Because lymphoma comprises numerous histologic subtypes, understanding the reasons for ongoing increases in its incidence requires surveillance and etiologic study of these subtypes. However, this research has been hindered by many coexisting classification schemes. The Revised European American classification of Lymphoid Neoplasms (REAL)/WHO system developed in 1994 and now used in clinical settings was not incorporated into the International Classification of Diseases-Oncology (ICD-O), used by cancer registries, until the release of the third edition (ICD-O-3) in 2001. Studies including patients diagnosed before 2001 may have codes from earlier ICD-O versions that must be converted to ICD-O-3 and have higher proportions of unclassified (e.g., lymphoma and not otherwise specified) cases. To better understand (a) the agreement of computer-converted ICD-O-3 codes to ICD-O-3 codes generated directly from diagnostic pathology reports and (b) the reproducibility of unclassified status, we reviewed a population-based series of diagnostic pathology reports for lymphoma patients diagnosed before (1988-1994; n = 1,493) and after (1998-2000; n = 1,527) the REAL/WHO scheme was introduced. Overall, computer- and coder-assigned ICD-O-3 codes agreed for 77% of patients in both groups and improved slightly (82%) when codes were grouped. The most common lymphoma subtypes, diffuse large B cell and follicular, had relatively good reliability (84-89%) throughout the study period. T-cell and natural killer cell lymphomas had worse agreement than B-cell lymphomas, even when grouped. Many (42-43%) lymphomas reported as unclassifiable could be assigned a subtype upon pathology report review. These findings suggest that the study of lymphoma subtypes could be improved by (a) use of more standardized terminology in pathology reports, (b) grouping individual ICD-O-3 codes to reduce misclassification bias, and (c) routine secondary editing of unclassified lymphomas by central cancer registries. (Cancer Epidemiol Biomarkers Prev 2006;15(4):630–8)

Lymphomas represent a large group of distinct B- and T-cell neoplasms arising from the lymphoid tissue. Overall lymphoma incidence has increased substantially in prior decades, with a 63% increase reported between 1973 and 2002 (1). Given the heterogeneity of lymphoma, understanding the reasons for this troubling increase will require the characterization and detailed etiologic study of distinct subtypes, a goal that has been hindered by the coexistence over the past three decades of multiple schemes for lymphoma classification. Although clinic-based studies used schemes like the Kiel, Lukes-Collins, Working Formulation, and the Revised European American classification of Lymphoid Neoplasms (REAL; ref. 2), which was later expanded and renamed the WHO classification, population-based studies and cancer registries used three different editions of the International Classification of Diseases-Oncology (ICD-O). The first (ICD-O-1) and second (ICD-O-2) editions (3) incorporated the Working Formulation and other now-obsolete schemes to classify lymphomas, whereas the most current third edition (ICD-O-3; ref. 4) draws heavily from the REAL/WHO scheme. ICD-O-3 has been used by cancer registries to classify cases diagnosed since the year 2001, but for cases diagnosed before 2001 and classified according to earlier versions of ICD-O, a computer algorithm has been devised to convert codes to ICD-O-3 (5).

The WHO classification represents an international consensus scheme (6) and is substantially more reliable and reproducible than prior schemes (7). Thus, the use of WHO and ICD-O-3 will simplify future monitoring and epidemiologic study of lymphoma subtypes. However, studies using historical data, such as assessments of time trends in cancer registry data, remain complicated by two issues. First, it is uncertain how well computer-generated ICD-O-3 codes agree with those directly assigned based on the original diagnostic pathology report. Of particular interest are diagnoses between 1995 and 2000 originally described by pathologists using WHO designations but coded by cancer registries to ICD-O-2. Second, cancer registry cases that are not subtyped [i.e., classified only as lymphoma and not otherwise specified (NOS)] complicate interpretation of distributions and trends of cases that are subtyped. NOS status (ICD-O-3 histology codes 9590-9591) was reported for 19% of all lymphomas reported to the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) program between 1973 and 2002 (1). To inform the implications of these issues for future research, we obtained and reviewed a population-based series of original diagnostic pathology reports for a selection of lymphoma patients diagnosed between the years 1988 and 2000, before and after the clinical dissemination of the REAL/WHO scheme, to (a) compare the agreement of computer-assigned ICD-O-3 to that assigned from the original diagnostic text and (b) determine the reproducibility of and reasons for assigning NOS status.

Included in this study were two distinct population-based groups of lymphoma patients reported to the Greater Bay Area Cancer Registry, a participant in the (SEER) program. The first group, hereafter referred to as the “1988 to 1994 study group,” included non-Hodgkin lymphoma (NHL) patients who participated in a population-based case-control study described in detail elsewhere (8-20). Patients initially eligible for the NHL study were ages 21 to 74 years and living in one of six San Francisco Bay Area counties (Alameda, Contra Costa, Marin, Santa Clara, San Francisco, or San Mateo) when newly diagnosed with NHL (ICD-O-2 morphology codes 9590-9595 and 9670-9711) from 1988 to 1994 and were identified through rapid case ascertainment procedures by the Greater Bay Area Cancer Registry. Pathology reports were collected as part of initial case ascertainment, but they were not obtained for 1,047 patients who at the time of interview were deceased or declined to participate (21). As detailed elsewhere (14), nonparticipating patients were more likely than participating patients to have more rapidly fatal lymphoma types. Of the participating patients, 61 had missing or incomplete (e.g., pages missing) pathology reports and were excluded from this analysis. Thus, the 1988 to 1994 study group included 1,493 lymphoma patients.

Patients eligible for the second group, hereafter called the “1998 to 2000 study group,” included 3,673 persons of all ages living in the six aforementioned counties when diagnosed with lymphoma (ICD-O-2 histology codes 9590-9719) during the years 1998 to 2000. We sampled patients on the basis of ICD-O-2 histology code, including 100% samples of those with codes (9590, 9591, 9593, 9595, 9650, 9653-9655, 9657-9659, 9671-9675, 9677, 9680-9686, 9688, 9690, 9692, 9705, 9708-9713, and 9715-9719) that represented new or updated entities in ICD-O-3 and smaller (n = 50) random samples of those with codes (9652, 9663-9667, 9670, 9691, 9693, 9695-98, 9700-9701, 9702-9704, 9606, 9607, and 9714) that represented nearly identical entities to those described in ICD-O-3, and thereby for whom conversion was code to code or otherwise considered straightforward. Sampling yielded a total of 2,834 patients. From this group, we excluded 196 patients for whom registry data suggested that the pathology report was unavailable (i.e., due to case ascertainment from death certificate/autopsy or lack of histologic confirmation) and requested the remaining 2,638 diagnostic pathology reports. We were able to retrieve successfully 1,693 (64%) reports. Of these, 166 were deemed during review to be incomplete or otherwise inadequate, resulting in a final 1998 to 2000 study group of 1,527 lymphoma patients for whom pathology reports were successfully reviewed. These patients differed somewhat from those for whom pathology reports could not be obtained or successfully reviewed. In particular, included patients were significantly (P < 0.05) more likely to have been diagnosed in 2000 (37% versus 31%), to be residents of Contra Costa (22% versus 10%) or San Francisco counties (21% versus 16%), to have a diffuse large B-cell lymphoma (DLBCL; 49% versus 42%), and to have evidence of AIDS (10% versus 8%), but were comparable with respect to age, sex, vital status, cause of death, and diagnosis at a teaching hospital.

ICD-O-3 Code Assignment

From the Greater Bay Area Cancer Registry records, we obtained demographic information, tumor characteristics (including original ICD-O-2 histology code and the computer-converted ICD-O-3 histology code), and patient vital status as of December 31, 2003 for all study subjects. Patients were considered to have evidence of AIDS based on cancer registry AIDS-positive status or HIV-related cause of death (21). Reviewers trained by Greater Bay Area Cancer Registry–certified tumor registrars abstracted diagnostic pathology reports for information regarding type of biopsy done (surgical, core needle, fine-needle aspiration, bone marrow, peripheral blood/other), type of diagnostic specimen (fresh/frozen tissue, preserved tissue, cells from fine-needle aspiration, or tissue from fine-needle aspiration), mention of malignant cell type [B, T, natural killer (NK), other], mention of diagnostic studies done [immunohistochemistry, flow cytometry, PCR-based methods, cytogenetic studies (e.g., fluorescent in situ hybridization assays for EBV)], and whether or not the diagnosis was a referral from another physician. Finally, ICD-O-3 histology code was assigned directly from the original diagnostic text using standard ICD-O-3 documentation and guidelines and without knowledge of the original ICD-O-2 code.

For cases that were coded as NOS (ICD-O-3 histology codes 9590-9591), reviewers were asked to classify the reason why a more specific code could not be assigned (i.e., diagnostic studies not done due to poor tissue quality/sufficiency, studies not done for other reasons, studies done but not interpretable, or disease considered truly unclassifiable). Quality control measures included consultation with certified tumor registrars or a consulting pathologist when needed and reabstraction and recoding of initial reviews for 2% of patients selected randomly.

Statistical Analysis

Cross-tabulations were used for all comparisons. Agreement was quantified using the percentage of all registry diagnoses confirmed by the coder to measure the positive predictive value (22). Kappa statistics could not be used because the two rating schemes (registry and coder) did not have identical categories due to uncommon ICD-O-3 codes assigned by one but not both of the raters. χ2 tests were used to describe univariate differences and compare them statistically. Logistic regression was used to examine associations with multiple variables. All calculations were done using SAS version 9. For some analyses, we grouped ICD-O-3 morphology codes together into broader categories, including Burkitt lymphoma (9687 and 9826), DLBCL (9678-9680 and 9684), marginal zone lymphoma (9689 and 9699), follicular lymphoma (9690, 9691, 9695, and 9698), lymphoplasmacytic lymphoma (9671 and 9761), mantle cell lymphoma (9673), small lymphocytic lymphoma/chronic lymphocytic leukemia (9670 and 9823), mycosis fungoides/Sezary disease (9700-9701), peripheral T-cell (9702, 9705, 9708, 9714, 9716, and 9827), NK/other T-cell (9709, 9717-9719, 9831, and 9948), lymphoblastic (9727-9729 and 9835-9837) and lymphoma, NOS (9590-9591, 9596, 9675, 9820, and 9970).

We reviewed 1,493 and 1,527 pathology reports from the 1988 to 1994 and 1998 to 2000 study groups, respectively. Table 1 describes demographic and other characteristics of patients comprising each study group. Patients in the 1998 to 2000 group were more likely than patients in the 1988 to 1994 group to be older, female, and non-White but were less likely to be deceased or to have evidence of AIDS, reflecting in some part the different selection criteria for the two groups. With respect to diagnostic characteristics, the 1988 to 1994 group was mostly diagnosed with a surgical biopsy (88%), with nearly half (45%) having some kind of immunohistochemical assay, but low proportions having flow cytometry (2%), assays for EBV (2%), cytogenetic studies (>1%), or molecular diagnostics (>1%). For the 1998 to 2000 group, a lower proportion (68%) was diagnosed with a surgical biopsy but had higher proportions having had immunohistochemical tests (77%), flow cytometry (13%), assays for EBV (3%), cytogenetic studies (1%), or molecular diagnostics (1%).

Table 1.

Demographic and other characteristics of lymphoma patients included in the 1988 to 1994 and 1998 to 2000 study groups, San Francisco Bay Area

1988-1994, n (%)1998-2000, n (%)
Age at diagnosis (y)   
    <20 0 (0) 24 (2) 
    20-29 73 (5) 63 (4) 
    30-39 232 (16) 143 (9) 
    40-49 286 (19) 192 (13) 
    50-59 292 (20) 233 (15) 
    60-69 417 (28) 259 (17) 
    70-79 193 (13) 344 (23) 
    ≥80 0 (0) 269 (18) 
Sex   
    Male 946 (63) 876 (57) 
    Female 547 (37) 651 (43) 
Race/ethnicity   
    White (non-Hispanic) 1,192 (80) 1,045 (68) 
    Black (non-Hispanic) 71 (5) 94 (6) 
    Hispanic 132 (9) 154 (10) 
    Asian/Pacific Islander 97 (7) 212 (14) 
    Other 1 (0) 22 (1) 
County of residence   
    San Francisco 359 (24) 314 (21) 
    Other 1,134 (76) 1,213 (79) 
AIDS status   
    Positive evidence 269 (18) 154 (10) 
    No evidence 1,224 (82) 1,373 (90) 
Vital status/cause of death   
    Alive at last report 522 (35) 780 (51) 
    Deceased 971 (65) 747 (49) 
Cause of death (% among deceased)   
    NHL 435 (45) 422 (57) 
    Hodgkin lymphoma 2 (0) 16 (2) 
    AIDS 233 (24) 69 (9) 
    Other 301 (31) 240 (32) 
Biopsy type   
    Surgical 1,309 (88) 1,039 (68) 
    Core needle 36 (2) 36 (2) 
    Fine-needle aspiration 91 (6) 91 (13) 
    Bone marrow 55 (4) 106 (7) 
    Other/unknown 2 (<1) 7 (<1) 
Total 1,493 1,527 
1988-1994, n (%)1998-2000, n (%)
Age at diagnosis (y)   
    <20 0 (0) 24 (2) 
    20-29 73 (5) 63 (4) 
    30-39 232 (16) 143 (9) 
    40-49 286 (19) 192 (13) 
    50-59 292 (20) 233 (15) 
    60-69 417 (28) 259 (17) 
    70-79 193 (13) 344 (23) 
    ≥80 0 (0) 269 (18) 
Sex   
    Male 946 (63) 876 (57) 
    Female 547 (37) 651 (43) 
Race/ethnicity   
    White (non-Hispanic) 1,192 (80) 1,045 (68) 
    Black (non-Hispanic) 71 (5) 94 (6) 
    Hispanic 132 (9) 154 (10) 
    Asian/Pacific Islander 97 (7) 212 (14) 
    Other 1 (0) 22 (1) 
County of residence   
    San Francisco 359 (24) 314 (21) 
    Other 1,134 (76) 1,213 (79) 
AIDS status   
    Positive evidence 269 (18) 154 (10) 
    No evidence 1,224 (82) 1,373 (90) 
Vital status/cause of death   
    Alive at last report 522 (35) 780 (51) 
    Deceased 971 (65) 747 (49) 
Cause of death (% among deceased)   
    NHL 435 (45) 422 (57) 
    Hodgkin lymphoma 2 (0) 16 (2) 
    AIDS 233 (24) 69 (9) 
    Other 301 (31) 240 (32) 
Biopsy type   
    Surgical 1,309 (88) 1,039 (68) 
    Core needle 36 (2) 36 (2) 
    Fine-needle aspiration 91 (6) 91 (13) 
    Bone marrow 55 (4) 106 (7) 
    Other/unknown 2 (<1) 7 (<1) 
Total 1,493 1,527 

Subtype Agreement

Overall, computer- and coder-assigned ICD-O-3 histologic subtype agreed for 1,145 (77%) of the 1988 to 1994 NHL study group and for 1,177 (77%) of the 1998 to 2000 group. For the 1988 to 1994 study group, Table 2 shows that agreement of computer- and coder-assigned subtype ranged from 0% to 100% for individual codes. Agreement was highest for mycosis fungoides (code 9700, 89%); follicular lymphoma, grade 1 (code 9695, 88%), DLBCL, NOS (code 9680, 88%); and Burkitt lymphoma (code 9687, 85%), and both cases originally coded as marginal zone B-cell lymphoma (code 9699) were also recoded as such (agreement of 100%). Most codes had agreement exceeding 70%. For immunoblastic DLBCL (code 9684), agreement was 81%, but 11% of cases were assigned by the coder to NOS. For mantle cell lymphoma (code 9673), agreement was mediocre (52%) with 10% of cases assigned to NOS. Subtype categories with very poor agreement (below 10%), including follicular lymphoma, NOS (code 9690, 4%) and cutaneous T-cell lymphoma, NOS (code 9709, 0%, based on one case only), were NOS categories within specific subtype categories for which most cases could be assigned to a more detailed subtype within the category. For the 1998 to 2000 group, which included both NHL and Hodgkin's lymphomas, Table 3 shows that subtype agreement also ranged from 0% to 100%. Agreement was high for angioimmunoblastic T-cell lymphoma (code 9705, 100%), nodular sclerosis type Hodgkin lymphoma (code 9663, 95%), mycosis fungoides (code 9700, 88%), DLBCL (code 9680; 87%), and immunoblastic DLBCL (code 9684, 88%). Subtypes with poorer agreement included mixed small and large cell diffuse lymphoma (code 9675, 40%; with 21% assigned by the coder to NOS); cutaneous T-cell lymphoma, NOS (code 9709, 26%; with 21% assigned by the coder to NOS); and the nasal NK/T-cell lymphoma (code 9719) subtype for which 0 of 2 cases (0 %) were recoded as the same entity.

Table 2.

Comparison of computer-assigned ICD-O-3 classification to coder-assigned classification for 1493 patients with lymphoma reported to the Greater Bay Area Cancer Registry 1988 to 1994, including case counts and positive predictive value (%)

Registry generated ICD-O-3 classificationCoder-assigned ICD-O-3 classification, n (%)
Total, n (%)
NOS95969670967196739675968096849687969096919695969896999700970297099714972797289835
NOS 133 (57)  13 (6)  7 (3) 9 (4) 20 (9) 12 (5) 6 (3) 1 (0) 9 (4) 9 (4) 6 (3) 7 (3)     1 (0)   233 (100) 
9670 5 (5)  77 (79) 8 (8) 4 (4)  1 (1)     1 (1)  1 (1)       1 (1) 98 (700) 
9671   2 (25) 5 (63)       1 (13)           8 (100) 
9673 5 (19)  5 (19)  14 (52)  2 (7)             1 (4)  27 (100) 
9675 5 (6)  1 (1)   52 (62) 8 (10) 1 (1)   10 (12) 1 (1) 3 (4) 1 (1)  1 (1) 1 (1)     84 (100) 
9680 18 (4) 2 (0) 3 (1)   1 (0) 410 (88) 11 (2) 1 (0)  1 (0) 2 (0) 4 (1)   3 (1)  8 (2)    464 (100) 
9684 15 (11)      6 (4) 114 (82)       1 (1) 1 (1)  2 (1)    139 (700) 
9687 3 (12)      1 (4)  22 (85)             26 (100) 
9690 1 (4)  2 (8)  1 (4) 1 (4) 2 (8)   1 (4) 3 (13) 8 (33) 4 (17)      1 (4)   24 (100) 
9691 3 (3)     5 (4) 3 (3)    94 (82) 4 (3) 6 (5)         115 (100) 
9695 3 (2)  1 (1) 1 (1) 1 (1) 1 (1)    3 (2) 5 (4) 110 (88)          125 (100) 
9698 2 (3) 1 (1) 1 (1)    8 (10)   4 (5) 1 (1)  60 (78)         77 (700) 
9699              2 (100)        2 (100) 
9700 2 (5)              39 (89)  2 (5) 1 (2)    44 (100) 
9701               1 (100)       1 (100) 
9702 2 (18)     2 (18)  2 (18)        3 (27) 1 (9) 1 (9)    11 (100) 
9709                1 (100) 0 (0)     1 (100) 
9727 2 (14)                  9 (64) 3 (21)  14 (100) 
Total 199 105 14 27 71 461 140 29 124 135 83 11 41 12 11 1,493 
Registry generated ICD-O-3 classificationCoder-assigned ICD-O-3 classification, n (%)
Total, n (%)
NOS95969670967196739675968096849687969096919695969896999700970297099714972797289835
NOS 133 (57)  13 (6)  7 (3) 9 (4) 20 (9) 12 (5) 6 (3) 1 (0) 9 (4) 9 (4) 6 (3) 7 (3)     1 (0)   233 (100) 
9670 5 (5)  77 (79) 8 (8) 4 (4)  1 (1)     1 (1)  1 (1)       1 (1) 98 (700) 
9671   2 (25) 5 (63)       1 (13)           8 (100) 
9673 5 (19)  5 (19)  14 (52)  2 (7)             1 (4)  27 (100) 
9675 5 (6)  1 (1)   52 (62) 8 (10) 1 (1)   10 (12) 1 (1) 3 (4) 1 (1)  1 (1) 1 (1)     84 (100) 
9680 18 (4) 2 (0) 3 (1)   1 (0) 410 (88) 11 (2) 1 (0)  1 (0) 2 (0) 4 (1)   3 (1)  8 (2)    464 (100) 
9684 15 (11)      6 (4) 114 (82)       1 (1) 1 (1)  2 (1)    139 (700) 
9687 3 (12)      1 (4)  22 (85)             26 (100) 
9690 1 (4)  2 (8)  1 (4) 1 (4) 2 (8)   1 (4) 3 (13) 8 (33) 4 (17)      1 (4)   24 (100) 
9691 3 (3)     5 (4) 3 (3)    94 (82) 4 (3) 6 (5)         115 (100) 
9695 3 (2)  1 (1) 1 (1) 1 (1) 1 (1)    3 (2) 5 (4) 110 (88)          125 (100) 
9698 2 (3) 1 (1) 1 (1)    8 (10)   4 (5) 1 (1)  60 (78)         77 (700) 
9699              2 (100)        2 (100) 
9700 2 (5)              39 (89)  2 (5) 1 (2)    44 (100) 
9701               1 (100)       1 (100) 
9702 2 (18)     2 (18)  2 (18)        3 (27) 1 (9) 1 (9)    11 (100) 
9709                1 (100) 0 (0)     1 (100) 
9727 2 (14)                  9 (64) 3 (21)  14 (100) 
Total 199 105 14 27 71 461 140 29 124 135 83 11 41 12 11 1,493 

NOTE: Malignant lymphoma, NOS (codes 9590 and 9591); composite Hodgkin's lymphoma and NHL (code 9596); Hodgkin's lymphoma, NOS (code 9650); Hodgkin's lymphoma, lymphocyte rich (code 9651); Hodgkin's lymphoma, mixed cellularity, NOS (code 9652); Hodgkin's lymphoma, lymphocyte depletion, NOS (code 9653); Hodgkin's lymphoma, lymphocyte depletion, reticular (code 9655); Hodgkin's lymphoma, nodular lymphocyte predominance (code 9659); Hodgkin's lymphoma, nodular sclerosis, NOS (code 9663); Hodgkin's lymphoma, nodular sclerosis, cellular phase (code 9664); Hodgkin's lymphoma, nodular sclerosis, grade 2 (code 9667); malignant lymphoma, small B lymphocytic, NOS (code 9670); malignant lymphoma, lymphoplasmacytic (code 9671); mantle cell lymphoma (code 9673); malignant lymphoma, mixed small and large cell, diffuse (code 9675); malignant lymphoma, large B cell, diffuse, NOS (code 9680); malignant lymphoma, large B cell, diffuse, immunoblastic, NOS (code 9684); Burkitt lymphoma, NOS (code 9687); splenic marginal zone B-cell lymphoma (code 9689); follicular lymphoma, NOS (code 9690); follicular lymphoma, grade 2 (code 9691); follicular lymphoma, grade 1 (code 9695); follicular lymphoma, grade 3 (code 9698); marginal zone B-cell lymphoma, NOS (code 9699); mycosis fungoides (code 9700); mature T-cell lymphoma, NOS (code 9702); angioimmunoblastic T-cell lymphoma (code 9705); s.c. panniculitis-like T-cell lymphoma (code 9708); cutaneous T-cell lymphoma, NOS (code 9709); anaplastic large cell lymphoma, T-cell and null cell type (code 9714); NK/T-cell lymphoma, nasal and nasal type (code 9719); precursor cell lymphoblastic lymphoma, NOS (code 9727); precursor T-cell lymphoblastic lymphoma (code 9729); Waldenstrom macroglobulinemia (code 9761); B-cell chronic lymphocytic leukemia/small lymphocytic lymphoma (code 9823); adult T-cell leukemia/lymphoma (HTLV-1 positive, code 9827); acute myelomonocytic leukemia (code 9867); lymphoproliferative disorder, NOS (code 9970).

Table 3.

Comparison of computer-assigned ICD-O-3 classification to coder-assigned classification for 1,527 patients with lymphoma reported to the Greater Bay Area Cancer Registry 1998 to 2000, including case counts and positive predictive value (%)

Registry generated ICD-O-3 classCoder-assigned ICD-O-3 classification, n (%)
Total, n (%)
NOS959696509651965296539655965996639667967096719673967596809684968796899690969196959698969997009702970597089709971497199727972997619823982798679970<9590
NOS 126 (58)       1 (0) 1 (0)  11 (5) 5 (2)  2 (1) 27 (13) 1 (0) 7 (3)  1 (0) 1 (0) 6 (3)  2 (1)  6 (3) 1 (0) 1 (0)  4 (2) 5 (2)     1 (0)  5 (2) 2 (1) 216 (100) 
9650 1 (3)  26 (67) 2 (5) 1 (3) 1 (3) 1 (3)  6 (15) 1 (3)                             39 (700) 
9651        1 (100)                               1 (100) 
9652 1 (3) 1 (3) 5 (14)  28 (80)                                  35 (700) 
9653 1 (33)     1 (33)    1 (33)                             3 (100) 
9659    2 (22)    4 (44) 2 (22)          1 (11)                    9 (100) 
9663   1 (5)      21 (95)                              22 (100) 
9664         1 (100)                              1 (100) 
9667         1 (33) 2 (67)                             3 (100) 
9670           25 (83)    2 (7)        1 (3)           1 (3)   1 (3)  30 (700) 
9671           1 (5) 18 (82)    1 (5)                 1 (5)    1 (5)  22 (700) 
9673 1 (2)          1 (2)  35 (81) 1 (2) 1 (2)        2 (5)           1 (2)   1 (2)  43 (99) 
9675 9 (21)    1 (2)        1 (2) 17 (40) 5 (12)    1 (2) 5 (12)  1 (2) 1 (2)   1 (2)             42 (100) 
9680 35 (5) 1 (0) 1 (0)     1 (0)   1 (0)   1 (0) 590 (87) 3 (0) 1 (0)  1 (0) 1 (0)  6 (1) 2 (0)  2 (0) 2 (0)   27 (4)        2 (0) 1 (0) 678 (100) 
9684 1 (3)              3 (9) 28 (88)                       32 (100) 
9687 3 (11)              2 (7)  23 (82)                      28 (100) 
9690 2 (7)          1 (4)    1 (4)    18 (64) 3 (11)  3 (11)                 28 (100) 
9691           1 (6)   2 (12)      11 (65) 1 (6) 1 (6)              1 (6)   17 (100) 
9695 2 (20)                    8 (80)                  10 (100) 
9698 1 (4)              2 (8)    3 (12)   20 (77)                 26 (100) 
9699 11 (9)          1 (1) 1 (1) 1 (1)  3 (2)   4 (3) 1 (1)   1 (1) 101 (80)     1 (1)          1 (1) 26 (100) 
9700 1 (6)                       15 (88)    1 (6)           17 (100) 
9702 1 (5)                        16 (80)     3 (15)         20 (100) 
9705                          9 (100)             9 (100) 
9708             1 (50)              1 (50)            2 (100) 
9709 4 (21)    1 (5)       2 (11)   1 (5) 1 (5)    1 (5)    2 (11)   1 (5) 5 (26) 1 (5)          19 (100) 
9714               6 (19)              25 (78) 1 (3)         32 (100) 
9719               1 (50)          1 (50)     0 (0)         2 (100) 
9727                               3 (21) 11 (79)       14 (100) 
9729                                1 (100)       1 (100) 
Total 200 33 31 32 42 26 38 23 644 34 31 26 22 15 32 109 17 25 13 57 12 10 1,527 
Registry generated ICD-O-3 classCoder-assigned ICD-O-3 classification, n (%)
Total, n (%)
NOS959696509651965296539655965996639667967096719673967596809684968796899690969196959698969997009702970597089709971497199727972997619823982798679970<9590
NOS 126 (58)       1 (0) 1 (0)  11 (5) 5 (2)  2 (1) 27 (13) 1 (0) 7 (3)  1 (0) 1 (0) 6 (3)  2 (1)  6 (3) 1 (0) 1 (0)  4 (2) 5 (2)     1 (0)  5 (2) 2 (1) 216 (100) 
9650 1 (3)  26 (67) 2 (5) 1 (3) 1 (3) 1 (3)  6 (15) 1 (3)                             39 (700) 
9651        1 (100)                               1 (100) 
9652 1 (3) 1 (3) 5 (14)  28 (80)                                  35 (700) 
9653 1 (33)     1 (33)    1 (33)                             3 (100) 
9659    2 (22)    4 (44) 2 (22)          1 (11)                    9 (100) 
9663   1 (5)      21 (95)                              22 (100) 
9664         1 (100)                              1 (100) 
9667         1 (33) 2 (67)                             3 (100) 
9670           25 (83)    2 (7)        1 (3)           1 (3)   1 (3)  30 (700) 
9671           1 (5) 18 (82)    1 (5)                 1 (5)    1 (5)  22 (700) 
9673 1 (2)          1 (2)  35 (81) 1 (2) 1 (2)        2 (5)           1 (2)   1 (2)  43 (99) 
9675 9 (21)    1 (2)        1 (2) 17 (40) 5 (12)    1 (2) 5 (12)  1 (2) 1 (2)   1 (2)             42 (100) 
9680 35 (5) 1 (0) 1 (0)     1 (0)   1 (0)   1 (0) 590 (87) 3 (0) 1 (0)  1 (0) 1 (0)  6 (1) 2 (0)  2 (0) 2 (0)   27 (4)        2 (0) 1 (0) 678 (100) 
9684 1 (3)              3 (9) 28 (88)                       32 (100) 
9687 3 (11)              2 (7)  23 (82)                      28 (100) 
9690 2 (7)          1 (4)    1 (4)    18 (64) 3 (11)  3 (11)                 28 (100) 
9691           1 (6)   2 (12)      11 (65) 1 (6) 1 (6)              1 (6)   17 (100) 
9695 2 (20)                    8 (80)                  10 (100) 
9698 1 (4)              2 (8)    3 (12)   20 (77)                 26 (100) 
9699 11 (9)          1 (1) 1 (1) 1 (1)  3 (2)   4 (3) 1 (1)   1 (1) 101 (80)     1 (1)          1 (1) 26 (100) 
9700 1 (6)                       15 (88)    1 (6)           17 (100) 
9702 1 (5)                        16 (80)     3 (15)         20 (100) 
9705                          9 (100)             9 (100) 
9708             1 (50)              1 (50)            2 (100) 
9709 4 (21)    1 (5)       2 (11)   1 (5) 1 (5)    1 (5)    2 (11)   1 (5) 5 (26) 1 (5)          19 (100) 
9714               6 (19)              25 (78) 1 (3)         32 (100) 
9719               1 (50)          1 (50)     0 (0)         2 (100) 
9727                               3 (21) 11 (79)       14 (100) 
9729                                1 (100)       1 (100) 
Total 200 33 31 32 42 26 38 23 644 34 31 26 22 15 32 109 17 25 13 57 12 10 1,527 

NOTE: Malignant lymphoma, NOS (codes 9590 and 9591); composite Hodgkin's lymphoma and NHL (code 9596); Hodgkin's lymphoma, NOS (code 9650); Hodgkin's lymphoma, lymphocyte rich (code 9651); Hodgkin's lymphoma, mixed cellularity, NOS (code 9652); Hodgkin's lymphoma, lymphocyte depletion, NOS (code 9653); Hodgkin's lymphoma, lymphocyte depletion, reticular (code 9655); Hodgkin's lymphoma, nodular lymphocyte predominance (code 9659); Hodgkin's lymphoma, nodular sclerosis, NOS (code 9663); Hodgkin's lymphoma, nodular sclerosis, cellular phase (code 9664); Hodgkin's lymphoma, nodular sclerosis, grade 2 (code 9667); malignant lymphoma, small B lymphocytic, NOS (code 9670); malignant lymphoma, lymphoplasmacytic (code 9671); mantle cell lymphoma (code 9673); malignant lymphoma, mixed small and large cell, diffuse (code 9675); malignant lymphoma, large B cell, diffuse, NOS (code 9680); malignant lymphoma, large B cell, diffuse, immunoblastic, NOS (code 9684); Burkitt lymphoma, NOS (code 9687); splenic marginal zone B-cell lymphoma (code 9689); follicular lymphoma, NOS (code 9690); follicular lymphoma, grade 2 (code 9691); follicular lymphoma, grade 1 (code 9695); follicular lymphoma, grade 3 (code 9698); marginal zone B-cell lymphoma, NOS (code 9699); mycosis fungoides (code 9700); mature T-cell lymphoma, NOS (code 9702); angioimmunoblastic T-cell lymphoma (code 9705); s.c. panniculitis-like T-cell lymphoma (code 9708); cutaneous T-cell lymphoma, NOS (code 9709); anaplastic large cell lymphoma, T-cell and null cell type (code 9714); NK/T-cell lymphoma, nasal and nasal type (code 9719); precursor cell lymphoblastic lymphoma, NOS (code 9727); precursor T-cell lymphoblastic lymphoma (code 9729); Waldenstrom macroglobulinemia (code 9761); B-cell chronic lymphocytic leukemia/small lymphocytic lymphoma (code 9823); adult T-cell leukemia/lymphoma (HTLV-1 positive, code 9827); acute myelomonocytic leukemia (code 9867); lymphoproliferative disorder, NOS (code 9970).

To gauge how agreement for specific subtypes changed over time, we compared the agreement within specific subtypes between the two study groups. For DLBCL (code 9680), the most common subtype, agreement was consistently high (88% versus 87%) in both groups. Follicular lymphoma, NOS (code 9690) showed a substantial improvement in agreement over time (4% in 1988-1994 versus 64% in 1998-2000). Although based on low numbers of observations, improvements were also seen for lymphoplasmacytic lymphoma (code 9671, 63-82%); mantle cell lymphoma (code 9673, 52-81%); mature T-cell lymphoma, NOS (code 9702, 27-80%); and cutaneous T-cell lymphoma, NOS (code 9709, 0-26%). Agreement decreased substantially over time for precursor cell lymphoblastic lymphoma, NOS (code 9727, 64-21%), mixed small and large cell diffuse lymphoma (code 9675, 62-40%), and follicular lymphoma grade 2 (code 9691, 82-65%).

Agreement generally improved when codes were grouped together into a pathologist-approved grouping scheme. Tables 4 and 5 show that overall, computer-and coder-assigned subtypes agreed for 1,219 of the 1,493 patients (82%) comprising the 1988 to 1994 study group and for 1,250 of the 1,527 (82%) patients in the 1998 to 2000 group. Agreement was similar between study groups for DLBCL (at 90% and 88%) and for follicular lymphoma (at 89% and 84%). Agreement seemed to improve substantially across time periods for the mantle cell lymphoma grouping, which increased from 52% to 81%, and for the peripheral T-cell lymphoma grouping, which increased from 36% to 81%.

Table 4.

Agreement of registry-assigned to coder-assigned ICD-O-3 classification groupings for lymphoma patients diagnosed 1988-1994, San Francisco Bay Area

Registry-assignedCoder assigned, n (%)
Total, n (%)
BurkittDLBCLMarginal zoneFollicularLymphoplasmacyticMantle cellSLL/CLLMF/SezaryPeripheral T cellNK/other T cellLymphoblasticOther/unknown
Burkitt 22 (85) 1 (4)          3 (12) 26 (100) 
DLBCL 1 (0) 541 (90)  7 (1)   3 (1) 1 (0) 14 (2)   36 (6) 603 (100) 
Marginal zone   2 (100)          2 (100) 
Follicular  13 (4)  303 (89) 1 (0) 2 (1) 4 (1)    1 (0) 17 (5) 341 (100) 
Lymphoplasmacytic    1 (13) 5 (63)  2 (25)      8 (100) 
Mantle cell  2 (7)    14 (52) 5 (19)    1 (4) 5 (19) 27 (100) 
SLL/CLL  1 (1) 1 (1) 1 (1) 8 (8) 4 (4) 77 (79)    1 (1) 5 (5) 98 (100) 
MF/Sezary        40 (89) 1 (2) 2 (4)  2 (4) 45 (100) 
Peripheral T cell  2 (18)       4 (36) 1 (9)  4 (36) 11 (100) 
NK/other T cell         1 (100) 0 (0)   1 (100) 
Lymphoblastic           12 (86) 2 (14) 14 (100) 
Other/unknown 6 (2) 41 (13) 8 (3) 39 (12)  7 (2) 14 (4)  1 (0) 1 (0) 1 (0) 199 (63) 317 (100) 
Total 29 601 11 351 14 27 105 41 21 16 273 1,493 
Registry-assignedCoder assigned, n (%)
Total, n (%)
BurkittDLBCLMarginal zoneFollicularLymphoplasmacyticMantle cellSLL/CLLMF/SezaryPeripheral T cellNK/other T cellLymphoblasticOther/unknown
Burkitt 22 (85) 1 (4)          3 (12) 26 (100) 
DLBCL 1 (0) 541 (90)  7 (1)   3 (1) 1 (0) 14 (2)   36 (6) 603 (100) 
Marginal zone   2 (100)          2 (100) 
Follicular  13 (4)  303 (89) 1 (0) 2 (1) 4 (1)    1 (0) 17 (5) 341 (100) 
Lymphoplasmacytic    1 (13) 5 (63)  2 (25)      8 (100) 
Mantle cell  2 (7)    14 (52) 5 (19)    1 (4) 5 (19) 27 (100) 
SLL/CLL  1 (1) 1 (1) 1 (1) 8 (8) 4 (4) 77 (79)    1 (1) 5 (5) 98 (100) 
MF/Sezary        40 (89) 1 (2) 2 (4)  2 (4) 45 (100) 
Peripheral T cell  2 (18)       4 (36) 1 (9)  4 (36) 11 (100) 
NK/other T cell         1 (100) 0 (0)   1 (100) 
Lymphoblastic           12 (86) 2 (14) 14 (100) 
Other/unknown 6 (2) 41 (13) 8 (3) 39 (12)  7 (2) 14 (4)  1 (0) 1 (0) 1 (0) 199 (63) 317 (100) 
Total 29 601 11 351 14 27 105 41 21 16 273 1,493 

NOTE: ICD-O-3 code groupings for lymphoma subtypes are as follows: Burkitt (codes 9687 and 9826), DLBCL (codes 9678-9680 and 9684), marginal zone (codes 9689 and 9699), follicular (codes 9690, 9691, 9695, and 9698),lymphoplasmacytic (codes 9671 and 9761), mantle cell (codes 9673), SLL/CLL (codes 9670 and 9823), mycosis fungoides/Sezary disease (codes 9700-9701), peripheral T cell (codes 9702, 9705, 9708, 9714, 9716, and 9827), NK/other T cell (codes 9709, 9717-9719, 9831, and 9948), lymphoblastic (codes 9727-9729 and 9835-9837), other/unknown type (codes 8000, 8140, 8263, 8586, 9590-9591, 9675, 9596, 9820, 9867, and 9970).

Abbreviations: CLL, chronic lymphocytic leukemia; SLL, small lymphocytic lymphoma; MF, mycosis fungoides.

Table 5.

Agreement of registry-assigned to coder-assigned ICD-O-3 classification groupings for lymphoma patients diagnosed 1998-2000, San Francisco Bay Area

Registry-assignedCoder-assigned, n (%)
Total, n (%)
BurkittDLBCLMarginal zoneFollicularLymphoplasmacyticMantle cellSLL/CLLMF/SezaryPeripheral T cellNK/other T cellHodgkin lymphoma, classicHodgkin lymphoma, nodularLymphoblasticOther/unknown
Burkitt 23 (82) 2 (7)            3 (11) 28 (100) 
DLBCL 1 (0) 624 (88) 2 (0) 8 (1)   1 (0)  31 (4)  1 (0) 1 (0)  41 (6) 710 (100) 
Marginal zone  3 (2) 105 (83) 2 (2) 1 (1) 1 (1) 1 (1)   1 (1)    12 (10) 126 (100) 
Follicular  3 (4)  68 (84)   2 (2)       8 (10) 81 (100) 
Lymphoplasmacytic  1 (5)   19 (86)  1 (5)       1 (5) 22 (100) 
Mantle cell  1 (2) 2 (5)   35 (81) 2 (5)       3 (7) 43 (100) 
SLL/CLL  2 (7) 1 (3)    26 (87)       1 (3) 30 (100) 
MF/Sezary        15 (88)  1 (6)    1 (6) 17 (100) 
Peripheral T cell  6 (10)    1 (2)   51 (81) 4 (6)    1 (2) 63 (100) 
NK/other T cell  3 (14)  1 (5) 2 (10)   2 (10) 3 (14) 5 (24) 1 (5)   4 (19) 21 (100) 
Hodgkin's lymphoma, classic           99 (95) 1 (1)  4 (4) 104 (100) 
Hodgkin's lymphoma, nodular    1 (11)       4 (44) 4 (44)   9 (100) 
Lymphoblastic             15 (100)  15 (100) 
Other/unknown 7 (3) 33 (13) 3 (1) 15 (6) 5 (2) 1 (0) 11 (4)  14 (5) 5 (2) 2 (1) 1 (0)  161 (62) 258 (100) 
Total 31 678 113 95 27 38 44 17 99 16 107 15 240 1,527 
Registry-assignedCoder-assigned, n (%)
Total, n (%)
BurkittDLBCLMarginal zoneFollicularLymphoplasmacyticMantle cellSLL/CLLMF/SezaryPeripheral T cellNK/other T cellHodgkin lymphoma, classicHodgkin lymphoma, nodularLymphoblasticOther/unknown
Burkitt 23 (82) 2 (7)            3 (11) 28 (100) 
DLBCL 1 (0) 624 (88) 2 (0) 8 (1)   1 (0)  31 (4)  1 (0) 1 (0)  41 (6) 710 (100) 
Marginal zone  3 (2) 105 (83) 2 (2) 1 (1) 1 (1) 1 (1)   1 (1)    12 (10) 126 (100) 
Follicular  3 (4)  68 (84)   2 (2)       8 (10) 81 (100) 
Lymphoplasmacytic  1 (5)   19 (86)  1 (5)       1 (5) 22 (100) 
Mantle cell  1 (2) 2 (5)   35 (81) 2 (5)       3 (7) 43 (100) 
SLL/CLL  2 (7) 1 (3)    26 (87)       1 (3) 30 (100) 
MF/Sezary        15 (88)  1 (6)    1 (6) 17 (100) 
Peripheral T cell  6 (10)    1 (2)   51 (81) 4 (6)    1 (2) 63 (100) 
NK/other T cell  3 (14)  1 (5) 2 (10)   2 (10) 3 (14) 5 (24) 1 (5)   4 (19) 21 (100) 
Hodgkin's lymphoma, classic           99 (95) 1 (1)  4 (4) 104 (100) 
Hodgkin's lymphoma, nodular    1 (11)       4 (44) 4 (44)   9 (100) 
Lymphoblastic             15 (100)  15 (100) 
Other/unknown 7 (3) 33 (13) 3 (1) 15 (6) 5 (2) 1 (0) 11 (4)  14 (5) 5 (2) 2 (1) 1 (0)  161 (62) 258 (100) 
Total 31 678 113 95 27 38 44 17 99 16 107 15 240 1,527 

NOTE: ICD-O-3 code groupings for lymphoma subtypes are as follows: Burkitt (codes 9687 and 9826), DLBCL (codes 9678-9680 and 9684), marginal zone (codes 9689 and 9699), follicular (codes 9690, 9691, 9695, and 9698), lymphoplasmacytic (codes 9671 and 9761), mantle cell (code 9673), SLL/CLL (codes 9670 and 9823), mycosis fungoides/Sezary disease (codes 9700-9701), peripheral T-cell (codes 9702, 9705, 9708, 9714, 9716, and 9827), NK/other T cell (codes 9709, 9717-9719, 9831, and 9948), lymphoblastic (codes 9727-9729 and 9835-9837), other/unknown type (codes 8000, 8140, 8263, 8586, 9590-91, 9596, 9675, 9820, 9867, and 9970).

Abbreviations: CLL, chronic lymphocytic leukemia; SLL, small lymphocytic lymphoma; MF, mycosis fungoides.

Completeness of malignant cell type information (e.g., B or T cell) on pathology reports seemed to improve over time. For cases diagnosed during 1988 to 1994, cell type was not specified on the pathology report for 68% of cases, whereas 26% specified B cell, 4% T cell, and 2% both T and B cell. For cases diagnosed during 1998 to 2000, only 36% of pathology reports did not specify cell type, whereas 54% specified B cell, 7% T cell, 3% both T and B cell, <1% NK cell, and <1% T/NK cell.

Lymphoma, NOS

Originally recorded by the cancer registry as lymphoma, NOS (ICD-O-3 codes 9590-9595) were 16% of the 1988 to 1994 study group and 14% of the 1998 to 2000 study group. After coder review, 14% and 13% of these groups, respectively, were so classified, including 57% of the 1988 to 1994 cases and 58% of the 1998 to 2000 cases that were originally classified as NOS. Compared with patients assigned a specific subtype, patients originally unclassified in the cancer registry were significantly (P < 0.01) more likely to have evidence of AIDS, to be deceased, male, older, and to have lived in San Francisco county. In a multivariate regression including these factors, those that were significantly and independently associated with unclassified status included evidence of AIDS (odds ratio, 2.8; 95% confidence interval, 2.0-3.9), residence in San Francisco county (odds ratio, 1.5; 95% confidence interval, 1.1-2.1), and age at diagnosis (odds ratio per year of age above 40, 1.01; 95% confidence interval, 1.00-1.02). Table 6 shows the distributions of detailed subtype assigned on re-review for 99 patients in the 1988 to 1994 study group and 80 patients in the 1998 to 2000 group that were originally unclassified. These distributions differed from those in the larger lymphoma patient population, with lower proportions of DLBCL. When compared with patients assigned a more detailed subtype upon review, the patients who remained unclassified after review (133 patients from the 1988-1994 group and 126 patients from the 1998-2000 group) did not differ by age, sex, race, diagnosis year, vital status, cause of death, and diagnosis at a teaching hospital but were significantly (P < 0.05) more likely to have evidence of AIDS (33% versus 22%) and be residents of San Francisco county (41% versus 28%). However, in multivariate regression, evidence of AIDS was the only independent predictor of remaining unclassified (odds ratio, 2.0; 95% confidence interval, 1.3-2.6). Sixty-six patients in the 1988 to 1994 study group (4% of total) and 74 patients in the 1998 to 2000 group (5% of total) were originally assigned a specific subtype but were assigned NOS status by the coder. These patients did not differ significantly with respect to demographic or hospital characteristics from patients assigned a specific code by both the registry and the coder but were more likely to have cancer registry subtype codes with mediocre or poor agreement, as described above.

Table 6.

Distributions of coder-assigned histologic subtype grouping for patients originally reported with unclassified subtype compared with SEER program data from the same time period

Coder-assigned ICD-O-3 grouping*1988-1994
1998-2000
Coder, n (%)SEER (%)Coder, n (%)SEER (%)
Burkitt 6 (6) 7 (9) 
DLBCL 41 (41) 54 30 (38) 50 
Marginal zone 7 (7) 2 (3) 
Follicular 25 (25) 23 8 (10) 21 
Lymphoplasmacytic 0 (0) 5 (6) 
Mantle cell 7 (7) 0 (0) 
SLL/CLL 13 (13) 11 11 (14) 
MF/Sezary 0 (0) 0 (0) 
Peripheral T cell 0 (0) 12 (15) 
Other T cell/NK 0 (0) 5 (6) 
Total 99 (100) 100 80 (100) 100 
Coder-assigned ICD-O-3 grouping*1988-1994
1998-2000
Coder, n (%)SEER (%)Coder, n (%)SEER (%)
Burkitt 6 (6) 7 (9) 
DLBCL 41 (41) 54 30 (38) 50 
Marginal zone 7 (7) 2 (3) 
Follicular 25 (25) 23 8 (10) 21 
Lymphoplasmacytic 0 (0) 5 (6) 
Mantle cell 7 (7) 0 (0) 
SLL/CLL 13 (13) 11 11 (14) 
MF/Sezary 0 (0) 0 (0) 
Peripheral T cell 0 (0) 12 (15) 
Other T cell/NK 0 (0) 5 (6) 
Total 99 (100) 100 80 (100) 100 

Abbreviations: CLL, chronic lymphocytic leukemia; SLL, small lymphocytic lymphoma; MF, mycosis fungoides; SEER, Surveillance, Epidemiology, and End Results.

*

ICD-O-3 code groupings for lymphoma subtypes are as follows: Burkitt (codes 9687 and 9826), DLBCL (codes 9675, 9678-9680, and 9684), marginal zone (codes 9689 and 9699), follicular (codes 9690, 9691, 9695, and 9698), lymphoplasmacytic (codes 9671 and 9761), mantle cell (codes 9673), SLL/CLL (codes 9670 and 9823), MF/Sezary disease (codes 9700-9701), peripheral T cell (codes 9702, 9705, 9708, 9714, 9716, and 9827), NK/other T cell (codes 9709, 9717-9719, 9831, and 9948), other/unknown type (codes 8000, 8140, 8263, 8586, 9590-9591, 9596, 9820, 9867, and 9970).

Cases ages 21 to 74 years diagnosed with NHL (codes 9670-9719) from 1988 to 1994 while residing in six county geographic region.

Cases of all ages diagnosed with NHL (codes 9670-9719) from 1998 to 2000 while residing in six county geographic region.

Our findings suggest that computer-converted ICD-O-3 codes for lymphoma subtypes are generally reliable at the single code level, and that agreement is improved when codes are grouped. The two most common types of lymphoma, DLBCL and follicular lymphoma, had relatively good agreement (84-89%) across the entire time period. T and NK cell lymphomas had generally worse agreement than B-cell lymphomas, even after they were grouped together. Variations in single code agreement may result from deviations in terminology used by pathologists to describe the same entity. Despite the detailed guidance provided in the ICD-O-3 coding materials, ambiguous terminology on pathology reports can result in nonuniform interpretation and coding of lymphoma subtypes by cancer registrars, as suggested previously (23). Even a pathologist reviewing pathology reports to assign WHO codes to cases in an epidemiologic study could assign a classification with “high confidence” to only 58% of cases (24). Thus, improving reliability of lymphoma subtype classification will depend mostly on pathologists using more standardized terminology and including more detail about immunohistochemical reports when diagnosing or classifying lymphoma subtypes. Short of this, grouping lymphoma subtypes together into meaningful groups for surveillance or analysis will reduce potential bias due to subtype misclassification.

Our findings also suggest that a substantial proportion (42-43%) of lymphomas initially reported to the cancer registry as NOS or unclassifiable in fact could be classified as a more detailed subtype after re-review of the diagnostic pathology report. We also observed that a small proportion of cases originally reported as a specific code were assigned NOS by the coder, but that the total proportion of cases assigned as NOS was only slightly lower for the coder-reviewed versus registry-assigned groups. Of those cases that remained unclassifiable after coder review, pathology report mention of inadequate specimen quality was the most common reason why a more detailed subtype could not be assigned. Less than 10% of pathology reports did not yield a detailed subtype because the pathologist considered the specimen truly unclassifiable. This proportion is lower than our prior finding of 25% of lymphoma NOS cases considered as unclassifiable after microscopic re-review by an expert hematopathologist (21). Cases originally classified as NOS varied with respect to age, geography, and particularly evidence of AIDS, and the distributions of subtypes for “resolved” NOS cases varied from the expected distribution. These variations, coupled with the high prevalence of NOS lymphomas, which represented about one in five of all NHLs reported to the Surveillance, Epidemiology, and End Results program, suggest that the misclassification of classifiable lymphomas as NOS may have biased prior Surveillance, Epidemiology, and End Results data-based assessments of NHL subtype-specific incidence and survival (25-29). Therefore, the surveillance of lymphoma subtypes could be improved by routine secondary editing or review of histologic coding by cancer registry personnel for all lymphomas initially reported as NOS.

Between the 1988 to 1994 and 1998 to 2000 study groups, which were chosen to differ temporally in relation to the adoption of the REAL and WHO classification schemes, we observed only subtle differences in overall or subtype-specific agreement for the most part. Overall agreement of computer-assigned and re-reviewed ICD-O-3 code was identical (77%) in both groups, as was the proportion of resolvable NOS cases (42-43%). For specific subtype groupings, agreement did seem to improve over time for the mantle cell, lymphoplasmacytic, and some of the T-cell lymphoma groups but was comparable for most other subtype groupings. In addition, completeness of the malignant cell type variable improved from 32% to 64% between the two study time periods.

Several factors may affect the generalizability of our findings. First, lymphoma subtype distributions in the San Francisco Bay Area generally differ from those in other regions during the study years, particularly with respect to HIV-associated lymphomas. The higher incidence of these lymphomas may influence local classification practices (e.g., classification of central nervous system lymphomas as NOS). However, reasons for NOS classification did not vary significantly between patients with and without AIDS, although a slightly higher percentage of AIDS lymphoma patients (21% versus 15%) were classified as NOS because “assay information was not available” from the pathology report, suggesting differences in diagnostic procedures for HIV and non–HIV-associated lymphomas. Second, we delineated our case series according to ICD-O-2–defined codes for lymphomas and thereby did not include cases originally coded as leukemias or other lymphoid neoplasms in our case series. Thus, our data cannot speak well to the reliability of codes for lymphoma entities with leukemic presentations (e.g., 9820). Third, for the 1998 to 2000 study group, we were unable to obtain a substantial proportion of pathology reports requested from hospitals. As much of this nonparticipation occurred at the hospital rather than patient level, our final study group might be of biased representation with respect to pathologist diagnostic or cancer registrar coding practices that differ among hospitals irrespective of their teaching status. This effect is probably minor, as histologic subtype distributions were generally comparable between the participating and nonparticipating groups, with only a slightly greater proportion of large B-cell lymphomas (ICD-O-3 code 9680) in the participating group.

The dissemination of the consensus REAL/WHO classification system marked a hopeful moment in our quest to learn more about the etiologies of specific lymphoma subtypes, especially subtypes shown to be increasing in incidence. However, as tracking of lymphoma subtype trends over time remains an important activity, more research should be done to understand and implement reliable means of consistent subtype classification over time in cancer registry and other data. Ensuring that uniform means of subtype classification and grouping are being used in all future surveillance and etiologic studies of lymphoid neoplasms is an explicit goal of InterLymph, the International Lymphoma Epidemiology Consortium (30). Although the REAL/WHO represents the most reliable classification system devised thus far, ongoing improvements in molecular characterization of lymphoid neoplasms will surely lead to even better classification schemes that may further enlighten the etiologies of these cancers.

Grant support: Rapid Response Surveillance Study program of the National Cancer Institute Surveillance Epidemiology and End Results program contract N01-CN-65107. National Cancer Institute grants CA45614, CA66529, and CA89745 (E.A. Holly) supported the primary work on the case-control study that collected the pathology reports for the 1988 to 1994 group. Cancer registry data were collected by the Northern California Cancer Center under contract N01-CN-65107 with the National Cancer Institute, NIH and with the support of the California Cancer Registry, a project of the Cancer Surveillance Section, California Department of Health Services, under subcontract 1006128 with the USPHS.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Note: The content of this publication does not necessarily reflect the views or policies of the U.S. Department of Health and Human Services nor the California Department of Health Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government or State of California.

We thank Elizabeth Traynor, M.D.; Paige Bracci; Jennifer Kristianson; Kathleen Torres; and Jennifer Yang for their contributions.

1
Surveillance, Epidemiology, and End Results (SEER) Program Public-Use Data (1973–2002), released April 2005, based on the November 2004 submission. Silver Spring (MD): Information Management Services, Inc.
2
Harris NL, Jaffe ES, Stein H, et al. A revised European-American classification of lymphoid neoplasms: a proposal from the International Lymphoma Study Group.
Blood
1994
;
84
:
1361
–92.
3
Percy C, Van Holten V, Muir C. International classification of diseases for oncology. Geneva: WHO; 1990.
4
Fritz A, Percy C, Jack A, et al., editors. International Classification of Diseases for Oncology, 3rd ed. Geneva: WHO; 2000.
5
Percy C, Fritz A, Ries L, editors. Conversion of neoplasms by topography and morphology from the International Classification of Diseases for Oncology, 2nd ed. to the International Classification of Diseases for Oncology, 3rd ed. Rockville (MD): National Cancer Institute; 2001.
6
Harris NL, Jaffe ES, Diebold J, et al. World Health Organization classification of neoplastic diseases of the hematopoietic and lymphoid tissues: report of the Clinical Advisory Committee meeting-Airlie House, Virginia, November 1997.
J Clin Oncol
1999
;
17
:
3835
–49.
7
International Lymphoma Study Group. A clinical evaluation of the International Lymphoma Study Group classification of non-Hodgkin's lymphoma. The Non-Hodgkin's Lymphoma Classification Project.
Blood
1997
;
89
:
3909
–18.
8
Holly EA, Lele C, Bracci P. Non-Hodgkin's lymphoma in homosexual men in the San Francisco Bay Area: occupational, chemical, and environmental exposures.
J Acquir Immune Defic Syndr Hum Retrovirol
1997
;
15
:
223
–31.
9
Holly EA, Lele C. Non-Hodgkin's lymphoma in HIV-positive and HIV-negative homosexual men in the San Francisco Bay Area: allergies, prior medication use, and sexual practices.
J Acquir Immune Defic Syndr Hum Retrovirol
1997
;
15
:
211
–22.
10
Holly EA, Lele C, Bracci PM. Hair-color products and risk for non-Hodgkin's lymphoma: a population-based study in the San Francisco bay area.
Am J Public Health
1998
;
88
:
1767
–73.
11
Holly EA, Lele C, Bracci PM, et al. Case-control study of non-Hodgkin's lymphoma among women and heterosexual men in the San Francisco Bay Area, California.
Am J Epidemiol
1999
;
150
:
375
–89.
12
Kamel OW, Holly EA, van de Rijn M, et al. A population based, case control study of non-Hodgkin's lymphoma in patients with rheumatoid arthritis.
J Rheumatol
1999
;
26
:
1676
–80.
13
Tiirikainen MI, Mullaney BP, Holly EA, et al. DNA copy number alterations in HIV-positive and HIV-negative patients with diffuse large-cell lymphomas.
J Acquir Immune Defic Syndr
2001
;
27
:
272
–6.
14
Holly EA, Gautam M, Bracci PM. Comparison of interviewed and non-interviewed non-Hodgkin's lymphoma (NHL) patients in the San Francisco Bay Area.
Ann Epidemiol
2002
;
12
:
419
–25.
15
Chow EJ, Holly EA. Blood transfusions as a risk factor for non-Hodgkin's lymphoma in the San Francisco Bay Area: a population-based study.
Am J Epidemiol
2002
;
155
:
725
–31.
16
Holly EA, Bracci PM. San Francisco population-based study of non-Hodgkin's lymphoma histology and medical history among HIV-negative participants.
Am J Epidemiol
2003
;
158
:
316
–27.
17
Bracci PM, Holly EA. Tobacco use and non-Hodgkin lymphoma: results from a population-based case-control study in the San Francisco Bay Area, California.
Cancer Causes Control
2005
;
16
:
333
–46.
18
Holly EA, Eberle CA, Bracci PM. Prior history of allergies and pancreatic cancer in the San Francisco Bay area.
Am J Epidemiol
2003
;
158
:
432
–41.
19
Holly EA, Bracci PM. Population-based study of non-Hodgkin lymphoma, histology, and medical history among human immunodeficiency virus-negative participants in San Francisco.
Am J Epidemiol
2003
;
158
:
316
–27.
20
Skibola CF, Forrest MS, Coppede F, et al. Polymorphisms and haplotypes in folate-metabolizing genes and risk of non-Hodgkin lymphoma.
Blood
2004
;
104
:
2155
–62.
21
Clarke CA, Glaser SL, Dorfman RF, et al. Expert review of non-Hodgkin's lymphomas in a population-based cancer registry: reliability of diagnosis and subtype classifications.
Cancer Epidemiol Biomarkers Prev
2004
;
13
:
138
–43.
22
Hennekens CH, Buring JE. Epidemiology in medicine. Boston: Little, Brown and Company; 1987.
23
Dick F, VanLier S, Banks P, et al. Use of the working formulation for non-Hodgkin's lymphoma in epidemiologic studies: agreement between reported diagnoses and a panel of experienced pathologists.
J Natl Cancer Inst
1987
;
78
:
1137
–44.
24
Turner JJ, Hughes AM, Kricker A, et al. Use of the WHO lymphoma classification in a population-based epidemiological study.
Ann Oncol
2004
;
15
:
631
–7.
25
Greiner TC, Medeiros LJ, Jaffe ES. Non-Hodgkin's lymphoma.
Cancer
1995
;
75
:
370
–80.
26
Herrinton LJ. Epidemiology of the Revised European-American Lymphoma Classification subtypes.
Epidemiol Rev
1998
;
20
:
187
–203.
27
Groves FD, Linet MS, Travis LB, et al. Cancer surveillance series: non-Hodgkin's lymphoma incidence by histologic subtype in the United States from 1978 through 1995.
J Natl Cancer Inst
2000
;
92
:
1240
–51.
28
Eltom MA, Jemal A, Mbulaiteye SM, et al. Trends in Kaposi's sarcoma and non-Hodgkin's lymphoma incidence in the United States from 1973 through 1998.
J Natl Cancer Inst
2002
;
94
:
1204
–10.
29
Ries LAG, Eisner MP, Kosary CL, et al., editors. SEER cancer statistics review, 1973–2002. Bethesda (MD): National Cancer Institute; 2005.
30
Boffetta P, Linet M, Armstrong B. The Interlymph collaboration: a consortium of molecular epidemiologic studies of non-Hodgkin's lymphoma [abstract 1579].
Proc Am Assoc Cancer Res
2003
;
44
:
308
.