Low Proportion of False-Negative Smears in the Finnish Program for Cervical Cancer Screening

Cervical cancer remains an important cause of morbidity and mortality worldwide, with an estimated global incidence of half a million and nearly a quarter of a million deaths annually (1). In Finland, population-based screening with conventional Pap smears has resulted in a nearly 80% decrease in cervical cancer incidence, from 15/100,000 woman years in 1963 to 4/100,000 in 2007 (2, 3). In most other countries, the effect of screening has been smaller, prompting the call for systematic audits of the different aspects of screening (4-8).

Population-based cervical cancer screening has high preventive potential but performance and detection rates of precancerous lesions vary considerably between programs. Reported diagnostic false-negative rates for cervical cytology range from 15% to 63% in different study settings, illustrating potentially differential validity as well as difficulty in nonblinded retrospective assessment of the Pap smear (9-17). There is variation also in specificity and related positive predictive value, but previous rereading studies have not used controls together with potential false-negative smears and have thus not assessed these validity problems (7, 18). The difficulties stem from the lack of an unbiased reference method, as screening primarily targets largely reversible precursors to prevent the future development of cancer. This is the strength of cervical cancer screening but at the same time renders direct evaluation problematic.

The purpose of the current study was to assess the performance and validity, specifically the false-negative rate, sensitivity, specificity, and reproducibility, of the cytologic diagnosis in cytologic screening in Finland and establish a procedure for continuing quality control. We have used clinical outcome to choose potential false-negative screening smears together with control smears from the healthy screening population and, unlike in any earlier study, blinded laboratory rereading and an expert panel to establish final cytologic results.

The cervical cancer screening program under study invites all women between the ages of 30 and 60 y with a personal letter for a conventional Pap smear with 5-y intervals. Some municipalities have extended the range by also including women ages 25 and 65 y (19).

Screening invitations and visits, including histologically confirmed findings, are registered in the Mass Screening Registry of the Finnish Cancer Registry. The Cancer Registry itself includes all reported cervical cancer, cervical intraepithelial neoplasia grade 3 (CIN3), and adenocarcinoma in situ (AIS) cases. Practising physicians and pathologists are requested to report these diagnoses directly to the Cancer Registry. In addition, death certificate diagnoses are obtained yearly and additional data are subsequently requested of those cases not previously reported (∼1% of all cancers; ref. 19).

The current study is based on a linkage between screening and cancer registry records over the period of 1990-1999, which involved 2.3 million visits among 1.3 million women. Six of the laboratories active throughout the study period participated with archives representing at least 5 of the 10 y covered by this study. These laboratories are located in Helsinki, Kotka, Kuopio, Oulu, Turku, and Vaasa and represent a good geographic cross-section of Finland. The study material amounted to a total of 828,514 screen samples.

Audit cases were defined as cervical cancers and CIN3/AIS lesions (CIN3+) diagnosed during 1990-1999 among women with Pap class I or class II [equal to reactive changes or atypical squamous intraepithelial lesion with undetermined significance (ASC-US) in Bethesda 2001, and borderline changes in British terminology] at previous screen without a referral for further assessment. Four hundred twenty-one such cases were identified. These included symptomatic cases, cases diagnosed through opportunistic screening, and, in the event of a preceding borderline screening test, cases diagnosed during the intensified follow-up recommended by the program. The participating laboratories were asked to locate the corresponding smears along with two control smears (the one preceding and the one following the case smear in sequence in their smear archives). Controls were thought particularly useful for the assessment of specificity of rereading. Three hundred forty-five (82%) of the requested case smears were eventually retrieved along with 689 controls (Fig. 1).

The smears were collected at the Mass Screening Registry and arranged into a random sequence and blinded to the status of case or control. The rereading was done in the original laboratories as well as in a reference laboratory and recorded using the standard Bethesda 2001 nomenclature (20). When these two reading results were clearly discrepant (Table 1), the final cytologic diagnosis was assessed by an expert panel. This involved 312 of 1,034 smears (180 case smears and 132 controls). The panel consisted of five members: two from the original laboratory (a cytotechnician and a cytopathologist), two from the reference laboratory, and the last member was an external cytopathologist experienced in screening. The panel worked blinded to the case history in question and each member reported his or her findings independently. In cases where the independent assessments were discrepant, the final result was determined in a discussion until a unanimous verdict was achieved. Due to the fact that in the Bethesda system, squamous and glandular cells are separately accounted for, it is possible to have two diagnoses for each smear. For statistical and presentational reasons, only the higher-grade atypia was considered and this being equal, squamous cell lesions took precedence over glandular.

Distribution of the cytologic diagnoses, detailed and grouped according to clinical relevance, were separately tabulated among case and control smears and by the different phases of rereading. Sensitivity and specificity were calculated for the rescreening results from the different sets of laboratories (screening and reference) with the final cytology as gold standard. These validity measures were estimated with 95% confidence intervals (95% CI) assuming binomial distribution. Cumulative results of final cytology were tabulated by case and control, as well as by original screening Pap classification (Pap class I or class II); 95% CIs are also reported. The false-negative rate was calculated as the proportion of case smears with a final cytologic diagnosis warranting referral for colposcopy. Reproducibility was assessed by cross-tabulating the cytologic diagnoses of screening and reference laboratories. Linearly weighted and unweighted κ values for interobserver reliability were calculated with 95% CIs.

The attendance rate of the study population in organized screening during the observed time period was 72% (Fig. 1), with a range between laboratories of 66% to 80%. On average, 0.8% of the screened women per one screening round were referred for colposcopy. In addition,5.4% of screened women had borderline cytologic findings, warranting, according to the routine, a repeated smear after 12 months instead of after the normal 5 years. Of a total of 6,949 referrals, 915 cases of CIN3+ were detected,87 of which were invasive cancers. Cases with a preceding negative screen, thus qualifying for the rereading phase, amounted to 421, 115 of which were invasive cancers. Smears for 345 (82%) of these cases, including 54 (47%) of the invasive cancers, were retrieved for rereading. Of the retrieved case smears, 230 (67%) were originally classified as Pap I and the rest as Pap II. The mean time interval from negative smear to CIN3+ diagnosis was 31 months, with a range of 1 to 109 months.

There was a large difference in the rates of high-grade squamous intraepithelial lesion (HSIL) in the case material between the rereading phases, with frequency proportions of 7.5%, 13%, and 19% according to screen laboratory, reference laboratory, and final cytology, respectively. Of 345 case smears, 164 (48%) were classified as normal according to final cytology (Table 2). No smears in this study were deemed unreadable due to poor smear quality.

In rereading, ASC-US+ was detected in 35% of the audit cases by the screening laboratories, 51% by the reference laboratory, and 52% according to the final cytologic result (Table 2; Fig. 2). The corresponding rates among controls were 7.3%, 19%, and 12%. The proportion of case smears that were classified as low-grade squamous intraepithelial lesion or worse (LSIL+, warranting referral) by final cytology, thus satisfying our criteria for false negatives, amounted to 35% (30-40%). For controls, the corresponding proportion was 2.8% (1.8-4.3%).

Assuming a false-negative rate of 35% to hold for all 421 identified cases with a preceding negative screen, false-negative screening results accounted for 11% (147 of 1,336) of the total observed CIN3+ cases in the screened population. On the other hand, many of the CIN3+ cases with a preceding Pap II screening result were diagnosed within the intensified screening activity done after borderline cytologic findings. The estimate of the false-negative rate of only those CIN3+ cases diagnosed after a Pap I result in the program was 23%. These cases accounted for 5% of the total observed CIN3+ cases.

There was no difference in the false-negative rates according to case severity (CIN3/AIS or invasive cancer). For those case smears that were originally classified as normal (Pap I), the rate of ASC-US+ in final cytology was 39%, whereas it was 80% in those originally classified as borderline (Pap II; Table 3).

Sensitivity of rereading was estimated as the proportion of screening, or reference, laboratory positives relative to the positives according to the final cytologic results (Table 4). At a threshold of ASC-US relative to final cytology of LSIL+, the sensitivity of the screening laboratories in rereading was 75% (67-82%) with a specificity of 93% (91-94%). Sensitivity with a threshold of LSIL+ was 50% and specificity 99%. The reference laboratory was more sensitive overall at considerable expense of specificity.

Cross-tabulation of the reevaluated specific cytologic diagnoses of rereading in the screening laboratories against those of the reference laboratory revealed only a fair correlation (Table 5). Unweighted κ was 0.24 (0.19-0.29). The unweighted κ value for management recommendation according to the principles of follow-up for ASC-US and referral for colposcopy for LSIL+ was 0.30. The linearly weighted κ for this three-class correlation was 0.38. Reproducibility of the ASC-H (atypical squamous cells, cannot exclude HSIL) diagnosis was particularly poor. Both glandular cell classes also had poor reproduction.

The current study, in which we calculated the screen-detected CIN3+ cases and reevaluated the smears of women diagnosed following a negative screen, showed that only a small proportion of CIN3+ and particularly of cervical cancer cases were due to screen failure. These results confirm previous conclusions that the low disease burden in Finland is mainly due to organized screening and that further prevention of cervical cancer with new screening methods and technologies has a fairly small potential for additional effect.

We found that the cytologic false-negative rate was 35%, as defined by final cytology of LSIL+, among smears preceding diagnosis of CIN3+ but in which no referral had been recommended. The corresponding false-negative rate in the Pap I subpopulation was lower at 23%. An additional 18% of all case smears had cytologic changes of undetermined significance [ASC-US and atypical glandular cells, not otherwise specified (AGC-NOS)]. Previously reported false-negative rates range from 15% to 63% in comparable studies (9-17). The 48% of case smears that were normal according to final cytology probably represent sampling errors but can also include cases where neoplasia has developed rapidly after the collection of the screening sample.

Assuming that the false-negative rate in the final study material would hold for all the audit cases identified, false-negative screening results accounted for 11% of the total observed CIN3+ cases in the screened population. Those CIN3+ cases diagnosed after a Pap I screening result within the program and with LSIL+ diagnosis in rereading constituted 5% of all CIN3+ cases. Furthermore, every treated CIN3 lesion is not a prevented cancer case. An early Finnish study estimated that 28% to 39% of high-degree (severe) dysplasia and carcinoma in situ cases combined will progress to invasive cancer (21). Progression rates estimated in a study of untreated CIN3 cases in New Zealand (22) and in a study using data from the British Columbia screening program (23) also fall within this range (31% and 38%, respectively). Because the average duration of the precancer phase is 10 to 12 years before progression to invasive cancer, many of the CIN3/AIS cases with a preceding negative screen would not have progressed before the next scheduled invitation. Taking this natural history into account, it is likely that the proportion with a false-negative diagnosis of the progressive CIN3/AIS lesions is less than that among all those lesions.

Of 421 CIN3+ cases identified for rereading, 115 were cervix carcinomas (0.01% of screening samples). Eighty-two percent of all requested case smears were recovered for reevaluation but only 47% of the invasive case cases. An explanation might be that on diagnosis of cervix cancer, the treating hospital often requests any preceding smears, sometimes failing to return them to the screening archives. Validity measures and false-negative rates did not vary with the eventual histologic diagnosis, however.

In the literature, sensitivity of a single Pap smear has been reported to range from 30% to 87% (24). Our estimate for the original screening laboratories during the rereading phase of the study was 75% at the lowest cutoff of ASC-US and 50% at the cutoff of LSIL among “true” LSIL+ cases. Validity measures for the reference laboratory differed significantly from those of the screening laboratories. This highlights the difficulty of finding cytologic truth by using just a single laboratory as gold standard.

Reproducibility of the cytologic diagnoses was fair at best when using the Bethesda 2001 classification system. This was true also when analyzing results using only three outcome categories and the linearly weighted κ value as indicator. Specificity variation between laboratories was important, leading to local variations in adverse effects and cost of screening but not necessarily in effectiveness. The difficulty of diagnosing glandular lesions with the conventional Pap smear seems to be another important issue for poor reproducibility. To achieve and maintain consistent screening quality, it is necessary to continue the evaluation and feedback process between and with the laboratories.

Knowledge that the rereading material contained a large proportion of potential false negatives, differing in this respect from normal screening material, was a possible source of bias. These pre-expectations could have influenced the evaluation by enhancing sensitivity and decreasing specificity, although this effect may be assessed by using controls, randomization, and blinding. The LSIL+ rates in general screening and in the rereading of controls were similar (0.8% versus 1.0%), suggesting the level of bias being low. Another study method would be to embed the study smears in day-to-day screening while keeping the laboratory personnel ignorant of their participation in a study. A panel working blinded and making individual judgements before a consensus discussion was assumed, however, to be the most valid cytologic gold standard and better than a single assessor.

Improving the sensitivity of screening by changing diagnostic criteria should still be possible but hardly feasible, considering the small gain on offer in exchange for a large loss of specificity or large incremental costs in the case of double reading of smears. We conclude that both the sensitivity and the specificity of the cytologic laboratories involved in the program are satisfactory, with very low cancer incidence and low referral rates achieved over the last few decades. Still, attention should be placed on the reproducibility of cytologic diagnoses, as well as on ensuring high attendance.

In a study by Bulk et al. (10), cytologic rereading of originally normal smears preceding a diagnosis of CIN2/3 resulted in an upgrade to borderline/mild dyskaryosis or worse (corresponding to ASC-US+) in 33% of cases. This figure primarily compares with the 39% ASC-US+ results according to final cytology of those case smears in our study that were originally classified as Pap I. In the Dutch study, there were no controls and, hence, no specificity calculations, which probably leads to overestimation of the false-negative cytology rate. They also reported high-risk human papillomavirus (hrHPV) test positivity in 80% of these case smears, which suggests significantly higher sensitivity compared with cytology. Again, no specificity calculations were made. The careful cytologic evaluation of hrHPV positives that Bulk et al. recommend could best be implemented in a primary HPV screening program combined with cytology triage (25). An experimental screening design is ongoing in the Finnish program (26, 27). A systematic cytology triage with careful independent evaluation phases within the cytology laboratory and in a panel can be recommended based on our experiences from the current study.

Considering the large variation in current cervix cancer burden between countries and in historical trends, regular and comprehensive audits, as recommended in the current quality assurance guidelines for cancer screening programs, should be done to monitor cervical cancer screening programs. To optimize the performance and validity of cytologic screening, register-based feedback and audits could also be included in the certifications and accreditation standards of screening units and laboratories.

No potential conflicts of interest were disclosed.

We thank Kaija Halonen and Sirkku Vuorma from the Mass Screening Registry of the Finnish Cancer Registry for help with the preparation and handling of the register data; Kaija Knuuti, Merja Herttola-Hämäläinen, and Tuomo Timonen from the Department of Pathology at the University Hospital of Helsinki for providing facilities and participation in the rereading; and the cytotechnologists from the screening laboratories that participated in the panel sessions: Päivi Kelkka, Anna-Maija Korhonen, Päivi Häkkinen, Benita Nyqvist, Regina Lääkkö, Eila Hanelius, and Berit Hällfors.

Grant Support: European Commission and the Finnish Cancer Organisation.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.