Higher Fecal Immunochemical Test Cutoff Levels: Lower Positivity Rates but Still Acceptable Detection Rates for Early-Stage Colorectal Cancers

This article is featured in Highlights of This Issue, p. 217

Screening for colorectal cancer (CRC) using guaiac-based fecal occult blood tests (G-FOBT) has been shown to reduce CRC-related mortality (1–3). In recent years, a growing body of literature lends support to the notion that fecal immunochemical tests (FIT) are superior to G-FOBT in CRC screening (4–8). This superiority does not only imply higher participation rates and sensitivity for advanced neoplasia, but also better reproducibility and quality control due to the automated analysis and quantitative test output (9). The quantitative test output allows for adjusting the threshold for the definition of a positive test. This is important because several recent studies comparing G-FOBT and FIT have reported a lower specificity of FIT when a cutoff level of 50 to 100 ng hemoglobin/mL was used (5–8). Once this test is applied in a CRC screening program, a lower cutoff level will result in more screenees being referred for colonoscopy, and due to lower specificity, a higher number of futile colonoscopies. Higher FIT cutoff levels will decrease strain on colonoscopy resources, but might also be associated with more curable CRCs being undetected. To test this hypothesis, a study design is needed in which all FIT-negative individuals undergo the reference test, that is, complete colonoscopy. However, in most population-based screening studies, only FIT-positive individuals undergo colonoscopy (4–7, 10, 11). Although these screening studies reflect the target population for screening, sensitivity cannot be calculated. Specificity can be calculated, but only indirectly and based on less accurate rare disease assumptions (6). Moreover, these studies often have a low yield of CRCs, which restricts the power to stratify these cancers by stage (6, 7, 12, 13). When aiming at CRC mortality reduction, detection of early-stage cancers is much more relevant than detecting late-stage cancers. In a referral population, like in the present study, a higher prevalence of CRC and its precursors will allow for stratification of quantitative FIT results for different phases of the natural history of the disease. We therefore assessed the effect of a higher cutoff level of a quantitative FIT on positivity rates and on detection rates of curable, early-stage CRCs and advanced adenomas in a colonoscopy-controlled population.

Details of study design and of most materials and methods relevant for this study have been published previously in a report on the direct comparison of a FIT and a G-FOBT (8).

All ambulatory subjects over the age of 40 years scheduled to undergo elective colonoscopy from June 2006 to January 2009 at 1 of the 5 participating hospitals were invited to participate in this study. Invitation was either in person by the referring gastroenterologist or through telephone by 1 of 5 research workers stationed at each of the participating centers. Once subjects consented in participation, they received an envelope containing background information on the study, the FIT with extensive instructions, and an informed consent form. When an individual could not be reached by telephone, the same package was sent but with an additional explanatory letter. Two of these 5 participating hospitals are situated in rural areas, another two are large teaching hospitals with an urban population. One of the centers is an academic medical center with a predominantly urban population. In all centers, local Medical Ethics Review Board approval was obtained prior to the start of the study.

All eligible individuals were asked to sample one FIT on stool from a bowel movement on the day prior to colonoscopy. Patients with a documented history of inflammatory bowel disease (IBD), subjects who failed to complete the test and subjects in whom no written informed consent was obtained were excluded from further analysis. We also excluded subjects with incomplete colonoscopies and subjects with inadequate bowel cleansing, as judged by the endoscopist.

The FIT used in the present study is the automated quantitative OC-sensor test (Eiken Chemical Co.). The FIT was sampled from stool produced the day before colonoscopy and bowel preparation had started. Subjects were excluded when the FIT was sampled after initiation of bowel preparation. Illustrated instructions guided the participants to sample their stool ensuring that contact with water and urine was prevented. No restrictions were made with regard to diet during the week in which the stool sample was taken (14). Participants were asked to discontinue anticoagulants and NSAIDs (nonsteroidal anti-inflammatory drugs) 5 days prior to colonoscopy. On the day of colonoscopy, the completed test and the signed informed consent form were handed over to the nursing staff at the endoscopy department. All FITs were stored at −5°C on arrival. Tests were analyzed using the OC sensor MICRO desktop analyzer (Eiken Chemical Co.) according to the manufacturer's instructions (15). Tests were analyzed within 1 week by 1 of the 2 experienced technicians who were unaware of the clinical data. Both technicians received special training for analyzing the tests.

Colonoscopy was the standard of reference for the presence, size, and location of colorectal neoplasia. Colonoscopies were performed or supervised by experienced gastroenterologists. Endoscopists were blinded to the FIT result. Conscious sedation using midazolam was offered to all patients. A complete colonoscopy was defined as intubation of the cecum with identification of the ileocecal valve or appendiceal orifice, or intubation up to an obstructing neoplasm. The results of histopathologic analysis of tissue samples obtained during colonoscopy were the standard of reference for the diagnosis of adenoma or cancer. Surgical resection specimens were used for the standard of reference for CRC staging. If no surgical resection had been performed, the results of histopathologic biopsy specimens were used instead. Adenomas of size 1.0 cm or greater, with any villous features (i.e., tubulovillous or villous adenoma) or high-grade dysplasia, were considered advanced adenomas (16, 17). Advanced neoplasia included all cases of CRC and all advanced adenomas. Colorectal carcinomas were staged according to the AJCC (American Joint Committee on Cancer) cancer staging manual (18). Early-stage CRC was defined as AJCC stage I or II, whereas late-stage CRC was defined as AJCC stage III or IV. Because the ultimate goal of screening is the detection of early stages of diseases, we defined screen-relevant neoplasia as one or more advanced adenoma(s) or early-stage CRC (8, 19). If multiple lesions were present, classification was based on the most advanced lesion found.

Taking colonoscopy as the reference test, sensitivities and specificities of FIT at 6 cutoff levels were calculated for the following colonoscopy outcomes: (i) the presence of CRC; (ii) the presence of early-stage CRC; (iii) the presence of advanced adenoma; (iv) the presence of screen-relevant neoplasia; and (v) the presence of advanced neoplasia. The sensitivity is calculated as the proportion of positive test results in patients with the colonoscopy outcome under consideration. The specificity is calculated as the proportion of negative test results in patients with an outcome less severe than the colonoscopy outcome under consideration. Note that, therefore, the same specificity results from choosing either outcome 1 (all stages of CRC) or outcome 2 (early-stage CRC), and from choosing either outcome 3, 4, or 5. For dichotomizing the FIT results, we used cutoff levels of 50, 75, 100, 125, 150, and 200 ng hemoglobin/mL, which are levels frequently used in FIT studies (7, 20, 21). The calculations were repeated for the subgroup of patients that are considered at low risk for colonic neoplasia (procedure indications: abdominal pain, constipation, and screening colonoscopy in average risk individuals) as well as for the high-risk subgroup separately.

We used receiver operator characteristic (ROC) curve analysis, including calculation of the area under the curve (AUC) with 95% CIs to evaluate the relation between the quantitative FIT outcome and (i) the presence of CRC; (ii) the presence of early-stage CRC; (iii) the presence of advanced adenoma; and (iv) the presence of screen-relevant neoplasia. All analyses were performed with SPSS for Windows Version 15 (SPSS Inc.).

Overall 2,525 individuals who underwent colonoscopy sampled a FIT. In total, 380 individuals were excluded (Fig. 1). The mean age of the 2,145 individuals that were included for final analysis was 61.8 years (range = 40–89 years) and 53.8% of these were female. Colonoscopy was performed because of gastrointestinal symptoms in 1,109 individuals (51.7%), whereas screening or surveillance for CRC was the indication for colonoscopy in 955 asymptomatic individuals (44.5%). Of 81 individuals (3.8%), the primary indication remained unspecified (Table 1).

CRCs were found in 79 individuals (3.7%). Of these CRCs, 38 (48.1%) were classified as early stage (AJCC stage I or II) and 36 (45.6%) were classified late stage (AJCC stage III or IV). For 5 rectal cancers (6.3%), stage could not be determined accurately due to the effects of neoadjuvant radiotherapy. In 236 individuals (11.0%), at least one advanced adenoma was found. This resulted in 315 individuals with advanced neoplasia (either advanced adenoma or CRC) and 274 individuals with screen-relevant neoplasia (either advanced adenoma or early-stage CRC).

The overall FIT positivity rates at the different cutoff levels varied from 16.5% (n = 354 at cutoff ≥50 ng/ml) to 10.2% (n = 218 at cut off ≥200 ng/ml). Table 2 shows the test characteristics of FIT at different cutoff levels to detect CRC, early-stage CRC, late-stage CRC, advanced neoplasia, screen-relevant neoplasia, and advanced adenomas. Table 3 summarizes the sensitivities and positivity rates for early-stage CRC and for screen-relevant neoplasia.

The AUC of the ROC curve for the detection of CRC (n = 79) was 0.93 (95% CI: 0.89–0.96). For the detection of early-stage CRC (n = 38), an AUC of 0.89 was found (95% CI: 0.82–0.95). When all screen-relevant neoplasia were considered (n = 274), the AUC was 0.72 (95% CI: 0.68–0.76). The AUC for the detection of advanced adenomas separately (n = 236) was 0.69 (95% CI: 0.65–0.73).

Sensitivities of FIT for CRC, early-stage CRC, and screen-relevant neoplasia were compared between indication groups that were considered at low risk for colonic neoplasia versus at high risk. Patients with procedure indications like abdominal pain, constipation, and screening colonoscopy in average risk individuals were considered to belong to a low-risk population (n = 374). The remaining procedure indications were considered to reflect a high risk population (n = 1,771; Table 1). Sensitivity and the yield of screen-relevant lesions in these 2 populations are shown in Table 4.

In the present study, test performance of one of the most commonly used FITs was evaluated at different cutoff levels in a large cohort of individuals undergoing colonoscopy. It was found that by increasing the cutoff level specificity increased substantially, whereas the effects on detection rates of curable, early-stage CRCs were only limited.

Although many other aspects have to be taken into account when deciding on the most suiting cutoff level, this study has its focus on sensitivity and specificity. In general, the FIT showed to have good test characteristics for detecting both CRC and early-stage CRC, as reflected by the AUC in the ROC curves. Adjusting the cutoff level from ≥50 to ≥200 ng/mL resulted in a substantial decrease in the number of positive tests (16.5%–10.2%). Compared with a cutoff level of ≥50 ng/mL, 2 early-stage cancers would have been missed at a cutoff level of greater than 200 ng/mL. In fact, from a cutoff level of ≥125 ng/mL upward, no further decrease in sensitivity was found. Specificity, however, increased from 86.4% to 92.8% with increasing cutoff levels. Focusing on all screen-relevant neoplasia, 47.1% were detected with the lowest cutoff level of 50 ng/mL or greater, whereas the highest cutoff level of 200 ng/mL or greater yielded only 37.2% of all screen-relevant lesions.

Consequences of these findings depend on the setting in which FIT is applied. The choice for a higher FIT threshold may be particularly relevant when a screening program is to be implemented, like is planned for the Netherlands (22). The Dutch Health Council advised to start screening at a cutoff level of 75 ng/mL, even though using 50 ng/mL might be more cost-effective (22). However, current colonoscopy capacity is insufficient to cope with positive screenees at this cutoff level. A higher FIT cutoff level will limit the number of colonoscopy referrals. In the first round of a screening program, both early- and late-stage CRCs, so called prevalent cancers, will be detected in a range consistent with their respective prevalences in a screening-naive population. So, in the first round, yield and thus strain on the health care system, will be inflated by the prevalent advanced-stage CRCs. In later rounds of screening, however, less advanced CRCs will be left in the population and the performance characteristics of the screening program will largely depend on the potential to detect early-stage CRCs, that is, incident cancers. In this respect, it is highly relevant to know that increasing the cutoff level to 200 ng/mL or greater has a relatively small effect on the sensitivity to detect early-stage cancers when starting a CRC screening program. The lower positivity rate and the higher specificity will result in less referrals for colonoscopy with an acceptable decrease in detection rates. When the prevalence of target lesions would decrease in later screening rounds, cutoff levels can easily be adjusted to lower values to achieve a more sensitive program. The miss rate for advanced adenomas at a higher cutoff level is somewhat higher (70% at >200 ng/mL vs. 59% at 50 ng/mL). Given the natural history of the disease, an advanced adenoma that would be missed in the initial screening round would have multiple opportunities to be detected in a consecutive round, either still at the stage of an advanced adenoma or as an early-stage cancer.

The colonoscopy-controlled referral population used in the present study has 2 advantages compared with a screening population. Firstly, in most FIT studies to date, only individuals with a positive test underwent subsequent colonoscopy. This precluded the determination of sensitivity, false-negative rates and thus specificity of the investigated tests. The present study design provides accurate data on direct sensitivity of the FIT at different cutoff levels. Secondly, the referral population contained a higher number of individuals with CRC or advanced adenomas compared with an average risk screening population. Consequently, FIT results could be stratified by stage of the disease. More precise data on sensitivity and specificity, that is with smaller CIs than data from screening studies in which there is a lower prevalence of target lesions, could be calculated (4, 6, 7, 20, 23). In screening studies, specificities are calculated on the basis of rare disease assumptions. This may lead to overestimation of specificity (24). Although sensitivity and specificity are characteristics of a diagnostic test and are not dependent on the prevalence of disease, specificity can still be underestimated in the present study because the subjects are at higher risk of other potentially bleeding disorders than the general population (25). When comparing data on specificity for advanced neoplasia of the present study with those reported in screening study designs using the same FIT, the present series shows lower specificities (89.9%–95.8% in the present study compared with 95.5%–98.8% in other studies with cutoff levels increasing from ≥50 ng/ml to ≥200 ng/ml; refs. 7, 20). Interestingly, in another colonoscopy-controlled study performed in a referral population with a smaller sample size, specificities were comparable with the present findings (21). These different specificities probably reflect the range of true specificity of the OC Sensor.

The higher positivity rate of the FIT and higher prevalence of advanced neoplasia in the present referral population make it impossible to extrapolate the positive and negative predictive values (PPV and NPV) of this study to a screening population. However, by applying Bayes' theorem, the sensitivity and specificity from this study can be combined with observed prevalences of CRC and advanced adenomas found in the general population to estimate the NPV and PPV in the general population (26). Although these computed values for NPV and PPV should be interpreted with caution, it allows us to explore the effect of increasing cutoff levels. The prevalence of CRC found in Dutch screening studies is 0.8% and advanced adenomas are found in 6.7% (22). In the time period from 2003 to 2007, 54% of all newly diagnosed CRC patients presented with early-stage disease (27). Increasing the cutoff level of FIT from ≥50 ng/mL to ≥200 ng/mL hardly affected NPV for early-stage CRC (99.9%). The PPV for detection of early-stage cancer, however, increases substantially from 2.6% to 4.5%. The number needed to scope reduced significantly by increasing the cutoff level to 200 ng/mL or greater, resulting in a 42% decrease in required colonoscopies and only a 6% reduction in detection rates of early-stage CRCs.

An important issue is whether the present findings can be generalized to a screening population. The use of a referral population to evaluate a screening test carries the risk of introducing spectrum bias. Spectrum bias refers to the situation that the spectrum of the disease phenotype differs from that in the population in which the test ultimately will be applied (28). This might lead to overestimation of sensitivity. An ultimate answer to this question can only come from a colonoscopy-controlled screening population. To accrue a similar number of cancers in such a study design as in the present study would require a very large sample size which might frustrate such a study design. According to the number of CRCs found in the 2 large screening trials in the Netherlands, such a study should invite 30,275 average risk individuals for FIT screening to obtain the same CRC yield as in the present study (6, 7). This sample size is based on the assumption of a 60% to 62% participation rate for FIT and 84% to 95% compliance to colonoscopy after a positive FIT (6, 7).

Three other lines of evidence provide indications that the effect of spectrum bias in the present study may be limited. Firstly, when comparing sensitivities of FIT for screen-relevant neoplasia in patients from the present study population who could be considered to have a low risk for colonic neoplasia to those that would be at higher risk, only minor differences in sensitivity were found (29,30). Secondly, spectrum bias could also be explained by a different tumor stage distribution in the referral population compared with those in a screening population. A comparison of CRCs from a referral and a screening population indeed revealed a higher prevalence of advanced cancers. After stratifying for T stage, however, no differences in FIT results were found between the screening and referral population (31). Thirdly, test characteristics found in screening studies remain debatable, as 70% of screen detected CRCs in the British NHS National Bowel Cancer Screening Program appeared to be symptomatic (32).

In conclusion, in the present study higher cutoff levels turned out to result in only a 5.3% decrease in detection rate for early-stage CRC, whereas at the same time substantially reducing the number of positive FITs with 6.3%. Overestimation of sensitivity, however, due to potential spectrum bias can not be ruled out completely. When a higher cutoff level would be used as a first step preceding colonoscopy in a CRC screening program, lower numbers of colonoscopies would be required. This may facilitate the appropriate allocation of available resources. The lower detection rates of advanced adenomas may be overcome by the fact that these lesions are likely to be detected in a later screening round while probably still being at a stage of disease at which death from CRC can be prevented.

J. Terhaar sive Droste: commercial research support from Nycomed BV Hoofddorp. There were no conflicts of interest for the authors of this study.

We gratefully acknowledge the excellent technical assistance of Edwin van Hengel. The assistance of all medical and nursing staff who cared for the patients in the study is much appreciated.

This research project was supported by an unrestricted grant of Nycomed BV, Hoofddorp to “The Amsterdam Gutclub,” the Netherlands. This company had no influence on any aspect relevant to this study. The OC sensor MICRO desktop analyzer was provided by Eiken Chemical Co. This company had no influence on any aspect relevant to this study.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.