Demand for Colonoscopy in Colorectal Cancer Screening Using a Quantitative Fecal Immunochemical Test and Age/Sex-Specific Thresholds for Test Positivity Free

Screening for colorectal cancer with a quantitative fecal immunochemical test (FIT) has been increasingly used in population-based screening for colorectal cancer (1). Most of these programs use population-average thresholds of fecal hemoglobin (f-Hb) concentration to define test positivity (2, 3), while some adjustments have been made by the receiver operating characteristic (ROC) curve (4, 5) or cost-effectiveness analyses (6, 7). It should be noted that age/sex-specific variations in the f-Hb have been shown in previous studies (8–12).

In population-based screening for colorectal cancer with FIT, the thresholds could be varied by age and gender because interval cancers that may be missed at previous screen when a universal threshold is used predominate in females (13) and the elderly (13, 14).

Nonetheless, as subgroup-specific or individual-specific optimized thresholds have been barely addressed by using different criteria on sensitivity, specificity, or both, the impact of age/sex-specific threshold on the accuracy of FIT remains unknown.

In terms of the viewpoint of resource allocation, the change of the threshold may also affect test positivity rate (colonoscopy workload).

Assessing the relationships between the colonoscopy workload and the selected thresholds, making allowance for the capacity of colonoscopy, plays a crucial role in determining the optimal thresholds of age–gender groups. It is therefore interesting to assess their impact on colonoscopy demand using different criteria on sensitivity and specificity by using the newly proposed colonoscopy-capacity-adjusted method.

Using the large empirical data from Taiwanese nationwide screening program with FIT (15), which provide an opportunity to collect f-Hb concentrations in all program participants and information on interval cancer for several years of follow-up after the last test negative screen to achieve the following two objectives, the first objective of this study was to assess the impact of age/sex-specific threshold on the accuracy of FIT by using four approaches; the ROC-derived threshold, targeted sensitivity, targeted false-positive rate, and a new colonoscopy-capacity-adjusted method integrating colonoscopy workload in comparison with the conventional “universal” method (the method for the current program). The second objective was to evaluate the effect on colonoscopy workload of the four methods with and without age and sex adjustment compared with the universal method.

Data used for the following analyses were derived from the Taiwanese nationwide screening program for colorectal cancer with FIT. The details on the screened population have been described in full elsewhere (16, 17). In brief, a single specimen of feces was collected directly from participants aged between 50 and 69 years who attended the biennial nationwide colorectal cancer screening program with FIT during the period of January 1, 2004, and December 31, 2009, and then were delivered test devices provided by two manufacturers, namely, OC-Sensor (Eiken Chemical Company Ltd) and HM-JACK (Kyowa Medex Company Ltd.). Each screening setting at city level used either OC-Sensor or HM-Jack. A total of 1,160,895 subjects accounting for 21.4% of the eligible population participated in this screening program. Complete information on f-Hb was available in 723,113 (78%) of those who used the OC-Sensor test and 202,658 (22%) who used the HM-Jack test at their prevalent screen. In this study, only participants who used OC-Sensor have been analyzed.

The sample collection device for OC-Sensor consisted of a probe that holds approximately 10 mg of feces and 2 mL of hemoglobin stabilization buffer. Participants were requested to return the specimens for testing immediately after using the device. Quantitative FIT testing was performed at qualified laboratories, which is supervised by Taiwan Society of Laboratory Medicine to provide these laboratories with hemoglobin solutions and hemoglobin-spiked, stool-like matrix samples to test occult blood using both FITs every half a year.

The study design is shown in Supplementary Fig. S1. The hemoglobin concentration threshold used for the OC-sensor was 20 μg Hb/g feces (100 ng Hb/mL test buffer) and is referred to as the “universal” threshold. This threshold was chosen by the Health Promotion Administration on the basis of published data and manufacturer recommendations (17). It should be noted that the manufacturer of the OC-Sensor only endorses analytical reliability for f-Hb concentrations above 10 μg Hb/g. In addition to reporting a positive or negative FIT result, concentration data were stored in the central database. Participants with positive tests were referred for confirmatory investigation by using either total colonoscopy or sigmoidoscopy and a barium enema. All interval cancers that may be missed at previous screen were identified using the nationwide cancer registry (17) throughout the follow-up period from the entry until the end of 2009. The cancer registry is a nationwide program with a coverage rate of 98.6% and an accuracy greater than 99% (18).

In addition to the preselected universal threshold, four methods of determining FIT thresholds were used to estimate the colonoscopy workload for the overall group and the age/sex-specific subgroups. The first method used the ROC curve to identify the optimal threshold for distinguishing subjects diagnosed as colorectal cancer from subjects free of colorectal cancer by using the Youden index (19). Despite widespread use of the ROC curve, choice of the “optimal” threshold for clinical practice depends the relative clinical importance of sensitivity and specificity, which have a reciprocal relationship. Two alternative methods for determining threshold were therefore examined and were based on targeted specificity (false-positive rate) or targeted sensitivity. The targeted sensitivity method used 70%, 75%, and 80%. Eighty percent was considered desirable because the pooled sensitivity of FIT for colorectal cancer was estimated as 0.79 (0.69–0.86) from 19 studies by a meta-analysis (20). The targeted false-positive rates of 5%, 7.5%, and 10% were used for specificity and 5% was preferred. An additional new approach to determining the optimal threshold used colonoscopy-capacity-adjustment, which was based on the relationships between colonoscopy workload and sensitivity or specificity (from the false-positive rate) and used the inflection point of the curve as the maximum allowable capacity of colonoscopy, see Fig. 1. The details of the procedure are given in Supplementary Part I and Supplementary Table S1. Note that although this curve is mainly used for identifying inflection point of the curve for the overall group and age/sex-specific subgroups, such relationships are useful for determining demand (positive rate of FIT or the proportion of colonoscopy) for three other methods.

For each method, the so-determined threshold would yield the positivity rate that in turn determines the colonoscopy workload. The colonoscopy workloads were compared for the four methods using age/sex-specific thresholds. We compared the unadjusted estimates of colonoscopy workloads with the corresponding age–sex-adjusted estimates for each method. All data management and statistical analyses were performed with SAS software (version 9.4; SAS Institute Inc.).

The basic characteristics of participants including their distribution of f-Hb and screen-detected and interval cancers and demographic characteristics are presented in Table 1. The mean f-Hb concentration (μg Hb/g) was higher in males (7.74, IQR = 1.6) than in females (4.58, IQR = 1.2) and in younger participants ages between 50 and 59, and it was also higher in males (9.76, IQR = 2.8) than in females (6.42, IQR = 2.2) in older participants ages between 60 and 69. The f-Hb concentration increased with age in both males and females. At the currently used “universal” threshold of 20 μg Hb/g feces, positivity rates were higher in males (4.3% for ages 50–59, 6.0% for ages 60–69) than females (3.0% for ages 50–59, 4.0% for ages 60–69) and higher in the older age groups. The sensitivity and specificity for cancer at the universal threshold were 78.7% (76.9%–80.4%) and 96.2% (96.1%–96.3%), respectively. Sensitivity was lower in females (79.8% for ages 50–59, 74.5% for ages 60–69) than males (80.7% for ages 50–59, 79.7% for ages 60–69) and the disparity in sensitivity between the age extremes was larger in females than males. The lowest sensitivity was seen in the elderly females. The lower specificity was seen in males (95.9% for ages 50–59, 94.4% for ages 60–69). Specificity was highest in females and younger participants (97.1% for ages 50–59, 96.2% for ages 60–69).

As described earlier, we propose four alternative methods for determining the threshold of f-Hb used in population-based screening for colorectal cancer using FIT. These methods used are the ROC-derived threshold method, targeted sensitivity method, targeted false-positive rate method, and the colonoscopy-capacity-adjusted method. The results of the bottom panel of Table 2 show the optimal thresholds and positive rates for the overall group determined by the four methods.

The relationships between colonoscopy workload and sensitivity or specificity (from the false-positive rate) are presented in Fig. 1. A nonlinear relationship was observed between sensitivity and colonoscopy workload rising disproportionately to sensitivity (80%) at 16 μg Hb/g, whereas a linear relationship was noted for specificity. The sensitivity and specificity for cancer with colonoscopy-capacity-adjusted method were 79.5% (77.7%–81.2%) and 95.5% (95.4%–95.6%), respectively. At this “colonoscopy-capacity-adjusted” threshold, the positivity rate was 4.92% (see Table 3).

Results of the first method using ROC-derived threshold by age group and sex are shown in Fig. 2. In men, the area under the curve in age group 50 to 59 years was 0.92 (0.90–0.94) and 0.90 (0.89–0.92) in 60 to 69 years (Fig. 2A). Similar findings were noted in women (Fig. 2B). Performance was better in the younger age group. Age/sex-specific thresholds determined by the Youden index from ROC curves are presented in Table 2. The optimal thresholds were lower in younger males (12 μg Hb/g) and older females (10 μg Hb/g) than the current universal threshold (20 μg Hb/g).

The results of age/sex-specific thresholds using targeted sensitivity (80%, see Materials and Methods) are also shown in Table 2. Age makes a larger contribution to the determination of thresholds than does sex using targeted sensitivity compared with the ROC-derived threshold method. The threshold was lower in elderly females (10 μg Hb/g) compared with the universal threshold (20 μg Hb/g).

The thresholds with the application of targeted false-positive rate with 5% (see Table 2) were lower for younger males (15 μg Hb/g) and for all females (12 μg Hb/g for ages 50–59, 16 μg Hb/g for ages 60–69) compared with the universal threshold, but, interestingly, to reduce the false-positive rate in older males, a higher threshold (24 μg Hb/g) was required.

Table 2 also shows age/sex-specific thresholds and their corresponding estimates of sensitivity and specificity that are determined by the inflection points of each curve presented in Supplementary Fig. S2 in the light of the proposed colonoscopy-capacity-adjusted method. The thresholds determined by this new method did not show a large variation across different age and sex differences compared with other methods.

Table 3 presents the proportion of screening participants, unadjusted and age/sex-adjusted, with a positive test subject for whom colonoscopic follow-up is recommended by each of the four methods. The details of demand for colonoscopy by age/sex-specific results are given in Supplementary Table S2 and also described in the Supplementary Methods (Part II).

Demand for colonoscopy with age–sex-adjusted estimates was lower than the unadjusted estimates using either the ROC derived threshold method or targeted sensitivity at 80% (Table 3). Using the Youden index of the ROC, the unadjusted estimate for a positive test and thus colonoscopy was 6.48% (46,829 of 723,113 screened) whereas age- and sex-adjusted yielded 5.56% (40,219 of 723,113). Varying FPR, demand was higher for 7.5% and 10% but similar for 5%. These observations show that using age- and sex-specific thresholds based on the colonoscopy-capacity-adjusted method, colonoscopy workload was lower than that achieved with the ROC derived threshold method, 80% targeted sensitivity, or 5% targeted FPR. However, despite lower demand after adjusting for sex and age in the four main models, when estimates are compared with the universal threshold workload estimate of 4.0%, the age–sex-adjusted estimates increased the proportion of follow-up colonoscopies in all four methods. For the ROC-derived threshold method, age–sex adjusting of the f-Hb concentration resulted in a colonoscopy demand of 5.56% representing a 39% increase in estimated workload. Similarly, workload increased for 80% targeted sensitivity (80% targeted sensitivity to universal threshold representing a 25% increase), 5% targeted FPR (5% targeted FPT to universal threshold representing a 27% increase), and colonoscopy-capacity-adjusted method (colonoscopy-capacity-adjusted method to universal threshold representing a 17% increase).

Our empirical data show that with a universal threshold, sensitivity and specificity vary with age and gender and might justify the use of age/sex-specific thresholds. At any chosen threshold concentration with the universal approach, more females and older people will be considered for referral and further investigation. Use of a single threshold hemoglobin concentration in population-based FIT screening programs with qualitative or quantitative FIT is not an optimal strategy because it gives different rates of false-negative and false-positive cases in different age/gender population subgroups. This conclusion is also evident from the ROC-derived threshold method to find age/sex-specific optimal thresholds with Youden index, targeted sensitivity, and 5% false-positive rate. Under constrained colonoscopy capacity, the age/sex-adjusted estimates were generally lower than if unadjusted as seen in Table 3, which supports further the need for age/sex-specific thresholds.

We have examined four methods using sensitivity and specificity criteria, starting with the universal threshold and then using an optimized threshold with the tradeoff between sensitivity and specificity based on the ROC-derived threshold method, targeted sensitivity, and targeted false-positive rate. These approaches depend on which is considered most important, sensitivity or specificity. Optimization of accuracy of disease screening using sensitivity and specificity has usually been conducted using the ROC curve. By means of the Youden index, whether or not it was adjusted for the optimal age and sex thresholds, the thresholds were found to be lower than that using the universal method and therefore demand more colonoscopy resource. With limited colonoscopy capacity, determination of optimal threshold using an ROC method might be unacceptable.

Rozen and colleagues found that for screening average-risk subjects 95% specificity for colorectal cancer was achieved by using a one-sample quantitative FIT with a 20 μg Hb/g threshold (21). Our study shows that the optimal threshold changes when considering age and sex. The same observation is made when the targeted sensitivity is used.

Since age/sex-specific thresholds, determined by three methods other than the universal threshold, were generally lower than the universal threshold, these alternatives will demand increased colonoscopy. Our findings show that 24.8% more colonoscopies are needed when targeting 80% sensitivity. An extra 39.0% colonoscopy workload is needed when using the ROC derived threshold method and an extra 26.9% at 5% false positivity. The determination of age/sex-specific thresholds also needs to take colonoscopy capacity for into consideration. This development been achieved by using the colonoscopy-capacity-adjusted method targeting an efficient sensitivity at an acceptable false-positive rate.

The threshold chosen determines the colonoscopy workload. In Scotland, a 2% positivity rate was suggested as threshold for FIT screening because of limited colonoscopy resources (8). However, if our threshold were to be set according to 2.0% positive rate, the sensitivity of OC-sensor for cancer is lowered to 61% and 60% for 50 to 59 and 60 to 69 years groups in males according to Supplementary Fig. S2. If such low sensitivity is unacceptable, then more colonoscopy resource will be necessary.

When targeting a desired sensitivity, the number of colonoscopies required was lower at 75% sensitivity than when using the current universal threshold. Importantly, we showed a nonlinear relationship between sensitivity and colonoscopy workload with workload rising sharply at 80% sensitivity, while there was a linear relationship for specificity. When sensitivity is above 80% there is a substantial increase in demand for colonoscopy. The proposed colonoscopy-capacity-adjusted method optimizes the demand for colonoscopy and also gives comparable sensitivity reported from the meta-analysis of previous studies (20) and an acceptable false-positive rate.

The optimal age/sex-specific derived thresholds see a small gain of sensitivity but have sizable impact on colonoscopy workload regardless of which method is applied. The demand for colonoscopy (17%) using the colonoscopy-capacity-adjusted method compared well with the three other methods which rely on sensitivity and specificity criteria.

It should be noted that our test positivity rates were lower than those reported in previous studies, although they still varied by age and sex. The lower positivity rate is probable reflects lower colorectal cancer incidence rate in the Taiwanese population compared with other Western countries. The observations on age/sex-specific thresholds in our study may not be generalized to other countries but the proposed method, particularly the colonoscopy-capacity-adjusted approach, is applicable to various countries with different baseline incidence rates of colorectal cancer.

In conclusion, we have investigated four different methods, with and without colonoscopy capacity constraints, to determine the optimal age/sex-specific thresholds of f-Hb concentration in population-based colorectal cancer screening with FIT. While age/sex-specific optimal thresholds improve the performance of FIT, the positive rates determined by them increase the demand of colonoscopy. The colonoscopy-capacity-adjusted method reduced the impact of optimized age/sex-specific thresholds on colonoscopy workload.

G.P. Young reports receiving commercial research funding from Eiken Chemical Company. No potential conflicts of interest were disclosed by the other authors.

Conception and design: S.L.-S. Chen, G.P. Young, S.Y.-H. Chiu, Y.-C. Lee, H.-M. Chiu, S.-T. Chiou, H.-H. Chen

Development of methodology: S.L.-S. Chen, Y.-C. Lee, H.-H. Chen

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): S.Y.-H. Chiu, Y.-C. Lee, H.-M. Chiu, S.-T. Chiou, H.-H. Chen

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): S.L.-S. Chen, C.-Y. Hsu, A.M.-F. Yen, G.P. Young, J.C.-Y. Fann, Y.-C. Lee, S.-T. Chiou, H.-H. Chen

Writing, review, and/or revision of the manuscript: S.L.-S. Chen, C.-Y. Hsu, G.P. Young, S.Y.-H. Chiu, J.C.-Y. Fann, H.-M. Chiu, H.-H. Chen

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): C.-Y. Hsu, S.Y.-H. Chiu, Y.-C. Lee, S.-T. Chiou

Study supervision: H.-H. Chen

This work was supported by the Ministry of Science and Technology, Taiwan (MOST 103-2118-M-002-005-MY3).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.