Development and Validation of a Clinical Score for Predicting Risk of Adenoma at Screening Colonoscopy

Colorectal cancer is the second leading cause of cancer-related death in the United States (1). Current guidelines recommend initiating screening for asymptomatic men and women at age 50, using a menu of screening options (2). Most colorectal cancers are thought to arise from precursor lesions called adenomas (3–5). The 2008 U.S. Multi-Society Task Force screening guidelines emphasized that the primary goal of screening should be prevention of colorectal cancer by detection and removal of asymptomatic adenomas (2). Recent guidelines on colorectal cancer screening by the American College of Physicians recommend that individualized risk assessment for risk of colorectal cancer should be performed in all adults, and a screening modality should be selected based on their risk (6).

Several demographic and clinical risk factors for harboring adenomas in asymptomatic men and women age 50 and over have been identified in large cohort and case–control studies and include increasing age, male sex, race, and a family history of colorectal cancer in a first-degree relative (7–10). Other identified risk factors include higher body mass index (BMI), current smoking, and heavy alcohol use (11–18). However, there is a lack of clinical tools to reliably risk-stratify men and women based on these factors. Several authors (19) have reported developing and validating risk scores for advanced neoplasia. However, there are no such tools for risk of adenomas, or that have been developed or validated in a U.S. cohort. Such clinical risk-stratification tools, or risk scores, are used not only for breast cancer (20) but also in several other areas of medicine—such as for stratifying individuals by risk of heart disease (21), for organ allocation (MELD score; ref. 22), for severity of liver disease (Child–Pugh score; ref. 23), and for hospital mortality (APACHE II; ref. 24)—where they have diagnostic or prognostic value.

An adenoma risk score would identify the absolute risk for an individual for harboring advanced neoplasia. Based on their absolute risk, individuals could be stratified into low- and or high-risk groups, and those in the high-risk group could be prioritized for screening colonoscopy, whereas those in low-risk groups can be offered a choice of modalities of screening including colonoscopy. Given the limited capacity for colonoscopy in the United States, along with its cost and complications, the ability to risk-stratify men and women adequately would be a first step in improving resource utilization, allocating capacity, and reducing costs and complications. The objective of our study was to develop and validate a risk-prediction model by using data from a randomized multicenter clinical trial to combine the risk factors associated with adenomas into an adenoma risk score among men and women undergoing colonoscopic screening.

We used data from phase I and II of the National Colonoscopy Study (NCS), a randomized trial of colonoscopy screening for model development and validation. The study, comparing the clinical results of colonoscopic screening compared to usual care, was conducted in two phases, between 2000 and 2002 (phase I) and between 2004 and 2007 (phase II) on a general population of men and women at three clinical centers: Group Health Cooperative, a managed care organization in the Puget Sound area of Washington State; a collaboration of the University of Minnesota and Minnesota Gastroenterology, a large group practice in Minneapolis, MN; and a wellness clinic for underserved and minorities at Louisiana State University in Shreveport, LA. The study sites were chosen to represent different areas of the country and different healthcare delivery systems. The study coordinating center was Memorial Sloan Kettering Cancer Center (MSKCC), and the central pathology center was at the Department of Pathology, Boston Medical Center (Mallory Institute of Pathology), Boston University.

Men and women age 50 years and older (40 years and older for Louisiana State University) were invited to participate. Eligibility criteria were as follows: no personal history of colorectal cancer, familial adenomatous polyposis or inflammatory bowel disease; no prior colonoscopy (phase I and II) and no prior flexible sigmoidoscopy in the previous 5 years (phase II); no serious active comorbidities such as myocardial infarction, congestive heart failure, active treatment for cancer, or on anticoagulation; no currently implanted cardioverter/defibrillator; and ability and willingness to provide informed consent. After giving consent, those eligible were randomized to colonoscopy or usual care. For the purposes of this study, we restricted our analysis to the cohort at least age 50 undergoing screening colonoscopy in phase I or II. All colonoscopies were performed by gastroenterologists with expertise in endoscopy and clinical research. Colonoscopies were performed using standard preparations and conscious sedation, with removal of all polyps. The polyps underwent histopathologic review locally and by the study pathologist (M.J. O'Brien) at the central pathology center.

Participants were asked to complete detailed questionnaires on various demographic, dietary and lifestyle factors, and on personal and family history of cancer before or shortly after their colonoscopy. The family history data were entered into a database and linked to a pedigree drawing program. Each pedigree was reviewed by a Genetics Review Committee at MSKCC to identify participants with possible hereditary non polyposis colon cancer or familial adenomatous polyposis, who were subsequently counseled to see their physicians and discuss colorectal cancer screening.

For the purpose of risk-score derivation and validation, only those subjects over the age of 50 at time of colonoscopy with a complete colonoscopy to the cecum and adequate bowel preparation and who completed all the relevant items on the baseline questionnaire were included.

The outcome variable was one or more adenomas or colorectal cancer found by the screening colonoscopy. Because the analysis was predictive rather than causal, we restricted our analysis to individuals that had complete information for variables of interest. The independent variables were age at colonoscopy, sex, race, BMI, smoking history, current use of aspirin or NSAIDS at least once per week for a year or more, history of colorectal cancer in at least one first-degree relative, and history of colorectal cancer in at least one second-degree relative (if no first-degree relative is affected). The effects of these variables, along with interactions selected before model fitting, were estimated by logistic regression. Likelihood ratio tests, combining the main effects with the interaction effects, measured the overall effect of variables involved with interactions. In addition to the listed variables, effects of the three NCS clinical centers on the risk of adenomatous polyps were estimated in a separate regression. The receiver operating characteristic curve (ROCC) was estimated with the convex hull approach (25), and the area under this curve (AUROCC) was computed. To mitigate bias induced by such reuse of the data, the bootstrap method for estimation of prediction error was applied (26). Because the adjustments are based on bootstrap sample means of the parameter estimates, 200 iterations of bootstrap resampling were used. Confidence intervals (CI) for the adjusted estimates were produced by a second, outer bootstrap applied to the entire estimation and adjustment process, yielding a double bootstrap. To obtain adequate precision for the CIs, the outer bootstrap consisted of 2,000 iterations.

The risk score developed on phase I participants was applied to phase II participants. To verify the bias adjustment procedure nonparametrically, another 2,000-iteration bootstrap was applied to estimate the difference between the adjusted AUROCC from phase I and the AUROCC of the same risk score applied to phase II. This difference was not statistically significantly different from zero, establishing the adequacy of the bias correction. A new logistic regression model was then fit using combined phase I and II data, and the same double bootstrap was applied to estimate the final adjusted AUROCC and CI.

Deciles of predicted probabilities were plotted against observed relative frequencies of participants with adenomas to examine the calibration of our model. The cumulative distribution of estimated risk was plotted to determine proportions of the population falling below any particular risk.

In phase I (derivation cohort), a total of 700 men and women were enrolled at the three sites, of whom 691 filled out some part of the baseline questionnaire, and 622 underwent a colonoscopy. One person who underwent colonoscopy did not fill out the baseline questionnaire, so 621 individuals had both a colonoscopy and a baseline questionnaire. Of these, 541 individuals remained in the final derivation cohort after excluding those 40 to 49 years old (n = 56) and those with missing information on their baseline questionnaire on one or more variables included in the model (n = 34). A few individuals age 49 at randomization but 50 at colonoscopy were included.

In phase II (validation cohort), 1,763 were enrolled, of whom 1,693 were 50 years or older and 1,450 filled some part of the baseline questionnaire and underwent a colonoscopy. Of these, 1,334 had complete information on baseline variables and colonoscopy, and were included in the validation analyses. The demographic and baseline characteristics of the derivation and validation cohorts are presented in Tables 1 and 2.

Of the 541 individuals in the derivation cohort, one or more adenomas were found in 106 (19%) individuals. One or more advanced adenomatous polyps (villous histology, size larger than 1 cm, high-grade dysplasia, or cancer) were found in 33 participants (6%). Of the 1,334 individuals in the validation cohort, one or more adenomas were found in 307 (23%) individuals, and one or more advanced adenomas were found in 76 (5%) individuals. The histopathology findings at colonoscopy are presented in Table 3 for the individuals in the derivation and validation cohort included in the analyses.

The adjusted AUROCC for the phase I derivation model was 0.67 (95% CI, 0.61–0.74). That risk score applied to phase II resulted in an AUROCC of 0.61 (95% CI, 0.59–0.65). The bootstrap 95% CI for the difference of these two AUROCCs was (−0.13, +0.01), which was not significantly different from zero. Thus, we combined phase I and II participants and re-estimated and bias-adjusted the risk score using the entire sample.

In multiple logistic regression, we found age (likelihood ratio test for overall contribution to the model, P < 0.001), male sex (P < 0.001), BMI (P < 0.001), family history of at least one first-degree relative with colorectal cancer (P = 0.036), and smoking history (P < 0.001) to be individually associated with risk of harboring one or more adenomatous polyps in the entire cohort. The effects of clinical centers were examined in a separate regression and found to be insignificant. After expanding the model and adding interaction terms, the fitted values for all coefficients are illustrated in Fig. 1.

The unadjusted, all-data ROCC based on our model is plotted in Fig. 2. The AUROCC was 0.64 (95% CI, 0.60–0.67). The predicted probability ranges from 0.03 to 0.7. For example, the predicted risk score for a 50-year-old female with a BMI of 20 who is a non-smoker, has no family history of colorectal cancer, and uses aspirin daily is low (0.1), whereas that for a 61-year-old male with a BMI of 46 and 62 pack-year history, with no family history of colorectal cancer who does not use aspirin is high (0.69).

A plot of predicted risk of adenoma detection (horizontal axis) by fraction of the population at or below that risk (vertical axis) is shown in Fig. 3. This figure illustrates the impact of potential use of such a model in clinical practice: for example, if we define “high risk” as individuals where predicted probability of an adenoma is ≥0.2, 58% would be classified as high risk and prioritized for colonoscopy, whereas 42% of individuals would be classified as low risk, and could be offered other modalities of screening. This classification would accurately capture 75% of all adenomas and 73% of advanced adenomas. Of the high-risk individuals undergoing colonoscopy, an adenoma or advanced adenoma will be found in 35%, improving the number of therapeutic screening colonoscopies, compared with all comers. Of the entire cohort, 7% of individuals harboring an adenoma or advanced adenoma would be classified as “low risk” and not be screened with colonoscopy initially. If the cutoff for “high risk” is changed to a ≥ 0.15 predicted probability of an adenoma, 78% would be classified as high risk and prioritized for colonoscopy, whereas 22% of individuals would be classified as low risk. This classification would accurately capture 88% of all adenomas and 90% of advanced adenomas. Of the high-risk individuals undergoing colonoscopy, an adenoma or advanced adenoma will be found in 32%. Also, of the entire cohort, 3% of individuals harboring an adenoma or advanced adenoma would be classified as “low risk” and not be screened with colonoscopy initially. As the risk criterion increases, so does the proportion offered alternatives to colonoscopy. Careful cost-benefit analyses would determine the criterion actually used.

Our study aimed to create and validate a risk model for quantifying an individual's risk of harboring adenomas. Our final AUROCC was 0.64, indicating that the model has good predictive utility.

Such a model could be used to determine which men and women are most likely to harbor adenomas and would likely benefit from a therapeutic colonoscopy. Although we did not directly compare colonoscopic screening with other modalities of screening, our results are a first step toward allowing payers, patients, and physicians to use risk thresholds to decide who should be offered screening colonoscopy as opposed to other modalities of screening, or who should be prioritized for screening colonoscopy, based on their estimated risk of harboring adenomas. Once externally validated in sufficient populations, and possibly improved, we envision a model such as this to be a web-based or mobile application tool readily accessible and easy to use. As illustrated in Fig. 3, a selected threshold for the cumulative probability of harboring adenomas can be used to stratify individuals for priority colonoscopy versus other modalities of screening. Combined with appropriate cost-effectiveness analysis, this information can determine the appropriate thresholds for which colonoscopic screening should be prioritized versus some other modality. For example, if we set the risk threshold for harboring an adenoma at 0.2 or above (Fig. 3), the cumulative fraction of the population below this cutoff would be about 40%, and would be offered another less invasive modality of screening, such as fecal occult blood test or even no screening. Only about 60% of the population would be prioritized to colonoscopic screening, which may greatly improve efficiency. Of course, the actual cutoff for predicted risk would have to be based on a careful analysis and comparison of the cost-effectiveness for each potential cutoff, or multiple cutoffs.

Others have used similar approaches to predict an individual's risk for harboring colorectal cancer (27). Imperiale and colleagues (28) developed a score based on three clinical factors to predict risk of harboring proximal lesions based on findings at flexible sigmoidoscopy. Others have developed risk scores for advanced neoplasia, but not adenoma, in Asian and European cohorts. Tao and colleagues (19) have used nine risk factors to derive a risk-stratification tool for a German cohort. They also took into account prior colonoscopy and detection of polyps. Kaminski and colleagues (29) used factors similar to ours: age, sex, family history of colorectal cancer, cigarette smoking, and BMI. Others have reported similar approaches in Asian patients (30–32). However, there are no clinical tools with systematic evaluation of risk factors for predicting an individual's risk of harboring an adenoma that have been performed in a U.S. cohort.

We included demographic and baseline variables that would be practical and easy to obtain for the score to be clinically useful. We also favored a parsimonious model, and included variables that others have shown to be risk factors for adenomas (8–10, 33, 34), and variables that are easy to obtain from the chart or from the patient. Also, since the risk score changes with time, it can be used to recalculate an individual's risk for harboring adenoma periodically, and perhaps changing the individual's priority for screening colonoscopy.

Because our analysis is predictive, not causal (35), our estimated coefficients for individual factors are not necessarily unbiased estimates of their causal effects (36). Although we made no attempt to adjust for potential confounding in our model, the results were similar to findings by others, i.e., increasing age and male sex were associated with risk of adenomas (9, 27). We also found risk associations with family history of colorectal cancer in one or more first-degree relatives. In their large cohort of 3,121 mostly male veteran patients undergoing screening colonoscopy, Lieberman and colleagues (34) reported family history of colorectal cancer in one or more first-degree relatives and current smoking as risk factors for advanced adenomas (10 mm or more, 25% villous histology or more, high-grade dysplasia, or invasive cancer), whereas use of NSAIDs including aspirin was inversely associated with risk of advanced adenomas, and no association was found with BMI. Others have not found family history in a first-degree relative to be associated with adenomas (10). We did not find use of aspirin to be associated with risk of adenomatous polyps, but did find higher BMI to be associated with risk, as has been reported by others (17). We also did not find an association with non-white race and did not examine physical activity, the evidence for both of which in the literature is mixed (13, 37, 38).

The strengths of our study are the multiple sites from diverse and different geographic locations, inclusion of community-dwelling men and women in the United States, non-white races, complete colonoscopy and histology information with one central pathologist review, comprehensive collection and review of family cancer history for both first- and second-degree relatives, and community-dwelling individuals, rigorous derivation and validation of the risk score, good model calibration. The adenoma/advanced adenoma prevalence rates of 25% and 28% observed in our cohorts are similar to the rates reported by others (39, 40) and to those recommended by guidelines as indicating high quality in colonoscopy examinations (41). The high-quality colonoscopy exam, along with risk factors comparable with the general population, also supports the validity of our modeling and risk-score assessment. Our robust statistical approach included both internal and second-sample validation and a double bootstrap to address both overfitting bias and variance.

Limitations of our study include a small number of advanced adenomatous polyps precluding a separate robust model to predict only advanced adenomatous polyps and cancers. Our study is cross-sectional and predictive and thus cannot identify causal factors for progression of adenomatous polyps to cancer. Finally, there is possibility of a recall error in self-reporting of risk factors, although this error is unlikely to be biased by outcomes because nearly all of the baseline data were collected before colonoscopy.

Although our risk score is validated and well calibrated, it would benefit from further validation in other cohorts of average risk men and women in the United States. Although the AUROCC for our model of 0.64 is good, and similar to that reported by others (29), improvements will enhance its efficiency. Development of a clinically useful risk-stratification score is important to enhancing our capacity to deliver effective colonoscopic screening targeted toward those who may benefit the most. Cost-effectiveness analyses should be performed to determine appropriate thresholds. Similar work needs to be applied to surveillance colonoscopy intervals.

No potential conflicts of interest were disclosed.

Conception and design: A. Shaukat, T.R. Church, J.A. Allen, A.D. Feld, S.J. Winawer

Development of methodology: A. Shaukat, T.R. Church, A.G. Zauber

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): A. Shaukat, T.R. Church, M.J. O'Brien, P.A. Jordan, J.A. Allen, A. Kim, A.D. Feld, A.G. Zauber, S.J. Winawer

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): A. Shaukat, T.R. Church, R. Shanley, N.D. Kauff, S.J. Winawer

Writing, review, and/or revision of the manuscript: A. Shaukat, T.R. Church, R. Shanley, N.D. Kauff, G.M. Mills, J.A. Allen, A. Kim, A.D. Feld, A.G. Zauber, S.J. Winawer

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A. Shaukat, T.R. Church, A.G. Zauber, S.J. Winawer

Study supervision: A. Shaukat, T.R. Church, S.J. Winawer

Other (pathology review): M.J. O'Brien

Other (principal investigator): A.G. Zauber

This study was supported by grant R01 CA079572, National Cancer Institute, Screening Colonoscopy Feasibility Trial (National Colonoscopy Study; to A.G. Zauber and S.J. Winawer), Center for Chronic Disease Outcomes Research, and a VA HSR&D Center of Innovation (CIN 13-406; to A. Shaukat).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.