Long-term Follow-up of Patients Having False-Positive Multitarget Stool DNA Tests after Negative Screening Colonoscopy: The LONG-HAUL Cohort Study

This article is featured in Highlights of This Issue, p. 441

Colorectal cancer is the second leading cause of cancer-related deaths in the United States (1). Because colorectal cancer deaths are preventable (2–4), the U.S. Preventive Services Task Force (USPSTF) currently recommends that all adults ages 50 to 75 years and select adults ages 76 to 85 years undergo screening (5). In their updated guidelines, the USPSTF now includes the new multitarget stool DNA test (MT-sDNA) as a fully endorsed screening option; MT-sDNA has also been endorsed by the American College of Gastroenterology (6) and the American Cancer Society (7). More recently, the National Committee for Quality Assurance has included MT-sDNA among its Healthcare Effectiveness Data and Information Set for 2017 (8).

The MT-sDNA test is now available for general colorectal cancer screening (Cologuard; Exact Sciences). In August 2014, this noninvasive test was jointly approved by the FDA and the Centers for Medicare and Medicaid Services for colorectal cancer screening in average-risk patients after demonstration of significantly greater sensitivity for detecting colorectal cancer and advanced precancerous lesions compared with fecal immunochemical testing (FIT) alone (9). In the cross-sectional screening studies (9, 10), MT-sDNA sensitivities for colorectal cancer were 92% to 100% and for adenomas ≥1, >1, ≥2, and ≥3 cm were 40% to 42%, 51 to 52%, 62% to 66%, and 68% to 80%, respectively. In these same screen-setting studies (9, 10), colorectal cancer or advanced precancerous lesions (advanced adenoma or sessile serrated polyps ≥1 cm) were absent in 7% to 13% of follow-up colonoscopies after positive MT-sDNA tests (9–12).

The MT-sDNA test incorporates both DNA (methylated BMP3 and NDRG4 and mutant KRAS, all normalized by β-actin) and hemoglobin markers. A logistic algorithm converts quantitative data from these component assays into a binomial “positive” or “negative” result, and a false-positive (FP) rate approximating 10% was prespecified in development studies (11, 12). The biological and clinical contributions underlying these apparent FPs remain to be established. Molecular preneoplastic field changes in the colorectum may exist in the absence of gross lesions and could potentially be detected by stool testing. Colorectal lesions could have been overlooked on colonoscopic evaluation of positive MT-sDNA results, as neoplasm misses by colonoscopy are well documented (13, 14). And, theoretically, neoplasms above the colon could bleed or exfoliate markers that are excreted in stool. DNA abnormalities among the MT-sDNA panel have been observed in tissues of other airway and digestive tract (collectively, aerodigestive) neoplasms (15–17). Yet minimal decreases in MT-sDNA specificity have been observed as a result of cross-reactivity with aerodigestive neoplasms in cross-sectional experiments (18). Furthermore, supracolonic neoplasms were not detected by additional endoscopic and cross-sectional imaging testing in a previous small cohort study evaluating patients with positive stool DNA tests results (19).

It is uncertain whether or not significant long-term adverse outcomes are associated with FP by the now available MT-sDNA test. Specifically, are FP MT-sDNA patients at sufficiently increased risk of mortality or subsequent cancer diagnosis to justify additional follow-up testing or intensification of cancer surveillance? This area of uncertainty is an important concern for patients and providers, notably the American Academy of Family Physicians (20). As such, we aimed to address this question by examining long-term outcomes among participants from three preapproval studies of MT-sDNA (9, 11, 21) who had either nonadvanced adenomas only or negative findings on baseline colonoscopy. We measured differences in mortality rate, subsequent cancer incidence, and development of alarm symptoms among patients with apparent FP and true-negative (TN) MT-sDNA results.

This study was conducted after institutional review board approval. Methods and results are reported in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines (22). We identified a retrospective cohort of patients at Mayo Clinic (Rochester, MN; Scottsdale, AZ; and Jacksonville, FL) who participated in any of three MT-sDNA preapproval clinical studies and whose study colonoscopy showed neither colorectal cancer nor advanced precancerous lesions (Table 1).

Study 1 (“Specificity”) cross-sectionally sampled asymptomatic patients at average risk for colorectal cancer between February 2010 and August 2010 (21). Study 2 (“Cutoff”) was a case–control sampling of higher risk patients referred for known colorectal neoplasia and asymptomatic average-risk control patients who were scheduled for screening at the time of study enrollment, between August 2011 and December 2011 (11). Study 3 (“DeeP-C”) was a cross-sectional cohort sample of asymptomatic, average-risk patients who were scheduled to undergo screening colonoscopy at the time of study enrollment, between June 2011 and November 2012 (NCT01397747; ClinicalTrials.gov; ref. 9). At the time of enrollment in each of the original studies, patients were excluded for personal history of cancer in the lungs or gastrointestinal tract, family history of colorectal cancer in a first-degree relative, a comorbid inflammatory bowel disease diagnosis, or a known genetic cancer syndrome. In addition, exposures to tobacco and alcohol were collected on all patients. Prototype molecular marker panels performed in each study are listed in Table 1.

After an Accurint database (LexisNexis) search to determine patients' vital status, patients or next of kin were invited by mail for a structured telephone interview to document new cancer or precancer diagnoses, subsequent esophagogastroduodenoscopy (EGD) or colonoscopy results, and the development of alarm symptoms (anemia, unintentional weight loss, dysphagia, early satiety, bowel habit change, or gross gastrointestinal bleeding) since study participation (Supplementary Information). Chart review was performed on all who declined interview and those who did not respond; chart review was also performed on a random subset of those interviewed to measure potential reporting bias. Patient follow-up was censored at the time of death, loss to follow-up, or finalization of the study database on December 31, 2015. Deaths were ascertained from the Accurint database, next-of-kin interview, and chart review. Subsequent cancer diagnoses were confirmed, wherever possible, by review of endoscopic, radiographic, and histopathologic reports.

Because each of the three studies used a different MT-sDNA prototype marker panel, baseline FP MT-sDNA results were determined by two methods. First, the FP and TN rates were measured according to per-protocol definitions, reported separately for each study. Briefly, 90th percentile values for each marker panel were applied to all patients in the “Specificity” study and “Cutoff” study controls to optimize panel specificity to a goal of 90%, whereas “DeeP-C” used a prespecified logistic regression algorithm, with a calculated score of ≥183, indicating a positive result (9). In the second method, the results of MT-sDNA across each of the three studies were calibrated using three DNA markers—methylated BMP3, methylated NDRG4, and β-actin—measured in common for all patients. To scale all three markers, raw data for each were standardized to the mean and SD within the study of origin. Then, a smoothing spline logistic regression model was used to calibrate marker values from “Specificity” and “Cutoff” to “DeeP-C” test results. From this calibrated dataset, 90th and 95th percentile rates for all three markers combined were used to establish 10% and 5% FP rates across all three studies.

Assuming MT-sDNA test specificity of 90%, 850 patients were anticipated to provide 80% power to detect a difference between a 6% cumulative event rate in the FP group from a 1% event rate in the TN group, at the 5% significance level. Primary study endpoints included rates of mortality, subsequent aerodigestive cancer diagnosis, any subsequent cancer diagnosis, and development of alarm symptoms assessed from the time of stool collection using the Kaplan–Meier method. Association of FP status with each of these rates was assessed by proportional hazards, reported as hazard ratios (HR) with 95% confidence intervals (CI). Cumulative incidence of aerodigestive cancers in FP and TN groups were also compared with expected Surveillance, Epidemiology, and End Results (SEER) Program cumulative incidence (23). Potential differences in proportions or continuous distributions of baseline characteristics between patients in each study were assessed with χ², Fisher exact, or Wilcoxon rank-sum tests, where applicable. Geographic distance from the site of study participation to each patient's primary address was measured as a surrogate for ongoing access to the study site for clinical follow-up. A distance of 100 miles or less was used as a surrogate for access to primary care at the study site.

There were 1,050 eligible participants. Of these, 595 (57%) participated in the structured interview. Six patients declined both interview and chart review; however, their vital status was publicly available (Fig. 1). Using the per-protocol method, MT-sDNA was FP in 160 (15%) and TN in 890 (85%) patients. The stool biomarkers common to all three studies, before and after calibration, are shown in Fig. 2. Using the calibrated method, MT-sDNA was falsely positive in 113 (11%) and 51 (5%) patients at 90th and 95th percentile thresholds, respectively, and this was not significantly different by study (Table 2). Baseline demographic and clinical features of all patients are shown in Table 2. Median age of the “DeeP-C” cohort was 69 years [interquartile range (IQR), 65–74]; this was significantly older than the median age in the other two studies (P < 0.0001). Although sex was similar across all three studies, the “Specificity” study participants were more racially diverse. Of the 1,050 patients, 570 (54%) were geographically defined as having access to primary care at Mayo Clinic. This was similar across MT-sDNA test result groups: 490/890 (55%) of TN versus 80/160 (50%) of FP (P = 0.26), using the per-protocol cutoff; 514/937 (55%) of TN versus 56/113 (50%) of FP (P = 0.32), using the calibrated 90% cutoff; and 544/999 (54%) of TN versus 26/51 (51%) of FP (P = 0.67), using the calibrated 95% cutoff. There was also no difference in geographic variation when distance to Mayo Clinic was analyzed as a continuous variable (P = 0.06), using the per protocol cutoff.

Median follow-up time was 4 years (IQR, 3.5–5.3) and was similar between FP and TN groups (P = 0.10, per-protocol, P > 0.9, calibrated). The proportion of patients who underwent a subsequent invasive gastrointestinal diagnostic procedure (any of colonoscopy, flexible sigmoidoscopy, or EGD) during follow-up was: 187/885 (21%) of TN versus 36/159 (23%) of FP (P = 0.67), using the per-protocol cutoff; 199/932 (21%) of TN versus 24/112 (21%) of FP (P = 1.00), using the calibrated 90% cutoff; and 215/993 (22%) of TN versus 8/51 (16%) of FP (P = 0.38), using the calibrated 95% cutoff.

During study follow-up, a total of 25 patients were reported deceased. Of these, all were confirmed by chart review or telephone interview with next of kin. A random sample of patients who had both interview and chart review performed were analyzed to assess agreement on cancer events (yes vs. no) between the two methods of data abstraction. There was agreement in 51 of the 53 patients assessed. The 2 that were not in agreement had noted skin cancer in the interview, but not in the chart review. Of 48 total subsequent cancer events, 36 were reported during interview and 12 additional events were found on chart review. All cancer events, except non-melanoma skin neoplasms, were confirmed by histopathology review. Subsequent alarm signs were common, occurring in 13% of the cohort after 5 years of follow-up.

By either cutoff method, FP status was not associated with mortality, subsequent cancer of all types, or alarm symptoms (Table 3). FP status by calibrated, but not per-protocol, cutoffs was associated with subsequent aerodigestive cancer (Table 3). There were 8 incident aerodigestive cancers (1 colorectal, 3 lung, 3 pancreatic, and 1 bile duct) in the study cohort (Supplementary Table S1). One patient considered FP by all three cutoff definitions subsequently developed colorectal cancer; this individual's multiple subcentimeter adenoma lesions at index colonoscopy did not meet prespecified advanced colorectal neoplasia endpoint criteria (9). By any cutoff method, cumulative incidence of aerodigestive cancer in the FP group fell below the expected age- and gender-specific SEER incidence, and all aerodigestive cancers discovered on follow-up were diagnosed beyond 3 years from MT-sDNA testing (Fig. 3).

This is the first study on the long-term follow-up of patients who underwent MT-sDNA testing for colorectal cancer screening. Five-year survival is similar between patients with FP and TN MT-sDNA results. Only 8 subsequent aerodigestive cancer events were observed in the cohort of 1,043 patients, a rate far below that expected based on SEER incidence data. By per-protocol assignments, these were not associated with FP MT-sDNA tests. However, after calibrated cutoffs were applied, the aerodigestive cancer rates were statistically increased among those with FP MT-sDNA. However, temporal association of these events with MT-sDNA appears to be lacking, as the earliest aerodigestive cancer among FP patients presented 3 years after MT-sDNA testing. The first event, an early-stage colorectal cancer, would have been detected at 93% sensitivity by MT-sDNA at the subsequent recommended 3-year screening interval. The other aerodigestive cancers were detected over 4 years after MT-sDNA testing. Given the rapid growth rate of lung cancers, current guidelines recommend annual screening of patients at increased risk (24). There is no population-level screening for other aerodigestive cancers, but even those at risk for familial pancreatic cancer might be offered only annual surveillance (25). Therefore, it is doubtful that the positive MT-sDNA test was attributable to a preclinical aerodigestive cancer.

For patient inclusion in each study, index colonoscopies were performed by endoscopists whose adenoma detection rates are regularly monitored. Study procedures also required documented evidence of cecal intubation, annotation of withdrawal time, and bowel preparation sufficient for examination of ≥90% of the mucosal surface. These standards of care for screening colonoscopy are supported by jointly issued guidelines from the American Society for Gastrointestinal Endoscopy and American College of Gastroenterology (26).

For follow-up of other noninvasive colorectal cancer screening methods, little data are available on the yield of additional testing beyond colonoscopy. Despite high cumulative FP rates of programmatic fecal occult blood test (FOBT) screening (27), there is insufficient evidence to recommend additional invasive testing (28). Although there are no U.S. guidelines to direct management in this subgroup of patients, a recent microsimulation study (29) concluded that these individuals could resume screening with annual FOBT, 10 years after negative follow-up colonoscopy (30).

Extrapolation of screen-setting observations to a theoretical population of 10,000 persons revealed that 16% of MT-sDNA tests will come back positive, with 45.4% of these having negative colonoscopy results (9). On the basis of the SEER incidence and studies in stools of patients with liver, pancreatic, and gynecologic cancers, it was estimated that specificity of MT-sDNA for colorectal cancer would only be reduced by 0.02% by cross-reactivity to these neoplasms (18).

We have opportunistically utilized a ready pool of patients from three preapproval studies performed sufficiently long ago to calculate 5-year event rates. The strengths of this study include the large sample size, high response rate, high agreement between survey and chart review, and low loss to follow-up. The majority of patients in our cohort were local to the Mayo Clinic system, optimizing the chances of capturing subsequent cancer events. The Kaplan–Meier analysis method also was used to account for variable rates of follow-up during the study period. Hard outcomes such as cancer incidence can be difficult to ascertain, as there is no national-level cancer registry to completely enumerate cancer events. We feel that our method of event ascertainment by structured survey and electronic medical records review was adequate but could have also missed events. As with all telephone interviews, recall bias has to be considered. However, it is unlikely that our primary outcomes, death and subsequent cancer diagnosis, would be affected. This was confirmed by measurement of agreement between chart review and survey for a randomly selected subset of patients. The development and reporting of subsequent alarm signs may have been influenced by recall bias. This secondary outcome was measured to avoid missing potentially undiagnosed cancers and was not significantly different between groups.

We acknowledge that the total sample size limited the ability to measure between-group differences in event rates of <5%. A multivariate analysis of subsequent cancer events in larger sets of patients will also be needed to measure for interaction of FP MT-sDNA results and exposures to other cancer risk factors, especially smoking. In addition, the now FDA-approved version of the MT-sDNA test was used in only 337 patients, representing 32% of our overall combined cohort. To account for this, data were logistically calibrated using three markers common to all studies. Although this method also permitted assessment of more stringent hypothetical specificity thresholds, direct extrapolation of these findings to the test specificity of the FDA-approved MT-sDNA test, which uses a larger panel of markers, is potentially limited. Also, these observations may not be applicable to patient populations at higher risk of cancer. As the MT-sDNA test becomes more widely used in the population, additional study of clinical outcomes of patients screened by MT-sDNA is warranted.

Clinical MT-sDNA test specificity in clinical use may differ from that observed in clinical trials. This difference may be attributable to patient factors. In the largest screen-setting study of MT-sDNA, subgroup analyses showed that specificity was 94% in participants ages 65 years or less but 87% in those over 65 (P < 0.0001; ref. 9). Moreover, MT-sDNA specificity was 89.8% (95% CI, 88.9–90.7) in the subset of patients with completely negative colonoscopies; this was higher than the observed specificity of 86.6% (95% CI, 85.9–87.2) from the entire study cohort, which included colonoscopy findings of nonadvanced adenomas or other nonneoplastic pathology among “negative” results (9). Provider factors may also influence MT-sDNA yield. Early observations from the first year of clinical use in a referral center practice suggest that polyp detection rates and colonoscopy withdrawal times were significantly higher among endoscopists aware of a positive MT-sDNA result in clinical practice compared with those blinded in preapproval studies (31). Specificity also needs to be examined over the duration of a screening program. Used every 3 years, MT-sDNA is expected to generate fewer FP results and, therefore, fewer follow-up colonoscopies than annual FIT screening (32) or other modalities over a lifetime (33). This directly influences the risk-to-benefit ratio of programmatic screening. A recent article analyzing the comparative and cost-effectiveness of colorectal cancer screening methods using a Markov model demonstrated that colonoscopy and FIT were more effective and less costly than MT-sDNA, assuming equal adherence (34). However, the same study acknowledged that consistent participation in yearly colorectal cancer screening by FIT is only 15%; if MT-sDNA yielded colorectal cancer participation rates more than 1.7-fold relative to FIT, then MT-sDNA every 3 years cost less than $100,000 per quality-adjusted life-year gained compared with yearly FIT. In addition, another recent study commissioned by the USPSTF showed that MT-sDNA performed at 3-year intervals yielded the greatest number of life-years gained and averted the most colorectal cancer deaths relative to the harms from complications of screening and follow-up tests generated by all other USPSTF-endorsed screening approaches (33).

In summary, patients with positive MT-sDNA tests are unlikely to benefit from additional follow-up evaluations after negative high-quality colonoscopy. We observed similar 5-year mortality and subsequent all-cancer event rates between those with positive and negative MT-sDNA, following a negative study colonoscopy. FP tests were associated with a low but statistically significant increase in aerodigestive cancers between groups, all of which presented after 3 to 5 years of follow-up. The clinical significance of this observation appears questionable, as events do not show temporal association and fall below SEER estimates of aerodigestive cancer incidence in the U.S. population. Larger studies are needed to confirm these observations.

D.W. Mahoney has ownership interest (including patents) in Exact Sciences. D.A. Ahlquist reports receiving a commercial research grant from, has ownership interest (including patents) in, and is a consultant/advisory board member for Exact Sciences. J.B. Kisiel is a consultant/advisory board member for and has provided expert testimony for Exact Sciences. No potential conflicts of interest were disclosed by the other authors.

The funding sources had no role in the design or conduct of the study.

Conception and design: T.G. Cotter, D.W. Mahoney, D.A. Ahlquist, J.B. Kisiel

Development of methodology: T.G. Cotter, D.W. Mahoney, D.A. Ahlquist, J.B. Kisiel

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): T.G. Cotter, M.E. Devens, J.A. Simonson, K.L. Lowrie, R.I. Heigh, D.H. Johnson

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): T.G. Cotter, K.N. Burger, D.W. Mahoney, D.H. Johnson, D.A. Ahlquist, J.B. Kisiel

Writing, review, and/or revision of the manuscript: T.G. Cotter, K.N. Burger, R.I. Heigh, D.W. Mahoney, D.H. Johnson, D.A. Ahlquist, J.B. Kisiel

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): M.E. Devens, D.W. Mahoney, D.A. Ahlquist

Study supervision: D.W. Mahoney, D.A. Ahlquist, J.B. Kisiel

Other (helped procure funding): D.A. Ahlquist

This research was supported by a grant from the Maxine and Jack Zarrow Family Foundation and the Paul Calabresi Program in Clinical-Translational Research (NCI CA90628). Additional partial support was provided by the Carol M. Gatton endowment for Digestive Diseases Research.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.