Comparison of Performance of Two Stool DNA Tests and a Fecal Immunochemical Test in Detecting Colorectal Neoplasm: A Multicenter Diagnostic Study

Colorectal cancer continues to be a major public health threat worldwide. Colorectal cancer is the fourth most common cancer among adults in the United States (1). In 2015, colorectal cancer was the third most commonly diagnosed cancer and the fifth most common cause of death by cancer in China (2). There is strong evidence supporting that the colorectal cancer mortality can effectively be reduced through screening (3). Updated colorectal cancer screening guidelines from the American Cancer Society (ACS) recommend testing asymptomatic subjects aged older than 45 years by either stool-based tests or structural examinations, such as sigmoidoscopy or colonoscopy (4). Although colonoscopy is the current gold standard for the detection of colorectal cancer and precancerous lesions, this method is associated with high costs and low compliance for colorectal cancer screening, especially in resource-limited countries (5).

The most widely used noninvasive screening tests are fecal occult blood tests (FOBT), such as guaiac-based FOBTs (gFOBT) or fecal immunochemical tests (FIT). Compared with gFOBTs, the sensitivity and specificity of FITs tend to be superior, and FITs also eliminate the need for dietary restrictions before testing (6). However, the use of FITs in colorectal cancer screening is still limited for its low sensitivity in detecting advanced adenomas (AA), which is only approximately 25% (7). The FIT which automatically quantifies the fecal hemoglobin concentrations has adjustable thresholds to best serve specific screening programs (8). Appropriate threshold adjustments would improve the sensitivity for detection of colorectal cancer and AA at acceptable specificity (9).

Multitarget stool DNA (mt-sDNA) testing was developed in recent years. A large cross-sectional study showed that the sensitivity of mt-sDNA for both colorectal cancer (92.3%) and advanced precancerous lesion (42.4%) exceeded that of the FIT by an absolute difference of nearly 20%, while the specificity was significantly lower than FIT (10). The American guidelines recommend mt-sDNA as an option for colorectal cancer screening (4). However, the mt-sDNA test marketed in the United States (Cologuard) is not currently available in China. At present, several stool DNA tests for colorectal cancer screening have been established in China and pilot studies have been carried out, but the effectiveness of these tests remains to be verified in a large population (11, 12). Moreover, direct comparative analyses of these stool DNA tests and FITs in the same study are still sparse, which makes the reported differences in diagnostic performance from different studies difficult to interpret. In this study, we aimed to perform head-to-head evaluations and to compare the diagnostic performance of two different stool DNA tests and FIT by using colonoscopy along with histologic confirmation as reference standards in a multicenter study.

The primary objective of this study was to estimate the ability of stool DNA tests to detect colorectal cancer, especially sensitivity and followed by specificity. The secondary objective was to compare the sensitivity of various markers for the detection of colorectal cancer, if the specificity of them was comparable.

We enrolled participants (n = 2,842) from August 2017 to April 2019 at 8 clinical centers in China. Subjects who visited outpatient clinics or cancer screening centers were eligible for enrollment. The inclusion criteria included the following: age ≥ 50 years, or a positive family history of colorectal cancer and age ≥ 40 years. The exclusion criteria included unable or unwilling to accept colonoscopy, older than 85 years, known inflammatory bowel disease, Lynch syndrome, familial adenomatous polyposis, Peutz-Jeghers syndrome, history of colectomy, known or highly suspected colorectal cancer, or other malignant diseases. This study was conducted in accordance with Declaration of Helsinki, and was approved by the ethics committee at each participating hospital and was registered at chictr.org.cn (Registration number: ChiCTR-DDD-17011169). The result was reported following the STARD guidelines (13). All laboratory examinations and consultations for screening were free of charge for the participants and funded by the Beijing Municipal Science and Technology Commission (BMSTC).

During the study period, subjects who potentially met the eligibility criteria were approached about volunteering for the study consecutively at each clinical site. After these individuals were informed about the details of the study, especially that all participants would undergo verification colonoscopy, all subjects were asked to provide written informed consent. Then, trained study staff assessed eligibility and helped the participants complete the baseline information questionnaire. Subsequently, colonoscopy was scheduled for each subject at one of the 8 clinical centers within 4 weeks. At least 1 day before colonoscopy preparation, a stool sample were collected from each subject. Diet and medication were not restricted before collecting the samples.

All endoscopists were blinded to the laboratory detection of the subjects. To ensure the quality of colonoscopy, we asked all participating clinical centers to perform bowel preparation in accordance with the guidelines of the Chinese Society of Digestive Endoscopy (2013) and spend more than 6 minutes during scope withdrawal (14). The extent of colonoscopy, cecal intubation, bowel preparation quality, and the size and location of any observed lesions were recorded on a standard case report form. The participants without adequate bowel preparation were scheduled for another colonoscopy within 3 days.

During colonoscopy, biopsies were taken for histologic examination. Adenomatous polyps with ≥ 25% villous component, high-grade dysplasia (HGD), or a diameter ≥ 10 mm were considered AA. Sessile serrated lesions with diameters ≥ 10 mm were also categorized as AA. Cases of carcinoma in situ were grouped as HGD cases. The stages of invasive adenocarcinoma were determined from the surgically resected specimens according to the American Joint Committee on Cancer (AJCC) staging system. Only the most advanced colorectal epithelia lesions were used for categorization. Negative results included nonneoplastic polyps, other nonneoplastic findings, or no evidence of disease. Thus, each participant with an evaluable colonoscopy outcome (those with adequate bowel preparation, achievement of cecal intubation, and pathology results if polyps existed) was assigned to one of four clinical classes: colorectal cancer, AA, nonadvanced adenomas (NAA), and negative results on colonoscopy. Furthermore, colorectal cancer and AA comprised colorectal ANs.

Stool DNA test-I (KRAS, BMP3, NDRG4, and hemoglobin immunochemical tests): The kits were manufactured by New Horizon Health Technology Co., Ltd. (Hangzhou, China). Similar to the mt-sDNA test kit (Cologuard) applied in the United States, the stool DNA test-I included analysis of KRAS gene mutation, BMP3 and NDRG4 methylation, β-actin as a reference gene, and a separate fecal hemoglobin immunochemical assay using ELISA. The primers and probes used in the PCR assays are described in Supplementary Table S1. The cycle threshold (Ct) values of mutated KRAS, methylated NDRG4 and BMP3, as well as results of the fecal immunochemical test were then fed into the risk prediction model, which provided a risk score as a single output with a cut-off value of ≥ 165 considered as positive. The details about the risk scoring system were described by Mu and colleagues previously (11).

Stool DNA test-II (methylated SDC2 and SFRP2 tests): The kit was provided by Realbio Technology Co., Ltd. (Shanghai, China) and was previously reported by Bai and colleagues in a Chinese publication (12). DNA was isolated from stool samples, and was subjected to a bisulfite reaction for the identification of aberrantly methylated SDC2 and SFRP2. β-actin was used as a reference gene for human DNA quantification. The Ct values of SDC2 and SFRP2 in methylation-specific PCR were put into a prespecified risk score calculation formula. If the result was equal to or greater than 297, the test was called “positive”, otherwise it was negative. The primers and probes used in the PCR assays, as well as the algorithm, are described in Supplementary Table S2.

Quantitative FIT: The OC-SENSOR Micro instrument and corresponding stool collection bottles and reagents (Eiken Chemical Co., Ltd., Tokyo, Japan) were used to process the samples and quantify the fecal hemoglobin by latex-enhanced immunoturbidimetry. The results, given as nanograms hemoglobin per milliliter of buffer, were automatically printed a few minutes after inserting the stool collection bottles. We used a threshold of 100 ng/mL as suggested by the manufacturer (15).

Stool collection, buffering, storage, and transport were performed according to the manufacturers' instructions. Participants defecated into special toilet boxes at home. Then, on the same day, the box with whole stools was sent to each clinical site where the participant was registered. Trained staff completed stool sample collection for all the three tests on the same feces. In brief, for FIT, the sampling probe was inserted into 6 different areas of the stool and then reinserted into the fecal sample tube which was filled with buffer for hemoglobin stabilization. For stool DNA test-I, the gene mutation and methylation detection need 5 gram feces collected by sampling scoops and preserved in nucleic acid stabilization buffer, while fecal immunochemical assay need stool sampling as same as FIT. For stool DNA test-II, 2 gram feces were collected by sampling scoops and were preserved in nucleic acid stabilization buffer. Watery diarrhea and blood-stained stools were not suitable for sampling. Stool aliquots were subsequently sent to the following 3 laboratories under room temperature within 48 hours: Clinical Laboratory of The Seventh Medical Center of Chinese People's Liberation Army General Hospital (Beijing, China) for FIT, New Horizon Health Technology (Hangzhou, China) for stool DNA test-I, and Realbio Technology (Shanghai, China) for stool DNA test-II. All laboratories were blinded to the clinical results of the subjects. Stool DNA test-II was not performed on the first 298 subjects due to unavailability from the manufacturer at that time.

PASS 11 software (NCSS, LLC., Utah) was used to find the sample size. In preliminary case–control studies, stool DNA test-I detected 97.5% colorectal cancer with specificity of 89.1% in normal controls, while stool DNA test-II detected 97.7% colorectal cancer with specificity of 94.2% in controls (11, 12). Although reported sensitivities for colorectal cancer of FITs varied widely (25%–100%), a recent multicenter study in China and the Cologuard study in the United States both showed a sensitivity of ∼70% for FIT (9, 10, 16). Therefore, both stool DNA tests were hypothesized to have a sensitivity of more than 70% for the detection of colorectal cancer. The study was designed to have a power of 90% to detect a change in sensitivity from 70% (under the null hypothesis) to 90% (under the alternative hypothesis) using a two-sided binomial test under a significance level of 0.05. The analysis required the diagnosis of 40 colorectal cancers in the prospective trial. We calculated the incidence of colorectal cancer among outpatients in 2015 at 4 participating clinical centers (The Seventh Medical Center of Chinese People's Liberation Army General Hospital, Peking University Third Hospital, Peking University People's Hospital, and Peking University First Hospital) and obtained a rate of 17 cases per 1,000 population. Thus, at least 2,353 enrolled outpatient participants were required to achieve the primary objective.

The sensitivity and specificity of each test were calculated with 95% confidence intervals (CI) on the basis of the exact binomial distribution. The sensitivity for detecting colorectal cancer, AA, and AN was calculated by the proportion of diseased subjects that yield a positive test result. The specificity was calculated by the proportion of negative test results among subjects with negative results on colonoscopy. McNemar test and receiver operating characteristic (ROC) curves was used to compare the different tests. All statistical tests were performed two-sided, and a statistically significant difference was established when P < 0.05. R software version 4.0.3 (www.r-project.org) was used for ROC curves analyses, and OpenEpi software version 3.01 (www.openepi.com) was used for the other statistical analyses.

The data generated in this study are available upon request from the corresponding author.

A total of 2,842 participants were enrolled with written informed consent provided and 2,561 (90.1%) participants had colonoscopy outcomes that could be evaluated (Fig. 1). Among these individuals, 2,240 (78.8%) participants undergoing both stool DNA test-I, test-II, and FIT were included in the primary analysis. The demographic data for these subjects are shown in Table 1. The age of participants was 60.0 ± 6.5 years old (mean ± SD, range from 40 to 85 years old). Of these 2,240 subjects, 42 (1.9%) and 302 (13.5%) participants had colorectal cancer and AA, respectively. Among 42 participants who had colorectal cancer, there were 13 cases of stage I, 12 cases of stage II, 8 cases of stage III, and 2 cases of stage IV according to the AJCC staging system, while the other 7 cases had no staging information.

Among 42 participants diagnosed with colorectal cancer, stool DNA test-I detected 38 cancers (90.5%; 95% CI, 77.9–96.2), while stool DNA test-II detected 39 cancers (92.9%; 95% CI, 81.0–97.5; Table 2). The two DNA tests were accordantly positive in most colorectal cancer cases (36/42). For curable colorectal cancer without metastasis (Stage I–II), stool DNA test-I detected 84.0% (21/25) and stool DNA test-II detected 92.0% (23/25) cases. The sensitivity of both DNA tests for colorectal cancer did not vary significantly according to cancer staging, location within the colon, or age and gender of patients (P>>0.05 by Fisher exact test, Table 3).

For detecting AA, stool DNA test-I had a sensitivity of 27.2% (95% CI, 22.5–32.4), whereas the sensitivity of stool DNA test-II was 35.1% (95% CI, 29.9–40.6). Stool DNA test-I detected 47.2% HGD while stool DNA test-II detected 50.0% cases of HGD. However, for detecting sessile serrated lesions ≥ 10 mm, both stool DNA tests showed poor sensitivities (9.5% for test-I and 14.3% for test-II). Among 344 participants with AN, stool DNA test-I detected 120 (34.9%; 95% CI, 30.0–40.1) and stool DNA test-II detected 145 participants (42.2%; 95% CI, 37.1–47.4). Among the participants with negative results on colonoscopy, the specificity of stool DNA tests -I and -II was 91.4% (95% CI, 89.8–92.8) and 93.3% (95% CI, 91.9–94.5), respectively.

Under the predetermined cut-off values, both stool DNA test -I and -II were more sensitive for detecting AN than FIT alone (34.9% and 42.2% vs. 25.9%, respectively, both P < 0.001 by McNemar test; Table 2). However, the specificity of both stool DNA tests was significantly lower than that of FIT (91.4% and 93.3% vs. 96.8%, respectively, both P < 0.001 by McNemar test). So we used two different methods to compare the performance of the two stool DNA tests and FIT. Firstly, ROC curves of these three methods were created and the areas under the curves (AUC) were compared between them (Fig. 2). For detecting colorectal cancer, the difference of the AUC was not statistically significant between stool DNA test-I, test-II, and FIT (0.923, 0.927, and 0.924, respectively, P > 0.9). For AN, no significant difference was seen between FIT and DNA test-II (0.689 vs. 0.671, P = 0.276), but the AUC of FIT was superior to DNA test-I (0.689 vs. 0.635, P = 0.007).

Moreover, we compared the different sensitivities after adjusted the cut-off value of FIT to yield the same specificity as the other two DNA tests (Table 4). Instead of 100 ng/mL suggested by the manufacturer, a threshold of 22 ng/mL was used to match the specificity of stool DNA test-II. Under this adjusted threshold, FIT detected 88.1% colorectal cancer and 36.6% AN, which was not significantly different from stool DNA test-II (P = 0.414 and 0.054 respectively by McNemar test). When the cut-off value of FIT was set to 12 ng/ML to match the specificity of stool DNA test-I, the sensitivity for AN of FIT alone was even greater than DNA test-I (41.3% vs. 34.9%, P = 0.006 by McNemar test) while the sensitivities for colorectal cancer were similar (88.1% vs. 90.5%, P = 0.480 by McNemar test).

Over the past decade, the Chinese government gradually initiated population-based screening for colorectal cancer in some urban areas. However, poor adherence is one major barrier to colorectal cancer screening in China. Recently, a population-based colorectal cancer screening program using colonoscopy as the main modality in China reported a relatively low participation rate for colonoscopy (14.0%) in those who were at high risk of colorectal cancer (17). Another community-based colorectal cancer screening program in Shanghai using FOBT as the primary screening method also reported a similar situation that the participation rate (35.2%), follow-up rate after positive primary screening (26.3%) and rate of diagnostic colonoscopy (6.4%) were all relatively low (18). While some studies showed that noninvasive tests prior to colonoscopy could increase adherence to colorectal cancer screening, the clinician recommendation emerged as one of the most important factors (17, 19, 20). Therefore, the Beijing government intended to establish an outpatient clinic-based opportunistic colorectal cancer screening program using a noninvasive test as primary method and follow-up colonoscopy as an alternative to community-based screening. Our study was launched to evaluate the performance of various tests head-to-head in such a screening setting.

Our results showed that both stool DNA tests are competent for colorectal cancer screening (both sensitivities > 90% for colorectal cancer and specificities > 90%). However, the ability of both stool DNA tests to detect AA was not outstanding (27.2% and 35.1% respectively). Furthermore, FIT showed similar performance for detecting colorectal cancer and AN compared with the DNA tests after appropriate threshold adjustment in this study. To our knowledge, this is the first study to present a direct comparison of efficacy of two different mt-sDNA and automatically quantitative FIT in the diagnosis of colorectal cancer within the same population. Although these colorectal cancer screening methods have been evaluated in previous studies, a head-to-head evaluation in the same population makes the diagnostic performances comparable (8, 11, 12, 15).

A large cross-sectional study in average-risk North American individuals showed that the sensitivity of mt-sDNA (Cologuard, Exact Sciences) and FIT (OC FIT-CHEK, Polymedco, with a threshold of 100 ng/mL) in detecting colorectal cancer was 92.3% and 73.8%, respectively (10). The sensitivity of mt-sDNA for detection of advanced adenomas and sessile serrated polyps in this study was also higher compared with FIT alone (42.4% vs. 23.8%), while the specificity of mt-sDNA was significantly lower than that of FIT (89.8% vs. 96.4%). Given that Cologuard is not available in China at this moment, we applied the stool DNA test-I using the same target genes and hemoglobin immunoassay component as Cologuard. The method of processing data of the stool DNA test-I was also similar to Cologuard, that is, the quantitative measurements of each marker were incorporated into a prespecified algorithm, except that the threshold was different (183 for Cologuard but 165 for stool DNA test-I). As shown in previous case–control study, the stool DNA test-I had a sensitivity of 97.5% for colorectal cancer and 53.1% for AA as specificity of 89.1% for normal controls (11). In our study, the stool DNA test-I showed similar sensitivity (90.7%) and specificity (91.3%) for colorectal cancer detection as the results reported in the Cologuard study; while the sensitivity for AA is lower (26.3%). However, the sensitivity for colorectal cancer of DNA test-I was not significantly different from FIT alone, and the sensitivity for AN was even lower than FIT under the same specificity. The previous case–control study reported that the fecal hemoglobin assay embedded in this kit identified most colorectal cancer and AA which were detected by the kit (11). Although there was no available information about how many cases detected by the DNA test-I were identified exclusively by fecal hemoglobin assay, one would expect that most of the diagnostic performance of this test was driven by the fecal hemoglobin component. This may be one of the reasons why there is no difference between the performance of FIT alone and stool DNA test-I.

Syndecam-2 (SDC2) and secreted frizzled-related protein 2 (SFRP2) are reported to be aberrantly methylated in colorectal cancer and could be used as biomarkers for the early detection of colorectal cancer (21). Previous small case–control studies showed the sensitivity of stool SDC2 in detecting colorectal cancer ranged from 81.1% to 90.2%, with specificity ranging from 90.2% to 93.3% (22–24). The sensitivity and specificity of stool SFRP2 in detecting colorectal cancer range from 82.6% to 94.0%, and 76.9% to 94.0%, respectively (25–27). While the stool DNA test-II used in our study analyzed SDC2 and SFRP2 simultaneously and showed a sensitivity of 97.2% for colorectal cancer and specificity of 94.2% in previous case–control study, our results showed that it was superior to stool DNA test-I in detecting AA (35.1% vs. 27.2%), with a higher specificity (93.3% vs. 91.4%) and a similar sensitivity for colorectal cancer detection (12). However, the sensitivity of DNA test-II in detecting colorectal cancer and AN was not significantly different from FIT under the same specificity in our study.

A systematic review of FITs showed an excellent potential of FITs to detect most colorectal cancers at high levels of specificity by appropriate threshold adjustments (9). An indirect comparison of FIT and mt-sDNA in two independent studies among the screening population showed that the diagnostic performance of FIT could be equivalent to that of mt-sDNA if an adjusted cutoff yielding the same specificity was used for FIT (28). In the current study, we directly compared FIT with two different mt-sDNA tests and found the performance of them was almost equal. Although the differences between the sensitivities are not statistically significant, it should be noted that this result is at least partially related to the design of the study (the power to compare 0.9 with 0.7 but not to compare sensitivities directly). In addition, it is still challenging to determine the best cutoff of FIT to serve a specific screening program instead of simply applying the threshold recommended by the manufacturer. However, given the cost and inconvenience of DNA-based stool tests, the results of our study do not support an extensive use of stool DNA tests for colorectal cancer screening instead of FIT. The cost of stool DNA tests is about 20 times that of FIT (RMB 2,000 per test vs. RMB 100 per test in our study approximately). And DNA tests require specialized laboratories, which increases the difficulty of sample collection and transportation.

There were some limitations in our study. First, our participants in the opportunistic screening setting were not average-risk individuals. The incidence of colorectal cancer in our study (1.9%) appeared higher than in general population (0.25%) as reported in a recent population-based colorectal cancer screening program in China (17). The difference in prevalence will affect positive predictive values and negative predictive values of tests. The potential discrepancy in characteristics of colorectal cancer such as stage and location between opportunistic setting and population-based screening setting may affect the performance of tests, too. However, the direct comparison of sensitivity and specificity in the same subjects can still reflect the pros and cons of these tests themselves. However, the comparison of sensitivity and specificity still reflects the abilities of these tests themselves. Second, some important factors for screening tests, such as intervals, patient adherence and costs are beyond the scope of this study, as the focus of this study was to assess the sensitivity and specificity of various tests. In fact, the mt-sDNA test is recommended every 3 years and FIT is recommended every year for colorectal cancer screening in the ACS guidelines (4). Further research is needed to confirm whether our findings suggest that changing the threshold of FIT could allow a same interval as DNA test for colorectal cancer screening.

In conclusion, although the sensitivity and specificity of the two types of stool DNA tests are competent for colorectal cancer screening, there was no significant advantage of both stool DNA tests compared with FIT in detecting colorectal cancer or AN in this study.

No disclosures were reported.

P. Jin: Conceptualization, data curation, formal analysis, supervision, funding acquisition, methodology, writing–original draft. P. You: Data curation, formal analysis, investigation. J. Fang: Data curation, investigation, writing–review and editing. Q. Kang: Data curation, investigation. F. Gu: Data curation, investigation. Y. Cai: Data curation, investigation. H. Zhai: Data curation. B. Wang: Data curation. Y. Li: Data curation. J. Xu: Data curation, formal analysis, writing–original draft. J. Wang: Data curation. Y. He: Data curation. Y. Wang: Data curation, methodology, writing–review and editing. M. Dai: Methodology, writing–review and editing. J. Sheng: Conceptualization, supervision, funding acquisition, investigation, methodology, writing–review and editing.

This work was supported by the BMSTC (Fund number: D171100002617001 to J. Sheng). We gratefully acknowledge all the patients, their families, and the institutions for supporting these studies.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.