Accuracy of Self-Reported Reason for Colorectal Cancer Testing

It has become increasingly difficult to obtain medical records for research since the implementation of the Health Insurance Portability and Accountability Act in 1996 (1). Many researchers now rely on self-report, ascertained from mail surveys or interviews, for health-related information. Self-reported health behaviors are often compared with medical or administrative records to examine accuracy. Overall, self-reported colorectal cancer (CRC) testing behaviors are reasonably accurate (2, 3). Less is known, however, about the accuracy of self-reported reason for testing.

Qualitative studies have underscored that patients are confused about the definition of screening when asked to report the reason for a test (4). From a public health standpoint, being able to distinguish whether a test is done for screening (i.e., a test to identify “previously unrecognized disease or a disease precursor”; ref. 5) or for diagnosis (i.e., a test to evaluate the cause of signs or symptoms or to follow-up an earlier abnormal test) is important to assess the patients' knowledge and understanding of the need for a CRC test and to monitor trends in screening behaviors. For example, we know from several decades of research on cancer screening adherence that a common reason given for not being screened is not having symptoms (6, 7). From a clinical perspective, it is important for patients to understand the reason for a test to better communicate with their health care provider about their medical history.

Only four published studies have examined the accuracy of self-reported reason for CRC testing (8-11). These studies were limited in the measures of agreement they assessed, and none examined correlates of accurate self-reported reason for testing. To fill this gap, we examined the accuracy and correlates of self-reported reason for CRC testing using multiple measures of agreement in a racially diverse sample of patients from a large multispecialty clinic in Houston, TX, who had received at least one recent endoscopy: sigmoidoscopy or colonoscopy.

We used data from a randomized controlled trial designed to evaluate the reliability and validity of a self-report questionnaire of CRC testing behaviors using three modes of survey administration: mail, phone, or face-to-face (3). Self-reported data were compared with information in the medical record and administrative databases, hereafter called the combined medical record. To be eligible to participate in the trial, patients must have been English speaking, ages 51 to 74 y, and have been receiving primary care at the study clinic for at least 5 y. Patients with a history of CRC were excluded. From September 2005 through August 2007, 1,040 patients were recruited for the parent study, and 857 completed a baseline questionnaire. Further details on recruitment, eligibility, and study design of the trial are described elsewhere (3).

Data for this study consisted of 326 of the 857 patients who completed a baseline questionnaire (Fig. 1). To be included in these analyses (a) self-report and the combined medical record must have agreed that an endoscopy was done during the same time interval (i.e., within the past 5 y for sigmoidoscopy and colonoscopy), and (b) self-report and the combined medical record both provided a reason for testing. Only the most recent test was counted for patients with more than one of the same endoscopic procedures (e.g., more than one sigmoidoscopy during the study period). Twelve of the 326 patients in our study had received both endoscopic procedures; as such, they are included in both the sigmoidoscopy and colonoscopy analyses. Although a 10-y interval is recommended for colonoscopy, a 5-y interval was used in the parent study because of the difficulty of identifying a sufficient number of patients who had received care at the clinic for >10 y.

Data were available for sigmoidoscopy (n = 145) and colonoscopy (n = 193). Although data were available from the parent study on barium enemas and fecal occult blood tests (FOBT), they were not evaluated in this analysis due to sample size constraints (barium enemas) and because the reason for FOBT was not consistently recorded in the medical record. Reason for CRC testing was dichotomized as screening (part of a routine exam or checkup, or reasons unrelated to symptoms or an earlier abnormal test) or diagnostic (because of a symptom or health problem or follow-up to an earlier abnormal test).

Descriptive statistics included cross-tabulations and the following measures of agreement: concordance, positive predictive value, negative predictive value, sensitivity, and specificity. Predictive values were provided because we were interested in whether self-report was clinically useful as an accurate measure of reason for testing. Logistic regression was conducted to explore the association between concordance and the following variables: sex (female/male), age (continuous), marital status (not married/married), race/ethnicity (non-Hispanic white/African-American/Hispanic), education (less than high school/high school/general education diploma or some college/at least a college degree), and number of CRC tests that a patient had during the 5-y study period (one test/two or more tests; tests could include FOBT, barium enemas, sigmoidoscopy, and/or colonoscopy). Concordance with regard to reason for testing was dichotomized as agreement (coded one) between self-report and the combined medical record or as disagreement (coded zero). Measures of agreement and correlates of accurate self-reported reason for testing were calculated separately for sigmoidoscopy and colonoscopy. In univariable logistic regression models, we used a P value of 0.25 to identify correlates for inclusion in multivariable analyses (12). Variables in the multivariable model were considered statistically significant at P < 0.05. SPSS version 17 was used for all analyses.

The characteristics of our study sample were comparable with those of the parent study. The mean age of study participants was 58.5 y, and women comprised the majority of the sample (69%). The study sample was 59% non-Hispanic white, 25% African-American, 10% Hispanic (10%), and 6% other. The majority of participants reported receiving at least a college degree (57%), and most were married (78%). About 61% had received only one CRC test (i.e., FOBT, sigmoidoscopy, colonoscopy, or barium enemas) during the study period.

Frequency distributions by test type are shown in Table 1; measures of agreement are shown in Table 2. Concordance between self-report and the combined medical record on reason for testing was 92% for sigmoidoscopy and 78% for colonoscopy. Sigmoidoscopy and colonoscopy had high positive predictive values (97% and 80%, respectively), meaning that the majority of patients correctly reported having a sigmoidoscopy and/or colonoscopy for screening. Fewer patients could accurately report having a diagnostic sigmoidoscopy or colonoscopy, as shown by the negative predictive values of 27% and 71%, respectively. Sensitivity of self-reports was >90% for sigmoidoscopy and colonoscopy; specificity was ∼43% for both sigmoidoscopy and colonoscopy (Table 2).

Using our criterion of P < 0.25 in univariable analysis, we identified three correlates for inclusion in multivariable analysis: age (sigmoidoscopy and colonoscopy models), number of tests (sigmoidoscopy and colonoscopy models), and race/ethnicity (colonoscopy model). Age was positively associated with accurately reporting the reason for sigmoidoscopy (P = 0.07) and inversely associated for colonoscopy (P = 0.19). Compared with patients who had only one sigmoidoscopy during the study period, patients who had a sigmoidoscopy plus one or more additional CRC tests (i.e., FOBT, colonoscopy, and/or barium enemas) were less likely to accurately report the reason for having a sigmoidoscopy (P < 0.01). The same pattern was observed for colonoscopy (P = 0.23). Lastly, compared with non-Hispanic whites, African-Americans were more likely to accurately report the reason for having a colonoscopy (P = 0.18).

Age and number of tests were significant at P < 0.05 in the multivariable sigmoidoscopy model [odds ratio (OR), 1.20; 95% confidence interval (95% CI), 1.01-1.43; and OR, 0.05; 95% CI, 0.01-0.44, respectively); however, the number of tests was substantially skewed. Of the 12 patients who did not accurately report the reason for sigmoidoscopy, 11 had two or more CRC tests. For colonoscopy, although the patterns were similar to the univariable estimates, none of the correlates were significant at the 0.05 level in multivariable analysis: age (OR, 0.97; 95% CI, 0.91-1.03), number of tests (more than two tests compared with one test; OR, 0.62; 95% CI, 0.30-1.28), and race/ethnicity (African-Americans compared with non-Hispanic whites; OR, 2.00; 95% CI, 0.76-5.22).

Using the combined medical record as the gold standard, the majority of patients who reported getting a CRC test for screening were correct. Fewer correctly reported getting a CRC test for diagnosis, suggesting that some patients may not be informed by their health care provider or understand the reason for the test. Participants in our study were better able to report the reason for obtaining sigmoidoscopy than colonoscopy (92% versus 78%), perhaps because colonoscopy is recommended for both screening and diagnosis, whereas sigmoidoscopy is more frequently recommended for screening. Data from the combined medical record showed that diagnostic testing is more common for colonoscopy than sigmoidoscopy (30% versus 5%, respectively).

Given our relatively small sample size and because this is the first study to examine correlates of accurately reporting reason for CRC testing, the associations we observed need to be confirmed and further explored in future studies. The positive association between age and correctly reporting reason for sigmoidoscopy could be due to more experience with the health care system as one ages, thus resulting in greater awareness of one's medical history. Our finding that having two or more CRC tests during the 5 y study period decreased the likelihood of accurately reporting the reason for the most recent sigmoidoscopy or colonoscopy may indicate that patients become confused about the reason when they have multiple tests in a relatively short period of time.

Positive and negative predictive values, sensitivity, and specificity were not examined in any other studies, limiting our ability to compare our results with other studies. Additionally, no other studies assessed correlates of accurate self-reported CRC testing. The concordance estimates observed in our sample, specifically 92% for sigmoidoscopy and 78% for colonoscopy, were comparable or slightly better than the estimates found by Hall et al. (>70% for sigmoidoscopy; ref. 9), Gordon et al. (76% for sigmoidoscopy and colonoscopy; ref. 10), and Schenck et al. (65% for endoscopy; ref. 11). Khoja et al. (8) did not report concordance or provide the data to calculate it.

In this article, we aimed to answer the question: can we rely on self-reported reason for CRC testing? Based on our data, the answer is: it depends. If the goal is to identify individuals who have received a CRC test for screening, self-report is a reasonable choice. Self-report may not be a good choice, however, for clinicians who want to ascertain a patient's CRC testing history or for researchers trying to identify individuals who received a CRC test for diagnosis.

A problem with assessing the accuracy of self-reported reason for testing is limitations of the “gold standard,” i.e., medical or administrative records. Billing codes to capture the reason for testing may not be available and physicians may not record the reason for the test in the medical record, resulting in missing data. To increase accuracy and completeness, we used multiple record sources to measure reason for the test. Although it is possible that some CRC tests, particularly colonoscopy, were recorded as diagnostic for insurance purposes, this explanation is unlikely because of the legislation requiring insurance providers to cover the costs of CRC screening. The 2008 Colorectal Cancer Legislation Report Card (13) found that half of all U.S. states, including Texas, have the legislation in place mandating coverage of specific CRC tests for screening. In Texas, mandated coverage for CRC screening began for most health plans on January 1, 2002, before data collection for this study (14). Thus, it is unlikely that CRC tests were misclassified as diagnostic for insurance purposes.

Knowing the limitations of such databases, Haque et al. (15) constructed an automated data algorithm to distinguish between CRC tests obtained for screening versus diagnosis. Similar to our findings, compared with the medical record, the algorithm missed most of the diagnostic endoscopies, but performed well for tests obtained for screening purposes. Using data algorithms to distinguish between tests obtained for screening versus diagnosis is time consuming and, as shown by Haque et al. (15), not always accurate. Future studies should assess the accuracy of self-reported reason for testing in other populations and settings, as well as explore how the accuracy of patients' self-reported reason for CRC testing (especially for diagnosis) could be improved through better patient-provider communication. Providing information to patients regarding why specific CRC tests are recommended/ordered may facilitate the patients' understanding of the prescribed course of action and may result in better recall of one's CRC testing history when requested for research or clinical purposes.

No potential conflicts of interest were disclosed.

We thank Anthony Greisinger and the Kelsey-Seybold Clinic staff for supporting this project.

We also acknowledge the Doctoral Behavioral Science Seminar at the University of Texas School of Public Health, for their editing assistance on this manuscript.

Grant Support: PRC SIP 19-04 U48 DP000057. J.M. Eberth, A. White, and P.N. Abotchie are the recipients of National Cancer Institute Cancer Education and Career Development Predoctoral Fellowships (R25-CA057712).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.