A Comparison of Self-reported Colorectal Cancer Screening with Medical Records¹

CRC³ screening is recommended for individuals at both average and increased risk by various agencies in both Canada and the United States (1, 2, 3, 4). The most common modalities for CRC screening are FOBTs, sigmoidoscopy, and colonoscopy; the recommended type and frequency of test vary with the risk level of the individual. Recommendations also vary depending on the health care system, and ultimately the choice of screening modality is made by the patient and/or their physician.

Many studies of CRC screening rely on self-reports of the type, reasons, and frequencies of various tests; yet there is a scant literature on the accuracy of such self-reports. The most thorough examination of the accuracy of self-reported CRC screening was carried out in a HMO in the United States (5). This study found very high accuracy for reporting of ever having a FOBT (96%), sigmoidoscopy (95%), and colonoscopy (89%), and for both endoscopic tests combined (96%). However, this study group was quite homogeneous (predominantly Caucasian and had belonged to the HMO for at least 4 years). This is in contrast to a study that compared self-reported FOBT with medical records in an African-American community health center. This study found that participants reported having FOBTs more frequently than they actually did [74% reported versus 11% verified (6)].

One study that compared the self-reports of FOBT in women over the age of 50 years with laboratory records found that self-reports overestimated the amount of screening, with self-reported rates exceeding actual rates by almost 14% (7). A survey of HMO members with regard to six different cancer screening practices, including FOBT and sigmoidoscopy, demonstrated that self-reports for FOBT and sigmoidoscopy were about 90% sensitive, but there was again a trend toward overreporting the procedures (8). A similar study found overreporting of FOBT in a telephone survey of specific ethnic groups (white versus Hispanic) but relatively accurate reporting of sigmoidoscopy (9). Whereas the literature to date indicates that there is a trend toward overreporting of FOBT, there have been few studies that include sigmoidoscopy, and only one that assesses the reporting of colonoscopy.

In general, relying on the reports of survey respondents to determine the frequency of cancer screening behaviors can lead to overestimations of screening activity (6, 7, 8, 9). There is also a need to assess the accuracy of CRC cancer screening self-reports in the context of a multiprovider health care system (where medical records must be separately obtained from physician’s offices, hospitals, and clinics) rather than within a contained health records setting such as that of a HMO. To date, the accuracy of self-reported CRC screening has not been reported in Canada, which has a universal publicly insured health system, with services delivered in a multiprovider setting. We assess the availability of records from various sources to determine whether they can be used as a “gold standard” for comparison with self-reports.

Subjects were accrued from an existing resource, the Ontario Familial Colon Cancer Registry (OFCCR). For a detailed description of the accrual protocol of the Registry, see the article by Cotterchio et al. (10). Briefly, the OFCCR is a population-based collection of information and biospecimens from newly diagnosed CRC patients in the province of Ontario. Patients who elect to participate are invited to provide specimens for genetic analysis, to provide further information about their diet and medical history, and to involve their family members. Patients provide the names and addresses of eligible relatives (adults who are parents, children, or siblings of a family member with cancer). These relatives are then invited by mail to participate in the OFCCR, and if they agree, they are sent a PHQ and diet questionnaire.

For this study, eligible participants are adults age 35 years and older unaffected by CRC who have returned a PHQ to the OFCCR. Individuals who completed the PHQ but who had indicated to the OFCCR that they wished no further involvement in research were excluded. As of April 2000, 722 relatives had sent back a completed PHQ to the OFCCR.

The “Bowel Screening and Health” section of the PHQ contained questions asking whether the subject had ever had a colonoscopy, sigmoidoscopy, or FOBT, along with a description of each of the tests. Subjects were also asked the reasons for any tests, when the first test was, when the most recent test was, and how many of each test they have had.

The package inviting eligible relatives to participate in this study included a cover letter, a reply form, a ROI form giving permission to obtain medical records, an information sheet summarizing the study, and a stamped, addressed return envelope. For participants who completed and signed a ROI form, medical records for FOBTs, sigmoidoscopies, and colonoscopies were requested from the source(s) indicated on the ROI form. Participants were asked to indicate a primary care physician in all cases, and a specialist, hospital, or clinic, if appropriate. The request was sent to physician’s offices by fax and/or mail and included a cover letter and copy of the signed ROI form. Hospitals were mailed the request, which included the cover letter and the original signed ROI form.

Physician’s offices were provided with a stamped, addressed return envelope to facilitate responses and were also provided with a fax number to which records could be sent. Hospitals were provided with the mailing address for returning records only. Different approaches were taken for physician’s offices and hospitals based on previous experience in requesting CRC screening records through the OFCCR. Hospital medical records departments have standard procedures and requirements for responding to requests for medical records, and records are normally received with little or no follow-up within 2 months. However, physician’s offices tend to have poorer response, thus the provision of a fax number and stamped envelope was made in an attempt to facilitate the return of medical records. Nonresponding physician’s offices and hospital medical records departments were called 3 months after the ROI form was sent to follow-up on the initial request.

Medical records received from physician’s offices and hospitals were reviewed to determine the type of tests, dates of tests, and, where possible, reasons for tests. The source of records received was documented. Because records were requested for putative “non-screeners” in addition to those self-reporting screening and diagnostic tests, telephone calls from physician’s offices stating that there were no CRC screening records filed for their patient were accepted as confirmation that no screening had taken place.

When the self-report indicated that a test had been done but the records did not confirm a test, cases were reviewed individually to determine possible reasons for the disagreement. Two-by-two tables were constructed for FOBT, sigmoidoscopy, and colonoscopy to compare the self-reports of having ever had the tests with the information in the medical records. For all three tests, the raw percentage agreement was calculated, as well as the κ statistic for agreement beyond chance. The sensitivity and specificity of self-reports using medical records as the gold standard were also calculated for each test.

Of 722 eligible subjects who were sent an invitation to participate and a ROI form, 612 replied (85%), and 398 agreed by completing and returning the ROI form (55% of 722). All participants had been asked to provide the name and address of a primary care physician where any screening records would be located; in all but 12 cases, the information was sufficient to send a request for medical records to a specific physician. A hospital or specialist clinic was identified on 231 ROI forms that were returned, and the information was sufficient in all but 7 cases. Examples of insufficient information include cases where the physician had retired or moved, and no alternate physician name was provided; incorrect address information; physician names for whom a current address could not be located; or hospitals that had closed.

Records were obtained from 285 family physicians (74% of those requested); 97 physicians did not reply, and 4 indicated that there was no such patient on file. For hospitals and specialists, 207 provided records (85% of those requested), 24 did not reply, 11 replied that there was no such patient on file, and 2 did not accept the ROI form because their hospital policy required a specific form to be used. Of the 398 subjects who participated, at least one medical record was obtained for 335 subjects (84%). The medical record was defined as either a copy of a chart note, procedural note, or laboratory report as well as direct correspondence or contact indicating that no such records were in the patient’s chart. A summary diagram of subject, physician, and hospital response rates is shown as Fig. 1.

Because completion of the PHQ was one of the eligibility requirements for entry into this study, there were near-complete data for each of the three tests included in the PHQ (FOBT, sigmoidoscopy, and colonoscopy). There were missing data in only four cases for FOBT information, three cases for sigmoidoscopy, and two cases for colonoscopy. No subject had missing data for more than one of the three tests.

Of the 722 eligible participants, 234 (32.4%) responded that they had undergone a FOBT, and 42 (5.8%) did not know whether they had ever had this test. For sigmoidoscopy, 166 (23.1%) of subjects reported having had the test, whereas 17 (2.4%) did not know. Two hundred and eighty-five subjects (39.6%) reported having ever had a colonoscopy, and only 6 (0.8%) did not know if they had ever had the test.

Of the 335 subjects with medical records, 306 indicated “yes” or “no” to having ever had a FOBT; 29 indicated that they did not know. The two-by-two table showing the self-reported FOBT versus medical record of FOBT is shown in Table 1.

A substantial number of subjects indicated that they had at some time had a FOBT (38%, n = 116), but this was not confirmed by the medical record (n = 82). Of these 82, the majority (n = 49) indicated that the FOBT was within the last 5 years (33 of the 49 were further confirmed as having had a FOBT by describing the procedure in detail during a telephone interview in a separate study). An additional 17 did not know when the FOBT was, and 16 indicated that the FOBT was more than 5 years ago (mean number of years ago = 16 years). Thus, most of the discrepancy between the PHQ report and the medical record for FOBT reporting would seem to be due to not receiving these records from the physician.

Of the 335 subjects with medical records, 323 indicated “yes” or “no” to having ever had a sigmoidoscopy; 12 indicated that they did not know. The two-by-two table showing the self-reported sigmoidoscopy versus medical record of sigmoidoscopy is shown in Table 2.

A substantial number of subjects indicated that they had at some time undergone a sigmoidoscopy (24%; n = 79), but this was not confirmed by the medical record (n = 60). Unlike the FOBT results, the majority of this group indicated that the sigmoidoscopy was more than 5 years ago (n = 30), and the mean number of years ago was 17.5 years. An additional 17 indicated that they had undergone a colonoscopy at the same time as the sigmoidoscopy, suggesting that there may have been confusion in self-reporting these two types of tests. An additional 10 subjects did not know when the sigmoidoscopy was. In the case of sigmoidoscopy, most of the discrepancy appears to be due to the length of time that had elapsed since having the test (and thus difficulty in retrieving these records). Indeed, only 8% of sigmoidoscopies reported 5 or more years ago could be confirmed, compared with 45% of those within the last 5 years. In addition, there may have been some confusion on the part of the subjects in distinguishing between colonoscopy and sigmoidoscopy.

Of the 335 subjects with medical records, 332 indicated “yes” or “no” to having ever had a colonoscopy; 3 indicated that they did not know. The two-by-two table showing the self-reported colonoscopy versus medical record of colonoscopy is shown as Table 3. Approximately half of the subjects reported having a colonoscopy, and there was very little discrepancy between the two sources with regard to colonoscopy reporting.

A comparison of the agreement statistics for each of the three screening procedures is presented in Table 4. Whereas overall agreement, chance-corrected agreement, sensitivity, and specificity were all high for colonoscopy (e.g., κ = 0.87), the chance-corrected agreement for FOBT (κ = 0.32) and sigmoidoscopy (κ = 0.29) was much lower.

The ability to compare self-reported screening with medical records is highly dependent on the type of screening test. For FOBT, there are a substantial number of subjects who very likely had the test, but for whom the medical record was not received. This could be due to FOBT reports (which are usually a single-page laboratory report) being “lost” within the patient’s file or difficult to retrieve. No reply was received from 25% of physician offices, despite follow-up attempts, and in some cases a participant’s entire medical record was received, despite clearly requesting only those records that pertained to CRC screening; these observations suggest that many physician offices are too busy to respond to research requests from studies such as this one. Whatever the reason, there is clearly a difficulty in tracking FOBT at the level of the family physician file.

In the case of sigmoidoscopy, however, the difficulty in obtaining records is primarily due to the length of time that had elapsed since the procedure or to the subject not recalling when the procedure was (which is most likely the case if the procedure was a long time ago). Records in Ontario are not required to be kept on file after 10 years, and even if they are kept, such records may go into archival filing, which is not readily accessible by the physician’s office. Whereas there is a good possibility that many subjects can accurately recall having had a sigmoidoscopy many years ago, these procedures are difficult to confirm with medical records. In addition, a subset of subjects may confuse the sigmoidoscopy procedure with colonoscopy; this indicates that perhaps better descriptions of the tests could be developed to help subjects to more accurately describe which test they may have had. This confusion between the two endoscopic procedures has also been noted in other studies (5, 11) and demonstrates the need to carefully consider the definitions of these procedures in survey and interview research.

Colonoscopy is the only test that proved feasible for directly comparing the self-report with the medical record; the agreement between the two sources was very high for this test. This may be because colonoscopies require an operative note to be dictated, and most colonoscopies take place in outpatient hospital clinics such that there will usually be two sources for the same record, the hospital chart and the family physician chart. Thus, the chances of confirming the procedure are higher than for FOBT and sigmoidoscopy, which are often only recorded in the family physician’s office chart. In addition, the majority of subjects who reported having ever had a colonoscopy also indicated that they had had a test within the last 5 years (data not shown), and so the recency of most colonoscopic procedures likely contributes to the availability of these records.

The only previous study that examined the accuracy of colonoscopy reporting found a high level of accuracy for reporting colonoscopy specifically and an even higher level of accuracy when all endoscopic procedures were combined (5). This is comparable with the current results, which found a high level of accuracy for colonoscopy alone. For FOBT self-reporting, other studies have consistently found that this test is overreported when compared with medical records. Although that also appears to be the case in the current study, it is not possible to determine whether there is true overreporting of the test or whether the discrepancy exists because of difficulty in obtaining FOBT records.

The population used for this study was accrued through a familial CRC registry, and participants had already indicated a willingness to take part in research by completing a PHQ and further agreeing to grant permission to have their medical records reviewed. In addition, the study cohort was restricted to relatives of CRC patients. Thus, the results of this study may not be applicable to research carried out in a broader population; it is possible that those who participated in the study are more health conscious with regard to CRC and have a more accurate recall of CRC screening tests.

Overall, using self-reported colonoscopy data should be acceptable for future studies when researchers need to determine whether or not a subject did indeed undergo the procedure and when the most recent colonoscopy took place. The self-report of FOBT is probably accurate in the majority of cases, but this is difficult to confirm due to the lack of medical records received from physician’s offices for this particular test. For sigmoidoscopy, it may be helpful to further describe the procedure to help subjects differentiate between it and colonoscopy.

We thank the individuals who participated in this study. We are grateful to Ferhana Khan and Sandy Passarelli for work with the data collection. We also thank Dr. Steve Gallinger (Principal Investigator of the OFCCR) for facilitating access to the PHQ data and for helpful comments.