Validity of Self-Reports of Return for Routine Repeat Screening in an Ovarian Cancer Screening Program

Ovarian cancer is the fourth leading cause of death from cancer in American women, with 22,220 new cases and 16,210 estimated deaths in 2005 (1). Early detection of ovarian cancer is critical. Whereas the 5-year survival rate is 93.6% when ovarian cancer is diagnosed at the local stage, 5-year survival rates decline to 68.1% and 29.1% when ovarian cancer is diagnosed at the regional and distal stages, respectively. Unfortunately, as ovarian cancer is associated with few reliable signs and symptoms, only 19% of ovarian cancer cases are detected at the localized stage when prognosis is excellent (1).

Given early diagnosis is critical to prognosis in ovarian cancer, development of an appropriate screening test for ovarian cancer is an important research goal. Current approaches to the detection of ovarian cancer include transvaginal sonography (TVS), CA-125 bioassay, and a multimodal approach, including both CA-125 bioassay and TVS (2, 3). TVS and CA-125 bioassay have been tested, both alone and in combination, and have shown some value in the early detection of ovarian cancer (3-9). For example, TVS screening alone in asymptomatic women has been shown to have 81% sensitivity, 98.9% specificity, 9.4% positive predictive value, and 99.97% negative predictive value (9). Although promising, data from prospective, randomized, and controlled trials linking ovarian cancer screening to reduced ovarian cancer-related mortality is lacking at the present time. Although the ongoing Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial (10) may clarify the value of ovarian cancer screening, mass screening of asymptomatic women for ovarian cancer is not currently recommended by any professional organization (11). However, TVS screening for ovarian cancer, either alone or in combination with CA-125 testing, is presently recommended for symptomatic women and women at high risk for ovarian cancer (1, 12).

The effectiveness of any approach to cancer screening is predicated on appropriate uptake of that screening modality. Research on the uptake of cancer screening relies on reliable and valid measurement of cancer screening behavior. One approach to measurement of screening behavior involves inspection of medical records. However, this approach can be costly and may be limited when individuals can potentially undergo screening at multiple clinical sites. An alternative approach relies on self-reports of screening. Self-reports have been used in a variety of studies of screening for breast, prostate, colorectal, and cervical cancers (e.g., refs. 13-17). Self-reports of screening behavior are less expensive to collect and eliminate the need to examine clinical records at multiple sites. However, although self-report represents a convenient and economical means of assessing uptake of cancer screening, self-reports can introduce measurement error into the data through intentional or unintentional inaccuracies in the reports. Respondents might err in their recollection of whether and when they participated in a particular screening activity. Alternatively, respondents might intentionally misreport their screening behavior to appear more socially desirable or simply to please the researcher. Consequently, it is important to establish the validity of self-reports of cancer screening.

Previous research has generally supported the validity of self-reports of screening in the breast, colorectal, prostate, and cervical cancer settings, with concordance between self-reported screening behavior and program records typically >70% (e.g., refs. 18-23). However, overreporting of screening behavior is often noted, and the validity of self-report often seems to vary based on the salience of the screening experience or length of recall period. Research has also shown that individuals often underestimate the amount of time that has passed since their last screening test (i.e., forward telescoping; refs. 19, 21-23).

No research has examined the validity of self-reports of participation in ovarian cancer screening. Consequently, the primary aim of the present study is to examine the validity of self-reports of participation in TVS screening for ovarian cancer among asymptomatic women participating in a well-established TVS screening program for ovarian cancer. The secondary aim of the present study is to examine demographic and clinical variables that may be associated with greater or lesser validity of self-reports of TVS screening.

The study sample consisted of new participants in the ovarian cancer screening program at the University of Kentucky Markey Cancer Center (9, 24). This program, established in 1987, offers free, annual TVS screening to all asymptomatic women ≥50 years of age and to asymptomatic women 25 to 50 years of age who are postmenopausal or who possess a family history of ovarian cancer (9). Participants in the current validity study were enrolled in a larger behavioral study of response to ovarian cancer screening (25, 26). Women were eligible to participate in the behavioral study if they were ≥25 years of age, had no history of TVS screening for ovarian cancer, and were able to read and understand English. Using these criteria, 707 women in a consecutive series were invited to participate between April 2000 and April 2002. All women were asymptomatic, meaning they were not participating in the ovarian cancer screening program due to symptoms that might be indicative of ovarian cancer. Of these 707 women, 624 (88.26%) provided consent for study participation and 622 underwent an initial TVS screening test. Thirty-seven women subsequently received an abnormal TVS test result and were excluded from all study analyses. The remaining 585 women received a normal TVS test result and were given a recommendation to return in 1 year for a routine, repeat TVS screening test. These 585 women constituted the sample of interest in this validity study.

Potential participants were approached by a member of the research staff in the clinic waiting area of the University of Kentucky Ovarian Cancer Screening Program. Eligible women were invited to participate in a study of psychological and behavioral response to participation in an ovarian cancer screening program. Information about study procedures was provided to interested women, and written informed consent was obtained from those agreeing to participate. Enrolled women completed a baseline questionnaire packet before undergoing a routine TVS screening test. Participants then completed a set of follow-up questionnaires by telephone interview 1, 4, and 18 months following their initial, baseline TVS screening test. The baseline and follow-up assessments collected a variety of demographic and psychosocial information (25, 26). Recent ovarian cancer screening behavior, the variable of primary interest in the present study, was assessed during the 18-month telephone interview. Following the 18-month interview, computerized records from the screening clinic program were examined and information about participation in TVS screening following a woman's initial, baseline TVS test was recorded.

Demographic information obtained at the baseline assessment included age, race, marital status, and education. Objective ovarian cancer risk information was also gathered at baseline, specifically, whether a woman had a first-degree relative (i.e., mother, sister, and daughter) with ovarian cancer.

During the 18-month follow-up interview, participants were asked how much they agreed with the following item on a four-point scale from 0 (strongly disagree) to 3 (strongly agree): “A TVS screening test can find ovarian cancer early.” This item was used as an index of perceived efficacy of the TVS screening approach. Recent ovarian cancer screening behavior was also assessed. Participants were reminded of the date they completed their initial TVS screening test at the University of Kentucky. They were then asked three questions about their receipt of additional ovarian cancer screening since that date. They were asked whether they had received any additional screening tests for ovarian cancer since that date (yes versus no). If yes, they were asked when (month and year) and where they had received this additional ovarian cancer screening. The latter question identified women who reported they had undergone ovarian cancer screening since their initial TVS test at the University of Kentucky but might have done so at another screening site.

Self-reports of return for repeat ovarian cancer screening since a woman's initial baseline TVS screening test were compared with University of Kentucky Ovarian Cancer Screening Program records of return for repeat ovarian cancer screening. Based on this comparison, a woman's self-report of return for repeat ovarian cancer screening was classified as accurate (true positive or true negative) or inaccurate (false positive or false negative). Overall agreement was defined as the percentage of study participants whose self-report of return for repeat ovarian cancer screening was accurate. To correct for chance agreement, the κ statistic was calculated. The sensitivity of ovarian cancer screening self-reports was calculated as the proportion of participants correctly reporting having undergone an additional ovarian cancer screening test among all those in the sample who had actually undergone additional ovarian cancer screening, according to ovarian cancer screening program records. The specificity of ovarian cancer screening self-reports was calculated as the proportion of participants correctly reporting that they had not undergone an additional ovarian cancer screening test among all those in the sample who had not actually undergone additional ovarian cancer screening, according to ovarian cancer screening program records. The positive predictive value was calculated as the proportion of participants who correctly reported that they had undergone additional screening among all participants who said that they had undergone additional screening. The negative predictive value was calculated as the proportion of participants who correctly reported that they had not undergone additional screening among all participants who said that they had not undergone additional screening.

Of the 585 women eligible to participate in the study based on receiving a normal TVS test result, 44 did not complete the 18-month follow-up interview and thus self-reports of participation in ovarian cancer screening were not available from these women. Consequently, these 44 women were excluded from study analyses. Additionally, 13 participants reported that they had undergone additional ovarian cancer screening since their initial TVS test at a site other than the University of Kentucky. Of these 13 participants, a record of return for repeat ovarian cancer screening was found in University of Kentucky screening program records for 7 of these women, and these participants were included in analyses. As there was no objective means of determining the accuracy of their reports of return for repeat ovarian cancer screening for the additional six participants, they were excluded from all subsequent analyses. Thus, the final study sample on which the validity analyses were based consisted of 535 women.

The sample was predominantly Caucasian (n = 523; 98%) with a mean age of 61 years (SD, 10) at the time of the 18-month interview. Mean years of education was 14 years (SD, 3). Forty-nine (9%) women reported a first-degree relative with ovarian cancer. There were no significant differences between the final study sample (n = 535) and the 44 women excluded from study analyses due to a failure to complete the 18-month telephone interview with regard to age, education level, or minority status. Women who failed to complete the 18-month follow-up interview were more likely to report a first-degree relative with ovarian cancer (23%) than women in the final study sample (9%; P < 0.01). Of the 44 women who failed to complete the 18-month interview, program records indicated that only 22 (50%) returned for a repeat TVS screening test compared with 81% in the final study sample (P < 0.001).

Perceived efficacy of TVS screening was very high. Four hundred sixty-two (86%) participants indicated strong agreement with the statement “A TVS screening test can find ovarian cancer early,” with an additional 13% of study participants indicating agreement with this statement. Thus, 99% of study participants strongly agreed or agreed with this statement.

Overall agreement between self-reports of return for repeat ovarian cancer screening and screening clinical program records was 98% (P < 0.001). To correct for chance agreement, the κ statistic was calculated and found to be 0.94. Negative self-reports were confirmed in 159 of 166 cases (specificity, 0.96; negative predictive value, 0.96), whereas positive self-reports were confirmed in 363 of 369 cases (sensitivity, 0.98; positive predictive value, 0.98). These results are shown in Table 1. Of the 363 participants who reported additional screening and for whom this report was confirmed with program records, 290 (80%) correctly identified the exact month of screening. An additional 38 (10%) participants were off by 1 month in their identification of the screening month. Of these 38, 17 reported screening in the month before that recorded in program records, whereas 21 reported screening in the month after that recorded in program records. Thirty-three (9%) participants were >1 month off compared with program record date of screening, whereas 2 (1%) women reported that they did not remember the month of their screening.

Additional analyses were conducted to identify demographic and clinical variables associated with greater or lesser accuracy of self-reports of return for repeat ovarian cancer screening. Results indicated no significant differences in likelihood of agreement between self-reports and objective reports of return for repeat ovarian cancer screening as a function of age (P = 0.57), education (P = 0.98), minority status (P = 0.18), or presence of a first-degree relative with ovarian cancer (P = 0.08).

A recent review of literature on the validity of self-report of cancer screening suggests that, across studies, weighted averages of concordance rates have ranged from 0.71 to 0.89, with sensitivity ranging from 0.68 to 0.91 and specificity from 0.40 to 0.90 (27). Thus, our data suggest that the validity of self-report of TVS screening is generally superior to the validity of screening self-reports in other modalities, although the high level of validity here must be considered in light of methodologic factors which affect accuracy. In this study, female volunteers underwent an initial TVS screening test and received a recommendation to return for repeat routine screening in 12 months. During the study interview, women were reminded of the date of their initial TVS test and were only required to recall whether they had engaged in another screening within a relatively narrow period, as interviews were completed at an average of 18.35 months following initial screening (range, 18-24 months). Thus, the task was not difficult relative to other potential screening report questions, such as asking whether someone has ever undergone a prostate-specific antigen screening test or how many mammograms a woman has received over an extended period. Prior research on self-report validity in other cancer modalities over similar short recall periods suggests that concordance between self-report and medical records is very high. For example, the accuracy of self-reports of mammography utilization in the past year was 98% in one study (22), whereas the accuracy of self-reports of mammography and Pap smear screening in the past year was 96% and 94%, respectively, in another (23). Thus, although the validity of TVS screening seems to be superior to the validity of self-reports of screening in general, the 98% concordance rate is clearly comparable with that for other screening modalities over similar recall periods. Future studies examining the validity of self-report of TVS screening in the general population may find lower concordance rates if longer recall periods or fewer recall cues are used.

Of note, the current data do not suggest that forward telescoping (19, 21-23) occurred. This conclusion is based on the high rate of accurate identification of the month of the most recent screening test and the approximately equal distribution of incorrect responses before and after the actual screening month. This may suggest that the relatively short recall period used in this study does not allow enough temporal expanse for forward telescoping to occur.

Accurate recall of TVS screening might be fostered because participation in TVS screening is a relatively salient event. Although TVS testing does not generally involve some potentially salient elements involved in other screening procedures, such as pain, discomfort, sedation, or a strong educational component, TVS testing is different from other cancer screening tests in ways that can enhance its salience. Specifically, TVS screening is not conducted in the context of a routine physical exam, as, for instance, screening for prostate or cervical cancer might. Rather, a woman has to schedule an appointment specifically to undergo TVS screening and make a special trip to the clinic to undergo that screening. From a memory recall standpoint, this likely makes whether a woman participated in TVS screening in our study fairly easy to recall accurately.

Self-presentation motives can also affect the validity of self-reports of cancer screening. Individuals may be motivated to make socially desirable responses when asked to report whether they had undergone repeat ovarian cancer screening. Presuming adherence with recommendations for annual ovarian cancer screening is socially desirable, this might suggest a tendency for some women to intentionally report that they had returned for repeat ovarian cancer screening when in fact they had not (i.e., false positive). Social desirability motivation might be enhanced in an interview, in which screening behavior is reported directly to another person, as opposed to more impersonal methods of data collection, such as completion of a mail or internet-based questionnaire. Although such intentional dissembling is possible, our data suggest that it is infrequent. Only six women, <2% of our sample, reported a return for repeat ovarian cancer screening at the University of Kentucky when, in fact, no objective record of their doing so was available.

Several additional methodologic factors affecting the interpretation of the current results should be noted. As this study examined only the validity of self-reports of TVS screening, the validity of self-reports of CA-125 screening for ovarian cancer might differ. In fact, as CA-125 screening involves only a blood draw, a perhaps less salient event than TVS screening, we might speculate that validity may be slightly lower for reports of uptake of CA-125 screening. However, this is an empirical question that remains to be determined. In addition, our sample consisted of a homogeneous group of predominantly Caucasian women. Consequently, whether our results generalize to other, more heterogeneous, samples is unknown. Additionally, although we found that the likelihood of an accurate self-report of return for ovarian cancer screening was not associated with demographic variables or family history of ovarian cancer, our base rate of screening accuracy was very high with few instances of inaccurate recall, so our ability to identify variables associated with the accuracy of self-reports of screening behavior was consequently limited. However, this should not be viewed as a weakness of our study per se but is a by-product of the low base rate of inaccurate self-reports in this screening context. Finally, as routine screening for ovarian cancer by average risk women is not currently recommended by any professional organization, the importance of establishing the validity of reports of ovarian cancer screening in general, or TVS screening in particular, might be questioned. However, screening for ovarian cancer using TVS is currently recommended for women at high risk for ovarian cancer (1, 12) and the ongoing Prostate, Lung, Colorectal, and Ovarian Cancer screening trial might well establish the usefulness of routine screening for ovarian cancer in average risk women (10). Thus, there is both current as well as potential future value in establishing the validity of self-reports of TVS screening.

In conclusion, further research is necessary to confirm and extend these initial findings. However, at this time, the use of self-reports of TVS screening for ovarian cancer seems to be an efficient substitute for more objective, yet more costly, methods of assessing ovarian cancer screening behavior.