Response Rates to a Mailed Survey Targeting Childhood Cancer Survivors: A Comparison of Conditional versus Unconditional Incentives

Mailed surveys are frequently used to collect data for epidemiologic and other health-related research. Poor response rates pose challenges to investigators because they may interject bias and threaten study validity. Strategies for increasing response rates have been recently reviewed by Edwards et al. (1) with consensus that monetary incentives more than double response rates (odds ratio, 2.02; 95% confidence interval, 1.79-2.27) in surveys conducted in adults as well as in children. The timing of incentives, however, is not as well-characterized and questions remain whether incentives should be included within initial mailings or whether better response rates are realized when incentives are provided upon survey completion. To date, 13 studies have been conducted and the majority show that response rates are higher when incentives are provided at the time of survey administration rather than upon return (1-4). However, the majority of these studies were conducted in healthy populations; to date, no studies have explored conditional (incentive provided upon survey completion) versus unconditional (incentive provided at the time of survey administration) incentives among clinic-based pediatric populations and/or their parents or guardians.

Our goal was to determine if response rate to a mailed survey aimed at childhood cancer survivors differs if a monetary incentive ($10) is inserted within the initial mailing or is mailed to subjects after receipt of their completed survey. Like previous studies, we hypothesized that within this group of childhood cancer survivors and their parents, response rates would also be higher among those incentivized before completing the survey.

This study of immediate versus delayed incentives was embedded within a mailed survey study aimed at assessing health-related behaviors among childhood cancer survivors and their parents (Institutional Review Board protocol 4635-03-3R0ER). The sample for this study was comprised of childhood cancer (lymphoma, leukemia, or central nervous system cancer) survivors who were between the ages of 11 and 33 and who were at least 1 year out from completion of treatment with either no evidence of disease or stable disease. The time from the end of treatment ranged from 1 to 28 years. Cases were identified through the various registries at Duke University Medical Center (i.e., Central Cancer Registry, Brain Tumor Center, Young Cancer Survivors Program, and Children's Cancer Center), and approved for contact by their primary oncology care physician. The sample, comprised of 497 cases, was block randomized into two groups based on gender, age (<18 versus ≥18 years), and race (White versus non-White) because these parameters have been shown to influence response rates in previous studies (1). In addition, the sample was block randomized on cancer type. Assuming response rates similar to those reported by Edwards et al. (1), the study had 83% power (two-sided, α = 0.05) to detect a difference between the incentive arms. The power calculations were based upon seeing an effect size difference of between 50% and 64%; these calculations were based upon an assumption that we would achieve response rates that were similar in magnitude to previous studies (2) and would allow us to detect differences between groups given a response rate of 50% in one group versus 64% in the other.

One group received the incentive ($10 bill) attached to the cover letter, “as a gesture of thanks.” The other group received a cover letter with information that they would receive *$10 in cash upon receipt of their completed survey. Both groups received an 11-page survey along with a postage-paid, preaddressed envelope for return. If the childhood survivor was under 18 years of age, the survey was included within a mailing sent to the parents. Parents also received a survey and were incentivized in an identical manner as the child; a separate envelope for return was also provided. We did not specify which parent should fill out the questionnaire nor did we include an item to indicate which had done so. Data for parent responses was not stratified by the type of disease their child had due to sample size considerations.

We allowed a significant amount of time for the surveys to be returned to us; after a 2-month period, nonrespondents received a follow-up telephone call using a standardized script. In the case where additional surveys were mailed, no additional incentive was supplied. Additional surveys were mailed to those who stated that they did not receive the original survey. In cases where phone numbers were inactive or subjects were unreachable, it was assumed that the survey never met its intended target (n = 61). This number was added to the number of surveys returned by the post office with no forwardable address (n = 29), as well as those going to subjects who later were identified as deceased (n = 10); this total was subtracted from the denominator yielding an evaluable sample of 397. Whereas we cannot exclude the possibility that some (or perhaps all) of the nonrespondents may have received the surveys, kept the incentive, and then denied either living at the contact address or phone number, we thought this to be unlikely and thus excluded these patients from our totals. An analysis of each of the subgroups that we were not able to contact showed that there no significant differences between assignments to the conditional or unconditional incentive arms of the study (data not shown). The χ² test of proportions was used to compare the response rates in the two arms (as well as within strata).

The evaluable sample was comprised of 215 cancer survivors who were currently >18 years of age, and 182 survivors who were younger (plus their parents). A majority of the survivor sample was White (83%) and male (55%), and the distribution among cancer types was roughly 40% central nervous system cancers, 21% lymphomas, and 39% leukemias. There was also no association with response rate with time from completion of therapy.

No significant differences in response rates were observed on the basis of age, gender, race, or disease category. However, there was a significant difference (P < 0.0001) in overall response rates between incentive arms, with a 44.8% overall response rate noted among those who were offered the conditional incentive versus a 63.9% response rate among those who received the unconditional incentive. Significant differences in response rates were also detected within the adult survivors, child survivors, and parents. Odds ratios (95% confidence intervals) for unconditional versus conditional incentives for adult survivors and child survivors and their parents were 1.31 (1.03, 1.67), 1.43 (1.08, 1.90), and 1.56 (1.17, 2.08), respectively. Response rates for each of these groups of subjects, as well as groups stratified by race, gender, and disease type are provided in Table 1.

Survey research standards for studies conducted within the general population purport a 60% response rate as a standard for “acceptability”—a standard that often requires additional effort and resources associated with precontact, incentives, or reminder postcards or calls (5, 6). In this study of childhood cancer survivors and their parents, 60% response rates were achieved via a follow-up phone call, plus the use of a *$10 incentive—but only if the incentive was provided concurrently with the survey and not conditional upon survey completion and return. These findings are in direct contrast to those reported in a recent survey study of 855 prostate cancer survivors, where a 60% response rate was achieved regardless of conditional or unconditional incentive, and without the use of follow-up telephone calls (2). The discrepancy between studies may be attributable to obvious age differences between these two populations of cancer survivors. This explanation is supported by data of Caan et al. (7) who confirm a positive association between age and response rates to health behavior surveys. Further, the lack of response rate differences in the previous study may have been due to participants' valuing the incentive, a 30-minute prepaid phone card, <*$10 in cash. Indeed, our findings are more similar to the previous studies reviewed by Edwards et al. (1), which indicate that response rates are higher among those who received unconditional compared with conditional incentives. It must be noted, however, that our odds ratios were more modest (1.3-1.5 compared with 1.7), suggesting that either this survey or this sample taking it differed somewhat from surveys and samples included within the prior review.

A previous methods paper by Mertens et al. (8) describes a host of barriers that must be overcome to conduct survey research in childhood cancer populations. Ensuring that substantial numbers of children and their parents are not to be lost to follow-up in today's highly mobile society requires resources such as comprehensive databases and intensive tracking systems. The fact that we were unable to locate 20% of our original sample is testimony to this need and acknowledged as a limitation of this study. Although our sample provided ample power to detect differences in response rates overall and within the major age groups of patients and their parents, we lacked adequate power to test for differences once these groups were further stratified by race, gender, or cancer type. Mertens et al. (8) also described an intensive campaign using reminder postcards and several follow-up telephone calls to achieve response rates that are deemed acceptable. Findings of this study suggest that the provision of study incentives is helpful in eliciting mailed survey responses, but that incentives need to be provided unconditionally to have optimal impact. These findings can inform future survey research conducted in other populations of childhood cancer survivors, as well as pediatric populations, in general.

We thank the efforts of Dr. Bercedis Peterson, Denise Snyder, Shelley Rusincovitch, Amy Hensley, Meredith Wood, and Cheri Willard.