An Experimental Test of the Effect of Incentives on Recruitment of Ethnically Diverse Colorectal Cancer Cases and Their First-Degree Relatives into a Research Study

Optimizing participant response rates is important for obtaining representative samples and for completing research studies in a timely manner. It is a common practice to use participant incentives and other strategies to boost response and retention rates, but few studies have systematically examined the effectiveness of such strategies, particularly among minority groups.

The literature indicates that offering monetary incentives or using recorded delivery methods such as registered mail can be effective strategies for boosting participation rates in survey research. A systematic review of 49 trials suggests that monetary incentives double the odds of response to a mailed questionnaire when compared with no incentive and that inclusion of the incentive with the initial survey is more effective than providing the incentive upon return of the survey (1, 2). Another systematic review focusing on strategies to improve recruitment to research studies (3) identified one trial recruiting adolescents into a health care randomized controlled trial that achieved significantly improved recruitment with a $2 enclosure (RR 1.43) and a $15 promise upon completion (RR 1.53) when compared with the no incentive group (4). Several studies suggest that the motivating power of such incentives is not so much in terms of their monetary value, but instead in their symbolic or token value (5, 6). Similarly, delivering recruitment invitations via registered mail may symbolize the importance of the research and serve to increase response rates (7). A review of six trials suggests that the odds of response are more than doubled when recorded delivery was used versus standard delivery (1). Typically, participation in health research is lower among minority groups compared with White populations (8) and boosting response rates in these groups could increase representation of these groups in research studies. Yet, the effect of incentives has rarely been studied in non-White populations (9-11).

This article focuses on the effect of incentives and delivery mechanisms for the recruitment of both cases and their first-degree relatives in the context of a randomized trial to increase colorectal cancer screening among first-degree relatives of colorectal cancer cases. Because our study included a large number of minority respondents, we were able to examine the effect of incentives separately among four ethnic groups: African Americans, Latinos, Asians, and non-Latino Whites.

Data were collected as part of a study testing a mail and telephone risk notification and barrier reduction intervention on colorectal cancer screening among first-degree relatives of colorectal cancer cases. The statewide California Cancer Registry was used to obtain random samples of African American, Latino, Asian, and White colorectal cancer cases diagnosed in California between 1996 and 1999. Cases were asked to enumerate and provide contact information for all their first-degree relatives. Relatives between the ages of 40 to 80 years, living in the United States, Mexico, or Canada, and English or Spanish speaking were invited to participate in the study. Both cases and relatives were informed that the research involved telephone interviews with relatives and mail and telephone interventions promoting colorectal cancer screening among relatives, and that results of the study would be used to develop better educational materials for colorectal cancer cases and their relatives. Initial recruitment of both cases and relatives was conducted via mail, followed by up to 10 telephone attempts if no response was received within 10 days. Letters to Latino cases and relatives were mailed in English and Spanish, and telephone contacts were made by English/Spanish bilingual interviewers. We did not have the resources to conduct the study in any of the Asian languages. Verbal consent was required for study participation. Cases and relatives had the opportunity to refuse to participate by mail or telephone. The study was approved by the University of California at Los Angeles Office for the Protection of Research Subjects.

The incentive experiment included cases and relatives with valid address information because the incentives were delivered through the initial mail contact. Based on previous research that suggested that registered mail and monetary incentives can increase response rates (1-7), we experimentally tested three different participant incentive strategies for their effectiveness in improving response rates among both cases and relatives in our study. For index cases, a 2 × 2 × 2 factorial design for the initial recruitment letter was created by crossing the following three conditions: (a) registered mail, which requires a signature indicating receipt, versus ordinary first class mail, (b) $5 cash enclosure versus no cash, and (c) $20 promise upon completion of the relative information form versus no promise. The outcome measure of interest for index cases was the provision of contact information on first-degree relatives. For relatives, a 2 × 2 × 2 design for the initial mailing was created by crossing the following three conditions: (a) registered versus first class mail, (b) $5 cash enclosure versus no cash, and (c) $10 promise upon completion of the baseline survey versus no promise. For the $10 promise, relatives had a choice of several equivalent alternatives: cash, a nationally valid telephone card, stamps, or participation in a lottery. The incentive amounts and choices were determined by our pilot work and budgetary resources. The reason for providing $20 to cases and $10 to relatives for completing of the required information was based on our prior work with breast cancer cases and relatives (12, 13), and information from the current trial before introduction of the incentives, indicating that response rates among cases were relatively low compared with response rates among relatives, suggesting that boosting response rates among cases would be a more productive strategy for achieving a more representative study sample. The outcome measure of interest for relatives was the completion of the baseline survey, which was the point of enrollment for the randomized trial to increase colorectal cancer screening. Cases and relatives were contacted in batches of ∼200 at a time, and subjects in each batch were randomized to one of the eight incentive conditions over a 24-mo recruitment period (see Table 1). The variation in sample size among the different incentive conditions within each ethnic group is due to the fact that all family members of an index case were randomized to the same incentive condition and family size varied. Although participants were unaware of incentive structures for other invitees, interviewers who did the telephone follow-up attempts were aware of the incentive status of all subjects.

We compared response rates among cases and relatives for the different incentive conditions for the total sample and within ethnic groups, using χ² tests. Because incentive strategies were not effective among cases, we limited subsequent multivariate analyses to relatives. We examined covariates associated with response rates including age, gender, relationship to case (child or sibling), stage at diagnosis (local, regional, or remote), registered versus first class mail, and four monetary incentive conditions ($5 cash plus $10 promise, $5 cash only, $10 promise only, and no monetary incentive). We also compared receipt of both monetary incentives ($5 and $10) to the no monetary incentive group. Finally, we inspected two interaction terms in the multivariate analyses, (a) between the four monetary incentive conditions and gender, and (b) between the four monetary incentive conditions and ethnicity. P values for both interaction terms were >0.4 and they were excluded from the final model. Because randomization was by family unit (the average family size was 2.2 and the maximum was 11), we used the GENMOD procedure, with a logit link, in SAS (Windows version 9.1), for our multivariate analyses. This procedure fits models to correlated responses using generalized estimating equations equivalent to logistic regression with SEMs adjusted for correlated data (14).

As shown in Table 2, a total of 3,816 cases were contacted by mail, of which 1,242 provided information on their first-degree relatives, for an overall response rate of 33%. Cases who were never reached (34%), who refused to participate (14%), who did not have eligible relatives (age 40-80 years, English or Spanish-speaking, and living in the United States, Mexico, or Canada, 9%), and who were deceased (6%) were included in the denominator. Overall, response rates ranged from 28% among Asian cases to 37% among Latino cases, and differences among the four ethnic groups were statistically significant at P value of <0.0001. The incentive strategies that were tested among cases did not result in increases in response rates. The only significant finding was that in the overall sample, registered mail seemed to have a slight advantage over regular first class mail.

A total of 2,353 relatives were contacted and 1,678 completed the telephone baseline survey, for an overall response rate of 71%. Thirteen percent were never reached, 10% refused, and 6% were age or language ineligible or had a history of colorectal cancer. Overall, response rates among relatives ranged from 66% among Asians to 76% among Whites, and differences among the four ethnic groups were statistically significant at a P value of <0.004. In bivariate analyses, registered versus regular first class mail had no effect on response rates in any of the ethnic groups or in the overall sample. Both the $5 cash upfront and the $10 promise upon completion of the baseline interview resulted in small but statistically significant increases in response rates in the total sample, but not within any ethnic group. A comparison of those who received any monetary incentive ($5 cash upfront or $10 promise or both) versus no monetary incentive showed statistically significant increases of about 13 percentage points among Asians and an increase of 8 percentage points in the total sample.

Table 3 shows response rates among relatives in each cell of the 2 × 2 design (the third factor, registered mail versus first class mail, was combined for this table because it did not affect response rates). There was a 10% absolute difference between the group that received both monetary incentives compared with the group that received no monetary incentive. This was the largest overall observed difference between incentive strategies in the study. Differences between other cell pairs in the factorial design were small.

Table 4 shows the results of multivariate analyses for the total sample of first-degree relatives and separately within each ethnic group. In the total sample of first-degree relatives (n = 2,184 with complete data on all variables), two payment schedules significantly increased the response rate: $5 cash plus $10 promise upon completion of the survey [odds ratio (OR), 1.65] and $5 cash only (OR, 1.47). In addition, females were more likely to respond than males (OR, 1.60) and Asians were significantly less likely to respond than Whites (OR, 0.66). The $10 promise only incentive had no significant effect on response rate (OR, 1.31).

Stratified analyses within ethnic groups indicated that the two payment schedules that included $5 cash upfront significantly increased the response rates among Latinos (OR, 1.94 and 1.74), but not in any of the other ethnic groups. African-Americans did not show an increase in response rate in for any of the payment schedules. In addition to the monetary incentive, female gender was significantly associated with increased response rate among Latinos (OR, 1.77) and Whites (OR, 1.80). Age, the relationship to case, stage at diagnosis, and registered versus first class mail did not significantly affect the response rates in any of the ethnic groups.

A separate analysis comparing the effect of $5 cash only versus $10 promise only upon completion did not indicate that one payment schedule was superior to the other in the total sample or in any of the ethnic groups (data not shown).

First-degree relatives who were given a choice of the $10 promise preferred to receive cash (57%), followed by postage stamps (21%), a chance to win a $200 lottery (11%), and prepaid phone cards (4%). Seven percent returned their incentive to us, and indicated that we should consider it their donation to cancer research.

This study provided the opportunity to test the effect of different incentives on response rates among colorectal cancer cases and their first-degree relatives who were recruited into a randomized trial. Because of the large number of cases and relatives recruited and oversampling of minority groups, we were able to examine the effect of incentives separately among White, African American, Latino, and Asian cases and relatives.

Contrary to findings from a review of six trials that found that the odds of response more than doubled when recorded delivery was used versus standard delivery (1), our data indicated that sending study materials by registered mail was not effective for improving response rates among colorectal cancer cases who were asked to provide contact information on their first-degree relatives or among first-degree relatives who were asked to participate in a study involving mail and a telephone contact. In fact, according to our telephone staff, some study participants were annoyed to have to go to the post office to retrieve the letter sent by registered mail, only to find out that it was pertaining to our study. Gibson et al. (7) also reported that some of their study participants expressed annoyance at receiving surveys via certified mail.

Different populations, incentives, and outcome measures (e.g., participation in a randomized trial over several months versus completion of a short mailed questionnaire) make comparisons to other studies that tested incentive strategies difficult. Coogan and Rosenberg (15) tested incentives in a population similar to ours, colorectal cancer cases, and healthy controls. Comparable to their findings, incentives failed to increase response rates among colorectal cancer cases in our study. We attribute this to the fact that we were requesting contact information on their first-degree relatives, rather than interviewing them about their own cancer experience. Some cases did not have family members that were eligible for our study, and some did not feel comfortable providing contact information for relatives due to concerns regarding privacy. Still, considering that we made cold contact with cases and that we were requesting fairly intrusive information regarding their relatives, a 38% overall response rate from cases is respectable. Among relatives identified by cases in our study, monetary incentives did influence response rates, although the effects were quite modest. Two prior studies (15, 16) reported significantly increased response rates with $1 and $5 cash enclosures in the initial mailing among healthy adults, similar to our healthy first-degree relatives. However, our response rate was much higher than in both of these studies (71% versus 56% and 22%), probably due to the fact that relatives in our study had been referred by their sibling, parent, or child and because of the cancer experience in the family.

There are a few studies in the literature that have examined the effect of incentives in specific ethnic groups. Whiteman and colleagues (10) reported findings separately for women 40 to 60 years of age living in minority and nonminority zip codes in the Baltimore Metropolitan Statistical Area. Minority zip codes had at least 30% residents of race/ethnicity other than non-Latino White. They found that response rates were lower within the minority zip codes than in the nonminority zip codes, and that inclusion of $1 in the initial mailing increased completion of a mailed questionnaire among women who were living in minority zip codes, but not in women living in nonminority zip codes. Although this study suggests that the enclosure of a small amount of cash in the initial mailing can increase response rates in a diverse sample of women that includes minorities, our study provides much more detailed information regarding the effect of incentives in the four major ethnic groups and for different payment schedules (cash upfront versus promise of cash upon completion). Our results conflict with two studies that assessed the impact of incentives in specific ethnic groups. The enclosure of $2 in the initial mailing among Medicaid enrollees by ethnic groups resulted in larger and more consistent increases in response rates among African-Americans than among Latinos (17). In another study among Chinese- and Korean-Americans, the enclosure of $1 in the initial mailing resulted in significantly increased response rates. In that study, only 14% of Korean-Americans and 24% of Chinese-Americans spoke English well and all study materials were provided in English and the native language (11). However, a recent study that tested the effects of different monetary incentives among African-American and White/other subjects also found that a small prepaid incentive but not a $10 or $20 promise resulted in the best enrollment rates (18). This suggests that the value of incentives in influencing response rates may vary by the type of incentives and payment schedules as well as by the ethnic groups being targeted.

We also compared our response rates with those achieved in two studies that recruited first-degree relatives of breast cancer cases using a similar approach as the one we used (19, 20). Both studies achieved high response rates among relatives (86-88%) and among cases (46-54%) without using incentives. Their response rates exceeded our rates, which may be due to a number of factors including the following: all cases and relatives were women who are usually more willing to participate in studies than men; a large number of cases were approached in person while undergoing treatment (compared with mail and telephone contacts in our sample); relatives were approached closer to the case's date of diagnosis, which may have increased their perceived risk and breast cancer worries, both of which were shown to be associated with participation (19). Overall, these high response rates in the literature in the absence of incentives coupled with the very small effects of incentives in our study suggest that cancer cases and first-degree relatives are motivated to participate in cancer control research, even without incentive payments, probably due to their personal cancer experience or that of a close relative. The two studies discussed above as well as our study all showed higher response rates among relatives than among cases. This suggests that incentives may be less important in achieving reasonable response rates compared with the context within which a study is being conducted and the saliency of the study goals for the target population.

Our study participants were recruited from a population-based sample of colorectal cancer cases and included large numbers of African-Americans, Latinos, and Asians, which is a major strength. The randomized factorial design used to systematically test the effects of various incentives among cases and relatives is a strength. A limitation is that due to the low response rate among the index cases, our final study sample of relatives may be nonrepresentative and the generalizability of findings from our study population to others may be limited.

Our randomized trial failed to show a major influence of incentives on response rates either among colorectal cancer cases or their first-degree relatives. The few positive findings among relatives were very modest. These findings call into question the value of providing incentives solely to boost response rates, at least among individuals who may be motivated to participate in a study due to the special salience of the topic for them.

No potential conflicts of interest were disclosed.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.