Mailing Strategies and Recruitment into an Intervention Trial of the Exercise Effect on Breast Cancer Biomarkers¹

Participant recruitment for intervention trials is often time-consuming and costly (1). Thus, developing and studying efficient, inexpensive, and high-yield recruitment strategies are crucial. Some common recruitment methods include advertisements in newspapers, radio, and television and presentations to targeted groups with potentially eligible individuals. Recently, targeted mailings, using DMV³ or voter registration lists, have been used to recruit participants in some intervention trials. This recruitment method has the advantage of reaching a large number of healthy people at a relatively low cost (2, 3, 4).

The contents and presentation of materials in mass mailings influence the response rate (5). For example, the initial response to the CARET trial was higher among those who received the recruitment packet via first-class versus bulk-rate mail (1). Other factors found to influence response include preliminary notification of the mailing, sponsorship, personalization, and a persuasive cover letter (4, 5).

As recruitment of the general public into intervention studies becomes more common, it is important to understand the effectiveness of different recruitment strategies, especially through pilot studies in the population of interest (6). This article discusses two components of the recruitment for an exercise trial targeting postmenopausal women (7): (a) a randomized pilot study of four mailing strategies; and (b) the results from the entire recruitment process.

The intervention study is investigating the effect of a 1-year moderate intensity exercise intervention on serum levels of endogenous sex hormones in postmenopausal women (7). Secondary end points include changes in weight, body mass index, percentage of body fat, fat distribution, and immune function. We randomized 173 postmenopausal women, ages 50–75 years, who were sedentary (<20 min of exercise 3 times/week) and overweight (body mass index > 25 or percentage of body fat > 33%). Participants resided in the Seattle, Washington area. Recruitment began in January 1998 and finished in July 2000. The study design is described in detail elsewhere (7).

The original goal was to randomize 168 women. The recruitment process identified potentially eligible women via mass mailings, media advertisements, and other sources, such as presentations to target groups, flyers, and word of mouth. Interested women contacted the study by returning a mailed interest survey or calling a designated telephone line, were screened for eligibility by a phone interview, and were asked to attend an information session. Major ineligibility criteria included declining participation, using hormone replacement therapy, not wanting to be randomized, being too physically active, and having medical conditions contraindicating moderate exercise. Eligible women were scheduled for a screening clinic visit and two treadmill tests to collect baseline data; those who successfully completed the screening process were randomized to an aerobic exercise intervention or stretching control group.

Women targeted for recruitment were identified via lists obtained from the Washington State DMV. Lists included the name, age, and address of persons with DMV activity over a 1–2-year period. We obtained three lists during the recruitment period. Mailings targeted women between the ages of 50 and 75 years, who lived in zip code areas that were less than a 30-min drive from our exercise facilities, where the intervention was implemented. To maintain a constant flow of potential participants we sent out weekly or biweekly mailings during the recruitment period.

Recruitment materials were personally addressed to potential participants and included the organization logo on the envelope. All packets contained a brochure about the study, a 2-page interest and eligibility survey, and a prepaid business-reply return envelope. The brochure outlined the purpose of the study and several key eligibility criteria. Women were asked to return the interest survey or call the study telephone number to learn more about the study.

During the first and second years of recruitment, we were unable to obtain a new DMV list, so we re-mailed women from our original list. We mailed packets to nonresponding women living in the zip code areas nearest to the intervention site up to three times. Re-mailings occurred from June 1998 to September 1999 and were interspersed with primary mailings to maintain a constant influx of potential subjects. Re-mailings were discontinued after we obtained new mailing lists from the DMV in September 1999.

Previous research suggests that different mailing strategies can affect the response rate (1, 4, 5, 8, 9, 10). Therefore, at the beginning of the recruitment period, we conducted a pilot study of four mailing strategies and used the results to determine the mailing method for use during the remainder of recruitment. We randomized 4,999 women into one of four groups, using a factorial design. The first factor was the inclusion or exclusion of an invitation letter from the principal investigator, and the second factor was the use of first-class stamps versus bulk mail. All four mailing types included the study brochure, the 2-page interest survey, and a reply envelope. The invitation letter personally invited the woman by name to join the study, outlined several eligibility criteria, and ensured confidentiality of any information supplied by the participant.

The recruitment packets for the pilot study were mailed 3 weeks apart in February and March 1998; block randomization assured that an equal number of mailings from each group was sent at both times. The cutoff date for response from either mailing was 24 weeks after the initial mailing because a second mailing was sent to nonrespondents at this time. A valid response included the return of the interest survey or a telephone call to the study office.

The pilot study data were analyzed approximately 2 months after the second mailing to determine the strategy for use during the remainder of study recruitment. Based on the results at that time and the cost of each mailing strategy, we chose to send the recruitment packets via bulk mail. We decided to include the personal invitation letter because of the low cost of incorporating it into the mailing.

To supplement mailing recruitment, we used several other strategies. We placed advertisements in a number of local newspapers, magazines, and newsletters for seniors. We also sent out Public Service Announcements to area newspapers, television, and radio stations. One major Seattle newspaper ran the announcement monthly from April 1999 to April 2000, and three major television stations did a segment on the local evening news in August and September 1999.

When analyzing the data for the pilot study of mailing strategies, responses were classified into four categories: (a) interested; (b) refused; (c) deceased; and (d) returned by the post office. We were primarily concerned with the proportion of women in each arm who were interested in participating in the study. We used the total number of women mailed as the denominator for this proportion because significantly more first-class packets than bulk mail packets were returned by the post office. This is an artifact of the post office’s policy to always return undeliverable first-class mail, but not bulk-rate mail, artificially inflating the overall response rate in the first-class mail group. Given the randomized design, we expected that the proportion of women reached in each group was similar, and therefore the total number of women mailed provided a reasonable estimate of the response rate of interested women.

We also determined the cost of randomizing one participant using first-class versus bulk-rate mailings. All prices used in this report are in 1998 dollars. In 1998, first-class postage was $0.320, and bulk-rate mailings cost $0.191 each, when sending 3000 packets at one time. The number of mailings needed to randomize one participant was determined by multiplying the number of mailings needed to obtain one response (the inverse of the response rate of interest) by the number of responses needed to randomize one subject (the inverse of the randomization rate). We used the overall recruitment randomization rate of 2.1% in this calculation. Numbers were rounded up to the nearest integer. The cost to randomize one participant equals the number of mailings needed to randomize one subject times the cost for that type of mailing.

We evaluated possible associations between mailing type or the inclusion of a letter and responses of interested women by estimating ORs using logistic regression models. The model initially included a binary variable for mailing type (bulk-rate versus first-class mail), a binary variable for inclusion of the letter (letter versus no letter), and a variable for the interaction between the two factors. Based on the Likelihood Ratio test, we found that the interaction was not significant (P > 0.50). The final model dropped the interaction term from the analysis.

For the total recruitment period, we examined response rates to mass mailings in three age groups (50–59, 60–69, and 70–75 years). Response rates for each group are the number of respondents divided by the total women reached by a mailing. The latter is the number of women mailed minus the number of recruitment packets returned by the post office. Because the post office does not return all nondeliverable bulk-rate mail packets, the response rate denominator is overestimated, and therefore the response rate is lower than the true value. Among those who replied, we examined the form of response (screening form, phone call, refused, or deceased).

We also considered response rates to first, second, and third mailings for women living in zip codes near the intervention site. Similar to the pilot study analysis, we determined the number of mailings needed to randomize one participant for first, second, and third mailings; costs to randomize one subject were determined using 1998 mailing prices. An exploratory analysis of the response rates by quarter of the year was conducted by examining the response of women to their first mailing; women who received a second mailing were considered to be nonrespondents to the first mailing. All comparisons were made using χ² tests.

Among responders to the different recruitment strategies (mailing, media, other), we examined the number of women who were initially eligible to enter the screening process and were ultimately randomized. The randomization rate for each recruitment strategy is the number of randomized women divided by the number of respondents. We use this denominator because the number of women reached by media and other sources is unknown. Comparisons were made using χ² tests.

A total of 4,999 recruitment packets were sent to potential participants, approximately 1,250 recruitment packets for each mailing strategy. We received 580 responses (11.6%) within 24 weeks: 184 packets were returned by the post office, 10 women were deceased (per notification by the family), 10 women refused, and 376 women were interested in learning more about the study (Table 1). There was a nonsignificant increase in the response rate from interested women between the first-class and bulk-rate mailings (OR, 1.19; P = 0.10), with response rates of 8.1% and 6.9%, respectively. We found no difference in the response rates of interested women between the mailings that included a letter and those that included only a brochure (OR, 1.00; P > 0.50).

Based on the response rate of interested women, 588 first-class mailings or 691 bulk-rate mailings would need to be sent to randomize one participant. Thus, in 1998 dollars, the cost to randomize one subject using first-class mail is $188.12, whereas the cost to do so using bulk-rate mail is only $131.98. Therefore, although first-class mailings yielded more responses than bulk-rate mailings, the cost of using first-class postage exceeded that of bulk-rate mail by $56.14/randomized participant. Using first-class mail would have added $9,500 to the recruitment costs needed to randomize 168 women.

Between January 1998 and April 2000, we mailed recruitment packets to 103,577 women in the Seattle area. Including multiple mailings to the same women, a total of 140,177 packets were mailed. The post office returned 1,118 packets, and we received 6,987 responses for a response rate of 6.8% (Table 2). Response rates between age categories were slightly different (P < 0.001), with the highest response in 60–69-year-old women (7.9%) and the lowest response in 50–59-year-old women (6.2%).

Most women (86.8%) who responded to a mailing did so by returning a completed screening form (Table 2). A significant difference existed in the response methods for women in different age groups (P < 0.001). Women in the oldest age group were slightly more likely to call the study phone line than the younger women (14.8% versus 11.3%). We also received a higher percentage of notifications from a family member that the woman was deceased in the oldest age category (2.3%); as expected, this percentage was the lowest in the youngest category (0.5%). The percentage of respondents who were randomized was not significantly different between age groups (P = 0.45).

Among women who received more than one mailing, response rates decreased with an increased number of mailings (Table 3). Of women living in Seattle who received their first mailing before July 1999, 7.8% responded to the first mailing, 5.2% responded to the second mailing, and 2.2% responded to the third mailing. These response rates are significantly different (P < 0.001). Also, there is a significant difference for the randomization rates of respondents to different mailings (P = 0.01). Women who responded to the first mailing had the highest randomization rate of 3.2%, whereas only 0.8% of those responding to their third mailing were randomized.

The responses to the first mailing received by potential subjects were significantly different between quarters of the year. The response to packets mailed during the second quarter (April through June) was the highest, with 6.4% of 43,222 women responding. The rates in the third (July through September) and fourth (October through December) quarters were 5.6% of 16,555 women and 24,810 women, respectively. The first quarter (January through March) had the lowest response rate of 4.9% of 43,222 women responding.

Most women (88.3%) who contacted the study responded to a mailing, whereas an additional 6.5% of women contacted the study in response to media promotions, and 5.2% replied in response to other recruitment efforts (Table 4). The proportion of women who completed the initial step in the screening process was similar across the three recruitment strategies (P > 0.50). The different strategies also yielded similar percentages (2.1–3.3%) of randomized participants (P = 0.10). About 0.14% of the 102,459 women reached by a mailing were eventually randomized. Similar values are not available for the other recruitment strategies because we do not know the total number of women reached by these methods.

This study explored a pilot study of mailing methods for recruitment into an exercise intervention trial and the subsequent recruitment experience. Results from the pilot study indicate that first-class postage leads to a nonsignificant, small increase in response over bulk-rate postage when recruiting older women into an exercise trial. This is somewhat inconsistent with findings from the CARET study, two meta-analyses of postage type on mailed surveys, and several randomized studies of postage on return envelopes that found that first-class postage yielded significantly higher response rates than bulk-rate mail (1, 8, 11, 12, 13, 14).

Despite the higher response rate to first-class mailings in this population, we found that bulk-rate mailing was the more cost-effective strategy when considering the cost to randomize one participant. The savings of using bulk-rate mail was just under $60 (in 1998 dollars) per randomized participant. Although a previous study indicated that the increased response to first-class stamps may offset the extra cost (15), the CARET study also had a lower cost per randomized participant when using bulk mail (1).

Thus two trials, recruiting different populations, reported that bulk-rate mail may be more cost-effective for recruitment into intervention trials than first-class mail, despite the higher response rate to the latter. Research from survey studies suggests that first-class mail is more cost-effective than bulk-rate postage; however, the different requirements of intervention trials make it difficult to translate survey research directly. This discrepancy exemplifies the need for additional studies of mailing response strategies specifically for intervention trials.

Our pilot study also showed that the inclusion of a personal invitation letter did not affect response rates in this population. This differs from two earlier studies in which a letter plus brochure had a higher response than a brochure only (16) or a letter only (17). This may be due to the visual appeal of our professionally designed brochures or because the letter did not outline additional information from the brochure.

During the entire recruitment process, we found that the older age groups had a significantly higher response rate to mailings than did the youngest age group. We initially expected that it would be more difficult to recruit the older women because of the higher incidence of failing health and lower social support. However, women over 60 years old may be more willing than women between 50 and 60 years old to participate in an exercise intervention trial. The older women are more likely to be retired and therefore have more free time to participate in a time-consuming study than younger, working women. A pilot study for the Systolic Hypertension in the Elderly Program reported that individuals over age 60 years are motivated to volunteer for clinical research trials (18), which is consistent with our findings. Additionally, older respondents had a randomization rate similar to that of the younger women. This suggests that intervention studies among women over age 60 years are feasible.

Due to logistical constraints, we mailed packets to some women multiple times. Results from those mailings suggest that second and third mailings have a lower yield than first mailings. Moreover, the women who responded to their second or third mailing were less eligible than those who responded to their first mailing. It is possible that women who responded to later mailings had less interest or were borderline for particular eligibility criteria. Our experience suggests that it may not be cost-effective to mail the same potential participant multiple times. However, because this was a convenience sample, further research is needed to confirm this result.

Although we found a statistical difference in response to a mailing by quarter of the year, the rates were qualitatively similar. Our results suggest that women sent mailings during April, May, and June may be more likely to respond than women mailed at other times of the year. This analysis is limited because the recruitment procedures were not designed to answer this question, and we were unable to control for possible confounding factors.

Although mass mailing was shown to be a successful method of recruitment in our study, media promotions and other methods also contributed substantially. The randomization rate was similar for women who responded to various recruitment methods, which reduces the concern that respondents to media and other methods might be less eligible than respondents to mailings. It is difficult to determine the cost of non-mailing strategies because most of the media exposure used for recruitment in this study was free. However, to randomize the 29 women from alternate sources via mass mailings, we would have had to send almost 21,000 more mailings over a 6–7-month period. Therefore, intervention trials using mass mailing as the primary recruitment strategy may benefit from including some alternative strategies.

This study has several limitations. First, the post office does not always return undeliverable bulk-rate mail, which affects the response rates for the pilot study and the total mailing recruitment. The response rates to bulk-rate mailings will be smaller than the true values. Second, media promotions and other recruitment methods were conducted concurrently with the mailing recruitment. We cannot estimate the effect of the other methods on women responding to the mailing recruitment. However, during the screening process, we verified the primary source of information that led the woman to contact the study.

This study illustrates the importance of understanding the effectiveness of various recruitment methods for intervention trials because recruitment success, cost, and staff time are directly affected. Although this study is recruiting older women, our results are generally consistent with findings from other intervention trials. More research about recruitment strategies for exercise and other behavioral intervention trials is needed to maximize the response rates from various recruitment strategies.