Objectives: We compared strategies and costs associated with recruiting unmarried middle-aged and older women who partner with women (WPW) and women who partner with men (WPM) into an observational study about experiences with cancer screening.

Methods: We used targeted and respondent-driven sampling methods to recruit potential participants. Comparable recruitment strategies were used for WPW and WPM.

Results: During 25 months (June 1, 2003, through June 30, 2005), 773 women were screened for study eligibility; 630 were enrolled (213 WPW, 417 WPM). Average staff time spent for recruitment was 100 min per participant. There were no differences by partner gender in average recruitment time (WPW, 90 min; WPM, 100 min). Print media was the most efficient recruitment mode (time per participant: 10 min for WPW, 15 min for WPM). Recruitment costs differed by partner gender ($140 for WPW, $110 for WPM). Costs associated with print media were $10 per WPW and $20 per WPM. Recruitment through community events had higher costs ($490 per WPW, $275 per WPM) but yielded more women with less education and lower incomes, who identified as a racial or ethnic minority, and self-reported a disability. Compared with WPM, WPW had more education and higher incomes, but were less likely to identify as a racial minority and self-report a disability.

Conclusions: There was a trade-off between cost and sample diversity for the different recruitment methods. The per-person costs were lowest for print media, but recruitment through community events ensured a more diverse representation of unmarried heterosexual and sexual minority women. (Cancer Epidemiol Biomarkers Prev 2007;16(12):2605–14)

Since the 1994 NIH mandate for more diversity in research populations (1), the number of studies about strategies and costs associated with recruiting hard-to-reach, underserved populations has increased. The majority of these studies have focused on strategies for enrollment of racial and ethnic minorities (2), and focused on recruitment related to particular disease conditions. Fewer studies have focused on individuals without a specific health condition. Individuals with a particular health condition may have different motivations to participate than healthy individuals. Finally, most published recruitment information is from large-scale clinical trials or intervention studies. Implementation of findings from these studies is often impractical for observational research efforts with more limited budgets.

Sexual minority (e.g., lesbian and bisexual) women are an important segment of the unmarried female population and represent another difficult-to-reach minority population (3). Sexual minority women may be living in committed relationships comparable with married heterosexual couples but are unable to legally marry in any state except Massachusetts. The proportion of women who identify as a sexual minority is assumed to be low. In a national, random probability sample, Laumann and colleagues (4) reported 0.9% of respondents identified as lesbian, 0.5% identified as bisexual, and 4.3% reported having sex with a woman sometime in their lifetime. Population prevalence estimates of middle-aged and older sexual minority women range from 1% to 5% in large community- and population-based studies (5-7).

Small-scale studies suggest that sexual minority women use fewer cancer screening services than their heterosexual counterparts and may be at higher risk for cancer (8, 9). Larger sample sizes are needed to compare sexual minority women to heterosexual women to fully assess the extent of health disparities based on sexual orientation. Unfortunately, to date, quantification of cancer-related disparities is lacking because the majority of studies about cancer detection and control have not collected sexual orientation data. Further, many previous studies focusing on sexual minority women have had study design limitations. For example, some studies were limited by selection bias because participants were recruited through lesbian community organizations or at events targeting the lesbian, gay, bisexual, and transgender community (10-12) and were not likely representative of the sexual minority population. Second, although some studies of cancer risk factors have matched sexual minority women with a heterosexual sister (9, 13), other studies have lacked comparison groups (14, 15). Therefore, it was difficult to assess the experiences of sexual minority women relative to other groups of women. Similar to racial and ethnic minorities, increased representation of sexual minorities would allow sufficient sample sizes for comparative and minority-specific analyses as well as increased generalizability of study findings (2).

The ideal approach to produce a representative sample of sexual minority women is to draw from a list of population members where minority status is known for all entries on the sampling frame (16). Unfortunately, such a list is not available. Furthermore, sexual minority women are a hidden population; identification of sexual minority status requires self-disclosure and women may be reluctant to disclose their sexual orientation because of risks involved.

Another approach is to add questions about sexual orientation to large-scale surveys to produce a sample of the population of interest (17). In 1999, the Institute of Medicine recommended this approach to obtain samples of sexual minority women (3). Measures of sexual orientation have recently been included in some population-based surveys (18, 19). However, these surveys have not asked questions about sexual orientation of the oldest respondents. Large surveys specifically focusing on women such as the Women's Health Initiative (5) and the Nurses Health Study (6) have included questions about sexual orientation. However, older sexual minority women may be underrepresented in these large-scale studies because older adults are less likely than younger individuals to respond to questions about sexual orientation (20).

In the absence of large representative samples, there are at least three alternative sampling strategies for hard-to-reach minority groups. One option is to use area-based sampling frames where households are sampled within geographic areas that have high concentrations of the minority of interest (21, 22). However, the cost of this approach can be particularly prohibitive in areas where residential distributions of unmarried women are diffuse and there are lower concentrations of sexual minorities.

A second option is targeted sampling in settings where the population of interest gather (23, 24). Participants are recruited at sites identified by ethnographic mapping, ensuring that respondents from different areas appear in the final sample. The adequacy of a targeted sample depends on the accuracy and comprehensiveness of the ethnographic mapping. Unfortunately, information needed for the mapping is often only obtained during actual recruitment of participants (24). Furthermore, members of the population that do not gather at the identified settings are not represented.

A third option is chain-referral methods. Using this strategy, a few “seed” participants are asked to recruit other eligible persons from their social networks. Unfortunately, these samples tend to be biased toward more cooperative persons (25). Furthermore, ethical considerations limit the practice of asking seed respondents for the contact information of network members. Alternatively, using respondent-driven approaches (25), participants are asked to inform network members about the study. Network members are then included if they take the initiative of contacting research staff.

The Cancer Screening Project for Women was a survey project about the experiences of unmarried women with breast, cervical, and colorectal cancer examinations. The designation of “unmarried” refers to women who were legally separated, divorced, widowed, or never married. A primary objective was to compare the experiences of heterosexual and sexual minority women. We used principles of targeted and respondent-driven sampling techniques to enroll at least 600 sexual minority and heterosexual women, providing an ideal context to evaluate the strategies and costs for recruiting comparably aged unmarried women. We present results of an evaluation of six recruitment methods in terms of yield, staff time allocation, cost, and sample characteristics. To our knowledge, there have been no studies to date about the expected yield and costs for recruiting sexual minority women for research studies.

Overview of the Cancer Screening Project for Women

The aims of the Cancer Screening Project for Women were to (a) evaluate survey protocols for collecting data about cancer-related attitudes and practices among unmarried sexual minority and majority women; and (b) determine factors that influence breast, cervical, and colorectal examinations among unmarried women. Women were eligible if they were not currently legally married, were ages 40 to 75 years, received the majority of their health care in Rhode Island, and had never been diagnosed with cancer other than nonmelanoma skin cancer. The project was approved by the University Institutional Review Board.

Study participation involved a baseline assessment followed by a reliability assessment collected 4 to 8 weeks later. Women were randomly assigned to one of three data collection protocols: (a) A 28-page booklet-form baseline self-administered mailed questionnaire with a 24-page questionnaire at follow-up; (b) a 35 to 40 min baseline computer-assisted telephone interview with a 15 to 20 min follow-up interview; and (c) a baseline and follow-up computer-assisted self-interviews completed either on laptops provided by research staff at locations chosen by the study or by a mailed computer-disk that included the computer-assisted self-interview program. The lengths of the computer-assisted self-interview assessments were comparable with computer-assisted telephone interviews. Participants were compensated $10 per assessment.

Recruitment

Before study initiation, members of the research team met with key community leaders (for details, see ref. 26). The leaders were asked about sources and methods of recruitment and perceived comfort levels with specific vocabulary related to sexual orientation. The consensus among community leaders was to avoid using labels of heterosexual, homosexual, lesbian, or sexual minority. Instead, they recommended the terminology of women who partner with men (WPM), women who partner with women (WPW), and women who partner with both women and men (WPWM). This terminology was used exclusively in all subsequent recruitment messages.

Four part-time research staff were responsible for participant recruitment and enrollment. Research staff (two WPW and two WPM) were comparably aged women representative of potential participants. With assistance from the community leaders, staff identified recruitment strategies that were classified into six methods for analysis:

  1. Community Settings: face-to-face contact with individuals at community events other than health fairs, such as social organizations and clubs; festivals; cultural events; senior housing; and food pantries.

  2. Health Fairs: face-to-face contact with women attending health events sponsored by hospitals or community-based service agencies.

  3. Mailings and Flyers: brochures provided to potential participants through targeted distribution lists, community-based organizations, and physician offices; flyers displayed in businesses, supermarkets, health clinics, and hospitals.

  4. Print Media: articles printed in the statewide lesbian, gay, bisexual, and transgender monthly newspaper; local town newspapers; local magazines and circulars directed to women and seniors; advertisements in the newspaper with the largest statewide circulation.

  5. Personal Networks: social networks of research staff and recruited participants.

  6. Other Sources: recruitment channels not classified by the other methods.

Potential participants were provided a brief overview of the study and ways to contact study staff. Interested women responded by returning prepaid reply cards, calling a toll-free number, or sending a message to a dedicated, study e-mail account.

Data

All potential participants were first asked, “Can you tell me where you heard about the project?” Research staff noted all recruitment methods mentioned. Next, as part of eligibility screening, women were asked their marital status, followed by the gender of a current partner or gender preference of a partner if they were not currently in a relationship (hereafter referred to as partner gender). Women were then allocated into one of six marital status/partner gender strata: (a) never married women who partner with women (WPW) or with either women or men (WPWM) (hereafter referred to as WPW); (b) previously married WPW (includes WPWM); (c) never married women who partner with men (WPM); (d) previously married WPM; (e) never married women with no partner preference; and (f) previously married women with no partner preference. Strata e and f included never married and previously married women who reported no current interest in having a partner and refused to select the gender of a potential future partner. Research staff entered eligibility screening information into a recruitment database and each case was assigned a code that reflected the recruitment method identified by the potential participant. This allowed us to compare our recruitment experiences for sexual minority (WPW) to those for sexual majority (WPM) women.

Analysis

We allocated data into one of six recruitment methods and four marital status–partner gender strata. For the six women (0.8%) who indicated hearing about the study from more than one source, we used the first recruitment method mentioned. We then conducted our analyses in seven phases. First, we calculated the yield of each recruitment method as the proportion of women contacted by the method who subsequently enrolled. Yield provided a measure of the number of women needed to be screened for eligibility to gain a participant. This measure of yield included only those who made contact with our study team. It was impossible for us to determine the total number of women exposed to our recruitment messages for all types of recruitment (e.g., an unknown number of women viewed flyers). Second, staff time allocation measured the amount of staff resources devoted to recruiting participants through a particular method. It was assessed by two calculations: (a) percent of total staff time spent on each recruitment method and (b) staff time spent in recruitment per participant enrolled. Third, we computed cost per participant enrolled by recruitment method. Fourth, using data collected from the baseline questionnaire, we compared selected demographic characteristics of study participants by recruitment method. Fifth, we compared selected demographic characteristics by partner gender. Sixth, we conducted one-way sensitivity analyses (27-29) to examine how changes in the mix of recruitment methods affected overall study yield, staff time allocation, and costs. Finally, we conducted one-way sensitivity analyses for the following cost model inputs: indirect rate, labor costs, supervision levels, and recruitment yield.

Costs of recruitment were estimated using a two-step microcosting method (28-30) and are presented from the perspective of a researcher. We do not account for costs to other parties such as participants.

Microcosting: Step 1. Because the majority of costs associated with recruitment involved personnel, in the first step, we allocated personnel costs to each recruitment method based on staff time (in minutes) spent in each recruitment activity. This component of microcosting involved process mapping of the recruitment activities, tracking of staff time, and record keeping of individual contacts (28-31). Process mapping provided a list of subcomponent activities that were involved in each activity (e.g., preparatory tasks, travel, on-site tasks). For example, for Community Events and Health Fairs, the amount of time preparing for, traveling to, and attending the event were recorded.

Based on type of recruitment activity or known information about the recruitment setting, staff estimated the proportion of recruitment time spent per month with women from each marital status–partner gender stratum. For example, staff noted the amount of time spent: preparing letters to a list of Catholic nuns (never married WPM); attending a support group for previously widowed women (previously married WPM); or participating in events with a gay and lesbian senior organization (never married and previously married WPW). As much as possible, staff attempted to determine the marital status and/or partner gender of age-eligible women at a recruitment event. If this information was unavailable (e.g., public events such as health fairs), we allocated the proportion of contact time with potential participants to marital status–partner gender strata based on best estimates using Census data of the age-eligible unmarried female population and unmarried same-sex partner households in Rhode Island, with an additional assumption that 10% of contacted women were ineligible: 5% never married WPW, 10% previously married WPW, 15% never married WPM, 60% previously married WPM, and 10% ineligible (32, 33).

For participants who reported learning about the study through personal networks or other unknown sources, we applied a set of assumptions regarding staff time. To estimate time in personal network recruitment, staff estimated the time that they spent per month in personal activities not directly associated with recruitment but during which information about the study was discussed. This time was then allocated to one or more of the partner gender strata based on information about the social network. For example, staff noted the partner gender strata of friends and acquaintances with whom they described the study while engaged in social activities. To estimate the intensity of resources for women who could not specify the recruitment source, we applied a weighted average of total time devoted to all other recruitment activities, assuming that this unknown source of information would approximate the level of resources for all other women combined.

We calculated personnel costs as the sum of costs of direct staff time with potential participants, administrative costs, and supervision costs. Personnel costs were then estimated by recruitment type based on detailed records of time spent across each. We estimated administrative and supervision time as 20% and 15%, respectively of every hour of direct contact with potential participants. Personnel costs were based on staff salaries of $28.00 per hour plus a 30% fringe rate and a supervisor salary of $40.00 per hour plus 30% fringe (34, 35). The salary estimate of $28.00 per hour is at the 100th percentile of Bureau of Labor Statistics national wage estimates for a social science research assistant in 2005. We used budget records to estimate all other direct costs associated with recruitment including travel, advertising, event fees, office supplies, postage and mailing, printing, and telephone charges.

Microcosting: Step 2. In the second step of microcosting, we assigned other nonpersonnel direct costs to each recruitment method based on specific expenditures. These direct costs were either assigned by recruitment type (e.g., advertising in targeted publications) or allocated by proportion of time devoted to each recruitment method. We then added personnel costs and other direct costs together, allocated the total direct costs into the partner gender strata, and computed costs per participant enrolled.

Consistent with recommendations for cost-effectiveness analyses (28), we conducted a series of sensitivity analyses in which we varied our cost-model assumptions. Our goal was to inform other studies that may be considering similar recruitment methods but face different resource costs or recruitment-strategy choices. First, we addressed how much the overall average costs would vary if the mix of recruitment methods changed. We recalculated the average cost models multiple times, each time removing one of the recruitment methods and then examining the average yield and cost for the combined remaining methods. Second, we conducted sensitivity analyses across a set of key cost model variables (see Table 5 for variable list). Each variable was changed to an alternate value and the cost model was recomputed. Overhead rates as a proportion of direct costs vary by institution so we examined a lower and higher rate than we used in the original cost models. Salaries were varied across the range reported by the Bureau of Labor Statistics for research assistants. We varied staff time dedicated to nonparticipant-specific administrative tasks associated with recruitment to account for differences in administrative structures across institutions. Next, we examined the effect of changes in amount of supervisor oversight on total recruitment costs. Next, we recalculated the cost model assuming a 20% lower yield for all recruitment strategies. Finally, we replicated the analyses removing the six women who indicated hearing about the study from more than one source.

We recruited participants for 25 months from June 1, 2003, through June 30, 2005. There was an average of eight recruitment activities per month with an enrollment average of 25 women per month into the study. We obtained contact information for 773 women of whom 630 were enrolled: 144 never married WPW, 69 previously married WPW, 158 never married WPM, and 236 previously married WPM. Eighteen never married and five previously married women reported that they were not interested in partnering in the future and refused to select the gender of a potential future partner. These 23 women were comparable with WPM on demographic characteristics and included in the WPM strata for all analyses. Of the 143 women who were not enrolled, 60.8% (n = 87) were ineligible, 36.3% (n = 52) were unable to be contacted or refused contact, and 2.8% (n = 4) were not interested in participation. Characteristics of recruitment methods are shown in Table 1.

Yield

Mailings/Flyers, Community Events, and Personal Networks produced the largest absolute numbers of respondents with contact information available. However, Print Media produced the highest total yield (Table 1, Section A). Of the 135 women who responded to Print Media, 133 (98.5%) enrolled. Community Events (70.5%) and Other Sources (52.2%) resulted in the lowest total yield. Print Media and Personal Networks yielded the largest percent of WPW. Health Fairs and Mailings/Flyers yielded the largest percent of WPM.

Staff Time Allocation

As shown in Table 1, Section B, almost half of the total staff time spent in recruitment was for Community Events (47.8%), whereas only 2.4% of recruitment time was spent in Print Media activities. For WPW, the largest percentage of total recruitment time was for Community Events (40.7%) followed by Mailings/Flyers (19.4%) and Personal Networks (19.1%), whereas the least amount of time was spent on Print Media (3.8%). Half of the total recruitment time for WPM was in Community Events (51.3%). The majority of the remaining time was in Health Fairs (22.7%). Similar to WPW, the least amount of time for WPM was for Print Media activities (1.7%).

We spent 80 min on average recruiting women for whom contact information was available and 100 min per woman enrolled (Table 1, Section B). Overall, we spent 95 min per WPW enrolled, with an average of 60 and 170 min for never married WPW and previously married WPW, respectively. Similarly, 100 min were spent on average recruiting WPM. Of this, 60 and 125 min were spent recruiting never married and previously married WPM, respectively. However, the amount of time spent in recruitment activities varied substantially by recruitment method. We spent an average of 10 and 15 min per WPW and WPM, respectively, using Print Media. Alternatively, we averaged 355 and 260 min per WPW and WPM, respectively, for Community Events.

Cost

As shown in Table 1, Section C, the cost was about $100 per woman contacted, whereas the cost was $120 per participant enrolled. Costs for WPW were about $30 higher per participant than for WPM, a result of the high costs associated with recruiting previously married WPW ($235 per participant). The cost per participant recruited was lowest for Print Media followed by Personal Networks. For WPW, costs for all other recruitment methods were at least $200 per participant enrolled. For WPM, only the costs for Community Events exceeded $200.

Participant Characteristics

Characteristics of study participants by recruitment method are shown in Table 2. The average age was 53.1 years (range 40-75 years). Younger women were more likely to be recruited using Print Media and Personal Networks. Compared with other methods, women with a high school degree or less, a household income of less than $30,000, self-rated Hispanic ethnicity, or a self-reported disability were most likely to be recruited at Community Events. Alternatively, those with at least a college degree, employed, a household income of $75,000 or more, or self-reported White race were more likely to be recruited through Print Media or Personal Networks. Women who self-reported a non-White racial group were most likely to be recruited at Community Events and Health Fairs.

Characteristics of study participants by partner gender are shown in Table 3. WPW were more likely to be younger, have more than a high school degree, be employed, report a household income of $75,000 or more, and have health insurance. In contrast, WPM were more likely to live alone, identify as a racial minority, and have a self-reported disability.

Sensitivity Analyses

Table 4 shows results of the sensitivity analyses of average yield and cost. Overall, the average yield was 82% and the average cost per participant was $100. All other factors being constant, if Print Media had not been used, the average yield would have been 78% and the average cost would have been $115 per participant. Alternatively, without Community Events, the average yield and cost would have been 84% and $70 per participant, respectively.

Table 5 presents results of the sensitivity analyses of costs per participant by marital status–partner gender strata. Total observed costs for all participants enrolled varied from $160 with a 35% indirect cost rate to $185 with a 55% indirect cost rate. When we reduced staff salaries to $20.00, $16.00, and $12.50, per hour, direct costs for all participants enrolled were lowered by $20, $30, and $40 per participant, respectively. Increasing administrative time from 20% to 50% of direct participant contact raised direct costs per participant enrolled by $5 on average. Increasing supervision from 15% to 30% of direct participant contact raised direct costs by $10 per participant enrolled. Direct costs per participant increased by $60 for previously married WPW but $30 or less for all other strata, assuming a 20% reduction in yield.

Results did not differ when we replicated the analyses after removing the six women who reported hearing about the study from more than one recruitment source (data not shown).

This is one of the first studies to describe strategies and costs associated with recruiting unmarried middle-aged and older sexual minority and heterosexual women. It contributes to other published reports documenting recruitment methods associated with an observational rather than clinical trial or intervention study. Key features of our success included collaboration with key community leaders, knowledgeable and respected research staff, and readiness to adopt new recruitment strategies as opportunities arose.

Cost-effective recruitment strategies are essential in a climate of increasing funding constraints. Our total direct recruitment costs were $120 per participant enrolled. This is comparable with costs of $113 per participant screened for eligibility reported for a cross-sectional interview study of racial and ethnic minority women's preferences regarding tamoxifen use for primary prevention of breast cancer (36). Alternatively, our costs were lower than the $145 per participant enrolled for Black women recruited to focus groups about screening examinations for cervical cancer (37). Other studies describing costs generally reported amounts comparable with our data for WPW ($143-$439 per participant). However, these were clinical trials or intervention studies with more demanding protocols, requiring significantly more commitment from participants (38-40).

Overall, our costs were about $30 higher per participant to recruit WPW than WPM. However, differences in average costs by marital status–partner gender strata varied dramatically depending on recruitment method. Most of these differences can be explained by smaller numbers of WPW relative to WPM enrolled by each method.

Similar to other studies (41-45), we found that Print Media resulted in the highest yield of participants (98.5%) overall and had the lowest associated costs ($15 per participant enrolled). Seeing the study in print may have provided credibility to the project and reassured potential participants that the research was legitimate. Because of limited staff responsibilities, only 2% of total recruitment time were spent in preparation of these print materials (average of 10 min per participant). Print Media activities included several newspaper reports and press releases that were not paid for by the study. Therefore, other than paid advertisements, direct costs for Print Media were low. However, women recruited through Print Media were younger, predominantly White, well educated, employed, and with higher incomes.

Personal network recruitment resulted in the second highest yield (93.8%) with modest associated costs ($60 per participant enrolled). Other studies, particularly focused on recruitment of racial and ethnic minorities, have also shown success using research staff and participant referrals (37, 42, 46-48). We are unaware of other studies that have used comparable methods to estimate research staff time for network recruitment. We employed members of the target population to gain community trust and legitimacy. Therefore, when calculating staff time allocation and cost, we used an approach that acknowledged that staff may interact with potential participants outside of employment-related responsibilities. Our results are consistent with others that have found that having research staff with ties in the community is important for establishing trust. However, it is unclear the extent to which our analytic approach affected the generalizability of the findings. It is likely that any bias, if present, was stronger for WPW than WPM because the sexual minority community is smaller, with members more well known to each other.

There were high costs associated with recruitment at Community Events for both WPW ($490) and WPM ($275). The cost per participant was at least $200 within each of the four marital status–partner gender strata. The majority of these costs were associated with personnel time. Similar to Tarlow and colleagues (43), we found that community-based groups frequently had small memberships and less than half of members regularly attended meetings. Therefore, group leaders' estimates of potential participants were often unreliable, and substantial staff time was spent at events with very small yield. However, women who were recruited from community events were more likely to be older, a member of a minority group, with lower incomes, and with a self-reported disability. This is comparable with other studies that documented higher yields of racial and ethnic participants when face-to-face contact methods were used (37, 46, 48-52).

Costs associated with mailings and flyers were reasonable ($60 per participant) for recruiting WPM. This was partially due to the availability of a directory of Catholic nuns (never married WPM), which we used to target direct mailings. Conversely, the average per participant cost for mailings and flyers for WPW was $360. Unlike some areas such as Boston, Chicago, and San Francisco, Rhode Island has no distinct neighborhoods that include higher proportions of sexual minority women. Consequently, we found no convenient distribution channels for targeting WPW with flyers or mailings, making these methods very labor-intensive and costly.

Direct mass mailings have been shown to be effective in recruiting participants in other studies (38, 46, 48, 53-59). We bought a mailing list of Rhode Island home owners, requesting the names and addresses of unmarried age-eligible women. In addition to the sample bias (i.e., only home owners), the response rate to this mailing was only 2.3% (18 of 796) and only 14 of the 18 women who responded were eligible to participate. Furthermore, 9.2% (73 of 796) of the mailings were returned as nondeliverable because of incorrect addresses or no forwarding information. Concerns about individual privacy prohibited other types of direct mass mailings such as from the Department of Motor Vehicles, voter registration lists, or magazine subscriptions. Therefore, our experiences suggest that commercial mailing lists are not reasonable approaches for recruiting middle-aged and older unmarried women.

This study has limitations. First, our sample was restricted to one geographic area. Findings would likely differ in areas with substantially different proportions of unmarried middle-aged and older women. Second, because of sample size, it was not feasible to separate WPWM and WPW. Third, our category of Community Events covered many disparate activities. The mix of events provided an appropriate representation of activities involving unmarried women, but findings may have differed with a different combination of activities, or if we had classified the activities into more homogeneous categories.

Although we presented the evaluation of each recruitment method separately, in practice, women may have been exposed to more than one approach. Our analytic technique likely overestimated the effect of any single recruitment method and underestimated the costs involved. Although less than 1% of our participants reported learning about the study from more than one source, there could have been an interaction between methods. For example, we identified the method that ultimately prompted participants to contact the research team, but they could have heard about the study through other methods before deciding to participate. The small percentage of women reporting more than one recruitment source also may have been due to question wording. We recommend that future investigators use an approach, such as that by Schoenfeld and colleagues (42), in which participants were asked how they first learned about the study, where else they might have heard about the study, and which of all the mentioned sources specifically led them to contact the study.

Despite study limitations, our data provide important information about staff time allocation and sample yield of recruitment methods for unmarried middle-aged and older sexual minority and heterosexual women. Our results show tradeoffs between intensity of staff resources, cost, and diversity of the samples recruited. Methods that required the least staff time resulted in the most homogeneous samples. Alternatively, women most at risk for being underrepresented in research were recruited by methods requiring the most personnel time. Our findings highlight the need for a comprehensive strategy that uses multiple recruitment methods to ensure diversity of research participants. Finally, cost data, such as that presented here, are critical for developing appropriate budget projections for future observational studies.

Grant support: National Cancer Institute grant K07-CA87070 (M.A. Clark).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Note: M.A. Clark was responsible for planning the research, analyzing the data, and writing the first draft of the article; M.R. Wasserman contributed to the literature review and study design and assisted in interpretation of study findings; C.J. Neighbors directed the analyses and assisted in the interpretation of the data and presentation of the results; and G.F. Armstrong, M.L. Drnach, S.L. Howie, and T.L. Hawthorne recruited participants and collected all of the data. All authors actively contributed to writing and editing of the article.

Human Participant Protection: The study was approved by the Brown University Institutional Review Board.

1
NIH. NIH Guidelines on the inclusion of women and minorities as subjects in clinical research. NIH Guide, Volume 23, Number 11, March 18, 1994. Available from http://grants.nih.gov/grants/guide/notice-files/not94-100.html. Accessed: 2007 Jul 23.
2
Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants.
Annu Rev Public Health
2006
;
27
:
1
–28.
3
Institute of Medicine. Lesbian health: current assessment and directions for the future. Washington (DC): National Academy Press; 1999.
4
Laumann EO, Gagnon JH, Michael RT, Michaels S. Homosexuality. In: Laumann EO, Gagnon JL, Michael RT, Michaels S, editors. The social organization of sexuality: sexual practices in the United States. Chicago (IL): University of Chicago Press; 1994. p. 283–320.
5
Valanis BG, Bowen DJ, Bassford T, Whitlock E, Charney P, Carter RA. Sexual orientation and health: comparisons in the women's health initiative sample.
Arch Fam Med
2000
;
9
:
843
–53.
6
Case P, Austin SB, Hunter DJ, et al. Disclosure of sexual orientation and behavior in the Nurses' Health Study II: results from a pilot study.
J Homosex
2006
;
51
:
13
–31.
7
Bowen DJ, Bradford J, Powers D. Comparing sexual minority status across sampling methods and populations.
Women Health
2006
;
44
:
121
–34.
8
Cochran SD, Mays VM, Bowen D, et al. Cancer-related risk indicators and preventive screening behaviors among lesbians and bisexual women.
Am J Public Health
2001
;
91
:
591
–7.
9
Dibble SL, Roberts SA, Nussey B. Comparing breast cancer risk between lesbians and their heterosexual sisters.
Womens Health Issues
2004
;
14
:
60
–8.
10
Wells BE, Bimbi DS, Tider D, Van Ora J, Parsons JT. Preventive health behaviors among lesbian and bisexually identified women.
Women Health
2006
;
44
:
1
–13.
11
White JC, Dull VT. Health risk factors and health-seeking behavior in lesbians.
J Womens Health
1997
;
6
:
103
–12.
12
Fish J, Anthony D. UK national lesbians and health care survey.
Women Health
2005
;
41
:
27
–45.
13
Rothblum ED, Factor R. Lesbians and their sisters as a control group: demographic and mental health factors.
Psychol Sci
2001
;
12
:
63
–9.
14
Grindel CG, McGehee LA, Patsdaughter CA, Roberts SJ. Cancer prevention and screening behaviors in lesbians.
Women Health
2006
;
44
:
15
–39.
15
Roberts SJ, Patsdaughter CA, Grindel CG, Tarmina MS. Health related behaviors and cancer screening of lesbians: results of the Boston Lesbian Health Project II.
Women Health
2004
;
39
:
41
–55.
16
Kalsbeek WD. Sampling minority groups in health surveys.
Stat Med
2003
;
22
:
1527
–49.
17
Sudman S, Kalton G. New developments in the sampling of special populations.
Annu Rev Sociol
1986
;
12
:
401
–29.
18
Centers for Disease Control and Prevention (CDC), National Center for Health Statistics (NCHS). National Survey of Family Growth Cycle 6. Available from: http://www.cdc.gov/nchs/nsfg.htm. Last updated: 2002. Accessed 2007 Jul 23.
19
Centers for Disease Control and Prevention (CDC), National Center for Health Statistics (NCHS). National Health and Nutrition Examination Survey Data, 1999+. Available from: http://www.cdc.gov/nchs/nhanes.htm. Last updated: 2003. Accessed 2007 Jul 23.
20
Gruskin E, Geiger AM, Gordon N, Ackerson L. Characteristics of nonrespondents to questions on sexual orientation and income in a HMO survey.
J Gay Lesbian Med Assoc
2001
;
5
:
21
–4.
21
Bowen DJ, Bradford JB, Powers D, et al. Comparing women of differing sexual orientations using population-based sampling.
Women Health
2004
;
40
:
19
–34.
22
Mills TC, Stall R, Pollack L, et al. Health-related characteristics of men who have sex with men: a comparison of those living in “gay ghettos” with those living elsewhere.
Am J Public Health
2001
;
91
:
980
–3.
23
Muhib FB, Lin LS, Stueve A, et al. A venue-based method for sampling hard-to-reach populations.
Public Health Rep
2001
;
116
Suppl 1:
S216
–22.
24
Watters JK, Biernacki P. Targeted sampling: options for the study of hidden populations.
Soc Probl
1989
;
36
:
416
–30.
25
Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations.
Soc Probl
1997
;
44
:
174
–99.
26
Clark MA, Bonacore L, Wright SJ, Armstrong G, Rakowski W. The cancer screening project for women: experiences of women who partner with women and women who partner with men.
Women Health
2003
;
38
:
19
–33.
27
Drummond MF, O'Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. New York: Oxford University Press; 1997.
28
Gold MR, Siegel J, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.
29
Muenning P. Designing and conducting cost-effectiveness analyses in medicine and health care. San Francisco: Jossey-Bass; 2002.
30
Kaplan RS, Atkinson AA. Advanced management accounting. Upper Saddle River (NJ): Prentice Hall; 1998.
31
Hunt VD. Process mapping: how to reengineer your business process. New York: John Wiley and Sons; 1996.
32
U.S. Census Bureau. American Community Survey, 2003. Table PCT 013: Sex by marital status by age for the population 15 years and over. Available from: http://factfinder.census.gov/servlet/DTTable?_bm=y&-context=dt&-ds_name=ACS_2003_EST_G00_&-mt_name=ACS_2003_EST_G2000_PCT013&-CONTEXT=dt&-tree_id=4001&-redoLog=true&-all_geo_types=N&-currentselections=ACS_2003_EST_G2000_PCT013&-geo_id=04000US44&-search_results=01000US&-format=&-_lang=en. Last updated: 2004. Accessed 2006 Apr 19.
33
34
Bureau of Labor Statistics. National occupational employment and wage estimates, May 2005: Social science research assistants. Available from: http://www.bls.gov/oes/2005/may/oes194061.htm. Last Updated: May 24, 2006. Accessed 2007 Apr 19.
35
Bureau of Labor Statistics. Employer costs for employee compensation-December 2005. Available from: http://www.bls.gov/ncs/ect/sp/ecnr0064.pdf. Last Updated: March 14, 2006. Accessed 2007 Apr 19.
36
Keyzer JF, Melnikow J, Kuppermann M, et al. Recruitment strategies for minority participation: challenges and cost lessons from the POWER interview.
Ethn Dis
2005
;
15
:
395
–406.
37
Wilson KM, Orians CE. Considerations in recruiting underscreened women to focus groups on screening for cervical cancer.
Health Promot Pract
2005
;
6
:
379
–84.
38
Katula JA, Kritchevsky SB, Guralnik JM, et al. Lifestyle Interventions and Independence for Elders pilot study: recruitment and baseline characteristics.
J Am Geriatr Soc
2007
;
55
:
674
–83.
39
Harris KJ, Ahluwalia JS, Catley D, Okuyemi KS, Mayo MS, Resnicow K. Successful recruitment of minorities into clinical trials: The Kick It at Swope project.
Nicotine Tob Res
2003
;
5
:
575
–84.
40
McIntosh S, Ossip-Klein DJ, Spada J, Burton K. Recruitment strategies and success in a multi-county smoking cessation study.
Nicotine Tob Res
2000
;
2
:
81
–4.
41
Maurer BT, Moreno SI, Pickard AR, Wurst BE, Norden DK, Schumacher Jr HR. A comparison of recruitment methods for an osteoarthritis exercise study.
Arthritis Care Res
1995
;
8
:
161
–6.
42
Schoenfeld ER, Greene JM, Wu SY, O'Leary E, Forte F, Leske MC. Recruiting participants for community-based research: the Diabetic Retinopathy Awareness Program.
Ann Epidemiol
2000
;
10
:
432
–40.
43
Tarlow BA, Mahoney DF. The cost of recruiting Alzheimer's disease caregivers for research.
J Aging Health
2000
;
12
:
490
–510.
44
Shuhatovich OM, Sharman MP, Mirabal YN, Earle NR, Follen M, Basen-Engquist K. Participant recruitment and motivation for participation in optical technology for cervical cancer screening research trials.
Gynecol Oncol
2005
;
99
Suppl 1:
S226
–31.
45
Davey R, Edwards SM, Cochrane T. Recruitment strategies for a clinical trial of community-based water therapy for osteoarthritis.
Br J Gen Pract
2003
;
53
:
315
–7.
46
Kennedy BM, Conlin PR, Ernst D, Reams P, Charleston JB, Appel LJ. Successfully recruiting a multicultural population: the DASH-Sodium experience.
Ethn Dis
2005
;
15
:
123
–9.
47
Brown CS, Mitchel JT. Recruiting patients for female sexual arousal disorder studies.
J Sex Marital Ther
2001
;
27
:
443
–51.
48
Fouad MN, Corbie-Smith G, Curb D, et al. Special populations recruitment for the Women's Health Initiative: successes and limitations.
Control Clin Trials
2004
;
25
:
335
–52.
49
Ford ME, Havstad SL, Davis SD. A randomized trial of recruitment methods for older African American men in the Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial.
Clin Trials
2004
;
1
:
343
–51.
50
Fuqua SR, Wyatt SB, Andrew ME, et al. Recruiting African-American research participation in the Jackson Heart Study: methods, response rates, and sample description.
Ethn Dis
2005
;
15
Suppl 6:
S18
–29.
51
Gilliss CL, Lee KA, Gutierrez Y, et al. Recruitment and retention of healthy minority women into community-based longitudinal research.
J Womens Health Gend Based Med
2001
;
10
:
77
–85.
52
Blumenthal DS, Sung J, Coates R, Williams J, Liff J. Recruitment and retention of subjects for a longitudinal cancer prevention study in an inner-city black community.
Health Serv Res
1995
;
30
:
197
–205.
53
Resio MA, Baltch AL, Smith RP. Mass mailing and telephone contact were effective in recruiting veterans into an antibiotic treatment randomized clinical trial.
J Clin Epidemiol
2004
;
57
:
1063
–70.
54
Rudick C, Anthonisen NR, Manfreda J. Recruiting healthy participants for a large clinical trial.
Control Clin Trials
1993
;
14
Suppl 1:
S68
–79.
55
McFarland LV, Reiber GE, Norman JE. Recruitment of Medicaid and dual-enrolled Medicare beneficiaries with diabetes mellitus into a randomized controlled trial.
Am J Manag Care
2005
;
11
:
443
–8.
56
Connett JE, Bjornson-Benson WM, Daniels K. Recruitment of participants in the Lung Health Study, II: assessment of recruiting strategies.
Control Clin Trials
1993
;
14
Suppl 1:
S38
–51.
57
Unson CG, Ohannessian C, Kenyon L, Case A, Reisine S, Prestwood K. Barriers to eligibility and enrollment among older women in a clinical trial on osteoporosis: effects of ethnicity and SES.
J Aging Health
2004
;
16
:
426
–43.
58
Hollis JF, Satterfield S, Smith F, et al. Effective strategies and predictors of randomization. Trials of Hypertension Prevention (TOHP) Collaborative Research Group. Recruitment for phase II of the Trials of Hypertension Prevention.
Ann Epidemiol
1995
;
5
:
140
–8.
59
Rubin RR, Fujimoto WY, Marrero DG, et al. The Diabetes Prevention Program: recruitment methods and results.
Control Clin Trials
2002
;
23
:
157
–71.