National Cancer Institute's Small Grants Program for Behavioral Research in Cancer Control Boosts Careers for New Investigators and Fulfills NIH Research Priorities

In 1998, the Behavioral Research Program (BRP) was created within the Division of Cancer Control and Population Sciences. The BRP provides resources and funding opportunities for cancer-related research in the social and behavioral sciences through its five branches: Applied Cancer Screening, Basic and Biobehavioral Research, Health Communications and Informatics, Health Promotion, and Tobacco Control. The decision to create this new research program reflected the recognition of the National Cancer Institute (NCI) of the effect of health behaviors on cancer research and also spoke to NCI's strong commitment to advance social, behavioral, and population sciences as integral to the conduct of applied research in cancer prevention and control (NIH, 1998; ref. 1).

Another objective in creating BRP was to encourage and facilitate the growth of new investigators whose interests focus on behavioral research aimed at helping people control behaviors such as tobacco use, physical inactivity, and sun exposure, which increase cancer risk. In so doing, BRP aimed to attract behavioral scientists to cancer control and build research capacity among new investigators. This led BRP to develop the Small Grants Program (SGP) for Behavioral Research in Cancer Control. Using the R03 funding mechanism, which funds small, investigator-initiated research projects that can be carried out in a short period with limited resources, the SGP was designed to support pilot projects, efforts to develop and test new methodologies, secondary data analyses, and innovative studies that provide a basis for more extended research. The main purpose of this grants program is to encourage new investigators to enter the field of behavioral research in cancer control while also facilitating their long-term career development as principal investigators within the field. Since its inception in 1999, the SGP has made 122 awards, primarily to psychologists.

The evaluation of the SGP, which was conducted from 2004 to 2005, was based on information from three sources-grantee surveys, grantee curriculum vitae, and mentor interviews. The study was limited to those grantees (n = 64) who were awarded funding under the earliest program announcement (PAR 99-006) from 1999 to 2001 because they had the most post-grant time (maximum of 2 years) in the field at the onset of data collection in August 2004.

A 47-item survey was developed that covered five topics, including grantee background, additional NIH and external funding, grant oversight, mentoring, and effect of the grant on grantees' career. A total of 54 of 64 grantees were surveyed, for a response rate of 84%. The remaining 10 grantees were excluded because either repeated attempts to contact them failed (n = 9) or the grantee refused to participate (n = 1). Research associates from a public health research organization conducted the surveys by phone over a 3-week period in November 2004.

A total of 47 of 64 grantees provided a copy of their curriculum vitae at the end of the 8-week data collection period. The remaining 17 did not respond within the data collection period or could not be contacted.

For each grantee, citations data were compiled. A series of descriptive bibliometric analyses were conducted. The analyses included: (a) total number and percentage of all grantees who had published, (b) average number of publications per grantee, (c) total number and percent distribution of publications by all grantees, and (d) total number and percentage of grantee publications cited by other publications. Additional analyses were conducted using effect factors taken from the Institute for Scientific Information and other sources, such as the Journal Citation Reports. Both analyses were an attempt to assess grantee productivity in publishing research related to their grant subject specifically as well as in the field of behavioral research generally and to assess the influence of the research. When using effect factors to assess the influence of research, it is important to be mindful of the fact that an effect factor is a measure of citations to a journal only and not a measure of quality per se.

During a pilot study phase of the project, interviews with eight individuals who served as formal mentors of the PAR 99-006 cohort of grantees for the years 1999 to 2001 were conducted using a 16-item open-ended interview guide. These eight mentors were a small subset of the total group of 66 mentors assigned to this cohort but were typical of the group in that they served as resource to one or more grantees during their award. The health research firm that conducted the grantee survey also conducted the mentor interviews, which covered three main topics: mentor background and experience, effect of the award on grantees' careers, and effect of the award on the field as a whole.

The SGP evaluation focused on grantee background and post-award activities.

The majority of the 54 R03 grantees surveyed have a Ph.D. (93%; n = 50), are affiliated with a university or university medical school (80%; n = 43), and consider themselves junior investigators in the field of behavioral research in cancer control (59%; n = 32).

All the surveyed grantees reported that they intended to apply for funds to support additional behavioral research in cancer control. Table 1 summarizes the number of applications submitted by grantees to NIH to fund studies in behavioral research in cancer control (including those linked and not linked to their R03) and the status of each application. After obtaining the R03s, each grantee, on average, submitted two applications overall and one application linked to the R03 topic. Grantees reported submitting a total of 127 grant applications. Seventy-five of the 127 applications were for continued funding of their R03 research topic, and 52 were to fund other behavioral research in cancer control (Table 1). Of these 127 applications, 37% (n = 47) were selected to be funded, including 18% (n = 23) for an R01 and 21% (n = 26) for an R21.

During 2001 to 2005 (the 4 years following the conclusion of the 1999-2001 grant cycle), 60% (n = 27) of the 47 grantees who submitted curriculum vitae published and had in press or in preparation at least one article resulting from their R03 research. Grantees published a total of 45 articles, had 12 in press and 32 in preparation, and presented their work on 91 occasions (Table 2). Of grantees who had an article published, in press, or in preparation (n = 27), more than half (n = 19) were the lead author. A total of 57 published and in press articles were cited 134 times, whereas those published with the grantee as lead author (n = 40) were cited 123 times. These citations occurred in 85 peer-reviewed journals, and less than 10% were self-citations. The overall average effect factor of 2.44 for journals where grantees published falls within the range of average effect factors (0.877-3.980) for comparable journals.

An additional analysis of grantees' publication histories from 1999 to 2005 found that, after their R03 research was completed, 47 grantees published a total of 701 articles in 315 peer-reviewed journals covering 125 disciplines, including cancer control or behavioral research. More than one half of these articles (53.6%) related to behavioral research and/or cancer control, and the grantee was the lead author of ∼42% of the articles. The number of publications per grantee ranged from 0 to 59, with an average of 14 articles per grantee. Forty percent (n = 19) published between 10 and 25 articles, 38% (n = 18) published between 5 and 10, and 17% (n = 8) published between 1 and 4 (Table 3).

Nearly all of the 54 grantees who responded to the grantee survey (85%; n = 46) attributed their decision to conduct subsequent behavioral research in cancer control to their experience in applying for and obtaining an R03 through this BRP's grants program, and >89% (n = 48) considered their research to be interdisciplinary. All grantees reported being involved in other activities related to their behavioral research in cancer control (Table 4).

All the mentors interviewed viewed the SGP as having a positive effect on the careers of new investigators because, in their view, it provides funding opportunities for investigators at an early stage in their careers. A common response from the mentor's view was that the R03 award is one of the few means available to fund collection of pilot data to support a subsequent application for an R01 or other funding opportunity. The mentor respondents also cited the interdisciplinary nature of the R03 grants as an important characteristic.

Additionally, mentor respondents commonly described the specific focus of the R03 on innovative research as important in bringing potentially pioneering studies to the field and thereby moving the field of cancer control forward. The key benefit identified was the focus on funding innovative projects that “may be a little riskier and not quite ready for an R01.” For example, one respondent mentioned the funded study entitled, “Efficacy Study of an Internet Tobacco Non-use Program”. At the time this study was funded, there was a paucity of research evaluating Internet-based interventions as an effective medium for lifestyle change. This project proposed to evaluate an Internet-based smoking cessation intervention that used multiple methods, spanned quantitative and qualitative domains, and included a cost-effectiveness component.

The purpose of this evaluation was to assess the value of the SGP as part of NCI's overall grant award portfolio intended to meet NCI's strategic priorities, such as encouraging investigators to develop careers in this field. The Final Evaluation Report can be found on the Web site.³

A particularly valuable aspect of the program is that it is designed to fund smaller studies to collect pilot data that can be used to support future grant applications. For example, the program has provided grantees with funding to pursue novel approaches for addressing biomedical research problems that otherwise would not typically be funded as an R01. The grant, “Effects of Yoga on Quality of Life during Breast Cancer,” is one such example of innovative research. The grant's Principal Investigator expanded the scope of research of the study beyond the small grants award and secured higher level NCI funding mechanism. This is important in light of decreased funding levels and the increasing competition to secure research support. Findings most encouraging to NCI are those that reflect the success of the grantees in continuing their research. Grantees' post-award activities, such as obtaining additional funding (particularly R01) and publishing their work in comparable peer-reviewed journals, suggest that this small grants award program serves as a bridge for new investigators to continue their innovative research. Lastly, it is important to note a majority of grantees surveyed responded that, without the R03 funding, they would not have been able to conduct and continue their research. They recommend that others apply for this NCI funding award.

This evaluation posed several challenges. Significant among them was identifying appropriate measures of the effect of the SGP and linking these measures to the broader goals of the program. For this evaluation, data were limited to self-reports of grantee post-award activities, such as obtaining additional funding and publishing in the field. Because NCI retains data only on applicants who are awarded funding, comparisons of this cohort of grantees with those who were not successful or with new investigators outside the program were not possible. Despite these limitations, we hope this evaluation will encourage other programs at NIH and other federal agencies to evaluate the effect of their funded research.

In summary, during this challenging NIH budget climate, it is important to systematically monitor the progress of extramural funding programs in achieving their stated goals. Lessons learned from this outcome evaluation study will serve to strengthen the management and improve the delivery of this SGP to new investigators in cancer prevention and control.

We thank Erica Breslau, Denise Buckley, and Helen Meissner (NCI, Rockville, MD) for their careful reading and comments on the manuscript and Holly Carmichael (Battelle) for her efforts on this project.