Increasing HPV Vaccination among Low-Income, Ethnic Minority Adolescents: Effects of a Multicomponent System Intervention through a County Health Department Hotline

Human papillomavirus (HPV)-associated cancers, including cervical, oropharyngeal, anal, penile, vulvar, and vaginal cancers, pose a heavy burden on the population and disproportionately impact low-income, ethnic minority, and immigrant populations (1, 2). Introduction of the prophylactic HPV vaccine in 2006 represented a breakthrough in cancer prevention. Despite the availability of the HPV vaccine for over a decade, uptake among adolescents, the target population for the vaccine, remains suboptimal. In 2019, only 54% of U.S. adolescents were “up to date” with the HPV vaccine series (3). The vaccine's benefits at the population level will only be realized if uptake is high, particularly in subgroups at highest risk of HPV-associated cancers.

The majority of studies evaluating interventions to improve HPV vaccination in adolescents using a controlled design have taken place in clinical settings (4, 5), where systematic reviews have documented the effectiveness of patient and provider reminders and system-based interventions in improving HPV vaccine initiation and completion rates (5). However, fewer studies have been conducted in nonclinical, community settings, which may allow for greater reach into vulnerable communities facing barriers to accessing the health care system.

In 2006, the UCLA study team established a partnership with the Office of Women's Health (OWH) of the Los Angeles County Department of Public Health to conduct HPV vaccination research. The collaboration resulted in the conduct of a survey of nearly 500 low-income, ethnic minority mothers of girls who used a multi-lingual telephone hotline operated by OWH. Study results revealed substantially lower HPV vaccine initiation rates among the adolescent children of hotline users compared to the general U.S. population in 2009 (29% vs. 44%; ref. 6). The study also revealed that a high proportion of mothers of unvaccinated girls did not have enough information to make a decision about vaccination (66%) and did not know where to get their child the vaccine (74%). On the basis of these findings, we developed a multicomponent, system-based intervention (brief telephone education, print materials, referral to vaccine provider) delivered through the telephone hotline by operators, as part of their routine operations. We report the outcomes of this intervention on HPV vaccine uptake.

The setting for this study was the Office of Women's Health's multi-language hotline, which targeted predominantly low-income residents and had a call volume of over 10,000 annually. The hotline provided callers information and referrals for health and social services, such as family planning, health coverage, and breast and cervical cancer screening, and callers were routinely called back for reminders about upcoming appointments, scheduling for annual screenings, and follow-up to educational programs. For this study, hotline users who reported being caregivers for unvaccinated adolescents were randomized into the intervention or control condition on an alternating (on/off) weekly schedule. The system-delivered intervention was conducted between November 2013 and June 2016. Eligible participants completed a brief baseline telephone interview. Immediately following the baseline interview, participants calling during intervention weeks received the multicomponent intervention, including brief telephone education tailored to constructs (i.e., beliefs about vaccine effectiveness, perceived HPV risk and severity, and barriers to HPV vaccination) assessed in the baseline survey and referral to a clinic where the HPV vaccine could be received for free or at a reduced cost. Intervention group participants were also sent brief print materials reflecting the information and referral shared during the telephone call. All participants were recontacted 3 months and 9 months after baseline to assess HPV vaccine initiation and completion. Given the extensive language capabilities of regular hotline operations, participants were given the opportunity to complete study elements (surveys and intervention) in English, Spanish, Chinese, Korean, or Armenian. This study protocol was approved by the Institutional Review Boards of both UCLA and the Los Angeles County Department of Public Health and registered with ClinicalTrials.gov (NCT02067507).

The Multilevel Health Outcomes Framework (MHOF) guided the development of the surveys, telephone education scripts, and print intervention text (7–9). The MHOF is a comprehensive conceptual framework that synthesizes constructs from several theoretical models, including implementation frameworks, and posits individual health behavior is influenced by a complex myriad of individual, health system, community, and societal level factors. This study utilized existing infrastructure to implement an intervention to address mutable factors at the individual level (i.e., the parent/caregiver), such as HPV vaccine attitudes and barriers to increase HPV vaccine uptake for their adolescent children. Additionally, the intervention modified the system to change it from a passive provider of services into a proactive system that took the additional step of offering a service (HPV vaccine education) that the caller did not directly request. Although our study did not intervene to modify societal or community factors, our intervention was targeted and tailored to take these factors into consideration in how the service was provided.

Eight OWH hotline operators administered all baseline and follow-up interviews and implemented the brief telephone education intervention over the study period. Operators received 20 hours of training, which was developed and administered by study investigators and the project coordinator. Training topics included HPV and the HPV vaccine, conducting research interviews and intervention delivery, weekly automated randomization protocol, and human subjects research training. The telephone education was scripted and delivery was supported through a computer-assisted platform, which helped to enhance fidelity to the study protocol. The UCLA study team also collected detailed caller tracking data on a weekly basis to identify and address any issues related to participant recruitment, refusal rates, screening, enrollment, and intervention delivery, and provided booster training as needed.

Eligible caregivers were recruited through the OWH's telephone hotline. Caregivers were eligible if they were between ages 18 and 70 and reported being caregivers and/or medical decision-makers for an unvaccinated adolescent between ages 11 and 17. For caregivers with more than one unvaccinated child in the age range, the caregiver was asked to report on the youngest child. Caregivers were asked about previous study participation to ensure no one participated more than once. Upon receiving an incoming call, operators first responded to the caller's request using routine hotline protocols. The most common reasons for hotline calls included requests for assistance with scheduling preventive health services such as mammograms and Pap tests or requests for information about health insurance or other women's health issues. After addressing the caller's initial request, operators invited the caller to be screened for study eligibility. In addition to recruitment through incoming calls, participants were recruited through outgoing calls using the OWH's hotline user database. OWH routinely utilizes their database to follow-up with previous hotline users to assess current need for health information or health care referrals or to offer new programs or services. Operators invited women to be screened for study eligibility during these routine outgoing calls and also specifically called back clients who had utilized the hotline over the past two years to inform them of the opportunity to be screened for and participate in the study.

The multicomponent intervention included brief (5-minute) telephone education, referral to a local HPV vaccine provider, and a mailed brochure. Operators provided basic information about the HPV vaccine and addressed the main vaccine barriers endorsed by the caregiver during the baseline survey (sample telephone education scripts in Supplementary Table S1). Operators also provided caregivers with a customized referral to a local clinic, in a location convenient to the family, where the adolescent could receive the HPV vaccine for free or at a low cost. For adolescents with a usual source of care, operators recommended caregivers return to the usual source of care, often health care facilities outside of the County system such as a community health center, to ensure continuity of care. Operators also accounted for the adolescent's insurance status and attempted to link adolescents without a usual source of care to clinics that could provide comprehensive primary care services. The mailed brochure, titled “Why is the HPV vaccine important for my daughter/son?”, included basic information about the vaccine and scripts addressing MHOF constructs (e.g., perceived susceptibility, perceived efficacy) and common concerns parents endorsed during the baseline survey (e.g., child too young; sample brochure in Supplementary Fig. S1). The brochure was provided in the caregiver's preferred language and was tailored by insurance status, sex of child, and race/ethnicity (e.g., images, risk information). For example, the brochure for caregivers of Latina adolescents mentioned Latinas have higher rates of cervical cancer compared with the U.S. population. The brochure also included information about the clinic to which the operator referred the caregiver for the HPV vaccine. Control group participants did not receive telephone education or HPV provider referral, but were mailed a brief in-language Centers for Disease Control & Prevention (CDC) fact sheet about HPV and the vaccine. This approach was consistent with OWH's desire to provide some information on HPV to all participants, particularly considering the high cervical cancer risk in our target population and that participants had just completed an HPV vaccination-focused survey.

The OWH hotline is a computer-assisted system into which the study data collection and intervention elements were integrated. Baseline telephone surveys lasting 10 to 15 minutes were administered after determining eligibility and obtaining verbal informed consent. All caregivers who completed the baseline survey were recontacted and invited to complete follow-up telephone surveys 3 and 9 months later. Participants received a $20 gift card after completing the baseline survey and $30 after each follow-up survey. The instruments were adapted from English into the target languages using methodology employed in prior studies to assure parallel versions in all languages (7, 10–12). Given OWH operators were community experts with training and experience communicating with members of the target study population, they provided additional input on item order, question phrasing, terminology used, and reading level to ensure survey items were appropriate for the target audience.

Survey items were adapted from the literature and the study team's prior research (7). The baseline survey included 34 items and assessed demographics, health insurance, and usual source of care for the child and caregiver. Using 5-point Likert scales, the survey also assessed caregiver's attitudes and beliefs about HPV and the vaccine including perceived severity of HPV (getting infected is a serious health problem); perceived susceptibility of child acquiring HPV (child likely will be infected with HPV in life); perceived efficacy of the vaccine (vaccine is effective at preventing cervical cancer); perceived risks of the vaccine (HPV vaccine may cause health problems in future, getting vaccine may cause problems getting pregnant among girls, child more likely to think it is alright to have sex if vaccinated among girls); barriers to vaccination (child is too young to get HPV vaccine, concerned about vaccine side effects and cost); HPV vaccine self-efficacy, and intentions to vaccinate the adolescent in the next 6 months (child will likely get HPV vaccine in next 6 months among unvaccinated). The survey also asked caregivers if they felt they had sufficient information to decide about vaccination for their child (yes/no, among unvaccinated). The 3-month follow-up survey included 13 items and assessed receipt of any HPV vaccine doses, number of doses, and location where doses were received. The 3-month survey did not include caregiver attitude and belief items. The 9-month follow-up survey included 37 items and assessed caregiver attitudes and beliefs about HPV and the vaccine (similar to baseline), and number, timing, and location of HPV vaccine doses received. The primary outcome, HPV vaccine initiation, was defined as receipt of at least one dose within 9 months. HPV vaccine completion was defined as receipt of all required doses according to the most recent age-based dosing recommendations published in 2016: two doses for adolescents who initiated the vaccine prior to age 15 and three doses for those who initiated vaccination at age 15 years or older (13).

Participants in the intervention and control groups were compared on baseline characteristics using t tests and χ² tests. We examined the effect of the intervention on HPV vaccine initiation (primary outcome) using several approaches, including analyses of all randomized participants that assumed participants with missing outcome data did not initiate vaccination and analyses of all participants with follow-up data; the latter approach has been described as a type of modified intent-to-treat (ITT) analysis (14). We chose to present the results of the modified ITT analyses, given the proportion of patients with missing outcome data was low (6%) and all analysis approaches gave similar results.

For the modified ITT analysis, participants who reported initiation at either the 3- or 9-month follow-up were classified as initiated. Participants who reported no initiation at both the 3- and 9-month follow-ups, or were missing at one assessment and reported no initiation at the other, were classified as uninitiated. Participants missing both follow-up assessments were not included. The test for an intervention effect on initiation was conducted using a mixed effects logistic regression model that included a random intercept for week to account for potential clustering of responses by week of enrollment. The model adjusted for caregiver gender and household income due to baseline differences between conditions on these factors. HPV vaccine completion (receipt of required doses based on age) was analyzed using the same modeling approach. For each attitude and belief item assessed at baseline and 9 months, we tested for a difference in change over time between the intervention and control groups using a linear mixed model with an interaction between group and time and a random intercept for week. These analyses were conducted with the subsample of participants with assessments at both baseline and 9 months.

Sample size calculations assuming a 10% increase in control group and 25% increase in intervention group suggested a sample of 462 participants was needed to achieve 80% power. However, after interim data analysis, we made a decision to stop recruitment early, given the substantially higher than expected increase in vaccine initiation rates in the control group and equivalency in outcome rates in both study groups.

A total of 2,786 participants were screened for eligibility (Fig. 1), of whom 238 (9%) met eligibility criteria and were enrolled in the study (Intervention condition n = 138; control condition n = 100). Of the 238 participants, 60% were recruited through incoming calls and 40% through outgoing calls operators made using the hotline user database. Three-month follow-up interviews were completed with 215 participants (90% of enrolled; intervention condition n = 122; control condition n = 93). Nine-month follow-up interviews were completed with 204 participants (86% of enrolled; Intervention condition n = 118; control condition n = 86). Of the 238 participants enrolled, 225 provided some follow-up data and were included in the primary outcome analyses (91%).

Table 1 outlines sample characteristics. Caregivers (n = 238) were on average 44 years old (SD 8) and mostly female (93%), Latino (71%), Spanish-speaking (63%), foreign-born (86%), low-income (80%), publicly insured (52%), and married (68%). Respondents' children were on average 13 years old (SD 2) and mostly female (67%), Latino (71%), publicly insured (82%), had usual source of care (90%), and had excellent health (40%). Characteristics of children were comparable between study groups, but a greater proportion of males (vs. female) and lower- (vs. higher-) income caregivers were enrolled in the intervention group (P < 0.05).

Caregivers omitted from outcome analyses due to incomplete data (n = 13) were younger, more likely to be male versus female, have a younger child, have a child without a usual source of care, and have a child whose usual source of care was a private hospital or urgent care, compared with caregivers/adolescents included in outcome analyses (n = 225; P < 0.05).

There were no differences in HPV vaccine initiation at 3-month follow-up (n = 215; 28% in both study conditions). A substantial proportion of adolescents in both conditions (n = 225; 45% intervention, 42% control) initiated vaccination by 9-month follow-up. Table 2 displays the results of the logistic regression analysis that assessed the effect of the intervention, controlling for small baseline differences in caregiver gender and household income observed between the intervention and control condition; n = 225. Results confirm lack of an intervention effect for HPV vaccine initiation [OR 1.09, 95% confidence interval (CI), 0.52–2.29; P = 0.83]. At 9-month follow-up, 24% of participants in both groups had completed the series, consistent with age recommendations for the 2- or 3-dose schedule. No significant difference was observed for series completion between the intervention and control condition (OR 1.15; 95% CI, 0.49–2.66; P = 0.75). Analyses including all adolescents randomized (n = 238), assuming those lost to follow-up had not initiated, yielded similar results for both initiation (41% intervention, 41% control) and completion (22% intervention, 23% control).

Figure 2 displays changes in caregiver attitudes and beliefs related to HPV and the HPV vaccine by study group among participants who completed the 9-month follow-up survey (n = 204). Many measures significantly changed over time in the expected direction within study groups, but none of these changes differed between study conditions (P > 0.05). The largest pre-post changes were observed in the proportion of caregivers who believed they had enough information to decide about the vaccine (30 percentage point [pp] increase in intervention, 29pp increase in control; P < 0.05), those who did not perceive their child as too young to get the HPV vaccine (29pp increase in intervention, 22pp increase in control; P < 0.05), and those who perceived their child would likely be infected with HPV in lifetime (28pp increase in intervention, 22pp increase in control; P < 0.05). In both study groups, increases were also observed in the proportion of caregivers who were not concerned about vaccine costs and side effects (P < 0.05). The proportion of caregivers who disagreed that their daughter would more likely think it is alright to have sex if vaccinated increased in the control group only (P < 0.05). There were no significant changes in perceived severity of HPV, perceived vaccine efficacy, perceived risks of the vaccine (belief vaccine would cause future health problems or problems getting pregnant for girls), or intention to vaccinate their child.

The goal of this study was to implement and evaluate a low-intensity intervention, embedded in an existing multi-lingual county hotline, to improve HPV vaccine initiation rates among low-income, ethnic minority adolescents. By 9-month follow-up, 45% of adolescents in the intervention group and 42% in the control group had received at least one dose of the vaccine. This difference between study groups was not statistically significant, indicating lack of an intervention effect. The average 44% increase across the two groups is substantially larger than what would have been expected from a secular trend. On the basis of repeated cross-sectional data between 2013 and 2016 from the National Immunization Survey-Teen (NIS-Teen), the average annual percentage point change in the prevalence of HPV initiation increased by less than 3% for girls and 8% for boys ages 13 to 17 years over the same time period (15, 16). Consistent with the initiation rate increase observed in both study groups, we found significant improvements in both groups in factors from the MHOF hypothesized to influence HPV vaccine acceptance including caregiver attitudes towards HPV and the vaccine.

Although there was no intervention effect, achieving a 44% vaccine initiation rate over a 9-month period among previously unvaccinated adolescents can be considered a public health success. The reason vaccine uptake increased so much in the control group is unclear. One hypothesis is that it may be due to a priming effect prompted by participation in a HPV vaccine study and completion of a baseline survey assessing HPV related attitudes, beliefs, and vaccine receipt (17, 18). Also, although caregivers in the control condition did not receive the brief telephone education or referral to a vaccine provider, they received a basic in-language HPV vaccine fact sheet. It is possible that participation in the study, completion of the baseline survey, and receipt of the fact sheet served as a low-dose intervention prompting caregivers in the usual care condition to seek out the vaccine for their child. Non-English speaking caregivers in our sample may otherwise have had limited access to in-language information about the HPV vaccine. The fact that almost all adolescents in both groups had a usual source of care likely facilitated vaccine receipt. This observation suggests that a very simple generic intervention may be sufficient for increasing vaccine receipt among a subset of the low-income population characterized by having a usual source of care and fairly low level of barriers to vaccine receipt.

An alternate explanation is that the control group may have been contaminated if participants received education from hotline operators during weeks assigned to usual care. We believe this explanation is unlikely because our computer-assisted telephone software prompted operators to deliver relevant intervention messages during intervention weeks only. Research staff regularly visited the study site and observed that operators relied on these prompts and scripts for intervention delivery. It is therefore unlikely that operators memorized and delivered intervention scripts to control participants unprompted on usual care weeks. In addition, to avoid any contamination in dissemination of print educational information by hotline operators, our research staff facilitated the mailings of either the tailored intervention brochure to the intervention group or brief fact sheet to the control group.

Our study had limitations. First, we relied on parent report of HPV vaccine receipt as our outcome variable. Medical record validation or direct medical record abstraction was not feasible because adolescents obtained the vaccine at a very wide range of clinic sites located throughout Los Angeles County versus in a single, or even a small set of, clinic systems. Prior studies have found parental recall is comparable to provider report for HPV vaccination, particularly for HPV vaccine initiation (19), although we recognize social desirability bias may contribute to overreporting. We considered comparing parent report to state immunization registry data, but we were unable to obtain the individually-identifiable data needed for verification due to confidentiality concerns. Our intervention was completed prior to publication of the 2016 ACIP guidelines, so may not be directly comparable with studies conducted more recently. Our intervention was delivered through a county telephone hotline focused on women's health issues and a relatively small proportion of callers fell within the age-range that would typically have an adolescent child. Such hotlines are not universally available across health departments and are likely diminishing in popularity given the general transition away from telephone-based communications towards electronic modalities. Hotline callers requesting assistance with health-related concerns may have been more engaged in their health, activated to pursue preventive care, and receptive to vaccine messaging for their child compared with the community at large. This study may not be directly comparable with other intervention studies that use hotlines connected to a broader set of health and social services (20, 21). In addition, we did not assess exposures to HPV vaccine information outside of the study but do not expect study groups would have been differentially exposed. Finally, given the unique characteristics of Los Angeles County, such as the structure of its safety-net system and its population characteristics, outcomes of initiatives such as ours to improve HPV vaccination rates may not generalize to low-income, predominantly minority adolescents in other regions.

It should be noted that over half of adolescents in our sample had still received no doses of the HPV vaccine over the study period. Although nearly 90% of adolescents had health insurance, there was no straightforward system to refer adolescents for vaccination. Adolescents who already had a usual source of care they were satisfied with were referred back to that source. Although caregivers were using a county-funded hotline, very few adolescents in our sample would be eligible to receive care through county-funded vaccination programs given they only serve uninsured patients or those who are ineligible for any public insurance. As a result, adolescents without a usual source of care were referred to a patchwork of safety-net clinics, primarily federally qualified health centers or look-alikes. For these individuals, reaching out and scheduling an appointment with a new clinic likely represented a barrier. More intensive interventions that address the logistical barriers associated with accessing health services in a large, complex, decentralized, urban safety-net setting may ultimately be more successful in increasing HPV vaccine initiation and completion among low-income adolescents.

R. Bastani reports grants from NIH during the conduct of the study. B.A. Glenn reports grants from NCIR01CA154549-01A1 during the conduct of the study. C.M. Crespi reports grants from NIH during the conduct of the study. N.J. Nonzee reports grants from NIH during the conduct of the study. L.C. Chang reports grants from NCI R01 CA154549 and grants from NIH/NCI P30 CA16042 during the conduct of the study. V.M. Taylor reports grants from NIH during the conduct of the study. No disclosures were reported by the other authors.

R. Bastani: Conceptualization, resources, supervision, funding acquisition, investigation, methodology, writing–review and editing. B.A. Glenn: Conceptualization, supervision, investigation, methodology, writing–original draft, writing–review and editing. R. Singhal: Conceptualization, resources, data curation, supervision, investigation, methodology, writing–review and editing. C.M. Crespi: Conceptualization, formal analysis, investigation, methodology, writing–review and editing. N.J. Nonzee: Data curation, investigation, visualization, writing–original draft, project administration, writing–review and editing. J. Tsui: Data curation, investigation, project administration, writing–review and editing. L.C. Chang: Data curation, formal analysis, visualization, writing–review and editing. A.K. Herrmann: Investigation, methodology, writing–review and editing. V.M. Taylor: Conceptualization, investigation, methodology, writing–review and editing.

This research was funded by NIH/National Cancer Institute (NCI) R01 CA154549 (PI: R. Bastani) and NIH/NCI P30 CA16042 (PI: M. Teitell/J. Gasson). N.J. Nonzee was also supported by NIH/National Center for Advancing Translational Science (NCATS) UCLA CTSI TL1 TR000121 (S. Dubinett) and TL1 TR001883 (PI: S. Ettner/M. Inkelas) and Agency for Healthcare Research and Quality (AHRQ) T32 HS000046 (PI: T. Rice). We would like to acknowledge the vital contributions of the Office of Women's Health staff and leadership, without whom the study would not have been possible. We would also like to extend our appreciation to Ms. Leah Sanchez and Ms. Mallika Bhandarkar, who helped to coordinate study activities.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.