Results of a Multilevel Intervention Trial to Increase Human Papillomavirus (HPV) Vaccine Uptake among Adolescent Girls

This article is featured in Highlights of This Issue, p. 567

Studies have documented the association between human papillomavirus (HPV) and six types of cancers, including cancers of the cervix, oropharynx, anus, vagina, vulva, and penis (1–5). In the United States., a quadrivalent HPV vaccine was approved for females aged 9 to 26 years in 2006 and for males of the same age in 2009 (6, 7) that protects against two oncogenic HPV types (16 and 18) responsible for most anogenital cancers and two nononcogenic HPV types (6 and 11) responsible for most cases of genital warts (7). A bivalent HPV vaccine was approved in 2009 for females, and in 2014, a 9-valent HPV vaccine was approved for females and males (7–9).

Uptake of HPV vaccines is higher in Australia (10–12) and some countries in Western Europe (13–15) than in the United States, with current rates of 39.7% for girls and 21.6% for boys aged 13 to 17 completing the three-dose vaccine series (16). Recommendations focus on vaccination of girls and boys 11 to 12 years of age and as early as age 9, with catch-up vaccination suggested for 13- to 26-year-old females and 13- to 21-year-old males (17, 18). While rates of HPV vaccine uptake are universally low, vaccine uptake should be more vigorously pursued among populations that have higher cervical cancer rates. One such population (19–22) is residents of Appalachia. Appalachia is a federally designated region of the United States, with deficits in income, educational attainment, business growth, and access to healthcare (23–25); moreover, cancer is the number one cause of mortality in this region (26, 27).

Previous research in Appalachia found negative attitudes and poor knowledge about HPV, the HPV vaccine, and cervical cancer among parents (28, 29). Using community-based participatory research strategies, our team, in partnership with community members from local cancer coalitions (advocacy organizations), developed a multilevel intervention (MLI) to improve HPV vaccination among girls aged 9 to 17 in Appalachian Ohio (30). Because previous studies have mainly focused on addressing one or two levels of influence (e.g., providers or parents; refs.31, 32), and our formative research (30) suggested that multiple levels influence the decision to get the HPV vaccine in this population, we thought that an MLI, addressing three levels most influential (health clinics, providers, and parents of girls aged 9 to 17 years), was appropriate. The theoretical framework for the intervention was guided by several theories, including the Health Belief Model (33), Theory of Reasoned Action (34), Extended Parallel Process Model (35), and Organizational Developmental Theory (36). The goal of this article is to describe and report the results of a group-randomized trial (GRT) at the county-level testing of the efficacy of the MLI to improve HPV vaccination among this population.

This study was conducted from 2010 to 2015. Originally, the study design called for one clinic from each county to participate and provide access to providers and patients to approach for recruitment. Clinics provided letters of support for the grant application, but when asked to provide or contact patients, clinics declined participation. The study design, therefore, was modified, as described below. CONSORT diagrams outline the number of parent participants (Fig. 1), and clinics and providers recruited (Fig. 2) by study arm and followed. This study was approved by the Institutional Review Board of the Ohio State University (OSU).

The OSU research team and community coalition members identified 12 counties to participate in the study. Counties were pair-matched based on cervical cancer incidence rates and location. One county from each pair was randomly assigned to receive the HPV vaccine intervention (n = 6), whereas the other county was assigned to the comparison condition (n = 6), an educational intervention (attention control) about influenza (flu) and the flu vaccine.

Coalition members and study staff identified all health clinics in each of the 12 counties. Clinics were contacted by study staff, eligibility determined, and invited those eligible to participate. Clinics were eligible if they (i) were located in one of the 12 counties in the study; (ii) provided care to girls 9 to 17 years old; and (iii) either provided immunizations or referrals for immunizations. A total of 98 clinics in Ohio Appalachia were approached to participate (Fig. 2). Of these, 5 were not reached despite multiple attempts via phone, 14 were ineligible, and 41 (49%) declined to participate (too busy, no time, not interested). Thus, 24 clinics agreed (10 intervention, 14 comparison) to participate fully, and 14 clinics participated only in the clinic-level intervention.

To be eligible for participation, providers had to be employed at one of the 24 fully participating clinics. Potential provider participants (staff involved in educating patients and/or administering vaccines) were recruited by their clinic manager or lead physician to attend a study education session conducted at regularly scheduled meetings.

Eligible parents had to (i) be age ≥18; (ii) be able to speak, read, and write English; (iii) be a resident of one of the participating 12 Ohio Appalachian counties; (iv) be a parent or a legal guardian of a young girl (age 9 to 17 years) who had not received the HPV vaccine (if a parent had more than one daughter, they were asked questions about their eldest daughter); and (v) have no children who have received the HPV vaccine [as confirmed by medical record review (MRR)].

Parents were sampled using the following procedures. A master list was first obtained from Survey Sampling International (37) consisting of all households from the 12 counties that had at least one female resident aged 9 to 17 years old. A random sample of households was drawn for each county, and the names and addresses were given to study staff, who then mailed a recruitment letter and informational sheet introducing the study to the selected households. Study staff made at least 10 attempts (calling at different times of the day, weekday, and weekend, as well as sending an “unable to contact” letter) to reach the listed household. Once the household was reached, the interviewer asked to speak to the parent who was most involved in making health decisions for their daughter. The study was then described, questions answered, informed consent obtained, and the baseline survey administered.

Development of the MLI has been previously described (30); however, each level is briefly detailed below.

The clinic-level intervention focused on providing an environment where information about HPV vaccination was visible and readily available. We developed study-specific waiting room and examination room posters, brochures, and tabletop tent cards for the HPV vaccine intervention. Study staff regularly checked in with clinic staff to answer questions and check common areas and exam rooms to see if study or other educational materials were visible (on HPV vaccination) and replenished depleted materials.

The educational session for providers, facilitated by a member of the research team, included a 1-hour PowerPoint presentation and handouts on the HPV vaccine, focusing on current evidence-based HPV vaccine information and strategies designed to assist physicians in discussing HPV vaccination with parents. For the comparison arm, providers were given information on the flu and flu vaccine. For the HPV vaccine education session, we modified an evidence-based tobacco cessation program (38) focused on the “5 A's” and “5 R's.” Providers completed surveys that assessed HPV vaccine knowledge and attitudes, as well as self-efficacy to talk to parents and patients about both vaccines, before and after the educational session. At 12 months following the educational session, a survey was administered to providers to assess the durability of intervention effects.

After the initial phone call, intervention participants were mailed a packet that included a $10 thank you gift card for completing the telephone-administered baseline survey, an educational brochure and DVD video about HPV and HPV vaccination, a magnet reminder to get the 2nd and 3rd HPV vaccine shots, and a Centers for Disease Control and Prevention (CDC) HPV vaccine information statement. The comparison group was mailed a packet that included similar items, a flu vaccine information statement from the CDC and flu information sheets from Ohio Department of Health. The packet for both groups also included a medical record release form and a self-administered survey which they were instructed to return in an enclosed self-addressed prepaid. Participants were mailed a $5 gift upon receipt of these items.

Health educators conducted an education session about the HPV vaccine (or flu vaccine to the comparison group) via telephone to reinforce the message in the educational materials regarding the need for a vaccine and addressed any vaccination barriers or questions.

An administrator from each clinic was asked to complete a questionnaire measuring patient volume, revenue sources, number of insured patients, and number of providers, prior to the introduction of the provider and patient components.

To determine knowledge about HPV and the HPV vaccine, providers were asked about state and national cervical cancer incidence and mortality rates, their knowledge about parental concerns regarding the HPV vaccine, and what they knew about patient-level predictors of HPV vaccination. Provider knowledge (in both groups) was measured by five true/false statements with correct answers representing higher levels of knowledge. Self-efficacy to talk to patients and parents about the HPV vaccine was assessed with two items. Provider demographic characteristics (age, gender, race, ethnicity, education, and job title) were also assessed.

Whether daughters received the first HPV vaccination shot within 3 months after mailing the intervention materials was determined by MRR. This timeframe was selected as individuals usually act close to cues to action, i.e., the educational intervention, than later (33). Medical records from any clinic a parent listed on the survey or release form were obtained, regardless if the clinic was participating in the study. Shortly after receiving the health educator call and one month later, a research interviewer contacted participants and administered a short phone survey to collect information on intent to vaccinate and vaccination status, for the specific vaccine (e.g., HPV vaccine in intervention group, etc.). One additional phone call 6 months later was conducted to collect information on HPV vaccination in both groups. A gift card for each follow-up survey was sent to the participant as a thank you for their time. Participants could receive up to $40 in gift cards for returning all forms and surveys.

Secondary outcomes included HPV vaccine uptake at 6 months and uptake of the second and third HPV vaccine shots by MRR and parental report on the six-month phone survey. Six months was chosen as this is the recommended timeframe to receive the three shot series; thus, we also assessed receipt of the second and third HPV vaccine dose among those who received the first dose.

Parents provided information about their daughter's age and race, and their own age, race, ethnicity, marital status, education, employment status, household income, and health insurance on the self-administered survey. General health and smoking status were asked in the posteducation survey.

Parents were asked (yes/no) at baseline if their daughter had ever received a flu, tetanus, diphtheria, and pertussis (Tdap), or meningococcal vaccine (MCV) at baseline. At the six-month survey, parents were asked about HPV and flu vaccination for their daughter.

Parents were asked at baseline and after education if they intended to have their daughter get the HPV vaccine. Responses for this item were measured on a five-point Likert scale (1 = strongly disagree to 5 = strongly agree). At the six-month survey, parents of unvaccinated daughters were asked (yes/no) if they intended to have their daughter get the first dose of the HPV vaccine within the next month and in the next year.

Intervention group participants were asked at baseline and after education about their daughter's susceptibility to getting HPV by responding to the item, “There is a high chance that my daughter will get HPV.” To measure perceived severity, participants were asked to respond to the item, “If my daughter were to get HPV it would be a very serious threat to her health.” Responses for these two items were measured on a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree).

Intervention group participants were asked at baseline and after education about their comfort in (self-efficacy) and intention to encourage their daughter to speak to a doctor about the HPV vaccine. Social influence on a participant's intent to encourage their daughter to speak to a doctor/nurse was also assessed. Responses for these items were measured on a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree).

Intervention participants were asked at baseline and after education about their knowledge, attitudes, and beliefs about the HPV and the HPV vaccine. They were asked to respond to 10 true or false statements about HPV and HPV vaccine knowledge, and for beliefs and attitudes about HPV and the HPV vaccine, they responded to 16 items measured on a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree).

The primary outcome was receipt of the first HPV vaccine shot within 3 months of the mailing of intervention materials, as determined by MRR. The sample size for the study was chosen based on power calculations assuming an HPV vaccination rate of 28% in the comparison counties (based on estimates from respective health departments) and a 58% vaccination rate in the intervention counties, based on previous studies that used educational and provider/system interventions to increase vaccine uptake (39). The interclass correlation coefficient used was 0.07, using the formula in Donner (40) and vaccination rates from the respective county health departments. Based on these parameters, a sample size of six counties per treatment arm and 23 participants per county was chosen to provide 85% power. We expected 15% attrition from baseline to follow-up, and thus recruited an additional five participants per county to provide a final sample size of 28 per county. Loss to follow-up over 6 months was actually 13.8% among intervention parents and 11.7% among comparison parents (P = 0.557).

Since “missingness” of medical record data was 25.3% among intervention and 23.9% among comparison groups (P = 0.644), the outcome was analyzed in two ways: (i) an analysis of only participants with MRR data (complete case) and (ii) an intent-to-treat analysis assuming participants without MRR data did not receive the shot, as is commonly done in studies where a drop-out is considered a treatment failure (41). In each analysis, a clustered permutation test was used to determine if the outcome distribution differed by study group. The test was implemented using a two-stage approach recommended by Gail and colleagues (42) to account for baseline factors that were unevenly distributed across treatment arms. The same analysis strategy was used to analyze several secondary outcomes: receipt of the first HPV vaccine shot within 6 months based on MRR, parent-reported first shot status, and receipt of the second and third shots.

Changes in attitudes and beliefs of parents in the intervention counties were examined by comparing the proportion who agreed/strongly agreed with statements about HPV and the HPV vaccine at baseline and at the posteducation survey. Change in knowledge about HPV and the HPV vaccine after receiving the video was analyzed using a Wilcoxon sign-rank test. Linear mixed models were used to test for changes in provider knowledge about HPV and HPV vaccine within each arm and to compare changes across arms from baseline to the 12-month survey.

Of the 24 participating clinics, 22 representatives completed the questionnaire about clinic characteristics (Table 1). Clinics employed, on average, two physicians, four nurses, three nurses who administer vaccines, and four support personnel. The clinics provided care for, on average, approximately 3,400 adult and 1,300 child patients. On average, 40.7% of the practice revenue was from Medicaid, followed by Medicare (14.6%), other managed care (13.0%), and other insurers (12.2%). Lastly, the average percentage of uninsured patients was 22.4% for adults and 18.8% for children. The average number of physicians in the intervention clinics was considerably higher (3.1 vs. 1.2) than the comparison clinics. All other clinic-level factors were similar across arms. All clinics displayed study-specific materials as directed, and material was restocked by study staff at all clinics.

Providers in the intervention counties were more likely to have a professional degree (28.1% vs. 21.0%) and have a title other than physician or nurse (45.6% vs. 37.7%). All other provider-level demographic variables were similar between groups. Loss to follow-up over 12 months was 33.3% among intervention and 24.2% among comparison providers (P = 0.270).

The demographic characteristics of parent and child participants (n = 337) are presented in Table 2. The mean age was 43.5 years, and the majority of them (98.5%) were white, representative of the Ohio Appalachian region. Most parents reported being married or living as a couple (89.3%), being employed full or part-time (63.4%), having health insurance (91.1%), annual household incomes <$70,000/year (63%), and at least some college education (71.2%). Parents in the intervention group were, on average, 2 years older (42.5 vs. 44.5). All other demographic factors were similar across study arms for both parents and daughters.

Twenty parents in the intervention counties listed a participating intervention county clinic as their usual source of care and 30 parents in the comparison counties likewise listed a clinic in the comparison counties. None of the parents listed a clinic from the opposite group's county as a source of care. Of parents responding to the question at 6 months (147/155), 39% reported discussing the HPV vaccine with a healthcare provider, 44% reported they visited a healthcare provider but did not discuss the HPV vaccine, and 17% did not visit a healthcare provider since the baseline survey. Moreover, only 5.7% of parents reported visiting one of the participating clinics during the study.

Of the 337 parents enrolled in the study, we were able to obtain MRR data on the outcome for 254 (75%) daughters. The proportion of daughters with MRR data did not differ by study group (76.1% comparison, 74.7% intervention, age-adjusted P = 0.644). Among the participants with MRR data, 10 (7.7%) daughters of intervention participants received the first dose of the HPV vaccine within 3 months compared with 4 (3.2%) daughters of comparison participants (age-adjusted P = 0.061; Table 3). Vaccine uptake estimates were lowered when it was assumed that daughters missing MRR data did not have the shot (age-adjusted P = 0.067).

Using MRR data, 17 (13.1%) daughters of intervention participants received the first HPV vaccine shot within 6 months compared with 8 (6.5%) daughters of comparison participants (P = 0.002). Parent-reported HPV vaccinations at 6 months was also significantly higher among daughters of intervention participants (17.3% vs. 9.0%, P = 0.009). Results were similar for each outcome when we assumed those with missing data did not receive the vaccine.

Among those who received the first shot by 6 months according to MRR, 12 (70.6%) daughters of intervention participants and 3 (37.5%) daughters of comparison participants received the second dose (P = 0.119). Among those who received the second HPV vaccine shot according to MRR, six (50%) daughters of intervention participants and two (66.7%) daughters of comparison participants received the third dose (P = 0.524).

For intervention parents who completed the baseline and posteducation survey (n = 155), 60 (38.7%) agreed/strongly agreed with the statement “I intend to encourage my daughter to get the HPV vaccine” before the intervention which significantly increased to 91 (58.7%) following the posteducation survey (P < 0.001). At 6 months, 124 parents of unvaccinated daughters in the intervention group provided information about intent. Of those 124, 40 (32.3%) indicated that they intend to have their daughter receive the first dose of the HPV vaccine within the next year. Parents who did discuss the HPV vaccine with their healthcare provider were more likely (OR = 3.43; 95% confidence interval, 1.19–9.87) to have their daughter receive the HPV vaccine than those who did not discuss the HPV vaccine with their healthcare provider.

Among the 155 intervention participants who responded to questions about viewing the intervention materials, 139 (89.7%) reported that they watched the video and 146 (94.2%) reported that they read the brochure and vaccine informational statement. Intervention group parents averaged 9.4 ± 1.0 correct answers to HPV and HPV vaccine knowledge questions (out of 10) on the posteducation survey, which was significantly higher than their knowledge score at baseline (7.4 ± 2.1, P < 0.001 from Wilcoxon sign-rank test) and the baseline score of comparison group parents (7.3 ± 1.9, P = 0.001 from clustered permutation test). Several belief and attitude items about HPV and the HPV vaccine also changed between the baseline and posteducation surveys (See Table 4).

Prior to the in-clinic education session, intervention group providers averaged 4.4 correct answers to HPV and HPV vaccine questions of five (N = 57, SD = 0.7) which significantly (P < 0.001) increased to an average of 4.9 correct of five following the education session (N = 55, SD = 0.3). The average number of correct responses by these providers decreased to 4.6 (N = 37, SD = 0.7) at 12 months, which was still significantly better than their baseline knowledge (P = 0.035), but was similar to the 12-month change observed in the comparison group (P = 0.87). Self-efficacy to talk to parents and patients about the HPV vaccine in the intervention group was similar at the baseline (89% for both questions) and 12-month surveys (83% to talk to parents and 92% to talk to patients).

The goal of this study was to test the efficacy of an MLI to improve the uptake of the HPV vaccine among Appalachian girls aged 9 to 17 years old. The primary outcome, receipt of the first HPV vaccine shot within 3 months, was more common in the intervention group though vaccination in both groups was very low (less than 10%). These numbers nearly doubled by 6 months to 13.1%, but still remained low, making it difficult to assess completion of the second and third shots among those who received the first shot.

MLIs are gaining more interest as a way to reduce disparities among underserved populations (43, 44). While these interventions have been tested for smoking cessation and healthy diet, none has been directed at HPV vaccine uptake among girls. Perkins (31) found that a provider-focused intervention at two federally qualified community health centers consisting of six to eight education sessions, feedback about providers' and practices' HPV vaccination rates, and quality improvement incentives (credits for maintaining board certification), significantly increased HPV vaccine initiation among females in the active intervention period compared with comparison clinics; however, differences did not remain significant at the 6-month postintervention period. Two studies directed at parents have also shown positive effects. Aragones (45) used a nonequivalent group design to test an education session plus text-messaging intervention compared with an education session intervention among Mexican-American parents with a child who needed the HPV vaccine. Based on parental report, there was an 88% series completion rate among those receiving the first HPV vaccine dose in the education plus text-messaging group compared with 40% in the education-only group (P = 0.004). Parra-Medina (46) tested a promotora outreach, education, and navigation program for HPV vaccination among Hispanic women with a daughter who did not receive the HPV vaccine. Compared with the brochure-only parent participants, those parents who received the promotora navigation program were more likely to complete the HPV vaccine series. Thus, it appears from these studies that more intensive, multicomponent interventions directed at either clinicians and/or parents are effective in increasing vaccination rates.

The current study, while demonstrating a greater effect in the intervention compared with the comparison group, did not have a large effect on the uptake of the HPV vaccine. Due to difficulties in accessing patients at the clinics where the intervention was delivered, parents were recruited from county-level commercial lists. Thus, few daughters of parents recruited visited the participating clinics, resulting in the small effect we found. Moreover, while knowledge about HPV and HPV vaccine increased among providers in participating intervention clinics, the impact of this increase on vaccination rates was not able to be assessed because of study design limitations. However, data indicate that the majority of intervention parents (83% among respondents) did have an opportunity to discuss HPV vaccination with a provider during the study period. This did not result in increased uptake of the vaccine, again perhaps due to the small percentage of participants who visited study clinics (5.7%).

Among parents who did not vaccinate their daughter, only 32% intended to have their daughter vaccinated in the next year. This suggests that additional levels of influence, e.g., policy (in the form of school mandates) or societal level (changing norms to accept the vaccine), should be included in future MLI studies. Another possible reason for the low uptake could have been that the intervention may require a “lag time” for effects on behaviors to be realized (43, 47).

This study has several strengths. We used a GRT design that allowed for comparisons of outcomes across similar groups. Moreover, the primary outcome, receipt of the first HPV vaccine shot, was assessed by MRR and showed that participant self-reports overestimated the receipt of the shot in both groups. Our study used MRR as the gold standard, as we have found MRR to be reliable in the past (44, 48), as have other studies (49, 50). The MLI test was developed with significant input from providers, parents, and relevant community members, and thus, it was culturally appropriate and addressed the misconceptions and questions parents and providers have about HPV, the HPV vaccine, and cervical cancer. The intervention also changed knowledge, attitudes, and beliefs about HPV and the HPV vaccine at both the provider and parent levels.

There were, however, several weaknesses. Changes in rules at participating clinics in relation to Health Insurance Portability and Accountability Act interpretation and enforcement limited our ability to recruit parents from participating clinics, as initially planned. We also were not able to “tease out” which of the levels of the MLI had an effect, an inherent weakness of MLI's, in general. More in-depth process evaluation in future studies may assist in teasing out effects of each component of MLI's. In addition, we were limited in what behavioral determinants we intervened on and did not address provider communication that has been addressed in other interventions (31, 51). Another weakness is that the sample only included parents of adolescent girls as at the time the study began, the HPV vaccine was only approved for girls. Future studies should examine how parents of adolescent boys and girls respond to interventions to improve HPV vaccine uptake. Lastly, most participants were white and living in Ohio Appalachia, limiting generalizability across other parts of the country.

In summary, an MLI designed with input from the Appalachian community increased uptake of the HPV vaccine among girls aged 9 to 17 years compared with a comparison group; however, the increase was much lower than expected. The results suggest that future studies should focus on patients within clinics where all levels of the planned MLI can be implemented. Additional levels of influence and components within each level should be tested, such as using the features of electronic medical record systems and automated reminder letters and/or phone calls, as some studies have found successful (52, 53), or interventions focused on provider communication strategies and other behavioral determinants impacting parental actions. Efforts to address low uptake of the HPV vaccine among this population need to continue.

E.D. Paskett reports receiving commercial research support from Merck. P.L. Reiter reports receiving commercial research grants from Merck Sharp & Dohme Corp. and Cervical Cancer-Free America, via an unrestricted educational grant from GlaxoSmithKline. No potential conflicts of interest were disclosed by the other authors.

Conception and design: E.D. Paskett, M.L. Pennell, C.M. Tatum, J. Peng, M.L. Katz

Development of methodology: E.D. Paskett, C.M. Tatum, P.L. Reiter, M.L. Katz

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): E.D. Paskett

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): E.D. Paskett, M.L. Pennell, P.L. Reiter, J. Peng, M.L. Katz

Writing, review, and/or revision of the manuscript: E.D. Paskett, J.L. Krok-Schoen, M.L. Pennell, P.L. Reiter, J. Peng, B.M. Bernardo, R.C. Weier, M.S. Richardson, M.L. Katz

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): E.D. Paskett, M.S. Richardson

Study supervision: E.D. Paskett, C.M. Tatum, M.S. Richardson

This study was funded by a grant from the NCI (P50 CA105632, to E.D. Paskett) and the Behavioral Measurement Shared Resource at The Ohio State University Comprehensive Cancer Center (P30CA016058, to M.A. Caligiuri).