Abstract ED04-05: Research and statistical design issues in interventions to reduce cancer health disparities

This workshop will cover research and statistical design issues in community intervention trials, with examples from an on-going NCI-funded program project using three different study designs to test three different interventions to promote hepatitis B testing in three different Asian American populations.

Development of an appropriate study design is critical to the evaluation of an intervention. In particular, it is necessary to develop testable hypotheses, define measurable outcomes, identify the target population, and determine how study participants will be selected from the population. When a controlled trial is planned, it is also necessary to determine how participants will be allocated to the intervention and control arms of the study. An additional consideration is whether longitudinal or cross-sectional samples will be used to assess changes over time. Finally, the data analysis methods must take the design features into account. Throughout the entire planning process community participation is essential to ensure that an appropriate study design is chosen and implemented.

Outcome: The primary outcome should be an indicator of health disparity that the intervention can affect within the study timeframe. If the aim of a 2–3 year intervention is to reduce liver cancer deaths by promoting hepatitis B testing, receipt of a hepatitis B test is a more appropriate primary outcome than the liver cancer mortality rate.

Target Population: A subset of the population in which the health disparity exists must be targeted for the intervention study. The age and gender of the target population are determined by the primary outcome; the target geographic area is determined by the distribution of the population. Hepatitis B serological testing is indicated for adults who have a significant chance of being hepatitis B positive and at risk of transmitting the virus to others. Hepatitis B and liver cancer affect both males and females, so it is appropriate for adults of either gender to be serologically tested. Asian Americans, who are disproportionately affected by hepatitis B and liver cancer, have relatively largeWest Coast populations, so California is an appropriate place in which to address this disparity.

Type of Trial: Different interventions in different communities require different types of controlled trials. Media interventions are designed to impact an entire community and may cover a large geographic area. For this reason, control areas must be geographically distant from intervention areas in order to avoid contamination. In addition, the geographic distribution of the target population determines the feasibility of randomizing communities to the study arms. As well as raising concerns about diffusion of the intervention to geographically adjacent areas, randomization of a very small number of communities is likely to result in unbalanced treatment groups (Atienza and King, 2002). As a result of these constraints, a quasi-experimental design may be chosen for a media intervention in an ethnic population whose geographic distribution in the U.S. is limited to a few areas. In contrast, individual and small group interventions are ideally suited for randomized trials, although care must be taken to define the randomization unit and deliver the intervention so as to avoid contamination of control groups.

Contamination: Several methods can be employed to minimize contamination of the study arms, including geographic separation of intervention and control groups, designing control activities that are unrelated to intervention themes; and training and monitoring interviewers and individuals delivering the intervention. Nevertheless, contamination may occur; therefore it is necessary to monitor community media and include questions on exposure to the intervention in post-test surveys.

Type of Sample: As with the type of trial, the type of intervention determines whether to select a cross-sectional or longitudinal sample. For interventions affecting the community as whole, a cross-sectional design is appropriate for measuring community-wide effects. Although cohort designs increase statistical power when baseline and follow-up measurements on an individual are positively correlated, loss to follow-up or changes in the community over time can introduce bias (Atienza and King, 2002; Diehr et al., 1995). However, for interventions delivered to individuals or small groups a longitudinal design is preferred.

Sampling Frame: The way in which study participants are selected depends upon characteristics of the population and the intervention. The evaluation of community-wide interventions requires random samples of members of the community. Although it is desirable for every member of the community to be sampled with positive probability, a list of telephone numbers of persons with Vietnamese surnames must be used in order to reach households in this ethnic population without prohibitive expense. However, a virtually complete listing of Hmong families referred to community organizations and persons with Hmong surnames can be compiled in certain geographic areas. In a study taking place in Korean churches, the site of the intervention (determined in part by the characteristics of the population) indicates that churches will be sampled from appropriate lists.

Survey Medium: The method of survey administration depends both on characteristics of the population, including experience and comfort with surveys, and on the site of data collection. Previous experience indicates that telephone surveys work well in the Vietnamese population, but it may be best to administer in-person surveys to Hmong participants, who are relative newcomers to the U.S. Although in-person surveys can help establish trust at the beginning of a study and may be conveniently administered at the intervention site, at the end of the study a less costly telephone survey is likely to be acceptable to participants.

Hypothesis: The primary evaluation of an intervention is performed by testing a statistical hypothesis. Therefore, it is essential to ensure that the hypothesis is testable and stated appropriately. The statement of the hypothesis test must be consistent with the study design. A study using cross-sectional surveys can test the hypothesis that the proportion serologically tested increases more from pre- to post-test in the intervention group than in the control group; a longitudinal cohort study can test the hypothesis that, among individuals not serologically tested at pre-test, the proportion who are serologically tested at post-test will be greater in the intervention group than in the control group.

Effect Size: Determination of an appropriate effect size depends on the method of intervention, the desired outcome, and the population in which it is delivered. Community-wide interventions typically have smaller effects than individual interventions (Thompson et al., 2003). A larger effect size may be selected for a lay health worker intervention, which is relatively intense, compared to a media campaign or group educational session.

Power and Sample Size: Key to the evaluation of an intervention by means of hypothesis testing is a sample that is large enough to detect an achievable, clinically significant effect size with adequate power. The achievable effect is usually estimated based on reports of the effects of similar interventions on similar populations. The power to detect the effect depends, of course, on the sample size and level of the test. In addition, power may depend on various other factors, including the pre-test level of the outcome, the presence or absence of clustering, independence or dependence of pre- and post-test samples, the study dropout rate, and the anticipated gain in the control group. A group randomized design requires that intra-class correlation among group members be taken into account. A longitudinal design means that pre- and post-test observations are correlated, whereas cross-sectional samples are assumed to be independent. Sample sizes in cohort studies must be chosen to ensure an adequate number of participants at post-test after anticipated dropouts. Finally, gain in the control arm, due to temporal trends and/or study participation of the individual or community, may be an issue.

Data Analysis: Differences in study design mandate differences in statistical methods and models. As noted by Murray (1997), “The proper unit of analysis is determined entirely by the design of the study.” Murray also emphasizes the importance of correctly accounting for all sources of variation associated with the study design. In particular, when analyzing data from an intervention trial, it is necessary to account for any clustering due to the sampling process or the allocation of groups to study arms. If a study samples groups of respondents and administers a group randomized trial, it is necessary for the analyses to use methods for correlated data, such as generalized estimating equations (GEE). Analysis of data from a quasi-experimental design must also account for any clustering in the sampling method. If a study randomly selects individuals, who are then randomly assigned to the study arms, observations are independent at pre- or post-test, and standard methods for individual-level data will suffice. However, analyses of longitudinal data using both pre- and post-test observations on the same individuals must account for intra-individual correlation.

These issues can pose challenges in the design of an intervention trial; nevertheless, it is essential to consider all aspects of the study design carefully in order to conduct a successful intervention trial and obtain valid results.