Why Smoking Prevention Programs Sometimes Fail. Does Effectiveness Depend on Sociocultural Context and Individual Characteristics?

Evidence-based school smoking prevention programs represent an important strategy for reducing tobacco use among adolescents (1-9). However, the failure of some studies to produce demonstrable effects has led some to question whether these programs are worthwhile (10-12). Although negative findings could reflect insufficient quality of program content or delivery, evidence-based programs might sometimes fail for other reasons. Previously, we showed an interaction between the manipulated environment (program content) and the fixed environment (ethnic composition of the school) in a prevention program. A program framed in a collectivist rationale (avoid smoking for the good of your family and community) prevented smoking progression for youth in predominantly Hispanic schools, but not in ethnically heterogeneous schools (13). A similar program framed in an individualist rationale (look after yourself, don't let others coerce you to do what's bad for you) prevented smoking progression in culturally heterogeneous schools but not in predominantly Hispanic schools. This interaction indicates that program-environment fit may be one source of variation in prevention program effects.

Another potential source of variation in program effects is individual characteristics. Depression and hostility are associated with smoking (14-17) and might affect readiness to respond to smoking prevention interventions. For example, a program that addresses the issues most relevant to depressed adolescents will be most effective in samples containing large numbers of depressed adolescents. Program effects might be moderated by dispositional phenotypes and by the sociocultural context.

Neuropsychological, neurologic, and brain imaging studies indicate that prefrontal structural and functional deficits are related to antisocial behavior (18, 19). Depression and hostility involve activation of the serotonin-dopamine system and reward pathways (20, 21) and are related to cigarette smoking. Specific genes control action on these pathways, and expression is environmentally determined (22). We hypothesize that dispositional phenotypes might moderate receptivity to interventions by attentional and motivational mechanisms.

Although few studies have investigated variations in program effects according to participants' dispositional phenotypes, such variation is plausible. Programs that teach personal and social skills such as refusal of cigarette offers, interpersonal communication, and decision-making may be most effective for adolescents who lack those skills, and their lack of those skills may be a manifestation of measurable dispositional phenotypes such as depression or hostility. It is also plausible that effects of prevention curricula could vary according to the social/cultural context of the group in which they are implemented. Programs that emphasize individual achievement may be more effective in sociocultural settings where individual achievement is a valued social norm, whereas programs that emphasize group cooperation may be more effective in sociocultural settings where group cooperation is valued.

This study tests the interactions of two dispositional phenotypes (depression and hostility) with program frame of a smoking prevention curriculum (collectivist or individualist) and sociocultural context (predominantly Hispanic or culturally diverse schools). The results could make it possible to increase prevention program effectiveness by matching program characteristics to the characteristics of the target population.

Data are from a longitudinal school-based experimental trial of smoking prevention strategies in an ethnically diverse, urban population of adolescents in California (13). The trial focused on Hispanic and Asian-American students, the two largest immigrant groups to the United States. Schools were eligible to participate if they contained at least 25% Hispanic and/or at least 25% Asian-American students (some schools were both >25% Hispanic and >25% Asian-American). A total of 36 public and private school districts in and near the Los Angeles area were invited to participate. Of those, 26 agreed to participate. The 26 districts contained 150 middle schools, of which 68 agreed to participate. Of those, 33 met the criteria for ethnic distribution, geographic distance from the research center, inclusion of grades 6, 7, and 8, and ability to obtain sufficient parental consent. Nine schools participated in pilot research, and the remaining 24 schools participated in this study. The catchment areas of the participating schools had a median income of $44,590, as compared with the median household income of $47,493 throughout California in 2000 (23).

Clusters of public schools were grouped together based on similar ethnic composition and geographic location. Schools within each cluster were randomly selected to receive one of the two curricula or a wait-list control. Because the Catholic schools were geographically diverse, assignments were done using a principal components analytic method to assign generally dissimilar schools to conditions (24, 25).

Descriptions of the study and consent forms were sent to the parents/guardians of all 6th-grade students. Repeated attempts were made over a 4-week period to obtain either active consent or active refusal from parents who did not respond to the initial request for consent. Students were eligible to participate if a parent/guardian provided active written consent and the student provided active written assent. Students who provided neither active parental consent nor active parental refusal participated in curriculum activities but did not provide identifiable data. The procedure was approved by the university's Institutional Review Board.

Baseline Survey. Students completed paper-and-pencil surveys in their classrooms. Trained data collectors distributed the surveys. The classroom teacher was not present during survey administration. The surveys were identified by code numbers so that the students could be followed longitudinally, but the students' names were not written on the surveys.

Measures. Past-month smoking (smoked on at least 1 of the past 30 days) was the dichotomous outcome measure. Analyses of smoking initiation between 6th and 8th grade included all respondents who did not report past-month smoking in 6th grade.

Age, gender, ethnicity, generation in the United States, and Socioeconomic status (SES) were included as demographic covariates. Ethnicity was coded as Asian, Hispanic, White, or Other/Multiethnic. African-Americans were included in the Other/Multiethnic category because they represented only 1% of the sample. Generation was coded as 1st/2nd if the student or either parent were born outside the United States, and 3rd+ if neither the student nor either parent was born outside the United States. We used a standardized average of several indicators as a measure of SES. These included parents' education, the number of rooms per resident in the home (a measure of overcrowding, which is correlated with poverty; ref. 26), the median annual household income in the student's zip code (from 2000 U.S. Census data), and dichotomous indicators of having a computer and Internet access at home. The Cronbach's α of this scale was 0.82.

As recommended by Murray (27), variables were identified as potential covariates based on their associations with smoking in previous research. Covariates were included if they were significantly associated with smoking behavior and/or experimental condition, and if their inclusion in the model altered the program effect by at least 10%. The variables considered as covariates included peer influence (best friends' smoking, prevalence estimate of peers' smoking, and cigarette offers in past month); parental variables (parents' smoking, parental monitoring, and communication with parents); and psychological/cognitive variables (cognitive susceptibility to smoking and perceived access to cigarettes). These psychosocial constructs were assessed with scales that had been validated in previous adolescent smoking surveys (28, 29). Consistent with previous research (1, 2, 28, 29), all of these variables were significantly associated with lifetime smoking prevalence. All except cigarette offers altered the program effect by at least 10% and were included in the models.

Depression and hostility were examined as potential moderators of program effects. Depression was measured with a five-item version of the CES-D (30), which correlates 0.89 with the full scale and has a Cronbach's α of 0.82. Hostility was assessed with a four-item version of the Irritability subscale of the Buss-Durkee Hostility Inventory (31), which was correlated (0.67) with the Anger subscale of the newer Buss and Perry (32) Aggression Questionnaire and had a Cronbach's α of 0.66. The wording of the items is available by request from the authors.

To control for pretest variation across schools, school-level measures of 6th-grade smoking prevalence and ethnic composition were included in the model as level 2 covariates. School smoking prevalence (proportion of the students who reported ever-smoking in 6th grade) was included as a covariate because baseline smoking prevalence was found to differ slightly across the three groups, despite random assignment. School ethnic composition was included in the model to evaluate the effects of the school ethnic context on program effects. The proportion of respondents in each school who self-identified as Hispanic was used to represent school-level ethnic composition. Because this variable was correlated (−0.95) with the proportion of Asian students and correlated (−0.74) with the proportion of White students, this variable can be interpreted as a high concentration of Hispanic students or a low concentration of Asian and White students.

Two programs were created for this study, each consisting of 8 classroom activities conducted by a trained health educator (13). The theoretical content of both programs was based on social influences models of prevention, which have been effective in preventing smoking in multiple studies (1, 33-35). The multicultural collectivist-framed program was designed to include cultural values from Hispanic and Asian cultures. Emphasis was placed on avoiding cigarette smoking for the good of one's family and community (collectivist objectives). Lessons emphasized not smoking in the context of the Hispanic value of familism (interdependence of family members), the Asian value of filial piety (respect for ancestors), the Asian value of saving face, and the Hispanic value of simpatia (harmonious interpersonal relations). Other lessons addressed acculturative stress, discrimination, and family conflicts that can occur during the acculturation process.

The individualist-framed program was an updated version of Project SMART (33) with heightened individualist content. It presented the same general theoretical perspective (e.g., physical and social consequences of smoking, social norms, refusal skills, general personal/social skills), but without references to specific family, community, and cultural values. Emphasis was on “looking after yourself.” In the control condition, schools did not receive a smoking prevention program, so the classroom teachers implemented their usual curricula. Because California schools are required to teach antitobacco education in grades 4 to 8, the usual curricula should have included lessons about tobacco. However, in a recent study of a statewide sample of California schools (28), only half of the teachers reported teaching antitobacco lessons, and only half of the students recalled receiving antitobacco lessons.

Students completed a follow-up survey in 8th grade, using the same procedure used in the baseline survey.

Pretest Equivalence of Groups. χ² and ANOVA analyses were conducted to assess the equivalence of the three groups before the intervention. The groups were compared on age, gender, ethnicity, generation in the United States, SES, and smoking prevalence.

Attrition Analysis. Attrition from longitudinal school-based studies typically is higher among students who engage in risky behaviors such as smoking (36, 37). If the attrition patterns differ between experimental conditions, estimates of program effects can be biased. A logistic regression model was used to evaluate differential attrition. Attrition (coded as 0/1) was regressed on the predictor variables of experimental condition, baseline smoking status, and the experimental condition × smoking interaction. A significant experimental condition × smoking interaction would indicate differential attrition (i.e., that the proportion of smokers versus nonsmokers who dropped out differed between experimental conditions).

Exploratory Analysis of Three-Way Interactions in Program Effects. Generalized Linear Mixed Models (GLIMMIX), were used to assess the three-way interactions between school environment, dispositional phenotypes, and intervention. Because the data contain a hierarchical structure (students nested within schools), individual-level models that do not control for intraclass correlation within schools might result in artificially small confidence intervals (38). The SAS GLIMMIX procedure, with school included as a random effect variable, was used to correct for the intraclass correlation. Past-month smoking initiation from 6th to 8th grade was the outcome variable. The independent variable was experimental condition, which was coded as a three-level class variable (collectivist, individualist, or control).

Because the dispositional phenotype measures are nearly continuous scales (a mean of several categorical items, ranging from 1 to 4), we explored both linear and nonlinear interactions. The method to explore the nonlinear three-way interaction is an intensive computation method adopted from the technique of data mining (39). When exploring the nonlinear three-way interaction, we did not use any prior knowledge about the cut-point of the program effect across the phenotype scales. Each candidate cut-point was a binary transformation of the linear scale of the phenotype. The scale ranged from 1 to 4, and the distance from one candidate cut-point to the next is 0.2; therefore, there were 16 candidate cut-points for each phenotype. At each candidate cut-point, the algorithm would compute the estimate (β) of the program effect within four different strata: (a) students in mixed-culture schools with low scores on the phenotype scale; (b) students in mixed-culture schools with high scores on the phenotype scale; (c) students in Latino schools with low scores on the phenotype scale; and (d) students in Latino schools with high scores on the phenotype scale.

A plot with four curves was generated to visualize the disparity of estimates (β) across the four strata as the cut-point moved across the phenotype scale. This pattern of disparity change represents the nonlinear three-way interaction. A statistical test for the three-way interaction at the cut-point with a large estimate disparity and “optimal cell size” was also reported to provide supportive evidence. A cell with optimal cell size must meet the following two conditions: (a) the sample size must be relatively large and balanced across strata, and (b) the number of smokers must be relatively large and balanced across strata.

Within the 24 schools, all 6th grade students (N = 4,427) were invited to participate. Of those students, 3,358 (76%) provided active parental consent, 420 (9%) provided active parental refusal, and 649 (15%) provided no parental response. No data were collected from students who provided parental refusal. Some limited anonymous demographic data were collected from students who provided no parental response under an Institutional Review Board–approved implied consent protocol. A comparison of the students with active parental consent and the students with no parental response (40) revealed that the students with no parental response were significantly more likely to be male, African-American, smokers, and have lower grade-point averages.

Of the 3,358 students who provided active consent, 168 (5%) were absent from school on the day of the baseline survey or elected not to participate, and 33 (1%) had extensive missing data. The analytic sample consisted of the 3,157 students who provided active consent and completed the baseline survey.

The demographic characteristics of the 3,157 students are shown in Table 1. The groups did not differ significantly on age, gender, or ethnicity. However, they did differ significantly on generation in the United States, median household income, lifetime smoking status, and past 30-day smoking status.

Of the 3,157 students surveyed in 6th grade, 2,412 (76.4%) were surveyed again in 8th grade. The attrition rates were 22.9% in the multicultural collectivist intervention group, 24.8% in the individualist intervention group, and 23.3% in the control group. There was no differential attrition rate of smokers versus nonsmokers associated with experimental condition (13), although attrition did differ significantly by smoking status; 35.2% of the 6th grade ever-smokers were lost to follow-up, compared with 22.4% of the 6th grade never-smokers (Wald X² = 24.25, P <.0001). Of the 2,412 students surveyed in both waves, 2,200 baseline nonsmokers were included in these analyses.

As reported previously, (13) there was a significant main effect for the collectivist-framed program. Nonsmokers who received the collectivist program were significantly less likely to try smoking between 6th and 8th grade, relative to those in the control condition [odds ratio (OR), 0.77; 95% confidence interval (95% CI), 0.61, 0.98]. The individualist program was no more effective than the control (OR, 0.97; 95% CI, 0.75, 1.26). The collectivist-frame program was associated with a significantly lower risk of past-month smoking in 8th grade (OR, 0.40; 95% CI, 0.18, 0.90), whereas the individualist framed program was not significantly more effective than the control. Hence, one program might have been judged by main effect as “successful” and one might have been judged as “unsuccessful.”

Interaction terms were added to the models to determine whether the program effects were moderated by dispositional phenotype and school composition. Separate analyses were done for depression and hostility as moderators. Linear models revealed a significant three-way interaction for depression (P < 0.05 for both programs) and a marginally significant three-way interaction for hostility (P = 0.11 for individualist program, P = 0.16 for collectivist program).

Figure 1 shows the nonlinear interaction effects with depression and reveals significant three-way (program × school composition × depression) interactions for both the collectivist (P = 0.016) and the individualist (P = 0.013) programs at the same optimal cut-point of 2.8. Figure 1 presents the interaction effects with hostility in the model and shows that the three-way interaction was significant for the individualist program (P = 0.01) and marginally significant for the collectivist program (P = 0.057). The optimal cut-points were different for the two intervention programs (3.4 for individualist and 3.6 for collectivist).

Negative βs in the figure indicate protective effects of the intervention program. The program was protective for more depressed and hostile students in the predominantly Hispanic schools. However, it was protective for less depressed and hostile students in the culturally mixed schools. In other words, the impact of dispositional phenotypes on program effects differed according to the cultural context of the school.

Odds ratios for program participants' progressing to 30-day smoking between 6th and 8th grade are presented by cell for the three-way interactions in Tables 2 and 3 (depression and hostility, respectively). Although the sample sizes are too small to test hypotheses by cell, most ORs are <1, consistent with a positive program effect, except among those in the culturally mixed schools with high depression and high hostility scores.

Although school-based smoking prevention programs have been effective in numerous studies, other studies have failed to show prevention effects, raising the question of which combinations of program characteristics, personal characteristics, and social environments are most conducive to effective prevention. In this study, three-way linear and nonlinear interactions were found for prevention programs by individual dispositional phenotypes and school composition. These findings support the hypothesis that smoking prevention effects depend both on characteristics of the individual and on the sociocultural environmental context. We found (13) that both a collectivist and an individualist framed social influences program were sometimes effective in preventing progression in cigarette smoking among adolescents and sometimes not, the success of each dependent on different sociocultural contexts. The current findings show that program effectiveness can depend on dispositional characteristics of the person as well, and prevention programs can involve complex interactions among disposition, sociocultural context, and program content. The impact of a specific prevention program in a given social context depends on dispositional phenotype, and the impact of a program on a person with specific dispositional characteristics depends on the social context.

We first tested linear interactions, then examined whether the interactions were truly linear by using a nonlinear exploratory method. The pattern of the three-way interaction effect was illustrated by a plot with four curves. The differences across the curves indicate the degree of the interaction. Finally, we chose one optimal cut-point to report the statistical test of the interaction as well as the four strata-specific ORs. A significant interaction at the chosen cut-point provides additional evidence of the overall three-way interaction, although the choice of cut-point was arbitrary.

How do we interpret these interactions relative to the design of better smoking prevention interventions? The collectivist program was effective in preventing smoking progression use among youth with higher levels of depression or hostility who attended primarily Hispanic schools. But the individualist-framed program seems to have been about as effective as the collectivist-framed program among those with higher levels of depression and hostility. Overall, the individualist framed program was more effective with those scoring low on depression and hostility in the culturally mixed schools. These differential effects suggest that the selection of school-based smoking prevention curricula should be guided by an analysis of the sociocultural characteristics of the school as well as the psychological characteristics of the students. If future research finds these psychological characteristics to have easily identifiable genetic markers, it may be possible to tailor intervention strategies to students' genotypes. Of course, the ethical implications of assigning students to interventions based on their genotypes must be considered carefully.

Depression and hostility have been linked to dopaminergic activity in the accumbens and its related brain reward circuitry (41). Persons exhibiting these phenotypes have been described as suffering from a “reward deficiency syndrome,” (42) which involves depressed mood and derivation of less positive affect from rewards. Perhaps to compensate for this reward deficiency, they engage in self-stimulating activities including substance use (41, 43). Reward deficiency syndrome is hypothesized to relate to the reward-mediating brain circuits, which play a unique role during adolescence (41, 43). Dopamine shifts balance toward cortical predominance, resulting in lower intracellular levels of dopamine in the mesolimbic regions during adolescence relative to during childhood and adulthood. Low baseline dopamine may cause boredom and dissatisfaction with moderate to low rewards and may sensitize the mesolimbic reward system in such a way that it becomes hyperactive in reaction to stronger challenges (44-47). The consistent pattern of results for prevention programs of both types in predominantly Hispanic schools indicates that a collectivist program might be especially effective for exhibiting hypoarousal of the dopaminergic system within a relatively homogeneous cultural context.

Perhaps those with suppressed (low-baseline dopamine) arousal states are particularly attentive to novel programs, which might account for the effectiveness of both programs observed among students high in depression and hostility. The curricula were designed to address challenging social situations including social acceptance/rejection, perhaps making them particularly stimulating for those students who suffered from depression, hostility, and doubts about their social competence. In the challenge of a culturally mixed context where social support and reinforcement might be more unpredictable (or an individualistic context where there might be more motivation to stand out from the crowd), programs seem most effective for those with higher levels of baseline arousal (low depression and low hostility). This explanation is speculative, but it is plausible that the roots of the dispositional component of the interaction might be found in underlying neural mechanisms driven by gene environment interactions. Future research should explore these underlying neurocognitive mechanisms, as well as other plausible mechanisms such as γ-aminobutyric acid, endogenous opiods, and the endocannabinoid system that may be involved in smoking initiation and nicotine dependence.

We should not expect prevention program effects to be consistent across social environments or across individuals. Program × individual characteristic × environmental interactions are not uncommon in clinical medicine (48). Social influences and individual characteristics can greatly determine how a given intervention is experienced, interpreted, and acted upon.

Other school-based substance use prevention programs have shown significant main effects across ethnic groups, arguing for universal rather than tailored approaches to prevention (49-54). However, no other study has examined interactions among the cultural context, the curriculum content, and the students' dispositional characteristics in predicting program effects. Future prevention studies should go beyond asking whether programs work and instead ask under what conditions, and for which adolescents, they work. A program that shows robust effects at the population level still may be more effective in certain social contexts than in others and/or more effective for certain individuals than for others.

The nonlinear three-way interactions resulted from relatively new exploratory analytic methods. Although these analyses are exploratory, results from more traditional linear interactions were generally consistent. It is also possible that exploration increased the likelihood of chance findings, but the consistent patterns across cut-points shown in Figure 1 argue against that explanation. Nevertheless, confirmatory replication is important. These results are based on adolescents' self-reports of their smoking behavior, which are generally accurate under confidential survey conditions (55, 56). These findings may not generalize to students who did not obtain parental consent or students who were lost to attrition, two groups that were characterized by their high smoking prevalence. Additional research is needed to understand intervention effects among these high-risk students. Because this study did not genotype students or measure their neurotransmitter activity, additional research will be needed to show the biological mechanisms suggested by the pattern of results. Studies of smoking cessation among adults (57) have not generally found depression to be a good predictor of long-term cessation success. Additional research on both prevention and cessation is needed to understand the complex interactions among depression, intervention content, and social context in predicting the success of interventions.

To dismiss prevention program strategies as failed or proclaim a single prevention approach as proven is misguided. Prevention programs should not be expected to act universally across all environmental settings and individuals. Prevention research should address which interventions produce which specific results in which people under which environmental circumstances. A transdisciplinary approach that addresses these questions from the perspectives of multiple levels of science holds the best promise for an integrated understanding of what motivates risk behavior and how to promote healthy outcomes in all segments of the population.

The authors thank Greg Molina, Kari-Lyn Sakuma, and the administrators and teachers of the participating school districts.