Assessing Consumer Responses to Potential Reduced-Exposure Tobacco Products: A Review of Tobacco Industry and Independent Research Methods

Background: Internal tobacco industry documents and the mainstream literature are reviewed to identify methods and measures for evaluating tobacco consumer response. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing consumer response, and consider how these methods might be applied to evaluate potentially reduced exposure tobacco products and new products.

Methods: Internal industry research reviewed included published articles, manuscript drafts, presentations, protocols, and instruments relating to consumer response measures were identified and analyzed. Peer-reviewed research was identified using PubMed and Scopus.

Results: Industry research on consumer response focuses on product development and marketing. To develop and refine new products, the tobacco industry has developed notable strategies for assessing consumers' sensory and subjective responses to product design characteristics. Independent research is often conducted to gauge the likelihood of future product adoption by measuring consumers' risk perceptions, responses to product, and product acceptability.

Conclusions: A model that conceptualizes consumer response as comprising the separate, but interacting, domains of product perceptions and response to product is outlined. Industry and independent research supports the dual domain model and provides a wide range of methods for assessment of the construct components of consumer response. Further research is needed to validate consumer response constructs, determine the relationship between consumer response and tobacco user behavior, and improve reliability of consumer response measures. Scientifically rigorous consumer response assessment methods will provide a needed empirical basis for future regulation of potentially reduced-exposure tobacco products and new products, to counteract tobacco industry influence on consumers, and enhance the public health. (Cancer Epidemiol Biomarkers Prev 2009;18(12):3225–40)

The commercial success of tobacco products depends largely on cigarette manufacturers designing and promoting products to ensure that they retain their existing customers, attract the customers of their competitors, and replenish the market with new smokers who choose those brands. An extensive body of evidence derived from analysis of internal tobacco industry documents has shown the use of sophisticated and subtle design variations by the tobacco industry to promote product acceptance among tobacco consumers. The design features of tobacco products may prompt sensory and other subjective responses that are important in the determination of taste and smoking satisfaction (1, 2), psychological reward (3), and craving diminution (4, 5): all of which may contribute to consumer acceptance. Tobacco companies have also strived to communicate the advantages to their consumers of new design features through new packaging and branding, sophisticated marketing messages, and even the form of the tobacco product itself (6).

Advertising and other messaging strategies are used by the tobacco industry to convey product information, and so positively influence consumer perceptions. Advertising and marketing of lower tar cigarette brands over the past 50 years has emphasized “satisfaction” (taste and sensory effects) and “pleasure” (nicotine effects), while simultaneously communicating reduced cigarette strength and, by implication, lower product risk (1). The misguided belief that a low-tar cigarette is less risky is undoubtedly a deterrent to smoking cessation, because health concerns are an important factor in motivating smoking cessation (7). A case in point is the Marlboro Lights brand. Marlboro Lights was one of first brands to use the “Lights” descriptor to communicate its lower tar (and, by implication, lowered harm) feature. The Marlboro Lights pack achieves a lighter look with a white pack and a gold chevron instead of the distinctive red chevron found on Marlboro Red. Consumer research conducted by Philip Morris has shown that smokers perceived different tastes when they smoked identical cigarettes in red versus gold packs (8). Marlboro Lights advertising also used more white space, less action, and muted colors: all intended to reinforce the lower tar delivery message of the brand. The cigarette itself also has been designed to reinforce the perception of a safer product, with the use of white tipping paper on the filter tip, in contrast to the cork colored paper used on most full-flavored cigarettes (8). The use of chemical additives and milder tobacco blends in low-tar cigarettes also helps reinforce the consumer's perception that a “light” cigarette is safer, when in fact it has been designed in a way that simply enhances the smoker's ability to puff more freely on the cigarette (9, 10). Despite the intensive efforts taken by the industry to implicitly communicate lowered harm, internal industry studies show that lower machine yield cigarettes have greater toxicity compared with higher yield (full flavor) cigarettes (8).

The proliferation of novel tobacco products that purport to be safer than conventional products has created significant challenges for the public health community. So-called potential reduced exposure tobacco products (PREP) use novel design features or ingredients that are intended to reduce exposure to tobacco-related toxicants, and thus lower disease risk for the consumer (11). The Institute of Medicine has stated the importance of understanding how consumers perceive and respond to PREPs to avoid misperceptions among consumers, such as those that resulted from the misleading marketing of so-called low-tar/light cigarettes (11). There are well-founded concerns that consumers could be misled by implicit or explicit marketing messages that communicate health benefits of PREPs, since official standards for evaluating claims for tobacco products have not yet been developed. Explicit marketing messages and claims made by tobacco manufacturers for reduced exposure tobacco products include messaging and claims about reduced exposure to harmful constituents (12). Not all reduced risk claims are explicit: implicit claims can be communicated through product descriptors, including the colors and graphics used in product packaging, together with qualifier terms such as “ultra” and “lights.”

Most PREPs introduced in the last decade are no longer in the market, in part because products designed to reduce exposure tend to have low acceptability among consumers. Poor sensory and subjective characteristics of PREPs may have contributed to their low consumer acceptance, despite a substantial research and development program by major tobacco companies to address this (i.e., Marlboro UltraSmooth; ref. 12). The modification of tobacco products to reduce harmful toxicants can affect design features that facilitate positive consumer responses (CR). For example, activated carbon is sometimes added to cigarette filters to reduce harmful gas phase constituents, but this also reduces or alters flavor characteristics (13). Similarly, products designed to heat tobacco electrically may be too dissimilar to a cigarette to promote broad consumer interest. However, new or future PREPs may appeal to consumers and could receive broad acceptance and corresponding market share. In the new tobacco regulatory environment provided under the Family Smoking Prevention and Tobacco Control Act of 2009 (14), the Food and Drug Administration will have the capacity to restrict or ban product design features, including flavorants and additives designed to enhance consumer acceptability, and product messaging that promotes perceptions of lowered risk. Thus, there is a great need for scientifically validated methods to allow rapid and early assessment of CRs to PREPs and other new tobacco products, and to provide an empirical basis for this new regulatory framework.

CR to tobacco products is defined as a set of subjective and behavioral responses that convey information, affect behavior and likelihood of long-term product use by the consumer, and his or her future intentions for product adoption (see Fig. 1). The Theory of Reasoned Action (15, 16) provides a basis for understanding the link between attitudes and behavior, and may provide a heuristic for conceptualizing of CRs to tobacco products. Theory of Reasoned Action posits that the likelihood of a behavioral outcome is influenced by an individual's attitude combined with subjective norms about that behavior. Consistent with this theory, CR might be seen as a complex set of attitudes and norms that influence the likelihood of product trial (the behavioral response).

In the model proposed here, CRs are represented by two broad categories: perception of the product and response to the product. Product perceptions encompass subjective responses to product information, and include individual perceptions of risk, attitudes and beliefs about the product, social acceptability, and outcome expectancies such as satisfactory nicotine and sensory effects. Responses to product encompass acute nicotine and sensory effects (including drug reward; relief of craving and withdrawal; taste, harshness/smoothness, and other sensory characteristics), conditioned motivational responses to sensory cues, afffective responses, and product personal acceptability. Also represented in Fig. 1 are two general domains within which tobacco manufacturers seek to positively influence CR: product advertising and marketing (“messaging”), and product design. In general, tobacco industry messaging is aimed at directly influencing product perceptions, whereas design manipulations are used to influence product responses. A hypothetical link exists between product perceptions and response, such that each exist in a mutual feedback loop. The combined influence of product perceptions and product response determine the likelihood of initial and subsequent use. Also assumed in this model are pre-existing personal factors, such as degree of tobacco dependence, and social and personal norms, in the context of which industry messaging and product design must operate to be effective. The tobacco industry has designed and marketed products to appeal to target groups with specific characteristics such as age, gender, ethnicity/race, and degree of tobacco dependence (2). This model thus allows a basis not only for predicting future use of the product, but for understanding relationships between the range of factors that contribute to positive CRs.

This article reviews both internal tobacco industry documents and the mainstream literature to identify methods and measures for evaluating tobacco CR. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing CR, and consider how these methods might be applied to evaluate new products, including PREPs. Although CR research has been conducted with conventional products, the present review focuses on products designed to reduce exposure, owing to the special challenges associated with assessing their potential for consumer acceptance and appropriately communicating their risks. Strategies used to evaluate PREPs have largely evolved from those used to evaluate conventional products, thus making a broader review largely redundant. For reasons of economy, the current review therefore will focus on PREPs.

Internal tobacco industry documents were identified using databases at Tobacco Documents Online,⁶

the Legacy Tobacco Documents Library,⁷ and the British American Tobacco Documents Archive.⁸ A snowball sampling design was used for text-based and index searches, with an initial set of keywords and phrases (i.e., consumer perception, reduced harm, test market), which resulted in the development of hundreds of further search terms. Search terms may have been combined for more targeted searches, initially yielding large numbers of documents (see examples in Table 1). These results were visually skimmed, and duplicates and obviously irrelevant documents (i.e., deposition transcripts, popular newspaper and magazine articles, smoke chemistry data) were discarded. The remaining documents were carefully reviewed for relevant methods and measures of CR.

Relevant documents included (a) product development efforts to address issues of consumer perception of PREPs, and (b) market research and surveillance conducted following the launch of novel tobacco products. Of the ∼8,000,000 documents available in the archives, the final sample was narrowed to include ∼970 relevant documents dating from 1980 to present. A total of 60 documents contained sufficiently detailed descriptions of industry methods, instruments, and measures to be used as citations in this article.

A previous literature review conducted by Pederson and Nelson (17) summarized the findings of studies on consumer perceptions of PREPs. In contrast, this article focused on cataloguing the methods used by independent researchers to measure CR, including responses to product, and compared them with research strategies used by the tobacco industry. The published literature within PubMed, ranging from January 1995 to August 2008 was searched using keywords such as reduced exposure, reduced harm, novel tobacco products, sensory, smoker perception, and subjective response.

Selected manuscripts examined smokers' knowledge, attitudes, beliefs and intentions about, and subjective responses to, PREPs. The criteria used by Pederson and Nelson (17) were used to define PREPs; specifically, tobacco products that resemble conventional cigarettes but do not necessarily require both tobacco and nicotine and are heated or burned, or is some other oral nicotine-containing product made by a tobacco company (i.e., electrically heated cigarettes, snus, denicotinized cigarettes, charcoal-filtered cigarettes).

Manufacturers of PREPs are concerned with consumer adoption of their proposed product, and the first phases of product development or marketing efforts include measuring perceptions of the product “concept” using a variety of methods. The product concept can be assessed independently of actual product use, or combined in a product-concept test, where subjects view and rate the concept, and sample and rate the product (18). Methods used by the tobacco industry for PREP concept and message testing include focus groups, public intercept surveys, and random digit dial surveys. Independent researchers have applied all of these methods for postmarket PREP assessment, with the inclusion of Web surveys (Table 2). The following describes methods used by the tobacco industry, at premarket and postmarket stages, to assess consumers' perceptions of product and product concepts, and methods used by independent researchers, primarily at the postmarket stage, to assess perceptions of product.

Focus groups are commonly used in market research to assess consumer reactions to new ideas or products, and social scientists have increasingly applied the method to gain insight into specific research questions and frame formative issues for further study (19). Typically, focus groups consist of groups of 6 to 10 respondents screened to represent the target market of interest, gathered in the same room, and guided by a moderator through a discussion on topics of interest (20, 21).

Focus group research conducted by the industry has included both product concept–naïve participants as well as participants who have previously used a PREP and can offer feedback about their experience. RJ Reynolds (22) conducted focus groups among current users of Premier in 1990, to determine which features of the PREP they found most appealing and to catalogue reasons for continued use. Similar sessions were held by the company in test markets for Eclipse in 1996, where perceived benefits and drawbacks of the product concept and actual product were discussed among smokers (23).

Among independent researchers, focus groups have been used to assess motivations and use patterns of PREPs, and consumer's perceptions of PREP messaging. Carabello et al. (24) asked smokers ages 30 to 50 years questions about how they learned about PREPs, their reasons for trying in the first place, which products they had tried, their perceptions of the product at first trial, their reasons for continuing or discontinuing use, and whether they would recommend PREPs to others. Other studies have used focus groups to assess consumer's perceptions of PREP messaging by showing advertisements for products currently on the market to participants. O'Hegarty et al. (25) used recruitment criteria described by Carabello et al. (24) in cities where Eclipse was in test market. Following a discussion about PREP use, participants were shown a series of print advertisements and promotional materials for specific products. A discussion about the participants' reactions followed, including their impressions of the intended message of the advertisement and its influence on their interest in PREPs.

Both tobacco industry and independent researchers recruit and/or classify focus group subjects by demographic or social background, smoking history, or an individual's knowledge, attitudes, and beliefs regarding smoking. Some of these characteristics may be used as inclusion criteria, but the industry has used them also to group subjects in the analysis of data collected in consumer perception studies. Tobacco companies maintain extensive databases of current and former customers, to expedite recruitment of participants in PREP consumer perception studies. In studies of Eclipse, RJ Reynolds contacted smokers by telephone or mail to invite them to participate in focus groups and surveys; samples of the test product were sometimes sent beforehand for the consumer to try (23). In the independent literature, Hughes et al. (26) utilized a similar resource in their investigation of Eclipse; the researchers contacted RJ Reynolds for access to the database of smokers who had ever purchased Eclipse, as well as those who had received Eclipse promotional materials. The company provided the names and addresses to a third party mailing service, which sent study recruitment materials to the smokers, identifying the study investigators as independent of the manufacturer.

Public intercept surveys use a convenience sampling strategy to administer a structured questionnaire to participants who are recruited at a public place (i.e., a shopping mall). Participants may also be shown examples of products or concepts to evaluate (25, 27-30). Questionnaires are written according to the objectives of the study, are administered in a structured interview format, and may also function as a screening tool to recruit participants for follow-up surveys or tracking studies on tobacco use by the company (31-33). Public intercept surveys for tobacco products may face unique limitations that compromise the representativeness of the final sample; for example, in 1997, RJ Reynolds discontinued its use of public intercept research for product Eclipse because of a high rate of refusal among mall patrons (34). It is unclear whether the high rate of refusal was inherent to the method, or whether it was due to the fact that consumers were resistant to participating in a tobacco industry study.

Independent researchers have used intercept surveys to investigate the nature and impact of implicit and explicit health messages contained in PREP promotional materials. Hamilton and colleagues (35) asked adult smokers to review a single advertisement for each type of product (one of two print advertisements for regular cigarettes, light cigarettes, and PREPs). Health messages, which promoted the health-related benefits of using PREPs instead of regular cigarettes, were distinguished from health risk warnings (35). In addition to Surgeon General's warnings (which were contained in advertisements for all types of tobacco products), PREPs contained specific warnings that were evaluated by respondents in the public intercept interview. Respondents were asked to assess the health risk (i.e., risk of lung or heart disease caused by the product), amount of “tar,” and “level of things that might cause cancer” on a scale from 1 to 10. Participants were asked whether the ads contained any set of specified messages, including whether the message was indicating that the product safer or healthier than other cigarettes, and whether it will help someone quit smoking. Questions intended to deemphasize the focus on health issues were also included, which elicited information on the perceived potential benefits or expectations of product response (i.e., whether the product would taste better and whether it would be a good product to smoke with friends).

Shadel and colleagues (36) conducted an intercept survey in 14 states in which the test product (Quest) was not being marketed. A sample of 200 regular smokers, most of whom (92%) had never heard of Quest, were asked about their perception of Quest after viewing a single advertisement. Eight questions assessed beliefs on a five-point Likert scale (1, definitely untrue; 5, definitely true). One question, whether Quest had lower nicotine than regular cigarettes, was based on information that was explicitly stated in the advertisement. The remaining questions asked about health-related product attributes that were not mentioned in the advertisement but may have been inferred (i.e., Quest cigarettes are less addictive than regular cigarette; Quest cigarettes make smoking safer). Additional measures were taken on respondents' need for cognition, using a nine-item scale to determine respondents' levels of need for understanding and preference for thinking about complex issues (37, 38). Three items asked about smokers' perceived vulnerability to smoking-related health harms on a five-point scale: (a) “How much do you think you can smoke without harming your health?”; (b) “To what extent do you feel that your overall health has been affected by smoking”; and (c) “How much do you think that quitting smoking could help your health?”

An alternative approach involves door-to-door surveys: one independent study exposed smokers to one of four experimental conditions, in which they were shown advertisements for PREPs with or without a box containing health information superimposed onto the ad (39). Respondents completed a 30-minute survey about the type of information they used to evaluate advertisements for tobacco products. Materials included print advertisements for two PREPs (Eclipse and Accord) with or without product health information, as well as a second page with a bar graph demonstrating levels of three carcinogens in PREPs compared with those found in a standard “light” cigarette.

Tobacco companies commonly select cities or regions as “test markets” to launch new products before their national introduction. These test markets are used to validate results from consumer testing and confirm industry projections of consumer demand. They provide important data on consumer adoption processes and extended use behaviors (i.e., where consumers buy products, the level of promotion and marketing necessary to achieve a desired trial rate, and the level of product experience needed to “acclimate tryers and stimulate conversion”; refs. 40, 41).

Test markets for PREPs are carefully selected according to several criteria that potentially impact outcomes (40). The geographic distribution of an area (i.e., population density, demographics) is of interest to tobacco companies, particularly as it corresponds to target groups. These markets must be nationally representative in terms of low-tar smokers, as well as smokers of discount brands (to minimize bias resulting from extremely high or low coupon use). The tobacco company's existing share of the market is evaluated to assess whether “cannibalization” (switching of consumers from existing company brands to the new product, rather than attracting new consumers from other brands or among nonsmokers) may occur.

The industry uses random-digit dial (RDD) methods to conduct surveys, often in multiple waves, through market research firms and follow standard RDD methods to obtain a sample of that is representative of the test market or the national population. Quota controls may be implemented according to gender, age, ethnicity, geography, price (i.e., generic/full price), and flavor (menthol/nonmenthol; refs. 42-44). Mail surveys are a common method used in test markets as well as national markets; household names may be purchased from a direct mail company. Mail surveys and telephone surveys may also be conducted among eligible households that have previously participated in market research conducted by the tobacco companies, or among customers in a preexisting company database. Recruitment for studies of PREPs in test market have included offers in local newspapers to “try Eclipse for free” by calling a toll-free number; respondents receive mail surveys or telephone surveys 2 to 5 weeks after receiving the product (45, 46).

In the independent literature, population-based surveys using random-digit dialing have asked respondents whether they are aware of specific PREPs, providing them with name of the product (i.e., Advance, Omni, Accord, or Eclipse; ref. 47). Others have asked respondents to name any specific PREP that they were aware of, but did not provide brand names (48). This is an important consideration in study design, as awareness of the concept of reduced-exposure products does not necessarily equate awareness of specific brands: in the study by O'Connor et al. (48), ∼27% of the respondents who expressed a general knowledge of such products were able to identify a specific product. In addition to questions on brand awareness, questions about demographic information, history of product use, interest in trial, and attitudes about products and government regulation of tobacco products are posed to respondents (47, 49). Sample sizes for RDD surveys of PREPs by independent researchers ranged from 1,174 (all current smokers) to 9,736 (inclusive of current smokers, former smokers, and nonsmokers; ref. 49).

Tobacco industry surveys of consumers' attitude and product awareness are conducted by telephone, mail, or email. Measures have included current awareness of the test product at baseline and after product launch, feelings about smoking generally, opinions about PREP benefits or claims, product expectations, and influence of media reports of test product on consumer expectations and interest (44, 45, 50-52). In a 1997, telephone survey within test market, RJ Reynolds evaluated whether consumers responded with higher ratings of acceptance of the test PREP, Eclipse, when they received a promotional video before product trial versus receiving only the product by mail (53).

Web surveys provide a means to access large numbers of survey participants in a rapid and cost-effect manner. This method has been used by independent researchers to assess consumers' perceptions to PREP messaging from specific populations, such as college students. O'Connor et al. (6) asked a sample of 424 participants to rate advertisements in terms of positive and negative expectancies about the product. Expectancies were measured on a Likert-type scale ranging from −3 (completely unlikely) to +3 (completely likely). Participants were asked to rate their product expectations when presented with a range of adjectives and phrases (see Table 3). No evidence of the use of Web surveys for assessment of PREP product perceptions was found in the industry documents archive, although Freeman and Chapman (54) recently conducted content analyses of tobacco companies' interactive marketing and consumer feedback campaigns conducted on their promotional Web sites.

Although not found in the independent literature, it is common for tobacco companies to measure consumer perceptions after a participant has sampled a product following a single, rather than repeated, use. “One-stick” studies are conducted among targeted consumers before large-scale consumer testing (55). Industry validation of the one-stick method compared with longer trials (i.e., studies where respondents smoke one to two packs before being asked to evaluate the product) revealed that one-stick tests are valid methods for widespread, quick screening of products for further development work. However, these tests are less valid for certain attributes (i.e., harshness, blend characteristics), where sensory characteristics become perceptible only after several cigarettes (56). They are occasionally administered within focus groups, public intercept surveys, or other concept/product testing methods, and measure smokers' opinions and reactions to the product after smoking a single sample of the product (approximate n = 300; ref. 57).

Before sampling the product, smokers may be asked a series of questions regarding their perceptions of the brand based on prior awareness and/or exposure to an advertisement. Measures of perceptions specific to PREPs include likes, dislikes, taste rating, aroma rating, tar level, ease of lighting, feel in the hand, ease of draw, extinguishing cues (i.e., “how they know when it's out”), ease of disposal, reactions to accompanying instruction booklet, and reactions to packaging (58). For a prototype to be recommended for further acceptance testing, it must achieve significantly higher ratings on overall taste preference and on most monadic taste ratings (59).

Tobacco companies have provided prerecruited respondents with product samples to use at home for a limited period of time. Some studies provide a single pack of the test product (32), whereas others entail longer term use (i.e., for 3days to 1 week) or more complex switching study designs. In a switching study, two products (the test product and a standard control) are given to participants to compare over the course of the study period (60, 61), and usually the products are “blinded” whenever possible to avoid response bias (46). Before the in-home phase, subjects may complete a questionnaire on ideal product ratings to assess baseline sensory preferences (62). Participants provide ratings after using the product in an environment that is more naturalistic than the smoking panel study design.

Products may be mailed or personally distributed to subjects at a research site. Advantages of mailing products for the in-home testing method, compared with laboratory-based sensory panels, include the ability to test a larger number of subjects, potentially over a wider geographic area. However, this method has distinct disadvantages, including a longer turnaround time before receiving results, researchers' loss of control over a subject's smoking behavior or use of the product, a substantial commitment by the subject, and the possibility that test products may be shared with nonsubjects (63). In-home product tests that require subjects to return to a laboratory or other research site for follow-up visits provide the opportunity for longer product use as with mailed in-home product tests, but ensure faster turnaround time, lower likelihood of attrition due to the fact that subjects return to the research facility for payment, and the opportunity for sensory or other laboratory-based procedures at follow-up (63).

Final sample sizes range from 100 to 350 complete responses per test product, with an expected response rate of about 50% (60, 63-67). Participants rate products for overall satisfaction, purchase interest, and on key sensory attributes, in a process known as monadic testing (68). Subjects are presented with one test product at a time and asked to rate them on selected attributes, rather than smoking a number of products at once and comparing them (69). These ratings are compared with subjects' initial ideal product ratings from the beginning of the study. Although respondents may be instructed to use the test product in lieu of their regular cigarette brand, follow-up questionnaires do not necessarily measure smoking behaviors such as frequency and combined use (62).

Sensory evaluation panels are used to measure the influence of specific product design characteristics on sensory responses. These properties may include taste and aftertaste, odor, tactile properties of product use (e.g., mouthfeel; “kick” or “bite”), tactile properties of the product itself (e.g., ease of lighting, firmness of rod), and strength (amount of nicotine content or delivery). This testing is typically conducted by tobacco companies before broader scale acceptance testing (70). The tobacco industry often recruits among a company's own employees for sensory evaluation panels, although these panelists are also recruited from the community (71-75). Panels typically consist of 15 to 50 participants who have been trained to use descriptive terms (i.e., taste and aroma) in specified ways when evaluating cigarettes (70). A controlled “triangle” testing strategy has been used in industry research to control for the potential for a given product to “mask” the sensory effects of the next product sampled. Under this regimen, three products are presented (two identical and one different) and participants are asked to identify and rate the one that differed. Order of presentation is varied in five combinations (i.e., AAB, ABA, BAA, BBA, BABA).

Quantitative descriptive analysis is conducted by tobacco industry laboratories also using trained sensory evaluation panelists (76). Prospective panelists are recruited based on “verbal articulation, social interaction, and most importantly, sensory acuity.” Approximately 30% to 40% of respondents are invited to be trained as panelists. Ten, 1-hour training sessions are held over a period of 2 weeks, where panelists are taught to identify sensory terminology. Development of a common set of terms describing taste and aroma characteristics, and associated definitions, are agreed to by the panel (77). Panelists rate the sensory attributes of each product on a structured questionnaire known as a “ballot.” Before each study, a warm-up session is conducted to expose the panelist to a subset of test products to calibrate the sensory scales and, if necessary, add additional terms to the ballot (77). Overall acceptance of the product can be measured using a structured scale (i.e., 0, dislike a lot; 50, indifferent; 100, like a lot) often called a “thermometer rating” (Fig. 2A; ref. 78), and recording perceptions of a variety of sensory attributes on a computer-administered unstructured line scale (Fig. 2B; ref. 79). For each study, two to six products are assessed, with no more than six products evaluated within a 1-week period. Three replicate evaluations of each product are made by the panelists, with a randomized complete block design being implemented to prevent order bias (77).

Studies by independent scientists also have incorporated sensory assessment into studies of smoking behavior, although the focus of this research has tended to be on measures of nicotine effect, smoking urge, and withdrawal relief and abuse liability rather than consumer acceptability. In an earlier study of denicotinized cigarettes (80), subjects were asked to rate sensory and reinforcing properties of the products, with anchor points ranging from 0 to 100 reflecting extreme responses to each question (i.e., 0, “no taste”; 100, “a lot of taste”). A study examining differences in subjective effects between moist snuff, medicinal nicotine lozenge, and three smokeless tobacco lozenge PREPs measured drug effects and liking through a 17-item scale ranging from 1 to 10 (“not at all” to “extremely”; ref. 81). Items include good drug effect, harshness of cigarette, heaviness of cigarette, desire to smoke the cigarette just smoked, intensity of flavor, and freshness of cigarette (82). The same drug effect and liking measures were used to compare a carbon-filtered PREP and conventional control cigarettes (83). The carbon-filtered PREP was rated as having less drug effect despite similar smoke machine nicotine yield, and was less liked than the conventional product.

Independent scientists have also assessed the influence of the sensory attributes of PREPs on use behavior and product adoption. These studies have involved switching designs, in which subjects are randomly assigned to PREP and non-PREP comparison conditions. Comparison conditions have included nicotine replacement inhalers (84, 85), nicotine lozenge (81), subjects' regular cigarette brand (4, 80, 86), a light cigarette brand (80), moist snuff (81), or cessation (86). Sample sizes in these studies have ranged from 10 (4, 80, 81) to 50 (38, 84, 85). Subjects may be enrolled based on their smoking status (i.e., regular brand is light or ultralight; refs. 4, 86) and/or level of nicotine dependence (4). Smokers may be asked to try a product in a single laboratory session (82), a series of laboratory sessions (2-5; refs. 4, 81), or completely have a forced switch over the course of days (3-5) or weeks (2-6) per product (80, 84, 86).

Studies have also measured the capacity for PREPs to relieve urge to smoke and symptoms of nicotine withdrawal, compared with conventional cigarettes. In laboratory-based smoking experiments, a 3-item craving/satisfaction measure has been used after each puff, as well as a separate 4-item measure at 15-minute increments in the 90 minutes following smoking, and a widely used 11-item tobacco withdrawal scale (87) may be administered after 90 minutes of deprivation (see Table 4 for further information on measures; refs. 4, 80, 81, 86). These scales may use a visual analogue scale, with items consisting of a word or phrase above a horizontal line anchored by phrases such as “not at all” and “extremely.” The Questionnaire on Smoking Urges measures intention to smoke and anticipation of relief from withdrawal (4, 86, 88). Questionnaires are administered immediately before of after the product is presented to subjects (81).

Evidence from both industry and independent research has confirmed the utility of conceiving tobacco CR as comprised of two distinct but related domains: product perceptions and response to product. Product perceptions, which encompass consumers' awareness, attitudes and beliefs to product concept, marketing, and/or messaging, are typically assessed via multiple measures that are contained in a structured custom-designed survey. In contrast, responses to the product—usually determined through actual use—involve measurement of sensory responses (including taste and aftertaste, mouth feel, and bite/kick), other subjective responses (including nicotine effect, urge, and withdrawal relief), and ratings of product acceptability, using sensory panels or switching study designs. The highly varied nature of the components that comprise CR has produced a broad array of methods, which are chosen according to the specific questions being addressed. Methods used by the tobacco industry fall into three major areas: concept testing, messaging studies, and sensory assessment. In contrast, independent research on PREPs is conducted postmarket, after the product has been refined by industry CR research, and includes assessment of product messaging and sensory and subjective responses. The findings from independent studies are used to inform the scientific community on the public health implications of PREPs and new products. Further work should guide the development of public health interventions and regulation of PREPs and new products.

The range of research methods identified provides a potential basis for developing a systematic set of PREP CR assessment strategies. Comprehensive listings of available information on CR outcome measures used in research on PREPs are provided, and are organized according to the two broad domains of product perception (Table 3) and response to product (Table 4).

Research on product perceptions generally makes use of custom-designed, large scale surveys. Telephone surveys are a standard methodology for tracking trends in risk factors and behaviors over time, evaluating the effects of interventions, and examining cross-sectional and longitudinal characteristics of target populations. The future of random-digit dial surveys is compromised by the decrease in landline telephone coverage due to the popularity of cell phones, as well as technology such as voice mail or call screening that reduces the likelihood of reaching participants directly. Mainstream scientists have developed strategies for addressing these methodologic challenges, including using multimode designs, incorporating cell phone users into sampling frames or applying weights based on the demographic characteristics of the cell phone users, using text messaging and voice mail, and sending advance letters and scheduling calls (89). Web surveys have become an increasingly popular tool among mainstream researchers, because of their capacity to rapidly recruit and screen participants, present visual information, and collate survey data. Nevertheless, adapting telephone surveys to a Web-based format requires consideration about issues about recruitment of random samples, question format, and visual design (90). The independent scientific community is continually testing Web-based measurement issues and their effects on survey error, but the method may present considerable benefits to researchers. In addition to being cost effective, researchers have found that in instances when respondents are asked to report sensitive information, computer-based surveys may yield more accurate data (90). Intercept surveys, particularly in malls and other public places, have been a popular survey method among commercial marketing researchers since the 1970s. Issues of validity and reliability arise in public intercept surveys, whereas more traditional door-to-door survey methodologies allow randomization and stratification, public intercept surveys are subject to response bias yielding a potentially nonrepresentative sample (91). Intercept surveys nonetheless provide a low-cost option for researchers seeking to directly expose consumers to a product and obtain immediate feedback. However, specific locations may have a tendency to “burn out” after a period of time, if too many researchers repeatedly sample from the same pool of respondents (91).

Research aimed at assessing responses to product must provide smokers with an opportunity to sample the product, restricting research methods to laboratory or home-based studies. Tobacco industry and independent scientists take slightly different approaches. The tobacco industry considers its users' characteristics according to target markets and/or product preferences, whereas mainstream scientists consider subjects according to other measures commonly found in the published literature (i.e., socioeconomic status, addiction and abuse liability, smoking behavior). Although measures of sensory response are important in determining product acceptability, other measures are required. Scales used by the industry to measure overall acceptability and sensory response are useful for informing product developers (and those anticipating the likelihood of future adoption) because they help to determine why consumers may or may not find a product appealing (i.e., low overall liking scores due to a product being too light; ref. 92).

Such scales require further refinement by independent scientists to confirm the validity of key sensory constructs and their predictive relationship with product acceptability and future intentions for trial. In this context, a measure can be validated through demonstration of concordance with widely understood or accepted examples of the construct in question (“face validity”), or by demonstrating concordance with an established measure. In addition, however, validation of measures requires attention to the purpose of the assessment and the special circumstances under which it might be applied. Such circumstances include the type of product tested, the claims made or implied by the manufacturer or the product design, and special populations to whom assessment is directed. The reliability of sensory scales also requires empirical demonstration.

Strategies to assess CR should, of course, be guided by their appropriateness for providing answers to a specific research question. Sets of questions or response items generally have been developed in response the specific demands of a research program and are generally not standardized for general use.

Further research is needed to develop a more complete understanding of CR and methods for its assessment. In particular, research to better understand the relationship between positive CR components and use behavior, especially the progression from trial use to patterns of established use, is needed. Studies that assess the effect of manipulating product information on risk perceptions and use behavior are also required. Likewise, manipulation of product design, emissions on CR, and use behavior outcomes would be valuable. The proposed inter-relationship between product perceptions and response to product has not been systematically investigated by industry or independent scientists. Further investigation of the relationships between product perceptions and response is needed, and future research on CR should, where possible, address these dual domains. This article provided a descriptive overview of CR methods used in both independent and industry research; however, a comparison of the effectiveness of methods for demonstrating specific outcomes is still needed. CR assessment methods should be examined and compared for validity, particularly according to type of PREP and the range of claims that could be presented. PREPs such as Omni have referred to reduced cancer risk in messaging materials, whereas Eclipse has also referred to reduced risk for respiratory illness. Other products, such as Camel Snus, present information about the convenience of the product and pleasure associated with its use. Further research is also required on consumer perceptions of implicit product messages, and the relationship between the consumer's understanding of actual message and the influence on risk-benefit ratio.

Research has shown that implicit messages are perceived similarly to explicit messages, which further underscores the need for research in this area. Looking more broadly, the applicability of existing or future methods for special populations requires investigation. Currently, PREPs have been shown to have greater appeal to women, older smokers, exsmokers, and health conscious smokers (35, 93). Further research should address PREP interest and use among ethnic and racial minorities and youth. Finally, the changing nature of PREP and new product design, marketing strategies, and consumer needs means that assessment methods must be able to respond to the “moving target” by adapting appropriately, but while maintaining empirical validity.

Strategies for comprehensive assessment of CR have previously been proposed by Stratton (2001) and Hatsukami et al. (2005; ref. 5). The present review provides a basis for establishing methods and measures for assessing CR to PREPs using a conceptual model. This model allows testing of predicted CR outcomes that may arise through future modification of product design or messaging. The tobacco industry has focused on CR assessment to ensure that its products meet with market success. Industry strategy now appears intent not only on successfully modifying product design and messaging, but tailoring those modifications to specific subgroups of smokers, such as youth, women, and racial/ethnic minorities (2). The proposed model of CR may provide a basis for anticipating CR among subgroups by accounting for social norms and personal factors such as tobacco use history and personal acceptability. The present model also provides a systematic strategy for those charged with regulation of tobacco product design and marketing, such as the Food and Drug Administration.

To ensure the commercial success of new PREPs, we can be sure that tobacco manufacturers will attempt to make those products acceptable alternatives for current smokers and may also enhance their appeal for new and extobacco users. Tobacco products that purport to reduce risk, whether or not they succeed in reducing human exposure to harmful toxicants, may perpetuate tobacco-related harms if their acceptance and use at a population level is enhanced. Consumer acceptance is conceived as the related components of product perceptions and sensory and other responses to product. In response, there has been a substantial research effort by the public health community to understand consumer perceptions and responses to product to PREP messaging and product design. As methods and measures for PREP assessment are adopted and standardized, their policy and regulatory function must be heeded. The present model provides an empirically based strategy for understanding and predicting CR, trial, and likelihood of future adoption of new PREPs. With further development of research methods, the model may provide a basis for understanding and predicting CRs of targeted smoker subgroups. This information is essential for effective regulation of new tobacco products by authorities such as Food and Drug Administration and states' attorneys general.

No potential conflicts of interest were disclosed.