Background: Internal tobacco industry documents and the mainstream literature are reviewed to identify methods and measures for evaluating tobacco consumer response. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing consumer response, and consider how these methods might be applied to evaluate potentially reduced exposure tobacco products and new products.

Methods: Internal industry research reviewed included published articles, manuscript drafts, presentations, protocols, and instruments relating to consumer response measures were identified and analyzed. Peer-reviewed research was identified using PubMed and Scopus.

Results: Industry research on consumer response focuses on product development and marketing. To develop and refine new products, the tobacco industry has developed notable strategies for assessing consumers' sensory and subjective responses to product design characteristics. Independent research is often conducted to gauge the likelihood of future product adoption by measuring consumers' risk perceptions, responses to product, and product acceptability.

Conclusions: A model that conceptualizes consumer response as comprising the separate, but interacting, domains of product perceptions and response to product is outlined. Industry and independent research supports the dual domain model and provides a wide range of methods for assessment of the construct components of consumer response. Further research is needed to validate consumer response constructs, determine the relationship between consumer response and tobacco user behavior, and improve reliability of consumer response measures. Scientifically rigorous consumer response assessment methods will provide a needed empirical basis for future regulation of potentially reduced-exposure tobacco products and new products, to counteract tobacco industry influence on consumers, and enhance the public health. (Cancer Epidemiol Biomarkers Prev 2009;18(12):3225–40)

The commercial success of tobacco products depends largely on cigarette manufacturers designing and promoting products to ensure that they retain their existing customers, attract the customers of their competitors, and replenish the market with new smokers who choose those brands. An extensive body of evidence derived from analysis of internal tobacco industry documents has shown the use of sophisticated and subtle design variations by the tobacco industry to promote product acceptance among tobacco consumers. The design features of tobacco products may prompt sensory and other subjective responses that are important in the determination of taste and smoking satisfaction (1, 2), psychological reward (3), and craving diminution (4, 5): all of which may contribute to consumer acceptance. Tobacco companies have also strived to communicate the advantages to their consumers of new design features through new packaging and branding, sophisticated marketing messages, and even the form of the tobacco product itself (6).

Advertising and other messaging strategies are used by the tobacco industry to convey product information, and so positively influence consumer perceptions. Advertising and marketing of lower tar cigarette brands over the past 50 years has emphasized “satisfaction” (taste and sensory effects) and “pleasure” (nicotine effects), while simultaneously communicating reduced cigarette strength and, by implication, lower product risk (1). The misguided belief that a low-tar cigarette is less risky is undoubtedly a deterrent to smoking cessation, because health concerns are an important factor in motivating smoking cessation (7). A case in point is the Marlboro Lights brand. Marlboro Lights was one of first brands to use the “Lights” descriptor to communicate its lower tar (and, by implication, lowered harm) feature. The Marlboro Lights pack achieves a lighter look with a white pack and a gold chevron instead of the distinctive red chevron found on Marlboro Red. Consumer research conducted by Philip Morris has shown that smokers perceived different tastes when they smoked identical cigarettes in red versus gold packs (8). Marlboro Lights advertising also used more white space, less action, and muted colors: all intended to reinforce the lower tar delivery message of the brand. The cigarette itself also has been designed to reinforce the perception of a safer product, with the use of white tipping paper on the filter tip, in contrast to the cork colored paper used on most full-flavored cigarettes (8). The use of chemical additives and milder tobacco blends in low-tar cigarettes also helps reinforce the consumer's perception that a “light” cigarette is safer, when in fact it has been designed in a way that simply enhances the smoker's ability to puff more freely on the cigarette (9, 10). Despite the intensive efforts taken by the industry to implicitly communicate lowered harm, internal industry studies show that lower machine yield cigarettes have greater toxicity compared with higher yield (full flavor) cigarettes (8).

The proliferation of novel tobacco products that purport to be safer than conventional products has created significant challenges for the public health community. So-called potential reduced exposure tobacco products (PREP) use novel design features or ingredients that are intended to reduce exposure to tobacco-related toxicants, and thus lower disease risk for the consumer (11). The Institute of Medicine has stated the importance of understanding how consumers perceive and respond to PREPs to avoid misperceptions among consumers, such as those that resulted from the misleading marketing of so-called low-tar/light cigarettes (11). There are well-founded concerns that consumers could be misled by implicit or explicit marketing messages that communicate health benefits of PREPs, since official standards for evaluating claims for tobacco products have not yet been developed. Explicit marketing messages and claims made by tobacco manufacturers for reduced exposure tobacco products include messaging and claims about reduced exposure to harmful constituents (12). Not all reduced risk claims are explicit: implicit claims can be communicated through product descriptors, including the colors and graphics used in product packaging, together with qualifier terms such as “ultra” and “lights.”

Most PREPs introduced in the last decade are no longer in the market, in part because products designed to reduce exposure tend to have low acceptability among consumers. Poor sensory and subjective characteristics of PREPs may have contributed to their low consumer acceptance, despite a substantial research and development program by major tobacco companies to address this (i.e., Marlboro UltraSmooth; ref. 12). The modification of tobacco products to reduce harmful toxicants can affect design features that facilitate positive consumer responses (CR). For example, activated carbon is sometimes added to cigarette filters to reduce harmful gas phase constituents, but this also reduces or alters flavor characteristics (13). Similarly, products designed to heat tobacco electrically may be too dissimilar to a cigarette to promote broad consumer interest. However, new or future PREPs may appeal to consumers and could receive broad acceptance and corresponding market share. In the new tobacco regulatory environment provided under the Family Smoking Prevention and Tobacco Control Act of 2009 (14), the Food and Drug Administration will have the capacity to restrict or ban product design features, including flavorants and additives designed to enhance consumer acceptability, and product messaging that promotes perceptions of lowered risk. Thus, there is a great need for scientifically validated methods to allow rapid and early assessment of CRs to PREPs and other new tobacco products, and to provide an empirical basis for this new regulatory framework.

CR to tobacco products is defined as a set of subjective and behavioral responses that convey information, affect behavior and likelihood of long-term product use by the consumer, and his or her future intentions for product adoption (see Fig. 1). The Theory of Reasoned Action (15, 16) provides a basis for understanding the link between attitudes and behavior, and may provide a heuristic for conceptualizing of CRs to tobacco products. Theory of Reasoned Action posits that the likelihood of a behavioral outcome is influenced by an individual's attitude combined with subjective norms about that behavior. Consistent with this theory, CR might be seen as a complex set of attitudes and norms that influence the likelihood of product trial (the behavioral response).

Figure 1.

Hypothetical model of tobacco CR.

Figure 1.

Hypothetical model of tobacco CR.

Close modal

In the model proposed here, CRs are represented by two broad categories: perception of the product and response to the product. Product perceptions encompass subjective responses to product information, and include individual perceptions of risk, attitudes and beliefs about the product, social acceptability, and outcome expectancies such as satisfactory nicotine and sensory effects. Responses to product encompass acute nicotine and sensory effects (including drug reward; relief of craving and withdrawal; taste, harshness/smoothness, and other sensory characteristics), conditioned motivational responses to sensory cues, afffective responses, and product personal acceptability. Also represented in Fig. 1 are two general domains within which tobacco manufacturers seek to positively influence CR: product advertising and marketing (“messaging”), and product design. In general, tobacco industry messaging is aimed at directly influencing product perceptions, whereas design manipulations are used to influence product responses. A hypothetical link exists between product perceptions and response, such that each exist in a mutual feedback loop. The combined influence of product perceptions and product response determine the likelihood of initial and subsequent use. Also assumed in this model are pre-existing personal factors, such as degree of tobacco dependence, and social and personal norms, in the context of which industry messaging and product design must operate to be effective. The tobacco industry has designed and marketed products to appeal to target groups with specific characteristics such as age, gender, ethnicity/race, and degree of tobacco dependence (2). This model thus allows a basis not only for predicting future use of the product, but for understanding relationships between the range of factors that contribute to positive CRs.

This article reviews both internal tobacco industry documents and the mainstream literature to identify methods and measures for evaluating tobacco CR. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing CR, and consider how these methods might be applied to evaluate new products, including PREPs. Although CR research has been conducted with conventional products, the present review focuses on products designed to reduce exposure, owing to the special challenges associated with assessing their potential for consumer acceptance and appropriately communicating their risks. Strategies used to evaluate PREPs have largely evolved from those used to evaluate conventional products, thus making a broader review largely redundant. For reasons of economy, the current review therefore will focus on PREPs.

Review of Internal Industry Methods

Internal tobacco industry documents were identified using databases at Tobacco Documents Online,6

the Legacy Tobacco Documents Library,7 and the British American Tobacco Documents Archive.8 A snowball sampling design was used for text-based and index searches, with an initial set of keywords and phrases (i.e., consumer perception, reduced harm, test market), which resulted in the development of hundreds of further search terms. Search terms may have been combined for more targeted searches, initially yielding large numbers of documents (see examples in Table 1). These results were visually skimmed, and duplicates and obviously irrelevant documents (i.e., deposition transcripts, popular newspaper and magazine articles, smoke chemistry data) were discarded. The remaining documents were carefully reviewed for relevant methods and measures of CR.

Table 1.

Example terms used in searches of tobacco industry document archives

Search termsTotal number of resultsNo. of documents containing relevant methods and measures
Trial and PREPs 730 
“Trial/usage” and “safer cigarette” 25 
“Product trial” and monitoring 306 
“Consumer expectations” and safer 508 
“Acceptance evaluation” 125 
“Product claims” and “safer alternative” 31 
“Inclusion criteria” and “safer cigarette” 35 
Search termsTotal number of resultsNo. of documents containing relevant methods and measures
Trial and PREPs 730 
“Trial/usage” and “safer cigarette” 25 
“Product trial” and monitoring 306 
“Consumer expectations” and safer 508 
“Acceptance evaluation” 125 
“Product claims” and “safer alternative” 31 
“Inclusion criteria” and “safer cigarette” 35 

Relevant documents included (a) product development efforts to address issues of consumer perception of PREPs, and (b) market research and surveillance conducted following the launch of novel tobacco products. Of the ∼8,000,000 documents available in the archives, the final sample was narrowed to include ∼970 relevant documents dating from 1980 to present. A total of 60 documents contained sufficiently detailed descriptions of industry methods, instruments, and measures to be used as citations in this article.

Review of Independent Scientific Literature

A previous literature review conducted by Pederson and Nelson (17) summarized the findings of studies on consumer perceptions of PREPs. In contrast, this article focused on cataloguing the methods used by independent researchers to measure CR, including responses to product, and compared them with research strategies used by the tobacco industry. The published literature within PubMed, ranging from January 1995 to August 2008 was searched using keywords such as reduced exposure, reduced harm, novel tobacco products, sensory, smoker perception, and subjective response.

Selected manuscripts examined smokers' knowledge, attitudes, beliefs and intentions about, and subjective responses to, PREPs. The criteria used by Pederson and Nelson (17) were used to define PREPs; specifically, tobacco products that resemble conventional cigarettes but do not necessarily require both tobacco and nicotine and are heated or burned, or is some other oral nicotine-containing product made by a tobacco company (i.e., electrically heated cigarettes, snus, denicotinized cigarettes, charcoal-filtered cigarettes).

Methods to Assess Product Perceptions

Manufacturers of PREPs are concerned with consumer adoption of their proposed product, and the first phases of product development or marketing efforts include measuring perceptions of the product “concept” using a variety of methods. The product concept can be assessed independently of actual product use, or combined in a product-concept test, where subjects view and rate the concept, and sample and rate the product (18). Methods used by the tobacco industry for PREP concept and message testing include focus groups, public intercept surveys, and random digit dial surveys. Independent researchers have applied all of these methods for postmarket PREP assessment, with the inclusion of Web surveys (Table 2). The following describes methods used by the tobacco industry, at premarket and postmarket stages, to assess consumers' perceptions of product and product concepts, and methods used by independent researchers, primarily at the postmarket stage, to assess perceptions of product.

Table 2.

Strategies to assess PREP CR domains

Response domainMethod
Perception of product 
    Health/safety claims •Focus groups 
•Public intercept surveys 
•Door-to-door surveys 
•Web surveys 
•Mail and telephone surveys 
    Cosmetic benefits •Focus groups 
•Public intercept surveys 
•Web surveys 
Consumer interest 
    Brand awareness •Public intercept surveys 
•Mail and telephone surveys 
    Purchase intent •Focus groups 
•Public intercept surveys 
•Mail and telephone surveys 
Response to product 
    Taste, sensation •Field test (single use) 
•Laboratory-based smoking panels 
•In-home product testing (extended use) 
•Randomized switching studies 
•Laboratory-based smoking sessions (randomized crossover design) 
    Nicotine craving, withdrawal •Randomized switching studies 
•Laboratory-based smoking sessions (randomized crossover design) 
    Drug (nicotine) effects •Laboratory-based smoking sessions (randomized crossover design) 
    “Lightability” and other usability issues •Focus groups (in industry, “Discovery Groups” used for consumer education) 
•Field test (single use) 
•Mail and telephone surveys 
Response domainMethod
Perception of product 
    Health/safety claims •Focus groups 
•Public intercept surveys 
•Door-to-door surveys 
•Web surveys 
•Mail and telephone surveys 
    Cosmetic benefits •Focus groups 
•Public intercept surveys 
•Web surveys 
Consumer interest 
    Brand awareness •Public intercept surveys 
•Mail and telephone surveys 
    Purchase intent •Focus groups 
•Public intercept surveys 
•Mail and telephone surveys 
Response to product 
    Taste, sensation •Field test (single use) 
•Laboratory-based smoking panels 
•In-home product testing (extended use) 
•Randomized switching studies 
•Laboratory-based smoking sessions (randomized crossover design) 
    Nicotine craving, withdrawal •Randomized switching studies 
•Laboratory-based smoking sessions (randomized crossover design) 
    Drug (nicotine) effects •Laboratory-based smoking sessions (randomized crossover design) 
    “Lightability” and other usability issues •Focus groups (in industry, “Discovery Groups” used for consumer education) 
•Field test (single use) 
•Mail and telephone surveys 

Focus Groups

Focus groups are commonly used in market research to assess consumer reactions to new ideas or products, and social scientists have increasingly applied the method to gain insight into specific research questions and frame formative issues for further study (19). Typically, focus groups consist of groups of 6 to 10 respondents screened to represent the target market of interest, gathered in the same room, and guided by a moderator through a discussion on topics of interest (20, 21).

Focus group research conducted by the industry has included both product concept–naïve participants as well as participants who have previously used a PREP and can offer feedback about their experience. RJ Reynolds (22) conducted focus groups among current users of Premier in 1990, to determine which features of the PREP they found most appealing and to catalogue reasons for continued use. Similar sessions were held by the company in test markets for Eclipse in 1996, where perceived benefits and drawbacks of the product concept and actual product were discussed among smokers (23).

Among independent researchers, focus groups have been used to assess motivations and use patterns of PREPs, and consumer's perceptions of PREP messaging. Carabello et al. (24) asked smokers ages 30 to 50 years questions about how they learned about PREPs, their reasons for trying in the first place, which products they had tried, their perceptions of the product at first trial, their reasons for continuing or discontinuing use, and whether they would recommend PREPs to others. Other studies have used focus groups to assess consumer's perceptions of PREP messaging by showing advertisements for products currently on the market to participants. O'Hegarty et al. (25) used recruitment criteria described by Carabello et al. (24) in cities where Eclipse was in test market. Following a discussion about PREP use, participants were shown a series of print advertisements and promotional materials for specific products. A discussion about the participants' reactions followed, including their impressions of the intended message of the advertisement and its influence on their interest in PREPs.

Both tobacco industry and independent researchers recruit and/or classify focus group subjects by demographic or social background, smoking history, or an individual's knowledge, attitudes, and beliefs regarding smoking. Some of these characteristics may be used as inclusion criteria, but the industry has used them also to group subjects in the analysis of data collected in consumer perception studies. Tobacco companies maintain extensive databases of current and former customers, to expedite recruitment of participants in PREP consumer perception studies. In studies of Eclipse, RJ Reynolds contacted smokers by telephone or mail to invite them to participate in focus groups and surveys; samples of the test product were sometimes sent beforehand for the consumer to try (23). In the independent literature, Hughes et al. (26) utilized a similar resource in their investigation of Eclipse; the researchers contacted RJ Reynolds for access to the database of smokers who had ever purchased Eclipse, as well as those who had received Eclipse promotional materials. The company provided the names and addresses to a third party mailing service, which sent study recruitment materials to the smokers, identifying the study investigators as independent of the manufacturer.

Public Intercept Surveys

Public intercept surveys use a convenience sampling strategy to administer a structured questionnaire to participants who are recruited at a public place (i.e., a shopping mall). Participants may also be shown examples of products or concepts to evaluate (25, 27-30). Questionnaires are written according to the objectives of the study, are administered in a structured interview format, and may also function as a screening tool to recruit participants for follow-up surveys or tracking studies on tobacco use by the company (31-33). Public intercept surveys for tobacco products may face unique limitations that compromise the representativeness of the final sample; for example, in 1997, RJ Reynolds discontinued its use of public intercept research for product Eclipse because of a high rate of refusal among mall patrons (34). It is unclear whether the high rate of refusal was inherent to the method, or whether it was due to the fact that consumers were resistant to participating in a tobacco industry study.

Independent researchers have used intercept surveys to investigate the nature and impact of implicit and explicit health messages contained in PREP promotional materials. Hamilton and colleagues (35) asked adult smokers to review a single advertisement for each type of product (one of two print advertisements for regular cigarettes, light cigarettes, and PREPs). Health messages, which promoted the health-related benefits of using PREPs instead of regular cigarettes, were distinguished from health risk warnings (35). In addition to Surgeon General's warnings (which were contained in advertisements for all types of tobacco products), PREPs contained specific warnings that were evaluated by respondents in the public intercept interview. Respondents were asked to assess the health risk (i.e., risk of lung or heart disease caused by the product), amount of “tar,” and “level of things that might cause cancer” on a scale from 1 to 10. Participants were asked whether the ads contained any set of specified messages, including whether the message was indicating that the product safer or healthier than other cigarettes, and whether it will help someone quit smoking. Questions intended to deemphasize the focus on health issues were also included, which elicited information on the perceived potential benefits or expectations of product response (i.e., whether the product would taste better and whether it would be a good product to smoke with friends).

Shadel and colleagues (36) conducted an intercept survey in 14 states in which the test product (Quest) was not being marketed. A sample of 200 regular smokers, most of whom (92%) had never heard of Quest, were asked about their perception of Quest after viewing a single advertisement. Eight questions assessed beliefs on a five-point Likert scale (1, definitely untrue; 5, definitely true). One question, whether Quest had lower nicotine than regular cigarettes, was based on information that was explicitly stated in the advertisement. The remaining questions asked about health-related product attributes that were not mentioned in the advertisement but may have been inferred (i.e., Quest cigarettes are less addictive than regular cigarette; Quest cigarettes make smoking safer). Additional measures were taken on respondents' need for cognition, using a nine-item scale to determine respondents' levels of need for understanding and preference for thinking about complex issues (37, 38). Three items asked about smokers' perceived vulnerability to smoking-related health harms on a five-point scale: (a) “How much do you think you can smoke without harming your health?”; (b) “To what extent do you feel that your overall health has been affected by smoking”; and (c) “How much do you think that quitting smoking could help your health?”

An alternative approach involves door-to-door surveys: one independent study exposed smokers to one of four experimental conditions, in which they were shown advertisements for PREPs with or without a box containing health information superimposed onto the ad (39). Respondents completed a 30-minute survey about the type of information they used to evaluate advertisements for tobacco products. Materials included print advertisements for two PREPs (Eclipse and Accord) with or without product health information, as well as a second page with a bar graph demonstrating levels of three carcinogens in PREPs compared with those found in a standard “light” cigarette.

Population-Based Surveys within Test Markets

Tobacco companies commonly select cities or regions as “test markets” to launch new products before their national introduction. These test markets are used to validate results from consumer testing and confirm industry projections of consumer demand. They provide important data on consumer adoption processes and extended use behaviors (i.e., where consumers buy products, the level of promotion and marketing necessary to achieve a desired trial rate, and the level of product experience needed to “acclimate tryers and stimulate conversion”; refs. 40, 41).

Test markets for PREPs are carefully selected according to several criteria that potentially impact outcomes (40). The geographic distribution of an area (i.e., population density, demographics) is of interest to tobacco companies, particularly as it corresponds to target groups. These markets must be nationally representative in terms of low-tar smokers, as well as smokers of discount brands (to minimize bias resulting from extremely high or low coupon use). The tobacco company's existing share of the market is evaluated to assess whether “cannibalization” (switching of consumers from existing company brands to the new product, rather than attracting new consumers from other brands or among nonsmokers) may occur.

The industry uses random-digit dial (RDD) methods to conduct surveys, often in multiple waves, through market research firms and follow standard RDD methods to obtain a sample of that is representative of the test market or the national population. Quota controls may be implemented according to gender, age, ethnicity, geography, price (i.e., generic/full price), and flavor (menthol/nonmenthol; refs. 42-44). Mail surveys are a common method used in test markets as well as national markets; household names may be purchased from a direct mail company. Mail surveys and telephone surveys may also be conducted among eligible households that have previously participated in market research conducted by the tobacco companies, or among customers in a preexisting company database. Recruitment for studies of PREPs in test market have included offers in local newspapers to “try Eclipse for free” by calling a toll-free number; respondents receive mail surveys or telephone surveys 2 to 5 weeks after receiving the product (45, 46).

In the independent literature, population-based surveys using random-digit dialing have asked respondents whether they are aware of specific PREPs, providing them with name of the product (i.e., Advance, Omni, Accord, or Eclipse; ref. 47). Others have asked respondents to name any specific PREP that they were aware of, but did not provide brand names (48). This is an important consideration in study design, as awareness of the concept of reduced-exposure products does not necessarily equate awareness of specific brands: in the study by O'Connor et al. (48), ∼27% of the respondents who expressed a general knowledge of such products were able to identify a specific product. In addition to questions on brand awareness, questions about demographic information, history of product use, interest in trial, and attitudes about products and government regulation of tobacco products are posed to respondents (47, 49). Sample sizes for RDD surveys of PREPs by independent researchers ranged from 1,174 (all current smokers) to 9,736 (inclusive of current smokers, former smokers, and nonsmokers; ref. 49).

Tobacco industry surveys of consumers' attitude and product awareness are conducted by telephone, mail, or email. Measures have included current awareness of the test product at baseline and after product launch, feelings about smoking generally, opinions about PREP benefits or claims, product expectations, and influence of media reports of test product on consumer expectations and interest (44, 45, 50-52). In a 1997, telephone survey within test market, RJ Reynolds evaluated whether consumers responded with higher ratings of acceptance of the test PREP, Eclipse, when they received a promotional video before product trial versus receiving only the product by mail (53).

Web Surveys

Web surveys provide a means to access large numbers of survey participants in a rapid and cost-effect manner. This method has been used by independent researchers to assess consumers' perceptions to PREP messaging from specific populations, such as college students. O'Connor et al. (6) asked a sample of 424 participants to rate advertisements in terms of positive and negative expectancies about the product. Expectancies were measured on a Likert-type scale ranging from −3 (completely unlikely) to +3 (completely likely). Participants were asked to rate their product expectations when presented with a range of adjectives and phrases (see Table 3). No evidence of the use of Web surveys for assessment of PREP product perceptions was found in the industry documents archive, although Freeman and Chapman (54) recently conducted content analyses of tobacco companies' interactive marketing and consumer feedback campaigns conducted on their promotional Web sites.

Table 3.

Subject characteristics and outcomes variables used for consumer perception of product, by method of assessment

Method of assessmentSubject characteristicsOutcome variables/measures
Focus group Age (24, 94) Prior awareness of PREP (95) 
 Gender (24, 94) First awareness of PREP (23, 24, 94) 
 Race/ethnicity (24, 94) Reasons for first trial (24, 94) 
 Geographic region (24, 94) First impressions upon trial (24) 
 Age of onset/years smoked (24, 94) Whether they would recommend PREP to others (24, 94) 
 Previous use of PREP (24) Whether health claims would influence trial (94) 
 Children living at home (24) Factors that would influence continued use (23, 24, 94) 
  Influence of quit intentions on willingness to switch (94) 
  Dual use vs switching (94) 
  Reaction to Surgeon General's warning (94) 
  Perceived benefits (personal, social; refs. 23, 95) 
  Perceived drawbacks (23) 
  Anticipated taste/sensation (95) 
  Reaction to instructions (95) 
  Reaction to name and pack design (95) 
  Reaction to “cleaner smoking” products (95) 
  Price expectations (95) 
  Type of person to use product (95) 
Face-to-face interview (public intercept or door-to-door canvassing) Age (i.e., 21-34 y, 35-49 y, 50+ y; refs. 25, 34-36, 39) Brand awareness (34) 
 Gender (34-36, 39) First awareness (mail, advertisement, etc; ref. 34) 
 Education level (35, 36, 39) Ever used product (34) 
 Race/ethnicity (35, 36, 39) Source of trial product (34) 
 Tar/Nicotine of usual brand (35) Purchase interest (28, 96) 
 Cigarettes per day (35) Repurchase of PREP after trial (34) 
 Usual brand menthol/nonmenthol (25) Perceived benefits (28, 34) 
 Usual brand price category (25) Different from cigarettes (28, 96) 
 Appearance (i.e., “consistent with clientele of venue”; ref. 34) Importance of perceived differences (28, 96) 
 Quit attempt in past year (35) Taste/claim trade-off (96) 
 Scores on Fagerstrom Test for Nicotine Dependence (36) Price/claim trade-off (96) 
  Situations where smoking is restricted (96) 
  Switching behavior (price, brand; ref. 96) 
  Perceived toxicity (scale 0-10; ref. 39) 
  Belief whether switching would expose smoker to fewer cancer-causing chemicals (y/n; refs. 36, 39) 
  Perception of how many cancer-causing chemicals reduced (all, most, some, none; refs. 35, 39) 
  Perceived health risk of PREP (i.e., scale 0-10; refs. 35, 39) 
  Whether switching would reduce smokers' chance of cancer (y/n; ref. 39) 
  Perception of addictiveness of PREP (36) 
  Need for understanding of product benefits (36) 
Telephone survey (RDD) Age (34, 45, 48, 53, 97) General awareness of PREPs (47, 48) 
 Gender (34, 45, 48, 53, 97) Awareness of specific PREP brands (unaided recall; refs. 47, 48) 
 Race/ethnicity (48) Ever use of PREP (44) 
 Socioeconomic status (34, 44, 45, 48, 97) First awareness (mail, advertisement, etc; ref. 44) 
 Education (26) Interest in trial (47) 
 Occupation (58) First trial (purchase, friend, etc; ref. 44) 
 Geographic region (34) Number of sticks used (44) 
 Usual brand T/N (53, 98) Read instructions before trying (y/n; ref. 44) 
 Cigarettes per day (48) Adjustment required for regular use (44) 
 Time to first cigarette of the day (48) Biggest adjustment (taste, lighting, keeping lit; ref. 44) 
 PREP awareness (unaided and aided; ref. 48) Expectations of future product innovations (98) 
 PREP trial (44) Pearson Novelty Experiencing Scale (98) 
 Quit attempts (i.e., in past year; reason; refs. 47, 98) Conservatism/Experimenting Scale (98) 
 Current quit intentions (47, 98) Price sensitivity vs health/emission benefits (98) 
 Stage of change (26) Credibility of benefits by type/source (98) 
 T/N of usual brand (lights, ultralights) Tastes different than cigarettes (44) 
 T/N levels of brand used pre/post-previous quit attempt (98) Lighting is different than cigarettes (44) 
 Previously switched to lights (26) Perceived benefits (less smoke, less odor, good taste; ref. 25) 
 Degree of health concern (48, 98) Perceived negatives (taste, hard to light, smokeless; ref. 25) 
 Beliefs about smoking causing cancer (48) Perception of reduced harm of PREPs (48) 
 Beliefs about filters or lights reducing harm (48) Belief that PREPs are regulated by government (47) 
 Disease type/site of highest concern (98) Belief that PREPs should be regulated by government (47) 
   
 Source of concern (product components; ref. 98) Among triers of PREP 
 Fitness/weight perceptions (98) Future purchase intentions (52, 58) 
 “Hypochondriasis” scale (98) Different from cigarettes (52) 
 Awareness of social pressure (98) Importance of differences (reasons) 
 In-home smoking policies (44, 45) Believability of claims of reduced risk (26) 
   
 Smoking restrictions at work (58) Among users of PREP 
 Exposure to second-hand smoke (47) Previous brand (58) 
 Spouse's smoking status (44) Length of time they smoked previous brand (58) 
 Children in household (58)  
 Age of onset/years smoked (26) First awareness (mail, advertisement, etc;* ref. 58) 
 Whether most of friends, relatives, coworkers smoke (58) Reason(s) for first trial* (58) 
  How first obtained trial product (friend, purchase, etc;* ref. 58) 
  Use of coupons (value, where obtained; ref. 58) 
  Amount of product purchased (58) 
  Price of product at purchase (58) 
  Packs used (26) 
  Duration of PREP use (i.e., weeks; ref. 26) 
  Pack rating* (58) 
  Reactions to instruction booklet* (58) 
  Reactions to packaging* (58) 
  Advantages* (26, 58) 
  Disadvantages* (26, 58) 
  Whether PREP use is full time (58) 
  When smoke conventional cigarettes (58) 
  Whether number of cigarettes smoked has changed (26, 58) 
  Whether feel higher price is justified* (58) 
  Comments from other smokers* (58) 
  Comments from nonsmokers* (58) 
  Shared PREP with others* (58) 
  Whether friends use PREP* (58) 
  Reasons for using PREP* (58) 
  Overall rating* (58) 
  Likes* (58) 
  Dislikes* (58) 
  Taste rating* (58) 
  Aroma rating* (58) 
  Tar level* (58) 
  Ease of lighting* (58) 
  Feel in hand* (58) 
  Ease of draw* (58) 
  How know when it is out* (58) 
  Disposal* (58) 
Web survey Age (6) Perception of low tar (6) 
 Gender (6) Product expected to taste good (SCQ-A; ref. 6) 
 Race/ethnicity (6) Expected addictiveness (SCQ-A; ref. 6) 
 Smoking status (regular smokers, nonsusceptible nonsmokers, susceptible nonsmokers/experimenters; ref. 6) Expected stimulation (SCQ-A; ref. 6) 
  Expected satisfaction (SCQ-A; ref. 6) 
  Expectation of difficulty to quit using product (SCQ-A; ref. 6) 
  Product expected to cause cancer (SCQ-A; ref. 6) 
  Expectation that product would suppress appetite [SCQ-A, Smoking Effects Questionnaire (SEQ); ref. 6] 
  Expectation that product would cause coughing (SCQ-A; ref. 6) 
  Fun (6) 
  Exciting (6) 
  Dangerous (6) 
  Interesting (6) 
  Stupid (6) 
  Bad breath (6) 
  Sophisticated (SEQ; ref. 6) 
  Feminine (SEQ; ref. 6) 
  Macho (SEQ; ref. 6) 
  Friends would like (6) 
  Good with a drink (6) 
  Worth trying (6) 
  Expectation that product would smell good (6) 
  Expectation that product would cause nausea (6) 
  Perception of PREP as a “kids' cigarette” (6) 
  Expectation of mildness (6) 
  Mature (6) 
  Expectation of harshness (6) 
  Perception of PREP as being for older smokers (6) 
  Expectation of strength (6) 
Test market selection criteria Geographic dispersion (40, 41)  
 Average low tar representation (40, 41)  
 Average discount brand representation (40, 41)  
 Average company (i.e., RJR) representation (40, 41)  
 Advertising not present if product unavailable (40, 41)  
 Ease of product sales tracking (40, 41)  
 Exclusion of other test market activity (by company or competitors; refs. 40, 41)  
 Exclusion of high levels of smokingrestrictions (40, 41)  
 Exclusion of smoking litigation in area (40, 41)  
Method of assessmentSubject characteristicsOutcome variables/measures
Focus group Age (24, 94) Prior awareness of PREP (95) 
 Gender (24, 94) First awareness of PREP (23, 24, 94) 
 Race/ethnicity (24, 94) Reasons for first trial (24, 94) 
 Geographic region (24, 94) First impressions upon trial (24) 
 Age of onset/years smoked (24, 94) Whether they would recommend PREP to others (24, 94) 
 Previous use of PREP (24) Whether health claims would influence trial (94) 
 Children living at home (24) Factors that would influence continued use (23, 24, 94) 
  Influence of quit intentions on willingness to switch (94) 
  Dual use vs switching (94) 
  Reaction to Surgeon General's warning (94) 
  Perceived benefits (personal, social; refs. 23, 95) 
  Perceived drawbacks (23) 
  Anticipated taste/sensation (95) 
  Reaction to instructions (95) 
  Reaction to name and pack design (95) 
  Reaction to “cleaner smoking” products (95) 
  Price expectations (95) 
  Type of person to use product (95) 
Face-to-face interview (public intercept or door-to-door canvassing) Age (i.e., 21-34 y, 35-49 y, 50+ y; refs. 25, 34-36, 39) Brand awareness (34) 
 Gender (34-36, 39) First awareness (mail, advertisement, etc; ref. 34) 
 Education level (35, 36, 39) Ever used product (34) 
 Race/ethnicity (35, 36, 39) Source of trial product (34) 
 Tar/Nicotine of usual brand (35) Purchase interest (28, 96) 
 Cigarettes per day (35) Repurchase of PREP after trial (34) 
 Usual brand menthol/nonmenthol (25) Perceived benefits (28, 34) 
 Usual brand price category (25) Different from cigarettes (28, 96) 
 Appearance (i.e., “consistent with clientele of venue”; ref. 34) Importance of perceived differences (28, 96) 
 Quit attempt in past year (35) Taste/claim trade-off (96) 
 Scores on Fagerstrom Test for Nicotine Dependence (36) Price/claim trade-off (96) 
  Situations where smoking is restricted (96) 
  Switching behavior (price, brand; ref. 96) 
  Perceived toxicity (scale 0-10; ref. 39) 
  Belief whether switching would expose smoker to fewer cancer-causing chemicals (y/n; refs. 36, 39) 
  Perception of how many cancer-causing chemicals reduced (all, most, some, none; refs. 35, 39) 
  Perceived health risk of PREP (i.e., scale 0-10; refs. 35, 39) 
  Whether switching would reduce smokers' chance of cancer (y/n; ref. 39) 
  Perception of addictiveness of PREP (36) 
  Need for understanding of product benefits (36) 
Telephone survey (RDD) Age (34, 45, 48, 53, 97) General awareness of PREPs (47, 48) 
 Gender (34, 45, 48, 53, 97) Awareness of specific PREP brands (unaided recall; refs. 47, 48) 
 Race/ethnicity (48) Ever use of PREP (44) 
 Socioeconomic status (34, 44, 45, 48, 97) First awareness (mail, advertisement, etc; ref. 44) 
 Education (26) Interest in trial (47) 
 Occupation (58) First trial (purchase, friend, etc; ref. 44) 
 Geographic region (34) Number of sticks used (44) 
 Usual brand T/N (53, 98) Read instructions before trying (y/n; ref. 44) 
 Cigarettes per day (48) Adjustment required for regular use (44) 
 Time to first cigarette of the day (48) Biggest adjustment (taste, lighting, keeping lit; ref. 44) 
 PREP awareness (unaided and aided; ref. 48) Expectations of future product innovations (98) 
 PREP trial (44) Pearson Novelty Experiencing Scale (98) 
 Quit attempts (i.e., in past year; reason; refs. 47, 98) Conservatism/Experimenting Scale (98) 
 Current quit intentions (47, 98) Price sensitivity vs health/emission benefits (98) 
 Stage of change (26) Credibility of benefits by type/source (98) 
 T/N of usual brand (lights, ultralights) Tastes different than cigarettes (44) 
 T/N levels of brand used pre/post-previous quit attempt (98) Lighting is different than cigarettes (44) 
 Previously switched to lights (26) Perceived benefits (less smoke, less odor, good taste; ref. 25) 
 Degree of health concern (48, 98) Perceived negatives (taste, hard to light, smokeless; ref. 25) 
 Beliefs about smoking causing cancer (48) Perception of reduced harm of PREPs (48) 
 Beliefs about filters or lights reducing harm (48) Belief that PREPs are regulated by government (47) 
 Disease type/site of highest concern (98) Belief that PREPs should be regulated by government (47) 
   
 Source of concern (product components; ref. 98) Among triers of PREP 
 Fitness/weight perceptions (98) Future purchase intentions (52, 58) 
 “Hypochondriasis” scale (98) Different from cigarettes (52) 
 Awareness of social pressure (98) Importance of differences (reasons) 
 In-home smoking policies (44, 45) Believability of claims of reduced risk (26) 
   
 Smoking restrictions at work (58) Among users of PREP 
 Exposure to second-hand smoke (47) Previous brand (58) 
 Spouse's smoking status (44) Length of time they smoked previous brand (58) 
 Children in household (58)  
 Age of onset/years smoked (26) First awareness (mail, advertisement, etc;* ref. 58) 
 Whether most of friends, relatives, coworkers smoke (58) Reason(s) for first trial* (58) 
  How first obtained trial product (friend, purchase, etc;* ref. 58) 
  Use of coupons (value, where obtained; ref. 58) 
  Amount of product purchased (58) 
  Price of product at purchase (58) 
  Packs used (26) 
  Duration of PREP use (i.e., weeks; ref. 26) 
  Pack rating* (58) 
  Reactions to instruction booklet* (58) 
  Reactions to packaging* (58) 
  Advantages* (26, 58) 
  Disadvantages* (26, 58) 
  Whether PREP use is full time (58) 
  When smoke conventional cigarettes (58) 
  Whether number of cigarettes smoked has changed (26, 58) 
  Whether feel higher price is justified* (58) 
  Comments from other smokers* (58) 
  Comments from nonsmokers* (58) 
  Shared PREP with others* (58) 
  Whether friends use PREP* (58) 
  Reasons for using PREP* (58) 
  Overall rating* (58) 
  Likes* (58) 
  Dislikes* (58) 
  Taste rating* (58) 
  Aroma rating* (58) 
  Tar level* (58) 
  Ease of lighting* (58) 
  Feel in hand* (58) 
  Ease of draw* (58) 
  How know when it is out* (58) 
  Disposal* (58) 
Web survey Age (6) Perception of low tar (6) 
 Gender (6) Product expected to taste good (SCQ-A; ref. 6) 
 Race/ethnicity (6) Expected addictiveness (SCQ-A; ref. 6) 
 Smoking status (regular smokers, nonsusceptible nonsmokers, susceptible nonsmokers/experimenters; ref. 6) Expected stimulation (SCQ-A; ref. 6) 
  Expected satisfaction (SCQ-A; ref. 6) 
  Expectation of difficulty to quit using product (SCQ-A; ref. 6) 
  Product expected to cause cancer (SCQ-A; ref. 6) 
  Expectation that product would suppress appetite [SCQ-A, Smoking Effects Questionnaire (SEQ); ref. 6] 
  Expectation that product would cause coughing (SCQ-A; ref. 6) 
  Fun (6) 
  Exciting (6) 
  Dangerous (6) 
  Interesting (6) 
  Stupid (6) 
  Bad breath (6) 
  Sophisticated (SEQ; ref. 6) 
  Feminine (SEQ; ref. 6) 
  Macho (SEQ; ref. 6) 
  Friends would like (6) 
  Good with a drink (6) 
  Worth trying (6) 
  Expectation that product would smell good (6) 
  Expectation that product would cause nausea (6) 
  Perception of PREP as a “kids' cigarette” (6) 
  Expectation of mildness (6) 
  Mature (6) 
  Expectation of harshness (6) 
  Perception of PREP as being for older smokers (6) 
  Expectation of strength (6) 
Test market selection criteria Geographic dispersion (40, 41)  
 Average low tar representation (40, 41)  
 Average discount brand representation (40, 41)  
 Average company (i.e., RJR) representation (40, 41)  
 Advertising not present if product unavailable (40, 41)  
 Ease of product sales tracking (40, 41)  
 Exclusion of other test market activity (by company or competitors; refs. 40, 41)  
 Exclusion of high levels of smokingrestrictions (40, 41)  
 Exclusion of smoking litigation in area (40, 41)  

*Also asked of triers of PREP.

Methods for Assessing Response to Product

Field Testing

One-Stick Studies

Although not found in the independent literature, it is common for tobacco companies to measure consumer perceptions after a participant has sampled a product following a single, rather than repeated, use. “One-stick” studies are conducted among targeted consumers before large-scale consumer testing (55). Industry validation of the one-stick method compared with longer trials (i.e., studies where respondents smoke one to two packs before being asked to evaluate the product) revealed that one-stick tests are valid methods for widespread, quick screening of products for further development work. However, these tests are less valid for certain attributes (i.e., harshness, blend characteristics), where sensory characteristics become perceptible only after several cigarettes (56). They are occasionally administered within focus groups, public intercept surveys, or other concept/product testing methods, and measure smokers' opinions and reactions to the product after smoking a single sample of the product (approximate n = 300; ref. 57).

Before sampling the product, smokers may be asked a series of questions regarding their perceptions of the brand based on prior awareness and/or exposure to an advertisement. Measures of perceptions specific to PREPs include likes, dislikes, taste rating, aroma rating, tar level, ease of lighting, feel in the hand, ease of draw, extinguishing cues (i.e., “how they know when it's out”), ease of disposal, reactions to accompanying instruction booklet, and reactions to packaging (58). For a prototype to be recommended for further acceptance testing, it must achieve significantly higher ratings on overall taste preference and on most monadic taste ratings (59).

In-home Product Testing

Tobacco companies have provided prerecruited respondents with product samples to use at home for a limited period of time. Some studies provide a single pack of the test product (32), whereas others entail longer term use (i.e., for 3days to 1 week) or more complex switching study designs. In a switching study, two products (the test product and a standard control) are given to participants to compare over the course of the study period (60, 61), and usually the products are “blinded” whenever possible to avoid response bias (46). Before the in-home phase, subjects may complete a questionnaire on ideal product ratings to assess baseline sensory preferences (62). Participants provide ratings after using the product in an environment that is more naturalistic than the smoking panel study design.

Products may be mailed or personally distributed to subjects at a research site. Advantages of mailing products for the in-home testing method, compared with laboratory-based sensory panels, include the ability to test a larger number of subjects, potentially over a wider geographic area. However, this method has distinct disadvantages, including a longer turnaround time before receiving results, researchers' loss of control over a subject's smoking behavior or use of the product, a substantial commitment by the subject, and the possibility that test products may be shared with nonsubjects (63). In-home product tests that require subjects to return to a laboratory or other research site for follow-up visits provide the opportunity for longer product use as with mailed in-home product tests, but ensure faster turnaround time, lower likelihood of attrition due to the fact that subjects return to the research facility for payment, and the opportunity for sensory or other laboratory-based procedures at follow-up (63).

Final sample sizes range from 100 to 350 complete responses per test product, with an expected response rate of about 50% (60, 63-67). Participants rate products for overall satisfaction, purchase interest, and on key sensory attributes, in a process known as monadic testing (68). Subjects are presented with one test product at a time and asked to rate them on selected attributes, rather than smoking a number of products at once and comparing them (69). These ratings are compared with subjects' initial ideal product ratings from the beginning of the study. Although respondents may be instructed to use the test product in lieu of their regular cigarette brand, follow-up questionnaires do not necessarily measure smoking behaviors such as frequency and combined use (62).

Laboratory-Based Assessments

Sensory Evaluation Panels

Sensory evaluation panels are used to measure the influence of specific product design characteristics on sensory responses. These properties may include taste and aftertaste, odor, tactile properties of product use (e.g., mouthfeel; “kick” or “bite”), tactile properties of the product itself (e.g., ease of lighting, firmness of rod), and strength (amount of nicotine content or delivery). This testing is typically conducted by tobacco companies before broader scale acceptance testing (70). The tobacco industry often recruits among a company's own employees for sensory evaluation panels, although these panelists are also recruited from the community (71-75). Panels typically consist of 15 to 50 participants who have been trained to use descriptive terms (i.e., taste and aroma) in specified ways when evaluating cigarettes (70). A controlled “triangle” testing strategy has been used in industry research to control for the potential for a given product to “mask” the sensory effects of the next product sampled. Under this regimen, three products are presented (two identical and one different) and participants are asked to identify and rate the one that differed. Order of presentation is varied in five combinations (i.e., AAB, ABA, BAA, BBA, BABA).

Quantitative Descriptive Analysis

Quantitative descriptive analysis is conducted by tobacco industry laboratories also using trained sensory evaluation panelists (76). Prospective panelists are recruited based on “verbal articulation, social interaction, and most importantly, sensory acuity.” Approximately 30% to 40% of respondents are invited to be trained as panelists. Ten, 1-hour training sessions are held over a period of 2 weeks, where panelists are taught to identify sensory terminology. Development of a common set of terms describing taste and aroma characteristics, and associated definitions, are agreed to by the panel (77). Panelists rate the sensory attributes of each product on a structured questionnaire known as a “ballot.” Before each study, a warm-up session is conducted to expose the panelist to a subset of test products to calibrate the sensory scales and, if necessary, add additional terms to the ballot (77). Overall acceptance of the product can be measured using a structured scale (i.e., 0, dislike a lot; 50, indifferent; 100, like a lot) often called a “thermometer rating” (Fig. 2A; ref. 78), and recording perceptions of a variety of sensory attributes on a computer-administered unstructured line scale (Fig. 2B; ref. 79). For each study, two to six products are assessed, with no more than six products evaluated within a 1-week period. Three replicate evaluations of each product are made by the panelists, with a randomized complete block design being implemented to prevent order bias (77).

Figure 2.

A. Thermometer rating scale for overall acceptance (78). B. Unstructured line scale for sensory attribute measurement (79).

Figure 2.

A. Thermometer rating scale for overall acceptance (78). B. Unstructured line scale for sensory attribute measurement (79).

Close modal

Independent Sensory and Subjective Response Assessment

Studies by independent scientists also have incorporated sensory assessment into studies of smoking behavior, although the focus of this research has tended to be on measures of nicotine effect, smoking urge, and withdrawal relief and abuse liability rather than consumer acceptability. In an earlier study of denicotinized cigarettes (80), subjects were asked to rate sensory and reinforcing properties of the products, with anchor points ranging from 0 to 100 reflecting extreme responses to each question (i.e., 0, “no taste”; 100, “a lot of taste”). A study examining differences in subjective effects between moist snuff, medicinal nicotine lozenge, and three smokeless tobacco lozenge PREPs measured drug effects and liking through a 17-item scale ranging from 1 to 10 (“not at all” to “extremely”; ref. 81). Items include good drug effect, harshness of cigarette, heaviness of cigarette, desire to smoke the cigarette just smoked, intensity of flavor, and freshness of cigarette (82). The same drug effect and liking measures were used to compare a carbon-filtered PREP and conventional control cigarettes (83). The carbon-filtered PREP was rated as having less drug effect despite similar smoke machine nicotine yield, and was less liked than the conventional product.

Independent scientists have also assessed the influence of the sensory attributes of PREPs on use behavior and product adoption. These studies have involved switching designs, in which subjects are randomly assigned to PREP and non-PREP comparison conditions. Comparison conditions have included nicotine replacement inhalers (84, 85), nicotine lozenge (81), subjects' regular cigarette brand (4, 80, 86), a light cigarette brand (80), moist snuff (81), or cessation (86). Sample sizes in these studies have ranged from 10 (4, 80, 81) to 50 (38, 84, 85). Subjects may be enrolled based on their smoking status (i.e., regular brand is light or ultralight; refs. 4, 86) and/or level of nicotine dependence (4). Smokers may be asked to try a product in a single laboratory session (82), a series of laboratory sessions (2-5; refs. 4, 81), or completely have a forced switch over the course of days (3-5) or weeks (2-6) per product (80, 84, 86).

Studies have also measured the capacity for PREPs to relieve urge to smoke and symptoms of nicotine withdrawal, compared with conventional cigarettes. In laboratory-based smoking experiments, a 3-item craving/satisfaction measure has been used after each puff, as well as a separate 4-item measure at 15-minute increments in the 90 minutes following smoking, and a widely used 11-item tobacco withdrawal scale (87) may be administered after 90 minutes of deprivation (see Table 4 for further information on measures; refs. 4, 80, 81, 86). These scales may use a visual analogue scale, with items consisting of a word or phrase above a horizontal line anchored by phrases such as “not at all” and “extremely.” The Questionnaire on Smoking Urges measures intention to smoke and anticipation of relief from withdrawal (4, 86, 88). Questionnaires are administered immediately before of after the product is presented to subjects (81).

Table 4.

Subject characteristics and outcomes variables used for CR to product, by method of assessment

Method of assessmentSubject characteristicsOutcome variables/measures
Field testing Age (99) Overall acceptance (thermometer) rating (99) 
 Gender (99) Perceived tar/nicotine (28) 
 Consumption (cigarette/day; ref. 99) Positive attributes (open-ended response; ref. 99) 
 Usual brand (99) Negative attributes (open-ended response; ref. 99) 
 T/N of usual brand (99) Willingness to use again (99) 
  Situations where/when PREP would be used (99) 
  Description of “typical user” (gender, age, life-style; ref. 99) 
  Different from cigarettes (57) 
  Importance of perceived differences (57) 
  Health perceptions (less harmful, less addicting; ref. 28) 
  Liking of taste (28, 99) 
  Natural tobacco taste (28, 99) 
  Smoking pleasure (99) 
  Mildness (28, 99) 
  Spiciness (99) 
  Dryness (99) 
  Perfuming (99) 
  Bitter (99) 
Sensory and subjective responses Age (4, 59, 80-82, 84, 86) No. of packs used (84, 100) 
 Gender (4, 59, 80-82, 86) Similarity to regular cigarette or own brand (50, 82) 
 Race/ethnicity (82) Purchase intent (50, 80, 100) 
 Number of cigarettes smoked per day (82) Subjective effects 
 Number of years smoking (82) Heart rate (82) 
 Usual brand menthol or nonmenthol (80, 82, 86) Blood pressure (diastolic and systolic; ref. 82) 
 T/N of usual brand (4, 86) Carbon monoxide (82) 
 Cigarettes per day (86) Good drug effect (82) 
 Interest in quitting (/no interest; ref. 80) Bad drug effect (82) 
  Drug strength, product effects, satisfaction from last puff (80-82, 100) 
  Drug liking (82) 
  Desire to smoke your regular cigarette (4, 80, 82, 86) 
  Desire to smoke product you just smoked (82) 
  Relaxed (82) 
  Dizziness (82) 
  Enjoyment (82) 
  Stimulated (82) 
  Symptoms of tobacco withdrawal (Hughes & Hatsukami, 1986) 
  Irritability/frustration/anger (4, 80, 81, 86) 
  Anxious (4, 80, 81, 86) 
  Difficulty concentrating (4, 80, 81, 86) 
  Restlessness (4, 80, 81, 86) 
  Hunger (4, 80, 86) 
  Impatient (4, 80, 81, 86) 
  Craving cigarette/nicotine(4, 80, 81, 86) 
  Insomnia/disturbed sleep (4, 86) 
  Increased eating (4, 86) 
  Drowsiness (4, 80, 86) 
  Depression/feeling blue (4, 80, 81, 86) 
  Desire for sweets (4, 80, 86) 
  Urges/Craving (Questionnaire of Smoking Urges; ref. 80) 
  Intention to smoke (Questionnaire of Smoking Urges; ref. 4) 
  Anticipation of relief from withdrawal (Questionnaire of Smoking Urges; refs. 4, 80) 
  Sensory effects 
  Overall acceptance (thermometer) rating, product liking (50, 80, 81, 100) 
  Ease of lighting (70, 100) 
  Draw (55, 80, 100) 
  Strength (50, 55, 80, 100) 
  Harshness (50, 55, 80, 82) 
  Smoothness (100) 
  Coolness (82) 
  Freshness (82) 
  Dryness (82) 
  Heaviness (82) 
  Tobacco taste (55, 82) 
  Quality of flavor (82) 
  Intensity of flavor (82) 
  Artificial taste (55) 
  Pleasant taste (100) 
  Pleasant aftertaste (50) 
  Staying lit (100) 
  Number of puffs (100) 
  Heat on lips (80, 100) 
  Length of time lasted (compared with usual brand; refs. 55, 100) 
  Product extinguished unexpectedly (100) 
  Negative taste (strong, weak, old; ref. 100) 
  Difficult to light, keep lit (100) 
  Difficult to draw (100) 
  Product strength (80) 
  Air dilution (80) 
  Identification of PREPs with different levels of nicotine (38) 
In-home product testing Age (65) Overall acceptance (thermometer), liking (60, 62, 67) 
 Usual brand (65) Purchase intent (60, 62) 
 Purchase intent (65) Frequency of use of test product (62) 
 Amount smoked per week (67) Preference vs regular brand (62) 
 Type of cigarette smoked (full flavor, light, ultralight; ref. 67) Satisfaction (62) 
  Strength (66) 
  Tobacco taste (66) 
  Harshness (66) 
  Smoothness (66) 
  Mildness (66) 
  Draw (66) 
  Woody taste (66) 
  Artificial taste (66) 
  Papery (66) 
  Coffee taste (66) 
  Fresh (66) 
  Strong (66) 
  Drying (66) 
  Long-lasting (66) 
  Sweet (66) 
Method of assessmentSubject characteristicsOutcome variables/measures
Field testing Age (99) Overall acceptance (thermometer) rating (99) 
 Gender (99) Perceived tar/nicotine (28) 
 Consumption (cigarette/day; ref. 99) Positive attributes (open-ended response; ref. 99) 
 Usual brand (99) Negative attributes (open-ended response; ref. 99) 
 T/N of usual brand (99) Willingness to use again (99) 
  Situations where/when PREP would be used (99) 
  Description of “typical user” (gender, age, life-style; ref. 99) 
  Different from cigarettes (57) 
  Importance of perceived differences (57) 
  Health perceptions (less harmful, less addicting; ref. 28) 
  Liking of taste (28, 99) 
  Natural tobacco taste (28, 99) 
  Smoking pleasure (99) 
  Mildness (28, 99) 
  Spiciness (99) 
  Dryness (99) 
  Perfuming (99) 
  Bitter (99) 
Sensory and subjective responses Age (4, 59, 80-82, 84, 86) No. of packs used (84, 100) 
 Gender (4, 59, 80-82, 86) Similarity to regular cigarette or own brand (50, 82) 
 Race/ethnicity (82) Purchase intent (50, 80, 100) 
 Number of cigarettes smoked per day (82) Subjective effects 
 Number of years smoking (82) Heart rate (82) 
 Usual brand menthol or nonmenthol (80, 82, 86) Blood pressure (diastolic and systolic; ref. 82) 
 T/N of usual brand (4, 86) Carbon monoxide (82) 
 Cigarettes per day (86) Good drug effect (82) 
 Interest in quitting (/no interest; ref. 80) Bad drug effect (82) 
  Drug strength, product effects, satisfaction from last puff (80-82, 100) 
  Drug liking (82) 
  Desire to smoke your regular cigarette (4, 80, 82, 86) 
  Desire to smoke product you just smoked (82) 
  Relaxed (82) 
  Dizziness (82) 
  Enjoyment (82) 
  Stimulated (82) 
  Symptoms of tobacco withdrawal (Hughes & Hatsukami, 1986) 
  Irritability/frustration/anger (4, 80, 81, 86) 
  Anxious (4, 80, 81, 86) 
  Difficulty concentrating (4, 80, 81, 86) 
  Restlessness (4, 80, 81, 86) 
  Hunger (4, 80, 86) 
  Impatient (4, 80, 81, 86) 
  Craving cigarette/nicotine(4, 80, 81, 86) 
  Insomnia/disturbed sleep (4, 86) 
  Increased eating (4, 86) 
  Drowsiness (4, 80, 86) 
  Depression/feeling blue (4, 80, 81, 86) 
  Desire for sweets (4, 80, 86) 
  Urges/Craving (Questionnaire of Smoking Urges; ref. 80) 
  Intention to smoke (Questionnaire of Smoking Urges; ref. 4) 
  Anticipation of relief from withdrawal (Questionnaire of Smoking Urges; refs. 4, 80) 
  Sensory effects 
  Overall acceptance (thermometer) rating, product liking (50, 80, 81, 100) 
  Ease of lighting (70, 100) 
  Draw (55, 80, 100) 
  Strength (50, 55, 80, 100) 
  Harshness (50, 55, 80, 82) 
  Smoothness (100) 
  Coolness (82) 
  Freshness (82) 
  Dryness (82) 
  Heaviness (82) 
  Tobacco taste (55, 82) 
  Quality of flavor (82) 
  Intensity of flavor (82) 
  Artificial taste (55) 
  Pleasant taste (100) 
  Pleasant aftertaste (50) 
  Staying lit (100) 
  Number of puffs (100) 
  Heat on lips (80, 100) 
  Length of time lasted (compared with usual brand; refs. 55, 100) 
  Product extinguished unexpectedly (100) 
  Negative taste (strong, weak, old; ref. 100) 
  Difficult to light, keep lit (100) 
  Difficult to draw (100) 
  Product strength (80) 
  Air dilution (80) 
  Identification of PREPs with different levels of nicotine (38) 
In-home product testing Age (65) Overall acceptance (thermometer), liking (60, 62, 67) 
 Usual brand (65) Purchase intent (60, 62) 
 Purchase intent (65) Frequency of use of test product (62) 
 Amount smoked per week (67) Preference vs regular brand (62) 
 Type of cigarette smoked (full flavor, light, ultralight; ref. 67) Satisfaction (62) 
  Strength (66) 
  Tobacco taste (66) 
  Harshness (66) 
  Smoothness (66) 
  Mildness (66) 
  Draw (66) 
  Woody taste (66) 
  Artificial taste (66) 
  Papery (66) 
  Coffee taste (66) 
  Fresh (66) 
  Strong (66) 
  Drying (66) 
  Long-lasting (66) 
  Sweet (66) 

Evidence from both industry and independent research has confirmed the utility of conceiving tobacco CR as comprised of two distinct but related domains: product perceptions and response to product. Product perceptions, which encompass consumers' awareness, attitudes and beliefs to product concept, marketing, and/or messaging, are typically assessed via multiple measures that are contained in a structured custom-designed survey. In contrast, responses to the product—usually determined through actual use—involve measurement of sensory responses (including taste and aftertaste, mouth feel, and bite/kick), other subjective responses (including nicotine effect, urge, and withdrawal relief), and ratings of product acceptability, using sensory panels or switching study designs. The highly varied nature of the components that comprise CR has produced a broad array of methods, which are chosen according to the specific questions being addressed. Methods used by the tobacco industry fall into three major areas: concept testing, messaging studies, and sensory assessment. In contrast, independent research on PREPs is conducted postmarket, after the product has been refined by industry CR research, and includes assessment of product messaging and sensory and subjective responses. The findings from independent studies are used to inform the scientific community on the public health implications of PREPs and new products. Further work should guide the development of public health interventions and regulation of PREPs and new products.

The range of research methods identified provides a potential basis for developing a systematic set of PREP CR assessment strategies. Comprehensive listings of available information on CR outcome measures used in research on PREPs are provided, and are organized according to the two broad domains of product perception (Table 3) and response to product (Table 4).

Research on product perceptions generally makes use of custom-designed, large scale surveys. Telephone surveys are a standard methodology for tracking trends in risk factors and behaviors over time, evaluating the effects of interventions, and examining cross-sectional and longitudinal characteristics of target populations. The future of random-digit dial surveys is compromised by the decrease in landline telephone coverage due to the popularity of cell phones, as well as technology such as voice mail or call screening that reduces the likelihood of reaching participants directly. Mainstream scientists have developed strategies for addressing these methodologic challenges, including using multimode designs, incorporating cell phone users into sampling frames or applying weights based on the demographic characteristics of the cell phone users, using text messaging and voice mail, and sending advance letters and scheduling calls (89). Web surveys have become an increasingly popular tool among mainstream researchers, because of their capacity to rapidly recruit and screen participants, present visual information, and collate survey data. Nevertheless, adapting telephone surveys to a Web-based format requires consideration about issues about recruitment of random samples, question format, and visual design (90). The independent scientific community is continually testing Web-based measurement issues and their effects on survey error, but the method may present considerable benefits to researchers. In addition to being cost effective, researchers have found that in instances when respondents are asked to report sensitive information, computer-based surveys may yield more accurate data (90). Intercept surveys, particularly in malls and other public places, have been a popular survey method among commercial marketing researchers since the 1970s. Issues of validity and reliability arise in public intercept surveys, whereas more traditional door-to-door survey methodologies allow randomization and stratification, public intercept surveys are subject to response bias yielding a potentially nonrepresentative sample (91). Intercept surveys nonetheless provide a low-cost option for researchers seeking to directly expose consumers to a product and obtain immediate feedback. However, specific locations may have a tendency to “burn out” after a period of time, if too many researchers repeatedly sample from the same pool of respondents (91).

Research aimed at assessing responses to product must provide smokers with an opportunity to sample the product, restricting research methods to laboratory or home-based studies. Tobacco industry and independent scientists take slightly different approaches. The tobacco industry considers its users' characteristics according to target markets and/or product preferences, whereas mainstream scientists consider subjects according to other measures commonly found in the published literature (i.e., socioeconomic status, addiction and abuse liability, smoking behavior). Although measures of sensory response are important in determining product acceptability, other measures are required. Scales used by the industry to measure overall acceptability and sensory response are useful for informing product developers (and those anticipating the likelihood of future adoption) because they help to determine why consumers may or may not find a product appealing (i.e., low overall liking scores due to a product being too light; ref. 92).

Such scales require further refinement by independent scientists to confirm the validity of key sensory constructs and their predictive relationship with product acceptability and future intentions for trial. In this context, a measure can be validated through demonstration of concordance with widely understood or accepted examples of the construct in question (“face validity”), or by demonstrating concordance with an established measure. In addition, however, validation of measures requires attention to the purpose of the assessment and the special circumstances under which it might be applied. Such circumstances include the type of product tested, the claims made or implied by the manufacturer or the product design, and special populations to whom assessment is directed. The reliability of sensory scales also requires empirical demonstration.

Strategies to assess CR should, of course, be guided by their appropriateness for providing answers to a specific research question. Sets of questions or response items generally have been developed in response the specific demands of a research program and are generally not standardized for general use.

Further research is needed to develop a more complete understanding of CR and methods for its assessment. In particular, research to better understand the relationship between positive CR components and use behavior, especially the progression from trial use to patterns of established use, is needed. Studies that assess the effect of manipulating product information on risk perceptions and use behavior are also required. Likewise, manipulation of product design, emissions on CR, and use behavior outcomes would be valuable. The proposed inter-relationship between product perceptions and response to product has not been systematically investigated by industry or independent scientists. Further investigation of the relationships between product perceptions and response is needed, and future research on CR should, where possible, address these dual domains. This article provided a descriptive overview of CR methods used in both independent and industry research; however, a comparison of the effectiveness of methods for demonstrating specific outcomes is still needed. CR assessment methods should be examined and compared for validity, particularly according to type of PREP and the range of claims that could be presented. PREPs such as Omni have referred to reduced cancer risk in messaging materials, whereas Eclipse has also referred to reduced risk for respiratory illness. Other products, such as Camel Snus, present information about the convenience of the product and pleasure associated with its use. Further research is also required on consumer perceptions of implicit product messages, and the relationship between the consumer's understanding of actual message and the influence on risk-benefit ratio.

Research has shown that implicit messages are perceived similarly to explicit messages, which further underscores the need for research in this area. Looking more broadly, the applicability of existing or future methods for special populations requires investigation. Currently, PREPs have been shown to have greater appeal to women, older smokers, exsmokers, and health conscious smokers (35, 93). Further research should address PREP interest and use among ethnic and racial minorities and youth. Finally, the changing nature of PREP and new product design, marketing strategies, and consumer needs means that assessment methods must be able to respond to the “moving target” by adapting appropriately, but while maintaining empirical validity.

Strategies for comprehensive assessment of CR have previously been proposed by Stratton (2001) and Hatsukami et al. (2005; ref. 5). The present review provides a basis for establishing methods and measures for assessing CR to PREPs using a conceptual model. This model allows testing of predicted CR outcomes that may arise through future modification of product design or messaging. The tobacco industry has focused on CR assessment to ensure that its products meet with market success. Industry strategy now appears intent not only on successfully modifying product design and messaging, but tailoring those modifications to specific subgroups of smokers, such as youth, women, and racial/ethnic minorities (2). The proposed model of CR may provide a basis for anticipating CR among subgroups by accounting for social norms and personal factors such as tobacco use history and personal acceptability. The present model also provides a systematic strategy for those charged with regulation of tobacco product design and marketing, such as the Food and Drug Administration.

To ensure the commercial success of new PREPs, we can be sure that tobacco manufacturers will attempt to make those products acceptable alternatives for current smokers and may also enhance their appeal for new and extobacco users. Tobacco products that purport to reduce risk, whether or not they succeed in reducing human exposure to harmful toxicants, may perpetuate tobacco-related harms if their acceptance and use at a population level is enhanced. Consumer acceptance is conceived as the related components of product perceptions and sensory and other responses to product. In response, there has been a substantial research effort by the public health community to understand consumer perceptions and responses to product to PREP messaging and product design. As methods and measures for PREP assessment are adopted and standardized, their policy and regulatory function must be heeded. The present model provides an empirically based strategy for understanding and predicting CR, trial, and likelihood of future adoption of new PREPs. With further development of research methods, the model may provide a basis for understanding and predicting CRs of targeted smoker subgroups. This information is essential for effective regulation of new tobacco products by authorities such as Food and Drug Administration and states' attorneys general.

No potential conflicts of interest were disclosed.

1
Pollay
R
,
Dewhirst
T
. 
The dark side of marketing seemingly "Light" cigarettes: successful images and failed fact
.
Tob Control
2002
;
11
:
i18
31
.
2
Carpenter
C
,
Wayne
G
,
Connolly
G
. 
The role of sensory perception in the development and targeting of tobacco products
.
Addiction
2007
;
102
:
136
47
.
3
Wayne
G
,
Connolly
G
,
Henningfield
J
. 
Assessing internal tobacco industry knowledge of the neurobiology of tobacco dependence
.
Nicotine Tob Res
2004
;
6
:
927
40
.
4
Buchhalter
A
,
Eissenbert
T
. 
Preliminary evaluation of a novel smoking system: effects on subjective and physiological measures and on smoking behavior
.
Nicotine Tob Res
2000
;
2
:
39
43
.
5
Hatsukami
D
,
Giovino
A
,
Eissenberg
T
,
Clark
P
,
Lawrence
D
,
Leischow
S
. 
Methods to assess potential reduced exposure products
.
Nicotine Tob Res
2005
;
7
:
827
44
.
6
O'Connor
R
,
Ashare
R
,
Fix
B
,
Hawk
L
,
Cummings
K
,
Schmidt
W
. 
College students' expectancies for light cigarettes and potential reduced exposure products
.
Am J Health Behav
2007
;
31
:
402
10
.
7
Riedel
B
,
Robinson
L
,
Klesges
R
,
McLain-Allen
B
. 
What motivates adolescent smokers to make a quit attempt?
Drug Alcohol Depend
2002
;
68
:
167
74
.
8
Cummings
K
,
Hyland
A
,
Bansal
M
,
Giovino
G
. 
What do Marlboro Lights smokers know about low-tar cigarettes?
Nicotine Tob Res
2004
;
Suppl 3
:
323
32
.
9
Kozlowski
L
,
O'Connor
R
. 
Cigarette filter ventilation is a defective design because of misleading taste, bigger puffs, and blocked vents
.
Tob Control
2002
;
11
:
I40
50
.
10
Shiffman
S
,
Pillitteri
J
,
Burton
S
,
Rohay
JG
. 
Effect of health messages about "Light" and "Ultra Light" cigarettes on beliefs and quitting intent
.
Tob Control
2001
;
10
:
i24
32
.
11
Institute of Medicine
.
Clearing the smoke: Assessing the science base for tobacco harm reduction
.
Washington (DC)
:
National Academy Press
; 
2001
.
12
Rees
V
,
Connolly
G
.
Potentially reduced exposure tobacco products
.
Boston (MA)
:
Harvard School of Public Health
; 
2008
.
13
Rees
V
,
Wayne
G
,
Thomas
B
,
Connolly
G
. 
Physical design analysis and mainstream smoke constituent yields of the new potential reduced exposure product, Marlboro UltraSmooth
.
Nicotine Tob Res
2007
;
9
:
1197
206
.
14
Rep Waxman
HA
.
Family Smoking Prevention and Tobacco Control Act. H.R. 1256
. 
2009
.
15
Fishbein
M
,
Ajzen
I
.
Belief, attitude, intention and behavior: An introduction to theory and research
.
Reading (MA)
:
Addison-Wesley
; 
1975
.
16
Ajzen
I
,
Fishbein
M
.
Understanding attitudes and predicting social behavior
.
Englewood Cliffs (NJ)
:
Prentice-Hall
; 
1980
.
17
Pederson
L
,
Nelson
D
. 
Literature review and summary of perceptions, attitudes, beliefs and marketing of potentially reduced exposure products: Communication implications
.
Nicotine Tob Res
2007
;
9
:
525
34
.
18
Hawkins
SC
.
Perspective on concept-product and product-concept tests
.
RJ Reynolds
; 
1988f
,
Bates No. 507261406/1410
.
19
Kreuger
R
,
Casey
M
.
Focus Groups: A practical guide for applied research
. 3rd ed
Thousand Oaks (CA)
:
Sage Publications, Inc.
; 
2000
.
20
Caples
J
.
Tested advertising methods
. 5th ed.
Paramus (NJ)
:
Prentice Hall
; 
1997
.
21
Centers for Disease Control and Prevention
.
Designing and implementing an effective tobacco counter-marketing campaign
.
Atlanta (GA)
:
US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health
; 
2003
.
22
RJ Reynolds
.
Survey proposal
. 
1990
,
Bates #510484034
.
23
Brown & Williamson
.
Analytic Insight, Inc—Eclipse focus group Chattanooga
. 
1996
,
Bates #575391163/575391164
.
24
Carabello
R
,
Pederson
L
,
Gupta
N
. 
New tobacco products: do smokers like them?
Tob Control
2006
;
15
:
39
44
.
25
O'Hegarty
M
,
Richter
P
,
Pederson
LL
. 
What do adult smokers think about ads and promotional materials for PREPs?
Am J Health Behav
2007
;
31
:
526
34
.
26
Hughes
J
,
Keely
J
,
Callas
P
. 
Ever users versus never users of a "less risky" cigarette
.
Psychol Addict Behav
2005
;
19
:
439
42
.
27
Pratkanis
A
.
Report on research investigating the perception of health benefit claims in Eclipse marketing communications
.
University of California, Santa Cruz
; 
2006
,
Bates #552734992/552735140
.
28
Hunter
C
,
Weber
J
,
Tinsley
C
.
Marketing research proposal (BIAD #94-2353-19. Fourth quarter 1994 (940000) product claims development and screening research - prisms I
.
RJ Reynolds
; 
1994
,
Bates #510353297/510353300
.
29
Dennis & Co
.
Project EW communications test
.
RJ Reynolds
; 
1994
,
BIAD #94-11125. DAC #2204. Bates #513856979/513857147
.
30
RJ Reynolds
.
EW ad screening mall intercept screening questionnaire
. 
1994
,
Bates #514080617/514080648
.
31
Brace
I
.
Questionnaire design: How to plan, structure and write survey material for effective market research
.
Philadelphia (PA)
:
Kogan Page US
; 
2004
.
32
RJ Reynolds
.
Product positioning and development - status
. 
1986
,
Bates #505497712/505497721
.
33
Tobacco Institute
.
TI-Mall intercept master questionnaire
.
Undated. Bates #TIMN0030936-0030941
.
34
RJ Reynolds
.
Market research proposal. (97-6303-18). Atlanta consumer tracking - new methods
. 
1997
,
Bates #520584398/520584399
.
35
Hamilton
W
,
diStefano Norton
G
,
Ouellette
T
,
Rhodes
W
,
Kling
R
,
Connolly
G
. 
Smokers' responses to advertisements for regular and light cigarettes and potential reduced-exposure tobacco products
.
Nicotine Tob Res
2004
;
6
:
S353
62
.
36
Shadel
W
,
Lerman
C
,
Capella
J
,
Strasser
A
,
Pinto
A
,
Hornik
R
. 
Evaluating smokers' reactions to advertising for new lower nicotine Quest cigarettes
.
Psychol Addict Behav
2006
;
20
:
80
4
.
37
Cacioppo
J
,
Petty
R
,
Quintanar
L
. 
Individual differences in relative hemispheric α abundance and cognitive responses to persuasive communications
.
J Pers Soc Psychol
1982
;
43
:
623
36
.
38
Strasser
A
,
Lerman
C
,
Capella
J
. 
Lower nicotine cigarettes may not lower harm
.
LDI Issue Brief
2006
;
12
.
39
Biener
L
,
Bogen
K
,
Connolly
G
. 
Impact of corrective health information on consumers' perceptions of “reduced exposure” tobacco products
.
Tob Control
2007
;
16
:
306
11
.
40
Martin
C
.
Premier test market plan - II
.
RJ Reynolds
; 
1989
,
Bates #514099137/514099139
.
41
Shouse
D
.
Lead markets
.
RJ Reynolds
; 
1987
,
Bates #506403062/506403070
.
42
MARC
.
Database and tracker analysis
.
RJ Reynolds
; 
1995
,
Bates #514655933/514655957
.
43
Still
M
,
Etzel
E
,
Whitlach
W
, et al
.
Marketing research proposal (MRD #89-43502). TF test market research—awareness, usage, and perceptions study
.
RJ Reynolds
; 
1989
,
Bates #507339627/507339629
.
44
RJ Reynolds
.
Lincoln consumer tracking—wave I
. 
1997
,
Bates #519958365/519958372
.
45
RJ Reynolds
.
Indianapolis ad study for Eclipse
. 
1997
,
Bates #519958357/519958364
.
46
Benson
S
.
MPID consumer research group overview
.
Lorillard
; 
1998
,
Bates #86289074/86289086
.
47
Hund
L
,
Farrelly
M
,
Allen
J
, et al
. 
Findings and implications from a national study on potential reduced exposure products (PREPs)
.
Nicotine Tob Res
2006
;
8
:
791
7
.
48
O'Connor
R
,
Hyland
A
,
Giovino
A
,
Fong
G
,
Cummings
K
. 
Smoker awareness of and beliefs about supposedly less-harmful tobacco products
.
Am J Prev Med
2005
;
29
:
85
90
.
49
Parascadola
M
,
Hurd
A
,
Augustson
E
. 
Consumer awareness and attitudes related to new potential reduced-exposure tobacco products
.
Am J Health Behav
2008
;
32
:
431
7
.
50
Philip Morris
.
SCoR program: carbon filter technology
. 
2005
,
Bates #3006670071/3006670162
.
51
RJ Reynolds
.
MDD perspective on Project SPA test marketing
. 
1986
,
Bates #506036242/506036266
.
52
RJ Reynolds
.
Project SPA research program
. 
1988
,
Bates #506432640/506432649
.
53
Bellis
J
.
Market research proposal (96-6303-36). Direct marketing product test
.
RJ Reynolds
; 
1997
,
Bates #520192383/520192386
.
54
Freeman
BC
. 
S. Open Source Marketing: Camel cigarette brand marketing in the Web 2.0 world
.
Tob Control
2009
,
[Epub ahead of print.]
.
55
RJ Reynolds
.
Reduced CO product test
. 
1994
,
Bates #514304798/514304809
.
56
RJ Reynolds
.
1989 (890000) consumer research summaries annual report
. 
1989
,
Bates #:516411847/516412071
.
57
Gurry
N
.
Nicotine-extracted ad/one stick study
.
Philip Morris
; 
1989
,
Bates #2021503949/2021503951
.
58
Philip Morris
.
Areas of questioning for Premier tracking
. 
1988
,
Bates #2040929857/2040929859
.
59
Brown
A
.
Consumer and sensory product research. Non-menthol Eclipse alternate substrates: Acceptance panel evaluation. Consumer acceptance evaluation. Final report
.
RJ Reynolds
; 
1997
,
Bates #522144121/522144133
.
60
Phillips
A
,
Dryden
M
,
Leary
E
,
Potter
D
,
Weber
J
,
Swaim
M
.
Proposal. MM: Winston select product test
.
RJ Reynolds
; 
1995
,
Bates #513950744/513950746
.
61
Day
D
.
New flavor prototypes in-home product testing
.
Lorillard
; 
1999
,
Bates #82341351/82341353
.
62
American Tobacco
.
American Tobacco, new Lucky Strike filters in-home product use test
. 
1981
,
Bates #990891741/990891744
.
63
Gravely
L
,
Wilhelm
J
.
“Mail Panel” and “Take Home Testing” discussion at RD&E - 870403
.
Brown & Williamson
; 
1987
,
Bates #674116794/674116809
.
64
RJ Reynolds
.
Proposal approval needed. EW full flavor consumer acceptance product test
. 
1994
,
Bates #512347867/512347869
.
65
Dryden
M
,
Hunter
C
,
Potter
D
,
Swaim
M
,
Gordin
H
.
Prism I CC concept product test - II. Product research proposal BAID#93-2352-16. Prism-I-Project CC - technology screenign concept/product test
.
RJ Reynolds
; 
1993
,
Bates #508470018/508470020
.
66
Scism
A
.
Product research report. Project XGT FF LT NM 85 prototype test final report
.
RJ Reynolds
; 
1989
,
Bates #514343880/514343895
.
67
Burke Marketing Services Inc
.
In-home cigarette testing - different products - for B&W, BMR #48-513-516
.
Brown & Williamson
; 
1983
,
Bates #620310135/620310263
.
68
Gannon
T
,
Smith
A
.
Old sensory 101 presentation early 900000's
.
Philip Morris
; 
1993
,
Bates #2075663262/2075663316
.
69
American Society for Testing and Materials
.
Standard guide for sensory claim substantiation
.
Philip Morris
; 
1998
,
Bates #2082155878/2082155898
.
70
RJ Reynolds
.
Proposed Eclipse testing
. 
1998
,
Bates #530772466/530772467
.
71
Smith
K
,
Pence
D
.
SED panel
.
RJ Reynolds
; 
1997
,
Bates #530977258
.
72
Reynolds
RJ
.
Brands R&D. Business units & competitive claims
. 
1995
,
Bates #518064256/518064273
.
73
DiMarco
G
.
NPT sensory panel participation
.
RJ Reynolds
; 
1987
,
Bates #505940104
.
74
DiMarco
G
.
NPT sensory panel participation
.
RJ Reynolds
; 
1987
,
Bates #505973439/505973440
.
75
DiMarco
G
.
NPT panelist support
.
RJ Reynolds
; 
1987
,
Bates #505944377
.
76
Norman
A
.
HRRC approval for consumer and SED panel evaluations of XA prototypes
.
RJ Reynolds
; 
1990
,
Bates #510744270/510744273
.
77
RJ Reynolds
.
Sensory evaluation methodologies
. 
2004
,
Bates # 551074115/551074116
.
78
RJ Reynolds
. (
Place “thermometer” scale in front of respondent
). 
1988
,
Bates #506904751/506904752
.
79
Ryan
G
.
SED92 1780. QDA and extended use evaluation of XA fflt menthol 100 prototypes
.
RJ Reynolds
; 
1993
,
Bates #510697256/510697281
.
80
Gross
J
,
Lee
J
,
Stitzer
M
. 
Nicotine-containing versus de-nicotinized cigarettes: effects on craving and withdrawal
.
Pharmacol Biochem Behav
1997
;
57
:
159
65
.
81
Kotlyar
M
,
Mendoza-Baumgart
M
,
Li
Z
, et al
. 
Nicotine pharmacokinetics and subjective effects of three potential reduced exposure products, moist snuff and nicotine lozenge
.
Tob Control
2007
;
16
:
138
42
.
82
Schuh
L
,
Stamat
H
,
Downey
K
,
Schuh
K
. 
Subjective and discriminative stimulus effects of two de-nicotinized cigaretes with different tar yields
.
Nicotine Tob Res
2001
;
3
:
77
83
.
83
Rees
V
,
Wayne
G
,
Connolly
G
. 
Puffing style and human exposure minimally altered by switching to a carbon-flavored cigarette
.
Cancer Epidemiol Biomarkers Prev
2008
;
17
:
2995
3003
.
84
Fagerstrom
K
,
Hughes
J
,
Rasmussen
T
,
Callas
P
. 
Randomised trial investigating the effect of a novel nicotine delivery device (Eclipse) and a nicotine oral inhaler on smoking behaviour, nicotine and carbon monoxide exposure, and motivation to quit
.
Tob Control
2000
;
9
:
327
33
.
85
Fagerstrom
K
,
Hughes
J
,
Callas
P
. 
Long-term effects of the Eclipse cigarette substitute and the nicotine inhaler in smokers not interested in quitting
.
Nicotine Tob Res
2002
;
4
:
S141
45
.
86
Breland
A
,
Acosta
M
,
Eissenberg
T
. 
Tobacco specific nitrosamines and potential reduced exposure products for smokers: a preliminary evaluatoin of AdvanceTM
.
Tob Control
2003
;
12
:
317
21
.
87
Hughes
J
,
Hatsukami
D
. 
Signs and symptoms of tobacco withdrawal
.
Arch Gen Psychiatry
1986
;
43
:
289
94
.
88
Tiffany
S
,
Drobes
D
. 
The development and initial validation of a questionnaire on smoking urges
.
Br J Addict
1991
;
86
:
1467
76
.
89
AM
K
,
PL
R
. 
New challenges for telephone survey research in the twenty-first century
.
Annu Rev Public Health
2007
;
28
:
113
26
.
90
Dillman
DA
,
Smyth
JD
. 
Design effects in the transition to web-based surveys
.
Am J Prev Med
2007
;
32
:
S90
6
.
91
Gates
R
,
Solomon
PJ
. 
Research using mall intercept: state of the art
.
J Advert Res
1982
;
22
:
43
9
.
92
Reitter
RN
. 
Product testing in segmented markets
.
J Mark Res
1969
;
6
:
179
84
.
93
O'Connor
R
,
McNeill
A
,
Borland
R
, et al
. 
Smokers' beliefs about the relative safety of other tobacco products: findings from the ITC Collaboration
.
Nicotine Tob Res
2007
;
9
:
1033
42
.
94
O'Hegarty
M
,
Richter
P
,
Pederson
L
. 
What do adult smokers think about ads and promotional materials for PREPs?
Am J Health Behav
2007
;
31
:
526
34
.
95
Philip Morris
.
Premier focus groups
. 
1988
,
Bates #2049258301/2049258312
.
96
RJ Reynolds
.
Second quarter, product claims, development ND screening research prisms I
. 
1993
,
Bates #511214460/511214516
.
97
British American Tobacco Company
.
Project Greendot - review of progress
.
Brown & Williamson
; 
1988
,
Bates #620215601/620215611
.
98
British American Tobacco Company
.
Project Greendot - Attitudinal Monitor
.
Brown & Williamson
;
Undated. Bates #465663530/465663583
.
99
Naasner
W
.
Concept/product test
.
RJ Reynolds
; 
1992
,
Bates #515472766/515472801
.
100
Bellis
J
.
Panel topline on hiq vs a product acceptance test and carton results on hiq vs 12 mm
.
RJ Reynolds
; 
1996
,
Bates #516759481/516759486
.