In Response: Gwinn et al. describe the risk of type I error when using genetic data from clinical trials, such as those which we have recently reported (1, 2) from the Patch randomized clinical trial of the nicotine replacement therapy transdermal patch (3-9). This program of pharmacogenetic research was initiated in 1999, based on a list of potential promising candidate genes, identified based on a priori relevance to tobacco addiction and known function. This program is now complete, and we are in the process of disseminating the findings. We intend to publish a review article summarizing the program of research and reporting the results from the entire list of candidate genes, which we have genotyped in due course, to achieve transparency.

Nevertheless, the issue of multiple testing in general, and subgroup analyses in particular, is a salient one in the context of pharmacogenetic studies, for the reasons highlighted by Gwinn et al. Several factors are likely to introduce bias into the genetic epidemiology literature and contribute to the risk of false-positive results. These include publication bias (10), longer time to publish for results that do not achieve statistical significance (11), the trend for effect sizes to decrease with year of publication (12), the poor predictive value of initial reports of genetic association (13), the post hoc study of further subgroups defined by sex or environmental factors (14), the excess of results that fall just below the 0.05 α level (15), etc. Although the pharmacogenetic literature is relatively new and modest in scale by comparison, it is likely that similar biases will operate here.

There are ways in which these risks may be ameliorated, however, the most obvious of which being replication in large-scale studies. To this end, we have recently completed a follow-up open label trial of the nicotine replacement therapy patch (Patch in Practice), where DNA for genotyping was collected at trial entry (16). Perhaps ironically, the first published pharmacogenetic results of this study are now available and appear to replicate the association of COMT genotype with response to nicotine replacement therapy patch (17). In particular, research questions relating to specific genotypes were included in the study protocol (ISRCTN 05689186). We are now in the process of preparing other results from this study for publication and will certainly not shy away from publishing failures to replicate or contradictory findings. Other measures include meta-analysis of multiple studies (18), which may also serve to highlight the presence of various biases in the published literature.

We agree entirely that results from studies such as ours should be regarded as hypothesis-generating and should subsequently contribute to the design of explicit pharmacogenetic trials and laboratory studies, which seek to confirm the role of specific genetic variants in treatment response. Such studies would benefit from the registration of the study protocol before data collection as is now required for clinical trials. Archiving of study data once primary hypotheses have been tested and reported would also aid transparency. What is required are explicitly designed pharmacogenetic trials conducted in the context of smoking cessation; emerging data such as those published by our group set the stage for this to be done in a hypothesis-testing way.

1
Imperial Cancer Research Fund. Effectiveness of a nicotine patch in helping people stop smoking: results of a randomised trial in general practice.
Br Med J
1993
;
306
:
1304
–8.
2
Imperial Cancer Research Fund. Randomised trial of nicotine patches in general practice: results at one year.
Br Med J
1994
;
308
:
1476
–7.
3
David SP, Munafo MR, Murphy MF, Proctor M, Walton RT, Johnstone EC. Genetic variation in the dopamine D4 receptor (DRD4) gene and smoking cessation: follow-up of a randomised clinical trial of transdermal nicotine patch. Pharmacogenomics J 2007.
4
David SP, Munafo MR, Murphy MF, Walton RT, Johnstone EC. The serotonin transporter 5-HTTLPR polymorphism and treatment response to nicotine patch: follow-up of a randomized controlled trial.
Nicotine Tob Res
2007
;
9
:
225
–31.
5
Johnstone EC, Elliot KM, David SP, Murphy MF, Walton RT, Munafo MR. Association of COMT Val108/158Met genotype with smoking cessation in a nicotine replacement therapy randomized trial.
Cancer Epidemiol Biomarkers Prev
2007
;
16
:
1065
–9.
6
Johnstone EC, Yudkin PL, Hey K, et al. Genetic variation in dopaminergic pathways and short-term effectiveness of the nicotine patch.
Pharmacogenetics
2004
;
14
:
83
–90.
7
Munafo MR, Elliot KM, Murphy MF, Walton RT, Johnstone EC. Association of the μ-opioid receptor gene with smoking cessation.
Pharmacogenomics J
2007
;
7
:
353
–61.
8
Munafo MR, Johnstone EC, Wileyto EP, Shields PG, Elliot KM, Lerman C. Lack of association of 5-HTTLPR genotype with smoking cessation in a nicotine replacement therapy randomized trial.
Cancer Epidemiol Biomarkers Prev
2006
;
15
:
398
–400.
9
Yudkin P, Munafo MR, Hey K, et al. Effectiveness of nicotine patches in relation to genotype in women versus men: randomised controlled trial.
Br Med J
2004
;
328
:
989
–90.
10
Ioannidis JP. Journals should publish all “null” results and should sparingly publish “positive” results.
Cancer Epidemiol Biomarkers Prev
2006
;
15
:
186
.
11
Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.
JAMA
1998
;
279
:
281
–6.
12
Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG, Ioannidis JP. Establishment of genetic associations for complex diseases is independent of early study findings.
Eur J Hum Genet
2004
;
12
:
762
–9.
13
Ioannidis JP, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG. Replication validity of genetic association studies.
Nat Genet
2001
;
29
:
306
–9.
14
Patsopoulos NA, Tatsioni A, Ioannidis JP. Claims of sex differences: an empirical assessment in genetic associations.
JAMA
2007
;
298
:
880
–93.
15
Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings.
Clin Trials
2007
;
4
:
245
–53.
16
Aveyard P, Brown K, Saunders C, et al. Weekly versus basic smoking cessation support in primary care: a randomised controlled trial.
Thorax
2007
;
62
:
898
–903.
17
Munafo MR, Johnstone EC, Guo B, Murphy MF, Aveyard P. Association of COMT Val108/158Met genotype with smoking cessation.
Pharmacogenet Genomics
2008
;
18
:
121
–8.
18
Munafo MR, Flint J. Meta-analysis of genetic association studies.
Trends Genet
2004
;
20
:
439
–44.