In Response: Gwinn et al. describe the risk of type I error when using genetic data from clinical trials, such as those which we have recently reported (1, 2) from the Patch randomized clinical trial of the nicotine replacement therapy transdermal patch (3-9). This program of pharmacogenetic research was initiated in 1999, based on a list of potential promising candidate genes, identified based on a priori relevance to tobacco addiction and known function. This program is now complete, and we are in the process of disseminating the findings. We intend to publish a review article summarizing the program of research and reporting the results from the entire list of candidate genes, which we have genotyped in due course, to achieve transparency.
Nevertheless, the issue of multiple testing in general, and subgroup analyses in particular, is a salient one in the context of pharmacogenetic studies, for the reasons highlighted by Gwinn et al. Several factors are likely to introduce bias into the genetic epidemiology literature and contribute to the risk of false-positive results. These include publication bias (10), longer time to publish for results that do not achieve statistical significance (11), the trend for effect sizes to decrease with year of publication (12), the poor predictive value of initial reports of genetic association (13), the post hoc study of further subgroups defined by sex or environmental factors (14), the excess of results that fall just below the 0.05 α level (15), etc. Although the pharmacogenetic literature is relatively new and modest in scale by comparison, it is likely that similar biases will operate here.
There are ways in which these risks may be ameliorated, however, the most obvious of which being replication in large-scale studies. To this end, we have recently completed a follow-up open label trial of the nicotine replacement therapy patch (Patch in Practice), where DNA for genotyping was collected at trial entry (16). Perhaps ironically, the first published pharmacogenetic results of this study are now available and appear to replicate the association of COMT genotype with response to nicotine replacement therapy patch (17). In particular, research questions relating to specific genotypes were included in the study protocol (ISRCTN 05689186). We are now in the process of preparing other results from this study for publication and will certainly not shy away from publishing failures to replicate or contradictory findings. Other measures include meta-analysis of multiple studies (18), which may also serve to highlight the presence of various biases in the published literature.
We agree entirely that results from studies such as ours should be regarded as hypothesis-generating and should subsequently contribute to the design of explicit pharmacogenetic trials and laboratory studies, which seek to confirm the role of specific genetic variants in treatment response. Such studies would benefit from the registration of the study protocol before data collection as is now required for clinical trials. Archiving of study data once primary hypotheses have been tested and reported would also aid transparency. What is required are explicitly designed pharmacogenetic trials conducted in the context of smoking cessation; emerging data such as those published by our group set the stage for this to be done in a hypothesis-testing way.
