Data-Dredging Gene-Dose Analyses in Association Studies: Biases and their Corrections

To examine the joint effect of multiple loci on disease risk, many case-control association studies used “gene-dose analyses” (we found 40 articles with gene-dose analysis from January 1998 to May 2005 in six esteemed cancer journals: Cancer Epidemiology Biomarkers and Prevention, Cancer Research, Carcinogenesis, Clinical Cancer Research, International Journal of Cancer, and the Journal of the National Cancer Institute). In these studies, the researchers first defined the high-risk genotype (or allele) at each locus. They then calculated the total number of the high-risk genotypes (alleles) a subject has for the cases and for the controls in their study. They categorized the total number of the high-risk genotypes (alleles) into a number of categories, say, “0 to 2”, “3 to 4”, and “5+”, and they calculated the odds ratios (OR) and the associated confidence intervals (CI) for these categories. Finally, they did a trend test to see if the OR increases as the total number of the high-risk genotypes (alleles) increases. “Dose-response relationship” is one of the causal criteria put forward by Hill (1). A gene-dose effect is corroborative of a cause-and-effect relation between the genes and the disease.

However, some researchers defined the high-risk genotypes (alleles) as those that have higher genotypic (allelic) frequencies in the case group than in the control group in the study, or equivalently, as those with OR > 1 in the study [among the aforementioned 40 articles, researchers from 10 of these articles defined their high-risk genotypes (alleles) in this way; see Table 1 for a summary]. They then did the usual gene-dose analysis as if those defined are the bona fide high-risk genotypes (alleles)—the “data-dredging” gene-dose analysis. The definition guarantees the frequency of the “high-risk” genotypes (alleles) being higher in the case group than in the control group at each locus. Intuitively, this will lead to the total number of the high-risk genotypes (alleles) tending to be higher for the cases than for the controls as well, even if, in fact, none of the studied loci were related to the disease. Consequently, an excess of false-positive gene-dose effects may result.

A simulation study is presented in Table 2. The loci are assumed to be unlinked or in linkage equilibrium with one another. We see that although none of the simulated loci is related to the disease, the OR increases as the total number of the high-risk genotypes increases, displaying a false sense of gene-dose effects. The coverage probabilities of the CIs are too low and the type I error rates of the trend test are inflated (α = 0.05). As the number of the studied loci increases, the problem becomes more acute; with 10 loci, the OR can be as high as ∼2.00, the coverage probability, as low as ∼0.38, and the type I error rate, as high as ∼0.64 (the data-dredging articles in Table 1 with ≥5 loci and borderline level of significance are thus particularly vulnerable). The overestimation of the OR is less severe when sample sizes reached 1,000 as compared with studies with 500 samples. However, the undercoverage of the CIs and the inflation of the type I error rates are irrespective of sample size. Similar biases can be found when the studied loci are in linkage disequilibrium (less severe) and when the analysis is based on alleles (more severe; results not shown).

To correct the biases, we let each and every study subject retain his/her genetic data. Whereas we randomly permute the disease status (cases or controls) of all the study subjects (keeping the total number of cases and the total number of controls constant). A new round of data-dredging gene-dose analyses is done for this permuted data—to determine a new set of the “high-risk” genotypes (alleles), and then to calculate a new set of the ORs and a new χ² statistic of the trend test. The procedure is to be repeated 1,000 times (from our limited simulation experiences, this number of permutations is sufficient).

To get a permutation-corrected OR for a particular category of the total number of high-risk genotypes (alleles), we divide the OR of that category in the original data by the geometric mean of the ORs of the same category in the 1,000 permuted data (the original log OR minus the arithmetic mean of the permutation log ORs). To get a permutation-corrected upper (lower) limit of the 95% CI for a particular category, we divide the OR of that category in the original data by the 2.5th (97.5) percentile of the ORs of the same category in the 1,000 permuted data (the 95% CI constructed in this way will cover 1.00, if the original OR does not differ significantly from 1.00, using permutation tests at α = 0.05; it will not cover 1.00, if the OR differs significantly from 1.00). To get a permutation-corrected P value for the trend test, we calculate the proportion in the 1,000 permuted data that have the χ² statistic of the trend test larger than the same statistic in the original data (alternatively, we declare the trend test to be significant at α = 0.05, if the original χ² statistic is larger than the 95th percentile of the permutation χ² statistics).

With regard to the data in Table 2, it can now be seen in Table 3 that the permutation-corrected ORs are very close to their true values of 1.00, the permutation-corrected CIs have coverage probabilities of ∼0.95, and the permutation-corrected trend tests have type I error rates of ∼0.05. The permutation correction achieves similarly satisfactory results when the studied loci are in linkage disequilibrium and when the analysis is based on alleles (data not shown).

In a study in which we have a priori knowledge about the high-risk genotypes (alleles) for some of the loci but not for the rest, we may resort to performing a gene-dose analysis with a mixture of the “predetermined” high-risks and the “data-dredged” high-risks. The correction method can deal with this; by copying the predetermining and the data-dredging (of the original data) to each and every round of the permutations. The method also works when other covariates beyond the studied loci (i.e., sex, age, smoking, etc.) need to be considered; by performing a logistic regression to derive the “covariates-adjusted” ORs (2) in each and every round of the permutations. Finally, the method can also be used for a matched case-control study; the ORs are to be calculated using the matched-data methods (2) and the permutations (of the disease status) are to be restricted to subjects in the same matching set.

In conclusion, the data-dredging gene-dose analysis as commonly employed in the literature will produce grossly biased results. The biases, however, could very effectively be corrected by the permutation correction method as described in this report.