- Open Access
A new statistical approach to combining p-values using gamma distribution and its application to genome-wide association study
© Chen et al.; licensee BioMed Central Ltd. 2014
- Published: 16 December 2014
Combining information from different studies is an important and useful practice in bioinformatics, including genome-wide association study, rare variant data analysis and other set-based analyses. Many statistical methods have been proposed to combine p-values from independent studies. However, it is known that there is no uniformly most powerful test under all conditions; therefore, finding a powerful test in specific situation is important and desirable.
In this paper, we propose a new statistical approach to combining p-values based on gamma distribution, which uses the inverse of the p-value as the shape parameter in the gamma distribution.
Simulation study and real data application demonstrate that the proposed method has good performance under some situations.
- Fisher test
- Lancaster method
- rare variant association test
To combine information from individual studies, many statistical approaches have been proposed. For example, meta-analysis with fixed or random effects has been intensively used to combine information from separate relevant genome-wide association studies (GWASs). However, in practice sometimes it may not be able to get all the statistics that we need, such as odds ratio and its 95% confidence interval; instead, only p-value from each study is available. In this case, combining p-values from independent studies should be used. In the literature, many statistical methods have been proposed to combine p-values [1–14]. For example, it has been shown that the Fisher test is more robust than the z-test and is commonly used for genetic data [15–26]. On the other hand, if the effects have the same direction and/or similar sizes, z-test is more powerful than the Fisher test. Some studies have shown that the weighted z-tests with weight equals to the sample size or the inverse of the standard error may perform better than the unweighted z-test under certain situations . However, it has also been shown that there is no uniformly most powerful method . Therefore, it is desirable to find a test which is more powerful than others for given situations. For instance, in GWAS meta-analysis, it is very common that the genetic effects of the same single-nucleotide polymorphism (SNP) from different studies are heterogeneous due to various environmental factors and study populations. Therefore, the fixed effect model cannot be applied and a p-value combining method is preferred.
Lancaster generalized Fisher test by giving certain degrees of freedom to individual studies when combine p-values based on the chi-square distribution. When the degrees of freedom (df) equal to two for each study, Lancaster's test is identical to the Fisher test. Recently, Chen and Nadarajah have studied another special case of Lancaster's test where the df is one for each study . They have shown that their test can also be viewed as a weighted z-test with the "weight" equals to the estimated effect, defined as the estimated mean difference divided by the estimated standard error, which can be calculated by , where p i is the one-sided p-value from the ith study and is the inverse of the cumulative density function (CDF) of the standard normal distribution, N(0,1).
Methods based on the gamma distribution (GDM) are also available in the literature. In fact, Lancaster's methods are special cases of GDMs. GDMs are more flexible and potentially can be more powerful in some situations when appropriate parameters (e.g., the shape parameter in the gamma distribution) are chosen. However, it is usually difficult to set appropriate parameters before we see the data. In this paper, we propose a GDM, which adaptively chooses the shape parameter of the gamma distribution for each individual study. We compare the performance of the proposed test with existing methods through simulation studies. We also use real data application to illustrate the use of the new approach.
where w i is the weight for study i. When all w i = 1, the above test is the unweighted z-test, also called the Stouffer test . When wi = ni, where ni is the sample size for study i , it is called the Mosteller-Bush test . Other researchers suggested the use of the square root of the sample size or the inverse of the estimated standard error as weight .
where is the inverse of .
where is the inverse gamma distribution with shape parameter α i and scale parameter β. Due to the property of the gamma distribution, for constant shape parameter α i , T will have a gamma distribution with shape parameter equals to and scale parameter equals to β. When all α i = 1, T has an exponential distribution under the null hypothesis. When all α i = v/2 and β = 2, the null distribution of T is a chi-square distribution with . When v = 1, it is the Chen-Nadarajah test ; when v = 2, it is the Fisher test.
Notice that since the gamma distribution with shape parameter α i and scale parameter 1 has expected value α i , a small p-value of p i results in a large expected value. Therefore, the proposed test gives larger "weights" to smaller p-values. In addition, since p i is a random variable, the proposed test doesn't follow a gamma distribution any more. However, the p-value can be easily estimated by resampling method. Under the null hypothesis, p i is uniformly distributed between 0 and 1. For the given number of studies, K, we can generate K numbers from uniform distribution U(0,1) and then calculate the statistic t defined in (4). We repeat this step N times (say, N = 108), then the null distribution of T can be approximated by those numbers and the p-value can be estimated by the proportion of the N values which are greater than the observed statistic.
To assess the performance of the proposed test, we conduct a simulation study by comparing it with some existing methods, including the z-test (denoted by Z), weighted z-tests with weights equal to the sample size (Z_n) or the estimated standard error (Z_se), the Chen-Nadarajah (CN) method, the Fisher test (Fisher). In the simulation study, we assume there are K independent studies, where K = 2, 10, or 100. For each study, we simulate data from two normal distributions: , and with sample sizes n1 = n2 = n, respectively. Of the K studies, there are different numbers of studies that have none-zero effects (i.e., ), which may have different values among studies but their sum is a constant. We consider several conditions for allocating effect sizes among the K studies. We first consider sample sizes and variances are fixed. We then assume the sample size, or the variance, or both the sample size and the variance are randomly sampled from given distributions. For random sample size, we assume it follows a Poisson distribution, Poi(λ); for random variances, we assume the standard deviation follows a gamma distribution with shape parameter α and scale parameter β, gamma(α, β). A p-value from a two-sample t-test to compare two group means for each study is obtained and is used to combine those K studies. When K is small (i.e., 2, and 10), we consider situations where there are 1 to K studies having none-zero effects. For K = 100, we consider i (i = 1, 2,..., 10) studies having the same effect size while the remaining 100-i studies having zero effect. We choose significance level 0.05 in the simulation study and use 105 replicates to estimate the type I error rate and the power.
Real data application
Count data from the five independent studies investigating the association between SNP rs17110747-A and major depression.
Discussion and conclusions
Combining information from individual studies is an important and useful tool, especially for set-based approaches. For example, in studying the effect of rare variants on diseases, a set of rare variants are tested simultaneously, and their p-values are combined to test for the association between the rare variants and the disease . However, most of the rare variants may have no or little effects while a few of them may have large effects. In this case, the proposed test will be more powerful than other methods if combining p-value methods are used. However, it should be pointed out that, the rare variants from a set (e.g., gene) maybe correlated, and the proposed test needs to be modified accordingly. A permutation-based test can be applied to estimate the p-value. We first calculate the statistic based on the proposed test (4). Then we permute the disease status (case or control); for each permutation, we use the proposed test to calculate a statistic. After a large number of permutations, the p-value will be estimated as the proportion of the statistics from the permutations excessing the observed statistic from the original data. To assess the performance of the proposed test in rare variant association studies, real data are needed. This will be a topic of our future research.
As mentioned earlier, no method is uniformly most powerful when combing p-values. However, based on our simulation studies, the proposed test is more powerful when the effects among the studies are more heterogeneous. When the effects are homogeneous, perhaps the Z-based tests are more powerful. Without the information about the effect sizes, robust methods, such as the CN and Fisher tests are recommended.
The research was supported by Indiana University Institutional Research Awards and National Institutes of Health (NIH). ZC would like to thank the institutional faculty research grant awards from the Indiana University Bloomington School of Public Health. MQY would like to thank the support from NIH/NIGMS 5P20GM10342913 and ASTA Award # 15-B-23.
The funding for publication of the article has come from Indiana University Bloomington institutional research grant awards to ZC.
This article has been published as part of BMC Bioinformatics Volume 15 Supplement 17, 2014: Selected articles from the 2014 International Conference on Bioinformatics and Computational Biology. The full contents of the supplement are available online at http://www.biomedcentral.com/bmcbioinformatics/supplements/15/S17.
- Birnbaum A: Combining independent tests of significance. J Amer Statist Assoc. 1954, 49 (267): 559-574.Google Scholar
- Good IJ: On the weighted combination of significance tests. J R Stat Soc Ser B Stat Methodol. 1955, 17 (2): 264-265.Google Scholar
- Lancaster H: The combination of probabilities: an application of orthonormal functions. Austral J Statist. 1961, 3: 20-33. 10.1111/j.1467-842X.1961.tb00058.x.View ArticleGoogle Scholar
- Pearson ES: The probability integral transformation for testing goodness of fit and combining independent tests of significance. Biometrika. 1938, 30 (1/2): 134-148. 10.2307/2332229.View ArticleGoogle Scholar
- Stouffer SA, Suchman EA, DeVinney LC, Star SA, Williams RMJ. In: The American Soldier, Vol 1: Adjustment during Army Life. 1949, Princeton: Princeton University PressGoogle Scholar
- Tippett L: The Methods of Statistics. 1931, London: Williams and Norgate LtdGoogle Scholar
- Whitlock MC: Combining probability from independent tests: the weighted Z-method is superior to Fisher's approach. J Evol Biol. 2005, 18 (5): 1368-1373. 10.1111/j.1420-9101.2005.00917.x.View ArticlePubMedGoogle Scholar
- Mosteller F, Bush RR: Selected quantitative techniques. Handbook of Social Psychology. Edited by: Lindzey G. Cambridge. 1954, Mass: Addison-Wesley, 289-334.Google Scholar
- Fisher RA: Statistical Methods for Research Workers. 1932, Edinburgh: Oliver and BoydGoogle Scholar
- Liptak T: On the combination of independent tests. Magyar Tud Akad Mat Kutato Int Kozl. 1958, 3: 171-197.Google Scholar
- Chen Z, Nadarajah S: Comments on 'Choosing an optimal method to combine p - values' by Sungho Won, Nathan Morris, Qing Lu and Robert C. Elston, Statistics in Medicine 2009; 28: 1537-1553. Statistics in Medicine. 2011, 30 (24): 2959-2961. 10.1002/sim.4222.View ArticlePubMedGoogle Scholar
- Chen Z: Is the weighted z - test the best method for combining probabilities from independent tests?. Journal of Evolutionary Biology. 2011, 24 (4): 926-930. 10.1111/j.1420-9101.2010.02226.x.View ArticlePubMedGoogle Scholar
- Loughin TM: A systematic comparison of methods for combining p-values from independent tests. Computational statistics & data analysis. 2004, 47 (3): 467-485. 10.1016/j.csda.2003.11.020.View ArticleGoogle Scholar
- Chen Z, Nadarajah S: On the optimally weighted z-test for combining probabilities from independent studies. Computational Statistics & Data Analysis. 2014, 70: 387-394.View ArticleGoogle Scholar
- Chen Z: A new association test based on Chi - square partition for case - control GWA studies. Genetic Epidemiology. 2011, 35 (7): 658-663. 10.1002/gepi.20615.View ArticlePubMedGoogle Scholar
- Chen Z: Association tests through combining p-values for case control genome-wide association studies. Statistics and Probability Letters. 2013, 83 (8): 1854-1862. 10.1016/j.spl.2013.04.021.View ArticleGoogle Scholar
- Chen Z, Huang H, Liu J, Ng HKT, Nadarajah S, Huang X, Deng Y: Detecting differentially methylated loci for Illumina Array methylation data based on human ovarian cancer data. BMC Medical Genomics 2013,. 2013, 6 (Suppl 1): S9-Google Scholar
- Chen Z, Huang H, Ng HKT: Design and Analysis of Multiple Diseases Genome-wide Association Studies without Controls. GENE. 2012, 510 (1): 87-92. 10.1016/j.gene.2012.07.089.PubMed CentralView ArticlePubMedGoogle Scholar
- Chen Z, Huang H, Ng HKT: Testing for Association in Case-Control Genome-wide Association Studies with Shared Controls. Statistical Methods in Medical Research, Published online before print February 1, 2013, doi: 101177/0962280212474061. 2013Google Scholar
- Chen Z, Liu Q, Nadarajah S: A new statistical approach to detecting differentially methylated loci for case control Illumina array methylation data. Bioinformatics. 2012, 28 (8): 1109-1113. 10.1093/bioinformatics/bts093.PubMed CentralView ArticlePubMedGoogle Scholar
- Chen Z, Ng HKT: A Robust Method for Testing Association in Genome-Wide Association Studies. Human Heredity. 2012, 73 (1): 26-34. 10.1159/000334719.PubMed CentralView ArticlePubMedGoogle Scholar
- Huang H, Chen Z, Huang X: Age-adjusted nonparametric detection of differential DNA methylation with case--control designs. BMC Bioinformatics. 2013, 14 (1): 86-10.1186/1471-2105-14-86.PubMed CentralView ArticlePubMedGoogle Scholar
- Chen Z, Ng HKT, Li J, Liu Q, Huang H: Detecting associated single-nucleotide polymorphisms on the × chromosome in case control genome-wide association studies. Statistical methods in medical research. 2014, Published online before print September 24, 2014, doi: 2010.1177/0962280214551815Google Scholar
- Chen Z, Huang H, Ng HKT: An Improved Robust Association Test for GWAS with Multiple Diseases. Statistics & Probability Letters. 2014, 91: 153-161.View ArticleGoogle Scholar
- Chen Z, Huang H, Liu Q: Detecting differentially methylated loci for multiple treatments based on high-throughput methylation data. BMC Bioinformatics. 2014, 15: 142-10.1186/1471-2105-15-142.PubMed CentralView ArticlePubMedGoogle Scholar
- Chen Z: A new association test based on disease allele selection for case-control genome-wide association studies. BMC Genomics. 2014, 15: 358-10.1186/1471-2164-15-358.PubMed CentralView ArticlePubMedGoogle Scholar
- Zaykin D: Optimally weighted Z - test is a powerful method for combining probabilities in meta - analysis. Journal of Evolutionary Biology. 2011, 24 (8): 1836-1841. 10.1111/j.1420-9101.2011.02297.x.PubMed CentralView ArticlePubMedGoogle Scholar
- Gao J, Pan Z, Jiao Z, Li F, Zhao G, Wei Q, Pan F, Evangelou E: TPH2 gene polymorphisms and major depression-a meta-analysis. PloS one. 2012, 7 (5): e36721-10.1371/journal.pone.0036721.PubMed CentralView ArticlePubMedGoogle Scholar
- Chen LS, Hsu L, Gamazon ER, Cox NJ, Nicolae DL: An exponential combination procedure for set-based association tests in sequencing studies. The American Journal of Human Genetics. 2012, 91 (6): 977-986. 10.1016/j.ajhg.2012.09.017.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.