Proportion statistics to detect differentially expressed genes: a comparison with logratio statistics
 Tracy L Bergemann^{1, 2}Email author and
 Jason Wilson^{3}Email author
DOI: 10.1186/1471210512228
© Bergemann and Wilson; licensee BioMed Central Ltd. 2011
Received: 28 October 2010
Accepted: 7 June 2011
Published: 7 June 2011
Abstract
Background
In genetic transcription research, gene expression is typically reported in a test sample relative to a reference sample. Laboratory assays that measure gene expression levels, from QRTPCR to microarrays to RNASeq experiments, will compare two samples to the same genetic sequence of interest. Standard practice is to use the log_{2}ratio as the measure of relative expression. There are drawbacks to using this measurement, including unstable ratios when the denominator is small. This paper suggests an alternative estimate based on a proportion that is just as simple to calculate, just as intuitive, with the added benefit of greater numerical stability.
Results
Analysis of two groups of mice measured with 16 cDNA microarrays found similar results between the previously used methods and our proposed methods. In a study of liver and kidney samples measured with RNASeq, we found that proportion statistics could detect additional differentially expressed genes usually classified as missing by ratio statistics. Additionally, simulations demonstrated that one of our proposed proportionbased test statistics was robust to deviations from distributional assumptions where all other methods examined were not.
Conclusions
To measure relative expression between two samples, the proportion estimates that we propose yield equivalent results to the log_{2}ratio under most circumstances and better results than the log_{2}ratio when expression values are close to zero.
Background
Several different bioinformatics technologies exist to quantify gene expression. Regardless of technological platform, laboratory assays of gene expression first extract mRNA from a test sample and a control sample. These samples may be labeled with a tag or dye and hybridized to amplified cloned sequences that represent a gene of interest. The amount of mRNA in each sample is usually measured by examining the amount of dye remaining after hybridization. Researchers use QRTPCR to measure expression when there are only one or a few genes of interest. Several lab protocols from various companies exist to quantify gene expression such as RTPCR assays using intercalating dyes like SYBR Green, the TaqMan Gene Expression Assays, LightCycler, and QuantiGene [1–3]. When genomewide levels of expression are of interest, microarrays can measure expression for thousands of genes of interest. Microarray platforms employ either cDNA clones [4, 5] or nmer oligonucleotide probes for many genes at once [6].
or other similar variants on the theme. The log_{2}ratio is commonly interpreted as the average "logfoldchange" in gene expression between the reference sample and the test sample. Its estimate will be denoted by . If r_{ j } = 1, then the ratio between the two samples is 2^{1} = 2, meaning that the expression of gene j in the test sample is twofold that of the reference sample on average. If r_{ j } = 2, then the ratio between the two samples is 2^{2} = 4, meaning that on average the expression in the test sample is fourfold that of the reference sample. Other values of r_{ j } are interpreted similarly.
While the interpretation of the log_{2}ratio is appealing, the statistic has an important drawback. When expression in the reference sample is low, is numerically unstable because the denominators R_{ ij } are small. As R_{ ij } approaches zero, r_{ j } increases drastically, approaching infinity. When R_{ ij } = 0, then r_{ j } is undefined. Thus, when reference sample expression is low, we get extreme estimates or missing values for r_{ j } . This phenomenon is especially common when measuring gene expression in simple organisms. In bacteria, for example, transcription may be binary; either on or off. The log_{2}ratio is least reliable for these systems. This problem persists in human genomics research for certain experimental conditions and genes of interest.
More generally, p_{ j } can be interpreted as the proportion of mRNA from gene j expressed in the test sample. As p_{ j } deviates from 0.5, then there is differential expression between the test and reference samples. As p_{ j } approaches one, then gene j is upregulated in the test sample. As p_{ j } approaches zero, then gene j is downregulated in the test sample. The proportion statistic p_{ j } can also be transformed into a percentage: p_{ j } × 100% for reporting. For example, if p_{ j } = 0.75 then we can say that 75% of the mRNA expressed in the experiment comes from the test sample. The proportion estimate can easily be used to test for differential expression between groups. Under the null hypothesis of no gene expression, p_{ j } = 0.5. The alternative hypothesis is differential expression, p_{ j } ≠ 0.5. The log_{2}ratio estimate requires a different hypothesis test. Under the null hypothesis, r_{ j } = 0 and under the alternative, r_{ j } ≠ 0.
Using a proportion p_{ j } to describe relative expression for gene j instead of the log_{2}ratio r_{ j } maintains the ability to interpret differential expression and test for differences. The added benefit of the proportion is the ability to preserve all data points, even for experiments with very low expression values. Typically when values of R_{ ij } are very small, researchers eliminate the j^{ th } probe of the i^{ th } experiment from their analysis. Eliminating missing data results in a loss of information and potential bias and loss of power. The proportion estimate does not require the removal of extreme, but legitimate, data points.
The Results section provides details that describe the estimation of statistics for p_{ j } . The section also provides several test statistics for hypothesis tests of p_{ j } . Estimation and testing are developed in the frequentist context but the Bayesian context can also be used, as described in the Appendix. The Results section compares the testing scenarios in simulations and two datasets. The first dataset consists of expression data from a cDNA microarray platform and the second dataset uses RNASeq. Both the log_{2}ratio and proportion statistics achieve roughly equivalent results under usual conditions, but one of the proportion statistics performs better across a variety of distributional assumptions. Proportion statistics also detect differentially expressed genes that would typically be classified as missing data.
Results
Parameter Estimates and Hypothesis Testing
We propose a new strategy for the comparison of expression values that is tied to the underpinnings of the hybridization process and its natural interpretation using a binomial distribution. Figure 1 illustrates the hybridization process in a way that justifies the use of a binomial distribution. The description is specific to a twocolor hybridization platform. The same concept extends to any system where both test samples and reference samples are assayed.
Simulation comparing test statistics for , , , limma/EBA, edgeR, and DESeq with a sample size of n = 20 under four distributional assumptions.
Exponential  Poisson  Binomial  Normal  

fc = 1  fc = 3  fc = 1  fc = 3  fc = 1  fc = 3  fc = 1  fc = 3  
 0.051  0.742  0.004  0.116  0.047  1.000  0.050  1.000 
+ 0.05  0.051  0.742  0.038  0.757  0.047  1.000  0.050  1.000 
+ 0.5  0.051  0.742  0.044  0.943  0.047  1.000  0.050  1.000 
 0.975  1.000  0.045  1.000  0.048  1.000  0.003  1.000 
 0.055  0.773  0.047  0.881  0.047  1.000  0.050  1.000 
 0.975  1.000  0.045  1.000  0.048  1.000  0.003  1.000 
EBA  0.051  0.781  0.048  1.000  0.047  1.000  0.052  1.000 
edger  NA  NA  0.033  1.000  0.014  1.000  NA  NA 
DESeq  NA  NA  0.042  1.000  0.047  1.000  NA  NA 
Calculating corresponding confidence intervals for each of the test statistics above is straightforward. Previous research suggests adjusting confidence intervals for binomial proportions. The most popular adjustment of these intervals uses the AgrestiCoull procedure [10, 11]. We recommend this procedure to estimate confidence intervals for both of the proportion estimates above.
The proposed statistics are evaluated within a frequentist framework. A Bayesian framework is provided in the Appendix.
Simulation Results
We ran a series of simulations to compare the inference behavior of proportion based statistics, and , to logratio based statistics, and . The proportion statistics and are introduced in equations 2 and 3 above and their test statistics are given in equations 4 and 5. The ratiobased statistics that have been used in the literature previously are described in equation 1 ( ) and equation 6 in the Methods section ( ). In preliminary simulation exercises, we found that the performance of some test statistics was heavily dependent on the distribution used to generate the expression data. Thus, we generated expression data under four different distributions. The simulation results in Table 1 present a subset of the sample sizes and fold changes examined. More extensive tables are in Additional file 1.
Comparison of estimators from the simulations.
Good  Acceptable  Unacceptable  

Exponential  , EBA 
 
Poisson  , EBA  , edgeR, DESeq 

Binomial  , EBA  edgeR  DESeq 
Normal  , EBA 

Analysis of Gene Expression in Mice with apoAI Knockout
Table of raw pvalues for Welch tstatistics and limma methods using and from the apoAI control treatment expression data.
rank  pvalue ( )  rank  pvalue ( )  rank  pvalue(limma ( ))  rank  pvalue(limma ( )) 

1  7.3 × 10^{7}  1  4.2 × 10^{6}  1  3.8 × 10^{12}  1  1.5 × 10^{9} 
2  2.4 × 10^{5}  2  2.4 × 10^{5}  4  5.2 × 10^{7}  4  1.3 × 10^{6} 
3  3.4 × 10^{5}  4  4.0 × 10^{5}  2  2.6 × 10^{8}  3  1.2 × 10^{7} 
4  5.0 × 10^{5}  3  2.8 × 10^{5}  3  5.1 × 10^{8}  2  7.6 × 10^{8} 
5  1.0 × 10^{4}  6  1.2 × 10^{4}  5  1.4 × 10^{6}  6  7.5 × 10 ^{6} 
6  1.0 × 10^{4}  5  5.8 × 10^{5}  7  9.6 × 10^{6}  7  1.2 × 10^{5} 
7  2.9 × 10^{4}  7  2.7 × 10^{4}  12  4.5 × 10^{5}  11  3.7 × 10^{5} 
8  5.9 × 10^{4}  9  6.4 × 10^{4}  8  1.3 × 10^{5}  16  4.8 × 10^{5} 
9  7.4 × 10^{4}  8  5.8 × 10^{4}  10  1.5 × 10^{4}  61  3.3 × 10^{4} 
10  1.3 × 10^{3}  10  1.1 × 10^{3}  6  2.0 × 10^{6}  5  1.4 × 10^{6} 
rank  tstat ( )  rank  tstat ( )  rank  tstat(limma ( ))  rank  tstat(limma ( )) 
1  16.5  1  12.8  1  23.1  1  14.4 
2  9.8  2  9.8  4  9.0  4  8.3 
3  9.3  4  9.1  2  11.5  3  10.1 
4  8.8  3  9.6  3  10.9  2  10.5 
5  7.9  6  7.7  5  8.2  6  7.0 
6  7.8  5  8.5  7  6.7  7  6.7 
7  6.6  7  6.7  12  5.9  11  6.0 
8  5.9  9  5.8  8  6.7  16  5.9 
9  5.7  8  5.9  10  5.2  61  4.8 
10  5.2  10  5.3  6  8.0  5  8.2 
When using , there were 158 (2.5%) unanalyzable probes because one or more of the samples had both G_{ ij } = 0 and R_{ ij } = 0, which made undefined. The statistic was defined for all probes because G_{ ij } and R_{ ij } were never zero for all samples of a specific probe. For this data, none of the 158 unanalyzed probes were in the top eight when using , although if they were a potential discovery they would have been missed using . To avoid this problem, one may add an arbitrary constant to all probes before taking the logratio. If merely raw pvalues were selected at α = 0.05, then would have selected 850, but there would have been 9 more significant pvalues if an arbitrary 0.05 were added to the data to avoid zero denominators when using logratios. By comparison, would have selected 871 probes.
Therefore, is able to give comparable results to for this cDNA microarray experiment, with the slight advantage that it provided information for 158 more probes in the study, without an arbitrary constant.
Analysis of Differential Expression in Human Kidney and Liver Cells
To examine the performance of our methods on RNASeq data, we analyzed the expression values reported in Marioni et al (2008) [9]. This data compared the expression of human kidney and liver cells sampled from the same person. Concentrations of 3 pM of cDNA were sequenced using the Illumina platform in five lanes. The original paper analyzed the expression of 32,000 sequences and reported that 11,493 of the sequences were differentially expressed with qvalues less than 0.001 (FDR < 0.1%) [14]. Supplemental Table 3 from Marioni et al (2008) provides the results of 17,708 sequences analyzed with both RNASeq technology and Affymetrix microarrays. They reported that 8,113 of Affyymetrix probe sets were differentially expressed with qvalues less than 0.001.
In order to compare the methods in the original paper to those we are proposing, we used a type I error rate of α = 0.05/32000 for all tests. In this way, the threshold can be universally applied to all genes and methods while controlling the genomewide error rate.
Significant genes detected from the dataset in Marioni et al (2008).
Significant Genes in Intersecting Sets  

Method  EBA (Affymetrix)  (LRT)  EBA (RNASeq)  edgeR  DESeq 
 
EBA (Affymetrix)  3641  3096  672  3127  3037  251  3183  960 
(LRT)  8641  790  8461  5400  308  8641  1116  
EBA (RNASeq)  790  790  789  275  790  700  
edgeR  8697  5516  308  8697  1351  
DESeq  7083  301  5644  1305  
 315  308  315  
 11915  3324  
 3331 
A summary of the missing values for each of the tests and the number of significant genes detected by other methods within those missing values.
Significant Genes from Sets of Missing Genes  

Method  EBA (Affymetrix)  (LRT)  EBA (RNASeq)  edgeR  DESeq  
EBA (Affymetrix)  14292  561  17  589  450  16  2550  1316 
(LRT)  75  3979  0  235  197  0  3064  2208 
EBA (RNASeq)  81  0  4726  235  197  0  3064  2208 
edgeR  0  0  0  0  0  0  0  0 
DESeq  0  0  0  0  0  0  0  0 
 422  973  5  1166  997  8947  4171  2406 
 2  0  0  0  0  0  915  0 
 3  0  0  0  0  0  856  1832 
Discussion
Although log_{2}ratios are widely used to compare two groups of expression data, there are limitations to using these statistics. The largest drawback to ratio statistics is that they are unstable as the denominator gets closer to zero. In addition, frequentist methods for constructing a corresponding variance and formally testing hypotheses of differential expression are unsatisfying and more complicated than typical scenarios [15, 16]. Due to these drawbacks, we proposed an alternative to testing for differential expression with all of the advantages of a log_{2}ratio statistic and none of its disadvantages.
We examined the proposed alternative, a proportion statistic, in four sets of simulations and two different sets of expression data. In simulations, the statistic , plus a constant and limma/EBA were robust to changes in distributional assumptions and the others were not. For the case of the Poisson distribution with rate parameter λ = 3, the statistic was underpowered, but otherwise and performed similarly well in simulations. The simulations suggest the use of in differential expression analyses because it uniformly preserved type I error and had competitive power. Note, however, that is not uniformly most powerful, and statistics derived from specific distributions can beat it when the distributional assumptions hold. The performance of the empirical Bayes analysis was competitive with in simulations but not in the analysis of the RNASeq dataset. Future research of interest may extend the test within an empirical Bayes framework, akin to what already exists for the log_{2}ratio. This may even further extend the clearly demonstrated feasibility of to detect differentially expressed genes.
Additionally, while the popular statistic performs sufficiently well under many simulation conditions, it suffers from problems with missing data in real data analysis problems when expression values are low. The addition of constants 0.05 and 0.5 appreciably improve simulation results and make the performance of nearly as good as (see Additional file 1). Nevertheless, this ad hoc procedure can be avoided using . In both the analysis of a cDNA microarray set and an RNASeq dataset, the log_{2}ratio based statistics led to missing values. Of these genes with missing ratio values, the proportion statistic was able to detect instances of statistically significant differential expression. We therefore recommend for general use over the other statistics discussed.
Conclusions
The use of the log_{2}ratio statistic to compare two expression values is challenged by denominators with near zero values. Thus, a reasonable alternative is to suggest a statistic that is not constrained by problems with very low expression values that still provides a meaningful test of differential expression. Using a proportion estimate instead of a ratio estimate does exactly that. The methods of this paper may only be used when data is naturally paired in test and reference samples, i.e. when logratios have traditionally been used. Our research provides several alternatives based on estimates of a proportion in both a frequentist and a Bayesian inference framework. We showed the performance of these alternatives and compared them to log_{2}ratio based tests in simulations and two gene expression datasets. In the gene expression analysis, all of the proportion methods performed better than ratio based methods for genes with low expression. For normal expression levels, inferential conclusions are similar, with the average proportion method, , plus a constant and the augmented log_{2}ratio method in limma/EBA, performing the best overall. The statistic has the added advantage that it does not require adjusting for an arbitrary constant that introduces bias in the estimate. Thus, tests of differential expression should consider proportion statistics over log_{2}ratios in future scientific studies.
Methods
This section describes the data generation process in our simulations and the data collection in the two datasets analyzed in this paper.
Simulations
The proposed test statistics were evaluated under four different distributions. Though sophisticated simulations can be used to mimic expression data, the simulations below use simple scenarios so as to examine the performance of test statistics under basic distributions and to compare the eight different methods clearly and meaningfully. The first set of simulated intensity values were sampled from an exponential distribution that mimics the values from a 16bit TIFF image of a cDNA microarray with respect to center and spread. The reference sample was taken from an Exp(1/4000) and the test sample was taken from a c × Exp(1/4000) where c was the foldchange value, c = 1,2,3,4,5. The four statistics, , and , were calculated for each value of c and sample sizes n = 3, 5, 10, 15, 20, 25, 30, 40, 50. Additionally we evaluated the ratio statistic after shifting values for an arbitrarily small constant set at either 0.05 and 0.5. For further comparison, a standard implementation of the limma/empirical Bayes method of Smyth (2004) was perfomed [17]. For simulations of count data values, we evaluated methods that account for overdispersion in the tests of differential expression using the edgeR and DESeq packages in Bioconductor [18, 19]. The implementation in both packages fixes a constant library size for each sample so that normalization is not executed. Sample sizes larger than n = 50 give simulation results similar to those for sample sizes of 50. In order to compare results, the pvalue for an independent ttest was computed, with a null hypothesis of no difference between the two sample means. The null hypothesis was rejected if the pvalue was below α = 0.05 and the proportion of rejections out of 1000 simulations was recorded (Table 1 and Additional file 1). The null hypothesis of no differential expression is equivalent to a fold change of one, c = 1. When the fold change is greater than one, we are calculating the power to detect differential expression. In this way, type I error and power were compared across the different methods. The results would be equivalent when using reciprocal fold changes instead. The simulations for an exponential distribution were repeated for an Exp(1/400) distribution, to study the effects of changing the scale.
A second set of simulated sampled intensity values from a Binomial(M = 10000, p = 0.5) distribution were obtained. The choice of this distribution was motivated by the derivation behind the maximum likelihood estimate of the proportion . The size was chosen to mimic the values from a 16bit TIFF image of a cDNA microarray with respect to center. Analogous simulations to the exponential above were conducted with respect to statistics, sample sizes, and fold changes. For the binomial distribution, foldchanges of 2, 3, 4, and 5 correspond to binomial probabilities of 2/3, 3/4, 4/5, and 5/6 respectively. The simulations were repeated for a Binomial(M = 100, p = 0.5) distribution, to study the effects of change in the size parameter. A third set of simulated sampled intensity values from a Poisson(λ = 3) distribution were obtained. This distribution is motivated by the derivation behind the likelihood ratio test used in Marioni et al (2008) [9]. The parameter λ = 3 was chosen to mimic the number of categories arising from a smooth histogram of values from the RNASeq data. The simulations were repeated for a Poisson(λ = 30) distribution, to study the effects of change in the rate parameter. Analogous simulations to the exponential above were conducted with respect to statistics, sample sizes, and fold changes.
A fourth set of simulated sampled intensity values from a Normal(μ = 5, σ = 1) distribution were obtained. This distribution was included since many analyses assume expression data to be normally distributed. The center was chosen to mimic values from cDNA data with mean 5,000 and standard deviation 1,000, scaled to Normal(μ = 5, σ = 1). Analogous simulations to the exponential above were conducted with respect to statistics, sample sizes, and fold changes. For the normal distribution, foldchanges of 2, 3, 4, and 5 correspond to test samples of Normal(c × μ, σ = 1). The simulations were repeated for a Normal(μ = 10, σ = 2) distribution, to study the impact of changing the parameters.
All simulations were conducted using R http://www.rproject.org and the code is available in Additional file 2.
Gene Expression Data from Mice using cDNA Microarrays
where X_{ trt } and X_{ cont } were either our proportion estimators, and or the usual log_{2}ratio estimators, and . Since the variability of the cDNA data resembles the exponential distribution, the assumptions for methods and do not hold and therefore they were not used. To account for multiple testing, the original analysis used the maxT stepdown procedure based on the tstatistics and found eight significantly differentially expressed probe sequences [12]. In order to explore the performance of alternative methods with both of the test statistics, the limma/EBA method of Smyth (2004) was computed [17]. Although this method was developed for log_{2}ratio values, we used the same programs on the proportion values as well.
Gene Expression Data from Human Kidney and Liver Cells using RNASeq
In order to examine the performance of our new approach on a sequencebased technology, we analyzed a set of RNASeq data discussed in Marioni et al (2008) [9]. This set of data compared the expression of 32,000 sequences in human kidney and liver cells extracted from the same person. The expression was also measured using Affymetrix U133 oligonucleotide arrays. Data was obtained from Supplemental Table 2 in the original manuscript. To compare our methods with those reported in the Supplemental Table 3 of their manuscript, we extracted the same five lanes of Illumina sequencing data corresponding to 3 pM concentrations of cDNA. We calculated both of the proportion tests outlined in the Results section, the ratiobased test provided in the Background section, and compared them to the methods from the original paper and more recent methods that account for overdispersion [18, 19].
for gene j. The maximum likelihood estimate for the alternative hypothesis from the above LRT is denoted by . The original paper also tested for differential expression on the Affymetrix platform for the same tissue samples. The methods employed were an empirical Bayes analysis with a false discovery rate of 0.1% [17]. More recent developments that account for overdispersion in the tests of differential expression were implemented using the edgeR and DESeq packages in Bioconductor [18, 19].
Appendix: Bayesian Estimation and Inference
[21]. To compare the performance of the Bayesian with frequentist statistics , and , credible intervals and confidence intervals can be constructed and coverage can be examined in simulations. For data where the difference of two proportions is required, the posterior distribution derived in [22] can be used.
Declarations
Acknowledgements
The authors would like to thank Suzanne Grindle in the Department of Microbiology at the University of Minnesota for her feedback, as well as that of the anonymous reviewers, which has resulted in improvements to the manuscript. Support for this research was provided by the Institute for Pure and Applied Mathematics at UCLA.
Authors’ Affiliations
References
 Canales R, Luo Y, Willey J, Austermiller B, Barbacioru C, Boysen C, Hunkapiller K, Jensen R, Knight C, Lee K, Ma Y, Maqsodi B, Papallo A, Peters E, Poulter K, Ruppel P, Samaha R, Shi L, Yang W, Zhang L, Goodsaid F: Evaluation of DNA microarray results with quantitative gene expression platforms. Nature Biotechnology 2006, 24: 1115–1122. 10.1038/nbt1236View ArticlePubMed
 Ramakers C, Ruijter J, LekanneDeprez R, Moorman A: Assumptionfree analysis of quantitative realtime polymerase chain reaction (PCR) data. Neuroscience Letters 2003, 339: 62–66. 10.1016/S03043940(02)014234View ArticlePubMed
 Zipper H, Brunner H, Bernhagen J, Vitzthum F: Investigations on DNA intercalation and surface binding by SYBR Green I, its structure determination and methodological implications. Nucleic Acids Research 2004, 32: e103. 10.1093/nar/gnh101PubMed CentralView ArticlePubMed
 DeRisi J, Iyer V, Brown P: Exploring the metabolic and genetic control of gene expression on a genomic scale. Science 1997, 278: 680–686. 10.1126/science.278.5338.680View ArticlePubMed
 Shalon D, Smith S, Brown P: A DNA microarray system for analyzing complex DNA samples using twocolor fluorescent probe hybridization. Genome Research 1996, 6: 639–645. 10.1101/gr.6.7.639View ArticlePubMed
 Irizarry R, Wu Z, Jaffee H: Comparison of Affymetrix GeneChip expression measures. Bioinformatics 2006, 22: 789–794. 10.1093/bioinformatics/btk046View ArticlePubMed
 Wang Z, Gerstein M, Snyder M: RNASeq: a revolutionary tool for transcriptomics. Nature Reviews Genetics 2009, 10: 57–63. 10.1038/nrg2484PubMed CentralView ArticlePubMed
 Morazavi A, Williams B, McCue K, Schaeffer L, Wold B: Mapping and quantifying mammalian transcriptomes by RNASeq. Nature Methods 2008, 5: 621–628. 10.1038/nmeth.1226View Article
 Marioni J, Mason C, Mane S, Stephens M, Gilad Y: RNASeq: an assessment of technical reproducibility and comparison with gene expression arrays. Genome Research 2008, 18: 1509–1517. 10.1101/gr.079558.108PubMed CentralView ArticlePubMed
 Agresti A, Coull B: Approximate is better than "exact" for interval estimation of binomial proportions. American Statistician 1998, 52: 119–126. 10.2307/2685469
 Brown L, Cai T, Dasgupta A: Interval estimation for binomial proportion. Statistical Science 2001, 16: 101–133.
 Ge Y, Dudoit S, Speed T: Resamplingbased multiple testing for microarray data analysis. Tech. rep., U. C. Berkeley Statistics Department 2003.
 Dudoit S, Shaffer J, Boldrick C: Multiple hypothesis testing in microarray experiments. Statistical Science 2003, 18: 71–103. 10.1214/ss/1056397487View Article
 Storey J, Tibshirani R: Statistical significance for genomewide studies. Proc Natl Acad Sci USA 2003, 100: 9440–9445. 10.1073/pnas.1530509100PubMed CentralView ArticlePubMed
 Fieller E: The biological standardization of insulin. Suppl to J R Statist Soc 1940, 7: 1–64. 10.2307/2983630View Article
 Kendall M, Stuart A: Advanced Theory of Statistics. London: Charles Griffin & Company; 1977.
 Smyth G: Linear models and empirical bayes methods for assessing differential expression in microarray experiments. Stat Appl Genet Mol Biol 2004, 3: Article 3.
 Anders S, Huber W: Differential expression analysis for sequence count data. Genome Biology 2010, 11: R106. 10.1186/gb20101110r106PubMed CentralView ArticlePubMed
 Robinson M, McCarthy D, Smyth G: edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics 2010, 26: 139–140. 10.1093/bioinformatics/btp616PubMed CentralView ArticlePubMed
 Callow MJ, Dudoit S, Gong EL, Speed TP, Rubin EM: Microarray expression profiling identifies genes with altered expression in HDL deficient mice. Genome Research 2000, 10(12):2022–2029. 10.1101/gr.10.12.2022PubMed CentralView ArticlePubMed
 Press S: Subjective and Objective Bayesian Statistics. Hoboken, NJ: Wiley; 2003.
 PhamGia T, Turkkan N, Eng P: Bayesian analysis of the difference of two proportions. Communications in Statistics  Theory and Methods 1993, 22(6):1755–1771. 10.1080/03610929308831114View Article
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.