Open Access

Differential expression analysis for paired RNA-seq data

BMC Bioinformatics201314:110

DOI: 10.1186/1471-2105-14-110

Received: 16 October 2012

Accepted: 1 March 2013

Published: 27 March 2013



RNA-Seq technology measures the transcript abundance by generating sequence reads and counting their frequencies across different biological conditions. To identify differentially expressed genes between two conditions, it is important to consider the experimental design as well as the distributional property of the data. In many RNA-Seq studies, the expression data are obtained as multiple pairs, e.g., pre- vs. post-treatment samples from the same individual. We seek to incorporate paired structure into analysis.


We present a Bayesian hierarchical mixture model for RNA-Seq data to separately account for the variability within and between individuals from a paired data structure. The method assumes a Poisson distribution for the data mixed with a gamma distribution to account variability between pairs. The effect of differential expression is modeled by two-component mixture model. The performance of this approach is examined by simulated and real data.


In this setting, our proposed model provides higher sensitivity than existing methods to detect differential expression. Application to real RNA-Seq data demonstrates the usefulness of this method for detecting expression alteration for genes with low average expression levels or shorter transcript length.


Gene expression profiles are routinely collected to identify differentially expressed genes and pathways across various individuals and cellular states. Sequencing-based technologies offer more accurate quantification of expression levels compared to other technologies. Early sequence-based expression measured transcript abundance by counting short segments, known as tags, generated from the 3’ end of a transcript. Tag-based methods include the Serial Analysis of Gene Expression (SAGE, [1]), Cap Analysis of Gene Expression (CAGE), LongSAGE, and massively parallel signature sequencing (MPSS). The development of deep sequencing technology enables simultaneous sequencing of millions of molecules and has led to advanced approaches for expression measurement [2, 3]. Digital gene expression - tag profiling [4] adapted the tag-based approach for use with the ‘next-generation’ sequencing platform. RNA-Seq is an alternative approach, that is an application of ‘whole genome shotgun sequencing’. Briefly, it entails generating a cDNA library by random priming off of fragmented RNA. The cDNA library is then subject to next-generation sequencing to generate short nucleotide sequences (reads) that correspond to the ends of the cDNA fragments. RNA-Seq aims to measure the entire transcriptome and is preferable to microarrays and tag-based approaches since it provides more information such as alternative splicing and isoform-specific gene expression with very low background signal and a wider dynamic range of quantification [5]. Moreover, recent experiments revealed that the RNA-Seq measures expression level with high accuracy and reproducibility [6-9].

Sequence-based approaches quantify gene expression as a ‘digital’ count and require modeling suitable for a count random variable. The Poisson distribution has been central in modelling expression data [10-12] and commonly applied to RNA-Seq data [6, 13]. In particular, Li et al. (2012) proposed a permutation-based approach to generate the null distribution [14]. However, Poisson-based approaches may not take all the variations between biological samples into account. The Beta-Binomial hierarchical model [15, 16], overdispersed logistic [17], and overdispersed log-linear models [18] were proposed to capture extra variance for each gene separately. Negative Bionomial models have been proposed to estimate the overdispersion parameter by shrinkage estimation [19-21], mean-dependent local regression [22], or empirically derived prior distribution [23]. Alternatively, beta-binomial [24] and Poisson mixture [25] models were proposed under the Bayesian modeling framework. Nonparametric method with resampling was also considered [26]. These approaches generally assume that samples under two groups are obtained independently. Recently, some of these approaches have been extended to deal with multi-factor design structures [14, 16, 21, 22].

Many practical RNA sequencing studies collect data with a paired structure, where the global expression profiles are measured before and after a treatment is applied to the same individual. Appropriate modeling of such data requires taking this design structure as well as the distributional property of the data into account. The Poisson model has been used to test the effect of drugs when the observation occurs as paired data, such as predrug and postdrug counts [27]. Lee [28] considered a mixture model to account for extra variance among individuals over the level that would be expected under the Poisson model. These approaches assume independence of the paired observation conditional on the individual mean. Bivariate Poisson or negative binomial distribution are alternative choices to model correlations between observations [29, 30].

In this paper, we propose a Bayesian hierarchical approach to modeling paired count data that separately accounts for the within and between individual variability from a paired data structure. Our work adopts the Poisson-Gamma mixture model [28] and utilizes a Bayesian approach to evaluate the expression difference. We note that the Bayesian models are widely utilized in microarray studies and have improved sensitivity to detect differential gene expression by sharing information among genes [31]. Mixture models are also commonly used to model differential expression, where non-differentially expressed and differently expressed genes correspond to different mixture components. Various mixture model specifications have been considered in the literature. The gamma and log-normal distribution were used to model the expression levels [32, 33]. Smyth [34] assumed a point mass at zero for log scaled fold change for null genes and a normal distribution centered at zero for non-null genes. Lonnstedt et al. [35] and Gottardo et al. [36] proposed a mixture of two (null and non-null) or three normal (null, over, and under expression) distributions. Non-parametric approaches have also been utilized [31, 37]. Lewin et al. [38] discussed various choices of mixture component priors and model checking.

The rest of this manuscript is organized as follows. Data Section introduces the biological problem and data that motivated this study. Methods Section presents our parametric model and the Bayesian method to identify genes with differential expression levels. The performance of the proposed model is examined by Simulations. Two sets of simulation studies are conducted: (1) those based on the model assumption to investigate the accuracy of the proposed method on parameter estimation, and (2) those based on mimicking the motivating data set to examine the robustness of the proposed method. Finally, the proposed method is applied to real data with detailed discussion of the results and comparisons with other methods.


Qian et al. (Qian F. et al.: Identification of genes critical for resistance to infection by West Nile virus using RNA-Seq analysis, submitted) designed an RNA-Seq experiment to study human West Nile virus (WNV) infection. One objective of this study was to identify altered genes/transcripts from viral infection of primary human macrophages in comparison to uninfected samples. This study naturally has a paired design structure. A total of 10 healthy donors were recruited according to the guidelines of the human research protection program of Yale University and cells were isolated from fresh heparinized blood samples for infection with WNV (strain CT 2741, MOI=1, for 24 hours) as described previously [39]. PolyA+ RNA was prepared from uninfected and WNV-infected primary macrophages, fragmented, and subjected to sequencing using the Illumina Genome Analyzer 2. Approximately 50 million quality filtered reads were obtained from each sample, and about 85% were mapped to the human transcriptome (hg19) with ENSEMBL transcript annotations (Release 57) using TOPHAT v.1.1.4 [40]. Genes and transcript isoforms were scored for expression by a maximum likelihood based method implemented in Cufflinks v.0.9.3 [41]. To analyze differential expression, the data were first converted from the FPKM unit (fragments per kilobase of exon per million fragments mapped) to the number of reads originated from each transcript isoform. The trimmed-mean method [42] was applied to further normalize the count expression values. The processed data contains transcript-level expression counts from a total of 20 samples consisting of 10 pairs of uninfected and virus infected samples. For differential expression analysis, we removed transcripts with less than 10 total counts across 10 uninfected samples or no observed count from 6 or more individuals in the uninfected conditions. After these steps, 37,111 transcripts were considered for data analysis.


Bayesian mixture model for paired counts

We now describe our Bayesian hierarchical mixture model to identify differentially expressed genes/transcripts from paired RNA-seq data. As noted above, such data arise naturally from experiments measuring the biological change from treatments. We start with an overdispersed count model [28]. The observations are denoted by a pair (Ygi1,Ygi2), for gene g = 1, …, G and individual i = 1, …, n, where Ygi1 is the observed baseline expression level and Ygi2 is the observed level after treatment. The sizes of the libraries are denoted as Ni1 and Ni2, respectively. Let λ gi denote the true baseline expression relative to the library size. Then, Ygi1 can be modeled as a Poisson random variable with mean λ g i Ni1. Let χ g denote the expression level fold change after treatment so the true expression level is χ g λ gi Ni2, then Ygi2 can be modeled as a Poisson random variable with mean χ g λ gi Ni2. Our goal is to test whether there is any treatment effect, i.e., χ g  ≠ 1, where
Y gi 1 | λ gi , χ g Poisson ( N i 1 λ gi ) , Y gi 2 | λ gi , χ g Poisson ( N i 2 λ gi χ g ) .
It has been shown that the variability among technical replicates for RNA-Seq data can be captured by the Poisson distribution [6]. However, greater variance can be expected, if observations are collected from individuals with differences in the underlying biological system. One way to model the overdispersion among the Poisson counts is to mix it with a Gamma distribution [28]. In this model, we use a Gamma distribution to model the baseline expected expression, λ gi , across individuals with shape parameter α g and rate β g ;
f λ ( λ gi ) = β g α g Γ ( α g ) λ gi α g - 1 e - β g λ gi .

This model allows us to obtain a simpler form of the predictive density, i.e., the λ g i ’s can be integrated out (see Appendix).

Assuming independence between the baseline expression and treatment effect, we use a two-component mixture model to characterize the fold change distribution, where the expression change state of each gene is defined by a latent variable z g , with z g  = 0 corresponding to no change and z g  = 1 otherwise. We assume that z g has a probability of Π0 for equal expression, i.e., z g  = 0, and a probability of Π1 = 1 - Π0 for differential expression. Given a state, 0 or 1, the log-scaled fold change is assumed to follow a normal distribution. Under equal expression, the log-fold change is assumed to arise from a normal distribution centered at zero and variance σ 0 2 For genes with differential expressions, if we assume their log-fold changes follow a normal distribution centered around zero, we implicitly assume that there is equal chance for a gene to be either over or under expressed. However, more genes were under-expressed after the viral infection for the data set described earlier, with 3.2% of transcripts showing increased expression by more than 4 fold after the infection whereas 4.3% showing reduction by more than 4 folds. To accommodate this asymmetry, we assume the log-fold change for non-null genes arises from a normal distribution with mean μ1, which may be different from 0, and variance σ 1 2
log ( χ g ) | ( z g = 0 ) Normal ( 0 , σ 0 2 ) log ( χ g ) | ( z g = 1 ) Normal ( μ 1 , σ 1 2 )
Collecting all the components discussed so far, the model can be summarized in Figure 1. Under this set-up, the goal is to estimate the posterior probability that a specific gene is differentially expressed after treatment, i.e., P r(z g  = 1|data). Genes can then be inferred to DE (Differential Expression) or EE (Equal Expression) according to these probabilities.
Figure 1

Diagram illustrating the hierarchical model for paired RNA-Seq data.

To complete our model description, we need to specify prior assumptions for the unknown model parameters, θ = ( { α g } , { β g } , Π 0 , Π 1 , σ 0 2 , μ 1 , σ 1 2 ) In our implementation, we assume non-informative priors for these unknown parameters:
  1. 1.

    (Π 0, Π 1) Dirichlet(1,1), i.e., Π 0 Uniform(0, 1).

  2. 2.

    Each α g and β g has a non-informative prior.

  3. 3.

    p ( σ 0 2 ) 1 / σ 0 2 and p ( σ 1 2 ) 1 / σ 1 2

  4. 4.

    μ 1 has an improper prior.

  5. 5.

    Joint independency among all the parameters.


Parameter estimation via Markov chain Monte-Carlo (MCMC)

In this section, we describe the Gibbs sampling algorithm [43] that we use to iteratively sample model parameters from their conditional distributions given the other parameters and the observed data. First, we evaluate the conditional distribution of parameters (α g , β g ) characterizing the baseline expression distribution (λ gi ). These parameters are separately updated using the Metropolis-Hastings algorithm. For the latent state z g and expression level change χ g , the state z g is first proposed and then χ g is sampled given the state. Lewin et al. [38] discussed this type of move with various choices of the mixture distribution. Details of our updates on the pair of (χ g , z g ) are described in the Appendix. Mixing proportions (Π0, Π1) and hyper-parameters for the mixture distribution ( σ 0 2, σ 1 2, μ1) are sampled from their conditional posterior distributions which can be derived in closed forms.

DE classification and false discovery rate estimation

The MCMC algorithm generates random samples from the joint posterior distribution of all model parameters. These samples are then used to infer the status of differential expression. One way to select a set of interesting genes is to rank genes using estimated posterior-mean fold change
χ g ̂ exp 1 T t = 1 T log χ g ( t ) ,
where T is the number of iterations used for inference after the burn-in period and χ g ( t ) is the sampled value for the fold change on iteration t of the Gibbs sampling algorithm. Another way to select DE genes is to consider the latent variable, z g . During the MCMC iteration, the expression state is sampled along with the fold change estimates. These MCMC samples can be used to approximate the posterior probability of differential expression by counting the proportion of sampled states being differentially expressed:
p g = P ( z g = 1 | data ) 1 T t = 1 T I z g ( t ) = 1 .
The Bayes’ rule assigns a gene’s expression status according to the largest posterior probability. An alternative is to classify a gene if the posterior probability of being non-null is greater than a threshold (p thres ) : p g  > p thres . For example, one choice would be p thres  = 0.5. The false discovery rate can be estimated from these posterior probabilities [31]:
FDR ̂ = 1 ( p g > p thres ) g : p g > p thres ( 1 - p g )

The method was implemented in R and is available at

Results and discussion


Simulations based on the model assumptions

The first part of the simulation was conducted to examine the performance of the proposed approach when the data are generated under the model assumptions. For 10,000 genes and 10 individuals, we simulate expression counts both before and after treatment according to Equation 1. Library sizes are sampled uniformly from 7 to 18 millions and relative expected baseline expression λ g i are drawn from a Gamma distribution with shape 0.1 and rate 1,000. For simplicity, we consider a two-component log-normal mixture model for effect size. For the null genes (90%), the log-scaled effect is sampled from a normal distribution with a mean 0 and a standard deviation (σ0) 0.1, whereas the log-effects are sampled from a normal distribution with mean (μ1) of 1.5 and standard deviation (σ1) of 0.5 for the non-null genes. For the simulation studies, the true library sizes are used for the parameter estimation.

Results in Table 1 show that the proposed approach estimates the model paramters well. With a posterior probability cutoff of 0.5, the algorithm identified more than 97% of true DE genes with an FDR of approximately 1%. Figure 2 illustrates the estimated fold changes showing the good performance of our algorithm.
Table 1

Posterior means of the parameters in the model


True parameter

Posterior mean

σ 0 2


0.013 (0.002)

μ 1


1.501 (0.015)

σ 1 2


0.238 (0.015)

Π 1


0.099 (0.001)
Figure 2

Estimated fold change. The left panel shows the distribution of the estimated fold changes under EE and DE by the Bayes’ rule. The red lines are the true fold change distributions. The right panel displays the relationship between the estimated and true fold change.

Simulations based on the empirical data

In the second part of the simulation, we assume that the expression abundance is measured for 5,000 genes simultaneously before and after a given treatment. The number of individuals is set to be 10 for the relatively larger sample case (cases 1 and 4), 5 for the medium (cases 2 and 5), and 3 for the relatively smaller sample case (cases 3 and 6). The size of each library is randomly sampled from 1.8 to 3 million to have simulated count distribution compatible with the real data distribution. The infected set of the RNA-seq data (Data Section, Qian F. et al. for details) was used as the expected baseline count data to mimic the observed mean-specific dispersion. First, we sample 5,000 gene indices with replacement to get the expected baseline expression. Expression counts from the selected indices are summarized by a matrix where rows from this data matrix correspond to the selected genes in the original data matrix and columns correspond to individuals. Then, the relative expression (λgi, i = 1, …, N, Equation 1) is computed proportional to the total counts in each sample.

Among 5,000 genes, the first 4,000 are assumed to have no change (z g  = 0) and their log-fold changes, log(χ g ), are sampled from a normal distribution with a mean of 0 and a variance of 4 × 10-4. For the rest of non-null genes, we considered the following two scenarios. An empirical set-up (cases 1, 2, 3) utilizes nominal fold change from the uninfected data set. Cases 4, 5, and 6 consider a theoretical setup, where the log-scaled fold change is drawn from a normal distribution with a mean of zero and a variance of 1. We further filter out non-null genes whose true fold changes are less than 1.4.

Each case was repeated 100 times. We compare the performance of our approach with DESeq (version 1.8.3) [22] and edgeR (version 2.6.10) [21], two widely used methods for RNA-seq data for the purpose of identifying differentially expressed genes. These two methods assume a negative binomial distribution to explain the variance due to the replicate. DESeq utilizes a smoothing curve to compute the overdispersion as a function of the average expression level. An option ‘pooled-CR’ is used to estimate the overdispersion parameter [44]. In edgeR, a common dispersion setting is used which assumes a consistent overdispersion across all the features and estimates the parameter using a common likelihood function. A paired design can be incorporated by utilizing generalized linear model. For each application, the true library sizes are used as the library size inputs.

Table 2 summarizes the results of our approach. Overall, we see excellent performance of our method in inferring the expression change status (reflected in a high correlation with the true status) as well as the parameters characterizing the distributions for the null and non-null genes. Since true expression states are known in the simulation, we call a feature to be differentially expressed if p g  > p thres and compare the estimated false discovery rate with the true value (Figure 3). The FDR is estimated well for cases with large sample sizes as p thres increases, while it is slightly under-estimated for small sample sizes. Figure 4 illustrates the receiver operating characteristics averaged across 100 simulations under four different simulation settings. For each setting, the true positive rate is plotted against the false positive rate. The corresponding rates are computed by ranking genes from the largest posterior probability by the Bayesian approach (then, the largest fold change, if tied) or from the smallest p-value by each of the other methods. The Bayesian approach shows higher sensitivity at the same level of false positive rates than the edgeR and DESeq. Especially, the Bayesian model achieves better performance for smaller sample size and empirical fold change setting (case 2 or 3).
Table 2

Estimated posterior means and results for empirical simulation


Case 1

Case 2

Case 3





μ 1

-0.170 (0.037)

-0.169 (0.041)

-0.157 (0.041)

σ 0 2








σ 1 2

0.984 (0.104)

0.968 (0.115)

0.955 (0.110)

Π 1

0.151 (0.004)

0.153 (0.005)

0.156 (0.006)

cor ( χ g , χ g ̂ )

0.972 (0.006)

0.993 (0.003)

0.953 (0.011)


0.030 (0.008)

0.046 (0.011)

0.068 (0.013)


0.024 (0.004)

0.037 (0.005)

0.049 (0.006)


0.928 (0.014)

0.866 (0.020)

0.802 (0.025)


0.995 (0.001)

0.994 (0.002)

0.991 (0.002)


Case 4

Case 5

Case 6





μ 1

0.007 (0.035)

0.006 (0.038)

-0.002 (0.037)

σ 0 2








σ 1 2

1.172 (0.048)

1.151 (0.059)

1.140 (0.050)

Π 1

0.179 (0.003)

0.183 (0.004)

0.188 (0.005)

cor ( χ g , χ g ̂ )

0.990 (0.002)

0.979 (0.004)

0.965 (0.007)


0.030 (0.008)

0.044 (0.009)

0.064 (0.012)


0.021 (0.004)

0.031 (0.005)

0.042 (0.006)


0.953 (0.011)

0.906 (0.015)

0.862 (0.020)


0.995 (0.001)

0.992 (0.002)

0.989 (0.002)

Operating characteristics are based on the Bayes rule. cor ( χ g , χ g ̂ ) is the correlation coefficient between the true difference and the estimated difference.
Figure 3

False discovery rate from the simulation. True and estimated false discovery rates are compared across different threshold for posterior probability. Solid lines are true values and dashed lines are estimated values averaged over all simulations. Left panel shows the result from simulation cases 1, 2, and 3, where non-null fold change is empirically generated. Results for cases 4, 5, 6 and 7,8 are illustrated on the middle panel and right panel, respectively.
Figure 4

Simulation results. Operating characteristics for 8 simulation settings are plotted with red, green, and blue lines for the Bayes, DESeq, and edgeR methods, respectively.

We further considered a simulation scenario similar with the real data. As shown in the data application, the log-scaled fold change estimated from the data has larger variance under null component. We set the null component variance to be 0.35 and repeated the simulation 50 times. For features in the non-null group, log-fold change was sampled from a normal distribution with a mean of -0.45 and a variance of 4. Simulation was performed with the sample size of 10 (case 7) and the size of 5 (case 8). Averages of the parameter estimates ( μ 1 , σ 0 2 , σ 1 2 , Π 1 ) for cases 7 and 8 are (-0.42, 0.35, 3.92, 0.20) and (-0.42, 0.35, 3.85, 0.21), respectively. Similarly with the cases 1 through 6, the estimated false discovery rate is examined (Figure 3) and performance of the proposed approach is compared with two existing methods (Figure 4).


Differential expression analysis with the Bayesian modeling

In this section, we apply our method to the motivating data set described in the Data Section. Initial values of the model parameters are calculated directly from the data. The MCMC sampling is run 4,000 iterations after discarding the first 8,000 iterations. On average, computational time was around 5 minutes per every 100 iterations. The number of total iterations and burn-in period are determined by monitoring trace plots of MCMC samples (Figure 5 (a)). We estimate the mixing proportion to be 0.88 and 0.12 for EE and DE group, respectively. The posterior means for the parameters μ1 and σ 1 2 are -0.45 and 4.04, respectively. The null group has a variance of 0.35. Under the Bayes rule (p thres  = 0.5), 2,352 transcripts are classified into DE after the West Nile virus infection. The estimated FDR is 16.2% from Equation 4. Figure 5 (b) illustrates the fold change distributions under DE and EE based on the Bayes rule classification. The estimated fold changes are plotted in Figure 6 (a) against their DE posterior probabilities.
Figure 5

Trace of parameters regarding the mixture distrubution. Trace of parameters regarding the mixture distrubution (a) and distributions of fold change estimates for genes classified into EE and DE groups, respectively, by the Bayes’ rule (b).
Figure 6

Result of the Bayesian approach and comparison with other existing methods. Posterior probabilities against estimated fold change (a) and consistency between the Bayesian approach and existing approaches when the same number of top-ranked transcripts are chosen (b).

Comparisons with existing methods

In this section, we compare DE analysis results between our approach and existing methods. The DESeq or edgeR is applied to the same data set and top 2,352 DE transcripts are selected by their p-values. The edgeR shows higher consistency with our Bayesian model with 63.5% of overlap than the DESeq having 34.3% of overlapping transcripts. Specifically, 832, 632, and 1,364 transcripts are detected uniquely by the Bayes, edgeR, and DESeq, respectively (Figure 6). Our approach detects those having low average expression and high fold change. In contrast, other approaches tend to identify more transcripts with high expression level and low fold change (Figure 7). Transcripts which have evidence of differential expression only by the proposed model often have large inter-individual variation. Their fold changes are high after the treatment except a few low expressed individuals. Figure 8 illustrates an example of uniquely identified transcript by our proposed approach. This transcript is a product of SLAMF7, which is known to play a role in natural killer cell activation [45]. Another interesting feature of the proposed method is that the proportion of DE genes is consistent across transcript length. Among the bottom 10% of the short transcripts, 4.6% are detected by the proposed approach while 2.4% are found by other methods. Among the top 10% of the long transcripts, 6.5% are detected by the proposed method whereas 7.4 and 8.9% are detected by DESeq and edgeR, respectively. To investigate more details, Figure 9 illustrates the DE proportion when the transcripts partitioned into 10 equal-sized bins based on their length.
Figure 7

Comparison of DE transcripts. Commonly detected transcripts by all three methods are labeled in purple: log-scaled Bayesian estimated fold change against log-scaled average expression. Other three panels show DE transcripts detected by each of three methods. They are labeled in red, green, and blue for the Bayes, DESeq, and edgeR methods, respectively.
Figure 8

Example of uniquely selected by the proposed Bayesian model. Illustration of expresseion values from a transcript detected by the proposed method only.
Figure 9

DE proportion and transcript length. Proportion of DE transcripts over their average expression level. Transcripts are partitioned into 10 equal-sized bins by their expression levels. The proportion of transcripts inferred to be DE is plotted on the y-axis. Red, green, and blue lines are from Bayes, DESeq, and edgeR methods, respectively.

Bioinformatics annotations of the results

Pathway-level analysis is one effective way to summarize biological relevance of differentially expressed genes. We perform gene enrichment analysis using DAVID (http://​david.​abcc.​ncifcrf.​gov/​). 2,352 DE transcripts inferred from our approach are mapped to 1,518 DAVID IDs for functional annotation clustering. Cluster 1 (DAVID enrichment score: 11.39) represents cellular response to the WNV infection. Specifically, pathways in cluster 10 (score: 2.72) are related to the activation of the macrophage after the virus infection. Molecular functions of these transcripts are characterized by their cytokine production (GO:0001817) in cluster 3. Cluster 8 (score: 2.89) consists of transcripts that are involved in apoptosis (GO:0042981) and the regulation of programmed cell death (GO:0043067). The induction of apoptosis by WNV is essential in the regulation of pro-inflammatory responses, and has been previously reported in cell lines and neuronal cell types [46, 47]. These clusters and related top pathways are reported on Table 3 with enrichment scores and p-values.
Table 3

Selected pathways from the functional analysis





Cluster 1

score: 11.39


Defense response




Response to wonding




Inflammatory response




Cluster 2

score: 5.43


Response to molecule of lipopolysaccharide




Response to cytokine stimulus




Response to bacterium




Cluster 3

score: 5.19


Regulation of cytokine production




Positive regulation of cytokine production




positive regulation of multicellular organismal process




Cluster 8

score: 2.89


Regulation of apoptosis




Regulation of programmed cell death




Regulation of cell death




Cluster 10

score: 2.72


Leukocyte activation




Cell activation




T cell activation




Count column indicates the number of DAVID IDs associated with each pathway.


In this paper, we have presented a hierarchical mixture model for the identification of differential gene expression from RNA-Seq data motivated by a West Nile Virus study, which collected samples as multiple pairs, i.e. pre- vs. post-treatment for each individual. While such design is common in biological investigations, few existing methods analyze such data appropriately. With a hierarchical Bayesian mixture model coupled with inference through MCMC, our approach incorporates variability across genes, individuals, and treatment effects in the context of a paired experiment. Application to both simulated and real data demonstrates that our model and implementation is suitable for paired design, having distinct advantages compared to the existing methods.

Simulation study suggests that our Bayesian setting can have better power to detect differential gene expression. In the real data application, our proposed is able to identify transcripts with large treatment effects but low expression levels, whereas these transcripts were not inferred to be differentially expressed by other approaches. This is likely due to the more flexible and adaptable modeling of variance across individuals in our approach. Further examination of the characteristics of these top-ranked transcripts shows that the proportion of top-ranked transcripts in the short transcript group is consistent with the proportion in the long transcript group. On the other hand, the gene sets detected by the existing approaches show a bias towards longer transcripts, as has been noted in the literature before [48, 49]. Our model reduces this bias and as a result facilitates detection of some short-length differentially expressed transcripts that the other approaches miss.

We have assumed that the log-fold change arises from a mixture of two normal distributions. Under DE, the model allows the mean of log-fold change distribution not to be restricted at zero. By doing so, our proposed model can be applied to the data showing asymmetry between over and under expression. A normal distributional assumption is shown to be robust from simulation study under empirical fold change scenarios. Other possible choices for the null genes include a point mass at 0 [50], uniform distribution around 0, and a log-Gamma distribution with a mean 0. Similar distributional assumptions can be made for the non-null genes under the two-component mixture set-up. Alternatively, one can consider a mixture of three components consisting of equal, over, and under expression states. Further extension can be considered by allowing variation in the magnitude of expression change across individuals.


Variability across individuals

The Poisson-Gamma setting (Equation 1 and 2) allows extra variance among count expression values [28]. The variance of the count is given as
Var ( Y gi 1 ) = E ( Var ( Y gi 1 | λ gi ) ) + Var ( E ( Y gi 1 | λ gi ) ) = E ( N i 1 λ gi ) + Var ( N i 1 λ gi ) = N i 1 α g β g 1 + N i 1 β g .

Modeling details

The joint density of z g and χ g is
p ( χ g , z g ) Π 0 LogNormal ( 0 , σ 0 2 ) I ( z g = 0 ) + Π 1 LogNormal ( μ 1 , σ 1 2 ) I ( z g = 1 ) .
Let θ be a vector of all model parameters, θ = α g , β g , π 0 , π 1 , σ 0 2 , μ 1 , σ 1 2 The complete likelihood of the model is
p ( Y , χ , z | θ ) = g p ( Y g , χ g , z g | θ ) = g p ( Y g | χ g , z g , θ ) p ( χ g | z g , θ ) p ( z g | θ ) = g i p ( Y gi | χ g , z g , θ ) p ( χ g | z g , θ ) p ( z g | θ ) = g i p ( Y gi | λ gi , χ g , z g , θ ) p ( λ gi | θ ) d λ gi × p ( χ g | z g , θ ) p ( z g | θ )
Here, some details on the integral over λ gi follow.
p ( Y gi | λ gi , χ g , z g , θ ) p ( λ gi | θ ) d λ gi = ( N i 1 λ gi ) y gi 1 ( N i 2 λ gi χ g ) y gi 2 y gi 1 ! y gi 2 ! e - N i 1 λ gi - N i 2 λ gi χ g × f λ ( λ gi ) d λ gi = Γ ( y gi 1 + y gi 2 + α g ) y gi 1 ! y gi 2 ! Γ ( α g ) β g β g + N i 1 + N i 2 χ g α g × N i 1 β g + N i 1 + N i 2 χ g y gi 1 N i 2 χ g β g + N i 1 + N i 2 χ g y gi 2
p ( Y , χ , z | θ ) = g i Γ ( y gi 1 + y gi 2 + α g ) y gi 1 ! y gi 2 ! Γ ( α g ) × β g β g + N i 1 + N i 2 χ g α g × N i 1 β g + N i 1 + N i 2 χ g y gi 1 × N i 2 χ g β g + N i 1 + N i 2 χ g y gi 2 × p ( χ g | z g , θ ) p ( z g | θ ) .
After integrating over the expected gene- and individual-specific relative baseline expression (λ gi’s), the posterior density of unknown parameters is proportional to the product of likelihood and prior density.
p ( χ , z , θ | Y ) p ( Y , χ , z | θ ) p ( θ )

We use the non-informative prior distributions for the unknown model parameters specified in the Methods Section.

Parameter estimates by the Metropolis-Hastings algorithm (MCMC)

We infer the posterior distributions using the Gibbs sampling [43], which iteratively samples model paramters from the conditional distribution of each patermter given the other parameters. In this section, we describe the procedure for the posterior inference.


Update α g . The conditional distribution for α g does not have a closed form expression. We use the Metropolis-Hastings algorithm to sample this parameter. More specifically, we update the parameter by proposing α g new N ( α g old , σ α 2 ) at each iteration, where σ α is set to be 0.1. The proposal is accepted with probability min{1, r}, where r is the acceptance ratio.
r = p ( z , χ , α g new , θ - α g old | Y ) p ( z , χ , α g old , θ - α g old | Y )
where θ - α g old is the current values of the parameters except α g and
p ( z , χ , α g , θ - α g | Y ) i Γ ( y gi 1 + y gi 2 + α g ) Γ ( α g ) × β g β g + N i 1 + N i 2 χ g α g .

If the proposal is accepted, we replace the old α g with the new one. Otherwise, α g stays at the current value.


Update β g . Similar to sample α g , we propose β new N(β old , σ β2), where σ β is set to be 1. The acceptance ratio is calculated as
p ( z , χ , β g , θ - β g | Y ) i β g α g ( β g + N i 1 + N i 2 χ g ) y gi 1 + y gi 2 + α g .

Similarly, θ - β g is the vector of parameters except β g . For the evaluation of the acceptance probability, updated value of α g in the Step 1 will be used.


Update (χ g , z g ) by utilizing generalized Metropolis-Hastings. Lewin et al. [38] pointed out that χ g and z g have to be jointly estimated since the supporting space of χ g depends on the choice of z g . For example, χ g is a point mass at one if z g  = 0. To estimate a pair of (χ g , z g ), they proposed the state z g first and then updated χ g |z g . By utilizing this approach, we adopt the following steps to sample (χ g , z g ).

(Step 3-1) Generate z g new from the Bernoulli distribution, with P ( z g new = 0 ) = Π 0 old

(Step 3-2) Then, χ g new is proposed from LogNormal(0, V g ) if z g new = 0 Otherwise, it is sampled from LogNormal(M g , V g ). The mean and variance of the log-normal proposal distribution are computed from the observed counts. First, we collect individuals whose pre- and post-treatment counts are non-zero for each gene, separately. Then, M g is computed as a median of log ( y gi 1 N i 1 y gi 2 N i 2 ) for such individuals. The variance of these values can be used as V g , however, this estimate often gives an extreme value. In data analysis, we trim the estimates at 25th and 75th percentiles when the sample size is 10. For small sample case, the median of V g ’s is used as the proposal variance.

Alternative description

Define Q ( χ g new , z g new | χ g old , z g old ) to be a proposal density from the current values ( χ g old , z g old ) to the proposed values. In our approach, the proposal density does not depend on the current values, i.e., we use the independence chain Metropolis-Hastings. The proposal distribution is given by
Q χ g new , z g new | χ g old , z g old Π 0 old LN ( 0 , V g ) I z g new = 0 + Π 1 old LN ( M g , V g ) I z g new = 1
The acceptance probabilty is min{1, r} where r is one of followings:
z g old = 1 , z g new = 1 : r = L N 1 ( χ g new ) t ( χ g new ) L N 1 ( χ g old ) t ( χ g old ) × LN ( χ g old ; M g , V g ) LN ( χ g new ; M g , V g ) z g old = 1 , z g new = 0 : r = L N 0 ( χ g new ) t ( χ g new ) L N 1 ( χ g old ) t ( χ g old ) × LN ( χ g old ; M g , V g ) LN ( χ g new ; 0 , V g ) z g old = 0 , z g new = 1 : r = L N 1 ( χ g new ) t ( χ g new ) L N 0 ( χ g old ) t ( χ g old ) × LN ( χ g old ; 0 , V g ) LN ( χ g new ; M g , V g ) z g old = 0 , z g new = 0 : r = L N 0 ( χ g new ) t ( χ g new ) L N 0 ( χ g old ) t ( χ g old ) × LN ( χ g old ; 0 , V g ) LN ( χ g new ; 0 , V g )

where t ( χ g ) = i χ g y gi 2 ( β g + N i 1 + χ g N i 2 ) y gi 1 + y gi 2 + α g, LN0 is a probability density function for log-normal distribution with mean zero and variance σ 0 2 , old Similarly, LN1 is a log-normal density centered at μ 1 new and variance σ 1 2 , old


Update σ 0 2, μ1, σ 1 2, which are hyper-paramaters from the distribution of χ g . Since it has a closed form for the posterior density conditional on all other parameters, we can directly sample those parameters.
σ 0 2 , new InvGamma ( z g = 0 ) 2 , 1 2 z g = 0 log ( χ g ) 2 μ 1 new Normal z g = 1 log ( χ g ) 2 ( z g = 1 ) , σ 1 2 ( z g = 1 ) σ 1 2 , new InvGamma ( z g = 1 ) 2 , 1 2 z g = 1 ( log χ g - μ 1 ) 2

where ( z g = 0 ) = g I ( z g = 0 ) and ( z g = 1 ) = g I ( z g = 1 )


Update the mixing proportions (Π0, Π1). We assume a Dirichilet prior for the mixture probabilities. Using Gibbs sampling scheme, these weight parameters are updated from Dir(1 + (z g  = 0), 1 + (z g  = s1)).



This work was supported in part by Grant GM59507 from NIH, 5T15LM007056-25 from PHS/DHHS, UL1 RR024139 from Yale CTSA grant, and awards from the NIH (HHS N272201100019C, AI 070343, AI 089992).

Authors’ Affiliations

Department of Biostatistics, Yale School of Public Health
Department of Statistics, George Washington University
Novartis Institutes for BioMedical Research
Section of Rheumatology, Yale School of Medicine
Department of Microbiology and Immunology, University of Maryland School of Medicine


  1. Velculescu V, Zhang L, Vogelstein B, Kinzler K: Serial analysis of gene expression. Science. 1995, 270: 484-487. 10.1126/science.270.5235.484.View ArticlePubMed
  2. Margulies M, Egholm M, Altman W, Attiya S, Bader J, Bemben L, Berka J, Braverman M, Chen Y: Genome sequencing in microfabricated high-density picolitre reactors. Nature. 2005, 437: 376-380.PubMed CentralPubMed
  3. Bennett S, Barnes C, Cox A, Davies L, Brown C: Toward the 1,000 dollars human genome. Pharmacogenomics. 2005, 6: 373-382. 10.1517/14622416.6.4.373.View ArticlePubMed
  4. ‘t Hoen P, Ariyurek Y, Thygesen H, Vreugdenhil E, Vossen R, de Menezes R, Boer G, van Ommen G, den Dunnen J: Deep sequencing-based expression analysis shows major advances in robustness, resolution and inter-lab portability over five microarray platforms. Nucleic Acids Res. 2008, 36: e141-10.1093/nar/gkn705.PubMed CentralView ArticlePubMed
  5. Wang GMZ, Snyder M: RNA-Seq: a revolutionary tool for transcriptomics. Nat Rev Genet. 2009, 10: 57-63. 10.1038/nrg2484.PubMed CentralView ArticlePubMed
  6. Marioni J, Mason C, Mane S, Stephens M, Gilad Y: RNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays. Genome Res. 2008, 18: 1509-1519. 10.1101/gr.079558.108.PubMed CentralView ArticlePubMed
  7. Mortazavi A, Williams B, McCue K, Schaeffer L, Wold B: Mapping and quantifying mammalian transcriptomes by RNA-Seq. Nat Methods. 2008, 5: 621-628. 10.1038/nmeth.1226.View ArticlePubMed
  8. Miller N, Kingsmore S, Farmer A, Langley R, Mudge J, Crow J, Gonzalez A, Schilkey F, Kim R, van Velkinburgh J, May G, Black C, Myers M, Utsey J, Frost N, Sugarbaker D, Bueno R, Gullans S, Baxter S, Day S, Retzel E: Management of high-throughput DNA sequencing projects: Alpheus. J Comput Sci Syst Biol. 2008, 1: 132-148. 10.4172/jcsb.1000013.PubMed CentralPubMed
  9. Fu X, Fu N, Guo S, Yan Z, Xu Y, Hu H, Menzel C, Chen W, Li Y, Zeng R, Khaitovich P: Estimating accuracy of RNA-Seq and microarrays with proteomics. BMC Genomics. 2009, 10: 161-10.1186/1471-2164-10-161.PubMed CentralView ArticlePubMed
  10. Audic S, Claverie J: The significance of digital gene expression profiles. Genome Res. 1997, 7: 986-995.PubMed
  11. Madden S, Galella E, Zhu J, Bertelsen A, Beaudry G: SAGE transcript profiles for p53-dependent growth regulation. Oncogene. 1997, 15: 1079-1085. 10.1038/sj.onc.1201091.View ArticlePubMed
  12. Kal A, van Zonneveld A, Benes V, van den Berg M, Koerkamp M, Albermann K, Strack N, Ruijter J, Richter A, Dujon B, Ansorge B, Tabak H: Dynamics of gene expression revealed by comparison of serial analysis of gene expression transcript profiles from yeast grown on two different carbon sources. Mol Biol Cell. 1999, 10: 1859-1872.PubMed CentralView ArticlePubMed
  13. Bullard JH, Purdom E, Hansen KD, Dudoit S: Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments. BMC Bioinformatics. 2010, 11: 94-10.1186/1471-2105-11-94.PubMed CentralView ArticlePubMed
  14. Li WDJIJ, Tibshirani R: Normalizing, testing, and false discovery rate estimation for RNA-sequencing data. Biostatistics. 2012, 13: 523-538. 10.1093/biostatistics/kxr031.PubMed CentralView ArticlePubMed
  15. Baggerly K, Deng L, Morris J, Marcelo Aldaz C: Differential expression in SAGE: accounting for normal between-library variation. Bioinformatics. 2003, 19: 1477-1483. 10.1093/bioinformatics/btg173.View ArticlePubMed
  16. Zhou XKY, Wright F: A Powerful and flexible approach to the analysis of RNA sequence count data. Bioinformatics. 2011, 27: 2672-2678. 10.1093/bioinformatics/btr449.PubMed CentralView ArticlePubMed
  17. Baggerly K, Deng L, Morris J, Marcelo Aldaz C: Overdispersed logistic regression for SAGE: modelling multiple groups and covariates. BMC Bioinformatics. 2004, 5: 144-10.1186/1471-2105-5-144.PubMed CentralView ArticlePubMed
  18. Lu J, Tomfohr J, Kepler T: Identifying differential expression in multiple SAGE libraries: an overdispersed log-linear model approach. Bioinformatics. 2005, 6: 165-PubMed CentralPubMed
  19. Robinson M, Smyth G: Moderated statistical tests for assessing differences in tag abundance. Bioinformatics. 2007, 23: 2881-2887. 10.1093/bioinformatics/btm453.View ArticlePubMed
  20. Robinson M, Smyth G: Small sample estimation of negative binomial dispersion, with applications to SAGE data. Biostatistics. 2008, 9: 321-332.View ArticlePubMed
  21. McCarthy D, Chen Y, Smyth G: Differential expression analysis of multifactor RNA-Seq experiments with respect to biological variation. Nucleic Acids Res. 2012, Epub
  22. Anders S, Huber W: Differential expression analysis for sequence count data. Genome Biol. 2010, 11: R106-10.1186/gb-2010-11-10-r106.PubMed CentralView ArticlePubMed
  23. Hardcastle T, Kelly K: baySeq: Empirical Bayesian methods for identifying differential expression in sequence count data. BMC Bioinformatics. 2010, 11: 422-10.1186/1471-2105-11-422.PubMed CentralView ArticlePubMed
  24. Vencio RZ, Brentani H, Patrao DF, Pereira CA: Bayesian model accounting for within-class biological variability in Serial Analysis of Gene Expression (SAGE). BMC Bioinformatics. 2004, 31: 119-View Article
  25. Zuyderduyn S: Statitical analyis and significance testing of serial analysis of gene expression data using a Poisson mixture model. BMC Bioinformatics. 2007, 8: 282-10.1186/1471-2105-8-282.PubMed CentralView ArticlePubMed
  26. Li J, Tibshirani R: Finding consistent patterns: A nonparametric approach for identifying differential expression in RNA-Seq data. Stat Methods Med Res. 2011, Epub November 28, 2011 http://​www.​ncbi.​nlm.​nih.​gov/​pubmed/​22127579
  27. Farewell VT, Sprott DA: The use of a mixture model in the analysis of count data. Biometrics. 1988, 44: 1191-1194. 10.2307/2531746.View ArticlePubMed
  28. Lee HS: Analysis of overdispersed paired count data. Canadian J Stat. 1996, 24: 319-326. 10.2307/3315742.View Article
  29. Karlis D, Ntzoufras I: Bayesian analysis of the differences of count data. Stat Med. 2006, 25: 1885-1905. 10.1002/sim.2382.View ArticlePubMed
  30. Khafrim S, Kazemnejad A, Eskandari F: Hierarchical Bayesian analysis of bivariate poisson regression model. World Appl Sci J. 2008, 4: 667-675.
  31. Newton MA, Noueiry A, Sarkar D, Ahlquist P: Detecting differential gene expression with a semiparametric hierarchical mixture method. Biostat. 2004, 5 (2): 155-176. 10.1093/biostatistics/5.2.155.View Article
  32. Newton MA, Kendziorski CM, Richmond CS, Blattner FR, Tsui KW: On differential variability of expression ratios: improving statistical inference about gene expression changes from microarray data. J Comput Biol. 2001, 8: 37-52. 10.1089/106652701300099074.View ArticlePubMed
  33. Kendziorski CM, Newton MA, Lan H, Gould M: On parametric empirical bayes methods for comparing multiple groups using replicated gene expression profiles. Stat Med. 2003, 22: 3899-3914. 10.1002/sim.1548.View ArticlePubMed
  34. Smyth GK: Linear models and empirical Bayes methods for assessing differential expression in microarray experiments. Stat Appl Genet Mol Biol. 2004, 3: 3-
  35. Loennstedt I, Britton T: Hierarchical Bayes models for cDNA microarray gene expression. Biostatistics. 2005, 6: 279-291. 10.1093/biostatistics/kxi009.View Article
  36. Gottardo R, Raftery AE, Yeung KY, Bumgarner RE: Bayesian robust inference for differential gene expression in microarrays with multiple Samples. Biometrics. 2006, 62: 10-18.View ArticlePubMed
  37. Do K, Mueller P, Tang F: A Bayesian mixture model for differential gene expression. Appl Stat. 2005, 54: 627-644.
  38. Lewin A, Bochkina N, Richardson S: Fully Bayesian mixture model for differential gene expression: simulations and model checks. Stat Appl Genet Mol Biol. 2007, 6: 36-
  39. Kong K, Delroux K, Wang X, Qian F, Arjona A, Malawista S, Fikrig E, Montgomery R: Dysregulation of TLR3 impairs the innate immune response to west Nile virus in the elderly. J Virol. 2008, 82: 7613-7623. 10.1128/JVI.00618-08.PubMed CentralView ArticlePubMed
  40. Trapnell C, Pachter L, Salzberg S: TopHat: discovering splice junctions with RNA-Seq. Bioinformatics. 2009, 25: 1105-1111. 10.1093/bioinformatics/btp120.PubMed CentralView ArticlePubMed
  41. Trapnell C, Williams B, Pertea G, Mortazavi A, Kwan G, van Baren M, Salzberg S, Wold B, Pachter L: Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation. Biotechnol. 2010, 28: 511-515. 10.1038/nbt.1621.
  42. Robinson M, Oshlack A: A scaling normalization method for differential expression analysis of RNA-seq data. Genome Biol. 2010, 11: R25-10.1186/gb-2010-11-3-r25.PubMed CentralView ArticlePubMed
  43. Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell. 1984, 6: 721-741.View ArticlePubMed
  44. Anders S, Huber W: Differential expression of RNA-Seq data at the gene level - the DESeq package. 2013, [http://​www.​bioconductor.​org/​packages/​devel/​bioc/​vignettes/​DESeq/​inst/​doc/​DESeq.​pdf]
  45. Bouchon CMGHCJA, Colonna M: Activation of NK cell-mediated cytotoxicity by a SAP-independent receptor of the CD2 family. J Immunol. 2001, 167: 5517-5521.View ArticlePubMed
  46. Parquet M, Kumatori A, Hasebe F, Morita K, Igarashi A: West Nile virus-induced bax-dependent apoptosis. FEBS letters. 2001, 500: 17-24. 10.1016/S0014-5793(01)02573-X.View ArticlePubMed
  47. Medigeshi G, Lancaster A, Hirsch A, Briese T, Lipkin W, DeFilippis V, Frueh K, Mason P, Nikolich-Zugich J, Nelson J: West Nile virus infection activates the unfolded protein response, leading to CHOP induction and apoptosis. J Virol. 2007, 81: 10849-10860. 10.1128/JVI.01151-07.PubMed CentralView ArticlePubMed
  48. Oshlack A, Wakefield M: Transcript length bias in RNA-seq data confounds systems biology. Biol Direct. 2009, 4: 14-10.1186/1745-6150-4-14.PubMed CentralView ArticlePubMed
  49. Zheng W, Chung L, Zhao H: Bias detection and correction in RNA-Sequencing data. BMC Bioinformatics. 2011, 12: 290-10.1186/1471-2105-12-290.PubMed CentralView ArticlePubMed
  50. Gottardo R, Raftery A: Markov chain Monte Carlo computations with mixture of singular distributions. Technical Report 470, Statistics Department. Seattle: University of Washington; 2004


© Chung et al.; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://​creativecommons.​org/​licenses/​by/​2.​0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.