Robust PCA based method for discovering differentially expressed genes

How to identify a set of genes that are relevant to a key biological process is an important issue in current molecular biology. In this paper, we propose a novel method to discover differentially expressed genes based on robust principal component analysis (RPCA). In our method, we treat the differentially and non-differentially expressed genes as perturbation signals S and low-rank matrix A, respectively. Perturbation signals S can be recovered from the gene expression data by using RPCA. To discover the differentially expressed genes associated with special biological progresses or functions, the scheme is given as follows. Firstly, the matrix D of expression data is decomposed into two adding matrices A and S by using RPCA. Secondly, the differentially expressed genes are identified based on matrix S. Finally, the differentially expressed genes are evaluated by the tools based on Gene Ontology. A larger number of experiments on hypothetical and real gene expression data are also provided and the experimental results show that our method is efficient and effective.


Background
One of the challenges in current molecular biology is how to find the genes associated with key cellular processes. Up to date, using microarray technology, these genes associated with a special biological process have been detected more comprehensively than ever before.
DNA microarray technology has enabled highthroughput genome-wide measurements of gene transcript levels [1,2], which is promising in providing insight into biological processes involved in gene regulation [3]. It allows researchers to measure the expression levels of thousands of genes simultaneously in a microarray experiment. Gene expression data usually contain thousands of genes (sometimes more than 10,000 genes), and yet only a small number of samples (usually less than 100 samples). Gene expression is believed to be regulated by a small number of factors (compared to the total number of genes), which act together to maintain the steady-state abundance of specific mRNAs. Some of these factors could represent the binding of one (or more) transcription factor(s) (TFs) to the promoter region(s) of the gene [4]. So, it can be assumed that the genes associated with a biological process are influenced only by a small subset of TFs [5]. Although the expression levels of thousands of genes are measured simultaneously, only a small number of genes are relevant to a special biological process. Therefore, it is important how to find a set of genes that are relevant to a biological process.
Various methods have been proposed for identifying differentially expressed genes from gene expression data. These methods can be roughly divided into two categories: univariate feature selection (UFS) and multivariate feature selection (MFS). The commonest scheme of UFS is utilized as follows. First, a score for each gene is independently calculated. Then the genes with high scores were selected [6]. The main virtues of UFS are simple, interpretable and fast. However, UFS has some drawbacks. For example, if each gene is independently selected from gene expression data, a large part of the mutual information contained in the data will be lost.
To overcome the drawbacks of UFS, the methods of MFS use all the features simultaneously to select the genes. So far, many mathematical methods for MFS, such as principal component analysis (PCA), independent component analysis (ICA), nonnegative matrix factorization (NMF), lasso logistic regression (LLR) and penalized matrix decomposition (PMD), have been devised to analyze gene expression data. For example, Lee et al. applied PCA to analyze gene expression data [7]. Liu et al. proposed a method of weighting principal components by singular values to select characteristic genes [8]. Probabilistic PCA was used to analyze gene expression data by Nyamundanda et al. [9]. Huang et al. used ICA to analyze gene expression data [10]. NMF was used to select the gene by Zheng et al. [11]. Liu et al. used LLR to select characteristic gene using gene expression data [12]. In [13], Witten et al. proposed penalized matrix decomposition (PMD), which was used to extract plant core genes by Liu et al. [14]. However, the brittleness of these methods with respect to grossly corrupted observations often puts its validity in jeopardy.
Recently, a new method for matrix recovery, namely robust PCA, has been introduced in the field of signal processing [15]. The problem of matrix recovery can be described as follows, assume that all the data points are stacked as column vectors of a matrix D, and the matrix (approximately) have low rank: where A 0 has low-rank and S 0 is a small perturbation matrix. The robust PCA proposed by Candes et al. can recover a low-rank matrix A 0 from highly corrupted measurements D [15]. Here, the entries in S 0 can have arbitrary large magnitude, and their support is assumed to be sparse but unknown.
Although the method has been successfully applied to model background from surveillance video and to remove shadows from face images [15], it's validity for gene expression data analysis is still need to be studied. The gene expression data all lie near some low-dimensional subspace [16], so it is natural to treat these genes data of non-differential expression as approximately low rank. As mentioned above, only a small number of genes are relevant to a biological process, so these genes with differential expression can be treated as sparse perturbation signals.
In this paper, based on robust PCA, a novel method is proposed for identifying differentially expressed genes. The differentially and non-differentially expressed genes are treated as perturbation signals S and low-rank matrix A. Firstly, the matrix D of expression data is decomposed into two adding matrices A and S by using RPCA. Secondly, the differentially expressed genes are discovered according to the matrix S. Finally, the differentially expressed genes are evaluated by the tools based on Gene Ontology. The main contributions of our work are as follows: firstly, it proposes, for the first time, the idea and method based on RPCA for discovery of differentially expressed genes; secondly, it provides a larger number of experiments of gene selection.

Methods
The definition of Robust PCA (RPCA) This subsection simply introduces robust PCA (RPCA) proposed by Candes et al. [15]. Let A * := i σ i (A) denote the nuclear norm of the matrix A, that is, the sum of its singular values, and let S 1 := ij S ij denote the L 1 -norm of S. Supposing that D denotes the observation matrix given by Eq.(1), RPCA solves the following optimization problem: where λ is a positive regulation parameter. Due to the ability to exactly recover underlying low-rank structure in the data, even in the presence of large errors or outliers, this optimization is referred to as Robust Principal Component Analysis (RPCA).
For the RPCA problem Eq.(2), a Lagrange multiplier Y is introduced to remove the equality constraint. According to [17], the augmented Lagrange multiplier method on the Lagrangian function can be applied: where μ is a positive scalar and • 2 F denotes the Frobenius norm. Lin et al. gave a method for solving the RPCA problem, which is referred to as the inexact ALM (IALM) method [17]. The details of this algorithm can be seen in [17].

The RPCA model of gene expression data
Considering the matrix D of gene expression data with size m × n, each row of D represents the transcriptional responses of a gene in all the n samples, and each column of D represents the expression levels of all the m genes in one sample. Without loss of generality, m >> n, so it is a classical small-sample-size problem.
Our goal of using RPCA to model the microarray data is to identify these significant genes. As mentioned in Introduction, it is reasonable to view the significant genes as sparse signals, so the differential ones are viewed as the sparse perturbation signals S and the non-differential ones as the low-rank matrix A. Consequently, the genes of differential expression can be identified according to the perturbation signals S. The RPCA model of microarray data is shown in Figure 1. The white and yellow blocks denote zero and near-zero in Figure 1. Red and blue blocks denote the perturbation signals. As shown in Figure 1, the matrix S of differentially expressed genes (red or blue block) can be recovered from the matrix D of gene expression data.
Suppose the matrix decomposition D = A + S has been done by using RPCA. By choosing the appropriate parameter λ, the sparse perturbation matrix S can be obtained, i.e., most of entries in S are zero or near-zero (as white and yellow blocks shown in Figure 1). The genes corresponding to non-zero entries can be considered as ones of differential expression.

Identification of differentially expressed genes
After observation matrix has been decomposed by using RPCA, sparse perturbation matrix S can be obtained. Therefore the differentially expressed genes can be identified according to sparse matrix S.
Denote the perturbation vector associated with i-th sample as: Then the sparse matrix S can be expressed as follows: So the sparse matrix S can be denoted as: The differentially expressed genes can be classified into two categories: up-and down-regulated ones [18], which are reflected by the positive and negative entries in the sparse matrix S. Here, to discover the differentially expressed genes, only the absolute value of entries in S need to be considered. Then the following two steps are executed: firstly, the absolute values of entries in the sparse matrix S are find out; secondly, to get the evaluating vectorS, the matrix is summed by rows. Mathematically, it can be expressed as follows: Consequently, to obtain the new evaluating vectorŜ, which is sorted in descending order. Without loss of generality, suppose that the first c 1 entries inŜ are nonzero, that is, Generally, the larger the element inŜ is, the more differential the gene is. So, the genes associated with only the first num (num ≤ c 1) entries inŜ are picked out as differentially expressed ones.

Results and discussion
This section gives the experimental results. Firstly, in the first subsection, hypothetical data are exploited to clarify how to set the parameterλ. Secondly, in the second subsection, our method is compared with the following methods on the real gene expression data of plants responding to abiotic stresses: (a) PMD method using the left singular vectors {u k } to identify the differentially expressed genes (proposed by Witten et al. [13]); (b) SPCA method using all the PCs of SPCA (proposed by Journée et al. [19]) to identify the differentially expressed genes. Finally, in the third subsection, the three methods are compared on the real gene expression data of colon tumor.

Experimental results on hypothetical data
Matrices randomly generated will be used for the simulation experiments. The true solution is denoted by the ordered pairs A * , S * , which are generated by using the method in [17]. The rank-r matrix A * ∈ R m×n is generated as A * = LR T , where L and R are independent m × r and n × r matrices, respectively. Elements of L and R are i.i.d. Gaussian random variables with zero mean and unit variance. S * ∈ {−1, 0, 1} m×n is a sparse matrix whose support is chosen uniformly at random, and whose non-zero entries are i.i.d. uniformly in the space R m×n . μ denotes the sparse degree of matrix S * , which is defined as the number of nonzero entries divided by the number of all the entries. The matrix D = A * + S * is the input data to the RPCA. To evaluate the identification performance of RPCA, Acc S denotes the recognition accuracy of matrix S, which is defined as follows.
where correct identified entries mean that the identified entries in S approximately equal to the ones in S * .
In [17,20], a fixed regulation parameter λ = c * max(m, n) −1 / 2 is used, where c = 1.0. In order to clarify how to set λ, the following two different cases are considered: first, m = n; second, m > n, the smallsize-sample problem.

Results while m=n
In this experiment, let m = n = 500, 1000 or 2000, μ = 0.05 or 0.1, μ = 0.05 or 0.1. Table 1 lists the recognition results with different c. As Table 1 listed, when c = 0.2, the recognition accuracy Acc S can be achieved above 90%. When c ≥ 0.3, the matrix S can be completely identified, i.e. Acc S = 100%.

Results while m>n
In this experiment, let m = 10000,rank = 5 or 10 , μ = 0.05 or 0.1 and n increase from 10 to 100 with an interval 10. Table 2, 3, 4, 5 list the results. As tables 2 and 3 listed with rank = 5, when n ≥ 20, the recognition accuracy Acc S can be achieved above 90%. As tables 4 and 5 listed with rank = 10 , when n ≥ 30, the recognition accuracy Acc S can be achieved above 90%. In words, to achieve the recognition accuracy Acc S above 90%, n must be equal to or larger than three times of rank (n ≥ 3 * rank). As tables 2, 3, 4, 5 listed, by rows, the larger the number of column n is, the higher the recognition accuracy Acc S can be achieved. Now, we investigate how different c influences the recovery accuracy Acc S . For example, when n = 40, Figure 2 shows the recognition accuracy Acc S with different c. As shown in Figure 2, when c = 0.3, the recognition of matrix S can reach highest accuracy. With c increasing, the recovery accuracy Acc S drops. For example, when c = 1.0, s3 and s4 are degraded to 90%.
From these experiments, a conclusion can be drawn that when the optimal empirical value of λ is given as: where the size of data matrix D is m × n, the highest identification accuracy Acc S can be obtained.

Experimental results on gene expression data of plants responding to abiotic stresses
Along with other two state-of-the-art methods, namely PMD and SPCA, used as comparison, three methods, including RPCA, are used to discover the differentially expressed genes responding to abiotic stresses based on real gene expression data.

Data source
The raw data were downloaded from NASCArrays [http://affy.arabidopsis.info/] [21], which include two classes: roots and shoots in each stress. The reference numbers are: control, NASCArrays-137; cold stress, NASCArrays-138; osmotic stress, NASCArrays-139; salt stress, NASCArrays-140; drought stress, NASCArrays-141; UV-B light stress, NASCArrays-144; heat stress, NASCArrays-146. Table 6 lists the sample number of each stress type. There are 22810 genes in each sample. The data are adjusted for background of optical noise using the GC-RMA software by Wu et al. [22] and normalized using quartile normalization. The results of GC-RMA are gathered in a matrix for further processed.

Selection of the parameters
In this paper, for PMD method, the L 1 -norm of u is taken as the penalty function, i.e. u 1 ≤ α 1 . Because of For simplicity, let p = 1, that is, only one factor is used. The results with L 1 -norm ( z 1 = i |z i |) and L 0 -norm ( z 0 , i.e. the number of nonzero coefficients, or cardinality) penalty in SPCA are similar, which is also shown in [19], so L 0 -norm penalty and the parameter γ are taken in SPCA. For a fair comparison, 500 genes are roughly selected by these methods via choosing appropriate parameters α and γ of the two methods, PMD and SPCA, which are listed in Table 7 for different data set. As the first subsection of experiments mentioned, while c = 0.3, RPCA gives the optimization results. Then, according to methods section, the first 500 genes are selected.

Gene ontology (GO) analysis
Recently, many tools have been developed for the functional analysis of large lists of genes [23,24]. Most of them focus on the evaluation of Gene Ontology (GO)   Table 4 The recognition accuracy Acc S with rank = 10 and μ = 0.05 c  Table 5 The recognition accuracy Acc S with rank = 10 and μ = 0.1 c annotations. GOTermFinder is a web-based tool that finds the significant GO terms shared among a list of genes, helping us discover what these genes may have in common. The analysis of GOTermFinder provides significant information for the biological interpretation of high-throughput experiments. In this subsection, the genes identified by these methods, RPCA, PMD and SPCA, are sent to GOTermFinder [24], which is publicly available at http://go.princeton. edu/cgi-bin/GOTermFinder. Its threshold parameters are set as following: minimum number of gene products = 2 and maximum P-value = 0.01. Here, the key results are shown. Table 8 lists the terms of Response to abiotic stimulus (GO:0009628), whose background frequency in TAIR set is 1539/29556 (5.2%). Response to abiotic stimulus is the ancestor term of all the abiotic stresses.
In GOTermFinder, a p-value is calculated using the hyper-geometric distribution, its details can be seen in [24]. Sample frequency denotes the number of genes hit in the selected genes, such as 107/500 denotes 107 genes associated with the GO term in 500 ones selected by these methods. As listed in Table 8, all the three experimented methods, PMD, SPCA and RPCA, can extract the significant genes with very lower P-value, as well as very higher sample frequency. In Table 8, the superior results are in bold type. In the twelve items, there is only one of them (cold on root) that PMD is equal to our Figure 2 The recognition accuracy of matrix S with different c. s1 denotes the recognition accuracy series with rank = 5 and μ = 0.05. s2 denotes the recognition accuracy series with rank = 5 and μ = 0.1. s3 denotes the recognition accuracy series with rank = 10 and μ = 0.05. s4 denotes the recognition accuracy series with rank = 10 and μ = 0.1.  method. In other items, our method is superior to SPCA and PMD. Figure 3 shows the sample frequency of response to abiotic stimulus (GO:0009628) given by the three methods. From Figure 3(a), RPCA method outperforms others in all the data sets of shoot samples with six different stresses. Figure 3(b) shows that only in cold-stress data set of root samples, PMD is equal to our method and they are superior to SPCA. In other data sets, our method is superior to the others.
The characteristic terms are listed in Table 9, in which the superior results are in bold type. As listed in Table 9, PMD method outperforms SPCA and our method in three items, such as drought in shoot, salt in root and cold in root, among the whole items. However, it shows that, on one of the twelve items (osmotic in shoot), our method has the same competitive result as PMD, while both methods are superior to SPCA. In other eight items, our method excels PMD and SPCA methods. In addition, on all the characteristic items, our method has superiority over SPCA.
From the results of experiments, it can be concluded that our method is efficient and effective.

Experimental results on colon data
The three methods, SPCA, PMD and RPCA, are compared on colon cancer data set [25]. Colon cancer is the fourth most common cancer for males and females and the second most frequent cause of death.  Figure 3 The sample frequency of response to abiotic stimulus.

Data source
The raw data were downloaded from http://genomicspubs.princeton.edu/oncology/affydata/I2000.html, which include gene expression levels for 2000 gene and contain 40 tumor and 22 normal tissue samples.

Selection of the parameters
In this subsection, for PMD method, the L 1 -norm of u is taken as the penalty function, i.e. u 1 ≤ α 1 . Let For SPCA method, let p = 1, that is, only one factor is used. L 0 -norm penalty and the parameter γ are taken in SPCA. For a fair comparison, 100 genes are roughly selected by these methods via choosing appropriate parameters. PMD and SPCA use the parameters α = 0.2351 and γ = 0.4306 on colon data set, respectively. As the first subsection of experiments mentioned, while c = 0.3, RPCA gives the optimization results. Then, according to Methods section,the first 100 genes are selected using our method.

Gene ontology (GO) analysis
The genes identified by these methods, RPCA, PMD and SPCA, are evaluated by using AmiGO [26]. Its threshold parameters are set as following: minimum number of gene products = 2 and maximum P-value = 0.1. A number of lines of evidence suggest that immune, stimulus and tumor have affinity, so Table 10 lists the key results: the terms of Response to stimulus (GO:0050896) and Immune system process (GO:0002376). As listed in Table 10, RPCA outperforms its competitive methods with higher sample frequency. Table 11 lists the top 30 genes selected by using RPCA. To further study the biology functions of the selected genes, we also make the network analysis of the top 100 genes selected by our algorithm using the GeneMANIA tool [27] on the Web sitehttp://genemania.org/. The result is listed in Table 12. From the table it can be seen that there are 215 genes of this chip participating in the cytokine-mediated signalling pathway, in which there are 21 genes discovered by our method. This pathway has the lowest p-value. It is considered as the most probable pathway with these top 100 genes. Recent findings also indicate that cytokine receptors can regulate immune cell functions by transcription-independent mechanisms [28]. Some other pathways with the most significance are also listed in Table 12.