Discrimination of outer membrane proteins with improved performance

Discrimination between OMPs and non-OMPs

Comparisons with previously published methods

We compare the proposed method with previously published methods. As discussed in Baldi et al. [17], in a two-class classification, if the numbers of examples in the two classes are not equal, MCC is a better measure for evaluating the classification performance. In the discrimination of OMPs and non-OMPs, the numbers of examples in the two classes are not equal. Therefore, we will use MCC as the primary measure in the comparison of different methods. At the same time, we also report accuracy, specificity, and sensitivity.

BOMP [14], TMB-Hunt [12, 13] and PROFtmb [9] are three top-scoring on-line servers that can discriminate OMPs. BOMP and TMB-Hunt are based on the K-nearest neighbor method, and PROFtmb is based on a hidden Markov model (HMM). We compared the proposed method with these methods by submitting the datasets used in this study to these servers. The comparisons (Table 1, rows 4–7) show that the proposed method outperforms all the other methods. It is worth to point out that the datasets used in this study are likely to have a big overlap with the datasets that were used to train BOMP, TMB-Hunt and PROFtmb servers. Thus, when we evaluated these methods by submitting our datasets to their web servers, the performance of these methods might have been overestimated. Remarkably, our method still outperforms the others under this condition.

Receiver Operating Characteristic (ROC) Curve

In the proposed method, a protein is classified as OMP or non-OMP based on the comparison of D _omp (its distance to the OMP group), D _glo (its distance to the globular protein group), and D _imp (its distance to the IMP group). A protein is predicted to be OMP if D _omp < Min{D _imp , D _glo }, where Min { } returns the minimal value of a set. This criteria is equal to evaluating D _omp - Min{D _imp , D _glo } < 0. In general, we can introduce a threshold parameter α, such that a protein is predict to be OMP if D _omp - Min{D _imp , D _glo } < α. Figure 1 shows the ROC curve of the proposed method obtained by varying α. The ROC curve shows how the method can trade off between specificity and sensitivity by changing α. When applying a prediction method to identify OMPs, some researchers may prefer to identify more potential OMPs (high sensitivity) at the cost of relatively low specificity; others may want to identify OMPs with very high specificity at the cost of relatively low sensitivity. The advantage of introducing this parameter α to the proposed method is that users can chose a threshold based on their need. When α is set to a lower value, the method can achieve higher specificity. On other hand, when a high value of α is chosen, the method can achieve higher sensitivity.

Identification of OMPs in the Proteome of E. coli

We applied the proposed method to search for OMPs in the proteome of E. Coli using α = -0.05, which corresponds to 98% specificity in the ROC curve. The E. Coli proteome consists of 4,319 proteins. 107 of them were predicted to be OMPs. That accounts for 2.5% of the whole proteome. This ratio is consistent with the previous estimation that 2–3% of the genes in Gram-negative bacteria encodes OMPs [2]. Among these 107 proteins, 49 are annotated as OMP proteins in Swiss-Prot [20] or ePSORTdb [21], a database of protein subcellular locations that have been determined by laboratory experiments, and 15 share very high similarities with some OMPs in the training set (with E ≤ 0.0001 in BLAST comparison). Thus, we have very high confidence in believing that these 64 hits are true positives. In addition, 13 proteins are annotated with "Membrane", "Cell membrane" and "Multi-pass membrane protein" in Swiss-Prot. We submitted these proteins to the TMHMM [22], a server for predicting the topology of trasmembrane α-helical proteins, and PSORTb [23], a server for predicting subcellular locations. None of them was predicted to be trasmembrane α-helical proteins (or inner membrane proteins) by both methods. Thus, most of these 15 proteins are very likely OMP proteins. The remaining 30 proteins may suggest new OMP proteins that have not been previously discovered.

We also compare our method's predictions with the proteome scanning results obtained by BOMP [14]. We choose BOMP for comparison because BOMP's predictions for E. Coli proteins are available on its server. In the E. Coli proteome, BOMP predicted 103 OMP proteins. Comparisons show that 59 proteins were predicted to be OMP by both our method and BOMP. 48 proteins were predicted to be OMP by our method but not by BOMP. Among them, 15 proteins are true positives. Additionally, 9 proteins are annotated with "membrane", "Cell membrane" or "multi-pass membrane protein" in SwissProt. 44 proteins were predicted to be TMB proteins by BOMP but not by our method. Among them, 16 are true positives. Additionally, 7 are annotated with "membrane", "Cell membrane" or "multi-pass membrane protein" in SwissProt. This comparison shows that there is a big overlap between the predictions of the proposed method and BOMP. It also shows that each of the two methods can identify some OMP proteins missed by the other. This suggests the possibility of achieving better performance by combining these two methods. Another possible direction to improve the performance is to combine the current method with other methods that predict signal peptides in proteins, since OMP proteins contain a signal peptide that leads them to the outer membrane.

Simple Methods versus Complicated Methods

It is estimated that 2–3% of the genes in Gram-negative bacteria encodes OMPs [2]. Identifying all OMPs ("OMPome") from bacterial genome is an urgent and challenging task. Compared with other complicated methods, such as k-nearest neighbor method, neural network and SVM, that have been used to identify OMPs, one merit of the proposed method resides in its simplicity and fast speed. The training data set is read only once. The calculation of residue composition and weighted Euclidean distance (WED) can be done with a very fast speed. The method proposed here will be very helpful to the discovery of "OMPome" in a genome scale.

Euclidean distance versus weighted Euclidean distance

We used a WED (i.e., $\sqrt{{\displaystyle \sum _i\frac{{({\overline{x}}_i-x_i)}^2}{{\overline{x}}_i}}}$) to discriminate OMPs from non-OMPs. Our results show that this method achieves better performance than a published method that discriminate OMPs and non-OMPs based on a deviation distance (i.e., ${\displaystyle \sum _i|{\overline{x}}_i-x_i|}$) [15]. In this study, we also tried Euclidean distance (i.e., $\sqrt{{\displaystyle \sum _i{({\overline{x}}_i-x_i)}^2}}$) instead of WED. But, the performance is not so good as using WED. Compared with Euclidean distance and the deviation distance, WED can better reveal the relation between a protein and a group. Intuitively, for the same amount of difference (i.e., |${\overline{x}}_i$ - x _i |), when ${\overline{x}}_i$ becomes smaller, the difference will become more significant. For example, for the same amount of difference 0.01 (i.e., |${\overline{x}}_i$ - x _i | = 0.01), if the composition of residue i in OMPs is 90% (i.e., ${\overline{x}}_i$ = 0.90 and x _i = 0.89), then 0.01 does not imply a significant difference between the test protein and OMPs. But, if ${\overline{x}}_i$ = 0.001 (then x _i = 0.011), then |${\overline{x}}_i$ - x _i | = 0.01 will suggest a significant difference between the test protein and OMPs.

Datasets

We compiled a set of outer membrane proteins (OMPs) that have been experimentally confirmed. It includes 118 proteins that are classified as "Transmembrane beta-barrels" in the SCOP database [24] and 188 proteins from the "β-Barrel porins" subclass in Transport Proteins Database [25]. We removed redundant proteins so that the mutual identity in the dataset was less than 25%. First the proteins were clustered by running BLASTCLUST with parameters "-S 25 -L 0.9 -b F". This step ensured that any two proteins from different clusters shared less than 25% identical residues over 90% coverage of any protein. Then, one protein was chosen from one cluster. Proteins with less than 50 amino acids and proteins that were not from Gram-negative bacterial were also removed. The final dataset consists of 119 OMPs. Globular proteins and α-helical membrane proteins (inner membrane proteins, IMPs) were obtained from a previous study by Park et al. [16]. We filtered the datasets so that the identity between any two proteins is less than 25%. After the filtering, 673 globular proteins, and 178 IMPs were left.

Residue composition

Residue composition of a protein was calculated using $x_i=n_i/{\displaystyle \sum _i{n}_i}$, where n _i was the number of residues of type i in the protein. Average residue composition of OMPs was calculated using ${\overline{x}}_{i\_omp}=n_{i\_omp}/{\displaystyle \sum _i{n}_{i\_omp}}$, where n_{i _omp}was the total number of residues of type i in OMPs. The average residue composition of globular proteins was calculated using ${\overline{x}}_{i\_glo}=n_{i\_glo}/{\displaystyle \sum _i{n}_{i\_glo}}$, where n_{i _glo}was the total number of residues of type i in globular proteins. The average residue composition of inner membrane proteins was calculated using ${\overline{x}}_{i\_imp}=n_{i\_imp}/{\displaystyle \sum _i{n}_{i\_imp}}$, where n_{i _imp}was the total number of residues of type i in globular proteins.

Weighted Euclidean distance (WED)

For each test protein, its distance to OMPs was calculated using $D_{Omp}=\sqrt{{\displaystyle \sum _i\frac{{({\overline{x}}_{i\_omp}-x_i)}^2}{{\overline{x}}_{i\_omp}}}}$, where x _i was the composition of residue type i in the test protein, ${\overline{x}}_{i\_omp}$ was the average composition of residue type i in OMPs. Note that $\sqrt{{\displaystyle \sum _i{({\overline{x}}_{i\_omp}-x_i)}^2}}$ gives the Euclidean distance between the OMP group and the test protein. In this study, we weighted each term inside the summation with $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\overline{x}}_{i\_omp}$}\right.$. Thus, we call D _omp a weighted Euclidean distance (WED). The WED between a test protein and globular proteins (D _glo ) and the WED between a test protein and inner membrane proteins (D _imp ) were calculated in a similar way.

Classification

Proteins were classified into the three groups based on the least WED. A test protein was predicted to be an OMP if D _omp ≤ D _imp and D _omp ≤ D _glo ; otherwise, it was predicted to be non-OMP.

Five-fold cross-validations

Five-fold cross-validations were used to evaluate the proposed method. The overall dataset was divided into five subsets. OMPs, globular proteins and IMPs were distributed into the subsets evenly. In each round of experiment, four subsets were used as the training set and the remaining subset was used as the test set. This procedure was repeated five times with each subset being used as test set once. The average performance was reported.

Including homologous sequences into the calculation of residue composition

For each protein, the BLAST program [26] was used to search for homologous sequences in the NCBI non-redundant database using an E-value of 0.0001. 50 best hits were chosen from the return result. If less than 50 hits were return, then all of the hits were chosen. These proteins plus the query protein were used to calculate the residue composition for the query protein.

Feature selection

We extended the proposed method by including the composition of di-peptides. We used a feature selection approach to search for residues and di-peptides that are useful for OMP prediction. For the feature selection, we implemented the Bestfirst method in the Weka package [27]. The feature selection was conducted with bi-directional search, with a starting set that include the 20 amino acids. In the end, we obtained a set of features that include the compositions of 14 amino acids and 130 di-peptides.

Performance measures

Sensitivity shows the fraction of OMPs that are correctly identified. Specificity shows the fraction of negative proteins that are correctly identified. Accuracy is the total accurate rate of the predictions. MCC (Matthews correlation coefficient) measures the correlation between predictions and actual class labels, which is in the range of [-1, 1], with 1 denoting perfect predictions. In a two-class classification, if the numbers of examples of the two classes are not equal, MCC is a better measure than accuracy [17]. Therefore, in the comparisons of different methods, we focus on the comparison of MCC. In addition to MCC, we also report the performance in other measures.

Availability

The software and data sets are available online [28].