Volume 6 Supplement 4
Italian Society of Bioinformatics (BITS): Annual Meeting 2005
A new decoding algorithm for hidden Markov models improves the prediction of the topology of allbeta membrane proteins
 Piero Fariselli^{1}Email author,
 Pier Luigi Martelli^{1} and
 Rita Casadio^{1}
DOI: 10.1186/147121056S4S12
© Fariselli et al; licensee BioMed Central Ltd 2005
Published: 1 December 2005
Abstract
Background
Structure prediction of membrane proteins is still a challenging computational problem. Hidden Markov models (HMM) have been successfully applied to the problem of predicting membrane protein topology. In a predictive task, the HMM is endowed with a decoding algorithm in order to assign the most probable state path, and in turn the labels, to an unknown sequence. The Viterbi and the posterior decoding algorithms are the most common. The former is very efficient when one path dominates, while the latter, even though does not guarantee to preserve the HMM grammar, is more effective when several concurring paths have similar probabilities. A third good alternative is 1best, which was shown to perform equal or better than Viterbi.
Results
In this paper we introduce the posteriorViterbi (PV) a new decoding which combines the posterior and Viterbi algorithms. PV is a two step process: first the posterior probability of each state is computed and then the best posterior allowed path through the model is evaluated by a Viterbi algorithm.
Conclusion
We show that PV decoding performs better than other algorithms when tested on the problem of the prediction of the topology of betabarrel membrane proteins.
Background
Allbeta membrane proteins constitute a well structurally conserved class of proteins, that span the outer membrane of Gramnegative bacteria with a barrellike structure. In all cases known so far with atomic resolution, the barrel consists of an even number of antiparallel beta strands, whose number ranges from 8 to 22 strands, depending on the protein and/or its functional role [1, 2]. In eukaryotes, it is known that similar architectures are present in the outer membrane of chloroplasts and mitochondria, although so far none of the socalled "porins", mainly acting as Voltage Dependent Anion Channels (VDAC), have been solved with atomic resolution ([3] and references therein). It is therefore urgent to devise methods for the prediction of the topology of this class of membrane proteins. Indeed the correct prediction of the protein topology, given the conservation of the barrel architecture may greatly help in threading procedures, especially when sequence homology is low. Furthermore reliable methods, endowed with a low rate of false positives, can also help in genome annotation on the basis of protein structure prediction [3, 4]. The problem of predicting beta barrel membrane proteins has been recently addressed with machine learning approaches, and among them Hidden Markov Models (HMMs) have been shown to outperform previously existing methods [5]. HMMs were developed for alignments [6, 7], pattern detection [8, 9] and also for predictions, as in the case of the topology of allalpha and allbeta membrane proteins [10–17]. When HMMs are implemented for predicting a given feature, a decoding algorithm is needed. With decoding we refer to the assignment of a path through the HMM states (which is the best under a suitable measure) given an observed sequence O. In this way, we can also assign a label to each sequence element [18, 19]. More generally, as stated in [20], the decoding is the prediction of the labelsof an unknownpath. The most famous decoding procedure is the Viterbi algorithm, which finds the most probable allowed path through the HMM model. Viterbi decoding is particularly effective when there is a single best path among others much less probable. When several paths have similar probabilities, the posterior decoding or the 1best algorithms are more convenient [20]. The posterior decoding assigns the state path on the basis of the posterior probability, although the selected path might be not allowed.
In this paper we address the problem of preserving the automaton grammar and concomitantly exploiting the posterior probabilities, without the need of the postprocessing algorithm [12, 21]. Prompted by this, we design a new decoding algorithm, the posteriorViterbidecoding (PV), which preserves the automaton grammars and at thesame time exploits the posterior probabilities. A related idea, that is specific for pairwise alignments was previously introduced to improve the sequence alignment accuracy [22]. We show that PV performs better than the other algorithms when we test it on the problem of predicting the topology of betabarrel membrane proteins.
Results and Discussion
Testing the decoding algorithms on allbeta membrane proteins
In order to test our decoding algorithm on real biological data, we used a previously developed HMM, devised for the prediction of the topology of betabarrel membrane proteins [12]. The hidden Markov model is a sequenceprofilebased HMM and takes advantage of emitting vectors instead of symbols, as described in [12].
Since the previously designed and trained HMM [12] emits profile vectors, sequence profiles have been computed from the alignments as derived with PSIBLAST [23] on the nonredundant database of protein sequences ftp://ftp.ncbi.nlm.nih.gov/blast/db/.
Q_{ ok }accuracy obtained with the four different decoding algorithms
Proteins  Viterbi  1best  posterior  posteriorViterbi 

crossvalidation  
1a0spTOT        OK 
1bxwaTOT    OK  OK  OK 
1e54      OK  OK 
lek9aTOT      OK  OK 
1fcpaTOT         
1fepTOT        OK 
1i78a      OK  OK 
1k24        OK 
1kmoaTOT      OK  OK 
1prn         
1qd5a      OK  OK 
1qj8a      OK  OK 
2mpra      OK  OK 
2omf      OK  OK 
2por         
<Q_{ ok }>  0.0  0.07  0.60  0.80 
blindtest  
1mm4      OK   
1nqf        OK 
1p4t  OK  OK  OK  OK 
1uyn        OK 
1t16         
<Q_{ ok }>  0.20  0.20  0.40  0.60 
From Table 1 it evident that the new PV decoding is the best performing decoding achieving 80% and 60% accuracy in crossvalidation and on the blind set, respectively. This is done ensuring that predictions are consistent with the designed automaton grammar.
Comparison with other available HMMs
PV accuracy compared with other algorithms and HMM models
Method  Q _{2}  SOV  SOV(BetaTM)  SOV(Loop)  Q _{ ok } 

crossvalidation  
PosteriorViterbi^{l}  0.82  0.87  0.92  0.81  0.80 
Viterbi^{1}  0.63  0.33  0.27  0.35  0.0 
1best^{1}  0.65  0.41  0.37  0.41  0.07 
HMMB2HTMR^{2}  0.83  0.87  0.88  0.84  0.73 
PROFTmb^{3}  0.83  0.87  0.88  0.84  0.73 
predtmbb^{4} (Viterbi)  0.78  0.83  0.81  0.82  0.60 
predtmbb^{4} (1best)  0.78  0.83  0.81  0.82  0.60 
predtmbb^{4} (posterior)  0.78  0.82  0.80  0.82  0.60 
blindtest  
PosteriorViterbi^{1}  0.80  0.81  0.84  0.74  0.60 
Viterbi^{1}  0.62  0.38  0.35  0.40  0.20 
1best^{1}  0.63  0.38  0.36  0.40  0.20 
HMMB2HTMR^{2}  0.80  0.81  0.84  0.74  0.60 
PROFTmb^{3}  0.72  0.65  0.72  0.58  0.40 
predtmbb^{4} (Viterbi)  0.71  0.73  0.79  0.71  0.20 
predtmbb^{4} (1best)  0.71  0.73  0.79  0.71  0.20 
predtmbb^{4} (posterior)  0.72  0.75  0.81  0.71  0.20 
Finally, the third server HMMB2TMR [21] achieves a performance quite similar to that obtained with PV decoding. To do that HMMB2TMR takes advantage of the MaxSubSeq algorithm on top of the posterior sum decoding. However, although MaxSubSeq is a very general twoclass segment optimization algorithm, it is a post processing procedure that has to be applied after a HMM decoding. On the contrary, PV is a general decoding algorithm and it is more useful when the underlying predictor is a HMM, where more than two labels and different constraints can be introduced into the automaton grammars.
Conclusion
The new PV decoding algorithm is more convenient in that it overcomes the difficulties of introducing a problemdependent optimization algorithm when the automaton grammar is to be recast. When onestatepath dominates we may expect that PV does not perform better than the other decoding algorithms, and in these cases the 1best is preferred [20]. Nevertheless, we show that when several concurring paths are present, as in the case of our betabarrel HMM, PV performs better than the others. Although PV takes more time than other algorithms (the posterior + the Viterbi time), the PV asymptotic computational timecomplexity still remains O(N^{2}·L) (where L and N are the protein length and the number of states, respectively) as for the other decodings. As far as the memory requirement is concerned, PV needs the same spacecomplexity of the Viterbi and posterior (O(N·L)), while 1best in the average case requires less memory, and can also be reduced [20]. When computational speed is an issue, Viterbi algorithm is the fastest and the time complexity order is time(viterbi) ≤ time(l  best) ≤ time(PV). Finally, PV satisfies any HMM grammar structures, including automata containing silent states, and it is applicable to all the possible HMM models with an arbitrary number of labels and without having to work out a problemdependent optimization algorithm.
Methods
The hidden Markov model definitions
For sake of clarity and compactness, in what follows we make use of explicit BEGIN (B) and END states and we do not treat the case of the silent (null) states. However, their inclusion in the algorithms is only a technical matter and can be done following the prescriptions indicated in [18, 19].
An observed sequence of length L is indicated as O (= O_{1}...O_{ L }) both for a singlesymbolsequence (as in the standard HMMs) or for a vectorsequence as described before [12]. λ(s) indicates the label associated to the state s, while Λ (= Λ_{ i },... Λ_{ L }) is the list of the labels associated to each sequence position i obtained after the application of a decoding algorithm. Depending on the problem at hand, the labels may identify transmembrane regions, loops, secondary structures of proteins, coding/non coding regions, intergenic regions, etc. A HMM consisting of N states (indicated below with s and k) is therefore defined by three probability distributions:
Starting probabilities
a_{B,k}= P(kB) (1)
Transition probabilities
a_{k,s}= P(sk) (2)
Emission probabilities
e_{ k }(O_{ i }) = P(O_{ i }k) (3)
The forward probability is
f_{ k }(i) = P(O_{1},O_{2}...O_{ i },π_{ i }= k) (4)
which is the probability of having emitted the first partial sequence up to position i ending at state k. The backward probability is:
b_{ k }(i) = P(O_{i+1},... O_{L  1}, O_{ L }π_{ i }= k) (5)
which is the probability of having emitted the sequence starting from the last element back to the (i+l)th element, given that we end at position i in state k. The probability of emitting the whole sequence can be computed using either the forward or backward probabilities according to:
P(OM) = f_{ END }(L + 1) = b_{ B }(0) (6)
Forward and backward probabilities are also necessary for updating the HMM parameters, using the BaumWelch algorithm [18, 19]. Alternatively a gradientbased training algorithm can be applied [18, 20].
Viterbi decoding
Viterbi decoding finds the path (π) through the model which has the maximal probability [18, 19]. This means that we look for the path which is
π^{ v }= argmax_{{π}}P(πO, M) (7)
where O(= O_{1},... O_{ L }) is the observed sequence of length L and M is the trained HMM model. Since the P(OM) is independent of a particular path π, Equation 7 is equivalent to
π^{ v }= argmax_{{π}}P(π, OM) (8)
P(π, OM) can be easily computed as
where by construction π(0) is always the BEGIN state (B).
Defining v_{ k }(i) as the probability of the most likely path ending in state k at position i, and p_{ i }(k) as the traceback pointer, π^{ v }can be obtained running the following dynamic programming algorithm called Viterbi decoding:
• Initialization
v_{ B }(0) = 1 v_{ k }(0) = 0 for k ≠ B
• Recursion
• Termination
• Traceback
• Label assignment
where λ(s) is the label associated to the s state.
1best decoding
The 1best labeling algorithm described here is Krogh's previously described variant of the Nbest decoding [20]. Since there is no exact algorithm for finding the most probable labeling, 1best is an approximate algorithm which usually achieves good results in solving this task [20]. Differently from Viterbi, the 1best algorithm ends when the most probable labeling is computed, so that no traceback is needed.
For sake of clarity, here we present a redundant description, in which we define H_{ i }as the set of all labeling hypotheses surviving as 1best for each state s up to sequence position i. In the worst case the number of distinct labelinghypotheses is equal to the number of states, is the current partial labeling hypothesis associated to the s state from the beginning to the ith sequence position. In general several states may share the same labeling hypothesis. Finally, we use ⊕ as the string concatenation operator, so that 'AAAA'⊕'B' = 'AAAAB' (the empty string is " and the empty set is ∅). Thus 1best algorithm can be described as
• Initialization
v_{ B }(") = 1 v_{ k }(") = 0 for k ≠ B
v_{ k }(λ(k)) = a_{B,k}·e_{ k }(O_{1})
H_{1} = {λ(k) : a_{B,k}≠ 0} H_{ i }= ∅ for i ≠ 1
• Recursion
• Termination
With 1best decoding, we do not need to keep a backtrace matrix since Λ is computed during the forward steps.
Posterior decoding
The posterior decoding finds the path which maximizes the product of the posterior probability of the states [18, 19]. Using the usual notation for forward (f_{ k }(i)) and backward (b_{ k }(i)) we have
P(π_{ i }= kO,M) = f_{ k }(i)b_{ k }(i)/P(OM) (10)
The path π^{ p }which maximizes the posterior probability is then computed as
for i = 1... L. The corresponding label assignment is
If we have more than one state sharing the same label, labeling can be improved by summing over the states that share the same label (posterior sum). In this way we can have a path through the model which maximizes the posterior probability of being in a state with label λ when emitting the observed sequence element, or more formally:
Λ_{ i }= argmax_{{λ}}P(label(O_{ i }) = λ O, S) (14)
where i ranges from 1 to L.
The posteriordecoding drawback is that the state path sequences π^{ p }or Λ may be not feasible paths.
However, this decoding can perform better than Viterbi, when more than one highly probable path exists [18, 19]. In this case a postprocessing algorithm that recasts the original topological constraints is recommended [21].
In the sequel, if not differently indicated, with the term posterior we mean the posterior sum.
PosteriorViterbi decoding
PosteriorViterbi decoding is based on the combination of the Viterbi and posterior algorithms. After having computed the posterior probabilities we use a Viterbi algorithm to find the best allowed posterior path through the model. A related idea, specific for pairwise alignments was previously introduced to improve the sequence alignment accuracy [22].
In the PV algorithm, the basic idea is to compute the path π^{ PV }
where A_{ p }is the set of the allowed paths through the model, and P(π_{ i }O,M) is the posterior probability of the state assigned by the path π at position i (as computed in Eq. 10).
Defining a function δ*(s, t) equal to 1 if s → t is an allowed transition of the model M, 0 otherwise; v_{ k }(i) as the probability of the most probable allowedposterior path ending at state k having observed the partial O_{1},... O_{ i }and p_{ i }as the traceback pointer, we can compute the best path π^{ PV }using the Viterbi algorithm:
• Initialization
v_{ B }(0) = 1 v_{ k }(0) = 0 for k ≠ B
• Recursion
• Termination
• Traceback
• Label assignment
An alternative approach, that directly maximizes the most probable labelling, is to substitute the posterior probability of a given state P(π_{ i }= kO, M), with the posterior sum P(label(O_{ i }) = λO, M) (equation 14). In this case all the states that share the same label have the same probability for each sequence position. However, since the performances of this second version are slightly worse we do not show them.
Datasets
The problem of the prediction of the allbeta transmembrane regions is used to test the algorithm on a real data application. In this case we use a set that includes 20 constitutive betabarrel membrane proteins whose sequences are less than 25% homologous and whose 3D structure have been resolved. The number of betastrands forming the transmembrane barrel ranges from 2 to 22. Among the 20 proteins, 15 were used to train a circular HMM (described in [12]), and here are tested in crossvalidation (1a0sP, 1bxwA, 1e54, 1ek9A, 1fcpA, 1fep, 1i78A, 1k24, 1kmoA, 1prn, 1qd5A, 1qj8A, 2mprA, 2omf, 2por). Since there is no detectable sequence identity among the selected 15 proteins, we adopted a leaveoneout approach for training the HMM and testing it. All the reported results are obtained during the testing phase, and the complete set of results is available at http://www.biocomp.unibo.it/piero/posvit. The other 5 new proteins (1mm4, 1nqf, 1p4t, 1uyn, 1t16) are used as a blind new test. Since our goal is to predict the betastrands that span the membrane we score the methods using the annotations derived from the PDB files. An alternative approach not addressed here, is to predict the portion of the transmembrane betastrands in contact with the lipid bilayer. This prediction is however out of the scope of our approach, since in real porins the localization of the betastrands in contact with the membrane, has been so far estimated by means of different computational methods and assumptions [25].
Measures of accuracy
We used three indices to score the accuracy of the algorithms. The first one is Q_{2} which computes the number of correctly assigned labels divided by the total number of observed symbols. Then we use the SOV index [26] to evaluate the segment overlaps. Finally, we also adopt a very stringent measure called Q_{ ok }: a prediction is considered correct only if the number of transmembrane segments coincides with the observed one and the corresponding segments have a minimal overlap of m residues [21]. The value m is segmentdependent and for each segment pairs, is computed as
m = min{seg_{ pr }/ 2, seg_{ ob }/2} (16)
where seg_{ pr } and seg_{ ob } are the predicted and observed segment lengths, respectively.
List of abbreviations
 • HMM:

hidden Markov model.
 • PV:

PosteriorViterbi.
Declarations
Acknowledgements
We thank Anders Krogh for the help with the 1best algorithm. This work was partially supported by the BioSapiens Network of Excellence, two grants of the Ministero della Istruzione dellUniversitá e della Ricerca (MIUR) 'Hydrolases from Thermophiles: Structure, Function and Homologous and Heterologous Expression' delivered to R.C. and 'Large scale modelling of proteases' delivered to P.F., a PNR 2001–2003 (FIRB art.8) and a PNR 2003–2007 (FIRB art.8).
Authors’ Affiliations
References
 Schulz G: Betabarrel membrane proteins. Curr Opin Struct Biol 2000, 10: 443–447. 10.1016/S0959440X(00)001202View ArticlePubMedGoogle Scholar
 Casadio R, Fariselli P, PL M: In silico prediction of the structure of membrane proteins: Is it feasible. Brief Bioinf 2003, 4: 341–348. 10.1093/bib/4.4.341View ArticleGoogle Scholar
 Casadio R, Jacoboni I, Messina A, V DP: A 3D model of the voltagedependent anion channel (VDAC). FEBS Lett 2003, 520: 1–7. 10.1016/S00145793(02)027588View ArticleGoogle Scholar
 Casadio R, Fariselli P, Finocchiaro G, Martelli P: Fishing new proteins in the twilight zone of genomes: The test case of outer membrane proteins in Escherichia coli K12, Escherichia coli O157:H7, and other Gramnegative bacteria. Protein Sci 2003, 11: 1158–1168. 10.1110/ps.0223603View ArticleGoogle Scholar
 Bagos P, Liakopoulos T, SJ H: Evaluation of methods for predicting the topology of barrel outer membrane proteins and a consensus prediction method. BMC Bioinformatics 2005, 1: 1–7.Google Scholar
 Krogh A, Brown M, Mian I, Sjolander K, Haussler D: Hidden Markov models in computational biology: Applications to protein modeling. Journal of Molecular Biology 1994, 235: 1501–1531. 10.1006/jmbi.1994.1104View ArticlePubMedGoogle Scholar
 Baldi P, Chauvin Y, Hunkapiller T, McClure M: Hidden Markov Models of Biological Primary Sequence Information. PNAS USA 1994, 91: 1059–1063. 10.1073/pnas.91.3.1059PubMed CentralView ArticlePubMedGoogle Scholar
 Mamitsuka H: Predicting peptides that bind to MHC molecules using supervised learning of hidden Markov models. Proteins 1998, 33: 460–474. 10.1002/(SICI)10970134(19981201)33:4<460::AIDPROT2>3.0.CO;2MView ArticlePubMedGoogle Scholar
 Bateman A, Birney E, Cerruti L, Durbin R, Etwiller L, Eddy S, GriffithsJones S, Howe K, Marshall M, Sonnhammer E: The Pfam Protein Families Database. Nucleic Acids Research 2002, 30: 276–280. 10.1093/nar/30.1.276PubMed CentralView ArticlePubMedGoogle Scholar
 Tusnady G, Simon I: Principles governing amino acid composition of integral membrane proteins: application to topology prediction. J Mol Biol 1998, 283: 489–506. 10.1006/jmbi.1998.2107View ArticlePubMedGoogle Scholar
 Krogh A, Larsson B, von Heijne G, Sonnhammer E: Predicting transmembrane protein topology with a hidden Markov model: application to complete genomes. J Mol Biol 2001, 305: 567–580. 10.1006/jmbi.2000.4315View ArticlePubMedGoogle Scholar
 Martelli P, Fariselli P, Krogh A, Casadio R: A sequenceprofilebased HMM for predicting and discriminating beta barrel membrane proteins. Bioinformaticsi 2002, 18: S46S53.View ArticleGoogle Scholar
 Martelli P, Fariselli P, Casadio R: An ENSEMBLE machine learning approach for the prediction of allalpha membrane proteins. Bioinformatics 2003, 19: i205i211. 10.1093/bioinformatics/btg1027View ArticlePubMedGoogle Scholar
 Liu Q, Zhu Y, Wang B, Li Y: A HMMbased method to predict the transmembrane regions of betabarrel membrane proteins. Comput Biol Chem 2003, 27: 69–76. 10.1016/S00978485(02)000517View ArticlePubMedGoogle Scholar
 Viklund H, Elofsson A: Best alphahelical transmembrane protein topology predictions are achieved using hidden Markov models and evolutionary information. Protein Sci 2004, 13: 1908–1917. 10.1110/ps.04625404PubMed CentralView ArticlePubMedGoogle Scholar
 Bagos P, Liakopoulos T, Spyropoulos I, SJ H: PREDTMBB: a web server for predicting the topology of betabarrel outer membrane proteins. Nucleic Acids Res 2004, 32: W400W404. 10.1093/nar/gkh417PubMed CentralView ArticlePubMedGoogle Scholar
 Bigelow H, Petrey D, Liu J, Przybylski D, B R: Predicting transmembrane betabarrels in proteomes. Nucleic Acids Res 2004, 32: 2566–2577. 10.1093/nar/gkh580PubMed CentralView ArticlePubMedGoogle Scholar
 Baldi P, Brunak S: Bioinformatics: the Machine Learning Approach. Cambridge: MIT Press; 2001.Google Scholar
 Durbin R, Eddy S, Krogh A, Mitchinson G: Biological sequence analysis: probabilistic models of proteins and nucleic acids. Cambridge: Cambridge Univ Press; 1998.View ArticleGoogle Scholar
 Krogh A: Two methods for improving performance of a HMM and their application for gene finding. In Proceedings of the Fifth International Conference on Intelligent Systems for Molecular Biology. ISCB, AAAI Press; 1997:179–186.Google Scholar
 Fariselli P, Finelli M, Marchignoli D, Martelli P, Rossi I, R C: MaxSubSeq: an algorithm for segmentlength optimization. The case study of the transmembrane spanning segments. Bioinformatics 2003, 19: 500–505. 10.1093/bioinformatics/btg023View ArticlePubMedGoogle Scholar
 Holmes I, Durbin R: Dynamic programming alignment accuracy. J Comput Biol 1998, 5: 493–504.View ArticlePubMedGoogle Scholar
 Altschul S, Madden T, Schaffer A, Zhang J, Zhang Z, Miller W, DJ L: Gapped BLAST and PSIBLAST: A new generation of protein database search programs. Nucleic Acid Res 1997, 25: 3389–3402. 10.1093/nar/25.17.3389PubMed CentralView ArticlePubMedGoogle Scholar
 Krogh A: Hidden Markov models for labeled sequences. In Proceedings 12th International Conference on Pattern Recognition. Singapore. IEEE Comp Soc Press; 1994:140–144.Google Scholar
 Tusnady G, Dosztanyi Z, Simon I: Transmembrane proteins in the Protein Data Bank: identification and classification. Bioinformatics 2004, 20: 2964–2972. 10.1093/bioinformatics/bth340View ArticlePubMedGoogle Scholar
 Zemla A, Venclovas C, Fidelis K, B R: A modified definition of Sov, a segmentbased measure for protein secondary structure prediction assessment. Proteins 1999, 34: 220–223. 10.1002/(SICI)10970134(19990201)34:2<220::AIDPROT7>3.0.CO;2KView ArticlePubMedGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.