A mass accuracy sensitive probability based scoring algorithm for database searching of tandem mass spectrometry data
© Xu and Freitas; licensee BioMed Central Ltd. 2007
Received: 07 December 2006
Accepted: 20 April 2007
Published: 20 April 2007
Liquid chromatography coupled with tandem mass spectrometry (LC-MS/MS) has become one of the most used tools in mass spectrometry based proteomics. Various algorithms have since been developed to automate the process for modern high-throughput LC-MS/MS experiments.
A probability based statistical scoring model for assessing peptide and protein matches in tandem MS database search was derived. The statistical scores in the model represent the probability that a peptide match is a random occurrence based on the number or the total abundance of matched product ions in the experimental spectrum. The model also calculates probability based scores to assess protein matches. Thus the protein scores in the model reflect the significance of protein matches and can be used to differentiate true from random protein matches.
The model is sensitive to high mass accuracy and implicitly takes mass accuracy into account during scoring. High mass accuracy will not only reduce false positives, but also improves the scores of true positive matches. The algorithm is incorporated in an automated database search program MassMatrix.
Liquid chromatography coupled with tandem mass spectrometry (LC-MS/MS) has become one of the most used tools in mass spectrometry based proteomics . In shotgun proteomics, peptides are separated using liquid chromatography and introduced into a mass spectrometer via an ionization interface. In tandem mass spectrometry, the peptide precursor ions are isolated and fragmented via collision-induced dissociation (CID)  with inert gas, electron capture dissociation (ECD) , surface induced dissociation (SID)  and/or electron transfer dissociation (ETD) . The resulting tandem MS spectra contain product ion signatures that relate back to the identity of the peptide precursor ions [2, 6, 7].
Various algorithms have since been developed to automate the process for modern high-throughput LC-MS/MS experiments. These algorithms fall under two categories: de novo sequence inference and database searching . The first approach identifies peptide sequences directly from the tandem MS data [9, 10]. This type of algorithm is usually computationally expensive and limited by the mass accuracy of the tandem MS data . The database searching algorithms identify peptides by comparison with a protein sequence database . In this approach, all potential peptides are created from the sequence database via digestion with proteases. Theoretical spectra containing product ion series appropriate for the given fragmentation technique are created for the peptides. All tandem MS spectra in the data set are then compared with the theoretical spectra . Because of their relatively lower computation expense and higher compatibility with low mass accuracy spectra, database searching programs are more commonly used at this time .
There are also various probability based post-search methods used to statistically curate search results from database search algorithms [13, 14]. These methods estimate the accuracy of protein/peptide identifications and compare search results from different algorithms based on a common standard. However, many models involve empirical parameters such as score from correlative scoring algorithms. Therefore they may possess biases as a result of parameter optimization or model training.
The key comparison between different algorithms lies in how each approach scores a potential match between experimental and theoretical spectra [11, 15–25]. We recently developed a database searching program, MassMatrix that uses a mass accuracy sensitive statistical model for scoring. This approach is separate and distinct from algorithms that filter matches based on mass accuracy. In the latter high mass accuracy can be used to filter spectra by only searching tandem mass spectra whose precursor ion falls within the stated mass tolerance, and filtering product ions by high mass accuracy can further reduce the likelihood of a random match [26, 27]. However, a score sensitive model implicitly takes mass accuracy into account during scoring. The model is rigorously derived and sensitive to the searching tolerance determined by the accuracy of mass spectrometer. High accuracy improves the sensitivity and selectivity of searches. The statistical scores represent the probability that a match is a random occurrence. In addition, a novel statistically derived algorithm to rigorously calculate protein scores from the statistically based peptide scores has been developed. Thus the protein scores reflect the significance of protein hits and can be used to differentiate true protein hits from random ones. Herein we describe the statistical models.
Multiple scoring algorithms
The peptide matching algorithm contains two independent scoring models, including a descriptive model and a statistical model. These models are used to calculate three distinct scores for a peptide match. Each of the scores may be independently used to ascertain the quality of the match. Because each score is distinct, the combination of scores is useful for validating each peptide match. The two models and the application to calculating peptide match scores are described in detail in the following.
Descriptive peptide scoring model
I i is defined as the standardized abundance of the ith product ion in the experimental spectrum (calculated by dividing the abundance of the ith product ion by the maximum abundance in the spectrum), is the total standardized abundance of matched product ions, nmatch is the number of matched product ions, rmatch is the ratio of standardized abundance of matched product ions to total standardized abundance of the experimental spectrum, and Lpep is the length of the peptide in the number of amino acids. Each of these factors contributes to the overall score as follows: evaluates the quality of the match, introduces a penalty for unmatched product ions, is an arbitrary penalty for matches with poor fragmentation, is an additional penalty for peptides with long sequences and the constant 100 is used arbitrarily to scale the scores. By default, scores for a spectrum with less than three matched product ions will be 0 due to the arbitrary penalty. However, the minimum number of matched ions may be changed to any value. Reducing this number is especially valuable for the analysis of singly charged peptides that have characteristic C-terminal aspartic acid fragmentation . The penalty for peptide length is included to normalize the scores. Peptides with longer sequences have more fragment ions and higher empirical scores than shorter sequences. The penalty results in long and short sequences both have similar scores for matches of similar quality. The choices of incorporating squared and square root for the terms nmatch and Lpep were empirically determined from the evaluation of tandem MS data sets collected from LCQ and LTQ-Orbitrap mass spectrometers.
Descriptive protein score
Probability based peptide scoring model
In addition to the empirical score, a mass accuracy sensitive probability based scoring model was derived to evaluate peptide matches. The model determines the likelihood that an experimental spectrum match to a theoretical spectrum is a random occurrence. Consider a pair of spectra: one experimental and one theoretical. W e and W t denote their precursor masses respectively. In addition, the experimental data contains information regarding the abundance of product ions I i for each precursor, W e . The model ultimately tests the following two hypotheses: the null hypothesis, H0, states that a match is random, i.e. the theoretical spectrum is independent of the experimental; and the alternative hypothesis, HA, states that the match is not random, i.e. the theoretical spectrum is related to the experimental one.
Scoring the match is performed in two stages: 1) match W e against W t within the specified precursor ion mass accuracy and 2) match all product ions in the experimental spectrum against the theoretical within the specified mass accuracy. Both stages rely on calculating the probability that the occurrence of an ion within a fixed mass window could be a random occurrence ().
In the above equation, τpep is the mass accuracy of the precursor ion and Π is the detection range for the precursor ion. For each precursor ion the mass window is defined as ± τpep (2 × τpep). Thus q has a bernoulli (p1) distribution under H0. If the precursor ion masses of the pair of spectra do not match (q = 0) then the second stage is skipped. If q = 1 we proceed to stage 2 where we test the match of the experimental product ion spectrum against the theoretical spectrum.
where Πtheo is the total coverage of the detection range for all product ions in the theoretical spectrum and Π is the MS/MS detection range. It is assumed that Π is the same as the precursor ion mass range. However, for instruments that have a dynamic detection range assuming a fixed value Π will result in more conservative scores. For each product ion in the theoretical spectrum, the mass window is ± τmsms (2 × τmsms). If we assume there is no overlap in the product ion mass windows, then Πtheo is calculated using the following equation
Πtheo = 2 m × τmsms.
where τmsms is the product ion mass accuracy and m is the number of product ions within the detection range in the theoretical spectrum. Because the theoretical spectrum is independent of the experimental under H0, all b i (i = 1, 2 ..., n) are assumed to have an identical and independent bernoulli (p2) distribution under H0. The model is then used to perform two distinct tests. Each uses a different approach to evaluate the quality of a match: number of matched product ions x and total abundance of matched product ions Y.
The pp score
where p2 is calculated from eqn. 6. The p-value, α, is defined as the probability that the quality of a random match between a pair of spectra is greater than or equal to a match observed under H0. The pp value, β, is defined as the negative common logarithm of the p-value:
β = -log(α)
The pp2 score
However, to complete the test we must know the inherent distribution of Y. This distribution is unknown and thus pp2 values can not be precisely calculated as were the pp values based on the total number of matched product ions. In order to estimate the pp2 value, three assumptions are needed:
1. I i is identically and independently distributed across product ions in the experimental spectrum,
2. b i is uncorrelated with I i in the experimental spectrum,
3. the number of product ions, n, in the experimental spectrum is large (n > 30).
The pp2 value can be estimated by equation 17 very efficiently. However, the real distribution of Y is more tailed to larger values than the normal distribution. Therefore, pp2 values are overestimated when they are large.
Distribution of pp value for random matches
In statistical hypothesis testing, a p-value for a null hypothesis H0 is always a uniform distribution on the interval [0, 1]. Therefore, the cumulative distribution function for p-value of a random match is continuously distributed as
Fα|q = 1(α) = α (0 ≤ α≤ 1)
when q = 1. According to the definition of pp value (eqn. 11), the cumulative distribution function for pp value when q = 1 is
Fβ|q = 1(β) = 1 - 10-β (β ≥ 0)
Confidence level for pp and pp2 values
The confidence level, ψ, is defined as
ψ = -log(φ) = -log(r p1 10-β) = β -log(r)-log(p1)
where β is either the pp or pp2 value, r is the number of theoretical spectra within the protein sequence database, and p1 is given in eqn. 4. Confidence levels calculated from pp value and pp2 value are referred as confidence level and confidence level2 respectively.
The confidence level is the negative common logarithm of the expected number of random matches with a pp value bigger than or equal to the one we observe for the corresponding experimental spectrum. Therefore, if the confidence level is below 0, more than one random match for the spectrum is expected and the corresponding match is highly suspect. From eqn. 38, the pp value is directly related to the confidence value. The confidence level is dependent upon the size of the database and degrades as the number of peptide created from the database increases.
The protein pp score
The statistical model is used to test the following hypotheses: H0 – All peptide matches for a given protein are random and HA – At least one peptide match for a given protein is not random. We assume that rspectra theoretical spectra created from the protein sequence are uncorrelated to each other and that nspectra experimental spectra from the data set are uncorrelated to each other. Since nmatch_protein is normally very large, B is approximately a normal distribution with a mean of μ B = nmatch_protein × μ β and a variance of σ B 2 = nmatch_protein × σ β 2 according to the central limit theorem.
Effect of various spectral characteristics on scoring
Empirical and statistical scores along with associated parameters (p 1 and p 2) for each spectrum shown in Figure 1. The data were obtained for mass accuracies of 1.0 Da, 0.1 Da and 0.01 Da. Confidence levels were calculated based on a search space of 2726345 theoretical peptides. The confidence levels for the pp and pp2 scores are denotes as confidence level and confidence level2
Confidence level 2
2.1. × 10-5
7.9. × 10-4
2.1. × 10-5
2.1. × 10-3
2.1. × 10-5
9.2. × 10-4
2.1. × 10-5
1.5. × 10-3
2.1. × 10-5
6.5. × 10-4
2.1. × 10-4
7.8. × 10-3
2.1. × 10-4
2.1. × 10-2
2.1. × 10-4
9.1. × 10-3
2.1. × 10-4
1.5. × 10-2
2.1. × 10-4
6.5. × 10-3
2.1. × 10-3
7.8. × 10-2
2.1. × 10-3
2.1. × 10-1
2.1. × 10-3
9.1. × 10-2
2.1. × 10-3
1.5. × 10-1
2.1. × 10-3
6.4. × 10-2
Comparison between pp and pp2 values
The pp value is the primary discriminator for quality of matches. The pp2 value can provide a complementary assessment of quality when pp values are suspect. Although the pp and pp2 value have the same statistical basis, there are several differences between them: The pp value is based on the number of matched product ions and the pp2 value is based on the total abundance of matched product ions. The pp value can be underestimated when noise is present in the experimental spectrum especially at low mass accuracy. Because noise normally has lower abundance than product ions, the pp2 value, on the other hand, is generally unaffected. As shown in Table 1, pp value for the spectrum with low signal to noise ratio and majority of noise peaks (Figure 1c) were affected negatively by the noise peaks and relative low compared with those for normal spectra at mass accuracy of 1.0 Da. However, pp2 value was not affected by the noise peaks.
While pp value can be precisely calculated, there are three assumptions needed to estimate the pp2 values. Assumption 1 and 3 for the pp2 test are not plausible when the number of product ions in the experimental spectrum, n is small. Therefore, pp2 value estimated by the central limit theorem cannot evaluate the quality of matches with a small number of product ions. Furthermore the normal distribution under the central limit theorem is less tailed than the true distribution of Y, pp2 value is normally overestimated when it is large (> 16) as shown in Table 1.
From the above discussion, the pp value is more reliable and accurate than the pp2 value under most circumstances, but it can be affected by noise. Under these circumstances, the number of product ions in the spectrum is normally large and pp2 value can be well estimated and complementary to pp value. Thus the combination of the two scores provides an excellent means to ascertain the quality of matches under conditions where one might fail.
Effect of mass accuracy on pp values
In the pp model, the two most important parameters (p1, the probability that a theoretical precursor randomly matches the experimental and p2, the probability that a theoretical product ion randomly matches any product ions in the MS/MS spectrum) are set in accordance with the predetermined mass accuracy of mass spectrometer. These parameters' values decrease as mass accuracy increases. This effect is shown in Table 1. A more thorough list of all parameters used in calculating the empirical and statistical scores is provided as supplementary material [see Additional file 1]. The statistical model specifically takes each parameter into account when calculating the statistical scores. Therefore, these two parameters have a substantial effect on the pp values for both random matches and true matches. Consequently, pp and pp2 values are very sensitive to the accuracy of mass spectrometer.
A new statistically derived scoring algorithm was developed for characterization of peptides, proteins and their posttranslational modifications from tandem MS data. The probability based algorithm implicitly incorporates mass accuracy into scoring the potential peptide and protein matches. This approach is separate and distinct from algorithms that filter precursor and product ion matches based on mass accuracy. The statistical model involves no empirical parameters and its scores correlate to the probability that a match is a random occurrence. A novel statistically derived algorithm to rigorously calculate protein scores from the probability based peptide scores was also developed. Thus the protein scores reflect the significance of protein matches and can be used to differentiate true protein matches from random matches. The algorithm is incorporated in an automated database search program MassMatrix.
The study was funded by the Ohio State University, the National Institutes of Health CA107106, the V Foundation/American Association for Cancer Research Translational Cancer Research Grant and the Leukemia & Lymphoma Society.
- Sadygov RG, Cociorva DC, Yates JR: Large-scale database searching using tandem mass spectra: Looking up the answer in the back of the book. Nature Methods. 2004, 1 (3): 195-202. 10.1038/nmeth725.View ArticlePubMed
- Hunt DF, Yates JR, Shabanowitz J, Winston S, Hauer CR: Protein sequencing by tandem mass-spectrometry. Proc Natl Acad Sci U S A. 1986, 57: 6233-6237. 10.1073/pnas.83.17.6233.View Article
- Bakhtiar R, Guan ZQ: Electron capture dissociation mass spectrometry in characterization of peptides and proteins. Biotechnol Lett. 2006, 28 (14): 1047-1059. 10.1007/s10529-006-9065-z.View ArticlePubMed
- Nikolaev EN, Somogyi A, Smith DL, Gu CG, Wysocki VH, Martin CD, Samuelson GL: Implementation of low-energy surface-induced dissociation (eV SID) and high-energy collision-induced dissociation (keV CID) in a linear sector-TOF hybrid tandem mass spectrometer. Int J Mass Spectrom. 2001, 212: 535-551. 10.1016/S1387-3806(01)00462-6.View Article
- Syka JEP, Coon JJ, Schroeder MJ, Shabanowitz J, Hunt DF: Peptide and protein sequence analysis by electron transfer dissociation mass spectrometry. Natl Acad Sci USA. 2004, 101: 9528-9533. 10.1073/pnas.0402700101.View Article
- Hunt DF, Buko AM, Ballard JM, Shabanowitz J, Giordani AB: Sequence-analysis of polypeptides by collision activated dissociation on a triple quadrupole mass-spectrometer. Biomed Mass Spectrom. 1981, 53: 397-408. 10.1002/bms.1200080909.View Article
- Biemann K: Contributions of mass-spectrometry to peptide and protein-structure. Biomed Environ Mass Spectrom. 1988, 16: 99-111. 10.1002/bms.1200160119.View ArticlePubMed
- Nesvizhskii AI, Aebersold R: Analysis, statistical validation and dissemination of large-scale proteomics datasets generated by tandem MS. Drug Discov Today. 2004, 9 (4): 173-181. 10.1016/S1359-6446(03)02978-7.View ArticlePubMed
- Dancík V, Addona TA, Clauser KR, Vath JE, Pevzner PA: De novo peptide sequencing via tandem mass spectrometry. J Comput Biol. 1999, 6: 327-342. 10.1089/106652799318300.View ArticlePubMed
- Standing KG: Peptide and protein de novo sequencing by mass spectrometry. Curr Opin Struc Biol. 2003, 13 (5): 595-601. 10.1016/j.sbi.2003.09.005.View Article
- Eng JK, McCormack AL, Yates JR: An approach to correlate tandem mass spectral data of peptides with amino acid sequences in a protein database. J Am Soc Mass Spectrom. 1994, 5: 976-989. 10.1016/1044-0305(94)80016-2.View ArticlePubMed
- Kapp EA, Schütz F, Connolly LM, Chakel JA, Meza JE, Miller CA, Fenyo D, Eng JK, Adkins JN, Omenn GS, Simpson RJ: An evaluation, comparison, and accurate benchmarking of several publicly available MS/MS search algorithms: sensitivity and specificity analysis. Proteomics. 2005, 5 (13): 3475-3490. 10.1002/pmic.200500126.View ArticlePubMed
- Keller A, Nesvizhskii A, Kolker E, Aebersold R: Empirical statistical model to estimate the accuracy of peptide identifications made by MS/MS and database search. Anal Chem. 2002, 74: 5383-5392. 10.1021/ac025747h.View ArticlePubMed
- Qian W, Liu T, Monroe ME, Strittmatter EF, Jacobs JM, Kangas LJ, Petritis K, Camp DG, Smith RD: Probability-Based Evaluation of Peptide and Protein Identifications from Tandem Mass Spectrometry and SEQUEST Analysis: The Human Proteome. J Proteome Res. 2005, 4: 53-62. 10.1021/pr0498638.View ArticlePubMed
- Zhang N, Aebersold R, Schwikowshi B: ProbID: a probabilistic algorithm to identify peptides through sequence database searching using tandem mass spectral data. Proteomics. 2002, 2: 1406-1412. 10.1002/1615-9861(200210)2:10<1406::AID-PROT1406>3.0.CO;2-9.View ArticlePubMed
- Perkins DN, Pappin DJC, Creasy DM, Cottrell JS: Probability-based protein identification by searching sequence database using mass spectrometry data. Electrophoresis. 1999, 20: 3551-3567. 10.1002/(SICI)1522-2683(19991201)20:18<3551::AID-ELPS3551>3.0.CO;2-2.View ArticlePubMed
- Hansen BT, Jones JA, Mason DE, Liebler DC: SALSA: a pattern recognition algorithm to detect electrophile-adducted peptides by automated evaluation of CID spectra in LC-MS-MS analyses. Anal Chem. 2001, 73: 1676-1683. 10.1021/ac001172h.View ArticlePubMed
- Mann M, Wilm M: Error tolerant identification of peptides in sequence databases by peptide sequence tags. Anal Chem. 1994, 66: 4390-4399. 10.1021/ac00096a002.View ArticlePubMed
- Bafna V, Edwards N: SCOPE: a probabilistic model for scoring tandem mass spectra against a peptide database. Bioinformatics. 2001, 17: S13-S21. 10.1093/bioinformatics/17.1.13.View ArticlePubMed
- Colinge J, Masselot A, Giron M, Dessingy T, Magnin J: OLAV: towards high-throughput tandem mass spectrometry data identification. Proteomics. 2003, 3: 1454-1463. 10.1002/pmic.200300485.View ArticlePubMed
- MacCoss MJ, Wu CC, Yates JR: Probability-based validation of protein identifications using a modified SEQUEST algorithm. Anal Chem. 2002, 74: 5593-5599. 10.1021/ac025826t.View ArticlePubMed
- Havilio M, Haddad Y, Smilansky Z: Intensity-based statistical scorer for tandem mass spectrometry. Anal Chem. 2003, 75: 435-444. 10.1021/ac0258913.View ArticlePubMed
- Sadygov RG, Yates JR: A hypergeometric probability model for protein identification and validation using tandem mass spectral data and protein sequence databases. Anal Chem. 2003, 75: 3792-3798. 10.1021/ac034157w.View ArticlePubMed
- Sadygov RG, Liu H, Yates JR: Statistical models for protein validation using tandem mass spectral data and protein amino acid sequence databases. Anal Chem. 2004, 76: 1664-1671. 10.1021/ac035112y.View ArticlePubMed
- Geer LY, Markey SP, Kowalak JA, Wagner L, Xu M, Maynard DM, Yang X, Shi W, Bryant SH: Open mass spectrometry search algorithm. J Proteome Res. 2004, 3: 958-964. 10.1021/pr0499491.View ArticlePubMed
- Olsen JV, de Godoy LMF, Li G, Macek B, Mortensen P, Pesch R, Makarov A, Lange O, Horning S, Mann M: Parts per million mass accuracy on an Orbitrap mass spectrometer via lock mass injection into a C-trap. Mol Cell Proteomics. 2005, 4 (12): 2010-2021. 10.1074/mcp.T500030-MCP200.View ArticlePubMed
- Clauser KR, Baker P, Burlingame AL: Role of accurate mass measurement (±10 ppm) in protein identification strategies employing MS or MS/MS and database searching. Anal Chem. 1999, 71: 2871-2882. 10.1021/ac9810516.View ArticlePubMed
- Sullivan AG, Brancia FL, Tyldesley R, Bateman R, Sidhu K, Hubbard SJ, Oliver SG, Gaskell SJ: The exploitation of selective cleavage of singly protonated peptide ions adjacent to aspartic acid residues using a quadrupole orthogonal time-of-flight mass spectrometer equipped with a matrix-assisted laser desorption/ionization source. Int J Mass Spectrom. 2001, 210/211: 665-676. 10.1016/S1387-3806(01)00430-4.View Article
- Xu H, Freitas AF: MassMatrix: A Database Searching Program for Rapid Characterization of Proteins and Peptides from Tandem Mass Spectrometry Data. To be submitted.
- Paizs B, Suhai S: Fragmentation pathways of protonated peptides. Mass Spectrom Rev. 2005, 24: 508-548. 10.1002/mas.20024.View ArticlePubMed
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.