Skip to main content

Table 1 Per-protein performance. *

From: TMbed: transmembrane proteins predicted through language model embeddings

  β-TMP (57) α-TMP (571) Globular (5654)
Recall (%) FPR (%) Recall (%) FPR (%) Recall (%) FPR (%)
TMbed 93.8 ± 7.5 0.1 ± 0.1 97.5 ± 0.7 0.5 ± 0.2 99.5 ± 0.2 2.8 ± 1.2
DeepTMHMM 77.9 ± 12.7 0.1 ± 0.1 95.8 ± 1.3 0.5 ± 0.2 99.5 ± 0.2 5.9 ± 2.2
TMSEG 96.5 ± 1.0 2.3 ± 0.3 97.7 ± 0.3 3.5 ± 1.0
TOPCONS21 94.2 ± 1.3 2.6 ± 0.3 97.4 ± 0.3 5.8 ± 1.3
OCTOPUS1 94.2 ± 1.9 9.1 ± 0.7 90.9 ± 0.7 5.8 ± 1.9
Philius1 92.5 ± 1.4 2.6 ± 0.2 97.4 ± 0.2 7.5 ± 1.4
PolyPhobius1 97.2 ± 1.1 5.3 ± 0.4 94.7 ± 0.4 2.8 ± 1.1
SPOCTOPUS1 97.5 ± 1.6 17.2 ± 0.8 82.8 ± 0.8 2.5 ± 1.6
SCAMPI2 (MSA) 94.2 ± 1.6 5.6 ± 0.3 94.4 ± 0.3 5.8 ± 1.6
CCTOP2    96.1 ± 2.1 3.7 ± 0.6 96.3 ± 0.6 3.9 ± 2.1
HMM-TM (MSA)3 97.3 ± 1.6 21.4 ± 0.5 78.6 ± 0.5 2.7 ± 1.6
BOCTOPUS2 84.0 ± 13.3 4.2 ± 0.5 95.8 ± 0.5 16.0 ± 13.3
BetAware-Deep 85.1 ± 9.3 4.7 ± 0.3 95.3 ± 0.3 14.9 ± 9.3
PRED-TMBB24 88.8 ± 12.1 7.1 ± 0.4 92.9 ± 0.4 11.2 ± 12.1
PROFtmb 91.9 ± 9.0 6.1 ± 0.5 93.9 ± 0.5 8.1 ± 9.0
  1. *Evaluation of the ability to distinguish between 57 beta barrel TMPs (β-TMP), 571 alpha helical TMPs (α-TMP) and 5654 globular, water-soluble non-TMP proteins in our data set. Recall and false positive rate (FPR) were averaged over the five independent cross-validation test sets; error margins given for the 95% confidence interval (1.96*standard error); bold: best values for each column; italics: differences statistically significant with over 95% confidence (only computed between best and 2nd best, or all methods ranked 1 and those ranked lower)
  2. 1Evaluation missing for one of 5,654 globular proteins
  3. 2Evaluation missing for one of 571 α-TMPs and six of 5,654 globular proteins
  4. 3Evaluation includes only 51 β-TMPs, 552 α-TMPs, and 5,524 globular proteins due to runtime errors
  5. 4The local PRED-TMBB2 version did not include the pre-filtering step of the web server. This caused a FPR for β-TMP of almost 78%. Thus, we listed the statistics for the web server predictions, which did not include MSA input