Skip to main content

Table 1 Per-protein performance. *

From: TMbed: transmembrane proteins predicted through language model embeddings

 

β-TMP (57)

α-TMP (571)

Globular (5654)

Recall (%)

FPR (%)

Recall (%)

FPR (%)

Recall (%)

FPR (%)

TMbed

93.8 ± 7.5

0.1 ± 0.1

97.5 ± 0.7

0.5 ± 0.2

99.5 ± 0.2

2.8 ± 1.2

DeepTMHMM

77.9 ± 12.7

0.1 ± 0.1

95.8 ± 1.3

0.5 ± 0.2

99.5 ± 0.2

5.9 ± 2.2

TMSEG

–

–

96.5 ± 1.0

2.3 ± 0.3

97.7 ± 0.3

3.5 ± 1.0

TOPCONS21

–

–

94.2 ± 1.3

2.6 ± 0.3

97.4 ± 0.3

5.8 ± 1.3

OCTOPUS1

–

–

94.2 ± 1.9

9.1 ± 0.7

90.9 ± 0.7

5.8 ± 1.9

Philius1

–

–

92.5 ± 1.4

2.6 ± 0.2

97.4 ± 0.2

7.5 ± 1.4

PolyPhobius1

–

–

97.2 ± 1.1

5.3 ± 0.4

94.7 ± 0.4

2.8 ± 1.1

SPOCTOPUS1

–

–

97.5 ± 1.6

17.2 ± 0.8

82.8 ± 0.8

2.5 ± 1.6

SCAMPI2 (MSA)

–

–

94.2 ± 1.6

5.6 ± 0.3

94.4 ± 0.3

5.8 ± 1.6

CCTOP2

  

96.1 ± 2.1

3.7 ± 0.6

96.3 ± 0.6

3.9 ± 2.1

HMM-TM (MSA)3

–

–

97.3 ± 1.6

21.4 ± 0.5

78.6 ± 0.5

2.7 ± 1.6

BOCTOPUS2

84.0 ± 13.3

4.2 ± 0.5

–

–

95.8 ± 0.5

16.0 ± 13.3

BetAware-Deep

85.1 ± 9.3

4.7 ± 0.3

–

–

95.3 ± 0.3

14.9 ± 9.3

PRED-TMBB24

88.8 ± 12.1

7.1 ± 0.4

–

–

92.9 ± 0.4

11.2 ± 12.1

PROFtmb

91.9 ± 9.0

6.1 ± 0.5

–

–

93.9 ± 0.5

8.1 ± 9.0

  1. *Evaluation of the ability to distinguish between 57 beta barrel TMPs (β-TMP), 571 alpha helical TMPs (α-TMP) and 5654 globular, water-soluble non-TMP proteins in our data set. Recall and false positive rate (FPR) were averaged over the five independent cross-validation test sets; error margins given for the 95% confidence interval (1.96*standard error); bold: best values for each column; italics: differences statistically significant with over 95% confidence (only computed between best and 2nd best, or all methods ranked 1 and those ranked lower)
  2. 1Evaluation missing for one of 5,654 globular proteins
  3. 2Evaluation missing for one of 571 α-TMPs and six of 5,654 globular proteins
  4. 3Evaluation includes only 51 β-TMPs, 552 α-TMPs, and 5,524 globular proteins due to runtime errors
  5. 4The local PRED-TMBB2 version did not include the pre-filtering step of the web server. This caused a FPR for β-TMP of almost 78%. Thus, we listed the statistics for the web server predictions, which did not include MSA input