A benchmark server using high resolution protein structure data, and benchmark results for membrane helix predictions

Rath, Emma M; Tessier, Dominique; Campbell, Alexander A; Lee, Hong Ching; Werner, Tim; Salam, Noeris K; Lee, Lawrence K; Church, W Bret

doi:10.1186/1471-2105-14-111

Table 1 Benchmark results showing prediction methods and their scores for benchmark measures

From: A benchmark server using high resolution protein structure data, and benchmark results for membrane helix predictions

	Benchmark measure (and subset of data used for benchmark)
	Sensitivity		Specificity		Correctly predicted sequences		N-terminal topology		Non-membrane topology segments	Helix boundaries
Prediction Method	(1)	(2)	(3)	(1)	(4)	(5)	(4)	(1)	(1)	(1)	(6)
DAS-TMfilter	81	77	91	96	51	88				62	63
DAS1997 (loose)	82	78	67	89	55	72				57	64
DAS1997 (strict)	89	84	39	85	42	37				67	69
DAS2002	82	78	92	97	56	90				62	62
deltaG	89	87	75	96	63	76				72	68
Eisen (11,10)	59	57	11	58	10	5				31	27
Eisen (19,10)	41	36	9	51	2	1				17	14
Eisen (7,10)	74	71	13	60	16	7				45	39
ENSEMBLE (in MemPype)	81	81	90	95	52	89	68	62	80	67	62
HMM-TM	63	43	82	97	60	85	76	77	66	53	53
HMMTOP (in TOPCONS-single)	89	86	95	96	65	94	76	75	79	72	71
HMMTOP2	90	86	84	96	66	86	79	77	81	72	72
KyteD (11,10)	75	70	21	71	26	16				53	46
KyteD (19,10)	58	50	16	66	6	1				30	24
KyteD (7,10)	84	78	25	75	36	23				62	59
MemBrain	94	93	79	95	69	82				80	75
MEMSAT (in TOPCONS-single)	87	86	93	94	57	92	69	71	79	71	71
MEMSAT-SVM	91	88	46	97	72	11	88	84	83	82	81
MEMSAT3	88	88	46	96	72	15	93	93	94	50	66
OCTOPUS	94	93	83	98	77	85	91	90	91	83	84
OCTOPUS (in TOPCONS)	90	88	92	97	70	91	89	88	86	75	76
OHM (11,10)	72	65	16	69	16	10				42	35
OHM (19,10)	51	46	12	62	7	2				23	17
OHM (7,10)	83	79	22	73	23	16				53	47
PHDhtm (at PBIL)	77	69	70	94	47	74				58	56
PHDThtm (at PBIL)	82	78	93	95	48	91	54	55	69	66	67
Philius	90	86	96	97	69	94	79	79	83	77	73
Phobius	87	86	95	97	60	93	66	65	78	73	69
PolyPhobius	91	89	92	96	67	92	69	70	80	78	75
PRED-TMR	83	81	89	96	55	88				67	68
PRO-TMHMM (in TOPCONS)	90	88	95	97	66	94	89	89	85	72	73
PRODIV-TMHMM (in TOPCONS)	94	91	43	96	74	32	89	89	89	78	78
S-TMHMM (in TOPCONS-single)	85	85	95	96	57	93	81	83	82	69	68
SCAMPI	88	87	94	96	65	93	89	88	84	72	71
SCAMPI-multi (in TOPCONS)	90	88	92	97	70	91	89	88	86	75	76
SCAMPI-sequence (in TOPCONS)	88	87	94	96	65	93	84	84	82	72	71
SCAMPI-sequence (in TOPCONS-single)	88	87	94	96	65	94	84	84	82	71	72
SOSUI	82	83	93	96	49	90				64	62
SPLIT4	75	74	87	96	40	84				60	56
SVMtm	81	84	95	97	56	92				65	67
SVMtop	86	88	95	96	52	92	72	75	70	72	65
TMAP	84	81	76	97	55	77				59	52
TMHMM2	86	86	96	97	59	93	73	74	80	72	71
TMMOD	85	85	96	97	56	93	69	72	81	70	68
TMpred	86	81	64	94	52	65	61	65	69	70	67
TOPCONS	91	88	92	98	72	91	89	88	87	77	77
TOPCONS-single	89	87	95	96	66	94	81	80	82	75	75
TOPPRED2	85	82	62	94	55	62	73	72	73	70	69
VALPRED	76	70	61	94	45	64				42	33
VALPRED2	93	90	54	80	40	65				55	50
waveTM	88	84	79	96	52	80				62	58

Numbers in brackets refer to the subset of data used in the benchmark as specified in Table 3. The benchmark server metric used for sensitivity is “Qhtm %obs”, for specificity is “Qhtm %prd”, for correctly predicted sequences is “Qok”, for N-terminal topology is “Nterm”, for non-membrane topology segments is “Qio %obs”, and for helix boundaries is “QHb %obs”. The scores of the 5 highest scoring prediction methods are marked in bold (and if there is more than one method having the score of the 5^th highest scoring method then all methods having that score are marked in bold). Methods that do not predict topology do not have topology scores.

Back to article page

ISSN: 1471-2105

Contact us

General enquiries: journalsubmissions@springernature.com

BMC Bioinformatics

Contact us