The Protein-Protein Interaction tasks of BioCreative III: classification/ranking of articles and linking bio-ontology concepts to full text

Krallinger, Martin; Vazquez, Miguel; Leitner, Florian; Salgado, David; Chatr-aryamontri, Andrew; Winter, Andrew; Perfetto, Livia; Briganti, Leonardo; Licata, Luana; Iannuccelli, Marta; Castagnoli, Luisa; Cesareni, Gianni; Tyers, Mike; Schneider, Gerold; Rinaldi, Fabio; Leaman, Robert; Gonzalez, Graciela; Matos, Sergio; Kim, Sun; Wilbur, W John; Rocha, Luis; Shatkay, Hagit; Tendulkar, Ashish V; Agarwal, Shashank; Liu, Feifan; Wang, Xinglong; Rak, Rafal; Noto, Keith; Elkan, Charles; Lu, Zhiyong; Dogan, Rezarta Islamaj; Fontaine, Jean-Fred; Andrade-Navarro, Miguel A; Valencia, Alfonso

doi:10.1186/1471-2105-12-S8-S3

Table 4 ACT participant results

From: The Protein-Protein Interaction tasks of BioCreative III: classification/ranking of articles and linking bio-ontology concepts to full text

Team	Run/Srvr	Accuracy	Specificity	Sensitivity	F-Score	MCC	AUC iP/R	Time_half
TC	RUN _1	89.03	93.87	61.98	63.16	0.56733	68.98	30.13
T65	RUN_1	88.68	97.64	38.57	50.83	0.48297	63.85	40.19
T65	RUN_2	87.93	93.07	59.23	59.82	0.52727	63.89	40.19
T65	RUN_3	67.05	64.19	83.08	43.34	0.34244	41.74	55.95
T65	RUN_4	73.68	74.13	71.21	45.08	0.34650	41.74	55.95
T65	RUN_5	88.00	94.40	52.20	56.89	0.50255	62.39	40.83
T70	RUN_1	56.45	49.70	94.18	39.62	0.31789	56.76	42.12
T70	RUN_2	87.41	96.11	38.79	48.32	0.43346	56.76	42.13
T70	RUN_3	81.92	83.61	72.53	54.91	0.46563	56.76	42.12
T70	RUN_4	47.77	39.04	96.59	35.95	0.27060	56.76	42.12
T70	RUN_5	86.84	98.62	20.99	32.62	0.34488	56.76	42.13
T73	RUN_1	87.55	91.81	63.74	60.83	0.53524	65.91	38.33
T73	RUN_2	89.15	94.95	56.70	61.32	0.55306	67.96	37.10
T73	RUN_3	87.78	92.61	60.77	60.14	0.52932	65.89	38.19
T73	RUN_4	88.88	94.34	58.35	61.42	0.55054	67.98	37.15
T73	RUN_5	87.62	92.18	62.09	60.33	0.53031	65.37	38.40
T81	RUN_1	59.03	58.76	60.55	30.96	0.13949	19.93	82.27
T81	RUN_2	58.47	57.86	61.87	31.12	0.14219	19.69	82.76
T81	RUN_3	25.37	14.72	84.95	25.66	-0.00344	15.66	102.73
T81	RUN_4	63.45	69.16	31.54	20.74	0.00538	16.20	104.95
T81	RUN_5	69.17	77.35	23.41	18.72	0.00645	15.63	98.72
T81	SRVR_9	84.88	99.98	0.44	0.88	0.05220	44.19	50.11
T81	SRVR_10	85.38	99.61	5.82	10.78	0.17771	50.25	45.11
T81	SRVR_11	84.73	99.86	0.11	0.22	-0.00272	46.02	48.23
T81	SRVR_12	84.30	98.86	2.86	5.23	0.05244	32.11	56.89
T81	SRVR_13	84.88	99.92	0.77	1.52	0.05791	18.59	113.11
T88	RUN_1	42.63	35.11	84.73	30.94	0.15238	21.97	84.90
T88	RUN_2	56.92	53.73	74.73	34.47	0.20417	26.04	75.33
T89	RUN_1	80.02	80.90	75.06	53.26	0.44911	61.29	41.31
T89	RUN_2	81.00	81.75	76.81	55.08	0.47242	62.13	40.99
T89	RUN_3	82.40	83.85	74.29	56.15	0.48180	60.48	41.72
T89	RUN_4	87.73	94.79	48.24	54.40	0.47967	43.76	43.09
T89	RUN_5	87.27	91.81	61.87	59.58	0.52082	48.47	44.57
T89	SRVR_4	77.80	77.84	77.58	51.46	0.43152	57.44	44.63
T89	SRVR_5	78.05	78.15	77.47	51.71	0.43424	57.56	45.20
T89	SRVR_6	79.90	81.00	73.74	52.67	0.44073	54.97	45.93
T89	SRVR_7	86.25	92.06	53.74	54.24	0.46156	41.58	45.94
T89	SRVR_8	86.87	90.39	67.14	60.80	0.53336	47.40	45.55
T90	RUN_1	88.73	95.15	52.86	58.73	0.52736	51.14	39.02
T90	RUN_2	88.70	94.97	53.63	59.01	0.52890	51.65	39.14
T90	RUN_3	88.32	93.93	56.92	59.64	0.52914	65.24	39.29
T90	RUN_4	88.93	96.03	49.23	57.44	0.52237	49.26	70.68
T90	RUN_5	88.60	95.05	52.53	58.29	0.52204	50.83	39.27
T92	RUN_1	86.22	90.77	60.77	57.22	0.49155	50.99	42.40
T100	RUN_1	88.77	96.82	43.74	54.15	0.50005	61.62	42.57
T100	RUN_2	88.27	93.89	56.81	59.49	0.52732	61.86	39.05
T100	RUN_3	81.13	82.69	72.42	53.80	0.45256	60.25	41.60
T100	RUN_4	81.85	82.85	76.26	56.04	0.48270	63.75	38.41
T104	RUN_1	80.12	80.69	76.92	53.99	0.45999	53.67	48.21
T104	RUN_2	80.07	80.47	77.80	54.21	0.46370	53.67	48.21
T104	RUN_3	64.93	59.86	93.3049	44.66	0.38161	53.67	48.21
T104	RUN_4	69.78	66.25	89.56	47.34	0.40530	53.67	48.21
T104	RUN_5	86.27	98.47	18.02	28.47	0.30064	53.67	142.95

Evaluation results based on the unrefined Gold Standard, in terms of Accuracy, MCC Score and AUC iP/R. The highest score for each evaluation column is show in bold typeface. Run/Srvr (RUN = offline run/SRVR = online run via the BCMS), MCC (Matthew's Correlation Coefficient), AUC iP/R (Area under the interpolated Precion/Recall curve). Time_half is the fraction of time needed to classify half of the positive abstracts using the output of that run when compared to unranked results. Note that some runs submitted by mistake the opposite ranking as requested for the negative records, which explains higher classification time (e.g. Team 104, RUN 5 with Time_half of 142.95), inverting in these cases the order of negative articles resulted in comparable time savings to the other systems.

Back to article page

ISSN: 1471-2105

Contact us

General enquiries: journalsubmissions@springernature.com

BMC Bioinformatics

Contact us