Overview of the ID, EPI and REL tasks of BioNLP Shared Task 2011

BMC Bioinformatics

Table 11 EPI task primary evaluation results: F-scores by event type

	UTurku	FAUST	MSR- NLP	UMass	Stanford	CCP- BTMG	ConcordU	Size
HYDROXYLATION	42.25	10.26	10.20	12.80	9.45	12.84	6.32	139
DEHYDROXYLATION	-	-	-	-	-	-	-	1
PHOSPHORYLATION	67.12	51.61	50.00	49.18	40.98	47.06	44.44	130
DEPHOSPHORYLATION	0.00	0.00	0.00	0.00	0.00	50.00	0.00	3
UBIQUITINATION	75.34	72.95	67.88	72.94	67.44	70.87	69.97	340
DEUBIQUITINATION	54.55	40.00	0.00	31.58	0.00	42.11	14.29	17
DNA METHYLATION	60.21	31.21	34.54	23.82	31.02	15.65	8.22	416
DNA DEMETHYLATION	26.67	0.00	0.00	0.00	0.00	0.00	0.00	21
Simple event total	63 . 05	45.17	44.97	43.01	40.96	40.62	37.84	1067
GLYCOSYLATION	49.43	41.10	38.87	40.00	37.22	25.62	25.94	347
DEGLYCOSYLATION	40.00	35.29	0.00	38.10	30.00	35.29	26.67	27
ACETYLATION	57.22	40.00	41.42	40.25	35.12	37.50	38.19	337
DEACETYLATION	54.90	28.00	31.82	29.17	21.74	24.56	27.27	50
METHYLATION	57.67	24.82	19.57	23.67	18.54	16.99	15.50	374
DEMETHYLATION	35.71	0.00	0.00	0.00	0.00	0.00	0.00	13
Non-simple event total	*54.36*	33.86	31.85	33.07	29.28	25.06	25.10	1148
CATALYSIS	7.06	6.58	7.75	5.00	2.84	7.58	1.74	238
Subtotal	*55.02*	36.93	36.17	35.30	32.85	30.58	28.92	2453
NEGATION	18.60	0.00	0.00	0.00	0.00	0.00	26.51	149
SPECULATION	37.65	0.00	0.00	0.00	0.00	0.00	6.82	103
Modification total	*28.07*	0.00	0.00	0.00	0.00	0.00	16.37	252
Total	*53.33*	35.03	34.27	33.52	31.22	28.97	27.88	2705
Addition total	*59.33*	40.27	39.05	38.65	36.03	32.75	31.50	2038
Removal total	*44.29*	22.41	15.73	22.76	14.41	23.53	17.48	132

The Size column gives the number of annotations of each type in the given data (training+development). Best result for each type shown in bold. For DEHYDROXYLATION, no examples were present in the test data and none were predicted by any participant.

ISSN: 1471-2105