A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools

Table 14 Precision/Recall/F1-score results for gene mention over CRAFT development set: ABNER with model retrained from the CRAFT public release data set

	ABNER CRAFT star-STAR			ABNER CRAFT gene-GENE			ABNER CRAFT genestar-GENESTAR			ABNER CRAFT poly-POLY			ABNER CRAFT polystar-POLYSTAR
	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1
overlap	0.72	0.40	0.51	0.86	0.33	0.48	0.78	0.40	0.53	0.56	0.04	0.07	0.64	0.06	0.11
shared	0.72	0.40	0.51	0.86	0.33	0.48	0.78	0.40	0.53	0.56	0.04	0.07	0.64	0.06	0.11
subspan	0.72	0.40	0.51	0.86	0.33	0.48	0.78	0.40	0.53	0.56	0.04	0.07	0.64	0.06	0.11
strict	0.63	0.35	0.45	0.83	0.31	0.46	0.73	0.38	0.50	0.50	0.03	0.06	0.63	0.06	0.11

ISSN: 1471-2105