A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools

Table 15 Precision/Recall/F1-score results for gene mention over CRAFT development set: LingPipe with model retrained from the CRAFT public release data set

	LingPipe CRAFT star-STAR			LingPipe CRAFT gene-GENE			LingPipe genestar-GENESTAR			LingPipe CRAFT poly-POLY			LingPipe polystar-POLYSTAR
	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1	Prec	Recall	F1
strict	0.60	0.64	0.62	0.50	0.73	0.59	0.49	0.75	0.59	0.21	0.34	0.26	0.23	0.34	0.27
subspan	0.62	0.67	0.64	0.52	0.77	0.62	0.52	0.79	0.62	0.21	0.34	0.26	0.23	0.35	0.28
shared	0.62	0.67	0.64	0.52	0.77	0.62	0.52	0.79	0.62	0.21	0.34	0.26	0.23	0.35	0.28
overlap	0.62	0.67	0.64	0.52	0.77	0.62	0.52	0.79	0.62	0.21	0.34	0.26	0.23	0.35	0.28

ISSN: 1471-2105