Skip to main content

Table 5 Best performance for each ontology-system pair

From: Large-scale biomedical concept recognition: an evaluation of current automatic annotators and their parameters

Cell Type Ontology (CL)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.32 0.76 0.20 1169 379 4591
MetaMap 0.69 0.61 0.80 4590 3010 1170
ConceptMapper 0.83 0.88 0.78 4478 592 1282
Gene Ontology - Cellular Component (GO_CC)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.40 0.75 0.27 2287 779 6067
MetaMap 0.70 0.67 0.73 6111 2969 2341
ConceptMapper 0.77 0.92 0.66 5532 452 2822
Gene Ontology - Molecular Function (GO_MF)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.08 0.47 0.04 173 195 4007
MetaMap 0.09 0.09 0.09 393 3846 3787
ConceptMapper 0.14 0.44 0.08 337 425 3834
Gene Ontology - Biological Process (GO_BP)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.25 0.70 0.15 2592 1120 14321
MetaMap 0.42 0.53 0.34 5802 4994 11111
ConceptMapper 0.36 0.46 0.29 4909 5710 12004
Sequence Ontology (SO)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.44 0.63 0.33 7056 4094 14231
MetaMap 0.50 0.47 0.54 11402 12634 9885
ConceptMapper 0.56 0.56 0.57 12059 9560 9228
ChEBI
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.56 0.7 0.46 3782 1595 4355
MetaMap 0.42 0.36 0.50 4424 8689 3717
ConceptMapper 0.56 0.55 0.56 4583 3687 3554
NCBI Taxonomy
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.04 0.16 0.02 157 807 7292
MetaMap 0.45 0.31 0.88 6587 14954 862
ConceptMapper 0.69 0.61 0.79 5857 3793 1592
Protein Ontology (PRO)
System F-measure Precision Recall # TP # FP # FN
NCBO Annotator 0.50 0.49 0.51 7958 8288 7636
MetaMap 0.36 0.39 0.34 5255 8307 10339
ConceptMapper 0.57 0.57 0.57 8843 6620 6751
  1. Maximum F-measure for each system on each ontology. Bolded systems produced the highest F-measure.
\