Skip to main content

Table 5 Inter-annotator agreement.

From: An evaluation of GO annotation retrieval for BioCreAtIvE and GOA

GO terms Curator 1 +2 Curator 1+3 Curator 2+ 3 Average
Exact 47 35 35 39
Same Lineage 15 20 19 18
New Lineage 56 39 35 43
Correct 107 91 85 94
Incorrect 11 3 4 18
TOTAL 118 94 89 100
Precision 0.91 0.96 0.96 0.94
Recall 0.70 0.72 0.73 0.72
F-measure 0.79 0.82 0.83 0.82
  1. Where precision is the fraction of manual GO term annotations that are correct (number of correct annotations / (number of correct annotations + number of incorrect annotations). Recall is defined as the fraction of correct GO term annotations that were successfully retrieved during manual annotation (number of correct annotations / number of correct annotations + (number of annotations from new lineage - number of incorrect annotations). New lineage annotations minus incorrect annotations represent total number of the GO terms that the curators should have correctly retrieved from the paper. F-measure = (balanced precision and recall) = 2 × P × R/(P+R).