Skip to main content

Table 11 Inference rule validation results

From: CoMAGC: a corpus with multi-faceted annotations of gene-cancer relations

Applied inference rules Full match One match No match Total Agreement rate
1 4 0 1 5 0.8
2 3 0 0 3 1
3 1 0 1 2 0.5
4 2 0 0 2 1
5 0 0 0 0 n/a
6 0 0 0 0 n/a
7 28 1 1 30 0.97
8 6 2 0 8 1
9 0 0 0 0 n/a
10 0 0 0 0 n/a
No rule applied 40 0 2 42 0.95
Total 82 3 5 92 0.95 (micro), 0.89 (macro)
  1. A gene class inferred by an inference rule is compared with two gene classes annotated by each of the two annotators. When the three gene classes are all the same, we refer to the case as a full match. When the inferred gene class agrees with only one of the two annotated gene classes, we refer to the case as a one match. When the inferred gene class is different from both of the annotated gene classes, it is a no match. Agreement rate is calculated as the ratio of full match and one match among the total cases.