Table 3 Statistics and IAA scores for entities, relation labels, and triggers. The IAA scores for entities and relation labels were calculated using a simple percent agreement. The IAA score for relation labels was calculated using Cohen’s kappa statistic
From: A corpus for plant-chemical relationships in the biomedical domain
Phases | Entity | # of entities | # of agreements for entities (IAA, Simple) | Relation | # of class labels | # of agreements for relation labels (IAA, Kappa) | Trigger | # of triggers | # of agreements for triggers (IAA, Simple) |
|---|---|---|---|---|---|---|---|---|---|
Phase 1 | Plants | 642 | 640 (99.7 %) | Plant–chemical | 642 | 570 (77.6 %) | Weak | 284 | 275 (96.8 %) |
Chemicals | 642 | 636 (99.1 %) | Strong | 284 | 271 (95.4 %) | ||||
Total | 1,284 | 1276 (99.4 %) | Total | 568 | 546 (96.1 %) | ||||
Phase 2 | Plants | 401 | 401 (100.0 %) | Plant–chemical | 401 | 369 (82.8 %) | Weak | 245 | 234 (95.5 %) |
Chemicals | 401 | 400 (99.8 %) | Strong | 245 | 223 (91.0 %) | ||||
Total | 802 | 801 (99.9 %) | Total | 490 | 457 (93.3 %) | ||||
Overall | 2,086 | 2,077 (99.6 %) | 1,043 | 939 (79.8 %) | 1,058 | 1,003 (94.8 %) |