Skip to main content

Table 2 Agreement rates for the three subcorpora. The chief annotator resolved just the cases where the first two annotators disagreed, cases of agreement were accepted without further checking. The numbers denote agreement between the two student annotators (first one), and the agreements between each student and the chief annotator (second and third numbers).

From: The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes

type   clinical records abstracts full articles
NEGATION     
  keyword 90.70/94.56/95.81 91.46/91.71/98.05 79.42/86.77/91.71
  left scope 86.27/86.86/97.95 97.78/97.90/100 83.44/82.42/95.87
  right scope 88.88/91.26/97.39 94.56/95.17/99.42 84.36/88.19/95.09
  full scope 76.29/79.32/95.35 92.46/93.07/99.42 70.86/73.35/91.21
SPECULATION     
  keyword 84.01/89.86/92.37 79.12/83.92/92.05 77.60/81.49/90.81
  left scope 89.36/88.90/97.60 87.52/88.37/97.58 75.49/80.13/92.15
  right scope 91.28/92.64/97.90 87.13/89.92/96.16 82.40/83.28/96.97
  full scope 81.90/82.88/95.54 76.72/80.07/94.04 62.50/66.72/89.67