Skip to main content

Table 3 Estimation of consistency in cases of initial agreement. We collected 200-200 randomly chosen examples from each type of corpus text to assess the level of consistency in cases when the two students provided identical annotation for the sentence (identical means here that all cues and scope boundaries were exactly the same) and they were compared to the annotation provided by the chief annotator. The agreement rates are given here.

From: The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes

NEGATION  
keyword: 98.65%
left scope: 97.27%
right scope: 98.64%
full scope: 95.91%
SPECULATION  
keyword: 99.63%
left scope: 99.25%
right scope: 99.63%
full scope: 98.88%