Skip to main content

Table 2 Estimated agreement across various levels of analysis

From: Developing a manually annotated clinical document corpus to identify phenotypic information for inflammatory bowel disease

Unit of Analysis Kappa (95% CI) F-measure
Document 0.87 (0.82,0.93)  
Concept   
Signs and Symptoms   0.61 (0.57, 0.86)
Diagnoses   0.83 (0.80, 0.87)
Procedures   0.63 (0.56, 0.68)
Medications   0.82 (0.76, 0.86)
all classes   0.72 (0.70, 0.74)
Attribute   
Granularity 0.34 (0.28,0.41)  
Ambiguity 0.08 (0.04,0.13)  
Relevance **  
Negation 0.67 (0.60,0.74)  
Temporality 0.67 (0.61,0.73)  
Experiencer 1  
Reason for Service 0.06 (0.03,0.09)  
Note Section 0.54 (0.50,0.59)  
  1. ** Kappa for relevance could not be estimated
  2. *** Only 1 concept was annotated as describing an experiencer other than the patient.