Skip to main content

Table 3 Inter-annotator agreement rates on the challenge train / test sets, in micro-averaged Fβ=1.

From: Automatic construction of rule-based ICD-9-CM coding systems

  A1 A2 A3 GS BasicRB Hybrid
A1 73.97/75.79 65.61/67.28 83.67/84.62 75.11/75.56 78.02/79.19
A2 73.97/75.79 70.89/72.68 88.48/89.63 78.52/78.43 83.40/82.84
A3 65.61/67.28 70.89/72.68 82.01/82.64 75.48/74.29 80.11/78.97
GS 83.67/84.62 88.48/89.63 82.01/82.64 85.57/84.85 90.26/88.93
BasicRB 75.11/75.56 78.52/78.43 75.48/74.29 85.57/84.85
Hybrid 78.02/79.19 83.40/82.84 80.11/78.97 90.26/88.93
  1. A1, A2 and A3 refers to Annotators 1, 2 and 3 respectively. GS stands for gold standard labeling, while BasicRB represents our basic rule-based system that models inter-label dependencies. Hybrid denotes our hybrid rule-based + MaxEnt statistical model.