Skip to main content

Table 3 Inter-annotator agreement rates on the challenge train / test sets, in micro-averaged Fβ=1.

From: Automatic construction of rule-based ICD-9-CM coding systems

 

A1

A2

A3

GS

BasicRB

Hybrid

A1

73.97/75.79

65.61/67.28

83.67/84.62

75.11/75.56

78.02/79.19

A2

73.97/75.79

70.89/72.68

88.48/89.63

78.52/78.43

83.40/82.84

A3

65.61/67.28

70.89/72.68

82.01/82.64

75.48/74.29

80.11/78.97

GS

83.67/84.62

88.48/89.63

82.01/82.64

85.57/84.85

90.26/88.93

BasicRB

75.11/75.56

78.52/78.43

75.48/74.29

85.57/84.85

Hybrid

78.02/79.19

83.40/82.84

80.11/78.97

90.26/88.93

  1. A1, A2 and A3 refers to Annotators 1, 2 and 3 respectively. GS stands for gold standard labeling, while BasicRB represents our basic rule-based system that models inter-label dependencies. Hybrid denotes our hybrid rule-based + MaxEnt statistical model.