Skip to main content

Table 16 Annotation confidence estimation in CHQA-web

From: Semantic annotation of consumer health questions

Method

Precision

Recall

F1

AGREE

0.99

0.66

0.79

IAA_RANK (Full frame w/ trigger)

0.78

0.80

0.79

IAA_RANK (Core frame w/ type)

0.85

0.84

0.84

MACE

0.82

0.86

0.84

MACE (Reliability Rank)

0.83

0.83

0.83

P&C

0.87

0.79

0.83

P&C (Reliability Rank)

0.81

0.82

0.82