Table 14 Inter-annotator agreement results

CHQA-email

CHQA-web

Unadjudicated

Avg. # of questions shared

81.9

36-120

540

540-540

Named entity

-

-

0.72

0.66-0.78

Question trigger

0.37

0.18-0.52

0.60

0.46-0.66

Question type

0.58

0.39-0.69

0.74

0.65-0.81

Full frame w/ trigger

0.22

0.08-0.34

0.41

0.33-0.48

Core frame w/ trigger

0.27

0.11-0.41

0.48

0.38-0.56

Full frame w/ type

0.32

0.15-0.46

0.54

0.47-0.58

Core frame w/ type

0.41

0.22-0.56

0.64

0.55-0.70

Question topic

0.71

0.61-0.87

-

-