|
Team
|
Run/Srvr
|
Precision
|
Recall
|
F-Score
|
MCC
|
AUC iP/R
|
|---|
|
Base _Top4
|
RUN _1
|
28.60
|
48.20
|
35.90
|
-
|
0.15924
|
|
Base _ Regex
|
RUN _1
|
32.45
|
22.34
|
26.46
|
-
|
-
|
|
T65
|
RUN_1
|
8.77
|
84.82
|
15.89
|
0.23552
|
0.27588
|
|
T65
|
RUN_2
|
2.45
|
100.00
|
4.78
|
0.06259
|
0.24484
|
|
T65
|
RUN_3
|
9.42
|
81.78
|
16.89
|
0.24172
|
0.27727
|
|
T65
|
RUN_4
|
33.48
|
42.32
|
37.39
|
0.36166
|
0.14169
|
|
T65
|
RUN_5
|
2.44
|
100.00
|
4.76
|
0.06193
|
0.29016
|
|
T69
|
RUN_1
|
52.07
|
55.03
|
53.51
|
0.52519
|
0.34302
|
|
T69
|
RUN_2
|
54.34
|
53.51
|
53.92
|
0.52958
|
0.33824
|
|
T69
|
RUN_3
|
57.36
|
50.29
|
53.59
|
0.52796
|
0.32539
|
|
T69
|
RUN_4
|
59.25
|
48.01
|
53.04
|
0.52456
|
0.31711
|
|
T69
|
RUN_5
|
61.33
|
43.64
|
51.00
|
0.50896
|
0.29373
|
|
T70
|
RUN_1
|
48.61
|
23.15
|
31.36
|
0.32617
|
0.12949
|
|
T70
|
RUN_2
|
70.00
|
11.95
|
20.42
|
0.28419
|
0.08731
|
|
T70
|
RUN_3
|
80.65
|
4.74
|
8.96
|
0.19270
|
0.03826
|
|
T70
|
RUN_4
|
31.22
|
36.43
|
33.63
|
0.32216
|
0.15688
|
|
T70
|
RUN_5
|
32.69
|
15.94
|
21.43
|
0.21717
|
0.05734
|
|
T81
|
RUN_1
|
4.54
|
66.03
|
8.50
|
0.11406
|
0.07716
|
|
T81
|
RUN_2
|
8.71
|
42.13
|
14.43
|
0.15560
|
0.06239
|
|
T81
|
RUN_3
|
13.51
|
28.46
|
18.33
|
0.17168
|
0.04657
|
|
T81
|
RUN_4
|
13.20
|
27.70
|
17.88
|
0.16667
|
0.05601
|
|
T81
|
RUN_5
|
21.35
|
22.20
|
21.77
|
0.20090
|
0.05283
|
|
T88
|
RUN_1
|
28.44
|
45.16
|
34.90
|
0.34146
|
0.20244
|
|
T88
|
RUN_2
|
28.17
|
45.92
|
34.92
|
0.34263
|
0.20069
|
|
T89
|
RUN_1
|
52.52
|
49.53
|
50.98
|
0.49997
|
0.28202
|
|
T89
|
RUN_2
|
52.02
|
48.96
|
50.44
|
0.49451
|
0.28589
|
|
T89
|
RUN_3
|
50.78
|
49.34
|
50.05
|
0.49016
|
0.27238
|
|
T89
|
RUN_4
|
52.50
|
49.91
|
51.17
|
0.50181
|
0.29220
|
|
T89
|
RUN_5
|
52.58
|
52.18
|
52.38
|
0.51382
|
0.29980
|
|
T89
|
SRVR_4
|
52.71
|
51.61
|
52.16
|
0.51163
|
0.29926
|
|
T89
|
SRVR_5
|
52.28
|
50.10
|
51.16
|
0.50168
|
0.30046
|
|
T89
|
SRVR_6
|
52.28
|
52.18
|
52.23
|
0.51226
|
0.30049
|
|
T89
|
SRVR_7
|
49.55
|
52.56
|
51.01
|
0.49972
|
0.29303
|
|
T89
|
SRVR_8
|
51.76
|
50.29
|
51.01
|
0.49999
|
0.29766
|
|
T90
|
RUN_1
|
53.33
|
47.06
|
50.00
|
0.49113
|
0.26805
|
|
T90
|
RUN_2
|
52.56
|
48.77
|
50.59
|
0.49625
|
0.28386
|
|
T90
|
RUN_3
|
52.30
|
58.25
|
0.5512
|
0.54201
|
0.35423
|
|
T90
|
RUN_4
|
61.09
|
38.14
|
46.96
|
0.47436
|
0.25209
|
|
T90
|
RUN_5
|
64.24
|
35.10
|
45.40
|
0.46707
|
0.24270
|
|
T100
|
RUN_1
|
44.59
|
51.61
|
47.85
|
0.46794
|
0.26055
|
|
T100
|
RUN_2
|
39.86
|
54.84
|
46.17
|
0.45448
|
0.26982
|
|
T100
|
RUN_3
|
35.29
|
44.59
|
39.40
|
0.38240
|
0.15734
|
|
T100
|
RUN_4
|
35.34
|
44.59
|
39.43
|
0.38271
|
0.15758
|
|
T100
|
RUN_5
|
54.86
|
18.22
|
27.35
|
0.30847
|
0.11109
|
- Micro-averaged results when evaluating all documents (i.e., measuring the overall performance of each run on the whole document set). The highest score for each evaluation column is show in bold typeface, the lowest in italics. Run/Srvr: RUN = offline run, SRVR = online server run via BCMS; MCC: Matthew's Correlation Coefficient; AUC iP/R: Area under the interpolated precision/recall curve (micro- averaged by iterating over the precision/recall values of the highest ranked annotation of all articles, then all second ranked annotations, etc.). Base_Top4: baseline system that assigns the four most frequent classes ordered by their frequencies in the training/development set. Base_Regex: simple matching strategy based on regular expressions.
