Table 1 Evaluation scores of event extraction services.
From: U-Compare bio-event meta-service: compatible BioNLP event extraction services
Participant | JULIE Lab JReX | UTurku | EventMine | BExtract | VIBGhent | TheBeast | UMich | Moara | CCP- BTMG | |
|---|---|---|---|---|---|---|---|---|---|---|
Rank in F1 score | # | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
Total | F1 | 51.09 | 49.91 | 48.20 | 44.48 | 42.44 | 37.19 | 36.34 | 29.50 | 22.03 |
PR | 57.69 | 56.32 | 64.00 | 61.56 | 59.05 | 48.15 | 35.57 | 31.99 | 70.03 | |
RC | 45.85 | 44.81 | 38.65 | 34.82 | 33.12 | 30.30 | 37.15 | 27.31 | 13.07 | |
Localization | F1 | 61.60 | 55.85 | 63.20 | 51.45 | 51.79 | 48.98 | 53.47 | 44.19 | 17.80 |
Binding | F1 | 49.24 | 45.43 | 39.86 | 26.97 | 34.42 | 34.50 | 31.75 | 28.36 | 20.92 |
Gene expression | F1 | 72.48 | 71.67 | 72.63 | 65.14 | 69.57 | 59.28 | 66.00 | 58.79 | 51.07 |
Transcription | F1 | 42.99 | 50.21 | 50.00 | 24.71 | 57.14 | 17.48 | 30.06 | 26.40 | 22.93 |
Protein catabolism | F1 | 80.00 | 50.00 | 60.87 | 60.00 | 68.97 | 72.00 | 58.06 | 50.00 | 40.00 |
Phosphorylation | F1 | 81.99 | 79.70 | 81.29 | 80.69 | 76.23 | 72.79 | 77.15 | 52.88 | 33.33 |
Regulation | F1 | 31.20 | 33.97 | 28.77 | 32.21 | 19.39 | 29.96 | 14.29 | 10.83 | 5.79 |
Positive Regulation | F1 | 40.39 | 38.66 | 28.25 | 35.83 | 23.34 | 29.57 | 21.50 | 14.68 | 6.69 |
Negative Regulation | F1 | 38.47 | 36.28 | 32.62 | 33.27 | 26.67 | 27.32 | 26.61 | 13.16 | 4.01 |
BioNLP '09 ST Total Evaluation | F1 | 46.66 | 51.95 | 36.88 | 44.62 | 40.54 | 44.35 | 19.28 | 24.15 | 22.66 |