Skip to main content

Table 5 Full end-to-end system evaluation on the core set comparing F1 score

From: Concept recognition as a machine translation problem

Ontology

CRF

BiLSTM

BiLSTM-CRF

Char-Embeddings

BiLSTM-ELMo

BioBERT

UZH@CRAFT-ST

ChEBI

0.7882

0.6394

0.5027

0.5942

0.0550

0.7885*

0.7700

CL

0.6779

0.5134

0.3859

0.5611

0.0526

0.6994*

0.6657

GO_BP

0.7505

0.5137

0.3642

0.6182

0.0720

0.7405

0.8037*

GO_CC

0.7225

0.1689

0.3049

0.3244

0.0506

0.7762*

0.7645

GO_MF

0.9778

0.9770

0.9778

0.8906

0.3704

0.9783

0.9838*

MOP

0.8129

0.7721

0.7158

0.5985

0.0930

0.8742*

0.8705

NCBITaxon

0.9026

0.7736

0.8391

0.8518

0.0948

0.8910

0.9694*

PR

0.4040

0.3136

0.2827

0.2732

0.0516

0.5295

0.8026*

SO

0.8987

0.4106

0.4096

0.7815

0.0813

0.9054*

0.9027

UBERON

0.7474

0.6812

0.5029

0.6901

0.0793

0.7670*

0.7488

  1. For all results shown here, the span detection algorithm is listed, and the concept normalization algorithm is OpenNMT. UZH@CRAFT-ST is the best performing system from Furrer et al. [4] in the CRAFT-ST, shown as a comparison to our methods. The best-performing algorithm is bolded with an asterisk*