Skip to main content

Table 5 Impact of co-reference strategy on co-reference resolution (only on documents with co-reference annotations)

From: The contribution of co-reference resolution to supervised relation detection between bacteria and biotopes entities

 

With gold mentions

With predicted mentions

 

Training

Development

Development

  

R

P

F

R

P

F

R

P

F

sim

bcube

0.927

0.756

0.833

0.980

0.640

0.775

0.824

0.667

0.737

 

ceafe

0.835

0.757

0.794

0.937

0.630

0.753

0.807

0.605

0.692

 

ceafm

0.928

0.799

0.859

0.974

0.676

0.798

0.872

0.705

0.780

 

muc

0.950

0.808

0.873

0.987

0.691

0.813

0.878

0.725

0.794

ana

bcube

0.006

0.167

0.012

0.012

0.167

0.022

0.009

0.124

0.017

 

ceafe

0.038

0.151

0.060

0.070

0.170

0.099

0.057

0.147

0.082

 

ceafm

0.029

0.286

0.052

0.046

0.300

0.080

0.041

0.258

0.071

 

muc

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

all

bcube

0.927

0.695

0.794

0.970

0.574

0.721

0.825

0.592

0.689

 

ceafe

0.823

0.708

0.761

0.910

0.582

0.710

0.791

0.541

0.643

 

ceafm

0.928

0.758

0.834

0.969

0.634

0.767

0.872

0.649

0.744

 

muc

0.950

0.766

0.848

0.981

0.646

0.779

0.878

0.668

0.759

 

With self-predicted mentions

 

rec

bcube

0.187

0.091

0.122

0.206

0.086

0.121

 
 

ceafe

0.230

0.053

0.086

0.379

0.065

0.111

 
 

ceafm

0.326

0.125

0.181

0.374

0.105

0.163

 
 

muc

0.319

0.147

0.201

0.308

0.102

0.153

 
  1. Impact of co-reference strategy on co-reference resolution (only on documents with co-reference annotations). Because no gold annotation has been removed, recall is unchanged with respect to Table 4. sim = Similarity-based co-reference resolution, ana = anaphora-based co-reference resolution, all = sim ∪ ana. bcube = B3, ceafe = CEAF-entities, ceafm = CEAF-mentions, muc = MUC. Boldface highlights F-measure of best strategy. For reference, we also show the results obtained with the off-the-shelf version of Reconcile (rec)