Skip to main content

Table 7 Comparison of predicted and actual Level 1 category assignments on independent dataset.

From: Cost sensitive hierarchical document classification to triage PubMed abstracts for manual curation

Classifier

Human Expert

 

Allergy

Autoimmunity

Infectious Disease

Transplantation

Cancer

HIV

Other

 

Allergy

11

1

0

0

0

0

0

 

Autoimmunity

0

58

0

0

0

0

1

 

Infectious

0

1

100

0

1

0

2

 

Disease

       
 

Transplantation

0

1

0

8

0

0

0

 

Cancer

0

1

0

1

41

0

2

 

HIV

0

0

2

0

0

33

0

 

Other

0

1

1

0

1

0

20

 

Total

0

5

3

1

2

0

5

 

Incorrect:16

       
 

Uncuratable

17

62

120

26

68

45

17

  1. Columns represent predictions by the classifier and rows represent the Level 1 category assigned by a human expert. For example, one reference predicted as Transplant was actually Cancer. The Total Incorrect row represents the total number of references that were predicted into Level 1 categories by the classifier that differed from the decision of the human expert. Of the 642 abstracts predicted to be curatable, 355 abstracts were overruled as uncuratable which can be seen in the Uncuratable row. Of the 287 curatable abstracts, 94.4% were assigned to the correct Level 1 category.