Skip to main content

Table 7 Comparison of predicted and actual Level 1 category assignments on independent dataset.

From: Cost sensitive hierarchical document classification to triage PubMed abstracts for manual curation

Classifier
Human Expert   Allergy Autoimmunity Infectious Disease Transplantation Cancer HIV Other
  Allergy 11 1 0 0 0 0 0
  Autoimmunity 0 58 0 0 0 0 1
  Infectious 0 1 100 0 1 0 2
  Disease        
  Transplantation 0 1 0 8 0 0 0
  Cancer 0 1 0 1 41 0 2
  HIV 0 0 2 0 0 33 0
  Other 0 1 1 0 1 0 20
  Total 0 5 3 1 2 0 5
  Incorrect:16        
  Uncuratable 17 62 120 26 68 45 17
  1. Columns represent predictions by the classifier and rows represent the Level 1 category assigned by a human expert. For example, one reference predicted as Transplant was actually Cancer. The Total Incorrect row represents the total number of references that were predicted into Level 1 categories by the classifier that differed from the decision of the human expert. Of the 642 abstracts predicted to be curatable, 355 abstracts were overruled as uncuratable which can be seen in the Uncuratable row. Of the 287 curatable abstracts, 94.4% were assigned to the correct Level 1 category.