Skip to main content

Table 3 Evaluation of submitted gene sets.

From: nGASP – the nematode genome annotation assessment project

Gene set Category Base Exon Isoform Gene
   Sn Sp Sn Sp Sn Sp Sn Sp
Agene 1 93.8 83.4 68.9 61.1 9.8 13.1 12.0 14.1
AUGUSTUS v1 1 97.0 89.0 86.1 72.6 50.1 28.7 61.1 38.4
AUGUSTUS v2 1 96.8 89.3 84.8 74.3 49.3 31.9 60.5 32.7
CRAIG 1 95.6 90.9 80.2 78.2 35.7 36.3 43.8 37.8
EUGENE 1 94.0 89.5 80.3 73.0 49.1 28.8 60.2 30.2
ExonHunter 1 95.4 86.0 72.6 62.5 15.5 18.6 19.1 19.2
Fgenesh 1 98.2 87.1 86.4 73.6 47.1 34.6 57.8 35.4
GeneID 1 93.9 88.2 77.0 68.6 36.2 22.8 44.4 25.1
GeneMark.hmm 1 98.3 83.1 83.2 65.6 37.7 24.0 46.3 24.5
GlimmerHMM 1 97.6 87.6 84.4 71.4 47.3 29.3 58.0 30.6
MGENE v1 1 97.2 91.5 84.6 78.6 44.6 40.9 54.8 42.3
MGENE v2 1 96.9 91.6 84.2 78.7 44.0 40.9 54.0 42.4
MGENE v3 1 96.9 91.6 84.2 78.6 43.5 40.5 53.4 44.8
SNAP 1 94.0 84.5 74.6 61.3 32.6 18.6 40.0 19.1
EUGENE 2 96.2 87.5 82.8 72.8 50.3 30.2 61.7 31.4
MGENE v1 2 97.7 90.9 85.8 78.4 51.6 41.2 63.3 42.5
MGENE v2 2 97.7 90.9 85.8 78.3 51.2 40.9 62.7 43.8
N-SCAN 2 97.4 88.1 83.5 70.8 39.2 27.7 48.1 28.4
SGP2 2 93.5 90.0 77.3 70.3 36.4 24.9 44.6 27.1
AUGUSTUS v1 3 99.0 90.5 92.5 80.2 68.3 47.1 80.1 51.8
EUGENE v1 3 97.3 85.3 88.5 72.2 55.7 33.7 68.4 34.2
EUGENE v2 3 98.5 85.1 92.1 70.3 60.8 31.5 68.8 36.1
ExonHunter v1 3 97.6 87.3 83.9 69.3 38.5 31.9 47.3 32.5
ExonHunter v2 3 93.7 92.0 81.2 76.9 37.2 39.7 45.6 40.5
Fgenesh++ 3 97.6 89.7 90.4 80.9 65.5 53.4 78.3 54.2
Gramene v11 3 98.2 95.4 88.5 71.8 41.7 19.6 48.7 37.2
Gramene v21 3 98.6 94.8 88.3 67.8 38.7 16.3 46.0 39.0
MAKER (using SNAP) v1 3 92.9 88.5 80.7 66.3 41.3 19.6 50.7 47.6
MAKER (using SNAP) v2 3 92.6 91.1 80.5 69.5 40.8 23.2 50.1 28.0
MGENE v1 3 98.7 91.9 91.0 80.7 57.7 48.0 70.8 48.9
MGENE v2 3 98.9 87.9 91.9 75.9 62.6 38.7 76.9 39.5
MGENE v3 3 98.7 91.9 91.0 80.6 57.7 48.0 70.6 51.1
EUGENE v1 4 98.5 85.6 90.5 75.1 60.4 39.3 75.9 39.5
EUGENE v2 4 99.4 85.4 94.3 72.6 63.9 35.9 74.7 42.0
EUGENE v3 4 98.6 85.6 90.6 74.2 63.3 36.9 79.5 37.4
EUGENE v4 4 99.2 85.3 94.0 71.8 67.1 33.9 77.9 39.8
Evigan 4 99.3 89.6 91.1 82.3 64.2 52.4 80.7 52.7
Fgenesh++C 4 98.7 89.7 91.1 82.7 66.1 56.3 80.3 57.1
GeneID v1 4 99.3 91.5 93.0 83.8 63.9 53.3 78.3 57.7
GeneID v2 4 99.0 92.0 90.7 85.0 61.7 55.5 77.5 57.1
GENOMIX v1 4 97.1 88.6 86.2 77.4 52.4 39.0 65.9 42.2
GENOMIX v2 4 98.1 90.4 89.7 83.5 60.4 53.3 75.9 56.1
GESECA 4 98.8 82.8 87.6 66.8 45.1 25.9 52.6 27.4
GLEAN 4 98.9 87.3 88.3 75.4 51.4 37.0 64.7 37.6
Gramene1 4 97.5 80.9 82.7 48.7 22.4 6.1 27.3 30.3
JIGSAW v1 4 98.9 93.2 90.5 87.4 63.6 60.2 79.9 61.0
JIGSAW v2 4 98.9 91.7 89.9 83.0 62.0 51.1 77.9 52.0
  1. 1After the evaluations were complete, the GRAMENE developers discovered an error in their pipeline which incorrectly moved the end of the coding region by 3 bp in a significant fraction of their gene predictions, and this negatively affected the overall performance of GRAMENE.
  2. The accuracy of the submitted gene sets evaluated using the reference gene sets ref1 and ref2. The sensitivity (Sn) results are given for reference set ref1, and the specificity (Sp) results are given for set ref2. The gene sets are divided according to nGASP category, where category 1 is ab initio gene-finders, 2 is gene-finders that used multi-genome alignments, 3 is gene-finders that used alignments of ESTs, mRNAS and proteins, and 4 is combiners.