Skip to main content

Table 1 Matthews correlation coefficient, perpendicular distance, true positive rate and false positive rate values

From: Performance evaluation of pipelines for mapping, variant calling and interval padding, for the analysis of NGS germline panels

 

Exonic SNVs

Exonic indels

Intronic SNVs

MCC

d

TPR

FPR

MCC

d

TPR

FPR

MCC

d

TPR

FPR

Null interval padding

BWA-MEM/GATK-HC

0.832

0.563

0.985

0.189

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

BWA-MEM/GATK-UG

0.956

0.665

0.997

0.057

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

BWA-MEM/SAMtools

0.945

0.663

0.994

0.057

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

Bowtie2/GATK-HC

0.672

0.608

0.860

0.000

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

Bowtie2/GATK-UG

0.300

0.167

0.953

0.717

0.000

0.000

0.000

0.000

− 0.332

− 0.088

0.000

0.125

Bowtie2/SAMtools

0.748

0.524

0.968

0.226

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

Stampy/GATK-HC

0.819

0.550

0.985

0.208

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

Stampy/GATK-UG

0.852

0.556

0.994

0.208

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

Stampy/SAMtools

0.946

0.674

0.991

0.038

1.000

0.707

1.000

0.000

0.000

0.000

0.000

0.000

50 bp interval padding

BWA-MEM/GATK-HC

0.819

0.550

0.985

0.208

1.000

0.707

1.000

0.000

0.828

0.666

0.942

0.000

BWA-MEM/GATK-UG

0.956

0.665

0.997

0.057

0.00

0.000

0.000

0.000

0.701

0.564

0.923

0.125

BWA-MEM/SAMtools

0.946

0.674

0.991

0.038

1.000

0.707

1.000

0.000

0.711

0.626

0.885

0.000

Bowtie2/GATK-HC

0.647

0.582

0.860

0.038

1.000

0.707

1.000

0.000

0.576

0.558

0.789

0.000

Bowtie2/GATK-UG

0.311

0.178

0.950

0.698

0.000

0.000

0.000

0.000

− 0.074

− 0.061

0.789

0.875

Bowtie2/SAMtools

0.767

0.529

0.974

0.226

1.000

0.707

1.000

0.000

0.515

0.483

0.808

0.125

Stampy/GATK-HC

0.819

0.550

0.985

0.208

1.000

0.707

1.000

0.000

0.828

0.666

0.942

0.000

Stampy/GATK-UG

0.852

0.545

0.997

0.226

0.000

0.000

0.000

0.000

0.574

0.462

0.904

0.250

Stampy/SAMtools

0.968

0.690

0.994

0.019

1.000

0.707

1.000

0.000

0.595

0.524

0.865

0.125

100 bp interval padding

BWA-MEM/GATK-HC

0.831

0.563

0.985

0.189

1.000

0.707

1.000

0.000

0.828

0.666

0.942

0.000

BWA-MEM/GATK-UG

0.934

0.650

0.994

0.075

0.000

0.000

0.000

0.000

0.746

0.578

0.942

0.125

BWA-MEM/SAMtools

0.956

0.676

0.994

0.038

1.000

0.707

1.000

0.000

0.711

0.626

0.885

0.000

Bowtie2/GATK-HC

0.647

0.582

0.860

0.038

1.000

0.707

1.000

0.000

0.576

0.558

0.789

0.000

Bowtie2/GATK-UG

0.338

0.194

0.953

0.679

0.000

0.000

0.000

0.000

− 0.059

− 0.048

0.808

0.875

Bowtie2/SAMtools

0.753

0.515

0.974

0.245

1.000

0.707

1.000

0.000

0.540

0.496

0.827

0.125

Stampy/GATK-HC

0.831

0.563

0.985

0.189

1.000

0.707

1.000

0.000

0.828

0.666

0.942

0.000

Stampy/GATK-UG

0.852

0.556

0.994

0.208

0.000

0.000

0.000

0.000

0.574

0.462

0.904

0.250

Stampy/SAMtools

0.945

0.663

0.994

0.057

1.000

0.707

1.000

0.000

0.679

0.612

0.865

0.000

BWA Enrichment

0.956

0.676

0.994

0.038

1.000

0.707

1.000

0.000

− 0.331

− 0.088

0.000

0.1250

 

Intronic indels

All types of variants

 

MCC

d

TPR

FPR

MCC

d

TPR

FPR

Null interval padding

BWA-MEM/GATK-HC

0.000

0.000

0.000

0.000

0.581

0.502

0.848

0.139

BWA-MEM/GATK-UG

0.000

0.000

0.000

0.000

0.653

0.572

0.851

0.042

BWA-MEM/SAMtools

0.000

0.000

0.000

0.000

0.660

0.576

0.856

0.042

Bowtie2/GATK-HC

0.000

0.000

0.000

0.000

0.551

0.524

0.741

0.000

Bowtie2/GATK-UG

0.000

0.000

0.000

0.000

0.233

0.192

0.813

0.542

Bowtie2/SAMtools

0.000

0.000

0.000

0.000

0.540

0.471

0.833

0.167

Stampy/GATK-HC

0.000

0.000

0.000

0.000

0.571

0.492

0.848

0.153

Stampy/GATK-UG

0.000

0.000

0.000

0.000

0.571

0.492

0.848

0.153

Stampy/SAMtools

0.000

0.000

0.000

0.000

0.666

0.584

0.853

0.028

50 bp interval padding

BWA-MEM/GATK-HC

0.000

0.000

1.000

1.000

0.749

0.487

0.980

0.292

BWA-MEM/GATK-UG

0.000

0.000

0.000

0.000

0.898

0.652

0.978

0.056

BWA-MEM/SAMtools

0.674

0.354

0.500

0.000

0.908

0.670

0.975

0.028

Bowtie2/GATK-HC

0.200

0.414

1.000

0.800

0.588

0.505

0.853

0.139

Bowtie2/GATK-UG

0.000

0.000

0.000

0.000

0.321

0.209

0.920

0.625

Bowtie2/SAMtools

0.674

0.354

0.500

0.000

0.741

0.544

0.950

0.181

Stampy/GATK-HC

0.135

0.071

1.000

0.900

0.758

0.497

0.980

0.278

Stampy/GATK-UG

0.000

0.000

0.000

0.000

0.790

0.542

0.975

0.208

Stampy/SAMtools

0.674

0.354

0.500

0.000

0.908

0.670

0.975

0.028

100 bp interval padding

BWA-MEM/GATK-HC

0.200

0.414

1.000

0.800

0.777

0.516

0.980

0.250

BWA-MEM/GATK-UG

0.00

0.000

0.000

0.000

0.888

0.642

0.978

0.069

BWA-MEM/SAMtools

0.674

0.354

0.500

0.000

0.922

0.673

0.980

0.028

Bowtie2/GATK-HC

0.200

0.414

1.000

0.800

0.588

0.505

0.853

0.139

Bowtie2/GATK-UG

0.000

0.000

0.000

0.000

0.344

0.222

0.925

0.611

Bowtie2/SAMtools

0.674

0.354

0.500

0.000

0.744

0.538

0.955

0.194

Stampy/GATK-HC

0.000

0.000

1.000

1.000

0.758

0.497

0.980

0.278

Stampy/GATK-UG

0.000

0.000

0.000

0.000

0.792

0.550

0.973

0.194

Stampy/SAMtools

0.674

0.354

0.500

0.000

0.906

0.662

0.978

0.042

BWA Enrichment

0.000

0.000

0.000

0.000

0.663

0.577

0.858

0.042

  1. For pipeline performance evaluation, variants were categorized in four groups; exonic single nucleotide variants (SNVs) and exonic indels (insertions or deletions), intronic SNVs (± 1- ± 10) and intronic indels (± 1– ± 10). Sanger sequencing was performed to validate the calls. d perpendicular distance, GATK Genome Analysis ToolKit, FPR false positive rate, HC HaplotypeCaller; indels, insertions & deletions; MCC Matthews correlation coefficient, SNVs single nucleotide variants, TPR true positive rate, UG UnifiedGenotyper; in bold, selected top tier performing pipelines as per MCC ranking