Skip to main content

Table 2 Descriptive statistics of GATK filters in the HC dataset, stratifying calls by type (SNV/Indels). Data are displayed as mean ± sd

From: GATK hard filtering: tunable parameters to improve variant calling for next generation sequencing targeted gene panel data

  SNVs Indels
TV
mean ± sd
FV
mean ± sd
p-value TV
mean ± sd
FV
mean ± sd
p-value
BQRS 0.11 ± 0.03 -0.6 ± 0.5 <0.0001 0.28 ± 0.12 0.15 ± 0.05 <0.0001
RPRS -0.067 ± 0.05 0.05 ± 0.4 0.0009 -0.74 ± 0.22 0.23 ± 0.05 <0.0001
CRS 0.0007 ± 0.02 -0.009 ± 0.29 0.72 0.001 ± 0.07 0.007 ± 0.03 0.6
DP 96.61 ± 0.58 49.25 ± 5.06 <0.0001 109.4 ± 9.57 96.01 ± 0.1 <0.0001
MQ 60 ± 0 59.99 ± 0.07 - 60 ± 0 60 ± 0 -
MQRS -0.03 ± 0.02 -0.05 ± 0.28 0.3 -0.02 ± 0.09 -0.21 ± 0.04 <0.0001
GQ 99 ± 0 79.15 ± 12.06 - 99 ± 0 73.16 ± 2.7 -
  1. The mean value is the mean value of the median value from each of the 100 replicates
  2. BQRS BaseQRankSum, RPRS ReadPosRankSum, CRS ClippingRankSum, DP depth of coverage, MQ MappingQuality, MQRS MappingQualityRankSum, GQ genotype quality, TV true variants, FV false variants