Skip to main content

Table 3 Descriptive statistics of GATK filters in the LC dataset, stratifying calls by type (SNV/Indels). Data are displayed as mean ± sd

From: GATK hard filtering: tunable parameters to improve variant calling for next generation sequencing targeted gene panel data

  SNVs Indels
TV
mean ± sd
FV
mean ± sd
p-value TV
mean ± sd
FV
mean ± sd
p-value
BQRS 0.02 ± 0.02 -0.27 ± 0.12 <0.0001 0.16 ± 0.16 0.01 ± 0.03 <0.0001
RPRS -0.19 ± 0.03 -0.31 ± 1.1 <0.0001 -1.29 ± 0.3 0.04 ± 0.04 <0.0001
CRS -0.02 ± 0.02 -0.05 ± 0.07 <0.0001 -0.004 ± 0.12 -0.04 ± 0.01 0.001
DP 19.97 ± 0.17 22.72 ± 1.3 <0.0001 21.83 ± 1.97 20.24 ± 0.42 <0.0001
MQ 60 ± 0 60 ± 0 - 60 ± 0 60 ± 0 -
MQRS -0.06 ± 0.01 -0.1 ± 0.08 <0.0001 -0.15 ± 0.15 -0.1 ± 0.04 0.03
GQ 99 ± 0 20.04 ± 2.9 - 99 ± 0 17.94 ± 0.92 -
  1. The mean value is the mean value of the median value from each of the 100 replicates
  2. BQRS BaseQRankSum, RPRS ReadPosRankSum, CRS ClippingRankSum, DP depth of coverage, MQ MappingQuality, MQRS MappingQualityRankSum, GQ genotype quality, TV true