Skip to main content

Table 2 Effect of BQSR in SNP and INDEL calling

From: Impact of post-alignment processing in variant discovery from whole exome data

Type

Metric

Div

5x coverage

40x coverage

(%)

Case

Min

Max

Mdn

Case

Min

Max

Mdn

SNP

Sensitivity

0.05–0.1

50

−0.1

1.6

0.6

50

0

0.2

0

SNP

Sensitivity

0.5–1

50

−4.5

0.9

−0.6

50

−0.5

0.1

−0.1

SNP

Sensitivity

5

20

−11.4

−2.6

−9.8

20

−1.8

0.2

−0.5

SNP

Sensitivity

10

20

−57.8

−19.5

−32.3

20

−89.4

3

−7.1

SNP

Precision rate

0.05–1

100

−0.1

0.3

0

100

−0.2

0.1

0

SNP

Precision rate

5

20

0

0.3

0.1

20

0

0.2

0

SNP

Precision rate

10

0

-

-

-

18

−0.3

1.5

0.2

INDEL

Sensitivity

0.05–0.1

40

−1.5

1.8

0.4

50

−0.2

0

0

INDEL

Sensitivity

0.5–1

40

−0.3

1

0.1

50

−0.1

0.1

0

INDEL

Sensitivity

5

12

−6.4

−0.1

−1.4

20

−0.9

2.8

0

INDEL

Sensitivity

10

12

−51.9

−2.4

−34.6

16

−75.3

0.1

−9.7

INDEL

Precision rate

0.05–1

80

−0.1

0

0

100

−0.1

0.2

0

INDEL

Precision rate

5

12

0

0.3

0

20

0

0.8

0

INDEL

Precision rate

10

0

-

-

-

13

−0.2

3.2

0

  1. Between 5 and 25 mapper-caller combinations were assessed at each of the six divergence levels and two coverage depths in simulation, excluding combinations with BWA at 5–10 % divergence. They were selected to have a sensitivity of at least 35 %; to estimate the change in precision rate, we also required that sensitivity should not decrease by more than 15 % after BQSR. Div divergence, Mdn median