Skip to main content

Table 2 Type-I error and power (α=0.05) of the four methods for data simulated from NBMM and CPMM

From: Model based heritability scores for high-throughput sequencing data

  Data from NB-sim   Data from CP-sim  
Method/ \(\sigma ^{2}_{g}\) 0 0.10 0.25 0.50 0.75 1 0 0.10 0.25 0.50 0.75 1
CP-fit 0.10 0.78 0.89 0.94 0.96 0.97 0.04 0.72 0.86 0.92 0.96 0.96
NB-fit 0.04 0.71 0.86 0.92 0.95 0.95 0.04 0.72 0.86 0.92 0.96 0.96
VST 0 0.56 0.76 0.86 0.90 0.92 0 0.55 0.76 0.86 0.90 0.92
  1. The value in a cell denotes the value of the power function (rounded off to 2 places after decimal) for each case. The value of \(\sigma ^{2}_{g}\) drives the power. The case \(\sigma ^{2}_{g}=0\) reflects the null hypothesis, therefore corresponding the cell value is the type-I error. Columns with \(\sigma ^{2}_{g} > 0\) indicates the power for increasing strain specific variance