From: Optimal alpha reduces error rates in gene expression studies: a meta-analysis approach
Critical effect size (CES) | Average of 100 iterations of 15,000 tests | α = 0.05 | Bonferroni FWER = 0.05 | Benjamini-Hochberg FDR = 0.05 | Optimal α |
---|---|---|---|---|---|
A. | |||||
CES = 1SD | # of significant results | 2046 | 0 | 1 | 6776 |
# of Type I errors | 376 | 0 | 0 | 2143 | |
# of Type II errors ≥ CES | 5829 | 7500 | 7499 | 2867 | |
# of Type I and II errors | 6205 | 7500 | 7499 | 5010 | |
% error reduction by using optimal α | 19.3% | 33% | 33% | - | |
CES = 2SD | # of significant results | 5298 | 3 | 1709 | 7659 |
# of Type I errors | 379 | 0 | 43 | 1130 | |
# of Type II errors ≥ CES | 2581 | 7497 | 5834 | 970 | |
# of Type I and II errors | 2960 | 7497 | 5876 | 2100 | |
% error reduction by using optimal α | 29% | 72% | 64% | - | |
CES = 4SD | # of significant results | 7848 | 61 | 7560 | 7608 |
# of Type I errors | 378 | 0 | 190 | 212 | |
# of Type II errors ≥ CES | 30 | 7439 | 130 | 105 | |
# of Type I and II errors | 408 | 7439 | 320 | 317 | |
% error reduction by using optimal α | 22% | 96% | 1% | - | |
B. | |||||
CES = 1SD | # of significant results | 1400 | 0 | 0 | 1456 |
# of Type I errors | 562 | 0 | 0 | 590 | |
# of Type II errors ≥ CES | 2912 | 3750 | 3750 | 2883 | |
# of Type I and II errors | 3474 | 3750 | 3750 | 3473 | |
% error reduction by using optimal α | 0.02% | 7% | 7% | - | |
CES = 2SD | # of significant results | 3032 | 1 | 119 | 3537 |
# of Type I errors | 562 | 0 | 5 | 791 | |
# of Type II errors ≥ CES | 1280 | 3749 | 3636 | 1004 | |
# of Type I and II errors | 1842 | 3749 | 3641 | 1795 | |
% error reduction by using optimal α | 3% | 52% | 51% | - | |
CES = 4SD | # of significant results | 4295 | 31 | 3665 | 3826 |
# of Type I errors | 560 | 0 | 136 | 200 | |
# of Type II errors ≥ CES | 15 | 3719 | 221 | 124 | |
# of Type I and II errors | 575 | 3719 | 358 | 324 | |
% error reduction by using optimal α | 44% | 91% | 9% | - | |
C. | |||||
CES = 1SD | # of significant results | 1012 | 0 | 0 | 10 |
# of Type I errors | 680 | 0 | 0 | 5 | |
# of Type II errors ≥ CES | 1167 | 1500 | 1500 | 1495 | |
# of Type I and II errors | 1847 | 1500 | 1500 | 1500 | |
% error reduction by using optimal α | 19% | 0% | 0% | - | |
CES = 2SD | # of significant results | 1662 | 1 | 3 | 1083 |
# of Type I errors | 677 | 0 | 0 | 334 | |
# of Type II errors ≥ CES | 515 | 1499 | 1497 | 752 | |
# of Type I and II errors | 1192 | 1499 | 1498 | 1086 | |
% error reduction by using optimal α | 9% | 28% | 27% | - | |
CES = 4SD | # of significant results | 2169 | 12 | 1261 | 1539 |
# of Type I errors | 675 | 0 | 56 | 143 | |
# of Type II errors ≥ CES | 6 | 1488 | 295 | 105 | |
# of Type I and II errors | 681 | 1488 | 350 | 248 | |
% error reduction by using optimal α | 64% | 83% | 29% | - |