Skip to main content

Table 4 A-C. A comparison of the mean number of significant results among four different procedures for evaluating significance of multiple comparisons: Type I errors, and Type II errors for 100 iterations of 15,000 simulated differential gene expression test using (1) α = 0.05 for all tests, (2) a Bonferroni correction to adjust the family-wise error rate (FWER) to 0.05, (3) the Benjamini-Hochberg procedure to adjust the false-discovery rate (FDR) to 0.05, and (4) optimal α

From: Optimal alpha reduces error rates in gene expression studies: a meta-analysis approach

Critical effect size (CES)

Average of 100 iterations of 15,000 tests

α = 0.05

Bonferroni FWER = 0.05

Benjamini-Hochberg FDR = 0.05

Optimal α

A.

CES = 1SD

# of significant results

2046

0

1

6776

# of Type I errors

376

0

0

2143

# of Type II errors ≥ CES

5829

7500

7499

2867

# of Type I and II errors

6205

7500

7499

5010

% error reduction by using optimal α

19.3%

33%

33%

-

CES = 2SD

# of significant results

5298

3

1709

7659

# of Type I errors

379

0

43

1130

# of Type II errors ≥ CES

2581

7497

5834

970

# of Type I and II errors

2960

7497

5876

2100

% error reduction by using optimal α

29%

72%

64%

-

CES = 4SD

# of significant results

7848

61

7560

7608

# of Type I errors

378

0

190

212

# of Type II errors ≥ CES

30

7439

130

105

# of Type I and II errors

408

7439

320

317

% error reduction by using optimal α

22%

96%

1%

-

B.

CES = 1SD

# of significant results

1400

0

0

1456

# of Type I errors

562

0

0

590

# of Type II errors ≥ CES

2912

3750

3750

2883

# of Type I and II errors

3474

3750

3750

3473

% error reduction by using optimal α

0.02%

7%

7%

-

CES = 2SD

# of significant results

3032

1

119

3537

# of Type I errors

562

0

5

791

# of Type II errors ≥ CES

1280

3749

3636

1004

# of Type I and II errors

1842

3749

3641

1795

% error reduction by using optimal α

3%

52%

51%

-

CES = 4SD

# of significant results

4295

31

3665

3826

# of Type I errors

560

0

136

200

# of Type II errors ≥ CES

15

3719

221

124

# of Type I and II errors

575

3719

358

324

% error reduction by using optimal α

44%

91%

9%

-

C.

CES = 1SD

# of significant results

1012

0

0

10

# of Type I errors

680

0

0

5

# of Type II errors ≥ CES

1167

1500

1500

1495

# of Type I and II errors

1847

1500

1500

1500

% error reduction by using optimal α

19%

0%

0%

-

CES = 2SD

# of significant results

1662

1

3

1083

# of Type I errors

677

0

0

334

# of Type II errors ≥ CES

515

1499

1497

752

# of Type I and II errors

1192

1499

1498

1086

% error reduction by using optimal α

9%

28%

27%

-

CES = 4SD

# of significant results

2169

12

1261

1539

# of Type I errors

675

0

56

143

# of Type II errors ≥ CES

6

1488

295

105

# of Type I and II errors

681

1488

350

248

% error reduction by using optimal α

64%

83%

29%

-

  1. Type II error rates and optimal α levels were evaluated using three different critical effect sizes (CES), representing effects as large as 1, 2, and 4 standard deviations (SD) of the data. The 15,000 simulated tests had 4 replicates in the experimental and control groups, and were constructed such that (A) H A prior probability = 0.50, H o prior probability = 0.50; (B) H A prior probability = 0.25, H o prior probability = 0.75; (C) H A prior probability = 0.10, H o prior probability = 0.90