Skip to main content

Table 3 Mean squared error of false discovery rates under variable library factors

From: A flexible count data model to fit the wide diversity of expression profiles arising from extensively replicated RNA-seq experiments

#DE

Rank

1.5-fold change

2-fold change

4-fold change

  

MSE

Method

MSE

Method

MSE

Method

 

1

0.030

DESeq - Pmax

0.059

DESeq - Cmax

0.046

tweeDEseq

 

2

0.099

tweeDEseq

0.064

tweeDEseq

0.055

DESeq - Pmax

 

3

0.189

DESeq - Cmax

0.072

DESeq - Pmax

0.057

DESeq - Cmax

 

4

0.194

DESeq - PgOn

0.116

DESeq - PgOn

0.106

DESeq - PgOn

100

5

0.258

edgeR - QLF Df1

0.129

edgeR - QLF Df1

0.124

edgeR - QLF Df1

 

6

0.348

edgeR - QLF (def)

0.129

edgeR - QLF (def)

0.153

edgeR - QLF (def)

 

7

0.581

edgeR - (def)

0.273

edgeR - (def)

0.290

edgeR - (def)

 

8

0.667

edgeR - Df1

0.420

edgeR - Df1

0.380

edgeR - Df1

 

9

8.882

DESeq - CgOn

6.429

DESeq - CgOn

5.217

DESeq - CgOn

 

1

0.005

tweeDEseq

0.006

tweeDEseq

0.008

tweeDEseq

 

2

0.009

DESeq - Cmax

0.012

DESeq - Cmax

0.010

DESeq - Cmax

 

3

0.013

DESeq - PgOn

0.012

DESeq - PgOn

0.011

edgeR - QLF Df1

 

4

0.016

edgeR - QLF Df1

0.016

edgeR - QLF Df1

0.013

edgeR - QLF (def)

1000

5

0.019

edgeR - QLF (def)

0.017

edgeR - QLF (def)

0.024

DESeq - PgOn

 

6

0.054

edgeR - (def)

0.051

edgeR - (def)

0.045

edgeR -(def)

 

7

0.083

DESeq - Pmax

0.082

DESeq - Pmax

0.067

DESeq - Pmax

 

8

0.087

edgeR - Df1

0.083

edgeR - Df1

0.077

edgeR - Df1

 

9

0.700

DESeq - CgOn

0.545

DESeq - CgOn

0.529

DESeq - CgOn

 

1

0.003

tweeDEseq

0.004

tweeDEseq

0.005

DESeq - Cmax

 

2

0.003

DESeq - PgOn

0.006

edgeR - QLF Df1

0.017

edgeR - QLF (def)

 

3

0.006

edgeR - QLF Df1

0.006

edgeR - QLF (def)

0.018

edgeR - QLF Df1

 

4

0.007

edgeR - QLF (def)

0.007

DESeq - PgOn

0.023

tweeDEseq

2000

5

0.025

DESeq - Cmax

0.024

DESeq - Cmax

0.029

DESeq - Pmax

 

6

0.028

edgeR - (def)

0.026

edgeR - (def)

0.053

edgeR - (def)

 

7

0.049

edgeR - Df1

0.047

edgeR - Df1

0.088

edgeR - Df1

 

8

0.091

DESeq - Pmax

0.077

DESeq - Pmax

0.092

DESeq - PgOn

 

9

0.267

DESeq - CgOn

0.238

DESeq - CgOn

0.465

DESeq - CgOn

  1. Data in this table correspond to the mean squared error (MSE) values between the empirical false discovery rates (eFDR) and the nominal q-values obtained from the simulation study shown in Figure 8 in which library factors were variable.