Skip to main content

Table 2 Relative performance of different ensemble methods using all level 1 inference methods’ results (i.e., regardless of kurtosis) as ensemble inputs and the same models while only using level 1 inference methods’ results with positive kurtosis, marked by plus signs (corresponding to positive kurtosis)

From: EnsInfer: a simple ensemble approach to network inference outperforms any single method

Ensemble model

Interval between time-series data points

10 min

20 min

25 min

50 min

100 min

Logistic regression

1.11 ± 0.45

0.99 ± 0.21

0.92 ± 0.34

1.33 ± 0.56

1.25 ± 0.44

Logistic regression\(^+\)

1.13 ± 0.48

1.12 ± 0.43

0.95 ± 0.35

1.31 ± 0.54

1.24 ± 0.46

Logistic regression with SGD

0.25 ± 0.06

0.4 ± 0.47

0.2 ± 0.11

0.2 ± 0.14

0.26 ± 0.19

Logistic regression with SGD\(^+\)

0.71 ± 0.23

0.63 ± 0.19

0.54 ± 0.08

0.55 ± 0.15

0.57 ± 0.21

Naive Bayes

0.83 ± 0.23

0.71 ± 0.17

0.65 ± 0.15

0.67 ± 0.16

0.68 ± 0.26

Naive Bayes\(^+\)

1.33 ± 0.58

1.09 ± 0.26

1.19 ± 0.5

1.14 ± 0.48

1.13 ± 0.35

Support vector machine

0.21 ± 0.09

0.29 ± 0.14

0.26 ± 0.09

0.25 ± 0.15

0.26 ± 0.22

Support vector machine\(^+\)

0.47 ± 0.2

0.51 ± 0.19

0.43 ± 0.12

0.86 ± 0.6

0.47 ± 0.25

K-nearest neighbors

0.59 ± 0.24

0.45 ± 0.22

0.51 ± 0.2

0.39 ± 0.34

0.57 ± 0.41

K-nearest neighbors\(^+\)

0.58 ± 0.23

0.68 ± 0.54

0.71 ± 0.47

0.6 ± 0.46

0.65 ± 0.31

Random forest

1.12 ± 0.56

0.9 ± 0.21

0.99 ± 0.29

1.73 ± 1.03

1.09 ± 0.38

Random forest\(^+\)

1.1 ± 0.51

0.97 ± 0.42

1.04 ± 0.37

1.73 ± 1.1

1.08 ± 0.34

Adaptive boosting

1.12 ± 0.5

1.13 ± 0.76

0.73 ± 0.26

1.27 ± 0.65

0.95 ± 0.46

Adaptive boosting\(^+\)

1.07 ± 0.43

1.11 ± 0.65

0.82 ± 0.27

1.3 ± 0.7

1.03 ± 0.49

XGBoost

0.65 ± 0.34

0.56 ± 0.16

0.61 ± 0.28

1.35 ± 0.86

0.66 ± 0.24

XGBoost\(^+\)

0.6 ± 0.25

0.58 ± 0.23

0.78 ± 0.63

1.31 ± 0.86

0.67 ± 0.26

Best level 1 method in training

0.88 ± 0.23

0.88 ± 0.2

0.88 ± 0.24

1 ± 0

0.81 ± 0.22

Best level 1 method evaluated on all samples

0.93 ± 0.26

1.11 ± 0.58

1.08 ± 0.34

1.03 ± 0.49

0.95 ± 0.44

Average rank of level 1 methods

0.95 ± 0.42

0.88 ± 0.52

0.78 ± 0.39

0.69 ± 0.16

0.61 ± 0.22

  1. Experiments were done across five different DREAM simulation settings for time-series intervals. The performance metric is the ratio of AUPRC score of the ensemble method compared to that of the best performing level 1 inference method in testing. Each cell is the mean value and standard deviation across the ten DREAM datasets with varying regulatory networks. The bold number in each column is the best performing value in that time interval setting. Logistic regression, random forest and adaptive boosting models yielded top level inference performance among all ensemble options when there was no filtering based on kurtosis. With kurtosis filtering, the Naive Bayes and logistic regression approaches yield the best overall results while performances from random forest and adaptive boosting are still competitive