Evaluating four major algorithms for identifying differential regulators in condition-specific transcriptional responses
BMC Bioinformatics volume 15, Article number: P33 (2014)
Identifying molecular regulators underlying condition-specific transcriptional responses is essential for our understanding of their underlying molecular mechanisms. So far, there have been several computational methods developed for this purpose. Specifically, four major algorithms, TFact , RIF , CSA , and DRrank , were released one after another. Each of these algorithms has its own features and evaluation strategies. Thus, these methods should be systematically evaluated so that the users can make the most appropriate method selection for their practical application needs.
Materials and methods
In this work, we evaluated the four algorithms using Escherichia coli transcription network models and synthetic expression datasets that were generated by SynTReN  and GeneNetWeaver . Specifically, we developed a simulation-based schema to evaluate each algorithm according to operatively defined, known “differential regulators.” In addition, we tested each method’s robustness against its key parameter(s) and explored factors that influence algorithm performance in general.
We found that TFact and DRrank stood out as the best methods in terms of both accuracy and robustness. In total, there were seven cases in which one single regulator was artificially perturbed. TFact attained the closest approximation four times, while DRrank had the best performance the other three times. In another 15 scenarios in which there were perturbations of multiple regulators, DRrank was ranked on average the best algorithm, while TFact had the second-place rank. Based on these observations, TFact and DRrank may each be best applied to different circumstances: TFact is better used for single regulator implication, while DRrank is better for multiple regulators’ simultaneous perturbation. In general, we observed that algorithms’ performances were negatively correlated with the number of regulation links per target, which may indicate that independent regulators are easier to recover than synergistic ones.
This work represents a preliminary benchmarking evaluation of four major currently-available algorithms for differential regulator identification. According to our simulation-based evaluation paradigm, two of the published algorithms, TFact and DRrank, are more robust than the other two. Further evaluation is needed, such as applying them to specific human disease expression datasets under different conditions or different platforms. Furthermore, these algorithms may be extended to rapidly emerging next-generation sequencing (NGS) data.
Essaghir A, Toffalini F, Knoops L, Kallin A, van Helden J, Demoulin JB: Transcription factor regulation can be accurately predicted from the presence of target gene signatures in microarray gene expression data. Nucleic Acids Res. 2010, 38 (11): e120-10.1093/nar/gkq149.
Reverter A, Hudson NJ, Nagaraj SH, Perez-Enciso M, Dalrymple BP: Regulatory impact factors: unraveling the transcriptional regulation of complex traits from expression data. Bioinformatics. 2010, 26 (7): 896-904. 10.1093/bioinformatics/btq051.
Huang CL, Lamb J, Chindelevitch L, Kostrowicki J, Guinney J, Delisi C, Ziemek D: Correlation set analysis: detecting active regulators in disease populations using prior causal knowledge. BMC Bioinformatics. 2012, 13: 46-10.1186/1471-2105-13-46.
Yang J, Yu H, Liu BH, Zhao Z, Liu L, Ma LX, Li YX, Li YY: DCGL v2.0: An R package for unveiling differential regulation from differential co-expression. PLoS One. 2013, 8 (11): e79729-10.1371/journal.pone.0079729.
Van den Bulcke T, Van Leemput K, Naudts B, van Remortel P, Ma H, Vershoren A, De Moor B, Marchal K: SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC Bioinformatics. 2006, 7: 43-10.1186/1471-2105-7-43.
Schaffter T, Marbach D, Floreano D: GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods. Bioinformatics. 2011, 27 (16): 2263-2270. 10.1093/bioinformatics/btr373.
This work was partially supported by grants (R01LM011177, R03CA167695, P30CA68485, and P50CA095103).
About this article
Cite this article
Yu, H., Zhao, Z. Evaluating four major algorithms for identifying differential regulators in condition-specific transcriptional responses. BMC Bioinformatics 15 (Suppl 10), P33 (2014). https://doi.org/10.1186/1471-2105-15-S10-P33