Volume 11 Supplement 3
Learning biological network using mutual information and conditional independence
© Gao et al; licensee BioMed Central Ltd. 2010
Published: 29 April 2010
Biological networks offer us a new way to investigate the interactions among different components and address the biological system as a whole. In this paper, a reverse-phase protein microarray (RPPM) is used for the quantitative measurement of proteomic responses.
To discover the signaling pathway responsive to RPPM, a new structure learning algorithm of Bayesian networks is developed based on mutual Information, conditional independence, and graph immorality. Trusted biology networks are thus predicted by the new approach. As an application example, we investigate signaling networks of ataxia telangiectasis mutation (ATM). The study was carried out at different time points under different dosages for cell lines with and without gene transfection. To validate the performance ofthe proposed algorithm, comparison experiments were also implemented using three well-known networks. From the experiment results, our approach produces more reliable networks with a relatively small number of wrong connection especially in mid-size networks. By using the proposed method, we predicted different networks for ATM under different doses of radiation treatment, and those networks were compared with results from eight different protein protein interaction (PPI) databases.
By using a new protein microarray technology in combination with a new computational framework, we demonstrate an application of the methodology to the study of biological networks of ATM cell lines under low dose ionization radiation.
Bayesian networks are widely applied to a variety of domains such as business, engineering, and medicine . The networks can be used to make optimal inference decisions based on Bayesian theory as well as to find the causal relationship between different entities as a graph model.
To perform an efficient inference and correct representation of the dependency relationship, an optimal structure is constructed to maximize the probabilistic fitness to the given data. Determining the optimal network through learning structures of Bayesian networks has been explored over the last decade, which contains the development of searching and scoring schemes. The searching is to find the structure that has the highest score among all possible ones. Since the searching space grows exponentially when the number of variables (nodes) increases, it is known as NP hard . The scoring computes the score of a structure to evaluate how well it fits a given data.
Until now, several scoring functions have been developed including the well known Cooper-Herskovits scoring function as in K2 algorithm , the likelihood equivalence Bayesian Dirichlet (BDe) scoring function , and the minimum description length (MDL) scoring function . In addition to serving as a scoring function, the K2 algorithm  functioning as a searching method has attracted attention from researchers due to its performance and efficiency till quite recently. However, the assumption of K2 algorithm is that the order of variables is correctly known. In other words, the performance highly depends on if the variables are well ordered. With regard to structure learning, Hill-Climbing greedy algorithm was used as a searching algorithm in . Conditional independence property and mutual information were also employed for structure learning .
The goal of this study is to infer the proteomic signaling pathways affected by DNA damage, DNA repair, cell cycle checkpoints, and cell apoptosis under the influence of different radiation dosages. An emerging protein microarray technology, called the revers-phase protein microarray (RPPM), in conjunction with the quantum dots (Qdot) nano-technology, is used as the detection system. We study the proteomic responses at different time points (1h, 6h, 24h, 48h, and 72h) under different dosages (4 cGy, 10 cGy, 50 cGy, 1 Gy, and 5 Gy).
To infer the signaling pathways under different radiation dosages, in this paper we propose a new Bayesian network structure learning algorithm using the mutual information, conditional independence, and property of immorality in graph. Our method has two important features. First, the algorithm does not provide the direction for every edge in a predicted network. Since a signaling pathway is composed of successive and oriented interactions of molecules, even a small number of edges that have incorrect directions can cause significant effect in biological network analysis. To avoid a misleading result, therefore, we aim to report the most trusted edges, though a complete directed graph is not produced. Second, we focus on reducing wrong edges even though price for missing edges is paid. In other words, reliable, though not complete, information is reported as opposed to complete but uncertain information. To achieve these two goals, we initially exclude edges with low mutual information, and strictly carry out conditional independence test and immorality test for each candidate edge in order to remove incorrect edges. In the following sections, we first introduce the main steps of the proposed methodology. Then we use well known standard networks to evaluate the performance of the algorithm. Finally proteomic networks for ATM cell lines under different radiation dosages are presented.
Bayesian networks and MDL scoring function
where r i is the number of states for variable X i , q i is the number of possible configurations of the parent set Pa(Xi) of X i with , and N ijk is the number of instances in the data set D where the variable X i takes the value X ik and the set of variables Pa(X i ) have the j th (j =1,2,…,q i ) configuration in the parent set of Pa(X i ). N ij is the total number of the j th configuration of Pa(X i ) .
Mutual information and conditional independence
where N is the number of samples and x i is a state of variable X.
In our algorithm, conditional independence (CI) is also used to find which edge is incorrect in a triangular structure. CI is defined as follows:
X i and X j is conditionally independent given X k if P(X i ,X j | X k ) = P(X i |X k )P(X j |X k )(5)
Therefore, once the edge we consider to connect makes a new triangle, we can test (5) for all three edges of the triangle. Based on the result of the CI test, we can update the network.
Property of equivalence class and immorality
The proposed algorithm initially starts from a non-connected network in which there is no edge between nodes. We calculate MI for two nodes of all edges, and the edges whose MI is less than threshold α are excluded from candidates of correct edges. In this paper, α is heuristically chosen as 0.0001. Based on these MI values, edges are ordered. After finishing the ordering, we sequentially decide the connection for each edge in the MI order. Since earlier decided edges can affect the decision for other edges, we choose edges which have higher chances to be correct edge as it is supposed that higher MI has higher probability of connection. Unlike greedy searching algorithms that normally involve many times of iterations, we test each edge one time only. Before we decide the connection edge by edge, each edge is categorized into three cases which have different decision rules as discussed in the following paragraphs.
The first case is when the current edge creates a triangular structure. Current edge indicates an edge currently to be decided in the algorithm. In this case, as we noted earlier, CI test is performed to find which edge is not necessary among the three edges of the triangle. For instance, suppose there are three nodes i, j, and k, and the edge E(i, j) is the current edge. Since E(i, j) creates a triangle structure, we test CI of three edges, E(i, j), E(j, k), and E(k, i). If only one pair of nodes is conditional independent, the edge between the two nodes is deleted. The other results of CI test are ignored which means we do not use current edge. After one edge is deleted or current edge is abandoned, we perform Immorality test with two remained edges in order to find the direction of them.
The second case is that the current edge creates a cycle in the graph which means there should be at least more than one Immorality in the cycle because Bayesian Network is an acyclic graph. Since most of the edges with relatively low MI creates cycles and are added after correct network is constructed already, we have to avoid the wrong edges with immorality test between current and other linked edges. If there is no immorality, we do not use the current edge. As an example, given a structure with four nodes, h, i, j, k and the current edge as E(i, j), if E(i, j) makes a new triangle structure, we perform two immorality tests with two pairs of edges like E(i, j), E(h, i) and E(i, j), E(j, k). If both of the two pairs have immorality, we choose one randomly. However, in most cases, only one of the two pairs has immorality. In our example, if E(i, j) and E(h, i) has immorality but not E(i, j) and E(j, k), we connect the current edge E(i, j) and give the head to head direction like h→i←j.
Results and discussion
Results of structure learning for known networks
Result for the Asia network
Result for the Car Diagnosis2 network
Result for the Alarm network
Result for Trustworthy Network
Learning structure of pathway in ATM cell
67 antibodies used in the reverse-phase protein array for ATM radiation study
The expression data is normalized with respect to Actin concentration on each microarray chip. The expression level of each antibody is discretized into 2 to 4 values using minimum entropy based discretization. For each IR dose, we have a total number of 30 samples for ATM+ and ATM- from the triplicate at different times. Among the 67 antibodies involved in the RPPM data, we select the most distinguishing ones between ATM+ and ATM- using a feature selection method developed in our early work .
Understanding the proteomic network structure reveals the inherent biological information flow which will lead to more effective therapies and disease treatments. In this paper, by using a new protein microarray technology in combination with a new computational framework, we demonstrate an application of the methodology to the study of biological networks of ATM cells under ionization radiation. Different networks were found through this study. The same technology can be extended to different biological problems. For future work, we intend to validate our discovery by carrying out biological experiments.
This research was supported by the Department of Energy under Grant No. DEFG02-07ER64335.
This article has been published as part of BMC Bioinformatics Volume 11 Supplement 3, 2010: Selected articles from the 2009 IEEE International Conference on Bioinformatics and Biomedicine. The full contents of the supplement are available online at http://www.biomedcentral.com/1471-2105/11?issue=S3.
- Pearl J: Probabilistic Reasoning in Interlligent Systems: Networks of Plausible Inference. Morgan Kaufmann; 1988.
- Chickering DM: Learning Bayesian Networks is NP-Complete. Springer-Verlag; 1996.View Article
- Cooper GF, Herskovits E: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 1992, 9(2):309–347.
- Heckerman D, Geiger D, Chickering DM: Learning Bayesian networks: The combination of knowledge and statistical data, Volume 20. 1995.View Article
- Lam W, Bacchus F: Learning Bayesian belief networks: An approach based on the MDL principle. Computational Intelligence 1994, 10: 269–293.View Article
- Tsamardinos I, Brown , Constantin A: The maxmin hillclimbing Bayesian network structure learning algorithm. Machine Learning 2006, 65: 31–78. [http://dx.doi.org/10.1007/s10994–006–6889–7]View Article
- Chen XW, Anantha G, Lin X: Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm. IEEE Trans. on Knowl. and Data Eng. 2008, 20(5):628–640.View Article
- Rissanen J: Modeling By Shortest Data Description. Automatica 1978, 14: 465–471.View Article
- de Campos LM: A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests. 2006.
- Verma T, Pearl J: A theory of inferred causation.1991. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.7221]
- Friedman N, Linial M, Nachman I, Pe'er D: Using Bayesian Networks to Analyze Expression Data. Journal of Computational Biology 2000, 7(34):601–620. [http://dx.doi.org/10.1089/106652700750050961]View ArticlePubMed
- Lauritzen SL, Spiegelhalter DJ: Local computations with probabilities on graphical structures and their application to expert systems. 1990, 415–448.
- Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF: The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks. 1989, 38: 247–256.
- Kim YB, Gao J, Dong Y, Yang CR: Functional Proteomic Pattern Identification under Low Dose Ionizing Radiation. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine. IEEE Computer Society; 2008:3–8.
- Chen XW, Anantha G, Wang X: An effective structure learning method for constructing gene networks. Bioinformatics 2006, 22(11):1367–1374. [http://view.ncbi.nlm.nih.gov/pubmed/16543279]View ArticlePubMed
- Chow C, Liu C: Approximating discrete probability distributions with dependence trees. Information Theory, IEEE Transactions on 1968, 14(3):462–467. [http://ieeexplore.ieee.org/xpls/abs\_all.jsp?arnumber=1054142]View Article
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.