Structural properties of the reconciliation space and their applications in enumerating nearly-optimal reconciliations between a gene tree and a species tree

Introduction A gene tree for a gene family is often discordant with the containing species tree because of its complex evolutionary course during which gene duplication, gene loss and incomplete lineage sorting events might occur. Hence, it is of great challenge to infer the containing species tree from a set of gene trees. One common approach to this inference problem is through gene tree and species tree reconciliation. Results In this paper, we generalize the traditional least common ancestor (LCA) reconciliation to define a reconciliation between a gene tree and species tree under the tree homomorphism framework. We then study the structural properties of the space of all reconciliations between a gene tree and a species tree in terms of the gene duplication, gene loss or deep coalescence costs. As application, we show that the LCA reconciliation is the unique one that has the minimum deep coalescence cost, provide a novel characterization of the reconciliations with the optimal duplication cost, and present efficient algorithms for enumerating (nearly-)optimal reconciliations with respect to each cost. Conclusions This work provides a new graph-theoretic framework for studying gene tree and species tree reconciliations.


Background
With much higher speed than the traditional Sanger sequencing technology, the ultra-deep sequencing technology has made huge amounts of molecular data available for genomics study [1]. It provides an unprecedented opportunity to infer phylogenetic trees from multilocus and genomics data. One approach to inferring phylogeny from multilocus data is to reconstruct a gene tree from each locus and then to combine the resulting trees into a phylogeny, called the containing species tree. Gene trees are often different since each gene family might undergo different mutational events such as gene duplication and loss, horizontal gene transfer, and incomplete lineage sorting [2,3]. Therefore, the containing species tree is inferred from gene trees by reconciling it with each gene tree to minimize the total number of hypothetical evolutionary events that are responsible for the discordance between the trees.
The gene tree and species tree reconciliation was first introduced by Goodman et al. [4] and formally defined by Page [5]. Given a gene tree for a gene family and a containing species tree, a reconciliation between them represents an evolutionary scenario of the gene family within the evolutionary history represented by the species tree [4]. To study gene duplication history, gene tree and species tree are reconciled to minimize the number of gene duplications and/or losses. The mathematical and algorithmic issues of gene tree and species tree reconciliations have been intensively studied in the past decade [6][7][8][9][10][11][12][13][14]. For example, it has been shown that the so-called least common ancestor (LCA) reconciliation has the minimum duplication and loss cost [9,15].
Although the LCA reconciliation is optimal in terms of the duplication cost, it may not represent the true evolution of the gene family being considered. Indeed, recent studies suggest that more than one reconciliations may occur with the highest probability [16,17]. Such studies [3,14,17,18] in the stochastic framework assume that the discordance between a gene tree and a species tree is caused by incomplete lineage sorting and adopt Kingman's coalescent theory from population genetics [19].
The fact that the LCA reconciliation may not be the unique optimal with respect to the duplication cost motivates researchers to study the space of all the reconciliations and develop algorithms to enumerate nearly-optimal reconciliations for a species tree and a gene tree [20,21]. In this paper, we take a different approach to these two issues. We generalize the LCA reconciliation to define an arbitrary reconciliation as a vertex-mapping from a gene tree to a species tree that preserves the hierarchical structure of the gene tree. Our approach is essentially different from the existing ones [20,22], where the specific mutation events are used and a gene tree vertex is mapped to a species tree branch to specify a duplication event. One advantage of our approach over the others is that we separate reconciliation concept from the cost models that are used to measure the tree discordance. Because of this, we are able to study the structural properties of the space of all reconciliations between a gene tree and a species tree in the same manner for each of the three cost models. We show that the LCA reconciliation has not only the minimum duplication and loss cost [9,15], but also the minimum deep coalescence cost. We also present a novel characterization of the reconciliations with the optimal duplication cost, and develop efficient algorithms for enumerating (nearly-) optimal reconciliations with respect to each cost model.

Basic notations
Species evolve from their common ancestor through a series of speciation events. A species tree represents the evolutionary history of a set of species. A gene family might evolve from its common ancestral gene through gene duplication and loss events. Here we will assume that no lateral gene transfer has occurred. Both gene and species trees are rooted trees with labeled leaves. In a species tree, a leaf x represents a species, the label of x. Hence, the species tree is uniquely leaf-labeled. In a gene tree, a leaf y represents a gene found in a species. To infer the duplication history of a gene family, its gene tree and the containing species tree is reconciled [4]. For this purpose, a leaf of a gene tree is labeled with the containing species. Since a species may contain duplicate genes, two leaves in a gene tree can have the same label.
Let T be a species or gene tree; its vertex set and edge set are denoted by V(T) and E(T), respectively. Given two vertices u and v in T, there exists a unique path P (u, v) from u to v. The number of edges in P(u, v), denoted by d(u, v), is called the distance between u and v. Note that d(u, v) = 0 if and only if u = v. The node v is a descendant of u or u is an ancestor of v, denoted by v ≤ u, if u is on the unique path from r(T), the root of T, to v. For simplicity, we also write v <u if v ≤ u and v ≠ u. Given a set A of vertices in T, u is a common ancestor of A if and only if v ≤ u for every v A. In addition, if u ≤ u′ for any other common ancestor u′ of A, then we say u is the least common ancestor of A, written as lca(A), or lca(u 1 , ..., u k ) if A = {u 1 , ..., u k }.
For each vertex u in T with u ≠ r(T), the parent of u, denoted by p(u), is the unique vertex in T that is adjacent to u and contained on the path from r(T) to u. In this case, u is also called a child of p(u). The out-degree of u, denoted by d(u), is defined as the number of the children of u. Obviously, a node is a leaf if and only if its out-degree is 0. Non-leaf nodes are internal nodes; they form a subset V°(T) of V(T). If every internal vertex has out-degree two, then T is binary. For an internal vertex u in a binary tree, its two children are denoted by u 1 and u 2 , unless stated otherwise. In this study, we will focus on the case that gene trees and species trees are binary. For a vertex u, we use L(u) to denote the set of the labels of its leaf descendants and call it the cluster induced by u. Finally, we use L(T) to denote the set of leaf labels, i.e., the cluster induced by the root of T.

Reconciliation between gene tree and species tree
Let S be a species tree over a set of species and G a gene tree such that L(G) ⊆ L(S), i.e., G is over all the homologous genes of a gene family found in some spe- is the unique leaf in S that has the same label.
A reconciliation between a gene tree G and a species tree S is a leaf-preserving and order-preserving map from V(G) to V(S). Clearly, a reconciliation f between G and S is necessarily an inclusion-preserving mapping (see [8]), that is, for each pair of vertices u, v in G, u ≤ v implies L(f(u)) ⊆ L(f(v)). However, the reverse statement is not true. For instance, the mapping that maps each vertex of G to the root of S is an inclusion-preserving mapping, but according to our definition, it is not leaf-preserving, and hence not a reconciliation.
Note that our definition is consistent with the one used in [20], where a reconciliation is defined as a mapping from V(G) to V(S) ∪ E(S) that satisfies three constraints: base constraint, tree mapping constraint and ancestor consistency constraint. Roughly speaking, our order-preserving condition corresponds to the ancestor consistency constraint, and the leaf-preserving condition is related to the base constraint, while the tree mapaaping constraint is not needed in our setting. The main difference between these two frameworks is the model used to interpret mappings. For example, in [20], a duplication event is associated to a vertex v in G if and only if v is mapped to an edge, while in our model, whether v is associated with a duplication event is not solely determined by the image of v.
A reconciliation represents a hypothetical evolutionary history of the gene family. In a gene tree, an internal vertex u represents the common ancestor of the genes represented by the leaves below it. The property just reflects the intuitive fact that u is an ancient gene appearing in some common ancestor of the species from which the genes are taken. Recall that in species tree each branch represents an ancestral species. Under the reconciliation f, we considered u as the gene ancestor found in the species represented by the branch entering f(u).
There is a canonical partial order ≼ on the set of reconciliations between G and S: for any f′ and f, f′ ≼ f if and only if f′(v) ≤ f(v) holds for every vertex v in G. Define a mapping M from G to S recursively as: M is called the least common ancestor (LCA) reconciliation between G and S. Note that we have M ≼ f for every reconciliation f between G and S, because it is easy to see that M(u) ≤ f(u) holds for all u V(G), by a bottom-up traversal.

Inference of gene duplications
If the discord of a gene tree G and its containing species tree S is due to gene duplication, a reconciliation f between them represents a plausible duplication history of genes. For an internal vertex u, a duplication event is associated with u if and only if one of the following two conditions holds: and P(f(u), f(u 2 )) contain a common edge. In the literature (see [5]), when the LCA reconciliation M is used for inferring gene duplications, the duplication condition used is (D-i). This is correct for the LCA reconciliation between a gene tree and a species tree. However, this stringent condition is no longer appropriate as the definition of duplication events for arbitrary reconciliations. For example, consider the reconciliation f between the gene tree G and the species tree S as in Figure 1. If the original definition is used, as proposed in [8], only one duplication is inferred, which is associated with r. However, one duplication cannot produce such a gene family having the gene tree G. On the other hand, if our proposed definition is used, two duplications are inferred, one associated with r and the other with b; the implied duplication scenario is given in Figure 2. Now, for an internal node u, we let δ f (u) = 1 if there is a duplication event associated with it, and δ f (u) = 0 otherwise. Then the gene duplication cost gd (f) of f is defined as:

Gene loss cost
Let G be a gene tree, S a species tree and f a reconciliation between G and S. Then the number of losses l f (u) associated to an internal vertex u is defined as: Note that our definition of l f (u) is a generalization of the one introduced by Ma et al. in [6], and is consistent with the one in [20]. When f is the LCA reconciliation, our definition agrees with the traditional one [5,6]. For later use, it is often convenient to combine the two formulae in the above definition, i.e., we have: For simplicity, we also set l f (x) = 0 for any leaf x of G. The gene loss cost gl(f) of f is defined as: For example, for the reconciliation f in Figure 1, we have gl(f) = 7 by noting that: 4 and which can also been observed from Figure 2.

Deep coalescence cost
If the discord of a gene tree G and a species tree S is due to incomplete lineage sorting, a reconciliation f Wu and Zhang BMC Bioinformatics 2011, 12(Suppl 9):S7 http://www.biomedcentral.com/1471-2105/12/S9/S7 between them is measured by the deep coalescence cost [3]. Given a branch e in S, we say that there are k (k > 0) extra lineages (with respect to f) failing to coalesce on e, denoted by τ f (e) = k, if there exist k + 1 distinct edges i.e., the total number of the extra lineages with respect to f on all branches of S. For example, for the reconciliation f in Figure 1, we have dc(f) = 3 by noting that:

Results
The monotonicity of the reconciliation costs We first have the following useful observations on the gene duplication cost.
Lemma 1 Let f be a reconciliation between a gene tree G and a species tree S. If u is an internal vertex in G with children u 1 and u 2 , then the following observations hold. ( Proof: Since δ f (u) is either 0 or 1, (ii) clearly follows from (i), and (v) follows from (iii) and (iv).
It remains to show (iv). Note first that we can assume lca(f(u 1 ), f(u 2 )) >M(u), because otherwise we have lca( f(u 1 ), f(u 2 )) = M(u) <f(u), and hence δ f (u) = 1 by (i). It follows that f(u i ) >M(u) for some i = 1, 2. Therefore, by switching u 1 and u 2 if necessary, we can further assume f(u 1 ) >M(u). Now we need to consider two cases: f(u 2 ) >M(u) and f(u 2 ) ≤ M(u). If f(u 2 ) >M(u), then f(u 1 ) and f(u 2 ) are both contained in the path P(f (u), M(u)), and thus L(f(u 1 )) ∩ L(f(u 2 )) ≠ ∅ holds. On the other hand, f(u 2 ) ≤ M(u) implies f(u 2 ) ≤ f(u 1 ), and hence also L(f(u 1 )) ∩ L(f(u 2 )) ≠ ∅. Since in both cases we have L(f(u 1 )) ∩ L(f(u 2 )) = ∅, by (iii) we obtain δ f (u) ≠ 1, as required. Q.E.D Note that (i) in the above lemma provides an additional characterization of gene duplication events. This characterization is easier for calculation while the original definition is more natural, from an evolutionary point of view. By (v) in the above lemma, if a speciation event happens at u, i.e., δ f (u) = 0, then we have f(u) = M(u). This agrees with the definition of reconciliation in [20]. Now we have the following main result.
Theorem 2 Let f and f′ be two distinct reconciliations between a gene tree G and a species tree S with f′ ≼ f; then we have: Lemma 3 Let f and f′ be the two reconciliations as given in the theorem. Then there exists a reconciliation f* between G and S that satisfies the following three conditions: Proof: To establish the above lemma, we select a minimal element v min (with respect to the partial order ≤ on V(G)) in the set {u V(G) : f(u) ≠ f′(u)}, which is necessarily non-empty by the assumption f ≠ f′. In other words, f(v) = f′(v) holds for any v such that v <v min . Now consider the map f* defined as: Then f* is a reconciliation between G and S. To see this, note first that f and f′ are reconciliations, and hence they are leaf-preserving. Therefore we know f* is also leaf-preserving. Let u and v be a pair of vertices in G with u ≤ v. If u, v V(G) -{v min }, then f*(u) ≤ f*(v) because f is order-preserving. On the other hand, if u = v min and v ≠ u, then we also have: where we use the fact that f′ is order-preserving and f′ ≼ f in the first and second inequality, respectively.
Finally, suppose v = v min and u ≠ v. By the way that v min is chosen, we have f′(u) = f(u), and hence also: This shows f* is order-preserving, and hence f* is indeed a reconciliation between G and S.
It remains to show that f* satisfies the three conditions required in the claim. Since f′ ≼ f, from the construction of f* we have f′ ≼ f* ≼ f. Noting that v min is the only vertex in V(G) that is mapped to different images by f and Now let v be the unique vertex in G with f(v) ≠ f′(v). Clearly, v is an internal vertex. If v is not the root, let v 0 := p(v) be its parent and v 3 be its sibling, that is, the other child of v 0 . The remainder argument will be divided into three cases, according to the cost measure considered.
Since v is the only vertex in G with f(v) ≠ f′(v), for each internal vertex g V(G) -{v, v 0 } and its two children g 1 and g 2 , we have: By definition, this implies d d

Gene loss case
Since f(v i ) ≤ f′(v) <f(v) holds for i = 1,2, we have: Together with the definition of l f , we obtain: , following the proof of the duplication cost case, and d(f(v), f′(v)) > 0, we can conclude that l f (v)l f′ (v) ≥ 0. If v is the root of G, then this leads to gl(f) > gl(f′), as required. Now we assume v is not the root of G. Then we have: Combining these results, we have: Since gd(f) ≥ gd(f′), following the proof of the duplication cost case, and d(f(v), f′(v)) > 0, we obtain gl(f) > gl(f ′), which completes the proof of this case.

Deep coalescence case
Let E f (S) be the set of edges e in S such that there exists an edge (u,u′) in G such that e is contained in the directed path from f(u) to f(u′). Now by counting extra lineages in terms of the edges contained in paths that have form P(f(u), f(u′)) for some edge (u, u′) in G, we have: Since E f (S) = E f′ (S) and f(u) = f′(u) for u ≠ v, the above formula implies: if v is not the root of G. Here in the second equality we use the observation that f(v) is on the directed path from f(v 0 ) to f′(v), and for i = 1,2, f′(v) is on the directed path from f(v) to f(v i ). If v is the root of G, then a similar argument leads to: which completes the proof. Q.E.D.
Since the LCA reconciliation is the minimal element in the space of reconciliations, the above theorem leads directly to the following result.
Corollary 4 Among all reconciliations between a gene tree G and a species tree S, the LCA reconciliation has (a) the minimum gene duplication cost [9], (b) the unique one with the optimal gene loss cost [15]and the optimal deep coalescence cost.
Note that there is a close relationship among the gene duplication, gene loss and deep coalescence costs [7]. From their relationship, one can easily obtained the fact that the LCA is the unique one with the optimal gene loss cost from that it is the unique one with the optimal deep coalescence, but the reverse is not clear.

Gd-optimal reconciliations
By Corollary 4, the LCA reconciliation is the unique optimal reconciliation for the gene loss cost, as well as the deep coalescence cost. However, the LCA reconciliation may not be the unique optimal one for the gene duplication cost (see [15]). For example, for the reconciliation f in Figure 1 and the LCA reconciliation M between the gene tree and species tree in Figure 1, we have gd(f) = gd(M) = 2. Since the reconciliations with the minimum gene duplication cost, which we shall refer to as gd-optimal reconciliations, may not be unique, in this section we will present a characterization of them, using the theoretical results developed above.
By Theorem 2, a reconciliation f is gd-optimal if and only if δ f (u) = δ M (u) holds for each vertex u in G. Based on it, we will show that there exists a unique maximal gd-optimal reconciliation M* so that f is gd-optimal if and only if f ≼ M* holds. The reconciliation M* between a gene tree G and a species tree S can be constructed as follows. For all u V(G) with δ M (u) = 0, M* maps u to M(u), i.e., M*(u) = M(u). For those u V(G) with δ M (u) = 1, we shall define M*(u) recursively. If u = r(G), i.e., it is the root of G, then M*(u) is defined as r(S), the root of S. Otherwise, M*(p(u)) has been defined, and M*(u) is defined as: If u is a vertex in G such that u ≠ r(G), then δ M (p(u)) = 0 implies M(u) <M(p(u)) ≤ M*(p(u)), hence the mapping M* is well defined. In addition, M* is also a reconciliation between G and S. To see this, note that if u is a leaf in G, then we have δ M (u) = 0, which implies M*(u) = M(u) and hence M* is leaf-preserving. On the other hand, by the construction of M*, it is order-preserving. For example, for the gene tree and species tree in Figure  1, the reconciliation M* is defined as: for 1 5 In this example, it is not difficult to check that gd(f) = gd(M*) holds for all f ≼ M*, which also follows directly from the following general result.

Enumerate nearly-optimal reconciliations
Recall that there are other reconciliations having the minimum duplication cost than the LCA reconciliation. Moreover, in a biological study, a nearly-optimal reconciliation could be the correct solution to its problem. Therefore, it is of interest to study the following problem [20]: Given a positive number ε, compute the set of nearly-optimal reconciliations that have the duplication cost less than or equal to gd(M) + ε, where gd(M) is the minimum duplication cost a reconciliation between the gene tree and the species tree can have. Such a subset of the nearly-optimal reconciliations is denoted by Γ ε (G, S, gd), which is also a subset of Γ(G, S), the set of all reconciliations between G and S.
In this section we will present an algorithm for enumerating Γ ε (G, S, gd). To this end, we need to introduce some additional definitions. Following [20], for a vertex u V(G), let id(u) be the number of vertices that precede u according to the prefix traversal of G, where the left child u 1 of a vertex u V°(G) is visited before the right child u 2 . For a reconciliation f in Γ(G, S), and a vertex u V°(G) with f(u) ≠ r(S), f[u] is a mapping defined as: ( ( )) , For an internal vertex u with u ≠ r(G), f[u] is a reconciliation if and only if f(u) <f(p(u)); for the root r(G) of G, f[r(G)] is a reconciliation if and only if f(r(G)) <r(S). In both cases, we will say that the reconciliation f[u] is obtained from f by applying a Nearest Mapping Change (NMC) operator on u; this operator is adapted from the one introduced in [20]. Similarly, we can define f[u 1 ,... , u k ] for a sequence of (not necessarily distinct) vertices in G. Note that for a reconciliation f in Γ(G, S) with f ≠ M, there exists a unique sequence u 1 , ... , u k so that f = M[u 1 , ... , u k ] and id(u i ) ≤ id(u i+1 ) for i = 1,..., k -1; now id(f) is defined as id(u k ), where u k is the last vertex in this sequence. For completeness, we will use the convention id(M) = 0. Finally, for a reconciliation f in Γ(G, S), we set: where K(f) will be regarded as an ordered list (with the order induced by id).
The NMC operator induces a tree structure on the set Γ(G, S): the root is M; f′ is a child of f if and only if f′ = f [u] for some u K(f). This tree, whose vertex set is Γ(G, S), will be denoted by T(G, S). The idea of considering a tree structure on the space of reconciliation was introduced in [20]. Clearly, by Theorem 2, the restriction of T (G, S) on Γ ε (G, S, gd) is a subtree, which will be referred to as T ε (G, S, gd). Now we can state our algorithm as follows, which enumerates Γ ε (G, S, gd) by a traversal of T ε (G, S, gd). Here ⊔ stands for disjoint union.

Algorithm for enumerating nearly -optimal reconciliations
A gene tree and a spec Input : G i ies tree with , and . The set

Set and
Output To see the running time of the above algorithm, note first that for a reconciliation f, K(f) is a subset of V°(G), and for each u V°(G), whether u K(f) or not can be determined in constant time, when id(u) and id(f) are known. In addition, if δ M is given, then line 5a and 5b can be computed in constant time; the proof of this observation will be presented in the full version of this paper. Therefore, the above algorithm runs in time O(|V (G)| . |Γ ε (G, S, gd)|), plus additional preprocessing time to compute id(u) and δ M (u) for each u V°(G).
Two facts prevent us from designing better algorithm for the enumeration problems. The first one concerns the boundary set B ε (G, S, gd), which consists of all reconciliation f in Γ(G, S) -Γ ε (G, S, gd) such that for some f* Γ ε (G, S, gd), f is a child of f* in T(G, S). In order to enumerate Γ ε (G, S, gd), an algorithm typically needs to visit not only the reconciliations in Γ ε (G, S, gd), but also those in B ε (G, S, gd). However, |B ε (G, S, gd)| could be as large as O(|V(G)| . |Γ ε (G, S, gd)|). For instance, if G and S have the same tree structure on n +1 leaves, then Γ 0 (G, S, gd) = {M} but |B 0 (G, S, gd)| contains n -1 reconciliations. Furthermore, we have |Γ 1 (G, S, gd)| = n and |B 1 (G, S, gd)| = Θ(n 2 ).
The other concern is about the set K ε (f) := {u V(G) : f[u] is in Γ ε (G, S, gd) and id(u) ≥ id(f)}, which is needed if we want to explore Γ ε (G, S, gd) without visiting the boundary set B ε (G, S, gd). However, some properties of these two sets, K(f) and K ε (f), are different. For instance, the following property of K(f) is crucial to the optimal algorithm for exploring Γ(G, S) (see Property 5 and Proposition 4 in [20]): If u is the first vertex in K(f) and f′ = f[u], then we have K(f) -K(f′) ⊆ {u}. However, this does not hold for K ε . To see it, considering the example mentioned in the previous paragraph, and denoting the first child of r(G) by r 1 , then we have K 1 (M) = V°(G) -{r(G)} while K 1 (M[r 1 ]) = ∅.
Since Γ 0 (G, S, gd) contains the gd-optimal reconciliations, the above algorithm also provides a method for enumerating all the optimal reconciliations between a gene tree and a species tree. Since T ε (G, S, gl), as well as T ε (G, S, dc), is also a subtree of T(G, S), we also remark that it can be modified to list nearly-optimal reconciliations with respect to the gene loss or deep coalescence cost. Due to the limited space, the details of these algorithms are omitted here and one is referred to the full version of this work appearing in our personal website. As our on-going work, the algorithms presented here will be coded in C++ and evaluated by comparing them with the existing ones on simulation data.

Conclusions
To investigate all reconciliations between a gene tree and a species tree, we have generalized the LCA reconciliation to define an arbitrary reconciliation as a vertex mapping from the gene tree to the species tree. This provides a new framework for investigating various mathematical issues of the reconciliation space. It allows us to give a unified approach to study reconciliations with each of the cost models. As applications, we show that the LCA reconciliation is the unique one having the smallest deep coalescence cost, and present a characterization of the reconciliations with the minimum gene duplication cost; we also develop efficient algorithms to enumerate nearly-optimal reconciliations with each cost models. In future, we shall incorporate other evolutionary forces behind the gene tree heterogeneity, such as horizontal gene transfer and recombination, into this framework.