### The monotonicity of the reconciliation costs

We first have the following useful observations on the gene duplication cost.

**Lemma 1***Let f be a reconciliation between a gene tree G and a species tree S. If u is an internal vertex in G with children u*_{1}*and u*_{2}, *then the following observations hold*.

*(i): δ*_{
f
}(*u*) = 1 *if and only if f*(*u*) ∈ {*f*(*u*_{1}), *f*(*u*_{2})} *or lca*(*f*(*u*_{1}), *f*(*u*_{2})) <*f*(*u*).

*(ii): δ*_{
f
}(*u*) = 0 *if and only if f*(*u*_{1}) ≠ *f*(*u*) ≠ *f*(*u*_{2}) *and lca*(*f*(*u*_{1}),*f*(*u*_{2})) = *f*(*u*).

*(iii): If L*(*f*(*u*_{1})) ∩ *L*(*f*(*u*_{2})) ≠ ∅, *then δ*_{
f
}(*u*) = 1.

*(iv): If f*(*u*) >*M*(*u*), *then δ*_{
f
}(*u*) = 1.

*(v): If δ*_{
f
}(*u*) = 0, *then f*(*u*) = *M*(*u*) *and L*(*f*(*u*_{1})) ∩ *L*(*f*(*u*_{2})) = ∅.

*Proof:* Since *δ*_{
f
}(*u*) is either 0 or 1, (ii) clearly follows from (i), and (v) follows from (iii) and (iv).

To establish (i), it suffices to show that lca(*f*(*u*
_{1}), *f*(*u*
_{2})) <*f*(*u*) if and only if *P*(*f*(*u*), *f*(*u*
_{1})) and *P*(*f*(*u*),*f*(*u*
_{2})) share a common edge. Indeed, if we have lca(*f*(*u*
_{1}),*f*(*u*
_{2})) <*f*(*u*), then *P*(*f*(*u*), *f*(*u*
_{1})) and *P*(*f*(*u*), *f*(*u*
_{2})) share the edge that is incident to *lc*a(*f*(*u*
_{1}), *f*(*u*
_{2})) and its parent. On the other hand, if *P*(*f*(*u*), *f*(*u*
_{1})) and *P*(*f*(*u*), *f*(*u*
_{2})) share a common edge (*s*, *s′*) with *s′* <*s*, then *s′* is a common ancestor of *f*(*u*
_{1}) and *f*(*u*
_{2}) such that *s′* <*s* ≤ *f*(*u*). Therefore we have *L*(*f*(*u*
_{1}), *f*(*u*
_{2})) ≤ *s′* <*f*(*u*), as required.

Now we proceed to prove (iii). If *L*(*f*(*u*
_{1})) ∩ *L*(*f*(*u*
_{2})) ≠ ∅, then we have either *f*(*u*
_{1}) ≤ *f*(*u*
_{2}) or *f*(*u*
_{2}) ≤ *f*(*u*
_{1}). By symmetry, we may assume *f*(*u*
_{1}) ≤ *f*(*u*
_{2}), and hence lca(*f*(*u*
_{1}), *f*(*u*
_{2})) = *f*(*u*
_{2}) holds. Now there are two cases to be considered, i.e., *f*(u_{2}) = *f*(*u*) and *f*(u_{2}) <*f*(*u*). By (i), we can conclude *δ*
_{
f
}(*u*) = 1 in both of them.

It remains to show (iv). Note first that we can assume lc*a*(*f*(*u*
_{1}), *f*(*u*
_{2})) >*M*(*u*), because otherwise we have lc*a*( *f*(*u*
_{1}), *f*(*u*
_{2})) = *M*(*u*) <*f*(*u*), and hence *δ*
_{
f
}(*u*) = 1 by (i). It follows that *f*(*u*
_{
i
}) >*M*(*u*) for some *i* = 1, 2. Therefore, by switching *u*
_{1} and *u*
_{2} if necessary, we can further assume *f*(*u*
_{1}) >*M*(*u*). Now we need to consider two cases: *f*(*u*
_{2}) >*M*(*u*) and *f*(*u*
_{2}) ≤ *M*(*u*). If *f*(*u*
_{2}) >*M*(*u*), then *f*(*u*
_{1}) and *f*(*u*
_{2}) are both contained in the path *P*(*f*(*u*), *M*(*u*)), and thus *L*(*f*(*u*
_{1})) ∩ *L*(*f*(*u*
_{2})) ≠ ∅ holds. On the other hand, *f*(*u*
_{2}) ≤ *M*(*u*) implies *f*(*u*
_{2}) ≤ *f*(*u*
_{1}), and hence also *L*(*f*(*u*
_{1})) ∩ *L*(*f*(*u*
_{2})) ≠ ∅. Since in both cases we have *L*(*f*(*u*
_{1})) ∩ *L*(*f*(*u*
_{2})) = ∅, by (iii) we obtain *δ*
_{
f
}(*u*) ≠ 1, as required. Q.E.D

Note that (i) in the above lemma provides an additional characterization of gene duplication events. This characterization is easier for calculation while the original definition is more natural, from an evolutionary point of view. By (v) in the above lemma, if a speciation event happens at *u*, i.e., *δ*
_{
f
}(*u*) = 0, then we have *f*(*u*) = *M*(*u*). This agrees with the definition of reconciliation in [20]. Now we have the following main result.

**Theorem 2***Let f and f′ be two distinct reconciliations between a gene tree G and a species tree S with f′* ≼

*f; then we have:*
*In addition*, *δ*_{
f′
}(*u*) ≤ *δ*_{
f
}(*u*) *for each u* ∈ *V*(*G*), *where the equality holds for each u* ∈ *V*(*G*) *if and only if* gd(*f′*) = gd(*f*).

*Proof:* Let *D*(*f′*, *f*) be the number of vertices *v* in *V*(*G*) with *f*(*v*) ≠ *f′*(*v*)*;* then the following observation plays an important role in our proof of the theorem.

**Lemma 3**
*Let f and f′ be the two reconciliations as given in the theorem. Then there exists a reconciliation f* between G and S that satisfies the following three conditions:*
*Proof:* To establish the above lemma, we select a minimal element

*v*_{min} (with respect to the partial order ≤ on

*V*(

*G*)) in the set {

*u* ∈

*V*(

*G*) :

*f*(

*u*) ≠

*f′*(

*u*)}, which is necessarily non-empty by the assumption

*f* ≠

*f′*. In other words,

*f*(

*v*) =

*f′*(

*v*) holds for any

*v* such that

*v* <

*v*_{min}. Now consider the map

*f** defined as:

Then

*f** is a reconciliation between

*G* and

*S*. To see this, note first that

*f* and

*f′* are reconciliations, and hence they are leaf-preserving. Therefore we know

*f** is also leaf-preserving. Let

*u* and

*v* be a pair of vertices in

*G* with

*u* ≤

*v*. If

*u*,

*v* ∈

*V*(

*G*)

*–* {

*v*
_{min}}, then

*f**(

*u*) ≤

*f**(

*v*) because

*f* is order-preserving. On the other hand, if

*u* =

*v*
_{min} and

*v* ≠

*u*, then we also have:

where we use the fact that *f′* is order-preserving and *f′* ≼ *f* in the first and second inequality, respectively.

Finally, suppose

*v* =

*v*
_{min} and

*u* ≠

*v*. By the way that

*v*
_{min} is chosen, we have

*f′*(

*u*) =

*f*(

*u*), and hence also:

This shows *f** is order-preserving, and hence *f** is indeed a reconciliation between *G* and *S*.

It remains to show that *f** satisfies the three conditions required in the claim. Since *f′* ≼ *f*, from the construction of *f** we have *f′* ≼ *f** ≼ *f*. Noting that *v*
_{min} is the only vertex in *V*(*G*) that is mapped to different images by *f* and *f**, we have *D*(*f**, *f*) = 1. Finally, for any *v* in *G*, *f′*(*v*) ≠ *f**(*v*) if and only if *v* ≠ *v*
_{min} and *f′*(*v*) ≠ *f*(*v*). In other words, we have *D*(*f′*, *f**) = *D*(*f′*, *f*) – 1, which completes the proof of Lemma 3. Q.E.D.

Now it suffices to prove the theorem for the special case *D*(*f*, *f′*) = 1. Indeed, if *D*(*f*, *f′*) = *m* > 1, then by Lemma 3, there exist *m* +1 reconciliations *f*
_{1} := *f′*, *f*
_{2}, *⋯*, *f*
_{
m
}
_{+1} := *f* so that *f*
_{
i
} ≼ *f*
_{
i
}
_{+1} and *D*(*f*
_{
i
}, *f*
_{
i
}
_{+1}) = 1 for 1 ≤ *i* ≤ *m*. Applying the theorem (in the special case mentioned above) for each pair of reconciliations *fi* and *f*
_{
i
}
_{+1}, we have gd(*f*
_{
i
}) ≤ gd(*f*
_{
i
}
_{+1}) for 1 ≤ *i* ≤ *m*, and hence gd(*f′*) = gd(*f*
_{1}) ≤ gd(*f*
_{
m
}
_{+1}) = gd(*f*). Similarly, we can show gl(*f′*) < gl(*f*), dc(*f′*) < dc(*f*), and *δ*
_{
f′
}(*u*) ≤ *δ*
_{
f
}(*u*) for each *u* ∈ *V*(*G*), among which the last one implies that gd(*f*) = gd(*f′*) if and only if *δ*
_{
f
}(*u*) = *δ*
_{
f′
}(*u*) for each *u* ∈ *V*(*G*).

Now let *v* be the unique vertex in *G* with *f*(*v*) ≠ *f′*(*v*). Clearly, *v* is an internal vertex. If *v* is not the root, let *v*
_{0} := *p*(*v*) be its parent and *v*
_{3} be its sibling, that is, the other child of *v*
_{0}. The remainder argument will be divided into three cases, according to the cost measure considered.

### Duplication cost case

Noting that

*f*(

*v*
_{
i
}) =

*f′*(

*v*
_{
i
}) ≤

*f′*(

*v*) <

*f*(

*v*) for

*i* = 1, 2, we have:

By (i) in Lemma 1, this shows *δ*
_{
f
}(*v*) = 1, and hence *δ*
_{
f
}(*v*) ≥ *δ*
_{
f′
}(*v*). If *v* is the root of *G*, then we have gd(*f*) – gd(*f′*) = *δ*
_{
f
}(*v*) *– δ*
_{
f′
}(*v*) ≥ 0, as required.

Now we assume

*v* is not the root, and proceed to show

*δ*
_{
f
}(

*v*
_{0}) ≥

*δ*
_{
f′
}(

*v*
_{0}). To begin with, we can assume

*δ*
_{
f′
} (

*v*
_{0}) = 1, because otherwise the inequality trivially holds. In addition, we can further assume

*f*(

*v*) <

*f*(

*v*
_{0}) and

*f*(

*v*
_{3}) <

*f*(

*v*
_{0}), because otherwise we have

*δ*
_{
f
}(

*v*
_{0}) = 1, which also implies the inequality. It follows that we have:

By (i) in Lemma 1, this leads to lca(*f′*(*v*), *f′*(*v*
_{3})) <*f′*(*v*
_{0}) = *f*(*v*
_{0}). Let *s* be the child of *f*(*v*
_{0}) so that lca(*f′*(*v*), *f′*(*v*
_{3})) ≤ s. Since *f′*(*v*) ≤ *f*(*v*) <*f*(*v*
_{0}) and *f′*(*v*
_{3}) = *f*(*v*
_{3}), *s* is also a common ancestor of *f*(*v*) and *f* (*v*
_{3}). Therefore we have lca (*f′* (*v*), *f′* (*v*
_{3})) ≤ s <*f*(*v*
_{0}). Using (i) in Lemma 1 again, we can conclude *δ*
_{
f
}(*v*
_{0}) = 1, as required.

Since

*v* is the only vertex in

*G* with

*f*(

*v*) ≠

*f′*(

*v*), for each internal vertex

*g* ∈

*V*(

*G*)

*–* {

*v*,

*v*
_{0}} and its two children

*g*
_{1} and

*g*
_{2}, we have:

By definition, this implies
for all *g* ∈ *V*(*G*) *–* {*v*, *v*
_{0}}. Combining the above observations, we can conclude that *δ*
_{
f
}(*u*) ≥ *δ*
_{
f′
}(*u*) for each *u* ∈ *V*(*G*). This leads to gd(*f*) ≥ gd(*f′*), where the equality holds if and only if *δ*
_{
f
}(*u*) = *δ*
_{
f′
}(*u*) for each *u* ∈ *V*(*G*).

### Gene loss case

Since

*f*(

*v*
_{
i
}) ≤

*f′*(

*v*) <

*f*(

*v*) holds for

*i* = 1,2, we have:

Together with the definition of

*l*
_{
f
}, we obtain:

Since *δ*
_{
f
}(*v*) ≥ *δ*
_{
f′
}(*v*), following the proof of the duplication cost case, and *d*(*f*(*v*), *f′*(*v*)) > 0, we can conclude that *l*
_{
f
}(*v*) *– l*
_{
f′
}(*v*) ≥ 0. If *v* is the root of *G*, then this leads to gl(*f*) > gl(*f′*), as required.

Now we assume

*v* is not the root of

*G*. Then we have:

where we use the observation that

*f′*(

*v*) <

*f*(

*v*) ≤

*f*(

*v*
_{0}) implies:

Combining these results, we have:

Since gd(*f*) ≥ gd(*f′*), following the proof of the duplication cost case, and *d*(*f*(*v*), *f′*(*v*)) > 0, we obtain gl(*f*) > gl(*f′*), which completes the proof of this case.

### Deep coalescence case

Let

*E*
_{
f
}(

*S*) be the set of edges

*e* in

*S* such that there exists an edge (

*u*,

*u′*) in

*G* such that

*e* is contained in the directed path from

*f*(

*u*) to

*f*(

*u′*). Now by counting extra lineages in terms of the edges contained in paths that have form

*P*(

*f*(

*u*),

*f*(

*u′*)) for some edge (

*u*,

*u′*) in

*G*, we have:

Since

*E*
_{
f
}(

*S*) =

*E*
_{
f′
}(

*S*) and

*f*(

*u*) =

*f′*(

*u*) for

*u* ≠

*v*, the above formula implies:

if

*v* is not the root of

*G*. Here in the second equality we use the observation that

*f*(

*v*) is on the directed path from

*f*(

*v*
_{0}) to

*f′*(

*v*), and for

*i* = 1,2,

*f′*(

*v*) is on the directed path from

*f*(

*v*) to

*f*(

*v*
_{
i
}). If

*v* is the root of

*G*, then a similar argument leads to:

which completes the proof. Q.E.D.

Since the LCA reconciliation is the minimal element in the space of reconciliations, the above theorem leads directly to the following result.

**Corollary 4***Among all reconciliations between a gene tree G and a species tree S*, *the LCA reconciliation has* (*a*) *the minimum gene duplication cost*[9], (*b*) *the unique one with the optimal gene loss cost*[15]*and the optimal deep coalescence cost*.

Note that there is a close relationship among the gene duplication, gene loss and deep coalescence costs [7]. From their relationship, one can easily obtained the fact that the LCA is the unique one with the optimal gene loss cost from that it is the unique one with the optimal deep coalescence, but the reverse is not clear.

### Gd-optimal reconciliations

By Corollary 4, the LCA reconciliation is the unique optimal reconciliation for the gene loss cost, as well as the deep coalescence cost. However, the LCA reconciliation may not be the unique optimal one for the gene duplication cost (see [15]). For example, for the reconciliation *f* in Figure 1 and the LCA reconciliation M between the gene tree and species tree in Figure 1, we have gd(*f*) = gd(*M*) = 2. Since the reconciliations with the minimum gene duplication cost, which we shall refer to as *gd-optimal reconciliations*, may not be unique, in this section we will present a characterization of them, using the theoretical results developed above.

By Theorem 2, a reconciliation

*f* is gd-optimal if and only if

*δ*
_{
f
}(

*u*) =

*δ*
_{
M
}(

*u*) holds for each vertex

*u* in

*G*. Based on it, we will show that there exists a unique maximal gd-optimal reconciliation

*M** so that

*f* is gd-optimal if and only if

*f* ≼

*M** holds. The reconciliation M* between a gene tree

*G* and a species tree

*S* can be constructed as follows. For all

*u* ∈

*V*(

*G*) with

*δ*
_{
M
}(

*u*) = 0,

*M** maps

*u* to

*M*(

*u*), i.e.,

*M**(

*u*) =

*M*(

*u*). For those

*u* ∈

*V*(

*G*) with

*δ*
_{
M
}(

*u*) = 1, we shall define

*M**(

*u*) recursively. If

*u* =

*r*(

*G*), i.e., it is the root of

*G*, then

*M**(

*u*) is defined as

*r*(

*S*), the root of

*S*. Otherwise,

*M**(

*p*(

*u*)) has been defined, and

*M**(

*u*) is defined as:

If

*u* is a vertex in

*G* such that

*u* ≠

*r*(

*G*), then

*δ*
_{
M
}(

*p*(

*u*)) = 0 implies

*M*(

*u*) <

*M*(

*p*(

*u*)) ≤

*M**(

*p*(

*u*)), hence the mapping

*M** is well defined. In addition,

*M** is also a reconciliation between

*G* and

*S*. To see this, note that if

*u* is a leaf in

*G*, then we have

*δ*
_{
M
}(

*u*) = 0, which implies

*M**(

*u*) =

*M*(

*u*) and hence

*M** is leaf-preserving. On the other hand, by the construction of

*M**, it is order-preserving. For example, for the gene tree and species tree in Figure

1, the reconciliation

*M** is defined as:

In this example, it is not difficult to check that gd(*f*) = gd(*M**) holds for all *f* ≼ *M**, which also follows directly from the following general result.

**Theorem 5***Given a gene tree G and a species tree S*, *a reconciliation f is gd-optimal if and only if M* ≼ *f* ≼ *M** *holds. In particular*, *M* is the unique maximal gd-optimal reconciliation between G and S*.

*Proof:* We need only to show that gd(*f*) = gd(*M*) for a reconciliation *f* if and only if *M* ≼ *f* ≼ *M** holds, because this implies *M** is indeed the unique maximal gd-optimal reconciliation.

To show that gd(

*M*) = gd(

*f*) holds for every reconciliation

*f* with

*M* ≼

*f* ≼

*M**, it suffices to prove gd(

*M**) = gd(

*M*), because together with Theorem 2, this implies gd(

*f*) = gd(

*M*) = gd(

*M**). To this end, we need only to show

*δ*
_{
M
}(

*u*) =

*δ*
_{
M*
}(

*u*) for each internal vertex

*u* in

*G*. Now fix an internal vertex

*u* in

*G*. Since

*M* ≼

*M**, we have

*δ*
_{
M*
}(

*u*) ≥

*δ*
_{
M
}(

*u*) by Theorem 2. If

*δ*
_{
M
}(

*u*) = 1, then we have

*δ*
_{
M*
}(

*u*) = 1 =

*δ*
_{
M
}(

*u*). Therefore it remains to consider the case

*δ*
_{
M
}(

*u*) = 0. By (ii) in Lemma 1,

*δ*
_{
M
}(

*u*) = 0 implies

*M*(

*u*
_{1}) ≠

*M*(

*u*) ≠

*M*(

*u*
_{2}). Together with the construction of

*M**, we have

*M**(

*u*
_{1}) ≠

*M**(

*u*) ≠

*M**(

*u*
_{2}). Since

*M*(

*u*
_{
i
}) ≤

*M**(

*u*
_{
i
}) ≤

*M**(

*u*) for

*i* = 1, 2, we have:

By the construction of

*M*, we know lca(

*M*(

*u*
_{1}),

*M*(

*u*
_{2})) =

*M*(

*u*), and hence:

By (ii) in Lemma 1, this shows *δ*
_{
M
}
_{*} (*u*) = 0, as required.

To establish the other direction, assume gd(

*f*) = gd(

*M*) for a reconciliation

*f*, and we shall show

*f* ≼

*M**, i.e.,

*f*(

*u*) ≤

*M**(

*u*) for each internal

*u* in

*V*(

*G*). To this end, fix an internal vertex

*u* in

*G*, and denote its two children by

*u*
_{1} and

*u*
_{2}. If

*δ*
_{
f
}(

*u*) = 0, then by (v) in Lemma 1 we have

*f*(

*u*) =

*M*(

*u*), and hence

*f*(

*u*) =

*M**(

*u*). Therefore, it remains to prove

*f*(

*u*) ≤

*M**(

*u*) for

*δ*
_{
f
}(

*u*) = 1, which will be established by induction. The base case is

*u* being the root of

*G;* then

*M**(

*u*) is the root of

*S*, and

*f*(

*u*) ≤

*M**(

*u*) trivially holds. For the induction step, let

*u*
_{0} :=

*p*(

*u*) be the parent of

*u;* then the induction assumption is

*f*(

*u*
_{0}) ≤

*M**(

*u*
_{0}). Now if

*δ*
_{
M
}(

*u*
_{0}) = 1, then by the definition of

*M** we have:

Otherwise, we have

*δ*
_{
M
}(

*u*
_{0}) = 0. Together with

*M* ≼

*f*,

*M* ≼

*M** and gd(

*M*) = gd(

*f*) = gd(

*M*), this leads to

*δ*
_{
f
}(

*u*
_{0}) =

*δ*
_{
M*
}(

*u*
_{0}) = 0 by Theorem 2. In view of (v) in Lemma 1, we obtain

*M*(

*u*
_{0}) =

*f*(

*u*
_{0}) =

*M**(

*u*
_{0}). Since

*δ*
_{
f
}(

*u*
_{0}) = 0, (ii) in Lemma 1 implies

*f*(

*u*) <

*f*(

*u*
_{0}), and hence also:

By definition, *M**(*u*) is the largest vertex in the set {*s* : *M*(*u*) ≤ *s* <*M**(*u*
_{0})}. Since *M**(*u*
_{0}) = *M*(*u*
_{0}), we can conclude *f*(*u*) ≤ *M**(*u*), which completes the proof. Q.E.D.

### Enumerate nearly-optimal reconciliations

Recall that there are other reconciliations having the minimum duplication cost than the LCA reconciliation. Moreover, in a biological study, a nearly-optimal reconciliation could be the correct solution to its problem. Therefore, it is of interest to study the following problem [20]: Given a positive number *ε*, compute the set of *nearly-optimal* reconciliations that have the duplication cost less than or equal to gd(*M*) + *ε*, where gd(*M*) is the minimum duplication cost a reconciliation between the gene tree and the species tree can have. Such a subset of the nearly-optimal reconciliations is denoted by Γ_{
ε
}(*G*, *S*, gd), which is also a subset of Γ(*G*, *S*), the set of all reconciliations between *G* and *S*.

In this section we will present an algorithm for enumerating Γ

_{
ε
}(

*G*,

*S*, gd). To this end, we need to introduce some additional definitions. Following [

20], for a vertex

*u* ∈

*V*(

*G*), let id(

*u*) be the number of vertices that precede

*u* according to the prefix traversal of

*G*, where the left child

*u*
_{1} of a vertex

*u* ∈

*V°*(

*G*) is visited before the right child

*u*
_{2}. For a reconciliation

*f* in Γ(

*G*,

*S*), and a vertex

*u* ∈

*V°*(

*G*) with

*f*(

*u*) ≠

*r*(

*S*),

*f*[

*u*] is a mapping defined as:

For an internal vertex

*u* with

*u* ≠

*r*(

*G*),

*f*[

*u*] is a reconciliation if and only if

*f*(

*u*) <

*f*(

*p*(

*u*))

*;* for the root

*r*(

*G*) of

*G*,

*f*[

*r*(

*G*)] is a reconciliation if and only if

*f*(

*r*(

*G*)) <

*r*(

*S*). In both cases, we will say that the reconciliation

*f*[

*u*] is obtained from

*f* by applying a Nearest Mapping Change (NMC) operator on

*u*; this operator is adapted from the one introduced in [

20]. Similarly, we can define

*f*[

*u*
_{1},

*⋯* ,

*u*
_{
k
}] for a sequence of (not necessarily distinct) vertices in G. Note that for a reconciliation

*f* in Γ(

*G*,

*S*) with

*f* ≠

*M*, there exists a unique sequence

*u*
_{1},

*⋯* ,

*u*
_{
k
} so that

*f* =

*M*[

*u*
_{1},

*⋯* ,

*u*
_{
k
}] and id(

*u*
_{
i
}) ≤ id(

*u*
_{
i
}
_{+1}) for

*i* = 1,...,

*k –* 1; now id(

*f*) is defined as id(

*u*
_{
k
}), where

*u*
_{
k
} is the last vertex in this sequence. For completeness, we will use the convention id(

*M*) = 0. Finally, for a reconciliation

*f* in Γ(

*G*,

*S*), we set:

where *K*(*f*) will be regarded as an ordered list (with the order induced by id).

The NMC operator induces a tree structure on the set Γ(

*G*,

*S*): the root is

*M*;

*f′* is a child of

*f* if and only if

*f′* =

*f*[

*u*] for some

*u* ∈

*K*(

*f*). This tree, whose vertex set is Γ(

*G*,

*S*), will be denoted by

*T*(

*G*,

*S*). The idea of considering a tree structure on the space of reconciliation was introduced in [

20]. Clearly, by Theorem 2, the restriction of

*T*(

*G*,

*S*) on Γ

_{
ε
}(

*G*,

*S*, gd) is a subtree, which will be referred to as

*T*
_{
ε
}(

*G*,

*S*, gd). Now we can state our algorithm as follows, which enumerates Γ

_{
ε
}(

*G*,

*S*, gd) by a traversal of

*T*
_{
ε
}(

*G*,

*S*, gd). Here ⊔ stands for disjoint union.

To see the running time of the above algorithm, note first that for a reconciliation *f*, *K*(*f*) is a subset of *V°*(*G*), and for each *u* ∈ *V°*(*G*), whether *u* ∈ *K*(*f*) or not can be determined in constant time, when id(*u*) and id(*f*) are known. In addition, if *δ*
_{
M
} is given, then line 5a and 5b can be computed in constant time; the proof of this observation will be presented in the full version of this paper. Therefore, the above algorithm runs in time *O*(|*V*(*G*)| . |Γ_{
ε
}(*G*, *S*, gd)|), plus additional preprocessing time to compute id(*u*) and *δ*
_{
M
}(*u*) for each *u* ∈ *V°*(*G*).

Two facts prevent us from designing better algorithm for the enumeration problems. The first one concerns the boundary set *B*
_{
ε
}(*G*, *S*, gd), which consists of all reconciliation *f* in Γ(*G*, *S*) *–* Γ_{
ε
}(*G*, S, gd) such that for some *f** ∈ Γ_{
ε
}(*G*, *S*, gd), f is a child of *f** in *T*(*G*, *S*). In order to enumerate Γ_{
ε
}(*G*, *S*, gd), an algorithm typically needs to visit not only the reconciliations in Γ_{
ε
}(*G*, *S*, gd), but also those in *B*
_{
ε
}(*G*, *S*, gd). However, *|B*
_{
ε
}(*G*, *S*, gd)| could be as large as *O*(|*V*(*G*)| . |Γ_{
ε
}(*G*, *S*, gd)|). For instance, if *G* and *S* have the same tree structure on *n* +1 leaves, then Γ_{0}(*G*, *S*, gd) = {*M*} but *|B*
_{0}(*G*, *S*, gd)| contains *n –* 1 reconciliations. Furthermore, we have |Γ_{1}(*G*, *S*, gd)| = *n* and *|B*
_{1}(*G*, *S*, gd)| = Θ(*n*
^{2}).

The other concern is about the set *K*
_{
ε
}(*f*) *:*= {*u* ∈ *V*(*G*) : *f*[*u*] is in Γ_{ε}(*G*, *S*, gd) and id(*u*) ≥ id(*f*)}, which is needed if we want to explore Γ_{
ε
}(*G*, *S*, gd) without visiting the boundary set *B*
_{
ε
}(*G*, *S*, gd). However, some properties of these two sets, *K*(*f*) and *K*
_{
ε
}(*f*), are different. For instance, the following property of *K*(*f*) is crucial to the optimal algorithm for exploring Γ(*G*, *S*) (see Property 5 and Proposition 4 in [20]): If *u* is the first vertex in *K*(*f*) and *f′* = *f*[*u*], then we have *K*(*f*) *– K*(*f*′) ⊆ {*u*}. However, this does not hold for *K*
_{
ε
}. To see it, considering the example mentioned in the previous paragraph, and denoting the first child of *r*(*G*) by *r*
_{1}, then we have *K*
_{1}(*M*) = *V°*(*G*) *–* {*r*(*G*)} while *K*
_{1}(*M*[*r*
_{1}]) = ∅.

Since Γ_{0}(*G*, *S*, gd) contains the gd-optimal reconciliations, the above algorithm also provides a method for enumerating all the optimal reconciliations between a gene tree and a species tree. Since *T*
_{
ε
}(*G*, *S*, gl), as well as *T*
_{
ε
}(*G*, *S*, dc), is also a subtree of *T*(*G*, *S*), we also remark that it can be modified to list nearly-optimal reconciliations with respect to the gene loss or deep coalescence cost. Due to the limited space, the details of these algorithms are omitted here and one is referred to the full version of this work appearing in our personal website. As our on-going work, the algorithms presented here will be coded in C++ and evaluated by comparing them with the existing ones on simulation data.