Related Work

Random graphs are graphs that are generated by some random process [26]. They are widely used as models of complex networks [5] and can assume various levels of complexity. The simplest model for generating random graphs, with only a single parameter, is the Bernoulli or Erdös-Renyi random graph model, which produces graphs that are completely defined by their average degree and are random in all other respects. A slightly more complex and general model is one that generates graphs with a specified degree distribution (or degree sequence) and ones which are random in all other respects [27]. These models can be extended to include additional structural constraints, such as degree correlations or the density of triangles or longer cycles, as we will demonstrate below.

Existing methods for generating clustered graphs, however, do not take this approach. One of the first examples is the seminal work of Watts and Strogatz [1]. They introduced a model that produces networks with high clustering and low average path length (typical distances between pairs of nodes in the network are small), now known as the small world property. Although not intended as a generative algorithm for clustered graphs, the model produces graphs with clustering spanning the range from 0 to 1. The graphs generated under this model, however, have rigid spatial structure and cannot accommodate varying degree distributions.

The first algorithms that were designed to generate graphs with a specified level of clustering for arbitrary degree distributions belonged to the class of projected bipartite graphs. Newman [20] introduced a three-step method that first builds a bipartite graph of individuals and affiliations, then projects the bipartite graph to a unipartite graph of individuals only, and finally runs a percolation process over the unipartite graph. This results in a clustered graph with a degree distribution that depends on the original distributions of numbers of individuals per group and groups per individual. The level of clustering in the final graph varies smoothly from 0 to 1 as a function of the percolation probability. In [28], Guillaume suggested a similar bipartite graph approach. Although these approaches can generate clustered graphs with diverse degree distributions, they lack straightforward methods for choosing parameters that yield graphs with not only a pre-specified clustering coefficient but also a pre-specified degree distribution. These algorithms also tends to produce graphs that leave a significant proportion of the graph vertices isolated.

A second class of clustered graph models use "growing network" algorithms [29–31]. The inputs to these models are a degree distribution and level of clustering. The method begins with a set of vertices with no edges; the graph is then "grown" by adding edges based on the degree and clustering constraints. Although the algorithms of this class allow for arbitrary degree distributions and levels of clustering, they either require a complex implementation [29], produce graphs of a highly specific structure [31] or introduce large amounts of degree correlations [31, 30].

Finally, the family of statistical models known as exponential random graph (ERG) models [32, 33] also provide tools to fit the structure of observed networks, for statistics such as degree distribution and number of triangles. These ERG model-based methods, although they have advanced significantly in recent years (e.g. [34]), still suffer from problems of degeneracy and computational intractability for large networks.

Our Approach

Here, we present a model that generates undirected, simple and connected graphs with prescribed degree sequences and a specified frequency of triangles, while maintaining a graph structure that is as random (uncorrelated) as possible. (A simple graph is one which contains no self-loops (edges from a node to itself) or multiedges (multiple edges between the same pair of nodes); and a connected graph is one where every node in the graph is reachable by a path of edges from every other graph node.) Prior models in this area were intended to generate clustered graphs that replicate the properties of real-world networks; our goal, on the other hand, is to generate a class of null networks with arbitrary degree distributions that are simple and connected and have a high density of triangles, but are random in all other respects.

This method thus leads to two valuable applications. First, network structure fundamentally influences the functions of and dynamical processes on networks. We can use clustered random graphs to systematically study the consequences of clustering, both independently and in combination with various degree patterns. Second, these networks can serve as null models for detecting whether an empirical network can be boiled down to its degree distribution and clustering values or, instead, contains substantial degree correlations or other important structures (beyond the byproducts of the degree distribution and clustering). One would first use the algorithm to generate an ensemble of networks that match the empirical degree sequences and clustering values, and then compare the structural, functional, or dynamical properties of the empirical network to those of the clustered random networks. We focus here on the role of these networks as null models as it is crucial to have appropriate random controls in the study of biological systems, as has been demonstrated in [24, 35, 36].

The rest of this article is organized as follows. In the Implementation section, we review common measures of clustering and introduce our Markov chain model and algorithm for generating clustered graphs with a specified degree sequence. In the Results section, we test our algorithm with numerical simulations and explore the structural properties of the generated graphs. The Discussion section is devoted to a demonstration of the randomly generated clustered networks as null networks for the analysis of empirical networks. We finish off with our conclusions, presenting the benefits of our Markov Chain simulation method for biological networks.

Measures of Clustering

We begin with a graph G = (V, E) which is undirected and simple. V is the set of vertices of G and E is the set of the edges. We let N = |V| and M = |E| denote the number of nodes and edges in G, respectively. The degree of a node i will be denoted d _i . The set of degrees for all nodes in the graph makes up the degree sequence, which follows a probability distribution called the degree distribution.

Clustering is the likelihood that two neighbors of a given node are themselves connected. In topological terms, clustering measures the density of triangles in the graph, where a triangle is the existence of the set of edges (i, j), (i, k), (j, k) between any triplet of nodes i, j, k (Figure 1b).

To quantify the local presence of triangles, δ(i) is defined as the number of triangles in which node i participates. Since each triangle consists of three nodes, it is counted thrice when we sum δ(i) for each node in the graph. Thus the total number of triangles in the graph is

A triple is a set of three nodes, i, j, k that are connected by edges (i, j) and (i, k), regardless of the existence of the edge (j, k) (Figure 1a). The number of triples of node i is simply

assuming d _i ≥ 2. To compute the total number of triples in the graph, τ(G), we sum τ(i) for all i ∈ V.

The clustering coefficient was introduced by Watts and Strogatz [1] as a local measure of triadic closure. For a node i with d _i ≥ 2, the clustering coefficient c(i) is the fraction of triples for node i which are closed, and can be measured as δ(i) = τ(i). The clustering coefficient of the graph is then given by:

where N₂ is the number of nodes with c(i) ≥ 0. Some authors do define the clustering coefficient for all nodes of G [43].

A more global measure of the presence of triangles is called the transitivity of graph G and is defined as:

Although they are often similar, T(G) and C(G) can vary by orders of magnitude [22]. They differ most when the triangles are heterogeneously distributed in the graph.

These traditional measures of clustering are degree-dependent and thus can be biased by the degree sequence of the network. The maximum number of possible triangles for a given node i is just its number of triples (τ(i)). For a node which is connected to only low degree neighbors, however, the maximum number of possible triangles may be much smaller than τ(i). To account for this, a new measure for clustering was introduced in [22] that calculates triadic closure as a function of degree and neighbor degree. Specifically, the Soffer-Vasquez clustering coefficient ( ) and transitivity ( ) are given by:

where ω(i) measures the number of possible triangles for node i, and N _ω is the number of nodes in G for which ω(i) > 0. We note that and are undefined if ω(G) = Σ _i ω(i) = 0. ω(i) is computed by counting the maximum number of edges that can be drawn among the d _i neighbors of a node i, given the degree sequence of i's neighbors; this value is often smaller than [22]. For example, consider a star network of five nodes, where four nodes have degree 1 and one node has degree 4. Although the total number of triples is τ(G) = 6, the number of possible triangles is ω(G) = 0 because the degree one nodes preclude their formation. The computation of ω(i) must be done algorithmically and is not possible in closed form. (From here on, we refer to as the SV-clustering coefficient and to as the SV-transitivity.)

Generative Model

Here we develop a model to generate a simply connected random graph with a specified degree sequence and a desired level of clustering. Generating random graphs uniformly from the set of simply connected graphs with a prescribed degree sequence is a well-studied problem with algorithmic solutions [37]. One of the simplest and most popular of these generative algorithms was suggested by Molloy and Reed and is known as the configuration model [27]. Given a specific realizable degree sequence [44], {d _i }, this method assigns d _j half-edges to each node j, and then randomly connects pairs half-edges to create edges until there are no half-edges left. (A realizable degree sequence is one which satisfies the Handshake Theorem (the requirement that the sum of the degrees be even) and the Erdos-Gallai criterion (which requires that for each subset of the k highest degree nodes, the degrees of these nodes can be "absorbed" within the subset and the remaining degrees.) Although the model sometimes produces graphs that are not simple or connected, this can be remedied by subsequently removing multiple edges and self loops from the constructed graph and keeping only the largest connected component [37]. Our method begins by using this approach to generate a simple, connected random graph G, with a specific realizable degree sequence D. We then introduce triangles into G using a Markov Chain process without disturbing the degree sequence until we achieve the desired level of clustering, as follows.

Let G _D be the set of all simple, connected graphs with degree sequence D. If are the graphs of G _D , then we let be the states of the Markov chain, P, where X _i represents the state in which our graph G = G _i . The states X _i and X_i+1are connected in the Markov Chain if G _i can be changed to G_i+1with the rewiring of one pair of edges. The state space of the Markov chain P is connected because there exists a path from X _i to X _j (for any pair i, j) by one or more rewiring moves that leave the degree sequence unchanged [45].

Our clustered graph generation algorithm involves starting with the random graph G (generated with the configuration model above) and transitioning from the state corresponding to G (X _G ) to other states of P until a halting condition is reached. A transition from one state of the Markov chain to another only occurs when the algorithm makes an edge rewiring that both increases the clustering of the graph and leaves the graph connected. Since a rewiring does not alter the degree sequence of the graph, the rewired graph is still in G _D . The transition probabilities of the Markov chain for a pair of connected states, X _i to X _j , are:

where clust(G _x ) is a clustering measure for graph G _x , which can be replaced by any of the measures introduced in Section. The algorithm continues searching for a feasible rewiring (one that increases the clustering and does not disconnect the graph) until one is found. If a feasible move is not found, a transition is not made and the process remains in the current state.

The Markov chain above is finite and aperiodic, but not irreducible as the process can never transition to a state in which the graph has lower clustering. It does, however, have an absorbing state, X_*, in which the transitivity of G_* is greater than or equal to the desired transitivity or is the maximum possible transitivity given the particular degree sequence and connectivity constraints.

Algorithm

To generate clustered graphs, we apply the above Markov Chain simulation model by iteratively applying rewirings that increase graph clustering. Each rewiring takes a set of five nodes {x, y₁, y₂, z₁, z₂}, connected by four edges {(x, y₁), (x, y₂), (y₁, z₁), (y₂, z₂)}, and swaps the outer edges: {(x, y₁), (x, y₂), (y₁, y₂), (z₁, z₂)}(illustrated in Figure 1d). This introduces a triangle among nodes {x, y₁, and y₂}, without perturbing the degree sequence. The algorithm proceeds as follows:

Input: A realizable degree sequence {d _i } a desired clustering value, target

Initialization: Generate a random graph G with degree sequence {d _i } (using the configuration model), and measure the clustering of G, clust(G).

while clust(G) <target do

1. uniformly select a random node, x, from the

set of all nodes of G such that d _x > 1.

2. uniformly select two random neighbors, y₁

and y₂, of x such that d_{y 1}> 1 and

   d_{y 2}> 1 and y₁≠y₂.

3. uniformly select a random neighbor, z₁

of y₁ and a random neighbor, z₂ of

   y₂ such that z₁ ≠ x, z₂ ≠ x,

   z₁ ≠ z₂.

4. G _cand : = G where G _cand is the candidate

graph to which the transition may be made.

5. if (y₁, y₂) and (z₁, z₂) do not exist then

Rewire two edges of G _cand : delete (y₁, z₁) and (y₂, z₂), add (y₁, y₂) and (z₁, z₂).

end

6. Update the value of clust(G _cand ) by measuring

   δ (i) (and ω (i) if relevant) for the nodes involved

   in the rewiring and their neighbors.

7. if clust(G _cand ) > clust(G) and G _cand

   is connected then

   G: = G _cand

   end

end

Output: A random graph, G with degree sequence {d _i } and clust(G) ≥ target.

The algorithm terminates when the graph attains at least the desired level of clustering or reaches a threshold number of unsuccessful rewiring attempts. In the latter case, the algorithm returns the graph with the maximum clustering achieved. For practical purposes, a threshold is placed on the number of unsuccessful attempts made by the algorithm in ClustRNet for the case that the desired clustering cannot be reached. Due to the random restarts made at every step, the algorithm is prevented from getting trapped in local minima.

The algorithm is designed to increase clustering while preserving both the degree sequence and connectedness of the graph. However, there are some cases where the desired clustering can only be reached by disconnecting the graph; and thus ClustRNet provides the option of removing the connectivity constraint (see Additional file 1, Figure S2).

Types of Graph Changes

As shown in Figure 2, there are six types of triangles that can be added or removed for every pair of edges that are rewired. As illustrated in Figure 1d, these additions and removals can occur in combination.

• Type A: The addition of the edge between vertices y₁ and y₂ guarantees the addition of one triangle in every rewiring event.

• Type B: The addition of the edge (y₁, y₂) could create new triangles with shared neighbors of y₁ and y₂.

• Type C: The addition of the edge (z₁, z₂) could add a triangle if there existed edges between x and z₁ and x and z₂.

• Type D: The addition of the edge between vertices z₁ and z₂ could create new triangles with shared neighbors of z₁ and z₂.

• Type E: The removal of edges (y₁, z₁) and (y₂, z₂) removes one triangle each if the edges (x, z₁) or (x, z₂) exist.

• Type F: The removal of the edges between vertices y₁ and z₁, and y₂ and z₂ could lead to the removal of existing triangles with shared neighbors of y₁ and z₁ or y₂ and z₂.

We note that although the type A addition is a special case of type B, the type C addition is a special case of type D, and the type E removals are a special case of type F, we distinguish them because they have different probabilities of occurrence. Our look-ahead strategy only allows rewiring moves when the total number of Type E and F losses is fewer than the total number of Type A, B, C, and D gains.

Computational Complexity

Like many heuristic search methods, the algorithm we propose can be computationally expensive. The method outlined in Section 2.2 requires O(M) steps to generate a connected graph, and up to O(M) steps to randomize the graph, where M is the number of edges in the graph. At each step of randomization, we test that the graph remains connected (an O(M) operation), resulting in an overall O(M²) random network generation process. A naive computation of the transitivity/clustering coefficient requires checking every node for the existence of edges between every pair of neighbors of the node. This step requires O( ) operations, where N is the number of nodes and d _max is the maximum degree of any node in the graph. The most expensive step of our algorithm is the introduction of triangles via rewiring. A single rewiring step requires O(M) operations for switching edges, checking for connectivity and updating the clustering measure. Although we cannot analytically calculate the number of attempted rewiring steps required to reach the desired transitivity, we have found it empirically to be O(M). Thus, the average complexity of the clustered network algorithm presented here is O(M²). This complexity has been computed for the most naive versions of our algorithms; and more efficient implementations may improve the complexity greatly. For example, we might improve efficiency by performing connectivity tests once every x rewirings (for some number x) rather than during every rewiring, as proposed in [46].

Performance

To test our algorithm, we generate networks with three different degree distributions and for a range of clustering target values. Specifically, we use Poisson (p _d = e^-λλ ^d /d!), exponential (p _d = (1 - e ^κ )e^-κ(d-1)) and a truncated scale-free (p _d = d^-γe^-d/κ/Li _γ (e^-1/κ)) degree distribution, each with a mean degree of five. Starting with random graphs with specific degree sequences matching these degree distributions, we rewire the networks towards (1) SV-transitivity (( )) targets and (2) transitivity (T) targets in addition to allowing the algorithm to generate disconnected graphs. These targets allow us to evaluate how the clustering measure and connectivity requirement constrain the results, and the second target, in particular, allows us to compare results to other algorithms. Figure 3 illustrates the rewiring of a network with a Poisson distributed degree sequence evolving towards higher transitivity.

We evaluate the performance of our algorithm in comparison to one representative network growth algorithm [30] and one representative bipartite network method [20]. Specifically, we measured the discrepancies between input and output degree distributions (Figure 4 left graphs) and transitivity values (Figure 4, right graphs). Our algorithm preserves the input degree sequence perfectly, while there are considerable mismatches between the input and output degree distributions in the Volz and Newman models. For both comparisons, the transitivity values of the output graphs from our algorithm exactly match the target transitivity values, when those values can be attained given the network topology and the requirements of the algorithm. Some values at the lower end of the clustering scales cannot be reached because the expected transitivity for random graphs of specified degree distributions scales as where p _k is the degree distribution [21, 8, 43]. This value is small for the Poisson degree distribution but can be quite high (especially when measured as SV-transitivity) for highly-skewed degree distributions such as the scale-free degree distribution. For the first comparison, the connectivity constraint imposes a maximum on the attainable clustering value, thus the highest SV-transitivity values cannot be reached without disconnecting the graphs. In these cases, our algorithm returns the graph with the largest attainable SV-transitivity that is less than the desired SV-transitivity. For the second comparison, (with requirements to match the other algorithms), our algorithm performs better in all cases compared to the Volz and Newman models. Due to the definition of the standard transitivity measure (T), however, we see that the networks reach a maximum T value, beyond which no further clustering can be accommodated by the network topology.

Structural Properties of Generated Networks

There are several other topological properties (besides degree sequence and clustering) that can strongly influence network function and dynamics. Among these are degree correlations (the dependence of a node's degree on its neighbors' degrees), community structure (groups of nodes that are highly intra-connected and only loosely inter-connected), and average path length (typical distances between pairs of nodes in the network). We have specifically developed this model to increase clustering with minimal structural byproducts. Thus, we confirm that we have reached this goal by measuring the above properties in the networks generated by our algorithm.

We evaluated the extent to which the algorithm introduces degree correlations by comparing random (unclustered) graphs to clustered random graphs generated by our algorithm and the Volz [30] and Newman [20] algorithms (Figure 5. While our algorithm essentially preserves the correlation structure of the random graph, the other algorithms produce highly correlated graphs. Results are not shown for scale-free graphs as initial transitivity values were larger than 0.5 for all generated graphs.

Several authors have discussed the relationship between clustering and community structure [8, 25, 47, 21]. As Figure 3 shows, the addition of triangles leads to modular structure. This behavior is not surprising: as the number of edges in the graph is constrained, sets of connected nodes with high ω(i) values (often high-degree nodes) must be brought together to create additional clustering. Although the presence of a significant proportion of triangles tends to separate the network into modules, it is not clear that clustering is always sufficient to explain the modular structure of a graph. We explore this further below.

Short average path lengths are a characteristic feature of random graphs [26]. To quantify the impact of our algorithm on path lengths, we calculated the average path length for each node to all other (N - 1) nodes, and then compared the distributions of these values for several random and random clustered graphs (Figure 5). While our algorithm mostly maintains short average path lengths, the mean of the path length distribution does tend to be slightly larger for the clustered graphs than for the corresponding random graphs. The intuition behind this increase in average path length may lie in the increased community structure: as graphs become more clustered and separate into subgroups, nodes in different groups require more links to reach each other (Figure 3). Given that our algorithm can generate graphs of high clustering while preserving short path lengths, this introduces a novel method of generating graphs with the small world property without the correlations of Watts-Strogatz graphs [1].

Application: Analysis of Empirical Networks

It is crucial to have random controls in the study of biological systems. Our algorithm can be used to generate null models and applied to the detection of structure in empirical biological networks. We can generate ensembles of clustered random networks with empirically estimated degree sequences and clustering values to ascertain whether empirical networks have significant non-random structure in other respects. We demonstrate this application using representatives from four classes of biological networks. We also analyze one non-biological network that is made of human transportation links as it provides contrast to the range of topological properties and design principles found in the biologically-motivated networks. The five real networks are as follows: a) a trophic exchange network for the Little Rock Lake in Wisconsin [48]; b) a protein interaction network for yeast [3]; c) a metabolic network for the eukaryote Caenorhabditis elegans [49]; d) a network made up of epidemiologically-relevant contacts for individuals in the city of Vancouver [13]; and e) a transportation network, made up of US metropolitan areas connected by air travel [50]. These networks represent a diverse set of applications and are systems that are well-studied in their respective literatures. The basic statistics of these networks, including clustering values, are listed in Table 1.

We use the following method to quantify deviations from randomness in these networks. First, we use our algorithm to generate 25 clustered random networks constrained to match the empirical degree sequence and clustering values. Second, we select a set of network topological measures (other than degree distribution and clustering), and compare these quantities for the empirical graph to the corresponding average quantities across the ensemble of generated graphs.

Of all the empirical networks analyzed, the random counterparts of the the US air traffic network are the only ones that have structural properties almost identical to the real network (with the network of Vancouver epidemiological contacts being the next closest). This suggests that the structure of the US air traffic network comes almost exclusively from its degree patterns. (In fact, even the high clustering is explained exclusively by the degree patterns.) We note that the US air traffic network is the only non-biological one and the most engineered of the networks we consider, and thus may have fewer emergent properties. The remaining empirical networks (all biological) differ considerably from their random counterparts, suggesting that there are important mechanistic features not captured in the random model.

Degree correlations vary somewhat systematically among the four biological networks (Table 2). The Vancouver human epidemiological contact network has significantly higher degree assortativity than our random networks, thus showing that the positive degree correlations are not just the result of degree distribution or clustering, both of which have been found to be positively correlated with assortativity [53]. This suggests the existence of social rules among humans that go beyond (a) variation in numbers of "friends" and (b) the tendency for "my friend's friend also to be my friend" [11]. The remaining biological networks (the yeast protein interactions, the Little Rock Lake foodweb, and the C. elegans metabolic networks), on the other hand, all have negative degree correlations. Our results show that the C. elegans metabolic network, in particular, has degree correlations approximately equal to the amount expected to arise as a random byproduct of degree distribution and clustering. One reason that a biological network only show random degree correlations might be due to the lack of a clear functional or structural advantage for strong correlations: negatively correlated networks are vulnerable to failures because functionality often depends on a few high degree nodes that provide essential connectivity. If any of these fail (e.g., because of a gene deletion in a metabolic network) the whole system fails [11, 12]. On the other hand, positively correlated networks, which have short distances between hub (high-degree) nodes, may be less favorable because they allow for the propagation of random perturbations (e.g., changes in the concentration of a protein in a protein-interaction network) [36].

All of the natural networks we study have significantly higher modularity than the corresponding clustered random networks, despite having a wide range of transitivity values. This suggests that clustering and community structure are not necessarily positively correlated, as has been previously suggested [52, 8]. The high modularity of the Little Rock foodweb, in particular, has been attributed to its high clustering [54]. Our generated clustered random graphs, however, indicate that the degree distribution and high transitivity only account for about half the modularity of the foodweb graph (Table 2). There is an extensive literature on the presence and evolution of modularity in protein, metabolic, and ecological networks highlighting its possible roles in functional specialization, innovation and robustness [55–60]. Since clustering and the mechanisms that give rise to it cannot fully account for the modularity of these empirical networks, such mechanistic explanations for the structure are warranted.