Biclustering

Biclustering [4], also known as co-clustering or two-mode clustering, is an emerging field of machine learning. In contrast to one-way and two-way clustering, biclustering is a category of algorithms in which the rows and columns are clustered simultaneously. Biclustering is also different from standard clustering because rows and columns may have multiple or no memberships. In this work, we focus on the visual analysis of biclustering results, as the characteristics of overlapping clusters pose a yet unsolved challenge for visualization.

The array of available biclustering methods ranges from algorithms that try to find a single bicluster, to algorithms that seek to find multiple overlapping biclusters. Madeira and Oliveira [5] surveyed different biclustering algorithms with respect to the structure of their output. Bicluster algorithms are often used to analyze gene expression data [6]. In the context of gene expression, a bicluster may correspond to a pathway that is activated in particular samples (the column members) and that contains certain genes (the row members). Each gene may belong to one bicluster, to more than one bicluster, or to no bicluster at all. The same holds for samples.

In general, clustering algorithms can additionally be differentiated by the kind of memberships they produce. In hard clustering, rows and columns are assigned to clusters in a binary way, i.e., they either belong to clusters or not. In soft clustering, the result consists of non-binary membership values that describe to what degree rows and columns belong to the clusters. As the assignment of rows and columns to clusters is fuzzy, this is also known as fuzzy clustering [7, 8].

Bicluster visualization

Let us consider the visualization of hard clustering results first. In order to understand and interpret hard clustering results, it is necessary to visualize the clusters together with the underlying data. Clustered heatmaps are the standard technique for visualizing both one-way and two-way clustering results. In clustered heatmaps, the rows or columns are reordered, such that clusters can be recognized as contiguous blocks consisting of adjacent cells. Showing clusters as contiguous blocks is highly desired, as it simplifies the detection and interpretation of patterns. However, for biclustering results, where clusters can overlap, rearranging the matrix this way is often impossible. Let us consider the example from Figure 1 that shows a 5x5 matrix with three clusters. In Figure 1(a), the columns are sorted such that the red and yellow clusters are represented as contiguous blocks, as indicated by a thick border. However, this sorting splits the blue cluster into two unconnected blocks. In Figure 1(b), columns B and E are swapped, which makes it possible to show the blue cluster as a contiguous block, but splits the red cluster. Consequently, even in small matrices there is often no optimal order of rows and columns where all clusters form contiguous blocks. The sorting problem can be solved by duplicating rows and/or columns, as demonstrated in Figure 1(c). However, the duplication approach does not scale, as it potentially produces large output matrices for comparably small input matrices.

Interpreting biclustering results is often time-consuming and tedious, as it is usually done statically by visually inspecting many separate plots. Adding fuzzy clustering to this equation makes the situation even more difficult.

Fuzzy biclustering is a visualization research problem that cannot be addressed by any of the existing tools. We will first elaborate on how biclustering results can be represented and then introduce the FABIA fuzzy biclustering algorithm [9]. We use FABIA to demonstrate the proposed technique; however, note that any other biclustering algorithm that produces overlapping clusters can be used in the same way. We continue by introducing general requirements for bicluster visualization, which we use to review existing work in this field. We then present Furby, an interactive visualization technique for analyzing fuzzy biclustering results. After a brief description of the implementation, we present how the tool can be used effectively to analyze a real-world dataset. Before concluding the paper, we discuss the scalability of our tool to large datasets.

Representation of biclustering results

Biclustering data can generally be represented by three matrices: X, L, and Z. The X matrix represents the input data to be clustered. The biclustering results are represented by L and Z. The L matrix contains the relationship information between rows and biclusters, and the Z matrix contains the same information for columns. While for hard biclustering results, L and Z hold binary values (1 = row/column is part of a cluster, 0 = row/column is not part of a cluster), for fuzzy biclustering results they contain real values that denote the degree of membership, starting with 0, which means that the row or column does not belong to the considered bicluster.

FABIA biclustering algorithm

FABIA [9] is an established biclustering algorithm which has been successfully applied not only to drug discovery and systems biology, but also to enhance recommender systems. FABIA, as a generative model, is based on factor analysis, but can be considered as a sparse matrix decomposition algorithm. As described above, the observed data matrix X is decomposed into two matrices: the L matrix describes memberships of rows (genes) to biclusters, and the Z matrix describes the memberships of columns (samples) to biclusters. Consequently, a bicluster is described by row and column memberships. The FABIA model assumes that biclusters have only few row and column members. This is the typical situation for gene expression data, where pathways contain only few genes (compared to all genes), which are activated in only few samples. This situation is also typical for recommender systems, where a customer buys only few products, and a certain product combination is chosen only by few customers. Another example is word-document matrices, where a bicluster is a certain topic (a document contains few topics and a topic contains few indicative words). Thus, in all these applications the data matrix is sparse, as are the matrix that describes row memberships, and the matrix that describes column memberships. In FABIA models, this sparsity is reflected by sparse row and column decomposition matrices enforced by sparse priors in a Bayesian framework. FABIA describes row and column memberships by real numbers. Hence, the bicluster memberships are fuzzy, and it is difficult to decide in the "twilight zone" whether a column or a row indeed belongs to a bicluster.

The memberships must often be inspected visually by an expert, who then decides how good the bicluster pattern is (gene pattern of a pathway) and how strong the signal is (gene expression). Assessing the relationship between biclusters is an even more complex task. Do two biclusters partially represent the same information? If yes, which columns and/or rows do they share? To comprehend the information in the biclusters and their mutual dependencies, a visual representation of this information is highly desired.

Requirements

Based on interviews with domain experts and surveying the body of previous work, we have elicited seven requirements that an effective fuzzy bicluster visualization needs to fulfill. We will assess existing work in this field against these requirements. Later sections will demonstrate how our technique addresses these requirements.

Related work on cluster visualization

The key to let analysts gain new insights in large and complex multi-dimensional datasets is to combine the strength of automated algorithmic techniques with the power of interactive visualization [10–13]. Numerous techniques for the interactive visual analysis of clustering results have been proposed over the last years.

In order to discuss interactive cluster visualization techniques, we split up the body of existing work according to types of clustering. The standard technique for oneand two-way clustering is the clustered heatmap, where rows and/or columns are reordered to reflect the similarities. Examples for visual analysis tools that provide interactive heatmaps are Mayday [14], Caleydo [15, 16] and the Dual Analysis framework [17]. For hierarchical clustering results, the clustered heatmap is commonly extended with a dendrogram that represents the similarities between the rows or columns [18]. The Hierarchical Cluster Explorer (HCE) [19] and MultiClusterTree [20] are both approaches that allow interactive analysis of hierarchical clustering results.

However, as mentioned at the beginning of the Bicluster visualization section, for biclustering results it is often not possible to rearrange the matrix in order to represent all clusters as contiguous blocks (see Figure 1), which is essential for interpreting the clusters. A simple approach to visualizing biclustering results is to create a separate plot for each bicluster, as implemented, for instance, in the Biclustering Analysis Toolbox (BicAT) [21], the BiClust R toolbox [21] and the BiVisu tool [22]. Showing every cluster as a separate plot allows analysts to inspect the clusters individually, which addresses requirement R I. However, this makes it impossible to see which rows and columns they share, which violates R II. Jin et al. [23] formulated the reordering issue as an optimization problem and proposed a reordering approach by exploiting analogies to the hypergraph vertex ordering problem. Grothaus et al. [24] propose to duplicate rows and columns to resolve situations where reordering is not possible. The BiCluster viewer [25] follows the same approach, but additionally allows analysts to interactively decide which clusters to show contiguously in order to minimize the number of duplicates. As this can, however, still result in very large matrices, scalability is limited (see R IV).

The work that is probably related most closely to ours is the BicOverlapper tool [26], which presents the biclustering result in a multiple-coordinated view setup. A parallel coordinates view and a heatmap show the individual biclusters, realizing R I. The overlapper view visualizes the bicluster network as a force-directed graph where biclusters are represented as overlapping groups. Although the BicOverlapper tool encodes the overlaps between clusters (R II) and the cluster assignment (R III), it does not scale well to many biclusters (R IV), as it creates occlusion problems, which renders obtaining an overview of the biclustering results as a whole impossible.

Only a small number of articles on visualizing fuzzy clustering results have been published. Most of them propose extensions to classical clustering visualizations, including parallel coordinate plots [27], heatmaps [28], and RadVis [29] - a radial visualization technique, in which membership values are projected to polar coordinates. A similar approach was developed by Rueda and Zhang [30], which maps membership values to a hyper-tetrahedron in the 2D or 3D space representing three or four fuzzy clusters. clusterMaker [31] takes a different approach by representing a one-way fuzzy clustering result as a force-directed graph where the clustered entities are shown as nodes and color is used to encode the cluster membership. However, all these methods focus on the membership or membership values of the rows and columns and ignore the underlying data of the clustering result (see R I).

In summary, none of the existing approaches are able to address the requirements in a satisfactory way. In particular, the visualization of fuzzy biclustering seems to be a blank area in the research map - a blank which Furby attempts to fill.

Cluster network overview

The overview visualization presents the biclustering result as a graph in which individual clusters are the nodes and the rows and columns overlapping between the clusters are the edges. Figure 3 shows an example biclustering result with 20 clusters. We layout the graph using a force-directed algorithm [32] in which overlapping clusters attract each other. The more rows and columns two bicluster share, the bigger is the attracting force. By default, all biclusters repulse each other, resulting in a layout in which clusters with a large overlap form groups.

Bicluster nodes in the graph represent the data as a heatmap, addressing requirement R I. By default, we apply a red-grey-blue color scheme. However, analysts can change and refine the color mapping on the fly during the analysis. The overlaps between biclusters are encoded using bands connecting the biclusters, which satisfies requirement R II. In previous work [33, 34], we have already made use of bands to visualize the relationships between clusters represented as heatmaps in the context of one-way clustering. In Furby, the same approach is applied in both dimensions, rows and columns. The thickness of the bands is proportional to the number of rows and columns shared by the clusters. The bands are attached to the bicluster heatmaps at the position of the shared rows and columns within the heatmap.

Selection and highlighting

Furby supports linking & brushing. Hence, when the user selects one or more rows and columns, all corresponding instances within all clusters and bands are automatically highlighted. This helps analysts to identify how often individual rows and columns are contained in the clusters (see requirement R III).

Keeping the visual clutter to a minimum and letting the analyst focus on the currently selected cluster are important aspects of a scalable visualization technique, formulated in requirement R IV. To achieve this, we use a combination of stubs [35, 36] and fading effects. Stubs are small indicators that replace the bands and point in the direction of the connected cluster, resulting in a significant reduction of visual clutter. While Figure 4(a) shows a screenshot of the regular overview, Figure 4(b) shows the adapted version in which the analyst hovers over a single cluster heatmap. All unconnected clusters within a distance of N hops in the underlying graph are faded out, and in a similar fashion all unconnected bands are replaced by stubs. Using stubs instead of bands lets analysts focus on the local graph neighborhood of the selected cluster of interest. Note that the distance variable N can be manipulated interactively, which allows the user to take smaller or larger portions of the cluster network into account.

In addition to direct interaction with the biclusters, analysts can globally manipulate parameters using a toolbar shown on the right side of the interface (see Figure 3). The toolbar allows users to turn off bands in both dimensions (rows and columns), for instance.

Detail visualization

In contrast to the overview visualization described in the previous section, the detail view focuses on a single bicluster and allows an in-depth exploration of its elements, addressing requirement R I. Analysts can bring a bicluster into the focus by doubleclicking its header, which shows its name. The bicluster will then be scaled up and put in the center of the visualization. The directly connected neighbor clusters are shown as thumbnails in the remaining area. The neighborhood degree, whether a bicluster should be shown in addition to the focused bicluster, will be defined again by the maximum distance parameter N, which can be specified in the toolbar (see Section Selection and highlighting). Figure 5(a) shows an example where N is set to 1. By setting N to 0, only the detail bicluster remains visible, which is useful when the context of a cluster is not of current interest to the user. In the detail mode, analysts can browse through the biclusters by using the left and right arrow keys. By double-clicking the header again, the visualization switches back to the overview cluster network.

The main area of a bicluster node is a multiform visualization, i.e., the applied visualization technique can be switched on demand. By default, we present the bicluster data as a heatmap, as it is the most commonly used technique for this kind of data. Besides the heatmap representation, we provide additional visualization techniques, including a bar chart and a histogram for all values. To ensure the readability of the identifier labels, we use a combination of scrollbars as well as orthogonal stretching, to magnify selected rows and columns. In addition to the actual data shown in the cluster visualization, we add extra rows and columns to visualize metadata as well as membership value information in case of fuzzy clustering results, as described in the next two sections.

Interactive membership value adjustment

Fuzzy biclustering produces soft memberships instead of binary cluster assignments, determining to what degree rows and columns belong to a bicluster. However, to visualize a bicluster, a certain membership value threshold needs to be chosen - this is the process of transforming a fuzzy clustering result into hard clusters. Furby allows analysts to interactively manipulate these thresholds, addressing requirement R VI. Thresholds can be defined independently per dimension, locally per bicluster, or globally via the toolbar. Analysts can use a slider to manipulate the thresholds, while the background of the sliders show a histogram of the underlying membership values. In addition to specifying a threshold value, users can restrict the bicluster to only contain the top M rows or columns.

We show the membership value for each row and column as an additional greyscale bar that is directly attached to the bicluster visualization, as shown in Figure 5. The darker the value, the higher the membership value. This membership value indicator addresses requirement R V. Some bicluster algorithms, including FABIA, produce positive as as well as negative membership values. We indicate 0 by a blue line.

Adding metadata

Often, users want to analyze the clusters in the context of externally loaded metadata, as described in requirement R VII. We enable analysts to load categorical or ordered numerical metadata for rows and columns. Examples for additional data defined on biological samples are gender, age, and tumor staging. In gene dimension, analysts can load metadata such as Gene Ontology (GO) terms [37] and the results of a gene set enrichment analysis [38] performed on KEGG pathways [39]. Furthermore, external cluster assignments can be loaded as additional categories to compare multiple clustering results.

In Furby, we visualize contextual data by attaching additional bars to the cluster visualization. In the case of categorical data, we assign a unique color to each category. For ordered numerical data, analysts can choose from a set of pre-defined color schemes. When the user hovers over an additional metadata bar, we show the name of the category as a tooltip.

Sorting strategy

We enable analysts to define a primary and a secondary sorting criterion for both the rows and columns. The secondary sorting criterion is only used if the first one produces a tie. The sorting is applied to both the overview and the detail visualization. Furby supports the following sorting criteria:

The examples in Figure 5 contain membership value bars in both dimensions and an additional external clustering assignment for the samples. In Figure 5(a) and 5(c), the sample ordering is determined by decreasing membership values, as indicated by the bars from dark to bright. However, this leads to fragmented clustering assignment bars. In Figure 5(b), the sorting is reversed, by grouping the samples according to their external cluster assignment and by the membership value within an assignment category.

Data loading

We integrated a data importer to simplify the loading of a biclustering result. The data can be loaded from CSV files. Three matrices, X, L, and Z, are needed to specify a clustering result. The X matrix contains the actual data, and the L and Z matrices hold the membership values. While in hard biclustering the L and Z matrices contain binary membership values, in fuzzy clustering the values are real numbers, with 0 indicating that rows or columns do not belong to the considered bicluster. Categorial cluster annotations and initial membership threshold guesses can be loaded in addition. Further, we provide an R script [40] for exporting FABIA result objects in the CSV file format required by Caleydo. The script can easily be adapted to load results of arbitrary bicluster algorithms from R.

Force-directed layout

To simplify and speed up the physical simulation of the forces, we internally use ellipses as shapes instead of the actual rectangular bounding boxes of the bicluster nodes. Nodes are positioned automatically following the force-directed layout approach, but they can also be freely repositioned using drag and drop. This way layout issues can be resolved, especially since we apply a damping factor within the layout algorithm to ensure that the layout quickly stabilizes. While this prevents the drifting of nodes caused by rounding errors of the physical simulation, it can produce sub-optimal layout results. However, according to user feedback, a stable layout is preferred over an optimal one which takes longer to be created. Layout stability is particularly important for users to maintain their mental map.