# Robust and automated three-dimensional segmentation of densely packed cell nuclei in different biological specimens with Lines-of-Sight decomposition

- B. Mathew
^{1}Email author, - A. Schmitz
^{1}, - S. Muñoz-Descalzo
^{2}, - N. Ansari
^{1}, - F. Pampaloni
^{1}, - E.H.K. Stelzer
^{1}and - S.C. Fischer
^{1}

**Received: **17 December 2014

**Accepted: **18 May 2015

**Published: **8 June 2015

## Abstract

### Background

Due to the large amount of data produced by advanced microscopy, automated image analysis is crucial in modern biology. Most applications require reliable cell nuclei segmentation. However, in many biological specimens cell nuclei are densely packed and appear to touch one another in the images. Therefore, a major difficulty of three-dimensional cell nuclei segmentation is the decomposition of cell nuclei that apparently touch each other. Current methods are highly adapted to a certain biological specimen or a specific microscope. They do not ensure similarly accurate segmentation performance, i.e. their robustness for different datasets is not guaranteed. Hence, these methods require elaborate adjustments to each dataset.

### Results

We present an advanced three-dimensional cell nuclei segmentation algorithm that is accurate and robust. Our approach combines local adaptive pre-processing with decomposition based on Lines-of-Sight (LoS) to separate apparently touching cell nuclei into approximately convex parts. We demonstrate the superior performance of our algorithm using data from different specimens recorded with different microscopes. The three-dimensional images were recorded with confocal and light sheet-based fluorescence microscopes. The specimens are an early mouse embryo and two different cellular spheroids. We compared the segmentation accuracy of our algorithm with ground truth data for the test images and results from state-of-the-art methods. The analysis shows that our method is accurate throughout all test datasets (mean F-measure: 91 %) whereas the other methods each failed for at least one dataset (F-measure ≤ 69 %). Furthermore, nuclei volume measurements are improved for LoS decomposition. The state-of-the-art methods required laborious adjustments of parameter values to achieve these results. Our LoS algorithm did not require parameter value adjustments. The accurate performance was achieved with one fixed set of parameter values.

### Conclusion

We developed a novel and fully automated three-dimensional cell nuclei segmentation method incorporating LoS decomposition. LoS are easily accessible features that ensure correct splitting of apparently touching cell nuclei independent of their shape, size or intensity. Our method showed superior performance compared to state-of-the-art methods, performing accurately for a variety of test images. Hence, our LoS approach can be readily applied to quantitative evaluation in drug testing, developmental and cell biology.

## Keywords

## Background

Biological processes rely on spatial cell-cell and cell-matrix interactions and are highly influenced by the microenvironment of the cells [1–3]. A large number of biomarkers and dyes are available to label distinct cellular structures. Well-defined protocols enable the observation and quantification of dynamic processes of cells within intrinsically three-dimensional structures. Several techniques have been explored to track cells *in vivo* or *in vitro*. The classical method is fluorescent labelling of cell nuclei [4, 5] and visualization by advanced three-dimensional fluorescence microscopy. This method enables the localization of cell nuclei and is also utilized to assess cell viability.

Optical imaging has experienced a significant progress during the past 20 years [6]. The most widely applied advanced three-dimensional fluorescence imaging technique is confocal microscopy [7, 8]. Light sheet-based fluorescence microscopy (LSFM) such as single plane illumination microscopy (SPIM) [9, 10] and digital scanned laser light sheet-based fluorescence microscopy (DSLM) ([11]) are becoming increasingly popular. They have demonstrated their applicability for imaging biological specimens ranging from a single cell to entire animals ([12, 13]). However, fluorescence images produced by all microscopes are affected by their optical properties. The brightness as well as the signal-to-noise ratio in images of large, thick and light-scattering multicellular specimens decreases with the penetration depth [14]. Furthermore, the axial resolution is at least three times worse than the lateral resolution, which often causes objects in the image to apparently touch one another.

The availability of three-dimensional microscopes for three-dimensional biological specimens has pushed the development of three-dimensional image segmentation methods. Three-dimensional cell nuclei segmentation methods commonly rely on pre-processing steps (e.g. filtering, initial thresholding or seed detection) combined with watershed ([15–17]), graph cut ([18, 19]), machine learning ([20, 21]), gradient flow tracking [22], active surface models [23], level set [24], or concavity-based segmentation ([25, 26]). But despite the continual progress in three-dimensional cell nuclei segmentation, there is still a need to improve accuracy, level of automation and adaptability.

A major challenge for segmentation is the separation of densely packed or apparently touching cell nuclei. Another difficulty arises from the diversity in terms of the imaged biological specimens or the imaging techniques used for acquisition. In general, segmentation methods are adopted to either a specific specimen (e.g. mouse embryo [8], *C. elegans* [27] or zebra fish [28]) or a specific imaging technique (e.g. DSLM or confocal microscope [29–31]); or their accuracy has only been tested for one particular application ([32, 33]). To achieve a satisfactory performance of such a method for images that contain another specimen or were obtained by a different imaging technique, sets of parameter values need to be optimized.

To tackle these problems and establish a method that does not rely on signal intensity, heavy parameterization or effective initialization of an algorithm, we chose approximate convex decomposition as the basis of our cell nuclei segmentation method. In the literature, several approaches exist for separating objects into approximately convex parts. However, none of them have been applied to biological images. In Lien’s and Amato’s approach [34], first the most concave regions are identified, and then they are partitioned such that the concavity is reduced below a specified threshold - resulting in approximately convex parts. Another approach is based on greedy region growing of a surface part until the distance to its convex hull decreases below a given threshold [27]. This again yields approximately convex parts. In Attene et al. [35], a shape is represented by a tetrahedral mesh. Decomposition is achieved by calculating a hierarchy of convex polyhedra that tightly enclose the shape. In a bottom-up manner, a single polyhedron is clustered into approximately convex parts.

In the work of Asafi *et al*. [36], approximate convex decomposition is achieved by analyzing pairs of surface points of a shape that are visible to each other. Lines between such mutually visible pairs of points are called “Lines-of-Sight” [36]. Neither a tetrahedralization nor the convex hull have to be calculated for an object. Therefore, Lines-of-Sight are easily accessible features for which neither shape and size information nor intensity distributions have to be considered. We apply this segmentation method to biological images for the very first time.

We combine the Lines-of-Sight (LoS) concept with a local adaptive pre-processing to separate apparently touching cell nuclei into approximately convex parts representing single cell nuclei. We show that our LoS implementation accurately segments three-dimensional cell nuclei due to effective separation of apparently touching cell nuclei. The accuracy of our proposed method exceeded 86 %. Furthermore, volumes of split cell nuclei were well represented. In a direct comparison with methods that are based on a graph cut algorithm (*FARsight*), seeded watershed transformation (ImageJ *3D Watershed*) or machine learning (*ilastik*) – commonly used techniques for three-dimensional cell nuclei segmentation – we demonstrate overall superior segmentation performance and volume measurements. Thereby, our automated LoS approach is applied with the same set of parameter values for real test images that differ in terms of biological specimen, size, imaging technique, and quality. Together, this demonstrates the high accuracy, adaptability and applicability of the proposed method.

## Method

### Approximately convex decomposition with Lines-of-Sight

*et al*. [36] and was adapted to cell nuclei. Two surface points of a shape are said to be in line-of-sight if the connecting line between those two points does not leave the inner volume of the shape. Such mutually visible points are in a convex position. Hence, an object is convex if all surface points are in a convex position. If few surface points are not in a convex position, an object is approximately convex. Fig. 2 illustrates Lines-of-Sight for a given surface point. As evident, not all pairs of points that are in line-of-sight belong to a connected part of the shape surface. Thus, this object is not convex.

Using this definition, we cluster surface points that are in line-of-sight. To ensure robust clustering we determine an initial guess of the number of clusters. For a number of *k* clusters, each cluster represents a first approximation of an individual cell nucleus. Subsequently, this rough estimation is optimized such that we obtain clearly separated cell nuclei. The method was implemented as a processing pipeline for three-dimensional gray level images. The raw images pass through four main stages: (1) local adaptive binarization, (2) collecting connected components, (3) determining the number of divisible parts and (4) decomposition of components into approximately convex parts with LoS.

### Algorithm

#### Pre-processing

- (1)
**Local adaptive binarization.**Foreground and background are separated by local intensity-based thresholding performed per slice. At each pixel, Otsu’s clustering method is applied to compute an intensity threshold within a defined neighborhood range of 18 pixels. Within this neighborhood, each pixel value above the local threshold is considered as foreground. This procedure returns an initial binary image, which might contain artefacts outside the region of interest (ROI). Therefore, a binary mask of the ROI is constructed by using a maximum filter with a large radius on the rough binarized raw image. Consequently, the ROI voxels are assigned a one and voxels outside the ROI a zero. Multiplication of the initial binary image with the ROI mask eliminates voxels outside the ROI. This yields an improved binary image. In a second step, holes within foreground objects are filled. Finally, a morphological opening removes objects that are smaller than a spherical structuring element of radius five. The resulting binary image provides the basis both for determining the number of divisible parts and for the decomposition of apparently touching nuclei. - (2)
**Collect connected components**. Connected foreground components of the given binary image are extracted and stored together with their bounding box specifying their exact location in the image. Each extracted foreground component corresponds to a possible clump of apparently touching cell nuclei and is individually processed by the LoS algorithm. - (3)
**Determine the number of divisible parts.**To determine the number of divisible parts for each connected component, we implemented an automatic detection method based on the Euclidean distance transformation. Thereby, the value of each voxel is replaced by its Euclidean distance to the closest background voxel. Consequently, the minimum distance from each voxel to the surface is returned. We determine the number of divisible parts by detecting local maxima in the transformed image that exceed the values of adjacent voxels by at least 0.1. For a convex object the local maximum is equivalent to the centroid. Objects that touch each other exhibit dents at split sites. Therefore, for a cell nuclei clump we detect one local maximum for each approximately convex part. Hence, the number of local maxima gives an approximation of the number of nuclei in a clump. An object with exactly one maximum represents a single cell nucleus.

#### LoS decomposition

- (i)
**Define surface points.**All foreground voxels that are directly adjacent to a background voxel are set to be the surface points of an object. - (ii)
**Define lines between pairs of surface points.**Pairs are formed from the determined surface points. To ensure good accuracy as well as fast processing, we perform a sampling. Thereby, the sample size of pairs depends on the number of surface points that are collected. We define a lower bound of 1000 surface points and an upper bound of 100000 surface points. If the number of surface points is below 1000, 10 % of the surface points are sampled. Exceeding 100000 surface points decreases the sample size to 2 %. If the number of surface points is in between the lower and upper bound, 3 % of the surface points are considered. Lines between pairs of surface points are generated. - (iii)
**Select LoS**. To determine whether a line between two surface points defines a Line-of-Sight, we check whether the line leaves the object. Points are sampled along each line at an interval of 0.1 % of its length and for each sample point we check whether this point is part of the foreground. If this is true, such a line is considered to not leave the inner volume of the object and thus defines a Line-of-Sight. All other lines are not LoS and are discarded. - (iv)
**Cluster LoS**. We perform a bottom-up hierarchical agglomerative clustering (HAC) of the LoS to subdivide them into*k*clusters, where*k*is given by the number of divisible parts determined in (3). The HAC algorithm always starts with each Line-of-Sight in one cluster. In each step, the similarity between two lines is computed with the chessboard distance and nearest clusters are fused using Ward’s method until*k*clusters are formed [37]. - (v)
**Label surface points**. Assuming the number of detected divisible parts for a nuclei clump is*k*, HAC outputs*k*clusters. LoS in the same cluster are assigned a common cluster label. LoS with different labels can originate from the same surface point. Therefore, the label of the surface point is set to the most common cluster label of the lines originating from that point. In case of an equal distribution of cluster labels, the cluster label is chosen randomly from the respective labels. - (vi)
**Optimize surface points labelling**. To achieve a labelling of the surface points such that the points in the individual parts are geometrically connected, i.e. form a coherent labelled surface and therefore an individual object, an optimization of the previous surface points labelling is performed. Therefore, each surface point is assigned the most common surface point label of the eighteen nearest surface points. - (vii)
**Label voxels**. Surface points labelling subdivides the surface of an object. To separate the whole object, we label the inner voxels according to the labelled surface points. For each foreground voxel the eighteen nearest surface points are determined. The voxel label is chosen as the most common label of these surface points. In case of an equal distribution of surface point labels, the label is determined randomly from the surface point labels found. The labelled voxels are then used to create a component matrix where each individual object obtains a unique identifier.

All previously described steps are performed for each nuclei clump. Subsequently, the already individual nuclei and the decomposed cell nuclei are reassembled and combined to a component matrix. Feature measurements such as the centroids and volumes of the objects are computed for each nucleus and stored in an Excel sheet.

#### Implementation

The method is implemented in *Mathematica* language (Version 9) and uses the *ImageJ* plugin for automatic local thresholding [38]. The source code is freely downloadable from [39]. Data processing using the complete pipeline with the test datasets took 8916 s for the mouse embryo, 3408 s for the breast cancer spheroid and 26487 s for the pancreatic cancer spheroid on a 2.67 GHz 12-core Intel Xeon E5650 with 96 GB of RAM. We expect to continue improving the performance of the LoS pipeline.

## Results

### Ethics statement

All mouse work was approved by the University of Bath Animal Welfare and Ethical Review Body (AWERB) and undertaken under UK Home Office license PPL 30/3219 in accordance with the Animals (Scientific Procedures) Act incorporating EU Directive 2010/63/EU. The human cell lines used in this study are listed in the *American Type Culture Collection* (ATCC) and therefore raise no ethical concerns (ATCC number for T47D: HTB-133 and for BxPC3: CRL-1687).

### Real test image data and ground truth data

### Comparison with state-of-the-art methods

The segmentation accuracy was not only compared with ground truth data, but also with results from the state-of-the-art segmentation methods *FARSight* ([40], [18]), *3D Watershed* in ImageJ ([41], [42]) and *ilastik* ([43],[20]). These are commonly used, open-source segmentation tools [6]. Additionally, we tested several other methods described in the literature, e.g. the method presented in [22], a three-dimensional cell nuclei segmentation named *“CellSegmentation3D”* based on gradient flow tracking and the software MINS [8]. All methods were supplied with the same raw images. A major drawback of many segmentation methods is their incapability to process huge datasets. MINS segmentation failed for the pancreatic spheroid and *“CellSegmentation3D”* as well as the command line tool for *FARSight* failed for all datasets, since they crashed during processing. Due to this issue with *FARSight’s* command line tool, we had to revert to the corresponding graphical user interface *“NucleusEditor”*. Furthermore, the segmentation tools do not support all image types. *“NucleusEditor”* e.g. did not operate with 16 bit images. Since the default parameter values of *FARSight* resulted in unsatisfactory segmentation results, we screened multiple sets of parameter values and chose the most reasonable one for each test image.

*3D Watershed* is a plugin for a seeded watershed within the 3D ImageJ suite. We used the automated seed detection implemented in this plugin. Further details are given in Additional file 1. Similar to *FARSight,* the parameter values had to be adjusted because the default values resulted in unsatisfactory segmentation results. For the machine learning-based segmentation tool *ilastik*, a training dataset had to be created for each test image. In contrast, the proposed LoS algorithm does not require a training dataset and uses the same parameter values for all test images.

### Accurate segmentation performance with LoS decomposition

*FARSight*,

*3D Watershed*and

*ilastik*in three dimensions and in a single two-dimensional slice. Additional file 2: Figure S2) shows further details of segmentation results along XZ and YZ.

*Ilastik*failed for all datasets.

*3D Watershed*and

*FARSight*suffered from over-segmentation (Fig. 5a and c and Additional file 2: Figure S2C). Furthermore,

*3D Watershed*segmentation resulted in nearly straight borders between cell nuclei (Fig. 5b and Additional file 2: Figure S2B). For all other methods the shapes of the splitting sites seem more natural. Note that cell nuclei that appear split incorrectly in the depicted two-dimensional slices are found to be correctly split if one considers the XZ direction (Fig. 6). Fig. 7 shows three-dimensional renderings of single clumps of apparently touching cell nuclei after decomposition with LoS. Clumps with nuclei of different sizes or shapes are similarly well split as clumps of homogenously sized or shaped nuclei. Furthermore, small and large clumps are split equally well.

For each segmentation method true positive (TP), false positive (FP) and false negative (FN) represent the number of correctly detected, falsely detected, and undetected nuclei relative to the GT [32].

To calculate these four metrics, we matched the centroids determined by the segmentation to centroids found in the GT. To do this, a three-dimensional spherical neighborhood of a range that corresponds to the average diameter of the respective cell nuclei was centered at each centroid position in the GT. If this local neighborhood included exactly one centroid in the segmentation, we counted it as a correct detection – thus true positive (TP). If the neighborhood included more than one centroid, the closest one was considered as TP. These calculations resulted in a TP score for each three-dimensional test image. Based on the number of TP, scores for false positive (FP) and false negative (FN) were obtained using FP = n_{seg} - TP and FN = n_{gt} - TP, where n_{seg} and n_{gt} are the number of centroids determined by the segmentation and centroids in the GT for each dataset [32].

*FARSight*,

*3D Watershed*, and

*ilastik*for all three test images. We observed that for recall all methods achieved comparable results. Hence, the number of detected nuclei for each method was similar to the number of cell nuclei in the GT. Thus, there was not much under-segmentation. However, compared to the LoS approach, all other methods exhibit lower precision. Specifically, we observed that single cell nuclei are mistakenly split into multiple parts, increasing the over-segmentation rate. The performance of the LoS algorithm was steadily accurate for the different datasets. In the worst case, we obtained an F-measure of 86 %, whereas the lowest F-measure for the other methods was 69 %, 34 %, and 5 % for

*FARSight*,

*3D Watershed*, and

*ilastik*, respectively (see Table 1).

Algorithmic performance of LoS, *FARSight*, *3D Watershed* and *ilastik*

Dataset | # cells GT | Algorithm | # cells Seg | Match | Recall | Precision | Accuracy | F-measure |
---|---|---|---|---|---|---|---|---|

| 61 | LoS | 59 | 58 | 0.95 | 0.98 | 0.94 | 0.97 |

FARSight | 62 | 60 | 0.98 | 0.97 | 0.95 | 0.98 | ||

3D Watershed | 299 | 61 | 1.00 | 0.20 | 0.20 | 0.34 | ||

ilastik | 274 | 61 | 1.00 | 0.22 | 0.22 | 0.36 | ||

| 240 | LoS | 247 | 216 | 0.90 | 0.87 | 0.80 | 0.89 |

FARSight | 338 | 236 | 0.98 | 0.70 | 0.69 | 0.82 | ||

3D Watershed | 288 | 220 | 0.92 | 0.76 | 0.71 | 0.83 | ||

ilastik | 112 | 74 | 0.31 | 0.66 | 0.27 | 0.42 | ||

| 531 | LoS | 690 | 523 | 0.98 | 0.76 | 0.75 | 0.86 |

FARSight | 997 | 524 | 0.99 | 0.53 | 0.52 | 0.69 | ||

3D Watershed | 734 | 518 | 0.98 | 0.70 | 0.69 | 0.81 | ||

ilastik | 19744 | 531 | 1.00 | 0.03 | 0.03 | 0.05 |

These results clearly demonstrate that our proposed segmentation method is capable of achieving a consistent segmentation performance for diverse datasets compared to an ideal ground truth. Additionally, no adjustments of parameter values are required, thus making the method robust for these datasets and straightforward to use.

To investigate the contribution of our pre-processing steps and the LoS decomposition to the performance of the whole LoS pipeline, we performed a more detailed comparison with *3D Watershed.* We considered two additional cases for *3D Watershed*: (1) we used the binary image generated by the LoS pipeline and applied the distance transformation implemented in ImageJ and (2) we supplied the binary image and the seeds both generated by the LoS pipeline. Comparison showed that the nuclear decomposition capabilities are similar (see Additional file 1: Table S1). Using either the binary image generated by the LoS pipeline as input or the local maxima from our detection of divisible parts as seeds lead to an improvement of the performance of *3D Watershed*. Although MINS crashed for the pancreatic spheroid, we analyzed the performance for the two other datasets. Results are summarized in Additional file 3: Table S2 shows similar performance for LoS and MINS.

### Reliable nuclear volume representation with LoS decomposition

^{3}(

*n*= 6), breast cancer spheroid: 1260 ± 426 μm

^{3}(

*n*= 20), pancreatic cancer spheroid: 271 ± 102 μm

^{3}(

*n*= 50)). We found that for the mouse embryo, the mean nuclear volume achieved by

*FARSight*segmentation was closer to the mean GT volume than the mean nuclear volume after LoS decomposition (Fig. 8a). However, after the LoS approach the nuclear volumes were more consistent with the ground truth than the result from

*FARSight*as indicated by a smaller standard deviation (Fig. 9a; LoS: 257 ± 86 μm

^{3}(

*n*= 59),

*FARSight*: 439 ± 180 μm

^{3}(

*n*= 62)). We observed, that compared to the standard deviation of the GT, the standard deviation of LoS was increased by 20 %. The other methods showed an increase in the standard deviation by 54 % for

*FARSight*, 189 % for

*3D Watershed*, and 8118 % for

*ilastik*.

For the spheroids, the nuclear volumes from our LoS algorithm reproduced the true nuclear volumes better than all other three methods (Figs. 8b and c, 9b and c). *FARSight* and *3D Watershed* produced enlarged nuclei compared to the GT. *Ilastik* throughout failed to resemble the volumes for all datasets due to its lacking capability of separating apparently touching cell nuclei. To show that our LoS algorithm yields a significant improvement over the other methods, a Wilcoxon rank sum test with Holm’s correction was performed. Thereby, nuclear volumes achieved by LoS were always significantly different from those determined by all other methods (*p*-value < 0.01).

In summary, our results show that nuclear volumes obtained by the LoS method are more consistent with the ground truths than all other tested methods. Additionally, less outliers indicate the robustness of the method. These results demonstrate that LoS decomposition yields nuclear volumes that resemble the actual volumes very closely.

## Discussion

We developed a segmentation pipeline for cell nuclei in three-dimensional fluorescence images. The algorithm consists of local adaptive pre-processing followed by decomposition into approximately convex objects. Our detailed analysis revealed the contributions of the two parts to the robustness of the algorithm with respect to differences in image properties like the biological specimen, imaging technique, signal quality, intensity distribution and size of the cell nuclei. These major parts are discussed in the following sections.

## Pre-processing

The key steps in our pre-processing are a local adaptive binarization and the determination of the number of divisible parts.

During binarization, many regions are difficult to classify as foreground or background in images that exhibit background noise or differences in contrast and intensity. For such cases, local thresholding is more appropriate and robust than global thresholding. Applying Otsu’s clustering method locally yielded fewer clumped nuclei than global thresholding. Additionally, for the chosen binarization method only a single parameter value had to be adjusted: the radius that defines the neighborhood during local Otsu thresholding. A value for the radius that is too high can increase the number of apparently touching nuclei and decrease the total number of extracted cell nuclei. We found that a value that corresponds to the approximate radius of a cell nucleus yielded the best results. It is an intuitive feature of the biological specimen. In contrast, pre-processing implemented in e.g. *3D Watershed* involves non-intuitive adjustments of parameter values like an intensity threshold for the image (see Additional file 1: Figure S1). Defining intensity thresholds is typically image dependent and therefore sensitive to the signal-to-noise ratio of the image [22]. Such parameters need to be carefully set to avoid poor segmentation results [45]. Besides the easy and intuitive parameterization for this step, our approach is insensitive to non-uniform intensity distributions as well as the decreasing signal quality with increasing penetration depth into the specimen (Fig. 4). Differences in terms of image origin and image quality is compensated by this. Summarizing the above, this part of the pre-processing is robust if cell nuclei sizes do not differ excessively. Thus, we obtained good results for all our test datasets. Further improvements are possible. By changing the parameter value for the local neighborhood radius e.g. for the pancreatic cancer spheroid to 30 voxels, yields a segmentation improvement of about 7 % (F-measure 93 %) compared to the result shown in Table 1.

The determination of the number of divisible parts of an object can be responsible for over- and under-segmentation. A small value for the local maxima detection in the distance transformed image increases the number of local maxima and hence the number of potential cell nuclei. We found our value gives an accurate and robust initial guess of the number of divisible parts. In our algorithm, the number of divisible parts determines the number of LoS clusters. However, it is only an upper threshold for the number of segmented cell nuclei due to the steps that follow the LoS clustering. These ensure an optimized number of cell nuclei in the final output. If the cell nuclei in the biological specimen exhibit a homogeneous size distribution, an improvement of the detection of divisible parts may be possible by adding an optimization that considers reasonable size descriptors. In this case, huge cell nuclei clumps for which not enough divisible parts are detected, would be split into parts of an appropriate size.

Consequently, our implementation of the intuitively parameterizable binarization and the calculation of the number of divisible parts with the potential of a subsequent optimization, contribute to the robustness of our approach.

### LoS decomposition

The fundamental difference between our decomposition method and existing methods is the application of lines that define approximate convex parts. The robustness is achieved because LoS are easily accessible features of cell nuclei clumps that do not require previous knowledge about object shape, size or intensity. Hence, a good throughout performance of the decomposition of cell nuclei for all datasets was obtained with the same set of parameter values.

Clumps containing nuclei of different sizes or shapes are equally well split as clumps consisting of equally sized or shaped parts (Fig. 7). This is especially advantageous in the case of homeostatic tissues where the stem cell nuclei are in general smaller than the differentiated cell nuclei. Furthermore, for separating complex clumps of many nuclei our LoS algorithm exhibits a better performance than e.g. *3D Watershed*, *ilastik* or methods used in [46] or [25] (Fig. 7, most right).

One reason for the robustness is, that the decomposition is performed directly on the three-dimensional objects rather than applying it on each two-dimensional slice independently like in [47] or [19]. Incorporating the three-dimensional information provides more accurate decomposition of apparently touching cell nuclei. Assuming we would have considered the slice in Fig. 6 independently, the object that is depicted in the smaller inset would not have been separated since it is already approximately convex. However, the view along the XZ direction in the larger inset clearly shows that this object had to be split. Although here, we focus on three spatial dimensions, the method works well for two-dimensional images and has the potential of being extended to three-dimensional time-lapse images.

If the split site of an object is pronounced, a reliable LoS clustering is guaranteed. Clumps with small concavities (i.e. wide split sites) are challenging. In this case, more LoS can pass through the split site and cause mis-clustering of the lines. To overcome this effect, we have introduced a parameter for the sampling size of LoS and the steps “label surface points and optimize surface points labelling” that follow the LoS clustering. The number of sampled LoS determines the spacing between the lines within an object. Lines with a small distance between them are clustered together with a higher probability. The more lines are sampled, the more pass through the wide split site and thus are clustered together. Consequently, surface points of one part of the object are hit more often by LoS with different labels. This can lead to a wrongly labelled surface point. If this happens for too many surface points, it is hard to eliminate them even with the steps following the clustering. However, too few lines would not cover the object and therefore would not be sufficient for an accurate clustering. Therefore, we introduced a LoS sample size that changes if the number of surface points drops below a lower threshold or exceeds an upper threshold. Nonetheless, the algorithm is less sensitive to the values for the lower and upper threshold than to the LoS sample size. For all three parameters, we chose values that were equally applicable to the various test images.

In general, the surface points labelling works well since a surface point is hit more often by LoS with the correct label. Optimization of the surface points labelling eliminates the few remaining mis-labelled surface points that are surrounded by correctly labelled surface points.

To sum up, the surface points labelling and the optimization following the LoS clustering improve the decomposition performance. The sample size of the lines is one key parameter for the LoS decomposition which can affect the segmentation output. This value can be adapted to further improve the results. In general, we found that our value enables a robust decomposition for all our test images.

### Complete LoS pipeline

Applying the entire pipeline as depicted in Fig. 3 on the different test datasets revealed a throughout robust performance with the same set of parameter values. In contrast, the other methods were supplied with adjusted parameter values for each image. Although each applied method failed for at least one dataset, *FARSight* and LoS achieved similar results for the mouse embryo (see Table 1). Nevertheless, cell nuclei volume measurements of our LoS algorithm are more consistent with the manually obtained volumes than those determined by *FARSight*. This shows that the segmentation with *FARSight* worked accurately for the mouse embryo but it produced a cell nuclei volume distribution that did not reflect the ground truth volume.

For *3D Watershed* pre-processing can be decoupled from the cell nuclei decomposition step. Improved results of watershed segmentation with parts of our pre-processing reveal that our pre-processing steps are more robust for our test datasets than those implemented in the *3D Watershed* plugin (see Table S1 in Additional file 1). Moreover, for *3D Watershed* the number of seeds and the number of cell nuclei in the output are exactly the same. This requires a more accurate seed detection as is needed for our LoS approach. Our end result does not only rely on the initial number of divisible parts. The additional optimization steps after LoS clustering decouple the final result from the initial calculation, and thereby contribute to the robustness and accuracy of our algorithm.

In our pipeline, we discarded segmented cell nuclei below a defined volume threshold, a post- processing step that is applicable to a wide range of biological specimens. However, our segmentation method extracts further properties such as the centroids or the fluorescence intensity of the cell nuclei. If only a specific biological specimen is considered, this information can be used for more sophisticated post-processing steps.

## Conclusion

The automated, robust and accurate detection of a single cell nucleus is a crucial and challenging step for a precise quantification in many biological applications. Up to now, several three-dimensional cell nuclei segmentation methods exist but in most cases they need further adjustments to the considered image data and biological specimen. We introduce a three-dimensional segmentation method that incorporates separating apparently touching cell nuclei. It combines local adaptive pre-processing with a Lines-of-Sight approach for approximately convex decomposition. Our method is readily applicable to three-dimensional microscopy images of different biological specimens acquired by diverse imaging techniques.

The segmentation quality is largely independent of cell type, size, intensity, cell density, and image quality. A comparison with state-of-the-art algorithms revealed an overall better performance of our method in terms of identification of individual cell nuclei and extraction of relevant features such as the nuclear volume.

Three-dimensional cell nuclei segmentation with our LoS method provides the crucial starting point for many applications. Cellular spheroids e.g. serve as in vitro tissue models, especially in tumur biology. They show an improved resemblance of the spatial arrangements compared to two-dimensional cell cultures [48]. Therefore, they have become increasingly important for drug testing assays. Volume changes of cell nuclei within such spheroids upon drug treatment might provide insights into drug efficacy. Developmental systems such as mouse embryos serve as a good model for exploring the coordination of cell linage specification and morphogenesis [49]. Thereby, a reliable identification of cell nuclei positions facilitates accurate tracking and lineaging of cell nuclei during different processes and hence opens up a variety of possibilities for investigating cellular dynamics. In general, the wide applicability of our proposed segmentation method enables a reliable and robust quantification of cell nuclei in fluorescence images that will contribute to our knowledge in many biological applications.

## Declarations

### Acknowledgements

Research in the Stelzer lab is supported by the Deutsche Forschungsgemeinschaft (DFG, EXC-115). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

## Authors’ Affiliations

## References

- Bissell MJ, Hines WC. Why don’t we get more cancer? A proposed role of the microenvironment in restraining cancer progression. Nat Med. 2011;17:320–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Harjanto D, Zaman MH. Matrix mechanics and receptor-ligand interactions in cell adhesion. Org Biomol Chem. 2010;8:299–304.View ArticlePubMedGoogle Scholar
- Sutherland RM. Cell and environment interactions in tumor microregions: the multicell spheroid model. Science. 1988;240:177–84.View ArticlePubMedGoogle Scholar
- Day RN, Schaufele F. Imaging molecular interactions in living cells. Mol Endocrinol. 2005;19:1675–86.View ArticlePubMedPubMed CentralGoogle Scholar
- Ponomarev V. Nuclear imaging of cancer cell therapies. J Nucl Med. 2009;50:1013–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Eliceiri KW, Berthold MR, Goldberg IG, Ibáñez L, Manjunath BS, Martone ME, et al. Biological imaging software tools. Nat Methods. 2012;9:697–710.View ArticlePubMedPubMed CentralGoogle Scholar
- Pawley J. Handbook of Biological Confocal Microscopy. 3rd ed. New York: Springer Science+Business Media, LLC; 2006. p. 988.View ArticleGoogle Scholar
- Lou X, Kang M, Xenopoulos P, Muñoz-Descalzo S, Hadjantonakis A-K. A rapid and efficient 2D/3D nuclear segmentation method for analysis of early mouse embryo and stem cell image data. Stem Cell Reports. 2014;2:382–97.View ArticlePubMedPubMed CentralGoogle Scholar
- Huisken J, Swoger J, Del Bene F, Wittbrodt J, Stelzer EHK. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science. 2004;305:1007–9.View ArticlePubMedGoogle Scholar
- Greger K, Swoger J, Stelzer EHK. Basic building units and properties of a fluorescence single plane illumination microscope. Rev Sci Instrum. 2007;78:23705.View ArticleGoogle Scholar
- Stelzer EHK. Light-sheet fluorescence microscopy for quantitative biology. Nat Methods. 2014;12:23–6.View ArticleGoogle Scholar
- Pampaloni F, Richa R, Ansari N, Stelzer EHK. Advanced Fluorescence Microscopy, Methods and Protocols. New York: Springer Science+Business Media, LLC; 2015. p. 43–57.Google Scholar
- Strobl F, Stelzer EHK. Non-invasive long-term fluorescence live imaging of Tribolium castaneum embryos. Development. 2014;141:2331–8.View ArticlePubMedGoogle Scholar
- Keller PJ, Stelzer EHK. Quantitative in vivo imaging of entire embryos with digital scanned laser light sheet fluorescence microscopy. Curr Opin Neurobiol. 2008;18:624–32.View ArticlePubMedGoogle Scholar
- Malpica N, de Solórzano CO, Vaquero JJ, Santos A, Vallcorba I, García-Sagredo JM, et al. Applying watershed algorithms to the segmentation of clustered nuclei. Cytometry. 1997;28:289–97.View ArticlePubMedGoogle Scholar
- Gniadek TJ, Warren G. WatershedCounting3D: a new method for segmenting and counting punctate structures from confocal image data. Traffic. 2007;8:339–46.View ArticlePubMedGoogle Scholar
- Wählby C, Bengtsson E. Segmentation of cell nuclei in tissue by combining seeded watersheds with gradient information. Springer Verlag Berlin Heidelb. 2003;408–414.Google Scholar
- Al-Kofahi Y, Lassoued W, Lee W, Roysam B. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans Biomed Eng. 2010;57:841–52.View ArticlePubMedGoogle Scholar
- Qi J. Dense nuclei segmentation based on graph cut and convexity-concavity analysis. J Microsc. 2014;253:42–53.View ArticlePubMedGoogle Scholar
- Sommer C, Straehle C, Kothe U, Hamprecht FA. Ilastik: Interactive learning and segmentation toolkit. IEEE Int Symp Biomed Imaging From Nano to Macro. 2011;2011:230–3.Google Scholar
- Chen C, Wang W, Ozolek JA, Lages N, Altschuler SJ, Wu LF, Rohde GK. A template matching approach for segmenting microscopy images. In International Symposium on Biomedical Imaging; 2012:768–771.Google Scholar
- Li G, Liu T, Tarokh A, Nie J, Guo L, Mara A, et al. 3D cell nuclei segmentation based on gradient flow tracking. BMC Cell Biol. 2007;8:40.View ArticlePubMedPubMed CentralGoogle Scholar
- Delgado-Gonzalo R, Chenouard N, Unser M. Spline-based deforming ellipsoids for interactive 3D bioimage segmentation. IEEE Trans Image Process. 2013;22:3926–40.View ArticlePubMedGoogle Scholar
- Harder N, Bodnar M, Eils R, Spector DL, Rohr K. 3D segmentation and quantification of mouse embryonic stem cells in fluorescence microscopy images. In: Bildverarbeitung für die Medizin. Berlin Heidelberg: Springer; 2011. p. 34–8. Informatik Aktuell.Google Scholar
- Yu D, Pham TD, Zhou X. Analysis and recognition of touching cell images based on morphological structures. Comput Biol Med. 2009;39:27–39.View ArticlePubMedGoogle Scholar
- Zhong Q, Zhou P, Yao Q, Mao K. A novel segmentation algorithm for clustered slender-particles. Comput Electron Agric. 2009;69:118–27.View ArticleGoogle Scholar
- Chen L, Chan LLH, Zhao Z, Yan H. A novel cell nuclei segmentation method for 3D C. elegans embryonic time-lapse images. BMC Bioinformatics. 2013;14:328.View ArticlePubMedPubMed CentralGoogle Scholar
- Mikula K, Remesiková M, Stasová O, Peyriéras N. Segmentation and analysis of 3D zebrafish cell image data. In International Congress on Image and Signal Processing; 2010:1444–48.Google Scholar
- Keller PJ, Schmidt AD, Wittbrodt J, Stelzer EHK. Reconstruction of zebrafish early light sheet microscopy. Science. 2008;322:1065–9.View ArticlePubMedGoogle Scholar
- Indhumathi C, Cai YY, Guan YQ, Opas M. An automatic segmentation algorithm for 3D cell cluster splitting using volumetric confocal images. J Microsc. 2011;243:60–76.View ArticlePubMedGoogle Scholar
- Zanella C, Campana M, Rizzi B, Melani C, Sanguinetti G, Bourgine P, et al. Cells segmentation from 3-D confocal images of early zebrafish embryogenesis. IEEE Trans Image Process. 2009;19:770–81.View ArticlePubMedGoogle Scholar
- Bashar MK, Yamagata K, Kobayashi TJ. Improved and robust detection of cell nuclei from four dimensional fluorescence images. PLoS One. 2014;9:e101891.View ArticlePubMedPubMed CentralGoogle Scholar
- Bilgin CC, Kim S, Leung E, Chang H, Parvin B. Integrated profiling of three dimensional cell culture models and 3D microscopy. Bioinformatics. 2013;29:3087–93.View ArticlePubMedPubMed CentralGoogle Scholar
- Lien J, Amato NM. Approximate convex decomposition of polyhedra. In ACM symposium on Solid and physical modeling; 2006:121–131.Google Scholar
- Attene M, Mortara M, Spagnuolo M, Falcidieno B. Hierarchical convex approximation of 3D shapes for fast region selection. Comput Graph Forum. 2008;27:1323–32.View ArticleGoogle Scholar
- Asafi S, Goren A, Cohen-Or D. Weak Convex Decomposition by Lines-of-sight. Comput Graph Forum. 2013;32:23–31.View ArticleGoogle Scholar
- Ward JJH. Hierarchical Grouping to Optimize an Objective Function. J Am Stat Assoc. 1963;58:236–44.View ArticleGoogle Scholar
- Plugin for auto local threshold (ImageJ). [http://fiji.sc/wiki/index.php/Auto_Local_Threshold]
- Physikalische Biologie. [https://www.physikalischebiologie.de/downloads]
- FARSight. [http://www.farsight-toolkit.org]
- 3D ImageJ Suite. [http://imagejdocu.tudor.lu/doku.php?id=plugin:stacks:3d_ij_suite:start]
- Ollion J, Cochennec J, Loll F, Escudé C, Boudier T. TANGO: A generic tool for high-throughput 3D image analysis for studying nuclear organization. Bioinformatics. 2013;29:1840–1.View ArticlePubMedPubMed CentralGoogle Scholar
- ilastik. [http://ilastik.org/]
- Song Y, Cai W, Huang H, Wang Y, Feng DD, Chen M. Region-based progressive localization of cell nuclei in microscopic images with data adaptive modeling. BMC Bioinformatics. 2013;14:173.View ArticlePubMedPubMed CentralGoogle Scholar
- Meijering E, Dzyubachyk O, Smal I. Methods for cell and particle tracking. Methods Enzymol. 2012;504:183–200.View ArticlePubMedGoogle Scholar
- Bai X, Sun C, Zhou F. Splitting touching cells based on concave points and ellipse fitting. Pattern Recognit. 2009;42:2434–46.View ArticleGoogle Scholar
- Latorre A, Alonso-Nanclares L, Muelas S, Peña J-M, Defelipe J. 3D segmentations of neuronal nuclei from confocal microscope image stacks. Front Neuroanat. 2013;7:49.View ArticlePubMedPubMed CentralGoogle Scholar
- Pampaloni F, Reynaud EG, Stelzer EHK. The third dimension bridges the gap between cell culture and live tissue. Nat Rev Mol Cell Biol. 2007;8:839–45.View ArticlePubMedGoogle Scholar
- Schrode N, Xenopoulos P, Piliszek A, Frankenberg S, Plusa B, Hadjantonakis A-K. Anatomy of a blastocyst: cell behaviors driving cell fate choice and morphogenesis in the early mouse embryo. Genesis. 2013;51:219–33.View ArticlePubMedPubMed CentralGoogle Scholar

## Copyright

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.