Semi-Automatic segmentation of multiple mouse embryos in MR images
© Baghdadi et al; licensee BioMed Central Ltd. 2011
Received: 20 February 2011
Accepted: 16 June 2011
Published: 16 June 2011
The motivation behind this paper is to aid the automatic phenotyping of mouse embryos, wherein multiple embryos embedded within a single tube were scanned using Magnetic Resonance Imaging (MRI).
Our algorithm, a modified version of the simplex deformable model of Delingette, addresses various issues with deformable models including initialization and inability to adapt to boundary concavities. In addition, it proposes a novel technique for automatic collision detection of multiple objects which are being segmented simultaneously, hence avoiding major leaks into adjacent neighbouring structures. We address the initialization problem by introducing balloon forces which expand the initial spherical models close to the true boundaries of the embryos. This results in models which are less sensitive to initial minimum of two fold after each stage of deformation. To determine collision during segmentation, our unique collision detection algorithm finds the intersection between binary masks created from the deformed models after every few iterations of the deformation and modifies the segmentation parameters accordingly hence avoiding collision.
We have segmented six tubes of three dimensional MR images of multiple mouse embryos using our modified deformable model algorithm. We have then validated the results of the our semi-automatic segmentation versus manual segmentation of the same embryos. Our Validation shows that except paws and tails we have been able to segment the mouse embryos with minor error.
This paper describes our novel multiple object segmentation technique with collision detection using a modified deformable model algorithm. Further, it presents the results of segmenting magnetic resonance images of up to 32 mouse embryos stacked in one gel filled test tube and creating 32 individual masks.
The mouse, due its genetic similarity to humans, the existence of sophisticated genetic tools to manipulate its genome, rapid reproduction time and comparatively low housing costs, has become an increasingly important model organism in mammalian biology. More recently, Magnetic Resonance Imaging (MRI) has emerged as tool for capturing three-dimensional anatomy to aid in phenotyping mice . There is, however, a resolution to time trade-off in MRI which impacts its application to the mouse, where high resolution is essential due to the small size of the animal but is achieved with very long scan time. Parallelization of MRI experiments is one technique to compensate for the long imaging time. This encompasses either separate transmit/receive coils for each mouse , or packing multiple specimen into the same field of view while using a single coil. The latter strategy was chosen by Schneider et al. for phenotyping mouse embryos, placing 32 fixed specimen embedded in gel into a single tube which was then imaged using high-resolution MRI at 9.4 Tesla .
Analysis of these datasets thus requires an algorithm which is not only capable of segmenting multiple embryos at the same time but can also separate adjacent touching embryos during segmentation. In this paper, we demonstrate a semi-automatic multiple embryo segmentation technique which allows us to create individual masks for all embryos starting from identical spheres. These masks can then be used to separate each embryo from the rest of the embryos in the tube for further analysis. Our method is a modified version of the Simplex Mesh Deformable Algorithm of Delingette  with additional balloon and collision forces which allow for segmentation of each embryo as well as its satisfactory separation from neighbouring embryos.
2.1.1 Related Work
The deformable simplex mesh algorithm is a general shape reconstruction technique which attempts to build a geometric model from a three-dimensional dataset. Its surface representation is a simplex mesh with topology dual to that of triangulation . In addition, each vertex of a simplex mesh has a fixed number of neighbouring vertices directly connected to it. Because of this geometry, simplex meshes defined discrete concepts at each vertex such as mean curvature and normal vectors which are used for the calculation of forces of deformation. This differs with triangle meshes in that each vertex of a triangle mesh can have many neighbours and concepts such as normal vectors are usually defined for each surface(made of three vertices) of the triangle rather than each vertex. For these reasons, simplex meshes are more preferable as deformable models rather than triangle meshes . On a three-dimensional simplex mesh, each vertex has three neighbouring vertices which define its tangent plane and normal vector. Further, the angle between adjacent edges can be defined as a simplex angle which defines the local shape around each vertex. This angle can then be used for the computation of mean curvature, a measure of surface bending. Finally, the position of a simplex vertex can be defined as a function of its neighbouring vertices and the shape parameters. Due to the large extent of shape control, simplex meshes are better suited for deformation and smoothing than triangulation [5, 7].
Both forces are calculated at time t, F int is the internal force which enforces geometric continuity of the mesh whereas F ext constrains the distance between the mesh and a three-dimensional dataset.
Internal forces are calculated based on the geometry of simplex meshes and can be decomposed into a tangential force F tangent and a normal force F normal .
The tangent force controls the position of each vertex with respect to its three neighbours in the tangent plane. The normal force constrains the mean curvature of the surface through the simplex angle of each vertex. For details of how these forces are calculated see .
The gradient force is determined from a search within a certain radius in the neighbourhood of each vertex for a voxel with the highest gradient intensity. The edge force is calculated by searching for the voxel with the highest intensity in the direction of the normal line of each vertex.
Despite their many intrinsic advantages, deformable model algorithms suffer from three main limitations : (1) they are sensitive to initialization problems, which necessitates that they begin their deformation in close vicinity to the final desired solution in order to avoid becoming trapped in local minima; (2) they are often unable to adapt to boundary concavities due to internal forces which keep the model smooth and minimize curvature; and (3) they are prone to self-intersection unless adequately constrained by internal and external forces.
We have modified the deformable model algorithm of Delingette by adding four components to the original model to adequately segment MRI scans of multiple embryos acquired within the same field of view. First, we have added a balloon force component to the definition of external forces. This allows us to start segmenting all embryos from identical spheres without having to worry about model getting trapped in close by local minima at stated in the above. Second, a collision force component was also added to the definition of external forces. Third, we have implemented the concept of a tube mask to exclude the plaster walls of the sample tube. The latter two concepts help solve the problem of touching embryos. Finally, instead of segmenting one embryo at a time, we have designed and implemented a deformable model program which segments up to N embryos simultaneously.
In the sections to follow, we explain the modifications to the original deformable model algorithm of Delingette. The solution proposed herein, while designed with a particular application in mind, should be generally applicable for using deformable models to segment multiple similar and potentially touching objects within one image.
2.1.2 Modified Deformable Model with Balloon Force
Initialization is one of the most challenging issues with deformable models. The success of most deformable model segmentation algorithms is determined by how closely the edges of the initial model follow that of the final result. If the initial model is much smaller than the object which is being segmented then the model will get stuck in local minima resulting in unsuccessful segmentation. In our case, since we start with identical initial models for all embryos, there is a need for a method to allow for local inflation of all models until they lie within a region of the image where they are close to the boundary of the embryos. To solve the multiple embryo initialization problem we implemented a two step solution wherein the image is fist coarsely classified into embryo versus gel, and secondly a balloon force is added to the deformable model to drive it towards the correct boundary.
It is important to mention that although the classification algorithm is capable of labeling the image with sufficient accuracy for initializing the deformable models, it has a tendency to label a few more voxels belonging to embryos than the actual embryos. As the problem of touching embryos already exist in the original MR images and is addressed by our algorithm, this does not create a problem for our technique.
where T low and T high are the lower and upper thresholds for desired intensity values of the objects of interest and in our case were chosen as T low = - 0.99 and T high = 1.0. Together, the balloon force and the classified image expand the initial models towards the correct boundaries of the embryos; this alone is, however, not sufficient for the final segmentation results. As explained before, the embryos are larger in the classified image than their actual size and without the gradient forces to stop the expansion of models at the correct embryo edges, the balloon force expands the models past the edges of embryos and segments the gel right around embryos as part of the embryos. Further, a look at the classified image reveals that even if the embryos are not touching each other in the original image, they do intersect in the classified image, and hence without any intervention models will leak into each other as there are no barriers in between the embryos.
2.1.3 Modified Deformable Model with Collision Force
After every 5 iterations we check the vertices of each mesh for points of collision. We have picked 5 since it allows the meshes to deform but not so much that there is a chance of collision before detection. At each collision point a small neighbourhood of the voxels (5 voxels in our case, we have tested our algorithm with different neighbourhood of the voxels and determined that values greater than 5 do not add more efficiency but result in poor performance) are searched for the voxel with the highest gradient intensity hence F gradient . Our collision energy is designed based on the assumption that the true boundary of the embryos is close by where the collision takes place between two meshes since they are expanding simultaneously. Therefore, a gradient search around the neighbourhood of the collision point reveals the correct boundary of the embryos.
Use seed points determined by the user as the centers of identical spherical simplex meshes to initialize all embryos (Figure 3a).
Simultaneously inflate all meshes on the classified image to avoid any local minima using the balloon force. This will help the models get close to the boundary of interest, in our case the edges of the embryos (Figure 3b).
While expanding the models, use the collision detection algorithm after every 5 iterations to detect any collision between the models. If collision occurs, stop the inflation for models in collision and continue inflation for the rest of the models. Continue until all models have stopped inflating.
At this point, all models are within the vicinity of their corresponding embryos, use the gradient search algorithm to find close by edges of embryos (Figure 3c).
Once the deformation have stopped for all the models, use a rasterization algorithm to create a binary mask for each embryo from the corresponding simplex deformable model.
2.1.4 Modified Deformable Model with Tube Mask
At each iteration the final displacement of the model points are created and checked to make sure the displacement lands every point within the tube mask. If it happens that any point is displaced outside the tube, the displacement for that point is set to zero and the previous coordinate is assigned to the point ensuring that no point leaves the tube mask area.
We have modified the original definition of the deformable simplex model algorithm with balloon force, collision force and tube mask to simultaneously deform up to N models while ensuring that they do not leak into each other or outside of the tube.
2.1.5 Multi-resolution Approach
The gradient forces which attract the model towards the correct boundaries of the target are computed around each model point within a local region in the image and often gets the model stuck to spurious image features or non-true boundaries of the object. Therefore, deformable models must be initialized close to the boundaries of the object to avoid unsuccessful segmentation. This feature of deformable model algorithms is also present in our modified version of the deformable model as it too searches the image within a local region of model points and can get the model stuck to non-true boundaries of the object.
The multi-resolution approach allows the deformable model to pass over non-true boundaries of the object and to quickly find a rough boundary approximation in the early stages of deformation .
Another important reason for starting the deformation with low resolution models is that because our deformable model algorithm does not have any explicit checking for self-intersection, starting the deformation with high resolution models will likely result in self-intersecting models due to having too many points at the first stage of deformation at which the models are expanding by the balloon force. Self-intersecting models result in binary masks with holes in them and since our collision detection algorithm relies on the binary masks created from the models, having non self-intersecting models is essential to the success of our algorithm.
Our deformable model algorithm starts with low resolution models. After the first stage of the deformation, the models go through major expansions with minimal or no self-intersection. However, low resolution models only recover approximate edges of the target boundaries. To recover finer boundary features of the embryos high resolution models are needed. We increase the resolution of the models by a two fold after every stage of the deformation and continue until achieving satisfactory results and the algorithm has reached the convergence criteria.
Initialization is one of the main problems with most deformable model algorithms. The deformable model algorithms usually can only find the close by edges once the model is placed within the close vicinity of the shape of interest. Although we use the balloon force (section 2.1.2) to locally inflate the initial meshes and get close to the boundaries of interest, we still need a starting point for each individual mesh. Thus, there is a need for a separate method to determine the initial location of the models before segmentation can take place . Delingette has proposed automatic methods for initialization, however, these algorithms have very limited range of application and are sensitive to image noise and most importantly are not designed for multiple object initialization which is required in our case since we are deforming multiple models simultaneously .
All forces that are governing our algorithm are strengthened or weakened through the use of weight factors (i.e., parameters). Every parameter ranges from zero (no strength) to one (highest strength). We have determined the particular values for each of our parameters through experimentation. At the start of the deformation, edge forces are kept at zero as it is not being used. Balloon forces and internal forces are kept high at 0.08 and 0.9 respectively, this is to ensure that the meshes are expanding rapidly while being kept smooth and hence free of self-intersection. The gradient and collision forces are kept at a middle range at 0.3 and 0.4 respectively to make sure mesh expansion is possible to the maximum limit but not beyond the embryo boundaries. Simplex meshes also benefit from having control over the scale of smoothness which is defined as the size of the neighborhood around each point used for smoothing of the mesh. To avoid self-intersection to the maximum level, the scale of the smoothness is also kept very high at 12 at the early stages of deformation.
After the first stage of deformation, the meshes are expanded dramatically and are close to the true edges of the embryos (Figure 7: column b). At this stage, the resolution of the mesh is increased by two fold, balloon and internal forces are decreased to 0.02 and 0.7 respectively. The balloon force is decreased because there is no need for further expansion of the meshes as they are already close to the edges of the embryos. The internal forces which keep the meshes smooth is also decreased so as to allow the meshes to recover finer edges of the embryos. The gradient force is increased to 0.4 to make sure the meshes are attracted to the edges of the embryos and the collision force is decreased to 0.3 to avoid collision between touching embryos while further deformation is still taking place. The smoothness scale is also decreased to 3, since self-intersecting meshes mostly occur during the first stage of deformation when meshes go through major expansion.
It has been observed that if the collision detection force is kept at a very high level of 0.8-0.9 to keep the segmentation of touching neighbouring embryos free of any intersection, the embryos will not be fully segmented. To overcome this problem, the collision detection force is reduced to mid range of 0.3-0.4 to allow slight touching of embryos so they can be fully segmented.
The stage of deformation with high resolution models can take place between 3 to 5 times in order to recover all fine feature of the embryos or when convergence is reached (Figure 7: column c). However, convergence does not imply that the modified deformable model algorithm is capable of recovering fine features such as paws or tails of the embryos. To recover such fine features using the simplex model deformable algorithm either the features have to be present in the initial model (i.e., to recover the tail, there has to be a tail present in the initial model) or there is a need for an algorithm which is capable of refining the model locally i.e., increase the resolution of the mesh only around areas with fine features.
The deformation continues until the models are stabilized. To determine the stability of each mesh, each point of the simplex mesh is classified as "active" or "inactive". A point is considered "inactive" if the magnitude of its displacement (the difference between its old position versus its new position) is less than a user defined threshold of 0.0001 in our case. The activity of each simplex mesh during the last n iterations is defined as the ratio of its inactive points over the total number of points. Any mesh with activity ratio of higher than a user determined threshold value of 0.5 is considered stabilized hence converged .
3 Results and Discussion
Mean and standard deviation of kappa, specificity and accuracy for two tubes totaling 37 embryos.
0.83 ± 0.07
0.83 ± 0.04
0.88 ± 0.03
0.78 ± 0.06
0.87 ± 0.03
0.88 ± 0.03
Mean and standard deviation of brain, liver, lung and ventricles for two tubes totalling 37 embryos.
0.998 ± 0.0
1 ± 0.0
1 ± 0.0
1 ± 0.0
0.991 ± 0.02
1 ± 0.0
1 ± 0.0
1 ± 0.0
We have developed a novel multi-resolution segmentation technique for semi-automatically segmenting multiple embryos simultaneously. Our multiple embryo segmentation technique can also be applied to other multiple object segmentation problems with possible touching of objects. Our algorithm was found to be highly accurate in capturing the whole embryo as well as its organs. Our deformation scheme follows a low to high resolution approach at which we start the deformation with low resolution meshes and continue increasing the resolution of the meshes through higher stages of deformation. The meshes are then deformed using a combination of internal and modified external forces (i.e., balloon and collision forces). The multi-resolution approach allows us to use identical initial models for all embryos while quickly and efficiently recovering the rough overall anatomical structure of the embryos at the first stage of deformation. The final shape of the embryo is then recovered by subsequently capturing the finer details of the embryos with higher resolution meshes at every stage of the deformation.
We have designed and implemented the concept of balloon force to allow us to start the deformation with small spheres and successfully reach the boundaries of the embryos. Using small initial models is a major issues with most deformable models as they can get trapped in local minima. However, our algorithm has overcome this problem by introducing the concept of balloon force.
To solve the problem of touching embryos, a novel collision detection algorithm was introduced. The collision detection method allows us to segment multiple embryos while avoiding major leaks into adjacent neighboring embryos. To our knowledge, most collision detection algorithms work on triangle meshes only whereas in our case binary masks created from the meshes are used. In order to recover the complete shape of all embryos, collision forces were kept in such a way that a small collision between the embryos was allowed.
Our multi-resolution deformable model algorithm with collision detection and balloon forces has overcome some of the difficulties of deformable model algorithms while allowing us to segment multiple embryos simultaneously. It has also enabled us to segment as many embryos without having to worry about the touching embryos. Our algorithm is written entirely in C++ as part of The Insight Toolkit(ITK) itk.org-InsightApplications/DeformableModelSimplexMesh open source software. It takes about half an hour to make the initial preparations including determining the seed points, creating the classified image and creating a single text file which holds all the information such as the location of the images and all parameters for the deformable model application. After the initial preparation, it takes approximately 6 hours to complete the deformation of 32 initial spheres, creating 32 individual masks on a 64-bit PC with 3600 MHz Intel Xeon CPU where each tube dataset is about 0.5 GB in size with 50 × 50 × 50 μ m resolution. This is computer time in comparison to segmenting embryos manually on high resolution datasets which are labour intensive with roughly 10 hours of operator time for segmenting one embryo.
While we were unable to find alternate solutions to the problem of segmenting multiple similar and touching objects in the literature, we acknowledge that there are other advanced segmentation methods, such as level sets, graph cuts, or similar region growing algorithms, that could have been used instead. The most likely alternate solution, level sets, involve numerical methods for tracking the evolution of contours and surfaces which uses image-based features such as mean, gradient and edges in the governing differential equations to segment the image . The same issues of preprocessing that we addressed with the pro-posed deformable model approach would likely have to be solved for a level-sets based implementation.
Although our multi-resolution deformable model algorithm is capable of successful segmentation of multiple embryos while avoiding major leaks into neigh-boring embryos, it is still not considered fully automatic as it requires user's input for seed points at the beginning. We would like to explore methods at which the initialization of multiple seed points can be done automatically and without any user intervention. Further, our algorithm does not use any methods to avoid self-intersecting meshes during the deformation. Currently, we have solved this issue by carefully choosing a high scale of the deformation at the first stage of deformation. Although successful most of the time, this stage of our algorithm is rather slow comparing to other stages of the deformation. For this reason, we would like to investigate algorithms which can be added to our program for automatically avoiding self-intersecting meshes. Finally, we would like to be able to recover finer details of embryos by using a local refinement technique. This implies that the meshes are not consistently refined everywhere but are locally refined at places of high curvature similar to -subdivision algorithm .
Acknowledgements and Funding
This project was funded by Genome Canada through the Ontario Genomics Institute. We acknowledge the helpful insights of Dr Hervé Delingette of INRIA, France and Dr David Gobbi of Atamai Inc, London, Canada.
- Nieman BrianJ, Bishop Jonathan, Dazai Jun, Bock NicholasA, Lerch JasonP, Feintuch Akiva, Josette Chen X, Sled JohnG, Mark Henkelman R: MR technology for biological studies in mice. Nuclear Magnetic Resonance in Biomedicine 2007, 20: 291–303.Google Scholar
- Bock NicholasA, Konyer NormanB, Mark Henkelman R: Multiple mouse MRI. Magnetic Resonance in Medicine 2003, 49(1):158–167. 10.1002/mrm.10326View ArticlePubMedGoogle Scholar
- Schneider JürgenE, Böse Jens, Bamforth SimonD, Gruber AchimD, Broadbent Carol, Clarke Kieran, Neubauer Stefan, Lengeling Andreas, Bhattacharya Shoumo: Identification of cardiac malformation in mice lacking ptdrs using a novel high-throughput magnetic resonance imaging technique. BMC Developmental Biology 2004., 16(4):Google Scholar
- Delingette Hervé: General object reconstruction based on simplex meshes. Int Journal of Computer Vision 1999, 32: 111–142. 10.1023/A:1008157432188View ArticleGoogle Scholar
- Delingette Hervé: Simplex meshes: a general representation for 3D shape reconstruction. Proc of Int Conf on Computer Vision and Pattern Recognition (CVPR'94), Seattle, USA 1995, 856–860.Google Scholar
- Galdames Francisco, Jaillet Fabrice: From Triangulation to Simplex Mesh: a Simple and Efficient Transformation. Technical Report RRLIRIS-2010–021, LIRIS UMR 5205 CNRS/INSA de Lyon/Universit Claude Bernard Lyon 1/Universit Lumire Lyon 2/cole Centrale de Lyon 2010.Google Scholar
- Tejos Cristian, Irarrazaval Pablo, Crdenas-Blanco Arturo: Simplex mesh diffusion snakes: Integrating 2d and 3d deformable moels and statistical shape knowledge in a variational framework. International Journal of Computer Vision 2009., 85(1):Google Scholar
- McInerney Tim, Terzopoulos Demetri: Deformable models in medical image analysis: A survey. Medical Image Analysis 1996, 1(2):91–108. 10.1016/S1361-8415(96)80007-7View ArticlePubMedGoogle Scholar
- Park Joo-Young, McInerney Tim, Terzopoulos Demetri, Kim Myoung-Hee: A non-self-intersecting adaptive deformable surface for boundary extraction from volumetric images. Graphics and Computers 2001, 25: 421–440. 10.1016/S0097-8493(01)00066-8View ArticleGoogle Scholar
- Zijdenbos AlexP, Forghani Reza, Evans AlanC: Automatic pipeline analysis of 3D MRI data for clinical trials: application to multiple sclerosis. IEEE Transactions on Medical Imaging 1999, 21(10):1280–1291.View ArticleGoogle Scholar
- Curtis Sean, Tamstorf Rasmus, Manocha Dinesh: Fast collision detection for deformable models using representative triangles. Proceedings of the 2008 Symposium on Interactive 3D graphics and games 2008, 61–69.View ArticleGoogle Scholar
- Smith StephenM: Fast robust automated brain extraction. Human Brain Mapping 2002, 17(3):143–155. 10.1002/hbm.10062View ArticlePubMedGoogle Scholar
- Zamyadi Mojdeh, Baghdadi Leila, Lerch JasonP, Bhattacharya Shoumo, Schneider JurgenE, Henkelman R Mark, Sled JohnG: Mouse embryonic phenotyping by morphometric analysis of MR images. Physiological Genomics 2010, 42A(2):89–95. 10.1152/physiolgenomics.00091.2010PubMed CentralView ArticlePubMedGoogle Scholar
- Malladi Ravikanth, Sethian JamesA, Vemuri BabaC: Vemuri. Shape modeling with front propagation: A level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995, 17(2):158–174. 10.1109/34.368173View ArticleGoogle Scholar
- Kobbelt Leif: -subdivision. Proceedings of SIGGRAPH 2000 Anual Conference on Computer Graphics, New Orleans, LA 2000, 103–112.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.