- Open Access
TOMOBFLOW: feature-preserving noise filtering for electron tomography
BMC Bioinformatics volume 10, Article number: 178 (2009)
Noise filtering techniques are needed in electron tomography to allow proper interpretation of datasets. The standard linear filtering techniques are characterized by a tradeoff between the amount of reduced noise and the blurring of the features of interest. On the other hand, sophisticated anisotropic nonlinear filtering techniques allow noise reduction with good preservation of structures. However, these techniques are computationally intensive and are difficult to be tuned to the problem at hand.
TOMOBFLOW is a program for noise filtering with capabilities of preservation of biologically relevant information. It is an efficient implementation of the Beltrami flow, a nonlinear filtering method that locally tunes the strength of the smoothing according to an edge indicator based on geometry properties. The fact that this method does not have free parameters hard to be tuned makes TOMOBFLOW a user-friendly filtering program equipped with the power of diffusion-based filtering methods. Furthermore, TOMOBFLOW is provided with abilities to deal with different types and formats of images in order to make it useful for electron tomography in particular and bioimaging in general.
TOMOBFLOW allows efficient noise filtering of bioimaging datasets with preservation of the features of interest, thereby yielding data better suited for post-processing, visualization and interpretation. It is available at the web site http://www.ual.es/%7ejjfdez/SW/tomobflow.html.
The advent of bioimaging technology has made it possible to observe the molecular and cellular architecture and interactions that underlie essential functions within cells and tissues. The availability of bioimaging techniques (e.g. light, confocal, X-ray, electron microscopies) in laboratories is growing rapidly. So is the need for advanced image processing methods that facilitate analysis and interpretation at different scales of resolution and complexity.
Electron tomography (ET), which combines electron microscopy with the power of three-dimensional (3D) imaging, is the leading technique to elucidate the molecular architecture of biological specimens in a close-to-native state [1–3]. ET produces extremely noisy and low contrast 3D density maps (known as "tomograms" in the field). The poor signal-to-noise ratio (SNR) severely hinders visualization and interpretation. Sophisticated filtering techniques are thus indispensable . Similar filtering needs arise in other bioimaging modalities (e.g. [5–8]).
Noise reduction is paramount for proper interpretation or post-processing of multidimensional images in bioimaging in general, and electron tomography in particular. Standard linear filtering techniques based on local averages or Gaussian kernels succeed in reducing the noise, but at the expense of blurring edges and features [4, 9]. Anisotropic nonlinear diffusion (AND) is currently one of the most powerful noise reduction techniques . It achieves feature preservation and enhancement as the strength and direction of the smoothing are adaptively tuned to the local structures [11, 12]. However, AND may be intensive in terms of computation time and memory consumption  and, moreover, there is need for tuning their parameters, which may be certainly far from trivial. These drawbacks have led to the proposal of other simpler, more practical, but less powerful, methods like iterative median filtering , or attempts for automated parameter tuning .
TOMOBFLOW is a program for noise reduction with feature-preserving capabilities based upon geometric flow, particularly the so-called Beltrami flow. The fact that this approach is parameter-free is one of its main advantages and makes it user-friendly. Therefore, TOMOBFLOW combines the power of diffusion-based noise filtering approaches with the easiness from the user's point of view. Furthermore, the program has been implemented efficiently in order to minimize the memory requirements and reduce the computation time.
Several approaches for noise reduction in multidimensional image processing are based on considering images as maps that are embedded into a higher dimension and that flow towards minimal surfaces . In these approaches, a 2D image is considered as a 2-manifold embedded in 3D, i.e. the image I(x, y) is regarded as a surface S = (x, y, I(x, y)) in a 3D space (see Figure 1). Similarly, a 3D volume I(x, y, z) is considered as a 3-manifold embedded in a 4D space S = (x, y, z, I(x, y, z)). Embedding the multidimensional image into a higher dimension allows the use of powerful differential geometry operators .
The Beltrami flow is an efficient geometric diffusion flow approach that aims to minimize the area of the image manifold, driving the flow towards a minimal surface solution while preserving edges. The Beltrami flow is formulated as follows :
where I t = ∂I/∂t denotes the derivative of the image density I with respect to the time t; ▽I is the gradient vector, that is ▽I ≡ (I x , I y ) for 2D images whereas ▽I = (I x , I y , I z ) for 3D volumes, being I x = ∂I/∂x the derivative of I with respect to x (similar applies for y and z); div is the divergence operator, defined for a vector function f = (f x , f y , f z ) as div(f) = ∂f x /∂x + ∂f y /∂y + ∂f z /∂z. Finally, g denotes the determinant of the first fundamental form of the surface, which is g = 1 + |▽I|2.
The term g comes from an induced metric for the Euclidean (n + 1)-D space where the density of a n-D image is embedded in the (n + 1)-th dimension  (with n = 2 for 2D images and n = 3 for 3D volumes). This g provides the measure of the area expansion between the image domain I and the surface domain S, and thus plays an important role to drive the flow towards a surface with the least area.
Moreover, the term in Equation (1) acts as an edge indicator since it is proven to be the projection of the normal-to-the-surface to the vector representing the (n + 1)-th dimension  (see Figure 1). Therefore, the Beltrami flow is a selective noise filtering method that preserves edges as minimizes diffusion at and across edges whereas it applies extensive diffusion elsewhere .
In TOMOBFLOW, the implementation of the partial differential equation derived from Equation (1) is based on finite differences, using an Euler forward difference approximation for I t and central differences to approximate the spatial derivatives (for brevity, only the numerical approximation for the 2D case is shown):
where Ikis the image in the k-th iteration, h t is the time step (for stability, the maximum value is the inverse of the squared number of dimensions, i.e. 0.25 for 2D images), I x is the derivative with respect to x (similar applies for y), I xx is the second order derivative with respect to x (similar applies for y) and I xy is the mixed second order partial derivative with respect to x and y. The derivatives are computed from the image in the previous iteration Ik-1, and are numerically approximated by central differences, as shown for x (similar applies for y):
where i, j are the indices of the pixel. TOMOBFLOW has the option of applying a slight Gaussian filtering (standard deviation typically in [0.5,1.0]) to the input dataset. This initial Gaussian filtering is employed for regularization purposes to yield better estimates of the derivatives, as commonly used in other diffusion approaches .
An efficient implementation has been carried out using single processor optimization  to reduce memory and time consumption. Only one copy of the input dataset, which is progressively updated during the processing, is held in memory. A sliding window (3 slices for 3D volumes, 3 rows for 2D images) is used to keep the data needed for the current slice/row in order to avoid overwriting. Figure 2 illustrates how the sliding window is used during the processing of 3D volumes. This optimization allows processing of huge datasets, as commonly found in ET, on computers with modest amounts of memory.
To make it suitable for bioimaging in general, TOMOBFLOW is capable of dealing with most image formats in electron microscopy (e.g. EM, MRC, Spider), in other microscopies (e.g. Biorad) and general formats (e.g. TIF, JPG, PNG) by using the Bsoft library . Furthermore, TOMOBFLOW is also able to filter 3D volumes, individual 2D images, or stacks of 2D images. Finally, it is available for most Unix platforms, including OS X and Windows (under cygwin). The command line user interface follows the Unix-style and the options follow the conventions of Bsoft . A comprehensive documentation is provided at the website http://www.ual.es/%7ejjfdez/SW/tomobflow.html.
The performance of TOMOBFLOW is illustrated with its application to a number of experimental datasets obtained from electron tomography. Tomograms (3D volumes) of (a) spiny dendrite, (b) algae chloroplast, (c) mitochondrion, (d) small unilamellar liposomes with integrin, (e) vaccinia virion and (f) human immunodeficiency virions (strain HIV-1) were tested. Different contrast and signal-to-noise ratio were present in those datasets as they were obtained by using different preparation techniques. The specimens in (a-c) were stained before imaged, hence their much better contrast in the original dataset compared to the other specimens in (d-f), which were imaged while frozen in close-to-native conditions without stain. The datasets in (a, b) were taken from the Cell Centered Database [20, 21] (accession codes 13 and 3408, respectively). The datasets in (d, f) were taken from the Electron Microscopy Data Bank (EMDB) at the European Bioinformatics Institute [22, 23] (accession codes 1487 and 1155, respectively). The Vaccinia virus dataset was obtained from a previous work . The mitochondrion dataset was kindly provided by Dr. G. Perkins. In order to compare TOMOBFLOW with other comparable standard (isotropic) nonlinear noise reduction technique, the datasets were also subjected to iterative median filtering  as implemented in Bsoft [19, 25]. This method is getting increasing interest in the electron microscopy field [8, 25–27]. The standard number of three iterations was used for all the experiments carried out in this work where the iterative median filtering was involved. For TOMOBFLOW, a number of iterations between 50 and 150 were used, which yielded a satisfactory level of smoothness for the background of the datasets. A Gaussian filtering with standard deviation of 1 was used to regularize the derivative estimation in TOMOBFLOW. For the results obtained with TOMOBFLOW, the median filtering was not used prior to the iterations of the Beltrami flow.
Figure 3 shows the effects of noise reduction with TOMOBFLOW and the iterative median filtering on the tomograms obtained from the stained specimens: spiny dendrite, algae chloroplast, mitochondrion, respectively. TOMOBFLOW and the median filtering succeed in reducing the noise and preserving details. However, the results obtained with TOMOBFLOW clearly show a background substantially smoothened while showing more sharpness in the structural features of interest. The results provided by the median filtering are not that sharp and the background still contains some noisy texture.
Figure 4 shows the results with the datasets of the unstained specimens: small unilamellar liposomes with integrin, vaccinia virion and HIV-1 virions, respectively. The behaviour of the methods is similar to that shown with stained specimens. Both methods yield datasets that overcome the extremely low contrast and signal-to-noise ratio, thereby facilitating the interpretation. However, the results provided by TOMOBFLOW have their background further smoothened and the structural features are better preserved. For a more exhaustive analysis of the performance of TOMOBFLOW, a representative electron cryotomography dataset was selected, HIV-1  (emd-1155), which was also used elsewhere to illustrate the performance of the iterative median filtering . Figure 5 shows the appearance of the tomograms (original and denoised with TOMOBFLOW and the median filtering) when isosurfaced. Both methods allow a clear visualization of the virions, with similar levels of residual noise. However, a noticeable difference is seen at the membranes of the virions. They appear quite flat with the median filtering whereas more corrugations and details are seen with TOMOBFLOW, which is indicative of better structural preservation of the latter method. This effect can also be seen on the slices shown in Figure 4c.
The evolution of the denoising with the iterations was then studied on the HIV-1 dataset. Figure 6 shows the result of TOMOBFLOW after 10, 25, 50, 100 and 150 iterations. It is clearly observed that the background is progressively flattened whereas the structural features are well preserved in general. However, the decrease in resolution at high number of iterations is expected to blur some details of interest. For instance, at 150 iterations some edges begin to look blurred. A more objective assessment of these results was carried out based on SNR (signal-to-noise ratio) measures, as defined in : SNR = (I s - I b )/σ b , where I s and I b denote the average intensity in the structure of interest and in the background, respectively, and σ b is the standard deviation of the background. This SNR metric is similar to the contrast-to-noise ratio (CNR) used in other disciplines . The background in this tomogram was determined based on the isosurfacing thresholds (calculated according to the optimal thresholding algorithm in ) used in Figure 5, and refined afterwards by means of some morphological operators applied in this sequence: flooding, dilation (2 cycles) and erosion (2 cycles). This background was used for computing all the SNR measures presented in this work. Table 1 summarizes the SNR results of the tomograms shown in Figure 6. The SNR measures quantitatively reflect the effects of the denoising seen in the visual results in Figure 6. The SNR for the original tomogram was 1.23. TOMOBFLOW outperforms the median filtering (SNR 2.63) at 50 or more iterations (SNR ≥ 2.79). The SNR was also computed for 200, 250 and 300 iterations of TOMOBFLOW (visual results not shown in Figure 6). As reflected by the SNR metric, the results begin to degrade at a number of TOMOBFLOW iterations between 150 and 200. Therefore, the SNR metric acts as an indicator of the degradation with high number of iterations.
The SNR metric was also used to assess the results shown in Figure 4c. The result from 70 iterations of TOMOBFLOW yielded SNR 3.03, higher than the result from the median filtering (SNR 2.63). These measures complement and confirm the visual results shown in Figure 4c. For comparison, the SNR was computed for a denoised version of the tomogram with anisotropic nonlinear diffusion, which is the leading denoising method in the field. The package TOMOAND http://www.ual.es/%7ejjfdez/SW/tomoand.html was used [4, 11] with the automated parameter tuning activated . The number of iterations (70) and the initial Gaussian filtering (std.dev.1) was set up as with TOMOBFLOW. The SNR of the TOMOAND-denoised tomogram turned out to be 4.11. Therefore, AND is superior to TOMOBFLOW, though at the expense of higher computation time and memory consumption. This behaviour was expected because the Beltrami flow is an isotropic nonlinear method and thus it is not equipped with the enhancement capabilities of anisotropic nonlinear methods, hence these two methods are not directly comparable.
TOMOBFLOW and the iterative median filtering were also compared in terms of computation time. The average time per iteration was computed in both methods (in a standard computer based on Intel Core 2 processor 2.4 GHz running under linux) and the ratio between both was then calculated. For the six datasets, which had very different sizes (from 14 MB to 390 MB), it turned out that a single iteration in the median filtering took around 20 times more than a single iteration of TOMOBFLOW, regardless of the data size. As the number of iterations of TOMOBFLOW is usually between 50 and 150, this involves that the computation times for both methods are of the same order of magnitude (1–3 minutes for the datasets and the computer tested here). As far as memory consumption is concerned, TOMOBFLOW only used space for one copy of the dataset, as described above. It thus required half the amount of memory allocated by the median filtering (two copies of the volume) as implemented in Bsoft.
TOMOBFLOW allows efficient noise reduction with levels of background smoothing and feature preservation better than other comparable standard nonlinear filtering methods. TOMOBFLOW applies an isotropic nonlinear filtering method based on the Beltrami flow, which tunes the strength of the smoothing according to a local edge indicator. In contrast to anisotropic nonlinear filtering (e.g. AND), there is no enhancement of features since the direction of the smoothing is not tuned. Therefore, it must not be expected that TOMOBFLOW will outperform AND. In this regard, the comparison with AND carried out in this work suggests that the method based on the Beltrami flow lies between the median filtering and the AND methods.
The main advantage of the method implemented in TOMOBFLOW stems from the fact that there is no need for complicated parameter tuning. Nevertheless, it is indeed an iterative method and one thus needs to specify a number of iterations. But this does not pose a serious inconvenience as the program easily allows an experiment to be continued with further iterations, if necessary. On the other hand, there has been intense investigation on objective stopping criteria for iterative noise reduction methods (e.g. [4, 11]). However, none of the proposed criteria have turned out to be generally applicable and the number of iterations still remains to be fixed subjectively by visual inspection of the results (e.g. [24, 31]).
On the other hand, the computational burden involved by sophisticated diffusion-based filtering methods precludes their integration on interactive environments . The fact that the method implemented in TOMOBFLOW is not computationally expensive along with the optimized implementation in terms of memory consumption makes this filtering method very appropriate to be embedded into interactive packages [32, 33].
TOMOBFLOW allows efficient noise filtering of datasets with preservation of the features of interest, thereby yielding data better suited for post-processing, visualization and interpretation. The program is versatile to deal with different types and formats of multidimensional images produced by bioimaging techniques.
Availability and requirements
Project name: TOMOBFLOW
Project home page: http://www.ual.es/%7ejjfdez/SW/tomobflow.html
Operating system(s): Unix-based (linux, OS X, cygwin under Windows).
Programming language: C.
Other requirements: none.
License: public domain binaries.
Any restrictions to use by non-academics: none.
Lucic V, Foerster F, Baumeister W: Structural studies by electron tomography: from cells to molecules. Annu Rev Biochem 2005, 74: 833–865.
Fernandez JJ, Sorzano COS, Marabini R, Carazo JM: Image processing and 3D reconstruction in electron microscopy. IEEE Signal Process Mag 2006, 23(3):84–94.
Leis AP, Beck M, Gruska M, Best C, Hegerl R, Baumeister W, Leis JW: Cryo-electron tomography of biological specimens. IEEE Signal Process Mag 2006, 23(3):95–103.
Fernandez JJ, Li S: Anisotropic nonlinear filtering of cellular structures in cryo-electron tomography. Computing in Science and Engineering 2005, 7(5):54–61.
Scharr H, Uttenweiler D: 3D anisotropic diffusion filtering for enhancing noisy actin filament fluorescence images. Lecture Notes in Computer Science 2001, 2191: 69–75.
Uttenweiler D, Weber C, Jahne B, Fink RH, Scharr H: Spatiotemporal anisotropic diffusion filtering to improve SNRs and object restoration in fluorescence microscopic image sequences. J Biomed Opt 2003, 8: 40–47.
Wang Y: Computational restoration of fluorescence images: noise reduction, deconvolution, and pattern recognition. Meth Cell Biol 2007, 81: 435–445.
Fontana J, Lopez-Montero N, Elliot RM, Fernandez JJ, Risco C: The unique architecture of Bunyamwera virus factories around the Golgi complex. Cell Microbiol 2008, 10: 2012–2028.
Gonzalez RC, Woods RE: Digital Image Processing. 3rd edition. Upper Saddle River, New Jersey: Prentice Hall; 2008.
Weickert J: Anisotropic Diffusion in Image Processing. Stuttgart: Teubner; 1998.
Fernandez JJ, Li S: An improved algorithm for anisotropic nonlinear diffusion for denoising cryo-tomograms. J Struct Biol 2003, 144: 152–161.
Frangakis AS, Hegerl R: Noise reduction in electron tomographic reconstructions using nonlinear anisotropic diffusion. J Struct Biol 2001, 135: 239–250.
Tabik S, Garzon EM, Garcia I, Fernandez JJ: High performance noise reduction for biomedical multidimensional data. Digital Signal Processing 2007, 17: 724–736.
Heide P, Xu XP, Marsh BJ, Hanein D, Volkmann N: Efficient automatic noise reduction of electron tomographic reconstructions based on iterative median filtering. J Struct Biol 2007, 158: 196–204.
Fernandez JJ, Li S, Lucic V: Three-dimensional anisotropic noise reduction with automated parameter tuning: application to electron cryotomography. Lecture Notes in Computer Science 2007, 4788: 60–69.
Kimmel R, Sochen NA, Malladi R: From high energy physics to low level vision. Lecture Notes in Computer Science 1997, 1252: 236–247.
Kimmel R, Malladi R, Sochen NA: Images as embedded maps and minimal surfaces: movies, color, texture, and volumetric medical images. Int J Comput Vis 2000, 39: 111–129.
Fernandez JJ: High performance computing in structural determination by electron cryomicroscopy. J Struct Biol 2008, 164: 1–6.
Heymann JB, Belnap DM: Bsoft: image processing and molecular modeling for electron microscopy. J Struct Biol 2007, 157: 3–18.
Martone ME, Gupta A, Wong M, Qian X, Sosinsky G, Ludaescher B, Ellisman MH: A cell centered database for electron tomographic data. J Struct Biol 2002, 138: 145–155.
The Cell Centered Database[http://ccdb.ucsd.edu]
Tagari M, Tate J, Swaminathan GJ, Newman R, Naim A, Vranken W, Kapopoulou A, Hussain A, Fillon J, Henrick K, Velankar S: E-MSD: improving data deposition and structure quality. Nucleic Acids Res 2006, 34: D287-D290.
The Electron Microscopy Data Bank at the EBI Macromolecular Structure Database[http://www.ebi.ac.uk/pdbe/emdb/]
Cyrklaff M, Risco C, Fernandez JJ, Jimenez MV, Esteban M, Baumeister W, Carrascosa JL: Cryo-electron tomography of Vaccinia virus. Proc Natl Acad Sci USA 2005, 102: 2772–2777.
Heymann JB, Cardone G, Winkler DC, Steven AC: Computational resources for cryo-electron tomography in Bsoft. J Struct Biol 2008, 161: 232–242.
Rouiller I, Xu XP, Amann KJ, Egile C, Nickell S, Nicastro D, Li R, Pollard TD, Volkmann N, Hanein D: The structural basis of actin filament branching by the Arp2/3 complex. J Cell Biol 2008, 180: 887–895.
McEwen BF, Renken C, Marko M, Mannella C: Principles and practice in electron tomography. Meth Cell Biol 2008, 89: 129–168.
Briggs JA, Grunewald K, Glass B, Forster F, Krausslich HG, Fuller SD: The mechanism of HIV-1 core assembly: insights from three-dimensional reconstructions of authentic virions. Structure 2006, 14: 15–20.
Yakushevska AE, Lebbink MN, Geerts WJC, Spek L, van Donselaar EG, Jansen KA, Humbel BM, Post JA, Verkleij AJ, Koster AJ: STEM tomography in cell biology. J Struct Biol 2007, 159: 381–391.
Geissler A, Gartus A, Foki T, Tahamtan AR, Beisteiner R, Barth M: Contrast-to-noise ratio (CNR) as a quality parameter in fMRI. J Magn Reson Imaging 2007, 25: 1263–1270.
Barcena M, Oostergetel GT, Bartelink W, Faas FG, Verkleij A, Rottier PJ, Koster AJ, Bosch BJ: Cryo-electron tomography of mouse hepatitis virus: insights into the structure of the coronavirion. Proc Natl Acad Sci USA 2009, 106: 582–587.
Pruggnaller S, Mayr M, Frangakis AS: A visualization and segmentation toolbox for electron microscopy. J Struct Biol 2008, 164: 161–165.
Messaoudi C, Boudier T, Sorzano C, Marco S: TomoJ: tomography software for three-dimensional reconstruction in transmission electron microscopy. BMC Bioinformatics 2007, 8: 288.
Dr. G. Perkins kindly provided the mitochondrion dataset. The anonymous reviewers provided helpful suggestions to improve the manuscript. Work partially supported by grants MCI-TIN2008-01117, MCI-PR2008-0273, JA-P06-TIC-01426 and EU-LSHG-CT-2004-502828.
JJF conceived and designed the work, developed the program, carried out the experiments, interpreted the resulting data and wrote the manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.