Fast processing of microscopic images using object-based extended depth of field
© The Author(s). 2016
Published: 22 December 2016
Microscopic analysis requires that foreground objects of interest, e.g. cells, are in focus. In a typical microscopic specimen, the foreground objects may lie on different depths of field necessitating capture of multiple images taken at different focal planes. The extended depth of field (EDoF) technique is a computational method for merging images from different depths of field into a composite image with all foreground objects in focus. Composite images generated by EDoF can be applied in automated image processing and pattern recognition systems. However, current algorithms for EDoF are computationally intensive and impractical, especially for applications such as medical diagnosis where rapid sample turnaround is important. Since foreground objects typically constitute a minor part of an image, the EDoF technique could be made to work much faster if only foreground regions are processed to make the composite image. We propose a novel algorithm called object-based extended depths of field (OEDoF) to address this issue.
The OEDoF algorithm consists of four major modules: 1) color conversion, 2) object region identification, 3) good contrast pixel identification and 4) detail merging. First, the algorithm employs color conversion to enhance contrast followed by identification of foreground pixels. A composite image is constructed using only these foreground pixels, which dramatically reduces the computational time.
We used 250 images obtained from 45 specimens of confirmed malaria infections to test our proposed algorithm. The resulting composite images with all in-focus objects were produced using the proposed OEDoF algorithm. We measured the performance of OEDoF in terms of image clarity (quality) and processing time. The features of interest selected by the OEDoF algorithm are comparable in quality with equivalent regions in images processed by the state-of-the-art complex wavelet EDoF algorithm; however, OEDoF required four times less processing time.
This work presents a modification of the extended depth of field approach for efficiently enhancing microscopic images. This selective object processing scheme used in OEDoF can significantly reduce the overall processing time while maintaining the clarity of important image features. The empirical results from parasite-infected red cell images revealed that our proposed method efficiently and effectively produced in-focus composite images. With the speed improvement of OEDoF, this proposed algorithm is suitable for processing large numbers of microscope images, e.g., as required for medical diagnosis.
Microscopic imaging is a widely used technique in life science in which two-dimensional images are acquired from three-dimensional cellular specimens. An important skill in microscopy is adjusting the focus in order to obtain clear images of biological features. A typical biological specimen will have several different features of interest that are located on different depths of field (DoF). Automated image acquisition can be used to acquire stacking images from different DoFs. The combined images can be processed using an algorithm to create a composite image that captures all features in-focus. This type of image is known as an extended depth of field (EDoF) image. Several algorithms have been proposed to generate EDoF images based on selecting regions with high saliency . The research efforts in [2–5] focused on improving the EDoF algorithm using pixel domain and transform domain methods. In 2004, Forster and colleagues  proposed a complex-valued wavelet transformation that can accurately measure the weight of each detail information from input images. Other computational methods for obtaining high-quality EDoF images have been proposed that involve sophisticated selection criteria based on geometric transformation techniques such as the ridgelet transform , wedgelet transform , contourlet transforms  and curvelet transform . Although all of these approaches are capable of generating high-quality EDoF images, the computational complexity of these algorithms grows quadratically with the number of pixels in each image. This high computational demand means that it is impractical to generate EDoF images from multiple specimens. In some applications of microscopy, for example medical diagnosis, sample turnaround time is very important. A more computationally efficient method for acquiring EDoF images could form the basis of a rapid automated image acquisition and diagnosis platform.
In a typical microscopic specimen, the features of biological interest are likely to be spread sparsely and unevenly over the field of view. Therefore, digital images of microscopic specimens will comprise mostly background and a minority of foreground pixels. If an image processing algorithm can identify foreground objects and selectively process only the pixels within these objects, the overall image processing time will be dramatically reduced. Microscopy-based medical diagnosis typical requires detailed observations of samples involving many fields of view, since features of interest, e.g., parasites, are sparsely distributed. Therefore, to confirm diagnosis, standard operating procedure requires processing of many images. For example, in diagnosis of malaria infection, greater than 100 fields of view must be examined . In this work, we present a novel image fusion technique based on the extended depth of field concept, called object-based extended depth of field (OEDoF). The proposed OEDoF workflow constructs the final EDoF composite image by focusing only on specific regions that contain objects of interest and thus dramatically cuts down the computational time. This algorithm is implemented as an ImageJ plugin and was used to reconstruct composite images from multiple optical sectioned images of biological specimens obtained from a malaria diagnostic laboratory. The implemented OEDoF software and the images used in this paper are publicly available for downloading from http://www4a.biotec.or.th/GI/tools/oedof.
The data used to test the algorithm comprised 250 images obtained from 45 thick film slides prepared from malaria-infected blood specimens. The images were obtained using an in-house automated image-capturing platform, and were published previously in . No new samples were collected for this study, and thus no ethical approval is required. The resolution of these images was set to 928 × 616 pixels with 24 bit depth. The computational processing of the stack images to create composite EDoF images was conducted on a MacBook Pro notebook (Apple Inc., USA) equipped with an Intel core 2 duo 2.4Ghz processor and 8 GB random access memory.
Object-based extended depth of field (OEDoF) algorithm
Preprocessing: color conversion
Object region identification
Good contrast pixel identification
The aim of this process is to identify good contrast pixels to be incorporated into the final composite EDoF image. This is because pixels with higher contrast are likely to be more in-focus than the ones with lower contrast. For each pixel, we calculate the value representing the pixel contrast by comparing with the eight adjacent pixel neighbors and the corresponding pixels on the top and bottom layer depths. We adopted the 3D Sobel kernel operator  to compute the underlying contrast value for each pixel.
Note that the above Sobel filtering step only operates on the foreground pixels; thus the selection of pixels with high contrast can be done quickly. The pixels of greatest contrast from all layers are used to reconstruct the final composite image in the next step.
Detail merging for image reconstruction
We perform the image reconstruction by combining the pixels of greatest contrast from individual depth fields together. In particular, the pixels from R, G and B components representing the highest gradient magnitude G are used to construct the final composite image. To complete the image reconstruction, we replace all pixels from the baseline image previously selected during the image preprocessing (color conversion) with the highest contrast pixels identified by the algorithm.
We created a software tool to generate a composite focused image using the proposed OEDoF technique. The tool is implemented as an ImageJ plugin using Java language. This plugin can be downloaded from http://www4a.biotec.or.th/GI/tools/oedof. The instruction on how to use the OEDoF plugin in ImageJ as well as the sample images used in our study are also available from this website.
Results and Discussion
On image quality
On processing time
Where m and n are the image dimensions and i, j are pixel positions in x and y directions. G x and G y are the 3 × 3 Sobel horizontal and vertical kernels, respectively.
The OEDoF algorithm can create composite images with foreground objects in focus that are comparable in quality to the state-of-the-art complex wavelet EDoF algorithm, but with four-fold faster processing time. The greater computational efficiency of OEDoF is achieved by selectively processing pixels from object regions only instead of the entire image. Although some image quality is sacrificed for speed, the composite images produced by OEDoF retain foreground details sufficient for downstream biomedical image processing, in applications such as counting infected cells, differentiating malaria species, etc. Furthermore, the threshold used to identify foreground objects will depend on the contrast, which will vary depending upon the type of specimens being examined. We identified a suitable threshold for thick film specimens of malaria cases; however, other biomedical specimens, e.g., histological specimens may require a different threshold. The proposed technique should also work well with microscopic images obtained from most stained biological specimens in which backgrounds are brighter than object regions. The marked improvement in image processing time is particularly important for medical diagnosis where rapid turnaround is required. For example, microscopy-based malaria diagnosis requires at least 100 images per specimen . The four-fold reduction in time using OEDoF technique could translate to improved diagnosis and treatment of diseases.
AI, SK and ST are partially supported by the grant from National Science and Technology Development Agency (NSTDA), Thailand. ST and PJS are also supported by the Thailand Research Fund mid-career research grants, RSA5860081 and RSA5780007, respectively.
About this supplement
This article has been published as part of BMC Bioinformatics Volume 17 Supplement 19, 2016. 15th International Conference On Bioinformatics (INCOB 2016): bioinformatics. The full contents of the supplement are available online https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement-19.
Publication of this article was funded by the National Electronics and Computer Center, National Science and Technology Development Agency, Thailand.
Availability of supporting data
The instruction of the software and the thick blood film images supporting the results of this paper are available to download from our website, http://www4a.biotec.or.th/GI/tools/oedof.
SK, AI and ST conceived the idea of the object based approach. SK processed the image data and drafted the manuscript. AI wrote the ImageJ plugin and verified the analysis results on the test data. PJS assisted with the biological interpretation of the analyzed data and drafting of the manuscript. MP assisted with image acquisition. ST and PJS revised the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Valdecasas GA, Marshall D, Becerra JM, Terrero JJ. On the extended depth of focus algorithms for bright field microscopy. Micron. 2001;32:559–69.View ArticlePubMedGoogle Scholar
- Dowski ER, Cathey WT. Extended depth of field through wavefront coding. Appl Opt. 1995;34(11):1859–66.View ArticlePubMedGoogle Scholar
- Li H, Manjunath BS, Mitra SK. Multisensor image fusion using the wavelet transform. Graphical Models and Image Processing. 1995;57(3):235–45.View ArticleGoogle Scholar
- Castro A, Ojeda-Castaneda J. Asymmetric phase masks for extended depth of field. Appl Opt. 2004;43(17):3474–9.View ArticlePubMedGoogle Scholar
- Forster B, Van De Ville D, Berent J, Sage D, Unser M. Complex wavelets for extended depth-of-field: A new method for the fusion of multichannel microscopy images. Microsc Res Tech. 2004;65(1–2):33–42.View ArticlePubMedGoogle Scholar
- Candes EJ, Donoho DL. Ridgelets: a key to higher dimensional intermittency. Philos Trans R Soc Lond A. 1999;357:2495–509.View ArticleGoogle Scholar
- Donoho DL. Wedgelets: nearly minimax estimation of edges. Ann Stat. 1999;27:859–97.View ArticleGoogle Scholar
- Do MN, Vetterli M. The contourlet transforms: An efficient directional multiresolution image represenation. IEEE Trans Image Process. 2005;14(12):2091–106.View ArticlePubMedGoogle Scholar
- Tessan L, Ledda A, Pizurica A, Philips W. Extending the depth of field in microscopy through curvelet-based frequency-adaptive image fusion. ICASSP 2007. doi:10.1109/ICASSP.2007.366044.
- World Health Organization. Basic Malaria Microscopy Part 1. Learner2019s Guide. 2010.Google Scholar
- Kaewkamnerd S, Intarapanich A, Pannarut M, Chaotheing S, Uthaipibull C, Tongsima S: Detection and Classification Device for Malaria Parasites in Thick-blood Films. BMC Bioinformatics. 2012;13(Suppl 17):S18. doi:10.1186/1471-2105-13-S17-S18.View ArticlePubMedPubMed CentralGoogle Scholar
- Vera E, Torres S. Adaptive color space transform using independent component analysis. Proc Of SPIE-IS& T Electronics Imaging. 2007;6497:64970P–12.Google Scholar
- Szeliski R. Computer Vision: Algorithms and Applications. Springer; 2010. ISBN:978-1-84882-935-0.Google Scholar
- Rangaraj MR, Biomedical Image Analysis. second edition. Wiley-IEEE; 2015. ISBN:978-0-470-91139-6.Google Scholar