Automated image computing reshapes computational neuroscience
BMC Bioinformatics volume 14, Article number: 293 (2013)
We briefly identify several critical issues in current computational neuroscience, and present our opinions on potential solutions based on bioimage informatics, especially automated image computing.
Computational neuroscience is undergoing a transformation. Traditionally, this field has focused on studying information processing in nervous systems by collecting, analyzing, and simulating neuronal electrophysiology data [1, 2]. Increasingly, the emphasis is shifting towards neuromorphological pattern analysis and brain atlas modeling using advanced microscopy and image computing. One important motivation for this changing focus is the need to ground our understating of animal behavior in real three-dimensional (3D) neuronal morphologies and connectivity. When dynamic imaging of living nervous tissue is possible, there is an additional opportunity to investigate the time-dependent molecular mechanisms in brain tissue in a data-driven manner. Advances in automated image computing are making such studies practical and informative.
Image computing is critical for mapping brain and building neuron databases
A common goal among many current studies is to map brain anatomy by systematically characterizing the distribution, projection, and connectivity of neurons throughout a nervous system. Pursuing this ambitious goal is increasingly practical, due to converging advances in neuron labeling [3, 4], histological preparation , multi-dimensional imaging [6, 7], and image computing [8, 9]. Several large-scale 3D imaging-based computational neuroscience initiatives, such as the HHMI Janelia FlyLight (http://janelia.org/team-project/fly-light) and FlyEM projects (http://janelia.org/team-project/fly-em), the Allen Mouse Brain “MindScope”  and Connectivity Atlas (http://alleninstitute.org/science/public_resources/atlases/connectivity_atlas.html), and the Harvard mouse brain connectome project (http://cbs.fas.harvard.edu/science/connectome-project), have been launched to acquire massive datasets, in which each image contains hundreds of millions of pixels.
The task of storing, organizing, and disseminating these vast datasets has driven the development of several neuronal databases and digital brain atlases. For example, http://NeuroMorpho.Org is a web-accessible database that allows researchers to share neuron reconstructions, quantify neuronal morphology in a standardized manner, mine the resulting morphological features, and use them in computational modeling and neuronal network simulation studies . The digital representations of light-level single neurons in http://NeuroMorpho.Org is associated with rich metadata, but only coarse information about the cellular location and orientation relative to surrounding brain structures. In contrast, 3D digital brain atlases contain rich and integrated information on the spatial coordinates of 3D reconstructed neurons, gene expression, functional modules, and networks, all mapped into a ‘standard’ view of brain anatomy. Examples of recent digital atlases include a C. elegans connectome  produced from electron microscopic images, a large-scale single-neuron atlas (http://flycircuit.org) and a stereotypy atlas of major neurite tracts  of the adult fruit fly brain, both produced from confocal microscopic images, a genome-wide gene expression map of the mouse brain produced using wide-field microscopic images (http://brain-map.org), and a high-resolution reconstruction of the rat hippocampus .
Producing databases and atlases of neuronal structures from microscopy data entails a pipeline of image processing and analysis (Figure 1) (also see ). Key steps include (1) image pre-processing: correct images for imaging artifacts and standardize the raw data; (2) registration: align brain morphology and stitch together separately imaged brain regions; (3) segmentation and validation: trace neuron morphology and ensure that the reconstructions are correct; (4) feature extraction: make detailed quantitative measurements of traced neurons; and (5) analytics: model, infer, and predict the identities of neurons or neuronal patterns using morphology, location, genetic, lineage, and other extracted features. Many of these steps, such as 3D neuron tracing, are also widely used in smaller-scale analysis of neuronal phenotypes.
The challenge of full automation
The long-standing need to automate the laborious and subjective manual analysis of light-microscopic and electron-microscopic images has motivated a large number of bioimage informatics efforts [8, 17-21]. The recent spurt in imaging throughput, combined with the desire for large-scale computational modeling, has added a sense of urgency to this need. For instance, manual tracing of neuron morphology is prohibitively expensive for analyzing image data that is approaching the scale of terabytes and thousands of image stacks, let alone mining the higher-order associations in these data.
For computational neuroscience, this great demand motivated several major institutions to conduct the worldwide DIADEM Challenge (http://diademchallenge.org) in 2010 as a way to stimulate progress and attract new computational researchers to join the technology development community. The challenge was to develop algorithms capable of automatically tracing stacks of images visualizing the tree-like shape of neuronal axons and dendrites into 3D digital reconstructions. This competition succeeded in stimulating a burst of progress. Five finalist-teams were selected from more than one hundred entries. A novel metric methodology was custom designed to compare automated and manual reconstructions . While none of the algorithms presented at the finishing stage reached the official goal of a 20-fold speed-up in the reconstruction process, all finalists’ software programs were made publicly downloadable along with the challenge data sets and the DIADEM metric, providing a useful toolbox for continuous and future development.
To achieve full automation, libraries of computer programs for neuronal pattern extraction, comparison and inference must be integrated into toolkits for multi-dimensional image visualization and analysis (e.g. http://vaa3d.org; http://farsight-toolkit.org). Optimizing each step of automated image computing (Figure 1) is also important to ensure a lower error rate compared to humans. In addition to a metric of precision, automation of image computing should also be assessed using measures of the speed-up compared to manual work, as well as the robustness of the method with respect to changes of its parameters and the type and level of noise in the data. These aspects relate to the scalability to large-scale applications and the generalization to similar or new problems. The above considerations apply to image data produced by most imaging modalities. Despite considerable progress, many computational problems remain open. For instance, the DIADEM neuron reconstruction challenge stands unmet. Part of the reason could be that only relatively small image data sets on neuron morphology were previously available to researchers. We hope this situation will change soon due to the availability of benchmark datasets and the strong demand of very large-scale single neuron screening projects in the next five years.
Making computer programs more intelligent
Increasingly, researchers have developed ways to make some of these algorithms more intelligent. Domain-specific prior knowledge can be used to improve the “intelligence” of an algorithm, akin to the observation that a “supervised” machine-learning algorithm generally performs better (e.g. predicts more accurately) than an “unsupervised” one. How to incorporate domain knowledge in an algorithm? One way is to code this information into explicit rules. For instance, the spatial location of cells coded as directed graphs has been used as constraints in training a nuclear image segmentation and recognition algorithm . In another example, particular types of neuron tracing errors could be propagated through a large set of results to identify other potential erroneous reconstruction loci . However, in many cases it is hard to enumerate all possible rules and code them explicitly. Rules may also become contradictory, biased, or “fuzzy” under different circumstances. Therefore, another more competent way to provide the domain knowledge is needed.
Interesting solutions may originate from comprehensive neuron databases and brain anatomy atlases. The reciprocal relationship that links neuroscience databases and atlases with image computing methods offers a synergistic opportunity to improve both of them iteratively. In both atlases and databases, it is possible to quantify neuronal patterns from many samples statistically. The statistics provide good “rules” or “priors”, and new paradigms, for improving image-computing algorithms. For example, since an atlas of cells is a comprehensive model of the expected yet spatially deformed objects (cells) in the observed image data, a model-based search approach can be designed to achieve registration, segmentation, and recognition at the same time. An automated algorithm for simultaneous cell segmentation and recognition of C. elegans cells  uses a standard 3D nuclear atlas of this animal to achieve much better performance than the more intuitive approach of performing these tasks in separate steps.
When neuron locations are stereotypical, atlas-based image-computing methods can be powerful to design novel biological experiments. The spatial variation of most nuclei, including many neurons, in freshly hatched C. elegans is about 2 microns , just about the average size of a nucleus. For adult fruit fly brains, this variation for the major neurite tracts is merely ~3 microns, orders of magnitude smaller than the dimensions of an entire brain . Therefore, it is possible to design new computer-driven instruments to repeatedly target the same neuron or neurite tract, locally perturb the neuronal circuits of these animals in real time experiments, and observe the response of the nervous system. Among ongoing efforts, the Janelia SmartScope project aims at leveraging new image-guided computer programs to drive a 3D laser-scanning-microscope to target and stimulate single cells precisely in 3D and observe their downstream activities, using optogenetics and calcium imaging.
Studying dynamics and functions of neural circuits
Functional connectivity and dynamics of neuronal connections are two other important topics of computational neuroscience. Neuronal dynamics are reflected in structure and activity at a variety of temporal scales, ranging from arbor growth and cell migration (days to years), through dendritic spine twitching and axonal bouton crawling (minutes to hours), to neuron firing (milliseconds to seconds). Intravital imaging  provides excellent means to visualize these temporal changes. The dynamics can then be studied using four-dimensional computational image analysis. For instance, tracking the 3D morphological change of neurons over time from time-lapse imaging data offers great opportunity to improve neuronal classification and characterize the structural plasticity of growing, degenerating, or regenerating neurons . Anchoring corresponding branching points of varying neuron morphology and detecting persistent and variable structures (branch tips, boutons, spines) from multi-dimensional image series are useful steps to analyze neuronal dynamics. Although the automation of these image analysis tasks is still new, it is highly desired to quantitatively study these problems at large scale.
In another example, 3D segmentation and tracking of somatic calcium imaging data provide a powerful way to study neuron activity patterns. Computational causal analysis might also enable interesting analysis of synaptic connections  (see however ). Imaging neuronal firing with possibly genetically encoded voltage-gated dyes or fast calcium indicators could enable the co-detection of neural network structure and activity. Using brain atlases and computational image-guided analysis, it is possible to pinpoint the firing sites in vivo and record the firing patterns in real-time experiments. This approach could complement the recent non-image-guided automatic whole-cell patch-clamp electrophysiology .
Correlative analysis of microscopic image data of neurons and other functional data, such as electrophysiology and optical-physiology recording and behavioral assays, can become a powerful approach to gain useful information and knowledge from a large amount of data, and may lead to the ultimate understanding of how individual neurons and their networks work. For instance, digitization of the dragonfly descending neurons’ morphology from confocal images has been correlated with the electrical recording and measurement of the receptive fields of these neurons . This analysis led to a new finding that the average population vector of the neuronal response of a surprisingly small number of descending neurons actually precisely controls the wings of the animal in targeting preys . On the other hand, for behavior assays, systematic video-analysis based high-throughput studies have yielded powerful new ways to associate the neuronal circuits of both insects and mammals with either the stimuli (such as the mouse whisker system ) or the behavior output (such as the Drosophila trajectories ). In the work of , a machine learning system enables biologists to automatically annotate the behaviors of animals in each frame of a video. It can be used to detect a wide variety of social and locomotion behaviors across flies, mice, and larvae.
Recent years have witnessed an acceleration of the ongoing transformation of computational neuroscience due to image computing automation. For example, neurite outgrowth assays employing automated reconstruction of neurons in high-throughput high-content environments are now common, and are used for testing hypotheses, screening, and toxicology studies [34-36]. At a more advanced level, new high-throughput methods for accurate neurite reconstruction  have allowed analysis of the specific wiring underlying computation of direction-selectivity in retinal circuits . More generally, these kinds of structural neurobiology studies may systematically provide the much needed mechanistic understanding of neural computation  if employed in large-scale automated applications  of new techniques for the anatomical and molecular interrogation of intact biological systems .
We are optimistic that in the next five years more automated and intelligent computer programs for analyzing large-scale brain images will yield tangible advancements in computational neuroscience. The neuron anatomy projects for various model animals will start to deliver integrated data sets, organized in neuron databases and brain atlases. This will provide invaluable data resources for the research community to analyze. This may lead to a paradigm shift similar to the massive rise of mining and reanalysis of ‘secondary’ (archived) data following the sudden abundance of genome sequences 10 years ago. Neuron databases and brain atlases are also going to incorporate more dynamic and functional information, such as from time-lapse and calcium imaging of interacting neurons. More realistic computational models of neurons and their networks may foster a deeper understanding of the functioning mechanisms of nervous systems, paving the path to re-create a brain in silico.
Schwartz E: Computational neuroscience. 1990, Cambridge, Mass: MIT Press, ISBN 0-262-19291-8
Markram H: The Blue Brain Project. Nat Rev Neurosci. 2006, 7: 153-160.
Livet J, et al: Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nat. 2007, 450: 56-62. 10.1038/nature06293.
Feinberg EH, et al: GFP Reconstitution Across Synaptic Partners (GRASP) defines cell contacts and synapses in living nervous systems. Neuron. 2008, 57: 353-363. 10.1016/j.neuron.2007.11.030.
Hama H, et al: Scale: a chemical approach for fluorescence imaging and reconstruction of transparent mouse brain. Nat Neurosci. 2011, 14: 1481-1488. 10.1038/nn.2928.
Micheva KD, Smith SJ: Array tomography: A new tool for imaging the molecular architecture and ultrastructure of neural circuits. Neuron. 2007, 55: 25-36. 10.1016/j.neuron.2007.06.014.
Sigrist SJ, Sabatini BL: Optical super-resolution microscopy in neurobiology. Curr Opin Neurobiol. 2011, 22: 86-93.
Peng H: Bioimage informatics: a new area of engineering biology. Bioinformatics. 2008, 24: 1827-1836. 10.1093/bioinformatics/btn346.
Helmstaedter M, Mitra PP: Computational methods and challenges for large-scale circuit mapping. Curr Opin Neurobiol. 2012, 22: 162-169. 10.1016/j.conb.2011.11.010.
Koch C, Reid C: Observatories of the mind. Nat. 2012, 483: 397-398. 10.1038/483397a.
Ascoli GA, Donohue DE, Halavi M: NeuroMorpho.Org - A central resource for neuronal morphologies. J Neurosci. 2007, 27: 9247-9251. 10.1523/JNEUROSCI.2055-07.2007.
Chen BL, Hall DH, Chklovskii DB: Wiring optimization can relate neuronal structure and function. PNAS. 2006, 103: 4723-4728. 10.1073/pnas.0506806103.
Chiang AS, et al: Three-dimensional reconstruction of brain-wide wiring networks in Drosophila at single-cell resolution. Curr Biol. 2011, 21: 1-11.
Peng H, et al: BrainAligner: 3D registration atlases of Drosophila brains. Nat Methods. 2011, 8: 493-498. 10.1038/nmeth.1602.
Lein ES, et al: Genome-wide atlas of gene expression in the adult mouse brain. Nat. 2007, 445: 168-176. 10.1038/nature05453.
Ropireddy D, Bachus SE, Ascoli GA: Non-homogeneous stereological properties of the rat hippocampus from high-resolution 3D serial reconstruction of thin histological sections. Neurosci. 2012, 205: 91-111.
Donohue DE, Ascoli GA: Automated reconstruction of neuronal morphology: an overview. Brain Res Rev. 2011, 67: 94-102. 10.1016/j.brainresrev.2010.11.003.
Türetken E, González G, Blum C, Fua P: Automated reconstruction of dendritic and axonal trees by global optimization with geometric priors. Neuroinformatics. 2011, 9: 279-302. 10.1007/s12021-011-9122-1.
Liu Y: The DIADEM and beyond. Neuroinformatics. 2011, 9: 99-102. 10.1007/s12021-011-9102-5.
He HY, Cline HT: Diadem X: Automated 4 dimensional analysis of morphological data. Neuroinformatics. 2011, 9: 107-112. 10.1007/s12021-011-9098-x.
Jain V, Seung H, Turaga SC: Machines that learn to segment images: a crucial technology for connectomics. Curr Opin Neurobiol. 2010, 20: 653-666. 10.1016/j.conb.2010.07.004.
Gillette TA, Brown KM, Ascoli GA: The DIADEM metric: comparing multiple reconstructions of the same neuron. Neuroinformatics. 2011, 9: 233-245. 10.1007/s12021-011-9117-y.
Peng H, Ruan Z, Long F, Simpson JH, Myers EW: V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat Biotechnol. 2010, 28: 348-353. 10.1038/nbt.1612.
Long F, Peng H, Liu X, Kim SK, Myers E: A 3D digital atlas of C. elegans and its application to single-cell analyses. Nat Methods. 2009, 6: 667-672. 10.1038/nmeth.1366.
Luisi J, Narayanaswamy A, Galbreath Z, Roysam B: The FARSIGHT trace editor: an open source tool for 3-D inspection and efficient pattern analysis aided editing of automated neuronal reconstructions. Neuroinformatics. 2011, 9: 305-315. 10.1007/s12021-011-9115-0.
Qu L, et al: Simultaneous recognition and segmentation of cells: application in C. elegans. Bioinformatics. 2011, 27: 2895-2902. 10.1093/bioinformatics/btr480.
Pittet M, Weissleder R: Intravital Imaging. Cell. 2011, 147: 983-991. 10.1016/j.cell.2011.11.004.
Gerhard F, Pipa G, Lima B, Neuenschwander S, Gerstner W: Extraction of network topology from multi-electrode recordings: is there a small-world effect?. Front. Comput. Neurosci. 2011, 5: doi:10.3389/fncom.2011.00004
Srinivasan S, Stevens CF: Robustness and fault tolerance make brains harder to study. BMC Biol. 2011, 9 (46): doi:10.1186/1741-7007-9-46
Kodandaramaiah SB, Franzesi GT, Chow BY, Boyden ES, Forest CR: Automated whole-cell patch-clamp electrophysiology of neurons in vivo. Nat Methods. 2012, doi:10.1038/nmeth.1993
Gonzalez-Bellido PT, Peng H, Yang J, Georgopoulos AP, Olberg RM: Eight pairs of descending visual neurons in the dragonfly give wing motor centers accurate population vector of prey direction. Proc Natl Acad Sci. 2013, 110 (2): 696-701. 10.1073/pnas.1210489109.
Clack NG, O’Connor DH, Huber D, Petreanu L, Hires A, Peron S, Myers EW: Automated tracking of whiskers in videos of head fixed rodents. PLoS Comput Biol. 2012, 8 (7): e1002591-10.1371/journal.pcbi.1002591.
Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K: Jaaba: interactive machine learning for automatic annotation of animal behavior. Nat Methods. 2012, 10 (1): 64-67. 10.1038/nmeth.2281. doi:10.1038/nmeth.2281
Frimat JP, Sisnaiske J, Subbiah S, Menne H, Godoy P, Lampen P, Leist M, Franzke J, Hengstler JG, van Thriel C, West J: The network formation assay: a spatially standardized neurite outgrowth analytical display for neurotoxicity screening. Lab Chip. 2010, 10 (6): 701-709. 10.1039/b922193j. Mar 21
Hutson TH, Buchser WJ, Bixby JL, Lemmon VP, Moon LD: Optimization of a 96-Well Electroporation Assay for Postnatal Rat CNS Neurons Suitable for Cost-Effective Medium-Throughput Screening of Genes that Promote Neurite Outgrowth. Front Mol Neurosci. 2011, 4: 55-
Klatt S, Fassold A, Straub RH: Sympathetic nerve fiber repulsion: testing norepinephrine, dopamine, and 17β-estradiol in a primary murine sympathetic neurite outgrowth assay. Ann N Y Acad Sci. 2012, 1261: 26-33. 10.1111/j.1749-6632.2012.06628.x. Jul
Helmstaedter M, Briggman KL, Denk W: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat Neurosci. 2011, 14 (8): 1081-1088. 10.1038/nn.2868. Jul 10
Briggman KL, Helmstaedter M, Denk W: Wiring specificity in the direction-selectivity circuit of the retina. Nature. 2011, 471 (7337): 183-8. 10.1038/nature09818. Mar 10
Denk W, Briggman KL, Helmstaedter M: Structural neurobiology: missing link to a mechanistic understanding of neural computation. Nat Rev Neurosci. 2012, 13 (5): 351-358. Nov 9
Kleinfeld D, Bharioke A, Blinder P, Bock DD, Briggman KL, Chklovskii DB, Denk W, Helmstaedter M, Kaufhold JP, Lee WC, Meyer HS, Micheva KD, Oberlaender M, Prohaska S, Reid RC, Smith SJ, Takemura S, Tsai PS, Sakmann B: Large-scale automated histology in the pursuit of connectomes. J Neurosci. 2011, 31 (45): 16125-16138. 10.1523/JNEUROSCI.4077-11.2011. Nov 9
Chung K, Wallace J, Kim SY, Kalyanasundaram S, Andalman AS, Davidson TJ, Mirzabekov JJ, Zalocusky KA, Mattis J, Denisin AK, Pak S, Bernstein H, Ramakrishnan C, Grosenick L, Gradinaru V, Deisseroth K: Structural and molecular interrogation of intact biological systems. Nat. 2013, 497 (7449): 332-337. 10.1038/nature12107. May 16
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Peng, H., Roysam, B. & Ascoli, G.A. Automated image computing reshapes computational neuroscience. BMC Bioinformatics 14, 293 (2013). https://doi.org/10.1186/1471-2105-14-293
- Image Computing
- Neuron Morphology
- Computational Neuroscience
- Neuron Reconstruction
- Allen Mouse Brain