Skip to main content

Automated image computing reshapes computational neuroscience


We briefly identify several critical issues in current computational neuroscience, and present our opinions on potential solutions based on bioimage informatics, especially automated image computing.

Computational neuroscience is undergoing a transformation. Traditionally, this field has focused on studying information processing in nervous systems by collecting, analyzing, and simulating neuronal electrophysiology data [1, 2]. Increasingly, the emphasis is shifting towards neuromorphological pattern analysis and brain atlas modeling using advanced microscopy and image computing. One important motivation for this changing focus is the need to ground our understating of animal behavior in real three-dimensional (3D) neuronal morphologies and connectivity. When dynamic imaging of living nervous tissue is possible, there is an additional opportunity to investigate the time-dependent molecular mechanisms in brain tissue in a data-driven manner. Advances in automated image computing are making such studies practical and informative.

Image computing is critical for mapping brain and building neuron databases

A common goal among many current studies is to map brain anatomy by systematically characterizing the distribution, projection, and connectivity of neurons throughout a nervous system. Pursuing this ambitious goal is increasingly practical, due to converging advances in neuron labeling [3, 4], histological preparation [5], multi-dimensional imaging [6, 7], and image computing [8, 9]. Several large-scale 3D imaging-based computational neuroscience initiatives, such as the HHMI Janelia FlyLight ( and FlyEM projects (, the Allen Mouse Brain “MindScope” [10] and Connectivity Atlas (, and the Harvard mouse brain connectome project (, have been launched to acquire massive datasets, in which each image contains hundreds of millions of pixels.

The task of storing, organizing, and disseminating these vast datasets has driven the development of several neuronal databases and digital brain atlases. For example, http://NeuroMorpho.Org is a web-accessible database that allows researchers to share neuron reconstructions, quantify neuronal morphology in a standardized manner, mine the resulting morphological features, and use them in computational modeling and neuronal network simulation studies [11]. The digital representations of light-level single neurons in http://NeuroMorpho.Org is associated with rich metadata, but only coarse information about the cellular location and orientation relative to surrounding brain structures. In contrast, 3D digital brain atlases contain rich and integrated information on the spatial coordinates of 3D reconstructed neurons, gene expression, functional modules, and networks, all mapped into a ‘standard’ view of brain anatomy. Examples of recent digital atlases include a C. elegans connectome [12] produced from electron microscopic images, a large-scale single-neuron atlas ([13]) and a stereotypy atlas of major neurite tracts [14] of the adult fruit fly brain, both produced from confocal microscopic images, a genome-wide gene expression map of the mouse brain produced using wide-field microscopic images ([15]), and a high-resolution reconstruction of the rat hippocampus [16].

Producing databases and atlases of neuronal structures from microscopy data entails a pipeline of image processing and analysis (Figure 1) (also see [9]). Key steps include (1) image pre-processing: correct images for imaging artifacts and standardize the raw data; (2) registration: align brain morphology and stitch together separately imaged brain regions; (3) segmentation and validation: trace neuron morphology and ensure that the reconstructions are correct; (4) feature extraction: make detailed quantitative measurements of traced neurons; and (5) analytics: model, infer, and predict the identities of neurons or neuronal patterns using morphology, location, genetic, lineage, and other extracted features. Many of these steps, such as 3D neuron tracing, are also widely used in smaller-scale analysis of neuronal phenotypes.

Figure 1

Major steps in building digital neuronal databases and brain atlases. After necessary image processing, the 3D morphology of a neuronal pattern is reconstructed from microscopy data. Multiple neuronal patterns need to be organized based on brain regions and neuronal types. For brain atlases, these reconstructions are mapped into a common 3D spatial coordinate system. This may involve an iterative analysis process.

The challenge of full automation

The long-standing need to automate the laborious and subjective manual analysis of light-microscopic and electron-microscopic images has motivated a large number of bioimage informatics efforts [8, 17-21]. The recent spurt in imaging throughput, combined with the desire for large-scale computational modeling, has added a sense of urgency to this need. For instance, manual tracing of neuron morphology is prohibitively expensive for analyzing image data that is approaching the scale of terabytes and thousands of image stacks, let alone mining the higher-order associations in these data.

For computational neuroscience, this great demand motivated several major institutions to conduct the worldwide DIADEM Challenge ( in 2010 as a way to stimulate progress and attract new computational researchers to join the technology development community. The challenge was to develop algorithms capable of automatically tracing stacks of images visualizing the tree-like shape of neuronal axons and dendrites into 3D digital reconstructions. This competition succeeded in stimulating a burst of progress. Five finalist-teams were selected from more than one hundred entries. A novel metric methodology was custom designed to compare automated and manual reconstructions [22]. While none of the algorithms presented at the finishing stage reached the official goal of a 20-fold speed-up in the reconstruction process, all finalists’ software programs were made publicly downloadable along with the challenge data sets and the DIADEM metric, providing a useful toolbox for continuous and future development.

To achieve full automation, libraries of computer programs for neuronal pattern extraction, comparison and inference must be integrated into toolkits for multi-dimensional image visualization and analysis (e.g.[23]; Optimizing each step of automated image computing (Figure 1) is also important to ensure a lower error rate compared to humans. In addition to a metric of precision, automation of image computing should also be assessed using measures of the speed-up compared to manual work, as well as the robustness of the method with respect to changes of its parameters and the type and level of noise in the data. These aspects relate to the scalability to large-scale applications and the generalization to similar or new problems. The above considerations apply to image data produced by most imaging modalities. Despite considerable progress, many computational problems remain open. For instance, the DIADEM neuron reconstruction challenge stands unmet. Part of the reason could be that only relatively small image data sets on neuron morphology were previously available to researchers. We hope this situation will change soon due to the availability of benchmark datasets and the strong demand of very large-scale single neuron screening projects in the next five years.

Making computer programs more intelligent

Increasingly, researchers have developed ways to make some of these algorithms more intelligent. Domain-specific prior knowledge can be used to improve the “intelligence” of an algorithm, akin to the observation that a “supervised” machine-learning algorithm generally performs better (e.g. predicts more accurately) than an “unsupervised” one. How to incorporate domain knowledge in an algorithm? One way is to code this information into explicit rules. For instance, the spatial location of cells coded as directed graphs has been used as constraints in training a nuclear image segmentation and recognition algorithm [24]. In another example, particular types of neuron tracing errors could be propagated through a large set of results to identify other potential erroneous reconstruction loci [25]. However, in many cases it is hard to enumerate all possible rules and code them explicitly. Rules may also become contradictory, biased, or “fuzzy” under different circumstances. Therefore, another more competent way to provide the domain knowledge is needed.

Interesting solutions may originate from comprehensive neuron databases and brain anatomy atlases. The reciprocal relationship that links neuroscience databases and atlases with image computing methods offers a synergistic opportunity to improve both of them iteratively. In both atlases and databases, it is possible to quantify neuronal patterns from many samples statistically. The statistics provide good “rules” or “priors”, and new paradigms, for improving image-computing algorithms. For example, since an atlas of cells is a comprehensive model of the expected yet spatially deformed objects (cells) in the observed image data, a model-based search approach can be designed to achieve registration, segmentation, and recognition at the same time. An automated algorithm for simultaneous cell segmentation and recognition of C. elegans cells [26] uses a standard 3D nuclear atlas of this animal to achieve much better performance than the more intuitive approach of performing these tasks in separate steps.

When neuron locations are stereotypical, atlas-based image-computing methods can be powerful to design novel biological experiments. The spatial variation of most nuclei, including many neurons, in freshly hatched C. elegans is about 2 microns [24], just about the average size of a nucleus. For adult fruit fly brains, this variation for the major neurite tracts is merely ~3 microns, orders of magnitude smaller than the dimensions of an entire brain [14]. Therefore, it is possible to design new computer-driven instruments to repeatedly target the same neuron or neurite tract, locally perturb the neuronal circuits of these animals in real time experiments, and observe the response of the nervous system. Among ongoing efforts, the Janelia SmartScope project aims at leveraging new image-guided computer programs to drive a 3D laser-scanning-microscope to target and stimulate single cells precisely in 3D and observe their downstream activities, using optogenetics and calcium imaging.

Studying dynamics and functions of neural circuits

Functional connectivity and dynamics of neuronal connections are two other important topics of computational neuroscience. Neuronal dynamics are reflected in structure and activity at a variety of temporal scales, ranging from arbor growth and cell migration (days to years), through dendritic spine twitching and axonal bouton crawling (minutes to hours), to neuron firing (milliseconds to seconds). Intravital imaging [27] provides excellent means to visualize these temporal changes. The dynamics can then be studied using four-dimensional computational image analysis. For instance, tracking the 3D morphological change of neurons over time from time-lapse imaging data offers great opportunity to improve neuronal classification and characterize the structural plasticity of growing, degenerating, or regenerating neurons [20]. Anchoring corresponding branching points of varying neuron morphology and detecting persistent and variable structures (branch tips, boutons, spines) from multi-dimensional image series are useful steps to analyze neuronal dynamics. Although the automation of these image analysis tasks is still new, it is highly desired to quantitatively study these problems at large scale.

In another example, 3D segmentation and tracking of somatic calcium imaging data provide a powerful way to study neuron activity patterns. Computational causal analysis might also enable interesting analysis of synaptic connections [28] (see however [29]). Imaging neuronal firing with possibly genetically encoded voltage-gated dyes or fast calcium indicators could enable the co-detection of neural network structure and activity. Using brain atlases and computational image-guided analysis, it is possible to pinpoint the firing sites in vivo and record the firing patterns in real-time experiments. This approach could complement the recent non-image-guided automatic whole-cell patch-clamp electrophysiology [30].

Correlative analysis of microscopic image data of neurons and other functional data, such as electrophysiology and optical-physiology recording and behavioral assays, can become a powerful approach to gain useful information and knowledge from a large amount of data, and may lead to the ultimate understanding of how individual neurons and their networks work. For instance, digitization of the dragonfly descending neurons’ morphology from confocal images has been correlated with the electrical recording and measurement of the receptive fields of these neurons [31]. This analysis led to a new finding that the average population vector of the neuronal response of a surprisingly small number of descending neurons actually precisely controls the wings of the animal in targeting preys [31]. On the other hand, for behavior assays, systematic video-analysis based high-throughput studies have yielded powerful new ways to associate the neuronal circuits of both insects and mammals with either the stimuli (such as the mouse whisker system [32]) or the behavior output (such as the Drosophila trajectories [33]). In the work of [33], a machine learning system enables biologists to automatically annotate the behaviors of animals in each frame of a video. It can be used to detect a wide variety of social and locomotion behaviors across flies, mice, and larvae.

Recent years have witnessed an acceleration of the ongoing transformation of computational neuroscience due to image computing automation. For example, neurite outgrowth assays employing automated reconstruction of neurons in high-throughput high-content environments are now common, and are used for testing hypotheses, screening, and toxicology studies [34-36]. At a more advanced level, new high-throughput methods for accurate neurite reconstruction [37] have allowed analysis of the specific wiring underlying computation of direction-selectivity in retinal circuits [38]. More generally, these kinds of structural neurobiology studies may systematically provide the much needed mechanistic understanding of neural computation [39] if employed in large-scale automated applications [40] of new techniques for the anatomical and molecular interrogation of intact biological systems [41].

Future prospects

We are optimistic that in the next five years more automated and intelligent computer programs for analyzing large-scale brain images will yield tangible advancements in computational neuroscience. The neuron anatomy projects for various model animals will start to deliver integrated data sets, organized in neuron databases and brain atlases. This will provide invaluable data resources for the research community to analyze. This may lead to a paradigm shift similar to the massive rise of mining and reanalysis of ‘secondary’ (archived) data following the sudden abundance of genome sequences 10 years ago. Neuron databases and brain atlases are also going to incorporate more dynamic and functional information, such as from time-lapse and calcium imaging of interacting neurons. More realistic computational models of neurons and their networks may foster a deeper understanding of the functioning mechanisms of nervous systems, paving the path to re-create a brain in silico.


  1. 1.

    Schwartz E: Computational neuroscience. 1990, Cambridge, Mass: MIT Press, ISBN 0-262-19291-8

    Google Scholar 

  2. 2.

    Markram H: The Blue Brain Project. Nat Rev Neurosci. 2006, 7: 153-160.

    Article  CAS  PubMed  Google Scholar 

  3. 3.

    Livet J, et al: Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nat. 2007, 450: 56-62. 10.1038/nature06293.

    Article  CAS  Google Scholar 

  4. 4.

    Feinberg EH, et al: GFP Reconstitution Across Synaptic Partners (GRASP) defines cell contacts and synapses in living nervous systems. Neuron. 2008, 57: 353-363. 10.1016/j.neuron.2007.11.030.

    Article  CAS  PubMed  Google Scholar 

  5. 5.

    Hama H, et al: Scale: a chemical approach for fluorescence imaging and reconstruction of transparent mouse brain. Nat Neurosci. 2011, 14: 1481-1488. 10.1038/nn.2928.

    Article  CAS  PubMed  Google Scholar 

  6. 6.

    Micheva KD, Smith SJ: Array tomography: A new tool for imaging the molecular architecture and ultrastructure of neural circuits. Neuron. 2007, 55: 25-36. 10.1016/j.neuron.2007.06.014.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  7. 7.

    Sigrist SJ, Sabatini BL: Optical super-resolution microscopy in neurobiology. Curr Opin Neurobiol. 2011, 22: 86-93.

    Article  PubMed  Google Scholar 

  8. 8.

    Peng H: Bioimage informatics: a new area of engineering biology. Bioinformatics. 2008, 24: 1827-1836. 10.1093/bioinformatics/btn346.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  9. 9.

    Helmstaedter M, Mitra PP: Computational methods and challenges for large-scale circuit mapping. Curr Opin Neurobiol. 2012, 22: 162-169. 10.1016/j.conb.2011.11.010.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  10. 10.

    Koch C, Reid C: Observatories of the mind. Nat. 2012, 483: 397-398. 10.1038/483397a.

    Article  CAS  Google Scholar 

  11. 11.

    Ascoli GA, Donohue DE, Halavi M: NeuroMorpho.Org - A central resource for neuronal morphologies. J Neurosci. 2007, 27: 9247-9251. 10.1523/JNEUROSCI.2055-07.2007.

    Article  CAS  PubMed  Google Scholar 

  12. 12.

    Chen BL, Hall DH, Chklovskii DB: Wiring optimization can relate neuronal structure and function. PNAS. 2006, 103: 4723-4728. 10.1073/pnas.0506806103.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  13. 13.

    Chiang AS, et al: Three-dimensional reconstruction of brain-wide wiring networks in Drosophila at single-cell resolution. Curr Biol. 2011, 21: 1-11.

    Article  CAS  PubMed  Google Scholar 

  14. 14.

    Peng H, et al: BrainAligner: 3D registration atlases of Drosophila brains. Nat Methods. 2011, 8: 493-498. 10.1038/nmeth.1602.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  15. 15.

    Lein ES, et al: Genome-wide atlas of gene expression in the adult mouse brain. Nat. 2007, 445: 168-176. 10.1038/nature05453.

    Article  CAS  Google Scholar 

  16. 16.

    Ropireddy D, Bachus SE, Ascoli GA: Non-homogeneous stereological properties of the rat hippocampus from high-resolution 3D serial reconstruction of thin histological sections. Neurosci. 2012, 205: 91-111.

    Article  CAS  Google Scholar 

  17. 17.

    Donohue DE, Ascoli GA: Automated reconstruction of neuronal morphology: an overview. Brain Res Rev. 2011, 67: 94-102. 10.1016/j.brainresrev.2010.11.003.

    PubMed Central  Article  PubMed  Google Scholar 

  18. 18.

    Türetken E, González G, Blum C, Fua P: Automated reconstruction of dendritic and axonal trees by global optimization with geometric priors. Neuroinformatics. 2011, 9: 279-302. 10.1007/s12021-011-9122-1.

    Article  PubMed  Google Scholar 

  19. 19.

    Liu Y: The DIADEM and beyond. Neuroinformatics. 2011, 9: 99-102. 10.1007/s12021-011-9102-5.

    Article  PubMed  Google Scholar 

  20. 20.

    He HY, Cline HT: Diadem X: Automated 4 dimensional analysis of morphological data. Neuroinformatics. 2011, 9: 107-112. 10.1007/s12021-011-9098-x.

    PubMed Central  Article  PubMed  Google Scholar 

  21. 21.

    Jain V, Seung H, Turaga SC: Machines that learn to segment images: a crucial technology for connectomics. Curr Opin Neurobiol. 2010, 20: 653-666. 10.1016/j.conb.2010.07.004.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  22. 22.

    Gillette TA, Brown KM, Ascoli GA: The DIADEM metric: comparing multiple reconstructions of the same neuron. Neuroinformatics. 2011, 9: 233-245. 10.1007/s12021-011-9117-y.

    PubMed Central  Article  PubMed  Google Scholar 

  23. 23.

    Peng H, Ruan Z, Long F, Simpson JH, Myers EW: V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat Biotechnol. 2010, 28: 348-353. 10.1038/nbt.1612.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  24. 24.

    Long F, Peng H, Liu X, Kim SK, Myers E: A 3D digital atlas of C. elegans and its application to single-cell analyses. Nat Methods. 2009, 6: 667-672. 10.1038/nmeth.1366.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  25. 25.

    Luisi J, Narayanaswamy A, Galbreath Z, Roysam B: The FARSIGHT trace editor: an open source tool for 3-D inspection and efficient pattern analysis aided editing of automated neuronal reconstructions. Neuroinformatics. 2011, 9: 305-315. 10.1007/s12021-011-9115-0.

    Article  PubMed  Google Scholar 

  26. 26.

    Qu L, et al: Simultaneous recognition and segmentation of cells: application in C. elegans. Bioinformatics. 2011, 27: 2895-2902. 10.1093/bioinformatics/btr480.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  27. 27.

    Pittet M, Weissleder R: Intravital Imaging. Cell. 2011, 147: 983-991. 10.1016/j.cell.2011.11.004.

    Article  CAS  PubMed  Google Scholar 

  28. 28.

    Gerhard F, Pipa G, Lima B, Neuenschwander S, Gerstner W: Extraction of network topology from multi-electrode recordings: is there a small-world effect?. Front. Comput. Neurosci. 2011, 5: doi:10.3389/fncom.2011.00004

    Google Scholar 

  29. 29.

    Srinivasan S, Stevens CF: Robustness and fault tolerance make brains harder to study. BMC Biol. 2011, 9 (46): doi:10.1186/1741-7007-9-46

    Google Scholar 

  30. 30.

    Kodandaramaiah SB, Franzesi GT, Chow BY, Boyden ES, Forest CR: Automated whole-cell patch-clamp electrophysiology of neurons in vivo. Nat Methods. 2012, doi:10.1038/nmeth.1993

    Google Scholar 

  31. 31.

    Gonzalez-Bellido PT, Peng H, Yang J, Georgopoulos AP, Olberg RM: Eight pairs of descending visual neurons in the dragonfly give wing motor centers accurate population vector of prey direction. Proc Natl Acad Sci. 2013, 110 (2): 696-701. 10.1073/pnas.1210489109.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  32. 32.

    Clack NG, O’Connor DH, Huber D, Petreanu L, Hires A, Peron S, Myers EW: Automated tracking of whiskers in videos of head fixed rodents. PLoS Comput Biol. 2012, 8 (7): e1002591-10.1371/journal.pcbi.1002591.

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  33. 33.

    Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K: Jaaba: interactive machine learning for automatic annotation of animal behavior. Nat Methods. 2012, 10 (1): 64-67. 10.1038/nmeth.2281. doi:10.1038/nmeth.2281

    Article  PubMed  Google Scholar 

  34. 34.

    Frimat JP, Sisnaiske J, Subbiah S, Menne H, Godoy P, Lampen P, Leist M, Franzke J, Hengstler JG, van Thriel C, West J: The network formation assay: a spatially standardized neurite outgrowth analytical display for neurotoxicity screening. Lab Chip. 2010, 10 (6): 701-709. 10.1039/b922193j. Mar 21

    Article  CAS  PubMed  Google Scholar 

  35. 35.

    Hutson TH, Buchser WJ, Bixby JL, Lemmon VP, Moon LD: Optimization of a 96-Well Electroporation Assay for Postnatal Rat CNS Neurons Suitable for Cost-Effective Medium-Throughput Screening of Genes that Promote Neurite Outgrowth. Front Mol Neurosci. 2011, 4: 55-

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  36. 36.

    Klatt S, Fassold A, Straub RH: Sympathetic nerve fiber repulsion: testing norepinephrine, dopamine, and 17β-estradiol in a primary murine sympathetic neurite outgrowth assay. Ann N Y Acad Sci. 2012, 1261: 26-33. 10.1111/j.1749-6632.2012.06628.x. Jul

    Article  CAS  PubMed  Google Scholar 

  37. 37.

    Helmstaedter M, Briggman KL, Denk W: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat Neurosci. 2011, 14 (8): 1081-1088. 10.1038/nn.2868. Jul 10

    Article  CAS  PubMed  Google Scholar 

  38. 38.

    Briggman KL, Helmstaedter M, Denk W: Wiring specificity in the direction-selectivity circuit of the retina. Nature. 2011, 471 (7337): 183-8. 10.1038/nature09818. Mar 10

    Article  CAS  PubMed  Google Scholar 

  39. 39.

    Denk W, Briggman KL, Helmstaedter M: Structural neurobiology: missing link to a mechanistic understanding of neural computation. Nat Rev Neurosci. 2012, 13 (5): 351-358. Nov 9

    CAS  PubMed  Google Scholar 

  40. 40.

    Kleinfeld D, Bharioke A, Blinder P, Bock DD, Briggman KL, Chklovskii DB, Denk W, Helmstaedter M, Kaufhold JP, Lee WC, Meyer HS, Micheva KD, Oberlaender M, Prohaska S, Reid RC, Smith SJ, Takemura S, Tsai PS, Sakmann B: Large-scale automated histology in the pursuit of connectomes. J Neurosci. 2011, 31 (45): 16125-16138. 10.1523/JNEUROSCI.4077-11.2011. Nov 9

    PubMed Central  Article  CAS  PubMed  Google Scholar 

  41. 41.

    Chung K, Wallace J, Kim SY, Kalyanasundaram S, Andalman AS, Davidson TJ, Mirzabekov JJ, Zalocusky KA, Mattis J, Denisin AK, Pak S, Bernstein H, Ramakrishnan C, Grosenick L, Gradinaru V, Deisseroth K: Structural and molecular interrogation of intact biological systems. Nat. 2013, 497 (7449): 332-337. 10.1038/nature12107. May 16

    Article  CAS  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Hanchuan Peng.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Peng, H., Roysam, B. & Ascoli, G.A. Automated image computing reshapes computational neuroscience. BMC Bioinformatics 14, 293 (2013).

Download citation


  • Image Computing
  • Neuron Morphology
  • Computational Neuroscience
  • Neuron Reconstruction
  • Allen Mouse Brain