TreeToReads - a pipeline for simulating raw reads from phylogenies
© The Author(s) 2017
Received: 11 June 2016
Accepted: 10 March 2017
Published: 20 March 2017
Using phylogenomic analysis tools for tracking pathogens has become standard practice in academia, public health agencies, and large industries. Using the same raw read genomic data as input, there are several different approaches being used to infer phylogenetic tree. These include many different SNP pipelines, wgMLST approaches, k-mer algorithms, whole genome alignment and others; each of these has advantages and disadvantages, some have been extensively validated, some are faster, some have higher resolution. A few of these analysis approaches are well-integrated into the regulatory process of US Federal agencies (e.g. the FDA’s SNP pipeline for tracking foodborne pathogens). However, despite extensive validation on benchmark datasets and comparison with other pipelines, we lack methods for fully exploring the effects of multiple parameter values in each pipeline that can potentially have an effect on whether the correct phylogenetic tree is recovered.
To resolve this problem, we offer a program, TreeToReads, which can generate raw read data from mutated genomes simulated under a known phylogeny. This simulation pipeline allows direct comparisons of simulated and observed data in a controlled environment. At each step of these simulations, researchers can vary parameters of interest (e.g., input tree topology, amount of sequence divergence, rate of indels, read coverage, distance of reference genome, etc) to assess the effects of various parameter values on correctly calling SNPs and reconstructing an accurate tree.
Such critical assessments of the accuracy and robustness of analytical pipelines are essential to progress in both research and applied settings.
KeywordsGenomics Phylogenetics Simulation
Whole genome sequencing (WGS) now allows researchers to trace ancestry among samples that differ by only a few mutations. Phylogenetic trees inferred from WGS data are a valuable tool for tracing the ancestry of closely related lineages in bacterial pathogen outbreaks. However, these estimates of shared ancestry may rely on only a handful of data points. When the resulting phylogenetic trees are used by public health agencies to make public health decisions, such as to define the scope of foodborne outbreaks , to identify the source of these outbreaks [2–4] and where appropriate to follow-up with regulatory or legal actions, it is particularly important to ensure that the WGS analysis methods used are validated.
One potential source of error is biases in which variable sites (single nucleotide polymorphisms, or SNPs) are detected from analysis of the sequencing read data. SNP ascertainment biases can be caused by various factors. These biases can be affected by analysis parameters, such as using missing data cutoffs  or read filtering artifacts . Read mapping issues due to choice of reference genome  and different mapping algorithms  can also result in biases in which variable sites are detected. Phylogenetic error can be exacerbated by interaction among dataset biases and analytic choices; for example, using a model of evolution developed for sequence data on a panel of exclusively variable sites , or choosing an inappropriate model of evolution .
Despite the sheer quantities of genomic data, it is possible that these types of biases could affect phylogenetic conclusions. If these errors are systematic, analyses can converge to support an incorrect result with high bootstrap confidence. In order to adopt data analysis pipelines for the regulatory environment it is necessary to understand potential biases in sequence analysis pipelines and validate their use. Simulations are a useful approach to investigating potential biases. Without validated in silico modeling, scientists have to rely on benchmark datasets where the truth can never be truly known.
To help solve this problem, we present TreeToReads, a software tool for simulating realistic patterns of sequence variation across phylogenies. This pipeline may be used to assess the robustness of evolutionary inferences from whole genome data against potential biases inherent in data collection and analysis pipelines. Two key aspects of TreeToReads differentiate it from existing simulation alternatives. First, the variable sites follow a user-specified phylogeny, resulting in more realistic evolutionary patterns. Second, those variable sites are placed in the context of an observed or ‘anchor genome’, which is represented as the tip on that phylogeny. Together, this combination allows researchers to make direct comparisons of WGS read mapping between their observed and simulated reads, and thus makes TreeToReads appropriate for testing the effects of distance to a reference genome on phylogenetic inference from genomic data. That anchor genome can be used as a reference genome for read mapping, or mapping can be tested against other empirically-observed genomes. In the case of empirically-observed genomes, what separates the reference data from the simulated data constitutes real evolutionary history. Making direct read mapping comparisons to assess reference genome effects cannot be done using alternative simulation software.
Results and discussion
To validate the performance of TreeToReads, we performed 5 replicate analyses under each of two case study conditions then compared the inferences to the simulation parameters. In the first case study we also assessed the effects of distance to reference genome on inference. All validation configuration files, input files, analysis scripts and output files are available with the software on GitHub https://github.com/snacktavish/TreeToReads.
The first case study used whole genome read data from ten Salmonella enterica sequences associated with a 2010 outbreak, and the observed phylogeny inferred from those data . We used a completed Salmonella enterica genome (CFSAN000189, GenBank: CP006053.1) as the anchor sequence, and simulated 200 variable sites under the generalized time reversible (GTR) model at an average coverage of 20 reads per site. Based on the locations of variable sites in the observed sequence data, the distances between locations for 20% of the variable sites were drawn from an exponential distribution with a 125bp mean. The locations for the rest of the variable sites were drawn from a uniform distribution across the genome. The read error profile was based on the observed outbreak sequence data.
For the second case study we simulated data using the reference genome and phylogeny from a Listeria outbreak. The tree and associated metadata are available in the Open Tree of Life data store (tree.opentreeoflife.org/curator/study/view/ot_301) ). For the validation tests we pruned out multiple replicates of clonal lineages, to create a resolved tree. We used a completed genome from the outbreak, Listeria monocytogenes (CFSAN023463, GenBank NZ_CP012021.1) as the anchor sequence. 500 variable sites were distributed uniformly across the genome. Insertions and deletions were simulated under the Lavalette distribution, with a = 1.7 and a maximum size of 541 base pairs . The insertion and deletion rates were equal and 0.1 times the nucleotide substitution rate. To demonstrate that sequence data can be simulated without empirical data in hand, for this case study the sequencing error rate was drawn from the defaults provided with ART. Reads were simulated at an average coverage of 40 reads per site .
For each replicate we analyzed the resulting short read datasets using the SNP pipeline from the Food and Drug Administration, Center for Food Safety And Nutrition (FDA CFSAN) to identify SNPs in each simulated dataset . The CFSAN SNP Pipeline uses reference-based alignments to create a matrix of SNPs for a given set of samples. We used the anchor genomes as the reference genomes for both case studies. For the first case study, Salmonella, we also ran these analyses using a distant reference genome, Salmonella enterica Typhimurium str. LT2 (GenBank AE006468.2). This reference differs from the anchor genome at around 0.7% percent of sites; approximately 36,490 nucleotide sites, across the 4,730,612 base pair Bareilly outbreak anchor genome (CFSAN000189), although the exact count depends on alignment choices. Using the called SNPs we inferred the maximum likelihood phylogeny for each data set using RAxML . For the analyses using closely related reference genomes we applied the ASC GTRGAMMA model with a Lewis ascertainment bias correction, which corrects for including only variable sites in the analysis . For the analyses using the distant reference genome many called SNPs were actually fixed differences from the reference and not variable within the alignment. We used the GTRGAMMA model with no ascertainment bias correction for phylogenetic analyses on these data.
We then compared the inferences from the data simulated using TreeToReads to the input parameters for the topology, number of variable sites, base frequencies, and GTR model of evolution. We compared the inferred topology to the input topology using Robinson–Foulds (RF) distances  - the symmetric difference in partitions between the input and inferred trees using Dendropy . For the Listeria example we also compared the indel locations in the simulated alignment to that generated by INDELible . We do not independently test the correctness of the results of INDELible, but the indel locations were in all cases were identical to those output by INDELible.
Simulation parameter values and mean of inferred parameter values across 5 simulation and inference replicates
Case Study 1 – Salmonella enterica Bareilly
Number of SNPs
154 (3) variable sites
Outbreak clade monophyletic
GTR rate Matrix
Case Study 2 – Listeria monocytogenes
Number of SNPs
GTR rate Matrix
Difference from INDELible gap distribution
In the Salmonella case study, where there were only 100 variable sites, inferences of base frequencies and GTR rate parameters did not match the simulated conditions (Table 1). Evolutionary rate parameters are difficult to estimate when the observed number of mutations in a single category are rare, and some of the relative rate parameters made certain rate categories (e.g. ac, cg) very rare, resulting in high variance across replicates in inferred rates. However, in the Listeria case study, where 400 variable sites were simulated, providing sufficient data to accurately infer rates, the simulated input parameters were within one standard deviation of the mean inferences of parameter values (Table 1).
These examples demonstrate that TreeToReads is effectively simulating data under complex evolutionary scenarios. In straightforward analysis cases, the inferred parameters match almost exactly the input parameters, although this is subject to data set size, and ability to resolve relationships of interest. Although many software tools can simulate sequence data, no existing tools combine phylogenetic relationships with observed genomic sequences. TreeToReads is designed to test the effects of multiple parameters on phylogenetic inference and provides a pipeline to simulate next-gen sequencing reads from a phylogenetic tree using an observed error model. Using an anchor genome as a tip in the simulated tree means that simulated and empirical data can be mapped to the same reference genome, providing direct comparisons of inferences. In addition, the reference genome does not need to be the anchor genome on which the simulations are based. If a different reference is used, as in the Salmonella analyses with a distance reference, the biological evolution separating the anchor genome from the reference genome includes real evolutionary processes affecting read mapping to genomes in a testable framework. This provides a greater realism than can be provided by even the most complex available models for simulating sequence evolution. Alternatively, the user can use the simulated data to test reference-free methods for phylogenetic inference, such as multi locus sequence typing (MLST) , or the k-mer based method kSNP .
Seq-Gen , used to generate the variable sites in the TreeToReads pipeline, uses a full GTR model of evolution with parameters specified by the user. However, on its own, Seq-Gen generates random sequences based on the model of evolution and therefore does not incorporate observed genomic context. Consequently, reads from Seq-Gen-simulated genomes cannot be mapped to observed reference genomes. This is also true for other simulators of more complex evolutionary processes, such as SWGE . Other sequence simulation software, such as ALF  and Indel Seq-gen , simulate evolution forward in time, starting from an input genome representing the root of the phylogeny. However, in empirical data the reference genome is not an ancestor - it is always a present day relative. Anchoring an observed genome to a tip in a tree using TreeToReads allows us to test choices about selection of reference genomes in a way that is directly comparable to empirical data. SWGE and ALF include complex evolutionary processes not simulated in TreeToReads, such as recombination across lineages and heterogeneity of the evolutionary model across the genome. While recombination and horizontal gene transfer (HGT) can be important processes, the expectations are challenging to parameterize, as even a few HGT events can lead to phylogenetic networks generating extremely complex ancestral recombination graphs . In order to simulate shifts in relationships along chromosomes it is possible to independently input into TreeToReads different tree topologies for different genomic regions, and then combine the reads for these simulations and attempt inference. TreeToReads is designed for testing the robustness of pipelines for estimating bifurcating relationships from next generation sequencing read data. While inference of branch lengths can be biased by recombination , as well as by selection and use of exclusively variable sites to infer phylogenies , inference of topology is often robust to recombination .
Existing genomic simulation software packages cannot provide a phylogenetic perspective in simulation testing of assembly and alignment tools. TreeToReads allows researchers to test the joint effects of multiple parameter values on the ability of any analysis pipeline to recover the signal and infer the correct tree. Simulating data that spans these parameters can validate methods for reconstructing phylogenies directly from short-read data, which is especially useful for public health agencies tracking emerging pathogens.
Availability and requirements
Project name: TreeToReads
Project home page: https://github.com/snacktavish/TreeToReads
Operating system(s): Linux, OSX
Programming language: Python
Licence: This project constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105.
Single nucleotide polymorphism
Whole genome sequence
We thank Lili Velez for her careful editorial improvements. We thank Torsten Seeman and Lee Katz for helpful comments and advice, as well as the comments of three anonymous reviewers.
We would like to thank the The Oak Ridge Institute for Science and Education (ORISE) for funding EJM’s work on this project. ORISE did not play any role in the design or conclusions of this study.
Availability of data and materials
All data and materials are available in a GitHub repository https://github.com/snacktavish/TreeToReads.
RET conceived of the study, and participated in its design and coordination. EJM wrote the software and participated in the overall design of the study. MA contributed text to the manuscript. JP, SD, HR, and ES contributed to the design and testing of the software. EJM and RET wrote the manuscript. All authors have read and approved the final version of this manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Jackson BR, Tarr C, Strain E, Jackson KA, Conrad A, Carleton H, et al.Implementation of Nationwide Real-time Whole-genome Sequencing to Enhance Listeriosis Outbreak Detection and Investigation. Clin Infect Dis. 2016:ciw242. Available from: http://cid.oxfordjournals.org/content/early/2016/05/17/cid.ciw242. Accessed 2016.
- Hoffmann M, Luo Y, Monday SR, Gonzales-Escalona N, Ottesen AR, Muruvanda T, et al.Tracing Origins of the Salmonella Bareilly strain causing a Foodborne Outbreak in the United States. J Infect Dis. 2015:jiv297. Available from: http://jid.oxfordjournals.org/content/early/2015/05/19/infdis.jiv297. Accessed 2015.
- Dallman T, Bernard H, Cleary P, Simon S, Ashton P, Jourdan da Silva N, et al.Phylogenetic structure of European Salmonella Enteritidis outbreak correlates with national and international egg distribution network. Microb Genom. 2016. Available from: http://mgen.microbiologyresearch.org/content/journal/mgen/10.1099/mgen.0.000070. Accessed 2016.
- Allard MW, Strain E, Melka D, Bunning K, Musser SM, Brown EW, et al.Practical Value of Food Pathogen Traceability through Building a Whole-Genome Sequencing Network and Database. J Clin Microbiol. 2016; 54(8):1975–83.View ArticlePubMedPubMed CentralGoogle Scholar
- Huang H, Knowles LL. Unforeseen consequences of excluding missing data from next-generation sequences: Simulation study of RAD sequences. Syst Biol. 2014:syu046. Available from: http://sysbio.oxfordjournals.org/content/early/2014/07/04/sysbio.syu046.abstract. Accessed 2016.
- Li H. Toward better understanding of artifacts in variant calling from high-coverage samples. Bioinformatics. 2014; 30(20):2843–2851. Available from: http://bioinformatics.oxfordjournals.org/content/30/20/2843.
- Bertels F, Silander OK, Pachkov M, Rainey PB, Nimwegen Ev. Automated reconstruction of whole-genome phylogenies from short-sequence reads. Mol Biol Evol. 2014; 31(5):1077–1088. Available from: http://mbe.oxfordjournals.org/content/31/5/1077.
- Pightling AW, Petronella N, Pagotto F. Choice of Reference Sequence and Assembler for Alignment of Listeria monocytogenes Short-Read Sequence Data Greatly Influences Rates of Error in SNP Analyses. PLoS ONE. 2014; 9(8):e104579. Available from: http://dx.doi.org/10.1371/journal.pone.0104579.
- Lewis PO. A likelihood approach to estimating phylogeny from discrete morphological character data. Syst Biol. 2001; 50(6):913–925. Available from: http://sysbio.oxfordjournals.org/content/50/6/913.short.
- Sullivan J, Swofford DL. Are Guinea Pigs Rodents? The Importance of Adequate Models in Molecular Phylogenetics. J Mamm Evol. 1997; 4(2):77–86. Available from: http://link.springer.com/article/10.1023/A%253A1027314112438.
- Rambaut A, Grass NC. Seq-Gen: an application for the Monte Carlo simulation of DNA sequence evolution along phylogenetic trees. Comput Appl Biosci: CABIOS. 1997; 13(3):235–238. Available from: https://academic.oup.com/bioinformatics/article/13/3/235/423110/Seq-Gen-an-application-for-the-Monte-Carlo.
- Fletcher W, Yang Z. INDELible: A Flexible Simulator of Biological Sequence Evolution. Mol Biol Evol. 2009; 26(8):1879–1888. Available from: http://mbe.oxfordjournals.org/content/26/8/1879.
- Huang W, Li L, Myers JR, Marth GT. ART: a next-generation sequencing read simulator. Bioinformatics. 2012; 28(4):593–594. Available from: http://bioinformatics.oxfordjournals.org/content/early/2011/12/23/bioinformatics.btr708.
- Lepais O, Weir JT. SimRAD: an R package for simulation-based prediction of the number of loci expected in RADseq and similar genotyping by sequencing approaches. Mol Ecol Resour. 2014; 14(6):1314–1321. Available from: http://onlinelibrary.wiley.com/doi/10.1111/1755-0998.12273/abstract.
- McTavish EJ, Hinchliff CE, Allman JF, Brown JW, Cranston KA, Holder MT, et al.Phylesystem: a git-based data store for community curated phylogenetic estimates. Bioinformatics. 2015:btv276. Available from: http://bioinformatics.oxfordjournals.org/content/early/2015/04/30/bioinformatics.btv276. Accessed 2015.
- Davis S, Pettengill JB, Luo Y, Payne J, Shpuntoff A, Rand H, et al.CFSAN SNP Pipeline: an automated method for constructing SNP matrices from next-generation sequence data. PeerJ Comput Sci. 2015; 1:e20. Available from: https://peerj.com/articles/cs-20. Accessed 2015.
- Stamatakis A. RAxML Version 8: A tool for Phylogenetic Analysis and Post-Analysis of Large Phylogenies. Bioinformatics. 2014; 30:1312–1313. Available from: http://bioinformatics.oxfordjournals.org/content/early/2014/01/21/bioinformatics.btu033.
- Robinson DF, Foulds LR. Comparison of phylogenetic trees. Math Biosci. 1981; 53(1):131–147. Available from: http://www.sciencedirect.com/science/article/pii/0025556481900432.
- Sukumaran J, Holder MT. DendroPy: a Python library for phylogenetic computing. Bioinformatics. 2010; 26(12):1569–1571. Available from: http://bioinformatics.oxfordjournals.org/content/26/12/1569.
- Pightling AW, Petronella N, Pagotto F. The Listeria monocytogenes Core-Genome Sequence Typer (LmCGST): a bioinformatic pipeline for molecular characterization with next-generation sequence data. BMC Microbiol. 2015; 15:224.View ArticlePubMedPubMed CentralGoogle Scholar
- Gardner SN, Hall BG. When Whole-Genome Alignments Just Won’t Work: kSNP v2 Software for Alignment-Free SNP Discovery and Phylogenetics of Hundreds of Microbial Genomes. PLoS ONE. 2013; 8(12):e81760. Available from: http://dx.doi.org/10.1371/journal.pone.0081760.
- Arenas M, Posada D. Simulation of Genome-Wide Evolution under Heterogeneous Substitution Models and Complex Multispecies Coalescent Histories. Mol Biol Evol. 2014; 31(5):1295–1301. Available from: http://mbe.oxfordjournals.org/content/31/5/1295.
- Dalquen DA, Anisimova M, Gonnet GH, Dessimoz C. ALF—A Simulation Framework for Genome Evolution. Mol Biol Evol. 2012; 29(4):1115–1123. Available from: http://mbe.oxfordjournals.org/content/29/4/1115.
- Strope CL, Scott SD, Moriyama EN. indel-Seq-Gen: a new protein family simulator incorporating domains, motifs, and indels. Mol Biol Evol. 2007; 24(3):640–649.View ArticlePubMedGoogle Scholar
- Yu Y, Degnan JH, Nakhleh L. The Probability of a Gene Tree Topology within a Phylogenetic Network with Applications to Hybridization Detection. PLOS Genet. 2012; 8(4):e002660. Available from: http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002660.
- Hedge J, Wilson DJ. Bacterial Phylogenetic Reconstruction from Whole Genomes Is Robust to Recombination but Demographic Inference Is Not. mBio. 2014; 5(6):e02158–14. Available from: http://mbio.asm.org/content/5/6/e02158-14.