- Open Access
SEQprocess: a modularized and customizable pipeline framework for NGS processing in R package
BMC Bioinformaticsvolume 20, Article number: 90 (2019)
Next-Generation Sequencing (NGS) is now widely used in biomedical research for various applications. Processing of NGS data requires multiple programs and customization of the processing pipelines according to the data platforms. However, rapid progress of the NGS applications and processing methods urgently require prompt update of the pipelines. Recent clinical applications of NGS technology such as cell-free DNA, cancer panel, or exosomal RNA sequencing data also require appropriate customization of the processing pipelines. Here, we developed SEQprocess, a highly extendable framework that can provide standard as well as customized pipelines for NGS data processing.
SEQprocess was implemented in an R package with fully modularized steps for data processing that can be easily customized. Currently, six pre-customized pipelines are provided that can be easily executed by non-experts such as biomedical scientists, including the National Cancer Institute’s (NCI) Genomic Data Commons (GDC) pipelines as well as the popularly used pipelines for variant calling (e.g., GATK) and estimation of allele frequency, RNA abundance (e.g., TopHat2/Cufflink), or DNA copy numbers (e.g., Sequenza). In addition, optimized pipelines for the clinical sequencing from cell-free DNA or miR-Seq are also provided. The processed data were transformed into R package-compatible data type ‘ExpressionSet’ or ‘SummarizedExperiment’, which could facilitate subsequent data analysis within R environment. Finally, an automated report summarizing the processing steps are also provided to ensure reproducibility of the NGS data analysis.
SEQprocess provides a highly extendable and R compatible framework that can manage customized and reproducible pipelines for handling multiple legacy NGS processing tools.
Next-Generation Sequencing (NGS) technology is now widely used in biomedical research fields, and is extensively being used in the clinic . Applications with NGS technology include identification of DNA or RNA sequence variants, and the quantitation of RNA abundances or DNA copy numbers. However, processing and analysis of NGS data remain difficult as data are generally processed through by multiple processing steps, and each step requires different legacy programs. To handle these complex processing steps, several pipeline programs have been released. For example, ‘NGS-pipe’  and ‘NEAT’  provide automated pipelines for NGS data analysis. Another tool ‘systemPipeR’ provides an NGS analysis workflow in R program that can be customized according to the various NGS applications such as whole-exome sequencing (WES), whole-genome sequencing (WGS) and transcriptome sequencing (RNA-seq) data . However, these tools do not handle the recently updated NCI Genomic Data Commons (GDC) pipelines, which have been used as standard pipelines to process The Cancer Genome Atlas (TCGA, https://cancergenome.nih.gov) data. Moreover, recent progress in clinical applications of the NGS data has generated new platform data, such as cell free DNAs, exosomes, and cancer panels. These applications require customized analysis for data quality control and processing.
With this concern, we developed a SEQprocess that provides fully customizable NGS processing pipelines covering the GDC pipelines and new data for clinical applications. SEQprocess is implemented in an R program, providing six pre-customized pipelines that are widely used as standards in NGS data processing and can be executed easily by non-experts such as biomedical scientists.
SEQprocess is a framework implemented in R package, providing pipelines for NGS data processing operated by multiple programs. It can be run from start-to-end with a single command in the R console, or through stepwise customization with an interactive mode. The pipelines are designed to support processing pipelines for DNA and RNA sequencing data, including the data processing steps for quality control of raw sequencing data, trimming, alignment, variant calling, annotation, DNA copy number estimation and RNA quantitation. Each pipeline is modularized to run sequentially or separately. The following programs are supported by the pipelines. Quality control of raw data is assessed by FastQC (https://www.bioinformatics.babraham.ac.uk). Sequence trimming is performed by TrimGalore (https://github.com/broadinstitute/picard) or Cutadapt . Sequence alignment is supported by BWA , STAR, TopHat2 , bowtie2 , or samtools . Removal of duplicates is performed by Picard (https://github.com/broadinstitute/picard) and re-alignment by GATK . Variants calling is supported by GATK, VarScan2 , MuSE , or SomaticSniper . Variant annotation is supported by VEP  or ANNOVAR . For RNA-seq data, SEQprocess performs RNA quantitation by HTSeq  or Cufflinks, and DNA copy number estimation is conducted by Sequenza . These programs are implemented as modularized functions with optimized default parameters. These external programs can be installed easily using Conda package manager (https://conda.io/en/latest). Subsequent steps for NGS data processing can be easily included or excluded in the pipeline. This modular framework provides a highly flexible and extendable platform; thus, new pipelines for upcoming data types such as single cell RNA-Seq data can be implemented.
The current version of SEQprocess provided six different pre-customized standard pipelines, including the pipelines for GDC processing and the newly adapted clinical applications for cell-free DNAs (cfDNA) or exosomal miRNAs (Fig. 1). These pipelines ran by a one-step command that could be executed easily by non-expert users. For WGS/WES, a GDC compatible pipeline of TrimGalore-BWA-Picard- VarScan2-VEP was implemented. We also implemented a popularly used standard Custom pipeline of TrimGalore-BWA-Picard-GATK–ANNOVAR. In addition, SEQprocess could estimate allele frequencies for each variant by calculating the sequence read depths of the mutated and wild-type sequences with a GATK function ‘DepthOfCoverage’. For liquid-biopsied cfDNA or targeted sequencing data, such as a cancer panel, an optimized pipeline excluding the duplicate removal step was provided, because cfDNA sequence reads usually have the same sequences. For barcoded data (BarSeq), the duplicate removal step was performed using the barcodes. For RNA-Seq data, a GDC pipeline (STAR-Samtools-HTSeq) was implemented. A popularly used standard pipeline Tuxedo (i.e., Tophat2-Cufflinks) was also implemented. For miR-Seq data from exosomes, cells, or tissues, the Cutadapt-BWA/bowtie2-HTSeq pipeline was implemented with optimized parameters.
SEQprocess operates multiple legacy programs and reference data, which might require installation in the system. Configuration of the installed programs and data could be managed simply by editing the ‘data/config.R’ file (Table 1). The current version of SEQprocess supported the Linux-operating system because some of the required programs only support the Linux-operating system. Parallel computation on multi-core machines was also supported by using the ‘parallel’ R package. In addition, multi-threading support in each program of GATK, TopHat2, BWA, STAR, and Cufflinks could be controlled by the program arguments.
Each step of these pipelines are modularized as a wrapper function in R package to provide an easy customization platform. Step-by-step pipelines could be conducted by a single command ‘SEQprocess’, and which could be readily customized by setting the function parameters (Table 2). The processed data were transformed into an R/Bioconductor compatible data type (i.e. ‘ExpressionSet’), which is popularly used for the subsequent NGS data analysis for biological interpretation -. Each data object for RNA expression, variant, and DNA copy number was provided with the filename extensions of ‘.eSet’, ‘.vSet’, or ‘cSet’, respectively. These ExpressionSet data types could be transformed into another data type ‘SummariazedExperiment’, i.e. a modified data type of ‘ExpressionSet’ containing ‘GenomicRanges’ data type (Fig. 2). These will serve as a framework facilitating the subsequent analyses in the R environment.
In addition, SEQprocess provided a report summarizing the processing steps and visualized tables and plots for the processed results (Fig. 3). The report file is automatically generated workflow records for data processing steps, arguments, and outcome results. Moreover, users can find error and processing messages from the log file in each program. These reporting systems will ensure the reproducibility of the data analysis. We have also provided an example data (‘inst/example’) and a script (‘example/example.R’).
In summary, SEQprocess provides a highly extendable and R-compatible framework that can be manage customized and reproducible pipelines for handling multiple legacy NGS processing tools.
Availability and requirements
Project name: SEQprocess.
Project home page: https://github.com/omicsCore/SEQprocess
Operating systems: Linux dependent.
Programming language: R language.
Other requirements: Java 1.8.0 or higher, Perl v5.10.1 or higher, Python 2.6.6 or higher.
Genomic Data Commons
Micro RNA sequencing
National Cancer Institute
Next Generation Sequencing
The Cancer Genome Atlas
Whole Exome Sequencing
Whole Genome Sequencing
Anders S, Pyl PT, Huber W. HTSeq--a Python framework to work with high-throughput sequencing data. Bioinformatics. 2015;31(2):166–9.
Backman TWH, Girke T. systemPipeR: NGS workflow and report generation environment. BMC Bioinformatics. 2016;17(1):388.
Dobin A, et al. STAR: ultrafast universal RNA-seq aligner. Bioinformatics. 2013;29(1):15–21.
Fan Y, et al. MuSE: accounting for tumor heterogeneity using a sample-specific error model improves sensitivity and specificity in mutation calling from sequencing data. Genome Biol. 2016;17(1):178.
Favero F, et al. Sequenza: allele-specific copy number and mutation profiles from tumor sequencing data. Annals of oncology : official journal of the European Society for. Med Oncol. 2015;26(1):64–70.
Huber W, et al. Orchestrating high-throughput genomic analysis with Bioconductor. Nat Methods. 2015;12(2):115–21.
Kim D, et al. TopHat2: accurate alignment of transcriptomes in the presence of insertions, deletions and gene fusions. Genome Biol. 2013;14(4):R36.
Koboldt DC, et al. VarScan 2: somatic mutation and copy number alteration discovery in cancer by exome sequencing. Genome Res. 2012;22(3):568–76.
Kwon SM, et al. Perspectives of integrative cancer genomics in next generation sequencing era. Genomics Inform. 2012;10(2):69–73.
Langmead B, Salzberg SL. Fast gapped-read alignment with bowtie 2. Nat Methods. 2012;9(4):357–9.
Larson DE, et al. SomaticSniper: identification of somatic point mutations in whole genome sequencing data. Bioinformatics. 2012;28(3):311–7.
Li H, Durbin R. Fast and accurate short read alignment with burrows-wheeler transform. Bioinformatics. 2009;25(14):1754–60.
Li H, et al. The sequence alignment/map format and SAMtools. Bioinformatics. 2009;25(16):2078–9.
Martin M. Cutadapt removes adapter sequences from high-throughput sequencing reads. 2011 2011;17(1):3.
McKenna A, et al. The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 2010;20(9):1297–303.
McLaren W, et al. The Ensembl variant effect predictor. Genome Biol. 2016;17(1):122.
Schorderet P. NEAT: a framework for building fully automated NGS pipelines and analyses. BMC Bioinformatics. 2016;17:53.
Singer J, et al. NGS-pipe: a flexible, easily extendable and highly configurable framework for NGS analysis. Bioinformatics. 2018;34(1):107–8.
Trapnell C, et al. Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation. Nat Biotechnol. 2010;28(5):511–5.
Wang K, Li M, Hakonarson H. ANNOVAR: functional annotation of genetic variants from high-throughput sequencing data. Nucleic Acids Res. 2010;38(16):e164.
This work was supported by grants from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare, Republic of Korea (H15C1551) and the National Research Foundation of Korea (NRF) funded by the Korea government (MSIP) (NRF-2017R1E1A1A01074733, NRF-2017M3C9A6047620, and NRF- 2017M3A9B6061509).
Funding institutes did not play any roles in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.