- Open Access
GFF3sort: a novel tool to sort GFF3 files for tabix indexing
BMC Bioinformaticsvolume 18, Article number: 482 (2017)
The traditional method of visualizing gene annotation data in JBrowse is converting GFF3 files to JSON format, which is time-consuming. The latest version of JBrowse supports rendering sorted GFF3 files indexed by tabix, a novel strategy that is more convenient than the original conversion process. However, current tools available for GFF3 file sorting have some limitations and their sorting results would lead to erroneous rendering in JBrowse.
We developed GFF3sort, a script to sort GFF3 files for tabix indexing. Specifically designed for JBrowse rendering, GFF3sort can properly deal with the order of features that have the same chromosome and start position, either by remembering their original orders or by conducting parent-child topology sorting. Based on our test datasets from seven species, GFF3sort produced accurate sorting results with acceptable efficiency compared with currently available tools.
GFF3sort is a novel tool to sort GFF3 files for tabix indexing. We anticipate that GFF3sort will be useful to help with genome annotation data processing and visualization.
In the recently released JBrowse version (v1.12.3), support for indexed GFF3 files has been added . In this strategy, the GFF3 file is compressed with bgzip and indexed with tabix , which generates only two data files: a compressed file (.gz) and an index file (.tbi or.csi). Compared with the traditional processing protocol, the whole compression and index process could be finished within a few seconds even for large datasets such as the human genome annotation data (Additional file 1). The tabix tool requires GFF3 files to be sorted by chromosomes and start positions, which could be performed in the GNU sort program or the GenomeTools  package (see ). When dealing with feature lines in the same chromosome and start position, both of these tools may break ties or return a sort order where child features are placed ahead of their parent feature (Fig. 1a). Although this is still valid for tabix indexing, it would causing erroneous rendering in JBrowse  (Fig. 1a). Currently there is no additional options or arguments for current tools to break such tied features by parent-child relationship. In the absence of a suitable bug fix to JBrowse, an alternative sorting tool is needed to resolve this problem.
Here, we present GFF3sort, a novel tool to sort GFF3 files for tabix indexing. Compared with GNU sort and GenomeTools, GFF3sort produces sorting results that can be correctly rendered by JBrowse while still has comparable time and memory requirements. We anticipate that GFF3sort will be a useful tool to help with processing and visualizing genome annotation data.
GFF3sort is a script written in Perl. It uses a hash table to store the input GFF3 annotation data (Fig. 1b). For each feature, the chromosome ID and the start position are stored in the primary and secondary key, respectively. Features with the same chromosome and start position are grouped in an array in the same order of their appearance in the original GFF3 data. After sorting the hash table by chromosome IDs and start positions, GFF3sort implemented two modes to sort features within the array: the default mode and the precise mode (Fig. 1b). In most situations, the original GFF3 annotations produced by genome annotation projects have already placed parent features before their children. Therefore, GFF3sort returns the feature lines in their original order, which is the default behavior. In some situations where orders in the input file has not yet placed parent features before child features, GFF3sort would re-place them according to the parent-child topology using the sorting algorithm of directed acyclic graph , which is the most precise behavior but costs a little more computational time.
In order to test the performance of GFF3sort, the GFF3 annotation files of seven species, Saccharomyces cerevisiae (R64–1-1), Aspergillus nidulans (ASM1142v1), Chlamydomonas reinhardtii (INSDC v3.1), Drosophila melanogaster (BDGP6), Arabidopsis thaliana (Araport11), Rattus norvegicus (Rnor_6.0), and Homo sapiens (GRCh38), were downloaded from the ENSEMBL database . All the tests were conducted on a SuperMicro® server equipped with 80 Intel® Xeon® CPUs (2.40GHz), 128 GB RAM, and running the CentOS 6.9 system. By default, CentOS 6.9 carries GNU sort v8.4, a relatively old version released in 2010. Therefore, we downloaded and installed a new version (v8.28) from the official repository of GNU Coreutils . Both the old and the new version of GNU sort are used in benchmarking.
Results and discussion
GFF3sort takes a GFF3 file as its input data and returns a sorted GFF3 file as output. Several optional parameters are provided such as turning on the precise mode, sorting chromosomes in different ways and properly dealing with inline FASTA sequences. Features sorted by GFF3sort are correctly rendered by JBrowse (Fig. 1c and Additional file 2).
In addition to providing a sort order that correctly renders in JBrowse, GFF3sort has also other advantages over traditional tools. Compared with the GNU sort program, GFF3sort can properly deal with GFF3-specific lines or directives that are preceded by the ‘##’ symbol, such as the topmost GFF version line, the sequence-region lines, and the embedded FASTA sequences. Compared with the GenomeTools, GFF3sort runs significantly faster (Additional file 1). In the default mode, GFF3sort saves ~70% running time in our seven test datasets. The precise mode takes longer to run but is still faster than GenomeTools, especially for large annotation data such as human. While keeping a high running speed, the memory consumption is still acceptable (Additional file 1). For the largest annotation dataset (the GRCh38 annotation version of human) with a ~400 MB GFF3 file, the memory usage of GFF3sort is ~758 MB, ~40% less than GenomeTools.
In conclusion, GFF3sort is a novel tool to sort GFF3 files for tabix indexing and therefore can be used to visualize annotation data in JBrowse appropriately. It has a fast running speed compared with similar, existing tools. We anticipate that GFF3sort will be a useful tool to simplify data processing and visualization.
Availability and requirements
Project name: GFF3sort.
Project home page: https://github.com/billzt/gff3sort
Operating system(s): Linux.
Programming language: Perl.
Other requirements: No.
License: No restrictions for academic users.
Any restrictions to use by non-academics: license needed.
General Feature Format, version 3
HyperText Markup Language, version 5
Skinner ME, Uzilov AV, Stein LD, Mungall CJ, Holmes IH. JBrowse: a next-generation genome browser. Genome Res. 2009;19(9):1630–8.
Buels R, Yao E, Diesh CM, Hayes RD, Munoz-Torres M, Helt G, Goodstein DM, Elsik CG, Lewis SE, Stein L, et al. JBrowse: a dynamic web platform for genome visualization and analysis. Genome Biol. 2016;17(1):66.
JBrowse Configuration Guide [http://gmod.org/wiki/JBrowse_Configuration_Guide] Accessed 26 May 2017.
JBrowse-1.12.3: Maintenance Release [http://jbrowse.org/jbrowse-1-12-3/] Accessed 26 May 2017.
Li H. Tabix: fast retrieval of sequence features from generic TAB-delimited files. Bioinformatics. 2011;27(5):718–9.
Gremme G, Steinbiss S, Kurtz S. GenomeTools: a comprehensive software library for efficient processing of structured genome annotations. IEEE/ACM Trans Comput Biol Bioinformatics. 2013;10(3):645–56.
JBrowse FAQ [http://gmod.org/wiki/JBrowse_FAQ] Accessed 26 May 2017.
Potential GFF3Tabix issues [https://github.com/GMOD/jbrowse/issues/780] Accessed 26 May 2017.
Aken BL, Achuthan P, Akanni W, Amode MR, Bernsdorff F, Bhai J, Billis K, Carvalho-Silva D, Cummins C, Clapham P, et al. Ensembl 2017. Nucleic Acids Res. 2017;45(D1):D635–42.
Coreutils - GNU core utilities [https://www.gnu.org/software/coreutils/coreutils.html] Accessed 15 Sept 2017.
We thank Dr. Miklos Csuros and other anonymous reviewers for their helpful comments.
This work is supported by grants from the National Natural Science Foundation of China (Grant No. 31771850) and the Ministry of Agriculture of China (Grant No. 2016ZX08005004).
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Benchmark data. This file displays: 1) the detailed running time of GFF3-to-JSON conversion and the bgzip-tabix process on our test datasets; 2) the detailed running time and 3) memory usage of GFF3sort, GNU sort (v8.4 and v8.28), and GenomeTools on our test datasets. (PDF 720 kb)