OTO: Ontology Term Organizer
© Huang et al.; licensee BioMed Central. 2015
Received: 28 September 2014
Accepted: 30 January 2015
Published: 15 February 2015
The need to create controlled vocabularies such as ontologies for knowledge organization and access has been widely recognized in various domains. Despite the indispensable need of thorough domain knowledge in ontology construction, most software tools for ontology construction are designed for knowledge engineers and not for domain experts to use. The differences in the opinions of different domain experts and in the terminology usages in source literature are rarely addressed by existing software.
OTO software was developed based on the Agile principles. Through iterations of software release and user feedback, new features are added and existing features modified to make the tool more intuitive and efficient to use for small and large data sets. The software is open source and built in Java.
Ontology Term Organizer (OTO; http://biosemantics.arizona.edu/OTO/) is a user-friendly, web-based, consensus-promoting, open source application for organizing domain terms by dragging and dropping terms to appropriate locations. The application is designed for users with specific domain knowledge such as biology but not in-depth ontology construction skills. Specifically OTO can be used to establish is_a, part_of, synonym, and order relationships among terms in any domain that reflects the terminology usage in source literature and based on multiple experts’ opinions. The organized terms may be fed into formal ontologies to boost their coverage. All datasets organized on OTO are publicly available.
OTO has been used to organize the terms extracted from thirty volumes of Flora of North America and Flora of China combined, in addition to some smaller datasets of different taxon groups. User feedback indicates that the tool is efficient and user friendly. Being open source software, the application can be modified to fit varied term organization needs for different domains.
The Ontology Term Organizer (OTO) tool was initially developed to facilitate consensus-based categorization of the terms used to describe the morphology or phenotype of organisms. The description of organisms new to science, and the continuous process of revising and improving them based on new evidence, is traditionally done by taxonomists and systematists. The publications that they produce contain descriptions that are typically highly detailed and contain terms which range from general to highly specific in their usages. When compared across resources, these terms can vary from identical to similar to non-overlapping within and between major groups of organisms. This lack of consistency presented a serious challenge to the authors when attempting to extract knowledge from legacy descriptive taxonomic literature as part of a National Science Foundation-funded project entitled: “Fine-Grained Semantic Markup of Descriptive Data for Knowledge Applications in Biodiversity Domains”.
Morphological descriptions are often composed in a telegrammatic style, for example:
Leaves alternate, spirally arranged, 2--3-ranked, simple; stipules deciduous, distinct; petioles present. Leaf blade sometimes lobed, pinnately veined, margins toothed, serrate to nearly entire; surfaces glabrous to tomentose, abaxially often with resinous glands. [Betulaceae, v.3. Flora of North America North of Mexico .
The description provides detailed information about the taxon, but knowledge beyond the text is required to fully understand the descriptions as it does not explicitly state that leaf is a plant organ (leaf is_a organ), that alternate is a way the leaves are arranged (alternate is_a arrangement), that stipules and petioles are part of the leaves (stipule part_of some leaf, petiole part_of some leaf), or what intermediate states between serrate and nearly entire for margin shape or between glabrous and tomentose for pubescence or hairiness are possible. To enable intelligent organization and use of organism-based morphological information (e.g., to generate phylogenetic matrices, to support machine reasoning, or to compare taxon morphological profiles), the detailed semantics need to be made explicit for computers to use. Among those the most fundamental semantics that need to be defined for the terms include is_a and part_of relationships as illustrated above, in addition, it is valuable to link synonyms and to define the semantic distance for the states that fall in a natural order. The latter will help the computer to reason that toothed leaves are more similar to serrate leaves than entire leaves.
To pin-down the semantics of domain terms, categorical glossaries and ontologies have been constructed (e.g. [2-6]), but they lack agreement. An evaluation of four glossaries/ontologies relevant to botany found that agreement was less than 50% .
Past research has shown that different taxon groups employ different vocabularies and new terminology are constantly encountered . In a recent experiment, a 2013 version of the Uberon ontology (a cross-taxon anatomy ontology, ), PATO , and BSPO  were used to annotate a set of 203 character descriptions taken from phylogenetic matrices. As much as 35% of the unique terms needed for the annotation were not found in the ontologies (manuscript in preparation). UMLS Metathesaurus  is arguably the most complete thesaurus/ontology for medicine and has been in active development for at least half of a century (one of its component MESH was introduced in 1963). A recent evaluation by Friedlin and Overhage  of the UMLS Metathesaurus found “a large portion of concepts [>70%] found in clinical narrative documents (admission, discharge, and chest x-rays) are either unrepresented [3-4%] or poorly represented [>66%] in the current version of the UMLS Metathesaurus”. The mismatch of terms used in the documents and those included in the thesaurus due to various form transformations have been cited as the main difficulty in using this and other large scale controlled vocabularies (e.g. Library of Congress Subject Headings ). These controlled vocabulary evaluations suggest that, while the coverage of the ontologies are constantly improving, the vocabulary control is a long term, continuous process. Care needs to be taken in transforming the terms for a controlled vocabulary and to provide a useful path to link the terms used in the literature to the transformed terms included in the controlled vocabularies.
Another factor that hinders the vocabulary control process is the set of diverse knowledge and skills required to perform the task. Individuals with rich knowledge in a subject domain for example biology are often not the individuals with the skills needed for ontology construction, yet both knowledge and skills are needed to perform the task of vocabulary control. Thus, knowledge modeling tools friendly to domain experts are urgently needed .
Taking the above issues into consideration, we developed the Ontology Terms Organizer (OTO; http://biosemantics.arizona.edu/OTO/) to assist a domain expert in organizing sets of terms extracted from their source literature with is_a, part_of and/or order semantics. The organized terms can be used by knowledge engineers and integrated into domain glossaries, thesauri, or ontologies. OTO is, to our knowledge, the first consensus-promoting, usage-informed, drag-and-drop based, online term organization tool designed for use by biologists or other domain experts who have rich domain knowledge but are not equipped to deal with the intricacy of formal knowledge representation using tools such as Protégé , TopBraid , and several dozen of other ontology editors (e.g. see Wikipedia page on “ontology editor”). OTO is not an ontology editor, but a tool that bridges the knowledge of domain experts and knowledge engineering.
Although OTO was initially created for biologists to categorize anatomical and morphological terms found in biodiversity literature, it can also be used generically to organize terms for any other domain, where is_a, part_of, and/or order relationships are required.
OTO also uses several web services to support its functionality. In OTO, a term is always accompanied by the source sentences where the term was found and a set of possible definitions provided for the term by other controlled vocabularies. For example, OTO uses the web services provided by the Ontology Lookup Service  to retrieve the definition of a term from the Phenotype Quality Ontology (PATO). Such information is available to the users in OTO to facilitate their decision-making process. OTO also allows a user to submit a term to a selection of existing ontologies in BioPortal and BioPortal REST Services  are employed for this purpose.
OTO delegates version control function to Github . When a term set is organized and approved by the responsible user, OTO will generate a version number for the term set, commit and push it to the biosemantics/glossaries repository on Github.
A user is anyone registered on the OTO website. A user is granted access to all the public datasets once her/his registration is approved by an administrator. A user can organize terms in any public term set, send term requests to the existing ontologies, and access finalized datasets either on OTO/via Github, as finalized term sets are publicly accessible.
A term set owner is the user who creates/uploads a term set. Every term set has an owner. In addition to the privileges of a user, a term set owner can dedicate the public/private status of an owned term set, delete an owned term set, merge owned datasets, review user term organization decisions, and finalize/reopen an owned term set.
Administrators of OTO are responsible for user management (i.e., approve/revoke a user) and have full privileges over all public and private datasets, including system reserved datasets.
A term set in OTO can be private, public, or system reserved. While private datasets are organized by the owner only and managed by the owner and administrators, any user can organize the terms in a public or a system reserved term set, and only an administrator can manage the system reserved term set (Figure 2). OTO currently holds five system reserved datasets, one for each of plants, hymenoptera, algae, porifera and fossil groups, that have already been reviewed by domain experts. System reserved datasets have a fixed naming format of Type_glossary in OTO, e.g. Plant_glossary, and is marked as “[System Reserved].” One goal of OTO is to progressively grow the system reserved glossaries from smaller datasets of the same group. When a term set (private, public, or system reserved) is finalized, it becomes accessible for all.
A MySQL database server is used to store all the data OTO uses or generates. As OTO holds multiple independent datasets, there are a set of tables that hold general data to support the general functionality of OTO, as well as a set of tables for each term set.
Results and discussion
In this section we describe OTO functionalities from the user’s perspective. Readers who are interested in learning OTO functionalities can log onto OTO with username OTOdemo and password OTOdemopass and work on the OTO_Demo dataset, which is one of many datasets currently hosted on OTO.
Term organization functionality
As OTO aims to promote consensus-based controlled vocabulary building, the most important design goal of OTO is to provide a user-friendly interface for them to express their opinions on how terms should be organized. OTO provides three ways to organize terms using drag-and-drop functionality. The first is term categorization that involves assigning one or more categories to a term, which is equivalent to assigning is_a relationships. This can be done on the Group Terms page of OTO. The second is to sort out part_of relationships among terms representing some entities, such as organs and their parts. The part_of relationships are rendered in a tree structure on the Structure Hierarchy page. The third term organization page, Term Order, serves to put range-valued categorical descriptors (e.g., terms describing varied levels of hairiness) in order. OTO records the order of terms in that group.
It is common for a term to belong to multiple categories as they may be homonyms or have different meanings under different contexts. OTO allows the user to copy a term from one category to another by holding down the Ctrl key while dragging (or the Command key on MAC). Terms that are found in multiple categories are renamed with a numerical index (e.g., sweet_1, sweet_2). There are often synonyms in a term set. Synonym relationships can be established by dragging one term onto another term. Note the synonym relations can be established only after the categorization decisions have been saved for both terms involved. Synonymy relationship can be removed by clicking on the blue “x” next to the synonym. Figure 5 shows examples of synonyms in the “Coating” category and multi-category terms (denoted by the underscore and number) in the “Coloration” category. The multi-category and synonym features used together help to move a set of unassigned terms toward a state that is better controlled. When synonyms exist, the term best representing the concept should be considered the preferred term and other synonyms should be synonymized to the preferred term. Whenever possible, avoid using terms with multiple categories (i.e. terms with a numerical index) as the preferred term because they are ambiguous. The combination of these practices will result in a sound controlled vocabulary (one term represents one concept and vice versa) with the maximized searchability (variations used in the literature are synonymized and linked to the preferred term). It also increases the chance for natural language processing techniques to perform Word Sense Disambiguation successfully on the ambiguous terms.
Two other features that make the term categorization process more efficient include Copy System Decisions and automatic display of the most recent decisions. Copy System Decisions (Figure 5, top right corner) copies the categorization decisions from the system reserved glossaries of the same taxon group to the current term grouping task, or phrased differently, terms that have matches in the system reserved glossaries are automatically dropped into the corresponding categories. The greater the coverage of the system reserved glossaries, the less effort required of the user in categorizing the terms. After the categorizations are copied, however, the user can still override the system provided decisions with his/her own categorization. The automatic display of the most recent decisions from all users who have worked on a term set allows a later user to simply review or change previous decisions (made by others) to his/her satisfaction and not have to start from the beginning.
The Term Order page is designed to sort the categorical values of entity attributes that fall in some natural order. For example, by wave length, colors can be sorted into the order: Violet, Blue, Cyan, Green, Yellow, Orange, and Red. Such orders provide useful semantic information that otherwise would be missing for computer or human agents to determine, for example, that Blue is more similar to Cyan than to Yellow. In morphological descriptions, such ordered categorical values are used but typically without precise definition, for example, the stems of a plant may be described as “… usually erect, sometimes prostrate to ascending”. The goal of the Term Order page is to invite domain experts to help define these orders, hence enabling a more precise understanding of prostrate to ascending and what intermediate states may be between prostrate and ascending.
Dataset finalization and management functionalities
The Admin Tasks page also allows dataset owners or administrators to click on the name of a dataset to view the term organization process of the dataset and to click on the “x” next to a dataset to delete a dataset. When a dataset is deleted, term sets associated with all three term organization tasks and all decisions made are permanently removed.
Dataset finalization and reopen
Dataset management: merge datasets
Merge unfinalized datasets
Merge finalized term set into system term sets
The term_category file contains all the is_a relationships between the terms and the categories. The syns file stores the synonyms. Note that synonyms are not included in the term_category file. The term_category file contains the following columns: term (string), category (string), hasSyn (1 or 0, 1 means the term has synonyms, 0 means no synonym in the syn file), sourceDataset (string, the source documents where the term is found), and termID (string, the UUID that is associated with the term/category pair). The syns file contains four columns: term (string), category (category of the term, string), termID (string, the UUID that is associated with the term/category pair), and synonym (string, the synonym). One term/category pair with multiple synonyms will result in multiple rows in the file. Note that a UUID is used to identify a ‘concept’ represented by a term/category pair, and not used to identify any synonym. In addition, both csv files contains comment lines that start with an “*” and hold metadata information about the file, such as its version number, release date, and the names of the persons who participated in the term categorization process, etc. Lastly, towards the end of the term_categorization files, we also included the natural language definitions of the categories.
Terms that are submitted to ontologies through the BioPortal web service are assigned a unique temporary ID by BioPortal upon submission and will be assigned a permanent ID if the submission is accepted by the ontology. These IDs are linked to the termID (UUID) OTO generated for each approved term categorization. Although OTO holds source sentences for each term, the BioPortal web service does not take them, so they are not included in the term submission form (Figure 19).
OTO Web services
OTO provides REST-compliant Web Services which support the functionalities of getting the available glossary types (i.e., taxon groups) in OTO, getting the most recent version of a glossary (categories and synonyms) for a certain group (Plant, etc.), getting the glossary categories and their definitions, and getting term information (categories, definitions, glossary group) associated with a term. Web services have also been implemented to support importing new term sets for various tasks. Detailed instructions can be found on the Web Service page on OTO.
Use and benefits
OTO was initially developed to be used with CharaParser to aid semantic markup of morphological descriptions and generation of taxon-character matrices from textual descriptions. CharaParser automatically extracts domain terms from textual descriptions and uploads terms directly to OTO where the terms are reviewed and categorized by the experts. Coming full circle, the finalized term set is automatically download by CharaParser and used to generate the final markup. We have been using the CharaParser-OTO combination in projects and it has proven to be a successful strategy. In this section we describe how OTO has been used to develop a rather comprehensive plant glossary incrementally (i.e., the system reserved Plant_glossary).
We started with the FNA Categorical Glossary (FNACG, ), which contained 2673 concepts with categories, definitions, and synonyms defined. Here we define concept as a term-category pair, for example, sweet (taste) and sweet (smell) are two different concepts. By using the FNACG with CharaParser to markup the morphological descriptions in FNA v.19, then using OTO to review the extracted terms, we discovered 830 new useful concepts. Interestingly, some categorizations of FNACG did not match how the terms were used in the source descriptions. For example, the shape of leaf margins are often described as entire, dentate, toothed, or lobed in description text, but in the glossary, the category for entire, dentate, and toothed is margin while the category for lobed is plane shape or solid shape. Putting terms that are alternative values for the same attribute (margin shape) into different categories artificially increases the semantic distance among these terms. The OTO Group Term page provides useful tools to detect these kinds of issues. The Glossaries panel allows the user to see if a term is included in the existing glossary and how it is categorized. The Context tab allows the user to see all the sentences a term appears in throughout all of the source documents. Thus, based on the presented knowledge of the term’s usage in reality, the user is better informed in determining a category for the term. In the example above, we decided to merge plane shape and solid shape to one shape category because in descriptions the 2-D and 3-D shape terms are often used in a mixed fashion. As a result, we moved shape terms (e.g., entire, dentate, toothed) from margin to the shape category. When terms have multiple meanings/senses a user can decide to make a copy allowing one term to be put into multiple categories.
The activities related to the categorization of the terms for Plant_glossary v0.1 and Plant_fna_foc term sets
# of unique terms
# of users
# of categorizations
# of conflict categorizations
# of comments posted
# of concepts
# of terms with multiple categories
merged (Plant_glossary v0.19)
Table 1 shows that there were 3293 unique terms to be categorized for Plant_glossary v0.1. Seven users posted 492 comments and made a total of 11781 categorizations on these terms and 44.9% (=5295/11781) of the categorizations were in conflict with some other users. After the term set had been finalized, 243 (out of 3293) terms ended up with multiple categories and the output term set contained 3559 unique concepts. The data on Plant_fna_foc suggests a similar pattern, that is, a significant portion of terms have different categorizations by different users. These differences were largely resolved in the end (a much smaller portion of terms have multiple categories in the finalized term sets). From users’ feedback, we know that the term report (comments) was a useful tool for the user, but our experience also showed that virtual and in-person meetings can help to resolve a lot of differences as well.
No formal usability test has been conducted to quantify the user-friendliness of OTO, however, the process of reviewing and finalizing these and other datasets generated numerous constructive suggestions by the users and resulted in many feature enhancements to OTO. User feedback we received suggests that the tool has become intuitive and efficient to use, especially the Group Terms functions as they are the most used in OTO to date. There are 56 registered users and 44 datasets related to plants, algae, and invertebrates (nematodes, porifera, hymenoptera, etc.) currently on OTO.
Future development plan
Our future development plan includes (1) Support for flexible ontology selection and use, for example, using a user specified anatomy ontology to initialize the Structure Hierarchy page. (2) Support other output formats, for example SKOS  or semantic wiki pages. (3) New web services for importing terms. We also plan to use Structure Hierarchy and Term Order functions to further organize FNA and FoC datasets and enhance their features based on user requests. In addition, we are working to make the term organization results available in a more user-friendly environment such as a wiki to encourage community involvement. We have created FloraTerms in TWDG’s terms wiki and made initial steps at http://terms.tdwg.org/wiki/FloraTerms.
The key features that separate OTO from other existing thesaurus [27-31] and ontology editors [15,16] are the usage-informed consensus building features, including access to source sentences, access to the decisions and comments made by other users, and visual cues signaling disagreements. Other differences are summarized as follows: OTO is not an ontology editor and it does not deal with the syntax of formal ontologies, but it supports the most fundamental ontological relations is_a and part_of in a user-friendly manner. Library-oriented controlled vocabulary construction software [27-30] relate terms using hierarchical (denoted by BT/NT or broader term/narrower term), associative (denoted by RT, or related term), and equivalence relationships (denoted by Use or Used For), without differentiating different types of hierarchical (i.e., instance, class/subclass, part_of) or associative (i.e., developed_from, created_by, etc.) relationships. OTO separates is_a and part_of relationships clearly. Existing tools require the user to type the terms in one by one, while OTO encourages manual or software batch imports, or the use of web services (future development).
A commercial platform that integrates text mining techniques with vocabulary control and information/knowledge organization is PoolParty . It enhances traditional thesaurus/taxonomy construction functions with text mining techniques and Semantic Web oriented features such as using the thesaurus/taxonomy to annotate enterprise documents and serve the annotated information as Linked Data. OTO is open source and much simpler to use as it is rather focused on consensus-based term organization.
Another tool that invites domain experts’ input to an existing ontology is NeuroLex , a semantic wiki for the neuroscience community and domain experts. “Essentially, NeuroLex is a place to accommodate the concepts and entities that are found in literatures and other legitimate sources that are not yet been realized within a formal ontology relevant to Neuroscience. NeuroLex allows a neuroscientist to add a new concept without having to worry about its deep semantic consequence due to incompleteness or partial truth about an asserted.” . Although the Wiki platform makes it possible for the users to track the editing history of a term and take part in any discussion, promoting consensus among domain experts is evidently not the primary concern of NeuroLex.
We have developed OTO to address the requirement for a user-friendly and non-technical tool to allow multiple domain experts to work collaboratively toward the creation of a controlled vocabulary that reflects the term usages in the source documents. OTO can be used to organize terms for any domain where is_a, part_of, and order relationships among terms are important for knowledge modeling and it is available online free of charge. It is open source so adapting it for other applications is possible and encouraged. Domain experts that have used OTO find the tool easy to use and have created several non-trivial glossaries which are used in ongoing projects. In the near future, we plan to release a version to the public in a wiki format to encourage further review and addition of terms and their respective definitions and illustrations.
Availability and requirements
Project Name: OTO
Project Home Page: Source Code is available at: https://github.com/biosemantics/oto. The OTO web application is running at http://biosemantics.arizona.edu/OTO/. A video introduction to OTO can be found at http://biosemantics.arizona.edu/OTO/demo.do. Short demos that show how to use OTO can be found in the Help section of OTO website after login. To test the system, use username OTOdemo and password OTOdemopass to login.
Operating Systems of Server: Linux, Windows
Other Requirements: Java JDK 6.0 or higher, MySQL Server 5.0 or higher, Apache Tomcat 5.0 or higher.
License: Open source, Apache License, Version 2.0
Restrictions to use by non-academics: None
aCharaParser  employs a bootstrap-based unsupervised machine learning method to categorize terms appearing in morphological descriptions into structure, character, and other groups. While the method does not require any training examples, it has an assumption that description sentences start with structure names and followed by characters, for example, “stems [structure] generally purple [character]”. Depending on how well the input text conforms to this assumption, in our evaluation on three real-world description collections, the accuracy of identifying structure and character terms ranged from 50% to over 80%.
bThe SQL dump contains the tables that hold the finalized results from Group Term, Term Order, and Structure Hierarchy tasks. The tables for Group Term results have the same structure as the csv files described in the paper. The table for Term Order result currently contains seven columns: orderID (varchar, not unique), orderName (varchar, user-provided name for an order), term (varchar, a term in the order), distance (int, position counted from the beginning of the order, starting from 0), accepted (tinyint, 1), userid (int, id of the user who made the decision on this term), confirmDate (datetime, time when the order is approved). The table structure makes it easy to find a term’s position in an order, but it also means that it takes N rows to make up an order, where N = the number of terms in the order. The table for Structure Hierarchy is call “paths” and contains five columns: term (varchar, the term), pathWithName (varchar, the path of the root to the term, for example “plant-flower-stamen-anther”), accepted(tinyint, 1), userid (int, id of the user who made the decision on this term), confirmDate (datetime, time when the order is approved). As the Term Order and Structure Hierarchy functions are used more often by more users, the table structures of the results may change based on user needs.
This material is based in part upon work supported by the National Science Foundation under Grant Numbers EF-0849982 and DBI-1147266. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
- Flora of North America Editorial Committee, editor. Flora of North America North of Mexico. New York: Oxford University Press; 1993.Google Scholar
- Kiger RW, Porter DM. Categorical glossary for the Flora of North America Project. 2001. ISBN 0-913196-70-3. Retrieved January 3, 2014, from http://huntbotanical.org/databases/show.php?4.Google Scholar
- Plant Characteristics, Virtual Field Herbarium http://herbaria.plants.ox.ac.uk/vfh/image/?glossary=show
- Radford AE, Dickison W, Massey J, Bell C. Vascular plant systematics. New York: Harper and Row; 1976. Retrieved Aug 4, 2014 from http://www.ibiblio.org/botnet/glossary/.Google Scholar
- PATO. Phenotypic Quality Ontology. http://wiki.obofoundry.org/wiki/index.php/PATO:Main_Page
- Harris JG, Harris MW. Plant identification terminology: an illustrated glossary. 2nd ed. Spring Lake, Utah: Spring Lake Publishing; 2001. ISBN 0964022168.Google Scholar
- Cui H. Competency evaluation of plant character ontologies against domain literature. J Am Soc Inf Sci Technol. 2010;61(6):1144–65.Google Scholar
- Cui H. Semantic annotation of morphological descriptions: an overall strategy. BMC Bioinformatics. 2010;11:278. doi:10.1186/1471-2105-11-278.View ArticlePubMedPubMed CentralGoogle Scholar
- Mungall CJ, Torniai C, Gkoutos GV, Lewis S, Haendel M. Uberon, an integrative multi-species anatomy ontology. Genome Biology. 2012;13:R5. doi:10.1186/gb-2012-13-1-r5.View ArticlePubMedPubMed CentralGoogle Scholar
- Dahdul WM, Cui H, Mabee P, Mungal CJ, Osumi-Sutherland D, Walls R, et al. Nose to tail, roots to shoots: spatial descriptors for phenotypic diversity in the Biological Spatial Ontology. J Biomed Semantics. 2014;5:34. doi:10.1186/2041-1480-5-34.View ArticlePubMedPubMed CentralGoogle Scholar
- Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004;32(Database issue):D267–70.View ArticlePubMedPubMed CentralGoogle Scholar
- Friedlin J, Overhage M. An evaluation of the UMLS in representing corpus derived clinical concepts. AMIA Symposium. 2011. p. 435–44.Google Scholar
- Library of Congress Authorities http://id.loc.gov/authorities/subjects.html
- Imam FT, Larson SD, Bandrowski A, Grethe JS, Gupta A, Martone ME. Development and use of ontologies inside the Neuroscience Information Framework: A practical approach. Front Genet. 2012;3:111. doi:10.3389/fgene.2012.00111.View ArticlePubMedPubMed CentralGoogle Scholar
- Tudorache T, Vendetti J, Noy NF. Web-Protégé: a lightweight OWL ontology editor for the web. In Proceedings of the Fifth OWLED Workshop on OWL: Experiences and Directions, 2008; Karlsruhe, Germany. Edited by Dolbear C, Ruttenberg A, Sattler U. http://ceur-ws.org/Vol-432/owled2008eu_submission_40.pdf
- Topbraid http://www.topquadrant.com/topbraid/
- Struts http://struts.apache.org/
- Model-View-Controller http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller
- Cui H. CharaParser for fine-grained semantic annotation of organism morphological descriptions. J Am Soc Inform Sci Technol. 2012;63(4):738–54.View ArticleGoogle Scholar
- Whetzel PL, Noy NF, Shah NH, Alexander PR, Nyulas C, Tudorache T, et al. BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res. 2011;39:W541–5.View ArticlePubMedPubMed CentralGoogle Scholar
- Côté R, Reisinger F, Martens L, Barsnes H, Vizcaino JA, Hermjakob H. Ontology Lookup Service: bigger and better. Nucleic Acids Research. 2010;38:W155–60. doi:10.1093/nar/gkq331. PMID: 20460452.View ArticlePubMedPubMed CentralGoogle Scholar
- BioPortal API Documentation http://data.bioontology.org/documentation
- Github https://github.com/
- A Universally Unique Identifier (UUID) URN Namespace http://www.ietf.org/rfc/rfc4122.txt
- Flora of China Editorial Committee, editor. Flora of China. Beijing: Science Press & St. Louis: Missouri Botanical Garden Press; 1994.Google Scholar
- SKOS Simple Knowledge Organization System http://www.w3.org/2004/02/skos/
- Carven T. The W32. Accessed on Aug 14, 2014 from http://publish.uwo.ca/~craven/thew32/thew32.htm
- MultiTes Pro http://www.multites.com/productspro.htm
- Cognatrix http://www.lgosys.com/products/Cognatrix/
- Term Tree http://www.acs121.com/
- PoolParty http://poolparty.biz
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.