OnEX: Exploring changes in life science ontologies
© Hartung et al; licensee BioMed Central Ltd. 2009
Received: 20 April 2009
Accepted: 13 August 2009
Published: 13 August 2009
Numerous ontologies have recently been developed in life sciences to support a consistent annotation of biological objects, such as genes or proteins. These ontologies underlie continuous changes which can impact existing annotations. Therefore, it is valuable for users of ontologies to study the stability of ontologies and to see how many and what kind of ontology changes occurred.
We present OnEX (Ontology Evolution EXplorer) a system for exploring ontology changes. Currently, OnEX provides access to about 560 versions of 16 well-known life science ontologies. The system is based on a three-tier architecture including an ontology version repository, a middleware component and the OnEX web application. Interactive workflows allow a systematic and explorative change analysis of ontologies and their concepts as well as the semi-automatic migration of out-dated annotations to the current version of an ontology.
OnEX provides a user-friendly web interface to explore information about changes in current life science ontologies. It is available at http://www.izbi.de/onex.
Life science ontologies
Ontologies have become increasingly important in life sciences [1, 2]. They consist of a set of concepts denoted by terms describing and structuring a domain of interest. Concepts are interconnected by different relationship types such as is_a and part_of relationships. A heavily used ontology is the Gene Ontology (GO)  providing sub-ontologies for molecular functions (MF), biological processes (BP) and cellular components (CC). A wide range of life science ontologies is made available by the OBO (Open Biomedical Ontologies) Foundry . The ontologies cover various life science disciplines, such as anatomy, health, biochemistry or phenotype. Other biomedical ontologies consider clinical and disease-related issues (for instance the NCI Thesaurus , SNOMED CT  or OMIM ). Due to their different focus and usage the developed ontologies vary in their size and complexity. For example, some OBO ontologies consist of only a few hundred concepts while others, such as the GO possess up to several ten thousand concepts.
There are different kinds of applications of life science ontologies. They are used for the annotation of biological objects, such as gene products and proteins. Particularly, biological objects are associated ("annotated") with ontology concepts to consistently and semantically describe their properties, for example the molecular functions and biological processes in which proteins are involved. For instance, the human protein Tubulin-specific chaperone D [Swiss-Prot:Q9BTW9] is associated with GO concepts GO:0007025, GO:0051087, GO:0005874, thereby expressing that the protein is involved in the biological process beta-tubulin folding (GO:0007025), is associated with the molecular function chaperone binding (GO:0051087) and that it acts in the cellular component microtubule (GO:0005874). Such annotations can be specified manually (for example based on experimental results) or derived automatically (for example by data mining techniques). There are different data sources providing GO annotations for various species, examples are GOA , Swiss-Prot , Ensembl , MGD  or AgBase . In a wide range of applications ontologies facilitate the structuring of and the focused search within large data sources. For instance, the GoPubMed application  makes use of MeSH  and GO to classify millions of articles of PubMed . Users can find relevant articles significantly faster by navigating and filtering along the applied ontologies. Another ontology application is the standardization of data exchange formats in heterogeneous environments by providing a common and explicit background. For example, the caBIG project  utilizes the NCI Thesaurus as a foundation for defining metadata and sharing data objects in their grid environment. Metadata stored in the central caDSR repository are semantically described by referring to concepts of the Thesaurus. Hence, ontology concepts are associated to metadata compared to the more common annotation of data objects (instances).
Usually, life science ontologies are explicitly modeled by ontology developers and scientists. The evolution of these ontologies is based on specific community agreements (at least among the ontology developers) and influenced by advances in the domain knowledge to be included in the ontologies. New research results/insights and new agreements may lead to additions or revisions of ontology elements. As a result ontologies evolve continuously and a sequence of ontology versions is provided where each version represents the state of an ontology at a specific point in time. The different versions are the basis of our change analysis. For instance, our analysis showed that in the last five years the number of concepts in GO and NCI Thesaurus more than doubled (from 13,163 to 28,250 and from 28,740 to 68,862, respectively). While most changes are additions of new concepts, many concepts have been deleted or declared obsolete (about 25 and 50 per month in GO and NCI Thesaurus, respectively). Setting an "obsolete status" for concepts is a common alternative to physically deleting a concept of an ontology. Both deleted and obsolete concepts result in a similar revision of the information represented in an ontology and may indicate a reduced stability of the ontology which negatively affects ontology usage. In particular, annotations referring to deleted or obsolete concepts are no longer valid and may have to be deleted or adapted. Other ontology changes such as additions also influence ontology usage, e.g., since they may trigger the addition of new annotations. Hence, frequent changes may impact the results of annotation analysis studies and indicate a need to rerun an analysis. Thus, there is a need to make ontology users aware of the evolution of an ontology including the quantity and quality of changes that occurred. Since not all parts of an ontology evolve at the same rate, it is useful to support the look-up of the change history for specific ontology concepts of interest, for example to see older concept names. This information may be used by curators when creating manual annotations. It would also be valuable to semi-automatically migrate out-dated annotations to newer ontology versions instead of performing a time-consuming manual adaptation.
There has been little research and support on ontology evolution and its impact for the life sciences. Several tools support change management and versioning of ontologies, e.g., the Protégé tool , the KAON infrastructure , or OBO edit  focusing on the OBO ontology file format. However, these approaches do not consider the impact of ontology changes on related data such as annotated biological objects or analysis results. Information on changes of current ontologies (history tracking) is primarily limited to mailing lists and reports by the ontology distributors. For instance, the GO consortium summarizes changes on the Gene Ontology in a monthly report which is available on their website http://www.geneontology.org/MonthlyReports/. Mailing lists notify interested users about changes and modifications of OBO ontologies at https://lists.sourceforge.net/lists/listinfo/obo-diffs. These approaches textually describe changes but cannot directly be used for automatic change processing such as an automatic migration of ontology-related data.
Theoretical aspects of ontology evolution have been studied for Semantic Web languages such as RDF  and OWL . Stojanovic et al. formalize the process of ontology evolution  and propose strategies to unambiguously handle critical changes during evolution . Klein investigates versioning of ontologies in a framework . Noy investigates change operations describing the evolution (difference) between ontology versions . Oliver et al.  present a concept model, typical change operations and a change-documentation model for change management of controlled medical terminologies.
There have been some studies to quantify the evolution of specific life science ontologies and terminologies. For instance, Ceusters has studied the evolution of concepts in SNOMED CT . Simple change statistics such as the evolution of number of concepts, relationships and paths have been determined for the Gene Ontology . In our previous work , we propose a generic evolution model and measures to study the evolution of ontologies, annotations and ontology-mappings. This framework serves as the basis for the OnEX system presented in this paper.
Online ontology tools
There are several online tools providing interfaces to query and search life science ontologies and related data. The well-known online browsers AmiGO  and QuickGO  allow to query the Gene Ontology including ontology concepts and associated gene products. Results are presented utilizing the graph structure of GO. Users can filter results and download them for further processing. Other online portals tend to integrate multiple ontologies into a single browser that offers a common query interface for all ontologies. For instance, the EBI Ontology Lookup Service (OLS)  is a web-based application allowing searching and browsing of approx. 60 life science ontologies within a central portal. The Terminology Browser of the NCI  offers a common interface to ontologies such as the NCI Thesaurus, Gene Ontology and SNOMED CT. However, the functionality of these browsers is limited to browsing and searching in one ontology version, mostly the latest one. Historical information about former ontology versions and changes on ontology concepts are rarely provided. For example, the NCI Terminology Browser merely provides the creation and modification dates for concepts.
There exist only few online tools or web services for comparing versions of an ontology. The OWS framework presented in  proposes an architecture for ontology access and manipulation through web services. A diff web service takes as input two versions of an OWL ontology and returns their structural diff (concepts that were added, deleted or changed). Another web service for comparison of ontology versions is described in . The service accepts a pair of ontologies written in OWL and returns a numeric value which represents their semantic difference. To the best of our knowledge GOChase  is the only online tool providing more detailed historical information about ontology concepts. However, the tool is specific to the Gene Ontology and is no longer maintained. Modification histories are based on GO accession numbers and the tool finds incorrect hyperlinks to GO concepts. Newer GO concepts introduced since 2006 are not captured. In contrast to GOChase, we focus on exploring and evaluating quantitative changes in many life science ontologies. Our system is not limited to one specific ontology but extensible so that further ontologies can be added in the future.
As mentioned in the previous sections, there is currently no online tool available for users to analyze the evolution of ontologies represented in different ontology versions. Ontology browsers do not support history tracking but at most provide creation and modification dates of concepts. To address these shortcomings we designed and implemented the OnEX toolkit to serve the community, especially bioinformaticians and biologists working with life science ontologies and ontology-related data.
The rest of the paper is organized as follows: The implementation section presents an overview of the OnEX system architecture. Next we discuss the import and versioning of ontologies within the system. The OnEX web application and its usage workflows are described and illustrated with sample application scenarios. The results section briefly describes the current state of the application. We finally conclude and outline possibilities for future work.
Ontology import and versioning
The import function utilizes the public archives of the ontology distributors, for example the CVS repositories of the OBO ontologies or the archive of Gene Ontology. Each ontology version integrated in OnEX is described by a version number and a release timestamp. A new ontology version is integrated by comparing elements of the new version with the latest available version in the repository. Elements include (1) concepts, (2) relationships between concepts, and (3) attributes, such as the obsolete status or alternative synonyms of a concept.
The comparison is based on our evolution model proposed in  which distinguishes between the following change types:
add : addition of an element
del : deletion of an element
value change: value of an element has changed
Change detection between two succeeding ontology versions is based on the comparison of accession numbers. An added element only exists in the new version. In contrast, a deleted element is available in the old version however not in the new version. A value change is detected if an element is available in both versions having different values, for example the name of a concept has been modified. Changes to the values of the obsolete status are especially important as they indicate whether an element has become obsolete.
The differences between two succeeding ontology versions are used to update the version repository of OnEX. Each element in the repository is assigned a start timestamp (date when the element was introduced) and an end timestamp describing the time period within which it is valid. An unspecified end timestamp indicates that an element is still valid. In case of an addition a new element is inserted with a start timestamp of its first occurrence. A deletion results in the adaptation of an element's end timestamp to the date of its last valid occurrence. In case of changes the old entries are assigned according end timestamps; furthermore new entries are created. Inspection of the obsolete attribute allows to detect concepts that have become obsolete. Another important change type are concept fusions which can be determined by studying alternative synonyms of concepts.
The repository is updated periodically with automatic routines which load and integrate the latest ontology versions from the distributor sites. While only few ontology distributors (for example the Gene Ontology consortium) release versions every day we currently consider at most one version per month (in case of several versions we pick the first one). Of course, all changes that occurred between two releases are captured by comparing the monthly ontology versions. The repository is not limited to a particular ontology format such as OBO, but we can customize the import to integrate ontologies of various formats (for example CSV, OWL, relational databases).
OnEX allows the exploration of ontology evolution by providing three interactive workflows: (1) the general Quantitative evolution analysis, (2) the Concept-based analysis and (3) the Annotation migration workflow. In the following these workflows are explained in detail.
The quantitative evolution analysis provides an overview about the evolution of the ontologies supported by OnEX. An overview table comparatively summarizes change statistics and the development of the ontology sizes. For a selected ontology, trend charts illustrate the evolution history. Different tables display the changes that occurred between ontology versions, in particular added, marked obsolete, fused and deleted concepts. Users can thus first compare the overall development of different ontologies and then focus on a specific ontology of interest. They can identify phases of ontology stability or instability to estimate the potential impact of ontology changes on annotations and analysis results of interest. Moreover, they can quickly find new or changed ontology concepts.
The concept-based evolution analysis workflow supports a detailed change analysis for specific concepts of interest. This workflow should also be relevant to curators of annotations to inspect the history of a specific concept to notice former changes such as re-definitions or new synonyms which may be useful for the manual annotation process. The workflow starts with a search for the relevant concepts, either by accession number or a keyword search across all or selected ontologies. The workflow can also be accessed from the quantitative evolution analysis workflow by selecting a concept of interest. The system provides current information about the selected concepts as well as the concept history in tabular form.
Annotation migration component
Finally, OnEX allows the migration of annotations after the release of a new ontology version. Out-dated annotations can be detected and automatically migrated to a newer (usually the latest) ontology version. The updated annotations may then be used to rerun an analysis that was originally performed for older ontology versions. The annotations to be updated are provided by an annotation file. The migration component accepts CSV annotation formats such as the GOA format. The migration workflow is based on the analysis of differences between the ontology version utilized in the user annotation file and the new ontology version. The adaptation of the annotations relies on the detected change types.
After submitting the data to the system, the Changed Concepts and Affected Annotations are reported based on the computation of changes between the relevant ontology versions. We focus on three change types that affect the annotations: deleted concepts, fused concepts and concepts marked as obsolete. A result table displays the observed ontology changes and the number of affected annotations. Now the user has different possibilities to migrate the out-dated annotations to the new ontology version. In case of deletions the corresponding annotations are deleted as well. Annotations to obsolete concepts can be deleted or updated. As some ontology distributors provide links to alternative concepts after setting them to obsolete, an annotation deletion can often be avoided. We thus integrated mappings between obsolete and alternative concepts in our repository. For instance, GO offers such mappings on their website http://www.geneontology.org/GO.downloads.files.shtml and some OBO ontologies specify alternatives within term descriptions or comments. In case of multiple migration alternatives the user can decide to which ontology concept affected annotations should be migrated. For concepts to be fused to another concept one may migrate the annotations to the fused concept or delete the annotations. Users can set migration options for a complete change type or can set individual specifications. The example of the pig gene product annotations of May 2008 possesses one fused and four obsolete ontology concepts resulting in 22 affected annotations. For instance, the discovered obsolete concept GO:0004246 (peptidyl-dipeptidase A activity) has two migration alternatives (GO:0008237 – metallopeptidase activity, GO:0008241 – peptidyl-dipeptidase activity) between which the user can choose. Finally, the specified migration settings are applied to migrate the out-dated annotations to the new ontology version. The Migrated Annotations results are provided in an output panel and can be downloaded for further processing.
As of April 2009 OnEX integrates about 560 versions of 16 life science ontologies from different ontology sources, such as Gene Ontology and OBO Foundry dating back to 2002. The version management presented in the implementation section supports an efficient storage of a large amount of ontology versions. Currently, the repository includes approx. 150,000 concepts, 260,000 relationship and 1,000,000 attribute entries which occur in different ontology versions. To avoid duplicate storage of unchanged ontology parts we only store differences between ontology versions after the first version. A specific ontology version can be retrieved by merely providing the timestamp. OnEX is not limited to a particular ontology format and can thus flexibly include further ontologies in the future.
The web application contains help pages describing the workflows step-by-step for novice users. Sample data is provided to follow the workflows and see how the system works. OnEX is running since June 2008 and has been tested by five different life science research groups at the University of Leipzig. Moreover, the system is also utilized within the German-wide MediGRID community .
OnEX provides an interactive and user-friendly access to valuable information about evolutionary changes in life science ontologies. Users are able to detect changes that occurred in an ontology version they utilize or plan to apply. For instance, one can estimate how intensive the analysis results are affected by ontology evolution and how frequent rerunning an analysis may be advised. OnEX is available at http://www.izbi.de/onex with currently approx. 560 versions of 16 life science ontologies.
Three workflows of the system address different analysis aspects: the comparative quantitative analysis of different life science ontologies, the detailed inspection of single ontology concept histories and the semi-automatic migration support for out-dated analysis results (especially annotations of molecular biological objects).
For future work, we plan to extend the system by additional change types and the identification of (un)stable ontology regions. We further plan to establish a "traffic-light" visualization for ontologies to display their stability or instability in an intuitive manner. Finally, the three-tier architecture can be extended by a web service interface to allow a software-based access to the stored ontology versions as well as the change analysis routines.
Availability and requirements
Project name: OnEX – Ontology Evolution Explorer
Project home page: http://www.izbi.de/onex
Operating system(s): Platform independent
Programming language: Java
Any restrictions to use by non-academics: none
This work is supported by Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung BMBF) grant 01AK803E "MediGRID – Networked Computing Resources For Biomedical Research" as well as the Interdisciplinary Centre for Bioinformatics founded by the German Research Foundation (Deutsche Forschungsgemeinschaft DFG). We are grateful to the anonymous reviewers for their suggestions that helped to improve the paper. We further thank David Aumüller for reading of the manuscript and providing us with language corrections.
- Bodenreider O, Stevens R: Bio-ontologies: current trends and future directions. Brief Bioinform 2006, 7(3):256–274. 10.1093/bib/bbl027PubMed CentralView ArticlePubMed
- Lambrix P, Tan H, Jakoniene V, Strömbäck L: Biological Ontologies. In Semantic Web: Revolutionizing Knowledge Discovery in the Life Sciences. Edited by: Baker CJO, Cheung KH. Springer Verlag; 2007:85–99. 10.1007/978-0-387-48438-9_5View Article
- Gene Ontology Consortium: The Gene Ontology project in 2008. Nucleic Acids Res 2008, (36 Database):D440-D444.
- Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, Goldberg LJ, Eilbeck K, Ireland A, Mungall CJ, The OBI Consortium, Leontis N, Rocca-Serra P, Ruttenberg A, Sansone SA, Scheuermann RH, Shah N, Whetzel PL, Lewis S: The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 2007, 25(11):1251–1255. 10.1038/nbt1346PubMed CentralView ArticlePubMed
- Sioutos N, de Coronado S, Haber MW: NCI Thesaurus: A semantic model integrating cancer-related clinical and molecular information. J Biomed Inform 2007, 40: 30–43. 10.1016/j.jbi.2006.02.013View ArticlePubMed
- Donnelly K: SNOMED-CT: The advanced terminology and coding system for eHealth. Stud Health Technol Inform 2006, 121: 279–290.PubMed
- Hamosh A, Scott AF, Amberger JS, Bocchini CA, McKusick VA: Online Mendelian Inheritance in Man (OMIM), a knowledgebase of human genes and genetic disorders. Nucleic Acids Res 2005, (33 Database):D514-D517.
- Barrell D, Dimmer E, Huntley RP, Binns D, O'Donovan C, Apweiler R: The GOA database in 2009-an integrated Gene Ontology Annotation resource. Nucleic Acids Res 2009, (37 Database):D396-D403. 10.1093/nar/gkn803
- Bairoch A, Boeckmann B, Ferro S, Gasteiger E: Swiss-Prot: juggling between evolution and stability. Brief Bioinform 2004, 5: 39–55. 10.1093/bib/5.1.39View ArticlePubMed
- Hubbard TJ, Aken BL, Ayling S, Ballester B, Beal K, Bragin E, Brent S, Chen Y, Clapham P, Clarke L, Coates G, Fairley S, Fitzgerald S, Fernandez-Banet J, Gordon L, Graf S, Haider S, Hammond M, Holland R, Howe K, Jenkinson A, Johnson N, Kahari A, Keefe D, Keenan S, Kinsella R, Kokocinski F, Kulesha E, Lawson D, Longden I, Megy K, Meidl P, Overduin B, Parker A, Pritchard B, Rios D, Schuster M, Slater G, Smedley D, Spooner W, Spudich G, Trevanion S, Vilella A, Vogel J, White S, Wilder S, Zadissa A, Birney E, Cunningham F, Curwen V, Durbin R, Fernandez-Suarez XM, Herrero J, Kasprzyk A, Proctor G, Smith J, Searle S, Flicek P: Ensembl 2009. Nucleic Acids Res 2009, (37 Database):D690-D697. 10.1093/nar/gkn828
- Bult CJ, Eppig JT, Kadin JA, Richardson JE, Blake JA: The Mouse Genome Database (MGD): mouse biology and model systems. Nucleic Acids Res 2008, (36 Database):D724-D728.
- McCarthy FM, Bridges SM, Wang N, Magee GB, Williams WP, Luthe DS, Burgess SC: AgBase: a unified resource for functional analysis in agriculture. Nucleic Acids Res 2007, (35 Database):D599-D603. 10.1093/nar/gkl936
- Doms A, Schroeder M: GoPubMed: exploring PubMed with the Gene Ontology. Nucleic Acids Res 2005, (33 Web Server):W783-W786. 10.1093/nar/gki470
- Medical Subject Headings[http://www.nlm.nih.gov/mesh/]
- caBIG Strategic Planning Workspace: The Cancer Biomedical Informatics Grid (caBIG): infrastructure and applications for a worldwide research community. Stud Health Technol Inform 2007, 129: 330–334.
- Knublauch H, Fergerson RW, Noy NF, Musen MA: The Protégé OWL Plugin: An Open Development Environment for Semantic Web Applications. In The Semantic Web – ISWC 2004: 3rd International Semantic Web Conference: 7–11 November 2004; Hiroshima, Japan Edited by: McIlraith SA, Plexousakis D, van Harmelen F. 2004, 229–243.
- Volz R, Oberle D, Staab S, Motik B: KAON server-a semantic web management system. Alternate Track Proceedings of the 12th International World Wide Web Conference, WWW2003: 20–24 May 2003; Budapest, Hungary 2003, 20–24.
- Day-Richter J, Harris MA, Haendel M, Gene Ontology OBO-Edit Working Group, Lewis S: OBO-Edit-an ontology editor for biologists. Bioinformatics 2007, 23(16):2198–2200. 10.1093/bioinformatics/btm112View ArticlePubMed
- Manola F, Miller E: RDF Primer. In W3C Recommendation. World Wide Web Consortium (W3C); 2004.
- McGuinness DL, van Harmelen F: OWL Web Ontology Language Overview. In W3C Recommendation. World Wide Web Consortium (W3C); 2004.
- Stojanovic L, Maedche A, Motik B, Stojanovic N: User-Driven Ontology Evolution Management. In Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, 13th International Conference, EKAW: 1–4 October 2002; Siguenza, Spain Edited by: Gómez-Pérez A, Richard Benjamins V. 2002, 285–300.View Article
- Stojanovic L, Motik B: Ontology Evolution within Ontology Editors. OntoWeb-SIG3 Workshop at the 13th International Conference on Knowledge Engineering and Knowledge Management EKAW 2002: 30th September 2002; Siguenza, Spain 2002, 53–62.
- Klein M: Change Management for Distributed Ontologies. PhD thesis. Vrije Universiteit Amsterdam; 2004.
- Noy NF, Klein M: Ontology evolution: Not the same as schema evolution. Knowl Inf Syst 2004, 6(4):428–440. 10.1007/s10115-003-0137-2View Article
- Oliver DE, Shahar Y, Shortliffe EH, Musen MA: Representation of change in controlled medical terminologies. Artif Intell Med 1999, 15: 53–76. 10.1016/S0933-3657(98)00045-1View ArticlePubMed
- Ceusters WM, Spackman KA, Smith B: Would SNOMED CT benefit from realism-based ontology evolution? AMIA Annu Symp Proc 2007, Oct 11: 105–109.
- Yang Z, Zhang D, Ye C: Ontology Analysis on Complexity and Evolution Based on Conceptual Model. In Proceedings 3rd International Workshop on Data Integration in the Life Sciences (DILS): 20–22 July 2004; Hinxton, UK Edited by: Leser U, Naumann F, Eckman BA. 2006, 216–223. 10.1007/11799511View Article
- Hartung M, Kirsten T, Rahm E: Analyzing the Evolution of Life Science Ontologies and Mappings. In Proceedings 5th International Workshop on Data Integration in the Life Sciences (DILS): 25–27 June 2008; Evry, France Edited by: Bairoch A, Cohen-Boulakia S, Froidevaux C. 2008, 11–27. 978-3-540-69827-2View Article
- Carbon S, Ireland A, Mungall CJ, Shu SQ, Marshall B, Lewis S, the AmiGO Hub, the Web Presence Working Group: AmiGO: online access to ontology and annotation data. Bioinformatics 2009, 25(2):288–289. 10.1093/bioinformatics/btn615PubMed CentralView ArticlePubMed
- Quick GO Browser[http://www.ebi.ac.uk/QuickGO/]
- Cote R, Jones P, Apweiler R, Hermjakob H: The Ontology Lookup Service, a lightweight cross-platform tool for controlled vocabulary queries. BMC Bioinformatics 2006, 7: 97. 10.1186/1471-2105-7-97PubMed CentralView ArticlePubMed
- NCI Terminology Browser[http://nciterms.nci.nih.gov/NCIBrowser]
- Dameron O, Noy NF, Knublauch H, Musen MA: Accessing and Manipulating Ontologies Using Web Services. In Proceedings of the ISWC 2004 Workshop on Semantic Web Services: Preparing to Meet the World of Business Applications, Hiroshima, Japan, Nov 8, Volume 119 of CEUR Workshop Proceedings. Edited by: Martin D, Lara R, Yamaguchi T. CEUR-WS.org, Aachen, Germany; 2004.
- Wang JZ, Ali F, Appaneravanda R: A Web Service for Efficient Ontology Comparison. ICWS '05: Proceedings of the IEEE International Conference on Web Services 2005, 843–844. 10.1109/ICWS.2005.22
- Park YR, Park CH, Kim JH: GOChase: correcting errors from Gene Ontology-based annotations for gene products. Bioinformatics 2005, 21(6):829–831. 10.1093/bioinformatics/bti106View ArticlePubMed
- Google Web Toolkit[http://code.google.com/webtoolkit/]
- Ext GWT: Rich Internet Application Framework for GWT[http://extjs.com/products/gxt]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.