Inferring ontology graph structures using OWL reasoning
© The Author(s) 2018
Received: 24 May 2017
Accepted: 13 December 2017
Published: 5 January 2018
Ontologies are representations of a conceptualization of a domain. Traditionally, ontologies in biology were represented as directed acyclic graphs (DAG) which represent the backbone taxonomy and additional relations between classes. These graphs are widely exploited for data analysis in the form of ontology enrichment or computation of semantic similarity. More recently, ontologies are developed in a formal language such as the Web Ontology Language (OWL) and consist of a set of axioms through which classes are defined or constrained. While the taxonomy of an ontology can be inferred directly from the axioms of an ontology as one of the standard OWL reasoning tasks, creating general graph structures from OWL ontologies that exploit the ontologies’ semantic content remains a challenge.
We developed a method to transform ontologies into graphs using an automated reasoner while taking into account all relations between classes. Searching for (existential) patterns in the deductive closure of ontologies, we can identify relations between classes that are implied but not asserted and generate graph structures that encode for a large part of the ontologies’ semantic content. We demonstrate the advantages of our method by applying it to inference of protein-protein interactions through semantic similarity over the Gene Ontology and demonstrate that performance is increased when graph structures are inferred using deductive inference according to our method. Our software and experiment results are available at http://github.com/bio-ontology-research-group/Onto2Graph.
Onto2Graph is a method to generate graph structures from OWL ontologies using automated reasoning. The resulting graphs can be used for improved ontology visualization and ontology-based data analysis.
An ontology is an explicit representation of a conceptualization of a domain [1, 2], and ontologies are widely applied in biology and biomedicine for annotation and integration of data . The BioPortal ontology repository alone now lists over 500 ontologies , with several more ontologies under development. In the past, ontologies in biology were widely developed as directed acyclic graphs (DAGs) in which nodes stand for classes of entities within a domain, and edges for relations between these classes. For example, the classes developmental cell growth (GO:0048588), cell growth (GO:0016049) and cell development (GO:0048468) in the Gene Ontology (GO)  would be represented as nodes, and the relations between them by an edge from developmental cell growth to cell growth with an is-a label, and from developmental cell growth to cell development with a part-of label .
More recently, many ontologies are implemented in the Web Ontology Language (OWL) . OWL is a formal language based on Description Logics  and offers a formal, model-theoretic semantics. Consequently, there have been several approaches for converting graph-based representations of ontologies into representations based on first order logic or description logic. For example, the OBO Relation Ontology  provides a systematic way to transform graphs into formal theories by giving explicit definitions for relations. Furthermore, approaches have been developed to convert graph-based representations of ontologies into OWL ontologies using an explicit translation relation [9, 10].
However, ontologies are not only used to express the knowledge within a domain but also for data analysis . In particular, ontology enrichment analysis and semantic similarity measures are applied for predicting protein-protein interactions [11, 12], finding candidate genes of diseases [13–15] or classifying chemicals . Most of these measures crucially rely on graph structures . For example, the majority of semantic similarity measures used in biology are graph similarity measures , and ontology enrichment analysis utilizes the graph structure of ontologies to detect over- or under-represented classes [19, 20]. Consequently, there is now a gap between the increasingly more formal representation languages used for ontologies in biology and the analysis methods that utilize them, and a need to generate graph structures from ontologies that also take into account the semantics of the axioms in ontologies.
One of the standard reasoning tasks in OWL ontologies is the generation of the backbone taxonomy underlying an ontology  based on the axioms provided. This classification task is used to generate graphs in which subsumption (i.e., is-a) relations are expressed, but cannot easily be used to generate different types of edges, such as those labeled part-of, which represent axioms involving complex class descriptions . In general, these edges can also not be created syntactically; an obvious example is a general concept inclusion axiom (i.e., an axiom in which a complex class description instead of a named class appears on both sides of a subclass axiom), in which axioms involving object properties cannot clearly be associated with a single class, or the inferences resulting from the use of inverse object properties or property hierarchies. While axioms in OWL may be arbitrarily complex and may not easily be representable in a graph-based form, they may imply axioms that can naturally be expressed in the form of a graph. For example, when nodes in a graph represent named classes, an axiom such as A or B SubClassOf: R some C cannot be represented (as A or B would not have a representation). However, this axiom implies that both A SubClassOf: R some C and B SubClassOf: R some C, and these inferences can be represented by two edges labeled R between A and C as well as between B and C.
Here, we describe a method to generate graph structures from OWL ontologies using only the semantic information (i.e., the axioms) contained in the ontologies combined with automated reasoning. We extend our previous work on visualizing ontologies in the AberOWL ontology repository  by improving our algorithm to generate sparser graphs (through the use of a transitive reduction) and making our conversion available as a stand-alone tool so that other researchers can integrate it in their analyses. Our method generates taxonomies as well as graphs containing other types of edges. We demonstrate that the graphs generated by our method outperform taxonomies and graphs generated using syntactic approaches when predicting protein-protein interactions through measures of similarity, demonstrating that our approach not only improves usability and representation of ontologies but also ontology-based data analysis methods. We implement our method in the Onto2Graph tool which is freely available at http://github.com/bio-ontology-research-group/Onto2Graph.
We obtained a list of all ontologies from the AberOWL ontology repository  to run our experiments. We downloaded all ontologies on 4 November 2015. We further perform a detailed evaluation on the Gene Ontology (GO) , and the GO extended with additional axioms and links to other ontologies, GO-Plus , also downloaded from the AberOWL ontology repository on 4 November 2015.
Interaction datasets and functional annotations
For evaluation of the performance of different types of graphs in computing semantic similarity, we selected the Biological General Repository for Interaction Datasets (BioGRID) , which contains over one million protein-protein interactions and genetic interactions that occur in different types of organisms. Particularly, we selected the protein-protein interactions and genetic interactions occurring in fruitfly (Drosophila melanogaster), mouse (Mus musculus), nematode worm (Caenorhabditis elegans), yeast (Saccharomyces cerevisiae) and zebrafish (Danio rerio) to evaluate our results. We downloaded all interaction data from BioGRID on 29/11/2015.
As second interaction dataset, we identified GO annotations with the IGI (inferred from genetic interaction) and IPI (inferred from protein interaction) evidence codes. These annotations contain the interaction partner as part of the annotation, and we use these as a second interaction dataset for evaluation (separated into protein-protein interactions for the IPI evidence code and genetic interactions for IGI).
Overview of the databases used in this work
Number of genetic interactions (IGI)
Number of physical interactions (IPI)
The Onto2Graph tool is developed in the Groovy language and implements the conversion algorithm (see Algorithm 1) to automatically transform OWL ontologies into graphs. Onto2Graph can generate graphs in different representation formats: RDF/XML , GraphViz , the OBO Flatfile Format , GraphML , and an output format used for the ontology enrichment tool OntoFUNC . Onto2Graph uses the OWLAPI  to process ontologies and integrates the Elk reasoner , HermiT  as well as the structural reasoner that is part of the OWLAPI . Output formats and reasoners can be selected as command line parameters and are generated using the Java Universal Network/Graph Framework (JUNG) .
In order to enable users to visualize the graphs, we generate graphs using OWLAPI’s structural reasoner and the Elk reasoner for all ontologies in AberOWL and store them in an OpenLink Virtuoso RDF store  for which we provide a public SPARQL endpoint at http://bio2vec.net/sparql/. Differences between syntactically generated graphs and graphs generated through the Elk reasoner can be retrieved through SPARQL queries. We further developed a visualisation environment to browse the structure of the graphs and analyse them easily. The visualization is based on LodLive project , and we modified the project so that it is possible to browse two graphs simultaneously for comparison. The resulting web interface is located in http://bio2vec.net/graphs/.
Computing similarity and evaluation
We compute semantic similarity over the GO using the Semantic Measures Library (SML) . We use the simGIC similarity measure  and Resnik’s measure  to compute pairwise semantic similarity between proteins within a species, using the best-match average as strategy to combine multiple pairwise similarity values. As the SML considers only subclass edges when computing semantic similarity, we rewrite other edge types generated through our algorithm as subclass edges before computing semantic similarity.
For each protein, we rank each protein by their similarity in descending order. Using our datasets of interactions as positive instances and all other pairs of proteins as negatives, we generate the ROC curves and compute the area under the ROC curve (ROCAUC) . When comparing the difference between two ROC curves, we compute the difference in ROCAUC and perform a Wilcoxon rank sum test to determine whether the difference is significant .
Results and discussion
Converting OWL ontologies into graphs
Furthermore, to build a more concise graph while considering the semantics of the axioms involving object properties, we have added the option to perform a transitive reduction of the resulting graph over edges resulting from transitive object properties, subclass edges, and any combinations thereof.
The conversion is performed in two different steps (see Algorithm 1). In the first step, the algorithm processes the ontology and pre-computes the candidate o-successors of each class. In the second step, the o-successors are identified and added to the output graph as edges; if required, a transitive reduction is performed at this stage. The backbone of the graph is formed by the taxonomy of the ontology, i.e., the subclass and equivalent class relations between named classes, and we add the o-successors generated for each class: if C has an o-successor D, we generate an edge from C to D labeled o. The algorithm can generate multiple edges with different labels between the same nodes. For example, if o1 is a sub-property of o2 and an o1-labeled edge is generated between nodes X and Y, then our algorithm will also generate an o2-labeled edge between X and Y unless this edge is removed due to a transitive reduction.
We implement two versions of the algorithm, one in which all operations are performed semantically through an OWL reasoner, and another in which operations are performed syntactically by analyzing the expression of the axioms. When using OWL reasoning, we currently use either the Elk reasoner  or HermiT , and plan to support further reasoners in the future.
When analyzing the OWL axioms syntactically, instead of using the Elk reasoner, we use the OWLAPI  to obtain all asserted subclass and equivalent class axioms in the ontology; within these, we identify the axioms in which a single class is asserted to be a subclass or equivalent class to a class expression C exp . We then examine whether C exp syntactically follows the pattern o some X to generate the candidate o-successors X.
Semantically generating graphs improves performance of semantic similarity
Graph structures generated from ontologies are used for visualization as well as by several data analysis methods, and we evaluate the generated graphs by applying a (graph-based) semantic similarity measure to genes and gene products annotated with GO and evaluating the results for their performance in predicting protein-protein interactions and genetic interactions. To perform this evaluation, we select the GO-Plus ontology . GO-Plus contains all the axioms in GO together with additional axioms, and may therefore be more suitable to demonstrate our approach as more edges can be inferred based on the additional axioms.
Runtime of the Onto2Graph algorithm and semantic similarity computation over the GO-Plus ontology
Semantic similarity runtime
1 min 37 s
4 min 59 s
0 min 58 s
4 min 54 s
SubClassOf + PartOf
10 min 50 s
5 min 42 s
10 min 37 s
5 min 44 s
SubClassOf + PartOf + Regulates
16 min 34 s
6 min 21 s
16 min 10 s
6 min 15 s
0 min 48 s
3 min 25 s
0 min 46 s
3 min 30 s
SubClassOf + PartOf
0 min 49 s
3 min 59 s
0 min 47 s
4 min 8 s
SubClassOf + PartOf + Regulates
0 min 48 s
5 min 8 s
0 min 49 s
4 min 42 s
Summary of results obtained from graphs based on asserted axioms vs graphs semantically generated
Type of interaction
SubClassOf PartOf Regulates
We find that performance in predicting both protein-protein interactions and genetic interactions generally improves when using graphs generated by the Elk reasoner compared to graphs generated syntactically. While the increase in ROCAUC is not very large, it is, however, significant for several of our evaluation datasets. For example, for genetic interactions in yeast, we observe an increase of 0.011 AUC, which is significant (p=1.2×10−24, Mann-Whitney U test).
We developed the Onto2Graph conversion algorithm and tool that enables users to convert ontologies into graphs efficiently and utilizes an automated reasoner to infer edges in an ontology graph based on the ontology’s deductive closure. The tool integrates two different ways to perform this conversion, by using OWL reasoning and by syntactically analyzing the ontology axioms. The Onto2Graph tool can output graphs generated from OWL ontologies in several file formats which can then be used for ontology-based data analysis, such as semantic similarity or ontology enrichment analysis.
We demonstrated that the graphs generated by Onto2Graph can outperform graph structures generated syntactically or based on the ontology’s taxonomy alone when applied to computation of semantic similarity and prediction of protein-protein interactions. While the observed differences are small, our results nevertheless demonstrate how inclusion of more information that is already present within ontologies can contribute to biological discovery.
A major limitation of our current approach is the reliance on a single (existential) pattern to generate edges while many ontologies now use more complex axioms. While the Onto2Graph method can be applied to other relational patterns that should represent an edge within a graph, we did not implement this due to the computational costs involved in using arbitrary OWL axiom patterns . In the future, the graph generated by our approach can also be used to infer additional edges used to build knowledge graph embeddings , and therefore contribute to applications of machine learning with ontologies.
This work has been supported by funding from King Abdullah University of Science and Technology (KAUST).
Availability of data and materials
Our software is freely available from http://github.com/bio-ontology-research-group/Onto2Graph. Analysis results are available as Supplementary materials.
MARG implemented the method and performed all experiments. RH and MARG designed the algorithm and experiments. RH and MARG drafted and revised the manuscript. RH supervised the research. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Gruber TR. Toward principles for the design of ontologies used for knowledge sharing. Int J Hum-Comput Stud. 1995; 43(5-6). doi:10.1006/ijhc.1995.1081.
- Guarino N. Formal ontology and information systems. In: Proceedings of the 1st International Conference on Formal Ontologies in Information Systems. Amsterdam: IOS Press: 1998. p. 3–15.Google Scholar
- Hoehndorf R, Schofield PN, Gkoutos GV. The role of ontologies in biological and biomedical research: a functional perspective. Brief Bioinforma. 2015; 16(6):1069–80.View ArticleGoogle Scholar
- Noy NF, Shah NH, Whetzel PL, Dai B, Dorf M, Griffith N, Jonquet C, Rubin DL, Storey MA, Chute CG, Musen MA. Bioportal: ontologies and integrated data resources at the click of a mouse. Nucleic Acids Res. 2009; 37(suppl_2):170–3. doi:10.1093/nar/gkp440. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/37/suppl_2/10.1093/nar/gkp440/2/gkp440.pdf.View ArticleGoogle Scholar
- Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, et al. Gene ontology: tool for the unification of biology. Nat Genet. 2000; 25(1):25–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Smith B, Ceusters W, Klagges B, Köhler J, Kumar A, Lomax J, Mungall C, Neuhaus F, Rector AL, Rosse C. Relations in biomedical ontologies. Genome Biol. 2005; 6(5):46. doi:10.1186/gb-2005-6-5-r46.View ArticleGoogle Scholar
- Grau BC, Horrocks I, Motik B, Parsia B, Patel-Schneider P, Sattler U. OWL 2: The next step for OWL. Web Semant Sci Serv Agents World Wide Web. 2008; 6(4):309–22. doi:10.1016/j.websem.2008.05.001.View ArticleGoogle Scholar
- Baader F, Calvanese D, McGuinness D, Nardi D, Patel-Schneider P. The Description Logic Handbook: Theory, Implementation and Applications. Cambridge: Cambridge University Press; 2003.Google Scholar
- Horrocks I. OBO Flat File Format Syntax and Semantics and Mapping to OWL Web Ontology Language. Technical report. University of Manchester: 2007. http://www.cs.man.ac.uk/~horrocks/obo/. Accessed 12 Oct 2017.
- Hoehndorf R, Oellrich A, Dumontier M, Kelso J, Rebholz-Schuhmann D, Herre H. Relations as patterns: Bridging the gap between OBO and OWL. BMC Bioinformatics. 2010; 11(1):441.PubMedPubMed CentralView ArticleGoogle Scholar
- Guzzi PH, Mina M, Guerra C, Cannataro M. Semantic similarity analysis of protein data: assessment with biological features and issues. Brief Bioinforma. 2011; 13(5):569–85. doi:10.1093/bib/bbr066. http://bib.oxfordjournals.org/content/early/2011/12/02/bib.bbr066.full.pdf+html.View ArticleGoogle Scholar
- Benabderrahmane S, Smail-Tabbone M, Poch O, Napoli A, Devignes MD. IntelliGO: a new vector-based semantic similarity measure including annotation origin. BMC Bioinformatics. 2010; 11(1):588. doi:10.1186/1471-2105-11-588.PubMedPubMed CentralView ArticleGoogle Scholar
- Köhler S, Schulz MH, Krawitz P, Bauer S, Doelken S, Ott CE, Mundlos C, Horn D, Mundlos S, Robinson PN. Clinical diagnostics in human genetics with semantic similarity searches in ontologies. Am J Hum Genet. 2009; 85(4):457–64.PubMedPubMed CentralView ArticleGoogle Scholar
- Hoehndorf R, Schofield PN, Gkoutos GV. Phenomenet: a whole-phenome approach to disease gene discovery. Nucleic Acids Res. 2011; 39(18):119.View ArticleGoogle Scholar
- Schlicker A, Albrecht M. FunSimMat update: new features for exploring functional similarity. Nucleic Acids Res. 2010; 38(suppl_1):244–8. doi:10.1093/nar/gkp979. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/38/suppl_1/10.1093_nar_gkp979/1/gkp979.pdf.View ArticleGoogle Scholar
- Ferreira JD, Couto FM. Semantic similarity for automatic classification of chemical compounds. PLoS Comput Biol. 2010; 6(9):1–11. doi:10.1371/journal.pcbi.1000937.View ArticleGoogle Scholar
- Harispe S, Ranwez S, Janaqi S, Montmain J. The semantic measures library and toolkit: fast computation of semantic similarity and relatedness using biomedical ontologies. Bioinformatics. 2014; 30(5):740–2.PubMedView ArticleGoogle Scholar
- Pesquita C, Faria D, Falcão AO, Lord P, Couto FM. Semantic Similarity in Biomedical Ontologies. PLoS Comput Biol. 2009; 5(7):1–12. doi:10.1371/journal.pcbi.1000443.View ArticleGoogle Scholar
- Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL, Golub TR, Lander ES, Mesirov JP. Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A. 2005; 102(43):15545–50. doi:10.1073/pnas.0506580102. http://www.pnas.org/content/102/43/15545.full.pdf+html.PubMedPubMed CentralView ArticleGoogle Scholar
- Wittkop T, TerAvest E, Evani U, Fleisch K, Berman A, Powell C, Shah N, Mooney S. STOP using just GO: a multi-ontology hypothesis generation tool for high throughput experimentation. BMC Bioinformatics. 2013; 14(1):53. doi:10.1186/1471-2105-14-53.PubMedPubMed CentralView ArticleGoogle Scholar
- Rodríguez-García MÁ, Slater L, O’Shea K, Schofield PN, Gkoutos GV, Hoehndorf R. Visualizing ontologies with AberOWL. In: Semantic Web Applications and Tools for Health Care and Life Sciences. SWAT4LS 2015, vol. 1546. Aachen: CEUR-WS.org: 2015. p. 183–92.Google Scholar
- Hoehndorf R, Slater L, Schofield PN, Gkoutos GV. Aber-OWL: a framework for ontology-based data access in biology. BMC Bioinformatics. 2015; 16(1):1.View ArticleGoogle Scholar
- Mungall CJ, Dietze H, Osumi-Sutherland D. Use of OWL within the gene ontology. In: OWL: Experiences and Directions Workshop 2014. OWLED2014, vol. 1256. Aachen: CEUR-WS.org: 2014. p. 25–36.Google Scholar
- Stark C, Breitkreutz BJ, Reguly T, Boucher L, Breitkreutz A, Tyers M. BioGRID: a general repository for interaction datasets. Nucleic Acids Res. 2006; 34(suppl_1):535–9.View ArticleGoogle Scholar
- Attrill H, Falls K, Goodman JL, Millburn GH, Antonazzo G, Rey AJ, Marygold SJ, Consortium F. Flybase: establishing a gene group resource for drosophila melanogaster. Nucleic Acids Res. 2016; 44(D1):786–92. doi:10.1093/nar/gkv1046. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/44/d1/10.1093_nar_gkv1046/3/gkv1046.pdf.View ArticleGoogle Scholar
- Eppig JT, Blake JA, Bult CJ, Kadin JA, Richardson JE, Group MGD. The mouse genome database (mgd): facilitating mouse as a model for human biology and disease. Nucleic Acids Res. 2015; 43(D1):726–36. doi:10.1093/nar/gku967. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/43/d1/10.1093_nar_gku967/2/gku967.pdf.View ArticleGoogle Scholar
- Harris TW, Antoshechkin I, Bieri T, Blasiar D, Chan J, Chen WJ, De La Cruz N, Davis P, Duesbury M, Fang R, Fernandes J, Han M, Kishore R, Lee R, Mller HM, Nakamura C, Ozersky P, Petcherski A, Rangarajan A, Rogers A, Schindelman G, Schwarz EM, Tuli MA, Van Auken K, Wang D, Wang X, Williams G, Yook K, Durbin R, Stein LD, Spieth J, Sternberg PW. Wormbase: a comprehensive resource for nematode research. Nucleic Acids Res. 2010; 38(suppl1):463–7. doi:10.1093/nar/gkp952. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/38/suppl_1/10.1093_nar_gkp952/1/gkp952.pdf.View ArticleGoogle Scholar
- Engel SR, Balakrishnan R, Binkley G, Christie KR, Costanzo MC, Dwight SS, Fisk DG, Hirschman JE, Hitz BC, Hong EL, Krieger CJ, Livstone MS, Miyasato SR, Nash R, Oughtred R, Park J, Skrzypek MS, Weng S, Wong ED, Dolinski K, Botstein D, Cherry JM. Saccharomyces genome database provides mutant phenotype data. Nucleic Acids Res. 2010; 38(suppl1):433–6. doi:10.1093/nar/gkp917. http://arxiv.org/abs//oup/backfile/content_public/journal/nar/38/suppl_1/10.1093_nar_gkp917/1/gkp917.pdf.View ArticleGoogle Scholar
- Bradford Y, Conlin T, Dunn N, Fashena D, Frazer K, Howe DG, Knight J, Mani P, Martin R, Moxon SA, et al. Zfin: enhancements and updates to the zebrafish model organism database. Nucleic Acids Res. 2011; 39(suppl 1):822–9.View ArticleGoogle Scholar
- Becket D, McBride B. RDF/XM syntax specification (revised). w3c recommendation. World Wide Web Consortium. 2004. http://www.w3.org/TR/rdf-syntax-grammar. Accessed 12 Oct 2017.
- Ellson J, Gansner E, Koutsofios L, North SC, Woodhull G. Graphviz– open source graph drawing tools. In: Graph Drawing. GD 2001, vol. 2265. Berlin: Springer: 2001. p. 483–4.Google Scholar
- Mungall CJ, Ireland A. OBO Flat File Format 1.4 Syntax and Semantics [DRAFT]. 2016. https://owlcollab.github.io/oboformat/doc/GO.format.obo-1_4.html. Accessed 12 Oct 2018.
- Brandes U, Eiglsperger M, Herman I, Himsolt M, Marshall MS. GraphML Progress Report Structural Layer Proposal. In: Graph Drawing. GD 2001, vol. 2265. Berlin: Springer: 2001. p. 501–12.Google Scholar
- Hoehndorf R, Dumontier M, Gkoutos GV. Identifying aberrant pathways through integrated analysis of knowledge in pharmacogenomics. Bioinformatics. 2012; 28(16):2169–75.PubMedPubMed CentralView ArticleGoogle Scholar
- Horridge M, Bechhofer S. The owl api: A java api for owl ontologies. Semant Web. 2011; 2(1):11–21.Google Scholar
- Kazakov Y, Krötzsch M, Simancik F. Elk reasoner: Architecture and evaluation. In: OWL Reasoner Evaluation Workshop 2012. ORE–2012, vol. 858. Aachen: CEUR-WS.org: 2012. p. 10.Google Scholar
- Shearer R, Motik B, Horrocks I. HermiT: A highly-efficient owl reasoner. In: OWL: Experiences and Directions Workshop. OWLED2008, vol. 432. Aachen: CEUR-WS.org: 2008. p. 11.Google Scholar
- O’Madadhain J, Fisher D, White S, Boey Y. The JUNG (java universal Network/Graph) framework. Technical report. UCI-ICS. 2003. http://www.datalab.uci.edu/papers/JUNG_tech_report.html. Accessed on 12 Oct 2017.
- Erling O, Mikhailov I. RDF Support in the Virtuoso DBMS. In: Networked Knowledge - Networked Media: Integrating Knowledge Management, New Media Technologies and Semantic Systems, vol. 221. Berlin: Springer: 2009. p. 7–24.Google Scholar
- Camarda DV, Mazzini S, Antonuccio A. LodLive, Exploring the Web of Data. In: Proceedings of the 8th International Conference on Semantic Systems. I-SEMANTICS ’12. New York: ACM: 2012. p. 197–200. doi: 10.1145/2362499.2362532. http://doi.acm.org/10.1145/2362499.2362532.Google Scholar
- Pesquita C, Faria D, Bastos H, Ferreira AE, Falcão AO, Couto FM. Metrics for GO based protein semantic similarity: a systematic evaluation. BMC Bioinformatics. 2008; 9(5):4. doi:10.1186/1471-2105-9-S5-S4.View ArticleGoogle Scholar
- Resnik P. Semantic similarity in a taxonomy: An Information-Based measure and its application to problems of ambiguity in natural language. J Artif Intell Res. 1999; 11:95–130.Google Scholar
- Fawcett T. An introduction to ROC analysis. Pattern Recogn Lett. 2006; 27(8):861–74.View ArticleGoogle Scholar
- Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143(1):29–36.PubMedView ArticleGoogle Scholar
- Kazakov Y, Krötzsch M, Simancik F. The incredible Elk. J Autom Reason. 2014; 53(1):1–61. doi:10.1007/s10817-013-9296-3.View ArticleGoogle Scholar
- Dudoit S, Shaffer JP, Boldrick JC. Multiple hypothesis testing in microarray experiments. Stat Sci. 2003; 18(1):71–103.View ArticleGoogle Scholar
- Alshahrani M, Khan MA, Maddouri O, Kinjo AR, Queralt-Rosinach N, Hoehndorf R. Neuro-symbolic representation learning on biological knowledge graphs. Bioinformatics. 2017; 33(17):2723–30. doi:10.1093/bioinformatics/btx275. http://arxiv.org/abs//oup/backfile/content_public/journal/bioinformatics/33/17/10.1093_bioinformatics_btx275/2/btx275.pdf.PubMedView ArticleGoogle Scholar