EXACT2: the semantics of biomedical protocols

The standardization of experimental protocols is at the heart of GLP (good laboratory practice) and GMP (good manufacturing practice) procedures [1]. These have been essential for generating data required by regulatory agencies for many years. A high-degree of rigor is essential to ensure the reproducibility and relevance of the observations on which experimental conclusions are based.

With the increasing complexity of experimental methods there is an increasing technological need for the representation of biomedical protocols in a way that ensures that sufficient and unambiguous information is recorded to enable another agent (human or machine) to replicate these protocols. The EXACT2 (EXperimental ACTions) ontology reported in this manuscript provides a representation of experimental protocols that ensures their reproducibility and is easily processable by computer programs.

We judged that manual analysis of actions would yield higher quality results compared with text mining methods. Therefore we manually inspected hundreds of published and commercial biomedical protocols from multiple areas of biomedicine, including neurology, epigenetics, metabolomics, stem cell biology, etc. [6]. We analysed instructions, notes, alerts, properties of experimental actions, what conditions are required and what goals are specified. We noticed after several rounds of analyses that newly considered protocols, even from different areas, did not add much information to that already formalised in EXACT2 knowledge. We therefore concluded that the representation of experimental actions in the scientific literature is limited and relatively consistent. There are of course differences in lexical patterns used to express information about experimental actions, but the number of distinct experimental actions mentioned in protocols is surprisingly low (<100). The language used to describe protocols is also considerably restricted compared with natural language used in other texts. As such, this apparent 'simplicity' would be expected to offer a consistent and reproducible means for presenting protocols.

We then updated the previous version of EXACT [7] using the entities and relation identified to be relevant to the representation of experimental procedures. We did this using the ontology editor Protégé 4.3, and the reasoner HermiT 1.3.8. to detect inconsistencies. There are a number of substantial changes in EXACT2 compared to EXACT (see the summary of changes in the results section), but the most important is the addition of descriptors to each experimental action. The reason for this is that it is not possible for biologists to safely reproduce a biomedical procedure without knowing the values of such descriptors as temperature, equipment, duration of experimental actions. Unfortunately, such information is frequently missing in published protocols, or is inconsistent between different protocols, leaving interpretation up to the reader and therefore dependent on personal or collective experience. For example, vital information is missing from this description of experimental actions:

"Reconstitute bFGF and EGF with 0.1% BSA solution at a concentration of 100 μg/mL. You will need 20 μL of each per 100 mL of complete medium. Freeze unused portions in aliquots".

The liquid component of the 0.1% BSA (Bovine Serum Albumin) solution is not identified (it can be distilled water, millipore water, phosphate-buffered saline, etc.). Furthermore, these actions must be performed under sterile conditions, otherwise experiments using these materials may fail. To overcome this difficulty we consulted with experts in various biomedical areas to identify what descriptors are required, and what descriptors are optional, for each experimental action included in EXACT2.

Following OBO Foundry recommendations (see [8]), EXACT2 imports classes and relations from external resources. We employed the principle of MIREOT (Minimum Information to Reference External Ontology Terms) for consistent reference of external terms [9]. MIREOT requires the inclusion of the following information: (1) source ontology URI (Unique Resource Identifier); (2) source term URI; and (3) target direct superclass URI. We used the OntoFox web application to import external terms to EXACT2 [10]. In total, 25 terms were imported into EXACT2 from BFO, OBI, IAO, PATO (see the section below for more detail).

EXACT2 is encoded in OWL-DL, a language widely used by the research community and recommended by OBO Foundry. We used github for versioning and depositing EXACT2. The latest version of the ontology and files with external imports are available at: https://github.com/larisa-soldatova/EXACT. EXACT2 is also available at BioPortal: http://bioportal.bioontology.org/ontologies/EXACT.

The texts of biomedical protocols, like many types of text in natural language, may be ambiguous and contain errors. The automated processing of biomedical protocols has additional challenges:

This framework can be implemented in many ways. EXACT2 is encoded in a standard W3C language OWL-DL, but it can be easily translated into other formats, i.e. RDF, XML, java, or txt. Biochemical entities should be recorded along with their IDs to disambiguate these terms. Lab-specific internal IDs or IDs of the suppliers can be provided. However, it is recommended to use external IDs provided by commonly used resources like ChEBI whenever possible. There are also Biolexicons available to serve the purpose [21].It is harder to assign external IDs to equipment items. However there are projects that aim to semantically define laboratory equipment. For example the eagle-i project provides a national (US) research resource discovery platform that helps biomedical scientists search for laboratory resources [22].

There are various converters from various formats to the txt-format (see for example Zamzar converter [23]). There are many high-quality POS taggers. For example POS tagger CLAWS has consistently achieved 96-97% accuracy [24]. The National (UK) Centre for Text Mining (NaCTeM) provides various text mining tools, including GENIA Sentence Splitter (GeniaSS) optimized for biomedical texts [25]. We used Apache OpenNLP tools to process biomedical protocols we worked with (see [26]).

To illustrate the process of the identification of the experimental actions and their descriptors in the text, suppose we have the following sentences:

Adjust to 10% TCA.

Incubate at 30°C overnight.

We assume that the text has been processed using appropriate text mining tools, and all NEs and POSs have been recognised and disambiguated. For example, TCA will be found in the text and checked against both the list of abbreviations and biochemical entities. The abbreviation will be replaced by the term Trichloroacetic acid and assigned with the ID NCBI Pubchem: CID 6421. All verbs then will be checked against subclasses of the class EXACT2: experimental action. The verb 'incubate' will be matched with the class EXACT2 000049: incubate and the verb 'adjust' will be matched with the class EXACT2 000089: adjust (see Figure 2). EXACT2 defines the following descriptors for the experimental action incubate:

A translating engine recognises the defined in EXACT2 descriptors in the given sentence. It is easy to recognize the value of the descriptor temperature by the clue '°C' and the value of the descriptor period as 16 hours (see Figure 3). However, the information about the participating biochemical entity and a condition is missing. This information is defined in EXACT2 as essential and therefore has to be specified. Therefore a translating engine will ask the user to input information about what is to be incubated, and under what condition. Thus all the essential information for the reproducibility of the protocol information will be captured and represented in a semantically defined form.

A more intelligent approach for the resolving of a missing descriptors issue would be to infer the missing values from previous actions, or to use default reasoning. The user should then be asked to check if the inference is correct. For example, we observed that a biochemical entity participating in an experimental action is often not specified in protocol text. However such an entity, e.g. 'yeast growth culture', can be easily identified from the preceding experimental actions. Also, information about typical scenarios, e.g. under what conditions an incubation with the use of the specified incubator for the specified biochemical entity typically takes place, can be captured and employed as default values.

The proposed framework ensures that biomedical protocols are translated to a machine-amenable format accurately, and EXACT2 and the supporting knowledge base of semantic clues are being continuously improved. Our evaluation suggests that at present EXACT2 supports the identification of 83-95% experimental actions in protocol texts (depending on if a protocol is from a research area that has been already used for the construction of EXACT2). The coverage of EXACT2 is improving through the analysis on new protocols. We propose that protocols translated to a machine amenable format should be deposited to public repositories for future re-use. Many operations on such protocols, like search, comparison and retrieval, will be enhanced and yield more accurate results.

Ontologies are well suited for the representation of declarative and static domain knowledge, but they generally struggle with the representation of complex sequences of events unfolding in time. However there is a need in the integration of a rigorous logical representation of key experimental steps, with the temporal sequence of those steps, so as to capture the explicit semantics of whole experimental procedures.

There were several attempts to integrate EXACT with the representation of experimental workflows. In the original work on EXACT [7] we encoded protocols in Python. This approach to the representation of workflows was not popular with biologists. Maccagnan et al [27] developed the COW (Combining Ontologies with Workflows) software to formalize workflows built on ontologies. The method was specifically set up to support the design of structured protocols for biological laboratory experiments. The workflows were enhanced with ontological concepts taken from the developed domain-specific ontologies, including EXACT [25]. Unfortunately this project has now been abandoned (personal communication).

In order to adress the need for rigorous representation of both experimental actions and their sequence, we have recently experimented with using the Petri net formalism to representat workflows of experimental procedures, with encouraging results. Petri nets are used as a formal and graphically appealing language for modelling systems. They are suitable for the representation of biochemical reactions in metabolism, signal transduction and gene expression, and in reconstructing complex molecular networks. For example, Petri nets have been applied to the regulation of the lac operon [28]. Duchenne muscular dystrophy [29], the response of S. cerevisiae to mating hormones [30], and the yeast cell cycle [31].

A general Petri net has the following main components [32]:

Petri net semantic describes the behaviour of nets [32]. We suggest to deepen the semantic representations of Petri nets by defining not only the behaviour of the system, but also the semantic meaning of each element of a Petri net representing an experimental workflow. Experimental actions can be encoded as transitions, the most essential descriptors of experimental actions such as time and vital experimental conditions as places may be marked with a token indicating whether or not the condition is fulfilled (true). Arcs can be used to represent the sequence of experimental actions.

For example, we considered the experimental procedures for isolation of Physarum polycephalum plasmodial mutants altered in sporulation obtained by chemical mutagenesis of flagellates [33].Corresponding genetic screens are run in the Magdeburg Centre for Systems Biology [34]. Physarum polycephalum belongs to the amoebozoa group of organisms. The experimental procedure is complex, non deterministic and takes ten days to complete (see the 'Growth of amoeba and preparation of flagellate suspension' procedure in the materials and methods section, [33]). Figure 4 shows a fragment of this procedure represented with a Petri net. The semantics of the places and transitions is defined with the use of the EXACT2 classes. The experimental actions, e.g. streak, transfer, are represented as transitions, and the key descriptors of those experimental actions are represented as places, e.g. amoebae microcolony, DSPB agar plate . In order to support the representation of such workflows, we will have to add to EXACT2 such descriptor as time point. Also, transition firing rules have to be defined appropriately in order to ensure the correct dynamic behaviour of the Petri net.

We suggest that an integration of the formalisms of Petri nets and EXACT2 will provide a powerful representation of experimental workflows. It would not only fully capture the semantics of experimental procedures, but also would enable one to simulate such procedures before physically executing them in the laboratory.

A key insight of the scientific revolution was the importance of experimental results that could be reproduced in different laboratories [35]. What was true in the 17^th century is still true today in the 21^st century: repeatable experiments are a hallmark of science. It is widely accepted that for new knowledge to be published in a scientific journal the protocols used to derive that new knowledge must also be published. The protocols are necessary to reproduce the observations upon which the knowledge is based, and to enable this the protocols need to be completely specified.

Modern laboratory science requires the use of sophisticated protocols [36, 37]. However, these are still generally expressed using natural language, and unfortunately use of natural language inevitably introduces ambiguities about how to reproduce experiments. The result is the failure to reproduce results, with the subsequent loss of time and money. While working on this paper we noticed that yet another research paper has been retracted from Nature because results could not be reproduced. Obokata et al. [38] in their paper titled 'Stimulus-triggered fate conversion of somatic cells into pluripotency' reported on a cheap and quick method of producing stem cells. The Reuters news agency reports Prof. Wakayama of the University of Yamanashi told Japanese TV: "When conducting the experiment, I believed it was absolutely right... But now that many mistakes have emerged, I think it is best to withdraw the research paper". We argue that if experimental procedures reported by [38] were expressed with the use of EXACT2 then mistakes could be identified earlier and other groups could reproduce their results more easily.

In this paper we present a fundamentally new version of the ontology EXACT2 designed to support the accurate and computer friendly recording of information about biomedical procedures. EXACT2 follows best practice in ontology development, and the recommendations of the OBO Foundry. It can therefore be directly integrated with other bio-medical ontologies.

We present a framework for the translation of biomedical protocols from natural text to a machine amenable semantically-defined format. The proposed framework employes EXACT2 as a reference model to identify experimental actions and their descriptors in protocol texts, and assigns them unique IDs.

We also demonstrate that the integration of EXACT2 with the formalism of Petri nets will enable the capture of explicit semantics of experimental workflows, and validate the workflow through simulations of the experimental procedure. We suggest that biomedical protocols represented in a formal machine friendly way should be submitted to public repositories for future re-use.

EXACT - an ontology of EXperimental ACTions; GLP - Good Laboratory Practice; GMP - Good Manufacturing Practice; OBI - the Ontology for Biomedical Investigations; BSA - Bovine Serum Albumin; OBOF - Open Biomedical Ontologies Foundry; MIREOT - Minimum Information to Reference External Ontology Terms; BFO - the Basic Formal Ontology; IAO - the Information Artifact Ontology; PATO - Phenotype And Trait Ontology; ChEBI - Chemical Entities of Biological Interest; SUMO - the Suggested Upper Merged Ontology; OED - Oxford English Dictionary; URI - Unique Resource Identifier; ID - Identifier; ON - Overnight; RT - Room Temperature; DMSO - DiMethylSulfOxide; NE - Named Entities; POS - Part Of Speech.