ChemEx: information extraction system for chemical data curation

Background Manual chemical data curation from publications is error-prone, time consuming, and hard to maintain up-to-date data sets. Automatic information extraction can be used as a tool to reduce these problems. Since chemical structures usually described in images, information extraction needs to combine structure image recognition and text mining together. Results We have developed ChemEx, a chemical information extraction system. ChemEx processes both text and images in publications. Text annotator is able to extract compound, organism, and assay entities from text content while structure image recognition enables translation of chemical raster images to machine readable format. A user can view annotated text along with summarized information of compounds, organism that produces those compounds, and assay tests. Conclusions ChemEx facilitates and speeds up chemical data curation by extracting compounds, organisms, and assays from a large collection of publications. The software and corpus can be downloaded from http://www.biotec.or.th/isl/ChemEx.


Background
Accurate chemical data curation is essential for cheminformatics. Nowadays, researchers or exploration software can access internal or external public databases [1,2] to retrieve necessary information. Still, the major source of knowledge is scientific literature. Unfortunately, information in the literature is unstructured or semi-structured, and written in natural language. Chemical structures were embedded in reports, journals, and patents in the form of images. These cannot be input into chemical databases or chemistry software directly. Manual reproducing the information is time-consuming and liable to errors. Furthermore, rapid growth of publications results in difficulty to maintain up-to-date data sets. To overcome these problems, automatic information extraction becomes a subject of interest.
Whereas there are numerous text-mining tools in biological domain [3][4][5][6], chemical information extraction had not received attention until recently. Existing techniques for the chemical information extraction can be broadly classified into two categories: visual and textual data extraction. The visual data extraction system, such as Kekulé [7], CLiDE [8,9], chemOCR [10], OSRA [11], and ChemReader [12], focuses on interpretation of images embedded in documents while the textual data extraction focuses on mining interested entities and their relations from text. Textual data extraction is varied based on a subject domain, such as chemical names [13,14], chemical formulae [14], or drug names [15]. Information extraction from either image or text content results in missing information or semantic links between text and images. Therefore, a technique for combining two media [16] could be applied to improve knowledge discovery.
ChemEx is a software developed to assists a chemical data curation process. While it can be used with general chemical information extraction, ChemEx is designed for extracting information of natural products which are a major source of novel bioactive compounds or structures [17]. It provides a framework to integrate optical structure recognition and chemical text-mining software. The extracted information can be then visualized and exported to a database. Enormous chemical libraries become available with minimum time and effort.

System overview
ChemEx processes a collection of publications in order to extract information of bioactive compounds as well as an organism that produces those compounds with their bioactivity from each publication as illustrated in Figure 1. The system consists of four main modules: (a) Document Preprocessor, (b) 2D Chemical Structure Image Recognition, (c) Text Annotator, and (d) Information Viewer. Figure 2 presents the workflow of the system. First, the Document Preprocessor transforms and segments each input literature into textual and visual data. The 2D Chemical Structure Image Recognition module then translates the visual data (images) into machine readable string whereas the Text Annotator module tags words in a subject domain. In the end, a user can visualize extracted information using the Information Viewer.

Document preprocessor module
This module pre-processes publications so that they can be input to 2D structure image recognition and text annotator modules. ChemEx works with both electronically-generated PDFs and scanned PDFs. Poppler [18] is used to segment a PDF file into a set of images and plain text. Converting full text PDF had layout errors, which are, the header and footer were mixed-up with the content, and a paragraph was sometimes broken to multiple discontinuous paragraphs. Hence, if a bibliography file is available, text content will be extracted from the abstract field in BibTeX instead. In case of the scanned PDF, which text cannot be extracted from the PDF file, BibTeX is required for the system to work properly.

2D chemical structure image recognition module
Structure images that are embedded in publications typically consist of two parts: 2D structure of chemical molecule and label of an identifier used for referencing later in the text content. The overview of this module is illustrated in Figure 3. This module consists of three following steps: (1) Structure Recognition which translates each 2D image of the chemical structure into machine readable format, (2) Label Recognition which identifies labels in a structure image, and (3) Structure-Label Mapper which constructs a mapping table between the label and file location of corresponding 2D structure. Step 1: structure recognition To retrieve machine readable structure from 2D chemical structure images, ChemEx uses an open source OSRA [11]. In this step, ChemEx recovers both SMILES [19] and MDL Molfile [20] from a 2D chemical structure image. Based on OSRA features, ChemEx recognizes atomic labels and charges, circle bond (old style aromatic rings), double and triple bonds, wedge and dash bonds, and bridge bonds.
Step 3: structure-label mapper One structure image may consist of multiple labels. Also, a label may contain the identification number used for reference the structure as well as others, for instance, a  compound name or R-group. To construct a structurelabel mapping table, ChemEx's Structure-Label Mapper assigns each 2D structure (from step 1) to a nearest label (from step 2) using minimum weight graph matching algorithm [16]. Successful mapping is written into a file to be used in Information Viewer.

Text annotator module
This module discovers interested entities and relations from textual information of publications. ChemEx employs a component called Analysis Engine (AE) from Unstructured Information Management Applications (UIMA) [22] to analyse document in four steps: (1) Tokenizer, (2) Tagger, (3) Phase Parser and Identification, and (4) Coordination Resolution. The processing flow among these steps is illustrated in Figure 4.
Step 1: tokenizer Tokenizer splits a text stream into tokens of words. ChemEx uses the tokenizer from OSCAR4 [23] which is able to handle hyphens or other symbols in chemical terms such as 2-Amino-2-(hydroxymethyl)-1,3-propanediol hydrochloride. ChemEx also extends OSCAR's tokenizer to handle scientific name abbreviation, such as Penicillium sp. or P. pacificum.
Step 2: tagger Tagger labels the interested word tokens in text. ChemEx tagger consists of Chemical Entities Tagger, Organism Entities Tagger, and Assay Entities Tagger.
ChemEx employs ChemicalTagger [24], which uses machine learning approach called Maximum Entropy Markov Model Recogniser [25], to (i) recognize chemical names, reaction names, enzymes, and chemistry-related terms such as experimental action verbs or units and (ii) tag general English word classes, such as a noun or a verb, which will be used in the phase parser. ChemEx uses all information from ChemicalTagger.
Organism and assay entities are tagged using dictionary-based approach. ChemEx extends ConceptMapper [26] which is a configurable dictionary UIMA-based annotator. The ConceptMapper allows a user to add or remove dictionaries according to domain of interest. The extended ConceptMapper keeps an identification number and database source so it is possible to retrieve further information of the entities. In case of organism, once scientific names, e.g., Escherichia coli, are detected, the tagger abbreviates all scientific names and searches the text again for abbreviated scientific names such as E. coli. Also, the organism tagger extends a term to cover "sp.", "spec.", or "spp." for unspecified species.
Step 3: phase parser and identification ChemicalTagger [24] also parses and identifies a sentence. Phase parser receives tagged token stream and builds grammatical structure based on predefined grammars. After text is parsed, experimental action phases, such as "Compound 1 was added to the solution" or "Compound 1 was extracted from compound 2", can be identified by analysing the grammatical structure. Numbers used for compound referencing (labels) are also identified in this step. Finally, ChemEx extracts natural products and their source organism from a ChemicalTagger's "yielded" phase, such as "Compound 1 was isolated from the fungus Xylaria multiplex".
Step 4: coordination resolution Sometimes, especially in an abstract, multiple compounds appear in one sentence. The compounds are joined with punctuation marks or coordinate conjunctions. Exploring semantic meaning of a noun group of compounds thus improves knowledge discovery. Coordination resolution is to identify each compound in a compound chunk mentioned in text content. For example, "multiplolides A (1) and B (2)" consists of two compounds: multiplolide A, labeled as 1, and multiplolide B, labeled as 2. "drechslerines C-G (6-10)" consists of five compounds: drechslerine C (6), drechslerine D (7), drechslerine E (8), drechslerine F (9), and drechslerine G (10).
ChemEx uses a state machine ( Figure 5) to recognize and interpret a compound group taking into account a label and series. The state machine processes on tagged token stream. Text state disregards non-chemical entity tokens. Chemical Name state accumulates a chemical name, either single or multiple words. Series and Label states are responsible for series and label token respectively. They also insert values in between two letters or numbers. For instance, "A-C" becomes "A, B, C", and "1-3" becomes "1, 2, 3". And/To state handles "and" and "to" token. For instance, "compounds A and B" becomes "compound A, compound B", and "compounds A to C" becomes "compound A, compound B, compound C". In the end, individual chemical names with series and label are generated as chemical entities.

Information viewer module
Information viewer provides graphical interface to user for viewing the integrated results from all modules. ChemEx summarizes natural products and their bioassay tests reported in a publication. The viewer includes UIMA CAS Annotation Viewer [22] to display annotated text and JChemPaint [31] to reproduce structure thumbnails from MOL files generated by 2D Chemical Structure Image Recognition module. Additionally, structure-label mapping tables generated by 2D Chemical Structure Image Recognition module is combined with chemical compound entities extracted from Text Annotator module as illustrated in Figure 6. Therefore, a chemical compound entity can be viewed and searched with its 2D chemical structure image and SMILES. A user can use the viewer to visualize results and export those results to an XML file which can be imported to sMOL Explorer [32], a web-enabled database and exploration tool for Small MOLecules datasets.

Results and discussion
ChemEx is able to extract compound, organism, and assay entities from text content automatically. It also finds 2D chemical structure of each compound from images embedded in full text, and converts 2D chemical structure images to machine readable format. Results from ChemEx can be visualized through the information viewer as demonstrated in Figure 7. A user can view annotated text together with publication information, compound list, organism that produces those compounds, and assay tests. Each compound can be also searched for additional information from external databases [2,30] as well as edited by 2D chemical structure editor (Figure 8). Moreover, a user can view and export extracted information of all publications in a collection in one place (Figure 9).
The system was tested using literatures from ACS Publications (accessed on 13th March 2012) [33]. The keywords   used for literature retrieval were "fungus Thailand". All accessible research articles with the abstract and full text PDF were downloaded. In total, 89 publications were obtained, but the test set contained only 74 publications that reports compounds with 2D chemical structures.
Each full text was retrieved with its bibliography. All images, including but not limited to 2D chemical structure images, were extracted from each PDF. Accuracy of information extraction and 2D chemical structure image recognition were evaluated.

Information extraction evaluation
Extracted information from text content, consisting of compounds, organisms, and assays were listed by the system and compared with manually listed entities. The results are shown in Table 1. Note that entities were evaluated regardless of natural products or not.
An exact match was an extracted entity matching the whole term of a manually listed entity, whereas a partial match was an extracted entity matching some part of a manually listed entity. False positive (FP) was an unexpected result. False negative (FN) was a missing result. The exact match was defined as true positive (TP). By default, the partial match was classified as false negative. Precision and recall were defined as:

Compound entities
The main purpose of the experiment was to extract compound entities from a content of abstracts and discover their 2D depiction from images embedded in full text. Thus, only compounds that have 2D structure images were considered. Partial matches were considered as mismatches.
The system extracted compound entities with 83.20% precision and 62.85% recall. ChemicalTagger achieved 61.34% precision and 22.60% recall. As demonstrated in Table 2, ChemEx increases precision and recall 21.85% and 40.25% respectively.
The main improvement resulted from ChemEx's Coordination Resolution, which, for example, recognized five instead of one compounds from "drechslerines C-G". The improvement could be smaller in case of full text where each compound is written separately.
Currently, Coordination Resolution recognizes a label that contains pure digits (i.e. 0, 1, 2). Future development could extend coordination resolution to recognize other types of label such as Roman digits or letters.

Organism entities
Organism recognition showed good performance with 96.81% precision and 77.78% recall. False negative were scientific names outside the domain of interest. Partial matches were scientific names that only a genus was detected without species. If partial matches were considered as true positive, the performance is up to 97.39% precision and 95.73% recall.
While dictionary-based text mining yields high precision, its recall may be low depending on dictionary size.

Assay entities
Assay recognition achieved 100.00% of precision and 84.21% of recall. False negative was due to assay terms does not exist in the corpus.
Currently, ChemEx recognized only one-word assay, such as, "antifungal" or "cytotoxic". However, some assays were reported in a sentence, for example, "Compound 1 inhibited activity against the malarial parasite Plasmodium falciparum". Future development could apply phase parsing to recognize these assay phases.

2D chemical structure image recognition evaluation
Compounds entities in publications were manually listed and used to search for corresponding chemical structure in PubChem [2]. The search was done automatically via web service and the most similar of each chemical structure was used in the evaluation. In total 204 structures were found and downloaded as the ground truth. Then CACTVS script [11,34] evaluated structure similarity between ground truth and regenerated structures based on standard InChI [35].
ChemEx was able to map 144 structures (70.59%) to compound entities extracted from text content. Mapping error comes from imperfect image segmentation, OCR errors, and incomprehensive pattern in label recognition. Table 3 shows number of structures according to similarity score. "T > 70%" indicates the number of structure with similarity above 70%. There were 72 structures (35.29%) with the similarity score is above 70%. The average similarity of these 72 structures was 91.42%. ChemEx reconstructed 28 identical structures (13.73%). The average similarity between ground truth and regenerated structures was 71.86%.
Our experiment found that sometimes OSRA [11] recognized a graph as chemical structure. Image classification prior to 2D Chemical Structure Image Recognition could improve accuracy and performance. Another major issue is that OSRA interests only structure images. Retrieving non-structure image components from OSRA may result in high segmentation error, which causes error in structure-label mapping. Future development could apply segments categorization [16] before using OSRA to cover this issue.

Conclusions
ChemEx automatically discovers chemical knowledge from a large collection of publications. It is built on top of multiple pieces of software [11,22,24] allowing information extraction from both visual and textual content. The system extracts compound, organism, and assay information with flexible framework. A user can add new dictionaries to customize results according to the domain of interest. ChemEx information viewer integrates and visualizes results. To the best of our knowledge, ChemEx is the first system that provides these functionalities.  The test set consisted of 89 publications with terms "fungus Thailand" from ACS Publications. Only 74 publications reported compounds with 2D chemical structures. Compounds, organisms, and assays were extracted from text content and compared with manually listed entities.
Although the accuracy needs to be improved, ChemEx increases information understanding and assists a user on chemical data curation process. We believe it is one step towards fully automatic chemical data curation, which is useful for constructing large chemical structure libraries.

Availability and requirements
• Project name: ChemEx -Chemical Information Extraction.
• Operating system(s): Windows and Linux.
• Other requirements: at least 2 GB of RAM. Other dependencies were listed in the home page.