{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "# Application ontology for metabolomics\n", "\n", "````{panels}\n", ":container: container-lg pb-3
\n", ":column: col-lg-3 col-md-4 col-sm-6 col-xs-12 p-1
\n", ":card: rounded
\n", "\n", "
\n", "^^^\n", "

Recipe metadata

\n", " identifier: RX.X\n", " version: v1.0\n", "\n", "---\n", "\n", "^^^\n", "

Difficulty level

\n", "\n", "\n", "\n", "\n", "\n", "\n", "---\n", "\n", "^^^\n", "

Reading Time

\n", " 30 minutes\n", "

Recipe Type

\n", " Hands-on\n", "

Executable Code

\n", " Yes\n", "\n", "---\n", "\n", "^^^\n", "

Intended Audience

\n", "\n", "

Data Manager

\n", "

Data Scientist

\n", "````\n", "---\n", "\n", "**title:** `MSIO build process`
\n", "**author:** `philippe.rocca-serra@oerc.ox.ac.uk`
\n", "**date:** `2020-05-11`
\n", "**license:** [`creative-commons-by-sa-4.0`](https://creativecommons.org/licenses/by/4.0/)
\n", "**keywords:** `ontology`
\n", "**description:** `MSIO an Application Ontology for Metabolomics. We detail how we use OBO Foundry ROBOT tool for generating the ontology from a set of seeds, which have been established by manual selection of the key classes to build a mild level ontology for experiments. MSIO specific classes are added the MSIO namespace but remain compatible with OBI for which MSIO can become a module.`
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Obtaining and installing robot\n", "\n", "The robot tool is available from: http://robot.obolibrary.org/ with detailed instructions available from the official pages.\n", "One may obtain the [latest official version](https://github.com/ontodev/robot/releases) or the [latest successful build](https://build.obolibrary.io/job/ontodev/job/robot/job/master/lastSuccessfulBuild/artifact/bin/robot.jar) from the obofoundry project. This the latter option we go for in this notebook as we want to test a new function available from version 1.7, onwards.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "robot_dir=\"robot_jar\"\n", "if not os.path.exists(robot_dir):\n", " os.makedirs(robot_dir)\n", " \n", "# Check if a newer versio exists a:\n", "\n", "get_robot_curl=\"curl -L https://build.obolibrary.io/job/ontodev/job/robot/job/master/lastSuccessfulBuild/artifact/bin/robot.jar > ./robot_jar/robot.jar\"\n", "try:\n", " os.system(get_robot_curl)\n", "except IOError as ioe_curl:\n", " print(ioe_curl)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "resources=[\n", "\"curl -L http://purl.obolibrary.org/obo/duo.owl > ./ontology-sources-test/duo.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/stato.owl > ./ontology-sources-test/stato.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/chmo.owl > ./ontology-sources-test/chmo.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/iao.owl > ./ontology-sources-test/iao.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/obi.owl > ./ontology-sources-test/obi.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/ms.owl > ./ontology-sources-test/ms.owl\",\n", "\"curl -L https://raw.githubusercontent.com/MSI-Metabolomics-Standards-Initiative/nmrcv/master/nmrCV-corrected-namespace.owl > ./ontology-sources-test/nmrcv.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/chebi.owl > ./ontology-sources-test/chebi.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/uo.owl > ./ontology-sources-test/uo.owl\",\n", "\"curl -L http://purl.obolibrary.org/obo/stato.owl > ./ontology-sources-test/stato.owl\" \n", "]" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/Users/philippe/Documents/git/FAIRplus-org/the-fair-cookbook/docs/content/recipes/ontology-robot\n" ] } ], "source": [ "cwd = os.getcwd()\n", "print(cwd)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Downloading Reference Ontologies & Key Resources:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "ontoresource_dir=\"ontology-sources-test\"\n", "if not os.path.exists(ontoresource_dir):\n", " os.makedirs(ontoresource_dir)\n", "else:\n", " for resource in resources:\n", " try:\n", " print(resource)\n", " os.system(resource)\n", " except IOError as ieo_curl:\n", " print(ioe_curl)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Module Extraction from seeds using the MIREOT approach:\n", "Using a number of seed files, which contains the classes manually selected by the ontology developers, we use the [MIREOT method](http://dx.doi.org/10.3233/AO-2011-0087)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "onto_outputdir=\"ontology-msio-build-modules\"\n", "\n", "extractors=[\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/chebi.owl --upper-terms ./ontology-select-classes/chebi-upper-terms.txt --lower-terms ./ontology-select-classes/chebi-lower-terms.txt --intermediates none --imports exclude --copy-ontology-annotations true --output ./ontology-msio-build-modules/chebi_mireot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/chmo.owl --lower-terms ./ontology-select-classes/chmo-lower-terms.txt --intermediates all --copy-ontology-annotations true --output ./ontology-msio-build-modules/chmo_mireot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/iao.owl --lower-terms ./ontology-select-classes/iao-lower-terms.txt --intermediates all --copy-ontology-annotations true --output ./ontology-msio-build-modules/iao_mireot_robot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/obi.owl --lower-terms ./ontology-select-classes/obi-lower-terms.txt --intermediates all --copy-ontology-annotations true --output ./ontology-msio-build-modules/obi_mireot_robot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/ms.owl --lower-terms ./ontology-select-classes/psims-lower-terms.txt --intermediates minimal --copy-ontology-annotations true --output ./ontology-msio-build-modules/psims_mireot_robot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/uo.owl --lower-terms ./ontology-select-classes/uo-lower-terms.txt --intermediates minimal --copy-ontology-annotations true --output ./ontology-msio-build-modules/uo_mireot_robot_module.owl\",\n", "\"java -jar ./robot_jar/robot.jar extract --method MIREOT --input ./ontology-sources-test/duo.owl --lower-terms ./ontology-select-classes/duo-lower-terms.txt --intermediates all --copy-ontology-annotations true --output ./ontology-msio-build-modules/duo_mireot_robot_module.owl\"\n", "] \n", "\n", "if not os.path.exists(onto_outputdir):\n", " os.makedirs(onto_outputdir)\n", "else:\n", " for extractor in extractors:\n", " try:\n", " print(extractor)\n", " os.system(extractor)\n", " except IOError as ioe:\n", " print(ioe) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Merging Extracted Modules into Application Ontology:\n", "This step simply fuses all the input OWL files into one single output, avoiding use of import statements.
\n", ":octocat: A `merge` can be *undone*( with `unmerge` command also available from `robot tool`." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/Users/philippe/Documents/git/FAIRplus-org/the-fair-cookbook/docs/content/recipes/ontology-robot\n" ] } ], "source": [ "if not os.path.exists(\"ontology-msio-build-results\"):\n", " os.makedirs(\"ontology-msio-build-results\")\n", " try:\n", " print(os.getcwd())\n", " os.system(\"java -jar ./robot_jar/robot.jar merge --input ./ontology-msio-build-modules/iao_mireot_robot_module.owl --input ./ontology-msio-build-modules/obi_mireot_robot_module.owl --input ./ontology-msio-build-modules/duo_mireot_robot_module.owl --input ./ontology-msio-build-modules/chmo_mireot_module.owl --input ./ontology-msio-build-modules/uo_mireot_robot_module.owl --input ./ontology-msio-build-modules/psims_mireot_robot_module.owl --input ./ontology-msio-build-modules/chebi_mireot_module.owl --output ./ontology-msio-build-results/msio-test-merge.owl\")\n", " except IOError as ioe_merge:\n", " print(ioe_merge)\n", "else:\n", " try:\n", " print(os.getcwd())\n", " os.system(\"java -jar ./robot_jar/robot.jar merge --input ./ontology-msio-build-modules/iao_mireot_robot_module.owl --input ./ontology-msio-build-modules/obi_mireot_robot_module.owl --input ./ontology-msio-build-modules/duo_mireot_robot_module.owl --input ./ontology-msio-build-modules/chmo_mireot_module.owl --input ./ontology-msio-build-modules/uo_mireot_robot_module.owl --input ./ontology-msio-build-modules/psims_mireot_robot_module.owl --input ./ontology-msio-build-modules/chebi_mireot_module.owl --output ./ontology-msio-build-results/msio-test-merge.owl\")\n", " except IOError as ioe_merge:\n", " print(ioe_merge) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Working with ELK reasoner with Robot Materialize:\n", "With this step, the goal is to ensure that the ontology is logically consistent. This can only be checked by running a `reasoner`, which will formally assess the various axioms used in the ontology and classify the classes bases on these constraints. Robot allows this step to be performed and provides various options as to which reasoner to offer. Here, the [ELK reasoner](https://www.cs.ox.ac.uk/isg/tools/ELK/) is used." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "if not os.path.exists(\"ontology-msio-materialize\"):\n", " os.makedirs(\"ontology-msio-materialize\")\n", "else: \n", " try:\n", " os.system(\"java -jar ./robot_jar/robot.jar materialize --reasoner ELK --input ./ontology-msio-build-results/msio-test-merge.owl reduce --output ./ontology-msio-materialize/msio-test-materialize.owl\")\n", " except IOError as ioe_reason:\n", " print(ioe_reason) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Convert to OBO Format:\n", "The module extraction, merging and reasoning function produce Ontology Web Language ([OWL](https://www.w3.org/TR/owl-features/)) documents. For reaching a wider audience and to allow use by some data entry tools, it is interesting to release the ontology in the [OBO format](https://owlcollab.github.io/oboformat/doc/GO.format.obo-1_4.html). The following section details how to invoke the command." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "if not os.path.exists(\"ontology-msio-convert\"):\n", " os.makedirs(\"ontology-msio-convert\")\n", " try:\n", " os.system(\"java -jar ./robot_jar/robot.jar convert --input ./ontology-msio-materialize/msio-test-materialize.owl --format obo --output ./ontology-msio-convert/msio-test-conversion.obo\")\n", " except IOError as ioe_reason:\n", " print(ioe_reason) \n", "else: \n", " try:\n", " os.system(\"java -jar ./robot_jar/robot.jar convert --input ./ontology-msio-materialize/msio-test-materialize.owl --format obo --output ./ontology-msio-convert/msio-test-conversion.obo\")\n", " except IOError as ioe_reason:\n", " print(ioe_reason) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Importing stato and nmrCV ontology" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# if not os.path.exists(\"ontology-msio-merge\"):\n", "# os.makedirs(\"ontology-msio-merge\")\n", "# else: \n", "# try:\n", "# os.system(\"java -jar robot.jar merge --input ./ontology-msio-materialize/msio-test-materialize.owl --input ./ontology-sources-modules/stato.owl --input ./ontology-sources-modules/nmrcv.owl --output ./ontology-msio-merge/msio-test-merge-round2.owl\")\n", "# except IOError as ioe_convert:\n", "# print(ioe_convert) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. Export as Tabular Representation in Excel format\n", "A new function in Robot tool (starting with version 1.7) provides the option of exporting the classes and properties of an ontology in a tabular representation, which can be useful to engage with users and curators for review and extension. The following section details how to invoke the command." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "if not os.path.exists(\"ontology-msio-export\"):\n", " os.makedirs(\"ontology-msio-export\")\n", " try:\n", " os.system(\"java -jar ./robot_jar/robot.jar export --input ./ontology-msio-materialize/msio-test-materialize.owl --header 'IRI|ID|LABEL|SubClass Of|definition' --format xlsx --export ./ontology-msio-export/msio-test-export.xlsx\")\n", " except IOError as ioe_convert:\n", " print(ioe_convert)\n", "else: \n", " try:\n", " os.system(\"java -jar ./robot_jar/robot.jar export --input ./ontology-msio-materialize/msio-test-materialize.owl --header 'IRI|ID|LABEL|SubClass Of|definition' --format xlsx --export ./ontology-msio-export/msio-test-export.xlsx\")\n", " except IOError as ioe_convert:\n", " print(ioe_convert)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References:\n", "\n", "1. Jackson R.C., Balhoff J.P., Douglass E.,Harris N.L., Mungall C.J. and Overton J.A. ROBOT: A tool for automating ontology workflows. BMC Bioinformatics, vol. 20, July 2019.[https://doi.org/10.1186/s12859-019-3002-3](https://doi.org/10.1186/s12859-019-3002-3)\n", "2. Hastings J., Owen G., Dekker A., Ennis M., Kale N., Muthukrishnan V., Turner S., Swainston N., Mendes P., Steinbeck C.(2016). ChEBI in 2016: Improved services and an expanding collection of metabolites. Nucleic Acids Res.[http://dx.doi.org/10.1093/nar/gks1146](http://dx.doi.org/10.1093/nar/gks1146)\n", "10. Haug K., Cochrane K., Nainala V.C., Williams M., Chang J., Jayaseelan K.N. and O’Donovan C. MetaboLights: a resource evolving in response to the needs of its scientific community. Nucleic Acids Research, gkz1019, [https://doi.org/10.1093/nar/gkz1019](https://doi.org/10.1093/nar/gkz1019)\n", "3. IAO: [https://github.com/information-artifact-ontology/IAO/](https://github.com/information-artifact-ontology/IAO/)\n", "4. The Ontology for Biomedical Investigations, PLoS One. 2016 Apr 29;11(4):e0154556. eCollection 2016. [https://doi.org/10.1371/journal.pone.0154556](https://doi.org/10.1371/journal.pone.0154556). \n", "5. STATO: [https:stato-ontology.org](https:stato-ontology.org)\n", "6. nmrCV: [http://nmrml.org/cv/](http://nmrml.org/cv/)\n", "7. CHMO: [https://github.com/rsc-ontologies/rsc-cmo](https://github.com/rsc-ontologies/rsc-cmo)\n", "8. UO: [https://github.com/bio-ontology-research-group/unit-ontology](https://github.com/bio-ontology-research-group/unit-ontology)\n", "9. PSI-MS: [http://www.psidev.info/groups/controlled-vocabularies](http://www.psidev.info/groups/controlled-vocabularies)\n" ] } ], "metadata": { "kernelspec": { "display_name": "venv374", "language": "python", "name": "venv374" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }