{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating an InterMine workflow using the API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by importing the Service class from InterMine's webservice module.  You will need to access your account on humanMine and you do this through an API token. You can get your token by logging into [HumanMine](http://www.humanmine.org/) and going to the account details tab within MyMine.  Cut and paste your token into the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from intermine.webservice import Service\n",
    "service = Service(\"https://www.humanmine.org/humanmine/service\", token = \"YOUR TOKEN HERE\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First Query: Pax6 Targets\n",
    "\n",
    "Our first query looked at whether the set of Pax6 targets (from list PL_Pax6_Targets) are expressed in the pancreas.  In the web interface we used a template to run this query, but here we will create a query object. We want to create a query against the `Gene` class in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new query against the root class \"Gene\"\n",
    "# Syntax: query = service.new_query(\"root_class_here\")\n",
    "\n",
    "query = "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we will define the output columns that we want in our result - i.e the view. We want to add fields (attributes) from both the `Gene` class and its child, the `proteinAtlasExpression` class. \n",
    "\n",
    "Note that we have started our query from the `Gene` class in the previous step. `symbol` and `primaryIdentifier` are attributes of this class. The `proteinAtlasExpression` class is referenced from the `Gene` class, so to return the protein atlas information we give the path from the `Gene` class - i.e `proteinAtlasExpression.level` etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now select the following views: \n",
    "#\n",
    "#    \"primaryIdentifier\", \"symbol\", \"proteinAtlasExpression.cellType\",\n",
    "#    \"proteinAtlasExpression.level\", \"proteinAtlasExpression.reliability\",\n",
    "#    \"proteinAtlasExpression.tissue.name\"\n",
    "#\n",
    "# The syntax to do so is query = query.add_view(\"comma\",\"separated\", \"set\", \"of\", \"views\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, add the constraints to your query. We want to constrain the Gene class to the genes in the PL_Pax6_Targets list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Syntax: query.add_constraint(\"view_name\", \"operator\", \"value\", code = \"SINGLE_CAPITAL_LETTER_HERE\")\n",
    "#\n",
    "# Let's set \"Gene\" to be \"IN\" the list named \"PL_Pax6_Targets\". \n",
    "# \n",
    "# note the final argument is the \"code\" - this is simply a unique identifier for\n",
    "# each constraint that allows tou to revisit them later if needed. Usually it's easiest \n",
    "# just to sequentially assign an alphabet letter - so in this case, set it to A next time to B, etc.\n",
    "\n",
    "# e.g. code = \"A\".\n",
    "\n",
    "# Here's the first one written for you as an example: \n",
    "\n",
    "query.add_constraint(\"Gene\", \"IN\", \"PL_Pax6_Targets\", code = \"A\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need to constrain the expression level to be \"high\" or \"medium\" and the tissue to be \"pancreas\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Syntax reminder: query.add_constraint(\"view_name\", \"operator\", \"value\")\n",
    "#\n",
    "# Let's add two constraints: \n",
    "# - Set \"proteinAtlasExpression.tissue.name\" to be equal to \"Pancreas\", with code \"B\"\n",
    "# - Set \"proteinAtlasExpression.level\" to be \"ONE OF\" the options \"Medium\" or \"High\", with code \"C\" \n",
    "#   (Note: when you use the ONE OF operator, each option needs to be part of an array,\n",
    "#          i.e. in this case we would write [\"Medium\", \"High\"] as the value.)\n",
    "#\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's check what the query returns by looping through the rows and printing the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We've filled this one out for you :) \n",
    "\n",
    "for row in query.rows():\n",
    "    print(row[\"primaryIdentifier\"], row[\"symbol\"], row[\"proteinAtlasExpression.cellType\"], \\\n",
    "        row[\"proteinAtlasExpression.level\"], row[\"proteinAtlasExpression.reliability\"], \\\n",
    "        row[\"proteinAtlasExpression.tissue.name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to save this set of genes (i.e genes from the Pax6 target set that are expressed in the pancreas) for further analysis. To do this we define our python list and loop through our results again - this time, instead of printing the results, we append just the primary identifiers returned to our list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's make an empty python list called \n",
    "UpinPancreas = list()\n",
    "\n",
    "# now let's use a for loop on query results and select just \n",
    "# the gene primary identifiers \n",
    "# then append them to our UpinPancreas list. \n",
    "#\n",
    "# To append an item to a python list, it's ListName.append(someValueHere)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and check that the list we have created looks correct:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(UpinPancreas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now need to save the list to our InterMine account so we can use it again in a later query. The `ListManager` python class provides methods to manage list contents and operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# first let's make a new list manager assigned to the variable lm\n",
    "# the syntax to make a list manager is service.list_manager()\n",
    " \n",
    "\n",
    "    \n",
    "# next, we want to put the contents of UpinAdipose into an InterMine list.\n",
    "# The syntax is lm.create_list(content=a_list_of_ids, list_type=\"identifier_class\", name=\"some name\")\n",
    "# In this case, you'll want to set the following arguments:\n",
    "# - content should be our python list UpinPancreas\n",
    "# - list_type is \"Gene\"\n",
    "# - name - could be anything you want, but let's be consistent and call it \"UpinPancreas\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Log in to HumanMine](http://www.humanmine.org/) and check your list has been created."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Second query: Diabetes genes\n",
    "\n",
    "Our second query (which we created using the query builder) found genes that are associated with the disease diabetes.  Re-create this query using code as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query2 = service.new_query(\"Gene\")\n",
    "\n",
    "# Let's add views for \"primaryIdentifier\" and \"symbol\" using query.add_view()\n",
    "\n",
    "\n",
    "# And let's give it some constraints using \n",
    "# query.add_constraint(\"view_name\", \"operator\", \"value\", code = \"constraint_code\")\n",
    "# \n",
    "# Constraint A: organism.name should equal Homo sapiens\n",
    "# Constraint B: diseases.name should contain diabetes (operator is CONTAINS) \n",
    "\n",
    "\n",
    "# We've written the code to print it out for you. \n",
    "for row in query2.rows():\n",
    "    print (row[\"primaryIdentifier\"], row[\"symbol\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and save the set of genes returned as a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a python list of gene identifiers\n",
    "diabetesGenes = list()\n",
    "for row in query2.rows():\n",
    "    diabetesGenes.append(row[\"primaryIdentifier\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# One last time, we'll create a list and save it to our HumanMine account\n",
    "#\n",
    "# syntax: lm.create_list(content=a_list_of_ids, list_type=\"identifier_class\", name=\"some name\")\n",
    "# \n",
    "# - content should be diabetesGenes\n",
    "# - list_type is \"Gene\"\n",
    "# - name  \"diabetesGenes\"\n",
    "# Try it now: \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we used a list intersect to find those genes that are upregulated in the pancreas that are also associated with the disease diabetes. We need to intersect the first (UpinPancreas) and second (diabetesGenes) lists that we created. We can do this using the intersect method from the ListManager class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The syntax to create an InterMine list intersection is\n",
    "# lm.intersect([\"comma_separated\", \"list\", \"of_intermine_lists\"], \"name for new list\")\n",
    "#\n",
    "# We want to intersect the last two lists we created - \n",
    "# \"UpinPancreas\" and \"diabetesGenes\"\n",
    "# try it now: \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last list intersection was stored in our HumanMine account, so we need to use the method `get_list` to retrieve it from HumanMine "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Syntax: lm.get_list(\"name of the intersected list you just created\")\n",
    "# Store it in a variable called intersectedList, so we can print it in the next step\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(intersectedList)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Query: GWAS \n",
    "\n",
    "Finally, we fed the intersected list from above back into another query to see if there was any association of these genes with diabetes phenotypes according to GWAS studies. Note that we now start our query from the GWAS class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We've supplied some code for you to get you started: \n",
    "query = service.new_query(\"GWAS\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding the columns we'd like to view: \n",
    "query.add_view(\n",
    "    \"results.associatedGenes.primaryIdentifier\",\n",
    "    \"results.associatedGenes.symbol\", \"results.associatedGenes.name\",\n",
    "    \"results.SNP.primaryIdentifier\", \"results.pValue\", \"results.phenotype\",\n",
    "    \"firstAuthor\", \"name\", \"publication.pubMedId\",\n",
    "    \"results.associatedGenes.organism.shortName\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We only want results with a significant pValue and the phenotype diabetes, and we also want all of these results to be part of the intersectedList we just created. So, a few more constraints:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Syntax reminder: query.add_constraint(\"view_name\", \"operator\", \"value\", code=\"constraint_code\")\n",
    "# \n",
    "# Let's add three constraints: \n",
    "# - Set \"results.pValue\" to be less than or equal to \"1e-04\" (the operator you'll want is \"<=\")\n",
    "# - Set \"results.phenotype\" to contain \"diabetes\" (the contain operator you're looking for here is \"CONTAINS\")\n",
    "# - Set \"results.associatedGenes\" to be IN the list \"intersectedList\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Right, let's take a look at the results of our query: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in query.rows():\n",
    "    print(row[\"results.associatedGenes.primaryIdentifier\"], row[\"results.associatedGenes.symbol\"], \\\n",
    "        row[\"results.associatedGenes.name\"], row[\"results.SNP.primaryIdentifier\"], \\\n",
    "        row[\"results.pValue\"], row[\"results.phenotype\"], row[\"firstAuthor\"], row[\"name\"], \\\n",
    "        row[\"publication.pubMedId\"], row[\"results.associatedGenes.organism.shortName\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in query.rows():\n",
    "    print(row[\"results.associatedGenes.symbol\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "That's it - all done! "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}