{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Abstract\n",
"\n",
"**Author:** [Charles Tapley Hoyt](https://github.com/cthoyt)\n",
"\n",
"**Estimated Run Time:** 4 minutes\n",
"\n",
"This notebook calculates the concordance between differential gene expression and the causal relationships between their protein products. This analysis assumes the forward hypothesis, under which gene expression is believed to be correlated with protein activity. While there are many faults to this assumption, it directly enables a very simple analysis of a knowledge assembly. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Notebook Imports"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import logging\n",
"import os\n",
"import time\n",
"\n",
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from pandas.tools.plotting import scatter_matrix\n",
"from matplotlib_venn import venn2\n",
"import seaborn as sns\n",
"\n",
"import pybel\n",
"import pybel_tools as pbt\n",
"from pybel_tools.analysis.concordance import *\n",
"from pybel.constants import *\n",
"from pybel.canonicalize import calculate_canonical_name\n",
"from pybel_tools.visualization import to_jupyter"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%config InlineBackend.figure_format = 'svg'\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Notebook Provenance\n",
"\n",
"The time of execution, random number generator seed, and the versions of the software packages used are displayed explicitly."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Sun Aug 27 11:39:19 2017'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"time.asctime()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# seed the random number generator\n",
"import random\n",
"random.seed(127)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'0.7.3-dev'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pybel.__version__"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'0.2.2-dev'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pbt.__version__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Local Path Definitions\n",
"\n",
"To make this notebook interoperable across many machines, locations to the repositories that contain the data used in this notebook are referenced from the environment, set in `~/.bashrc` to point to the place where the repositories have been cloned. Assuming the repositories have been `git clone`'d into the `~/dev` folder, the entries in `~/.bashrc` should look like:\n",
"\n",
"```bash\n",
"...\n",
"export BMS_BASE=~/dev/bms\n",
"...\n",
"```\n",
"\n",
"#### BMS \n",
"\n",
"The biological model store (BMS) is the internal Fraunhofer SCAI repository for keeping BEL models under version control. It can be downloaded from https://tor-2.scai.fraunhofer.de/gf/project/bms/"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"bms_base = os.environ['BMS_BASE']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### OwnCloud\n",
"\n",
"The differential gene expression data used in this notebook is currently not published, and is obfuscated with reference through our team's internal data storage system with [OwnCloud](https://owncloud.org/)."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"owncloud_base = os.environ['OWNCLOUD_BASE']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Data\n",
"\n",
"## Alzheimer's Disease Knowledge Assembly\n",
"\n",
"The Alzheimer's Disease Knowledge Assembly has been precompiled with the following command line script, and will be loaded from this format for improved performance. In general, derived data, such as the gpickle representation of a BEL script, are not saved under version control to ensure that the most up-to-date data is always used.\n",
"\n",
"```sh\n",
"pybel convert --path \"$BMS_BASE/aetionomy/alzheimers.bel\" --pickle \"$BMS_BASE/aetionomy/alzheimers.gpickle\"\n",
"```\n",
"\n",
"The BEL script can also be compiled from inside this notebook with the following python code:\n",
"\n",
"```python\n",
">>> import os\n",
">>> import pybel\n",
">>> # Input from BEL script\n",
">>> bel_path = os.path.join(bms_base, 'aetionomy', 'alzheimers.bel')\n",
">>> graph = pybel.from_path(bel_path)\n",
">>> # Output to gpickle for fast loading later\n",
">>> pickle_path = os.path.join(bms_base, 'aetionomy', 'alzheimers.gpickle')\n",
">>> pybel.to_pickle(graph, pickle_path)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"pickle_path = os.path.join(bms_base, 'aetionomy', 'alzheimers', 'alzheimers.gpickle')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"graph = pybel.from_pickle(pickle_path)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'4.0.3'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"graph.version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All orthologies are discarded before analysis."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"pbt.filters.remove_nodes_by_namespace(graph, {'MGI', 'RGD'})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Following the forward hypothesis, the knowledge graph is collapsed to genes using [pbt.mutation.collapse_by_central_dogma_to_genes](http://pybel-tools.readthedocs.io/en/latest/mutation.html#pybel_tools.mutation.collapse_by_central_dogma_to_genes)."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"pbt.mutation.collapse_by_central_dogma_to_genes(graph)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"An additional assumption is made about the activities of variants of proteins and genes. All variants are collapsed to the reference gene."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"pbt.mutation.rewire_variants_to_genes(graph)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Nodes: 3701\n",
"Edges: 19562\n",
"Citations: 1760\n",
"Authors: 9305\n",
"Network density: 0.0014285401315933604\n",
"Components: 70\n",
"Average degree: 5.2855984868954335\n",
"Compilation warnings: 2578\n"
]
}
],
"source": [
"pbt.summary.print_summary(graph)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"pbt.summary.plot_summary(graph, plt, figsize=(10, 4))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Differential Gene Expression\n",
"\n",
"Differential gene expression data can be obtained from many sources, including ADNI and other large clinical studies. This analysis is concerned with the log-fold-changes on each gene, and not necessarily the p-value. This is better as a data-driven process becuase it does not require a model or multiple hypothesis testing on raw data."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"data_path = os.path.join(owncloud_base, 'alzheimers', 'SevAD.csv')\n",
"target_columns = ['Gene.symbol', 'logFC']"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Gene.symbol | \n",
" logFC | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" ZNF616 | \n",
" -4.244691 | \n",
"
\n",
" \n",
" | 1 | \n",
" DEFB125 | \n",
" 3.974393 | \n",
"
\n",
" \n",
" | 3 | \n",
" SNAP23 | \n",
" 3.337636 | \n",
"
\n",
" \n",
" | 4 | \n",
" PHLDB2 | \n",
" 3.192559 | \n",
"
\n",
" \n",
" | 5 | \n",
" LOC389895 | \n",
" -4.296850 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Correct | \n",
" Incorrect | \n",
" Ambiguous | \n",
" Unassigned | \n",
"
\n",
" \n",
" \n",
" \n",
" | Interleukin signaling | \n",
" 20 | \n",
" 41 | \n",
" 25 | \n",
" 285 | \n",
"
\n",
" \n",
" | Tumor necrosis factor | \n",
" 20 | \n",
" 29 | \n",
" 16 | \n",
" 214 | \n",
"
\n",
" \n",
" | miRNA | \n",
" 10 | \n",
" 16 | \n",
" 80 | \n",
" 167 | \n",
"
\n",
" \n",
" | Insulin signal transduction | \n",
" 10 | \n",
" 19 | \n",
" 17 | \n",
" 761 | \n",
"
\n",
" \n",
" | Nuclear factor Kappa beta | \n",
" 8 | \n",
" 29 | \n",
" 19 | \n",
" 60 | \n",
"
\n",
" \n",
" | Inflammatory response | \n",
" 8 | \n",
" 17 | \n",
" 23 | \n",
" 488 | \n",
"
\n",
" \n",
" | Chemokine signaling | \n",
" 6 | \n",
" 12 | \n",
" 4 | \n",
" 168 | \n",
"
\n",
" \n",
" | Neurotrophic | \n",
" 6 | \n",
" 7 | \n",
" 1 | \n",
" 56 | \n",
"
\n",
" \n",
" | Nerve growth factor | \n",
" 6 | \n",
" 10 | \n",
" 5 | \n",
" 171 | \n",
"
\n",
" \n",
" | Caspase | \n",
" 5 | \n",
" 18 | \n",
" 15 | \n",
" 156 | \n",
"
\n",
" \n",
" | DKK1 | \n",
" 5 | \n",
" 5 | \n",
" 8 | \n",
" 77 | \n",
"
\n",
" \n",
" | Albumin | \n",
" 3 | \n",
" 1 | \n",
" 0 | \n",
" 26 | \n",
"
\n",
" \n",
" | Acetylcholine signaling | \n",
" 3 | \n",
" 3 | \n",
" 2 | \n",
" 361 | \n",
"
\n",
" \n",
" | Bcl-2 | \n",
" 3 | \n",
" 11 | \n",
" 5 | \n",
" 77 | \n",
"
\n",
" \n",
" | Complement system | \n",
" 3 | \n",
" 3 | \n",
" 3 | \n",
" 97 | \n",
"
\n",
" \n",
" | Amyloidogenic | \n",
" 3 | \n",
" 27 | \n",
" 62 | \n",
" 1324 | \n",
"
\n",
" \n",
" | Endosomal lysosomal | \n",
" 2 | \n",
" 16 | \n",
" 8 | \n",
" 281 | \n",
"
\n",
" \n",
" | Chaperone | \n",
" 2 | \n",
" 9 | \n",
" 6 | \n",
" 63 | \n",
"
\n",
" \n",
" | Notch signaling | \n",
" 2 | \n",
" 2 | \n",
" 8 | \n",
" 117 | \n",
"
\n",
" \n",
" | Metabolism of steroid hormones | \n",
" 2 | \n",
" 0 | \n",
" 0 | \n",
" 6 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Correct | \n",
" Incorrect | \n",
" Ambiguous | \n",
" Unassigned | \n",
" Concordance | \n",
"
\n",
" \n",
" \n",
" \n",
" | Plasminogen activator | \n",
" 1 | \n",
" 0 | \n",
" 2 | \n",
" 97 | \n",
" 1.000000 | \n",
"
\n",
" \n",
" | Metabolism of steroid hormones | \n",
" 2 | \n",
" 0 | \n",
" 0 | \n",
" 6 | \n",
" 1.000000 | \n",
"
\n",
" \n",
" | Cell cycle | \n",
" 1 | \n",
" 0 | \n",
" 7 | \n",
" 83 | \n",
" 1.000000 | \n",
"
\n",
" \n",
" | Vitamin | \n",
" 2 | \n",
" 0 | \n",
" 0 | \n",
" 41 | \n",
" 1.000000 | \n",
"
\n",
" \n",
" | Albumin | \n",
" 3 | \n",
" 1 | \n",
" 0 | \n",
" 26 | \n",
" 0.750000 | \n",
"
\n",
" \n",
" | Reactive oxygen species | \n",
" 2 | \n",
" 1 | \n",
" 4 | \n",
" 184 | \n",
" 0.666667 | \n",
"
\n",
" \n",
" | Interferon signaling | \n",
" 2 | \n",
" 1 | \n",
" 2 | \n",
" 75 | \n",
" 0.666667 | \n",
"
\n",
" \n",
" | Complement system | \n",
" 3 | \n",
" 3 | \n",
" 3 | \n",
" 97 | \n",
" 0.500000 | \n",
"
\n",
" \n",
" | Notch signaling | \n",
" 2 | \n",
" 2 | \n",
" 8 | \n",
" 117 | \n",
" 0.500000 | \n",
"
\n",
" \n",
" | DKK1 | \n",
" 5 | \n",
" 5 | \n",
" 8 | \n",
" 77 | \n",
" 0.500000 | \n",
"
\n",
" \n",
" | Acetylcholine signaling | \n",
" 3 | \n",
" 3 | \n",
" 2 | \n",
" 361 | \n",
" 0.500000 | \n",
"
\n",
" \n",
" | Neurotrophic | \n",
" 6 | \n",
" 7 | \n",
" 1 | \n",
" 56 | \n",
" 0.461538 | \n",
"
\n",
" \n",
" | Tumor necrosis factor | \n",
" 20 | \n",
" 29 | \n",
" 16 | \n",
" 214 | \n",
" 0.408163 | \n",
"
\n",
" \n",
" | Binding and Uptake of Ligands by Scavenger Receptors | \n",
" 2 | \n",
" 3 | \n",
" 2 | \n",
" 72 | \n",
" 0.400000 | \n",
"
\n",
" \n",
" | miRNA | \n",
" 10 | \n",
" 16 | \n",
" 80 | \n",
" 167 | \n",
" 0.384615 | \n",
"
\n",
" \n",
" | Nerve growth factor | \n",
" 6 | \n",
" 10 | \n",
" 5 | \n",
" 171 | \n",
" 0.375000 | \n",
"
\n",
" \n",
" | Insulin signal transduction | \n",
" 10 | \n",
" 19 | \n",
" 17 | \n",
" 761 | \n",
" 0.344828 | \n",
"
\n",
" \n",
" | Prostaglandin | \n",
" 1 | \n",
" 2 | \n",
" 1 | \n",
" 82 | \n",
" 0.333333 | \n",
"
\n",
" \n",
" | Axonal transport | \n",
" 1 | \n",
" 2 | \n",
" 4 | \n",
" 35 | \n",
" 0.333333 | \n",
"
\n",
" \n",
" | Chemokine signaling | \n",
" 6 | \n",
" 12 | \n",
" 4 | \n",
" 168 | \n",
" 0.333333 | \n",
"
\n",
" \n",
"
\n",
"