{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Systems Immunogenetics Project\n",
"\n",
"## WNV Cleaning Steps\n",
"\n",
"### McWeeney Lab, Oregon Health & Science University\n",
"\n",
"** Authors: Gabrielle Choonoo (choonoo@ohsu.edu) and Michael Mooney (mooneymi@ohsu.edu) **\n",
"\n",
"## Introduction\n",
"\n",
"This is the step-by-step process for cleaning the WNV qPCR data (ByLine and ByMouse).\n",
"\n",
"Required Files:\n",
"* qPCR ByLine and ByMouse Data Files\n",
"* This notebook** (`SIG_WNV_qPCR_Data_Cleaning.ipynb`): [[Download here]](https://raw.githubusercontent.com/biodev/SIG/master/SIG_WNV_qPCR_Data_Cleaning.ipynb)\n",
"* The R script (`qpcr_data_cleaning_functions.r`): [[Download here]](https://raw.githubusercontent.com/biodev/SIG/master/scripts/qpcr_data_cleaning_functions.r)\n",
"* The data dictionary containing all qPCR variables (`WNV_Data_Dictionary.xlsx`): [[Download here]](https://raw.githubusercontent.com/biodev/SIG/master/data/WNV_Data_Dictionary.xlsx)\n",
"\n",
"** Note: this notebook can also be downloaded as an R script (only the code blocks seen below will be included): [[Download R script here]](https://raw.githubusercontent.com/biodev/SIG/master/SIG_WNV_qPCR_Data_Cleaning.r)\n",
"\n",
"Required R packages:\n",
"- `gdata` - [https://cran.r-project.org/web/packages/gdata/index.html](https://cran.r-project.org/web/packages/gdata/index.html)\n",
"\n",
"** All code is available on GitHub: [https://github.com/biodev/SIG](https://github.com/biodev/SIG) **"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1. Load Necessary R Packages and Functions"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"gdata: read.xls support for 'XLS' (Excel 97-2004) files ENABLED.\n",
"\n",
"gdata: read.xls support for 'XLSX' (Excel 2007+) files ENABLED.\n",
"\n",
"Attaching package: ‘gdata’\n",
"\n",
"The following object is masked from ‘package:stats’:\n",
"\n",
" nobs\n",
"\n",
"The following object is masked from ‘package:utils’:\n",
"\n",
" object.size\n",
"\n"
]
}
],
"source": [
"source('./scripts/qpcr_data_cleaning_functions.r')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2. Read ByLine Data"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<table>\n",
"<thead><tr><th></th><th scope=col>UW_Line</th><th scope=col>Mating</th><th scope=col>Timepoint</th><th scope=col>Condition</th><th scope=col>Tissue</th><th scope=col>Experiment</th><th scope=col>N</th><th scope=col>dCt.mean</th><th scope=col>dCt.sd</th><th scope=col>baseline.dCt</th><th scope=col>ddCt.mean</th><th scope=col>ddCt.sd</th><th scope=col>fc.mean</th><th scope=col>fc.sd</th></tr></thead>\n",
"<tbody>\n",
"\t<tr><th scope=row>1</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFIT1</td><td>3</td><td>7.060444</td><td>4.44931</td><td>12.28855</td><td>-5.228111</td><td>4.44931</td><td>37.4816</td><td>4.44931</td></tr>\n",
"\t<tr><th scope=row>2</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFITM1</td><td>3</td><td>7.164444</td><td>1.29619</td><td>8.593667</td><td>-1.429223</td><td>1.29619</td><td>2.693016</td><td>1.29619</td></tr>\n",
"\t<tr><th scope=row>3</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFNb1</td><td>3</td><td>14.47589</td><td>5.691944</td><td>18.79678</td><td>-4.32089</td><td>5.691944</td><td>19.98561</td><td>5.691944</td></tr>\n",
"\t<tr><th scope=row>4</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IL12b</td><td>3</td><td>13.98678</td><td>5.186199</td><td>18.61556</td><td>-4.628778</td><td>5.186199</td><td>24.74008</td><td>5.186199</td></tr>\n",
"\t<tr><th scope=row>5</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>WNV</td><td>3</td><td>12.963</td><td>7.274413</td><td>19.51256</td><td>-6.549556</td><td>7.274413</td><td>93.67266</td><td>7.274413</td></tr>\n",
"\t<tr><th scope=row>6</th><td>4</td><td>16188x3252</td><td>12M</td><td>B_d12M</td><td>Brain</td><td>IFIT1</td><td>3</td><td>12.28855</td><td>0.7611464</td><td>12.28855</td><td>0</td><td>0.7611464</td><td>1</td><td>0.7611464</td></tr>\n",
"</tbody>\n",
"</table>\n"
],
"text/latex": [
"\\begin{tabular}{r|llllllllllllll}\n",
" & UW_Line & Mating & Timepoint & Condition & Tissue & Experiment & N & dCt.mean & dCt.sd & baseline.dCt & ddCt.mean & ddCt.sd & fc.mean & fc.sd\\\\\n",
"\\hline\n",
"\t1 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFIT1 & 3 & 7.060444 & 4.44931 & 12.28855 & -5.228111 & 4.44931 & 37.4816 & 4.44931\\\\\n",
"\t2 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFITM1 & 3 & 7.164444 & 1.29619 & 8.593667 & -1.429223 & 1.29619 & 2.693016 & 1.29619\\\\\n",
"\t3 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFNb1 & 3 & 14.47589 & 5.691944 & 18.79678 & -4.32089 & 5.691944 & 19.98561 & 5.691944\\\\\n",
"\t4 & 4 & 16188x3252 & 12 & B_d12 & Brain & IL12b & 3 & 13.98678 & 5.186199 & 18.61556 & -4.628778 & 5.186199 & 24.74008 & 5.186199\\\\\n",
"\t5 & 4 & 16188x3252 & 12 & B_d12 & Brain & WNV & 3 & 12.963 & 7.274413 & 19.51256 & -6.549556 & 7.274413 & 93.67266 & 7.274413\\\\\n",
"\t6 & 4 & 16188x3252 & 12M & B_d12M & Brain & IFIT1 & 3 & 12.28855 & 0.7611464 & 12.28855 & 0 & 0.7611464 & 1 & 0.7611464\\\\\n",
"\\end{tabular}\n"
],
"text/plain": [
" UW_Line Mating Timepoint Condition Tissue Experiment N dCt.mean\n",
"1 4 16188x3252 12 B_d12 Brain IFIT1 3 7.060444\n",
"2 4 16188x3252 12 B_d12 Brain IFITM1 3 7.164444\n",
"3 4 16188x3252 12 B_d12 Brain IFNb1 3 14.475888\n",
"4 4 16188x3252 12 B_d12 Brain IL12b 3 13.986777\n",
"5 4 16188x3252 12 B_d12 Brain WNV 3 12.963000\n",
"6 4 16188x3252 12M B_d12M Brain IFIT1 3 12.288555\n",
" dCt.sd baseline.dCt ddCt.mean ddCt.sd fc.mean fc.sd\n",
"1 4.4493099 12.288555 -5.228111 4.4493099 37.481600 4.4493099\n",
"2 1.2961898 8.593667 -1.429223 1.2961898 2.693016 1.2961898\n",
"3 5.6919441 18.796778 -4.320890 5.6919441 19.985608 5.6919441\n",
"4 5.1861991 18.615555 -4.628778 5.1861991 24.740076 5.1861991\n",
"5 7.2744126 19.512556 -6.549556 7.2744126 93.672658 7.2744126\n",
"6 0.7611464 12.288555 0.000000 0.7611464 1.000000 0.7611464"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>1430</li>\n",
"\t<li>14</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 1430\n",
"\\item 14\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 1430\n",
"2. 14\n",
"\n",
"\n"
],
"text/plain": [
"[1] 1430 14"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Set data directory\n",
"data_dir = \"/Users/mooneymi/Documents/SIG/WNV/qPCR\"\n",
"\n",
"## Read in data (byLine)\n",
"qpcr_data = read.xls(file.path(data_dir, \"16-May-2016/Gale_qPCR_byLine_5-16-16 %282%29.xlsx\"), sheet=1)\n",
"\n",
"head(qpcr_data)\n",
"dim(qpcr_data)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>1430</li>\n",
"\t<li>14</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 1430\n",
"\\item 14\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 1430\n",
"2. 14\n",
"\n",
"\n"
],
"text/plain": [
"[1] 1430 14"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Replace line 82 with fixed data (special case for May 16, 2016 data)\n",
"qpcr_data = qpcr_data[qpcr_data$UW_Line != 82, ]\n",
"\n",
"line_82 = read.xls(file.path(data_dir, \"18-May-2016/Gale_qPCR_byLine_5-18-16.xlsx\"), sheet=1)\n",
"qpcr_data = rbind(qpcr_data, line_82)\n",
"\n",
"dim(qpcr_data)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>1560</li>\n",
"\t<li>14</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 1560\n",
"\\item 14\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 1560\n",
"2. 14\n",
"\n",
"\n"
],
"text/plain": [
"[1] 1560 14"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Replace lines 54 and 58 with fixed data (special case for May 16, 2016 data)\n",
"qpcr_data = qpcr_data[!qpcr_data$UW_Line %in% c(54, 58), ]\n",
"\n",
"lines_54_58 = read.xls(file.path(data_dir, \"23-May-2016/Gale_qPCR_byLine_5-23-16.xlsx\"), sheet=1)\n",
"qpcr_data = rbind(qpcr_data, lines_54_58)\n",
"\n",
"dim(qpcr_data)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check for duplicates\n",
"sum(duplicated(qpcr_data))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<dl class=dl-horizontal>\n",
"\t<dt>4</dt>\n",
"\t\t<dd>16188x3252</dd>\n",
"\t<dt>42</dt>\n",
"\t\t<dd>8008x8016</dd>\n",
"\t<dt>45</dt>\n",
"\t\t<dd>16441x8024</dd>\n",
"\t<dt>46</dt>\n",
"\t\t<dd>8048x15155</dd>\n",
"\t<dt>48</dt>\n",
"\t\t<dd>13140x16680</dd>\n",
"\t<dt>61</dt>\n",
"\t\t<dd>8056x8033</dd>\n",
"\t<dt>62</dt>\n",
"\t\t<dd>8054x8036</dd>\n",
"\t<dt>70</dt>\n",
"\t\t<dd>8045x4410</dd>\n",
"\t<dt>71</dt>\n",
"\t\t<dd>3564x8027</dd>\n",
"\t<dt>72</dt>\n",
"\t\t<dd>5035x16785</dd>\n",
"\t<dt>73</dt>\n",
"\t\t<dd>5358x8046</dd>\n",
"\t<dt>74</dt>\n",
"\t\t<dd>8046x8004</dd>\n",
"\t<dt>75</dt>\n",
"\t\t<dd>8016x8004</dd>\n",
"\t<dt>76</dt>\n",
"\t\t<dd>8024x8048</dd>\n",
"\t<dt>77</dt>\n",
"\t\t<dd>8034x8043</dd>\n",
"\t<dt>78</dt>\n",
"\t\t<dd>13421x16034</dd>\n",
"\t<dt>79</dt>\n",
"\t\t<dd>16034x13067</dd>\n",
"\t<dt>80</dt>\n",
"\t\t<dd>16521x3260</dd>\n",
"\t<dt>81</dt>\n",
"\t\t<dd>16072x5346</dd>\n",
"\t<dt>113</dt>\n",
"\t\t<dd>8027x477</dd>\n",
"\t<dt>82</dt>\n",
"\t\t<dd>16557x3154</dd>\n",
"\t<dt>54</dt>\n",
"\t\t<dd>8036x18018</dd>\n",
"\t<dt>58</dt>\n",
"\t\t<dd>5346x16768</dd>\n",
"</dl>\n"
],
"text/latex": [
"\\begin{description*}\n",
"\\item[4] 16188x3252\n",
"\\item[42] 8008x8016\n",
"\\item[45] 16441x8024\n",
"\\item[46] 8048x15155\n",
"\\item[48] 13140x16680\n",
"\\item[61] 8056x8033\n",
"\\item[62] 8054x8036\n",
"\\item[70] 8045x4410\n",
"\\item[71] 3564x8027\n",
"\\item[72] 5035x16785\n",
"\\item[73] 5358x8046\n",
"\\item[74] 8046x8004\n",
"\\item[75] 8016x8004\n",
"\\item[76] 8024x8048\n",
"\\item[77] 8034x8043\n",
"\\item[78] 13421x16034\n",
"\\item[79] 16034x13067\n",
"\\item[80] 16521x3260\n",
"\\item[81] 16072x5346\n",
"\\item[113] 8027x477\n",
"\\item[82] 16557x3154\n",
"\\item[54] 8036x18018\n",
"\\item[58] 5346x16768\n",
"\\end{description*}\n"
],
"text/markdown": [
"4\n",
": 16188x325242\n",
": 8008x801645\n",
": 16441x802446\n",
": 8048x1515548\n",
": 13140x1668061\n",
": 8056x803362\n",
": 8054x803670\n",
": 8045x441071\n",
": 3564x802772\n",
": 5035x1678573\n",
": 5358x804674\n",
": 8046x800475\n",
": 8016x800476\n",
": 8024x804877\n",
": 8034x804378\n",
": 13421x1603479\n",
": 16034x1306780\n",
": 16521x326081\n",
": 16072x5346113\n",
": 8027x47782\n",
": 16557x315454\n",
": 8036x1801858\n",
": 5346x16768\n",
"\n"
],
"text/plain": [
" 4 42 45 46 48 61 \n",
" 16188x3252 8008x8016 16441x8024 8048x15155 13140x16680 8056x8033 \n",
" 62 70 71 72 73 74 \n",
" 8054x8036 8045x4410 3564x8027 5035x16785 5358x8046 8046x8004 \n",
" 75 76 77 78 79 80 \n",
" 8016x8004 8024x8048 8034x8043 13421x16034 16034x13067 16521x3260 \n",
" 81 113 82 54 58 \n",
" 16072x5346 8027x477 16557x3154 8036x18018 5346x16768 \n",
"23 Levels: 13140x16680 13421x16034 16034x13067 16072x5346 ... 8056x8033"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check that each line has a single mating\n",
"line_matings = sapply(unique(qpcr_data$UW_Line), function(x){unique(qpcr_data$Mating[qpcr_data$UW_Line==x])})\n",
"names(line_matings) = unique(qpcr_data$UW_Line)\n",
"line_matings"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3. Format and Clean ByLine Data"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Data_Altered and Notes columns\n",
"qpcr_data$Data_Altered = NA\n",
"qpcr_data$Notes = NA"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Annotate Virus\n",
"qpcr_data$Virus = NA\n",
"qpcr_data[grep(\"M\",qpcr_data[,\"Timepoint\"]),\"Virus\"] <- \"Mock\"\n",
"qpcr_data[-grep(\"M\",qpcr_data[,\"Timepoint\"]),\"Virus\"] <- \"WNV\""
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"## Remove 'M' from time points and convert to numeric\n",
"qpcr_data[,\"Timepoint\"] <- as.numeric(as.character(gsub(\"M\",\"\",qpcr_data[,\"Timepoint\"])))"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<dl class=dl-horizontal>\n",
"\t<dt>IFIT1</dt>\n",
"\t\t<dd>312</dd>\n",
"\t<dt>IFITM1</dt>\n",
"\t\t<dd>312</dd>\n",
"\t<dt>IFNb1</dt>\n",
"\t\t<dd>312</dd>\n",
"\t<dt>IL12b</dt>\n",
"\t\t<dd>312</dd>\n",
"\t<dt>WNV</dt>\n",
"\t\t<dd>312</dd>\n",
"</dl>\n"
],
"text/latex": [
"\\begin{description*}\n",
"\\item[IFIT1] 312\n",
"\\item[IFITM1] 312\n",
"\\item[IFNb1] 312\n",
"\\item[IL12b] 312\n",
"\\item[WNV] 312\n",
"\\end{description*}\n"
],
"text/markdown": [
"IFIT1\n",
": 312IFITM1\n",
": 312IFNb1\n",
": 312IL12b\n",
": 312WNV\n",
": 312\n",
"\n"
],
"text/plain": [
" IFIT1 IFITM1 IFNb1 IL12b WNV \n",
" 312 312 312 312 312 "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check experiment names\n",
"sum(names(summary(qpcr_data[,\"Experiment\"])) == c(\"IFIT1\",\"IFITM1\", \"IFNb1\", \"IL12b\", \"WNV\")) == 5\n",
"summary(qpcr_data[,\"Experiment\"])"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Group column: UW Line, Timepoint, Virus, Tissue, Experiment separated by \"_\"\n",
"qpcr_data$Group = paste(qpcr_data$UW_Line, qpcr_data$Timepoint, qpcr_data$Virus, qpcr_data$Tissue, qpcr_data$Experiment, sep=\"_\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Lab column\n",
"qpcr_data$Lab = \"Gale\""
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<table>\n",
"<thead><tr><th></th><th scope=col>UW_Line</th><th scope=col>Mating</th><th scope=col>Timepoint</th><th scope=col>Condition</th><th scope=col>Tissue</th><th scope=col>Experiment</th><th scope=col>N</th><th scope=col>dCt.mean</th><th scope=col>dCt.sd</th><th scope=col>baseline.dCt</th><th scope=col>ddCt.mean</th><th scope=col>ddCt.sd</th><th scope=col>fc.mean</th><th scope=col>fc.sd</th><th scope=col>Data_Altered</th><th scope=col>Notes</th><th scope=col>Virus</th><th scope=col>Group</th><th scope=col>Lab</th></tr></thead>\n",
"<tbody>\n",
"\t<tr><th scope=row>1</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFIT1</td><td>3</td><td>7.060444</td><td>4.44931</td><td>12.28855</td><td>-5.228111</td><td>4.44931</td><td>37.4816</td><td>4.44931</td><td>NA</td><td>NA</td><td>WNV</td><td>4_12_WNV_Brain_IFIT1</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>2</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFITM1</td><td>3</td><td>7.164444</td><td>1.29619</td><td>8.593667</td><td>-1.429223</td><td>1.29619</td><td>2.693016</td><td>1.29619</td><td>NA</td><td>NA</td><td>WNV</td><td>4_12_WNV_Brain_IFITM1</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>3</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IFNb1</td><td>3</td><td>14.47589</td><td>5.691944</td><td>18.79678</td><td>-4.32089</td><td>5.691944</td><td>19.98561</td><td>5.691944</td><td>NA</td><td>NA</td><td>WNV</td><td>4_12_WNV_Brain_IFNb1</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>4</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>IL12b</td><td>3</td><td>13.98678</td><td>5.186199</td><td>18.61556</td><td>-4.628778</td><td>5.186199</td><td>24.74008</td><td>5.186199</td><td>NA</td><td>NA</td><td>WNV</td><td>4_12_WNV_Brain_IL12b</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>5</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12</td><td>Brain</td><td>WNV</td><td>3</td><td>12.963</td><td>7.274413</td><td>19.51256</td><td>-6.549556</td><td>7.274413</td><td>93.67266</td><td>7.274413</td><td>NA</td><td>NA</td><td>WNV</td><td>4_12_WNV_Brain_WNV</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>6</th><td>4</td><td>16188x3252</td><td>12</td><td>B_d12M</td><td>Brain</td><td>IFIT1</td><td>3</td><td>12.28855</td><td>0.7611464</td><td>12.28855</td><td>0</td><td>0.7611464</td><td>1</td><td>0.7611464</td><td>NA</td><td>NA</td><td>Mock</td><td>4_12_Mock_Brain_IFIT1</td><td>Gale</td></tr>\n",
"</tbody>\n",
"</table>\n"
],
"text/latex": [
"\\begin{tabular}{r|lllllllllllllllllll}\n",
" & UW_Line & Mating & Timepoint & Condition & Tissue & Experiment & N & dCt.mean & dCt.sd & baseline.dCt & ddCt.mean & ddCt.sd & fc.mean & fc.sd & Data_Altered & Notes & Virus & Group & Lab\\\\\n",
"\\hline\n",
"\t1 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFIT1 & 3 & 7.060444 & 4.44931 & 12.28855 & -5.228111 & 4.44931 & 37.4816 & 4.44931 & NA & NA & WNV & 4_12_WNV_Brain_IFIT1 & Gale\\\\\n",
"\t2 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFITM1 & 3 & 7.164444 & 1.29619 & 8.593667 & -1.429223 & 1.29619 & 2.693016 & 1.29619 & NA & NA & WNV & 4_12_WNV_Brain_IFITM1 & Gale\\\\\n",
"\t3 & 4 & 16188x3252 & 12 & B_d12 & Brain & IFNb1 & 3 & 14.47589 & 5.691944 & 18.79678 & -4.32089 & 5.691944 & 19.98561 & 5.691944 & NA & NA & WNV & 4_12_WNV_Brain_IFNb1 & Gale\\\\\n",
"\t4 & 4 & 16188x3252 & 12 & B_d12 & Brain & IL12b & 3 & 13.98678 & 5.186199 & 18.61556 & -4.628778 & 5.186199 & 24.74008 & 5.186199 & NA & NA & WNV & 4_12_WNV_Brain_IL12b & Gale\\\\\n",
"\t5 & 4 & 16188x3252 & 12 & B_d12 & Brain & WNV & 3 & 12.963 & 7.274413 & 19.51256 & -6.549556 & 7.274413 & 93.67266 & 7.274413 & NA & NA & WNV & 4_12_WNV_Brain_WNV & Gale\\\\\n",
"\t6 & 4 & 16188x3252 & 12 & B_d12M & Brain & IFIT1 & 3 & 12.28855 & 0.7611464 & 12.28855 & 0 & 0.7611464 & 1 & 0.7611464 & NA & NA & Mock & 4_12_Mock_Brain_IFIT1 & Gale\\\\\n",
"\\end{tabular}\n"
],
"text/plain": [
" UW_Line Mating Timepoint Condition Tissue Experiment N dCt.mean\n",
"1 4 16188x3252 12 B_d12 Brain IFIT1 3 7.060444\n",
"2 4 16188x3252 12 B_d12 Brain IFITM1 3 7.164444\n",
"3 4 16188x3252 12 B_d12 Brain IFNb1 3 14.475888\n",
"4 4 16188x3252 12 B_d12 Brain IL12b 3 13.986777\n",
"5 4 16188x3252 12 B_d12 Brain WNV 3 12.963000\n",
"6 4 16188x3252 12 B_d12M Brain IFIT1 3 12.288555\n",
" dCt.sd baseline.dCt ddCt.mean ddCt.sd fc.mean fc.sd Data_Altered\n",
"1 4.4493099 12.288555 -5.228111 4.4493099 37.481600 4.4493099 NA\n",
"2 1.2961898 8.593667 -1.429223 1.2961898 2.693016 1.2961898 NA\n",
"3 5.6919441 18.796778 -4.320890 5.6919441 19.985608 5.6919441 NA\n",
"4 5.1861991 18.615555 -4.628778 5.1861991 24.740076 5.1861991 NA\n",
"5 7.2744126 19.512556 -6.549556 7.2744126 93.672658 7.2744126 NA\n",
"6 0.7611464 12.288555 0.000000 0.7611464 1.000000 0.7611464 NA\n",
" Notes Virus Group Lab\n",
"1 NA WNV 4_12_WNV_Brain_IFIT1 Gale\n",
"2 NA WNV 4_12_WNV_Brain_IFITM1 Gale\n",
"3 NA WNV 4_12_WNV_Brain_IFNb1 Gale\n",
"4 NA WNV 4_12_WNV_Brain_IL12b Gale\n",
"5 NA WNV 4_12_WNV_Brain_WNV Gale\n",
"6 NA Mock 4_12_Mock_Brain_IFIT1 Gale"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"head(qpcr_data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4. Read ByMouse Data"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<table>\n",
"<thead><tr><th></th><th scope=col>ID</th><th scope=col>Mating</th><th scope=col>RIX_ID</th><th scope=col>UW_Line</th><th scope=col>UWID</th><th scope=col>Sample_Name</th><th scope=col>Experiment</th><th scope=col>Tissue</th><th scope=col>Condition</th><th scope=col>Timepoint</th><th scope=col>Ct</th><th scope=col>Ct.sd</th><th scope=col>ref.Ct</th><th scope=col>ref.sd</th><th scope=col>dCt</th><th scope=col>dCt.linear</th><th scope=col>dCt.sd</th></tr></thead>\n",
"<tbody>\n",
"\t<tr><th scope=row>1</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>S 1.12</td><td>IFIT1</td><td>Spleen</td><td>WNV</td><td>12</td><td>30.31133</td><td>0.1537316</td><td>20.94267</td><td>0.08401435</td><td>9.368667</td><td>0.001512691</td><td>0.1751908</td></tr>\n",
"\t<tr><th scope=row>2</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFITM1</td><td>Brain</td><td>WNV</td><td>12</td><td>26.49567</td><td>0.1571378</td><td>20.519</td><td>0.076237</td><td>5.976665</td><td>0.01587978</td><td>0.174655</td></tr>\n",
"\t<tr><th scope=row>3</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>K 1.12</td><td>IFIT1</td><td>Kidney</td><td>WNV</td><td>12</td><td>29.47433</td><td>0.09113332</td><td>19.27533</td><td>0.04600315</td><td>10.199</td><td>0.0008507361</td><td>0.1020861</td></tr>\n",
"\t<tr><th scope=row>4</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>K 1.12</td><td>IL12b</td><td>Kidney</td><td>WNV</td><td>12</td><td>40</td><td>0</td><td>19.27533</td><td>0.04600315</td><td>20.72467</td><td>5.77e-07</td><td>0.04600315</td></tr>\n",
"\t<tr><th scope=row>5</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFNb1</td><td>Brain</td><td>WNV</td><td>12</td><td>29.12167</td><td>0.1936756</td><td>20.519</td><td>0.076237</td><td>8.602666</td><td>0.002572405</td><td>0.2081402</td></tr>\n",
"\t<tr><th scope=row>6</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFIT1</td><td>Brain</td><td>WNV</td><td>12</td><td>23.525</td><td>0.02586539</td><td>20.519</td><td>0.076237</td><td>3.006</td><td>0.1244812</td><td>0.08050527</td></tr>\n",
"</tbody>\n",
"</table>\n"
],
"text/latex": [
"\\begin{tabular}{r|lllllllllllllllll}\n",
" & ID & Mating & RIX_ID & UW_Line & UWID & Sample_Name & Experiment & Tissue & Condition & Timepoint & Ct & Ct.sd & ref.Ct & ref.sd & dCt & dCt.linear & dCt.sd\\\\\n",
"\\hline\n",
"\t1 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & S 1.12 & IFIT1 & Spleen & WNV & 12 & 30.31133 & 0.1537316 & 20.94267 & 0.08401435 & 9.368667 & 0.001512691 & 0.1751908\\\\\n",
"\t2 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFITM1 & Brain & WNV & 12 & 26.49567 & 0.1571378 & 20.519 & 0.076237 & 5.976665 & 0.01587978 & 0.174655\\\\\n",
"\t3 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & K 1.12 & IFIT1 & Kidney & WNV & 12 & 29.47433 & 0.09113332 & 19.27533 & 0.04600315 & 10.199 & 0.0008507361 & 0.1020861\\\\\n",
"\t4 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & K 1.12 & IL12b & Kidney & WNV & 12 & 40 & 0 & 19.27533 & 0.04600315 & 20.72467 & 5.77e-07 & 0.04600315\\\\\n",
"\t5 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFNb1 & Brain & WNV & 12 & 29.12167 & 0.1936756 & 20.519 & 0.076237 & 8.602666 & 0.002572405 & 0.2081402\\\\\n",
"\t6 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFIT1 & Brain & WNV & 12 & 23.525 & 0.02586539 & 20.519 & 0.076237 & 3.006 & 0.1244812 & 0.08050527\\\\\n",
"\\end{tabular}\n"
],
"text/plain": [
" ID Mating RIX_ID UW_Line UWID Sample_Name Experiment Tissue\n",
"1 16188x3252_248 16188x3252 248 4 1.12 S 1.12 IFIT1 Spleen\n",
"2 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFITM1 Brain\n",
"3 16188x3252_248 16188x3252 248 4 1.12 K 1.12 IFIT1 Kidney\n",
"4 16188x3252_248 16188x3252 248 4 1.12 K 1.12 IL12b Kidney\n",
"5 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFNb1 Brain\n",
"6 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFIT1 Brain\n",
" Condition Timepoint Ct Ct.sd ref.Ct ref.sd dCt\n",
"1 WNV 12 30.31133 0.15373160 20.94267 0.08401435 9.368667\n",
"2 WNV 12 26.49567 0.15713780 20.51900 0.07623700 5.976665\n",
"3 WNV 12 29.47433 0.09113332 19.27533 0.04600315 10.199001\n",
"4 WNV 12 40.00000 0.00000000 19.27533 0.04600315 20.724667\n",
"5 WNV 12 29.12167 0.19367563 20.51900 0.07623700 8.602666\n",
"6 WNV 12 23.52500 0.02586539 20.51900 0.07623700 3.006000\n",
" dCt.linear dCt.sd\n",
"1 0.0015126910 0.17519080\n",
"2 0.0158797774 0.17465500\n",
"3 0.0008507361 0.10208610\n",
"4 0.0000005770 0.04600315\n",
"5 0.0025724055 0.20814017\n",
"6 0.1244812292 0.08050527"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>4210</li>\n",
"\t<li>17</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 4210\n",
"\\item 17\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 4210\n",
"2. 17\n",
"\n",
"\n"
],
"text/plain": [
"[1] 4210 17"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Read in data (byMouse)\n",
"qpcr_data_mouse = read.xls(file.path(data_dir, \"16-May-2016/Gale_qPCR_byMouse_5-16-16 %281%29.xlsx\"), sheet=1)\n",
"\n",
"head(qpcr_data_mouse)\n",
"dim(qpcr_data_mouse)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>4215</li>\n",
"\t<li>17</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 4215\n",
"\\item 17\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 4215\n",
"2. 17\n",
"\n",
"\n"
],
"text/plain": [
"[1] 4215 17"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Replace line 82 with fixed data (special case for May 16, 2016 data)\n",
"qpcr_data_mouse = qpcr_data_mouse[qpcr_data_mouse$UW_Line != 82, ]\n",
"\n",
"line_82_mouse = read.xls(file.path(data_dir, \"18-May-2016/Gale_qPCR_byMouse_5-18-16.xlsx\"), sheet=1)\n",
"qpcr_data_mouse = rbind(qpcr_data_mouse, line_82_mouse)\n",
"\n",
"dim(qpcr_data_mouse)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>4600</li>\n",
"\t<li>17</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 4600\n",
"\\item 17\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 4600\n",
"2. 17\n",
"\n",
"\n"
],
"text/plain": [
"[1] 4600 17"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Replace lines 54 and 58 with fixed data (special case for May 16, 2016 data)\n",
"qpcr_data_mouse = qpcr_data_mouse[!qpcr_data_mouse$UW_Line %in% c(54, 58), ]\n",
"\n",
"lines_54_58_mouse = read.xls(file.path(data_dir, \"23-May-2016/Gale_qPCR_byMouse_5-23-16.xlsx\"), sheet=1)\n",
"qpcr_data_mouse = rbind(qpcr_data_mouse, lines_54_58_mouse)\n",
"\n",
"dim(qpcr_data_mouse)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5. Format and Clean ByMouse Data"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check if any CT < 15\n",
"length(which(qpcr_data_mouse[,\"Ct\"] < 15))"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check if any reference CT < 15\n",
"length(which(qpcr_data_mouse[,\"ref.Ct\"] < 15))"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check if an reference CT == 40\n",
"length(which(qpcr_data_mouse[,\"ref.Ct\"] == 40))"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<dl class=dl-horizontal>\n",
"\t<dt>4</dt>\n",
"\t\t<dd>16188x3252</dd>\n",
"\t<dt>42</dt>\n",
"\t\t<dd>8008x8016</dd>\n",
"\t<dt>45</dt>\n",
"\t\t<dd>16441x8024</dd>\n",
"\t<dt>46</dt>\n",
"\t\t<dd>8048x15155</dd>\n",
"\t<dt>48</dt>\n",
"\t\t<dd>13140x16680</dd>\n",
"\t<dt>61</dt>\n",
"\t\t<dd>8056x8033</dd>\n",
"\t<dt>62</dt>\n",
"\t\t<dd>8054x8036</dd>\n",
"\t<dt>70</dt>\n",
"\t\t<dd>8045x4410</dd>\n",
"\t<dt>71</dt>\n",
"\t\t<dd>3564x8027</dd>\n",
"\t<dt>72</dt>\n",
"\t\t<dd>5035x16785</dd>\n",
"\t<dt>73</dt>\n",
"\t\t<dd>5358x8046</dd>\n",
"\t<dt>74</dt>\n",
"\t\t<dd>8046x8004</dd>\n",
"\t<dt>75</dt>\n",
"\t\t<dd>8016x8004</dd>\n",
"\t<dt>76</dt>\n",
"\t\t<dd>8024x8048</dd>\n",
"\t<dt>77</dt>\n",
"\t\t<dd>8034x8043</dd>\n",
"\t<dt>78</dt>\n",
"\t\t<dd>13421x16034</dd>\n",
"\t<dt>79</dt>\n",
"\t\t<dd>16034x13067</dd>\n",
"\t<dt>80</dt>\n",
"\t\t<dd>16521x3260</dd>\n",
"\t<dt>81</dt>\n",
"\t\t<dd>16072x5346</dd>\n",
"\t<dt>113</dt>\n",
"\t\t<dd>8027x477</dd>\n",
"\t<dt>82</dt>\n",
"\t\t<dd>16557x3154</dd>\n",
"\t<dt>54</dt>\n",
"\t\t<dd>8036x18018</dd>\n",
"\t<dt>58</dt>\n",
"\t\t<dd>5346x16768</dd>\n",
"</dl>\n"
],
"text/latex": [
"\\begin{description*}\n",
"\\item[4] 16188x3252\n",
"\\item[42] 8008x8016\n",
"\\item[45] 16441x8024\n",
"\\item[46] 8048x15155\n",
"\\item[48] 13140x16680\n",
"\\item[61] 8056x8033\n",
"\\item[62] 8054x8036\n",
"\\item[70] 8045x4410\n",
"\\item[71] 3564x8027\n",
"\\item[72] 5035x16785\n",
"\\item[73] 5358x8046\n",
"\\item[74] 8046x8004\n",
"\\item[75] 8016x8004\n",
"\\item[76] 8024x8048\n",
"\\item[77] 8034x8043\n",
"\\item[78] 13421x16034\n",
"\\item[79] 16034x13067\n",
"\\item[80] 16521x3260\n",
"\\item[81] 16072x5346\n",
"\\item[113] 8027x477\n",
"\\item[82] 16557x3154\n",
"\\item[54] 8036x18018\n",
"\\item[58] 5346x16768\n",
"\\end{description*}\n"
],
"text/markdown": [
"4\n",
": 16188x325242\n",
": 8008x801645\n",
": 16441x802446\n",
": 8048x1515548\n",
": 13140x1668061\n",
": 8056x803362\n",
": 8054x803670\n",
": 8045x441071\n",
": 3564x802772\n",
": 5035x1678573\n",
": 5358x804674\n",
": 8046x800475\n",
": 8016x800476\n",
": 8024x804877\n",
": 8034x804378\n",
": 13421x1603479\n",
": 16034x1306780\n",
": 16521x326081\n",
": 16072x5346113\n",
": 8027x47782\n",
": 16557x315454\n",
": 8036x1801858\n",
": 5346x16768\n",
"\n"
],
"text/plain": [
" 4 42 45 46 48 61 \n",
" 16188x3252 8008x8016 16441x8024 8048x15155 13140x16680 8056x8033 \n",
" 62 70 71 72 73 74 \n",
" 8054x8036 8045x4410 3564x8027 5035x16785 5358x8046 8046x8004 \n",
" 75 76 77 78 79 80 \n",
" 8016x8004 8024x8048 8034x8043 13421x16034 16034x13067 16521x3260 \n",
" 81 113 82 54 58 \n",
" 16072x5346 8027x477 16557x3154 8036x18018 5346x16768 \n",
"23 Levels: 13140x16680 13421x16034 16034x13067 16072x5346 ... 8056x8033"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check that each line has a single mating\n",
"mouse_line_matings = sapply(unique(qpcr_data_mouse$UW_Line), function(x){unique(qpcr_data_mouse$Mating[qpcr_data_mouse$UW_Line==x])})\n",
"names(mouse_line_matings) = unique(qpcr_data_mouse$UW_Line)\n",
"mouse_line_matings"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Remove 'M' from time points and convert to numeric\n",
"qpcr_data_mouse[,\"Timepoint\"] <- as.numeric(as.character(gsub(\"M\",\"\",qpcr_data_mouse[,\"Timepoint\"])))"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"## Change Condition column name to Virus\n",
"names(qpcr_data_mouse)[which(names(qpcr_data_mouse) == \"Condition\")] <- \"Virus\""
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<dl class=dl-horizontal>\n",
"\t<dt>IFIT1</dt>\n",
"\t\t<dd>920</dd>\n",
"\t<dt>IFITM1</dt>\n",
"\t\t<dd>920</dd>\n",
"\t<dt>IFNb1</dt>\n",
"\t\t<dd>920</dd>\n",
"\t<dt>IL12b</dt>\n",
"\t\t<dd>920</dd>\n",
"\t<dt>WNV</dt>\n",
"\t\t<dd>920</dd>\n",
"</dl>\n"
],
"text/latex": [
"\\begin{description*}\n",
"\\item[IFIT1] 920\n",
"\\item[IFITM1] 920\n",
"\\item[IFNb1] 920\n",
"\\item[IL12b] 920\n",
"\\item[WNV] 920\n",
"\\end{description*}\n"
],
"text/markdown": [
"IFIT1\n",
": 920IFITM1\n",
": 920IFNb1\n",
": 920IL12b\n",
": 920WNV\n",
": 920\n",
"\n"
],
"text/plain": [
" IFIT1 IFITM1 IFNb1 IL12b WNV \n",
" 920 920 920 920 920 "
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Check experiment names\n",
"sum(names(summary(qpcr_data_mouse[,\"Experiment\"])) == c(\"IFIT1\",\"IFITM1\", \"IFNb1\", \"IL12b\", \"WNV\")) == 5\n",
"summary(qpcr_data_mouse[,\"Experiment\"])"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Group column: UW Line, Timepoint, Virus, Tissue, Experiment separated by \"_\"\n",
"qpcr_data_mouse$Group <- paste(qpcr_data_mouse$UW_Line, qpcr_data_mouse$Timepoint, qpcr_data_mouse$Virus, \n",
" qpcr_data_mouse$Tissue, qpcr_data_mouse$Experiment, sep=\"_\")"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"0"
],
"text/latex": [
"0"
],
"text/markdown": [
"0"
],
"text/plain": [
"[1] 0"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check that ByLine file contains data from only those animals in ByMouse file\n",
"length(setdiff(qpcr_data$Group, qpcr_data_mouse$Group))\n",
"length(setdiff(qpcr_data_mouse$Group, qpcr_data$Group))"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Data_Altered and Notes columns\n",
"qpcr_data_mouse$Data_Altered = NA\n",
"qpcr_data_mouse$Notes = NA"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add Lab column\n",
"qpcr_data_mouse$Lab = \"Gale\""
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<table>\n",
"<thead><tr><th></th><th scope=col>ID</th><th scope=col>Mating</th><th scope=col>RIX_ID</th><th scope=col>UW_Line</th><th scope=col>UWID</th><th scope=col>Sample_Name</th><th scope=col>Experiment</th><th scope=col>Tissue</th><th scope=col>Virus</th><th scope=col>Timepoint</th><th scope=col>Ct</th><th scope=col>Ct.sd</th><th scope=col>ref.Ct</th><th scope=col>ref.sd</th><th scope=col>dCt</th><th scope=col>dCt.linear</th><th scope=col>dCt.sd</th><th scope=col>Group</th><th scope=col>Data_Altered</th><th scope=col>Notes</th><th scope=col>Lab</th></tr></thead>\n",
"<tbody>\n",
"\t<tr><th scope=row>1</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>S 1.12</td><td>IFIT1</td><td>Spleen</td><td>WNV</td><td>12</td><td>30.31133</td><td>0.1537316</td><td>20.94267</td><td>0.08401435</td><td>9.368667</td><td>0.001512691</td><td>0.1751908</td><td>4_12_WNV_Spleen_IFIT1</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>2</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFITM1</td><td>Brain</td><td>WNV</td><td>12</td><td>26.49567</td><td>0.1571378</td><td>20.519</td><td>0.076237</td><td>5.976665</td><td>0.01587978</td><td>0.174655</td><td>4_12_WNV_Brain_IFITM1</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>3</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>K 1.12</td><td>IFIT1</td><td>Kidney</td><td>WNV</td><td>12</td><td>29.47433</td><td>0.09113332</td><td>19.27533</td><td>0.04600315</td><td>10.199</td><td>0.0008507361</td><td>0.1020861</td><td>4_12_WNV_Kidney_IFIT1</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>4</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>K 1.12</td><td>IL12b</td><td>Kidney</td><td>WNV</td><td>12</td><td>40</td><td>0</td><td>19.27533</td><td>0.04600315</td><td>20.72467</td><td>5.77e-07</td><td>0.04600315</td><td>4_12_WNV_Kidney_IL12b</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>5</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFNb1</td><td>Brain</td><td>WNV</td><td>12</td><td>29.12167</td><td>0.1936756</td><td>20.519</td><td>0.076237</td><td>8.602666</td><td>0.002572405</td><td>0.2081402</td><td>4_12_WNV_Brain_IFNb1</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"\t<tr><th scope=row>6</th><td>16188x3252_248</td><td>16188x3252</td><td>248</td><td>4</td><td>1.12</td><td>B 1.12</td><td>IFIT1</td><td>Brain</td><td>WNV</td><td>12</td><td>23.525</td><td>0.02586539</td><td>20.519</td><td>0.076237</td><td>3.006</td><td>0.1244812</td><td>0.08050527</td><td>4_12_WNV_Brain_IFIT1</td><td>NA</td><td>NA</td><td>Gale</td></tr>\n",
"</tbody>\n",
"</table>\n"
],
"text/latex": [
"\\begin{tabular}{r|lllllllllllllllllllll}\n",
" & ID & Mating & RIX_ID & UW_Line & UWID & Sample_Name & Experiment & Tissue & Virus & Timepoint & Ct & Ct.sd & ref.Ct & ref.sd & dCt & dCt.linear & dCt.sd & Group & Data_Altered & Notes & Lab\\\\\n",
"\\hline\n",
"\t1 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & S 1.12 & IFIT1 & Spleen & WNV & 12 & 30.31133 & 0.1537316 & 20.94267 & 0.08401435 & 9.368667 & 0.001512691 & 0.1751908 & 4_12_WNV_Spleen_IFIT1 & NA & NA & Gale\\\\\n",
"\t2 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFITM1 & Brain & WNV & 12 & 26.49567 & 0.1571378 & 20.519 & 0.076237 & 5.976665 & 0.01587978 & 0.174655 & 4_12_WNV_Brain_IFITM1 & NA & NA & Gale\\\\\n",
"\t3 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & K 1.12 & IFIT1 & Kidney & WNV & 12 & 29.47433 & 0.09113332 & 19.27533 & 0.04600315 & 10.199 & 0.0008507361 & 0.1020861 & 4_12_WNV_Kidney_IFIT1 & NA & NA & Gale\\\\\n",
"\t4 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & K 1.12 & IL12b & Kidney & WNV & 12 & 40 & 0 & 19.27533 & 0.04600315 & 20.72467 & 5.77e-07 & 0.04600315 & 4_12_WNV_Kidney_IL12b & NA & NA & Gale\\\\\n",
"\t5 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFNb1 & Brain & WNV & 12 & 29.12167 & 0.1936756 & 20.519 & 0.076237 & 8.602666 & 0.002572405 & 0.2081402 & 4_12_WNV_Brain_IFNb1 & NA & NA & Gale\\\\\n",
"\t6 & 16188x3252_248 & 16188x3252 & 248 & 4 & 1.12 & B 1.12 & IFIT1 & Brain & WNV & 12 & 23.525 & 0.02586539 & 20.519 & 0.076237 & 3.006 & 0.1244812 & 0.08050527 & 4_12_WNV_Brain_IFIT1 & NA & NA & Gale\\\\\n",
"\\end{tabular}\n"
],
"text/plain": [
" ID Mating RIX_ID UW_Line UWID Sample_Name Experiment Tissue\n",
"1 16188x3252_248 16188x3252 248 4 1.12 S 1.12 IFIT1 Spleen\n",
"2 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFITM1 Brain\n",
"3 16188x3252_248 16188x3252 248 4 1.12 K 1.12 IFIT1 Kidney\n",
"4 16188x3252_248 16188x3252 248 4 1.12 K 1.12 IL12b Kidney\n",
"5 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFNb1 Brain\n",
"6 16188x3252_248 16188x3252 248 4 1.12 B 1.12 IFIT1 Brain\n",
" Virus Timepoint Ct Ct.sd ref.Ct ref.sd dCt\n",
"1 WNV 12 30.31133 0.15373160 20.94267 0.08401435 9.368667\n",
"2 WNV 12 26.49567 0.15713780 20.51900 0.07623700 5.976665\n",
"3 WNV 12 29.47433 0.09113332 19.27533 0.04600315 10.199001\n",
"4 WNV 12 40.00000 0.00000000 19.27533 0.04600315 20.724667\n",
"5 WNV 12 29.12167 0.19367563 20.51900 0.07623700 8.602666\n",
"6 WNV 12 23.52500 0.02586539 20.51900 0.07623700 3.006000\n",
" dCt.linear dCt.sd Group Data_Altered Notes Lab\n",
"1 0.0015126910 0.17519080 4_12_WNV_Spleen_IFIT1 NA NA Gale\n",
"2 0.0158797774 0.17465500 4_12_WNV_Brain_IFITM1 NA NA Gale\n",
"3 0.0008507361 0.10208610 4_12_WNV_Kidney_IFIT1 NA NA Gale\n",
"4 0.0000005770 0.04600315 4_12_WNV_Kidney_IL12b NA NA Gale\n",
"5 0.0025724055 0.20814017 4_12_WNV_Brain_IFNb1 NA NA Gale\n",
"6 0.1244812292 0.08050527 4_12_WNV_Brain_IFIT1 NA NA Gale"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"head(qpcr_data_mouse)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6. Calculate and Check Summary Measures"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6a. Calculate dCt mean"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All dCt mean correct\"\n"
]
}
],
"source": [
"## Calculate dCt mean from byMouse data\n",
"dct_mean = aggregate(formula=qpcr_data_mouse[,\"dCt\"]~qpcr_data_mouse[,\"Group\"], data=qpcr_data_mouse, FUN=mean)\n",
"names(dct_mean) <- c(\"Group\",\"dCt.mean.V2\")\n",
"dct_mean[order(dct_mean[,1]),] -> dct_mean_order\n",
"\n",
"## Get dCt mean from byLine data\n",
"byline_dct_mean = unique(qpcr_data[,c(\"Group\",\"dCt.mean\")])\n",
"byline_dct_mean[order(byline_dct_mean[,1]),] -> byline_dct_mean_order\n",
"\n",
"## Check that the dCt mean calculations are the same\n",
"check_dct_mean_v2 = sapply(1:dim(dct_mean_order)[1], \n",
" function(x){isTRUE(all.equal(dct_mean_order[x,2], byline_dct_mean_order[x,2]))})\n",
"\n",
"## Print discrepancies, if they exist\n",
"if(sum(check_dct_mean_v2) == nrow(dct_mean_order)){\n",
" print(\"All dCt mean correct\")\n",
"} else {\n",
" print(\"Need to clean dCt mean\")\n",
" dct_mean_errs = cbind(byline_dct_mean_order[which(check_dct_mean_v2==F),], dct_mean_order[which(check_dct_mean_v2==F),])\n",
" dct_mean_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6b. Update dCt mean if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if (dim(dct_mean_errs)[1] > 0) {\n",
" for (i in 1:dim(dct_mean_errs)[1]) {\n",
" qpcr_data$dCt.mean[qpcr_data$Group==dct_mean_errs[i,3]] = dct_mean_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6c. Calculate N for each group"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All N correct\"\n"
]
}
],
"source": [
"## Calculate the N for each group from the byMouse data\n",
"data.frame(summary(as.factor(qpcr_data_mouse[,\"Group\"]),maxsum=8000)) -> bymouse_n\n",
"names(bymouse_n) <- c(\"N.V2\")\n",
"bymouse_n[,2] <- row.names(bymouse_n)\n",
"names(bymouse_n)[2] <- \"Group\"\n",
"bymouse_n = bymouse_n[,c(2,1)]\n",
"bymouse_n[order(bymouse_n[,\"Group\"]),] -> bymouse_n_order\n",
"\n",
"## Get the N for each group from the byLine data \n",
"qpcr_data[,c(\"Group\",\"N\")] -> byline_n\n",
"byline_n[order(byline_n[,\"Group\"]),] -> byline_n_order\n",
"\n",
"## Print discrepancies, if they exist\n",
"if(sum(bymouse_n_order[,2] == byline_n_order[,2]) == nrow(bymouse_n_order)){\n",
" print(\"All N correct\")\n",
"} else {\n",
" print(\"Need to clean N\")\n",
" n_errs = cbind(byline_n_order[bymouse_n_order[,2] != byline_n_order[,2],], \n",
" bymouse_n_order[bymouse_n_order[,2] != byline_n_order[,2],])\n",
" n_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6d. Update N if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if (dim(n_errs)[1] > 0) {\n",
" for (i in 1:dim(n_errs)[1]) {\n",
" qpcr_data$N[qpcr_data$Group==n_errs[i,3]] = n_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6e. Calculate dCt SD"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All dCt SD correct\"\n"
]
}
],
"source": [
"## Calculate dCt SD from the byMouse data \n",
"dct_sd = aggregate(formula=qpcr_data_mouse[,\"dCt\"]~qpcr_data_mouse[,\"Group\"], data=qpcr_data_mouse, FUN=sd)\n",
"names(dct_sd) <- c(\"Group\",\"dCt.sd.V2\")\n",
"dct_sd[order(dct_sd[,1]),] -> dct_sd_order\n",
"\n",
"## Get dCt SD from the byLine data\n",
"byline_dct_sd = qpcr_data[,c(\"Group\",\"dCt.sd\")]\n",
"byline_dct_sd[order(byline_dct_sd[,1]),] -> byline_dct_sd_order\n",
"\n",
"check_dct_sd = sapply(1:dim(dct_sd_order)[1],function(x){isTRUE(all.equal(dct_sd_order[x,2],byline_dct_sd_order[x,2]))})\n",
"\n",
"## Print discrepancies, if they exist\n",
"if(sum(check_dct_sd) == nrow(dct_sd_order)) {\n",
" print(\"All dCt SD correct\")\n",
"} else {\n",
" print(\"Need to clean dCt SD\")\n",
" dct_sd_errs = cbind(byline_dct_sd_order[which(check_dct_sd==F),], dct_sd_order[which(check_dct_sd==F),])\n",
" dct_sd_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6f. Update dCt SD if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if (dim(dct_sd_errs)[1] > 0) {\n",
" for (i in 1:dim(dct_sd_errs)[1]) {\n",
" qpcr_data$dCt.sd[qpcr_data$Group==dct_sd_errs[i,3]] = dct_sd_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6g. Calculate baseline dCt"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All baseline dCt correct\"\n"
]
}
],
"source": [
"## Check baseline.dct\n",
"## Add new group column to annotate baseline\n",
"qpcr_data$Group_g <- paste(qpcr_data[,\"UW_Line\"],qpcr_data[,\"Tissue\"], qpcr_data[,\"Experiment\"],sep=\"_\")\n",
"\n",
"## Calculate baseline, baseline is 12 for this data\n",
"qpcr_data$baseline.dCt.V2 = NA\n",
"baseline = 12\n",
"\n",
"for(i in unique(qpcr_data[,\"Group_g\"])){\n",
" qpcr_data[which(qpcr_data[,\"Group_g\"] == i),\"baseline.dCt.V2\"] <- \n",
" qpcr_data[which(qpcr_data[,\"Group_g\"] == i & qpcr_data[,\"Virus\"] == \"Mock\" & qpcr_data[,\"Timepoint\"] == baseline),\"dCt.mean\"]\n",
"}\n",
"\n",
"# Print discrepancies, if they exist\n",
"if(sum(qpcr_data[,\"baseline.dCt\"] == qpcr_data[,\"baseline.dCt.V2\"]) == nrow(qpcr_data)){\n",
" print(\"All baseline dCt correct\")\n",
"} else {\n",
" print(\"Need to clean baseline dCt\")\n",
" baseline_dct_errs = qpcr_data[qpcr_data[,\"baseline.dCt\"] != qpcr_data[,\"baseline.dCt.V2\"], \n",
" c(\"Group\", \"Group_g\", \"baseline.dCt\", \"baseline.dCt.V2\")]\n",
" baseline_dct_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6h. Update baseline dCt if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"if (dim(baseline_dct_errs)[1] > 0) {\n",
" for (i in 1:dim(baseline_dct_errs)[1]) {\n",
" qpcr_data$baseline.dCt[qpcr_data$Group==baseline_dct_errs[i,1]] = baseline_dct_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6i. Calculate ddCt mean"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All ddCt mean correct\"\n"
]
}
],
"source": [
"## Calculate ddCt mean \n",
"qpcr_data$ddCt.mean.V2 <- as.numeric(as.character(qpcr_data[,\"dCt.mean\"])) - as.numeric(as.character(qpcr_data[,\"baseline.dCt\"]))\n",
"\n",
"check_ddct_mean = sapply(1:dim(qpcr_data)[1],function(x)isTRUE(all.equal(qpcr_data[x,\"ddCt.mean\"], qpcr_data[x,\"ddCt.mean.V2\"], tolerance=5.5e-8)))\n",
"\n",
"## Print discrepancies, if they exist\n",
"if(sum(check_ddct_mean) == dim(qpcr_data)[1]){\n",
" print(\"All ddCt mean correct\")\n",
"} else {\n",
" print(\"Need to clean ddCt mean\")\n",
" ddct_errs = qpcr_data[which(check_ddct_mean==F), c(\"Group\", \"Group_g\", \"ddCt.mean\", \"ddCt.mean.V2\")]\n",
" ddct_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6j. Update ddCt mean if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if (dim(ddct_errs)[1] > 0) {\n",
" for (i in 1:dim(ddct_errs)[1]) {\n",
" qpcr_data$ddCt.mean[qpcr_data$Group==ddct_errs[i,1]] = ddct_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6k. Calculate fold change mean"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1] \"All FC mean correct\"\n"
]
}
],
"source": [
"# check fc mean correct\n",
"qpcr_data$fc.mean.V2 <- 2^-qpcr_data[,\"ddCt.mean\"]\n",
"\n",
"check_fc_mean = sapply(1:dim(qpcr_data)[1],function(x)isTRUE(all.equal(qpcr_data[x,\"fc.mean\"], qpcr_data[x,\"fc.mean.V2\"])))\n",
"\n",
"if(sum(check_fc_mean) == dim(qpcr_data)[1]){\n",
" print(\"All FC mean correct\")\n",
"} else {\n",
" print(\"Need to clean FC mean\")\n",
" fc_errs = qpcr_data[which(check_fc_mean==F), c(\"Group\", \"Group_g\", \"fc.mean\", \"fc.mean.V2\")]\n",
" fc_errs\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6l. Update fold change mean if necessary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"if (dim(fc_errs)[1] > 0) {\n",
" for (i in 1:dim(fc_errs)[1]) {\n",
" qpcr_data$fc.mean[qpcr_data$Group==fc_errs[i,1]] = fc_errs[i,4]\n",
" }\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 7. Remove Unused Columns and Add Baseline dCt.sd and ddCt.se"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Remove extra (unused) columns\n",
"remove_cols = c(\"ddCt.sd\", \"fc.sd\", \"baseline.dCt.V2\", \"ddCt.mean.V2\", \"fc.mean.V2\")\n",
"qpcr_data_v2 = qpcr_data[,-as.vector(unlist(sapply(remove_cols,function(x)which(x==names(qpcr_data)))))]"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Add baseline.dCt.sd column\n",
"qpcr_data_v2$baseline.dCt.sd = NA\n",
"\n",
"# compute baseline sd, use baseline = 12 saved above\n",
"for (i in unique(qpcr_data_v2[,\"Group_g\"])){\n",
" qpcr_data_v2[which(qpcr_data_v2[,\"Group_g\"] == i),\"baseline.dCt.sd\"] <- \n",
" qpcr_data_v2[which(qpcr_data_v2[,\"Group_g\"] == i & \n",
" qpcr_data_v2[,\"Virus\"] == \"Mock\" & \n",
" qpcr_data_v2[,\"Timepoint\"] == baseline),\"dCt.sd\"]\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"## Add ddCt.se\n",
"ddCt.se = sapply(1:dim(qpcr_data_v2)[1],function(x){\n",
" sqrt((qpcr_data_v2[x,\"dCt.sd\"]^2/qpcr_data_v2[x,\"N\"]) + \n",
" (qpcr_data_v2[x,\"baseline.dCt.sd\"]^2/qpcr_data_v2[which(qpcr_data_v2[,\"Group_g\"] == qpcr_data_v2[x,\"Group_g\"] & \n",
" qpcr_data_v2[,\"Virus\"] == \"Mock\" & \n",
" qpcr_data_v2[,\"Timepoint\"] == baseline),\"N\"])\n",
" )\n",
"})\n",
"qpcr_data_v2$ddCt.se <- ddCt.se"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 8. Finalize Data"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"TRUE"
],
"text/latex": [
"TRUE"
],
"text/markdown": [
"TRUE"
],
"text/plain": [
"[1] TRUE"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Order by group\n",
"qpcr_data_v3_final = qpcr_data_v2[order(qpcr_data_v2[,\"Group\"]),]\n",
"\n",
"## Double-check all calculations are correct\n",
"## dCt mean\n",
"isTRUE(all.equal(dct_mean_order[,2], qpcr_data_v3_final[,\"dCt.mean\"]))\n",
"\n",
"## N\n",
"isTRUE(all.equal(bymouse_n_order[,2], qpcr_data_v3_final[,\"N\"]))\n",
"\n",
"## dCt SD\n",
"isTRUE(all.equal(dct_sd_order[,2], qpcr_data_v3_final[,\"dCt.sd\"]))\n",
"\n",
"## baseline dCt\n",
"isTRUE(all.equal(qpcr_data[order(qpcr_data[,\"Group\"]),\"baseline.dCt.V2\"],qpcr_data_v3_final[,\"baseline.dCt\"]))\n",
"\n",
"## ddCt mean\n",
"isTRUE(all.equal(qpcr_data[order(qpcr_data[,\"Group\"]),\"ddCt.mean.V2\"],qpcr_data_v3_final[,\"ddCt.mean\"]))\n",
"\n",
"## FC mean\n",
"isTRUE(all.equal(qpcr_data[order(qpcr_data[,\"Group\"]),\"fc.mean.V2\"],qpcr_data_v3_final[,\"fc.mean\"]))\n",
"\n",
"## baseline SD, computed using corrected dCt SD at timepoint 12\n",
"\n",
"## ddCt SE, computed using correct dCt SD and baseline SD\n",
"\n",
"## Remove Group_g column\n",
"qpcr_data_final_format <- qpcr_data_v3_final[,!(names(qpcr_data_v3_final) %in% \"Group_g\")]"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"## Order columns according to data dictionary (byLine)\n",
"## Note: you may have to change the path to the data dictionary\n",
"data_dict <- read.xls(xls=\"./data/WNV_Data_Dictionary.xlsx\", sheet=\"qPCR Data - By Line\", as.is=T)\n",
"qpcr_data_final_format_order = qpcr_data_final_format[, data_dict[,1]]"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"## Order columns according to data dictionary (byMouse)\n",
"data_dict <- read.xls(xls=\"./data/WNV_Data_Dictionary.xlsx\", sheet=\"qPCR Data - By Mouse\", as.is=T)\n",
"qpcr_data_mouse_order = qpcr_data_mouse[, data_dict[,1]]"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"\t<li>TRUE</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\item TRUE\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. TRUE\n",
"2. TRUE\n",
"3. TRUE\n",
"4. TRUE\n",
"5. TRUE\n",
"6. TRUE\n",
"7. TRUE\n",
"8. TRUE\n",
"9. TRUE\n",
"10. TRUE\n",
"11. TRUE\n",
"12. TRUE\n",
"13. TRUE\n",
"14. TRUE\n",
"15. TRUE\n",
"16. TRUE\n",
"17. TRUE\n",
"18. TRUE\n",
"19. TRUE\n",
"20. TRUE\n",
"21. TRUE\n",
"22. TRUE\n",
"23. TRUE\n",
"\n",
"\n"
],
"text/plain": [
" [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE\n",
"[16] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check both byLine and byMouse have the same UW lines\n",
"names(summary(as.factor(qpcr_data_mouse_order[,\"UW_Line\"]))) == \n",
"names(summary(as.factor(qpcr_data_final_format_order[,\"UW_Line\"])))"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>1560</li>\n",
"\t<li>19</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 1560\n",
"\\item 19\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 1560\n",
"2. 19\n",
"\n",
"\n"
],
"text/plain": [
"[1] 1560 19"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>4600</li>\n",
"\t<li>21</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 4600\n",
"\\item 21\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 4600\n",
"2. 21\n",
"\n",
"\n"
],
"text/plain": [
"[1] 4600 21"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dim(qpcr_data_final_format_order)\n",
"dim(qpcr_data_mouse_order)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 9. Combine with Previously Cleaned Data"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>3990</li>\n",
"\t<li>19</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 3990\n",
"\\item 19\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 3990\n",
"2. 19\n",
"\n",
"\n"
],
"text/plain": [
"[1] 3990 19"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Read in previous version of byLine data\n",
"cleaned_data_dir = \"~/Documents/SIG/WNV/Cleaned_Data_Releases/23-Mar-2016/\"\n",
"prev_qpcr_byline = read.xls(xls=file.path(cleaned_data_dir, \"Gale_qPCR_byLine_23-Mar-2016_final.xlsx\"), sheet=1, as.is=T)\n",
"dim(prev_qpcr_byline)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"170"
],
"text/latex": [
"170"
],
"text/markdown": [
"170"
],
"text/plain": [
"[1] 170"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check for duplicates (new data will overwrite old)\n",
"dup_groups = intersect(qpcr_data_final_format_order$Group, prev_qpcr_byline$Group)\n",
"length(dup_groups)"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>3820</li>\n",
"\t<li>19</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 3820\n",
"\\item 19\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 3820\n",
"2. 19\n",
"\n",
"\n"
],
"text/plain": [
"[1] 3820 19"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Remove duplicated groups from previous data\n",
"prev_qpcr_byline = prev_qpcr_byline[!prev_qpcr_byline$Group %in% dup_groups, ]\n",
"dim(prev_qpcr_byline)"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>5380</li>\n",
"\t<li>19</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 5380\n",
"\\item 19\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 5380\n",
"2. 19\n",
"\n",
"\n"
],
"text/plain": [
"[1] 5380 19"
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>5380</li>\n",
"\t<li>19</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 5380\n",
"\\item 19\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 5380\n",
"2. 19\n",
"\n",
"\n"
],
"text/plain": [
"[1] 5380 19"
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Combine old and new data\n",
"qpcr_byline_updated = rbind(prev_qpcr_byline, qpcr_data_final_format_order)\n",
"dim(qpcr_byline_updated)\n",
"\n",
"# Set blanks to NA\n",
"qpcr_byline_updated_cleaned = clean_na(qpcr_byline_updated)\n",
"\n",
"# Remove duplicates\n",
"if(sum(duplicated(qpcr_byline_updated_cleaned)) != 0){\n",
" qpcr_byline_updated_cleaned = qpcr_byline_updated_cleaned[!duplicated(qpcr_byline_updated_cleaned),]\n",
"}\n",
"dim(qpcr_byline_updated_cleaned)"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>11564</li>\n",
"\t<li>21</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 11564\n",
"\\item 21\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 11564\n",
"2. 21\n",
"\n",
"\n"
],
"text/plain": [
"[1] 11564 21"
]
},
"execution_count": 47,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Read in previous version of byMouse data\n",
"prev_qpcr_bymouse = read.xls(xls=file.path(cleaned_data_dir, \"Gale_qPCR_byMouse_23-Mar-2016_final.xlsx\"), sheet=1)\n",
"dim(prev_qpcr_bymouse)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"495"
],
"text/latex": [
"495"
],
"text/markdown": [
"495"
],
"text/plain": [
"[1] 495"
]
},
"execution_count": 48,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Check for duplicates (new data will overwrite old)\n",
"prev_ids = paste(prev_qpcr_bymouse$ID, prev_qpcr_bymouse$Tissue, prev_qpcr_bymouse$Experiment, sep='_')\n",
"new_ids = paste(qpcr_data_mouse_order$ID, qpcr_data_mouse_order$Tissue, qpcr_data_mouse_order$Experiment, sep='_')\n",
"dup_ids = intersect(prev_ids, new_ids)\n",
"length(dup_ids)"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>11069</li>\n",
"\t<li>21</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 11069\n",
"\\item 21\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 11069\n",
"2. 21\n",
"\n",
"\n"
],
"text/plain": [
"[1] 11069 21"
]
},
"execution_count": 49,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Remove duplicated IDs from previous data\n",
"prev_qpcr_bymouse = prev_qpcr_bymouse[which(!prev_ids %in% dup_ids), ]\n",
"dim(prev_qpcr_bymouse)"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>15669</li>\n",
"\t<li>21</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 15669\n",
"\\item 21\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 15669\n",
"2. 21\n",
"\n",
"\n"
],
"text/plain": [
"[1] 15669 21"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"text/html": [
"<ol class=list-inline>\n",
"\t<li>15669</li>\n",
"\t<li>21</li>\n",
"</ol>\n"
],
"text/latex": [
"\\begin{enumerate*}\n",
"\\item 15669\n",
"\\item 21\n",
"\\end{enumerate*}\n"
],
"text/markdown": [
"1. 15669\n",
"2. 21\n",
"\n",
"\n"
],
"text/plain": [
"[1] 15669 21"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## Combine old and new data\n",
"qpcr_bymouse_updated = rbind(prev_qpcr_bymouse, qpcr_data_mouse_order)\n",
"dim(qpcr_bymouse_updated)\n",
"\n",
"## Set blanks to NA\n",
"qpcr_bymouse_updated_cleaned = clean_na(qpcr_bymouse_updated)\n",
"\n",
"## Remove duplicates\n",
"if(sum(duplicated(qpcr_bymouse_updated_cleaned)) != 0){\n",
" qpcr_bymouse_updated_cleaned = qpcr_byline_updated_cleaned[!duplicated(qpcr_bymouse_updated_cleaned),]\n",
"}\n",
"dim(qpcr_bymouse_updated_cleaned)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 10. Make Any Manual Corrections, If Necessary (Record These in README)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 11. Save Cleaned Data"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Save ByLine Data\n",
"write.table(qpcr_byline_updated_cleaned, file=file.path(data_dir, \"23-May-2016/Gale_qPCR_byLine_5-23-16_MM_updated.txt\"), \n",
" col.names=T, row.names=F, sep='\\t', quote=F, na=\"\")\n",
"## Save ByMouse Data\n",
"write.table(qpcr_bymouse_updated_cleaned, file=file.path(data_dir, \"23-May-2016/Gale_qPCR_byMouse_5-23-16_MM_updated.txt\"), \n",
" col.names=T, row.names=F, sep='\\t', quote=F, na=\"\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"#### Last Updated: 26-May-2016"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "R",
"language": "R",
"name": "ir"
},
"language_info": {
"codemirror_mode": "r",
"file_extension": ".r",
"mimetype": "text/x-r-source",
"name": "R",
"pygments_lexer": "r",
"version": "3.2.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}