{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Running in Docker container on Ostrich\n",
"\n",
"#### Started Docker container with the following command:\n",
"\n",
"```docker run -p 8888:8888 -v /Users/sam/data/:/data -v /Users/sam/owl_home/:/owl_home -v /Users/sam/owl_web/:/owl_web -v /Users/sam/gitrepos:/gitrepos -it f99537d7e06a```\n",
"\n",
"The command allows access to Jupyter Notebook over port 8888 and makes my Jupyter Notebook GitHub repo and my data files on Owl/home and Owl/web accessible to the Docker container.\n",
"\n",
"Once the container was started, started Jupyter Notebook with the following command inside the Docker container:\n",
"\n",
"```jupyter notebook```\n",
"\n",
"This is configured in the Docker container to launch a Jupyter Notebook without a browser on port 8888.\n",
"\n",
"The Docker container is running on an image created from this [Dockerfile (Git commit 443bc42)](https://github.com/sr320/LabDocs/blob/443bc425cd36d23a07cf12625f38b7e3a397b9be/code/dockerfiles/Dockerfile.bio)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wed Mar 1 20:13:26 UTC 2017\n"
]
}
],
"source": [
"%%bash\n",
"date"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Check computer specs"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0f2bca9c664b\n"
]
}
],
"source": [
"%%bash\n",
"hostname"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Architecture: x86_64\n",
"CPU op-mode(s): 32-bit, 64-bit\n",
"Byte Order: Little Endian\n",
"CPU(s): 8\n",
"On-line CPU(s) list: 0-7\n",
"Thread(s) per core: 1\n",
"Core(s) per socket: 8\n",
"Socket(s): 1\n",
"Vendor ID: GenuineIntel\n",
"CPU family: 6\n",
"Model: 26\n",
"Model name: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz\n",
"Stepping: 5\n",
"CPU MHz: 2260.998\n",
"BogoMIPS: 4521.99\n",
"Hypervisor vendor: KVM\n",
"Virtualization type: full\n",
"L1d cache: 32K\n",
"L1i cache: 32K\n",
"L2 cache: 256K\n",
"L3 cache: 8192K\n"
]
}
],
"source": [
"%%bash\n",
"lscpu"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Copy non-demultiplexed BGI FASTQ files to local computer (Ostrich) and verfiy checksums"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"\u001b[0m\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_35A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_3A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_3A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_4A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_4A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_5A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_5A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_6A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_6A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_7A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_7A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_8A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_8A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_9A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1HL_9A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_10A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_10A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_11A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_11A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_12A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_12A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_13A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_13A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_14A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_14A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_15A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_15A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_16A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_16A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_17A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_17A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_18A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_18A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_19A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_19A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_1A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_1A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_20A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_20A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_21A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_21A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_22A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_22A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_23A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_23A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_24A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_24A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_25A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_25A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_26A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_26A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_27A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_27A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_28A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_28A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_29A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_29A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_2A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_2A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_30A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_30A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_31A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_31A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_32A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_32A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_33A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_33A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_4A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_4A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_5A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_5A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_6A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_6A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_7A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_7A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_8A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_8A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_9A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1NF_9A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_10A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_10A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_11A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_11A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_12A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_12A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_13A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_13A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_14A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_14A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_15A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_15A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_16A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_16A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_17A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_17A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_18A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_18A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_19A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\n",
"\u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\n",
"\u001b[01;32mUnKnow_1.fq.gz\u001b[0m*\n",
"\u001b[01;32mUnKnow_2.fq.gz\u001b[0m*\n",
"\u001b[01;32mbgi_project_summary.tar.gz\u001b[0m*\n",
"checksums.md5\n",
"index.lst\n",
"\u001b[01;32mmd5.check\u001b[0m*\n",
"\u001b[01;32mmd5.txt\u001b[0m*\n",
"md51.txt\n",
"readme.md\n",
"split.sh\n"
]
}
],
"source": [
"ls /owl_web/nightingales/O_lurida/20160223_gbs/"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%bash\n",
"mkdir /data/20170301_fastqc_gbs"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/data/20170301_fastqc_gbs\n"
]
}
],
"source": [
"cd /data/20170301_fastqc_gbs/"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"real\t31m9.816s\n",
"user\t0m0.020s\n",
"sys\t5m31.590s\n"
]
}
],
"source": [
"%%bash\n",
"time for file in /owl_web/nightingales/O_lurida/20160223_gbs/160123*\n",
" do\n",
" cp \"$file\" .\n",
" done"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate MD5 checksums & compare to original checksums"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"bash: line 3: md5: command not found\n",
"bash: line 3: md5: command not found\n",
"\n",
"real\t0m0.020s\n",
"user\t0m0.010s\n",
"sys\t0m0.000s\n"
]
}
],
"source": [
"%%bash\n",
"time for file in *.gz\n",
" do\n",
" md5 \"$file\" >> temp_checksums.md5\n",
" done"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%bash\n",
"cat temp_checksums.md5_checksums.md5"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 37G\n",
"-rw-r--r-- 1 srlab staff 17G Mar 1 20:36 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"-rw-r--r-- 1 srlab staff 20G Mar 1 20:54 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"-rw-r--r-- 1 srlab staff 0 Mar 1 20:54 temp_checksums.md5\n"
]
}
],
"source": [
"%%bash\n",
"ls -lh"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Whoops! Typos in both of the above commands. Will re-run..."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"real\t7m1.077s\n",
"user\t0m3.950s\n",
"sys\t4m11.280s\n"
]
}
],
"source": [
"%%bash\n",
"time for file in *.gz\n",
" do\n",
" md5sum \"$file\" >> temp_checksums.md5\n",
" done"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"afd18614b9f5694af1a59672821cd0db 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"2894dd9b54c1388d3d74e8f8642aa267 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n"
]
}
],
"source": [
"%%bash\n",
"cat temp_checksums.md5"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"awk: line 1: syntax error at or near [\n"
]
}
],
"source": [
"%%bash\n",
"awk '/160123_I132/[print $0]' /owl_web/nightingales/O_lurida/20160223_gbs/checksums.md5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Bracket typo. Redo..."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MD5 (/Volumes/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz) = afd18614b9f5694af1a59672821cd0db\n",
"MD5 (/Volumes/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz) = 2894dd9b54c1388d3d74e8f8642aa267\n"
]
}
],
"source": [
"%%bash\n",
"awk '/160123_I132/{print $0}' /owl_web/nightingales/O_lurida/20160223_gbs/checksums.md5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Visual comparison looks good. Will proceed with FastQC"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### FastQC Analysis"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/usr/local/bioinformatics/FastQC/fastqc\n"
]
}
],
"source": [
"%%bash\n",
"which fastqc"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analysis complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Analysis complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Started analysis of 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 5% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 10% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 15% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 20% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 25% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 30% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 35% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 40% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 45% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 50% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 55% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 60% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 65% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 70% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 75% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 80% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 85% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 90% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Approx 95% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\n",
"Started analysis of 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 5% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 10% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 15% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 20% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 25% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 30% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 35% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 40% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 45% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 50% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 55% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 60% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 65% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 70% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 75% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 80% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 85% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 90% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"Approx 95% complete for 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\n",
"\n",
"real\t353m56.148s\n",
"user\t28m26.990s\n",
"sys\t215m31.450s\n"
]
}
],
"source": [
"%%bash\n",
"time fastqc --extract 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 37G\r\n",
"-rw-r--r-- 1 srlab staff 17G Mar 1 20:36 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"drwxr-xr-x 1 srlab staff 272 Mar 2 00:35 \u001b[0m\u001b[01;34m160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc\u001b[0m/\r\n",
"-rw-r--r-- 1 srlab staff 294K Mar 2 00:35 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.html\r\n",
"-rw-r--r-- 1 srlab staff 403K Mar 2 00:35 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.zip\r\n",
"-rw-r--r-- 1 srlab staff 20G Mar 1 20:54 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"drwxr-xr-x 1 srlab staff 272 Mar 2 03:12 \u001b[01;34m160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc\u001b[0m/\r\n",
"-rw-r--r-- 1 srlab staff 347K Mar 2 03:12 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.html\r\n",
"-rw-r--r-- 1 srlab staff 489K Mar 2 03:12 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.zip\r\n",
"-rw-r--r-- 1 srlab staff 186 Mar 1 21:10 temp_checksums.md5\r\n"
]
}
],
"source": [
"ls -lh"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[0m\u001b[01;34mIcons\u001b[0m/ \u001b[01;34mImages\u001b[0m/ fastqc.fo fastqc_data.txt fastqc_report.html summary.txt\r\n"
]
}
],
"source": [
"ls 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PASS\tBasic Statistics\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tPer base sequence quality\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"WARN\tPer tile sequence quality\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tPer sequence quality scores\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"WARN\tPer base sequence content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tPer sequence GC content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tPer base N content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tSequence Length Distribution\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"FAIL\tSequence Duplication Levels\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"PASS\tOverrepresented sequences\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"FAIL\tAdapter Content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"FAIL\tKmer Content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n"
]
}
],
"source": [
"cat 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/summary.txt"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[0m\u001b[01;34mIcons\u001b[0m/ \u001b[01;34mImages\u001b[0m/ fastqc.fo fastqc_data.txt fastqc_report.html summary.txt\r\n"
]
}
],
"source": [
"ls 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"cat: 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/: Is a directory\r\n"
]
}
],
"source": [
"cat 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/s"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PASS\tBasic Statistics\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tPer base sequence quality\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tPer tile sequence quality\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"PASS\tPer sequence quality scores\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tPer base sequence content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"PASS\tPer sequence GC content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"WARN\tPer base N content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"PASS\tSequence Length Distribution\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tSequence Duplication Levels\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"WARN\tOverrepresented sequences\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tAdapter Content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"FAIL\tKmer Content\t160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n"
]
}
],
"source": [
"cat 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/summary.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Move the FastQC Outputs to Nightingales"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc.fo' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc.fo'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_data.txt' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_data.txt'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/error.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/error.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/fastqc_icon.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/fastqc_icon.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/tick.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/tick.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/warning.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/warning.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/adapter_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/adapter_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/duplication_levels.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/duplication_levels.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/kmer_profiles.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/kmer_profiles.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_n_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_n_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_sequence_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_sequence_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_gc_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_gc_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_tile_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_tile_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/sequence_length_distribution.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/sequence_length_distribution.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/summary.txt' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/summary.txt'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc.fo'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_data.txt'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/error.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/fastqc_icon.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/tick.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons/warning.png'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Icons'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/adapter_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/duplication_levels.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/kmer_profiles.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_n_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_base_sequence_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_gc_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_sequence_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/per_tile_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images/sequence_length_distribution.png'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/Images'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/summary.txt'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc.fo' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc.fo'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_data.txt' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_data.txt'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/error.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/error.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/fastqc_icon.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/fastqc_icon.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/tick.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/tick.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/warning.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/warning.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/adapter_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/adapter_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/duplication_levels.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/duplication_levels.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/kmer_profiles.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/kmer_profiles.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_n_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_n_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_sequence_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_sequence_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_gc_content.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_gc_content.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_tile_quality.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_tile_quality.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/sequence_length_distribution.png' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/sequence_length_distribution.png'\n",
"'160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/summary.txt' -> '/owl_web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/summary.txt'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc.fo'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_data.txt'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/error.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/fastqc_icon.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/tick.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons/warning.png'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Icons'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/adapter_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/duplication_levels.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/kmer_profiles.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_n_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_base_sequence_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_gc_content.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_sequence_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/per_tile_quality.png'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images/sequence_length_distribution.png'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/Images'\n",
"removed '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/summary.txt'\n",
"removed directory: '160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/'\n"
]
}
],
"source": [
"%%bash\n",
"mv -v 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/ \\\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/ \\\n",
"-t /owl_web/nightingales/O_lurida/20160223_gbs/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### View FastQC Reports\n",
"\n",
"Easier viewing can be had by visiting the two folling links:\n",
"\n",
"- [20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc](http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html)\n",
"\n",
"- [20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc](http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<iframe width=\"100%\" height=\"500\" src=\"http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html\"></iframe>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%HTML\n",
"<iframe width=\"100%\" height=\"500\" src=\"http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/fastqc_report.html\"></iframe>"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<iframe width=\"100%\" height=\"500\" src=\"http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html\"></iframe>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%%HTML\n",
"<iframe width=\"100%\" height=\"500\" src=\"http://owl.fish.washington.edu/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/fastqc_report.html\"></iframe>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These results aren't terribly surprising since these data haven't been quality trimmed to remove adapter/barcode sequences."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/data/20170301_fastqc_gbs\n"
]
}
],
"source": [
"cd /data/20170301_fastqc_gbs/"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.html\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.zip\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.html\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.zip\r\n",
"temp_checksums.md5\r\n"
]
}
],
"source": [
"ls"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Forgot to move the .html files!"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%bash\n",
"mv *.html /owl_web/nightingales/O_lurida/20160223_gbs/"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.zip\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz\r\n",
"160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.zip\r\n",
"temp_checksums.md5\r\n"
]
}
],
"source": [
"ls"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Clean up."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/data\n"
]
}
],
"source": [
"cd /data/"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[0m\u001b[01;34m20161117_oly_gbs_vcf_analysis\u001b[0m/ test_file10.txt.gz test_file6.txt.gz\r\n",
"\u001b[01;34m20170227_jay_data_tmp\u001b[0m/ test_file2.txt.gz test_file7.txt.gz\r\n",
"\u001b[01;34m20170301_fastqc_gbs\u001b[0m/ test_file3.txt.gz test_file8.txt.gz\r\n",
"md5s.txt test_file4.txt.gz test_file9.txt.gz\r\n",
"test_file1.txt.gz test_file5.txt.gz wgetrc_berk_seq\r\n"
]
}
],
"source": [
"ls"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%bash\n",
"rm -rf 20170301_fastqc_gbs/"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[0m\u001b[01;34m20161117_oly_gbs_vcf_analysis\u001b[0m/ test_file2.txt.gz test_file7.txt.gz\r\n",
"\u001b[01;34m20170227_jay_data_tmp\u001b[0m/ test_file3.txt.gz test_file8.txt.gz\r\n",
"md5s.txt test_file4.txt.gz test_file9.txt.gz\r\n",
"test_file1.txt.gz test_file5.txt.gz wgetrc_berk_seq\r\n",
"test_file10.txt.gz test_file6.txt.gz\r\n"
]
}
],
"source": [
"ls"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python [default]",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
}