{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### Running in Docker container on EC2 instance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Started Docker container with the following command:\n", "\n", "```docker run -p 8888:8888 -v /home/ubuntu/gitrepos/LabDocs/jupyter_nbs/sam:/home/notebooks -v /home/ubuntu/data:/data/ -it kubu4/bioinformatics:v11 /bin/bash```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The command allows access to Jupyter Notebook over port 8888 and makes my Jupyter Notebook GitHub repo and my data files accessible to the Docker container" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the container was started, started Jupyter Notebook with the following command inside the Docker container:\n", "\n", "```jupyter notebook```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is configured in the Docker container to launch a Jupyter Notebook without a browser on port 8888." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Created a tunnel from my local computer to the Docker container:\n", "\n", "```ssh -i ~/Dropbox/Lab/Sam/bioinformatics.pem -N -L localhost:8888:localhost:8888 ubuntu@ec2.ip.address```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This command is run in a separate Terminal window than the one that is used to ssh into the EC2 instance to start Docker and all of that.\n", "\n", "This ssh command specifies to use my Amazon EC2 authentication file (bioinformatics.pem), along with the -N and -L options for port forwarding stuff (see man ssh for deets), and binds the port 8888 on my local computer to port 8888 on the EC2 isntance. \n", "\n", "The tunnel allows me to start the Jupyter Notebook in my web browser. I enter ```localhost:8888``` in as the URL." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Thu Jul 14 21:17:15 UTC 2016\n" ] } ], "source": [ "%%bash\n", "date" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "570c28713283\n" ] } ], "source": [ "%%bash\n", "hostname" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Check computer specs" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Architecture: x86_64\n", "CPU op-mode(s): 32-bit, 64-bit\n", "Byte Order: Little Endian\n", "CPU(s): 8\n", "On-line CPU(s) list: 0-7\n", "Thread(s) per core: 2\n", "Core(s) per socket: 4\n", "Socket(s): 1\n", "NUMA node(s): 1\n", "Vendor ID: GenuineIntel\n", "CPU family: 6\n", "Model: 63\n", "Model name: Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz\n", "Stepping: 2\n", "CPU MHz: 2900.106\n", "BogoMIPS: 5800.21\n", "Hypervisor vendor: Xen\n", "Virtualization type: full\n", "L1d cache: 32K\n", "L1i cache: 32K\n", "L2 cache: 256K\n", "L3 cache: 25600K\n", "NUMA node0 CPU(s): 0-7\n" ] } ], "source": [ "%%bash\n", "lscpu" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data\n" ] } ], "source": [ "cd /data/" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%%bash\n", "mkdir stacks" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%%bash\n", "mkdir /data/stacks/radtags_out" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data\n" ] } ], "source": [ "cd /data" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz \u001b[0m\u001b[01;32m1NF_25A_1.fq.gz\u001b[0m*\r\n", "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz \u001b[01;32m1NF_25A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_35A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_3A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_3A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_4A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_4A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_5A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_5A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_6A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_6A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_7A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_7A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_8A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_8A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_9A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_9A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_10A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_10A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_11A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_11A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_12A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_12A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_13A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_13A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_14A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_14A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_15A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_15A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_16A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_16A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_17A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_17A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_18A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_18A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_19A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_19A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_1A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_1A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_20A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_20A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_21A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_21A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_22A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_22A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_23A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_23A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1NF_24A_1.fq.gz\u001b[0m* \u001b[01;34mstacks\u001b[0m/\r\n", "\u001b[01;32m1NF_24A_2.fq.gz\u001b[0m*\r\n" ] } ], "source": [ "ls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Run ```process_radtags```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Cell below creates two lists (one list for each individual pair of FASTQs), loops through each as an array and assigns the corresponding FASTQ pairs to the variables \"i\" and \"j\". The values in \"i\" and \"j\" are used in the process_radtags command for matching the paired FASTQ files.\n", "\n", "Trims sequences to 90 bases (I believe uniform sequence length is needed for later Stacks steps)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\n", "real\t61m18.225s\n", "user\t50m30.730s\n", "sys\t0m7.866s\n" ] } ], "source": [ "%%bash\n", "seq1=( 1HL_10A_1.fq.gz 1HL_11A_1.fq.gz 1HL_12A_1.fq.gz 1HL_13A_1.fq.gz 1HL_14A_1.fq.gz 1HL_15A_1.fq.gz 1HL_16A_1.fq.gz 1HL_17A_1.fq.gz 1HL_19A_1.fq.gz 1HL_1A_1.fq.gz 1HL_20A_1.fq.gz 1HL_21A_1.fq.gz 1HL_22A_1.fq.gz 1HL_23A_1.fq.gz 1HL_24A_1.fq.gz 1HL_25A_1.fq.gz 1HL_26A_1.fq.gz 1HL_27A_1.fq.gz 1HL_28A_1.fq.gz 1HL_29A_1.fq.gz 1HL_2A_1.fq.gz 1HL_31A_1.fq.gz 1HL_33A_1.fq.gz 1HL_34A_1.fq.gz 1HL_35A_1.fq.gz 1HL_3A_1.fq.gz 1HL_4A_1.fq.gz 1HL_5A_1.fq.gz 1HL_6A_1.fq.gz 1HL_7A_1.fq.gz 1HL_8A_1.fq.gz 1HL_9A_1.fq.gz 1NF_10A_1.fq.gz 1NF_11A_1.fq.gz 1NF_12A_1.fq.gz 1NF_13A_1.fq.gz 1NF_14A_1.fq.gz 1NF_15A_1.fq.gz 1NF_16A_1.fq.gz 1NF_17A_1.fq.gz 1NF_18A_1.fq.gz 1NF_19A_1.fq.gz 1NF_1A_1.fq.gz 1NF_20A_1.fq.gz 1NF_21A_1.fq.gz 1NF_22A_1.fq.gz 1NF_23A_1.fq.gz 1NF_24A_1.fq.gz 1NF_25A_1.fq.gz 1NF_26A_1.fq.gz 1NF_27A_1.fq.gz 1NF_28A_1.fq.gz 1NF_29A_1.fq.gz 1NF_2A_1.fq.gz 1NF_30A_1.fq.gz 1NF_31A_1.fq.gz 1NF_32A_1.fq.gz 1NF_33A_1.fq.gz 1NF_4A_1.fq.gz 1NF_5A_1.fq.gz 1NF_6A_1.fq.gz 1NF_7A_1.fq.gz 1NF_8A_1.fq.gz 1NF_9A_1.fq.gz 1SN_10A_1.fq.gz 1SN_11A_1.fq.gz 1SN_12A_1.fq.gz 1SN_13A_1.fq.gz 1SN_14A_1.fq.gz 1SN_15A_1.fq.gz 1SN_16A_1.fq.gz 1SN_17A_1.fq.gz 1SN_18A_1.fq.gz 1SN_19A_1.fq.gz 1SN_1A_1.fq.gz 1SN_20A_1.fq.gz 1SN_21A_1.fq.gz 1SN_22A_1.fq.gz 1SN_23A_1.fq.gz 1SN_24A_1.fq.gz 1SN_25A_1.fq.gz 1SN_26A_1.fq.gz 1SN_27A_1.fq.gz 1SN_28A_1.fq.gz 1SN_29A_1.fq.gz 1SN_2A_1.fq.gz 1SN_30A_1.fq.gz 1SN_31A_1.fq.gz 1SN_32A_1.fq.gz 1SN_3A_1.fq.gz 1SN_4A_1.fq.gz 1SN_5A_1.fq.gz 1SN_6A_1.fq.gz 1SN_7A_1.fq.gz 1SN_8A_1.fq.gz 1SN_9A_1.fq.gz )\n", "seq2=( 1HL_10A_2.fq.gz 1HL_11A_2.fq.gz 1HL_12A_2.fq.gz 1HL_13A_2.fq.gz 1HL_14A_2.fq.gz 1HL_15A_2.fq.gz 1HL_16A_2.fq.gz 1HL_17A_2.fq.gz 1HL_19A_2.fq.gz 1HL_1A_2.fq.gz 1HL_20A_2.fq.gz 1HL_21A_2.fq.gz 1HL_22A_2.fq.gz 1HL_23A_2.fq.gz 1HL_24A_2.fq.gz 1HL_25A_2.fq.gz 1HL_26A_2.fq.gz 1HL_27A_2.fq.gz 1HL_28A_2.fq.gz 1HL_29A_2.fq.gz 1HL_2A_2.fq.gz 1HL_31A_2.fq.gz 1HL_33A_2.fq.gz 1HL_34A_2.fq.gz 1HL_35A_2.fq.gz 1HL_3A_2.fq.gz 1HL_4A_2.fq.gz 1HL_5A_2.fq.gz 1HL_6A_2.fq.gz 1HL_7A_2.fq.gz 1HL_8A_2.fq.gz 1HL_9A_2.fq.gz 1NF_10A_2.fq.gz 1NF_11A_2.fq.gz 1NF_12A_2.fq.gz 1NF_13A_2.fq.gz 1NF_14A_2.fq.gz 1NF_15A_2.fq.gz 1NF_16A_2.fq.gz 1NF_17A_2.fq.gz 1NF_18A_2.fq.gz 1NF_19A_2.fq.gz 1NF_1A_2.fq.gz 1NF_20A_2.fq.gz 1NF_21A_2.fq.gz 1NF_22A_2.fq.gz 1NF_23A_2.fq.gz 1NF_24A_2.fq.gz 1NF_25A_2.fq.gz 1NF_26A_2.fq.gz 1NF_27A_2.fq.gz 1NF_28A_2.fq.gz 1NF_29A_2.fq.gz 1NF_2A_2.fq.gz 1NF_30A_2.fq.gz 1NF_31A_2.fq.gz 1NF_32A_2.fq.gz 1NF_33A_2.fq.gz 1NF_4A_2.fq.gz 1NF_5A_2.fq.gz 1NF_6A_2.fq.gz 1NF_7A_2.fq.gz 1NF_8A_2.fq.gz 1NF_9A_2.fq.gz 1SN_10A_2.fq.gz 1SN_11A_2.fq.gz 1SN_12A_2.fq.gz 1SN_13A_2.fq.gz 1SN_14A_2.fq.gz 1SN_15A_2.fq.gz 1SN_16A_2.fq.gz 1SN_17A_2.fq.gz 1SN_18A_2.fq.gz 1SN_19A_2.fq.gz 1SN_1A_2.fq.gz 1SN_20A_2.fq.gz 1SN_21A_2.fq.gz 1SN_22A_2.fq.gz 1SN_23A_2.fq.gz 1SN_24A_2.fq.gz 1SN_25A_2.fq.gz 1SN_26A_2.fq.gz 1SN_27A_2.fq.gz 1SN_28A_2.fq.gz 1SN_29A_2.fq.gz 1SN_2A_2.fq.gz 1SN_30A_2.fq.gz 1SN_31A_2.fq.gz 1SN_32A_2.fq.gz 1SN_3A_2.fq.gz 1SN_4A_2.fq.gz 1SN_5A_2.fq.gz 1SN_6A_2.fq.gz 1SN_7A_2.fq.gz 1SN_8A_2.fq.gz 1SN_9A_2.fq.gz )\n", "time for pair in \"${!seq1[@]}\"; do\n", " i=${seq1[$pair]}\n", " j=${seq2[$pair]}\n", " /usr/local/bioinformatics/stacks-1.40/process_radtags \\\n", " -1 $i \\\n", " -2 $j \\\n", " -o /data/stacks/radtags_out/ \\\n", " -e apeKI \\\n", " -c \\\n", " -q \\\n", " -i gzfastq \\\n", " -t 16 \\\n", " >>radtags.stdout 2>>radtags.stderr\n", "done" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data/stacks/radtags_out\n" ] } ], "source": [ "cd /data/stacks/radtags_out" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1HL_10A_1.1.fq.gz 1NF_10A_1.rem.1.fq.gz 1SN_10A_2.2.fq.gz\r\n", "1HL_10A_1.rem.1.fq.gz 1NF_10A_2.2.fq.gz 1SN_10A_2.rem.2.fq.gz\r\n", "1HL_10A_2.2.fq.gz 1NF_10A_2.rem.2.fq.gz 1SN_11A_1.1.fq.gz\r\n", "1HL_10A_2.rem.2.fq.gz 1NF_11A_1.1.fq.gz 1SN_11A_1.rem.1.fq.gz\r\n", "1HL_11A_1.1.fq.gz 1NF_11A_1.rem.1.fq.gz 1SN_11A_2.2.fq.gz\r\n", "1HL_11A_1.rem.1.fq.gz 1NF_11A_2.2.fq.gz 1SN_11A_2.rem.2.fq.gz\r\n", "1HL_11A_2.2.fq.gz 1NF_11A_2.rem.2.fq.gz 1SN_12A_1.1.fq.gz\r\n", "1HL_11A_2.rem.2.fq.gz 1NF_12A_1.1.fq.gz 1SN_12A_1.rem.1.fq.gz\r\n", "1HL_12A_1.1.fq.gz 1NF_12A_1.rem.1.fq.gz 1SN_12A_2.2.fq.gz\r\n", "1HL_12A_1.rem.1.fq.gz 1NF_12A_2.2.fq.gz 1SN_12A_2.rem.2.fq.gz\r\n", "1HL_12A_2.2.fq.gz 1NF_12A_2.rem.2.fq.gz 1SN_13A_1.1.fq.gz\r\n", "1HL_12A_2.rem.2.fq.gz 1NF_13A_1.1.fq.gz 1SN_13A_1.rem.1.fq.gz\r\n", "1HL_13A_1.1.fq.gz 1NF_13A_1.rem.1.fq.gz 1SN_13A_2.2.fq.gz\r\n", "1HL_13A_1.rem.1.fq.gz 1NF_13A_2.2.fq.gz 1SN_13A_2.rem.2.fq.gz\r\n", "1HL_13A_2.2.fq.gz 1NF_13A_2.rem.2.fq.gz 1SN_14A_1.1.fq.gz\r\n", "1HL_13A_2.rem.2.fq.gz 1NF_14A_1.1.fq.gz 1SN_14A_1.rem.1.fq.gz\r\n", "1HL_14A_1.1.fq.gz 1NF_14A_1.rem.1.fq.gz 1SN_14A_2.2.fq.gz\r\n", "1HL_14A_1.rem.1.fq.gz 1NF_14A_2.2.fq.gz 1SN_14A_2.rem.2.fq.gz\r\n", "1HL_14A_2.2.fq.gz 1NF_14A_2.rem.2.fq.gz 1SN_15A_1.1.fq.gz\r\n", "1HL_14A_2.rem.2.fq.gz 1NF_15A_1.1.fq.gz 1SN_15A_1.rem.1.fq.gz\r\n", "1HL_15A_1.1.fq.gz 1NF_15A_1.rem.1.fq.gz 1SN_15A_2.2.fq.gz\r\n", "1HL_15A_1.rem.1.fq.gz 1NF_15A_2.2.fq.gz 1SN_15A_2.rem.2.fq.gz\r\n", "1HL_15A_2.2.fq.gz 1NF_15A_2.rem.2.fq.gz 1SN_16A_1.1.fq.gz\r\n", "1HL_15A_2.rem.2.fq.gz 1NF_16A_1.1.fq.gz 1SN_16A_1.rem.1.fq.gz\r\n", "1HL_16A_1.1.fq.gz 1NF_16A_1.rem.1.fq.gz 1SN_16A_2.2.fq.gz\r\n", "1HL_16A_1.rem.1.fq.gz 1NF_16A_2.2.fq.gz 1SN_16A_2.rem.2.fq.gz\r\n", "1HL_16A_2.2.fq.gz 1NF_16A_2.rem.2.fq.gz 1SN_17A_1.1.fq.gz\r\n", "1HL_16A_2.rem.2.fq.gz 1NF_17A_1.1.fq.gz 1SN_17A_1.rem.1.fq.gz\r\n", "1HL_17A_1.1.fq.gz 1NF_17A_1.rem.1.fq.gz 1SN_17A_2.2.fq.gz\r\n", "1HL_17A_1.rem.1.fq.gz 1NF_17A_2.2.fq.gz 1SN_17A_2.rem.2.fq.gz\r\n", "1HL_17A_2.2.fq.gz 1NF_17A_2.rem.2.fq.gz 1SN_18A_1.1.fq.gz\r\n", "1HL_17A_2.rem.2.fq.gz 1NF_18A_1.1.fq.gz 1SN_18A_1.rem.1.fq.gz\r\n", "1HL_19A_1.1.fq.gz 1NF_18A_1.rem.1.fq.gz 1SN_18A_2.2.fq.gz\r\n", "1HL_19A_1.rem.1.fq.gz 1NF_18A_2.2.fq.gz 1SN_18A_2.rem.2.fq.gz\r\n", "1HL_19A_2.2.fq.gz 1NF_18A_2.rem.2.fq.gz 1SN_19A_1.1.fq.gz\r\n", "1HL_19A_2.rem.2.fq.gz 1NF_19A_1.1.fq.gz 1SN_19A_1.rem.1.fq.gz\r\n", "1HL_1A_1.1.fq.gz 1NF_19A_1.rem.1.fq.gz 1SN_19A_2.2.fq.gz\r\n", "1HL_1A_1.rem.1.fq.gz 1NF_19A_2.2.fq.gz 1SN_19A_2.rem.2.fq.gz\r\n", "1HL_1A_2.2.fq.gz 1NF_19A_2.rem.2.fq.gz 1SN_1A_1.1.fq.gz\r\n", "1HL_1A_2.rem.2.fq.gz 1NF_1A_1.1.fq.gz 1SN_1A_1.rem.1.fq.gz\r\n", "1HL_20A_1.1.fq.gz 1NF_1A_1.rem.1.fq.gz 1SN_1A_2.2.fq.gz\r\n", "1HL_20A_1.rem.1.fq.gz 1NF_1A_2.2.fq.gz 1SN_1A_2.rem.2.fq.gz\r\n", "1HL_20A_2.2.fq.gz 1NF_1A_2.rem.2.fq.gz 1SN_20A_1.1.fq.gz\r\n", "1HL_20A_2.rem.2.fq.gz 1NF_20A_1.1.fq.gz 1SN_20A_1.rem.1.fq.gz\r\n", "1HL_21A_1.1.fq.gz 1NF_20A_1.rem.1.fq.gz 1SN_20A_2.2.fq.gz\r\n", "1HL_21A_1.rem.1.fq.gz 1NF_20A_2.2.fq.gz 1SN_20A_2.rem.2.fq.gz\r\n", "1HL_21A_2.2.fq.gz 1NF_20A_2.rem.2.fq.gz 1SN_21A_1.1.fq.gz\r\n", "1HL_21A_2.rem.2.fq.gz 1NF_21A_1.1.fq.gz 1SN_21A_1.rem.1.fq.gz\r\n", "1HL_22A_1.1.fq.gz 1NF_21A_1.rem.1.fq.gz 1SN_21A_2.2.fq.gz\r\n", "1HL_22A_1.rem.1.fq.gz 1NF_21A_2.2.fq.gz 1SN_21A_2.rem.2.fq.gz\r\n", "1HL_22A_2.2.fq.gz 1NF_21A_2.rem.2.fq.gz 1SN_22A_1.1.fq.gz\r\n", "1HL_22A_2.rem.2.fq.gz 1NF_22A_1.1.fq.gz 1SN_22A_1.rem.1.fq.gz\r\n", "1HL_23A_1.1.fq.gz 1NF_22A_1.rem.1.fq.gz 1SN_22A_2.2.fq.gz\r\n", "1HL_23A_1.rem.1.fq.gz 1NF_22A_2.2.fq.gz 1SN_22A_2.rem.2.fq.gz\r\n", "1HL_23A_2.2.fq.gz 1NF_22A_2.rem.2.fq.gz 1SN_23A_1.1.fq.gz\r\n", "1HL_23A_2.rem.2.fq.gz 1NF_23A_1.1.fq.gz 1SN_23A_1.rem.1.fq.gz\r\n", "1HL_24A_1.1.fq.gz 1NF_23A_1.rem.1.fq.gz 1SN_23A_2.2.fq.gz\r\n", "1HL_24A_1.rem.1.fq.gz 1NF_23A_2.2.fq.gz 1SN_23A_2.rem.2.fq.gz\r\n", "1HL_24A_2.2.fq.gz 1NF_23A_2.rem.2.fq.gz 1SN_24A_1.1.fq.gz\r\n", "1HL_24A_2.rem.2.fq.gz 1NF_24A_1.1.fq.gz 1SN_24A_1.rem.1.fq.gz\r\n", "1HL_25A_1.1.fq.gz 1NF_24A_1.rem.1.fq.gz 1SN_24A_2.2.fq.gz\r\n", "1HL_25A_1.rem.1.fq.gz 1NF_24A_2.2.fq.gz 1SN_24A_2.rem.2.fq.gz\r\n", "1HL_25A_2.2.fq.gz 1NF_24A_2.rem.2.fq.gz 1SN_25A_1.1.fq.gz\r\n", "1HL_25A_2.rem.2.fq.gz 1NF_25A_1.1.fq.gz 1SN_25A_1.rem.1.fq.gz\r\n", "1HL_26A_1.1.fq.gz 1NF_25A_1.rem.1.fq.gz 1SN_25A_2.2.fq.gz\r\n", "1HL_26A_1.rem.1.fq.gz 1NF_25A_2.2.fq.gz 1SN_25A_2.rem.2.fq.gz\r\n", "1HL_26A_2.2.fq.gz 1NF_25A_2.rem.2.fq.gz 1SN_26A_1.1.fq.gz\r\n", "1HL_26A_2.rem.2.fq.gz 1NF_26A_1.1.fq.gz 1SN_26A_1.rem.1.fq.gz\r\n", "1HL_27A_1.1.fq.gz 1NF_26A_1.rem.1.fq.gz 1SN_26A_2.2.fq.gz\r\n", "1HL_27A_1.rem.1.fq.gz 1NF_26A_2.2.fq.gz 1SN_26A_2.rem.2.fq.gz\r\n", "1HL_27A_2.2.fq.gz 1NF_26A_2.rem.2.fq.gz 1SN_27A_1.1.fq.gz\r\n", "1HL_27A_2.rem.2.fq.gz 1NF_27A_1.1.fq.gz 1SN_27A_1.rem.1.fq.gz\r\n", "1HL_28A_1.1.fq.gz 1NF_27A_1.rem.1.fq.gz 1SN_27A_2.2.fq.gz\r\n", "1HL_28A_1.rem.1.fq.gz 1NF_27A_2.2.fq.gz 1SN_27A_2.rem.2.fq.gz\r\n", "1HL_28A_2.2.fq.gz 1NF_27A_2.rem.2.fq.gz 1SN_28A_1.1.fq.gz\r\n", "1HL_28A_2.rem.2.fq.gz 1NF_28A_1.1.fq.gz 1SN_28A_1.rem.1.fq.gz\r\n", "1HL_29A_1.1.fq.gz 1NF_28A_1.rem.1.fq.gz 1SN_28A_2.2.fq.gz\r\n", "1HL_29A_1.rem.1.fq.gz 1NF_28A_2.2.fq.gz 1SN_28A_2.rem.2.fq.gz\r\n", "1HL_29A_2.2.fq.gz 1NF_28A_2.rem.2.fq.gz 1SN_29A_1.1.fq.gz\r\n", "1HL_29A_2.rem.2.fq.gz 1NF_29A_1.1.fq.gz 1SN_29A_1.rem.1.fq.gz\r\n", "1HL_2A_1.1.fq.gz 1NF_29A_1.rem.1.fq.gz 1SN_29A_2.2.fq.gz\r\n", "1HL_2A_1.rem.1.fq.gz 1NF_29A_2.2.fq.gz 1SN_29A_2.rem.2.fq.gz\r\n", "1HL_2A_2.2.fq.gz 1NF_29A_2.rem.2.fq.gz 1SN_2A_1.1.fq.gz\r\n", "1HL_2A_2.rem.2.fq.gz 1NF_2A_1.1.fq.gz 1SN_2A_1.rem.1.fq.gz\r\n", "1HL_31A_1.1.fq.gz 1NF_2A_1.rem.1.fq.gz 1SN_2A_2.2.fq.gz\r\n", "1HL_31A_1.rem.1.fq.gz 1NF_2A_2.2.fq.gz 1SN_2A_2.rem.2.fq.gz\r\n", "1HL_31A_2.2.fq.gz 1NF_2A_2.rem.2.fq.gz 1SN_30A_1.1.fq.gz\r\n", "1HL_31A_2.rem.2.fq.gz 1NF_30A_1.1.fq.gz 1SN_30A_1.rem.1.fq.gz\r\n", "1HL_33A_1.1.fq.gz 1NF_30A_1.rem.1.fq.gz 1SN_30A_2.2.fq.gz\r\n", "1HL_33A_1.rem.1.fq.gz 1NF_30A_2.2.fq.gz 1SN_30A_2.rem.2.fq.gz\r\n", "1HL_33A_2.2.fq.gz 1NF_30A_2.rem.2.fq.gz 1SN_31A_1.1.fq.gz\r\n", "1HL_33A_2.rem.2.fq.gz 1NF_31A_1.1.fq.gz 1SN_31A_1.rem.1.fq.gz\r\n", "1HL_34A_1.1.fq.gz 1NF_31A_1.rem.1.fq.gz 1SN_31A_2.2.fq.gz\r\n", "1HL_34A_1.rem.1.fq.gz 1NF_31A_2.2.fq.gz 1SN_31A_2.rem.2.fq.gz\r\n", "1HL_34A_2.2.fq.gz 1NF_31A_2.rem.2.fq.gz 1SN_32A_1.1.fq.gz\r\n", "1HL_34A_2.rem.2.fq.gz 1NF_32A_1.1.fq.gz 1SN_32A_1.rem.1.fq.gz\r\n", "1HL_35A_1.1.fq.gz 1NF_32A_1.rem.1.fq.gz 1SN_32A_2.2.fq.gz\r\n", "1HL_35A_1.rem.1.fq.gz 1NF_32A_2.2.fq.gz 1SN_32A_2.rem.2.fq.gz\r\n", "1HL_35A_2.2.fq.gz 1NF_32A_2.rem.2.fq.gz 1SN_3A_1.1.fq.gz\r\n", "1HL_35A_2.rem.2.fq.gz 1NF_33A_1.1.fq.gz 1SN_3A_1.rem.1.fq.gz\r\n", "1HL_3A_1.1.fq.gz 1NF_33A_1.rem.1.fq.gz 1SN_3A_2.2.fq.gz\r\n", "1HL_3A_1.rem.1.fq.gz 1NF_33A_2.2.fq.gz 1SN_3A_2.rem.2.fq.gz\r\n", "1HL_3A_2.2.fq.gz 1NF_33A_2.rem.2.fq.gz 1SN_4A_1.1.fq.gz\r\n", "1HL_3A_2.rem.2.fq.gz 1NF_4A_1.1.fq.gz 1SN_4A_1.rem.1.fq.gz\r\n", "1HL_4A_1.1.fq.gz 1NF_4A_1.rem.1.fq.gz 1SN_4A_2.2.fq.gz\r\n", "1HL_4A_1.rem.1.fq.gz 1NF_4A_2.2.fq.gz 1SN_4A_2.rem.2.fq.gz\r\n", "1HL_4A_2.2.fq.gz 1NF_4A_2.rem.2.fq.gz 1SN_5A_1.1.fq.gz\r\n", "1HL_4A_2.rem.2.fq.gz 1NF_5A_1.1.fq.gz 1SN_5A_1.rem.1.fq.gz\r\n", "1HL_5A_1.1.fq.gz 1NF_5A_1.rem.1.fq.gz 1SN_5A_2.2.fq.gz\r\n", "1HL_5A_1.rem.1.fq.gz 1NF_5A_2.2.fq.gz 1SN_5A_2.rem.2.fq.gz\r\n", "1HL_5A_2.2.fq.gz 1NF_5A_2.rem.2.fq.gz 1SN_6A_1.1.fq.gz\r\n", "1HL_5A_2.rem.2.fq.gz 1NF_6A_1.1.fq.gz 1SN_6A_1.rem.1.fq.gz\r\n", "1HL_6A_1.1.fq.gz 1NF_6A_1.rem.1.fq.gz 1SN_6A_2.2.fq.gz\r\n", "1HL_6A_1.rem.1.fq.gz 1NF_6A_2.2.fq.gz 1SN_6A_2.rem.2.fq.gz\r\n", "1HL_6A_2.2.fq.gz 1NF_6A_2.rem.2.fq.gz 1SN_7A_1.1.fq.gz\r\n", "1HL_6A_2.rem.2.fq.gz 1NF_7A_1.1.fq.gz 1SN_7A_1.rem.1.fq.gz\r\n", "1HL_7A_1.1.fq.gz 1NF_7A_1.rem.1.fq.gz 1SN_7A_2.2.fq.gz\r\n", "1HL_7A_1.rem.1.fq.gz 1NF_7A_2.2.fq.gz 1SN_7A_2.rem.2.fq.gz\r\n", "1HL_7A_2.2.fq.gz 1NF_7A_2.rem.2.fq.gz 1SN_8A_1.1.fq.gz\r\n", "1HL_7A_2.rem.2.fq.gz 1NF_8A_1.1.fq.gz 1SN_8A_1.rem.1.fq.gz\r\n", "1HL_8A_1.1.fq.gz 1NF_8A_1.rem.1.fq.gz 1SN_8A_2.2.fq.gz\r\n", "1HL_8A_1.rem.1.fq.gz 1NF_8A_2.2.fq.gz 1SN_8A_2.rem.2.fq.gz\r\n", "1HL_8A_2.2.fq.gz 1NF_8A_2.rem.2.fq.gz 1SN_9A_1.1.fq.gz\r\n", "1HL_8A_2.rem.2.fq.gz 1NF_9A_1.1.fq.gz 1SN_9A_1.rem.1.fq.gz\r\n", "1HL_9A_1.1.fq.gz 1NF_9A_1.rem.1.fq.gz 1SN_9A_2.2.fq.gz\r\n", "1HL_9A_1.rem.1.fq.gz 1NF_9A_2.2.fq.gz 1SN_9A_2.rem.2.fq.gz\r\n", "1HL_9A_2.2.fq.gz 1NF_9A_2.rem.2.fq.gz process_radtags.log\r\n", "1HL_9A_2.rem.2.fq.gz 1SN_10A_1.1.fq.gz\r\n", "1NF_10A_1.1.fq.gz 1SN_10A_1.rem.1.fq.gz\r\n" ] } ], "source": [ "ls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Concatenate process_radtags output files into single FASTQ file\n", "\n", "Creates four lists of each fastq file and processes them using arrays. Uses a for loop to precess each value (i.e. index) of the four arrays is assigned to four different variables. The contents of the four variables are concatentated into a single file. The output file is named utilizing parameter substitution: ${i/_1.1/}. This takes the file stored in the variable \"i\", matches the text \"_1.1\" in that file name and replaces it with nothing \"/\" (i.e. deletes it)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\n", "real\t6m24.711s\n", "user\t0m0.089s\n", "sys\t0m6.347s\n" ] } ], "source": [ "%%bash\n", "list1=( 1HL_10A_1.1.fq.gz 1HL_11A_1.1.fq.gz 1HL_12A_1.1.fq.gz 1HL_13A_1.1.fq.gz 1HL_14A_1.1.fq.gz 1HL_15A_1.1.fq.gz 1HL_16A_1.1.fq.gz 1HL_17A_1.1.fq.gz 1HL_19A_1.1.fq.gz 1HL_1A_1.1.fq.gz 1HL_20A_1.1.fq.gz 1HL_21A_1.1.fq.gz 1HL_22A_1.1.fq.gz 1HL_23A_1.1.fq.gz 1HL_24A_1.1.fq.gz 1HL_25A_1.1.fq.gz 1HL_26A_1.1.fq.gz 1HL_27A_1.1.fq.gz 1HL_28A_1.1.fq.gz 1HL_29A_1.1.fq.gz 1HL_2A_1.1.fq.gz 1HL_31A_1.1.fq.gz 1HL_33A_1.1.fq.gz 1HL_34A_1.1.fq.gz 1HL_35A_1.1.fq.gz 1HL_3A_1.1.fq.gz 1HL_4A_1.1.fq.gz 1HL_5A_1.1.fq.gz 1HL_6A_1.1.fq.gz 1HL_7A_1.1.fq.gz 1HL_8A_1.1.fq.gz 1HL_9A_1.1.fq.gz 1NF_10A_1.1.fq.gz 1NF_11A_1.1.fq.gz 1NF_12A_1.1.fq.gz 1NF_13A_1.1.fq.gz 1NF_14A_1.1.fq.gz 1NF_15A_1.1.fq.gz 1NF_16A_1.1.fq.gz 1NF_17A_1.1.fq.gz 1NF_18A_1.1.fq.gz 1NF_19A_1.1.fq.gz 1NF_1A_1.1.fq.gz 1NF_20A_1.1.fq.gz 1NF_21A_1.1.fq.gz 1NF_22A_1.1.fq.gz 1NF_23A_1.1.fq.gz 1NF_24A_1.1.fq.gz 1NF_25A_1.1.fq.gz 1NF_26A_1.1.fq.gz 1NF_27A_1.1.fq.gz 1NF_28A_1.1.fq.gz 1NF_29A_1.1.fq.gz 1NF_2A_1.1.fq.gz 1NF_30A_1.1.fq.gz 1NF_31A_1.1.fq.gz 1NF_32A_1.1.fq.gz 1NF_33A_1.1.fq.gz 1NF_4A_1.1.fq.gz 1NF_5A_1.1.fq.gz 1NF_6A_1.1.fq.gz 1NF_7A_1.1.fq.gz 1NF_8A_1.1.fq.gz 1NF_9A_1.1.fq.gz 1SN_10A_1.1.fq.gz 1SN_11A_1.1.fq.gz 1SN_12A_1.1.fq.gz 1SN_13A_1.1.fq.gz 1SN_14A_1.1.fq.gz 1SN_15A_1.1.fq.gz 1SN_16A_1.1.fq.gz 1SN_17A_1.1.fq.gz 1SN_18A_1.1.fq.gz 1SN_19A_1.1.fq.gz 1SN_1A_1.1.fq.gz 1SN_20A_1.1.fq.gz 1SN_21A_1.1.fq.gz 1SN_22A_1.1.fq.gz 1SN_23A_1.1.fq.gz 1SN_24A_1.1.fq.gz 1SN_25A_1.1.fq.gz 1SN_26A_1.1.fq.gz 1SN_27A_1.1.fq.gz 1SN_28A_1.1.fq.gz 1SN_29A_1.1.fq.gz 1SN_2A_1.1.fq.gz 1SN_30A_1.1.fq.gz 1SN_31A_1.1.fq.gz 1SN_32A_1.1.fq.gz 1SN_3A_1.1.fq.gz 1SN_4A_1.1.fq.gz 1SN_5A_1.1.fq.gz 1SN_6A_1.1.fq.gz 1SN_7A_1.1.fq.gz 1SN_8A_1.1.fq.gz 1SN_9A_1.1.fq.gz )\n", "list2=( 1HL_10A_1.rem.1.fq.gz 1HL_11A_1.rem.1.fq.gz 1HL_12A_1.rem.1.fq.gz 1HL_13A_1.rem.1.fq.gz 1HL_14A_1.rem.1.fq.gz 1HL_15A_1.rem.1.fq.gz 1HL_16A_1.rem.1.fq.gz 1HL_17A_1.rem.1.fq.gz 1HL_19A_1.rem.1.fq.gz 1HL_1A_1.rem.1.fq.gz 1HL_20A_1.rem.1.fq.gz 1HL_21A_1.rem.1.fq.gz 1HL_22A_1.rem.1.fq.gz 1HL_23A_1.rem.1.fq.gz 1HL_24A_1.rem.1.fq.gz 1HL_25A_1.rem.1.fq.gz 1HL_26A_1.rem.1.fq.gz 1HL_27A_1.rem.1.fq.gz 1HL_28A_1.rem.1.fq.gz 1HL_29A_1.rem.1.fq.gz 1HL_2A_1.rem.1.fq.gz 1HL_31A_1.rem.1.fq.gz 1HL_33A_1.rem.1.fq.gz 1HL_34A_1.rem.1.fq.gz 1HL_35A_1.rem.1.fq.gz 1HL_3A_1.rem.1.fq.gz 1HL_4A_1.rem.1.fq.gz 1HL_5A_1.rem.1.fq.gz 1HL_6A_1.rem.1.fq.gz 1HL_7A_1.rem.1.fq.gz 1HL_8A_1.rem.1.fq.gz 1HL_9A_1.rem.1.fq.gz 1NF_10A_1.rem.1.fq.gz 1NF_11A_1.rem.1.fq.gz 1NF_12A_1.rem.1.fq.gz 1NF_13A_1.rem.1.fq.gz 1NF_14A_1.rem.1.fq.gz 1NF_15A_1.rem.1.fq.gz 1NF_16A_1.rem.1.fq.gz 1NF_17A_1.rem.1.fq.gz 1NF_18A_1.rem.1.fq.gz 1NF_19A_1.rem.1.fq.gz 1NF_1A_1.rem.1.fq.gz 1NF_20A_1.rem.1.fq.gz 1NF_21A_1.rem.1.fq.gz 1NF_22A_1.rem.1.fq.gz 1NF_23A_1.rem.1.fq.gz 1NF_24A_1.rem.1.fq.gz 1NF_25A_1.rem.1.fq.gz 1NF_26A_1.rem.1.fq.gz 1NF_27A_1.rem.1.fq.gz 1NF_28A_1.rem.1.fq.gz 1NF_29A_1.rem.1.fq.gz 1NF_2A_1.rem.1.fq.gz 1NF_30A_1.rem.1.fq.gz 1NF_31A_1.rem.1.fq.gz 1NF_32A_1.rem.1.fq.gz 1NF_33A_1.rem.1.fq.gz 1NF_4A_1.rem.1.fq.gz 1NF_5A_1.rem.1.fq.gz 1NF_6A_1.rem.1.fq.gz 1NF_7A_1.rem.1.fq.gz 1NF_8A_1.rem.1.fq.gz 1NF_9A_1.rem.1.fq.gz 1SN_10A_1.rem.1.fq.gz 1SN_11A_1.rem.1.fq.gz 1SN_12A_1.rem.1.fq.gz 1SN_13A_1.rem.1.fq.gz 1SN_14A_1.rem.1.fq.gz 1SN_15A_1.rem.1.fq.gz 1SN_16A_1.rem.1.fq.gz 1SN_17A_1.rem.1.fq.gz 1SN_18A_1.rem.1.fq.gz 1SN_19A_1.rem.1.fq.gz 1SN_1A_1.rem.1.fq.gz 1SN_20A_1.rem.1.fq.gz 1SN_21A_1.rem.1.fq.gz 1SN_22A_1.rem.1.fq.gz 1SN_23A_1.rem.1.fq.gz 1SN_24A_1.rem.1.fq.gz 1SN_25A_1.rem.1.fq.gz 1SN_26A_1.rem.1.fq.gz 1SN_27A_1.rem.1.fq.gz 1SN_28A_1.rem.1.fq.gz 1SN_29A_1.rem.1.fq.gz 1SN_2A_1.rem.1.fq.gz 1SN_30A_1.rem.1.fq.gz 1SN_31A_1.rem.1.fq.gz 1SN_32A_1.rem.1.fq.gz 1SN_3A_1.rem.1.fq.gz 1SN_4A_1.rem.1.fq.gz 1SN_5A_1.rem.1.fq.gz 1SN_6A_1.rem.1.fq.gz 1SN_7A_1.rem.1.fq.gz 1SN_8A_1.rem.1.fq.gz 1SN_9A_1.rem.1.fq.gz )\n", "list3=( 1HL_10A_2.2.fq.gz 1HL_11A_2.2.fq.gz 1HL_12A_2.2.fq.gz 1HL_13A_2.2.fq.gz 1HL_14A_2.2.fq.gz 1HL_15A_2.2.fq.gz 1HL_16A_2.2.fq.gz 1HL_17A_2.2.fq.gz 1HL_19A_2.2.fq.gz 1HL_1A_2.2.fq.gz 1HL_20A_2.2.fq.gz 1HL_21A_2.2.fq.gz 1HL_22A_2.2.fq.gz 1HL_23A_2.2.fq.gz 1HL_24A_2.2.fq.gz 1HL_25A_2.2.fq.gz 1HL_26A_2.2.fq.gz 1HL_27A_2.2.fq.gz 1HL_28A_2.2.fq.gz 1HL_29A_2.2.fq.gz 1HL_2A_2.2.fq.gz 1HL_31A_2.2.fq.gz 1HL_33A_2.2.fq.gz 1HL_34A_2.2.fq.gz 1HL_35A_2.2.fq.gz 1HL_3A_2.2.fq.gz 1HL_4A_2.2.fq.gz 1HL_5A_2.2.fq.gz 1HL_6A_2.2.fq.gz 1HL_7A_2.2.fq.gz 1HL_8A_2.2.fq.gz 1HL_9A_2.2.fq.gz 1NF_10A_2.2.fq.gz 1NF_11A_2.2.fq.gz 1NF_12A_2.2.fq.gz 1NF_13A_2.2.fq.gz 1NF_14A_2.2.fq.gz 1NF_15A_2.2.fq.gz 1NF_16A_2.2.fq.gz 1NF_17A_2.2.fq.gz 1NF_18A_2.2.fq.gz 1NF_19A_2.2.fq.gz 1NF_1A_2.2.fq.gz 1NF_20A_2.2.fq.gz 1NF_21A_2.2.fq.gz 1NF_22A_2.2.fq.gz 1NF_23A_2.2.fq.gz 1NF_24A_2.2.fq.gz 1NF_25A_2.2.fq.gz 1NF_26A_2.2.fq.gz 1NF_27A_2.2.fq.gz 1NF_28A_2.2.fq.gz 1NF_29A_2.2.fq.gz 1NF_2A_2.2.fq.gz 1NF_30A_2.2.fq.gz 1NF_31A_2.2.fq.gz 1NF_32A_2.2.fq.gz 1NF_33A_2.2.fq.gz 1NF_4A_2.2.fq.gz 1NF_5A_2.2.fq.gz 1NF_6A_2.2.fq.gz 1NF_7A_2.2.fq.gz 1NF_8A_2.2.fq.gz 1NF_9A_2.2.fq.gz 1SN_10A_2.2.fq.gz 1SN_11A_2.2.fq.gz 1SN_12A_2.2.fq.gz 1SN_13A_2.2.fq.gz 1SN_14A_2.2.fq.gz 1SN_15A_2.2.fq.gz 1SN_16A_2.2.fq.gz 1SN_17A_2.2.fq.gz 1SN_18A_2.2.fq.gz 1SN_19A_2.2.fq.gz 1SN_1A_2.2.fq.gz 1SN_20A_2.2.fq.gz 1SN_21A_2.2.fq.gz 1SN_22A_2.2.fq.gz 1SN_23A_2.2.fq.gz 1SN_24A_2.2.fq.gz 1SN_25A_2.2.fq.gz 1SN_26A_2.2.fq.gz 1SN_27A_2.2.fq.gz 1SN_28A_2.2.fq.gz 1SN_29A_2.2.fq.gz 1SN_2A_2.2.fq.gz 1SN_30A_2.2.fq.gz 1SN_31A_2.2.fq.gz 1SN_32A_2.2.fq.gz 1SN_3A_2.2.fq.gz 1SN_4A_2.2.fq.gz 1SN_5A_2.2.fq.gz 1SN_6A_2.2.fq.gz 1SN_7A_2.2.fq.gz 1SN_8A_2.2.fq.gz 1SN_9A_2.2.fq.gz )\n", "list4=( 1HL_10A_2.rem.2.fq.gz 1HL_11A_2.rem.2.fq.gz 1HL_12A_2.rem.2.fq.gz 1HL_13A_2.rem.2.fq.gz 1HL_14A_2.rem.2.fq.gz 1HL_15A_2.rem.2.fq.gz 1HL_16A_2.rem.2.fq.gz 1HL_17A_2.rem.2.fq.gz 1HL_19A_2.rem.2.fq.gz 1HL_1A_2.rem.2.fq.gz 1HL_20A_2.rem.2.fq.gz 1HL_21A_2.rem.2.fq.gz 1HL_22A_2.rem.2.fq.gz 1HL_23A_2.rem.2.fq.gz 1HL_24A_2.rem.2.fq.gz 1HL_25A_2.rem.2.fq.gz 1HL_26A_2.rem.2.fq.gz 1HL_27A_2.rem.2.fq.gz 1HL_28A_2.rem.2.fq.gz 1HL_29A_2.rem.2.fq.gz 1HL_2A_2.rem.2.fq.gz 1HL_31A_2.rem.2.fq.gz 1HL_33A_2.rem.2.fq.gz 1HL_34A_2.rem.2.fq.gz 1HL_35A_2.rem.2.fq.gz 1HL_3A_2.rem.2.fq.gz 1HL_4A_2.rem.2.fq.gz 1HL_5A_2.rem.2.fq.gz 1HL_6A_2.rem.2.fq.gz 1HL_7A_2.rem.2.fq.gz 1HL_8A_2.rem.2.fq.gz 1HL_9A_2.rem.2.fq.gz 1NF_10A_2.rem.2.fq.gz 1NF_11A_2.rem.2.fq.gz 1NF_12A_2.rem.2.fq.gz 1NF_13A_2.rem.2.fq.gz 1NF_14A_2.rem.2.fq.gz 1NF_15A_2.rem.2.fq.gz 1NF_16A_2.rem.2.fq.gz 1NF_17A_2.rem.2.fq.gz 1NF_18A_2.rem.2.fq.gz 1NF_19A_2.rem.2.fq.gz 1NF_1A_2.rem.2.fq.gz 1NF_20A_2.rem.2.fq.gz 1NF_21A_2.rem.2.fq.gz 1NF_22A_2.rem.2.fq.gz 1NF_23A_2.rem.2.fq.gz 1NF_24A_2.rem.2.fq.gz 1NF_25A_2.rem.2.fq.gz 1NF_26A_2.rem.2.fq.gz 1NF_27A_2.rem.2.fq.gz 1NF_28A_2.rem.2.fq.gz 1NF_29A_2.rem.2.fq.gz 1NF_2A_2.rem.2.fq.gz 1NF_30A_2.rem.2.fq.gz 1NF_31A_2.rem.2.fq.gz 1NF_32A_2.rem.2.fq.gz 1NF_33A_2.rem.2.fq.gz 1NF_4A_2.rem.2.fq.gz 1NF_5A_2.rem.2.fq.gz 1NF_6A_2.rem.2.fq.gz 1NF_7A_2.rem.2.fq.gz 1NF_8A_2.rem.2.fq.gz 1NF_9A_2.rem.2.fq.gz 1SN_10A_2.rem.2.fq.gz 1SN_11A_2.rem.2.fq.gz 1SN_12A_2.rem.2.fq.gz 1SN_13A_2.rem.2.fq.gz 1SN_14A_2.rem.2.fq.gz 1SN_15A_2.rem.2.fq.gz 1SN_16A_2.rem.2.fq.gz 1SN_17A_2.rem.2.fq.gz 1SN_18A_2.rem.2.fq.gz 1SN_19A_2.rem.2.fq.gz 1SN_1A_2.rem.2.fq.gz 1SN_20A_2.rem.2.fq.gz 1SN_21A_2.rem.2.fq.gz 1SN_22A_2.rem.2.fq.gz 1SN_23A_2.rem.2.fq.gz 1SN_24A_2.rem.2.fq.gz 1SN_25A_2.rem.2.fq.gz 1SN_26A_2.rem.2.fq.gz 1SN_27A_2.rem.2.fq.gz 1SN_28A_2.rem.2.fq.gz 1SN_29A_2.rem.2.fq.gz 1SN_2A_2.rem.2.fq.gz 1SN_30A_2.rem.2.fq.gz 1SN_31A_2.rem.2.fq.gz 1SN_32A_2.rem.2.fq.gz 1SN_3A_2.rem.2.fq.gz 1SN_4A_2.rem.2.fq.gz 1SN_5A_2.rem.2.fq.gz 1SN_6A_2.rem.2.fq.gz 1SN_7A_2.rem.2.fq.gz 1SN_8A_2.rem.2.fq.gz 1SN_9A_2.rem.2.fq.gz )\n", "\n", "time for index in \"${!list1[@]}\"; do\n", " i=${list1[$index]}\n", " j=${list2[$index]}\n", " k=${list3[$index]}\n", " l=${list4[$index]}\n", " cat $i $j $k $l > ${i/_1.1/}\n", " \n", "done" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### List newly concatenated fastq files" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1HL_10A.fq.gz 1HL_11A.fq.gz 1HL_12A.fq.gz 1HL_13A.fq.gz 1HL_14A.fq.gz 1HL_15A.fq.gz 1HL_16A.fq.gz 1HL_17A.fq.gz 1HL_19A.fq.gz 1HL_1A.fq.gz 1HL_20A.fq.gz 1HL_21A.fq.gz 1HL_22A.fq.gz 1HL_23A.fq.gz 1HL_24A.fq.gz 1HL_25A.fq.gz 1HL_26A.fq.gz 1HL_27A.fq.gz 1HL_28A.fq.gz 1HL_29A.fq.gz 1HL_2A.fq.gz 1HL_31A.fq.gz 1HL_33A.fq.gz 1HL_34A.fq.gz 1HL_35A.fq.gz 1HL_3A.fq.gz 1HL_4A.fq.gz 1HL_5A.fq.gz 1HL_6A.fq.gz 1HL_7A.fq.gz 1HL_8A.fq.gz 1HL_9A.fq.gz 1NF_10A.fq.gz 1NF_11A.fq.gz 1NF_12A.fq.gz 1NF_13A.fq.gz 1NF_14A.fq.gz 1NF_15A.fq.gz 1NF_16A.fq.gz 1NF_17A.fq.gz 1NF_18A.fq.gz 1NF_19A.fq.gz 1NF_1A.fq.gz 1NF_20A.fq.gz 1NF_21A.fq.gz 1NF_22A.fq.gz 1NF_23A.fq.gz 1NF_24A.fq.gz 1NF_25A.fq.gz 1NF_26A.fq.gz 1NF_27A.fq.gz 1NF_28A.fq.gz 1NF_29A.fq.gz 1NF_2A.fq.gz 1NF_30A.fq.gz 1NF_31A.fq.gz 1NF_32A.fq.gz 1NF_33A.fq.gz 1NF_4A.fq.gz 1NF_5A.fq.gz 1NF_6A.fq.gz 1NF_7A.fq.gz 1NF_8A.fq.gz 1NF_9A.fq.gz 1SN_10A.fq.gz 1SN_11A.fq.gz 1SN_12A.fq.gz 1SN_13A.fq.gz 1SN_14A.fq.gz 1SN_15A.fq.gz 1SN_16A.fq.gz 1SN_17A.fq.gz 1SN_18A.fq.gz 1SN_19A.fq.gz 1SN_1A.fq.gz 1SN_20A.fq.gz 1SN_21A.fq.gz 1SN_22A.fq.gz 1SN_23A.fq.gz 1SN_24A.fq.gz 1SN_25A.fq.gz 1SN_26A.fq.gz 1SN_27A.fq.gz 1SN_28A.fq.gz 1SN_29A.fq.gz 1SN_2A.fq.gz 1SN_30A.fq.gz 1SN_31A.fq.gz 1SN_32A.fq.gz 1SN_3A.fq.gz 1SN_4A.fq.gz 1SN_5A.fq.gz 1SN_6A.fq.gz 1SN_7A.fq.gz 1SN_8A.fq.gz 1SN_9A.fq.gz\n" ] } ], "source": [ "%%bash\n", "echo *A.fq.gz" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in *A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Jupyter kernel crased while this was running, so no output to screen. I used the top command to verify that pyrad was still running despite Jupyter being down. Let's look at some files to see what happened...\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "20150313_LSU_Oil_Spill_IndexID_Comparisons.ipynb\n", "20150316_LSU_OilSpill_Adapter_ID.ipynb\n", "20150317_LSU_OilSpill_EpinextAdaptor1_ID.ipynb\n", "20150408_Install_Bismark_bisulfite_mapper.ipynb\n", "20150414_C_gigas_Larvae_OA_Trimmomatic_FASTQC.ipynb\n", "20150414_C_virginica_LSU_Oil_Spill_Trimmomatic_FASTQC.ipynb\n", "20150429_Gigas_larvae_OA_BLASTn.ipynb\n", "20150501_Cgigas_larvae_OA_BLASTn_nt.ipynb\n", "20150506_Cgigas_larvae_OA_trimmomatic_FASTQC.ipynb\n", "20150521_Cgigas_larvae_OA_Trimmomatic_FASTQC.ipynb\n", "20160114_wasted_space_synologies.ipynb\n", "20160126_Olurida_BGI_data_handling.ipynb\n", "20160126_Pgenerosa_BGI_data_handling.ipynb\n", "20160203_Olurida_Zymo_Data_Handling.ipynb\n", "20160308_find_rename_2bRAD_undetermined_fastqs.ipynb\n", "20160314_Olurida_GBS_data_management.ipynb\n", "20160406_Oly_GBS_STACKS.ipynb\n", "20160406_STACKS_install.ipynb\n", "20160411_Concatenate_Oly_MBDseq.ipynb\n", "20160418_Oly_GBS_PE-Pyrad_populations.ipynb\n", "20160418_pyrad_oly_PE-GBS.ipynb\n", "20160427_Oly_GBS_data_management.ipynb\n", "20160427_speed_comparison.ipynb\n", "20160428_Oly_GBS_STACKS.ipynb\n", "20160502_Oly_GBS_barcode_repair.ipynb\n", "20160516_Oly_Small_Insert_Library_Genome_Read_Counts.ipynb\n", "20160523_Oly_GBS_Stacks.ipynb\n", "20160525_pyrad_oly_gbs_bgi.ipynb\n", "20160609_pyrad_oly_gbs_bgi.ipynb\n", "20160714_EC2_Oly_GBS_stacks_analysis.ipynb\n", "20160715_ec2_oly_gbs_pyrad.ipynb\n", "\u001b[0m\u001b[01;32mInstallingBLAST.ipynb\u001b[0m*\n", "PE-GBS_empirical.ipynb\n", "\u001b[01;32mPhageNGS.ipynb\u001b[0m*\n", "\u001b[01;32mPhageNGS_ID.ipynb\u001b[0m*\n", "README.md\n", "Untitled.ipynb\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data/stacks\n" ] } ], "source": [ "cd /data/stacks/" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;34mradtags_out\u001b[0m/\r\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data\n" ] } ], "source": [ "cd .." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_35A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_2.fq.gz\u001b[0m* \u001b[01;34manalysis\u001b[0m/\r\n", "\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_1.fq.gz\u001b[0m* radtags.stderr\r\n", "\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_2.fq.gz\u001b[0m* radtags.stdout\r\n", "\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_1.fq.gz\u001b[0m* \u001b[01;34mstacks\u001b[0m/\r\n" ] } ], "source": [ "ls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Guess it didn't complete..." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data\n" ] } ], "source": [ "cd /data/" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '*A.fq.gz': No such file or directory.\n", "\n", "real\t0m0.002s\n", "user\t0m0.004s\n", "sys\t0m0.001s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in *A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_35A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\r\n", "\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_2.fq.gz\u001b[0m* \u001b[01;34manalysis\u001b[0m/\r\n", "\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_1.fq.gz\u001b[0m* radtags.stderr\r\n", "\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_2.fq.gz\u001b[0m* radtags.stdout\r\n", "\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_1.fq.gz\u001b[0m* \u001b[01;34mstacks\u001b[0m/\r\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '*A.fq.gz': No such file or directory.\n", "\n", "real\t0m0.001s\n", "user\t0m0.000s\n", "sys\t0m0.002s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in \"*A.fq.gz\"; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '*A.fq.gz': No such file or directory.\n", "\n", "real\t0m0.002s\n", "user\t0m0.000s\n", "sys\t0m0.002s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in *A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '*A.fq.gz': No such file or directory.\n", "\n", "real\t0m0.002s\n", "user\t0m0.000s\n", "sys\t0m0.001s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in *A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "Something weird was going on. Got an error message in the PyRad notebook I was running (see my [notebook entry](http://onsnetwork.org/kubu4/2016/07/17/computing-amazon-ec2-instance-out-of-space/) about being out of space. Maybe that's what's causing the issue here and preventing me from re-running this analysis? I've since expanded the storage space of this Amazon EC2 Instance. Let's try again..." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mon Jul 18 15:07:09 UTC 2016\n" ] } ], "source": [ "%%bash\n", "date" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data\n" ] } ], "source": [ "cd /data/" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_35A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_26A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_3A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_27A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_4A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_28A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_5A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_29A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_6A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_2A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_7A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_30A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_8A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_31A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m* \u001b[01;32m1HL_9A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_32A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_10A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_33A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_11A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_4A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_12A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_5A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_13A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_6A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_14A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_7A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_15A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_8A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_16A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_9A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_17A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_10A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_18A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_11A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_19A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_12A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_1A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_13A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_20A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_14A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_21A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_15A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_22A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_16A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_23A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\n", "\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_17A_2.fq.gz\u001b[0m* \u001b[01;34manalysis\u001b[0m/\n", "\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_24A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_1.fq.gz\u001b[0m* radtags.stderr\n", "\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_1.fq.gz\u001b[0m* \u001b[01;32m1SN_18A_2.fq.gz\u001b[0m* radtags.stdout\n", "\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m* \u001b[01;32m1NF_25A_2.fq.gz\u001b[0m* \u001b[01;32m1SN_19A_1.fq.gz\u001b[0m* \u001b[01;34mstacks\u001b[0m/\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '*A.fq.gz': No such file or directory.\n", "\n", "real\t0m4.045s\n", "user\t0m0.000s\n", "sys\t0m0.002s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in *A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Why do I keep getting this error???!! Try this with a different command..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "OMG... It's because the command isn't set to look in the correct directory! Duh! Grrrrr!!!!\n", "\n", "A good example of why I should always use absolute paths for everything..." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ustacks paramters selected:\n", " Min depth of coverage to create a stack: 3\n", " Max distance allowed between stacks: 2\n", " Max distance allowed to align secondary reads: 4\n", " Max number of stacks allowed per de novo locus: 3\n", " Deleveraging algorithm: enabled\n", " Removal algorithm: enabled\n", " Model type: SNP\n", " Alpha significance level for model: 0.05\n", " Gapped alignments: disabled\n", "Failed to open gzipped file '/data/*A.fq.gz': No such file or directory.\n", "\n", "real\t0m2.260s\n", "user\t0m0.000s\n", "sys\t0m0.003s\n" ] } ], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in /data/*A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Brain dead. Yeesh. I did it again! The data I want is NOT in the location listed in the above command! It's here:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### List newly concatenated fastq files" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1HL_10A.fq.gz 1NF_10A_1.1.fq.gz 1SN_10A_1.rem.1.fq.gz\r\n", "1HL_10A_1.1.fq.gz 1NF_10A_1.rem.1.fq.gz 1SN_10A_2.2.fq.gz\r\n", "1HL_10A_1.rem.1.fq.gz 1NF_10A_2.2.fq.gz 1SN_10A_2.rem.2.fq.gz\r\n", "1HL_10A_2.2.fq.gz 1NF_10A_2.rem.2.fq.gz 1SN_11A.fq.gz\r\n", "1HL_10A_2.rem.2.fq.gz 1NF_11A.fq.gz 1SN_11A_1.1.fq.gz\r\n", "1HL_11A.fq.gz 1NF_11A_1.1.fq.gz 1SN_11A_1.rem.1.fq.gz\r\n", "1HL_11A_1.1.fq.gz 1NF_11A_1.rem.1.fq.gz 1SN_11A_2.2.fq.gz\r\n", "1HL_11A_1.rem.1.fq.gz 1NF_11A_2.2.fq.gz 1SN_11A_2.rem.2.fq.gz\r\n", "1HL_11A_2.2.fq.gz 1NF_11A_2.rem.2.fq.gz 1SN_12A.fq.gz\r\n", "1HL_11A_2.rem.2.fq.gz 1NF_12A.fq.gz 1SN_12A_1.1.fq.gz\r\n", "1HL_12A.fq.gz 1NF_12A_1.1.fq.gz 1SN_12A_1.rem.1.fq.gz\r\n", "1HL_12A_1.1.fq.gz 1NF_12A_1.rem.1.fq.gz 1SN_12A_2.2.fq.gz\r\n", "1HL_12A_1.rem.1.fq.gz 1NF_12A_2.2.fq.gz 1SN_12A_2.rem.2.fq.gz\r\n", "1HL_12A_2.2.fq.gz 1NF_12A_2.rem.2.fq.gz 1SN_13A.fq.gz\r\n", "1HL_12A_2.rem.2.fq.gz 1NF_13A.fq.gz 1SN_13A_1.1.fq.gz\r\n", "1HL_13A.fq.gz 1NF_13A_1.1.fq.gz 1SN_13A_1.rem.1.fq.gz\r\n", "1HL_13A_1.1.fq.gz 1NF_13A_1.rem.1.fq.gz 1SN_13A_2.2.fq.gz\r\n", "1HL_13A_1.rem.1.fq.gz 1NF_13A_2.2.fq.gz 1SN_13A_2.rem.2.fq.gz\r\n", "1HL_13A_2.2.fq.gz 1NF_13A_2.rem.2.fq.gz 1SN_14A.fq.gz\r\n", "1HL_13A_2.rem.2.fq.gz 1NF_14A.fq.gz 1SN_14A_1.1.fq.gz\r\n", "1HL_14A.fq.gz 1NF_14A_1.1.fq.gz 1SN_14A_1.rem.1.fq.gz\r\n", "1HL_14A_1.1.fq.gz 1NF_14A_1.rem.1.fq.gz 1SN_14A_2.2.fq.gz\r\n", "1HL_14A_1.rem.1.fq.gz 1NF_14A_2.2.fq.gz 1SN_14A_2.rem.2.fq.gz\r\n", "1HL_14A_2.2.fq.gz 1NF_14A_2.rem.2.fq.gz 1SN_15A.fq.gz\r\n", "1HL_14A_2.rem.2.fq.gz 1NF_15A.fq.gz 1SN_15A_1.1.fq.gz\r\n", "1HL_15A.fq.gz 1NF_15A_1.1.fq.gz 1SN_15A_1.rem.1.fq.gz\r\n", "1HL_15A_1.1.fq.gz 1NF_15A_1.rem.1.fq.gz 1SN_15A_2.2.fq.gz\r\n", "1HL_15A_1.rem.1.fq.gz 1NF_15A_2.2.fq.gz 1SN_15A_2.rem.2.fq.gz\r\n", "1HL_15A_2.2.fq.gz 1NF_15A_2.rem.2.fq.gz 1SN_16A.fq.gz\r\n", "1HL_15A_2.rem.2.fq.gz 1NF_16A.fq.gz 1SN_16A_1.1.fq.gz\r\n", "1HL_16A.fq.gz 1NF_16A_1.1.fq.gz 1SN_16A_1.rem.1.fq.gz\r\n", "1HL_16A_1.1.fq.gz 1NF_16A_1.rem.1.fq.gz 1SN_16A_2.2.fq.gz\r\n", "1HL_16A_1.rem.1.fq.gz 1NF_16A_2.2.fq.gz 1SN_16A_2.rem.2.fq.gz\r\n", "1HL_16A_2.2.fq.gz 1NF_16A_2.rem.2.fq.gz 1SN_17A.fq.gz\r\n", "1HL_16A_2.rem.2.fq.gz 1NF_17A.fq.gz 1SN_17A_1.1.fq.gz\r\n", "1HL_17A.fq.gz 1NF_17A_1.1.fq.gz 1SN_17A_1.rem.1.fq.gz\r\n", "1HL_17A_1.1.fq.gz 1NF_17A_1.rem.1.fq.gz 1SN_17A_2.2.fq.gz\r\n", "1HL_17A_1.rem.1.fq.gz 1NF_17A_2.2.fq.gz 1SN_17A_2.rem.2.fq.gz\r\n", "1HL_17A_2.2.fq.gz 1NF_17A_2.rem.2.fq.gz 1SN_18A.fq.gz\r\n", "1HL_17A_2.rem.2.fq.gz 1NF_18A.fq.gz 1SN_18A_1.1.fq.gz\r\n", "1HL_19A.fq.gz 1NF_18A_1.1.fq.gz 1SN_18A_1.rem.1.fq.gz\r\n", "1HL_19A_1.1.fq.gz 1NF_18A_1.rem.1.fq.gz 1SN_18A_2.2.fq.gz\r\n", "1HL_19A_1.rem.1.fq.gz 1NF_18A_2.2.fq.gz 1SN_18A_2.rem.2.fq.gz\r\n", "1HL_19A_2.2.fq.gz 1NF_18A_2.rem.2.fq.gz 1SN_19A.fq.gz\r\n", "1HL_19A_2.rem.2.fq.gz 1NF_19A.fq.gz 1SN_19A_1.1.fq.gz\r\n", "1HL_1A.fq.gz 1NF_19A_1.1.fq.gz 1SN_19A_1.rem.1.fq.gz\r\n", "1HL_1A_1.1.fq.gz 1NF_19A_1.rem.1.fq.gz 1SN_19A_2.2.fq.gz\r\n", "1HL_1A_1.rem.1.fq.gz 1NF_19A_2.2.fq.gz 1SN_19A_2.rem.2.fq.gz\r\n", "1HL_1A_2.2.fq.gz 1NF_19A_2.rem.2.fq.gz 1SN_1A.fq.gz\r\n", "1HL_1A_2.rem.2.fq.gz 1NF_1A.fq.gz 1SN_1A_1.1.fq.gz\r\n", "1HL_20A.fq.gz 1NF_1A_1.1.fq.gz 1SN_1A_1.rem.1.fq.gz\r\n", "1HL_20A_1.1.fq.gz 1NF_1A_1.rem.1.fq.gz 1SN_1A_2.2.fq.gz\r\n", "1HL_20A_1.rem.1.fq.gz 1NF_1A_2.2.fq.gz 1SN_1A_2.rem.2.fq.gz\r\n", "1HL_20A_2.2.fq.gz 1NF_1A_2.rem.2.fq.gz 1SN_20A.fq.gz\r\n", "1HL_20A_2.rem.2.fq.gz 1NF_20A.fq.gz 1SN_20A_1.1.fq.gz\r\n", "1HL_21A.fq.gz 1NF_20A_1.1.fq.gz 1SN_20A_1.rem.1.fq.gz\r\n", "1HL_21A_1.1.fq.gz 1NF_20A_1.rem.1.fq.gz 1SN_20A_2.2.fq.gz\r\n", "1HL_21A_1.rem.1.fq.gz 1NF_20A_2.2.fq.gz 1SN_20A_2.rem.2.fq.gz\r\n", "1HL_21A_2.2.fq.gz 1NF_20A_2.rem.2.fq.gz 1SN_21A.fq.gz\r\n", "1HL_21A_2.rem.2.fq.gz 1NF_21A.fq.gz 1SN_21A_1.1.fq.gz\r\n", "1HL_22A.fq.gz 1NF_21A_1.1.fq.gz 1SN_21A_1.rem.1.fq.gz\r\n", "1HL_22A_1.1.fq.gz 1NF_21A_1.rem.1.fq.gz 1SN_21A_2.2.fq.gz\r\n", "1HL_22A_1.rem.1.fq.gz 1NF_21A_2.2.fq.gz 1SN_21A_2.rem.2.fq.gz\r\n", "1HL_22A_2.2.fq.gz 1NF_21A_2.rem.2.fq.gz 1SN_22A.fq.gz\r\n", "1HL_22A_2.rem.2.fq.gz 1NF_22A.fq.gz 1SN_22A_1.1.fq.gz\r\n", "1HL_23A.fq.gz 1NF_22A_1.1.fq.gz 1SN_22A_1.rem.1.fq.gz\r\n", "1HL_23A_1.1.fq.gz 1NF_22A_1.rem.1.fq.gz 1SN_22A_2.2.fq.gz\r\n", "1HL_23A_1.rem.1.fq.gz 1NF_22A_2.2.fq.gz 1SN_22A_2.rem.2.fq.gz\r\n", "1HL_23A_2.2.fq.gz 1NF_22A_2.rem.2.fq.gz 1SN_23A.fq.gz\r\n", "1HL_23A_2.rem.2.fq.gz 1NF_23A.fq.gz 1SN_23A_1.1.fq.gz\r\n", "1HL_24A.fq.gz 1NF_23A_1.1.fq.gz 1SN_23A_1.rem.1.fq.gz\r\n", "1HL_24A_1.1.fq.gz 1NF_23A_1.rem.1.fq.gz 1SN_23A_2.2.fq.gz\r\n", "1HL_24A_1.rem.1.fq.gz 1NF_23A_2.2.fq.gz 1SN_23A_2.rem.2.fq.gz\r\n", "1HL_24A_2.2.fq.gz 1NF_23A_2.rem.2.fq.gz 1SN_24A.fq.gz\r\n", "1HL_24A_2.rem.2.fq.gz 1NF_24A.fq.gz 1SN_24A_1.1.fq.gz\r\n", "1HL_25A.fq.gz 1NF_24A_1.1.fq.gz 1SN_24A_1.rem.1.fq.gz\r\n", "1HL_25A_1.1.fq.gz 1NF_24A_1.rem.1.fq.gz 1SN_24A_2.2.fq.gz\r\n", "1HL_25A_1.rem.1.fq.gz 1NF_24A_2.2.fq.gz 1SN_24A_2.rem.2.fq.gz\r\n", "1HL_25A_2.2.fq.gz 1NF_24A_2.rem.2.fq.gz 1SN_25A.fq.gz\r\n", "1HL_25A_2.rem.2.fq.gz 1NF_25A.fq.gz 1SN_25A_1.1.fq.gz\r\n", "1HL_26A.fq.gz 1NF_25A_1.1.fq.gz 1SN_25A_1.rem.1.fq.gz\r\n", "1HL_26A_1.1.fq.gz 1NF_25A_1.rem.1.fq.gz 1SN_25A_2.2.fq.gz\r\n", "1HL_26A_1.rem.1.fq.gz 1NF_25A_2.2.fq.gz 1SN_25A_2.rem.2.fq.gz\r\n", "1HL_26A_2.2.fq.gz 1NF_25A_2.rem.2.fq.gz 1SN_26A.fq.gz\r\n", "1HL_26A_2.rem.2.fq.gz 1NF_26A.fq.gz 1SN_26A_1.1.fq.gz\r\n", "1HL_27A.fq.gz 1NF_26A_1.1.fq.gz 1SN_26A_1.rem.1.fq.gz\r\n", "1HL_27A_1.1.fq.gz 1NF_26A_1.rem.1.fq.gz 1SN_26A_2.2.fq.gz\r\n", "1HL_27A_1.rem.1.fq.gz 1NF_26A_2.2.fq.gz 1SN_26A_2.rem.2.fq.gz\r\n", "1HL_27A_2.2.fq.gz 1NF_26A_2.rem.2.fq.gz 1SN_27A.fq.gz\r\n", "1HL_27A_2.rem.2.fq.gz 1NF_27A.fq.gz 1SN_27A_1.1.fq.gz\r\n", "1HL_28A.fq.gz 1NF_27A_1.1.fq.gz 1SN_27A_1.rem.1.fq.gz\r\n", "1HL_28A_1.1.fq.gz 1NF_27A_1.rem.1.fq.gz 1SN_27A_2.2.fq.gz\r\n", "1HL_28A_1.rem.1.fq.gz 1NF_27A_2.2.fq.gz 1SN_27A_2.rem.2.fq.gz\r\n", "1HL_28A_2.2.fq.gz 1NF_27A_2.rem.2.fq.gz 1SN_28A.fq.gz\r\n", "1HL_28A_2.rem.2.fq.gz 1NF_28A.fq.gz 1SN_28A_1.1.fq.gz\r\n", "1HL_29A.fq.gz 1NF_28A_1.1.fq.gz 1SN_28A_1.rem.1.fq.gz\r\n", "1HL_29A_1.1.fq.gz 1NF_28A_1.rem.1.fq.gz 1SN_28A_2.2.fq.gz\r\n", "1HL_29A_1.rem.1.fq.gz 1NF_28A_2.2.fq.gz 1SN_28A_2.rem.2.fq.gz\r\n", "1HL_29A_2.2.fq.gz 1NF_28A_2.rem.2.fq.gz 1SN_29A.fq.gz\r\n", "1HL_29A_2.rem.2.fq.gz 1NF_29A.fq.gz 1SN_29A_1.1.fq.gz\r\n", "1HL_2A.fq.gz 1NF_29A_1.1.fq.gz 1SN_29A_1.rem.1.fq.gz\r\n", "1HL_2A_1.1.fq.gz 1NF_29A_1.rem.1.fq.gz 1SN_29A_2.2.fq.gz\r\n", "1HL_2A_1.rem.1.fq.gz 1NF_29A_2.2.fq.gz 1SN_29A_2.rem.2.fq.gz\r\n", "1HL_2A_2.2.fq.gz 1NF_29A_2.rem.2.fq.gz 1SN_2A.fq.gz\r\n", "1HL_2A_2.rem.2.fq.gz 1NF_2A.fq.gz 1SN_2A_1.1.fq.gz\r\n", "1HL_31A.fq.gz 1NF_2A_1.1.fq.gz 1SN_2A_1.rem.1.fq.gz\r\n", "1HL_31A_1.1.fq.gz 1NF_2A_1.rem.1.fq.gz 1SN_2A_2.2.fq.gz\r\n", "1HL_31A_1.rem.1.fq.gz 1NF_2A_2.2.fq.gz 1SN_2A_2.rem.2.fq.gz\r\n", "1HL_31A_2.2.fq.gz 1NF_2A_2.rem.2.fq.gz 1SN_30A.fq.gz\r\n", "1HL_31A_2.rem.2.fq.gz 1NF_30A.fq.gz 1SN_30A_1.1.fq.gz\r\n", "1HL_33A.fq.gz 1NF_30A_1.1.fq.gz 1SN_30A_1.rem.1.fq.gz\r\n", "1HL_33A_1.1.fq.gz 1NF_30A_1.rem.1.fq.gz 1SN_30A_2.2.fq.gz\r\n", "1HL_33A_1.rem.1.fq.gz 1NF_30A_2.2.fq.gz 1SN_30A_2.rem.2.fq.gz\r\n", "1HL_33A_2.2.fq.gz 1NF_30A_2.rem.2.fq.gz 1SN_31A.fq.gz\r\n", "1HL_33A_2.rem.2.fq.gz 1NF_31A.fq.gz 1SN_31A_1.1.fq.gz\r\n", "1HL_34A.fq.gz 1NF_31A_1.1.fq.gz 1SN_31A_1.rem.1.fq.gz\r\n", "1HL_34A_1.1.fq.gz 1NF_31A_1.rem.1.fq.gz 1SN_31A_2.2.fq.gz\r\n", "1HL_34A_1.rem.1.fq.gz 1NF_31A_2.2.fq.gz 1SN_31A_2.rem.2.fq.gz\r\n", "1HL_34A_2.2.fq.gz 1NF_31A_2.rem.2.fq.gz 1SN_32A.fq.gz\r\n", "1HL_34A_2.rem.2.fq.gz 1NF_32A.fq.gz 1SN_32A_1.1.fq.gz\r\n", "1HL_35A.fq.gz 1NF_32A_1.1.fq.gz 1SN_32A_1.rem.1.fq.gz\r\n", "1HL_35A_1.1.fq.gz 1NF_32A_1.rem.1.fq.gz 1SN_32A_2.2.fq.gz\r\n", "1HL_35A_1.rem.1.fq.gz 1NF_32A_2.2.fq.gz 1SN_32A_2.rem.2.fq.gz\r\n", "1HL_35A_2.2.fq.gz 1NF_32A_2.rem.2.fq.gz 1SN_3A.fq.gz\r\n", "1HL_35A_2.rem.2.fq.gz 1NF_33A.fq.gz 1SN_3A_1.1.fq.gz\r\n", "1HL_3A.fq.gz 1NF_33A_1.1.fq.gz 1SN_3A_1.rem.1.fq.gz\r\n", "1HL_3A_1.1.fq.gz 1NF_33A_1.rem.1.fq.gz 1SN_3A_2.2.fq.gz\r\n", "1HL_3A_1.rem.1.fq.gz 1NF_33A_2.2.fq.gz 1SN_3A_2.rem.2.fq.gz\r\n", "1HL_3A_2.2.fq.gz 1NF_33A_2.rem.2.fq.gz 1SN_4A.fq.gz\r\n", "1HL_3A_2.rem.2.fq.gz 1NF_4A.fq.gz 1SN_4A_1.1.fq.gz\r\n", "1HL_4A.fq.gz 1NF_4A_1.1.fq.gz 1SN_4A_1.rem.1.fq.gz\r\n", "1HL_4A_1.1.fq.gz 1NF_4A_1.rem.1.fq.gz 1SN_4A_2.2.fq.gz\r\n", "1HL_4A_1.rem.1.fq.gz 1NF_4A_2.2.fq.gz 1SN_4A_2.rem.2.fq.gz\r\n", "1HL_4A_2.2.fq.gz 1NF_4A_2.rem.2.fq.gz 1SN_5A.fq.gz\r\n", "1HL_4A_2.rem.2.fq.gz 1NF_5A.fq.gz 1SN_5A_1.1.fq.gz\r\n", "1HL_5A.fq.gz 1NF_5A_1.1.fq.gz 1SN_5A_1.rem.1.fq.gz\r\n", "1HL_5A_1.1.fq.gz 1NF_5A_1.rem.1.fq.gz 1SN_5A_2.2.fq.gz\r\n", "1HL_5A_1.rem.1.fq.gz 1NF_5A_2.2.fq.gz 1SN_5A_2.rem.2.fq.gz\r\n", "1HL_5A_2.2.fq.gz 1NF_5A_2.rem.2.fq.gz 1SN_6A.fq.gz\r\n", "1HL_5A_2.rem.2.fq.gz 1NF_6A.fq.gz 1SN_6A_1.1.fq.gz\r\n", "1HL_6A.fq.gz 1NF_6A_1.1.fq.gz 1SN_6A_1.rem.1.fq.gz\r\n", "1HL_6A_1.1.fq.gz 1NF_6A_1.rem.1.fq.gz 1SN_6A_2.2.fq.gz\r\n", "1HL_6A_1.rem.1.fq.gz 1NF_6A_2.2.fq.gz 1SN_6A_2.rem.2.fq.gz\r\n", "1HL_6A_2.2.fq.gz 1NF_6A_2.rem.2.fq.gz 1SN_7A.fq.gz\r\n", "1HL_6A_2.rem.2.fq.gz 1NF_7A.fq.gz 1SN_7A_1.1.fq.gz\r\n", "1HL_7A.fq.gz 1NF_7A_1.1.fq.gz 1SN_7A_1.rem.1.fq.gz\r\n", "1HL_7A_1.1.fq.gz 1NF_7A_1.rem.1.fq.gz 1SN_7A_2.2.fq.gz\r\n", "1HL_7A_1.rem.1.fq.gz 1NF_7A_2.2.fq.gz 1SN_7A_2.rem.2.fq.gz\r\n", "1HL_7A_2.2.fq.gz 1NF_7A_2.rem.2.fq.gz 1SN_8A.fq.gz\r\n", "1HL_7A_2.rem.2.fq.gz 1NF_8A.fq.gz 1SN_8A_1.1.fq.gz\r\n", "1HL_8A.fq.gz 1NF_8A_1.1.fq.gz 1SN_8A_1.rem.1.fq.gz\r\n", "1HL_8A_1.1.fq.gz 1NF_8A_1.rem.1.fq.gz 1SN_8A_2.2.fq.gz\r\n", "1HL_8A_1.rem.1.fq.gz 1NF_8A_2.2.fq.gz 1SN_8A_2.rem.2.fq.gz\r\n", "1HL_8A_2.2.fq.gz 1NF_8A_2.rem.2.fq.gz 1SN_9A.fq.gz\r\n", "1HL_8A_2.rem.2.fq.gz 1NF_9A.fq.gz 1SN_9A_1.1.fq.gz\r\n", "1HL_9A.fq.gz 1NF_9A_1.1.fq.gz 1SN_9A_1.rem.1.fq.gz\r\n", "1HL_9A_1.1.fq.gz 1NF_9A_1.rem.1.fq.gz 1SN_9A_2.2.fq.gz\r\n", "1HL_9A_1.rem.1.fq.gz 1NF_9A_2.2.fq.gz 1SN_9A_2.rem.2.fq.gz\r\n", "1HL_9A_2.2.fq.gz 1NF_9A_2.rem.2.fq.gz process_radtags.log\r\n", "1HL_9A_2.rem.2.fq.gz 1SN_10A.fq.gz\r\n", "1NF_10A.fq.gz 1SN_10A_1.1.fq.gz\r\n" ] } ], "source": [ "ls /data/stacks/radtags_out/" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%%bash\n", "#Runs ustacks and appends sql ID to each file for downstream analysis.\n", "sql_id=0\n", "time for i in /data/stacks/radtags_out/*A.fq.gz; do\n", " ((sql_id++))\n", " /usr/local/bioinformatics/stacks-1.40/ustacks \\\n", " -t gzfastq \\\n", " -f $i \\\n", " -o /data/stacks/ \\\n", " -i $sql_id \\\n", " -d \\\n", " -r \\\n", " -p 16\n", "done" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.12" } }, "nbformat": 4, "nbformat_minor": 0 }