{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running in Docker container on EC2 instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Started Docker container with the following command:\n",
    "\n",
    "```docker run  -p 8888:8888 -v /home/ubuntu/gitrepos/LabDocs/jupyter_nbs/sam:/home/notebooks -v /home/ubuntu/data:/data/ -it kubu4/bioinformatics:v11 /bin/bash```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The command allows access to Jupyter Notebook over port 8888 and makes my Jupyter Notebook GitHub repo and my data files accessible to the Docker container"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the container was started, started Jupyter Notebook with the following command inside the Docker container:\n",
    "\n",
    "```jupyter notebook```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is configured in the Docker container to launch a Jupyter Notebook without a browser on port 8888."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Created a tunnel from my local computer to the Docker container:\n",
    "\n",
    "```ssh -i ~/Dropbox/Lab/Sam/bioinformatics.pem -N -L localhost:8888:localhost:8888 ubuntu@ec2.ip.address```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This command is run in a separate Terminal window than the one that is used to ssh into the EC2 instance to start Docker and all of that.\n",
    "\n",
    "This ssh command specifies to use my Amazon EC2 authentication file (bioinformatics.pem), along with the -N and -L options for port forwarding stuff (see man ssh for deets), and binds the port 8888 on my local computer to port 8888 on the EC2 isntance. \n",
    "\n",
    "The tunnel allows me to start the Jupyter Notebook in my web browser. I enter ```localhost:8888``` in as the URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fri Jul 15 16:56:46 UTC 2016\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "date"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "570c28713283\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "hostname"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check computer specs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Architecture:          x86_64\n",
      "CPU op-mode(s):        32-bit, 64-bit\n",
      "Byte Order:            Little Endian\n",
      "CPU(s):                8\n",
      "On-line CPU(s) list:   0-7\n",
      "Thread(s) per core:    2\n",
      "Core(s) per socket:    4\n",
      "Socket(s):             1\n",
      "NUMA node(s):          1\n",
      "Vendor ID:             GenuineIntel\n",
      "CPU family:            6\n",
      "Model:                 63\n",
      "Model name:            Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz\n",
      "Stepping:              2\n",
      "CPU MHz:               2900.088\n",
      "BogoMIPS:              5800.17\n",
      "Hypervisor vendor:     Xen\n",
      "Virtualization type:   full\n",
      "L1d cache:             32K\n",
      "L1i cache:             32K\n",
      "L2 cache:              256K\n",
      "L3 cache:              25600K\n",
      "NUMA node0 CPU(s):     0-7\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "lscpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data\n"
     ]
    }
   ],
   "source": [
    "cd /data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz  \u001b[0m\u001b[01;32m1NF_25A_2.fq.gz\u001b[0m*\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz  \u001b[01;32m1NF_26A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_10A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_26A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_10A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_27A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_11A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_27A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_11A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_28A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_12A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_28A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_12A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_29A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_13A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_29A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_13A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_2A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_14A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_2A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_14A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_30A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_15A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_30A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_15A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_31A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_16A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_31A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_16A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_32A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_17A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_32A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_17A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_33A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_19A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_33A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_19A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_4A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_1A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1NF_4A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_1A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1NF_5A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_20A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_5A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_20A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_6A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_21A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_6A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_21A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_7A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_22A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_7A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_22A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_8A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_23A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_8A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_23A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_9A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_24A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1NF_9A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_24A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_10A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_25A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_10A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_25A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_11A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_26A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_11A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_26A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_12A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_27A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_12A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_27A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_13A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_28A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_13A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_28A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_14A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_29A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_14A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_29A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_15A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_2A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_15A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_2A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_16A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_31A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_16A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_31A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_17A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_33A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_17A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_33A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_18A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_34A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_18A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_34A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_19A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_35A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_19A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_35A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_1A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_3A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_1A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_3A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_20A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_4A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_20A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_4A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_21A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_5A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_21A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_5A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_22A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_6A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_22A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_6A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_23A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_7A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_23A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_7A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_24A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_8A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_24A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_8A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_25A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_9A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_25A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1HL_9A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_26A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_10A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_26A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_10A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_27A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_11A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_27A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_11A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_28A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_12A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_28A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_12A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_29A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_13A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_29A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_13A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_2A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_14A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_2A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_14A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_30A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_15A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_30A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_15A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_31A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_16A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_31A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_16A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_32A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_17A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_32A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_17A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_3A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_18A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_3A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_18A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_4A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_19A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_4A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_19A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_5A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_1A_1.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_5A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_1A_2.fq.gz\u001b[0m*                                             \u001b[01;32m1SN_6A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_20A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_6A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_20A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_7A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_21A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_7A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_21A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_8A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_22A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_8A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_22A_2.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_9A_1.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_23A_1.fq.gz\u001b[0m*                                            \u001b[01;32m1SN_9A_2.fq.gz\u001b[0m*\r\n",
      "\u001b[01;32m1NF_23A_2.fq.gz\u001b[0m*                                            \u001b[01;34manalysis\u001b[0m/\r\n",
      "\u001b[01;32m1NF_24A_1.fq.gz\u001b[0m*                                            radtags.stderr\r\n",
      "\u001b[01;32m1NF_24A_2.fq.gz\u001b[0m*                                            radtags.stdout\r\n",
      "\u001b[01;32m1NF_25A_1.fq.gz\u001b[0m*                                            \u001b[01;34mstacks\u001b[0m/\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rename FASTQ files to match R1 and R2 requirements for pyrad demultiplexing¶"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "mv 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "mv 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "barcodes.txt  params.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "./                     ## 1. Working directory                                 (all)\n",
      "/data/oly_gbs/*.gz             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/analysis/20160609_pyrad/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "16                     ## 7. N processors (parallel)                           (all)\n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "pairgbs                 ## 11. Datatype: rad,gbs,pairgbs,pairddrad,(others:see docs)(all)\n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "                       ## 18.opt.: loc. of de-multiplexed data                      (s2)\n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "mv /data/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "mv /data/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  barcodes.txt\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  params.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Need to edit params file to match current locations of data files. Will do outside of notebook. BRB..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "16                     ## 7. N processors (parallel)                           (all)\n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "pairgbs                 ## 11. Datatype: rad,gbs,pairgbs,pairddrad,(others:see docs)(all)\n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "                       ## 18.opt.: loc. of de-multiplexed data                      (s2)\n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View barcodes.txt file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1NF_1A\tCTCC\n",
      "1NF_2A\tTGCA\n",
      "1NF_4A\tACTA\n",
      "1NF_5A\tCAGA\n",
      "1NF_6A\tAACT\n",
      "1NF_7A\tGCGT\n",
      "1NF_8A\tCGAT\n",
      "1NF_9A\tGTAA\n",
      "1NF_10A\tAGGC\n",
      "1NF_11A\tGATC\n",
      "1NF_12A\tTCAC\n",
      "1NF_13A\tTGCGA\n",
      "1NF_14A\tCGCTT\n",
      "1NF_15A\tTCACC\n",
      "1NF_16A\tCTAGC\n",
      "1NF_17A\tACAAA\n",
      "1NF_18A\tTTCTC\n",
      "1NF_19A\tAGCCC\n",
      "1NF_20A\tGTATT\n",
      "1NF_21A\tCTGTA\n",
      "1NF_22A\tACCGT\n",
      "1NF_23A\tGCTTA\n",
      "1NF_24A\tGGTGT\n",
      "1NF_25A\tAGGAT\n",
      "1NF_26A\tATTGA\n",
      "1NF_27A\tCATCT\n",
      "1NF_28A\tCCTAC\n",
      "1NF_29A\tGAGGA\n",
      "1NF_30A\tGGAAC\n",
      "1NF_31A\tGTCAA\n",
      "1NF_32A\tTAATA\n",
      "1NF_33A\tTACAT\n",
      "1SN_1A\tTCGTT\n",
      "1SN_2A\tGGTTGT\n",
      "1SN_3A\tCCAGCT\n",
      "1SN_4A\tTTCAGA\n",
      "1SN_5A\tTAGGAA\n",
      "1SN_6A\tGCTCTA\n",
      "1SN_7A\tCCACAA\n",
      "1SN_8A\tCTTCCA\n",
      "1SN_9A\tGAGATA\n",
      "1SN_10A\tATGCCT\n",
      "1SN_11A\tAGTGGA\n",
      "1SN_12A\tACCTAA\n",
      "1SN_13A\tATATGT\n",
      "1SN_14A\tATCGTA\n",
      "1SN_15A\tCATCGT\n",
      "1SN_16A\tCGCGGT\n",
      "1SN_17A\tCTATTA\n",
      "1SN_18A\tGCCAGT\n",
      "1SN_19A\tGGAAGA\n",
      "1SN_20A\tGTACTT\n",
      "1SN_21A\tGTTGAA\n",
      "1SN_22A\tTAACGA\n",
      "1SN_23A\tTGGCTA\n",
      "1SN_24A\tTATTTTT\n",
      "1SN_25A\tCTTGCTT\n",
      "1SN_26A\tATGAAAC\n",
      "1SN_27A\tAAAAGTT\n",
      "1SN_28A\tGAATTCA\n",
      "1SN_29A\tGAACTTC\n",
      "1SN_30A\tGGACCTA\n",
      "1SN_31A\tGTCGATT\n",
      "1SN_32A\tAACGCCT\n",
      "1HL_1A\tAATATGC\n",
      "1HL_2A\tACGTGTT\n",
      "1HL_3A\tATTAATT\n",
      "1HL_4A\tATTGGAT\n",
      "1HL_5A\tCATAAGT\n",
      "1HL_6A\tCGCTGAT\n",
      "1HL_7A\tCGGTAGA\n",
      "1HL_8A\tCTACGGA\n",
      "1HL_9A\tGCGGAAT\n",
      "1HL_10A\tTAGCGGA\n",
      "1HL_11A\tTCGAAGA\n",
      "1HL_12A\tTCTGTGA\n",
      "1HL_13A\tTGCTGGA\n",
      "1HL_14A\tACGACTAC\n",
      "1HL_15A\tTAGCATGC\n",
      "1HL_16A\tTAGGCCAT\n",
      "1HL_17A\tTGCAAGGA\n",
      "1HL_19A\tTGGTACGT\n",
      "1HL_20A\tTCTCAGTC\n",
      "1HL_21A\tCCGGATAT\n",
      "1HL_22A\tCGCCTTAT\n",
      "1HL_23A\tAACCGAGA\n",
      "1HL_24A\tACAGGGAA\n",
      "1HL_25A\tACGTGGTA\n",
      "1HL_26A\tCCATGGGT\n",
      "1HL_27A\tCGCGGAGA\n",
      "1HL_28A\tCGTGTGGT\n",
      "1HL_29A\tGCTGTGGA\n",
      "1HL_31A\tGGATTGGT\n",
      "1HL_33A\tGTGAGGGT\n",
      "1HL_34A\tTATCGGGA\n",
      "1HL_35A\tTTCCTGGA\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat barcodes.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: De-multiplex reads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\twarning: CUT site matches within one of the barcodes, I suggest double \n",
      "\tchecking the file to make sure it properly demultiplexes\n",
      "\n",
      "no stats file generated\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 1: sorting reads by barcode\n",
      "\t Traceback (most recent call last):\n",
      "  File \"/usr/local/bin/pyrad\", line 9, in <module>\n",
      "    load_entry_point('pyrad==3.0.66', 'console_scripts', 'pyrad')()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/pyRAD.py\", line 292, in main\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/sortandcheck2.py\", line 428, in main\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/sortandcheck2.py\", line 389, in writefunc\n",
      "ValueError: max() arg is an empty sequence\n",
      "\n",
      "real\t93m3.141s\n",
      "user\t81m1.775s\n",
      "sys\t0m19.037s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p params.txt -s 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hmmm, will re-run this since it seems to have failed..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\tfastq/ directory in working directory contains data, move/remove it before running step 1\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "real\t0m2.740s\n",
      "user\t0m0.304s\n",
      "sys\t0m0.115s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p params.txt -s 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis/fastq\n"
     ]
    }
   ],
   "source": [
    "cd fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd .."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  \u001b[0m\u001b[01;34mfastq\u001b[0m/\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  params.txt\r\n",
      "barcodes.txt                                                  \u001b[01;34mstats\u001b[0m/\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "rm -rf fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  params.txt\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  \u001b[0m\u001b[01;34mstats\u001b[0m/\r\n",
      "barcodes.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "time pyrad -p params.txt -s 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jupyter kernel crased while this was running, so no output to screen. I used the ```top``` command to verify that pyrad was still running despite Jupyter being down. Let's look at some files to see what happened..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'/home/notebooks'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file    \tNreads\tcut_found\tbar_matched\n",
      "\n",
      "\n",
      "sample\ttrue_bar\tobs_bars\tN_obs\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "head -n 100 stats/s1.sorting.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  \u001b[0m\u001b[01;34mfastq\u001b[0m/\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  params.txt\r\n",
      "barcodes.txt                                                  \u001b[01;34mstats\u001b[0m/\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls fastq/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guess it didn't complete..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rmdir: failed to remove 'fastq/': Directory not empty\r\n"
     ]
    }
   ],
   "source": [
    "rmdir fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis/fastq\n"
     ]
    }
   ],
   "source": [
    "cd fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rm -rf fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'/data/analysis/fastq'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd .."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rm -rf fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  params.txt\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  \u001b[0m\u001b[01;34mstats\u001b[0m/\r\n",
      "barcodes.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis/stats\n"
     ]
    }
   ],
   "source": [
    "cd stats/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s1.sorting.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rm s1.sorting.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd .."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\twarning: CUT site matches within one of the barcodes, I suggest double \n",
      "\tchecking the file to make sure it properly demultiplexes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 1: sorting reads by barcode\n",
      "\t ./bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "/bin/cat: write error: No space left on device\n",
      "\n",
      "real\t911m11.266s\n",
      "user\t718m35.394s\n",
      "sys\t4m32.133s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p params.txt -s 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Let's see if the de-multiplexing really failed or not. Based on the run time, it seems like things should be fine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis/fastq\n"
     ]
    }
   ],
   "source": [
    "cd /data/analysis/fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1HL_10A_R1.fq.gz  1HL_35A_R1.fq.gz  1NF_25A_R1.fq.gz  1SN_18A_R1.fq.gz\r\n",
      "1HL_10A_R2.fq.gz  1HL_35A_R2.fq.gz  1NF_25A_R2.fq.gz  1SN_18A_R2.fq.gz\r\n",
      "1HL_11A_R1.fq.gz  1HL_3A_R1.fq.gz   1NF_26A_R1.fq.gz  1SN_19A_R1.fq.gz\r\n",
      "1HL_11A_R2.fq.gz  1HL_3A_R2.fq.gz   1NF_26A_R2.fq.gz  1SN_19A_R2.fq.gz\r\n",
      "1HL_12A_R1.fq.gz  1HL_4A_R1.fq.gz   1NF_27A_R1.fq.gz  1SN_1A_R1.fq.gz\r\n",
      "1HL_12A_R2.fq.gz  1HL_4A_R2.fq.gz   1NF_27A_R2.fq.gz  1SN_1A_R2.fq.gz\r\n",
      "1HL_13A_R1.fq.gz  1HL_5A_R1.fq.gz   1NF_28A_R1.fq.gz  1SN_20A_R1.fq.gz\r\n",
      "1HL_13A_R2.fq.gz  1HL_5A_R2.fq.gz   1NF_28A_R2.fq.gz  1SN_20A_R2.fq.gz\r\n",
      "1HL_14A_R1.fq.gz  1HL_6A_R1.fq.gz   1NF_29A_R1.fq.gz  1SN_21A_R1.fq.gz\r\n",
      "1HL_14A_R2.fq.gz  1HL_6A_R2.fq.gz   1NF_29A_R2.fq.gz  1SN_21A_R2.fq.gz\r\n",
      "1HL_15A_R1.fq.gz  1HL_7A_R1.fq.gz   1NF_2A_R1.fq.gz   1SN_22A_R1.fq.gz\r\n",
      "1HL_15A_R2.fq.gz  1HL_7A_R2.fq.gz   1NF_2A_R2.fq.gz   1SN_22A_R2.fq.gz\r\n",
      "1HL_16A_R1.fq.gz  1HL_8A_R1.fq.gz   1NF_30A_R1.fq.gz  1SN_23A_R1.fq.gz\r\n",
      "1HL_16A_R2.fq.gz  1HL_8A_R2.fq.gz   1NF_30A_R2.fq.gz  1SN_23A_R2.fq.gz\r\n",
      "1HL_17A_R1.fq.gz  1HL_9A_R1.fq.gz   1NF_31A_R1.fq.gz  1SN_24A_R1.fq.gz\r\n",
      "1HL_17A_R2.fq.gz  1HL_9A_R2.fq.gz   1NF_31A_R2.fq.gz  1SN_24A_R2.fq.gz\r\n",
      "1HL_19A_R1.fq.gz  1NF_10A_R1.fq.gz  1NF_32A_R1.fq.gz  1SN_25A_R1.fq.gz\r\n",
      "1HL_19A_R2.fq.gz  1NF_10A_R2.fq.gz  1NF_32A_R2.fq.gz  1SN_25A_R2.fq.gz\r\n",
      "1HL_1A_R1.fq.gz   1NF_11A_R1.fq.gz  1NF_33A_R1.fq.gz  1SN_26A_R1.fq.gz\r\n",
      "1HL_1A_R2.fq.gz   1NF_11A_R2.fq.gz  1NF_33A_R2.fq.gz  1SN_26A_R2.fq.gz\r\n",
      "1HL_20A_R1.fq.gz  1NF_12A_R1.fq.gz  1NF_4A_R1.fq.gz   1SN_27A_R1.fq.gz\r\n",
      "1HL_20A_R2.fq.gz  1NF_12A_R2.fq.gz  1NF_4A_R2.fq.gz   1SN_27A_R2.fq.gz\r\n",
      "1HL_21A_R1.fq.gz  1NF_13A_R1.fq.gz  1NF_5A_R1.fq.gz   1SN_28A_R1.fq.gz\r\n",
      "1HL_21A_R2.fq.gz  1NF_13A_R2.fq.gz  1NF_5A_R2.fq.gz   1SN_28A_R2.fq.gz\r\n",
      "1HL_22A_R1.fq.gz  1NF_14A_R1.fq.gz  1NF_6A_R1.fq.gz   1SN_29A_R1.fq.gz\r\n",
      "1HL_22A_R2.fq.gz  1NF_14A_R2.fq.gz  1NF_6A_R2.fq.gz   1SN_29A_R2.fq.gz\r\n",
      "1HL_23A_R1.fq.gz  1NF_15A_R1.fq.gz  1NF_7A_R1.fq.gz   1SN_2A_R1.fq.gz\r\n",
      "1HL_23A_R2.fq.gz  1NF_15A_R2.fq.gz  1NF_7A_R2.fq.gz   1SN_2A_R2.fq.gz\r\n",
      "1HL_24A_R1.fq.gz  1NF_16A_R1.fq.gz  1NF_8A_R1.fq.gz   1SN_30A_R1.fq.gz\r\n",
      "1HL_24A_R2.fq.gz  1NF_16A_R2.fq.gz  1NF_8A_R2.fq.gz   1SN_30A_R2.fq.gz\r\n",
      "1HL_25A_R1.fq.gz  1NF_17A_R1.fq.gz  1NF_9A_R1.fq.gz   1SN_31A_R1.fq.gz\r\n",
      "1HL_25A_R2.fq.gz  1NF_17A_R2.fq.gz  1NF_9A_R2.fq.gz   1SN_31A_R2.fq.gz\r\n",
      "1HL_26A_R1.fq.gz  1NF_18A_R1.fq.gz  1SN_10A_R1.fq.gz  1SN_32A_R1.fq.gz\r\n",
      "1HL_26A_R2.fq.gz  1NF_18A_R2.fq.gz  1SN_10A_R2.fq.gz  1SN_32A_R2.fq.gz\r\n",
      "1HL_27A_R1.fq.gz  1NF_19A_R1.fq.gz  1SN_11A_R1.fq.gz  1SN_3A_R1.fq.gz\r\n",
      "1HL_27A_R2.fq.gz  1NF_19A_R2.fq.gz  1SN_11A_R2.fq.gz  1SN_3A_R2.fq.gz\r\n",
      "1HL_28A_R1.fq.gz  1NF_1A_R1.fq.gz   1SN_12A_R1.fq.gz  1SN_4A_R1.fq.gz\r\n",
      "1HL_28A_R2.fq.gz  1NF_1A_R2.fq.gz   1SN_12A_R2.fq.gz  1SN_4A_R2.fq.gz\r\n",
      "1HL_29A_R1.fq.gz  1NF_20A_R1.fq.gz  1SN_13A_R1.fq.gz  1SN_5A_R1.fq.gz\r\n",
      "1HL_29A_R2.fq.gz  1NF_20A_R2.fq.gz  1SN_13A_R2.fq.gz  1SN_5A_R2.fq.gz\r\n",
      "1HL_2A_R1.fq.gz   1NF_21A_R1.fq.gz  1SN_14A_R1.fq.gz  1SN_6A_R1.fq.gz\r\n",
      "1HL_2A_R2.fq.gz   1NF_21A_R2.fq.gz  1SN_14A_R2.fq.gz  1SN_6A_R2.fq.gz\r\n",
      "1HL_31A_R1.fq.gz  1NF_22A_R1.fq.gz  1SN_15A_R1.fq.gz  1SN_7A_R1.fq.gz\r\n",
      "1HL_31A_R2.fq.gz  1NF_22A_R2.fq.gz  1SN_15A_R2.fq.gz  1SN_7A_R2.fq.gz\r\n",
      "1HL_33A_R1.fq.gz  1NF_23A_R1.fq.gz  1SN_16A_R1.fq.gz  1SN_8A_R1.fq.gz\r\n",
      "1HL_33A_R2.fq.gz  1NF_23A_R2.fq.gz  1SN_16A_R2.fq.gz  1SN_8A_R2.fq.gz\r\n",
      "1HL_34A_R1.fq.gz  1NF_24A_R1.fq.gz  1SN_17A_R1.fq.gz  1SN_9A_R1.fq.gz\r\n",
      "1HL_34A_R2.fq.gz  1NF_24A_R2.fq.gz  1SN_17A_R2.fq.gz  1SN_9A_R2.fq.gz\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "192\r\n"
     ]
    }
   ],
   "source": [
    "ls -1 | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Correct number of files were produced from demultiplexing, so things look good on that front. Let's just glance at the stats file to see what it looks like...\n",
    "\n",
    "NOTE: Bash trick above. The ls command above is using the number one (1), NOT lowercase L (l). This provides an accurate line count when you pipe the output to the word count command (wc)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file    \tNreads\tcut_found\tbar_matched\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_.fq.gz\t364261046\t355811046\t354725421\n",
      "\n",
      "\n",
      "sample\ttrue_bar\tobs_bars\tN_obs\n",
      "1SN_27A    \tAAAAGTT    \tAAAAGTT\t3325108   \n",
      "1SN_27A    \tAAAAGTT    \tNAAAGTT\t8330      \n",
      "1SN_27A    \tAAAAGTT    \tAAAAATT\t4748      \n",
      "1SN_27A    \tAAAAGTT    \tANAAGTT\t1455      \n",
      "1SN_27A    \tAAAAGTT    \tAAAATTT\t782       \n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "head ../stats/s1.sorting.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Things look OK to me. Next step!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merge paired-reads\n",
    "\n",
    "The first thing we have to do is unzip the reads if they are gzipped. If you demultiplexed your data in pyrad they will be in the fastq/ directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "gzip: /data/analysis/fastq/1HL_19A_R2.fq.gz: unexpected end of file\n",
      "\n",
      "real\t52m44.818s\n",
      "user\t1m24.977s\n",
      "sys\t0m13.352s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time gunzip /data/analysis/fastq/*.fq.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "real\t25m34.538s\n",
      "user\t71m32.940s\n",
      "sys\t0m17.129s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time for gfile in /data/analysis/fastq/*_R1.fq;\n",
    "    do /usr/local/bioinformatics/pear-0.9.6-bin-64/pear-0.9.6-bin-64 -f $gfile \\\n",
    "            -r ${gfile/_R1.fq/_R2.fq} \\\n",
    "            -o ${gfile/_R1.fq/} \\\n",
    "            -n 33 \\\n",
    "            -t 33 \\\n",
    "            -q 10 \\\n",
    "            -j 20  >> pear.log 2>&1;\n",
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the data location to the de-multiplexed & merged data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "## set location of demultiplexed data that are 'pear' filtered\n",
    "sed -i '/## 18./c\\/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\r\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\r\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\r\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\r\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\r\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\r\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\r\n",
      "16                     ## 7. N processors (parallel)                           (all)\r\n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\r\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\r\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\r\n",
      "pairgbs                 ## 11. Datatype: rad,gbs,pairgbs,pairddrad,(others:see docs)(all)\r\n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\r\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\r\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\r\n",
      "==== optional params below this line ===================================  affected step ==\r\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\r\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\r\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\r\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \r\n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\r\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\r\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\r\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\r\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\r\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\r\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\r\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\r\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\r\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\r\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\r\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\r\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\r\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\r\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\r\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\r\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\r\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\r\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\r\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\r\n",
      "1HL\t5\t1HL*\r\n",
      "1NF\t5\t1NF*\r\n",
      "1SN\t5\t1SN*\r\n"
     ]
    }
   ],
   "source": [
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the datatype to \"merged\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sed -i '/## 11./c\\merged            ## 11. data type ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "16                     ## 7. N processors (parallel)                           (all)\n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "merged            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Quality filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\tsorted .fastq from /data/analysis/fastq/*.assembled.fastq being used\n",
      "\tstep 2: editing raw reads \n",
      "\t........\n",
      "real\t14m52.577s\n",
      "user\t39m28.511s\n",
      "sys\t0m7.845s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's take a look at the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample \tNreads\tpassed\tpassed.w.trim\tpassed.total\n",
      "1HL_10A.assembled\t2890999\t2127357\t214294\t2341651\n",
      "1HL_11A.assembled\t3088425\t2242089\t246561\t2488650\n",
      "1HL_12A.assembled\t2323231\t1728934\t163227\t1892161\n",
      "1HL_13A.assembled\t2701305\t2075000\t175929\t2250929\n",
      "1HL_14A.assembled\t1865526\t1407146\t129089\t1536235\n",
      "1HL_15A.assembled\t1837684\t1342114\t139898\t1482012\n",
      "1HL_16A.assembled\t2100838\t1568118\t147901\t1716019\n",
      "1HL_17A.assembled\t3106139\t2259592\t237499\t2497091\n",
      "\n",
      "    Nreads = total number of reads for a sample\n",
      "    passed = retained reads that passed quality filtering at full length\n",
      "    passed.w.trim= retained reads that were trimmed due to detection of adapters\n",
      "    passed.total  = total kept reads of sufficient length\n",
      "    note: you can set the option in params file to include trimmed reads of xx length. \n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/s2.rawedit.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Clustering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tde-replicating files for clustering...\n",
      "\n",
      "\tstep 3: within-sample clustering of 8 samples at \n",
      "\t        '.88' similarity. Running 8 parallel jobs\n",
      "\t \twith up to 16 threads per job. If needed, \n",
      "\t\tadjust to avoid CPU and MEM limits\n",
      "\n",
      "Process Worker-3:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-8:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-16:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-9:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-13:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-14:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 538, in final\n",
      "    makederepclust(outfolder, handle, w1, datatype)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 40, in makederepclust\n",
      "    line[5] = re.sub(\"\\..*\\n\",\"\\n\", line[5])\n",
      "IndexError: list index out of range\n",
      "Process Worker-11:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-4:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/local/bin/pyrad\", line 9, in <module>\n",
      "    load_entry_point('pyrad==3.0.66', 'console_scripts', 'pyrad')()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/pyRAD.py\", line 340, in main\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 721, in main\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 522, in call\n",
      "    return Popen(*popenargs, **kwargs).wait()\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "\n",
      "real\t38m28.984s\n",
      "user\t138m32.476s\n",
      "sys\t8m13.640s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Possibly overloading the CPUs. Let's change to reflect the actual number of cores (8)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sed -i '/## 7. /c\\8                   ## 7. N processors   ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "8                   ## 7. N processors   \n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "merged            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try this again..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tskipping 1HL_17A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_13A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_16A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_14A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_15A.assembled.clustS.gz already exists in /data/analysis/clust.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tde-replicating files for clustering...\n",
      "\n",
      "\tstep 3: within-sample clustering of 8 samples at \n",
      "\t        '.88' similarity. Running 8 parallel jobs\n",
      "\t \twith up to 16 threads per job. If needed, \n",
      "\t\tadjust to avoid CPU and MEM limits\n",
      "\n",
      "Process Worker-6:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-2:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "Process Worker-4:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/lib/python2.7/multiprocessing/process.py\", line 258, in _bootstrap\n",
      "    self.run()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/potpour.py\", line 31, in run\n",
      "    res = self.func(*job)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 545, in final\n",
      "    alignwrap(outfolder+\"/\"+handle.split(\"/\")[-1].replace(\".edit\",\".clust.gz\"), mindepth, muscle, w1)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 415, in alignwrap\n",
      "    stringnames = alignfast(names[0:200],seqs[0:200],muscle)\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/cluster7dp.py\", line 301, in alignfast\n",
      "    stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 710, in __init__\n",
      "    errread, errwrite)\n",
      "  File \"/usr/lib/python2.7/subprocess.py\", line 1231, in _execute_child\n",
      "    self.pid = os.fork()\n",
      "OSError: [Errno 12] Cannot allocate memory\n",
      "\n",
      "real\t21m28.279s\n",
      "user\t56m45.842s\n",
      "sys\t1m52.941s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well, I think the problem is that I'm running this at the same time as I'm running ustacks (a different program). It seems that ustacks is using all of the EC2 instance's resources (viewed using the Bash program \"htop\"). I will wait until that has completed and then run PyRad (or, upgrade the EC2 instance to a a greater number of CPUs or clone this instance and run two separate instances).\n",
    "\n",
    "![](http://eagle.fish.washington.edu/Arabidopsis/20160718_ec2_ustacks_cpus.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tskipping 1HL_17A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_10A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_13A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_12A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_16A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_14A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_15A.assembled.clustS.gz already exists in /data/analysis/clust.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tde-replicating files for clustering...\n",
      "\n",
      "\tstep 3: within-sample clustering of 8 samples at \n",
      "\t        '.88' similarity. Running 8 parallel jobs\n",
      "\t \twith up to 16 threads per job. If needed, \n",
      "\t\tadjust to avoid CPU and MEM limits\n",
      "\n",
      "\tsample 1HL_11A finished, 125407 loci\n",
      "\n",
      "real\t63m55.147s\n",
      "user\t44m17.743s\n",
      "sys\t29m14.039s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's take a look at the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "1HL_11A.assembled\t125407\t10.143\t106.054\t39824\t27.904\t186.958\t0\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "sample: 1HL_11A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                81060\n",
      "5 \t*****                               15999\n",
      "10 \t***                                 8344\n",
      "15 \t**                                  5099\n",
      "20 \t**                                  3282\n",
      "25 \t**                                  2270\n",
      "30 \t*                                   1646\n",
      "35 \t*                                   1247\n",
      "40 \t*                                   1846\n",
      "50 \t**                                  3628\n",
      "100 \t*                                   738\n",
      "250 \t*                                   160\n",
      "500 \t*                                   88\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/s3.clusters.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Parameter estimation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 4: estimating error rate and heterozygosity\n",
      "\t........\n",
      "real\t34m34.339s\n",
      "user\t60m29.346s\n",
      "sys\t0m0.623s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "taxa\tH\tE\n",
      "1HL_13A.assembled\t0.01234178\t0.00144187\t\n",
      "1HL_10A.assembled\t0.01399484\t0.00139753\t\n",
      "1HL_15A.assembled\t0.01242723\t0.0011879\t\n",
      "1HL_12A.assembled\t0.01240698\t0.00157456\t\n",
      "1HL_16A.assembled\t0.01381188\t0.00054237\t\n",
      "1HL_17A.assembled\t0.01332716\t0.00127763\t\n",
      "1HL_14A.assembled\t0.01266762\t0.00203182\t\n",
      "1HL_11A.assembled\t0.01216978\t0.00165352\t\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/Pi_E_estimate.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yeesh, this isn't right. Not all of the individuals are here. Should've noticed this before! Going back to unzipping files before merging paired reads."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "gzip: /data/analysis/fastq/1HL_19A_R2.fq.gz: unexpected end of file\n",
      "\n",
      "real\t0m35.063s\n",
      "user\t0m4.390s\n",
      "sys\t0m0.746s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time gunzip /data/analysis/fastq/*.fq.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 49G\r\n",
      "-rw-r--r-- 1 root root 762M Jul 18 16:17 1HL_10A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 384M Jul 18 16:17 1HL_10A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:14 1HL_10A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:14 1HL_10A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 969M Jul 17 06:24 1HL_10A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 1.0G Jul 17 06:25 1HL_10A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 822M Jul 18 16:22 1HL_11A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 433M Jul 18 16:22 1HL_11A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:17 1HL_11A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:17 1HL_11A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 1.1G Jul 17 06:25 1HL_11A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 1.1G Jul 17 06:26 1HL_11A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 612M Jul 18 16:26 1HL_12A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 299M Jul 18 16:26 1HL_12A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:22 1HL_12A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:22 1HL_12A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 771M Jul 17 06:26 1HL_12A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 815M Jul 17 06:26 1HL_12A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 697M Jul 18 16:31 1HL_13A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 328M Jul 18 16:31 1HL_13A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:26 1HL_13A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:26 1HL_13A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 882M Jul 17 06:27 1HL_13A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 932M Jul 17 06:27 1HL_13A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 487M Jul 18 16:32 1HL_14A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 251M Jul 18 16:32 1HL_14A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:31 1HL_14A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:31 1HL_14A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 622M Jul 17 06:27 1HL_14A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 663M Jul 17 06:28 1HL_14A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 490M Jul 18 16:34 1HL_15A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 277M Jul 18 16:34 1HL_15A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:33 1HL_15A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:33 1HL_15A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 636M Jul 17 06:28 1HL_15A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 677M Jul 17 06:28 1HL_15A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 551M Jul 18 16:37 1HL_16A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 290M Jul 18 16:37 1HL_16A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:34 1HL_16A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:34 1HL_16A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 707M Jul 17 06:29 1HL_16A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 753M Jul 17 06:29 1HL_16A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 826M Jul 18 16:40 1HL_17A.assembled.fastq\r\n",
      "-rw-r--r-- 1 root root 451M Jul 18 16:40 1HL_17A.discarded.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:37 1HL_17A.unassembled.forward.fastq\r\n",
      "-rw-r--r-- 1 root root    0 Jul 18 16:37 1HL_17A.unassembled.reverse.fastq\r\n",
      "-rw-r--r-- 1 root root 1.1G Jul 17 06:29 1HL_17A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 1.2G Jul 17 06:30 1HL_17A_R2.fq\r\n",
      "-rw-r--r-- 1 root root 1.1G Jul 17 06:31 1HL_19A_R1.fq\r\n",
      "-rw-r--r-- 1 root root 168M Jul 17 06:32 1HL_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 133M Jul 17 06:19 1HL_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 17 06:19 1HL_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 159M Jul 17 06:19 1HL_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 196M Jul 17 06:19 1HL_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_33A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_33A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_34A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_34A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_35A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root    0 Jul 17 06:32 1HL_35A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 129M Jul 17 06:20 1HL_3A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 17 06:20 1HL_3A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 17 06:20 1HL_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 186M Jul 17 06:21 1HL_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 118M Jul 17 06:21 1HL_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 145M Jul 17 06:21 1HL_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 119M Jul 17 06:21 1HL_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 149M Jul 17 06:22 1HL_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 162M Jul 17 06:22 1HL_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 220M Jul 17 06:22 1HL_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 148M Jul 17 06:23 1HL_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 182M Jul 17 06:23 1HL_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 165M Jul 17 06:24 1HL_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 203M Jul 17 06:24 1HL_9A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 17 05:39 1NF_10A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 17 05:39 1NF_10A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 134M Jul 17 05:39 1NF_11A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 163M Jul 17 05:39 1NF_11A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 132M Jul 17 05:40 1NF_12A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 160M Jul 17 05:40 1NF_12A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 229M Jul 17 05:40 1NF_13A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 278M Jul 17 05:41 1NF_13A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 17 05:41 1NF_14A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 192M Jul 17 05:42 1NF_14A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 150M Jul 17 05:42 1NF_15A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 184M Jul 17 05:42 1NF_15A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 17 05:42 1NF_16A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 199M Jul 17 05:43 1NF_16A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 184M Jul 17 05:43 1NF_17A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 224M Jul 17 05:44 1NF_17A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 102M Jul 17 05:44 1NF_18A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 125M Jul 17 05:44 1NF_18A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 120M Jul 17 05:44 1NF_19A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 147M Jul 17 05:45 1NF_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root  82M Jul 17 05:33 1NF_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root  99M Jul 17 05:34 1NF_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 146M Jul 17 05:45 1NF_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 177M Jul 17 05:45 1NF_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 182M Jul 17 05:46 1NF_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 221M Jul 17 05:46 1NF_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 17 05:47 1NF_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 17 05:47 1NF_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 192M Jul 17 05:47 1NF_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 234M Jul 17 05:48 1NF_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 175M Jul 17 05:48 1NF_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 214M Jul 17 05:48 1NF_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 17 05:49 1NF_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 17 05:49 1NF_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 17 05:49 1NF_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 17 05:50 1NF_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 137M Jul 17 05:50 1NF_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 165M Jul 17 05:50 1NF_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 141M Jul 17 05:51 1NF_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 17 05:51 1NF_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 224M Jul 17 05:51 1NF_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 274M Jul 17 05:52 1NF_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 204M Jul 17 05:34 1NF_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 246M Jul 17 05:34 1NF_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 167M Jul 17 05:52 1NF_30A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 203M Jul 17 05:52 1NF_30A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 207M Jul 17 05:53 1NF_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 253M Jul 17 05:53 1NF_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 208M Jul 17 05:54 1NF_32A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 251M Jul 17 05:54 1NF_32A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 17 05:54 1NF_33A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 189M Jul 17 05:55 1NF_33A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 212M Jul 17 05:35 1NF_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 256M Jul 17 05:35 1NF_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 17 05:35 1NF_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 197M Jul 17 05:36 1NF_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 145M Jul 17 05:36 1NF_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 175M Jul 17 05:36 1NF_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 154M Jul 17 05:36 1NF_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 187M Jul 17 05:37 1NF_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 163M Jul 17 05:37 1NF_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 198M Jul 17 05:37 1NF_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 199M Jul 17 05:38 1NF_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 240M Jul 17 05:38 1NF_9A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 17 06:02 1SN_10A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 185M Jul 17 06:02 1SN_10A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 154M Jul 17 06:02 1SN_11A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 188M Jul 17 06:03 1SN_11A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 197M Jul 17 06:03 1SN_12A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 242M Jul 17 06:04 1SN_12A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 17 06:04 1SN_13A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 210M Jul 17 06:04 1SN_13A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 206M Jul 17 06:05 1SN_14A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 252M Jul 17 06:05 1SN_14A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 144M Jul 17 06:05 1SN_15A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 176M Jul 17 06:06 1SN_15A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 152M Jul 17 06:06 1SN_16A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 189M Jul 17 06:06 1SN_16A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 17 06:07 1SN_17A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 17 06:07 1SN_17A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 188M Jul 17 06:08 1SN_18A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 233M Jul 17 06:08 1SN_18A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 177M Jul 17 06:08 1SN_19A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 216M Jul 17 06:09 1SN_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 176M Jul 17 05:55 1SN_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 214M Jul 17 05:56 1SN_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 130M Jul 17 06:09 1SN_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 160M Jul 17 06:09 1SN_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 201M Jul 17 06:10 1SN_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 248M Jul 17 06:10 1SN_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 179M Jul 17 06:11 1SN_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 219M Jul 17 06:11 1SN_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 17 06:11 1SN_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 261M Jul 17 06:12 1SN_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 110M Jul 17 06:12 1SN_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 134M Jul 17 06:12 1SN_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 17 06:13 1SN_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 212M Jul 17 06:13 1SN_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 17 06:14 1SN_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 17 06:14 1SN_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 148M Jul 17 06:14 1SN_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 183M Jul 17 06:15 1SN_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 162M Jul 17 06:15 1SN_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 200M Jul 17 06:15 1SN_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 17 06:15 1SN_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 161M Jul 17 06:16 1SN_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 178M Jul 17 05:56 1SN_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 220M Jul 17 05:56 1SN_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 219M Jul 17 06:16 1SN_30A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 271M Jul 17 06:17 1SN_30A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 190M Jul 17 06:17 1SN_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 235M Jul 17 06:18 1SN_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 147M Jul 17 06:18 1SN_32A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 181M Jul 17 06:18 1SN_32A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 140M Jul 17 05:57 1SN_3A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 17 05:57 1SN_3A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 191M Jul 17 05:57 1SN_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 234M Jul 17 05:58 1SN_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 174M Jul 17 05:58 1SN_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 17 05:58 1SN_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 17 05:59 1SN_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 239M Jul 17 05:59 1SN_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 173M Jul 17 05:59 1SN_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 17 06:00 1SN_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 17 06:00 1SN_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 237M Jul 17 06:01 1SN_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 185M Jul 17 06:01 1SN_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 227M Jul 17 06:01 1SN_9A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root  21K Jul 18 16:40 pear.log\r\n"
     ]
    }
   ],
   "source": [
    "ls -lh /data/analysis/fastq/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well, it turns out the the original demultiplexing actually failed. Many fq.gz files that are empty (i.e. 0 megabytes in size). Need to re-run demultiplexing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Need to change data type back to pairgbs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sed -i '/## 11./c\\pairgbs            ## 11. data type ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "8                   ## 7. N processors   \n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "pairgbs            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash \n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz\n",
      "barcodes.txt\n",
      "clust.88\n",
      "edits\n",
      "fastq\n",
      "params.txt\n",
      "stats\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "ls /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "rm -rf /data/analysis/fastq/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz\n",
      "barcodes.txt\n",
      "clust.88\n",
      "edits\n",
      "params.txt\n",
      "stats\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "ls /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/analysis\n"
     ]
    }
   ],
   "source": [
    "cd /data/analysis/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De-multiplex reads (again)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tskipping step 1: line 18 of input file shows seqs already sorted\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "real\t0m0.225s\n",
      "user\t0m0.202s\n",
      "sys\t0m0.023s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "## set location of demultiplexed data that are 'pear' filtered\n",
    "sed -i '/## 18./c\\ ## 18. demulti data loc ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "8                   ## 7. N processors   \n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "pairgbs            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      " ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash \n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\twarning: CUT site matches within one of the barcodes, I suggest double \n",
      "\tchecking the file to make sure it properly demultiplexes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 1: sorting reads by barcode\n",
      "\t .\n",
      "real\t771m53.005s\n",
      "user\t687m46.614s\n",
      "sys\t3m12.997s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Let's check the output and make sure the files actually contain data..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 35G\r\n",
      "-rw-r--r-- 1 root root 183M Jul 20 08:30 1HL_10A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 226M Jul 20 08:31 1HL_10A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 200M Jul 20 08:31 1HL_11A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 247M Jul 20 08:32 1HL_11A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 146M Jul 20 08:32 1HL_12A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 181M Jul 20 08:32 1HL_12A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 159M Jul 20 08:33 1HL_13A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 197M Jul 20 08:33 1HL_13A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 117M Jul 20 08:33 1HL_14A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 146M Jul 20 08:34 1HL_14A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 123M Jul 20 08:34 1HL_15A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 154M Jul 20 08:34 1HL_15A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 133M Jul 20 08:35 1HL_16A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 166M Jul 20 08:35 1HL_16A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 201M Jul 20 08:36 1HL_17A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 252M Jul 20 08:36 1HL_17A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 202M Jul 20 08:37 1HL_19A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 252M Jul 20 08:37 1HL_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 133M Jul 20 08:24 1HL_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 20 08:24 1HL_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 162M Jul 20 08:38 1HL_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 203M Jul 20 08:38 1HL_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 145M Jul 20 08:38 1HL_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 181M Jul 20 08:39 1HL_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 119M Jul 20 08:39 1HL_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 148M Jul 20 08:39 1HL_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 201M Jul 20 08:40 1HL_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 251M Jul 20 08:40 1HL_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 20 08:41 1HL_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 189M Jul 20 08:41 1HL_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 167M Jul 20 08:42 1HL_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 208M Jul 20 08:42 1HL_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 179M Jul 20 08:42 1HL_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 224M Jul 20 08:43 1HL_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 182M Jul 20 08:43 1HL_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 227M Jul 20 08:44 1HL_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 20 08:44 1HL_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 282M Jul 20 08:45 1HL_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 177M Jul 20 08:45 1HL_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 222M Jul 20 08:46 1HL_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 159M Jul 20 08:24 1HL_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 196M Jul 20 08:25 1HL_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 139M Jul 20 08:46 1HL_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 174M Jul 20 08:46 1HL_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 20 08:47 1HL_33A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 190M Jul 20 08:47 1HL_33A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 161M Jul 20 08:47 1HL_34A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 201M Jul 20 08:48 1HL_34A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 146M Jul 20 08:48 1HL_35A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 183M Jul 20 08:49 1HL_35A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 129M Jul 20 08:25 1HL_3A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 20 08:25 1HL_3A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 20 08:26 1HL_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 186M Jul 20 08:26 1HL_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 118M Jul 20 08:26 1HL_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 145M Jul 20 08:27 1HL_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 119M Jul 20 08:27 1HL_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 149M Jul 20 08:27 1HL_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 162M Jul 20 08:28 1HL_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 220M Jul 20 08:28 1HL_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 148M Jul 20 08:28 1HL_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 182M Jul 20 08:29 1HL_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 165M Jul 20 08:29 1HL_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 203M Jul 20 08:30 1HL_9A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 20 07:39 1NF_10A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 20 07:40 1NF_10A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 134M Jul 20 07:40 1NF_11A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 163M Jul 20 07:40 1NF_11A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 132M Jul 20 07:41 1NF_12A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 160M Jul 20 07:41 1NF_12A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 229M Jul 20 07:41 1NF_13A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 278M Jul 20 07:42 1NF_13A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 20 07:42 1NF_14A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 192M Jul 20 07:42 1NF_14A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 150M Jul 20 07:43 1NF_15A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 184M Jul 20 07:43 1NF_15A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 20 07:44 1NF_16A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 199M Jul 20 07:44 1NF_16A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 184M Jul 20 07:44 1NF_17A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 224M Jul 20 07:45 1NF_17A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 102M Jul 20 07:45 1NF_18A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 125M Jul 20 07:45 1NF_18A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 120M Jul 20 07:46 1NF_19A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 147M Jul 20 07:46 1NF_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root  82M Jul 20 07:34 1NF_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root  99M Jul 20 07:34 1NF_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 146M Jul 20 07:46 1NF_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 177M Jul 20 07:47 1NF_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 182M Jul 20 07:47 1NF_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 221M Jul 20 07:47 1NF_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 20 07:48 1NF_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 20 07:48 1NF_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 192M Jul 20 07:48 1NF_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 234M Jul 20 07:49 1NF_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 175M Jul 20 07:49 1NF_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 214M Jul 20 07:50 1NF_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 20 07:50 1NF_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 20 07:50 1NF_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 20 07:51 1NF_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 20 07:51 1NF_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 137M Jul 20 07:51 1NF_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 165M Jul 20 07:52 1NF_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 141M Jul 20 07:52 1NF_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 20 07:52 1NF_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 224M Jul 20 07:53 1NF_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 274M Jul 20 07:53 1NF_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 204M Jul 20 07:34 1NF_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 246M Jul 20 07:34 1NF_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 167M Jul 20 07:54 1NF_30A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 203M Jul 20 07:54 1NF_30A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 207M Jul 20 07:55 1NF_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 253M Jul 20 07:55 1NF_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 208M Jul 20 07:56 1NF_32A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 251M Jul 20 07:56 1NF_32A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 156M Jul 20 07:56 1NF_33A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 189M Jul 20 07:57 1NF_33A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 212M Jul 20 07:35 1NF_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 256M Jul 20 07:35 1NF_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 164M Jul 20 07:35 1NF_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 197M Jul 20 07:36 1NF_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 145M Jul 20 07:36 1NF_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 175M Jul 20 07:37 1NF_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 154M Jul 20 07:37 1NF_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 187M Jul 20 07:37 1NF_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 163M Jul 20 07:38 1NF_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 198M Jul 20 07:38 1NF_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 199M Jul 20 07:39 1NF_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 240M Jul 20 07:39 1NF_9A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 151M Jul 20 08:05 1SN_10A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 185M Jul 20 08:05 1SN_10A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 154M Jul 20 08:06 1SN_11A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 188M Jul 20 08:06 1SN_11A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 197M Jul 20 08:06 1SN_12A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 242M Jul 20 08:07 1SN_12A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 20 08:08 1SN_13A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 210M Jul 20 08:08 1SN_13A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 206M Jul 20 08:08 1SN_14A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 252M Jul 20 08:09 1SN_14A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 144M Jul 20 08:09 1SN_15A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 176M Jul 20 08:10 1SN_15A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 152M Jul 20 08:10 1SN_16A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 189M Jul 20 08:10 1SN_16A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 20 08:11 1SN_17A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 209M Jul 20 08:11 1SN_17A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 188M Jul 20 08:12 1SN_18A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 233M Jul 20 08:12 1SN_18A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 177M Jul 20 08:12 1SN_19A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 216M Jul 20 08:13 1SN_19A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 176M Jul 20 07:57 1SN_1A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 214M Jul 20 07:58 1SN_1A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 130M Jul 20 08:13 1SN_20A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 160M Jul 20 08:14 1SN_20A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 201M Jul 20 08:14 1SN_21A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 248M Jul 20 08:14 1SN_21A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 179M Jul 20 08:15 1SN_22A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 219M Jul 20 08:15 1SN_22A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 20 08:16 1SN_23A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 261M Jul 20 08:16 1SN_23A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 110M Jul 20 08:17 1SN_24A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 134M Jul 20 08:17 1SN_24A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 171M Jul 20 08:17 1SN_25A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 212M Jul 20 08:18 1SN_25A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 158M Jul 20 08:18 1SN_26A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 20 08:19 1SN_26A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 148M Jul 20 08:19 1SN_27A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 183M Jul 20 08:19 1SN_27A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 162M Jul 20 08:20 1SN_28A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 200M Jul 20 08:20 1SN_28A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 128M Jul 20 08:20 1SN_29A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 161M Jul 20 08:21 1SN_29A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 178M Jul 20 07:58 1SN_2A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 220M Jul 20 07:59 1SN_2A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 219M Jul 20 08:21 1SN_30A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 271M Jul 20 08:22 1SN_30A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 190M Jul 20 08:22 1SN_31A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 235M Jul 20 08:23 1SN_31A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 147M Jul 20 08:23 1SN_32A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 181M Jul 20 08:23 1SN_32A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 140M Jul 20 07:59 1SN_3A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 172M Jul 20 07:59 1SN_3A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 191M Jul 20 08:00 1SN_4A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 234M Jul 20 08:00 1SN_4A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 174M Jul 20 08:01 1SN_5A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 20 08:01 1SN_5A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 20 08:01 1SN_6A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 239M Jul 20 08:02 1SN_6A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 173M Jul 20 08:02 1SN_7A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 213M Jul 20 08:03 1SN_7A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 194M Jul 20 08:03 1SN_8A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 237M Jul 20 08:04 1SN_8A_R2.fq.gz\r\n",
      "-rw-r--r-- 1 root root 185M Jul 20 08:04 1SN_9A_R1.fq.gz\r\n",
      "-rw-r--r-- 1 root root 227M Jul 20 08:04 1SN_9A_R2.fq.gz\r\n"
     ]
    }
   ],
   "source": [
    "ls -lh /data/analysis/fastq/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, let's make sure there's the expected number of individuals (96 individuals, sequenced from each end, so 192 total files)..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "192\r\n"
     ]
    }
   ],
   "source": [
    "ls -1 /data/analysis/fastq/ | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Merge paired reads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing we have to do is unzip the reads if they are gzipped. If you demultiplexed your data in pyrad they will be in the fastq/ directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "real\t145m40.754s\n",
      "user\t14m53.245s\n",
      "sys\t2m6.261s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time gunzip /data/analysis/fastq/*.fq.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pi_E_estimate.txt  s1.sorting.txt  s2.rawedit.txt  s3.clusters.txt\r\n"
     ]
    }
   ],
   "source": [
    "ls /data/analysis/stats/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz  \u001b[0m\u001b[01;34medits\u001b[0m/\r\n",
      "160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz  \u001b[01;34mfastq\u001b[0m/\r\n",
      "barcodes.txt                                                  params.txt\r\n",
      "\u001b[01;34mclust.88\u001b[0m/                                                     \u001b[01;34mstats\u001b[0m/\r\n"
     ]
    }
   ],
   "source": [
    "ls /data/analysis/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "real\t173m24.008s\n",
      "user\t940m23.125s\n",
      "sys\t2m39.123s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time for gfile in /data/analysis/fastq/*_R1.fq;\n",
    "    do /usr/local/bioinformatics/pear-0.9.6-bin-64/pear-0.9.6-bin-64 -f $gfile \\\n",
    "            -r ${gfile/_R1.fq/_R2.fq} \\\n",
    "            -o ${gfile/_R1.fq/} \\\n",
    "            -n 33 \\\n",
    "            -t 33 \\\n",
    "            -q 10 \\\n",
    "            -j 20  >> pear.log 2>&1;\n",
    "done"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 263G\n",
      "-rw-r--r-- 1 root root  762M Jul 20 16:53 1HL_10A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  384M Jul 20 16:53 1HL_10A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:52 1HL_10A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:52 1HL_10A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  969M Jul 20 08:30 1HL_10A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.0G Jul 20 08:31 1HL_10A_R2.fq\n",
      "-rw-r--r-- 1 root root  822M Jul 20 16:55 1HL_11A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  433M Jul 20 16:55 1HL_11A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:53 1HL_11A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:53 1HL_11A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:31 1HL_11A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:32 1HL_11A_R2.fq\n",
      "-rw-r--r-- 1 root root  612M Jul 20 16:57 1HL_12A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  299M Jul 20 16:57 1HL_12A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:55 1HL_12A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:55 1HL_12A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  771M Jul 20 08:32 1HL_12A_R1.fq\n",
      "-rw-r--r-- 1 root root  815M Jul 20 08:32 1HL_12A_R2.fq\n",
      "-rw-r--r-- 1 root root  697M Jul 20 16:59 1HL_13A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  328M Jul 20 16:59 1HL_13A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:57 1HL_13A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:57 1HL_13A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  882M Jul 20 08:33 1HL_13A_R1.fq\n",
      "-rw-r--r-- 1 root root  932M Jul 20 08:33 1HL_13A_R2.fq\n",
      "-rw-r--r-- 1 root root  487M Jul 20 17:00 1HL_14A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  251M Jul 20 17:00 1HL_14A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:59 1HL_14A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 16:59 1HL_14A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  622M Jul 20 08:33 1HL_14A_R1.fq\n",
      "-rw-r--r-- 1 root root  663M Jul 20 08:34 1HL_14A_R2.fq\n",
      "-rw-r--r-- 1 root root  490M Jul 20 17:02 1HL_15A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  277M Jul 20 17:02 1HL_15A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:00 1HL_15A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:00 1HL_15A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  636M Jul 20 08:34 1HL_15A_R1.fq\n",
      "-rw-r--r-- 1 root root  677M Jul 20 08:34 1HL_15A_R2.fq\n",
      "-rw-r--r-- 1 root root  551M Jul 20 17:03 1HL_16A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  290M Jul 20 17:03 1HL_16A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:02 1HL_16A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:02 1HL_16A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  707M Jul 20 08:35 1HL_16A_R1.fq\n",
      "-rw-r--r-- 1 root root  753M Jul 20 08:35 1HL_16A_R2.fq\n",
      "-rw-r--r-- 1 root root  826M Jul 20 17:05 1HL_17A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  451M Jul 20 17:05 1HL_17A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:03 1HL_17A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:03 1HL_17A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:36 1HL_17A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:36 1HL_17A_R2.fq\n",
      "-rw-r--r-- 1 root root  841M Jul 20 17:07 1HL_19A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  437M Jul 20 17:07 1HL_19A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:05 1HL_19A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:05 1HL_19A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:37 1HL_19A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:37 1HL_19A_R2.fq\n",
      "-rw-r--r-- 1 root root  549M Jul 20 17:09 1HL_1A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  278M Jul 20 17:09 1HL_1A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:07 1HL_1A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:07 1HL_1A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  703M Jul 20 08:24 1HL_1A_R1.fq\n",
      "-rw-r--r-- 1 root root  743M Jul 20 08:24 1HL_1A_R2.fq\n",
      "-rw-r--r-- 1 root root  666M Jul 20 17:11 1HL_20A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  358M Jul 20 17:11 1HL_20A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:09 1HL_20A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:09 1HL_20A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  855M Jul 20 08:38 1HL_20A_R1.fq\n",
      "-rw-r--r-- 1 root root  911M Jul 20 08:38 1HL_20A_R2.fq\n",
      "-rw-r--r-- 1 root root  585M Jul 20 17:12 1HL_21A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  328M Jul 20 17:12 1HL_21A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:11 1HL_21A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:11 1HL_21A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  759M Jul 20 08:38 1HL_21A_R1.fq\n",
      "-rw-r--r-- 1 root root  809M Jul 20 08:39 1HL_21A_R2.fq\n",
      "-rw-r--r-- 1 root root  478M Jul 20 17:14 1HL_22A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  262M Jul 20 17:14 1HL_22A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:12 1HL_22A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:12 1HL_22A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  617M Jul 20 08:39 1HL_22A_R1.fq\n",
      "-rw-r--r-- 1 root root  658M Jul 20 08:39 1HL_22A_R2.fq\n",
      "-rw-r--r-- 1 root root  828M Jul 20 17:16 1HL_23A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  442M Jul 20 17:16 1HL_23A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:14 1HL_23A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:14 1HL_23A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:40 1HL_23A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:40 1HL_23A_R2.fq\n",
      "-rw-r--r-- 1 root root  609M Jul 20 17:17 1HL_24A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  344M Jul 20 17:17 1HL_24A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:16 1HL_24A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:16 1HL_24A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  791M Jul 20 08:41 1HL_24A_R1.fq\n",
      "-rw-r--r-- 1 root root  843M Jul 20 08:41 1HL_24A_R2.fq\n",
      "-rw-r--r-- 1 root root  688M Jul 20 17:19 1HL_25A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  362M Jul 20 17:19 1HL_25A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:17 1HL_25A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:17 1HL_25A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  878M Jul 20 08:42 1HL_25A_R1.fq\n",
      "-rw-r--r-- 1 root root  936M Jul 20 08:42 1HL_25A_R2.fq\n",
      "-rw-r--r-- 1 root root  762M Jul 20 17:21 1HL_26A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  381M Jul 20 17:21 1HL_26A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:19 1HL_26A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:19 1HL_26A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  964M Jul 20 08:42 1HL_26A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:43 1HL_26A_R2.fq\n",
      "-rw-r--r-- 1 root root  759M Jul 20 17:23 1HL_27A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  384M Jul 20 17:23 1HL_27A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:21 1HL_27A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:21 1HL_27A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  960M Jul 20 08:43 1HL_27A_R1.fq\n",
      "-rw-r--r-- 1 root root 1023M Jul 20 08:44 1HL_27A_R2.fq\n",
      "-rw-r--r-- 1 root root  847M Jul 20 17:25 1HL_28A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  472M Jul 20 17:25 1HL_28A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:23 1HL_28A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:23 1HL_28A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:44 1HL_28A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:45 1HL_28A_R2.fq\n",
      "-rw-r--r-- 1 root root  776M Jul 20 17:27 1HL_29A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  376M Jul 20 17:27 1HL_29A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:25 1HL_29A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:25 1HL_29A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  981M Jul 20 08:45 1HL_29A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:46 1HL_29A_R2.fq\n",
      "-rw-r--r-- 1 root root  664M Jul 20 17:29 1HL_2A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  325M Jul 20 17:29 1HL_2A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:27 1HL_2A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:27 1HL_2A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  838M Jul 20 08:24 1HL_2A_R1.fq\n",
      "-rw-r--r-- 1 root root  886M Jul 20 08:25 1HL_2A_R2.fq\n",
      "-rw-r--r-- 1 root root  579M Jul 20 17:31 1HL_31A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  297M Jul 20 17:31 1HL_31A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:29 1HL_31A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:29 1HL_31A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  736M Jul 20 08:46 1HL_31A_R1.fq\n",
      "-rw-r--r-- 1 root root  784M Jul 20 08:46 1HL_31A_R2.fq\n",
      "-rw-r--r-- 1 root root  624M Jul 20 17:32 1HL_33A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  333M Jul 20 17:32 1HL_33A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:31 1HL_33A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:31 1HL_33A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  799M Jul 20 08:47 1HL_33A_R1.fq\n",
      "-rw-r--r-- 1 root root  851M Jul 20 08:47 1HL_33A_R2.fq\n",
      "-rw-r--r-- 1 root root  654M Jul 20 17:34 1HL_34A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  360M Jul 20 17:34 1HL_34A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:32 1HL_34A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:32 1HL_34A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  843M Jul 20 08:47 1HL_34A_R1.fq\n",
      "-rw-r--r-- 1 root root  899M Jul 20 08:48 1HL_34A_R2.fq\n",
      "-rw-r--r-- 1 root root  591M Jul 20 17:36 1HL_35A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  328M Jul 20 17:36 1HL_35A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:34 1HL_35A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:34 1HL_35A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  764M Jul 20 08:48 1HL_35A_R1.fq\n",
      "-rw-r--r-- 1 root root  814M Jul 20 08:49 1HL_35A_R2.fq\n",
      "-rw-r--r-- 1 root root  525M Jul 20 17:37 1HL_3A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  272M Jul 20 17:37 1HL_3A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:36 1HL_3A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:36 1HL_3A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  674M Jul 20 08:25 1HL_3A_R1.fq\n",
      "-rw-r--r-- 1 root root  712M Jul 20 08:25 1HL_3A_R2.fq\n",
      "-rw-r--r-- 1 root root  648M Jul 20 17:39 1HL_4A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  310M Jul 20 17:39 1HL_4A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:37 1HL_4A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:37 1HL_4A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  819M Jul 20 08:26 1HL_4A_R1.fq\n",
      "-rw-r--r-- 1 root root  866M Jul 20 08:26 1HL_4A_R2.fq\n",
      "-rw-r--r-- 1 root root  497M Jul 20 17:40 1HL_5A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  246M Jul 20 17:40 1HL_5A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:39 1HL_5A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:39 1HL_5A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  635M Jul 20 08:26 1HL_5A_R1.fq\n",
      "-rw-r--r-- 1 root root  671M Jul 20 08:27 1HL_5A_R2.fq\n",
      "-rw-r--r-- 1 root root  473M Jul 20 17:42 1HL_6A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  269M Jul 20 17:42 1HL_6A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:40 1HL_6A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:40 1HL_6A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  621M Jul 20 08:27 1HL_6A_R1.fq\n",
      "-rw-r--r-- 1 root root  657M Jul 20 08:27 1HL_6A_R2.fq\n",
      "-rw-r--r-- 1 root root  671M Jul 20 17:44 1HL_7A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  346M Jul 20 17:44 1HL_7A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:42 1HL_7A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:42 1HL_7A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  861M Jul 20 08:28 1HL_7A_R1.fq\n",
      "-rw-r--r-- 1 root root  910M Jul 20 08:28 1HL_7A_R2.fq\n",
      "-rw-r--r-- 1 root root  610M Jul 20 17:45 1HL_8A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  327M Jul 20 17:45 1HL_8A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:44 1HL_8A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:44 1HL_8A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  794M Jul 20 08:28 1HL_8A_R1.fq\n",
      "-rw-r--r-- 1 root root  840M Jul 20 08:29 1HL_8A_R2.fq\n",
      "-rw-r--r-- 1 root root  687M Jul 20 17:47 1HL_9A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  345M Jul 20 17:47 1HL_9A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:45 1HL_9A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:45 1HL_9A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  876M Jul 20 08:29 1HL_9A_R1.fq\n",
      "-rw-r--r-- 1 root root  926M Jul 20 08:30 1HL_9A_R2.fq\n",
      "-rw-r--r-- 1 root root  518M Jul 20 17:49 1NF_10A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  272M Jul 20 17:49 1NF_10A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:47 1NF_10A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:47 1NF_10A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  681M Jul 20 07:39 1NF_10A_R1.fq\n",
      "-rw-r--r-- 1 root root  703M Jul 20 07:40 1NF_10A_R2.fq\n",
      "-rw-r--r-- 1 root root  535M Jul 20 17:50 1NF_11A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  285M Jul 20 17:50 1NF_11A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:49 1NF_11A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:49 1NF_11A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  703M Jul 20 07:40 1NF_11A_R1.fq\n",
      "-rw-r--r-- 1 root root  725M Jul 20 07:40 1NF_11A_R2.fq\n",
      "-rw-r--r-- 1 root root  540M Jul 20 17:52 1NF_12A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  268M Jul 20 17:52 1NF_12A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:50 1NF_12A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:50 1NF_12A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  701M Jul 20 07:41 1NF_12A_R1.fq\n",
      "-rw-r--r-- 1 root root  723M Jul 20 07:41 1NF_12A_R2.fq\n",
      "-rw-r--r-- 1 root root  970M Jul 20 17:54 1NF_13A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  460M Jul 20 17:54 1NF_13A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:52 1NF_13A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:52 1NF_13A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 07:41 1NF_13A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.3G Jul 20 07:42 1NF_13A_R2.fq\n",
      "-rw-r--r-- 1 root root  625M Jul 20 17:56 1NF_14A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  345M Jul 20 17:56 1NF_14A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:54 1NF_14A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:54 1NF_14A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  822M Jul 20 07:42 1NF_14A_R1.fq\n",
      "-rw-r--r-- 1 root root  855M Jul 20 07:42 1NF_14A_R2.fq\n",
      "-rw-r--r-- 1 root root  610M Jul 20 17:58 1NF_15A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  319M Jul 20 17:58 1NF_15A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:56 1NF_15A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:56 1NF_15A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  793M Jul 20 07:43 1NF_15A_R1.fq\n",
      "-rw-r--r-- 1 root root  825M Jul 20 07:43 1NF_15A_R2.fq\n",
      "-rw-r--r-- 1 root root  700M Jul 20 18:00 1NF_16A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  339M Jul 20 18:00 1NF_16A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:58 1NF_16A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 17:58 1NF_16A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  904M Jul 20 07:44 1NF_16A_R1.fq\n",
      "-rw-r--r-- 1 root root  941M Jul 20 07:44 1NF_16A_R2.fq\n",
      "-rw-r--r-- 1 root root  752M Jul 20 18:02 1NF_17A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  388M Jul 20 18:02 1NF_17A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:00 1NF_17A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:00 1NF_17A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  977M Jul 20 07:44 1NF_17A_R1.fq\n",
      "-rw-r--r-- 1 root root 1017M Jul 20 07:45 1NF_17A_R2.fq\n",
      "-rw-r--r-- 1 root root  396M Jul 20 18:03 1NF_18A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  221M Jul 20 18:03 1NF_18A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:02 1NF_18A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:02 1NF_18A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  521M Jul 20 07:45 1NF_18A_R1.fq\n",
      "-rw-r--r-- 1 root root  542M Jul 20 07:45 1NF_18A_R2.fq\n",
      "-rw-r--r-- 1 root root  472M Jul 20 18:04 1NF_19A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  265M Jul 20 18:04 1NF_19A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:03 1NF_19A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:03 1NF_19A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  627M Jul 20 07:46 1NF_19A_R1.fq\n",
      "-rw-r--r-- 1 root root  653M Jul 20 07:46 1NF_19A_R2.fq\n",
      "-rw-r--r-- 1 root root  310M Jul 20 18:05 1NF_1A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  179M Jul 20 18:05 1NF_1A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:04 1NF_1A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:04 1NF_1A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  417M Jul 20 07:34 1NF_1A_R1.fq\n",
      "-rw-r--r-- 1 root root  430M Jul 20 07:34 1NF_1A_R2.fq\n",
      "-rw-r--r-- 1 root root  596M Jul 20 18:07 1NF_20A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  308M Jul 20 18:07 1NF_20A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:05 1NF_20A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:05 1NF_20A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  776M Jul 20 07:46 1NF_20A_R1.fq\n",
      "-rw-r--r-- 1 root root  808M Jul 20 07:47 1NF_20A_R2.fq\n",
      "-rw-r--r-- 1 root root  764M Jul 20 18:09 1NF_21A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  357M Jul 20 18:09 1NF_21A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:07 1NF_21A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:07 1NF_21A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  966M Jul 20 07:47 1NF_21A_R1.fq\n",
      "-rw-r--r-- 1 root root 1005M Jul 20 07:47 1NF_21A_R2.fq\n",
      "-rw-r--r-- 1 root root  706M Jul 20 18:11 1NF_22A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  351M Jul 20 18:11 1NF_22A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:09 1NF_22A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:09 1NF_22A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  908M Jul 20 07:48 1NF_22A_R1.fq\n",
      "-rw-r--r-- 1 root root  944M Jul 20 07:48 1NF_22A_R2.fq\n",
      "-rw-r--r-- 1 root root  813M Jul 20 18:13 1NF_23A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  386M Jul 20 18:13 1NF_23A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:11 1NF_23A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:11 1NF_23A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:48 1NF_23A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:49 1NF_23A_R2.fq\n",
      "-rw-r--r-- 1 root root  709M Jul 20 18:15 1NF_24A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  376M Jul 20 18:15 1NF_24A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:13 1NF_24A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:13 1NF_24A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  922M Jul 20 07:49 1NF_24A_R1.fq\n",
      "-rw-r--r-- 1 root root  959M Jul 20 07:50 1NF_24A_R2.fq\n",
      "-rw-r--r-- 1 root root  503M Jul 20 18:17 1NF_25A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  283M Jul 20 18:17 1NF_25A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:15 1NF_25A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:15 1NF_25A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  666M Jul 20 07:50 1NF_25A_R1.fq\n",
      "-rw-r--r-- 1 root root  693M Jul 20 07:50 1NF_25A_R2.fq\n",
      "-rw-r--r-- 1 root root  695M Jul 20 18:19 1NF_26A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  360M Jul 20 18:19 1NF_26A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:17 1NF_26A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:17 1NF_26A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  900M Jul 20 07:51 1NF_26A_R1.fq\n",
      "-rw-r--r-- 1 root root  937M Jul 20 07:51 1NF_26A_R2.fq\n",
      "-rw-r--r-- 1 root root  567M Jul 20 18:20 1NF_27A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  284M Jul 20 18:20 1NF_27A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:19 1NF_27A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:19 1NF_27A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  733M Jul 20 07:51 1NF_27A_R1.fq\n",
      "-rw-r--r-- 1 root root  762M Jul 20 07:52 1NF_27A_R2.fq\n",
      "-rw-r--r-- 1 root root  583M Jul 20 18:22 1NF_28A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  297M Jul 20 18:22 1NF_28A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:20 1NF_28A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:20 1NF_28A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  758M Jul 20 07:52 1NF_28A_R1.fq\n",
      "-rw-r--r-- 1 root root  788M Jul 20 07:52 1NF_28A_R2.fq\n",
      "-rw-r--r-- 1 root root  929M Jul 20 18:24 1NF_29A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  485M Jul 20 18:24 1NF_29A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:22 1NF_29A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:22 1NF_29A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 07:53 1NF_29A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.3G Jul 20 07:53 1NF_29A_R2.fq\n",
      "-rw-r--r-- 1 root root  834M Jul 20 18:26 1NF_2A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  425M Jul 20 18:26 1NF_2A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:24 1NF_2A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:24 1NF_2A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:34 1NF_2A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:34 1NF_2A_R2.fq\n",
      "-rw-r--r-- 1 root root  725M Jul 20 18:28 1NF_30A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  335M Jul 20 18:28 1NF_30A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:26 1NF_30A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:26 1NF_30A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  930M Jul 20 07:54 1NF_30A_R1.fq\n",
      "-rw-r--r-- 1 root root  967M Jul 20 07:54 1NF_30A_R2.fq\n",
      "-rw-r--r-- 1 root root  848M Jul 20 18:30 1NF_31A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  439M Jul 20 18:30 1NF_31A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:28 1NF_31A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:28 1NF_31A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:55 1NF_31A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 07:55 1NF_31A_R2.fq\n",
      "-rw-r--r-- 1 root root  860M Jul 20 18:33 1NF_32A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  420M Jul 20 18:33 1NF_32A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:30 1NF_32A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:30 1NF_32A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:56 1NF_32A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 07:56 1NF_32A_R2.fq\n",
      "-rw-r--r-- 1 root root  649M Jul 20 18:34 1NF_33A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  321M Jul 20 18:34 1NF_33A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:33 1NF_33A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:33 1NF_33A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  837M Jul 20 07:56 1NF_33A_R1.fq\n",
      "-rw-r--r-- 1 root root  870M Jul 20 07:57 1NF_33A_R2.fq\n",
      "-rw-r--r-- 1 root root  868M Jul 20 18:37 1NF_4A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  438M Jul 20 18:37 1NF_4A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:34 1NF_4A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:34 1NF_4A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:35 1NF_4A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 07:35 1NF_4A_R2.fq\n",
      "-rw-r--r-- 1 root root  651M Jul 20 18:38 1NF_5A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  353M Jul 20 18:38 1NF_5A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:37 1NF_5A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:37 1NF_5A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  860M Jul 20 07:35 1NF_5A_R1.fq\n",
      "-rw-r--r-- 1 root root  887M Jul 20 07:36 1NF_5A_R2.fq\n",
      "-rw-r--r-- 1 root root  590M Jul 20 18:40 1NF_6A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  301M Jul 20 18:40 1NF_6A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:38 1NF_6A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:38 1NF_6A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  767M Jul 20 07:36 1NF_6A_R1.fq\n",
      "-rw-r--r-- 1 root root  791M Jul 20 07:37 1NF_6A_R2.fq\n",
      "-rw-r--r-- 1 root root  642M Jul 20 18:42 1NF_7A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  319M Jul 20 18:42 1NF_7A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:40 1NF_7A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:40 1NF_7A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  831M Jul 20 07:37 1NF_7A_R1.fq\n",
      "-rw-r--r-- 1 root root  858M Jul 20 07:37 1NF_7A_R2.fq\n",
      "-rw-r--r-- 1 root root  651M Jul 20 18:44 1NF_8A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  351M Jul 20 18:44 1NF_8A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:42 1NF_8A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:42 1NF_8A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  858M Jul 20 07:38 1NF_8A_R1.fq\n",
      "-rw-r--r-- 1 root root  886M Jul 20 07:38 1NF_8A_R2.fq\n",
      "-rw-r--r-- 1 root root  802M Jul 20 18:46 1NF_9A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  419M Jul 20 18:46 1NF_9A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:44 1NF_9A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:44 1NF_9A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:39 1NF_9A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 07:39 1NF_9A_R2.fq\n",
      "-rw-r--r-- 1 root root  631M Jul 20 18:47 1SN_10A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  308M Jul 20 18:47 1SN_10A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:46 1SN_10A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:46 1SN_10A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  804M Jul 20 08:05 1SN_10A_R1.fq\n",
      "-rw-r--r-- 1 root root  843M Jul 20 08:05 1SN_10A_R2.fq\n",
      "-rw-r--r-- 1 root root  647M Jul 20 18:49 1SN_11A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  308M Jul 20 18:49 1SN_11A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:47 1SN_11A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:47 1SN_11A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  823M Jul 20 08:06 1SN_11A_R1.fq\n",
      "-rw-r--r-- 1 root root  863M Jul 20 08:06 1SN_11A_R2.fq\n",
      "-rw-r--r-- 1 root root  809M Jul 20 18:51 1SN_12A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  412M Jul 20 18:51 1SN_12A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:49 1SN_12A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:49 1SN_12A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:06 1SN_12A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:07 1SN_12A_R2.fq\n",
      "-rw-r--r-- 1 root root  706M Jul 20 18:53 1SN_13A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  361M Jul 20 18:53 1SN_13A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:51 1SN_13A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:51 1SN_13A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  906M Jul 20 08:08 1SN_13A_R1.fq\n",
      "-rw-r--r-- 1 root root  950M Jul 20 08:08 1SN_13A_R2.fq\n",
      "-rw-r--r-- 1 root root  850M Jul 20 18:55 1SN_14A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  429M Jul 20 18:55 1SN_14A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:53 1SN_14A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:53 1SN_14A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:08 1SN_14A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:09 1SN_14A_R2.fq\n",
      "-rw-r--r-- 1 root root  572M Jul 20 18:57 1SN_15A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  311M Jul 20 18:57 1SN_15A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:55 1SN_15A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:55 1SN_15A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  743M Jul 20 08:09 1SN_15A_R1.fq\n",
      "-rw-r--r-- 1 root root  780M Jul 20 08:10 1SN_15A_R2.fq\n",
      "-rw-r--r-- 1 root root  615M Jul 20 18:58 1SN_16A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  325M Jul 20 18:58 1SN_16A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:57 1SN_16A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:57 1SN_16A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  797M Jul 20 08:10 1SN_16A_R1.fq\n",
      "-rw-r--r-- 1 root root  835M Jul 20 08:10 1SN_16A_R2.fq\n",
      "-rw-r--r-- 1 root root  688M Jul 20 19:00 1SN_17A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  371M Jul 20 19:00 1SN_17A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:58 1SN_17A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 18:58 1SN_17A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  893M Jul 20 08:11 1SN_17A_R1.fq\n",
      "-rw-r--r-- 1 root root  937M Jul 20 08:11 1SN_17A_R2.fq\n",
      "-rw-r--r-- 1 root root  782M Jul 20 19:02 1SN_18A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  391M Jul 20 19:02 1SN_18A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:00 1SN_18A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:00 1SN_18A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  998M Jul 20 08:12 1SN_18A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:12 1SN_18A_R2.fq\n",
      "-rw-r--r-- 1 root root  730M Jul 20 19:04 1SN_19A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  367M Jul 20 19:04 1SN_19A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:02 1SN_19A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:02 1SN_19A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  934M Jul 20 08:12 1SN_19A_R1.fq\n",
      "-rw-r--r-- 1 root root  980M Jul 20 08:13 1SN_19A_R2.fq\n",
      "-rw-r--r-- 1 root root  712M Jul 20 19:06 1SN_1A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  383M Jul 20 19:06 1SN_1A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:04 1SN_1A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:04 1SN_1A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  931M Jul 20 07:57 1SN_1A_R1.fq\n",
      "-rw-r--r-- 1 root root  968M Jul 20 07:58 1SN_1A_R2.fq\n",
      "-rw-r--r-- 1 root root  555M Jul 20 19:07 1SN_20A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  261M Jul 20 19:07 1SN_20A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:06 1SN_20A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:06 1SN_20A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  706M Jul 20 08:13 1SN_20A_R1.fq\n",
      "-rw-r--r-- 1 root root  740M Jul 20 08:14 1SN_20A_R2.fq\n",
      "-rw-r--r-- 1 root root  819M Jul 20 19:10 1SN_21A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  435M Jul 20 19:10 1SN_21A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:07 1SN_21A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:07 1SN_21A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:14 1SN_21A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:14 1SN_21A_R2.fq\n",
      "-rw-r--r-- 1 root root  746M Jul 20 19:11 1SN_22A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  367M Jul 20 19:11 1SN_22A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:10 1SN_22A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:10 1SN_22A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  950M Jul 20 08:15 1SN_22A_R1.fq\n",
      "-rw-r--r-- 1 root root  996M Jul 20 08:15 1SN_22A_R2.fq\n",
      "-rw-r--r-- 1 root root  877M Jul 20 19:14 1SN_23A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  439M Jul 20 19:14 1SN_23A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:11 1SN_23A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:11 1SN_23A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:16 1SN_23A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:16 1SN_23A_R2.fq\n",
      "-rw-r--r-- 1 root root  432M Jul 20 19:15 1SN_24A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  241M Jul 20 19:15 1SN_24A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:14 1SN_24A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:14 1SN_24A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  566M Jul 20 08:17 1SN_24A_R1.fq\n",
      "-rw-r--r-- 1 root root  599M Jul 20 08:17 1SN_24A_R2.fq\n",
      "-rw-r--r-- 1 root root  690M Jul 20 19:17 1SN_25A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  380M Jul 20 19:17 1SN_25A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:15 1SN_25A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:15 1SN_25A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  896M Jul 20 08:17 1SN_25A_R1.fq\n",
      "-rw-r--r-- 1 root root  947M Jul 20 08:18 1SN_25A_R2.fq\n",
      "-rw-r--r-- 1 root root  638M Jul 20 19:18 1SN_26A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  339M Jul 20 19:18 1SN_26A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:17 1SN_26A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:17 1SN_26A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  821M Jul 20 08:18 1SN_26A_R1.fq\n",
      "-rw-r--r-- 1 root root  868M Jul 20 08:19 1SN_26A_R2.fq\n",
      "-rw-r--r-- 1 root root  598M Jul 20 19:20 1SN_27A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  332M Jul 20 19:20 1SN_27A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:18 1SN_27A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:18 1SN_27A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  781M Jul 20 08:19 1SN_27A_R1.fq\n",
      "-rw-r--r-- 1 root root  825M Jul 20 08:19 1SN_27A_R2.fq\n",
      "-rw-r--r-- 1 root root  662M Jul 20 19:22 1SN_28A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  352M Jul 20 19:22 1SN_28A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:20 1SN_28A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:20 1SN_28A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  854M Jul 20 08:20 1SN_28A_R1.fq\n",
      "-rw-r--r-- 1 root root  903M Jul 20 08:20 1SN_28A_R2.fq\n",
      "-rw-r--r-- 1 root root  505M Jul 20 19:23 1SN_29A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  301M Jul 20 19:23 1SN_29A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:22 1SN_29A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:22 1SN_29A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  674M Jul 20 08:20 1SN_29A_R1.fq\n",
      "-rw-r--r-- 1 root root  713M Jul 20 08:21 1SN_29A_R2.fq\n",
      "-rw-r--r-- 1 root root  738M Jul 20 19:25 1SN_2A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  375M Jul 20 19:25 1SN_2A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:23 1SN_2A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:23 1SN_2A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  948M Jul 20 07:58 1SN_2A_R1.fq\n",
      "-rw-r--r-- 1 root root  994M Jul 20 07:59 1SN_2A_R2.fq\n",
      "-rw-r--r-- 1 root root  917M Jul 20 19:28 1SN_30A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  452M Jul 20 19:28 1SN_30A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:25 1SN_30A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:25 1SN_30A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:21 1SN_30A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.2G Jul 20 08:22 1SN_30A_R2.fq\n",
      "-rw-r--r-- 1 root root  790M Jul 20 19:30 1SN_31A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  409M Jul 20 19:30 1SN_31A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:28 1SN_31A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:28 1SN_31A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root 1012M Jul 20 08:22 1SN_31A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:23 1SN_31A_R2.fq\n",
      "-rw-r--r-- 1 root root  596M Jul 20 19:31 1SN_32A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  317M Jul 20 19:31 1SN_32A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:30 1SN_32A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:30 1SN_32A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  770M Jul 20 08:23 1SN_32A_R1.fq\n",
      "-rw-r--r-- 1 root root  814M Jul 20 08:23 1SN_32A_R2.fq\n",
      "-rw-r--r-- 1 root root  557M Jul 20 19:33 1SN_3A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  312M Jul 20 19:33 1SN_3A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:31 1SN_3A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:31 1SN_3A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  730M Jul 20 07:59 1SN_3A_R1.fq\n",
      "-rw-r--r-- 1 root root  765M Jul 20 07:59 1SN_3A_R2.fq\n",
      "-rw-r--r-- 1 root root  790M Jul 20 19:35 1SN_4A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  388M Jul 20 19:35 1SN_4A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:33 1SN_4A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:33 1SN_4A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root 1003M Jul 20 08:00 1SN_4A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:00 1SN_4A_R2.fq\n",
      "-rw-r--r-- 1 root root  716M Jul 20 19:37 1SN_5A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  359M Jul 20 19:37 1SN_5A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:35 1SN_5A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:35 1SN_5A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  912M Jul 20 08:01 1SN_5A_R1.fq\n",
      "-rw-r--r-- 1 root root  957M Jul 20 08:01 1SN_5A_R2.fq\n",
      "-rw-r--r-- 1 root root  806M Jul 20 19:39 1SN_6A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  402M Jul 20 19:39 1SN_6A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:37 1SN_6A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:37 1SN_6A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:01 1SN_6A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:02 1SN_6A_R2.fq\n",
      "-rw-r--r-- 1 root root  717M Jul 20 19:41 1SN_7A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  358M Jul 20 19:41 1SN_7A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:39 1SN_7A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:39 1SN_7A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  915M Jul 20 08:02 1SN_7A_R1.fq\n",
      "-rw-r--r-- 1 root root  959M Jul 20 08:03 1SN_7A_R2.fq\n",
      "-rw-r--r-- 1 root root  813M Jul 20 19:43 1SN_8A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  398M Jul 20 19:43 1SN_8A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:41 1SN_8A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:41 1SN_8A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:03 1SN_8A_R1.fq\n",
      "-rw-r--r-- 1 root root  1.1G Jul 20 08:04 1SN_8A_R2.fq\n",
      "-rw-r--r-- 1 root root  767M Jul 20 19:45 1SN_9A.assembled.fastq\n",
      "-rw-r--r-- 1 root root  379M Jul 20 19:45 1SN_9A.discarded.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:43 1SN_9A.unassembled.forward.fastq\n",
      "-rw-r--r-- 1 root root     0 Jul 20 19:43 1SN_9A.unassembled.reverse.fastq\n",
      "-rw-r--r-- 1 root root  976M Jul 20 08:04 1SN_9A_R1.fq\n",
      "-rw-r--r-- 1 root root 1023M Jul 20 08:04 1SN_9A_R2.fq\n"
     ]
    }
   ],
   "source": [
    "ls -lh /data/analysis/fastq/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the data location to the de-multiplexed & merged data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "## set location of demultiplexed data that are 'pear' filtered\n",
    "sed -i '/## 18./c\\/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "8                   ## 7. N processors   \n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "pairgbs            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the datatype to \"merged\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sed -i '/## 11./c\\merged            ## 11. data type ' /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==\n",
      "/data/analysis/                     ## 1. Working directory                                 (all)\n",
      "/data/analysis/             \t       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)\n",
      "/data/analysis/barcodes.txt\t               ## 3. Loc. of barcode file (if not line 18)             (s1)\n",
      "/usr/local/bioinformatics/vsearch-1.11.1-linux-x86_64/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)\n",
      "/usr/local/bioinformatics/muscle3.8.31_i86linux64                    ## 5. command (or path) to call muscle                  (s3,s7)\n",
      "CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)\n",
      "8                   ## 7. N processors   \n",
      "6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)\n",
      "4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)\n",
      ".88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)\n",
      "merged            ## 11. data type \n",
      "4                      ## 12. MinCov: min samples in a final locus             (s7)\n",
      "3                      ## 13. MaxSH: max inds with shared hetero site          (s7)\n",
      "oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)\n",
      "==== optional params below this line ===================================  affected step ==\n",
      "                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)\n",
      "                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)\n",
      "                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)\n",
      "/data/analysis/fastq/*.assembled.fastq ## 18. demulti data loc \n",
      "                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)\n",
      "                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)\n",
      "2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)\n",
      "                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)\n",
      "4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)\n",
      "8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)\n",
      "                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)\n",
      "                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)\n",
      "                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)\n",
      "                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)\n",
      "                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)\n",
      "*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)\n",
      "                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)\n",
      "50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)\n",
      "                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)\n",
      "                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)\n",
      "                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)\n",
      "                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)\n",
      "16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)\n",
      "==== optional: list group/clade assignments below this line (see docs) ==================\n",
      "1HL\t5\t1HL*\n",
      "1NF\t5\t1NF*\n",
      "1SN\t5\t1SN*\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/params.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Quality Filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t/data/analysis/edits/1HL_17A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_11A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_10A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_13A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_12A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_16A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_15A.assembled already in edits/\n",
      "\t/data/analysis/edits/1HL_14A.assembled already in edits/\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\tsorted .fastq from /data/analysis/fastq/*.assembled.fastq being used\n",
      "\tstep 2: editing raw reads \n",
      "\t........................................................................................\n",
      "real\t70m16.887s\n",
      "user\t453m14.412s\n",
      "sys\t1m16.616s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's take a look at the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample \tNreads\tpassed\tpassed.w.trim\tpassed.total\n",
      "1HL_10A.assembled\t2890999\t2127357\t214294\t2341651\n",
      "1HL_11A.assembled\t3088425\t2242089\t246561\t2488650\n",
      "1HL_12A.assembled\t2323231\t1728934\t163227\t1892161\n",
      "1HL_13A.assembled\t2701305\t2075000\t175929\t2250929\n",
      "1HL_14A.assembled\t1865526\t1407146\t129089\t1536235\n",
      "1HL_15A.assembled\t1837684\t1342114\t139898\t1482012\n",
      "1HL_16A.assembled\t2100838\t1568118\t147901\t1716019\n",
      "1HL_17A.assembled\t3106139\t2259592\t237499\t2497091\n",
      "\n",
      "    Nreads = total number of reads for a sample\n",
      "    passed = retained reads that passed quality filtering at full length\n",
      "    passed.w.trim= retained reads that were trimmed due to detection of adapters\n",
      "    passed.total  = total kept reads of sufficient length\n",
      "    note: you can set the option in params file to include trimmed reads of xx length. \n",
      "sample \tNreads\tpassed\tpassed.w.trim\tpassed.total\n",
      "1HL_19A.assembled\t3174884\t2334130\t238750\t2572880\n",
      "1HL_1A.assembled\t2098741\t1574224\t145441\t1719665\n",
      "1HL_20A.assembled\t2516582\t1852015\t188703\t2040718\n",
      "1HL_21A.assembled\t2203056\t1616145\t167632\t1783777\n",
      "1HL_22A.assembled\t1807508\t1343661\t129836\t1473497\n",
      "1HL_23A.assembled\t3117475\t2274454\t240445\t2514899\n",
      "1HL_24A.assembled\t2290319\t1671535\t175429\t1846964\n",
      "1HL_25A.assembled\t2606955\t1914854\t193248\t2108102\n",
      "1HL_26A.assembled\t2915184\t2177424\t206507\t2383931\n",
      "1HL_27A.assembled\t2887147\t2121325\t216634\t2337959\n",
      "1HL_28A.assembled\t3182673\t2272478\t240890\t2513368\n",
      "1HL_29A.assembled\t3004016\t2288387\t202722\t2491109\n",
      "1HL_2A.assembled\t2525685\t1858833\t189246\t2048079\n",
      "1HL_31A.assembled\t2203255\t1636345\t159656\t1796001\n",
      "1HL_33A.assembled\t2357587\t1733974\t178510\t1912484\n",
      "1HL_34A.assembled\t2461367\t1795163\t191006\t1986169\n",
      "1HL_35A.assembled\t2222135\t1621437\t168996\t1790433\n",
      "1HL_3A.assembled\t1994744\t1473692\t146406\t1620098\n",
      "1HL_4A.assembled\t2492589\t1890737\t171986\t2062723\n",
      "1HL_5A.assembled\t1914213\t1462615\t127309\t1589924\n",
      "1HL_6A.assembled\t1778884\t1322008\t130346\t1452354\n",
      "1HL_7A.assembled\t2555275\t1862044\t181746\t2043790\n",
      "1HL_8A.assembled\t2330048\t1756156\t162264\t1918420\n",
      "1HL_9A.assembled\t2620475\t1941412\t189818\t2131230\n",
      "1NF_10A.assembled\t1955600\t1455878\t144024\t1599902\n",
      "1NF_11A.assembled\t2004118\t1477037\t151957\t1628994\n",
      "1NF_12A.assembled\t2051515\t1531004\t146564\t1677568\n",
      "1NF_13A.assembled\t3671926\t2725655\t269586\t2995241\n",
      "1NF_14A.assembled\t2333572\t1708721\t178913\t1887634\n",
      "1NF_15A.assembled\t2296320\t1697717\t170815\t1868532\n",
      "1NF_16A.assembled\t2701311\t2082692\t176785\t2259477\n",
      "1NF_17A.assembled\t2846779\t2102287\t210452\t2312739\n",
      "1NF_18A.assembled\t1469499\t1065959\t116806\t1182765\n",
      "1NF_19A.assembled\t1775039\t1310095\t133947\t1444042\n",
      "1NF_1A.assembled\t1157346\t850786\t89996\t940782\n",
      "1NF_20A.assembled\t2261913\t1689225\t164421\t1853646\n",
      "1NF_21A.assembled\t2903774\t2133792\t221401\t2355193\n",
      "1NF_22A.assembled\t2674529\t1978558\t197203\t2175761\n",
      "1NF_23A.assembled\t3106447\t2328959\t222988\t2551947\n",
      "1NF_24A.assembled\t2650885\t1941071\t205135\t2146206\n",
      "1NF_25A.assembled\t1879642\t1378557\t146071\t1524628\n",
      "1NF_26A.assembled\t2616210\t1900022\t206048\t2106070\n",
      "1NF_27A.assembled\t2158729\t1630790\t151335\t1782125\n",
      "1NF_28A.assembled\t2221247\t1685941\t149698\t1835639\n",
      "1NF_29A.assembled\t3514784\t2617734\t257383\t2875117\n",
      "1NF_2A.assembled\t3135623\t2306002\t239165\t2545167\n",
      "1NF_30A.assembled\t2821297\t2180592\t179533\t2360125\n",
      "1NF_31A.assembled\t3191552\t2338303\t244951\t2583254\n",
      "1NF_32A.assembled\t3264224\t2413332\t239270\t2652602\n",
      "1NF_33A.assembled\t2473981\t1850165\t178316\t2028481\n",
      "1NF_4A.assembled\t3260931\t2396178\t247926\t2644104\n",
      "1NF_5A.assembled\t2436600\t1796651\t190767\t1987418\n",
      "1NF_6A.assembled\t2221155\t1650625\t164512\t1815137\n",
      "1NF_7A.assembled\t2431311\t1825704\t174568\t2000272\n",
      "1NF_8A.assembled\t2436005\t1788363\t187953\t1976316\n",
      "1NF_9A.assembled\t3008281\t2205187\t231508\t2436695\n",
      "1SN_10A.assembled\t2404256\t1798762\t169244\t1968006\n",
      "1SN_11A.assembled\t2486121\t1856100\t176791\t2032891\n",
      "1SN_12A.assembled\t3042628\t2223439\t229758\t2453197\n",
      "1SN_13A.assembled\t2667803\t1973010\t195834\t2168844\n",
      "1SN_14A.assembled\t3219771\t2373958\t236954\t2610912\n",
      "1SN_15A.assembled\t2138651\t1557006\t167519\t1724525\n",
      "1SN_16A.assembled\t2321176\t1702395\t174707\t1877102\n",
      "1SN_17A.assembled\t2578853\t1887817\t197186\t2085003\n",
      "1SN_18A.assembled\t2962022\t2185019\t219331\t2404350\n",
      "1SN_19A.assembled\t2768323\t2042906\t208850\t2251756\n",
      "1SN_1A.assembled\t2667969\t1966508\t204226\t2170734\n",
      "1SN_20A.assembled\t2144390\t1629218\t144236\t1773454\n",
      "1SN_21A.assembled\t3074446\t2240437\t241641\t2482078\n",
      "1SN_22A.assembled\t2834716\t2101705\t206100\t2307805\n",
      "1SN_23A.assembled\t3302973\t2390188\t260212\t2650400\n",
      "1SN_24A.assembled\t1634612\t1212113\t118238\t1330351\n",
      "1SN_25A.assembled\t2592529\t1894147\t197899\t2092046\n",
      "1SN_26A.assembled\t2405605\t1760829\t182699\t1943528\n",
      "1SN_27A.assembled\t2257668\t1681310\t160915\t1842225\n",
      "1SN_28A.assembled\t2505410\t1846471\t186214\t2032685\n",
      "1SN_29A.assembled\t1902038\t1395597\t145369\t1540966\n",
      "1SN_2A.assembled\t2797966\t2073626\t202843\t2276469\n",
      "1SN_30A.assembled\t3478736\t2550190\t265223\t2815413\n",
      "1SN_31A.assembled\t2993047\t2215634\t218769\t2434403\n",
      "1SN_32A.assembled\t2259662\t1666918\t167017\t1833935\n",
      "1SN_3A.assembled\t2078939\t1527695\t157980\t1685675\n",
      "1SN_4A.assembled\t2993530\t2177510\t234644\t2412154\n",
      "1SN_5A.assembled\t2699104\t1976652\t205514\t2182166\n",
      "1SN_6A.assembled\t3050907\t2243129\t227550\t2470679\n",
      "1SN_7A.assembled\t2716067\t1990379\t203176\t2193555\n",
      "1SN_8A.assembled\t3117883\t2332672\t221072\t2553744\n",
      "1SN_9A.assembled\t2905903\t2126562\t223282\t2349844\n",
      "\n",
      "    Nreads = total number of reads for a sample\n",
      "    passed = retained reads that passed quality filtering at full length\n",
      "    passed.w.trim= retained reads that were trimmed due to detection of adapters\n",
      "    passed.total  = total kept reads of sufficient length\n",
      "    note: you can set the option in params file to include trimmed reads of xx length. \n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/s2.rawedit.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Clustering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tskipping 1HL_17A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_11A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_10A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_13A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_12A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_16A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_14A.assembled.clustS.gz already exists in /data/analysis/clust.88\n",
      "\tskipping 1HL_15A.assembled.clustS.gz already exists in /data/analysis/clust.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tde-replicating files for clustering...\n",
      "\n",
      "\tstep 3: within-sample clustering of 96 samples at \n",
      "\t        '.88' similarity. Running 8 parallel jobs\n",
      "\t \twith up to 16 threads per job. If needed, \n",
      "\t\tadjust to avoid CPU and MEM limits\n",
      "\n",
      "\tsample 1NF_29A finished, 97131 loci\n",
      "\tsample 1NF_31A finished, 109056 loci\n",
      "\tsample 1NF_4A finished, 110918 loci\n",
      "\tsample 1NF_32A finished, 115584 loci\n",
      "\tsample 1SN_14A finished, 110527 loci\n",
      "\tsample 1SN_23A finished, 118344 loci\n",
      "\tsample 1SN_30A finished, 121564 loci\n",
      "\tsample 1NF_13A finished, 127575 loci\n",
      "\tsample 1HL_19A finished, 111346 loci\n",
      "\tsample 1HL_23A finished, 94229 loci\n",
      "\tsample 1NF_2A finished, 102160 loci\n",
      "\tsample 1SN_8A finished, 102124 loci\n",
      "\tsample 1NF_23A finished, 122732 loci\n",
      "\tsample 1HL_28A finished, 104605 loci\n",
      "\tsample 1SN_21A finished, 131359 loci\n",
      "\tsample 1SN_12A finished, 116140 loci\n",
      "\tsample 1HL_29A finished, 93081 loci\n",
      "\tsample 1SN_6A finished, 107796 loci\n",
      "\tsample 1NF_9A finished, 103833 loci\n",
      "\tsample 1SN_31A finished, 103367 loci\n",
      "\tsample 1SN_18A finished, 113397 loci\n",
      "\tsample 1HL_26A finished, 109215 loci\n",
      "\tsample 1SN_4A finished, 111639 loci\n",
      "\tsample 1NF_21A finished, 161246 loci\n",
      "\tsample 1NF_30A finished, 92056 loci\n",
      "\tsample 1SN_9A finished, 126666 loci\n",
      "\tsample 1HL_27A finished, 99679 loci\n",
      "\tsample 1NF_17A finished, 108371 loci\n",
      "\tsample 1SN_22A finished, 107304 loci\n",
      "\tsample 1SN_2A finished, 106745 loci\n",
      "\tsample 1SN_19A finished, 120143 loci\n",
      "\tsample 1NF_16A finished, 101457 loci\n",
      "\tsample 1NF_22A finished, 108890 loci\n",
      "\tsample 1SN_1A finished, 95372 loci\n",
      "\tsample 1SN_13A finished, 106500 loci\n",
      "\tsample 1SN_7A finished, 107945 loci\n",
      "\tsample 1SN_5A finished, 121723 loci\n",
      "\tsample 1HL_25A finished, 89569 loci\n",
      "\tsample 1NF_24A finished, 109692 loci\n",
      "\tsample 1NF_26A finished, 105799 loci\n",
      "\tsample 1SN_25A finished, 120465 loci\n",
      "\tsample 1HL_9A finished, 113633 loci\n",
      "\tsample 1SN_17A finished, 106838 loci\n",
      "\tsample 1HL_20A finished, 96673 loci\n",
      "\tsample 1HL_4A finished, 93231 loci\n",
      "\tsample 1HL_2A finished, 93559 loci\n",
      "\tsample 1SN_28A finished, 96449 loci\n",
      "\tsample 1NF_33A finished, 91154 loci\n",
      "\tsample 1SN_11A finished, 104454 loci\n",
      "\tsample 1HL_34A finished, 90703 loci\n",
      "\tsample 1NF_5A finished, 97753 loci\n",
      "\tsample 1HL_7A finished, 100958 loci\n",
      "\tsample 1NF_7A finished, 96247 loci\n",
      "\tsample 1NF_8A finished, 98509 loci\n",
      "\tsample 1SN_10A finished, 107961 loci\n",
      "\tsample 1SN_26A finished, 100318 loci\n",
      "\tsample 1HL_33A finished, 90312 loci\n",
      "\tsample 1NF_14A finished, 99152 loci\n",
      "\tsample 1HL_8A finished, 87336 loci\n",
      "\tsample 1NF_15A finished, 93177 loci\n",
      "\tsample 1HL_24A finished, 88576 loci\n",
      "\tsample 1SN_16A finished, 96334 loci\n",
      "\tsample 1NF_20A finished, 91488 loci\n",
      "\tsample 1SN_27A finished, 96854 loci\n",
      "\tsample 1NF_28A finished, 92468 loci\n",
      "\tsample 1SN_32A finished, 117048 loci\n",
      "\tsample 1NF_6A finished, 94940 loci\n",
      "\tsample 1HL_21A finished, 104256 loci\n",
      "\tsample 1HL_31A finished, 95291 loci\n",
      "\tsample 1NF_27A finished, 90432 loci\n",
      "\tsample 1HL_35A finished, 95108 loci\n",
      "\tsample 1SN_15A finished, 97901 loci\n",
      "\tsample 1SN_20A finished, 110647 loci\n",
      "\tsample 1NF_12A finished, 83679 loci\n",
      "\tsample 1HL_1A finished, 89466 loci\n",
      "\tsample 1NF_10A finished, 90811 loci\n",
      "\tsample 1NF_11A finished, 88047 loci\n",
      "\tsample 1SN_3A finished, 123243 loci\n",
      "\tsample 1HL_3A finished, 113436 loci\n",
      "\tsample 1HL_5A finished, 87447 loci\n",
      "\tsample 1SN_29A finished, 88849 loci\n",
      "\tsample 1NF_25A finished, 83423 loci\n",
      "\tsample 1NF_1A finished, 62149 loci\n",
      "\tsample 1NF_19A finished, 75939 loci\n",
      "\tsample 1SN_24A finished, 83024 loci\n",
      "\tsample 1HL_22A finished, 92252 loci\n",
      "\tsample 1HL_6A finished, 85254 loci\n",
      "\tsample 1NF_18A finished, 87277 loci\n",
      "\n",
      "real\t1003m49.345s\n",
      "user\t4473m36.422s\n",
      "sys\t3013m44.243s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's take a look at the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "1HL_11A.assembled\t125407\t10.143\t106.054\t39824\t27.904\t186.958\t0\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "sample: 1HL_11A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                81060\n",
      "5 \t*****                               15999\n",
      "10 \t***                                 8344\n",
      "15 \t**                                  5099\n",
      "20 \t**                                  3282\n",
      "25 \t**                                  2270\n",
      "30 \t*                                   1646\n",
      "35 \t*                                   1247\n",
      "40 \t*                                   1846\n",
      "50 \t**                                  3628\n",
      "100 \t*                                   738\n",
      "250 \t*                                   160\n",
      "500 \t*                                   88\n",
      "\n",
      "\n",
      "taxa\ttotal\tdpt.me\tdpt.sd\td>5.tot\td>5.me\td>5.sd\tbadpairs\n",
      "1HL_19A.assembled\t111346\t11.783\t258.313\t38571\t30.329\t438.285\t0\n",
      "1HL_1A.assembled\t89466\t9.948\t347.089\t29477\t26.013\t604.363\t0\n",
      "1HL_20A.assembled\t96673\t10.944\t193.672\t32643\t28.529\t332.587\t0\n",
      "1HL_21A.assembled\t104256\t9.054\t192.249\t30870\t26.107\t352.712\t0\n",
      "1HL_22A.assembled\t92252\t8.599\t259.696\t27812\t23.907\t472.615\t0\n",
      "1HL_23A.assembled\t94229\t13.942\t180.382\t39737\t30.281\t276.936\t0\n",
      "1HL_24A.assembled\t88576\t10.82\t149.102\t30992\t27.192\t251.243\t0\n",
      "1HL_25A.assembled\t89569\t12.118\t226.818\t34969\t27.81\t362.446\t0\n",
      "1HL_26A.assembled\t109215\t11.438\t420.149\t36553\t30.405\t725.87\t0\n",
      "1HL_27A.assembled\t99679\t12.006\t223.061\t38529\t27.889\t358.205\t0\n",
      "1HL_28A.assembled\t104605\t12.051\t197.342\t38344\t29.491\t325.205\t0\n",
      "1HL_29A.assembled\t93081\t13.658\t750.0\t35000\t32.963\t1222.843\t0\n",
      "1HL_2A.assembled\t93559\t11.288\t201.279\t34228\t27.314\t332.162\t0\n",
      "1HL_31A.assembled\t95291\t9.835\t251.465\t31182\t25.924\t439.152\t0\n",
      "1HL_33A.assembled\t90312\t10.396\t79.055\t32034\t25.631\t131.365\t0\n",
      "1HL_34A.assembled\t90703\t11.697\t246.572\t34024\t27.797\t402.071\t0\n",
      "1HL_35A.assembled\t95108\t10.032\t228.329\t32759\t25.228\t388.592\t0\n",
      "1HL_3A.assembled\t113436\t7.502\t184.256\t29682\t23.511\t359.717\t0\n",
      "1HL_4A.assembled\t93231\t11.593\t537.487\t33129\t28.966\t901.401\t0\n",
      "1HL_5A.assembled\t87447\t9.52\t487.743\t27033\t26.22\t877.004\t0\n",
      "1HL_6A.assembled\t85254\t8.901\t277.694\t27005\t23.706\t493.074\t0\n",
      "1HL_7A.assembled\t100958\t10.385\t361.718\t34619\t26.445\t617.387\t0\n",
      "1HL_8A.assembled\t87336\t13.276\t960.795\t30092\t34.623\t1636.609\t0\n",
      "1HL_9A.assembled\t113633\t9.831\t316.472\t33918\t28.622\t578.821\t0\n",
      "1NF_10A.assembled\t90811\t9.312\t299.189\t27231\t26.366\t545.981\t0\n",
      "1NF_11A.assembled\t88047\t9.677\t196.397\t28562\t25.554\t344.278\t0\n",
      "1NF_12A.assembled\t83679\t10.531\t310.007\t28530\t26.891\t530.534\t0\n",
      "1NF_13A.assembled\t127575\t11.923\t359.562\t43922\t30.945\t612.343\t0\n",
      "1NF_14A.assembled\t99152\t9.885\t154.482\t32182\t26.324\t270.413\t0\n",
      "1NF_15A.assembled\t93177\t10.451\t264.876\t32279\t26.332\t449.593\t0\n",
      "1NF_16A.assembled\t101457\t11.024\t607.705\t33035\t29.739\t1064.747\t0\n",
      "1NF_17A.assembled\t108371\t11.106\t255.998\t37017\t28.707\t437.478\t0\n",
      "1NF_18A.assembled\t87277\t7.406\t89.423\t24585\t21.141\t167.693\t0\n",
      "1NF_19A.assembled\t75939\t9.628\t190.419\t24889\t25.087\t332.074\t0\n",
      "1NF_1A.assembled\t62149\t8.108\t174.832\t18717\t22.047\t318.139\t0\n",
      "1NF_20A.assembled\t91488\t10.527\t312.509\t31878\t26.372\t529.052\t0\n",
      "1NF_21A.assembled\t161246\t7.618\t219.851\t39461\t25.754\t443.92\t0\n",
      "1NF_22A.assembled\t108890\t9.774\t85.584\t35333\t26.057\t148.922\t0\n",
      "1NF_23A.assembled\t122732\t10.619\t411.238\t40204\t28.507\t718.184\t0\n",
      "1NF_24A.assembled\t109692\t10.192\t149.97\t36529\t26.586\t259.098\t0\n",
      "1NF_25A.assembled\t83423\t9.536\t148.404\t27776\t24.477\t256.531\t0\n",
      "1NF_26A.assembled\t105799\t10.345\t146.292\t37361\t25.612\t245.441\t0\n",
      "1NF_27A.assembled\t90432\t10.227\t440.639\t29792\t26.848\t767.433\t0\n",
      "1NF_28A.assembled\t92468\t10.505\t517.33\t30337\t27.845\t902.936\t0\n",
      "1NF_29A.assembled\t97131\t15.049\t542.183\t38679\t34.741\t858.809\t0\n",
      "1NF_2A.assembled\t102160\t12.238\t124.042\t39165\t28.671\t199.235\t0\n",
      "1NF_30A.assembled\t92056\t13.305\t858.877\t33366\t33.131\t1426.392\t0\n",
      "1NF_31A.assembled\t109056\t12.184\t211.596\t40183\t29.644\t347.89\t0\n",
      "1NF_32A.assembled\t115584\t11.831\t295.114\t42115\t29.092\t488.418\t0\n",
      "1NF_33A.assembled\t91154\t11.593\t412.218\t33851\t27.73\t676.131\t0\n",
      "1NF_4A.assembled\t110918\t12.304\t250.034\t42340\t29.001\t404.131\t0\n",
      "1NF_5A.assembled\t97753\t10.474\t250.119\t34708\t25.809\t419.318\t0\n",
      "1NF_6A.assembled\t94940\t9.368\t118.351\t30978\t24.514\t206.359\t0\n",
      "1NF_7A.assembled\t96247\t10.695\t394.478\t32142\t27.94\t682.291\t0\n",
      "1NF_8A.assembled\t98509\t10.389\t241.035\t33332\t26.768\t413.876\t0\n",
      "1NF_9A.assembled\t103833\t11.85\t94.91\t38787\t28.323\t153.878\t0\n",
      "1SN_10A.assembled\t107961\t9.501\t325.43\t32953\t26.629\t588.677\t0\n",
      "1SN_11A.assembled\t104454\t10.049\t303.261\t33415\t27.275\t535.767\t0\n",
      "1SN_12A.assembled\t116140\t10.748\t152.998\t41460\t26.503\t255.311\t0\n",
      "1SN_13A.assembled\t106500\t10.499\t239.77\t36704\t26.636\t407.936\t0\n",
      "1SN_14A.assembled\t110527\t12.04\t247.494\t39987\t29.769\t410.869\t0\n",
      "1SN_15A.assembled\t97901\t9.263\t79.841\t32175\t24.095\t138.079\t0\n",
      "1SN_16A.assembled\t96334\t10.109\t192.251\t32132\t26.332\t332.283\t0\n",
      "1SN_17A.assembled\t106838\t10.042\t126.215\t36986\t25.224\t213.683\t0\n",
      "1SN_18A.assembled\t113397\t11.024\t258.755\t37830\t29.143\t447.44\t0\n",
      "1SN_19A.assembled\t120143\t9.694\t213.952\t37066\t27.205\t384.612\t0\n",
      "1SN_1A.assembled\t95372\t11.692\t243.657\t34549\t28.667\t404.267\t0\n",
      "1SN_20A.assembled\t110647\t8.501\t460.876\t28105\t27.957\t914.175\t0\n",
      "1SN_21A.assembled\t131359\t9.694\t116.903\t40472\t27.254\t209.541\t0\n",
      "1SN_22A.assembled\t107304\t11.036\t304.238\t37147\t28.069\t516.65\t0\n",
      "1SN_23A.assembled\t118344\t11.696\t134.392\t43564\t28.418\t220.498\t0\n",
      "1SN_24A.assembled\t83024\t8.485\t194.621\t24894\t23.525\t354.961\t0\n",
      "1SN_25A.assembled\t120465\t8.972\t171.923\t34291\t26.879\t321.535\t0\n",
      "1SN_26A.assembled\t100318\t10.0\t154.535\t33532\t25.88\t266.578\t0\n",
      "1SN_27A.assembled\t96854\t10.007\t270.905\t30897\t27.075\t479.192\t0\n",
      "1SN_28A.assembled\t96449\t11.092\t223.44\t34014\t27.717\t375.682\t0\n",
      "1SN_29A.assembled\t88849\t9.162\t283.094\t23680\t29.309\t547.853\t0\n",
      "1SN_2A.assembled\t106745\t11.138\t256.265\t36309\t28.841\t438.851\t0\n",
      "1SN_30A.assembled\t121564\t11.995\t182.973\t43872\t29.826\t303.755\t0\n",
      "1SN_31A.assembled\t103367\t11.972\t280.474\t37483\t29.472\t465.245\t0\n",
      "1SN_32A.assembled\t117048\t8.287\t217.588\t30447\t26.647\t426.085\t0\n",
      "1SN_3A.assembled\t123243\t7.339\t191.784\t30283\t24.092\t386.409\t0\n",
      "1SN_4A.assembled\t111639\t10.976\t67.766\t40547\t26.738\t110.685\t0\n",
      "1SN_5A.assembled\t121723\t9.342\t129.073\t38686\t25.228\t228.135\t0\n",
      "1SN_6A.assembled\t107796\t11.845\t209.367\t39610\t28.769\t344.727\t0\n",
      "1SN_7A.assembled\t107945\t11.257\t414.056\t37281\t28.845\t704.22\t0\n",
      "1SN_8A.assembled\t102124\t12.804\t475.968\t39290\t30.02\t767.048\t0\n",
      "1SN_9A.assembled\t126666\t9.535\t155.002\t39406\t26.521\t277.138\t0\n",
      "\n",
      "    ## total = total number of clusters, including singletons\n",
      "    ## dpt.me = mean depth of clusters\n",
      "    ## dpt.sd = standard deviation of cluster depth\n",
      "    ## >N.tot = number of clusters with depth greater than N\n",
      "    ## >N.me = mean depth of clusters with depth greater than N\n",
      "    ## >N.sd = standard deviation of cluster depth for clusters with depth greater than N\n",
      "    ## badpairs = mismatched 1st & 2nd reads (only for paired ddRAD data)\n",
      "\n",
      "HISTOGRAMS\n",
      "\n",
      "    \n",
      "sample: 1HL_19A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 68447\n",
      "5 \t*****                               15348\n",
      "10 \t***                                 8028\n",
      "15 \t**                                  4717\n",
      "20 \t**                                  3018\n",
      "25 \t**                                  2157\n",
      "30 \t*                                   1601\n",
      "35 \t*                                   1296\n",
      "40 \t**                                  1860\n",
      "50 \t**                                  3880\n",
      "100 \t*                                   762\n",
      "250 \t*                                   138\n",
      "500 \t*                                   94\n",
      "\n",
      "sample: 1HL_1A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                55800\n",
      "5 \t******                              14663\n",
      "10 \t***                                 6265\n",
      "15 \t**                                  3286\n",
      "20 \t**                                  2205\n",
      "25 \t**                                  1626\n",
      "30 \t*                                   1261\n",
      "35 \t*                                   963\n",
      "40 \t*                                   1453\n",
      "50 \t*                                   1454\n",
      "100 \t*                                   369\n",
      "250 \t*                                   71\n",
      "500 \t*                                   49\n",
      "\n",
      "sample: 1HL_20A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                59978\n",
      "5 \t*****                               14290\n",
      "10 \t***                                 6839\n",
      "15 \t**                                  3593\n",
      "20 \t**                                  2286\n",
      "25 \t**                                  1754\n",
      "30 \t*                                   1352\n",
      "35 \t*                                   1130\n",
      "40 \t**                                  1702\n",
      "50 \t**                                  2969\n",
      "100 \t*                                   584\n",
      "250 \t*                                   122\n",
      "500 \t*                                   74\n",
      "\n",
      "sample: 1HL_21A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               69366\n",
      "5 \t*****                               14289\n",
      "10 \t***                                 6552\n",
      "15 \t**                                  3529\n",
      "20 \t**                                  2351\n",
      "25 \t*                                   1616\n",
      "30 \t*                                   1265\n",
      "35 \t*                                   1038\n",
      "40 \t*                                   1625\n",
      "50 \t**                                  2010\n",
      "100 \t*                                   463\n",
      "250 \t*                                   93\n",
      "500 \t*                                   59\n",
      "\n",
      "sample: 1HL_22A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               60207\n",
      "5 \t******                              14409\n",
      "10 \t***                                 6005\n",
      "15 \t**                                  3259\n",
      "20 \t**                                  2225\n",
      "25 \t**                                  1558\n",
      "30 \t*                                   1193\n",
      "35 \t*                                   939\n",
      "40 \t*                                   1060\n",
      "50 \t*                                   959\n",
      "100 \t*                                   333\n",
      "250 \t*                                   65\n",
      "500 \t*                                   40\n",
      "\n",
      "sample: 1HL_23A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*****************                   50797\n",
      "5 \t*****                               13857\n",
      "10 \t***                                 7823\n",
      "15 \t***                                 5087\n",
      "20 \t**                                  3695\n",
      "25 \t**                                  2718\n",
      "30 \t**                                  2039\n",
      "35 \t*                                   1464\n",
      "40 \t**                                  2053\n",
      "50 \t**                                  3627\n",
      "100 \t*                                   794\n",
      "250 \t*                                   175\n",
      "500 \t*                                   100\n",
      "\n",
      "sample: 1HL_24A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 53755\n",
      "5 \t******                              13696\n",
      "10 \t***                                 6422\n",
      "15 \t**                                  3364\n",
      "20 \t**                                  2299\n",
      "25 \t**                                  1676\n",
      "30 \t*                                   1329\n",
      "35 \t*                                   1106\n",
      "40 \t**                                  1662\n",
      "50 \t**                                  2599\n",
      "100 \t*                                   500\n",
      "250 \t*                                   103\n",
      "500 \t*                                   65\n",
      "\n",
      "sample: 1HL_25A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  50626\n",
      "5 \t******                              14673\n",
      "10 \t***                                 7408\n",
      "15 \t**                                  4284\n",
      "20 \t**                                  2640\n",
      "25 \t**                                  1913\n",
      "30 \t*                                   1452\n",
      "35 \t*                                   1159\n",
      "40 \t**                                  1714\n",
      "50 \t**                                  2900\n",
      "100 \t*                                   612\n",
      "250 \t*                                   115\n",
      "500 \t*                                   73\n",
      "\n",
      "sample: 1HL_26A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                68621\n",
      "5 \t*****                               14751\n",
      "10 \t***                                 7815\n",
      "15 \t**                                  4704\n",
      "20 \t**                                  2935\n",
      "25 \t**                                  2068\n",
      "30 \t*                                   1503\n",
      "35 \t*                                   1186\n",
      "40 \t**                                  1854\n",
      "50 \t**                                  2966\n",
      "100 \t*                                   597\n",
      "250 \t*                                   139\n",
      "500 \t*                                   75\n",
      "\n",
      "sample: 1HL_27A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  57155\n",
      "5 \t******                              15022\n",
      "10 \t***                                 8191\n",
      "15 \t***                                 5108\n",
      "20 \t**                                  3421\n",
      "25 \t**                                  2289\n",
      "30 \t**                                  1699\n",
      "35 \t*                                   1257\n",
      "40 \t**                                  1791\n",
      "50 \t**                                  2886\n",
      "100 \t*                                   651\n",
      "250 \t*                                   134\n",
      "500 \t*                                   75\n",
      "\n",
      "sample: 1HL_28A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 62056\n",
      "5 \t*****                               14868\n",
      "10 \t***                                 7895\n",
      "15 \t**                                  4977\n",
      "20 \t**                                  3150\n",
      "25 \t**                                  2085\n",
      "30 \t*                                   1592\n",
      "35 \t*                                   1265\n",
      "40 \t**                                  1889\n",
      "50 \t**                                  3830\n",
      "100 \t*                                   774\n",
      "250 \t*                                   138\n",
      "500 \t*                                   86\n",
      "\n",
      "sample: 1HL_29A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  54102\n",
      "5 \t******                              14459\n",
      "10 \t***                                 7471\n",
      "15 \t**                                  4356\n",
      "20 \t**                                  2782\n",
      "25 \t**                                  1895\n",
      "30 \t*                                   1421\n",
      "35 \t*                                   1099\n",
      "40 \t**                                  1734\n",
      "50 \t**                                  2963\n",
      "100 \t*                                   599\n",
      "250 \t*                                   127\n",
      "500 \t*                                   72\n",
      "\n",
      "sample: 1HL_2A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 55293\n",
      "5 \t******                              14690\n",
      "10 \t***                                 7496\n",
      "15 \t**                                  4060\n",
      "20 \t**                                  2452\n",
      "25 \t**                                  1830\n",
      "30 \t*                                   1370\n",
      "35 \t*                                   1114\n",
      "40 \t**                                  1686\n",
      "50 \t**                                  2813\n",
      "100 \t*                                   569\n",
      "250 \t*                                   114\n",
      "500 \t*                                   72\n",
      "\n",
      "sample: 1HL_31A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                59878\n",
      "5 \t******                              14849\n",
      "10 \t***                                 6772\n",
      "15 \t**                                  3561\n",
      "20 \t**                                  2301\n",
      "25 \t**                                  1618\n",
      "30 \t*                                   1274\n",
      "35 \t*                                   1035\n",
      "40 \t*                                   1558\n",
      "50 \t**                                  1887\n",
      "100 \t*                                   415\n",
      "250 \t*                                   90\n",
      "500 \t*                                   53\n",
      "\n",
      "sample: 1HL_33A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 54388\n",
      "5 \t******                              14080\n",
      "10 \t***                                 6826\n",
      "15 \t**                                  3683\n",
      "20 \t**                                  2289\n",
      "25 \t**                                  1839\n",
      "30 \t*                                   1318\n",
      "35 \t*                                   1072\n",
      "40 \t**                                  1591\n",
      "50 \t**                                  2543\n",
      "100 \t*                                   520\n",
      "250 \t*                                   97\n",
      "500 \t*                                   66\n",
      "\n",
      "sample: 1HL_34A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  52753\n",
      "5 \t******                              14439\n",
      "10 \t***                                 7250\n",
      "15 \t**                                  4027\n",
      "20 \t**                                  2581\n",
      "25 \t**                                  1893\n",
      "30 \t*                                   1352\n",
      "35 \t*                                   1138\n",
      "40 \t**                                  1660\n",
      "50 \t**                                  2834\n",
      "100 \t*                                   588\n",
      "250 \t*                                   119\n",
      "500 \t*                                   69\n",
      "\n",
      "sample: 1HL_35A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 57982\n",
      "5 \t******                              15201\n",
      "10 \t***                                 7157\n",
      "15 \t**                                  3969\n",
      "20 \t**                                  2415\n",
      "25 \t**                                  1786\n",
      "30 \t*                                   1319\n",
      "35 \t*                                   1086\n",
      "40 \t**                                  1642\n",
      "50 \t**                                  1970\n",
      "100 \t*                                   446\n",
      "250 \t*                                   78\n",
      "500 \t*                                   57\n",
      "\n",
      "sample: 1HL_3A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t**********************              79456\n",
      "5 \t*****                               14740\n",
      "10 \t***                                 6701\n",
      "15 \t**                                  3537\n",
      "20 \t**                                  2253\n",
      "25 \t*                                   1605\n",
      "30 \t*                                   1215\n",
      "35 \t*                                   954\n",
      "40 \t*                                   1325\n",
      "50 \t*                                   1194\n",
      "100 \t*                                   340\n",
      "250 \t*                                   71\n",
      "500 \t*                                   45\n",
      "\n",
      "sample: 1HL_4A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 56050\n",
      "5 \t******                              14669\n",
      "10 \t***                                 7245\n",
      "15 \t**                                  4170\n",
      "20 \t**                                  2624\n",
      "25 \t**                                  1814\n",
      "30 \t*                                   1321\n",
      "35 \t*                                   1090\n",
      "40 \t**                                  1644\n",
      "50 \t**                                  1954\n",
      "100 \t*                                   485\n",
      "250 \t*                                   108\n",
      "500 \t*                                   56\n",
      "\n",
      "sample: 1HL_5A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                56227\n",
      "5 \t******                              14257\n",
      "10 \t***                                 5890\n",
      "15 \t**                                  3100\n",
      "20 \t**                                  2004\n",
      "25 \t**                                  1555\n",
      "30 \t*                                   1217\n",
      "35 \t*                                   933\n",
      "40 \t*                                   971\n",
      "50 \t*                                   935\n",
      "100 \t*                                   258\n",
      "250 \t*                                   60\n",
      "500 \t*                                   39\n",
      "\n",
      "sample: 1HL_6A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                54262\n",
      "5 \t******                              13951\n",
      "10 \t***                                 5618\n",
      "15 \t**                                  3062\n",
      "20 \t**                                  2158\n",
      "25 \t**                                  1600\n",
      "30 \t*                                   1277\n",
      "35 \t*                                   924\n",
      "40 \t*                                   1057\n",
      "50 \t*                                   968\n",
      "100 \t*                                   287\n",
      "250 \t*                                   55\n",
      "500 \t*                                   35\n",
      "\n",
      "sample: 1HL_7A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 61941\n",
      "5 \t******                              15596\n",
      "10 \t***                                 7832\n",
      "15 \t**                                  4635\n",
      "20 \t**                                  2781\n",
      "25 \t**                                  1897\n",
      "30 \t*                                   1388\n",
      "35 \t*                                   1043\n",
      "40 \t*                                   1592\n",
      "50 \t*                                   1657\n",
      "100 \t*                                   438\n",
      "250 \t*                                   98\n",
      "500 \t*                                   59\n",
      "\n",
      "sample: 1HL_8A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 53227\n",
      "5 \t******                              13942\n",
      "10 \t***                                 6123\n",
      "15 \t**                                  3357\n",
      "20 \t**                                  2316\n",
      "25 \t**                                  1653\n",
      "30 \t*                                   1288\n",
      "35 \t*                                   1023\n",
      "40 \t**                                  1609\n",
      "50 \t**                                  2152\n",
      "100 \t*                                   484\n",
      "250 \t*                                   99\n",
      "500 \t*                                   62\n",
      "\n",
      "sample: 1HL_9A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               75819\n",
      "5 \t*****                               14427\n",
      "10 \t***                                 7230\n",
      "15 \t**                                  4104\n",
      "20 \t**                                  2675\n",
      "25 \t*                                   1887\n",
      "30 \t*                                   1413\n",
      "35 \t*                                   1112\n",
      "40 \t*                                   1690\n",
      "50 \t**                                  2544\n",
      "100 \t*                                   559\n",
      "250 \t*                                   106\n",
      "500 \t*                                   66\n",
      "\n",
      "sample: 1NF_10A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               59486\n",
      "5 \t******                              13838\n",
      "10 \t***                                 5285\n",
      "15 \t**                                  2995\n",
      "20 \t**                                  2099\n",
      "25 \t**                                  1521\n",
      "30 \t*                                   1247\n",
      "35 \t*                                   1041\n",
      "40 \t*                                   1476\n",
      "50 \t*                                   1344\n",
      "100 \t*                                   357\n",
      "250 \t*                                   72\n",
      "500 \t*                                   50\n",
      "\n",
      "sample: 1NF_11A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                55342\n",
      "5 \t******                              13998\n",
      "10 \t***                                 5684\n",
      "15 \t**                                  3071\n",
      "20 \t**                                  2136\n",
      "25 \t**                                  1624\n",
      "30 \t*                                   1374\n",
      "35 \t*                                   1074\n",
      "40 \t**                                  1572\n",
      "50 \t**                                  1641\n",
      "100 \t*                                   401\n",
      "250 \t*                                   83\n",
      "500 \t*                                   47\n",
      "\n",
      "sample: 1NF_12A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 51252\n",
      "5 \t******                              13806\n",
      "10 \t***                                 5740\n",
      "15 \t**                                  3176\n",
      "20 \t**                                  2028\n",
      "25 \t**                                  1603\n",
      "30 \t*                                   1241\n",
      "35 \t*                                   989\n",
      "40 \t**                                  1554\n",
      "50 \t**                                  1760\n",
      "100 \t*                                   387\n",
      "250 \t*                                   91\n",
      "500 \t*                                   52\n",
      "\n",
      "sample: 1NF_13A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                78889\n",
      "5 \t*****                               17043\n",
      "10 \t***                                 8832\n",
      "15 \t**                                  5810\n",
      "20 \t**                                  3960\n",
      "25 \t**                                  2690\n",
      "30 \t*                                   1945\n",
      "35 \t*                                   1382\n",
      "40 \t*                                   2066\n",
      "50 \t**                                  3833\n",
      "100 \t*                                   867\n",
      "250 \t*                                   157\n",
      "500 \t*                                   100\n",
      "\n",
      "sample: 1NF_14A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                62650\n",
      "5 \t******                              14980\n",
      "10 \t***                                 6750\n",
      "15 \t**                                  3424\n",
      "20 \t**                                  2326\n",
      "25 \t**                                  1813\n",
      "30 \t*                                   1316\n",
      "35 \t*                                   1155\n",
      "40 \t**                                  1761\n",
      "50 \t**                                  2314\n",
      "100 \t*                                   485\n",
      "250 \t*                                   107\n",
      "500 \t*                                   71\n",
      "\n",
      "sample: 1NF_15A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 56710\n",
      "5 \t******                              14879\n",
      "10 \t***                                 7031\n",
      "15 \t**                                  3623\n",
      "20 \t**                                  2384\n",
      "25 \t**                                  1746\n",
      "30 \t*                                   1378\n",
      "35 \t*                                   1053\n",
      "40 \t**                                  1712\n",
      "50 \t**                                  2044\n",
      "100 \t*                                   447\n",
      "250 \t*                                   105\n",
      "500 \t*                                   65\n",
      "\n",
      "sample: 1NF_16A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                63999\n",
      "5 \t******                              15463\n",
      "10 \t***                                 7182\n",
      "15 \t**                                  3826\n",
      "20 \t**                                  2460\n",
      "25 \t**                                  1773\n",
      "30 \t*                                   1381\n",
      "35 \t*                                   1111\n",
      "40 \t*                                   1679\n",
      "50 \t**                                  1976\n",
      "100 \t*                                   442\n",
      "250 \t*                                   108\n",
      "500 \t*                                   56\n",
      "\n",
      "sample: 1NF_17A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                66920\n",
      "5 \t*****                               15679\n",
      "10 \t***                                 7793\n",
      "15 \t**                                  4436\n",
      "20 \t**                                  2751\n",
      "25 \t**                                  2007\n",
      "30 \t*                                   1587\n",
      "35 \t*                                   1203\n",
      "40 \t**                                  1834\n",
      "50 \t**                                  3288\n",
      "100 \t*                                   661\n",
      "250 \t*                                   133\n",
      "500 \t*                                   79\n",
      "\n",
      "sample: 1NF_18A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               58644\n",
      "5 \t******                              13303\n",
      "10 \t***                                 5111\n",
      "15 \t**                                  2923\n",
      "20 \t**                                  1980\n",
      "25 \t**                                  1457\n",
      "30 \t*                                   1091\n",
      "35 \t*                                   839\n",
      "40 \t*                                   859\n",
      "50 \t*                                   724\n",
      "100 \t*                                   252\n",
      "250 \t*                                   62\n",
      "500 \t*                                   32\n",
      "\n",
      "sample: 1NF_19A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                47341\n",
      "5 \t******                              12391\n",
      "10 \t***                                 4652\n",
      "15 \t**                                  2727\n",
      "20 \t**                                  2058\n",
      "25 \t**                                  1526\n",
      "30 \t*                                   1215\n",
      "35 \t*                                   1002\n",
      "40 \t**                                  1356\n",
      "50 \t*                                   1225\n",
      "100 \t*                                   341\n",
      "250 \t*                                   69\n",
      "500 \t*                                   36\n",
      "\n",
      "sample: 1NF_1A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               40415\n",
      "5 \t******                              9764\n",
      "10 \t***                                 3824\n",
      "15 \t**                                  2502\n",
      "20 \t**                                  1812\n",
      "25 \t**                                  1283\n",
      "30 \t*                                   850\n",
      "35 \t*                                   495\n",
      "40 \t*                                   467\n",
      "50 \t*                                   481\n",
      "100 \t*                                   198\n",
      "250 \t*                                   33\n",
      "500 \t*                                   25\n",
      "\n",
      "sample: 1NF_20A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 55363\n",
      "5 \t******                              14974\n",
      "10 \t***                                 6816\n",
      "15 \t**                                  3668\n",
      "20 \t**                                  2391\n",
      "25 \t**                                  1692\n",
      "30 \t*                                   1364\n",
      "35 \t*                                   1163\n",
      "40 \t**                                  1653\n",
      "50 \t**                                  1825\n",
      "100 \t*                                   430\n",
      "250 \t*                                   85\n",
      "500 \t*                                   64\n",
      "\n",
      "sample: 1NF_21A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t***********************             117090\n",
      "5 \t****                                16991\n",
      "10 \t***                                 8855\n",
      "15 \t**                                  5584\n",
      "20 \t**                                  3381\n",
      "25 \t*                                   2168\n",
      "30 \t*                                   1504\n",
      "35 \t*                                   1150\n",
      "40 \t*                                   1820\n",
      "50 \t*                                   1981\n",
      "100 \t*                                   536\n",
      "250 \t*                                   108\n",
      "500 \t*                                   78\n",
      "\n",
      "sample: 1NF_22A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                69222\n",
      "5 \t*****                               15624\n",
      "10 \t***                                 7399\n",
      "15 \t**                                  4179\n",
      "20 \t**                                  2636\n",
      "25 \t**                                  1857\n",
      "30 \t*                                   1442\n",
      "35 \t*                                   1142\n",
      "40 \t*                                   1743\n",
      "50 \t**                                  2885\n",
      "100 \t*                                   558\n",
      "250 \t*                                   138\n",
      "500 \t*                                   65\n",
      "\n",
      "sample: 1NF_23A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                78024\n",
      "5 \t*****                               16707\n",
      "10 \t***                                 8705\n",
      "15 \t**                                  5301\n",
      "20 \t**                                  3276\n",
      "25 \t**                                  2225\n",
      "30 \t*                                   1603\n",
      "35 \t*                                   1291\n",
      "40 \t*                                   1869\n",
      "50 \t**                                  2898\n",
      "100 \t*                                   613\n",
      "250 \t*                                   132\n",
      "500 \t*                                   87\n",
      "\n",
      "sample: 1NF_24A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                68448\n",
      "5 \t******                              16514\n",
      "10 \t***                                 7667\n",
      "15 \t**                                  4237\n",
      "20 \t**                                  2554\n",
      "25 \t**                                  1923\n",
      "30 \t*                                   1491\n",
      "35 \t*                                   1193\n",
      "40 \t**                                  1833\n",
      "50 \t**                                  3029\n",
      "100 \t*                                   602\n",
      "250 \t*                                   127\n",
      "500 \t*                                   74\n",
      "\n",
      "sample: 1NF_25A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                51529\n",
      "5 \t******                              13841\n",
      "10 \t***                                 5522\n",
      "15 \t**                                  3049\n",
      "20 \t**                                  2085\n",
      "25 \t**                                  1613\n",
      "30 \t*                                   1250\n",
      "35 \t*                                   1063\n",
      "40 \t**                                  1562\n",
      "50 \t*                                   1389\n",
      "100 \t*                                   393\n",
      "250 \t*                                   82\n",
      "500 \t*                                   45\n",
      "\n",
      "sample: 1NF_26A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 63962\n",
      "5 \t******                              16117\n",
      "10 \t***                                 8166\n",
      "15 \t**                                  4751\n",
      "20 \t**                                  2925\n",
      "25 \t**                                  2048\n",
      "30 \t*                                   1479\n",
      "35 \t*                                   1152\n",
      "40 \t**                                  1787\n",
      "50 \t**                                  2625\n",
      "100 \t*                                   590\n",
      "250 \t*                                   122\n",
      "500 \t*                                   75\n",
      "\n",
      "sample: 1NF_27A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                56374\n",
      "5 \t******                              15004\n",
      "10 \t***                                 6276\n",
      "15 \t**                                  3441\n",
      "20 \t**                                  2277\n",
      "25 \t**                                  1624\n",
      "30 \t*                                   1293\n",
      "35 \t*                                   1056\n",
      "40 \t*                                   1314\n",
      "50 \t*                                   1290\n",
      "100 \t*                                   350\n",
      "250 \t*                                   77\n",
      "500 \t*                                   55\n",
      "\n",
      "sample: 1NF_28A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                57828\n",
      "5 \t******                              14884\n",
      "10 \t***                                 6545\n",
      "15 \t**                                  3515\n",
      "20 \t**                                  2293\n",
      "25 \t**                                  1737\n",
      "30 \t*                                   1324\n",
      "35 \t*                                   1045\n",
      "40 \t*                                   1415\n",
      "50 \t*                                   1397\n",
      "100 \t*                                   356\n",
      "250 \t*                                   85\n",
      "500 \t*                                   43\n",
      "\n",
      "sample: 1NF_29A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  54455\n",
      "5 \t******                              14658\n",
      "10 \t***                                 7747\n",
      "15 \t**                                  4706\n",
      "20 \t**                                  2984\n",
      "25 \t**                                  2020\n",
      "30 \t*                                   1597\n",
      "35 \t*                                   1348\n",
      "40 \t**                                  1991\n",
      "50 \t**                                  4385\n",
      "100 \t*                                   944\n",
      "250 \t*                                   195\n",
      "500 \t*                                   100\n",
      "\n",
      "sample: 1NF_2A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  58774\n",
      "5 \t*****                               15245\n",
      "10 \t***                                 8081\n",
      "15 \t**                                  5015\n",
      "20 \t**                                  3276\n",
      "25 \t**                                  2233\n",
      "30 \t*                                   1592\n",
      "35 \t*                                   1288\n",
      "40 \t**                                  1940\n",
      "50 \t**                                  3691\n",
      "100 \t*                                   771\n",
      "250 \t*                                   170\n",
      "500 \t*                                   84\n",
      "\n",
      "sample: 1NF_30A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 54566\n",
      "5 \t******                              14727\n",
      "10 \t***                                 7037\n",
      "15 \t**                                  3924\n",
      "20 \t**                                  2587\n",
      "25 \t**                                  1891\n",
      "30 \t*                                   1367\n",
      "35 \t*                                   1090\n",
      "40 \t**                                  1742\n",
      "50 \t**                                  2416\n",
      "100 \t*                                   534\n",
      "250 \t*                                   99\n",
      "500 \t*                                   75\n",
      "\n",
      "sample: 1NF_31A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 64382\n",
      "5 \t*****                               15952\n",
      "10 \t***                                 8419\n",
      "15 \t**                                  5021\n",
      "20 \t**                                  3255\n",
      "25 \t**                                  2280\n",
      "30 \t*                                   1765\n",
      "35 \t*                                   1299\n",
      "40 \t**                                  1947\n",
      "50 \t**                                  3724\n",
      "100 \t*                                   749\n",
      "250 \t*                                   167\n",
      "500 \t*                                   96\n",
      "\n",
      "sample: 1NF_32A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 69054\n",
      "5 \t*****                               16199\n",
      "10 \t***                                 8932\n",
      "15 \t**                                  5662\n",
      "20 \t**                                  3875\n",
      "25 \t**                                  2672\n",
      "30 \t*                                   1860\n",
      "35 \t*                                   1294\n",
      "40 \t**                                  1984\n",
      "50 \t**                                  3095\n",
      "100 \t*                                   716\n",
      "250 \t*                                   148\n",
      "500 \t*                                   93\n",
      "\n",
      "sample: 1NF_33A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  53161\n",
      "5 \t******                              14931\n",
      "10 \t***                                 7387\n",
      "15 \t**                                  4128\n",
      "20 \t**                                  2666\n",
      "25 \t**                                  1788\n",
      "30 \t*                                   1436\n",
      "35 \t*                                   1085\n",
      "40 \t**                                  1745\n",
      "50 \t**                                  2167\n",
      "100 \t*                                   491\n",
      "250 \t*                                   102\n",
      "500 \t*                                   66\n",
      "\n",
      "sample: 1NF_4A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  64058\n",
      "5 \t*****                               16236\n",
      "10 \t***                                 8800\n",
      "15 \t***                                 5600\n",
      "20 \t**                                  3795\n",
      "25 \t**                                  2670\n",
      "30 \t*                                   1819\n",
      "35 \t*                                   1346\n",
      "40 \t**                                  2014\n",
      "50 \t**                                  3583\n",
      "100 \t*                                   734\n",
      "250 \t*                                   163\n",
      "500 \t*                                   100\n",
      "\n",
      "sample: 1NF_5A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 58654\n",
      "5 \t******                              15786\n",
      "10 \t***                                 7511\n",
      "15 \t**                                  4235\n",
      "20 \t**                                  2597\n",
      "25 \t**                                  1915\n",
      "30 \t*                                   1509\n",
      "35 \t*                                   1129\n",
      "40 \t**                                  1762\n",
      "50 \t**                                  2018\n",
      "100 \t*                                   467\n",
      "250 \t*                                   106\n",
      "500 \t*                                   64\n",
      "\n",
      "sample: 1NF_6A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                59692\n",
      "5 \t******                              15080\n",
      "10 \t***                                 6491\n",
      "15 \t**                                  3424\n",
      "20 \t**                                  2300\n",
      "25 \t**                                  1690\n",
      "30 \t*                                   1330\n",
      "35 \t*                                   1138\n",
      "40 \t*                                   1546\n",
      "50 \t**                                  1702\n",
      "100 \t*                                   401\n",
      "250 \t*                                   86\n",
      "500 \t*                                   60\n",
      "\n",
      "sample: 1NF_7A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                59628\n",
      "5 \t******                              15414\n",
      "10 \t***                                 6662\n",
      "15 \t**                                  3563\n",
      "20 \t**                                  2376\n",
      "25 \t**                                  1727\n",
      "30 \t*                                   1299\n",
      "35 \t*                                   1093\n",
      "40 \t**                                  1691\n",
      "50 \t**                                  2186\n",
      "100 \t*                                   438\n",
      "250 \t*                                   106\n",
      "500 \t*                                   63\n",
      "\n",
      "sample: 1NF_8A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                60915\n",
      "5 \t******                              15163\n",
      "10 \t***                                 7188\n",
      "15 \t**                                  3650\n",
      "20 \t**                                  2356\n",
      "25 \t**                                  1752\n",
      "30 \t*                                   1409\n",
      "35 \t*                                   1101\n",
      "40 \t**                                  1802\n",
      "50 \t**                                  2511\n",
      "100 \t*                                   492\n",
      "250 \t*                                   108\n",
      "500 \t*                                   62\n",
      "\n",
      "sample: 1NF_9A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 60663\n",
      "5 \t******                              15870\n",
      "10 \t***                                 8026\n",
      "15 \t**                                  4718\n",
      "20 \t**                                  3031\n",
      "25 \t**                                  2111\n",
      "30 \t*                                   1623\n",
      "35 \t*                                   1202\n",
      "40 \t**                                  1861\n",
      "50 \t**                                  3700\n",
      "100 \t*                                   780\n",
      "250 \t*                                   162\n",
      "500 \t*                                   86\n",
      "\n",
      "sample: 1SN_10A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               70432\n",
      "5 \t*****                               15850\n",
      "10 \t***                                 7226\n",
      "15 \t**                                  3644\n",
      "20 \t**                                  2291\n",
      "25 \t*                                   1757\n",
      "30 \t*                                   1395\n",
      "35 \t*                                   1023\n",
      "40 \t*                                   1733\n",
      "50 \t**                                  2009\n",
      "100 \t*                                   443\n",
      "250 \t*                                   101\n",
      "500 \t*                                   56\n",
      "\n",
      "sample: 1SN_11A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                66764\n",
      "5 \t*****                               14959\n",
      "10 \t***                                 7366\n",
      "15 \t**                                  4114\n",
      "20 \t**                                  2520\n",
      "25 \t**                                  1756\n",
      "30 \t*                                   1378\n",
      "35 \t*                                   1059\n",
      "40 \t*                                   1612\n",
      "50 \t**                                  2261\n",
      "100 \t*                                   500\n",
      "250 \t*                                   103\n",
      "500 \t*                                   62\n",
      "\n",
      "sample: 1SN_12A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 69908\n",
      "5 \t*****                               17243\n",
      "10 \t***                                 8883\n",
      "15 \t**                                  5452\n",
      "20 \t**                                  3474\n",
      "25 \t**                                  2344\n",
      "30 \t*                                   1669\n",
      "35 \t*                                   1335\n",
      "40 \t**                                  2003\n",
      "50 \t**                                  2933\n",
      "100 \t*                                   681\n",
      "250 \t*                                   134\n",
      "500 \t*                                   81\n",
      "\n",
      "sample: 1SN_13A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 65156\n",
      "5 \t******                              16331\n",
      "10 \t***                                 8085\n",
      "15 \t**                                  4570\n",
      "20 \t**                                  2775\n",
      "25 \t**                                  1889\n",
      "30 \t*                                   1462\n",
      "35 \t*                                   1140\n",
      "40 \t**                                  1854\n",
      "50 \t**                                  2500\n",
      "100 \t*                                   534\n",
      "250 \t*                                   128\n",
      "500 \t*                                   76\n",
      "\n",
      "sample: 1SN_14A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 66093\n",
      "5 \t*****                               16195\n",
      "10 \t***                                 8253\n",
      "15 \t**                                  5072\n",
      "20 \t**                                  3073\n",
      "25 \t**                                  2166\n",
      "30 \t*                                   1667\n",
      "35 \t*                                   1265\n",
      "40 \t**                                  1950\n",
      "50 \t**                                  3773\n",
      "100 \t*                                   759\n",
      "250 \t*                                   175\n",
      "500 \t*                                   86\n",
      "\n",
      "sample: 1SN_15A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                61429\n",
      "5 \t******                              15221\n",
      "10 \t***                                 6915\n",
      "15 \t**                                  3617\n",
      "20 \t**                                  2340\n",
      "25 \t**                                  1732\n",
      "30 \t*                                   1359\n",
      "35 \t*                                   1122\n",
      "40 \t**                                  1694\n",
      "50 \t**                                  1866\n",
      "100 \t*                                   458\n",
      "250 \t*                                   89\n",
      "500 \t*                                   59\n",
      "\n",
      "sample: 1SN_16A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                60032\n",
      "5 \t******                              14828\n",
      "10 \t***                                 6726\n",
      "15 \t**                                  3745\n",
      "20 \t**                                  2286\n",
      "25 \t**                                  1713\n",
      "30 \t*                                   1302\n",
      "35 \t*                                   1090\n",
      "40 \t**                                  1700\n",
      "50 \t**                                  2274\n",
      "100 \t*                                   472\n",
      "250 \t*                                   102\n",
      "500 \t*                                   64\n",
      "\n",
      "sample: 1SN_17A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 65231\n",
      "5 \t******                              16536\n",
      "10 \t***                                 8216\n",
      "15 \t**                                  4531\n",
      "20 \t**                                  2770\n",
      "25 \t**                                  1893\n",
      "30 \t*                                   1471\n",
      "35 \t*                                   1148\n",
      "40 \t**                                  1852\n",
      "50 \t**                                  2432\n",
      "100 \t*                                   566\n",
      "250 \t*                                   121\n",
      "500 \t*                                   71\n",
      "\n",
      "sample: 1SN_18A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                71002\n",
      "5 \t*****                               16111\n",
      "10 \t***                                 7962\n",
      "15 \t**                                  4690\n",
      "20 \t**                                  2809\n",
      "25 \t**                                  1930\n",
      "30 \t*                                   1497\n",
      "35 \t*                                   1204\n",
      "40 \t*                                   1860\n",
      "50 \t**                                  3430\n",
      "100 \t*                                   664\n",
      "250 \t*                                   155\n",
      "500 \t*                                   83\n",
      "\n",
      "sample: 1SN_19A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               78620\n",
      "5 \t*****                               15808\n",
      "10 \t***                                 8065\n",
      "15 \t**                                  4719\n",
      "20 \t**                                  2879\n",
      "25 \t**                                  2015\n",
      "30 \t*                                   1466\n",
      "35 \t*                                   1146\n",
      "40 \t*                                   1788\n",
      "50 \t**                                  2826\n",
      "100 \t*                                   614\n",
      "250 \t*                                   120\n",
      "500 \t*                                   77\n",
      "\n",
      "sample: 1SN_1A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 56574\n",
      "5 \t******                              15056\n",
      "10 \t***                                 7145\n",
      "15 \t**                                  3874\n",
      "20 \t**                                  2457\n",
      "25 \t**                                  1910\n",
      "30 \t*                                   1471\n",
      "35 \t*                                   1174\n",
      "40 \t**                                  1845\n",
      "50 \t**                                  3081\n",
      "100 \t*                                   588\n",
      "250 \t*                                   123\n",
      "500 \t*                                   74\n",
      "\n",
      "sample: 1SN_20A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t**********************              78249\n",
      "5 \t*****                               14274\n",
      "10 \t***                                 5786\n",
      "15 \t**                                  3176\n",
      "20 \t**                                  2132\n",
      "25 \t*                                   1572\n",
      "30 \t*                                   1213\n",
      "35 \t*                                   1014\n",
      "40 \t*                                   1420\n",
      "50 \t*                                   1326\n",
      "100 \t*                                   363\n",
      "250 \t*                                   80\n",
      "500 \t*                                   41\n",
      "\n",
      "sample: 1SN_21A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               86121\n",
      "5 \t*****                               16817\n",
      "10 \t***                                 8526\n",
      "15 \t**                                  5034\n",
      "20 \t**                                  3269\n",
      "25 \t*                                   2178\n",
      "30 \t*                                   1663\n",
      "35 \t*                                   1262\n",
      "40 \t*                                   1908\n",
      "50 \t**                                  3627\n",
      "100 \t*                                   727\n",
      "250 \t*                                   147\n",
      "500 \t*                                   80\n",
      "\n",
      "sample: 1SN_22A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 65559\n",
      "5 \t******                              16330\n",
      "10 \t***                                 7913\n",
      "15 \t**                                  4519\n",
      "20 \t**                                  2797\n",
      "25 \t**                                  2023\n",
      "30 \t*                                   1454\n",
      "35 \t*                                   1178\n",
      "40 \t**                                  1851\n",
      "50 \t**                                  2883\n",
      "100 \t*                                   596\n",
      "250 \t*                                   131\n",
      "500 \t*                                   70\n",
      "\n",
      "sample: 1SN_23A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 70248\n",
      "5 \t*****                               16403\n",
      "10 \t***                                 8889\n",
      "15 \t**                                  5871\n",
      "20 \t**                                  4049\n",
      "25 \t**                                  2695\n",
      "30 \t*                                   1948\n",
      "35 \t*                                   1411\n",
      "40 \t**                                  2062\n",
      "50 \t**                                  3678\n",
      "100 \t*                                   797\n",
      "250 \t*                                   185\n",
      "500 \t*                                   108\n",
      "\n",
      "sample: 1SN_24A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               54175\n",
      "5 \t******                              12989\n",
      "10 \t***                                 5054\n",
      "15 \t**                                  2834\n",
      "20 \t**                                  1985\n",
      "25 \t**                                  1504\n",
      "30 \t*                                   1183\n",
      "35 \t*                                   907\n",
      "40 \t*                                   1057\n",
      "50 \t*                                   973\n",
      "100 \t*                                   272\n",
      "250 \t*                                   53\n",
      "500 \t*                                   38\n",
      "\n",
      "sample: 1SN_25A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               81776\n",
      "5 \t*****                               15548\n",
      "10 \t***                                 7196\n",
      "15 \t**                                  3891\n",
      "20 \t**                                  2423\n",
      "25 \t*                                   1764\n",
      "30 \t*                                   1380\n",
      "35 \t*                                   1108\n",
      "40 \t*                                   1825\n",
      "50 \t**                                  2812\n",
      "100 \t*                                   560\n",
      "250 \t*                                   119\n",
      "500 \t*                                   63\n",
      "\n",
      "sample: 1SN_26A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                62225\n",
      "5 \t******                              15646\n",
      "10 \t***                                 7183\n",
      "15 \t**                                  3857\n",
      "20 \t**                                  2467\n",
      "25 \t**                                  1745\n",
      "30 \t*                                   1319\n",
      "35 \t*                                   1110\n",
      "40 \t**                                  1674\n",
      "50 \t**                                  2421\n",
      "100 \t*                                   488\n",
      "250 \t*                                   115\n",
      "500 \t*                                   68\n",
      "\n",
      "sample: 1SN_27A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                61562\n",
      "5 \t******                              15045\n",
      "10 \t***                                 6281\n",
      "15 \t**                                  3303\n",
      "20 \t**                                  2256\n",
      "25 \t**                                  1702\n",
      "30 \t*                                   1270\n",
      "35 \t*                                   1083\n",
      "40 \t**                                  1632\n",
      "50 \t**                                  2146\n",
      "100 \t*                                   419\n",
      "250 \t*                                   95\n",
      "500 \t*                                   60\n",
      "\n",
      "sample: 1SN_28A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 58170\n",
      "5 \t******                              15131\n",
      "10 \t***                                 7200\n",
      "15 \t**                                  3936\n",
      "20 \t**                                  2501\n",
      "25 \t**                                  1813\n",
      "30 \t*                                   1410\n",
      "35 \t*                                   1099\n",
      "40 \t**                                  1700\n",
      "50 \t**                                  2734\n",
      "100 \t*                                   560\n",
      "250 \t*                                   128\n",
      "500 \t*                                   67\n",
      "\n",
      "sample: 1SN_29A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t**********************              61992\n",
      "5 \t*****                               10749\n",
      "10 \t**                                  4344\n",
      "15 \t**                                  2512\n",
      "20 \t**                                  1897\n",
      "25 \t**                                  1564\n",
      "30 \t*                                   1215\n",
      "35 \t*                                   939\n",
      "40 \t*                                   1401\n",
      "50 \t**                                  1744\n",
      "100 \t*                                   377\n",
      "250 \t*                                   70\n",
      "500 \t*                                   45\n",
      "\n",
      "sample: 1SN_2A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 65816\n",
      "5 \t******                              16161\n",
      "10 \t***                                 7702\n",
      "15 \t**                                  4282\n",
      "20 \t**                                  2555\n",
      "25 \t**                                  1828\n",
      "30 \t*                                   1419\n",
      "35 \t*                                   1154\n",
      "40 \t**                                  1820\n",
      "50 \t**                                  3135\n",
      "100 \t*                                   647\n",
      "250 \t*                                   143\n",
      "500 \t*                                   83\n",
      "\n",
      "sample: 1SN_30A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 73268\n",
      "5 \t*****                               15757\n",
      "10 \t***                                 8439\n",
      "15 \t**                                  5612\n",
      "20 \t**                                  4288\n",
      "25 \t**                                  2976\n",
      "30 \t**                                  2274\n",
      "35 \t*                                   1628\n",
      "40 \t**                                  2260\n",
      "50 \t**                                  3888\n",
      "100 \t*                                   881\n",
      "250 \t*                                   199\n",
      "500 \t*                                   94\n",
      "\n",
      "sample: 1SN_31A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 61571\n",
      "5 \t******                              15614\n",
      "10 \t***                                 7864\n",
      "15 \t**                                  4528\n",
      "20 \t**                                  2742\n",
      "25 \t**                                  1990\n",
      "30 \t*                                   1504\n",
      "35 \t*                                   1220\n",
      "40 \t**                                  1839\n",
      "50 \t**                                  3626\n",
      "100 \t*                                   642\n",
      "250 \t*                                   142\n",
      "500 \t*                                   85\n",
      "\n",
      "sample: 1SN_32A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t**********************              82376\n",
      "5 \t*****                               14587\n",
      "10 \t***                                 6195\n",
      "15 \t**                                  3447\n",
      "20 \t**                                  2205\n",
      "25 \t*                                   1630\n",
      "30 \t*                                   1348\n",
      "35 \t*                                   1087\n",
      "40 \t*                                   1650\n",
      "50 \t*                                   1948\n",
      "100 \t*                                   413\n",
      "250 \t*                                   99\n",
      "500 \t*                                   63\n",
      "\n",
      "sample: 1SN_3A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t**********************              88130\n",
      "5 \t*****                               15945\n",
      "10 \t***                                 6261\n",
      "15 \t**                                  3358\n",
      "20 \t**                                  2263\n",
      "25 \t*                                   1647\n",
      "30 \t*                                   1354\n",
      "35 \t*                                   1095\n",
      "40 \t*                                   1382\n",
      "50 \t*                                   1300\n",
      "100 \t*                                   369\n",
      "250 \t*                                   98\n",
      "500 \t*                                   41\n",
      "\n",
      "sample: 1SN_4A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 66525\n",
      "5 \t*****                               16198\n",
      "10 \t***                                 8510\n",
      "15 \t**                                  5400\n",
      "20 \t**                                  3534\n",
      "25 \t**                                  2383\n",
      "30 \t*                                   1695\n",
      "35 \t*                                   1220\n",
      "40 \t**                                  1861\n",
      "50 \t**                                  3351\n",
      "100 \t*                                   701\n",
      "250 \t*                                   175\n",
      "500 \t*                                   86\n",
      "\n",
      "sample: 1SN_5A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t********************                78276\n",
      "5 \t*****                               17217\n",
      "10 \t***                                 8370\n",
      "15 \t**                                  4862\n",
      "20 \t**                                  3089\n",
      "25 \t**                                  2090\n",
      "30 \t*                                   1523\n",
      "35 \t*                                   1202\n",
      "40 \t*                                   1765\n",
      "50 \t**                                  2582\n",
      "100 \t*                                   558\n",
      "250 \t*                                   113\n",
      "500 \t*                                   76\n",
      "\n",
      "sample: 1SN_6A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 63699\n",
      "5 \t*****                               16069\n",
      "10 \t***                                 8480\n",
      "15 \t**                                  5081\n",
      "20 \t**                                  3165\n",
      "25 \t**                                  2176\n",
      "30 \t*                                   1587\n",
      "35 \t*                                   1268\n",
      "40 \t**                                  1886\n",
      "50 \t**                                  3463\n",
      "100 \t*                                   681\n",
      "250 \t*                                   153\n",
      "500 \t*                                   88\n",
      "\n",
      "sample: 1SN_7A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*******************                 66184\n",
      "5 \t*****                               16029\n",
      "10 \t***                                 8005\n",
      "15 \t**                                  4767\n",
      "20 \t**                                  3007\n",
      "25 \t**                                  2051\n",
      "30 \t*                                   1525\n",
      "35 \t*                                   1145\n",
      "40 \t**                                  1807\n",
      "50 \t**                                  2632\n",
      "100 \t*                                   587\n",
      "250 \t*                                   126\n",
      "500 \t*                                   79\n",
      "\n",
      "sample: 1SN_8A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t******************                  58494\n",
      "5 \t******                              15867\n",
      "10 \t***                                 8341\n",
      "15 \t***                                 5216\n",
      "20 \t**                                  3433\n",
      "25 \t**                                  2249\n",
      "30 \t*                                   1596\n",
      "35 \t*                                   1282\n",
      "40 \t**                                  1962\n",
      "50 \t**                                  2791\n",
      "100 \t*                                   686\n",
      "250 \t*                                   132\n",
      "500 \t*                                   74\n",
      "\n",
      "sample: 1SN_9A.assembled\n",
      "bins\tdepth_histogram\tcnts\n",
      "   :\t0------------50-------------100%\n",
      "0 \t*********************               82655\n",
      "5 \t*****                               16474\n",
      "10 \t***                                 8607\n",
      "15 \t**                                  5154\n",
      "20 \t**                                  3303\n",
      "25 \t**                                  2210\n",
      "30 \t*                                   1521\n",
      "35 \t*                                   1194\n",
      "40 \t*                                   1810\n",
      "50 \t**                                  2886\n",
      "100 \t*                                   644\n",
      "250 \t*                                   125\n",
      "500 \t*                                   83\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/s3.clusters.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Parameter estimation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 4: estimating error rate and heterozygosity\n",
      "\t................................................................................................\n",
      "real\t1033m58.941s\n",
      "user\t7810m54.513s\n",
      "sys\t0m26.031s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "taxa\tH\tE\n",
      "1HL_13A.assembled\t0.01234178\t0.00144187\t\n",
      "1HL_10A.assembled\t0.01399484\t0.00139753\t\n",
      "1HL_15A.assembled\t0.01242723\t0.0011879\t\n",
      "1HL_12A.assembled\t0.01240698\t0.00157456\t\n",
      "1HL_16A.assembled\t0.01381188\t0.00054237\t\n",
      "1HL_17A.assembled\t0.01332716\t0.00127763\t\n",
      "1HL_14A.assembled\t0.01266762\t0.00203182\t\n",
      "1HL_11A.assembled\t0.01216978\t0.00165352\t\n",
      "1HL_13A.assembled\t0.01234178\t0.00144187\t\n",
      "1HL_10A.assembled\t0.01399484\t0.00139753\t\n",
      "1HL_15A.assembled\t0.01242723\t0.0011879\t\n",
      "1HL_12A.assembled\t0.01240698\t0.00157456\t\n",
      "1HL_16A.assembled\t0.01381188\t0.00054237\t\n",
      "1HL_17A.assembled\t0.01332716\t0.00127763\t\n",
      "1HL_14A.assembled\t0.01266762\t0.00203182\t\n",
      "1NF_1A.assembled\t0.01302939\t0.00167192\t\n",
      "1NF_19A.assembled\t0.01302296\t0.00156732\t\n",
      "1SN_24A.assembled\t0.01280068\t0.00172812\t\n",
      "1NF_18A.assembled\t0.01298092\t0.00169576\t\n",
      "1SN_29A.assembled\t0.01288545\t0.0014629\t\n",
      "1HL_6A.assembled\t0.01312247\t0.00152568\t\n",
      "1NF_25A.assembled\t0.01280702\t0.00162634\t\n",
      "1NF_12A.assembled\t0.01233271\t0.00160106\t\n",
      "1HL_5A.assembled\t0.01210494\t0.00151496\t\n",
      "1NF_10A.assembled\t0.01256406\t0.00160585\t\n",
      "1NF_11A.assembled\t0.01268246\t0.00165147\t\n",
      "1HL_22A.assembled\t0.01296414\t0.00141904\t\n",
      "1HL_1A.assembled\t0.01264807\t0.00153935\t\n",
      "1NF_27A.assembled\t0.01276751\t0.00161456\t\n",
      "1HL_8A.assembled\t0.01278796\t0.00153183\t\n",
      "1NF_28A.assembled\t0.01270732\t0.0013666\t\n",
      "1HL_24A.assembled\t0.01238308\t0.00167162\t\n",
      "1NF_20A.assembled\t0.01238014\t0.0016131\t\n",
      "1HL_33A.assembled\t0.01272048\t0.00149538\t\n",
      "1NF_6A.assembled\t0.01228499\t0.00146483\t\n",
      "1NF_15A.assembled\t0.01227426\t0.0016802\t\n",
      "1NF_33A.assembled\t0.0122457\t0.0014486\t\n",
      "1HL_31A.assembled\t0.01280596\t0.00156314\t\n",
      "1NF_30A.assembled\t0.01249949\t0.00142214\t\n",
      "1HL_4A.assembled\t0.0124557\t0.00147379\t\n",
      "1HL_25A.assembled\t0.01237222\t0.00147768\t\n",
      "1NF_7A.assembled\t0.01265262\t0.0015292\t\n",
      "1SN_16A.assembled\t0.01240499\t0.00171252\t\n",
      "1HL_34A.assembled\t0.0125749\t0.00155212\t\n",
      "1SN_27A.assembled\t0.01287353\t0.00165881\t\n",
      "1HL_35A.assembled\t0.01271276\t0.00166711\t\n",
      "1SN_15A.assembled\t0.01229991\t0.00183908\t\n",
      "1HL_2A.assembled\t0.01213818\t0.00163783\t\n",
      "1HL_20A.assembled\t0.01231824\t0.00169476\t\n",
      "1NF_14A.assembled\t0.01255169\t0.00174661\t\n",
      "1NF_8A.assembled\t0.01200827\t0.00145281\t\n",
      "1NF_5A.assembled\t0.01213863\t0.00174276\t\n",
      "1SN_20A.assembled\t0.01273133\t0.00142206\t\n",
      "1HL_29A.assembled\t0.01255651\t0.00138799\t\n",
      "1SN_1A.assembled\t0.01224028\t0.00159679\t\n",
      "1HL_21A.assembled\t0.01272641\t0.00165356\t\n",
      "1SN_28A.assembled\t0.01232066\t0.00158669\t\n",
      "1NF_16A.assembled\t0.01270907\t0.00133209\t\n",
      "1SN_26A.assembled\t0.01229891\t0.00164706\t\n",
      "1HL_3A.assembled\t0.01232275\t0.00170013\t\n",
      "1SN_11A.assembled\t0.01217015\t0.0017145\t\n",
      "1SN_10A.assembled\t0.01259095\t0.00153324\t\n",
      "1HL_7A.assembled\t0.01250226\t0.00159767\t\n",
      "1HL_27A.assembled\t0.01243331\t0.00153194\t\n",
      "1SN_32A.assembled\t0.01259383\t0.00168319\t\n",
      "1NF_26A.assembled\t0.0123278\t0.0016356\t\n",
      "1HL_23A.assembled\t0.0120483\t0.00141033\t\n",
      "1NF_22A.assembled\t0.012368\t0.00153584\t\n",
      "1SN_13A.assembled\t0.01218486\t0.00168494\t\n",
      "1SN_8A.assembled\t0.0121543\t0.00123362\t\n",
      "1SN_17A.assembled\t0.01209382\t0.00169839\t\n",
      "1HL_9A.assembled\t0.01225037\t0.00138481\t\n",
      "1NF_29A.assembled\t0.01233428\t0.00133912\t\n",
      "1SN_2A.assembled\t0.01243021\t0.00158393\t\n",
      "1SN_31A.assembled\t0.01246079\t0.00163411\t\n",
      "1SN_7A.assembled\t0.01225447\t0.00159903\t\n",
      "1NF_2A.assembled\t0.01183364\t0.00155891\t\n",
      "1SN_22A.assembled\t0.01251389\t0.00158949\t\n",
      "1NF_9A.assembled\t0.01216885\t0.00156536\t\n",
      "1HL_26A.assembled\t0.01233653\t0.00150096\t\n",
      "1NF_17A.assembled\t0.01222267\t0.00155723\t\n",
      "1NF_24A.assembled\t0.01246688\t0.00156561\t\n",
      "1SN_3A.assembled\t0.01289977\t0.00173733\t\n",
      "1SN_6A.assembled\t0.01147223\t0.00148751\t\n",
      "1SN_18A.assembled\t0.01207448\t0.00169724\t\n",
      "1SN_25A.assembled\t0.01237121\t0.00153697\t\n",
      "1NF_31A.assembled\t0.0120439\t0.00169293\t\n",
      "1SN_4A.assembled\t0.01186629\t0.00181512\t\n",
      "1HL_19A.assembled\t0.01249681\t0.001469\t\n",
      "1SN_14A.assembled\t0.01188004\t0.00151311\t\n",
      "1HL_28A.assembled\t0.01284234\t0.00148895\t\n",
      "1SN_19A.assembled\t0.01226379\t0.00146784\t\n",
      "1NF_4A.assembled\t0.01189766\t0.00151837\t\n",
      "1SN_5A.assembled\t0.01218246\t0.00154433\t\n",
      "1NF_32A.assembled\t0.01184959\t0.00150245\t\n",
      "1SN_12A.assembled\t0.01194628\t0.0015164\t\n",
      "1NF_23A.assembled\t0.01202496\t0.00159625\t\n",
      "1SN_9A.assembled\t0.01227092\t0.00147231\t\n",
      "1SN_23A.assembled\t0.0119076\t0.00165804\t\n",
      "1HL_11A.assembled\t0.01216978\t0.00165352\t\n",
      "1SN_30A.assembled\t0.01212247\t0.00153805\t\n",
      "1SN_21A.assembled\t0.01214048\t0.00152167\t\n",
      "1NF_13A.assembled\t0.01200208\t0.00144921\t\n",
      "1NF_21A.assembled\t0.01227234\t0.00160427\t\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/Pi_E_estimate.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Consenus base calling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 5: creating consensus seqs for 96 samples, using H=0.01247 E=0.00154\n",
      "\t................................................................................................\n",
      "real\t283m28.123s\n",
      "user\t2225m48.454s\n",
      "sys\t0m21.896s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "taxon          \tnloci\tf1loci\tf2loci\tnsites\tnpoly\tpoly\n",
      "1SN_9A.assembled\t126666\t39351\t36531\t3608975\t27926\t0.0077379\n",
      "1NF_21A.assembled\t161246\t39443\t36671\t3636146\t28281\t0.0077777\n",
      "1HL_11A.assembled\t125407\t39736\t36789\t3659499\t28748\t0.0078557\n",
      "1SN_21A.assembled\t131359\t40405\t37391\t3710615\t29515\t0.0079542\n",
      "1NF_23A.assembled\t122732\t40195\t37205\t3685221\t27872\t0.0075632\n",
      "1SN_23A.assembled\t118344\t43478\t40280\t4003823\t30280\t0.0075628\n",
      "1SN_30A.assembled\t121564\t43820\t40498\t4028393\t30831\t0.0076534\n",
      "1NF_13A.assembled\t127575\t43904\t40669\t4061429\t31655\t0.0077941\n",
      "1SN_19A.assembled\t120143\t37039\t34302\t3391779\t26288\t0.0077505\n",
      "1SN_5A.assembled\t121723\t38622\t35831\t3535911\t27199\t0.0076922\n",
      "1SN_12A.assembled\t116140\t41408\t38556\t3826417\t29483\t0.0077051\n",
      "1HL_28A.assembled\t104605\t38307\t35400\t3494480\t29496\t0.0084407\n",
      "1NF_32A.assembled\t115584\t42091\t39211\t3910432\t29914\t0.0076498\n",
      "1NF_4A.assembled\t110918\t42309\t39302\t3937693\t30501\t0.0077459\n",
      "1HL_19A.assembled\t111346\t38543\t35529\t3516253\t28770\t0.008182\n",
      "1SN_14A.assembled\t110527\t39960\t37012\t3672750\t28933\t0.0078777\n",
      "1SN_3A.assembled\t123243\t30267\t27902\t2757712\t22382\t0.0081161\n",
      "1SN_25A.assembled\t120465\t34264\t31627\t3121698\t24600\t0.0078803\n",
      "1SN_4A.assembled\t111639\t40461\t37537\t3700940\t28499\t0.0077005\n",
      "1NF_31A.assembled\t109056\t40135\t37206\t3699649\t29395\t0.0079453\n",
      "1SN_18A.assembled\t113397\t37806\t34886\t3437679\t26836\t0.0078064\n",
      "1NF_24A.assembled\t109692\t36482\t33637\t3323227\t26392\t0.0079417\n",
      "1SN_6A.assembled\t107796\t39568\t36696\t3633773\t28379\t0.0078098\n",
      "1NF_17A.assembled\t108371\t36993\t34269\t3396706\t26808\t0.0078924\n",
      "1HL_26A.assembled\t109215\t36539\t33940\t3355241\t27093\t0.0080748\n",
      "1NF_9A.assembled\t103833\t38701\t35800\t3566248\t27750\t0.0077813\n",
      "1SN_22A.assembled\t107304\t37128\t34313\t3381298\t27082\t0.0080094\n",
      "1SN_7A.assembled\t107945\t37270\t34523\t3405066\t26766\t0.0078606\n",
      "1SN_31A.assembled\t103367\t37462\t34522\t3408204\t27267\t0.0080004\n",
      "1NF_2A.assembled\t102160\t39089\t36185\t3613096\t27291\t0.0075534\n",
      "1SN_2A.assembled\t106745\t36280\t33499\t3295202\t26624\t0.0080796\n",
      "1NF_29A.assembled\t97131\t38665\t35705\t3572818\t28739\t0.0080438\n",
      "1HL_9A.assembled\t113633\t33903\t31474\t3108523\t24533\t0.0078922\n",
      "1SN_17A.assembled\t106838\t36925\t34316\t3397345\t26326\t0.007749\n",
      "1SN_32A.assembled\t117048\t30430\t28152\t2765465\t22416\t0.0081057\n",
      "1NF_22A.assembled\t108890\t35268\t32593\t3205990\t25077\t0.0078219\n",
      "1SN_13A.assembled\t106500\t36679\t34119\t3377673\t26667\t0.0078951\n",
      "1SN_8A.assembled\t102124\t39277\t36419\t3606017\t27214\t0.0075468\n",
      "1NF_26A.assembled\t105799\t37307\t34584\t3417125\t27144\t0.0079435\n",
      "1HL_23A.assembled\t94229\t39679\t36679\t3670297\t28098\t0.0076555\n",
      "1HL_27A.assembled\t99679\t38496\t35515\t3506773\t27942\t0.007968\n",
      "1HL_3A.assembled\t113436\t29668\t27610\t2706497\t21429\t0.0079176\n",
      "1HL_7A.assembled\t100958\t34603\t32043\t3133581\t25103\t0.008011\n",
      "1SN_10A.assembled\t107961\t32943\t30498\t2992413\t23627\t0.0078956\n",
      "1SN_11A.assembled\t104454\t33399\t30952\t3032091\t23582\t0.0077775\n",
      "1NF_16A.assembled\t101457\t33028\t30579\t3023331\t24401\t0.0080709\n",
      "1SN_26A.assembled\t100318\t33495\t31062\t3054944\t24463\t0.0080077\n",
      "1SN_28A.assembled\t96449\t33988\t31452\t3103839\t24776\t0.0079824\n",
      "1SN_20A.assembled\t110647\t28100\t26023\t2546672\t20322\t0.0079798\n",
      "1HL_21A.assembled\t104256\t30845\t28517\t2804488\t22863\t0.0081523\n",
      "1SN_1A.assembled\t95372\t34524\t31959\t3175357\t25129\t0.0079138\n",
      "1NF_5A.assembled\t97753\t34686\t32182\t3206890\t24794\t0.0077315\n",
      "1HL_29A.assembled\t93081\t34994\t32305\t3196604\t25791\t0.0080682\n",
      "1NF_14A.assembled\t99152\t32144\t29593\t2920320\t23617\t0.0080871\n",
      "1NF_8A.assembled\t98509\t33312\t30789\t3052509\t23625\t0.0077395\n",
      "1HL_20A.assembled\t96673\t32611\t30108\t2953428\t23741\t0.0080385\n",
      "1SN_15A.assembled\t97901\t32116\t29737\t2929488\t22906\t0.0078191\n",
      "1HL_2A.assembled\t93559\t34198\t31675\t3106805\t24414\t0.0078582\n",
      "1HL_35A.assembled\t95108\t32745\t30314\t2981191\t24792\t0.0083161\n",
      "1SN_27A.assembled\t96854\t30880\t28482\t2808518\t22876\t0.0081452\n",
      "1SN_16A.assembled\t96334\t32102\t29553\t2900317\t22978\t0.0079226\n",
      "1NF_7A.assembled\t96247\t32132\t29763\t2948720\t23567\t0.0079923\n",
      "1HL_34A.assembled\t90703\t34004\t31368\t3103956\t24993\t0.008052\n",
      "1HL_25A.assembled\t89569\t34944\t32390\t3192139\t25739\t0.0080632\n",
      "1HL_4A.assembled\t93231\t33120\t30738\t3033354\t23946\t0.0078942\n",
      "1NF_30A.assembled\t92056\t33360\t30959\t3056959\t24605\t0.0080488\n",
      "1HL_31A.assembled\t95291\t31165\t28789\t2817102\t23125\t0.0082088\n",
      "1NF_33A.assembled\t91154\t33841\t31504\t3106099\t24494\t0.0078858\n",
      "1NF_6A.assembled\t94940\t30921\t28724\t2843186\t22585\t0.0079436\n",
      "1NF_15A.assembled\t93177\t32266\t29930\t2959421\t22973\t0.0077627\n",
      "1HL_33A.assembled\t90312\t31968\t29601\t2908044\t23843\t0.008199\n",
      "1NF_20A.assembled\t91488\t31865\t29615\t2925249\t23684\t0.0080964\n",
      "1HL_24A.assembled\t88576\t30943\t28613\t2823744\t22660\t0.0080248\n",
      "1NF_28A.assembled\t92468\t30327\t28091\t2782839\t21855\t0.0078535\n",
      "1HL_8A.assembled\t87336\t30088\t27767\t2756324\t22606\t0.0082015\n",
      "1HL_22A.assembled\t92252\t27801\t25710\t2521484\t20380\t0.0080825\n",
      "1NF_27A.assembled\t90432\t29783\t27582\t2715903\t21908\t0.0080666\n",
      "1HL_1A.assembled\t89466\t29467\t27230\t2658920\t21428\t0.0080589\n",
      "1NF_11A.assembled\t88047\t28538\t26386\t2609746\t21160\t0.0081081\n",
      "1NF_10A.assembled\t90811\t27220\t25173\t2485274\t19734\t0.0079404\n",
      "1HL_5A.assembled\t87447\t27026\t25137\t2467239\t18819\t0.0076276\n",
      "1NF_18A.assembled\t87277\t24553\t22760\t2237750\t18717\t0.0083642\n",
      "1SN_29A.assembled\t88849\t23673\t21743\t2168624\t17003\t0.0078405\n",
      "1NF_12A.assembled\t83679\t28518\t26486\t2601756\t20544\t0.0078962\n",
      "1NF_25A.assembled\t83423\t27750\t25672\t2533230\t20930\t0.0082622\n",
      "1HL_16A.assembled\t1494\t791\t736\t73247\t611\t0.0083416\n",
      "1HL_6A.assembled\t85254\t26998\t24895\t2465456\t20399\t0.0082739\n",
      "1HL_12A.assembled\t1492\t747\t689\t69158\t509\t0.00736\n",
      "1HL_15A.assembled\t1495\t759\t702\t69999\t542\t0.007743\n",
      "1HL_17A.assembled\t2990\t1793\t1631\t162299\t1403\t0.0086445\n",
      "1HL_13A.assembled\t497\t266\t247\t24456\t188\t0.0076873\n",
      "1HL_10A.assembled\t997\t528\t489\t48543\t414\t0.0085285\n",
      "1SN_24A.assembled\t83024\t24881\t22983\t2252342\t18213\t0.0080862\n",
      "1HL_14A.assembled\t6986\t3553\t3301\t328955\t2940\t0.0089374\n",
      "1NF_19A.assembled\t75939\t24872\t22971\t2267086\t18505\t0.0081625\n",
      "1NF_1A.assembled\t62149\t18708\t17365\t1733112\t14104\t0.008138\n",
      "\n",
      "    ## nloci = number of loci\n",
      "    ## f1loci = number of loci with >N depth coverage\n",
      "    ## f2loci = number of loci with >N depth and passed paralog filter\n",
      "    ## nsites = number of sites across f loci\n",
      "    ## npoly = number of polymorphic sites in nsites\n",
      "    ## poly = frequency of polymorphic sites\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/s5.consens.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cluster across samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vsearch v1.11.1_linux_x86_64, 14.7GB RAM, 8 cores\n",
      "https://github.com/torognes/vsearch\n",
      "\n",
      "\n",
      "\tfinished clustering\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tstep 6: clustering across 96 samples at '.88' similarity \n",
      "\n",
      "Reading file /data/analysis/clust.88/cat.haplos_ 100%\n",
      "291106371 nt in 2830448 seqs, min 32, max 234, avg 103\n",
      "Counting unique k-mers 100%\n",
      "Clustering 100%\n",
      "Sorting clusters 100%\n",
      "Writing clusters 100%\n",
      "Clusters: 125434 Size min 1, max 2641, avg 22.6\n",
      "Singletons: 25519, 0.9% of seqs, 20.3% of clusters\n",
      "\n",
      "real\t38m13.548s\n",
      "user\t60m4.549s\n",
      "sys\t0m14.356s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assemble final data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tingroup 1HL_10A.assembled,1HL_11A.assembled,1HL_12A.assembled,1HL_13A.assembled,1HL_14A.assembled,1HL_15A.assembled,1HL_16A.assembled,1HL_17A.assembled,1HL_19A.assembled,1HL_1A.assembled,1HL_20A.assembled,1HL_21A.assembled,1HL_22A.assembled,1HL_23A.assembled,1HL_24A.assembled,1HL_25A.assembled,1HL_26A.assembled,1HL_27A.assembled,1HL_28A.assembled,1HL_29A.assembled,1HL_2A.assembled,1HL_31A.assembled,1HL_33A.assembled,1HL_34A.assembled,1HL_35A.assembled,1HL_3A.assembled,1HL_4A.assembled,1HL_5A.assembled,1HL_6A.assembled,1HL_7A.assembled,1HL_8A.assembled,1HL_9A.assembled,1NF_10A.assembled,1NF_11A.assembled,1NF_12A.assembled,1NF_13A.assembled,1NF_14A.assembled,1NF_15A.assembled,1NF_16A.assembled,1NF_17A.assembled,1NF_18A.assembled,1NF_19A.assembled,1NF_1A.assembled,1NF_20A.assembled,1NF_21A.assembled,1NF_22A.assembled,1NF_23A.assembled,1NF_24A.assembled,1NF_25A.assembled,1NF_26A.assembled,1NF_27A.assembled,1NF_28A.assembled,1NF_29A.assembled,1NF_2A.assembled,1NF_30A.assembled,1NF_31A.assembled,1NF_32A.assembled,1NF_33A.assembled,1NF_4A.assembled,1NF_5A.assembled,1NF_6A.assembled,1NF_7A.assembled,1NF_8A.assembled,1NF_9A.assembled,1SN_10A.assembled,1SN_11A.assembled,1SN_12A.assembled,1SN_13A.assembled,1SN_14A.assembled,1SN_15A.assembled,1SN_16A.assembled,1SN_17A.assembled,1SN_18A.assembled,1SN_19A.assembled,1SN_1A.assembled,1SN_20A.assembled,1SN_21A.assembled,1SN_22A.assembled,1SN_23A.assembled,1SN_24A.assembled,1SN_25A.assembled,1SN_26A.assembled,1SN_27A.assembled,1SN_28A.assembled,1SN_29A.assembled,1SN_2A.assembled,1SN_30A.assembled,1SN_31A.assembled,1SN_32A.assembled,1SN_3A.assembled,1SN_4A.assembled,1SN_5A.assembled,1SN_6A.assembled,1SN_7A.assembled,1SN_8A.assembled,1SN_9A.assembled\n",
      "\taddon \n",
      "\texclude \n",
      "\t\n",
      "\tfinal stats written to:\n",
      "\t /data/analysis/stats/oly_gbs_pyrad.stats\n",
      "\toutput files being written to:\n",
      "\t /data/analysis/outfiles/ directory\n",
      "\n",
      "\tfiltering & writing to phylip file\n",
      "\twriting nexus file\n",
      "\tWriting gphocs file\n",
      "\t  + writing full SNPs file\n",
      "\t  + writing unlinked bi-allelic SNPs file\n",
      "\t  + writing STRUCTURE file\n",
      "\t  + writing geno file\n",
      "\t  + writing treemix file\n",
      "\t    data set reduced for group coverage minimums\n",
      "\t    1HL ['1HL_5A.assembled', '1HL_31A.assembled', '1HL_11A.assembled', '1HL_20A.assembled', '1HL_4A.assembled', '1HL_2A.assembled', '1HL_9A.assembled', '1HL_29A.assembled', '1HL_17A.assembled', '1HL_35A.assembled', '1HL_27A.assembled', '1HL_6A.assembled', '1HL_16A.assembled', '1HL_21A.assembled', '1HL_12A.assembled', '1HL_26A.assembled', '1HL_10A.assembled', '1HL_23A.assembled', '1HL_22A.assembled', '1HL_7A.assembled', '1HL_8A.assembled', '1HL_15A.assembled', '1HL_24A.assembled', '1HL_25A.assembled', '1HL_1A.assembled', '1HL_19A.assembled', '1HL_13A.assembled', '1HL_33A.assembled', '1HL_3A.assembled', '1HL_34A.assembled', '1HL_28A.assembled', '1HL_14A.assembled'] minimum= 5\n",
      "\t    1NF ['1NF_29A.assembled', '1NF_24A.assembled', '1NF_14A.assembled', '1NF_31A.assembled', '1NF_5A.assembled', '1NF_6A.assembled', '1NF_9A.assembled', '1NF_10A.assembled', '1NF_18A.assembled', '1NF_21A.assembled', '1NF_15A.assembled', '1NF_17A.assembled', '1NF_7A.assembled', '1NF_8A.assembled', '1NF_32A.assembled', '1NF_1A.assembled', '1NF_20A.assembled', '1NF_19A.assembled', '1NF_23A.assembled', '1NF_33A.assembled', '1NF_27A.assembled', '1NF_28A.assembled', '1NF_30A.assembled', '1NF_4A.assembled', '1NF_25A.assembled', '1NF_13A.assembled', '1NF_26A.assembled', '1NF_2A.assembled', '1NF_11A.assembled', '1NF_12A.assembled', '1NF_22A.assembled', '1NF_16A.assembled'] minimum= 5\n",
      "\t    1SN ['1SN_31A.assembled', '1SN_20A.assembled', '1SN_11A.assembled', '1SN_32A.assembled', '1SN_6A.assembled', '1SN_5A.assembled', '1SN_15A.assembled', '1SN_13A.assembled', '1SN_27A.assembled', '1SN_16A.assembled', '1SN_19A.assembled', '1SN_23A.assembled', '1SN_3A.assembled', '1SN_1A.assembled', '1SN_17A.assembled', '1SN_25A.assembled', '1SN_22A.assembled', '1SN_21A.assembled', '1SN_7A.assembled', '1SN_24A.assembled', '1SN_10A.assembled', '1SN_29A.assembled', '1SN_2A.assembled', '1SN_26A.assembled', '1SN_14A.assembled', '1SN_12A.assembled', '1SN_4A.assembled', '1SN_18A.assembled', '1SN_28A.assembled', '1SN_8A.assembled', '1SN_30A.assembled', '1SN_9A.assembled'] minimum= 5\n",
      "\twriting vcf file\n",
      "\twriting migrate-n file\n",
      "\t    data set reduced for group coverage minimums\n",
      "\t    1HL ['1HL_5A.assembled', '1HL_31A.assembled', '1HL_11A.assembled', '1HL_20A.assembled', '1HL_4A.assembled', '1HL_2A.assembled', '1HL_9A.assembled', '1HL_29A.assembled', '1HL_17A.assembled', '1HL_35A.assembled', '1HL_27A.assembled', '1HL_6A.assembled', '1HL_16A.assembled', '1HL_21A.assembled', '1HL_12A.assembled', '1HL_26A.assembled', '1HL_10A.assembled', '1HL_23A.assembled', '1HL_22A.assembled', '1HL_7A.assembled', '1HL_8A.assembled', '1HL_15A.assembled', '1HL_24A.assembled', '1HL_25A.assembled', '1HL_1A.assembled', '1HL_19A.assembled', '1HL_13A.assembled', '1HL_33A.assembled', '1HL_3A.assembled', '1HL_34A.assembled', '1HL_28A.assembled', '1HL_14A.assembled'] minimum= 5\n",
      "\t    1NF ['1NF_29A.assembled', '1NF_24A.assembled', '1NF_14A.assembled', '1NF_31A.assembled', '1NF_5A.assembled', '1NF_6A.assembled', '1NF_9A.assembled', '1NF_10A.assembled', '1NF_18A.assembled', '1NF_21A.assembled', '1NF_15A.assembled', '1NF_17A.assembled', '1NF_7A.assembled', '1NF_8A.assembled', '1NF_32A.assembled', '1NF_1A.assembled', '1NF_20A.assembled', '1NF_19A.assembled', '1NF_23A.assembled', '1NF_33A.assembled', '1NF_27A.assembled', '1NF_28A.assembled', '1NF_30A.assembled', '1NF_4A.assembled', '1NF_25A.assembled', '1NF_13A.assembled', '1NF_26A.assembled', '1NF_2A.assembled', '1NF_11A.assembled', '1NF_12A.assembled', '1NF_22A.assembled', '1NF_16A.assembled'] minimum= 5\n",
      "\t    1SN ['1SN_31A.assembled', '1SN_20A.assembled', '1SN_11A.assembled', '1SN_32A.assembled', '1SN_6A.assembled', '1SN_5A.assembled', '1SN_15A.assembled', '1SN_13A.assembled', '1SN_27A.assembled', '1SN_16A.assembled', '1SN_19A.assembled', '1SN_23A.assembled', '1SN_3A.assembled', '1SN_1A.assembled', '1SN_17A.assembled', '1SN_25A.assembled', '1SN_22A.assembled', '1SN_21A.assembled', '1SN_7A.assembled', '1SN_24A.assembled', '1SN_10A.assembled', '1SN_29A.assembled', '1SN_2A.assembled', '1SN_26A.assembled', '1SN_14A.assembled', '1SN_12A.assembled', '1SN_4A.assembled', '1SN_18A.assembled', '1SN_28A.assembled', '1SN_8A.assembled', '1SN_30A.assembled', '1SN_9A.assembled'] minimum= 5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "     ------------------------------------------------------------\n",
      "      pyRAD : RADseq for phylogenetics & introgression analyses\n",
      "     ------------------------------------------------------------\n",
      "\n",
      "\n",
      "\tCluster input file: using \n",
      "\t/data/analysis/clust.88/cat.clust_.gz\n",
      "\n",
      "........Traceback (most recent call last):\n",
      "  File \"/usr/local/bin/pyrad\", line 9, in <module>\n",
      "    load_entry_point('pyrad==3.0.66', 'console_scripts', 'pyrad')()\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/pyRAD.py\", line 527, in main\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/alignable.py\", line 1013, in main\n",
      "  File \"build/bdist.linux-x86_64/egg/pyrad/loci2mig.py\", line 75, in make\n",
      "IndexError: list index out of range\n",
      "\n",
      "real\t62m40.523s\n",
      "user\t199m9.651s\n",
      "sys\t18m54.962s\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "time pyrad -p /data/analysis/params.txt -s 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Well, that error is a bit disconcerting... Let's look at the results and output files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examine results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "71174       ## loci with > minsp containing data\n",
      "46377       ## loci with > minsp containing data & paralogs removed\n",
      "46377       ## loci with > minsp containing data & paralogs removed & final filtering\n",
      "\n",
      "## number of loci recovered in final data set for each taxon.\n",
      "taxon\tnloci\n",
      "1HL_10A.assembled\t175\n",
      "1HL_11A.assembled\t13726\n",
      "1HL_12A.assembled\t250\n",
      "1HL_13A.assembled\t89\n",
      "1HL_14A.assembled\t1124\n",
      "1HL_15A.assembled\t242\n",
      "1HL_16A.assembled\t253\n",
      "1HL_17A.assembled\t614\n",
      "1HL_19A.assembled\t13237\n",
      "1HL_1A.assembled \t9589\n",
      "1HL_20A.assembled\t10722\n",
      "1HL_21A.assembled\t10058\n",
      "1HL_22A.assembled\t8946\n",
      "1HL_23A.assembled\t13897\n",
      "1HL_24A.assembled\t10120\n",
      "1HL_25A.assembled\t11994\n",
      "1HL_26A.assembled\t12644\n",
      "1HL_27A.assembled\t13331\n",
      "1HL_28A.assembled\t13299\n",
      "1HL_29A.assembled\t11531\n",
      "1HL_2A.assembled \t11332\n",
      "1HL_31A.assembled\t10212\n",
      "1HL_33A.assembled\t10466\n",
      "1HL_34A.assembled\t11384\n",
      "1HL_35A.assembled\t10929\n",
      "1HL_3A.assembled \t9789\n",
      "1HL_4A.assembled \t10970\n",
      "1HL_5A.assembled \t8488\n",
      "1HL_6A.assembled \t8568\n",
      "1HL_7A.assembled \t11704\n",
      "1HL_8A.assembled \t9702\n",
      "1HL_9A.assembled \t11520\n",
      "1NF_10A.assembled\t8772\n",
      "1NF_11A.assembled\t9182\n",
      "1NF_12A.assembled\t9173\n",
      "1NF_13A.assembled\t15824\n",
      "1NF_14A.assembled\t10389\n",
      "1NF_15A.assembled\t10584\n",
      "1NF_16A.assembled\t10908\n",
      "1NF_17A.assembled\t12588\n",
      "1NF_18A.assembled\t7653\n",
      "1NF_19A.assembled\t7740\n",
      "1NF_1A.assembled \t5730\n",
      "1NF_20A.assembled\t10530\n",
      "1NF_21A.assembled\t13420\n",
      "1NF_22A.assembled\t11876\n",
      "1NF_23A.assembled\t14146\n",
      "1NF_24A.assembled\t12340\n",
      "1NF_25A.assembled\t8823\n",
      "1NF_26A.assembled\t12830\n",
      "1NF_27A.assembled\t9581\n",
      "1NF_28A.assembled\t9856\n",
      "1NF_29A.assembled\t13202\n",
      "1NF_2A.assembled \t13564\n",
      "1NF_30A.assembled\t11282\n",
      "1NF_31A.assembled\t13992\n",
      "1NF_32A.assembled\t15263\n",
      "1NF_33A.assembled\t11421\n",
      "1NF_4A.assembled \t15121\n",
      "1NF_5A.assembled \t11616\n",
      "1NF_6A.assembled \t10245\n",
      "1NF_7A.assembled \t10526\n",
      "1NF_8A.assembled \t10983\n",
      "1NF_9A.assembled \t13320\n",
      "1SN_10A.assembled\t10945\n",
      "1SN_11A.assembled\t11161\n",
      "1SN_12A.assembled\t14540\n",
      "1SN_13A.assembled\t12521\n",
      "1SN_14A.assembled\t13881\n",
      "1SN_15A.assembled\t10552\n",
      "1SN_16A.assembled\t10531\n",
      "1SN_17A.assembled\t12615\n",
      "1SN_18A.assembled\t12986\n",
      "1SN_19A.assembled\t12675\n",
      "1SN_1A.assembled \t11588\n",
      "1SN_20A.assembled\t9021\n",
      "1SN_21A.assembled\t14061\n",
      "1SN_22A.assembled\t12667\n",
      "1SN_23A.assembled\t15501\n",
      "1SN_24A.assembled\t7868\n",
      "1SN_25A.assembled\t11435\n",
      "1SN_26A.assembled\t11144\n",
      "1SN_27A.assembled\t10070\n",
      "1SN_28A.assembled\t11328\n",
      "1SN_29A.assembled\t7278\n",
      "1SN_2A.assembled \t12322\n",
      "1SN_30A.assembled\t15723\n",
      "1SN_31A.assembled\t12763\n",
      "1SN_32A.assembled\t9938\n",
      "1SN_3A.assembled \t9641\n",
      "1SN_4A.assembled \t14322\n",
      "1SN_5A.assembled \t13373\n",
      "1SN_6A.assembled \t13776\n",
      "1SN_7A.assembled \t12741\n",
      "1SN_8A.assembled \t13723\n",
      "1SN_9A.assembled \t13737\n",
      "\n",
      "\n",
      "## nloci = number of loci with data for exactly ntaxa\n",
      "## ntotal = number of loci for which at least ntaxa have data\n",
      "ntaxa\tnloci\tsaved\tntotal\n",
      "1\t-\n",
      "2\t-\t\t-\n",
      "3\t-\t\t-\n",
      "4\t5125\t*\t46377\n",
      "5\t4040\t*\t41252\n",
      "6\t3276\t*\t37212\n",
      "7\t2709\t*\t33936\n",
      "8\t2280\t*\t31227\n",
      "9\t1864\t*\t28947\n",
      "10\t1683\t*\t27083\n",
      "11\t1501\t*\t25400\n",
      "12\t1392\t*\t23899\n",
      "13\t1164\t*\t22507\n",
      "14\t1098\t*\t21343\n",
      "15\t1078\t*\t20245\n",
      "16\t832\t*\t19167\n",
      "17\t864\t*\t18335\n",
      "18\t773\t*\t17471\n",
      "19\t731\t*\t16698\n",
      "20\t611\t*\t15967\n",
      "21\t595\t*\t15356\n",
      "22\t593\t*\t14761\n",
      "23\t495\t*\t14168\n",
      "24\t479\t*\t13673\n",
      "25\t465\t*\t13194\n",
      "26\t411\t*\t12729\n",
      "27\t398\t*\t12318\n",
      "28\t368\t*\t11920\n",
      "29\t377\t*\t11552\n",
      "30\t338\t*\t11175\n",
      "31\t321\t*\t10837\n",
      "32\t313\t*\t10516\n",
      "33\t303\t*\t10203\n",
      "34\t302\t*\t9900\n",
      "35\t260\t*\t9598\n",
      "36\t256\t*\t9338\n",
      "37\t241\t*\t9082\n",
      "38\t234\t*\t8841\n",
      "39\t211\t*\t8607\n",
      "40\t228\t*\t8396\n",
      "41\t203\t*\t8168\n",
      "42\t211\t*\t7965\n",
      "43\t189\t*\t7754\n",
      "44\t179\t*\t7565\n",
      "45\t176\t*\t7386\n",
      "46\t162\t*\t7210\n",
      "47\t195\t*\t7048\n",
      "48\t154\t*\t6853\n",
      "49\t185\t*\t6699\n",
      "50\t177\t*\t6514\n",
      "51\t146\t*\t6337\n",
      "52\t152\t*\t6191\n",
      "53\t160\t*\t6039\n",
      "54\t144\t*\t5879\n",
      "55\t141\t*\t5735\n",
      "56\t140\t*\t5594\n",
      "57\t134\t*\t5454\n",
      "58\t137\t*\t5320\n",
      "59\t128\t*\t5183\n",
      "60\t146\t*\t5055\n",
      "61\t130\t*\t4909\n",
      "62\t129\t*\t4779\n",
      "63\t132\t*\t4650\n",
      "64\t128\t*\t4518\n",
      "65\t109\t*\t4390\n",
      "66\t117\t*\t4281\n",
      "67\t123\t*\t4164\n",
      "68\t137\t*\t4041\n",
      "69\t125\t*\t3904\n",
      "70\t128\t*\t3779\n",
      "71\t155\t*\t3651\n",
      "72\t122\t*\t3496\n",
      "73\t129\t*\t3374\n",
      "74\t123\t*\t3245\n",
      "75\t123\t*\t3122\n",
      "76\t133\t*\t2999\n",
      "77\t142\t*\t2866\n",
      "78\t156\t*\t2724\n",
      "79\t178\t*\t2568\n",
      "80\t165\t*\t2390\n",
      "81\t161\t*\t2225\n",
      "82\t165\t*\t2064\n",
      "83\t169\t*\t1899\n",
      "84\t192\t*\t1730\n",
      "85\t198\t*\t1538\n",
      "86\t195\t*\t1340\n",
      "87\t239\t*\t1145\n",
      "88\t283\t*\t906\n",
      "89\t462\t*\t623\n",
      "90\t142\t*\t161\n",
      "91\t17\t*\t19\n",
      "92\t2\t*\t2\n",
      "\n",
      "\n",
      "## nvar = number of loci containing n variable sites (pis+autapomorphies).\n",
      "## sumvar = sum of variable sites (SNPs).\n",
      "## pis = number of loci containing n parsimony informative sites.\n",
      "## sumpis = sum of parsimony informative sites.\n",
      "\tnvar\tsumvar\tPIS\tsumPIS\n",
      "0\t21211\t0\t29874\t0\n",
      "1\t7261\t7261\t5694\t5694\n",
      "2\t3835\t14931\t2967\t11628\n",
      "3\t2729\t23118\t2097\t17919\n",
      "4\t2015\t31178\t1448\t23711\n",
      "5\t1694\t39648\t1109\t29256\n",
      "6\t1371\t47874\t880\t34536\n",
      "7\t1176\t56106\t646\t39058\n",
      "8\t997\t64082\t476\t42866\n",
      "9\t787\t71165\t334\t45872\n",
      "10\t634\t77505\t226\t48132\n",
      "11\t508\t83093\t160\t49892\n",
      "12\t386\t87725\t126\t51404\n",
      "13\t337\t92106\t88\t52548\n",
      "14\t276\t95970\t68\t53500\n",
      "15\t208\t99090\t43\t54145\n",
      "16\t186\t102066\t35\t54705\n",
      "17\t140\t104446\t16\t54977\n",
      "18\t120\t106606\t15\t55247\n",
      "19\t90\t108316\t15\t55532\n",
      "20\t70\t109716\t10\t55732\n",
      "21\t56\t110892\t10\t55942\n",
      "22\t43\t111838\t6\t56074\n",
      "23\t46\t112896\t10\t56304\n",
      "24\t35\t113736\t5\t56424\n",
      "25\t27\t114411\t6\t56574\n",
      "26\t26\t115087\t3\t56652\n",
      "27\t22\t115681\t0\t56652\n",
      "28\t11\t115989\t2\t56708\n",
      "29\t12\t116337\t0\t56708\n",
      "30\t17\t116847\t2\t56768\n",
      "31\t9\t117126\t1\t56799\n",
      "32\t4\t117254\t0\t56799\n",
      "33\t4\t117386\t0\t56799\n",
      "34\t5\t117556\t0\t56799\n",
      "35\t3\t117661\t0\t56799\n",
      "36\t3\t117769\t3\t56907\n",
      "37\t4\t117917\t1\t56944\n",
      "38\t3\t118031\t0\t56944\n",
      "39\t2\t118109\t0\t56944\n",
      "40\t1\t118149\t0\t56944\n",
      "41\t0\t118149\t0\t56944\n",
      "42\t2\t118233\t0\t56944\n",
      "43\t3\t118362\t0\t56944\n",
      "44\t0\t118362\t0\t56944\n",
      "45\t2\t118452\t1\t56989\n",
      "46\t0\t118452\t0\t56989\n",
      "47\t0\t118452\t0\t56989\n",
      "48\t0\t118452\t0\t56989\n",
      "49\t2\t118550\t0\t56989\n",
      "50\t1\t118600\t0\t56989\n",
      "51\t0\t118600\t0\t56989\n",
      "52\t1\t118652\t0\t56989\n",
      "53\t0\t118652\t0\t56989\n",
      "54\t1\t118706\t0\t56989\n",
      "55\t1\t118761\t0\t56989\n",
      "total var= 118761\n",
      "total pis= 56989\n",
      "sampled unlinked SNPs= 25166\n",
      "sampled unlinked bi-allelic SNPs= 25039\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat /data/analysis/stats/oly_gbs_pyrad.stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### I'm not sure what all of that means..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "oly_gbs_pyrad.alleles\n",
      "oly_gbs_pyrad.excluded_loci\n",
      "oly_gbs_pyrad.gphocs\n",
      "oly_gbs_pyrad.loci\n",
      "oly_gbs_pyrad.migrate\n",
      "oly_gbs_pyrad.nex\n",
      "oly_gbs_pyrad.phy\n",
      "oly_gbs_pyrad.phy.partitions\n",
      "oly_gbs_pyrad.snps\n",
      "oly_gbs_pyrad.snps.geno\n",
      "oly_gbs_pyrad.str\n",
      "oly_gbs_pyrad.treemix.gz\n",
      "oly_gbs_pyrad.unlinked_snps\n",
      "oly_gbs_pyrad.usnps.geno\n",
      "oly_gbs_pyrad.vcf\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "ls /data/analysis/outfiles/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}