{ "metadata": { "name": "BiGill_Tran_Elements", "signature": "sha256:996533932f4a9173df9748da5cabf43ff8a1ee61fe62f061b5a55e810730110a" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "#Transposable Element Methylation \n", "\n", "Summary: using BSMAP to look at TE specific methylation \n", "Updated: June 21, 2013" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Previously Transposable Elements were identifed - [evernote link](https://www.evernote.com/shard/s10/sh/7dea995c-17ac-4bcf-bc38-963220e9e7c9/b28dacbbdbfe123960b88e42fa45a34a) \n", "\n", "Mac has taken ? and run fastafromBed however sequence name was uniformative.\n", "\n", "`\n", "./fastaFromBed -fi /Volumes/web-1/bivalvia/wholegenomefiles_MBDbsSeq_gill/oyster.v9.fa -bed /Volumes/web-1/bivalvia/wholegenomefiles_MBDbsSeq_gill/qDOD_RepeatProteinMask_v9_gffformat.gff -fo /Volumes/web-1/bivalvia/wholegenomefiles_MBDbsSeq_gill/RepeatProteinMask_v9.fa -s \n", "`\n", "\n", "\n", "re-run using `-name`" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!fastaFromBed -s -name -fi /Volumes/web/cnidarian/oyster.v9.fa -bed /Volumes/web/bivalvia/wholegenomefiles_MBDbsSeq_gill/qDOD_RepeatProteinMask_v9_gffformat.gff -fo /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 7 }, { "cell_type": "code", "collapsed": false, "input": [ "!head /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ ">DNA/hAT-Tip100\r\n", "AGCTCGACGATGGACAAGTTTGTAACAAAAACGAATAGAAAAAGTACAGATGATCAGACAGCTGATAAAAATAACAACGAAACTACATTAGCGAAACATGGTAAATACGAAAAGACCAAACGGAAAAGATGCTATTTGAAAACGTGGGAGGAAACCTGGCCCTAGGTACGACATGACGATATAAATGACACAATGTATTACACAATTTGCAGGGAATTTTCTCCTGCGGTTGGCAACGTAAAAAACAATTCTTTCTACGAAGGATGCAAACTCTTTCATGTGGATTCTTTTAAGGCCCATCAAACTAGCGATGTTCATGTGATATGCACCAGTTTCTATCGCCAGAAAAACGGAACTCCCGTGACTAAGGGTGGAAATCAAGCACATATAAGTGACAATGTCAATGTTAGAGCAACACCTATTATGACAGCATTGCTGAAGTTAGATGAAAACCAAATGAAAAGGCTTGATGCTCTTTTCAAAATAGCATATAAAATTGCAAAACATGGTAAACCACACAGTGATTTTGAAATCGATTGTAAATTGATTCAGAAACTTGGTGTTGATCTTGGGAACAATTATTTTAATACAAATAGATGTAAAGATTTTATAAAATCTATCGCTGATGCTATGACTGAAGCTGTGGCTGATGATTTGAAAGGAGCAAATTTTGTTTTTGTTCTCTCTGATGGTAGCACTGACTGTAGTAACCTTGAACAAGAGAATGTCCCTGTACGATATGTGTCTCCTAAGACTCGGAATCCTGTAACAATATTTTGTGGCATTGTAAATTTGGAACATGGTCATGAAGATGGCGTTTAAGATGGCATTTTTAGAGCCTTGAGTTTAGTTGGCTTAACGAAGGAGAATTTGAAAGCTAATGAAGCAGGTCCAACATTAATTTGCGCGAACTTTGACGATGGAAATGTCATGCAAGGTAAGAAAAATGGAGTTGTGGGAAAACTCGTAAAGGAATACAATCATGTTTTGGGAATGTGGTGAATTGCACACAAACTTCAACTGGCAGTGATGGATTCAGTCAAGGATGTTCAAAGTCTTCAGGAA\r\n", ">DNA/hAT-Tip100\r\n", "GTAGAAACAAACCCAGAACAAATATCTAGTCAGTGTACACCTGATACTTCCATAATTAAATTGCACCTCGTTCATTCAGATCCAAAGTTAGATTGGGACGTCTCTATCATTGGCAATGAACCAAACCAACCACAGAAGAGTTTTCCTGTAACAATAATTGCAAAACGAAAGAGATCATTTGTTTACAGTTGGTTTAGTACCTGGACATGGCTTCATTATGAAGAATATTCGGACAAAGCTTTCTGTTTCTATTGTATTAAAGCTTACAAGGAGCAAAAGTTGGCAAACATGTGCAAAGAAAAGGCTTTCATATCTGATGGATTCAAAAATTGGAAAAAAGCCACATTAAAATTCAGAGAACATGATAACTCTGATTGTCACAGGGAAGCAGTCGAACGC\r\n", ">DNA/hAT-Tip100\r\n", "CAAACATTGAGTGATGGAAATTCTTATACATGTCCAAAAAGTGTTAGTGAATTCCAAAGTGTGTGTGCTAAGGTTGTTTTAATCAATGTCATTGAAAAAGTTTTGAATGCAGGAGTATTTTCTCTAATGCTTGATGAAAGCACAGACAGAGGAAACCGTAAACGGCTGTTGGTGTACATCCAATATCTACATGAGAGAAAACTACAGACCAGCCTCCTTAGCAACATTGAAATTTTAACAGCAAAAGCGGATGCTGAAACTATAACAAACCATGTTTTGACAGAGCTGCGGATGAAAGGCTTAAGTGTTGCAGGACTTGTTGGCATTGGAACTGACGGTGCGAATGTTATGACAGGGAAAAAGAGTGGAGTTGTTGTGAGGCTTCGGGAGCACAGTCCCACATTGATAGGAACACATTGTGCCGCTCATAGATGTGCCCTGGCTGCTTCACAAGCATCAAAATCAATTCCAGAGCTAAAGAAGTATTCTGACACAGTGTCAAATATATTTTTCTATTTTAGTGGCTCTGCTCTCAGATCAAATAAACTGAGGGAGATTCAAATGCTACTGAACCTCCCCATGTTAAAATTTGCCCAGCTCCACTCAGTCCGGTGGTTAAGTCTTCAGAAATCTGTGGAAATTCTGTACAGGACTTACCCAGCACTTCTTGTGGCATTAGAACATGAGGGAACAGTGAACCCCGCTGCAAGAGGAATATATGCAGAAGTATCGCAGTACAGTTTCATTGCAATCACACATATGCTTATGGACATCCTCCCATTTCTTGGAAAACTTAGTCGAATTTTTCAGACAAAGGACTTGGATCTAAGTAAAGTAACACCCATAGTCAAAAGCACCTGTGATGCACTTCTAGATTTCAAAGAGGCAGAGGGAATGTATGTTGATAAACTAGATGAGTTCATAATTAAAGATGGGGACAGTGTTTTGTACAGGAGACCTGACACTGAAAGTAGAAAATCAGCTGTCCAGGGAGCCATAGAAGTAAACATGGATGGATTCAGTGGTTTTGAAAAAGAGCATGAAGAGTGTAACTTTGAAGAAGAAGACAATGGGGTTCAAATTAGGTATTATCAACAACAACAAAACGTCCTCAGAAGTATCATGCCCAAGTACTTGGATACTATTGTTGCTAACTTAGAAAACAGATTTCAGGAGAATGGTTTTATTGAAAAAATGCAAGTGCTTCAGCCTATTCACATTACAGCTGCCCACAAGAAAGGAGAGCTAGCAAAGTATGGAAGTGAAGAAATTGCTGTTATGGCCAACCACTTTGCTATCCATGGAATTAATCCAGAGGAAACTCCAATGGAATACAAACAGTATAAACGTCTTGTAGTTGGATCATATCAAACTTCTTCCCTCAGTGATATGTGCTTTCACCTTGGATCCGAGTATGAAGATATCATGCCCAATATTGTGAGACTGATTCACTGTTGTACTGTGATACCAGTCAGCAGTGCAACATGTGAAAGAGGGTTCTCAACCCAAAACAGAATTAAATCTAGACTGAGGACCAATTTGAACAACATTTCATTAAATGATTTAATGAGAATTTCTGAAGATGGTCCTTCCATGGCACTTTTTGACTTTCCAGAGGCTCTGAAAGTATGGAAGGAAGAAAAGAAGCGA\r\n", ">DNA/hAT-Tip100\r\n", "AGGAAGGGAAAATATTCTGTCAACTGGATTCTGCAACTGGAAAGATGCTACCTCAAAGTTCCAAAACATCAAGACTCCGACTGTCACAAAGAGGCCGTGGAAAGAAAGTTGAAACTTCCTACAGAAACCAAGGACATCGGTGAAGTTCTCTCAGCCGCCCACTCAGAGGAGAAAACCCCCAACAGACAGCAAATGTTGACCATCCTACGCAACATTAGATTTCGTGCTAGACAGGGACTACCAATCCGTGGCCATGACGATAACCAGAGCAACTTCATCCAACTC\r\n", ">DNA/hAT-Tip100\r\n", "CCCAATATTGTGAGACTGATTCACTGTTGTACTGTGATACCAGTCAGCAGTGCAACATGTGAAAGAGGGTTCTCAACCCAAAACAGAATTAAATCTAGACTGAGGACCAACTTGAACAACAGTTCATTAAATGATTTAATGAGAATTTCTGAAGATGGTCCTTTTTCATGGCACACTTTGACTTTCCAGAGGCTCTGAAAATATGGAAGGAAGGAAAGA\r\n" ] } ], "prompt_number": 9 }, { "cell_type": "markdown", "metadata": {}, "source": [ "also have a Pearl file; Descriptions not informative but can parse post.\n" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!head /Volumes/web/bivalvia/wholegenomefiles_MBDbsSeq_gill/oyster_v9_pearlTE.fa" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ ">scaffold42662:175722-176097(+)\r\n", "TTAGCTCACCTGAGCCAAAGGCTCAAGTGAGCTTTTCTGATCACAATTTGTCCGTTGTCTGTCGTTGTCGTTGTCGTTGTCGTCGTCGTTGATGTCGTCGTCGTTGTAAACTTTTCACATTTTCATCTTCTTCTCAAGAACCACAGGGCCAATTTCAATCCAACTTGCCATAAAGTATCCTTAGGTAAAGGGGTTTCAGGTTTGTTCAAATGAAGATCCTTGCCCTCTTTCAAGGGGAGATAATTAGGAATTAGTGAAAAAGTAATGGTGTATTTTAAAAATCTTCTTCTCAAGAACCACTTGGCCAAAAAAGATGAAACTCATAGTGAAGTATCCTCAGGTGGTGTAGATTCAAGTTTGTTCAAATCATGACCC\r\n", ">scaffold48:322270-322642(+)\r\n", "TTAGCTCACCTGAGCCAAAGGCTCAAGTGAGCTTTTCTGATCACAATTTGTCCGTTGTCTGTCGTTGTCGTTGTCGTTGTCGTCGTTGTCGTCGTCGTTGTTGTAAACTTTTCACATTTTCATCTTCTTCTCAAGAACCACTGGGCAGATTCCAACCGAATTTGGCACAAATCCCCACTAGGTGAAGGGGATCCAGGTTTGTTCAAATAAAGTGCCACGCCCTCTTTAAAGGGGAGATAATTGAGAATTATTGAAAATTTGTTGGTATTTTTCAAAAATCTTCTTCTCAAAAACTATTCGGCCTGAAAAGCTTGAACTTGTGTGGAGGCATCCTCAGGTAGTGTAGATTCAAGTTTGTTCAAATCATGATCC\r\n", ">scaffold544:193731-194102(+)\r\n", "TTAGCTCACCTGAGCTGAAAGCTCAAGTGAGCTATTCTGATCACATTTTGTCTGTCATCCATCTGTCCATTTGTCCATCTGTCCATGTCTGTACGTCTGTCTGTAAACTTTTACATTTTCAACATCTTCTCTAGAACCACTGGGCCAACTTTAACCAAATTTAGCACAAAGCATCTCTAAGCAAAGGGCATTCAAAGTTGTGAAAATTAAGGACCACACTGTTTTACATGTAGAGATAATTAGGAGTTATTGAAAATTTAAAAAAAAAGCCAGAAATCTTCTTCTCCAGAACTGTTTGGCCAGGAAAGCTGAAACTTATGTGGAAGCATCCTCAGGTAATGTAGATTCAAAGTTGTGAAAATCATGAACCA\r\n", ">scaffold1715:409762-410131(+)\r\n", "TTAGCTCACCTGAGCTGAAAGCTCAAGTGAGCTTTTCTGATCAAAATTTGTCCGTTGTCTGTCGTCGGCGTTGGCGTTGTCATTGGCAGCGTTGTCGTTGTAAACTTTTCACATTTTCTTCTTCTTCTCCAGAACCACACTGCAGATTTCAACCAAATTTGGCACAAAGCATCACTTGGTGAAGGGGATTCAAGTTTGTTCAAATGAAGGGCAACGCCCTCTTTAAAGGGGAAATAATTGAGAATTATTGAAAATTTATTAGTATTTTTCAAAAATCTTCTTCTCAAAAACTATTTGGCCAGAAAAGCTTTAACCTGTGTGGAGGCATCCTCAGGTATTGTTGATTCAAGTTTGTTCAAATCATGGACC\r\n", ">scaffold1834:443990-444359(+)\r\n", "TTAGCTCACCTGAGCTGAAAGCTGAAAGCTCAAGTGAGGTTTTTTCTGATCAAAATTTGTCCGTTGTCTGTTGTCTTTGGCGTTGGCGTTGTCATTGTTATAAACTTTTCACATTTTTATCTACTTCTCCAGAACCACTGGGTCGATTTCAACCAAACTTGGCACAAAGCATCACTGGGTGAAAGGGATTCAAGATTGTTCAAATGAAGGGCCCACCCATGTTAAAGGGGAGATAATTTAAAATCATTAAATAATTTGTTGGTATTTTTCGAAAAACTTCTTCTCAAAAAGTATTGGGCCAAAAAAGCTTTAACTTGTGTTGAAGCATTCTCAGGTTGTATAGATTTAAGTTTGTTCAAATCATGATCC\r\n" ] } ], "prompt_number": 10 }, { "cell_type": "markdown", "metadata": {}, "source": [ "`!bsmap -w 1000 -a /Volumes/NGS\\ Drive/NGS\\ Raw\\ Data/Oyster_BSseq/filtered_Unlabeled_NoIndex_L003_R1.fastq.gz -d /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa -o /Volumes/web/cnidarian/BiGill_BSMAP_TEonly_v1.sam -p 1` \n", "\n", "**aborted**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Running BSMAP with `-w 1000` on genome\n", "\n", "Previous analysis was as follows\n", "\n", "./bsmap -a //Volumes/NGS\\ Drive/NGS\\ Raw\\ Data/Oyster_BSseq/filtered_Unlabeled_NoIndex_L003_R1.fastq.gz -d /Volumes/web/cnidarian/oyster.v9.fa -o /Volumes/web/cnidarian/BiGill_BSMAP_GillMBD_genome_v9_v1.sam -p 16\n", "\n", "Total number of aligned reads: 120734940 (86%)\n", "Done.\n", "Finished at Wed Apr 17 09:09:59 2013\n", "Total time consumed: 8235 secs\n", "\n", "\n", "http://eagle.fish.washington.edu/cnidarian/BiGill_BSMAP_GiIlMBD_genome_v9_v1.sam\n", "\n", "redux.." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`!bsmap -w 1000 -a /Volumes/NGS\\ Drive/NGS\\ Raw\\ Data/Oyster_BSseq/filtered_Unlabeled_NoIndex_L003_R1.fastq.gz -d /Volumes/web/cnidarian/cnidarian/oyster.v9.fa -o /Volumes/web/cnidarian/BiGill_BSMAP_genome_v9_v1000.sam -p 2` \n", "\n", "**aborted**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "###Running on Hummingbird\n", "\n", "`d-128-95-149-219:bsmap-2.74 sr320$ ./bsmap -w 1000 -a /Volumes/NGS\\ Drive/NGS\\ Raw\\ Data/Oyster_BSseq/filtered_Unlabeled_NoIndex_L003_R1.fastq.gz -d /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa -o /Volumes/web/cnidarian/BiGill_BSMAP_TEonly_v2.sam -p 16 `\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", " BSMAP v2.74\n", " Start at: Fri Jun 14 13:19:06 2013\n", "\n", " Input reference file: /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa \t(format: FASTA)\n", " Load in 119787 db seqs, total size 39414569 bp. 2 secs passed\n", " total_kmers: 43046721\n", " Create seed table. 5 secs passed\n", " max number of mismatches: read_length * 8% \tmax gap size: 0\n", " kmer cut-off ratio: 5e-07\n", " max multi-hits: 1000\tmax Ns: 5\tseed size: 16\tindex interval: 4\n", " quality cutoff: 0\tbase quality char: '!'\n", " min fragment size:28\tmax fragemt size:500\n", " start from read #1\tend at read #4294967295\n", " additional alignment: T in reads => C in reference\n", " mapping strand: ++,-+\n", " Single-end alignment(16 threads)\n", " Input read file: /Volumes/NGS Drive/NGS Raw Data/Oyster_BSseq/filtered_Unlabeled_NoIndex_L003_R1.fastq.gz \t(format: gzipped FASTQ)\n", " Output file: /Volumes/web/cnidarian/BiGill_BSMAP_TEonly_v2.sam\t (format: SAM)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "BSMAP complete on Hummingbird \n", "`\n", "Total number of aligned reads: 2545683 (1.8%)\n", "Done.\n", "Finished at Sat Jun 15 20:45:07 2013\n", "Total time consumed: 113161 secs\n", "`" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "python methratio.py -d /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa -z -g -o /Volumes/web/cnidarian/BiGill_methratio_TEonly_A.txt -s /Volumes/Bay3/Software/BSMAP/bsmap-2.73/samtools /Volumes/web/cnidarian/BiGill_BSMAP_TEonly_v2.sam" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`\n", "@ Mon Jun 17 08:25:48 2013: reading reference /Volumes/web/cnidarian/TJGR_RepProMask_TE.fa ...\n", "@ Mon Jun 17 08:25:50 2013: reading /Volumes/web/cnidarian/BiGill_BSMAP_TEonly_v2.sam ...\n", "[samopen] SAM header is present: 119787 sequences.\n", "@ Mon Jun 17 08:26:57 2013: combining CpG methylation from both strands ...\n", "@ Mon Jun 17 08:26:57 2013: writing /Volumes/web/cnidarian/BiGill_methratio_TEonly_A.txt ...\n", "@ Mon Jun 17 08:26:58 2013: done.\n", "total 1486700 valid mappings, 3267 covered cytosines, average coverage: 536.48 fold.\n", "`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**OUTPUT** \n", "\n", "\n", "" ] }, { "cell_type": "code", "collapsed": false, "input": [], "language": "python", "metadata": {}, "outputs": [] } ], "metadata": {} } ] }