{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
An Introduction to Solving Biological Problems with Perl
\n",
"Graham Ritchie, John Davey & Feyruz Yalcin\n",
"Cambridge Graduate School of Life Sciences Bioinformatics Training\n",
"
\n",
" from Sofia Robb's original course: `http://sofiarobb.com/learning-perl-toc/`",
"
This page can be found at bit.ly/cambridgeperl. Solutions to all the exercises can be found on the course GitHub repository"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 1: Perl script skeleton"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl is widely used by biologists to solve problems we encounter when handling DNA and protein sequences, file format conversions, as well as many any other tasks that are computationally repetitive. Perl is not the only language for these tasks but one of many. \n",
"\n",
"Perl comes preinstalled on Macs and Linux distributions. It can be downloaded for PCs (see [perl.org](http://www.perl.org/get.html#win32) for details; [Strawberry Perl](http://strawberryperl.com) is a good choice). \n",
"\n",
"Perl is designed to work like a natural language. Just as there are many different ways to say roughly the same thing in a natural language, so there are many different ways to solve a problem using Perl. Perl's motto is [\"There's More Than One Way To Do It\"](http://en.wikipedia.org/wiki/There's_more_than_one_way_to_do_it). This has advantages and disadvantages. On the one hand, Perl can be easily adopted by many different communities and applied to many different problems; it is heavily used by organisations like the BBC and Amazon and was instrumental in the completion of the [Human Genome Project](http://www.bioperl.org/wiki/How_Perl_saved_human_genome). On the other hand, it can be horribly abused, just like [natural languages](http://icanhas.cheezburger.com/lolcats).\n",
"\n",
"In this course, we will teach the basic vocabulary and grammar of the Perl language, using some simple bioinformatics examples. However, as bioinformatics covers a multitude of different topics, we recommend that you consider the problems you are trying to solve in your research and try to apply Perl to these problems throughout the course.\n",
"\n",
"As some people on the course have not programmed before, we assume no previous experience throughout. The pace will therefore be slow for people who have used other programming languages. Please feel free to read ahead."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl programs (also known as scripts) are series of instructions that we save in plain text files. As computers do not understand Perl directly, our scripts must be processed by a program called the Perl interpreter, which translates the scripts into machine language. This program is run from the command line. To get started with Perl, we must get acquainted with the command line.\n",
"\n",
"1. Open the text editor gedit (icon in the menu on the left side of the screen) and save a file called \"`central_dogma.pl`\" containing the following line:\n",
" - `print \"Once information has passed into protein it cannot get out again\\n\";`\n",
"\n",
" Note the directory that you save the file into. Now open up the Terminal (also in the left hand menu). We want to move to the directory containing the file you just saved, and run the Perl interpreter on that file.\n",
"\n",
"\n",
"2. Enter the command `pwd` (on Linux or Mac) or `cd` (on Windows) to see which directory you are in. If it's not the directory that contains your script, use the command `ls` (on Linux or Mac) or `dir` (on Windows) to see what the directory contains. If you see your directory in the list - let's say it's called `mydir` - you can move into that directory by entering the command `cd mydir`. Check that `mydir` contains your `central_dogma.pl` script using `ls` or `dir`.\n",
"\n",
"3. Run the Perl interpreter by typing:\n",
" - `perl central_dogma.pl`\n",
"\n",
"The script you just ran contains one *command*, which `print`s the *string* \"Once information has passed into protein it cannot get out again\" to the screen, and moves the cursor to a new line (`\\n`). The semi-colon is used to punctuate the end of a command. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can work through the rest of the course using the interpreter in this way. However, you may find it useful to add the following features to your script. If you find them confusing, however, feel free to skip to Lesson 2 for now.\n",
"\n",
"1. If there is an error in your code that the interpreter can't make sense of, it will fail to run. But it can also warn you about problems in the code that it can interpret, but which are probably not going to work correctly. You can turn on these warnings by putting this at the top of your script:\n",
" - `use warnings;`\n",
"\n",
" You can also ask the Perl interpreter to turn warnings on for you:\n",
"\n",
" - `perl -w central_dogma.pl`\n",
"\n",
" `use warnings` and `-w` do the same thing, so you only need to use one of them.\n",
"2. It's useful to annotate your scripts with human-readable comments. Perl ignores anything on a line that follows a hash symbol. So we could have written:\n",
" - `# central_dogma.pl`\n",
" - `# Print the Central Dogma of Molecular Biology`\n",
" - `print \"Once information has passed into protein it cannot get out again\\n\" # See Crick 1970`\n",
"\n",
" In this case, the comments are not very useful, but when we get to writing more complicated code it is very useful to make notes in comments, to remind ourselves how the code works when we come to read it months or years later.\n",
"\n",
"3. Many Perl guides will tell you to add `use strict;` to every script you write (like `use warnings;` above). This tells the Perl interpreter to complain if you use some of the more dangerous Perl idioms in your code. It's not too important for the course, as we won't be using any of those idioms, but it's not a bad idea to use it in scripts you write and actually use."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Putting it all together, the complete script:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# central_dogma.pl\n",
"# Print the Central Dogma of Molecular Biology\n",
"use strict;\n",
"use warnings;\n",
"print \"Once information has passed into protein it cannot get out again\\n\"; # See Crick 1970"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Once information has passed into protein it cannot get out again\n"
]
}
],
"prompt_number": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 2: Values and variables"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Typically, we want to process some data using Perl and produce some output. This means we have a set of values and we want to transform these into some other, more useful information. In Perl, we create data structures to hold our values, which we can then manipulate. We call these variables, because we can change their values as we progress through our analysis.\n",
"\n",
"There are three major types of data structures we typically use in Perl: scalars, arrays and hashes. Scalars are single values, such as numbers or characters. Strings of characters also count as single values in Perl. Arrays are ordered lists of values, whereas hashes are unordered collections of values. Variable names of each type begin with a symbol that identifies the variable as a scalar, array or hash. The symbols are given in the table below.\n",
"\n",
"\n",
"Scalar | $ | single values |
\n",
"Array | @ | ordered lists of scalars |
\n",
"Hash | % | unordered collections of scalars |
\n",
"
\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Values can be assigned to variables using the assignment operator `=`. For example (all code needs to have the complete script skeleton from Lesson 1 but we will leave it out of the notes in further examples):"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# use 'my' the FIRST time you use a variable\n",
"my $serine = \"TCA\";\n",
"my $codonLength = 3;\n",
"print \"Codon $serine has $codonLength bases\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Codon TCA has 3 bases\n"
]
}
],
"prompt_number": 2
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can change the value of a variable by using the same assignment operator:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $serine = \"TCA\";\n",
"print \"Serine is coded by $serine\\n\";\n",
"$serine = \"TCC\";\n",
"print \"Serine is also coded by $serine\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Serine is coded by TCA\n",
"Serine is also coded by TCC\n"
]
}
],
"prompt_number": 3
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can even use the value of a variable when we change its value, for example if we wanted to count something:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $age = 18;\n",
"print \"I am $age years old\\n\";\n",
"\n",
"# two years later\n",
"$age = $age + 2;\n",
"print \"I will be $age years old two years later\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"I am 18 years old\n",
"I will be 20 years old two years later\n"
]
}
],
"prompt_number": 4
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another example of scalar reassignment:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $dogma = \"Once information gets into protein\";\n",
"print \"1: $dogma\\n\";\n",
" \n",
"# '.' text concatenation operator\n",
"$dogma = $dogma . \" it cannot get out again\";\n",
"print \"2: $dogma\\n\";\n",
" \n",
"$dogma = \"DNA makes RNA makes Protein\";\n",
"print \"3: $dogma\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"1: Once information gets into protein\n",
"2: Once information gets into protein it cannot get out again\n",
"3: DNA makes RNA makes Protein\n"
]
}
],
"prompt_number": 5
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Scalars\n",
"\n",
"\n",
"type of scalar | examples |
\n",
"number: integers | 1 2 -3 128 |
\n",
"number: floating-points | -0.2 0.1 3.14159 |
\n",
"string: letters | \"a\" \"A\" \"G\" \"c\" |
\n",
"string: words | \"Protein\" \"DNA\" |
\n",
"string: words and spaces and punctuation | \"DNA makes RNA makes Protein.\"\n",
" |
\n",
"reference | (see later lessons) |
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Single and Double Quotes\n",
"\n",
"If you put a variable such as `\u201c$codon\u201d` inside of double quotes in a `print` statement, the value of `$codon` will be printed, but if you put `\u2018$codon\u2019` inside of single quotes you will literally get `$codon`. The same is true for special characters like the newline `\u201c\\n\u201d`."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $codon = \"ACG\";\n",
"print \"$codon\\n\";\n",
"print '$codon\\n';"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ACG\n",
"$codon\\n"
]
}
],
"prompt_number": 6
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Look up the genetic code. Create a number of scalar variables that store possible DNA encodings of serine (S), leucine (L), tyrosine (Y) and cysteine (C). Where multiple codings are available, just pick one for now.\n",
"2. Create a variable containing a possible DNA sequence for the protein sequence `SYLYC`. Use the concatenation operator."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 3: Arrays"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"An array is an ordered list of scalars. The scalars do not all need to be of the same kind."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my @numbers = (22,103,1,0);\n",
"my @aminoAcids = (\"Serine\",\"Tyrosine\",\"Leucine\",\"Tyrosine\",\"Cysteine\");\n",
"my $serine = \"TCA\";\n",
"my @mixed = (23,$serine,0.3,\"Histone\");"
],
"language": "python",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"prompt_number": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are a number of different ways to print arrays. For example, we can dump the elements of an array one after the other with no delimiters just by directly printing the array:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my @aminoAcids = (\"Serine\",\"Tyrosine\",\"Leucine\",\"Tyrosine\",\"Cysteine\");\n",
"\n",
"print \"Amino acids: \" , @aminoAcids , \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Amino acids: SerineTyrosineLeucineTyrosineCysteine\n"
]
}
],
"prompt_number": 8
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"But this isn't very readable. We can do better by enclosing the array variable in double quotes, which inserts a space after each element in the array:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my @aminoAcids = (\"Serine\",\"Tyrosine\",\"Leucine\",\"Tyrosine\",\"Cysteine\");\n",
"print \"Amino acids, spaced: @aminoAcids\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Amino acids, spaced: Serine Tyrosine Leucine Tyrosine Cysteine\n"
]
}
],
"prompt_number": 9
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will see a third way later on when we look at the `join` function.\n",
"\n",
"We can access individual elements of an array by using their index. Each element is given a unique number, an index, corresponding to their place in the array. In Perl, counting starts from 0, so the first element of the array has index 0, the second has index 1 and so on. We access elements using square brackets:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"my @aminoAcids = (\"Serine\",\"Tyrosine\",\"Leucine\",\"Tyrosine\",\"Cysteine\");\n",
"print $aminoAcids[2], \"\\n\";\n",
"print $aminoAcids[0], \"\\n\";\n",
"print $aminoAcids[-1], \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Leucine\n",
"Serine\n",
"Cysteine\n"
]
}
],
"prompt_number": 10
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can assign new values to array elements by using their indices:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"my @aminoAcids = (\"Serine\",\"Tyrosine\",\"Leucine\",\"Tyrosine\",\"Cysteine\");\n",
"$aminoAcids[2] = \"Cysteine\";\n",
"print \"@aminoAcids\\n\";\n",
"$aminoAcids[5] = \"STOP\";\n",
"print \"@aminoAcids\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Serine Tyrosine Cysteine Tyrosine Cysteine\n",
"Serine Tyrosine Cysteine Tyrosine Cysteine STOP\n"
]
}
],
"prompt_number": 11
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Arrays can be included in other arrays, but the result is just one flat array. For example, if two arrays of three elements each are merged together, the result is an array containing the six elements, not an array containing two arrays."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"\n",
"my @serine = ('T','C','A');\n",
"print \"@serine\\n\";\n",
"\n",
"my @cysteine = ('T', 'G', 'T');\n",
"print \"@cysteine\\n\";\n",
"\n",
"my @sc = (@serine, @cysteine);\n",
"print \"@sc\\n\";\n",
"\n",
"print \"$sc[4]\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"T C A\n",
"T G T\n",
"T C A T G T\n",
"G\n"
]
}
],
"prompt_number": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can get the length of an array like this:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"my @serine = ('T','C','A');\n",
"my $length = @serine;\n",
"print \"$length\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"3"
]
}
],
"prompt_number": 13
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is called accessing an array in scalar context. `$length` is a scalar, so when we set the array `@serine` as the value of the scalar `$length`, Perl interprets this to mean we want the length of the array. This is just a convention - Perl could, for example, return the first element of the array instead - but it is a useful convention that we will use often."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Create an array for the protein sequence `CLYSY` where the elements of the array are DNA codon sequences. Use the codon variables you defined in the last lesson.\n",
"2. Print the DNA sequence of the protein to the screen, with and without spaces between each codon.\n",
"3. Print the DNA sequence of the last amino acid in the protein by accessing this element from the array.\n",
"4. Create an additional stop codon variable and add the DNA for stop to your DNA sequence."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 4: Hashes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Arrays are very useful but not sufficient for many purposes. Often, we have a data set with no inherent order but where each entry has a particular value. For example, we may have a set of genes with expression levels, where we want to look up expression levels by gene name. There is no biological order to the gene names and we want to rapidly access particular genes without trawling through the whole data collection looking for it. Perl has a very efficient data structure for dealing with this situation - the hash."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"# hash_assignment.pl\n",
" \n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 14
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hashes have keys and values, and work like a dictionary - we look up the keys to find the matching values. The keys are like indices of an array, but they can be any scalar, not just a number, as long as they are unique in the hash. In the hash above, the DNA codons are the keys (`TAT`, `TAC` and so on) and the amino acids names are the values. As with arrays, the values can be any type and do not need to all be the same type, as long as they are scalars.\n",
"\n",
"We can access values from a hash by looking up their keys. We use curly brackets to do this, rather than the square brackets we used for array indices. As for array elements, a hash value is a scalar, not a hash, so we say `$geneticCode{'TAT'}`, not `%geneticCode{'TAT'}`."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"# hash_assignment.pl\n",
" \n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );\n",
"\n",
"print $geneticCode{'TAT'};\n",
"print \"\\n\";\n",
"print \"$geneticCode{'CAG'}\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Tyrosine\n",
"Glutamine\n"
]
}
],
"prompt_number": 15
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can extract the keys or values from a hash using the `keys` and `values` functions. These will return an unsorted array of the keys or values of the hash, which we can use like any other array (we will see how to sort these arrays later in the course)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"# hash_assignment.pl\n",
" \n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );\n",
"\n",
"my @codons = keys %geneticCode;\n",
"my @aas = values %geneticCode;\n",
"print \"@codons\\n\";\n",
"print \"@aas\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"CAG CAA TAC TAT\n",
"Glutamine Glutamine Tyrosine Tyrosine\n"
]
}
],
"prompt_number": 16
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can add or update key-value pairs to the hash just as we could add new array elements, or delete them with the `delete` function. If we try to look up a non-existent key, Perl will complain, as long as we are using warnings (`-w`)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"# hash_assignment.pl\n",
" \n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );\n",
"\n",
"$geneticCode{'TGA'} = 'Cysteine';\n",
"print \"$geneticCode{'TGA'}\\n\";\n",
"$geneticCode{'TGA'} = 'STOP'; # Whoops! Got the assignment wrong the first time\n",
"print \"$geneticCode{'TGA'}\\n\";\n",
"delete $geneticCode{'TGA'};\n",
"print \"$geneticCode{'TGA'}\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Cysteine\n",
"STOP\n",
"\n"
]
},
{
"output_type": "stream",
"stream": "stderr",
"text": [
"Use of uninitialized value $geneticCode{\"TGA\"} in concatenation (.) or string at - line 14.\n"
]
}
],
"prompt_number": 17
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercise\n",
"1. Print out the names of the amino acids that would be produced by the DNA sequence `\"GTT GCA CCA CAA CCG GTT GTG\"`. Use a hash to do this, similar to the `%geneticCode` hash above.\n",
"2. Why couldn't we build a hash where the keys are names of amino acids and the values are the DNA codons?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 5: Loops"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the exercise for Lesson 4, perhaps you ended up writing something like this:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"# hash_assignment.pl\n",
" \n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );\n",
"\n",
"my @sequence = ('CAA','CAG','TAC');\n",
"print $geneticCode{$sequence[0]}, \"\\n\";\n",
"print $geneticCode{$sequence[1]}, \"\\n\";\n",
"print $geneticCode{$sequence[2]}, \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Glutamine\n",
"Glutamine\n",
"Tyrosine\n"
]
}
],
"prompt_number": 18
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Repeating the last line of code several times is not ideal. We have to type, or copy and paste, the same thing multiple times and if we wanted to change what we were doing with each amino acid name, we would have to change every line (for example, we forgot to print a new line or a space after each name). It would be much easier to just write the line once and then execute that line for every item in the array. We can do this with a for loop:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" );\n",
"\n",
"my @sequence = ('CAA','CAG','TAC');\n",
"\n",
"foreach my $codon (@sequence) {\n",
" print \"$geneticCode{$codon}\\n\";\n",
"}\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Glutamine\n",
"Glutamine\n",
"Tyrosine\n"
]
}
],
"prompt_number": 19
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Because '`keys %hash`' is an array, we can also loop over the entries in a hash in the same way:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneExpression = ('Beta-Catenin' => 2.5,\n",
" 'Beta-Actin' => 1.7,\n",
" 'Pax6' => 0,\n",
" 'HoxA2' => -3.2,\n",
" );\n",
"\n",
"foreach my $gene (keys %geneExpression) {\n",
" print \"$gene: $geneExpression{$gene}\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Beta-Catenin: 2.5\n",
"HoxA2: -3.2\n",
"Beta-Actin: 1.7\n",
"Pax6: 0\n"
]
}
],
"prompt_number": 20
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl has another type of loop which can also be used to iterate over a hash, among other things - the while loop:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneExpression = ('Beta-Catenin' => 2.5,\n",
" 'Beta-Actin' => 1.7,\n",
" 'Pax6' => 0,\n",
" 'HoxA2' => -3.2,\n",
" );\n",
"\n",
"while (my $gene = each %geneExpression) {\n",
" print \"$gene: $geneExpression{$gene}\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Beta-Catenin: 2.5\n",
"HoxA2: -3.2\n",
"Beta-Actin: 1.7\n",
"Pax6: 0\n"
]
}
],
"prompt_number": 21
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Like the for loop, the while loop executes the block of code within curly brackets. The difference is that, whereas the for loop takes an array and does something with each element of the array, the while loop takes a logical test and runs until that test is false. So here it runs until each of the genes in `%geneExpression` have been processed. Let's say we want to write out the first five codons of a gene's DNA sequence:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"my @sequence = ('CAA','CAG','TAG','CAT','GGT','GAG','GGC','CAG');\n",
"\n",
"my $i = 0;\n",
"while ($i<5) {\n",
" print \"$sequence[$i] \";\n",
" $i = $i + 1;\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"CAA CAG TAG CAT GGT "
]
}
],
"prompt_number": 22
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Create an array where each element is an individual base of DNA. Make the array 15 bases long.\n",
"2. Create a for loop to output every base of the sequence on a new line.\n",
"3. Create a while loop similar to the one above that starts at the third base in the sequence and outputs every third base until the 12th."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 6: If statements and loop control"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the while loop above, we saw an example of a logical test: `($i<5)`. While this statement was true, the while loop continued; when it became false, the while loop ended. Another powerful control structure that uses logical tests is `if..elsif..else`."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneExpression = ('Beta-Catenin' => 2.5,\n",
" 'Beta-Actin' => 1.7,\n",
" 'Pax6' => 0,\n",
" 'HoxA2' => -3.2,\n",
" );\n",
"\n",
"foreach my $gene (keys %geneExpression) {\n",
" if ($geneExpression{$gene} < 0) {\n",
" print \"$gene is downregulated\\n\";\n",
" }\n",
" elsif ($geneExpression{$gene} > 0) {\n",
" print \"$gene is upregulated\\n\";\n",
" }\n",
" else {\n",
" print \"No change in expression of $gene\\n\";\n",
" }\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Beta-Catenin is upregulated\n",
"HoxA2 is downregulated\n",
"Beta-Actin is upregulated\n",
"No change in expression of Pax6\n"
]
}
],
"prompt_number": 23
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `elsif` block is optional, and we can use as many as we like. The `else` block is also optional, but it is good practice to include it in most cases, so that you think to do something explicitly with whatever doesn't pass the test in the `if` block. Otherwise, some cases might not pass the condition, and Perl would pass over them silently, which may not be what you want. There are many different comparison operators we can use in our logical tests. \n",
"\n",
"Strings and numbers have different operators.\n",
"\n",
"\n",
"Operator | Description | Example |
\n",
"== | numeric equality | 1 == 2 # False |
\n",
"eq | string equality | 'a' eq 'b' # False |
\n",
"!= | numeric non equality | 1 != 2 # True |
\n",
"ne | string non equality | 'a' ne 'b' # True |
\n",
"< | numeric less than | 1 < 2 # True |
\n",
"<= | numeric equal or less than | 2 <= 2 # True |
\n",
"lt | string less than | 'a' lt 'b' # True by ASCII order |
\n",
"> | numeric greater then | 1 > 2 # False |
\n",
">= | numeric equal or greater than | 1 >= 1 # True |
\n",
"gt | string greater than | 'a' gt 'b' # False |
\n",
"
\n",
"\n",
"For example, we might want to output only the nonsynonymous bases of a codon (the first and second bases) and ignore the third, synonymous base (because most amino acids are determined by the first and second bases alone)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @sequence = ('T','A','C','G','G','C','A','T','C','T','A','G');\n",
"\n",
"my $i = 0;\n",
"foreach my $base (@sequence) {\n",
" $i = $i + 1;\n",
" if ($i == 3) {\n",
" $i = 0;\n",
" }\n",
" else {\n",
" print \"$base\";\n",
" }\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"TAGGATTA"
]
}
],
"prompt_number": 24
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl has a useful keyword that can simplify loops like this: `next`, which moves on to the next iteration of the loop if a condition is true:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @sequence = ('T','A','C','G','G','C','A','T','C','T','A','G');\n",
"\n",
"my $i = 0;\n",
"foreach my $base (@sequence) {\n",
" $i = $i + 1;\n",
" if ($i == 3) {\n",
" $i = 0;\n",
" next;\n",
" }\n",
" print \"$base\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"TAGGATTA"
]
}
],
"prompt_number": 25
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we have omitted the else statement, against our advice above, and in this simple example the first version was probably better. But imagine if we had complex processing of the base to do, with perhaps ten or twenty lines of code following the if statement. If we used an else, we would have to enclose all of these lines in the else block, which would have made our code less readable than just placing them after the if statement.\n",
"\n",
"Perl also has a `last` keyword, which exits a loop if a condition is met:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" 'TAG' => 'STOP',\n",
" );\n",
"\n",
"my @sequence = ('CAG','TAC','CAA','TAG','TAC','CAG','CAA');\n",
"\n",
"foreach my $codon (@sequence) {\n",
" if ($geneticCode{$codon} eq 'STOP') {\n",
" last;\n",
" }\n",
" else {\n",
" print \"$geneticCode{$codon}\\n\";\n",
" }\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Glutamine\n",
"Tyrosine\n",
"Glutamine\n"
]
}
],
"prompt_number": 26
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here the amino acids in the sequence are printed until the `STOP` codon (`TAG`) is reached. We used the `else` here, but actually Perl provides a more readable syntax for if statements when there is only one command in the block (as here, the `last` command). We can just put the command before the `if` statement, and forget about the blocks:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my %geneticCode = ('TAT' => 'Tyrosine',\n",
" 'TAC' => 'Tyrosine',\n",
" 'CAA' => 'Glutamine',\n",
" 'CAG' => 'Glutamine',\n",
" 'TAG' => 'STOP',\n",
" );\n",
"\n",
"my @sequence = ('CAG','TAC','CAA','TAG','TAC','CAG','CAA');\n",
"\n",
"foreach my $codon (@sequence) {\n",
" last if ($geneticCode{$codon} eq 'STOP');\n",
" print \"$geneticCode{$codon}\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Glutamine\n",
"Tyrosine\n",
"Glutamine\n"
]
}
],
"prompt_number": 27
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally for this lesson, we can chain multiple logical tests with `and` and `or` operators. We can either use the words `and` and `or`, or the symbols `&&` and `||`.\n",
"\n",
"\n",
"Operator (symbolic) | Operator (written) | Example | To be true |
\n",
"&& | and | i < 50 && i > 5 | Both conditions must be true |
\n",
"|| | or | str eq \"clock\" || str ne \"black\" | Only one condition needs to be true |
\n",
"
\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneExpression = ('Beta-Catenin' => 2.5,\n",
" 'Beta-Actin' => 1.7,\n",
" 'Pax6' => 0,\n",
" 'HoxA2' => -3.2,\n",
" );\n",
"\n",
"foreach my $gene (keys %geneExpression) {\n",
" if ($geneExpression{$gene} < 0 or $geneExpression{$gene} > 0) {\n",
" print \"$gene has changed expression\\n\";\n",
" }\n",
" else {\n",
" print \"$gene has not changed expression\\n\";\n",
" }\n",
"\n",
" if ($geneExpression{$gene} > 0 and $gene ne 'Beta-Actin') {\n",
" print \"$gene is upregulated and not a housekeeping gene\\n\";\n",
" }\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Beta-Catenin has changed expression\n",
"Beta-Catenin is upregulated and not a housekeeping gene\n",
"HoxA2 has changed expression\n",
"Beta-Actin has changed expression\n",
"Pax6 has not changed expression\n"
]
}
],
"prompt_number": 28
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"1. Let's calculate GC content of a DNA sequence. Use the 15-base array you created for the exercises in the lesson on loops. Create a variable, `$gc`, which we will use to count the number of Gs or Cs in our sequence.\n",
"2. Create a loop to iterate over the bases in your sequence. If the base is a G or the base is a C, add one to your `$gc` variable.\n",
"3. When the loop is done, divide the number of GC bases by the length of the sequence and multiply by 100 to get the GC percentage. Recall the array lesson to see how to get the length of the sequence. Just as we have been using '`+`' to add in Perl, to multiply, we use '`*`' and to divide, we use '`/`'. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 7: Introduction to functions: basic numeric operators"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the next few lessons, we will introduce many of Perl's basic functions and operators for processing numbers, strings, arrays and hashes. We will see how to load functions from Perl modules and finally we will show how you can construct your own functions.\n",
"\n",
"A function is something that takes some input and produces some output. We have already seen several Perl functions: `print`, `keys`, `values` and `each` are all examples of functions. For example, `keys` takes a hash as input and returns the keys of the hash as output.\n",
"\n",
"Perl has many core operators and functions built into the language. For example, the following operators all take two numbers as input and produce one number as output:\n",
"\n",
"\n",
"Operator | Summary | Example |
\n",
"+ | addition | 2 + 2 |
\n",
"- | subtraction | 5 - 3 |
\n",
"* | multiplication | 4 * 4 |
\n",
"/ | division | 8 / 2 |
\n",
"% | modulus, returns remainder | 10 % 2 |
\n",
"** | exponent | 4**2 |
\n",
"
"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# addition\n",
"print \"sum of 2 and 3 = \";\n",
"my $num = 2 + 3;\n",
"print \"$num\\n\";\n",
"\n",
"# subtraction\n",
"print \"$num minus 2 = \";\n",
"$num = $num - 2;\n",
"print \"$num\\n\";\n",
"\n",
"# multiplication\n",
"print \"product of $num and 3 = \";\n",
"$num = $num * 3;\n",
"print \"$num\\n\";\n",
"\n",
"# division\n",
"print \"$num divided by 2 = \";\n",
"$num = $num / 2;\n",
"print \"$num\\n\";\n",
"\n",
"# modulus\n",
"print \"the remainder of $num divided by 2 = \";\n",
"my $remainder = $num % 2;\n",
"print \"$remainder\\n\";\n",
"\n",
"# exponentiation\n",
"print \"$remainder raised to the power 2 = \";\n",
"$num = $remainder ** 2;\n",
"print \"$num\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"sum of 2 and 3 = 5\n",
"5 minus 2 = 3\n",
"product of 3 and 3 = 9\n",
"9 divided by 2 = 4.5\n",
"the remainder of 4.5 divided by 2 = 0\n",
"0 raised to the power 2 = 0\n"
]
}
],
"prompt_number": 29
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl has a couple of handy shortcuts for this kind of operation. Any of the above operators can be placed before the equals sign, so that instead of writing `$num = $num + 2` we can just write `$num += 2`. Also, to add or subtract one to or from a variable, we can use `++` or `--`:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $num = 1;\n",
"\n",
"print \"Number was $num,\";\n",
"$num *= 5;\n",
"$num += 3;\n",
"\n",
"print \" now $num\\n\";\n",
"\n",
"# increment in place\n",
"print \"$num + 1 = \";\n",
"$num++;\n",
"print \"$num\\n\";\n",
"\n",
"# decrement in place\n",
"print \"$num - 1 = \";\n",
"$num--;\n",
"print \"$num\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Number was 1, now 8\n",
"8 + 1 = 9\n",
"9 - 1 = 8\n"
]
}
],
"prompt_number": 30
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"++ | increment by 1 (same as $i = $i+1) | $i++ |
\n",
"-- | decrement by 1, (same as $i = $i-1) | $i-- |
\n",
"+= | in place addition | $i += 2 |
\n",
"-= | in place subtraction | $i -= 3 |
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, here are a couple of useful functions for calculating the absolute value of some number, and the integer portion of a floating-point number:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"Function | Summary | Example |
\n",
"abs | absolute value | abs(-32) |
\n",
"int | get the integer portion of a number | int(5.45) |
\n",
"
"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# absolute value\n",
"print \"the absolute value of 3 - 4.2 = \";\n",
"my $num = abs 3 - 4.2;\n",
"print \"$num\\n\";\n",
"\n",
"# conversion to integer\n",
"print \"int portion of $num = \";\n",
"print int $num, \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"the absolute value of 3 - 4.2 = 1.2\n",
"int portion of 1.2 = 1\n"
]
}
],
"prompt_number": 31
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A word about brackets here. If you are used to other languages, you may be surprised that we don't need brackets around function arguments. Perl is generally fairly lax about requiring brackets, in the hope that code can be read something like a natural language. However, it doesn't mind if you put the brackets in, so if you find it easier to read, you can write things like this:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# absolute value\n",
"print \"the absolute value of 3 - 4.2 = \";\n",
"my $num = abs(3 - 4.2);\n",
"print \"$num\\n\";\n",
"\n",
"# conversion to integer\n",
"print \"int portion of $num = \";\n",
"print(int($num), \"\\n\");"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"the absolute value of 3 - 4.2 = 1.2\n",
"int portion of 1.2 = 1\n"
]
}
],
"prompt_number": 32
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The rule of thumb is to do what you find most readable and clear. For example, the brackets around the `abs` are probably a good idea; it saves us from thinking Perl will calculate `abs 3` and then subtract `4.2` from that. But the brackets on the `int` line arguably clutter up the line rather than making it clearer. Sometimes the brackets will be necessary, however, to tell Perl which arguments belong to which function. As `int` only takes one argument, there is no ambiguity here, but we will shortly see some examples where the brackets are required. This is something you will figure out through experience."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"1. Can you simplify your GC calculation from the previous lesson using one of the shorthands here?\n",
"2. Did you get an ugly floating point number for your GC percentage? Turn it into an integer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 8: String functions and operators"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl also has a set of useful string functions. For example, we can add strings together using the concatenation operator, which has a prefix notation like the numerical operators:\n",
"\n",
"\n",
"Operator | Summary | Example |
\n",
". | concatenation | $str . \" \" . \"!!\" |
\n",
".= | concatenation & assignment | $str .= \"add this\" |
\n",
"
"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $str = \"ACG\" . \"AGA\" . \"GCG\";\n",
"print \"$str\\n\";\n",
"$str .= \"TGT\";\n",
"print \"$str\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ACGAGAGCG\n",
"ACGAGAGCGTGT\n"
]
}
],
"prompt_number": 33
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here are some more basic functions, which are hopefully fairly self-explanatory:\n",
"\n",
"Function | Summary | Example |
\n",
"length | length of string | print length $str |
\n",
"uc | uppercase | print uc $str |
\n",
"lc | lowercase | print lc $str |
\n",
"ucfirst | uppercase first character | print ucfirst $str |
\n",
"lcfirst | lowercase first character | print lcfirst $str |
\n",
"chop | remove last character | chop $str |
\n",
"chomp | remove new line characters | chomp $str |
\n",
"
"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"# change case\n",
"my $str = \"ACGAGAGCGTGT\";\n",
"print length $str, \"\\n\";\n",
"print \"$str\\n\";\n",
"$str = lc $str;\n",
"print \"$str\\n\";\n",
"print uc $str, \"\\n\";\n",
"print \"ucfirst: \" , ucfirst(lc($str)),\"\\n\";\n",
"print \"lcfirst: \" , lcfirst uc $str,\"\\n\";\n",
"chop $str;\n",
"print \"Chopped: $str\\n\";\n",
"$str .= \"\\n\";\n",
"print \"Str now has a new line: $str$str$str\";\n",
"chomp $str;\n",
"print \"Now it doesn't: $str$str$str\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"12\n",
"ACGAGAGCGTGT\n",
"acgagagcgtgt\n",
"ACGAGAGCGTGT\n",
"ucfirst: Acgagagcgtgt\n",
"lcfirst: aCGAGAGCGTGT\n",
"Chopped: acgagagcgtg\n",
"Str now has a new line: acgagagcgtg\n",
"acgagagcgtg\n",
"acgagagcgtg\n",
"Now it doesn't: acgagagcgtgacgagagcgtgacgagagcgtg"
]
}
],
"prompt_number": 34
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`chomp` is often used when processing files or user input, because lines from files or lines entered by users will usually have a new line character on the end. More on this later.\n",
"\n",
"Here are some more sophisticated string processing functions.\n",
"\n",
"\n",
"substr | sub string | print substr($str, 3, 5) |
\n",
"split | splits up a string, returns an array | my @array = split ' ', $str |
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`split` takes a string and breaks up the string at every instance of a particular character, returning an array containing each part of the string:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $seq = \"ACG AGA GCG TGT\";\n",
"my @codons = split ' ', $seq;\n",
"foreach my $codon (@codons) {\n",
" print \"$codon\\n\";\n",
"}\n",
"foreach my $base (split '', $seq) {\n",
" next if $base eq \" \";\n",
" print \"$base\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ACG\n",
"AGA\n",
"GCG\n",
"TGT\n",
"A\n",
"C\n",
"G\n",
"A\n",
"G\n",
"A\n",
"G\n",
"C\n",
"G\n",
"T\n",
"G\n",
"T\n"
]
}
],
"prompt_number": 35
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`substr` takes a string, a starting position and (optionally) a length, and returns a substring of the string of the desired length that starts at the given position. As with array indexing, `substr` starts counting from 0. It has a few odd variants, which are worth exploring. If the length argument is omitted, `substr` returns everything from the start position to the end of the string. If a negative number is used for the start position, `substr` counts from the end of the string. If a negative number is used for the length, `substr` again subtracts that number of characters from the end of the string. Make sure you understand the following examples."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $str = \"ACGAGAGCGTGT\";\n",
"print substr($str, 3, 3),\"\\n\";\n",
"print substr($str, 3), \"\\n\";\n",
"print substr($str, -3), \"\\n\";\n",
"print substr($str, 3, -3), \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"AGA\n",
"AGAGCGTGT\n",
"TGT\n",
"AGAGCG\n"
]
}
],
"prompt_number": 36
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercise\n",
"\n",
"1. Write a routine that takes a DNA sequence and uses `substr` and a `while` loop to print out each codon in the sequence on a new line. Before the `while` loop starts, check that the length of the sequence is divisible by three and only run the `while` loop if it is."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 9: Array and Hash Functions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We have seen how to create a new array containing a set of elements, and one way to add new elements to an array (by setting a particular array element to a chosen value). We need a bit more flexibility, however. Here are some functions to help us manipulate arrays we have already created.\n",
" \n",
"\n",
"Function | Description |
\n",
"pop @array | removes last element from array and returns it |
\n",
"push @array, $new_element, ... | appends one or more elements to the end of an array |
\n",
"shift @array | removes the first element from the array and returns it |
\n",
"unshift @array, $new_element, ... | add elements at the front of the array |
\n",
"splice | adds or removes elements anywhere in the array |
\n",
"join($str, @array) | joins the elements of the array together with the specified string, and returns the whole string |
\n",
"
\n",
"\n",
"`pop`, `push`, `shift` and `unshift` all remove or add one element from or to an array. `pop` and `push` work from the end of the array; `shift` and `unshift` work from the beginning. `pop` and `shift` return the element they have removed."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @array = ('ACT','GCT','GAG','CAG');\n",
"print \"Array is @array\\n\";\n",
"my $last = pop @array;\n",
"print \"Popped the last element off the array: @array. Here's the last element: $last\\n\";\n",
"push @array, 'CGT';\n",
"print \"Added CGT to the end of the array: @array\\n\";\n",
"my $first = shift @array;\n",
"print \"Shifted the first element off the array: @array. Here's the first element: $first\\n\";\n",
"unshift @array, 'AAC';\n",
"print \"Added AAC to the beginning of the array: @array\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Array is ACT GCT GAG CAG\n",
"Popped the last element off the array: ACT GCT GAG. Here's the last element: CAG\n",
"Added CGT to the end of the array: ACT GCT GAG CGT\n",
"Shifted the first element off the array: GCT GAG CGT. Here's the first element: ACT\n",
"Added AAC to the beginning of the array: AAC GCT GAG CGT\n"
]
}
],
"prompt_number": 37
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With `splice`, we can remove chunks of the array and replace them with something else if we want to. Like `substr`, `splice` takes an array as input, a starting position and an optional length, plus an optional replacement array. It removes the elements from the start position up to the given length, and replaces them with the replacement array. It returns the array elements it has removed. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @array = ('ACT','GCT','GAG','CAG');\n",
"print \"Array starts as @array\\n\";\n",
"my @spliced = splice @array, 1, 2;\n",
"print \"Removed @spliced to leave @array\\n\";\n",
"my @replacement = ('CGG','CCC','AGT','GAC');\n",
"@spliced = splice @array, 1, 1, @replacement;\n",
"print \"Removed @spliced and replaced with @replacement to leave @array\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Array starts as ACT GCT GAG CAG\n",
"Removed GCT GAG to leave ACT CAG\n",
"Removed CAG and replaced with CGG CCC AGT GAC to leave ACT CGG CCC AGT GAC\n"
]
}
],
"prompt_number": 38
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, `join` does the opposite of `split`; it takes an array and joins the elements together into a string, using a separator if given:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my @bases = ('A','C','G','T','A','C','G','G','T','T','G','A');\n",
"my $seq = join '', @bases;\n",
"print \"@bases is now $seq\\n\";\n",
"$seq = join ':', @bases;\n",
"print \"We can use separators too: $seq\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"A C G T A C G G T T G A is now ACGTACGGTTGA\n",
"We can use separators too: A:C:G:T:A:C:G:G:T:T:G:A\n"
]
}
],
"prompt_number": 39
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perl also provides some convenience hash functions. We've already met three of these.\n",
" \n",
"\n",
"Function | Description |
\n",
"keys %hash | retrieve an array of all keys in a hash |
\n",
"values %hash | retrieve an array of all values in a hash |
\n",
"each %hash | retrieve next key-value pair from a hash |
\n",
"exists $hash{'key'} | test whether a key is found in the hash |
\n",
"delete $hash{'key'} | delete a key, and any corresponding value, from a hash |
\n",
"
\n",
"\n",
"Here is a simple example of using `exists` and `delete`. We wouldn't normally write such code, with the hash and array defined next to each other, but suppose that the hash has been loaded from one file and the array from another, and we don't know whether the array contains the same genes as the hash."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my %geneExpression = ('Beta-Catenin' => 2.5,\n",
" 'Beta-Actin' => 1.7,\n",
" 'Pax6' => 0,\n",
" 'HoxA2' => -3.2,\n",
" );\n",
"\n",
"my @genes = ('Pax6','EphB2','HoxA2');\n",
"\n",
"for my $gene (@genes) {\n",
" if (exists $geneExpression{$gene}) {\n",
" print \"Found $gene in hash, has expression $geneExpression{$gene}\\n\";\n",
" delete $geneExpression{$gene};\n",
" }\n",
" else {\n",
" print \"Can't find $gene\\n\";\n",
" }\n",
"}\n",
"\n",
"print \"Hash now contains the following genes: \", \n",
" join(' ', keys %geneExpression), \" with the following values: \", \n",
" join(' ', values %geneExpression), \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Found Pax6 in hash, has expression 0\n",
"Can't find EphB2\n",
"Found HoxA2 in hash, has expression -3.2\n",
"Hash now contains the following genes: Beta-Catenin Beta-Actin with the following values: 2.5 1.7\n"
]
}
],
"prompt_number": 40
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercise\n",
"\n",
"Let's put all of this together. You are given the following data:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my %geneticCode = ('TCT' => 'Serine',\n",
" 'TCC' => 'Serine',\n",
" 'CTA' => 'Leucine',\n",
" 'CTG' => 'Leucine',\n",
" 'TGT' => 'Cysteine',\n",
" 'TAG' => 'STOP',\n",
" );\n",
"\n",
"my $sequence = 'TCCTGTCTACCCCTGAAGTCTTAG';"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 41
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Write a routine that will convert the given DNA sequence into an amino acid sequence, using the first letter of the amino acid name for each amino acid (eg output `S` for Serine, `L` for Leucine and so on). Where no codon is present in the hash, output '`?`'. Use `'*'` for the stop codon.\n",
"\n",
"You have all the tools to figure out how to do this, but if you need some hints:\n",
"\n",
"1. You're going to need to iterate through the sequence and split it into codons. You can do this with a while loop or a for loop.\n",
"2. There are many ways to get the codons. For example, you can either shift bases off the sequence into a `codon` variable until the codon is three bases long, or you could split the sequence up into an array and use splice to chop off three bases at a time. You may be able to think of another solution.\n",
"3. You'll need an `if` statement to check whether the codon exists in the hash. If it does, you'll want to add the first character of the amino acid value in the hash to an amino acid sequence (which you'll have to set up before the loop begins). You can use `substr` to get the first character. If it doesn't, you want to add '`?`' to the amino acid sequence.\n",
"4. You will need another `if` statement to check whether the codon is a `STOP` codon or not. If it is, you want to add '`*`' to the amino acid sequence, not the first character of `STOP`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 10: Core functions, modules and subroutines"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We have now made use of several core Perl functions. There are many more that we haven't covered here. See Perl functions for a full list. Perl also has a large number of core modules, which are packages containing additional functions with common goals. These are all documented on Perldoc too.\n",
"\n",
"The core functions are always available, but to use a module we need to import it. For example, one commonly used module is `List::Util`. This contains some commonly used list (array) functions. Here's how we can use it:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"use List::Util ('max', 'min');\n",
"\n",
"my @genome_mb = (100.3, 2700, 4.6, 3200, 12.1);\n",
"\n",
"print max(@genome_mb), \"\\n\";\n",
"print min(@genome_mb), \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"3200\n",
"4.6\n"
]
}
],
"prompt_number": 42
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We import the functions `max` and `min` from `List::Util`, and then we can use these functions just like any of Perl's core functions. We can find out what functions are contained in `List::Util` by looking up the `List::Util` Perldoc page.\n",
"\n",
"We can also create our own functions, or subroutines. If there is a block of code you use more than once in your script, or there are some fiddly details that you would rather hide away from the important parts of your algorithm, you should write a subroutine. Using subroutines reduces the chance of introducing errors by copying and pasting code, and if you have to change something you only change it in one place, not in many.\n",
"\n",
"Just like a normal function, subroutines can take arguments as input and return values as output. Here's an example:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @genome_mb = (100.3, 2700);\n",
"\n",
"sub maximum {\n",
" my ($a, $b)= @_;\n",
" \n",
" if ($a > $b) {\n",
" return $a;\n",
" }\n",
" else {\n",
" return $b;\n",
" }\n",
"}\n",
"\n",
"my $max = maximum(@genome_mb);\n",
"\n",
"print \"Maximum genome size in our list is $max Mb\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Maximum genome size in our list is 2700 Mb\n"
]
}
],
"prompt_number": 43
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we create a subroutine called `maximum` which takes two arguments, `$a` and `$b`. Perl has an odd syntax for subroutine arguments. It creates a special array, `@_`, to hold any arguments passed into a subroutine. To get at the arguments, we have to unpack `@_`. This is what we are doing in the first line of `maximum`.\n",
"\n",
"To return a value from the subroutine, we use the `return` keyword. Perl's subroutines can only return a single value or an array, but not separate multiple values, and not a hash.\n",
"\n",
"For interest, here's another take on this subroutine:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @genome_mb = (100.3, 2700);\n",
"\n",
"sub maximum2 {\n",
" my $a = shift;\n",
" my $b = shift;\n",
" return ($a > $b) ? $a : $b;\n",
"}\n",
"\n",
"my $max = maximum2 @genome_mb;\n",
"\n",
"print \"Maximum genome size in our list is $max Mb\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Maximum genome size in our list is 2700 Mb\n"
]
}
],
"prompt_number": 44
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Three things have changed here. Firstly, we are now calling the subroutine without brackets - this works for our own subroutines as well as core functions, although it is not recommended. Secondly, we are using `shift` to take arguments off the special `@_` array; as this array is always created for a subroutine, Perl doesn't require us to call it by name, and just assumes we mean this array when we don't give an argument for shift.\n",
"\n",
"Thirdly, we are using a nice shorthand conditional syntax: `(cond) ? A : B`. If `cond` is true, do whatever is after the `?`; if false, do whatever is after the `:`. This is very handy when we are trying to decide what value a variable should have based on some condition, as here, or when the processing we need to do as a result of the computation is very simple."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Exercises\n",
"\n",
"1. Write a subroutine based on `maximum` that takes two numerical arguments and returns their mean.\n",
"2. Our `mean` and `maximum` subroutines assume we only have two arguments, but often we have more or less. Write another subroutine that takes an array that could contain any number of elements and returns the mean of all the elements. Use the first `@genome_mb` array (with five elements) to test your subroutine.\n",
"3. Write a subroutine that takes a DNA sequence as input and returns the GC content of the sequence as a percentage. Round the percentage to the nearest integer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 11: User/Command-line input"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So far we have just been operating on data that we have defined ourselves within our scripts. In the real world we will want to get input data from users and files etc., operate on it, and frequently output some results to other files. Perl has lots of support for dealing with input and output (I/O in computer jargon), and we'll go through some of these in the next few lessons.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first form of user input we'll look at is reading in arguments passed to our scripts on the command line, these may include specific parameters for some analysis (e.g. gene names, or numerical cutoffs) or often the names of files that we will operate on later. Command line arguments are similar in some ways to arguments that we pass to subroutines.\n",
"\n",
"So far we have been running our scripts with command lines like:\n",
"\n",
"`perl my_script.pl`\n",
"\n",
"If we want to supply some arguments to the script we can do so by writing them after the name of the script:\n",
"\n",
"`perl my_script.pl 23 BRCA2 0.5`\n",
"\n",
"Perl will split these up according to whitespace (so if you want to pass in an argument that contains whitespace you'll need to put the string in quotation marks), and then put them all in another 'magic' array variable called `@ARGV`.\n",
"\n",
"We can then access arguments as values in this array just like any other, so `@ARGV` is equivalent to the `@_` array Perl uses to pass arguments to subroutines."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson11_example1.pl\n",
"\n",
"my @user_input = @ARGV;\n",
"print join(\"\\n\" , @user_input) , \"\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson11_example1.pl\n"
]
}
],
"prompt_number": 45
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson11_example1.pl 23 BRCA2 0.5"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"23\n",
"BRCA2\n",
"0.5\n"
]
}
],
"prompt_number": 46
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### `shift` and `@ARGV`\n",
"\n",
"Remember the array function `shift` removes and returns the first element of an array."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson11_example2.pl\n",
"\n",
"my $first_input = shift @ARGV;\n",
"print \"first input: $first_input\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson11_example2.pl\n"
]
}
],
"prompt_number": 47
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson11_example2.pl one two three"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"first input: one\n"
]
}
],
"prompt_number": 48
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Just like in subroutines where the argument array `@_` is implicitly used when calling array functions like `shift` so you don't have to explicitly name it, in the main body of a script `@ARGV` will be used implicitly, so a common idiom for reading in a command line argument is:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson11_example3.pl\n",
"\n",
"my $first = shift;\n",
"print \"First: $first\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson11_example3.pl\n"
]
}
],
"prompt_number": 49
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson11_example3.pl one two three"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"First: one"
]
}
],
"prompt_number": 50
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Using array indices\n",
"\n",
"Remember with arrays that each element can be accessed using indices."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson11_example4.pl\n",
"my $first_input = $ARGV[0];\n",
"my $second_input = $ARGV[1];\n",
"print \"$first_input and $second_input\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson11_example4.pl\n"
]
}
],
"prompt_number": 51
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson11_example4.pl one two three"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"one and two\n"
]
}
],
"prompt_number": 52
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"####Assigning an array to a list of variables "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson11_example5.pl\n",
"my ($in_1, $in_2 , $in_3) = @ARGV;\n",
"print \"$in_1 and $in_2 and $in_3\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson11_example5.pl\n"
]
}
],
"prompt_number": 53
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson11_example5.pl one two three"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"one and two and three\n"
]
}
],
"prompt_number": 54
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Write a script that takes 2 numbers from the command line, adds the 2 numbers and prints out the result.\n",
"2. Write a script that reads in a DNA sequence from the command line, and then prints out its length and GC percentage (using the subroutine you wrote earlier on to compute the GC content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 12: Sanity checks and testing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we are able to read in some input from users, it is important for your programs to check that the input matches any assumptions you have made about this input, and to deal with things gracefully if they do not. For example, say you want to compute the average of some numbers the user supplies on the command line, what should you do if the user gives you strings instead of numbers?\n",
"\n",
"An even more basic check to perform is that an expected argument has indeed been supplied.\n",
"\n",
"Another common check is that if the user supplies the name of a file to operate on (as we will see later), we should check that the file exists, and perhaps that it has some data in it.\n",
"\n",
"Perl provides a few useful functions to help with these sort of checks:\n",
"\n",
"\n",
"Test | Description |
\n",
"defined | checks if a variable is defined |
\n",
"-e | checks if a file exists |
\n",
"-s | checks the size of a file (returning the result as a number) |
\n",
"
\n",
"\n",
"If we find some problem with user input then Perl also has a couple of useful functions that allow us to print warning messages to the user, without interfering with the normal output of the program:\n",
"\n",
"\n",
"Action | Description |
\n",
"die $message | prints a message to STDERR and kills the program |
\n",
"warn $message | prints a message to STDERR, but the program will continue |
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The example below checks to see if an argument was provided, and if it is then the argument will be assigned to `$arg_1`. Otherwise the `else` block will be executed and the script will die and an error message will be printed."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson12_example1.pl\n",
"\n",
"my $arg_1;\n",
" \n",
"if (defined $ARGV[0]){\n",
" $arg_1 = $ARGV[0];\n",
" print \"Here's your number: $arg_1\\n\";\n",
"}\n",
"else {\n",
" die \"Please supply a number\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson12_example1.pl\n"
]
}
],
"prompt_number": 55
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson12_example1.pl 3.14\n",
"perl lesson12_example1.pl"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Here's your number: 3.14\n"
]
},
{
"output_type": "stream",
"stream": "stderr",
"text": [
"Please supply a number\n"
]
}
],
"prompt_number": 56
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### File tests\n",
"\n",
"\n",
"The example below does the same check, but also uses `-e` to check that a file with the supplied name exists. Here we are using negative conditions (NOT defined and so on) to check the basic sanity tests, so we can get them out of the way before we get on to the rest of the code. This way, we don't have to embed all of our code in if blocks. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson12_example2.pl\n",
"\n",
"if (!defined $ARGV[0]) {\n",
" die \"Please supply a file\\n\";\n",
"}\n",
"\n",
"if (!-e $ARGV[0]) {\n",
" die \"$ARGV[0] does not exist. Please check the filename and/or location.\\n\";\n",
"}\n",
"\n",
"my $filename = $ARGV[0];\n",
"\n",
"print \"$filename is a file that exists!\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson12_example2.pl\n"
]
}
],
"prompt_number": 57
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson12_example2.pl lesson12_example1.pl\n",
"perl lesson12_example2.pl\n",
"perl lesson12_example2.pl lesson12_example42.pl"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"lesson12_example1.pl is a file that exists!\n"
]
},
{
"output_type": "stream",
"stream": "stderr",
"text": [
"Please supply a file\n",
"lesson12_example42.pl does not exist. Please check the filename and/or location.\n"
]
}
],
"prompt_number": 58
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Modify the script you wrote for the last lesson that takes two numbers from the command line and adds them, add a check to make sure that two arguments are supplied. Die if there are too few or too many arguments (Hint: remember that `@ARGV` is just a normal array)\n",
"2. Write a script that takes a single filename as an argument and prints out the size of the named file. The script should check that an argument has been provided and that the file exists. Use any file that you can see in the same directory as your script (on the command line, use `ls` on Linux or Mac or `dir` on Windows to check for one)\n",
"3. Optional: what else could go wrong with your script that adds two numbers? For example, what happens if one of the arguments is not a number? See if you can find a function in a core module that will deal with this problem, load it into your script and use it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 13: Files and File handles"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Frequently the data we want to operate on or analyse will be stored in files, so in our programs we need to be able to open files, read through them (perhaps all at once, perhaps not), and then close them. We will also frequently want to be able to print output to files rather than always printing out results to the terminal.\n",
"\n",
"Perl supports all of these modes of operations on files, and provides a number of useful functions and syntax to make dealing with files fairly straightforward."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first thing we need to be able to do is open a file, for this Perl has the conveniently named `open` function.\n",
"\n",
"The syntax of this function is: `open(file handle, mode, file name)`\n",
"\n",
"A _file handle_ is a special scalar variable that perl uses to represent files in your program, and is distinct from the _file name_ which is just a string which contains the location of the file on your machine's hard disk.\n",
"\n",
"The _mode_ tells Perl if you want to open your file for reading or writing, or appending (another form of writing), and Perl uses special characters to represent each of these modes (note these are the same as the UNIX file redirection symbols):\n",
"\n",
"\n",
"Mode | Symbol |
\n",
"reading | < |
\n",
"writing | > |
\n",
"appending | >> |
\n",
"
\n",
"\n",
"If the call to `open` is successful it will return a true value, if it is not, perhaps because the file doesn't exist, or you don't have the correct permissions to open it, it will return a false value.\n",
" \n",
"Another very common idiom that relies on this is to include an `or die` statement directly after the call to open which will be executed if there was some problem opening the file."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file lesson13_example1.pl\n",
"\n",
"my $file = shift @ARGV; \n",
"\n",
"open (my $fh , '<' , $file) or die \"Can't open $file, $!\\n\";\n",
"\n",
"print \"opened file: $file\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing lesson13_example1.pl\n"
]
}
],
"prompt_number": 59
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl lesson13_example1.pl lesson13_example1.pl"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"opened file: lesson13_example1.pl\n"
]
}
],
"prompt_number": 60
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Iterating through a file\n",
"\n",
"Once we have successfully opened up our file we will probably want to start reading the contents of a file, which we typically do line by line. A common pattern, especially if we don't know in advance how many lines our file contains, is to use a `while` loop to loop over all the lines in the file operating on each line in turn - perhaps doing some analysis on each line as we proceed, or perhaps storing the data in some data structure to analyse later on in the script.\n",
"\n",
"Perl provides some special syntax to read files line by line, to access lines in a file we wrap our file handle in `<>` characters, e.g. `<$FILEHANDLE>`. This is another operator that is aware of the context in which it is called. In scalar context this will return the next line in the file (Perl keeps track of where in a file it is up to for you), but if used in array context it will return all lines from the file in an array (watch out for big files!)\n",
"\n",
"- scalar context: `my $line = <$FILEHANDLE>`\n",
"- array context: `my @lines = <$FILEHANDLE>`\n",
"\n",
"Each line in the file with have an invisible newline character `\\n` at the end, and generally we will want to remove this before operating on the line, and so we can use the `chomp` string function to remove it.\n",
"\n",
"The example file used below is in `Course_Materials/lesson_files` but any file you have will do. You can either change directory to lesson_files, or you can just specify the name of the directory when writing out the file. Here we are using the Unix command `cat` to print out the contents of the file."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"cat lesson_files/lesson13_file1.txt\n",
"cd lesson_files\n",
"cat lesson13_file1.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"This is a file\n",
"with\n",
"lines of words.\n",
"This is a file\n",
"with\n",
"lines of words.\n"
]
}
],
"prompt_number": 61
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $filename = 'lesson_files/lesson13_file1.txt';\n",
"\n",
"open(my $inputfh, '<' , $filename) or die \"can't open $filename\\n\";\n",
" \n",
"while (my $line = <$inputfh>) {\n",
" chomp $line;\n",
" print uc $line , \"\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"THIS IS A FILE\n",
"WITH\n",
"LINES OF WORDS.\n"
]
}
],
"prompt_number": 62
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Open and close a file for writing\n",
"\n",
"If we open a file for writing we to tell Perl to print output to that file handle, otherwise it will keep printing out to the terminal. We do this by putting the file handle between the `print` statement and the stuff we want to print to that file. It is good practice to then `close` your file when you are finished with it."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $newfilename = 'lesson13_file2.txt'; \n",
"open(my $newfh , '>' , $newfilename) or die \"Can't open $newfilename\\n\";\n",
" \n",
"print $newfh lc \"Print this TO a FILE in LOWER case\\n\";\n",
"close $newfh;"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 63
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"cat lesson13_file2.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"print this to a file in lower case\n"
]
}
],
"prompt_number": 64
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tab delimited files\n",
"\n",
"In biology we very often end up dealing with tab (or comma) delimited files. We can use some of the string functions we covered earlier such as `split`, in combination with the various file operations Perl supports, to deal with the sorts of files in a very straightforward way.\n",
"\n",
"Here is an example splitting the columns into elements of an array. The file we will use contains the chromosomal coordinates of a few genes downloaded from Ensembl, and has 4 columns: gene, chromosome, start, end. We want to read this file and then compute the length of each gene. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"cat lesson_files/lesson13_file3.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"BRCA2\t13\t32889611\t32973805\n",
"TNFAIP3\t6\t138188351\t138204449\n",
"TCF7\t5\t133450402\t133487556\n"
]
}
],
"prompt_number": 65
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $genefilename = 'lesson_files/lesson13_file3.txt';\n",
"\n",
"open(my $genefh, '<' , $genefilename) or die \"Can't open $genefilename\\n\";\n",
"\n",
"while (my $line = <$genefh>) {\n",
" chomp $line;\n",
" \n",
" my ($gene, $chr, $start, $end) = split \"\\t\", $line;\n",
" \n",
" ## for csv (comma separated files) split on the comma\n",
" # split ',' , $line;\n",
" \n",
" my $length = $end - $start + 1;\n",
" \n",
" print \"$gene is on chromosome $chr and is $length bps long\\n\";\n",
"}\n",
"close $genefh;"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"BRCA2 is on chromosome 13 and is 84195 bps long\n",
"TNFAIP3 is on chromosome 6 and is 16099 bps long\n",
"TCF7 is on chromosome 5 and is 37155 bps long\n"
]
}
],
"prompt_number": 66
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Create a script that takes the name of a file containing many lines of nucleotide sequence as a command line argument, checks that it exists with `-e`, and opens the file for reading. For each line in the file, print out the line number and the length of the corresponding line. You can use `lesson13_file4.txt` as an example input file, or you can create your own.\n",
"2. Write a script that reads in a FASTA file of DNA sequence and computes the GC content of each line, using the GC content subroutine from earlier lessons (see, for example, `lesson11_exercise2.pl`). The script should also open an output file for writing and then print the GC content for each input line on a separate line in the output file. You should check that you don't try to compute GC content on the header lines, which start with a '>' character. You can use `lesson13_file5.fasta` as an example input file, which contains BRCA2 sequences for human and mouse, or you can create your own."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash \n",
"cat lesson_files/lesson13_file4.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"CGGCTAGATCCAGAT\n",
"CGTGTAA\n",
"GTACACCCA\n",
"GTCAACACTTA\n"
]
}
],
"prompt_number": 67
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash \n",
"cat lesson_files/lesson13_file5.fasta"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
">13 dna:chromosome chromosome:GRCh37:13:32889611:32973805:1\n",
"GGGCTTGTGGCGCGAGCTTCTGAAACTAGGCGGCAGAGGCGGAGCCGCTGTGGCACTGCT\n",
"GCGCCTCTGCTGCGCCTCGGGTGTCTTTTGCGGCGGTGGGTCGCCGCCGGGAGAAGCGTG\n",
"AGGGGACAGATTTGTGACCGGCGCGGTTTTTGTCAGCTTACTCCGGCCAAAAAAGAACTG\n",
"CACCTCTGGAGCGGGTTAGTGGTGGTGGTAGTGGGTTGGGACGAGCGCGTCTTCCGCAGT\n",
"CCCAGTCCAGCGTGGCGGGGGAGCGCCTCACGCCCCGGGTCGCTGCCGCGGCTTCTTGCC\n",
"CTTTTGTCTCTGCCAACCCCCACCCATGCCTGAGAGAAAGGTCCTTGCCCGAAGGCAGAT\n",
"TTTCGCCAAGCAAATTCGAGCCCCGCCCCTTCCCTGGGTCTCCATTTCCCGCCTCCGGCC\n",
"CGGCCTTTGGGCTCCGCCTTCAGCTCAAGACTTAACTTCCCTCCCAGCTGTCCCAGATGA\n",
"CGCCATCTGAAATTTCTTGGAAACACGATCACTTTAACGGAATATTGCTGTTTTGGGGAA\n",
">gi|1905987|gb|U89503.1|MMU89503 Mus musculus Brca2 (BRCA2) gene, partial cds\n",
"GATATCCACTCTAAAAAATAAAAAAAGAAAGTTTATTTACTCTGTAAGTGATGATGCATCTCTTCAAGGA\n",
"AAAAAACTACAGACACACAGACAGTTAGAGCTTACTAATCTTTCAGCCCAGCTTGAAGCAAGTGCTTTTG\n",
"AAGTACCATTGACCTTTACAAATGTAAATTCAGGTATTATTCTTTTTTTCTTCTAATGTATATAGTCTTT\n",
"AGGATGAGATTAAGTTTTCCTATTTGAATATAAGTTTTTTGTTTTTTGAATTTTAAGAGAAACACTTGCT\n",
"TACTTTCTGTAGTTTTTCAGATCTCTTTTTGGTTTGGTTTGGTTTGGTTTGGTTTGGTTTGGTTTGGTGA\n",
"GAACCCTGTGTAGCCCTGGCTGTCTTAGAACTTAGGCTGGCCTGGAACTTAGATATC"
]
}
],
"prompt_number": 68
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 13b: Perl one-liners"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Very often in bioinformatics we will deal with line-oriented text files with multiple columns of information, and sometimes we want to convert between formats or compute simple statistics over a complete data file without having to write a full script (though that is a good idea if the computation we want to perform is complex, or we will be running the same analysis multiple times).\n",
"\n",
"There are several unix tools that can be used to do this kind of work, such as `cut` to extract particular columns, `awk` to filter data and perform simple calculations, and `sed` to substitute text. Perl was originally designed as a superset of all these tools, and supports working with data files in a very similar way to these tools in 'one-liner' mode.\n",
"\n",
"Various flags can be supplied to the perl interpreter to make it useful in this context:\n",
"\n",
"- `-e` execute the string passed on the command line as a perl program (the program should be supplied within quotes)\n",
"- `-l` automatically `chomp`s each line, and prints a trailing newline after each `print` command\n",
"- `-a` automatically `split`s each line from the standard input into a special array called `@F`, the delimiter defaults to whitespace, but can be controlled by specifying a delimiter with `-F`, e.g. `-F\",\"` will split on commas, or `-F\"\\t\"` to split on tabs.\n",
"- `-n` which tells perl to assume the following loop around your program: `while (<>) { # your program goes here }`\n",
"\n",
"These can all be combined to produce a powerful mechanism for processing files and other data piped into your program from the command line. If you want to keep track of some data throughout the run and then produce some summary results at the end, you can use an `END{}` block at the end of your code which will only be run once the whole file has been processed. You can also supply a `BEGIN{}` block which will be run before any lines are processed and can be used, for example, to print out a header line for your output.\n",
"\n",
"For example, we could solve the exercise from above with the following command line:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl -lane 'print $. , \": \", length $F[0]' < lesson_files/lesson13_file4.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"1: 15\n",
"2: 7\n",
"3: 9\n",
"4: 11\n"
]
}
],
"prompt_number": 69
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the special perl variable `$.` contains the current line number being processed. As another example we could print out the gene names and corresponding chromosome from the example file of gene coordinates with:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"perl -lane 'print \"$F[0] is on chromosome $F[1]\"' < lesson_files/lesson13_file3.txt"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"BRCA2 is on chromosome 13\n",
"TNFAIP3 is on chromosome 6\n",
"TCF7 is on chromosome 5\n"
]
}
],
"prompt_number": 70
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Write a perl one liner to compute the length of each gene from `lesson13_file3.txt` and print the gene name in lower case and it's length separated by a comma, you could print a header line as well with a `BEGIN{}` block.\n",
"\n",
"2. Advanced: write a perl one liner to compute the counts of all nucleotides occuring in `lesson13_file4.txt`"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 13c: System calls"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another common use of Perl, particularly on UNIX systems, is to automate running external programs or scripts.\n",
"\n",
"Perl makes this very easy to do, and there are two ways in which you can run an external command.\n",
"\n",
"- using `system` function\n",
"- using backticks ``"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### `system`\n",
"\n",
"`system` is a function that you can supply the command line statement to be executed as the first argument and the exit status is returned. Output cannot be captured in a variable using `system`, but see backticks `` below for this feature.\n",
"\n",
"`system` returns:\n",
"\n",
"- 0 for no errors\n",
"- -1 for errors\n",
"- if you get an error, you can print the variable `$!` to find out the reason"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $sys = system (\"date\");\n",
"print \"sys returned: $sys\\n\";\n",
"\n",
"if ($sys == -1) {\n",
" print \"Error: $!\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu 1 May 2014 20:38:40 BST\n",
"sys returned: 0\n"
]
}
],
"prompt_number": 71
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Backticks `` \n",
"\n",
"You can also wrap the command you want to run (which may include variables) in backticks\\`\\`. The advantage of this approach is that you can capture the output of the external command into a variable in your script, and then operate on this data. In array context backticks will give you all the output lines in an array, while in scalar context you will get all the output as a single string value."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $dir = `pwd`;\n",
"\n",
"print \"I am currently in $dir\\n\";\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"I am currently in /Users/grsr/git/perl-course\n",
"\n"
]
}
],
"prompt_number": 72
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"1. Write a script that runs the UNIX command `ls` to get a list the files in the current directory, then prints out all upper case versions of all the file names.\n",
"2. Modify your script to only print out Perl files, i.e. those ending with '.pl'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 14: Sorting with arrays and hashes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sorting data in one way or another is a very common operation. Perl provides us with a very flexible function called `sort` that we can use to sort any kind of array, and because we can access the keys and values of hashes as arrays, it allows us to control the order in which we output values stored in hashes etc.\n",
"\n",
"`sort` is very customisable, but its default behaviour is to sort in alphabetical order (or ASCII order to be precise).\n",
"\n",
"`sort` is a function that takes an array as an argument and returns a copy of the array sorted according to our criteria."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @array = ('Valine', 'Leucine','Cysteine','Tyrosine');\n",
"my @sorted_array = sort @array;\n",
" \n",
"foreach my $element (@sorted_array){\n",
" print \"$element\\n\";\n",
"}\n",
" \n",
"my @numbers = (3,400,1,100,4,20);\n",
"my @sorted_numbers = sort @numbers;\n",
" \n",
"foreach my $element (@sorted_numbers){\n",
" print \"$element\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Cysteine\n",
"Leucine\n",
"Tyrosine\n",
"Valine\n",
"1\n",
"100\n",
"20\n",
"3\n",
"4\n",
"400\n"
]
}
],
"prompt_number": 73
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Sorting numerically\n",
"\n",
"When we have numerical data we generally don't want this behaviour, we'd rather that the values were sorted by their values, possibly either ascending or descending.\n",
"\n",
"To do this we have to customise the behaviour of `sort` with a small block of code placed in between `sort` and the array we are sorting.\n",
"\n",
"The only thing `sort` needs to know to be able to sort any array in any complicated way you can imagine is whether one value is \"less than\", \"greater than\" or \"equal\" to another, and that what the block of code you supply to sort tells it. Inside this block of code the two values `sort` is asking you to compare are called `$a` and `$b`.\n",
"\n",
"So, to sort a list of numbers we need some function that tells `sort` how to compare two numbers. You could supply your own function, but this is a common requirement so Perl gives us the \"spaceship\" operator `<=>`, and the block of code we will use is: `{$a <=> $b}`.\n",
"\n",
"This block will sort numbers in ascending order, if we want to reverse the sort order we can just swap the order of `$a` and `$b`."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @numbers = (3,400,1,100,4,20);\n",
"\n",
"my @sorted_numbers = sort {$a <=> $b} @numbers;\n",
" \n",
"foreach my $element (@sorted_numbers){\n",
" print \"$element\\n\";\n",
"}\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"1\n",
"3\n",
"4\n",
"20\n",
"100\n",
"400\n"
]
}
],
"prompt_number": 74
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Sorting by length\n",
"\n",
"As well as sorting arrays according to the values of the elements, we can sort by any property of the values as well.\n",
"\n",
"A useful example is to sort an array of strings by the lenght of each string, instead of their alphabetical ordering. We can do this just by slightly modifying the sort block."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my @fragments = ( 'TCTCGAATC' , 'TTA' , 'A' , 'GCGTGATGTCGA' , 'GATC' );\n",
"\n",
"my @sorted_by_length = sort { length($b) <=> length($a) } @fragments;\n",
"\n",
"foreach my $element( @sorted_by_length ) {\n",
" print \"$element\\t\", length ($element), \"\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"GCGTGATGTCGA\t12\n",
"TCTCGAATC\t9\n",
"GATC\t4\n",
"TTA\t3\n",
"A\t1\n"
]
}
],
"prompt_number": 75
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Sorting a hash by values\n",
"\n",
"Because hashes are inherently unordered people often want to access hashes in more controlled way by sorting either the keys or the values.\n",
"\n",
"Sorting by key is easy, we just use `keys` to get an array of the keys, sort this array, and then use this sorted array to access all of out values. Sorting by value is a little trickier.\n",
"\n",
"Recall that to access the value corresponding to some key in a hash we use code like: `$value = $hash{$key}`\n",
"\n",
"So to access the values of the hash in a sorted order, we will in fact have to sort the keys.\n",
"\n",
"In our sort block `$a` and `$b` will hold the keys of the hash, but just as in the length example above we will be sorting by some property of the keys, namely their corresponding values. Here's an example:\n",
"\n",
"Order of operations:\n",
"\n",
"1. keys are retrieved with the `keys` function\n",
"2. the keys are used to retrieve the values\n",
"3. the keys are sorted and returned based on the value ( `$value = $hash{$key}` )\n",
"4. the list of newly sorted keys (based on the value) are passed 1 by 1 to `$nt` by the foreach loop"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my %nt_count = (\n",
" 'G' => 2,\n",
" 'C' => 1,\n",
" 'T' => 4,\n",
" 'A' => 3,\n",
" );\n",
"\n",
"@sorted_by_values = sort {$nt_count{$a} <=> $nt_count{$b}} keys %nt_count;\n",
"\n",
"print join ' - ', @sorted_by_values;\n",
"\n",
"print \"\\n\";\n",
"\n",
"foreach my $nt( @sorted_by_values ) {\n",
" print \"$nt\\t$nt_count{$nt}\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"C - G - A - T\n",
"C\t1\n",
"G\t2\n",
"A\t3\n",
"T\t4\n"
]
}
],
"prompt_number": 76
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Write a script that reads in some numbers from the command line and prints them in descending numerical order.\n",
"2. Write a program that will accept a DNA sequence from the command line and outputs the number of As, Cs, Gs and Ts in the sequence, sorted by number of occurrences of each nucleotide, smallest first. Hint: try using a hash to store the nucleotide counts, where the keys are the nucleotides and the values are the occurrences."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 15a: Regular Expressions I"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In all sorts of programming contexts we will want to manipulate and analyse text. We have seen a number of useful string functions built into Perl that let us do things like look at substrings of strings, chop them up in various ways, and we can already combine these with Perl's logical operators to manipulate strings in fairly complex ways.\n",
"\n",
"We don't need very complicated examples though to realise the limitations of manipulating text using what we've learned so far though. Consider the simple example of checking that a variable supplied by the user is a true number, and doesn't contain any non-numerical characters. How could you check for this?\n",
"\n",
"This is just a very simple example, there are all sorts of patterns we might want to identify in data - in biology we might want to check that a string contains only valid DNA bases, or amino acid codes, or even try to identify specific motifs in a DNA sequence where proteins bind or restriction enzymes cut.\n",
"\n",
"Perl provides us with a very powerful, but somewhat complicated, system to let us answer these kinds of questions called \"Regular Expressions\" (\"regexes\", \"REs\"). This is almost another language built inside Perl and we will only introduce some of the basics here.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When we write a regular expression we are essentially writing a template against which we will check values to see if they _match_ the RE. We typically write these templates inside forward slashes `//`, and we ask if a string matches the template with the `=~` operator which is a logical operator like `==`. There is also an equivalent to `!=` to check if the string does not match: `!~`.\n",
"\n",
"The simplest RE, that is still often very useful, just checks if a specific bit of text is found in the string, here we check if there is a stop codon anywhere in the DNA."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = 'ACTTGATAG';\n",
"\n",
"if ($dna =~ /TAG/) {\n",
" print \"$dna contains a stop codon\\n\";\n",
"}\n",
"else {\n",
" print \"No stop codons found\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ACTTGATAG contains a stop codon\n"
]
}
],
"prompt_number": 77
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As this example shows, inside the pattern, normal characters just match themselves, but we can also match more general classes of things:\n",
"\n",
"- Any normal character matches itself\n",
"- The `.` character matches anything except a newline\n",
"- A specific set of characters specified in square brackets, e.g. `[ACGT]` matches any DNA base, `[1234]` matches any number less than 5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By default, the string we are testing is allowed match the pattern anywhere, so in our example above we could have a stop codon as the first 3 bases and the expression would still match. We can however control where in the string the pattern can match with the following special characters:\n",
"\n",
"- `^` Matches the beginning of the string\n",
"- `$` Matches the end of the string\n",
"\n",
"We can use all of these features to look for common biological signals in DNA sequences:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = \"ACAGGAGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATAGGCAGACGAGAACGTAG\";\n",
"\n",
"if ($dna =~ /CTGCAG/) {\n",
" print \"Found PstI recognition site\\n\";\n",
"}\n",
"if ($dna =~ /GA.TC/) {\n",
" print \"Found HinfI recognition site\\n\";\n",
"}\n",
"if ($dna =~ /[AG]GATC[TC]/) {\n",
" print \"Found XhoII recognition site\\n\";\n",
"}\n",
"if ($dna =~ /TAG$/) {\n",
" print \"Found STOP codon at the end of the sequence\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Found PstI recognition site\n",
"Found HinfI recognition site\n",
"Found XhoII recognition site\n",
"Found STOP codon at the end of the sequence\n"
]
}
],
"prompt_number": 78
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- Perl provides various predefined character classes:\n",
" - `\\d` matches any digit (`[0-9]`)\n",
" - `\\w` matches any word character (`[a-zA-Z0-9_]`)\n",
" - `\\s` matches any whitespace (`[ \\t\\n\\r]`)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each of the _atoms_ we have described so far will match a single character, but we can quantify how many times any of them will match with some special characters:\n",
"\n",
"- `*` matches atom zero or more times\n",
"- `+` matches atom one or more times"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"foreach my $genename (\"BRCA1\",\"BRCA11\",\"BRCA 1\",\"BRCA1 gene\") {\n",
" if ($genename =~ /^\\w+\\d$/) {\n",
" print \"$genename contains one or more word characters, one digit, and nothing else\\n\";\n",
" }\n",
" else {\n",
" print \"$genename contains something other than one or more word characters, one digit and nothing else\\n\";\n",
" }\n",
" if ($genename =~ /\\w+\\s*\\d+/) {\n",
" print \"$genename contains one or more word characters, optional whitespace and one or more digits\\n\";\n",
" }\n",
" else {\n",
" print \"$genename contains something other than one or more word characters, optional whitespace and one or more digits\\n\";\n",
" }\n",
"}\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"BRCA1 contains one or more word characters, one digit, and nothing else\n",
"BRCA1 contains one or more word characters, optional whitespace and one or more digits\n",
"BRCA11 contains one or more word characters, one digit, and nothing else\n",
"BRCA11 contains one or more word characters, optional whitespace and one or more digits\n",
"BRCA 1 contains something other than one or more word characters, one digit and nothing else\n",
"BRCA 1 contains one or more word characters, optional whitespace and one or more digits\n",
"BRCA1 gene contains something other than one or more word characters, one digit and nothing else\n",
"BRCA1 gene contains one or more word characters, optional whitespace and one or more digits\n"
]
}
],
"prompt_number": 79
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Use a regular expression to check if a DNA sequence contains any `CG` dinucleotides.\n",
"2. Write a regular expression that will check that a string only contains DNA bases, but allow for either upper or lower case characters.\n",
"3. DNA can be methylated at specific sites where there is a purine base (either a G or an A), then a C, then any base, then a G. Write a regular expression to check if a DNA string contains such a methylation site.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"------"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 15b: Regular Expressions II"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Capturing matches\n",
"\n",
"So far we have only been checking that our values match the patterns we have been concocting. You can also use regular expressions to capture matching chunks of the string.\n",
"\n",
"To tell Perl you want to capture some bit of the match, just surround the bit you're after in parentheses `()`. You can do this multiple times if you want. Once you have checked that the pattern matches, Perl will then create some (more) \"magic\" variables that will contain the matched portion of the string. The first captured match will be in a variable called `$1`, the second in `$2` and so on.\n",
"\n",
"We can also look for alternate variations using the `|` operator. So, for example, we could check for any of the three possible STOP codons and capture which one we have found:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = \"ACGTGTAG\";\n",
"\n",
"if ($dna =~ /(TAG|TAA|TGA)/) {\n",
" print \"Found this stop codon: $1\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Found this stop codon: TAG\n"
]
}
],
"prompt_number": 80
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Subpatterns and Greediness\n",
"\n",
"By default, regular expressions are \"greedy\". They try to match as much as they can. For example, suppose we wanted to look for gene sequences that start with a Methionine (start codon, ATG) and end with a STOP codon. We might try this, to pull out the two gene sequences in the following sequence (genes in uppercase just so you can see where they are, we turn the whole sequence to upper case straight away before matching):"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = 'caccATGAGACAGAACAGTAGgggacagttgcacATGCCAGGACGACCCATATAATAGaca';\n",
"$dna = uc $dna;\n",
"$dna =~ /(ATG[ACGT]+(TAG|TAA|TGA))/;\n",
"print \"$1\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ATGAGACAGAACAGTAGGGGACAGTTGCACATGCCAGGACGACCCATATAATAG\n"
]
}
],
"prompt_number": 81
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Because the match is greedy, both of the genes have been extracted. To pull out just the first gene, we need to match the minimum number of times, not the maximum. We can do this using a `?`, which modifies the preceding quantifier and makes it non-greedy. Note that (unfortunately) the `?` is also a quantifier itself that specifies that the atom should match 0 or 1 times (i.e. make it optional)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = 'caccATGAGACAGAACAGTAGgggacagttgcacATGCCAGGACGACCCATATAATAGaca';\n",
"$dna = uc $dna;\n",
"$dna =~ /(ATG[ACGT]+?(TAG|TAA|TGA))/;\n",
"print \"$1\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ATGAGACAGAACAGTAG\n"
]
}
],
"prompt_number": 82
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"What if we wanted to get both of the genes? We could do this by asking for the regular expression to do a global search, with the `g` modifier. We can then use a while loop to get each match one at a time:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $dna = 'caccATGAGACAGAACAGTAGgggacagttgcacATGCCAGGACGACCCATATAATAGaca';\n",
"my $dna = uc $dna;\n",
"while ($dna =~ /(ATG[ACGT]+?(TAG|TAA|TGA))/g) {\n",
" print \"$1\\n\";\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ATGAGACAGAACAGTAG\n",
"ATGCCAGGACGACCCATATAA\n"
]
}
],
"prompt_number": 83
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### String Substitution\n",
"\n",
"So far we have only matched parts of strings, but we can also replace patterns with the `s///` function.\n",
"\n",
"`s///` has two parts: the regular expression and the string to replace it with: `s/expression/replacement/`. By default, `s///` will only substitute one instance of the match, if you want to substitute all instances then add a `g` at the end: `s/expression/replacement/g`.\n",
"\n",
"For example, we can simulate the degradation of methylated CpG sites in the genome to TpG as follows:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl \n",
"\n",
"my $dna = \"CGAGGAGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATAGGCAGACGAGAACGTCG\";\n",
"\n",
"print \"$dna\\n\";\n",
"$dna =~ s/CG/TG/;\n",
"print \"$dna\\n\"; # Whoops! only mutates the first CG\n",
"$dna =~ s/CG/TG/g;\n",
"print \"$dna\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"CGAGGAGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATAGGCAGACGAGAACGTCG\n",
"TGAGGAGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATAGGCAGACGAGAACGTCG\n",
"TGAGGAGGATCTACTGCAGGCCAGTGTGAAGAGACTCATATAGGCAGATGAGAATGTTG\n"
]
}
],
"prompt_number": 84
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Regular expression can also be used in several Perl built in functions that we have already seen, for example you can use a regular expression as the delimiter in a `split` command. Here's how we could simulate a set of PstI restriction fragments, for example:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my $psti = 'CTGCAG';\n",
"my $dna = \"CGAGGACTGCAGGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATACTGCAGCGTCG\";\n",
"my @psti_frags = split /$psti/, $dna;\n",
"print \"Original sequence\\n$dna\\n\";\n",
"print \"PstI($psti) fragments:\\n\", join(\"\\n\", @psti_frags), \"\\n\""
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Original sequence\n",
"CGAGGACTGCAGGGATCTACTGCAGGCCAGCGCGAAGAGACTCATATACTGCAGCGTCG\n",
"PstI(CTGCAG) fragments:\n",
"CGAGGA\n",
"GGATCTA\n",
"GCCAGCGCGAAGAGACTCATATA\n",
"CGTCG\n"
]
}
],
"prompt_number": 85
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Use the `s///` operator to transcribe a DNA sequence into an RNA sequence.\n",
"2. The consensus transcription start site sequence is `TATTAT`. Read in the FASTA file `lesson13_file5.fasta` and check each of the sequences in this file for transcription start sites. If a start site is found, print out the sequence downstream of the start site, including any lines after the one containing the start site."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Lesson 16: References and multidimensional data structures"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the lesson on subroutines, we saw that we can only pass a single array into a subroutine, and only return a single scalar or array. But we might want to pass in several data structures as input and return more than one value, or possibly more than one complex data structure. We know from looking at arrays earlier that if we put an array inside of an array, Perl will flatten the array and lose the internal structure. This will also happen if we pass in an array or a hash to a subroutine, which is no good.\n",
"\n",
"Also, although arrays and hashes are very useful, our real world data is often more complicated than these structures. We may want to store arrays of arrays, hashes of arrays, arrays of hashes, hashes of hashes. But arrays and hashes can only store scalar values.\n",
"\n",
"To take care of these problems, we can use references. All data and data structures have an address in memory. A reference is a Perl variable that refers to the memory address of some other variable. We never actually need to look at this address, we can just use the reference without thinking about what it points to explicitly. References are scalars, so we can use them to pass complex data structures into subroutines, by passing in references to these data structures. Similarly, to create an array of arrays, we create an array of references to other arrays.\n",
"\n",
"Just so you know what a reference looks like, let's print one out. Ideally, you will never see this in real program output, but in practice, it's not uncommon to accidentally print out some references because of some bug in your code, so it's worth just seeing what they look like."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"my @codons = ('ACG','GAG','GCG','CCC','AGT','GAA');\n",
"my $codon_ref = \\@codons;\n",
"print \"$codon_ref\\n\";\n",
"\n",
"my %geneticCode = ('TCT' => 'Serine',\n",
" 'TCC' => 'Serine',\n",
" 'TAG' => 'STOP',\n",
" );\n",
"\n",
"my $geneticCode_ref = \\%geneticCode;\n",
"print \"$geneticCode_ref\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"ARRAY(0x7ff95202dca0)\n",
"HASH(0x7ff952031e18)\n"
]
}
],
"prompt_number": 86
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see how we can use this to store an array reference as a hash value. Note that we are not copying the array; there is only one copy of the array, and when we change it, the hash will also see the change. We will use a module called `Data::Dumper` to print more complex data structures."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"use Data::Dumper;\n",
" \n",
"my @serine = ('TCA','TCC','TCG','TCT');\n",
"my @proline =('CCA','CCC','CCG','CCT');\n",
"\n",
"my %aas;\n",
"$aas{'serine'} = \\@serine;\n",
"$aas{'proline'} = \\@proline;\n",
"\n",
"print Dumper \\%aas;\n",
"\n",
"pop @serine;\n",
"shift @proline;\n",
"\n",
"print \"After array changes, the hash also changes\\n\";\n",
"print Dumper \\%aas;"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"$VAR1 = {\n",
" 'serine' => [\n",
" 'TCA',\n",
" 'TCC',\n",
" 'TCG',\n",
" 'TCT'\n",
" ],\n",
" 'proline' => [\n",
" 'CCA',\n",
" 'CCC',\n",
" 'CCG',\n",
" 'CCT'\n",
" ]\n",
" };\n",
"After array changes, the hash also changes\n",
"$VAR1 = {\n",
" 'serine' => [\n",
" 'TCA',\n",
" 'TCC',\n",
" 'TCG'\n",
" ],\n",
" 'proline' => [\n",
" 'CCC',\n",
" 'CCG',\n",
" 'CCT'\n",
" ]\n",
" };\n"
]
}
],
"prompt_number": 87
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the Dumper output, you can see that array elements are wrapped in square brackets `[]` and hash elements are wrapped in curly brackets `{}`. We can use these brackets directly to create anonymous arrays and anonymous hashes, which are references to arrays and hashes where the arrays and hashes themselves do not have names. This makes it easier to create multi-dimensional data structures.\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl -w\n",
"use Data::Dumper;\n",
"\n",
"my %aas = ('serine' => ['TCA','TCC','TCG','TCT'],\n",
" 'proline' => ['CCA','CCC','CCG','CCT'],\n",
" );\n",
"\n",
"my %codes;\n",
"$codes{'earth'} = \\%aas;\n",
"$codes{'mars'} = {'serine' => ['QWZ','QWX','QWQ','QWW'],\n",
" 'proline' => ['ZXZ','ZXX','ZXQ','ZXW'],\n",
" };\n",
"\n",
"print Dumper \\%codes;\n",
"\n",
"print \"$codes{'earth'}{'serine'}[0]\\n\";\n",
"print \"$codes{'mars'}{'proline'}[1]\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"$VAR1 = {\n",
" 'earth' => {\n",
" 'serine' => [\n",
" 'TCA',\n",
" 'TCC',\n",
" 'TCG',\n",
" 'TCT'\n",
" ],\n",
" 'proline' => [\n",
" 'CCA',\n",
" 'CCC',\n",
" 'CCG',\n",
" 'CCT'\n",
" ]\n",
" },\n",
" 'mars' => {\n",
" 'serine' => [\n",
" 'QWZ',\n",
" 'QWX',\n",
" 'QWQ',\n",
" 'QWW'\n",
" ],\n",
" 'proline' => [\n",
" 'ZXZ',\n",
" 'ZXX',\n",
" 'ZXQ',\n",
" 'ZXW'\n",
" ]\n",
" }\n",
" };\n",
"TCA\n",
"ZXX\n"
]
}
],
"prompt_number": 88
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note the subtle difference between creating the hash `%aa`, for which we use the normal brackets `()` as we have before, and creating the hash reference `$codes{'mars'}`, for which we use the curly brackets `{}`. Once we have created our multi-dimensional structure, we can access elements of it by chaining the hash and array indices. You will sometimes see people using arrows to make it clear we are chaining references (eg `$codes->{'earth'}->{'serine'}->[0]`), but this is not strictly necessary.\n",
"\n",
"We have seen how to create a reference for a data structure, but not how to get the data structure from a reference - how to dereference. This can be done by wrapping the reference in curly brackets."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"\n",
"my $mars_ref = {'serine' => ['QWZ','QWX','QWQ','QWW'],\n",
" 'proline' => ['ZXZ','ZXX','ZXQ','ZXW'],\n",
" };\n",
"\n",
"my @aas = keys %{$mars_ref};\n",
"\n",
"print \"@aas\\n\";"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"serine proline\n"
]
}
],
"prompt_number": 89
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, here's how we can use references to pass complex data structures in to a subroutine and get multiple values out."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%perl\n",
"use Data::Dumper;\n",
" \n",
"sub means {\n",
" my @means;\n",
" for my $array_ref (@_) {\n",
" my $length = @{$array_ref};\n",
" my $total = 0;\n",
" for my $num (@{$array_ref}) {\n",
" $total += $num;\n",
" }\n",
" push @means, $total / $length;\n",
" }\n",
" return @means;\n",
"}\n",
"\n",
"my @mean_out = means([1,2,3],[4,5,6],[7,8,9]);\n",
"\n",
"print Dumper \\@mean_out;"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"$VAR1 = [\n",
" '2',\n",
" '5',\n",
" '8'\n",
" ];\n"
]
}
],
"prompt_number": 90
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `means` subroutine takes a list of array references as input, with the references pointing to anonymous arrays, defined with square brackets. The subroutine itself loops through the array references and calculates the mean of each array. It then returns the `@means` array, which we then copy into our `@mean_out` output array."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Exercises\n",
"\n",
"1. Write a subroutine that takes the full `%codes` hash defined above as input, identifies the unique bases of each code from the given codons, and returns two arrays, one containing the four bases of the earth genetic code, and one containing the four bases of the mars genetic code.\n",
"\n",
"HINTS: use multiple for loops to iterate through the hash of hashes of arrays and split up each codon by base. You will need to do multiple dereferences to get at the inner hashes and arrays. To find the unique bases, see the answer to Lesson 14 Exercise II."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
}
],
"metadata": {}
}
]
}