{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Course 6: More on list like objects and iteration\n",
"=================================================\n",
"\n",
"Slicing\n",
"-------\n",
"\n",
"This slicing operation applies to \"list like\" Python object (`list`, `str`, `numpy.array`, etc). Here is an example.\n",
"```python\n",
"l = [19, 12, 24, -61, 15, 18, 13, -81, -33, 42]\n",
"l[1:5] # sublist made of the elements at position 1, 2, 3, 4\n",
"```\n",
"The general syntaxes are the following\n",
"- `l[start:stop]`: the sublist made of elements from `start` (included) to `stop` (not included)\n",
"- `l[start:stop:step]`: the sublist made of elements `start`, `start + step`, `start + 2step`, up to `stop` (not included)\n",
"\n",
"These are the same conventions as with the `range` and `np.arange` functions.\n",
"\n",
"**Exercise:**\n",
"- Try the above code.\n",
"- How do we get the sublist made of the elements `l[3]`, `l[4]` and `l[5]`?\n",
"- What is the code to make a copy of the whole list `l`?\n",
"- Could you reproduce the following output using the list `l` and a `while` loop?\n",
"```python\n",
"[19, 12, 24, -61, 15, 18, 13, -81, -33, 42]\n",
"[12, 24, -61, 15, 18, 13, -81, -33]\n",
"[24, -61, 15, 18, 13, -81]\n",
"[-61, 15, 18, 13]\n",
"[15, 18]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:**\n",
"- Construct the list of the first 100th Fibonacci numbers\n",
"- Using a slice, make the list of the terms $F_{2n}$\n",
"- Using a slice, make the list of the terms $F_{2n+1}$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:**\n",
"- Let `s` be the following string\n",
"```python\n",
"s = 'ilagiK ni evil I'\n",
"```\n",
"Could you find a command that returns the mirror of `s`? That is the string made of the letters `s[-1]`, `s[-2]`, `s[-3]` up to `s[-len(s)+1]`.\n",
"- Let `l` be the following list of strings\n",
"```python\n",
"l = ['amor', 'bridge', 'eye', 'civic', 'march', 'blinis', 'kayak',\n",
" 'level', 'mirror', 'radar', 'palindrome', 'redder', 'word',\n",
" 'stats', 'string', 'wow', 'road', 'zouk', 'scale', 'tree']\n",
"```\n",
"Using a list comprehension construct a list that is made of the elements of `l` that are palindromes (words that are equal to their mirror as `stats` or `bob`)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:**\n",
"- You remember about the function `range`? What does it do?\n",
"- Let `r` be defined as follow\n",
"```python\n",
"r = range(3, 41, 2)\n",
"```\n",
"- What do you see if you do `print(r)`? `len(r)`? `r[0]`? `r[-1]`?\n",
"- Try various slicing operation on `r`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pattern matching\n",
"----------------\n",
"\n",
"Pattern matching is a way of simplifying affectation of several variables. For example if you have a list `l` of length 3 you might want to assign the variables `a`, `b` and `c` to be these three elements as in\n",
"```python\n",
"a = l[0]\n",
"b = l[1]\n",
"c = l[2]\n",
"```\n",
"The shortcut to perform this operation is\n",
"```python\n",
"a,b,c = l\n",
"```\n",
"The above is called a pattern matching (since the left part describes the structure of the right part).\n",
"\n",
"It can be applied recursively if you have list of lists\n",
"```python\n",
"l = [[1,2], [3,4]]\n",
"(a,b), (c,d) = l # sets a=1, b=2, c=3 and d=4\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Copy/paste the code above\n",
"- Let `l` be the following list \n",
"```python\n",
"l = [[1,2,3], [-1,5], 19]\n",
"```\n",
"Use one pattern matching to set `a=1`, `b=2`, `c=3`, `d=-1`, `e=5` and `f=19` using only one line."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pattern matching is particularly convenient when doing iterations. For example if you have a list of pairs and want to iterate through the pairs using the variable `x` for the first coordinate and `y` for the second you can do\n",
"```python\n",
"l = [[1,2], [3,4], [5,6]]\n",
"s = 0\n",
"for (x,y) in l:\n",
" s = s + x*y\n",
"```\n",
"\n",
"**Exercise::**\n",
"- What is computing the above code?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:**\n",
"- Given two variables `x` and `y` could you swap their values using pattern matching (1 line of code)? In this question you assume that `x` and `y` are already defined. Their assignment do not count in the swap operation.\n",
"- Could you swap their values without using pattern matching (3 lines of code)?\n",
"- Using the swap with pattern matching implement a 4 lines code that prints the values of the first 30 Fibonacci numbers (without using a list)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Operations on `numpy` arrays\n",
"----------------------------\n",
"\n",
"For many problems using `numpy` would result in a much faster solution. One reason is that the data in a numpy array are homogeneous (each entry has the same type). You can try the following timing comparison\n",
"```python\n",
"%%timeit\n",
"a = np.arange(100) ** 2\n",
"```\n",
"versus\n",
"```python\n",
"%%timeit\n",
"b = [i**2 for i in range(100)]\n",
"```\n",
"(on my computer, the first one is x20 faster)\n",
"\n",
"The first operation that you know about is the sum of two arrays. This operations is done entrywise (i.e. vector addition). The same behavior holds for `*` or `/` or `//` as in\n",
"However, array/scalar operations are also allowed as in the following.\n",
"```python\n",
"a1 = np.array([1,2,5,3,4,1,2,2,5,2,4])\n",
"a2 = np.array([3,3,5,1,2,1,2,1,1,2,4])\n",
"a_add = a1 + a2\n",
"a_sub = a1 - a2\n",
"a_mul = a1 * a2\n",
"a_div = a1 / a2\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Copy/paste the above code and print the values of `a_add`, `a_sub`, `a_mul` and `a_div`.\n",
"- What would happen if `a1` and `a2` were lists and not numpy array?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With numpy array it is also possible to do array/scalar operations. The result is an array of the same shape with the scalar operation performed on each entry.\n",
"```python\n",
"a = np.array([1,2,5,3,4,1,2,2,5,2,4])\n",
"a_p1 = a + 1 # this is the array [2, 3, 6, 4, 5, 2, ...]\n",
"a_square = a ** 2 # this is the array [1, 4, 25, 9, 16, ...]\n",
"```\n",
"**Exercise:** Using numpy arrays can you compute the sum of cubes of all numbers between 1 and 100? (*hint: numpy arrays have a method sum*)\n",
"\n",
"\n",
"**Remark:** Recall that in Python (and any object oriented language) there is a distinction between *functions* and *methods*. Examples of functions are `print`, `len` and `math.cos`. Methods are attached to objects like `append` and `pop` to list or `replace` to string. Functions and methods can easily be distinguished from the syntax used\n",
"```python\n",
"l = [1,2,5,2]\n",
"print(l) # print is a function\n",
"l.append(5) # append is a method only available through the list l\n",
"```\n",
"It is often the case that the implementation of methods are more specialized and hence more efficient. You can try the following timings\n",
"```python\n",
"%%timeit\n",
"sum(np.arange(1000))\n",
"```\n",
"versus\n",
"```python\n",
"%%timeit\n",
"np.arange(1000).sum())\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Be careful that comparisons are particular cases of array/array and array/scalar operations:\n",
"```python\n",
"a1 = np.array([1,2,5,3,4,1,2,2,5,2,4])\n",
"a2 = np.array([3,3,5,1,2,1,2,1,1,2,4])\n",
"b1 = a1 < 3 # the boolean array [True, True, False, False, False, True ...]\n",
"b2 = a1 < a2 # the boolean array [True, True, False, False, False, False, ...]\n",
"```\n",
"**Exercise:**\n",
"- Copy/paste the code above and print the value of `b1` and `b2`.\n",
"- Construct two lists `l1` and `l2` that contain the same elements as `a1` and `a2`\n",
"- How can you obtain the same result as `a1 < 3` using the list `l1`?\n",
"- How can you obtain the same result as `a1 < a2` using the lists `l1` and `l2`?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**WARNING:** If you want to test the equality of two arrays you can not use equality for that purpose!\n",
"```python\n",
"np.array([1,2,1]) == np.aray([1,1,1]) # result is array([True, False, True])\n",
"```\n",
"To test the equality of arrays you need to use `np.array_equal` as in\n",
"```python\n",
"a1 = np.array([1,2,1])\n",
"a2 = np.array([1,1,2])\n",
"np.array_equal(a1,a2)\n",
"```\n",
"**Exercise:**\n",
"- Try the above code"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Indexing numpy arrays\n",
"---------------------\n",
"It is possible to index numpy array with integers (to access elements) or slice (to access subarrays) in the same way as lists.\n",
"```python\n",
"a = np.arange(0,100,2)\n",
"e1 = a[0] # the first element of the array\n",
"e2 = a[-1] # the last element of the array\n",
"b = a[3:30] # the subarray of elements at position from 3 to 29 included\n",
"```\n",
"**Exercise:**\n",
"- Copy/paste/execute the above code and print the values of `e1`, `e2` and `b`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Numpy arrays support two very useful other kinds of indexing using other arrays. The first one is with an array of indices as in\n",
"```python\n",
"a = np.array([1, 3, 5, 2, 4])\n",
"indices = np.array([0, 3, 4])\n",
"b = a[indices] # elements at position 0, 3 and 4\n",
"```\n",
"When using this operation the array `indices` does not need have the same length as `a`. It can even have repeated elements.\n",
"\n",
"The second way is with an array of booleans of the same shape.\n",
"```python\n",
"a = np.array([1, 3, 5, 2, 4])\n",
"condition = np.array([True,False,False,True,True])\n",
"c = a[condition] # elements positions where we have True\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Let `a` be the following array\n",
"```python\n",
"a = np.array([1, 3, 6, 2, 4, 1, 5, 10, 2, 34, 2, 1, 5, 6, 2, 1])\n",
"```\n",
"- Construct the sub-array of `a` that consists of the elements at position 0, 4, 5, 7, 9 (you are not allowed to use for loops here).\n",
"- Construct the sub-array of `a` that consists of the elements smaller than 5 (you are not allowed to use loops here).\n",
"- Construct a list `l` that contains the same elements as `a`\n",
"- What happens if you try this kind of indexing with lists?\n",
"- Could you make for loops that perform the same operations as these numpy array indexing?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Numpy arrays also support a syntax for setting sub-arrays to a fixed value\n",
"```python\n",
"a = np.ones(50, dtype=bool)\n",
"a[:2] = False # sets the elements at position 0 and 1 to False\n",
"a[4::2] = False # sets the elements at position 4, 6, 8, 10, 12, ... to False\n",
"a[9::3] = False # sets the elements at position 9, 12, 15, 18, ... to False\n",
"a[25::5] = False # sets the elements at position 25, 30, 35, 40, ... to False\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Copy/paste/execute the code above\n",
"- Prints the indices of elements that are set to `True` (*hint: use the nonzero method*). \n",
"- What are the numbers that you see?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Multi-dimensional numpy arrays\n",
"------------------------------\n",
"Numpy arrays can also handle multi-dimensional arrays. For example matrices are naturally 2d arrays. To make the parallel with Python lists, a list of integers would correspond to a 1-dimensional array, a list of list of integers would correspond to a 2-dimension array and so on.\n",
"\n",
"Given a numpy array, you can access to its shape with its *attribute* shape. Be careful that attributes are objects in themselves (not functions). You should not use parenthesis for them.\n",
"```python\n",
"a = np.array([[[0,1,0], [2,3,1], [1,1,1], [2,1,5]], [[4,5,4], [6,7,2], [8,7,6], [4,4,4]]])\n",
"print(a.shape)\n",
"```\n",
"In memory the above array is actually contiguous containing `0, 1, 0, 2, 3, 1, 1, etc`. As a consequence you can easily reshape a given array (as soon as it contains the same number of elements)\n",
"```python\n",
"b1 = a.reshape((24,)) # an 1d array with 24 elements\n",
"b2 = a.reshape((2,12)) # a 2d array with 2 rows of 12 elements\n",
"b3 = a.reshape((2,3,4)) # a 3d array of shape (2,3,4)\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Execute the above codes and print the values of `a`, `b1`, `b2`, `b3` together with their shapes\n",
"- USe the method `reshape` as above to construct an array equal to `a` starting from the following\n",
"```python\n",
"c = np.array([0,1,0,2,3,1,1,1,1,2,1,5,4,5,4,6,7,2,8,7,6,4,4,4])\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To access the elements of a (multidimensional) numpy array `a` there is a special syntax available.\n",
"```python\n",
"a = np.array([[0,1,0], [2,3,1], [1,1,1], [2,1,5]])\n",
"e = a[0,3] # element at position (0,3)\n",
"```\n",
"This also works when you want to consider slices\n",
"```python\n",
"r = a[0, :] # first row\n",
"c = a[:, 1] # second column\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Execute the above code and print the values of `a`, `e`, `r`, `c`.\n",
"- What happens if you try this syntax with lists?\n",
"- Could you find the comand to get the array consisting of the two first rows of `a`?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Many numpy methods operate on a given dimension of the array (i.e. either the first indices or the second indices). Try the following code\n",
"```python\n",
"a = np.array([[0,1,0], [2,3,1], [1,1,1], [2,1,5]])\n",
"print(a.sum()) # sum of all elements\n",
"print(a.sum(0)) # sums of rows\n",
"print(a.sum(1)) # sums of columns\n",
"```\n",
"**Exercise:**\n",
"- Could you find the method to compute the product of the elements of a numpy array?\n",
"- Using this method for product compute the following quantity\n",
"$$\n",
"(1+2+3)*(4+5+6)*(7+8+9)*(10+11+12)*(13+14+15)*(16+17+18)\n",
"$$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:** What is doing the code below\n",
"```python\n",
"import matplotlib.pyplot as plt\n",
"from scipy import stats\n",
"plt.hist(stats.uniform(0,1).rvs(50000).reshape(500,100).sum(0))\n",
"plt.show()\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Linear algebra with numpy\n",
"-------------------------\n",
"\n",
"The numpy library also comes with functions to deal with matrices. Recall that a matrix is nothing more than a 2-dimensional array. But as we saw the product of two arrays is an entrywise operation\n",
"```python\n",
"a = np.array([[1,2],[3,4]])\n",
"a*a # not matrix multiplication!\n",
"```\n",
"In order to construct a matrix from `a` you need to do\n",
"```python\n",
"b = np.matrix(a)\n",
"```\n",
"The object `b` is an object of different type. When doing the product it is now matrix product\n",
"```python\n",
"b*b # this is matrix multiplication\n",
"```\n",
"\n",
"**Exercise:**\n",
"- Try the above code\n",
"- Construct two 2-dimensional numpy arrays `a1` and `a2` with shapes respectively $2 \\times 4$ and $4 \\times 5$.\n",
"- Transform them into matrices `m1` and `m2` as explained above.\n",
"- Compute the product `m1 * m2`. What is its shape?\n",
"- What would happen if we try to compute the product `a1 * a2`?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The numpy library comes with many primitive to solve linear systems, computing matrix decomposition, computing eigenvalues and eigenvectors, etc. You are invited to have a look at the documentation available on internet if you are interested with linear algebra. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (C) 2016 Vincent Delecroix <vincent.delecroix@u-bordeaux.fr>\n",
"\n",
"This work is licensed under a Creative Commons Attribution-NonCommercial 4.0\n",
"International License (CC BY-NC-SA 4.0). You can either read the\n",
"[Creative Commons Deed](https://creativecommons.org/licenses/by-nc-sa/4.0/)\n",
"(Summary) or the [Legal Code](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode)\n",
"(Full licence)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 1
}