{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Direct Methods for Solving Linear Systems\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Motivating example\n",
    "\n",
    "Find a quadratic polynomial $p(x)$ such that $p(-1) = 0$, $p(1) = 0$ and $p'(0) = 0$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let $p(x) = a x^2 + b x + c$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Then we have to solve the following equations\n",
    "\n",
    "$$a - b + c = 0\\\\\n",
    "a + b + c  =0\\\\\n",
    "0 + b + 0 = 0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "To solve this linear system we perform row operations: \n",
    "\n",
    "$$\\begin{array}{cccc} \n",
    "a - b + c = 0 & &a - b + c = 0\\\\\n",
    "a + b + c  =0 & R_2 \\leftarrow R_2- R_1  &0 + 2b + 0  =0\\\\\n",
    "0 + b + 0 = 0 && 0 + b + 0 = 0\\\\ \\end{array}$$\n",
    "\n",
    "The row operation $R_2 \\leftarrow R_2- R_1 $ states that we should subtract row 1 from row 2, and put whatever we find in row 2.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The last two equations give $b = 0$, and then the top equation gives $c = -a$.  So, letting $a = \\alpha \\in \\mathbb R$, the polynomial $p(x)$ is given by:\n",
    "\n",
    "$$ p(x) = \\alpha(x^2 -1).$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Linear Algebra \n",
    "\n",
    "We develop the proper definitions so that we can write the above system in matrix form as\n",
    "\n",
    "$$\\begin{bmatrix} 1 & -1 & 1\\\\ 1 & 1 & 1 \\\\ 0 & 1 & 0  \\end{bmatrix}\\begin{bmatrix} a \\\\ b \\\\ c \\end{bmatrix} =  \\begin{bmatrix} 0 \\\\ 0 \\\\ 0 \\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Definition\n",
    "\n",
    "A __matrix__ is an $n \\times m$ array of real (or complex) numbers, where ordering matters.  Here $n$ refers to the number of rows in the matrix and $m$ is the number of columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For an $n \\times m$ matrix $A$ we use $a_{ij}$ to refer to the element that is in the $i$th row and $j$th column.  Rows and columns are counted from the top left entry.  We use the notation\n",
    "\n",
    "$$A = (a_{ij})_{1 \\leq i \\leq n, 1 \\leq j \\leq m}$$\n",
    "\n",
    "to refer to the the matrix $A$ by its entires.  We also use $A = (a_{ij})$ when $n$ and $m$ are implied."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The **diagonal** entries of a matrix $A$ are $a_{ii}$ for $1 \\leq i \\leq \\min\\{m,n\\}$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "A matrix $A$ is __triangular__ if either $a_{ij} = 0$ for $i < j$ (__lower triangular__) or $a_{ij} = 0$ for $j < i$ (__upper triangular__).  If a matrix is both lower and upper triangular it is said to be __diagonal__."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "A __column vector__ is an $n \\times 1$ matrix.\n",
    "\n",
    "A __row vector__ is an $1 \\times m$ matrix.\n",
    "\n",
    "If $x$ is is either a row or column vector, we use $x_i$ ($1 \\leq i \\leq n$ or $1 \\leq i \\leq m$, resp.) to refer to its entries.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Definition (matrix-vector multiplication)\n",
    "\n",
    "Let $A = (a_{ij})$ be an $n \\times m$ matrix and let $x$ be a $m \\times 1$ column vector.  The product $y = Ax$ is a $n \\times 1$ vector $y$ given by\n",
    "\n",
    "$$ y_i = \\sum_{j=1}^m a_{ij} x_j, \\quad 1 \\leq i \\leq n.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "With this notation, the following linear system of equations\n",
    "\n",
    "$$a_{11} x_1 + a_{12} x_2 + \\cdots + a_{1n} x_n = y_1\\\\\n",
    "a_{21} x_1 + a_{22} x_2 + \\cdots + a_{2n} x_n = y_2\\\\\n",
    "\\vdots ~~~~~~~~~~~~~~~ \\vdots\\\\\n",
    "a_{n1} x_1 + a_{n2} x_2 + \\cdots + a_{nn} x_n = y_n$$\n",
    "\n",
    "is equivalent to $Ax = y$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Matrix multiplication in `numpy`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A = \n",
      "[[1 2 3]\n",
      " [4 5 6]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "A = np.array([[1,2,3], [4,5,6]]) \n",
    "print 'A = '\n",
    "print A "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = \n",
      "[[1]\n",
      " [1]\n",
      " [1]]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([[1], [1], [1]])\n",
    "print 'x = '\n",
    "print x "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ax = \n",
      "[[ 6]\n",
      " [15]]\n"
     ]
    }
   ],
   "source": [
    "print 'Ax = ' \n",
    "print np.dot(A, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Gaussian elimination with backward substitution\n",
    "\n",
    "The goal of Gaussian elimination is to solve a system of equations by performing what are called __elementary row operations__ to reduce a general matrix to an upper-triangular matrix.  Then a procedure called __backward substitution__ applies to the upper-triangular matrix.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Elementary row operations are:\n",
    "1. Multiply any row by a non-zero constant, $R_1 \\leftarrow c R_1 $\n",
    "2. Given a row, any multiple of another row can be added, replacing the given row, $R_2 \\leftarrow R_2 - 3 R_1 $\n",
    "3. Interchange two rows, $R_1 \\leftrightarrow R_2$"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}