{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gaussian Elimination"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function performs LU decomposition of given matrix A."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def LU(A):\n",
    "    n = A.shape[0]\n",
    "    L = np.identity(n)\n",
    "    U = np.array(A)\n",
    "    for i in range(n-1):\n",
    "        L[i+1:n,i] = U[i+1:n,i]/U[i,i]\n",
    "        for j in range(i+1,n):\n",
    "            U[j,i+1:n] = U[j,i+1:n] - L[j,i] * U[i,i+1:n]\n",
    "        U[i+1:n,i] = 0.0\n",
    "    return L,U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This performs performs solution of\n",
    "$$\n",
    "LUx=b\n",
    "$$\n",
    "in two steps. In first step, solve\n",
    "$$\n",
    "Ly = b\n",
    "$$\n",
    "using\n",
    "$$\n",
    "y_i = \\frac{1}{L_{ii}} \\left[b_i - \\sum_{j=0}^{i-1} L_{ij} y_j\\right], \\qquad i=0,1,\\ldots,n-1\n",
    "$$\n",
    "Note that $L_{ii}=1$, so we can drop the division step. In second step, solve\n",
    "$$\n",
    "Ux = y\n",
    "$$\n",
    "using\n",
    "$$\n",
    "x_i = \\frac{1}{U_{ii}}\\left[y_i - \\sum_{j=i+1}^{n-1} U_{ij} x_j \\right], \\qquad i=n-1,n-2,\\ldots,0\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def LUSolve(L,U,b):\n",
    "    n = L.shape[0]\n",
    "    # solve Ly = b\n",
    "    y = np.empty_like(b)\n",
    "    for i in range(n):\n",
    "        y[i] = b[i] - L[i,0:i].dot(y[0:i])\n",
    "    # solve Ux = y\n",
    "    x = np.empty_like(b)\n",
    "    for i in range(n-1,-1,-1):\n",
    "        x[i] = (y[i] - U[i,i+1:n].dot(x[i+1:n]))/U[i,i]\n",
    "    return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we test the above function for LU decomposition. We initialize $A$ to a random matrix and compute its LU decomposition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A =\n",
      " [[0.2567139  0.41304159 0.45543895]\n",
      " [0.69575772 0.78135014 0.60077887]\n",
      " [0.0358914  0.72083532 0.88130568]]\n",
      "L =\n",
      " [[ 1.          0.          0.        ]\n",
      " [ 2.71024564  1.          0.        ]\n",
      " [ 0.13981091 -1.96125209  1.        ]]\n",
      "U =\n",
      " [[ 0.2567139   0.41304159  0.45543895]\n",
      " [ 0.         -0.33809402 -0.63357257]\n",
      " [ 0.          0.         -0.42496519]]\n",
      "[[ 0.00000000e+00  0.00000000e+00  0.00000000e+00]\n",
      " [ 0.00000000e+00  0.00000000e+00  0.00000000e+00]\n",
      " [ 0.00000000e+00  0.00000000e+00 -1.11022302e-16]]\n"
     ]
    }
   ],
   "source": [
    "n = 3\n",
    "A = np.random.rand(n,n)\n",
    "L,U = LU(A)\n",
    "print(\"A =\\n\",A)\n",
    "print(\"L =\\n\",L)\n",
    "print(\"U =\\n\",U)\n",
    "print(A - L.dot(U))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solve the linear system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00000000e+00 1.11022302e-16 0.00000000e+00]\n"
     ]
    }
   ],
   "source": [
    "b = np.random.rand(n)\n",
    "x = LUSolve(L,U,b)\n",
    "print(A.dot(x) - b)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}