{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "ChEn-3170: Computational Methods in Chemical Engineering Spring 2020 UMass Lowell; Prof. V. F. de Almeida **28Jan20**\n", "\n", "# 06. Computational Linear Algebra Fundamentals\n", "$ \n", " \\newcommand{\\Amtrx}{\\boldsymbol{\\mathsf{A}}}\n", " \\newcommand{\\Bmtrx}{\\boldsymbol{\\mathsf{B}}}\n", " \\newcommand{\\Cmtrx}{\\boldsymbol{\\mathsf{C}}}\n", " \\newcommand{\\Dmtrx}{\\boldsymbol{\\mathsf{D}}}\n", " \\newcommand{\\Mmtrx}{\\boldsymbol{\\mathsf{M}}}\n", " \\newcommand{\\Imtrx}{\\boldsymbol{\\mathsf{I}}}\n", " \\newcommand{\\Pmtrx}{\\boldsymbol{\\mathsf{P}}}\n", " \\newcommand{\\Qmtrx}{\\boldsymbol{\\mathsf{Q}}}\n", " \\newcommand{\\Lmtrx}{\\boldsymbol{\\mathsf{L}}}\n", " \\newcommand{\\Umtrx}{\\boldsymbol{\\mathsf{U}}}\n", " \\newcommand{\\xvec}{\\boldsymbol{\\mathsf{x}}}\n", " \\newcommand{\\avec}{\\boldsymbol{\\mathsf{a}}}\n", " \\newcommand{\\bvec}{\\boldsymbol{\\mathsf{b}}}\n", " \\newcommand{\\cvec}{\\boldsymbol{\\mathsf{c}}}\n", " \\newcommand{\\rvec}{\\boldsymbol{\\mathsf{r}}}\n", " \\newcommand{\\norm}[1]{\\bigl\\lVert{#1}\\bigr\\rVert}\n", " \\DeclareMathOperator{\\rank}{rank}\n", "$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Table of Contents\n", "* [Objectives](#obj)\n", "* [Theory](#theory)\n", "* [Matrix-vector and matrix-matrix product operations](#product)\n", "* [NumPy and SciPy Linear Algebra](#pylinalg)\n", " + [Matrix solve](#pysolve)\n", " + [$\\Lmtrx$ forward solve](#pyl)\n", " + [$\\Umtrx$ backward solve](#pyu)\n", " + [$\\Amtrx = \\Pmtrx\\,\\Lmtrx\\,\\Umtrx$ factorization](#pyplu)\n", "* [Course Linear Algebra](#courselinalg)\n", " + [$\\Lmtrx$ forward solve](#l)\n", " + [$\\Umtrx$ backward solve](#u)\n", " + [$\\Amtrx = \\Lmtrx\\,\\Umtrx$ factorization](#lu)\n", " + [$\\Pmtrx\\,\\Amtrx = \\Lmtrx\\,\\Umtrx$ factorization](#plu)\n", " + [$\\Pmtrx\\,\\Amtrx\\,\\Qmtrx = \\Lmtrx\\,\\Umtrx$ factorization](#pqlu)\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Objectives\n", "\n", " + Introduce the elements of computational linear algebra needed in this course to analyse and solve system of linear algebraic equations.\n", " + Implement a direct method of solution of linear algebraic equations (also known as matrix factorization)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "The course notes (OneNote [ChEn-3170-linalg](https://studentuml-my.sharepoint.com/:o:/g/personal/valmor_dealmeida_uml_edu/EkHM3pyx7T9JpikMFFET9XsBCu5gwmKdo7AMeoYqAq5utw?e=dZerTZ) cover basic elements of linear system of algebraic equations as applied to computational stoichiometry. Particular attention is given to conditions for the existance and uniqueness of solutions of general algebraic systems.\n", "\n", "Basic theoretical aspects of solving for $\\overset{(n)}{\\xvec}$ in the matrix equation $\\overset{(m\\times n)}{\\Amtrx}\\,\\overset{(n)}{\\xvec} = \\overset{(m)}{\\bvec}$ are covered. $\\overset{(m\\times n)}{\\Amtrx}$ is a matrix, $\\overset{(m)}{\\bvec}$ and $\\overset{(n)}{\\xvec}$ are vectors where $m$ indicates the number of rows (or equations) and $n$ number of columns (or unknowns)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrix-vector and matrix-matrix product operations\n", "The following operations between vectors and matrices are obtained directly from the buil-in functions in the `numpy` package." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "'''Import the NumPy package as usual'''\n", "\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Inner product of two vectors: $\\avec \\cdot \\bvec$." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a.b = 0.6381292212080929\n", "a@b = 0.6381292212080929\n" ] } ], "source": [ "'''Vector inner product or dot product of vectors'''\n", "\n", "a_vec = np.array( np.random.random(3) )\n", "b_vec = np.array( np.random.random(3) )\n", "\n", "np.set_printoptions( precision=3, threshold=20, edgeitems=12, linewidth=100 )\n", "\n", "a_vec_dot_b_vec = np.dot( a_vec, b_vec ) # clear linear algebra operation\n", "print('a.b =', a_vec_dot_b_vec)\n", "\n", "a_vec_x_b_vec = a_vec @ b_vec # consistent linear algebra multiplication\n", "print('a@b =', a_vec_x_b_vec )" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a.b = 6.3813e-01\n" ] } ], "source": [ "print( 'a.b = %10.4e'%a_vec_dot_b_vec ) # formatting with scientific notation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Matrix vector product: $\\Amtrx\\,\\bvec$." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A b = [17. 32. -6.]\n" ] } ], "source": [ "'''Matrix-vector product'''\n", "\n", "a_mtrx = np.array( [ [ 2., 1., 1.], # per course notes (NB 03/04)\n", " [ 4., -6., 0.],\n", " [-2., 7., 2.] \n", " ] )\n", "\n", "b_vec = np.array( [5., -2., 9.] ) # per course notes\n", "\n", "a_mtrx_x_b_vec = a_mtrx @ b_vec # linear algebra matrix-vector product\n", "\n", "print('A b =', a_mtrx_x_b_vec)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Matrix-vector product: $\\Imtrx\\,\\bvec = \\bvec$. Note: $\\begin{pmatrix}\n", "1 & 0 & 0 \\\\\n", "0 & 1 & 0 \\\\\n", "0 & 0 & 1\n", "\\end{pmatrix} \\, \\begin{pmatrix} b_1\\\\b_2\\\\b_3 \\end{pmatrix} = \\begin{pmatrix} b_1\\\\b_2\\\\b_3 \\end{pmatrix} $." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "b = [ 5. -2. 9.]\n", "I x b = [ 5. -2. 9.]\n" ] } ], "source": [ "'''Identity-matrix vector product'''\n", "\n", "i_mtrx = np.eye(3)\n", "\n", "i_mtrx_x_b_vec = i_mtrx @ b_vec # linear algebra matrix-vector product\n", "\n", "print('b =', b_vec)\n", "print('I x b =', i_mtrx_x_b_vec)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Matrix-matrix product: $\\Imtrx\\,\\Amtrx = \\Amtrx$. Note: $\\begin{pmatrix}\n", "1 & 0 & 0 \\\\\n", "0 & 1 & 0 \\\\\n", "0 & 0 & 1\n", "\\end{pmatrix} \\, \n", "\\begin{pmatrix} \n", "A_{1,1} & A_{1,2} & A_{1,3} \\\\\n", "A_{2,1} & A_{2,2} & A_{2,3} \\\\\n", "A_{3,1} & A_{3,2} & A_{3,3}\n", "\\end{pmatrix} = \n", "\\begin{pmatrix} \n", "A_{1,1} & A_{1,2} & A_{1,3} \\\\\n", "A_{2,1} & A_{2,2} & A_{2,3} \\\\\n", "A_{3,1} & A_{3,2} & A_{3,3}\n", "\\end{pmatrix}\n", "$." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "I x A =\n", " [[ 2. 1. 1.]\n", " [ 4. -6. 0.]\n", " [-2. 7. 2.]]\n", "A =\n", " [[ 2. 1. 1.]\n", " [ 4. -6. 0.]\n", " [-2. 7. 2.]]\n" ] } ], "source": [ "'''Matrix-matrix product IA = A'''\n", "\n", "i_mtrx_x_a_mtrx = i_mtrx @ a_mtrx # linear algebra matrix-matrix product\n", "\n", "print('I x A =\\n', i_mtrx_x_a_mtrx)\n", "print('A =\\n', a_mtrx)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Matrix-matrix product: $\\Amtrx\\,\\Bmtrx = \\Cmtrx$. Note: \n", "$\\begin{pmatrix}\n", "A_{1,1} & A_{1,2} & A_{1,3} \\\\\n", "A_{2,1} & A_{2,2} & A_{2,3} \\\\\n", "A_{3,1} & A_{3,2} & A_{3,3}\n", "\\end{pmatrix} \\, \n", "\\begin{pmatrix} \n", "B_{1,1} & B_{1,2} & B_{1,3} \\\\\n", "B_{2,1} & B_{2,2} & B_{2,3} \\\\\n", "B_{3,1} & B_{3,2} & B_{3,3}\n", "\\end{pmatrix} = \n", "\\begin{pmatrix} \n", "C_{1,1} & C_{1,2} & C_{1,3} \\\\\n", "C_{2,1} & C_{2,2} & C_{2,3} \\\\\n", "C_{3,1} & C_{3,2} & C_{3,3}\n", "\\end{pmatrix}\n", "$ where each $C_{i,j}$ is a vector product of the $i$th row of $\\Amtrx$ and the $j$th column of $\\Bmtrx$, *i.e.* \n", "$C_{i,j} = \\sum\\limits_{k=1}^3 A_{i,k}\\, B_{k,j}$." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A =\n", " [[ 2. 1. 1.]\n", " [ 4. -6. 0.]\n", " [-2. 7. 2.]]\n", "B =\n", " [[ 5. 5. 5.]\n", " [-2. -2. -2.]\n", " [ 9. 9. 9.]]\n", "C =\n", " [[17. 17. 17.]\n", " [32. 32. 32.]\n", " [-6. -6. -6.]]\n" ] } ], "source": [ "'''Matrix-matrix product AB = C'''\n", "\n", "b_mtrx = np.array( [ [ 5. , 5. , 5.],\n", " [-2. , -2. , -2.],\n", " [ 9. , 9. , 9.] ]\n", " )\n", "c_mtrx = a_mtrx @ b_mtrx # linear algebra matrix-matrix product\n", "\n", "print('A =\\n', a_mtrx)\n", "print('B =\\n', b_mtrx)\n", "print('C =\\n', c_mtrx)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "