{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Column Space and Null Space (18.06_L6)\n",
    "> Linear Algebra - Introduction to columns space and nullspace\n",
    "\n",
    "- toc: true \n",
    "- badges: true\n",
    "- comments: true\n",
    "- author: Isaac Flath\n",
    "- categories: [Linear Algebra]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Background\n",
    "\n",
    "We started on this topic in the last post, so much this section is review - but critical for the upcoming discussion.\n",
    "\n",
    "## Vector Spaces\n",
    "\n",
    "To have a vector space you need to be able to:\n",
    "\n",
    "1. Add any vectors and still be within the space\n",
    "1. Multiply by any number and stay within space\n",
    "1. Take any linear combination and stay within space\n",
    "\n",
    "## Examples\n",
    "\n",
    "$R^2$ is a vector space.  This is all real number vectors with 2 components.  Here's a few examples.\n",
    "\n",
    "$\\begin{bmatrix}3\\\\2\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}0\\\\0\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}\\pi\\\\e\\end{bmatrix}$\n",
    "\n",
    "$R^3$ is another vector space, but it contains all real 3D vectors.  Here's a few examples.\n",
    "\n",
    "$\\begin{bmatrix}3\\\\2\\\\5\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}3\\\\2\\\\0\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}0\\\\0\\\\0\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}\\pi\\\\e\\\\5\\end{bmatrix}$\n",
    "\n",
    "$R^n$ is another with all column vectors with $n$ real components."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Subspaces\n",
    "\n",
    "What if we just want a piece of $R^n$?  We still need to be able to add vectors and multiply it together.  So what are some subspaces?\n",
    "\n",
    "1. A line through $R^n$ that goes through the origin.  Any multiple of a point on a line is still on the line.  We need the origin because you can mutliply any vector by 0, which would give you the origin.\n",
    "1. Just the Origin is a subspace.  You can add it to itelf, multiply it by anything, and take any combination and you will stil lhave the origin.\n",
    "1. In $R^3$ and above, a plane through the origin is also a subspace."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combinations of Subspaces\n",
    "\n",
    "If I have 2 subspaces, then all vectors in P or L or both does not make a subspace.  I can't add all together.  In $R^3$ I would have to make a cube which would be all of $R^3$.  \n",
    "\n",
    "However, if you all points that are in both subspaces does make a subspace it is a subspace.  If you think about it, by definition 2 Vectors in the intersection are for the subspace 1 and they are both in subspace 2.  We already know that if you add 2 vectors in subspace 1 you get another vector in subspace 1.  We also know the same thing about subspace 2.  So really, any linear combination has to be in both subspaces (probably a smaller subspace)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Subspace from a matrix\n",
    "\n",
    "## Column Space\n",
    "\n",
    "$\\begin{bmatrix}1&1&2\\\\2&1&3\\\\3&1&4\\\\4&1&5\\end{bmatrix}$\n",
    "\n",
    "Columns are in $R^4$ and all their linear combinations are a subspace.  So we have 3 subspaces, 1 for each column.  So the full subspace for this matrix must have a linear combination of all columns, or all 3 subspaces.  With 3 columns do we get all of $R^4$? What exactly do we get?  How big is this subspace?  Does $Ax=b$ have a solution for every $b$?  Which $b$ values are ok?\n",
    "\n",
    "We know it doesn't, because we have 4 equations but only 3 unknowns.  In the proper form we see.\n",
    "\n",
    "$Ax=b$\n",
    "$\\begin{bmatrix}1&1&2\\\\2&1&3\\\\3&1&4\\\\4&1&5\\end{bmatrix}$\n",
    "$\\begin{bmatrix}x_1\\\\x_2\\\\x_3\\end{bmatrix}$\n",
    "$=$\n",
    "$\\begin{bmatrix}b_1\\\\b_2\\\\b_3\\\\b_4\\end{bmatrix}$\n",
    "\n",
    "Now, just because we can't solve for every b doesn't mean we can't solve it for any b.  So which $b$'s allow this to be solved?  Obviously we can solve for the 0 vector.  Obviously we can solve for (1,2,3,4) or any other column as we can have $x_1 = 1$ with the other x's being 0.  We could also solve for multiples of any column because $x_2$ could equal 2 or 3 with other uknowns being 0.\n",
    "\n",
    ">Note: To Summarize, Ax=b can be solved with $b$ is in $C(A)$, the column space of A.\n",
    "\n",
    "Now column 3 and column 1 are redundant, so we can throw away column 3.  Column 1 + column 2 = column 3.  So really linear combinations of the first 2 columns gives the same subspace as linear combinations of all 3 columns.  So in all practicality, we have 2 equations with 4 unknowns with 2 of the columns sitting on the same plane."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nullspace\n",
    "\n",
    "$\\begin{bmatrix}1&1&2\\\\2&1&3\\\\3&1&4\\\\4&1&5\\end{bmatrix}$\n",
    "\n",
    "Nullspace of $A$ = all solutions of $x$ where $Ax=0$.  Because there are 3 unknowns, it's a subspace in $R^3$.  This is opposed to the column space that is in $R^4$.  Naturally the 0 vector always satisfies this so is always in the nullspace.  Here's a few others\n",
    "\n",
    "$\\begin{bmatrix}0\\\\0\\\\0\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}1\\\\1\\\\-1\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}2\\\\2\\\\-2\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}-1\\\\-1\\\\1\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}-2\\\\-2\\\\2\\end{bmatrix}$\n",
    "\n",
    "To summarize, we really have a line in $R^3$, which is a subspace:\n",
    "$c\\begin{bmatrix}1\\\\1\\\\-1\\end{bmatrix}$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution Spaces\n",
    "\n",
    "Great!  So we have defined the nullspace, which is just a subspace when $b$ is the 0 vector.  Why not do the same thing with other $b$'s?  For example:\n",
    "\n",
    "$Ax=b$\n",
    "$\\begin{bmatrix}1&1&2\\\\2&1&3\\\\3&1&4\\\\4&1&5\\end{bmatrix}$\n",
    "$\\begin{bmatrix}x_1\\\\x_2\\\\x_3\\end{bmatrix}$\n",
    "$=$\n",
    "$\\begin{bmatrix}1\\\\2\\\\3\\\\4\\end{bmatrix}$\n",
    "\n",
    "Clearly it isn't a space because the zero vector is not a solution.\n",
    "\n",
    "Possible Solutions = \n",
    "$\\begin{bmatrix}1\\\\0\\\\0\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}2\\\\1\\\\-1\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}3\\\\2\\\\-2\\end{bmatrix}$,\n",
    "$\\begin{bmatrix}0\\\\-1\\\\1\\end{bmatrix}$\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}