{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "$$ \\LaTeX \\text{ command declarations here.}\n", "\\newcommand{\\R}{\\mathbb{R}}\n", "\\renewcommand{\\vec}[1]{\\mathbf{#1}}\n", "$$" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "from Lec04 import *" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# EECS 545: Machine Learning\n", "## Lecture 04: Linear Regression I\n", "* Instructor: **Jacob Abernethy**\n", "* Date: January 20, 2015\n", "\n", "\n", "*Lecture Exposition Credit: Benjamin Bray*" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Outline for this Lecture\n", "- Introduction to Regression\n", "- Solving Least Squares\n", " - Gradient Descent Method\n", " - Closed Form Solution\n", "\n", "\n", "## Reading List\n", "- Required: \n", " - **[PRML]**, §1.1: Polynomial Curve Fitting Example\n", " - **[PRML]**, §3.1: Linear Basis Function Models\n", "- Optional:\n", " - **[MLAPP]**, Chapter 7: Linear Regression" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "> In this lecture, we will cover linear regression. First some basic concepts and examples of linear regression will be introduced. Then we will show solving linear regression can be done by solving least squares problem. To solve least squares problem, two methods will be given: *gradient descent method* and *closed form solution*. Finally, the concept of Moore-Penrose pseudoinverse is introduced which is highly related to least squares." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Supervised Learning\n", "\n", "- Goal\n", " - Given data $X$ in feature sapce and the labels $Y$\n", " - Learn to predict $Y$ from $X$\n", "- Labels could be discrete or continuous\n", " - Discrete-valued labels: Classification\n", " - Continuous-valued labels: Regression\n", " \n", "