{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lesson 1\n", "The course will use wrappers and code for the most part, so use the usual Python methods to check what's going on." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0, 1, 2, 3, 4])" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# View the docstring/code using ??, or view info using shift + tab\n", "import numpy as np\n", "np.arange(5)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "??np.arange" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "?np.arange" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "~~## Running a notebook on AWS~~\n", "\n", "~~1. Open a terminal window and `ssh` into the AWS server (IP can be found from the console: https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Instances:sort=instanceId)~~\n", "\n", "~~2. Navigate to the nbs (notebooks) directory~~\n", "\n", "~~3. `wget` the notebook from https://github.com/fastai/courses/tree/master/deeplearning1/nbs (remember to get the RAW notebook)~~\n", "\n", "~~4. (Optional) You can use `tmux` to split the terminal window. This will let you run Jupyter from one part of the terminal, while leaving another part of the terminal free to execute commands, crunch data, etc.~~\n", "\n", "(I now have my own Ubuntu and Windows servers working)\n", "\n", "### tmux commands\n", "* `ctrl+b ?`: List commands\n", "* `ctrl+b \"`: Split terminal horizontally\n", "* `ctrl+b %`: Split terminal vertically\n", "* `ctrl+b left/right/up/down arrows`: Switch between split panes\n", "* `ctrl+b d`: Detatch from this tmux session (session will still be running)\n", "* `tmux attach`: Rejoin tmux session\n", "\n", "5. Run `jupyter notebook`\n", "\n", "6. In browser, put in the server_ip:8888 -- dl_course\n", "\n", "That's it!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Note on folder structure\n", "\n", "Folder structure is very important when training/running models.\n", "As well as keeping things organized, folder structure matters when it comes to Keras. Keras expects each class of object is in its own folder.\n", "\n", "__Sample structure__\n", "\n", "dogscats\n", "- training\n", " - dogs\n", " - dog1.jpg\n", " - ...\n", " - cats\n", " - cat1.jpg\n", " - ...\n", "- testing\n", " - img1.jpg\n", " - ...\n", " (you must determine which images in this directory belong to which class, hence there are no subfolders)\n", "- validation\n", "- (Like testing, except you know which are which?)\n", " - dogs\n", " - dog1.jpg\n", " - ...\n", " - cats\n", " - cat1.jpg\n", " - ...\n", "- sample\n", "- (This is like your top-level directory, but contains fewer images, so you can run through it more quickly)\n", " - training\n", " - dogs\n", " - dog1.jpg\n", " - ...\n", " - cats\n", " - cat1.jpg\n", " - ...\n", " - testing\n", " - img1.jpg\n", " - ..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## image-net.org\n", "http://www.image-net.org/ contains images you can use to train models, but beware: these images tend to have only the object in question, they have been sanitized. When you use a pre-trained model, you inherit all of the biases from the data it was trained on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Architecture\n", "\n", "__`VGG16`__: A simple neural net model that can be used to, e.g., classify images. This is built on...\n", "\n", "__`Keras`__: A neural net API written in Python. It can be used to produce `TensorFlow` or `Theano` code. Update `keras.json` to switch the backend to use either one. You can also use the file to switch between CPU and GPU processing.\n", "\n", "__` Theano`__: Converts Python code into compiled GPU code (`CUDA`)\n", "\n", "__`TensorFlow`__: Like a Google version of `Theano`. Can run on multiple GPUs.\n", "\n", "__`CUDA`__: The \"bottom\" layer. The NVIDIA GPU language for machine learning.\n", "\n", "__`CUDNN`__: CUDA Deep Neural Network Library. A part of `CUDA`.\n", "\n", "__NOTE:__ Now using `PyTorch` in V2. PyTorch is Facebook's open source machine learning library for Python." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Universal Approximation Theorem\n", "Thinking of a neural network as a function, this kind of function can solve any given problem to some close accuracy as long as you add enough parameters. (This is proven.)\n", "\n", "This links to the assertion that deep learning is (or should be):\n", "\n", "1. An infinitely flexible function\n", "2. All-purpose parameter fitting\n", "3. Fast and flexible" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Convolutional Neural Network\n", "http://setosa.io/ev/image-kernels/\n", "\n", "Multiplies each pixel value in 3x3 matrix by a kernel, and then add them all together.\n", "\n", "Low values become black, high values become white, and so you manage to detect the edges." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Nonlinearity\n", "\n", "Takes some input value and transforms it into another value in a non-linear way.\n", "\n", "E.g., the sigmoid function, or recitified linear unit (RLU).\n", "\n", "Using our linear and non-linear functions allows us to create complex shapes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learning rate\n", "Step size in gradient descent." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Images\n", "Images are held in 3D arrays.\n", "\n", "The first two dimensions represent the X and Y dimensions, and the third dimension represents the RGB vaules." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pretraining Models\n", "A model created by somebody else to solve another problem." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Probability\n", "When classifying between our two cat and dog objects, we measure things between 0-1.\n", "\n", "0 = cat\n", "1 = dog\n", "0.5 = not sure\n", "\n", "These numbers can then be used to view, for example, the most incorrect dog (scored close to 0, but was labelled dog) or the most dog-like dog (scored close to 1 and is a dog).\n", "\n", "The probability is returned using a __log scale__. On a log scale, each tick is the previous tick mark MULTIPLIED by a certain number. It's useful for compressing a large data range into a more smaller range." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overfitting & transforms\n", "When you run too many epochs (process all of the training data via a series of mini-batches), you run the risk of __overfitting__.\n", "\n", "This means the network has become very good at identifying the training data, but it cannot generalize enough to identify the validation data.\n", "\n", "One way to correct for overfitting is to create more test data.\n", "\n", "One way to do this is to create transforms of the data. For example, rotate, zoom, and scale your training images." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Confusion Matrix\n", "A confusion matrix shows you:\n", "\n", "* TN = true negatives, values that should be negative and are\n", "* FN = false negatives, values that shouldn't be negative but are\n", "* TP = true positives, values that should be positive and are\n", "* FP = false positives, values that shouldn't be positive but are\n", "\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting the model\n", "When we talk about fitting the model, we mean finding the best parameters for a layer in order to achieve the desired outputs using gradient descent.\n", "\n", "When we fit the mode, we get back:\n", "\n", "* Loss in the training set\n", "* Loss in the validation set\n", "* Accuracy in the validation set\n", "\n", "Accuracy = ratio of correct predictions\n", "\n", "Loss/cost function = represents the price paid for inaccuracy of predictions\n", "\n", "E.g., if your image label is 1 and your model gives it a prediction of 0.9, the loss should be small, and vice versa." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 2 }