{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Deep Learning Tutorial with Keras and Tensorflow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div>\n",
" <img style=\"text-align: left\" src=\"imgs/keras-tensorflow-logo.jpg\" width=\"40%\" />\n",
"<div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get the Materials"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"imgs/github.jpg\" />\n",
"```shell\n",
"\n",
"git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"# Outline at a glance\n",
"\n",
"- **Part I**: **Introduction**\n",
"\n",
" - Intro to Artificial Neural Networks\n",
" - Perceptron and MLP \n",
" - naive pure-Python implementation\n",
" - fast forward, sgd, backprop\n",
" \n",
" - Introduction to Deep Learning Frameworks\n",
" - Intro to Theano\n",
" - Intro to Tensorflow\n",
" - Intro to Keras\n",
" - Overview and main features\n",
" - Overview of the `core` layers\n",
" - Multi-Layer Perceptron and Fully Connected\n",
" - Examples with `keras.models.Sequential` and `Dense`\n",
" - Keras Backend\n",
" \n",
"- **Part II**: **Supervised Learning **\n",
" \n",
" - Fully Connected Networks and Embeddings\n",
" - Intro to MNIST Dataset\n",
" - Hidden Leayer Representation and Embeddings\n",
" \n",
" - Convolutional Neural Networks\n",
" - meaning of convolutional filters\n",
" - examples from ImageNet \n",
" - Visualising ConvNets \n",
"\n",
" - Advanced CNN\n",
" - Dropout\n",
" - MaxPooling\n",
" - Batch Normalisation\n",
"\n",
" - HandsOn: MNIST Dataset\n",
" - FC and MNIST\n",
" - CNN and MNIST\n",
" \n",
" - Deep Convolutiona Neural Networks with Keras (ref: `keras.applications`)\n",
" - VGG16\n",
" - VGG19\n",
" - ResNet50\n",
" - Transfer Learning and FineTuning\n",
" - Hyperparameters Optimisation \n",
" \n",
"- **Part III**: **Unsupervised Learning**\n",
"\n",
" - AutoEncoders and Embeddings\n",
"\t- AutoEncoders and MNIST\n",
" \t- word2vec and doc2vec (gensim) with `keras.datasets`\n",
" - word2vec and CNN\n",
" \n",
"- **Part IV**: **Recurrent Neural Networks**\n",
" - Recurrent Neural Network in Keras \n",
" - `SimpleRNN`, `LSTM`, `GRU`\n",
" - LSTM for Sentence Generation\n",
"\t\t\n",
"- **PartV**: **Additional Materials**: \n",
" - Custom Layers in Keras \n",
" - Multi modal Network Topologies with Keras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Requirements"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This tutorial requires the following packages:\n",
"\n",
"- Python version 3.5\n",
" - Python 3.4 should be fine as well\n",
" - likely Python 2.7 would be also fine, but *who knows*? :P\n",
" \n",
"- `numpy` version 1.10 or later: http://www.numpy.org/\n",
"- `scipy` version 0.16 or later: http://www.scipy.org/\n",
"- `matplotlib` version 1.4 or later: http://matplotlib.org/\n",
"- `pandas` version 0.16 or later: http://pandas.pydata.org\n",
"- `scikit-learn` version 0.15 or later: http://scikit-learn.org\n",
"- `keras` version 2.0 or later: http://keras.io\n",
"- `tensorflow` version 1.0 or later: https://www.tensorflow.org\n",
"- `ipython`/`jupyter` version 4.0 or later, with notebook support\n",
"\n",
"(Optional but recommended):\n",
"\n",
"- `pyyaml`\n",
"- `hdf5` and `h5py` (required if you use model saving/loading functions in keras)\n",
"- **NVIDIA cuDNN** if you have NVIDIA GPUs on your machines.\n",
" [https://developer.nvidia.com/rdp/cudnn-download]()\n",
"\n",
"The easiest way to get (most) these is to use an all-in-one installer such as [Anaconda](http://www.continuum.io/downloads) from Continuum. These are available for multiple architectures."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Python Version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'm currently running this tutorial with **Python 3** on **Anaconda**"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Python 3.5.2\r\n"
]
}
],
"source": [
"!python --version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configure Keras with tensorflow\n",
"\n",
"1) Create the `keras.json` (if it does not exist):\n",
"\n",
"```shell\n",
"touch $HOME/.keras/keras.json\n",
"```\n",
"\n",
"2) Copy the following content into the file:\n",
"\n",
"```\n",
"{\n",
" \"epsilon\": 1e-07,\n",
" \"backend\": \"tensorflow\",\n",
" \"floatx\": \"float32\",\n",
" \"image_data_format\": \"channels_last\"\n",
"}\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\r\n",
"\t\"epsilon\": 1e-07,\r\n",
"\t\"backend\": \"tensorflow\",\r\n",
"\t\"floatx\": \"float32\",\r\n",
"\t\"image_data_format\": \"channels_last\"\r\n",
"}"
]
}
],
"source": [
"!cat ~/.keras/keras.json"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Test if everything is up&running"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Check import"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import scipy as sp\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"import sklearn"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"import keras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Check installeded Versions"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"numpy: 1.11.1\n",
"scipy: 0.18.0\n",
"matplotlib: 1.5.2\n",
"iPython: 5.1.0\n",
"scikit-learn: 0.18\n"
]
}
],
"source": [
"import numpy\n",
"print('numpy:', numpy.__version__)\n",
"\n",
"import scipy\n",
"print('scipy:', scipy.__version__)\n",
"\n",
"import matplotlib\n",
"print('matplotlib:', matplotlib.__version__)\n",
"\n",
"import IPython\n",
"print('iPython:', IPython.__version__)\n",
"\n",
"import sklearn\n",
"print('scikit-learn:', sklearn.__version__)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"keras: 2.0.2\n",
"Theano: 0.9.0\n",
"Tensorflow: 1.0.1\n"
]
}
],
"source": [
"import keras\n",
"print('keras: ', keras.__version__)\n",
"\n",
"# optional\n",
"import theano\n",
"print('Theano: ', theano.__version__)\n",
"\n",
"import tensorflow as tf\n",
"print('Tensorflow: ', tf.__version__)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"<h1 style=\"text-align: center;\">If everything worked till down here, you're ready to start!</h1>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}