---
title: "Neural Networks and Deep Learning - Part II (Practice)"
author: "Dr. Hua Zhou"
date: "3/15/2018"
output: 
  html_document:
    toc: true
    toc_depth: 4
---

```{r setup, include=FALSE}
options(width = 120)
knitr::opts_chunk$set(echo = TRUE)
sessionInfo()
```

## Software

- High-level software focuses on user-friendly interface to specify and train models.  
[Keras](https://keras.io), [PyTorch](http://pytorch.org) (only Linux and MacOS), [scikit-learn](http://scikit-learn.org/stable/), ...

- Lower-level software focuses on developer tools for impelementing deep learning models.   
[TensorFlow](https://www.tensorflow.org), [Theano](http://deeplearning.net/software/theano/#), [CNTK](https://github.com/Microsoft/CNTK), [Caffe](http://caffe.berkeleyvision.org), [Torch](http://torch.ch), ...

- Most tools are developed in Python plus a low-level language.

## TensorFlow

- Developed by Google Brain team for internal Google use. Formerly DistBelief.

- Open sourced in Nov 2015.

- OS: Linux, MacOS, and Windows (since Nov 2016).

- GPU support: NVIDIA CUDA.

- TPU (tensor processing unit), built specifically for machine learning and tailored for TensorFlow.

- Mobile device deployment: TensorFlow Lite (May 2017) for Android and iOS.

- Used in a variety of Google apps: speech recognition (Google assistant), Gmail (Smart Reply), search, translate, self-driving car ...

> when you have a hammer, everything looks like a nail.  

<p align="center">
![](./hammer.jpg){width=200px}
</p>

- [Machine Learning Crash Course (MLCC)](https://developers.google.com/machine-learning/crash-course/?utm_source=google-ai&utm_medium=card-image&utm_campaign=training-hub&utm_content=ml-crash-course). A 15 hour workshop available to public since March 1, 2018.

## R/RStudio

R users can access Keras and TensorFlow via the `keras` and `tensorflow` packages. 

## Example: MNIST - MLP

[Rmd](http://raw.githubusercontent.com/Hua-Zhou/Hua-Zhou.github.io/master/teaching/biostatm280-2018winter/slides/16-nn/mnist_mlp.Rmd), [html](./mnist_mlp.html).

## Example: MNIST - CNN

[Rmd](http://raw.githubusercontent.com/Hua-Zhou/Hua-Zhou.github.io/master/teaching/biostatm280-2018winter/slides/16-nn/mnist_cnn.Rmd), [html](./mnist_cnn.html).

## Example: Generate text from Nietzsche’s writings

[Rmd](http://raw.githubusercontent.com/Hua-Zhou/Hua-Zhou.github.io/master/teaching/biostatm280-2018winter/slides/16-nn/nietzsche_lstm.Rmd), [html](./nietzsche_lstm.html).