{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Jupyter/Tensorflow Installation and Setup\n", "\n", "\n", "## Anaconda Setup\n", "\n", "First, [download and install Anaconda](https://www.continuum.io/downloads).\n", "\n", "This step takes a long time since the full anaconda distribution is >2G and takes forever to download, and the installation script isn't fast either.\n", "\n", "Once Anaconda is installed though, the rest only takes ~15 minutes.\n", "\n", "## Python Package Installation\n", "\n", "Next, create a new anaconda environment:\n", "\n", " conda create -n tf3.5 python=3.5\n", "\n", "Activate the environment \"tf3.5\":\n", "\n", " source activate tf3.5\n", "\n", "Install things that will be useful for data analysis in general:\n", "\n", " # The below will also install scipy, matplotlib, numpy and a variety of other things\n", " conda install scikit-learn jupyter pandas seaborn plotly\n", "\n", "[Install Tensorflow](https://www.tensorflow.org/install/):\n", "\n", " # Note that if you want to use the GPU-enabled version of TF, you should make deal with the\n", " # things mentioned below in the \"Using GPUs\" section FIRST (ie before running this command)\n", " pip install tensorflow-gpu\n", " \n", " # OR if you don't want to use the GPU enabled version (which can be a pain) just run this:\n", " # pip install tensorflow\n", "\n", "And that's it. If you want to check to see if a package is installed you can run something like this:\n", "\n", " pip freeze | grep tensorflow\n", "\n", "Also, if you ever want to delete an environment:\n", " \n", " rm -rf ~/anaconda/envs/tf3.5 # Or replace \"tf3.5\" with whatever you called your environment\n", "\n", "## Using GPUs\n", "\n", "If you want to use your GPU for running tensorflow graphs, at least on Mac OS, there are a few things that you'd have to do first:\n", "\n", "- Figure out if your machine has a CUDA-enabled NVIDIA graphics card (ie GPU)\n", " - This is probably true if you have a 2011 or later macbook pro with bigger than 13\" display\n", " - Older Macs have AMD GPUs that are not CUDA compatible\n", " - Some newer Macs only have an Intel, integrated graphics card (that is also not CUDA compatible)\n", "- Install CUDA Toolkit\n", "- Install cuDNN\n", "\n", "See here for some more details:\n", "\n", "https://www.tensorflow.org/install/install_mac#requirements_to_run_tensorflow_with_gpu_support\n", "\n", "\n", "## Creating a new Jupyter Notebook\n", "\n", "\n", "To create a new folder to contain a notebook (a good idea to do it within a git repo), and then create a notebook in that folder:\n", "\n", " > source activate tf3.5\n", " (tf3.5)> mkdir /tmp/test_notebooks\n", " (tf3.5)> cd /tmp/test_notebooks/\n", " (tf3.5)> jupyter notebook\n", " [I 08:00:15.291 NotebookApp] Serving notebooks from local directory: /private/tmp/test_notebook\n", " [I 08:00:15.291 NotebookApp] 0 active kernels\n", " [I 08:00:15.291 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=846ffbfdefd9ee084d60977e8d49ae1e7b1809154de2936d\n", " [I 08:00:15.291 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).\n", " \n", "Now your browser should automatically open up and bring you to a notebook to start working in.\n", "\n", "## Some Helpful Tips\n", "\n", "- On GPUs:\n", "\n", "When you launch a notebook and the tensorflow GPU setup is all working correctly, you should see some output like this in the console where you ran \"jupyter notebook\":\n", " \n", " I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:\n", " name: GeForce GT 750M\n", " major: 3 minor: 0 memoryClockRate (GHz) 0.9255\n", " pciBusID 0000:01:00.0\n", " Total memory: 2.00GiB\n", " Free memory: 76.35MiB\n", "\n", "In the new notebook, you can test the tensorflow installation by running \"import tensorflow as tf\".\n", "\n", "In the past, this would give me some errors that were ultimately corrected by making sure some necessary CUDA libraries were reachable by making sure to set these environment variables:\n", "\n", " export CUDA_HOME=/usr/local/cuda\n", " export DYLD_LIBRARY_PATH=\"$DYLD_LIBRARY_PATH:$CUDA_HOME/lib\"\n", " export PATH=\"$CUDA_HOME/bin:$PATH\"\n", "\n", "- On common notebook imports:\n", "\n", "I always thought this was a surprisingly difficult thing to google, but if you ever get tired of running the same import commands in ever single notebook you can create your \"initialization scripts\" by doing something like this:\n", "\n", "First, create a file in the Anaconda environment like this:\n", "\n", " vi ~/anaconda/envs/tf3.5/lib/python3.5/site-packages/local.pth\n", "\n", "Add a single line to this file indicating a directory that will contain python scripts (or multiple lines for multiple directories). For example, I use something like:\n", "\n", " /Users/eczech/repos/python/notebooks/startup_scripts\n", "\n", "Within this directory, I have small files with boilerplate code like:\n", "\n", "\n", " cat /Users/eczech/repos/python/notebooks/startup_scripts/ipy_startup.py\n", " \n", " import os\n", " import sys\n", " import numpy as np\n", " import pandas as pd\n", " import matplotlib.pyplot as plt\n", "\n", "Now in a notebook, you can run \"%run -m ipy_startup\" and the lines above will execute based on a single line of code rather than having to run all 5 of those lines every single time." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 2 }