{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "
\n", " \n", " \"QuantEcon\"\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Julia Tools and Editors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "- [Julia Tools and Editors](#Julia-Tools-and-Editors) \n", " - [Preliminary Setup](#Preliminary-Setup) \n", " - [The REPL](#The-REPL) \n", " - [Atom](#Atom) \n", " - [Package Environments](#Package-Environments) \n", " - [Docker](#Docker) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Co-authored with Arnav Sood\n", "\n", "While Jupyter notebooks are a great way to get started with the language, eventually you will want to use more powerful tools\n", "\n", "We’ll discuss a few of them here, such as\n", "\n", "- Text editors like Atom, which come with rich Julia support for debugging, documentation, git integration, plotting and inspecting data, and code execution \n", "- The Julia REPL, which has specialized modes for package management, shell commands, and help \n", "- A virtualized Docker setup which provides a painless pre-configured environment on your machine \n", "\n", "\n", "Note that we assume you’ve already completed the [getting started](https://lectures.quantecon.org/getting_started_julia/getting_started.html) and [interacting with Julia](https://lectures.quantecon.org/getting_started_julia/julia_environment.html) lectures" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preliminary Setup\n", "\n", "Follow the instructions for setting up Julia [on your local computer](https://lectures.quantecon.org/getting_started_julia/getting_started.html#jl-jupyterlocal)\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating a Startup File (Recommended)\n", "\n", "Whenever the Julia compiler or REPL starts, it will look for a file called `startup.jl` (see [Julia Manual](https://docs.julialang.org/en/v1/manual/getting-started/#man-getting-started-1))\n", "\n", "We provide a file here which does two things\n", "\n", "- Makes the REPL shell mode “sticky,” so you don’t need to keep running `;` for new commands \n", "- Loads the `Revise.jl` package on startup, which lets you see changes you make to a package in real-time (i.e., no need to quit the REPL, open again, and load again) \n", "\n", "\n", "The location for the file is relative to your default Julia environment (e.g. `~/.julia/config/startup.jl` or `C:\\Users\\USERNAME\\.julia\\config\\startup.jl` on Windows)\n", "\n", "Recall that you can find the location of the `~/.julia` directory by running" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hide-output": false }, "outputs": [ { "data": { "text/plain": [ "\"/home/qebuild/.julia\"" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "DEPOT_PATH[1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note:** On Mac, this won’t be visible in the Finder unless you specifically enable that option, but you can get to it by running `cd .julia; open .` from a new terminal\n", "\n", "To add the file:\n", "\n", "- In the `julia` terminal, type the following " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "] add Revise REPL; precompile\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "- Create the `~/.julia/config/` directory if necessary in the terminal or file explorer \n", "- Download the file [startup.jl](https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/jl/_static/includes/startup.jl) into that directory \n", "- For convenience, you may find it useful on your operating system to change the directory where the REPL starts \n", "\n", "\n", "On Windows, if you have a shortcut on your desktop or on the taskbar, you could: (1) right-click on the icon; (2) right click on the “julia” text; (3) choose “Properties”, and (4) change the “Start In” to be something such as `C:\\Users\\YOURUSERNAME\\Documents`\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The REPL\n", "\n", "Previously, we discussed basic use of the Julia REPL (“Read-Evaluate-Print Loop”)\n", "\n", "Here, we’ll consider some more advanced features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Shell Mode\n", "\n", "Hitting `;` brings you into shell mode, which lets you run bash commands (PowerShell on Windows)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed\n" ] } ], "source": [ "; pwd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also use Julia variables from shell mode" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "hide-output": false }, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = 2" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2\n" ] } ], "source": [ "; echo $x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Package Mode\n", "\n", "Hitting `]` brings you into package mode\n", "\n", "- `] add Expectations` will add a package (here, `Expectations.jl`) \n", "- Likewise, `] rm Expectations` will remove that package \n", "- `] st` will show you a snapshot of what you have installed \n", "- `] up` will (intelligently) upgrade versions of your packages \n", "- `] precompile` will precompile everything possible \n", "\n", "\n", "You can get a full list of package mode commands by running" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \u001b[1mWelcome to the Pkg REPL-mode\u001b[22m. To return to the \u001b[36mjulia>\u001b[39m prompt, either press" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", " backspace when the input line is empty or press Ctrl+C.\n", "\n", " \u001b[1mSynopsis\u001b[22m\n", "\n", "\u001b[36m pkg> cmd [opts] [args]\u001b[39m\n", "\n", " Multiple commands can be given on the same line by interleaving a \u001b[36m;\u001b[39m between\n", " the commands.\n", "\n", " \u001b[1mCommands\u001b[22m\n", "\n", " \u001b[36mactivate\u001b[39m: set the primary environment the package manager manipulates\n", "\n", " \u001b[36madd\u001b[39m: add packages to project\n", "\n", " \u001b[36mbuild\u001b[39m: run the build script for packages\n", "\n", " \u001b[36mdevelop\u001b[39m: clone the full package repo locally for development\n", "\n", " \u001b[36mfree\u001b[39m: undoes a \u001b[36mpin\u001b[39m, \u001b[36mdevelop\u001b[39m, or stops tracking a repo\n", "\n", " \u001b[36mgc\u001b[39m: garbage collect packages not used for a significant time\n", "\n", " \u001b[36mgenerate\u001b[39m: generate files for a new project\n", "\n", " \u001b[36mhelp\u001b[39m: show this message\n", "\n", " \u001b[36minstantiate\u001b[39m: downloads all the dependencies for the project\n", "\n", " \u001b[36mpin\u001b[39m: pins the version of packages\n", "\n", " \u001b[36mprecompile\u001b[39m: precompile all the project dependencies\n", "\n", " \u001b[36mpreview\u001b[39m: previews a subsequent command without affecting the current state\n", "\n", " \u001b[36mremove\u001b[39m: remove packages from project or manifest\n", "\n", " \u001b[36mresolve\u001b[39m: resolves to update the manifest from changes in dependencies of\n", " developed packages\n", "\n", " \u001b[36mstatus\u001b[39m: summarize contents of and changes to environment\n", "\n", " \u001b[36mtest\u001b[39m: run tests for packages\n", "\n", " \u001b[36mupdate\u001b[39m: update packages in manifest\n", "\n", " \u001b[36mregistry add\u001b[39m: add package registries\n", "\n", " \u001b[36mregistry remove\u001b[39m: remove package registries\n", "\n", " \u001b[36mregistry status\u001b[39m: information about installed registries\n", "\n", " \u001b[36mregistry update\u001b[39m: update package registries" ] } ], "source": [ "] ?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On some operating systems (such as OSX) REPL pasting may not work for package mode, and you will need to access it in the standard way (i.e., hit `]` first and then run your commands)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Help Mode\n", "\n", "Hitting `?` will bring you into help mode\n", "\n", "The key use case is to find docstrings for functions and macros, e.g." ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? print\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that objects must be loaded for Julia to return their documentation, e.g." ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? @test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "will fail, but" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "using Test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? @test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "will succeed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Atom\n", "\n", "As discussed [previously](https://lectures.quantecon.org/getting_started_julia/getting_started.html), eventually you will want to use a fully fledged text editor\n", "\n", "The most feature-rich one for Julia development is [Atom](https://atom.io/), with the [Juno](http://junolab.org/) package\n", "\n", "There are several reasons to use a text editor like Atom, including\n", "\n", "- Git integration (more on this in the [next lecture](https://lectures.quantecon.org/version_control.html)) \n", "- Painless inspection of variables and data \n", "- Easily run code blocks, and drop in custom snippets of code \n", "- Integration with Julia documentation and plots " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Installation and Configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Installing Atom\n", "\n", "1. Download and Install Atom from the [Atom website](https://atom.io/) \n", "1. (Optional, but recommended): Change default Atom settings \n", " \n", " - Use `Ctrl-,` to get the `Settings` pane \n", " - Choose the `Packages` tab \n", " - Type `line-ending-selector` into the Filter and then click “Settings” for that package \n", " \n", " - Change the default line ending to `LF` (only necessary on Windows) \n", " \n", " - Choose the Editor tab \n", " \n", " - Turn on `Soft Wrap` \n", " - Set the `Tab Length` default to `4` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Installing Juno\n", "\n", "1. Use `Ctrl-,` to get the Settings pane \n", "1. Go to the `Install` tab \n", "1. Type `uber-juno` into the search box and then click Install on the package that appears \n", "1. Wait while Juno installs dependencies \n", "1. When it asks you whether or not to use the standard layout, click `yes` \n", "\n", "\n", "At that point, you should see a built-in REPL at the bottom of the screen and be able to start using Julia and Atom" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Troubleshooting\n", "\n", "Sometimes, Juno will fail to find the Julia executable (say, if it’s installed somewhere nonstandard, or you have multiple)\n", "\n", "To do this\n", "1. `Ctrl-,` to get Settings pane, and select the Packages tab\n", "2. Type in `julia-client` and choose Settings\n", "3. Find the Julia Path, and fill it in with the location of the Julia binary\n", "\n", "> - To find the binary, you could run `Sys.BINDIR` in the REPL, then add in an additional `/julia` to the end of the screen \n", "- e.g. `C:\\Users\\YOURUSERNAME\\AppData\\Local\\Julia-1.0.1\\bin\\julia.exe` on Windows as `/Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia` on OSX \n", "\n", "\n", "\n", "See the [setup instructions for Juno](http://docs.junolab.org/latest/man/installation.html) if you have further issues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Standard Layout\n", "\n", "If you follow the instructions, you should see something like this when you open a new file\n", "\n", "If you don’t, simply go to the command palette and type “Julia standard layout”\n", "\n", "\n", "\n", " \n", "The bottom pane is a standard REPL, which supports the different modes above\n", "\n", "The “workspace” pane is a snapshot of currently-defined objects\n", "\n", "For example, if we define an object in the REPL" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "hide-output": false }, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our workspace should read\n", "\n", "\n", "\n", " \n", "The `ans` variable simply captures the result of the last computation\n", "\n", "The `Documentation` pane simply lets us query Julia documentation\n", "\n", "\n", "\n", " \n", "The `Plots` pane captures Julia plots output (the code is as follows)" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "using Plots\n", "gr(fmt = :png);\n", "data = rand(10, 10)\n", "h = heatmap(data)\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", " \n", "**Note:** The plots feature is not perfectly reliable across all plotting backends, see [the Basic Usage](http://docs.junolab.org/latest/man/basic_usage.html) page" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other Features\n", "\n", "- `` Shift + Enter `` will evaluate a highlighted selection or line (as above) \n", "- The run symbol in the left sidebar (or `Ctrl+Shift+Enter`) will run the whole file \n", "\n", "\n", "See [basic usage](http://docs.junolab.org/latest/man/basic_usage.html) for an exploration of features, and the [FAQ](http://docs.junolab.org/latest/man/faq.html) for more advanced steps (e.g. using with `Docker`)\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Package Environments\n", "\n", "Julia’s package manager lets you set up Python-style “virtualenvs,” or subsets of packages that draw from an underlying pool of assets on the machine\n", "\n", "This way, you can work with (and specify) the dependencies (i.e., required packages) for one project without worrying about impacts on other projects\n", "\n", "- An `environment` is a set of packages specified by a `Project.toml` (and optionally, a `Manifest.toml`) \n", "- A `registry` is a git repository corresponding to a list of (typically) registered packages, from which Julia can pull (for more on git repositories, see [version control](https://lectures.quantecon.org/version_control.html)) \n", "- A `depot` is a directory, like `~/.julia`, which contains assets (compile caches, registries, package source directories, etc.) \n", "\n", "\n", "Essentially, an environment is a dependency tree for a project, or a “frame of mind” for Julia’s package manager\n", "\n", "- We can see the default (`v1.1`) environment as such " ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[36m\u001b[1mProject \u001b[22m\u001b[39mquantecon-notebooks-julia v0.1.0\n", "\u001b[32m\u001b[1m Status\u001b[22m\u001b[39m `~/repos/lecture-source-jl/_build/website/jupyter/Project.toml`\n", " " ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[90m [6e4b80f9]\u001b[39m\u001b[37m BenchmarkTools v0.4.2\u001b[39m\n", " \u001b[90m [a134a8b2]\u001b[39m\u001b[37m BlackBoxOptim v0.5.0\u001b[39m\n", " \u001b[90m [324d7699]\u001b[39m\u001b[37m CategoricalArrays v0.5.5\u001b[39m\n", " \u001b[90m [34da2185]\u001b[39m\u001b[37m Compat v2.1.0\u001b[39m\n", " \u001b[90m [a93c6f00]\u001b[39m\u001b[37m DataFrames v0.19.1\u001b[39m\n", " \u001b[90m [1313f7d8]\u001b[39m\u001b[37m DataFramesMeta v0.5.0\u001b[39m\n", " \u001b[90m [5721bf48]\u001b[39m\u001b[37m DataVoyager v0.3.1\u001b[39m\n", " \u001b[90m [39dd38d3]\u001b[39m\u001b[37m Dierckx v0.4.1\u001b[39m\n", " \u001b[90m [31c24e10]\u001b[39m\u001b[37m Distributions v0.21.0\u001b[39m\n", " \u001b[90m [2fe49d83]\u001b[39m\u001b[37m Expectations v1.1.0\u001b[39m\n", " \u001b[90m [442a2c76]\u001b[39m\u001b[37m FastGaussQuadrature v0.3.3\u001b[39m\n", " \u001b[90m [1a297f60]\u001b[39m\u001b[37m FillArrays v0.6.4\u001b[39m\n", " \u001b[90m [9d5cd8c9]\u001b[39m\u001b[37m FixedEffectModels v0.8.1\u001b[39m\n", " \u001b[90m [587475ba]\u001b[39m\u001b[37m Flux v0.8.3\u001b[39m\n", " \u001b[90m [f6369f11]\u001b[39m\u001b[37m ForwardDiff v0.10.3\u001b[39m\n", " \u001b[90m [38e38edf]\u001b[39m\u001b[37m GLM v1.3.1\u001b[39m\n", " \u001b[90m [28b8d3ca]\u001b[39m\u001b[37m GR v0.40.0\u001b[39m\n", " \u001b[90m [a98d9a8b]\u001b[39m\u001b[37m Interpolations v0.12.2\u001b[39m\n", " \u001b[90m [b6b21f68]\u001b[39m\u001b[37m Ipopt v0.6.0\u001b[39m\n", " \u001b[90m [4076af6c]\u001b[39m\u001b[37m JuMP v0.20.0\u001b[39m\n", " \u001b[90m [5ab0869b]\u001b[39m\u001b[37m KernelDensity v0.5.1\u001b[39m\n", " \u001b[90m [b964fa9f]\u001b[39m\u001b[37m LaTeXStrings v1.0.3\u001b[39m\n", " \u001b[90m [0fc2ff8b]\u001b[39m\u001b[37m LeastSquaresOptim v0.7.2\u001b[39m\n", " \u001b[90m [76087f3c]\u001b[39m\u001b[37m NLopt v0.5.1\u001b[39m\n", " \u001b[90m [2774e3e8]\u001b[39m\u001b[37m NLsolve v4.0.0\u001b[39m\n", " \u001b[90m [429524aa]\u001b[39m\u001b[37m Optim v0.19.1\u001b[39m\n", " \u001b[90m [d96e819e]\u001b[39m\u001b[37m Parameters v0.10.3\u001b[39m\n", " \u001b[90m [14b8a8f1]\u001b[39m\u001b[37m PkgTemplates v0.6.2\u001b[39m\n", " \u001b[90m [91a5bcdd]\u001b[39m\u001b[37m Plots v0.25.3\u001b[39m\n", " \u001b[90m [f27b6e38]\u001b[39m\u001b[37m Polynomials v0.5.2\u001b[39m\n", " \u001b[90m [92933f4c]\u001b[39m\u001b[37m ProgressMeter v1.0.0\u001b[39m\n", " \u001b[90m [1fd47b50]\u001b[39m\u001b[37m QuadGK v2.1.0\u001b[39m\n", " \u001b[90m [fcd29c91]\u001b[39m\u001b[37m QuantEcon v0.16.1\u001b[39m\n", " \u001b[90m [1a8c2f83]\u001b[39m\u001b[37m Query v0.12.1\u001b[39m\n", " \u001b[90m [612083be]\u001b[39m\u001b[37m Queryverse v0.3.1\u001b[39m\n", " \u001b[90m [ce6b1742]\u001b[39m\u001b[37m RDatasets v0.6.3\u001b[39m\n", " \u001b[90m [295af30f]\u001b[39m\u001b[37m Revise v2.1.6\u001b[39m\n", " \u001b[90m [f2b01f46]\u001b[39m\u001b[37m Roots v0.8.1\u001b[39m\n", " \u001b[90m [90137ffa]\u001b[39m\u001b[37m StaticArrays v0.11.0\u001b[39m\n", " \u001b[90m [2913bbd2]\u001b[39m\u001b[37m StatsBase v0.31.0\u001b[39m\n", " \u001b[90m [3eaba693]\u001b[39m\u001b[37m StatsModels v0.6.3\u001b[39m\n", " \u001b[90m [f3b207a7]\u001b[39m\u001b[37m StatsPlots v0.11.0\u001b[39m\n", " \u001b[90m [112f6efa]\u001b[39m\u001b[37m VegaLite v0.7.0\u001b[39m\n", " \u001b[90m [37e2e46d]\u001b[39m\u001b[37m LinearAlgebra \u001b[39m\n", " \u001b[90m [9a3f8284]\u001b[39m\u001b[37m Random \u001b[39m\n", " \u001b[90m [2f01184e]\u001b[39m\u001b[37m SparseArrays \u001b[39m\n", " \u001b[90m [10745b16]\u001b[39m\u001b[37m Statistics \u001b[39m\n", " \u001b[90m [8dfed614]\u001b[39m\u001b[37m Test \u001b[39m\n" ] } ], "source": [ "] st" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- We can also create and activate a new environment " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mGenerating\u001b[22m\u001b[39m project ExampleEnvironment:\n", " ExampleEnvironment/Project.toml\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " ExampleEnvironment/src/ExampleEnvironment.jl\n" ] } ], "source": [ "] generate ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- And go to it " ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment\n" ] } ], "source": [ "; cd ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- To activate the directory, simply " ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mActivating\u001b[22m\u001b[39m environment at `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Project.toml`\n" ] } ], "source": [ "] activate ." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where “.” stands in for the “present working directory”\n", "\n", "- Let’s make some changes to this " ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m registry at `~/.julia/registries/General`\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m git-repo `https://github.com/JuliaRegistries/General.git`\n", "\u001b[?25l\u001b[2K" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[?25h\u001b[32m\u001b[1m Resolving\u001b[22m\u001b[39m package versions...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Project.toml`\n", " \u001b[90m [2fe49d83]\u001b[39m\u001b[92m + Expectations v1.1.1\u001b[39m\n", " \u001b[90m [d96e819e]\u001b[39m\u001b[92m + Parameters v0.11.0\u001b[39m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1m Updating\u001b[22m\u001b[39m `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Manifest.toml`\n", " \u001b[90m [7d9fca2a]\u001b[39m\u001b[92m + Arpack v0.3.1\u001b[39m\n", " \u001b[90m [9e28174c]\u001b[39m\u001b[92m + BinDeps v0.8.10\u001b[39m\n", " \u001b[90m [b99e7846]\u001b[39m\u001b[92m + BinaryProvider v0.5.6\u001b[39m\n", " \u001b[90m [34da2185]\u001b[39m\u001b[92m + Compat v2.1.0\u001b[39m\n", " \u001b[90m [9a962f9c]\u001b[39m\u001b[92m + DataAPI v1.0.1\u001b[39m\n", " \u001b[90m [864edb3b]\u001b[39m\u001b[92m + DataStructures v0.17.0\u001b[39m\n", " \u001b[90m [31c24e10]\u001b[39m\u001b[92m + Distributions v0.21.1\u001b[39m\n", " \u001b[90m [2fe49d83]\u001b[39m\u001b[92m + Expectations v1.1.1\u001b[39m\n", " \u001b[90m [442a2c76]\u001b[39m\u001b[92m + FastGaussQuadrature v0.4.0\u001b[39m\n", " \u001b[90m [e1d29d7a]\u001b[39m\u001b[92m + Missings v0.4.1\u001b[39m\n", " \u001b[90m [bac558e1]\u001b[39m\u001b[92m + OrderedCollections v1.1.0\u001b[39m\n", " \u001b[90m [90014a1f]\u001b[39m\u001b[92m + PDMats v0.9.9\u001b[39m\n", " \u001b[90m [d96e819e]\u001b[39m\u001b[92m + Parameters v0.11.0\u001b[39m\n", " \u001b[90m [1fd47b50]\u001b[39m\u001b[92m + QuadGK v2.0.3\u001b[39m\n", " \u001b[90m [79098fc4]\u001b[39m\u001b[92m + Rmath v0.5.0\u001b[39m\n", " \u001b[90m [a2af1166]\u001b[39m\u001b[92m + SortingAlgorithms v0.3.1\u001b[39m\n", " \u001b[90m [276daf66]\u001b[39m\u001b[92m + SpecialFunctions v0.7.2\u001b[39m\n", " \u001b[90m [2913bbd2]\u001b[39m\u001b[92m + StatsBase v0.32.0\u001b[39m\n", " \u001b[90m [4c63d2b9]\u001b[39m\u001b[92m + StatsFuns v0.8.0\u001b[39m\n", " \u001b[90m [30578b45]\u001b[39m\u001b[92m + URIParser v0.4.0\u001b[39m\n", " \u001b[90m [2a0f44e3]\u001b[39m\u001b[92m + Base64 \u001b[39m\n", " \u001b[90m [ade2ca70]\u001b[39m\u001b[92m + Dates \u001b[39m\n", " \u001b[90m [8bb1440f]\u001b[39m\u001b[92m + DelimitedFiles \u001b[39m\n", " \u001b[90m [8ba89e20]\u001b[39m\u001b[92m + Distributed \u001b[39m\n", " \u001b[90m [b77e0a4c]\u001b[39m\u001b[92m + InteractiveUtils \u001b[39m\n", " \u001b[90m [76f85450]\u001b[39m\u001b[92m + LibGit2 \u001b[39m\n", " \u001b[90m [8f399da3]\u001b[39m\u001b[92m + Libdl \u001b[39m\n", " \u001b[90m [37e2e46d]\u001b[39m\u001b[92m + LinearAlgebra \u001b[39m\n", " \u001b[90m [56ddb016]\u001b[39m\u001b[92m + Logging \u001b[39m\n", " \u001b[90m [d6f4376e]\u001b[39m\u001b[92m + Markdown \u001b[39m\n", " \u001b[90m [a63ad114]\u001b[39m\u001b[92m + Mmap \u001b[39m\n", " \u001b[90m [44cfe95a]\u001b[39m\u001b[92m + Pkg \u001b[39m\n", " \u001b[90m [de0858da]\u001b[39m\u001b[92m + Printf \u001b[39m\n", " \u001b[90m [3fa0cd96]\u001b[39m\u001b[92m + REPL \u001b[39m\n", " \u001b[90m [9a3f8284]\u001b[39m\u001b[92m + Random \u001b[39m\n", " \u001b[90m [ea8e919c]\u001b[39m\u001b[92m + SHA \u001b[39m\n", " \u001b[90m [9e88b42a]\u001b[39m\u001b[92m + Serialization \u001b[39m\n", " \u001b[90m [1a1011a3]\u001b[39m\u001b[92m + SharedArrays \u001b[39m\n", " \u001b[90m [6462fe0b]\u001b[39m\u001b[92m + Sockets \u001b[39m\n", " \u001b[90m [2f01184e]\u001b[39m\u001b[92m + SparseArrays \u001b[39m\n", " \u001b[90m [10745b16]\u001b[39m\u001b[92m + Statistics \u001b[39m\n", " \u001b[90m [4607b0f0]\u001b[39m\u001b[92m + SuiteSparse \u001b[39m\n", " \u001b[90m [8dfed614]\u001b[39m\u001b[92m + Test \u001b[39m\n", " \u001b[90m [cf7118a7]\u001b[39m\u001b[92m + UUIDs \u001b[39m\n", " \u001b[90m [4ec0a83e]\u001b[39m\u001b[92m + Unicode \u001b[39m\n" ] } ], "source": [ "] add Expectations Parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note the lack of commas\n", "\n", "- To see the changes, simply open the `ExampleEnvironment` directory in an editor like Atom \n", "\n", "\n", "The Project TOML should look something like this" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "name = \"ExampleEnvironment\"\n", "uuid = \"14d3e79e-e2e5-11e8-28b9-19823016c34c\"\n", "authors = [\"QuantEcon User \"]\n", "version = \"0.1.0\"\n", "\n", "[deps]\n", "Expectations = \"2fe49d83-0758-5602-8f54-1f90ad0d522b\"\n", "Parameters = \"d96e819e-fc66-5662-9728-84c9c7592b0a\"\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m project...\n", "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m Expectations\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Info: Recompiling stale cache file /home/qebuild/.julia/compiled/v1.2/Expectations/r1e2K.ji for Expectations [2fe49d83-0758-5602-8f54-1f90ad0d522b]\n", "└ @ Base loading.jl:1240\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m Parameters\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Info: Recompiling stale cache file /home/qebuild/.julia/compiled/v1.2/Parameters/ycYNs.ji for Parameters [d96e819e-fc66-5662-9728-84c9c7592b0a]\n", "└ @ Base loading.jl:1240\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m ExampleEnvironment\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Info: Precompiling ExampleEnvironment [a9fe0181-583a-484a-be3c-5186ec7d44fa]\n", "└ @ Base loading.jl:1242\n" ] } ], "source": [ "] precompile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note** The TOML files are independent of the actual assets (which live in `~/.julia/packages`, `~/.julia/dev`, and `~/.julia/compiled`)\n", "\n", "You can think of the TOML as specifying demands for resources, which are supplied by the `~/.julia` user depot\n", "\n", "- To return to the default Julia environment, simply " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mActivating\u001b[22m\u001b[39m environment at `~/repos/lecture-source-jl/_build/website/jupyter/Project.toml`\n" ] } ], "source": [ "] activate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "without any arguments\n", "\n", "- Lastly, let’s clean up " ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "hide-output": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed\n" ] } ], "source": [ "; cd .." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; rm -rf ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### InstantiateFromURL\n", "\n", "With this knowledge, we can explain the operation of the setup block" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "hide-output": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32m\u001b[1mActivated\u001b[0m /home/qebuild/repos/lecture-source-jl/_build/website/jupyter/Project.toml\u001b[39m\n", "\u001b[36m\u001b[1mInfo\u001b[0m quantecon-notebooks-julia 0.1.0 activated, 0.2.0 requested\u001b[39m\n" ] } ], "source": [ "using InstantiateFromURL\n", "github_project(\"QuantEcon/quantecon-notebooks-julia\", version = \"0.2.0\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What this `activate_github` function does is\n", "\n", "1. Download the TOML from that repo to a directory called `.projects` \n", "1. `] activate` that environment, and \n", "1. `] instantiate` and `] precompile`, if necessary \n", "\n", "\n", "_docker_main:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Docker\n", "\n", "Docker is a tool that lets you run preconfigured, lightweight environments as applications on your computer or in a computational cloud\n", "\n", "The advantage of a Docker-based workflow is that it’s perfectly reproducible, and that setup (of Julia versions and dependencies, etc.) is handled upstream by the image maintainer\n", "\n", "Here, we’ll walk through the setup and installation steps, along with the main features of the `quantecon/base` Docker image" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup\n", "\n", "- First, create an account for [Docker Hub](https://hub.docker.com/) and create a docker id \n", "- Next, download and install Docker for \n", " \n", " - [Mac](https://store.docker.com/editions/community/docker-ce-desktop-mac) \n", " - [Windows](https://store.docker.com/editions/community/docker-ce-desktop-windows) - **do not** choose to use Windows containers \n", " \n", "\n", "\n", "**Note:** For Windows\n", "\n", "> - Hyper-V support should be enabled. For Windows 10 users, this means you must use a Pro, Enterprise, or Education version (**not Home or Mobile**) \n", "- If you don’t meet these requirements, the [Docker Toolbox for Windows](https://docs.docker.com/toolbox/toolbox_install_windows/) may help \n", "\n", "\n", "\n", "- Next, to verify that there are no obvious errors in the installation, open a terminal (macOS/Linux) or Powershell (Windows) and run " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker pull hello-world\n", "docker run hello-world\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "You should see something like\n", "\n", "\n", "\n", " \n", "- Then, download the QuantEcon Docker image by running the following in your terminal (this may take some time depending on your internet connection) " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker pull quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "- Next, create a “data volume,” or a hidden directory where Docker will persist any changes to your Julia packages " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker volume rm quantecon\n", "docker volume create quantecon\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "The first line will delete any existing volume we had with that name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Usage\n", "\n", "The basic command is (Linux, OS/X)" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v \"$(pwd)\":/home/jovyan/local quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or on Powershell on Windows" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v ${PWD}:/home/jovyan/local quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The `rm` instructs Docker to delete the container on exit, \n", "- The `PWD` statement will mount the local directory (i.e., where the terminal is) to the Docker for exchanging files \n", "- The `p` flag is for browser access \n", "- The `quantecon:/home/jovyan/.julia` mount is for persisting changes we make to the Julia user depot \n", "\n", "\n", "You will see something like\n", "\n", "\n", "\n", " \n", "In the output, you should see some text near that bottom that looks like" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "127.0.0.1):8888/?token=7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Copy the text after `?token=` (e.g. `7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b`)\n", "\n", "In a browser, go to a URL like the following" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "http://127.0.0.1:8888/lab\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To see something like\n", "\n", "\n", "\n", " \n", "**Note**:\n", "\n", "- `Ctrl+C` is also the keyboard shortcut you use to kill the container, so be sure to copy using the mouse \n", "- When you call this command, Docker may require you to give it permissions to access the drive and the network. If you do not see the output within 20 or so seconds, then look for confirmation windows which may be hidden behind the terminal/etc. \n", "\n", "\n", "Paste the text into `Password or token:` and choose `Log in` to get the full window\n", "\n", "\n", "\n", " \n", "We can see that some packages are already pre-installed for our use\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Maintenance and Troubleshooting\n", "\n", "A few reminders\n", "\n", "- If you forget your token number, you may need to stop and restart the container \n", "- To stop the container, use `Ctrl-C` or type `docker stop \\$(docker ps -aq)` in a different terminal \n", "- To reset your Docker volume completely, redo the `docker volume rm quantecon` and `docker volume create quantecon` steps \n", "- To clean unnecessary Docker assets from your system, run `docker system prune` \n", "- If you can’t log in, make sure you don’t have an existing jupyter notebook occupying port 8888. To check, run `jupyter notebook list` and (if need be) `jupyter notebook stop 8888`. If you have difficulties, see [this git issue](https://github.com/jupyter/notebook/issues/2844) " ] } ], "metadata": { "filename": "tools_editors.rst", "kernelspec": { "display_name": "Julia 1.2", "language": "julia", "name": "julia-1.2" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.2.0" }, "title": "Julia Tools and Editors" }, "nbformat": 4, "nbformat_minor": 2 }