{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "
\n", " \n", " \"QuantEcon\"\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Julia Tools and Editors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "- [Julia Tools and Editors](#Julia-Tools-and-Editors) \n", " - [Preliminary Setup](#Preliminary-Setup) \n", " - [The REPL](#The-REPL) \n", " - [Atom](#Atom) \n", " - [Package Environments](#Package-Environments) \n", " - [Docker](#Docker) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Co-authored with Arnav Sood\n", "\n", "While Jupyter notebooks are a great way to get started with the language, eventually you will want to use more powerful tools\n", "\n", "We’ll discuss a few of them here, such as\n", "\n", "- Text editors like Atom, which come with rich Julia support for debugging, documentation, git integration, plotting and inspecting data, and code execution \n", "- The Julia REPL, which has specialized modes for package management, shell commands, and help \n", "- A virtualized Docker setup which provides a painless pre-configured environment on your machine \n", "\n", "\n", "Note that we assume you’ve already completed the [getting started](getting_started_julia/getting_started.ipynb) and [interacting with Julia](getting_started_julia/julia_environment.ipynb) lectures" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preliminary Setup\n", "\n", "Follow the instructions for setting up Julia [on your local computer](getting_started_julia/getting_started.ipynb#jl-jupyterlocal)\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating a Startup File (Recommended)\n", "\n", "Whenever the Julia compiler or REPL starts, it will look for a file called `startup.jl` (see [Julia Manual](https://docs.julialang.org/en/v1/manual/getting-started/#man-getting-started-1))\n", "\n", "We provide a file here which does two things\n", "\n", "- Makes the REPL shell mode “sticky,” so you don’t need to keep running `;` for new commands \n", "- Loads the `Revise.jl` package on startup, which lets you see changes you make to a package in real-time (i.e., no need to quit the REPL, open again, and load again) \n", "\n", "\n", "The location for the file is relative to your default Julia environment (e.g. `~/.julia/config/startup.jl` or `C:\\Users\\USERNAME\\.julia\\config\\startup.jl` on Windows)\n", "\n", "Recall that you can find the location of the `~/.julia` directory by running" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "DEPOT_PATH[1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note:** On Mac, this won’t be visible in the Finder unless you specifically enable that option, but you can get to it by running `cd .julia; open .` from a new terminal\n", "\n", "To add the file:\n", "\n", "- In the `julia` terminal, type the following " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "] add Revise REPL; precompile\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "- Create the `~/.julia/config/` directory if necessary in the terminal or file explorer \n", "- Download the file [startup.jl](https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/jl/_static/includes/startup.jl) into that directory \n", "- For convenience, you may find it useful on your operating system to change the directory where the REPL starts \n", "\n", "\n", "On Windows, if you have a shortcut on your desktop or on the taskbar, you could: (1) right-click on the icon; (2) right click on the “julia” text; (3) choose “Properties”, and (4) change the “Start In” to be something such as `C:\\Users\\YOURUSERNAME\\Documents`\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The REPL\n", "\n", "Previously, we discussed basic use of the Julia REPL (“Read-Evaluate-Print Loop”)\n", "\n", "Here, we’ll consider some more advanced features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Shell Mode\n", "\n", "Hitting `;` brings you into shell mode, which lets you run bash commands (PowerShell on Windows)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; pwd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also use Julia variables from shell mode" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "x = 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; echo $x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Package Mode\n", "\n", "Hitting `]` brings you into package mode\n", "\n", "- `] add Expectations` will add a package (here, `Expectations.jl`) \n", "- Likewise, `] rm Expectations` will remove that package \n", "- `] st` will show you a snapshot of what you have installed \n", "- `] up` will (intelligently) upgrade versions of your packages \n", "- `] precompile` will precompile everything possible \n", "\n", "\n", "You can get a full list of package mode commands by running" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] ?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On some operating systems (such as OSX) REPL pasting may not work for package mode, and you will need to access it in the standard way (i.e., hit `]` first and then run your commands)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Help Mode\n", "\n", "Hitting `?` will bring you into help mode\n", "\n", "The key use case is to find docstrings for functions and macros, e.g." ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? print\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that objects must be loaded for Julia to return their documentation, e.g." ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? @test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "will fail, but" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "using Test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "? @test\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "will succeed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Atom\n", "\n", "As discussed [previously](getting_started_julia/getting_started.ipynb), eventually you will want to use a fully fledged text editor\n", "\n", "The most feature-rich one for Julia development is [Atom](https://atom.io/), with the [Juno](http://junolab.org/) package\n", "\n", "There are several reasons to use a text editor like Atom, including\n", "\n", "- Git integration (more on this in the [next lecture](version_control.ipynb)) \n", "- Painless inspection of variables and data \n", "- Easily run code blocks, and drop in custom snippets of code \n", "- Integration with Julia documentation and plots " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Installation and Configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Installing Atom\n", "\n", "1. Download and Install Atom from the [Atom website](https://atom.io/) \n", "1. (Optional, but recommended): Change default Atom settings \n", " \n", " - Use `Ctrl-,` to get the `Settings` pane \n", " - Choose the `Packages` tab \n", " - Type `line-ending-selector` into the Filter and then click “Settings” for that package \n", " \n", " - Change the default line ending to `LF` (only necessary on Windows) \n", " \n", " - Choose the Editor tab \n", " \n", " - Turn on `Soft Wrap` \n", " - Set the `Tab Length` default to `4` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Installing Juno\n", "\n", "1. Use `Ctrl-,` to get the Settings pane \n", "1. Go to the `Install` tab \n", "1. Type `uber-juno` into the search box and then click Install on the package that appears \n", "1. Wait while Juno installs dependencies \n", "1. When it asks you whether or not to use the standard layout, click `yes` \n", "\n", "\n", "At that point, you should see a built-in REPL at the bottom of the screen and be able to start using Julia and Atom" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Troubleshooting\n", "\n", "Sometimes, Juno will fail to find the Julia executable (say, if it’s installed somewhere nonstandard, or you have multiple)\n", "\n", "To do this\n", "1. `Ctrl-,` to get Settings pane, and select the Packages tab\n", "2. Type in `julia-client` and choose Settings\n", "3. Find the Julia Path, and fill it in with the location of the Julia binary\n", "\n", "> - To find the binary, you could run `Sys.BINDIR` in the REPL, then add in an additional `/julia` to the end of the screen \n", "- e.g. `C:\\Users\\YOURUSERNAME\\AppData\\Local\\Julia-1.0.1\\bin\\julia.exe` on Windows as `/Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia` on OSX \n", "\n", "\n", "\n", "See the [setup instructions for Juno](http://docs.junolab.org/latest/man/installation.html) if you have further issues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Standard Layout\n", "\n", "If you follow the instructions, you should see something like this when you open a new file\n", "\n", "If you don’t, simply go to the command palette and type “Julia standard layout”\n", "\n", "![_static/figures/juno-standard-layout.png](_static/figures/juno-standard-layout.png) \n", "The bottom pane is a standard REPL, which supports the different modes above\n", "\n", "The “workspace” pane is a snapshot of currently-defined objects\n", "\n", "For example, if we define an object in the REPL" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "x = 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our workspace should read\n", "\n", "![_static/figures/juno-workspace-1.png](_static/figures/juno-workspace-1.png) \n", "The `ans` variable simply captures the result of the last computation\n", "\n", "The `Documentation` pane simply lets us query Julia documentation\n", "\n", "![_static/figures/juno-docs.png](_static/figures/juno-docs.png) \n", "The `Plots` pane captures Julia plots output (the code is as follows)" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```julia\n", "using Plots\n", "gr(fmt = :png);\n", "data = rand(10, 10)\n", "h = heatmap(data)\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![_static/figures/juno-plots.png](_static/figures/juno-plots.png) \n", "**Note:** The plots feature is not perfectly reliable across all plotting backends, see [the Basic Usage](http://docs.junolab.org/latest/man/basic_usage.html) page" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other Features\n", "\n", "- `` Shift + Enter `` will evaluate a highlighted selection or line (as above) \n", "- The run symbol in the left sidebar (or `Ctrl+Shift+Enter`) will run the whole file \n", "\n", "\n", "See [basic usage](http://docs.junolab.org/latest/man/basic_usage.html) for an exploration of features, and the [FAQ](http://docs.junolab.org/latest/man/faq.html) for more advanced steps (e.g. using with `Docker`)\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Package Environments\n", "\n", "Julia’s package manager lets you set up Python-style “virtualenvs,” or subsets of packages that draw from an underlying pool of assets on the machine\n", "\n", "This way, you can work with (and specify) the dependencies (i.e., required packages) for one project without worrying about impacts on other projects\n", "\n", "- An `environment` is a set of packages specified by a `Project.toml` (and optionally, a `Manifest.toml`) \n", "- A `registry` is a git repository corresponding to a list of (typically) registered packages, from which Julia can pull (for more on git repositories, see [version control](version_control.ipynb)) \n", "- A `depot` is a directory, like `~/.julia`, which contains assets (compile caches, registries, package source directories, etc.) \n", "\n", "\n", "Essentially, an environment is a dependency tree for a project, or a “frame of mind” for Julia’s package manager\n", "\n", "- We can see the default (`v1.1`) environment as such " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] st" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- We can also create and activate a new environment " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] generate ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- And go to it " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; cd ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- To activate the directory, simply " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] activate ." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where “.” stands in for the “present working directory”\n", "\n", "- Let’s make some changes to this " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] add Expectations Parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note the lack of commas\n", "\n", "- To see the changes, simply open the `ExampleEnvironment` directory in an editor like Atom \n", "\n", "\n", "The Project TOML should look something like this" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "name = \"ExampleEnvironment\"\n", "uuid = \"14d3e79e-e2e5-11e8-28b9-19823016c34c\"\n", "authors = [\"QuantEcon User \"]\n", "version = \"0.1.0\"\n", "\n", "[deps]\n", "Expectations = \"2fe49d83-0758-5602-8f54-1f90ad0d522b\"\n", "Parameters = \"d96e819e-fc66-5662-9728-84c9c7592b0a\"\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] precompile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note** The TOML files are independent of the actual assets (which live in `~/.julia/packages`, `~/.julia/dev`, and `~/.julia/compiled`)\n", "\n", "You can think of the TOML as specifying demands for resources, which are supplied by the `~/.julia` user depot\n", "\n", "- To return to the default Julia environment, simply " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "] activate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "without any arguments\n", "\n", "- Lastly, let’s clean up " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; cd .." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "; rm -rf ExampleEnvironment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### InstantiateFromURL\n", "\n", "With this knowledge, we can explain the operation of the setup block" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": true }, "outputs": [], "source": [ "using InstantiateFromURL\n", "github_project(\"QuantEcon/quantecon-notebooks-julia\", version = \"0.2.0\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What this `activate_github` function does is\n", "\n", "1. Download the TOML from that repo to a directory called `.projects` \n", "1. `] activate` that environment, and \n", "1. `] instantiate` and `] precompile`, if necessary \n", "\n", "\n", "_docker_main:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Docker\n", "\n", "Docker is a tool that lets you run preconfigured, lightweight environments as applications on your computer or in a computational cloud\n", "\n", "The advantage of a Docker-based workflow is that it’s perfectly reproducible, and that setup (of Julia versions and dependencies, etc.) is handled upstream by the image maintainer\n", "\n", "Here, we’ll walk through the setup and installation steps, along with the main features of the `quantecon/base` Docker image" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup\n", "\n", "- First, create an account for [Docker Hub](https://hub.docker.com/) and create a docker id \n", "- Next, download and install Docker for \n", " \n", " - [Mac](https://store.docker.com/editions/community/docker-ce-desktop-mac) \n", " - [Windows](https://store.docker.com/editions/community/docker-ce-desktop-windows) - **do not** choose to use Windows containers \n", " \n", "\n", "\n", "**Note:** For Windows\n", "\n", "> - Hyper-V support should be enabled. For Windows 10 users, this means you must use a Pro, Enterprise, or Education version (**not Home or Mobile**) \n", "- If you don’t meet these requirements, the [Docker Toolbox for Windows](https://docs.docker.com/toolbox/toolbox_install_windows/) may help \n", "\n", "\n", "\n", "- Next, to verify that there are no obvious errors in the installation, open a terminal (macOS/Linux) or Powershell (Windows) and run " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker pull hello-world\n", "docker run hello-world\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "You should see something like\n", "\n", "![_static/figures/docker-hello-world.png](_static/figures/docker-hello-world.png) \n", "- Then, download the QuantEcon Docker image by running the following in your terminal (this may take some time depending on your internet connection) " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker pull quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "- Next, create a “data volume,” or a hidden directory where Docker will persist any changes to your Julia packages " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker volume rm quantecon\n", "docker volume create quantecon\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "The first line will delete any existing volume we had with that name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Usage\n", "\n", "The basic command is (Linux, OS/X)" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v \"$(pwd)\":/home/jovyan/local quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or on Powershell on Windows" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v ${PWD}:/home/jovyan/local quantecon/base\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The `rm` instructs Docker to delete the container on exit, \n", "- The `PWD` statement will mount the local directory (i.e., where the terminal is) to the Docker for exchanging files \n", "- The `p` flag is for browser access \n", "- The `quantecon:/home/jovyan/.julia` mount is for persisting changes we make to the Julia user depot \n", "\n", "\n", "You will see something like\n", "\n", "![_static/figures/docker-basic-command.png](_static/figures/docker-basic-command.png) \n", "In the output, you should see some text near that bottom that looks like" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "127.0.0.1):8888/?token=7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Copy the text after `?token=` (e.g. `7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b`)\n", "\n", "In a browser, go to a URL like the following" ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "http://127.0.0.1:8888/lab\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To see something like\n", "\n", "![_static/figures/docker-jupyter-lab.png](_static/figures/docker-jupyter-lab.png) \n", "**Note**:\n", "\n", "- `Ctrl+C` is also the keyboard shortcut you use to kill the container, so be sure to copy using the mouse \n", "- When you call this command, Docker may require you to give it permissions to access the drive and the network. If you do not see the output within 20 or so seconds, then look for confirmation windows which may be hidden behind the terminal/etc. \n", "\n", "\n", "Paste the text into `Password or token:` and choose `Log in` to get the full window\n", "\n", "![_static/figures/docker-jlab-full.png](_static/figures/docker-jlab-full.png) \n", "We can see that some packages are already pre-installed for our use\n", "\n", "![_static/figures/docker-packages-preinstalled.png](_static/figures/docker-packages-preinstalled.png) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Maintenance and Troubleshooting\n", "\n", "A few reminders\n", "\n", "- If you forget your token number, you may need to stop and restart the container \n", "- To stop the container, use `Ctrl-C` or type `docker stop \\$(docker ps -aq)` in a different terminal \n", "- To reset your Docker volume completely, redo the `docker volume rm quantecon` and `docker volume create quantecon` steps \n", "- To clean unnecessary Docker assets from your system, run `docker system prune` \n", "- If you can’t log in, make sure you don’t have an existing jupyter notebook occupying port 8888. To check, run `jupyter notebook list` and (if need be) `jupyter notebook stop 8888`. If you have difficulties, see [this git issue](https://github.com/jupyter/notebook/issues/2844) " ] } ], "metadata": { "filename": "tools_editors.rst", "kernelspec": { "display_name": "Julia 1.2", "language": "julia", "name": "julia-1.2" }, "title": "Julia Tools and Editors" }, "nbformat": 4, "nbformat_minor": 2 }