{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "
\n", " \n", " \"QuantEcon\"\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Git, GitHub, and Version Control" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "- [Git, GitHub, and Version Control](#Git,-GitHub,-and-Version-Control) \n", " - [Setup](#Setup) \n", " - [Basic Objects](#Basic-Objects) \n", " - [Individual Workflow](#Individual-Workflow) \n", " - [Collaborative Work](#Collaborative-Work) \n", " - [Collaboration via Pull Request](#Collaboration-via-Pull-Request) \n", " - [Additional Resources and Troubleshooting](#Additional-Resources-and-Troubleshooting) \n", " - [Exercises](#Exercises) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Co-authored with Arnav Sood\n", "\n", "An essential part of modern software engineering is using version control\n", "\n", "We use version control because\n", "\n", "- Not all iterations on a file are perfect, and you may want to revert changes \n", "- We want to be able to see who has changed what and how \n", "- We want a uniform version scheme to do this between people and machines \n", "- Concurrent editing on code is necessary for collaboration \n", "- Version control is an essential part of creating reproducible research \n", "\n", "\n", "In this lecture, we’ll discuss how to use Git and GitHub" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "1. Make sure you create an account on [GitHub.com](http://github.com/) \n", " \n", " - If you are a student, be sure to use the GitHub [Student Developer Pack](https://education.github.com/pack/) \n", " - Otherwise, see if you qualify for a free [Non-Profit/Academic Plan](https://help.github.com/articles/about-github-education-for-educators-and-researchers/) \n", " - These come with things like unlimited private repositories, testing support, etc. \n", " \n", "1. Install `git` and the GitHub Desktop application \n", " \n", " 1. Install [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git/) \n", " 1. Install the [GitHub Desktop](https://desktop.github.com/) application \n", " \n", "1. Optionally (but strongly recommended): On Windows, change the default line-ending by: \n", "\n", "1. Opening a Windows/Powershell console, or the “Git Bash” installed in the previous step \n", "1. Running the following " ] }, { "cell_type": "markdown", "metadata": { "hide-output": false }, "source": [ "```text\n", "git config --global core.eol lf\n", "git config --global core.autocrlf false\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "\n", "### Git vs. GitHub vs. GitHub Desktop\n", "\n", "To understand the relationship\n", "\n", "- Git is an infrastructure for versioning and merging files (it is not specific to GitHub and does not even require an online server) \n", "- GitHub provides an online service to coordinate working with Git repositories, and adds some additional features for managing projects \n", "- GitHub Desktop is just one of many GUI-based clients to make Git and GitHub easier to use \n", "\n", "\n", "Later, you may find yourself using alternatives\n", "\n", "- GitHub is the market leader for open source projects and Julia, but there are other options, e.g. [GitLab](https://about.gitlab.com/) and [Bitbucket](https://bitbucket.org) \n", "- Instead of the GitHub Desktop, you may directly use the Git command line, [GitKraken](https://www.gitkraken.com/), or use the Git functionality built into editors such as [Atom](https://atom.io/) or [VS Code](https://code.visualstudio.com/) \n", "\n", "\n", "Since these lecture notes are intended to provide a minimal path to using the technologies, here we will conflate the workflow of these distinct products" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic Objects" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Repositories\n", "\n", "The fundamental object in GitHub is a *repository* (or “repo”) – this is the master directory for a project\n", "\n", "One example of a repo is the QuantEcon [Expectations.jl](https://github.com/quantecon/expectations.jl/) package\n", "\n", "On the machine, a repo is a normal directory, along with a subdirectory called `.git` which contains the history of changes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Commits\n", "\n", "GitHub stores history as a sequence of changes to text, called *commits*\n", "\n", "[Here](https://github.com/QuantEcon/lecture-source-jl/commit/ba59c3ea9a0dec10def3f4f3928af5e2827f3b92) is an example of a commit, which revises the style guide in a QuantEcon repo\n", "\n", "In particular, commits have the following features\n", "\n", "- An ID (formally, an “SHA-1 hash”) \n", "- Content (i.e., a before and after state) \n", "- Metadata (author, timestamp, commit message, etc.) \n", "\n", "\n", "**Note:** It’s crucial to remember that what’s stored in a commit is only the actual changes you make to text\n", "\n", "This is a key reason why git can store long and complicated histories without consuming massive amounts of memory" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Common Files\n", "\n", "In addition, each GitHub repository typically comes with a few standard text files\n", "\n", "- A `.gitignore` file, which lists files/extensions/directories that GitHub shouldn’t try to track (e.g., LaTeX compilation byproducts) \n", "- A `README.md` file, which is a Markdown file which GitHub puts on the repository website \n", "- A `LICENSE.txt` file, which describes the terms under which the repository’s contents are made available \n", "\n", "\n", "For an example of all three, see the [Expectations.jl](https://github.com/quantecon/expectations.jl/) repo\n", "\n", "Of these, the `README.md` is the most important, as GitHub will display it as [Markdown](https://guides.github.com/features/mastering-markdown/) when accessing the repository online\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Individual Workflow\n", "\n", "In this section, we’ll describe how to use GitHub to version your own projects\n", "\n", "Much of this will carry over to the collaborative section" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating a Repository\n", "\n", "In general, we will always want to repos for new projects using the following dropdown\n", "\n", "\n", "\n", " \n", "We can then configure repository options as such\n", "\n", "\n", "\n", " \n", "In this case, we’re making a public repo `github.com/quantecon_user/example_repository`, which will come with a `README.md`, is licensed under the MIT License, and will ignore Julia compilation byproducts\n", "\n", "**Note** This workflow is for creating projects *de novo*; the process for turning existing directories into git repos is a bit more complicated\n", "\n", "In particular, in that case we recommend that you create a new repo via this method, then copy in and commit your files (see below), and then delete the old directory" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Cloning a Repository\n", "\n", "The next step is to get this to our local machine\n", "\n", "\n", "\n", " \n", "This dropdown gives us a few options\n", "\n", "- “Open in Desktop” will call to the GitHub Desktop application that we’ve installed \n", "- “Download Zip” will download the directory *without the .git* subdirectory (avoid this option) \n", "- The copy/paste button next to the link lets us use the command line, i.e. `git clone https://github.com/quanteconuser/example_repository.git` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making and Managing Changes\n", "\n", "Now that we have the repository, we can start working with it\n", "\n", "For example, let’s say that we’ve amended the `README.md` (using our editor of choice), and also added a new file `economics.jl` which we’re still working on\n", "\n", "Returning to GitHub Desktop, we should see something like\n", "\n", "\n", "\n", " \n", "To select individual files for commit, we can use the check boxes to the left of each file\n", "\n", "Let’s say you select only the README to commit. Going to the history tab should show you our change\n", "\n", "\n", "\n", " \n", "The Julia file is unchanged" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pushing to the Server\n", "\n", "As of now, this commit lives only on our local machine\n", "\n", "To upload it to the server, you can simply click the “Push Origin” button at the top the screen\n", "\n", "The small “1^” to the right of the text indicates we have one commit to upload" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reading and Reverting History\n", "\n", "As mentioned, one of the key features of GitHub is the ability to scan through history\n", "\n", "By clicking the “commits” tab on the repo front page,\n", "we see [this page](https://github.com/quanteconuser/example_repository/commits/master)\n", "(as an example)\n", "\n", "Clicking an individual commit gives us the difference view, (e.g., [example commit](https://github.com/quanteconuser/example_repository/commit/d0b17f5ce0f8742e88da9b604bfed418d6a16884/))\n", "\n", "Sometimes, however, we want to not only inspect what happened before, but reverse the commit\n", "\n", "- If you haven’t made the commit yet, just right-click the file in the “changes” tab and hit “discard changes” to reset the file to the last known commit \n", "- If you have made the commit but haven’t pushed to the server yet, go to the “history” tab as above, right click the commit and click “revert this commit.” This will create the inverse commit, shown below \n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Working across Machines\n", "\n", "Generally, you want to work on the same project but across multiple machines (e.g., a home laptop and a lab workstation)\n", "\n", "The key is to push changes from one machine, and then to pull changes from the other machine\n", "\n", "Pushing can be done as above\n", "\n", "To pull, simply click pull under the “repository” dropdown at the top of the screen\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Collaborative Work" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding Collaborators\n", "\n", "First, let’s add a collaborator to the `quanteconuser/example_repository` lecture we created earlier\n", "\n", "We can do this by clicking “settings => collaborators,” as follows\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Project Management\n", "\n", "GitHub’s website also comes with project management tools to coordinate work between people\n", "\n", "The main one is an *issue*, which we can create from the issues tab\n", "\n", "You should see something like this\n", "\n", "\n", "\n", " \n", "Let’s unpack the different components\n", "\n", "- The *assignees* dropdown lets you select people tasked to work on the issue \n", "- The *labels* dropdown lets you tag the issue with labels visible from the issues page, such as “high priority” or “feature request” \n", "- It’s possible to tag other issues and collaborators (including in different repos) by linking to them in the comments – this is part of what’s called *GitHub-Flavored Markdown* \n", "\n", "\n", "For an example of an issue, see [here](https://github.com/quanteconuser/example_repository/issues/1)\n", "\n", "You can see open issues at a glance from the general issues tab\n", "\n", "\n", "\n", " \n", "The checkboxes are common in GitHub to manage project tasks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reviewing Code\n", "\n", "There are a few different ways to review people’s code in GitHub\n", "\n", "- Whenever people push to a project you’re working on, you’ll receive an email notification \n", "- You can also review individual line items or commits by opening commits in the difference view as [above](https://github.com/quanteconuser/example_repository/commit/d0b17f5ce0f8742e88da9b604bfed418d6a16884/) \n", "\n", "\n", "\n", "\n", " \n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Merge Conflicts\n", "\n", "Any project management tool needs to figure out how to reconcile conflicting changes between people\n", "\n", "In GitHub, this event is called a “merge conflict,” and occurs whenever people make conflicting changes to the same *line* of code\n", "\n", "Note that this means that two people touching the same file is OK, so long as the differences are compatible\n", "\n", "A common use case is when we try to push changes to the server, but someone else has pushed conflicting changes\n", "\n", "GitHub will give us the following window\n", "\n", "\n", "\n", " \n", "- The warning symbol next to the file indicates the existence of a merge conflict \n", "- The viewer tries to show us the discrepancy (I changed the word repository to repo, but someone else tried to change it to “repo” with quotes) \n", "\n", "\n", "To fix the conflict, we can go into a text editor (such as Atom)\n", "\n", "\n", "\n", " \n", "Let’s say we click the first “use me” (to indicate that my changes should win out), and then save the file\n", "\n", "Returning to GitHub Desktop gives us a pre-formed commit to accept\n", "\n", "\n", "\n", " \n", "Clicking “commit to master” will let us push and pull from the server as normal" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Collaboration via Pull Request\n", "\n", "One of the defining features of GitHub is that it is the dominant platform for *open source* code, which anyone can access and use\n", "\n", "However, while anyone can make a copy of the source code, not everyone has access to modify the particular version stored on GitHub\n", "\n", "A maintainer (i.e. someone with “write” access to directly modify a repository)\n", "might consider different contributions and “merge” the changes into the main\n", "repository if the changes meet their criteria\n", "\n", "A *pull request* (“PR”) allows **any** outsiders to suggest changes to open source repositories\n", "\n", "A PR requests the project maintainer to merge (“pull”) changes you’ve worked on into their repository\n", "\n", "There are a few different workflows for creating and handling PRs, which we’ll walk through below\n", "\n", "**Note:** If the changes are for a Julia Package, you will need to follow a different workflow – described in the [testing lecture](https://julia.quantecon.org/testing.html)\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Quick Fixes\n", "\n", "GitHub’s website provides an online editor for quick and dirty changes, such as fixing typos in documentation\n", "\n", "To use it, open a file in GitHub and click the small pencil to the upper right\n", "\n", "\n", "\n", " \n", "Here, we’re trying to add the QuantEcon link to the Julia project’s `README` file\n", "\n", "After making our changes, we can then describe and propose them for review by maintainers\n", "\n", "But what if we want to make more in-depth changes?\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### No-Access Case\n", "\n", "A common problem is when we don’t have write access (i.e. we can’t directly modify) the repo in question\n", "\n", "In that case, click the “Fork” button that lives in the top-right of every repo’s main page\n", "\n", "\n", "\n", " \n", "This will copy the repo into your own GitHub account\n", "\n", "For example, [this repo](https://github.com/ubcecon/example_repository) is a fork of our original [git setup](https://github.com/quanteconuser/example_repository/)\n", "\n", "Clone this fork to our desktop and work with it in exactly the same way as we would a repo we own\n", "(as the fork is in your account, you now have write access)\n", "\n", "That is, click the “clone” button on our fork\n", "\n", "\n", "\n", " \n", "You’ll see a new repo with the same name but different URL in your GitHub Desktop repo list, along with a special icon to indicate that it’s a fork\n", "\n", "\n", "\n", " \n", "Commit some changes by selecting the files and writing a commit message\n", "\n", "\n", "\n", " \n", "And push by using the dropdown\n", "\n", "\n", "\n", " \n", "Below, for example, we’ve committed and pushed some changes to the fork that we want to upstream into the main repo\n", "\n", "\n", "\n", " \n", "We should make sure that these changes are on the server (which we can get to by going to the [fork](https://github.com/ubcecon/example_repository) and clicking “commits”)\n", "\n", "\n", "\n", " \n", "Next, go to the pull requests menu and click “New Pull Request”\n", "\n", "You’ll see something like this\n", "\n", "\n", "\n", " \n", "This gives us a quick overview of the commits we want to merge in, as well as the overall differences\n", "\n", "Hit create and then click through the following form\n", "\n", "This opens a page like this on the main repo\n", "\n", "\n", "\n", " \n", "The key pieces are\n", "\n", "- A list of the commits we’re proposing \n", "- A list of reviewers, who can approve or modify our changes \n", "- Labels, Markdown space, assignees, and the ability to tag other git issues and PRs, just as with issues \n", "\n", "\n", "Here’s an [example pull request](https://github.com/quanteconuser/example_repository/pull/3)\n", "\n", "To edit a PR, simply push changes to the fork you cloned to your desktop\n", "\n", "For example, let’s say we commit a new change to the README *after* we create the PR\n", "\n", "\n", "\n", " \n", "After pushing to the server, the change is reflected on the PR [page](https://github.com/quanteconuser/example_repository/pull/3)\n", "\n", "\n", "\n", " \n", "That is, creating a pull request is not like bundling up your changes and delivering them, but rather like opening an *ongoing connection* between two repositories, that is only severed when the PR is closed or merged" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Write Access Case\n", "\n", "As you become more familiar with GitHub, and work on larger projects, you will find yourself making PRs even when it isn’t strictly required\n", "\n", "If you are a maintainer of the repo (e.g. you created it or are a collaborator) then you don’t need to create a fork, but will rather work with a *git branch*\n", "\n", "Branches in git represent parallel development streams (i.e., sequences of commits) that the PR is trying to merge\n", "\n", "First, load the repo in GitHub Desktop and use the branch dropdown\n", "\n", "\n", "\n", " \n", "Click “New Branch” and choose an instructive name (make sure there are no spaces or special characters)\n", "\n", "This will “check out” a new branch with the same history as the old one (but new commits will be added only to this branch)\n", "\n", "We can see the active branch in the top dropdown\n", "\n", "\n", "\n", " \n", "For example, let’s say we add some stuff to the Julia code file and commit it\n", "\n", "\n", "\n", " \n", "To put this branch (with changes) on the server, we simply need to click “Publish Branch”\n", "\n", "Navigating to the [repo page](https://github.com/quanteconuser/example_repository), we will see a suggestion about a new branch\n", "\n", "\n", "\n", " \n", "At which point the process of creating a PR is identical to the previous case" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Julia Package Case\n", "\n", "One special case is when the repo in question is actually a Julia project or package\n", "\n", "We cover that (along with package workflow in general) in the [testing lecture](https://julia.quantecon.org/testing.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Additional Resources and Troubleshooting\n", "\n", "You may want to go beyond the scope of this tutorial when working with GitHub\n", "\n", "For example, perhaps you run into a bug, or you’re working with a setup that doesn’t have GitHub Desktop installed\n", "\n", "Here are some resources to help\n", "\n", "- Kate Hudson’s excellent [git flight rules](https://github.com/k88hudson/git-flight-rules/), which is a near-exhaustive list of situations you could encounter, and command-line fixes \n", "- The GitHub [Learning Lab](https://lab.github.com/), an interactive sandbox environment for git \n", "- The docs for forking on [GitHub Desktop](https://help.github.com/desktop/guides/contributing-to-projects/cloning-a-repository-from-github-to-github-desktop/) and [the GitHub Website](https://guides.github.com/activities/forking/) \n", "\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Command-Line Basics\n", "\n", "Git also comes with a set of command-line tools\n", "\n", "They’re optional, but many people like using them\n", "\n", "Furthermore, in some environments (e.g. JupyterHub installations) you may only have access to the command line\n", "\n", "- On Windows, downloading `git` will have installed a program called `git bash`, which installs these tools along with a general Linux-style shell \n", "- On Linux/MacOS, these tools are integrated into your usual terminal \n", "\n", "\n", "To open the terminal in a directory, either right click and hit “open git bash” (in Windows), or use Linux commands like `cd` and `ls` to navigate\n", "\n", "See [here](https://www.git-tower.com/learn/git/ebook/en/command-line/appendix/command-line-101) for a short introduction to the command line\n", "\n", "As above, you can clone by grabbing the repo URL (say, GitHub’s [site-policy repo](https://github.com/github/site-policy/)) and running `git clone https://github.com/github/site-policy.git`\n", "\n", "This won’t be connected to your GitHub Desktop, so you’d need to use it manually (`File => Add Local Repository`) or drag-and-drop from the file explorer onto the GitHub Desktop\n", "\n", "\n", "\n", " \n", "From here, you can get the latest files on the server by `cd`-ing into the directory and running `git pull`\n", "\n", "When you `pull` from the server, it will never overwrite your modified files, so it is impossible to lose local changes\n", "\n", "Instead, to do a hard reset of all files and overwrite any of your local changes, you can run `git reset --hard origin/master`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercises" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1a\n", "\n", "Follow the instructions to create a [new repository](#new-repo-workflow) for one of your GitHub accounts\n", "In this repository\n", "\n", "- Take the code from one of your previous assignments, such as [Newton’s method](https://julia.quantecon.org/../getting_started_julia/julia_by_example.html#jbe-ex8a) in [Introductory Examples](https://julia.quantecon.org/../getting_started_julia/julia_by_example.html) (either as a `.jl` file or a Jupyter notebook) \n", "- Put in a `README.md` with some text \n", "- Put in a `.gitignore` file, ignoring the Jupyter files `.ipynb_checkpoints` and the project files, `.projects` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1b\n", "\n", "Pair-up with another student who has done Exercise 1a and find out their GitHub ID, and each do the following\n", "\n", "- Add the GitHub ID as a collaborators on your repository \n", "- Clone the repositories to your local desktop \n", "- Assign each other an issue \n", "- Submit a commit from GitHub Desktop which references the issue by number \n", "- Comment on the commits \n", "- Ensure you can run their code without any modifications " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1c\n", "\n", "Pair-wise with the results of Exercise 1b examine a merge-conflict by editing the `README.md` file for your repository that you have both setup as collaborators\n", "\n", "Start by ensuring there are multiple lines in the file so that some changes may have conflicts, and some may not\n", "\n", "- Clone the repository to your local desktops \n", "- Modify **different** lines of code in the file and both commit and push to the server (prior to pulling from each other)–and see how it merges things “automatically” \n", "- Modify **the same** line of code in the file, and deal with the [merge conflict](#merge-conflict) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2a\n", "\n", "Just using GitHub’s [web interface](#web-interface), submit a Pull Request for a simple change of documentation to a public repository\n", "\n", "The easiest may be to submit a PR for a typo in the source repository for these notes, i.e. `https://github.com/QuantEcon/lecture-source-jl`\n", "\n", "Note: The source for that repository is in `.rst` files, but you should be able to find spelling mistakes/etc. without much effort" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2b\n", "\n", "Following the [instructions](#fork-workflow) for forking and cloning a public repository to your local desktop, submit a Pull Request to a public repository\n", "\n", "Again, you could submit it for a typo in the source repository for these notes, i.e. `https://github.com/QuantEcon/lecture-source-jl`, but you are also encouraged to instead look for a small change that could help the documentation in another repository.\n", "\n", "If you are ambitious, then go to the Exercise Solutions for one of the Exercises in these lecture notes and submit a PR for your own modified version (if you think it is an improvement!)" ] } ], "metadata": { "date": 1570688168.1605358, "download_nb": 1, "download_nb_path": "https://julia.quantecon.org/", "filename": "version_control.rst", "filename_with_path": "more_julia/version_control", "kernelspec": { "display_name": "Julia 1.2.0", "language": "julia", "name": "julia-1.2" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.2.0" }, "title": "Git, GitHub, and Version Control" }, "nbformat": 4, "nbformat_minor": 2 }