{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with GitHub Actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About GitHub Actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions) help you automate tasks within your software development life cycle. GitHub say that:\n",
    "\n",
    "> \"*GitHub Actions are event-driven, meaning that you can run a series of commands after a specified event has occurred. For example, every time someone creates a pull request for a repository, you can automatically run a command that executes a software testing script.*\"\n",
    "\n",
    "They are a powerful system for automating many software development processes, and open source projects get unlimited compute hours for free! A sequence of steps in GitHub Actions are be combined together into a \"workflow\", which can furthermore contain multiple \"jobs\", which are run on multiple machines, with (if needed) multiple operating systems, in parallel. For more information, read GitHub's [Introduction to GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/introduction-to-github-actions).\n",
    "\n",
    "However, workflows can be difficult to create and debug, because:\n",
    "\n",
    "- They run in a special environment which is difficult to replicate locally\n",
    "- Each time you make a change it takes quite a few minutes before your updated workflow is run and you can see the results\n",
    "- It is difficult to develop interatively and iteratively\n",
    "- The actions environment and contexts are complex and the documentation, whilst detailed, provides few real examples to work from\n",
    "- The main \"language\" for building workflows is a YAML-based language which is quite verbose, and doesn't allow you to leverage your knowledge of other programming languages\n",
    "- Building full-featured extensions requires using TypeScript, which is not a language that most developers are used to using for scripting, sysadmin, and continuous integration tasks.\n",
    "\n",
    "However, with `ghapi` you can write your workflows in Python and shell, and can do nearly all your development on your local machine. We'll start by importing from the library, along with `fastcore`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ghapi.all import *\n",
    "from fastcore.utils import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Your first Python-based workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to help you get started by building a simple workflow which will add a comment to all new pull requests, saying \"thank you\" to the contributor.\n",
    "\n",
    "`GhApi` makes developing workflows easier, because your actual YAML file is created for you entirely automatically. To create the new workflow, run the `create_workflow` function, like so:\n",
    "\n",
    "```python\n",
    "create_workflow(name='thankyou', event=Event.pull_request)\n",
    "```\n",
    "\n",
    "Now it's time to create our python script. You'll find it in `.github/scripts`, named based on your `create_workflow` parameters. The initial skeleton just contains import statements.\n",
    "\n",
    "It will be much easier for us to iterate on our script if we have a sample *context* that the GitHub workflow runner will provide to us. We can get one by calling `example_payload`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['action', 'number', 'pull_request', 'repository', 'sender']"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example = example_payload(Event.pull_request)\n",
    "list(example)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For information about all the fields in a *payload*, use the GitHub [webhook payload](https://docs.github.com/en/free-pro-team@latest/developers/webhooks-and-events/webhook-events-and-payloads#pull_request) documentation. For instance, the docs tell us that the \"action\" field can be:\n",
    "\n",
    "> \"*opened, edited, closed, assigned, unassigned, review_requested, review_request_removed, ready_for_review, labeled, unlabeled, synchronize, locked, unlocked, or reopened*\"\n",
    "\n",
    "Let's see what it contains for our example payload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'opened'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example.action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our script, we should ensure that it's only run for the `opened` action. Next up, we need to find out how to add a comment to a pull request. First, we'll need to create our `GhApi` object. On your own PC, your GitHub token should be in the `GITHUB_TOKEN` environment variable, whereas when run in a workflow it will be part of the `github` context. The `github_token` function handles this for you automatically, so we can say:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api = GhApi(owner='fastai', repo='ghapi-test', token=github_token())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to find the correct operation to call is to search the [full API reference](https://ghapi.fast.ai/fullapi.html). Operations are generally named as `{verb}_{object}`, so search for `create_comment`. Alternatively, you can jump to the section that you expect to contain your required operation -- in this case, it's important to know that GitHub considers a \"pull request\" to be a special kind of \"issue\". After some looking through the page, we found this operation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[issues.create_comment](https://docs.github.com/rest/reference/issues#create-an-issue-comment)(issue_number, body): *Create an issue comment*"
      ],
      "text/plain": [
       "[issues.create_comment](https://docs.github.com/rest/reference/issues#create-an-issue-comment)(issue_number, body): *Create an issue comment*"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "api.issues.create_comment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The hyperlink provided will take you to the GitHub docs for this operation, so take a look at that now. We need to provide an `issue_number` and the `body` of the comment. We can look inside the payload to find the issue number we need to use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'url, id, node_id, html_url, diff_url, patch_url, issue_url, number, state, locked, title, user, body, created_at, updated_at, closed_at, merged_at, merge_commit_sha, assignee, assignees, requested_reviewers, requested_teams, labels, milestone, commits_url, review_comments_url, review_comment_url, comments_url, statuses_url, head, base, _links, author_association, draft, merged, mergeable, rebaseable, mergeable_state, merged_by, comments, review_comments, maintainer_can_modify, commits, additions, deletions, changed_files'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "', '.join(example.pull_request)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the docs call the parameter `issue_number`, and there is a `number` field here, that's probably what we should use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test this out, we'll first create a PR or issue, and then create our comment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "com = api.issues.create_comment(issue_number=1, body='Thank you for your *valuable* contribution')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view the comment by visiting its URL:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://api.github.com/repos/fastai/ghapi-test/issues/comments/737393228'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "com.url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and then delete it since we were just testing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "com = api.issues.delete_comment(com.id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The payload will be available in the [`github` context](https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions#github-context), which is provided automatically to you using the `context_github` variable. When running locally, an example github context will be used instead. (This example github context will not, in general, have the same payload as the event you're using, so use the `example payload` for that.)\n",
    "\n",
    "The `event` contains the actual payload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['inputs', 'organization', 'ref', 'repository', 'sender', 'workflow']"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(context_github.event)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we called `create_workflow`, the workflow it created can also be triggered using [workflow dispatch](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/), which is more convenient for testing. In this case, our payload will not have the same information, so we should write our function in a way that can handle workflow dispatch as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dispatch = example_payload(Event.workflow_dispatch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check what type of trigger we're responding to, by checking the keys of our payload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'workflow' in dispatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In that case, we'll use a fixed issue number, instead of using the actual payload number.\n",
    "\n",
    "We can now write our function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reply_thanks():\n",
    "    api = GhApi(owner='fastai', repo='ghapi', token=github_token())\n",
    "    payload = context_github.event\n",
    "    if 'workflow' in payload: issue = 1\n",
    "    else:\n",
    "        if payload.action != 'opened': return\n",
    "        issue = payload.number\n",
    "    api.issues.create_comment(issue_number=issue, body='Thank you for your *valuable* contribution')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, copy this to the script in `.github/scripts`, along with a line to run it (`reply_thanks()`), and commit it to GitHub.\n",
    "\n",
    "You can now test the workflow by going to GitHub, clicking the \"Actions\" tab, then clicking the workflow you just created, and then clicking the \"Run workflow\" button.\n",
    "\n",
    "Alternatively, you can run it using `GhApi`, which we'll do now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [],
      "text/plain": []
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wf = api.actions.get_workflow('thankyou-pull_request.yml')\n",
    "api.actions.create_workflow_dispatch(wf.id, ref='master')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find out if the dispatch succeeded, go to the Actions tab on your repo and look for a yellow circle, which means it's running, or a green tick, which means it succeeded. If you get a red cross, it failed - you can click the item for details. You can also get the results of the latest run through the API (but be sure to wait at least 30 seconds for the workflow to kick off):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'success'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_run = api.actions.list_workflow_runs(wf.id).workflow_runs[0]\n",
    "last_run.conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If it doesn't work, you can add `print(payload)` after the first line, and when GitHub runs your workflow you'll see the whole payload in your \"Actions\" panel.\n",
    "\n",
    "Once it's working OK using the manual workflow dispatch, try creating a pull request."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced multi-job workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, you need to join more than one *job* in a workflow. This isn't common, but one example it's absolutely needed is if you want to build software or run tests on multiple platforms, and then have some step that runs before or after all of them. For instance, building and releasing software that needs to be built on each platform requires one job to create a release, and then a separate job to do the builds and add artifacts to the release, since GitHub Actions runs all your steps in a job in a single platform. Furthermore, we need to set up [dependencies between jobs](https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#creating-dependent-jobs) using `needs`. `ghapi` provides helpers to set up multi-job workflows with depedencies for you, which we'll show an example of here.\n",
    "\n",
    "In this tutorial we're going to show how we created [hugo-mathjax](https://github.com/fastai/hugo-mathjax). The workflow in this project patches [Hugo](https://gohugo.io/) to add [Mathjax](https://www.mathjax.org/) support, then builds Mac, Ubuntu, and Windows versions of the software, creating a release which includes each of those builds as artifacts. It does the following:\n",
    "\n",
    "- On a Ubuntu runner:\n",
    "  - Find out what the latest release of Hugo is\n",
    "  - It we haven't yet built a patched version of this release, create a new release\n",
    "  - Output the release number for the next job to use\n",
    "- On Mac, Ubuntu, and Windows runners all running in parallel:\n",
    "  - Get the release number output by the previous step\n",
    "  - Download this release from the Hugo repo, and untar it\n",
    "  - Patch, build, and tar this release\n",
    "  - Add the tarred release as an artifact of the release created in the first job\n",
    "\n",
    "First, we'll create the workflow and script files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_workflow('create', Event.schedule, contexts='needs', opersys='macos, ubuntu, windows', prebuild=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This has some pieces we haven't seen before, so let's take a look at them now.\n",
    "\n",
    "The `prebuild` bool tells `ghapi` to include a prebuild job (see `create_workflow` for the full workflow created). After you run the above, take a look at the YAML that was created, to see how it all works. We're going to need to modify the YAML, to add the [schedule](https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#schedule) we want to run it on, using a `cron:` line. You can see the final YAML file [here](https://github.com/fastai/hugo-mathjax/blob/master/.github/workflows/python.yml).\n",
    "\n",
    "The prebuild job calls a prebuild script using a Ubuntu runner. We'll create the prebuild script now.\n",
    "\n",
    "The script needs to get the current Hugo release tag:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'v0.79.0'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag = GhApi().repos.get_latest_release('gohugoio', repo='hugo').name\n",
    "tag"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and check whether we have released that patched version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "api = GhApi(owner='fastai', repo='hugo-mathjax', token=github_token())\n",
    "exists = True\n",
    "try: api.repos.get_release_by_tag(tag)\n",
    "except HTTP404NotFoundError: exists = False\n",
    "exists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the case that the release already exists, we'll want to just exit, otherwise we'll create a new release:\n",
    "\n",
    "```python\n",
    "rel = api.repos.create_release(tag, name=tag)\n",
    "```\n",
    "\n",
    "The `build` job will run in a separate runner, so we need to create an output telling it the tag of the release we've created. The `actions_output` function does that, but print a special format that GitHub Actions uses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "::set-output name=tag::v0.79.0\n"
     ]
    }
   ],
   "source": [
    "actions_output('tag', tag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the completed prebuild script [here](https://github.com/fastai/hugo-mathjax/blob/master/.github/scripts/prebuild.py).\n",
    "\n",
    "Now we can move on to creating the `build` script. First, we can get the outputs of the `prebuild` job as a `dict` by using `context_needs`, which contains information from any jobs in the `needs` section (which `ghapi` will automatically set up for you if you enable `prebuild`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "out = loads(nested_idx(context_needs, 'prebuild', 'outputs', 'out'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the `tag` output of the job, we index into the `dict`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'v0.79.0'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag = nested_idx(out, 'step1', 'outputs', 'tag')\n",
    "tag"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get the release details using the same approach as before, then download, untar, patch, and build it. We won't discuss those steps here since they're not specific to `ghapi`.\n",
    "\n",
    "Once the software is built, we need to know the name of the created file, which depends on the operating system we're running on. We can use the following approach to check:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hugo'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "platform = dict(linux='linux', linux2='linux', win32='win', darwin='mac')[sys.platform]\n",
    "ext_nm = 'hugo.exe' if platform=='win' else 'hugo'\n",
    "ext_nm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can upload our file as an artifact:\n",
    "\n",
    "```python\n",
    "api = GhApi(owner='fastai', repo='hugo-mathjax', token=github_token())\n",
    "rel = api.repos.get_release_by_tag(tag)\n",
    "api.upload_file(rel, fn)\n",
    "```\n",
    "\n",
    "You can see the complete script [here](https://github.com/fastai/hugo-mathjax/blob/master/.github/scripts/build.py).\n",
    "\n",
    "We've now successfully built a workflow that automatically patches, builds, and releases a mixed C/Go project, across multiple platforms, in parallel!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}