{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#default_exp ghtop" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# ghtop API\n", "\n", "> API details" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "import time, sys, signal, shutil, os, json, enlighten, emoji, blessed\n", "from dashing import *\n", "from collections import defaultdict\n", "from warnings import warn\n", "from itertools import islice\n", "\n", "from fastcore.utils import *\n", "from fastcore.foundation import *\n", "from fastcore.script import *\n", "from ghapi.all import *" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "term = Terminal()\n", "logfile = Path(\"log.txt\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def github_auth_device(wb='', n_polls=9999):\n", " \"Authenticate with GitHub, polling up to `n_polls` times to wait for completion\"\n", " auth = GhDeviceAuth()\n", " print(f\"First copy your one-time code: {term.yellow}{auth.user_code}{term.normal}\")\n", " print(f\"Then visit {auth.verification_uri} in your browser, and paste the code when prompted.\")\n", " if not wb: wb = input(\"Shall we try to open the link for you? [y/n] \")\n", " if wb[0].lower()=='y': auth.open_browser()\n", "\n", " print(\"Waiting for authorization...\", end='')\n", " token = auth.wait(lambda: print('.', end=''), n_polls=n_polls)\n", " if not token: return print('Authentication not complete!')\n", " print(\"Authenticated with GitHub\")\n", " return token" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When we run this we'll be shown a URL to visit and a code to enter in order to authenticate. Normally we'll be prompted to open a browser, and the function will wait for authentication to complete -- for demonstrating here we'll skip both of these steps:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "First copy your one-time code: \u001b[33m004F-A5D6\u001b[m\n", "Then visit https://github.com/login/device in your browser, and paste the code when prompted.\n", "Waiting for authorization...Authentication not complete!\n" ] } ], "source": [ "github_auth_device('n',n_polls=0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _exit(msg):\n", " print(msg, file=sys.stderr)\n", " sys.exit()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#exports\n", "def limit_cb(rem,quota):\n", " \"Callback to warn user when close to using up hourly quota\"\n", " w='WARNING '*7\n", " if rem < 1000: print(f\"{w}\\nRemaining calls: {rem} out of {quota}\\n{w}\", file=sys.stderr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When creating `GhApi` we can pass a callback which will be called after each API operation. In this case, we use it to warn the user when their quota is getting low." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _get_api():\n", " path = Path.home()/\".ghtop_token\"\n", " if path.is_file():\n", " try: token = path.read_text().strip()\n", " except: _exit(\"Error reading token\")\n", " else: token = github_auth_device()\n", " path.write_text(token)\n", " return GhApi(limit_cb=limit_cb, token=token)\n", "\n", "api = _get_api()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "Events = dict(\n", " IssuesEvent_closed=('โญ', 'closed', noop),\n", " IssuesEvent_opened=('๐Ÿ“ซ', 'opened', noop),\n", " IssueCommentEvent=('๐Ÿ’ฌ', 'commented on', term.white),\n", " PullRequestEvent_opened=('โœจ', 'opened a pull request', term.yellow),\n", " PullRequestEvent_closed=('โœ”', 'closed a pull request', term.green),\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _to_log(e):\n", " login,repo,pay = e.actor.login,e.repo.name,e.payload\n", " typ = e.type + (f'_{pay.action}' if e.type in ('PullRequestEvent','IssuesEvent') else '')\n", " emoji,msg,color = Events.get(typ, [0]*3)\n", " if emoji:\n", " xtra = '' if e.type == \"PullRequestEvent\" else f' issue # {pay.issue.number}'\n", " d = try_attrs(pay, \"pull_request\", \"issue\")\n", " return color(f'{emoji} {login} {msg}{xtra} on repo {repo[:20]} (\"{d.title[:50]}...\")')\n", " elif e.type == \"ReleaseEvent\": return f'๐Ÿš€ {login} released {e.payload.release.tag_name} of {repo}'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def print_event(e, commits_counter):\n", " res = _to_log(e)\n", " if res: print(res)\n", " elif e.type == \"PushEvent\": [commits_counter.update() for c in e.payload.commits]\n", " elif e.type == \"SecurityAdvisoryEvent\": print(term.blink(\"SECURITY ADVISORY\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can pretty print a selection of event types using `print_event`, e.g:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "๐Ÿ“ซ erfanmola opened issue # 26 on repo gonzaarcr/Fildem (\"Global Menu for Jetbrains Apps...\")\n", "๐Ÿ“ซ baidua opened issue # 4436 on repo baidua/xmvavaciof (\"็ฆๅทž้ซ˜็ซฏๅ–่Œถ่”็ณปๆ–นๅผๅ“่Œถๅค–ๅ›ดไธŠ่ฏพ...\")\n", "๐Ÿ“ซ trnphocgqe opened issue # 1533 on repo trnphocgqe/tvtxtvxtv (\"ๅ…ณไบŽๆฑŸ้—จๅ“ช้‡Œๅฏไปฅๅผ€ๅŠ ๆฒนๅ‘็ฅจ-ๆฑŸ้—จๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ limjinsun opened issue # 1 on repo DestyNova/advent_of_ (\"Day1 solution....\")\n", "๐Ÿ“ซ fghyfuy36 opened issue # 113646 on repo ouyanxia2/hgmgmg (\"่Œ‚ๅๆ•™ๆ‰‘ๅ…‹็‰Œๆ‰‹ๆณ•...\")\n", "๐Ÿ“ซ bbb5426647 opened issue # 1617 on repo bbb5426647/uuigzoyap (\"ๆพณๆดฒๅนธ่ฟ5ๅฏ้ ๅ—...\")\n", "๐Ÿ“ซ 95sex opened issue # 614 on repo 95sex/ofrqtowfts (\"ๆž้€Ÿ่ต›่ฝฆๅพฎไฟก็š„็พคappไธ‹ๆณจ...\")\n", "๐Ÿ“ซ Misly16 opened issue # 4 on repo ourcord/ourcord (\"Find fix for zlib install....\")\n", "๐Ÿ“ซ hvva183 opened issue # 2355 on repo hvva183/huwchryzgn (\"ๅ…ณ๎‚ไบŽ๎‚ๆตŽ๎‚ๅ—๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-ๆตŽๅ—ๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ hvva183 opened issue # 2345 on repo hvva183/ktaaonudut (\"ๅ…ณ๎‚ไบŽ๎‚ๆˆ๎‚้ƒฝ๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-ๆˆ้ƒฝๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ bbb5426647 opened issue # 1625 on repo bbb5426647/dkegwbigm (\"ๅฎ‰ๅ…จ็š„ไธ€ๅˆ†้’ŸๅŠ ๆ‹ฟๅคง28ๅพฎไฟก็พค...\")\n", "๐Ÿ“ซ ok0003 opened issue # 113642 on repo ouyanxia2/hgmgmg (\"ๅ…ณไบŽ)ๆ–‡ๅฑฑไปฃๅผ€ไธ‰็”ฒๅŒป้™ข่ฏŠๆ–ญ่ฏๆ˜Žใ€โ•‹่–‡/่Šฏ:xrj3781ใ€‘...\")\n", "โญ TNelen closed issue # 242 on repo TNelen/BlackBox (\"change all icons to OMIcons...\")\n", "๐Ÿ“ซ hvva183 opened issue # 2362 on repo hvva183/oowsiqqguw (\"ๅ…ณ๎‚ไบŽ๎‚ๅฎ๎‚ๆณข๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-ๅฎๆณขๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ hvva183 opened issue # 2355 on repo hvva183/qthmkvmgtf (\"ๅ…ณ๎‚ไบŽ๎‚ๆญ๎‚ๅทž๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-ๆญๅทžๆœฌๅœฐๅฎ...\")\n", "\u001b[32mโœ” AliasIO closed a pull request on repo AliasIO/wappalyzer (\"Add Aimtell/Marketing automation...\")\u001b[m\n", "\u001b[32mโœ” akemimeka closed a pull request on repo KarinaFS/battle-win (\"Mergeia a master e realiza melhoras no Readme...\")\u001b[m\n", "๐Ÿ“ซ hvva183 opened issue # 2371 on repo hvva183/hcigoxucgz (\"ๅ…ณ๎‚ไบŽ๎‚้•ฟ๎‚ๆ˜ฅ๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-้•ฟๆ˜ฅๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ hvva183 opened issue # 2357 on repo hvva183/guubsqomuj (\"ๅ…ณ๎‚ไบŽ๎‚่‹๎‚ๅทž๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-่‹ๅทžๆœฌๅœฐๅฎ...\")\n", "๐Ÿ“ซ hvva183 opened issue # 2352 on repo hvva183/ltrubzaaap (\"ๅ…ณ๎‚ไบŽ๎‚ๆฒˆ๎‚้˜ณ๎‚ๅ“ช๎‚้‡Œ๎‚ๅฏ๎‚ไปฅ๎‚ๅผ€๎‚ๅ…ท๎‚้ค๎‚้ฅฎ๎‚่ดน๎‚ๅ‘๎‚็ฅจ-ๆฒˆ้˜ณๆœฌๅœฐๅฎ...\")\n" ] } ], "source": [ "gen = api.fetch_events(types=('IssuesEvent','ReleaseEvent','PullRequestEvent'))\n", "for e in islice(gen, 20): print_event(e,None)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def tail_events():\n", " \"Print events from `fetch_events` along with a counter of push events\"\n", " manager = enlighten.get_manager()\n", " commits = manager.counter(desc='Commits', unit='commits', color='green')\n", " for ev in fetch_events(): print_event(ev, commits)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _pr_row(*its): print(f\"{its[0]: <30} {its[1]: <6} {its[2]: <5} {its[3]: <6} {its[4]: <7}\")\n", "def watch_users(evts):\n", " \"Print a table of the users with the most events\"\n", " users,users_events = defaultdict(int),defaultdict(lambda: defaultdict(int))\n", " while True:\n", " for x in islice(evts, 10):\n", " users[x.actor.login] += 1\n", " users_events[x.actor.login][x.type] += 1\n", "\n", " print(term.clear())\n", " _pr_row(\"User\", \"Events\", \"PRs\", \"Issues\", \"Pushes\")\n", " sorted_users = sorted(users.items(), key=lambda o: (o[1],o[0]), reverse=True)\n", " for u in sorted_users[:20]:\n", " _pr_row(*u, *itemgetter('PullRequestEvent','IssuesEvent','PushEvent')(users_events[u[0]]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _push_to_log(e): return f\"{e.actor.login} pushed {len(e.payload.commits)} commits to repo {e.repo.name}\"\n", "def _logwin(title,color): return Log(title=title,border_color=2,color=color)\n", "\n", "def quad_logs(evts):\n", " \"Print 4 panels, showing most recent issues, commits, PRs, and releases\"\n", " term.enter_fullscreen()\n", " ui = HSplit(VSplit(_logwin('Issues', color=7), _logwin('Commits' , color=3)),\n", " VSplit(_logwin('Pull Requests', color=4), _logwin('Releases', color=5)))\n", "\n", " issues,commits,prs,releases = all_items = ui.items[0].items+ui.items[1].items\n", " for o in all_items: o.append(\" \")\n", "\n", " d = dict(PushEvent=commits, IssuesEvent=issues, IssueCommentEvent=issues, PullRequestEvent=prs, ReleaseEvent=releases)\n", " while True:\n", " for x in islice(evts, 10):\n", " f = [_to_log,_push_to_log][x.type == 'PushEvent']\n", " if x.type in d: d[x.type].append(f(x)[:95])\n", " ui.display()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def simple(evts):\n", " for ev in evts: print(f\"{ev.actor.login} {ev.type} {ev.repo.name}\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#export\n", "def _signal_handler(sig, frame):\n", " if sig != signal.SIGINT: return\n", " print(term.exit_fullscreen(),term.clear(),term.normal)\n", " sys.exit(0)\n", "\n", "_funcs = dict(tail=tail_events, quad=quad_logs, users=watch_users, simple=simple)\n", "_filts = str_enum('_filts', 'user', 'repo', 'org')\n", "_OpModes = str_enum('_OpModes', *_funcs)\n", "\n", "@call_parse\n", "def main(mode: Param(\"Operation mode to run\", _OpModes),\n", " include_bots: Param(\"Include bots (there's a lot of them!)\", store_true)=False,\n", " types: Param(\"Comma-separated types of event to include (e.g PushEvent)\", str)='',\n", " filt: Param(\"Filtering method\", _filts)=None,\n", " filtval: Param(\"Value to filter by (for `repo` use format `owner/repo`)\", str)=None):\n", " signal.signal(signal.SIGINT, _signal_handler)\n", " types = types.split(',') if types else None\n", " if filt and not filtval: _exit(\"Must pass `filter_value` if passing `filter_type`\")\n", " if filtval and not filt: _exit(\"Must pass `filter_type` if passing `filter_value`\")\n", " kwargs = {filt:filtval} if filt else {}\n", " evts = api.fetch_events(types=types, incl_bot=include_bots, **kwargs)\n", " _funcs[mode](evts)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Export -" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Converted 00_ghtop.ipynb.\n", "Converted index.ipynb.\n", "Converted rich_test.ipynb.\n" ] } ], "source": [ "#hide\n", "from nbdev.export import notebook2script\n", "notebook2script()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 4 }