{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Automatic discovery of community organizations for long-term package maintenance" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Python 3.7.4 (default, Jul 8 2019, 18:31:06) \n", "[GCC 7.4.0]\n", "IPython 7.6.1\n", "\n", "Libraries:\n", "\n", "matplotlib 3.1.1\n", "numpy 1.17.0\n", "pandas 0.25.0\n", "requests 2.22.0\n" ] } ], "source": [ "%matplotlib inline\n", "\n", "import sys\n", "print(f'Python {sys.version}')\n", "\n", "import IPython\n", "print(f'IPython {IPython.__version__}')\n", "\n", "print('\\nLibraries:\\n')\n", "\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "print(f'matplotlib {matplotlib.__version__}')\n", "\n", "import numpy as np\n", "print(f'numpy {np.__version__}')\n", "\n", "import pandas as pd\n", "from pandas.plotting import register_matplotlib_converters\n", "print(f'pandas {pd.__version__}')\n", "\n", "import requests\n", "print(f'requests {requests.__version__}')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "api_token = ''\n", "\n", "def send_rest_request(url):\n", " headers = {'Authorization': f'token {api_token}'}\n", " r = requests.get(url=url, headers=headers)\n", " r.raise_for_status() # Abort if unsuccessful request\n", " return r.json()\n", "\n", "def send_graphql_request(query, variables):\n", " headers = {'Authorization': f'token {api_token}'}\n", " url = 'https://api.github.com/graphql'\n", " json = {'query':query, 'variables':variables}\n", " r = requests.post(url=url, json=json, headers=headers)\n", " r.raise_for_status() # Abort if unsuccessful request\n", " return r.json()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Phase 1: get a preliminary list of organizations\n", "\n", "GitHub only provide two APIs to get a list of organization: a REST endpoint that allows to get the full list, but requires many requests, given that there are more than 2,000,000 organizations on GitHub (https://developer.github.com/changes/2015-06-17-organizations-endpoint/) and given that this first type of request will only provide the list of organization logins and descriptions, but nothing more, or the Search API that is limited to browsing 1000 results.\n", "\n", "We choose to use the second to limit the number of requests, but this imposes to find ways of querying for less than 1000 results at a time, using the limited filters that search queries provide.\n", "\n", "Our first restriction will be to limit ourselves to organizations with at least 5 public repositories.\n", "We are aware that this is an arbitrary restriction that will exclude community organizations that are just starting and have not yet reached that number.\n", "\n", "Our second restriction will be to search by keywords.\n", "We list as many keywords as we could think that could appear in the names or the descriptions of this type of organizations:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "75" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "keywords = [\n", " # To add next time: 'addon', 'addons',\n", " 'app', 'apps', 'application', 'applications',\n", " 'care', 'caring',\n", " 'collab', 'collaboration', 'collaborative',\n", " 'collection', 'collective',\n", " 'common', 'commons',\n", " 'community',\n", " 'component', 'components',\n", " # To add next time: 'contribs'\n", " 'contrib', 'contribution', 'contributions', 'contributing',\n", " 'distribute', 'distribution', 'distributions',\n", " 'ecosystem', 'ecosystems',\n", " 'extension', 'extensions',\n", " 'gather',\n", " 'give', 'giving',\n", " 'group',\n", " 'help', 'helper', 'helpers',\n", " 'library', 'libraries',\n", " 'maintain', 'maintainer', 'maintainers', 'maintenance', 'maintaining',\n", " 'member', 'members',\n", " 'module', 'modules',\n", " 'open source',\n", " 'org', 'organization',\n", " 'package', 'packages',\n", " 'participate', 'participant', 'participants', 'participation',\n", " 'people',\n", " 'place',\n", " 'plugin', 'plugins',\n", " 'projects',\n", " # Not project singular because that would give too many results\n", " # and this is not about organizations focused on a single project\n", " 'quality',\n", " 'repository', 'repositories',\n", " 'reuse', 'reusable',\n", " 'share', 'shared', 'sharing',\n", " 'support', 'supporter', 'supporters', 'supporting',\n", " 'together',\n", " # To add next time: tool, tools\n", " 'unofficial',\n", " 'user', 'users'\n", "]\n", "len(keywords)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For some keywords, this still gives too many results so we additionally partition using language filters:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "language_filters = [\n", " 'language:JavaScript',\n", " 'language:Java',\n", " 'language:Python',\n", " 'language:PHP',\n", " 'language:HTML',\n", " 'language:C#',\n", " 'language:C++',\n", " 'language:C',\n", " 'language:CSS',\n", " '-language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS'\n", "]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "query = '''\n", "query searchOrganizations($query: String!,$cursor: String) {\n", " search(type:USER,query:$query, first: 50, after: $cursor) {\n", " userCount\n", " pageInfo {\n", " endCursor\n", " hasNextPage\n", " }\n", " nodes {\n", " ... on Organization {\n", " login\n", " name\n", " description\n", " websiteUrl\n", " membersWithRole {\n", " totalCount\n", " }\n", " repositories(first: 1, orderBy: {field: STARGAZERS, direction: DESC}) {\n", " totalCount\n", " nodes {\n", " stargazers {\n", " totalCount\n", " }\n", " assignableUsers {\n", " totalCount\n", " }\n", " }\n", " }\n", " }\n", " }\n", " }\n", "}\n", "'''" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "columns = [\n", " 'name',\n", " 'description',\n", " 'url',\n", " 'members', # Number of public members\n", " 'repositories', # Number of public repositories\n", " 'stars', # Number of stars of the most starred repository\n", " 'collaborators' # Number of assignable users of the most starred repository\n", "]\n", "\n", "keyword_columns = list(map(lambda keyword: f'keyword {keyword}', keywords))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values = pd.DataFrame(columns=columns + keyword_columns).astype({\n", " 'members': 'UInt32',\n", " 'repositories': 'UInt32',\n", " 'stars': 'UInt32',\n", " 'collaborators': 'UInt32'\n", "})" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def paged_query(keyword, language=''):\n", " if keyword == 'repository' or keyword == 'user':\n", " exclude = 'NOT aur-archive'\n", " elif keyword == 'collaborative':\n", " exclude = 'NOT GITenberg'\n", " else:\n", " exclude = ''\n", " next_page = True\n", " cursor = None\n", " while next_page:\n", " searchQuery = f'type:organization repos:>=5 {keyword} {exclude} {language}'\n", " print(f'Search query: {searchQuery}')\n", " json = send_graphql_request(\n", " query,\n", " {'query': searchQuery, 'cursor': cursor}\n", " )\n", " search_json = json['data']['search']\n", " nb_results = search_json['userCount']\n", " if nb_results > 1000:\n", " raise ValueError('Query not restricted enough: more than 1000 results.')\n", " page_info = search_json['pageInfo']\n", " next_page = page_info['hasNextPage']\n", " cursor = page_info['endCursor']\n", " for node in search_json['nodes']:\n", " # Index\n", " login = node['login']\n", " # Fields\n", " name = node['name']\n", " values.loc[login, 'name'] = name\n", " values.loc[login, 'description'] = node['description']\n", " values.loc[login, 'url'] = node['websiteUrl']\n", " values.loc[login, 'members'] = node['membersWithRole']['totalCount']\n", " repos_json = node['repositories']\n", " repos_nb = repos_json['totalCount']\n", " values.loc[login, 'repositories'] = repos_nb\n", " if repos_nb > 0:\n", " repo_json = repos_json['nodes'][0]\n", " values.loc[login, 'stars'] = repo_json['stargazers']['totalCount']\n", " values.loc[login, 'collaborators'] = repo_json['assignableUsers']['totalCount']\n", " values.loc[login, f'keyword {keyword}'] = True" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Search query: type:organization repos:>=5 repository NOT aur-archive \n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:JavaScript\n", "Now fetched a total number of 27130 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Java\n", "Now fetched a total number of 27337 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:Python\n", "Now fetched a total number of 27532 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:PHP\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:PHP\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:PHP\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:PHP\n", "Now fetched a total number of 27632 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:HTML\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:HTML\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:HTML\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:HTML\n", "Now fetched a total number of 27728 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C#\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C#\n", "Now fetched a total number of 27785 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C++\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C++\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C++\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C++\n", "Now fetched a total number of 27894 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:C\n", "Now fetched a total number of 27993 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive language:CSS\n", "Now fetched a total number of 28033 organizations.\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repository NOT aur-archive -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Now fetched a total number of 28390 organizations.\n", "Search query: type:organization repos:>=5 repositories \n", "Search query: type:organization repos:>=5 repositories language:JavaScript\n", "Search query: type:organization repos:>=5 repositories language:JavaScript\n", "Search query: type:organization repos:>=5 repositories language:JavaScript\n", "Search query: type:organization repos:>=5 repositories language:JavaScript\n", "Search query: type:organization repos:>=5 repositories language:JavaScript\n", "Now fetched a total number of 28401 organizations.\n", "Search query: type:organization repos:>=5 repositories language:Java\n", "Search query: type:organization repos:>=5 repositories language:Java\n", "Search query: type:organization repos:>=5 repositories language:Java\n", "Search query: type:organization repos:>=5 repositories language:Java\n", "Now fetched a total number of 28421 organizations.\n", "Search query: type:organization repos:>=5 repositories language:Python\n", "Search query: type:organization repos:>=5 repositories language:Python\n", "Search query: type:organization repos:>=5 repositories language:Python\n", "Search query: type:organization repos:>=5 repositories language:Python\n", "Search query: type:organization repos:>=5 repositories language:Python\n", "Now fetched a total number of 28437 organizations.\n", "Search query: type:organization repos:>=5 repositories language:PHP\n", "Search query: type:organization repos:>=5 repositories language:PHP\n", "Now fetched a total number of 28443 organizations.\n", "Search query: type:organization repos:>=5 repositories language:HTML\n", "Search query: type:organization repos:>=5 repositories language:HTML\n", "Now fetched a total number of 28447 organizations.\n", "Search query: type:organization repos:>=5 repositories language:C#\n", "Now fetched a total number of 28447 organizations.\n", "Search query: type:organization repos:>=5 repositories language:C++\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Search query: type:organization repos:>=5 repositories language:C++\n", "Now fetched a total number of 28453 organizations.\n", "Search query: type:organization repos:>=5 repositories language:C\n", "Search query: type:organization repos:>=5 repositories language:C\n", "Search query: type:organization repos:>=5 repositories language:C\n", "Now fetched a total number of 28457 organizations.\n", "Search query: type:organization repos:>=5 repositories language:CSS\n", "Now fetched a total number of 28457 organizations.\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 repositories -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Now fetched a total number of 28480 organizations.\n", "Search query: type:organization repos:>=5 reuse \n", "Search query: type:organization repos:>=5 reuse \n", "Now fetched a total number of 28506 organizations.\n", "Search query: type:organization repos:>=5 reusable \n", "Search query: type:organization repos:>=5 reusable \n", "Now fetched a total number of 28513 organizations.\n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Search query: type:organization repos:>=5 share \n", "Now fetched a total number of 28926 organizations.\n", "Search query: type:organization repos:>=5 shared \n", "Search query: type:organization repos:>=5 shared \n", "Search query: type:organization repos:>=5 shared \n", "Now fetched a total number of 28934 organizations.\n", "Search query: type:organization repos:>=5 sharing \n", "Search query: type:organization repos:>=5 sharing \n", "Search query: type:organization repos:>=5 sharing \n", "Search query: type:organization repos:>=5 sharing \n", "Search query: type:organization repos:>=5 sharing \n", "Now fetched a total number of 28946 organizations.\n", "Search query: type:organization repos:>=5 support \n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Search query: type:organization repos:>=5 support language:JavaScript\n", "Now fetched a total number of 29667 organizations.\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Search query: type:organization repos:>=5 support language:Java\n", "Now fetched a total number of 29957 organizations.\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Search query: type:organization repos:>=5 support language:Python\n", "Now fetched a total number of 30246 organizations.\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Search query: type:organization repos:>=5 support language:PHP\n", "Now fetched a total number of 30700 organizations.\n", "Search query: type:organization repos:>=5 support language:HTML\n", "Search query: type:organization repos:>=5 support language:HTML\n", "Search query: type:organization repos:>=5 support language:HTML\n", "Search query: type:organization repos:>=5 support language:HTML\n", "Now fetched a total number of 30807 organizations.\n", "Search query: type:organization repos:>=5 support language:C#\n", "Search query: type:organization repos:>=5 support language:C#\n", "Search query: type:organization repos:>=5 support language:C#\n", "Search query: type:organization repos:>=5 support language:C#\n", "Now fetched a total number of 30919 organizations.\n", "Search query: type:organization repos:>=5 support language:C++\n", "Search query: type:organization repos:>=5 support language:C++\n", "Search query: type:organization repos:>=5 support language:C++\n", "Search query: type:organization repos:>=5 support language:C++\n", "Now fetched a total number of 31034 organizations.\n", "Search query: type:organization repos:>=5 support language:C\n", "Search query: type:organization repos:>=5 support language:C\n", "Search query: type:organization repos:>=5 support language:C\n", "Now fetched a total number of 31134 organizations.\n", "Search query: type:organization repos:>=5 support language:CSS\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Search query: type:organization repos:>=5 support language:CSS\n", "Search query: type:organization repos:>=5 support language:CSS\n", "Now fetched a total number of 31200 organizations.\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Search query: type:organization repos:>=5 support -language:JavaScript -language:Java -language:Python -language:PHP -language:HTML -language:C# -language:C++ -language:C -language:Ruby -language:CSS\n", "Now fetched a total number of 31751 organizations.\n", "Search query: type:organization repos:>=5 supporter \n", "Now fetched a total number of 31754 organizations.\n", "Search query: type:organization repos:>=5 supporters \n", "Now fetched a total number of 31754 organizations.\n", "Search query: type:organization repos:>=5 supporting \n", "Search query: type:organization repos:>=5 supporting \n", "Search query: type:organization repos:>=5 supporting \n", "Now fetched a total number of 31759 organizations.\n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Search query: type:organization repos:>=5 together \n", "Now fetched a total number of 31941 organizations.\n", "Search query: type:organization repos:>=5 unofficial \n", "Search query: type:organization repos:>=5 unofficial \n", "Search query: type:organization repos:>=5 unofficial \n", "Search query: type:organization repos:>=5 unofficial \n", "Search query: type:organization repos:>=5 unofficial \n", "Now fetched a total number of 32043 organizations.\n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Search query: type:organization repos:>=5 user NOT aur-archive \n", "Now fetched a total number of 32382 organizations.\n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Search query: type:organization repos:>=5 users \n", "Now fetched a total number of 32434 organizations.\n" ] } ], "source": [ "for keyword in keywords[60:]:\n", " try:\n", " paged_query(keyword)\n", " print(f'Now fetched a total number of {len(values)} organizations.')\n", " except ValueError:\n", " for language in language_filters:\n", " paged_query(keyword, language)\n", " print(f'Now fetched a total number of {len(values)} organizations.')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values.to_csv('community-organizations-phase-one.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Phase 2: filter the results and fetch more information\n", "\n", "### Filter\n", "\n", "We start with the organizations that we have fetched in phase 1.\n", "We have fetched more than 32,000 organizations, which is close to 15% of all GitHub organizations with at least 5 repositories." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/nix/store/l2drdy46nqd6kqqz3pv3hfmy4c64ixn9-python3.7-ipython-7.6.1/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3057: DtypeWarning: Columns (8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63) have mixed types. Specify dtype option on import or set low_memory=False.\n", " interactivity=interactivity, compiler=compiler, result=result)\n" ] } ], "source": [ "values = pd.read_csv('community-organizations-phase-one.csv', index_col=0, dtype={\n", " 'members': 'UInt32',\n", " 'repositories': 'UInt32',\n", " 'stars': 'UInt32',\n", " 'collaborators': 'UInt32'\n", "})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Apparently, the search filters are not fully efficient because about 1% of our search results have less than 5 repositories (four of them even having zero repositories):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.011839427760991552" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['repositories'] < 5]) / len(values)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values = values[values['repositories'] >= 5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because many members can decide to make their membership status private (and in fact this is even the default), the public members are just an inferiror bound on the actual number of members of an organization.\n", "To estimate an upper bound on organization membership, we have also retrieved the number of assignable users of the most starred repository.\n", "\n", "Assignable users are organization members with read access to the repository, or collaborators with write access specifically on this repository.\n", "In theory, it is possible for an organization member to not be an assignable user, if the organization owners have changed the default member permissions from \"read\" to \"none\".\n", "In this case, the number of public members of the organization could be larger than the number of assignable users on the most starred repository, but such situation is quite rare, it represents less than 3% of our dataset:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.021029641185647426" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['members'] > values['collaborators']]) / len(values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In most organizations, there are strictly more collaborators than public members:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3.0" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.median(values['collaborators'] - values['members'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Most organizations are not community organizations.\n", "Community organizations (at least established once) should have a strong membership.\n", "Thus we select organizations with at least 10 public members or collaborators on the most starred repository.\n", "This represents 25% of the remaining organizations:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.23051482059282372" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[(values['members'] >= 10) | (values['collaborators'] >= 10)]) / len(values)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values = values[(values['members'] >= 10) | (values['collaborators'] >= 10)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Most organizations do not maintain any popular projects. Community organization should host several popular projects. Stars are often used as a proxy for popularity on GitHub. It is especially relevant for libraries that are mainly targeted to other developers. We set an arbitrary low limit of 10 stars on the most starred project. This represents about 60% of the remaining organizations:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.5929886302111532" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['stars'] >= 10]) / len(values)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values = values[values['stars'] >= 10]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4381" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fetch more information\n", "\n", "For each organization in the remaining list, we fetch the creation date of the organization, and the number of repositories that were created before this date, as an under-approximation of the number of transferred repositories.\n", "The GraphQL API allows us to batch requests and thus to have much fewer requests:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def build_graphql_query(imin):\n", " query = \"\"\"\n", " query {\n", " \"\"\"\n", " if imin + 40 < len(values):\n", " next_imin = imin + 40\n", " isup = next_imin\n", " else:\n", " next_imin = None\n", " isup = len(values)\n", " index = values.index[imin:isup]\n", " for i, owner in enumerate(index):\n", " query += \"\"\"\n", " request%d: organization(login: \"%s\") {\n", " createdAt\n", " }\n", " \"\"\" % (i, owner)\n", " query += \"\"\"\n", " }\n", " \"\"\"\n", " return query, index, next_imin\n", "\n", "def save_testorg_result(json, index):\n", " data = json['data']\n", " i = 0\n", " while f'request{i}' in data:\n", " result = data[f'request{i}']\n", " if result is None:\n", " print(f'Warning: {values.loc[index[i]].name} has been deleted')\n", " else:\n", " values.loc[index[i],'creation date'] = result['createdAt']\n", " i += 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Warning: wedeploy has been deleted\n", "Warning: surging-cloud has been deleted\n", "Warning: ruby-gnome2 has been deleted\n", "Warning: BloomSoftware has been deleted\n", "Warning: SchibstedSpain has been deleted\n", "imin: 4360\r" ] } ], "source": [ "imin = 0\n", "while imin is not None:\n", " sys.stdout.write(f'imin: {imin}\\r')\n", " sys.stdout.flush()\n", " query, index, imin = build_graphql_query(imin)\n", " json = send_graphql_request(query, {})\n", " save_testorg_result(json, index)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def build_graphql_query(imin):\n", " query = \"\"\"\n", " query {\n", " \"\"\"\n", " if imin + 40 < len(values):\n", " next_imin = imin + 40\n", " isup = next_imin\n", " else:\n", " next_imin = None\n", " isup = len(values)\n", " index = values.index[imin:isup]\n", " for i, owner in enumerate(index):\n", " query += \"\"\"\n", " request%d: search(query: \"user:%s created:<%s\", type: REPOSITORY) {\n", " repositoryCount\n", " }\n", " \"\"\" % (i, owner, values.loc[owner, 'creation date'])\n", " query += \"\"\"\n", " }\n", " \"\"\"\n", " return query, index, next_imin\n", "\n", "def save_testorg_result(json, index):\n", " data = json['data']\n", " i = 0\n", " while f'request{i}' in data:\n", " result = data[f'request{i}']\n", " values.loc[index[i],'transferred repositories'] = result['repositoryCount']\n", " i += 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "imin: 4360\r" ] } ], "source": [ "imin = 0\n", "while imin is not None:\n", " sys.stdout.write(f'imin: {imin}\\r')\n", " sys.stdout.flush()\n", " query, index, imin = build_graphql_query(imin)\n", " json = send_graphql_request(query, {})\n", " save_testorg_result(json, index)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values[columns + [\n", " 'creation date',\n", " 'transferred repositories'\n", "] + keyword_columns ].to_csv(\n", " 'community-organizations-phase-two.csv'\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Phase 3: browse through organizations with transferred repos" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "values = pd.read_csv('community-organizations-phase-two.csv', index_col=0, parse_dates=['creation date'], dtype={\n", " 'members': 'UInt32',\n", " 'repositories': 'UInt32',\n", " 'stars': 'UInt32',\n", " 'collaborators': 'UInt32'\n", "}).sort_values('transferred repositories', ascending=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Organizations with one transferred repository from before their creation represent 35% of the remaining organizations:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.35151791828349693" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['transferred repositories'] > 0]) / len(values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And organizations with two transferred repositories from before their creation represent about 20% of the same organizations:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.21410636840903904" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['transferred repositories'] > 1]) / len(values)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "938" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(values[values['transferred repositories'] > 1])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namedescriptionurlmembersrepositoriesstarscollaboratorscreation datetransferred repositorieskeyword app...keyword sharedkeyword sharingkeyword supportkeyword supporterkeyword supporterskeyword supportingkeyword togetherkeyword unofficialkeyword userkeyword users
datadeskLos Angeles Times Data DeskAnalysis, applications and automation from a t...https://www.latimes.com8184313272010-07-02 02:04:07+00:006.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
collectiveCollectivePlone add-ons shared code repositorieshttps://collective.github.io26816745696282010-08-13 00:04:43+00:007.0NaN...TrueNaNNaNNaNNaNNaNNaNNaNNaNNaN
uncopenwebUNC Open Web GroupNaNhttp://sites.google.com/site/uncopenweb/11231522010-09-04 01:22:47+00:006.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
PerlDancerPerlDancerThe Dancer Developers grouphttp://perldancer.org1071708152010-09-21 12:27:49+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNTrue
symphonistsSymphony CommunityNaNhttps://www.getsymphony.com1210647132010-10-21 15:40:12+00:0056.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
libtomlibtomlibtom projectshttp://www.libtom.net37859222010-10-22 09:12:56+00:005.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
xcoreXCore open source projectNaNgithub.xcore.com261197572011-01-13 14:16:30+00:003.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
silverstripe-archiveSilverStripe ArchiveArchive of unsupported SilverStripe modules. I...http://silverstripe.org107172112011-01-17 00:22:34+00:004.0NaN...NaNNaNTrueNaNNaNNaNNaNNaNNaNNaN
mapboxMapboxMapbox is the location data platform for mobil...https://www.mapbox.com6281247004582011-02-04 19:02:13+00:004.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
openstateOpen State FoundationOpen State Foundation promotes digital transpa...https://openstate.eu1810723132011-03-15 21:42:43+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
charlotte-rubyCharlotte Ruby GroupCharlotte's local Ruby User Grouphttp://charlotteruby.org1018127772011-04-07 15:45:24+00:003.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNTrueNaN
pusherPusherPusher makes communication and collaboration A...https://pusher.com/192061488542011-04-19 17:16:38+00:004.0True...NaNNaNTrueNaNNaNNaNNaNNaNNaNNaN
culColumbia University LibrariesNaNhttp://library.columbia.edu516820162011-04-29 14:08:07+00:005.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
silexlabsSilex LabsSilex Labs is a foundation dedicated to helpin...http://www.silexlabs.org/657688132011-05-22 17:18:30+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
bbcBBCOpen source code used on public facing service...http://www.bbc.co.uk/opensource/105624100220362011-06-04 01:31:11+00:007.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
haikuHaikuAn open-source operating system that specifica...https://www.haiku-os.org101976692011-06-18 03:22:05+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
XCSoarXCSoar... the open-source glide computerhttps://xcsoar.org/79134112011-06-20 09:34:21+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
KozeaKozeaWe build open source software that you will love.https://community.kozea.fr/131032921392011-06-23 10:59:31+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
JetBrainsJetBrainsJetBrains open source projects.https://www.jetbrains.com9441028677942011-06-27 10:06:52+00:006.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
AutomatticAutomatticWe are passionate about making the web a bette...https://automattic.com150548192148922011-07-01 02:45:15+00:007.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
mantisbt-pluginsMantisBT Community PluginsNaNhttps://www.mantisbt.org2589160542011-07-12 14:07:02+00:005.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
neo4j-contribNeo4j ContribPublic, Open Source Contributions to the Neo4j...http://neo4j.com/developer15134915342011-07-14 23:15:43+00:0010.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
sandstormsandstormsandstorm - building great web applicationshttps://sandstorm.de/blog.html18440112011-08-03 08:20:12+00:003.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
KongKongNext-Generation API Platform for Microservices...https://konghq.com2513722961992011-08-06 02:08:16+00:006.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
CloudStack-extrasCollection of additional tools that are useful...NaNNaN1823260182011-08-26 14:40:35+00:008.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
hacsocHacSocThe organization for all things done by the CW...http://hacsoc.org/184216252011-09-11 04:47:22+00:002.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
vim-jpvim-jpVim community for Japanese developers and usershttps://vim-jp.org/47334041162011-09-15 02:44:30+00:003.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNTrueTrue
SitePenSitePenModernizing Apps, Tools & Teams for the Enterp...http://www.sitepen.com631614392011-09-21 08:39:10+00:003.0True...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
tikalkTikal Knowledge, Ltd.NaNwww.tikalk.com5195745992011-10-06 07:44:56+00:006.0NaN...NaNNaNTrueNaNNaNNaNNaNNaNNaNNaN
sbtsbtCommunity organization for all sbt plugin auth...https://www.scala-sbt.org361293911242011-10-28 14:16:17+00:0013.0NaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
\n", "

30 rows × 84 columns

\n", "
" ], "text/plain": [ " name \\\n", "datadesk Los Angeles Times Data Desk \n", "collective Collective \n", "uncopenweb UNC Open Web Group \n", "PerlDancer PerlDancer \n", "symphonists Symphony Community \n", "libtom libtom \n", "xcore XCore open source project \n", "silverstripe-archive SilverStripe Archive \n", "mapbox Mapbox \n", "openstate Open State Foundation \n", "charlotte-ruby Charlotte Ruby Group \n", "pusher Pusher \n", "cul Columbia University Libraries \n", "silexlabs Silex Labs \n", "bbc BBC \n", "haiku Haiku \n", "XCSoar XCSoar \n", "Kozea Kozea \n", "JetBrains JetBrains \n", "Automattic Automattic \n", "mantisbt-plugins MantisBT Community Plugins \n", "neo4j-contrib Neo4j Contrib \n", "sandstorm sandstorm \n", "Kong Kong \n", "CloudStack-extras Collection of additional tools that are useful... \n", "hacsoc HacSoc \n", "vim-jp vim-jp \n", "SitePen SitePen \n", "tikalk Tikal Knowledge, Ltd. \n", "sbt sbt \n", "\n", " description \\\n", "datadesk Analysis, applications and automation from a t... \n", "collective Plone add-ons shared code repositories \n", "uncopenweb NaN \n", "PerlDancer The Dancer Developers group \n", "symphonists NaN \n", "libtom libtom projects \n", "xcore NaN \n", "silverstripe-archive Archive of unsupported SilverStripe modules. I... \n", "mapbox Mapbox is the location data platform for mobil... \n", "openstate Open State Foundation promotes digital transpa... \n", "charlotte-ruby Charlotte's local Ruby User Group \n", "pusher Pusher makes communication and collaboration A... \n", "cul NaN \n", "silexlabs Silex Labs is a foundation dedicated to helpin... \n", "bbc Open source code used on public facing service... \n", "haiku An open-source operating system that specifica... \n", "XCSoar ... the open-source glide computer \n", "Kozea We build open source software that you will love. \n", "JetBrains JetBrains open source projects. \n", "Automattic We are passionate about making the web a bette... \n", "mantisbt-plugins NaN \n", "neo4j-contrib Public, Open Source Contributions to the Neo4j... \n", "sandstorm sandstorm - building great web applications \n", "Kong Next-Generation API Platform for Microservices... \n", "CloudStack-extras NaN \n", "hacsoc The organization for all things done by the CW... \n", "vim-jp Vim community for Japanese developers and users \n", "SitePen Modernizing Apps, Tools & Teams for the Enterp... \n", "tikalk NaN \n", "sbt Community organization for all sbt plugin auth... \n", "\n", " url members \\\n", "datadesk https://www.latimes.com 8 \n", "collective https://collective.github.io 268 \n", "uncopenweb http://sites.google.com/site/uncopenweb/ 11 \n", "PerlDancer http://perldancer.org 10 \n", "symphonists https://www.getsymphony.com 12 \n", "libtom http://www.libtom.net 3 \n", "xcore github.xcore.com 26 \n", "silverstripe-archive http://silverstripe.org 10 \n", "mapbox https://www.mapbox.com 62 \n", "openstate https://openstate.eu 18 \n", "charlotte-ruby http://charlotteruby.org 10 \n", "pusher https://pusher.com/ 19 \n", "cul http://library.columbia.edu 5 \n", "silexlabs http://www.silexlabs.org/ 6 \n", "bbc http://www.bbc.co.uk/opensource/ 105 \n", "haiku https://www.haiku-os.org 10 \n", "XCSoar https://xcsoar.org/ 7 \n", "Kozea https://community.kozea.fr/ 13 \n", "JetBrains https://www.jetbrains.com 94 \n", "Automattic https://automattic.com 150 \n", "mantisbt-plugins https://www.mantisbt.org 25 \n", "neo4j-contrib http://neo4j.com/developer 15 \n", "sandstorm https://sandstorm.de/blog.html 1 \n", "Kong https://konghq.com 25 \n", "CloudStack-extras NaN 18 \n", "hacsoc http://hacsoc.org/ 18 \n", "vim-jp https://vim-jp.org/ 47 \n", "SitePen http://www.sitepen.com 6 \n", "tikalk www.tikalk.com 5 \n", "sbt https://www.scala-sbt.org 36 \n", "\n", " repositories stars collaborators \\\n", "datadesk 184 313 27 \n", "collective 1674 569 628 \n", "uncopenweb 23 15 2 \n", "PerlDancer 71 708 15 \n", "symphonists 106 47 13 \n", "libtom 7 859 22 \n", "xcore 119 75 7 \n", "silverstripe-archive 71 72 11 \n", "mapbox 812 4700 458 \n", "openstate 107 23 13 \n", "charlotte-ruby 18 1277 7 \n", "pusher 206 1488 54 \n", "cul 168 20 16 \n", "silexlabs 57 688 13 \n", "bbc 624 1002 2036 \n", "haiku 19 766 9 \n", "XCSoar 9 134 11 \n", "Kozea 103 2921 39 \n", "JetBrains 410 28677 94 \n", "Automattic 548 19214 892 \n", "mantisbt-plugins 89 160 54 \n", "neo4j-contrib 134 915 34 \n", "sandstorm 84 40 11 \n", "Kong 137 22961 99 \n", "CloudStack-extras 23 260 18 \n", "hacsoc 42 16 25 \n", "vim-jp 33 404 116 \n", "SitePen 31 614 39 \n", "tikalk 195 745 99 \n", "sbt 129 3911 24 \n", "\n", " creation date transferred repositories \\\n", "datadesk 2010-07-02 02:04:07+00:00 6.0 \n", "collective 2010-08-13 00:04:43+00:00 7.0 \n", "uncopenweb 2010-09-04 01:22:47+00:00 6.0 \n", "PerlDancer 2010-09-21 12:27:49+00:00 2.0 \n", "symphonists 2010-10-21 15:40:12+00:00 56.0 \n", "libtom 2010-10-22 09:12:56+00:00 5.0 \n", "xcore 2011-01-13 14:16:30+00:00 3.0 \n", "silverstripe-archive 2011-01-17 00:22:34+00:00 4.0 \n", "mapbox 2011-02-04 19:02:13+00:00 4.0 \n", "openstate 2011-03-15 21:42:43+00:00 2.0 \n", "charlotte-ruby 2011-04-07 15:45:24+00:00 3.0 \n", "pusher 2011-04-19 17:16:38+00:00 4.0 \n", "cul 2011-04-29 14:08:07+00:00 5.0 \n", "silexlabs 2011-05-22 17:18:30+00:00 2.0 \n", "bbc 2011-06-04 01:31:11+00:00 7.0 \n", "haiku 2011-06-18 03:22:05+00:00 2.0 \n", "XCSoar 2011-06-20 09:34:21+00:00 2.0 \n", "Kozea 2011-06-23 10:59:31+00:00 2.0 \n", "JetBrains 2011-06-27 10:06:52+00:00 6.0 \n", "Automattic 2011-07-01 02:45:15+00:00 7.0 \n", "mantisbt-plugins 2011-07-12 14:07:02+00:00 5.0 \n", "neo4j-contrib 2011-07-14 23:15:43+00:00 10.0 \n", "sandstorm 2011-08-03 08:20:12+00:00 3.0 \n", "Kong 2011-08-06 02:08:16+00:00 6.0 \n", "CloudStack-extras 2011-08-26 14:40:35+00:00 8.0 \n", "hacsoc 2011-09-11 04:47:22+00:00 2.0 \n", "vim-jp 2011-09-15 02:44:30+00:00 3.0 \n", "SitePen 2011-09-21 08:39:10+00:00 3.0 \n", "tikalk 2011-10-06 07:44:56+00:00 6.0 \n", "sbt 2011-10-28 14:16:17+00:00 13.0 \n", "\n", " keyword app ... keyword shared keyword sharing \\\n", "datadesk NaN ... NaN NaN \n", "collective NaN ... True NaN \n", "uncopenweb NaN ... NaN NaN \n", "PerlDancer NaN ... NaN NaN \n", "symphonists NaN ... NaN NaN \n", "libtom NaN ... NaN NaN \n", "xcore NaN ... NaN NaN \n", "silverstripe-archive NaN ... NaN NaN \n", "mapbox NaN ... NaN NaN \n", "openstate NaN ... NaN NaN \n", "charlotte-ruby NaN ... NaN NaN \n", "pusher True ... NaN NaN \n", "cul NaN ... NaN NaN \n", "silexlabs NaN ... NaN NaN \n", "bbc NaN ... NaN NaN \n", "haiku NaN ... NaN NaN \n", "XCSoar NaN ... NaN NaN \n", "Kozea NaN ... NaN NaN \n", "JetBrains NaN ... NaN NaN \n", "Automattic NaN ... NaN NaN \n", "mantisbt-plugins NaN ... NaN NaN \n", "neo4j-contrib NaN ... NaN NaN \n", "sandstorm NaN ... NaN NaN \n", "Kong NaN ... NaN NaN \n", "CloudStack-extras NaN ... NaN NaN \n", "hacsoc NaN ... NaN NaN \n", "vim-jp NaN ... NaN NaN \n", "SitePen True ... NaN NaN \n", "tikalk NaN ... NaN NaN \n", "sbt NaN ... NaN NaN \n", "\n", " keyword support keyword supporter keyword supporters \\\n", "datadesk NaN NaN NaN \n", "collective NaN NaN NaN \n", "uncopenweb NaN NaN NaN \n", "PerlDancer NaN NaN NaN \n", "symphonists NaN NaN NaN \n", "libtom NaN NaN NaN \n", "xcore NaN NaN NaN \n", "silverstripe-archive True NaN NaN \n", "mapbox NaN NaN NaN \n", "openstate NaN NaN NaN \n", "charlotte-ruby NaN NaN NaN \n", "pusher True NaN NaN \n", "cul NaN NaN NaN \n", "silexlabs NaN NaN NaN \n", "bbc NaN NaN NaN \n", "haiku NaN NaN NaN \n", "XCSoar NaN NaN NaN \n", "Kozea NaN NaN NaN \n", "JetBrains NaN NaN NaN \n", "Automattic NaN NaN NaN \n", "mantisbt-plugins NaN NaN NaN \n", "neo4j-contrib NaN NaN NaN \n", "sandstorm NaN NaN NaN \n", "Kong NaN NaN NaN \n", "CloudStack-extras NaN NaN NaN \n", "hacsoc NaN NaN NaN \n", "vim-jp NaN NaN NaN \n", "SitePen NaN NaN NaN \n", "tikalk True NaN NaN \n", "sbt NaN NaN NaN \n", "\n", " keyword supporting keyword together keyword unofficial \\\n", "datadesk NaN NaN NaN \n", "collective NaN NaN NaN \n", "uncopenweb NaN NaN NaN \n", "PerlDancer NaN NaN NaN \n", "symphonists NaN NaN NaN \n", "libtom NaN NaN NaN \n", "xcore NaN NaN NaN \n", "silverstripe-archive NaN NaN NaN \n", "mapbox NaN NaN NaN \n", "openstate NaN NaN NaN \n", "charlotte-ruby NaN NaN NaN \n", "pusher NaN NaN NaN \n", "cul NaN NaN NaN \n", "silexlabs NaN NaN NaN \n", "bbc NaN NaN NaN \n", "haiku NaN NaN NaN \n", "XCSoar NaN NaN NaN \n", "Kozea NaN NaN NaN \n", "JetBrains NaN NaN NaN \n", "Automattic NaN NaN NaN \n", "mantisbt-plugins NaN NaN NaN \n", "neo4j-contrib NaN NaN NaN \n", "sandstorm NaN NaN NaN \n", "Kong NaN NaN NaN \n", "CloudStack-extras NaN NaN NaN \n", "hacsoc NaN NaN NaN \n", "vim-jp NaN NaN NaN \n", "SitePen NaN NaN NaN \n", "tikalk NaN NaN NaN \n", "sbt NaN NaN NaN \n", "\n", " keyword user keyword users \n", "datadesk NaN NaN \n", "collective NaN NaN \n", "uncopenweb NaN NaN \n", "PerlDancer NaN True \n", "symphonists NaN NaN \n", "libtom NaN NaN \n", "xcore NaN NaN \n", "silverstripe-archive NaN NaN \n", "mapbox NaN NaN \n", "openstate NaN NaN \n", "charlotte-ruby True NaN \n", "pusher NaN NaN \n", "cul NaN NaN \n", "silexlabs NaN NaN \n", "bbc NaN NaN \n", "haiku NaN NaN \n", "XCSoar NaN NaN \n", "Kozea NaN NaN \n", "JetBrains NaN NaN \n", "Automattic NaN NaN \n", "mantisbt-plugins NaN NaN \n", "neo4j-contrib NaN NaN \n", "sandstorm NaN NaN \n", "Kong NaN NaN \n", "CloudStack-extras NaN NaN \n", "hacsoc NaN NaN \n", "vim-jp True True \n", "SitePen NaN NaN \n", "tikalk NaN NaN \n", "sbt NaN NaN \n", "\n", "[30 rows x 84 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "values[values['transferred repositories'] > 1].sort_values('creation date')[0:30]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A list of instances of this model of community organizations" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name coq-community\n", "description A project for a collaborative, community-drive...\n", "url https://github.com/coq-community/manifesto\n", "members 27\n", "repositories 22\n", "stars 112\n", "collaborators 27\n", "creation date 2017-12-11 16:11:12+00:00\n", "Name: coq-community, dtype: object\n", "\n", "name D Community hub\n", "description Community hub for popular D projects\n", "url https://github.com/dlang-community/discussions\n", "members 8\n", "repositories 24\n", "stars 293\n", "collaborators 24\n", "creation date 2016-12-11 20:18:39+00:00\n", "Name: dlang-community, dtype: object\n", "\n", "name Elm Community\n", "description Unofficial group for shared work on Elm packag...\n", "url https://elm-community.github.io\n", "members 14\n", "repositories 53\n", "stars 885\n", "collaborators 36\n", "creation date 2015-11-20 20:59:45+00:00\n", "Name: elm-community, dtype: object\n", "\n", "name Elytra\n", "description A group of people who like making cool mods an...\n", "url https://elytradev.com/\n", "members 6\n", "repositories 56\n", "stars 23\n", "collaborators 13\n", "creation date 2016-05-06 09:11:12+00:00\n", "Name: elytra, dtype: object\n", "\n", "name fluent-plugins-nursery\n", "description Collaborate to maintain Fluentd plugins.\n", "url NaN\n", "members 2\n", "repositories 12\n", "stars 143\n", "collaborators 13\n", "creation date 2016-09-05 02:46:52+00:00\n", "Name: fluent-plugins-nursery, dtype: object\n", "\n", "name OCaml Community\n", "description A collaborative, community-driven project for ...\n", "url https://github.com/ocaml-community/meta\n", "members 6\n", "repositories 13\n", "stars 1742\n", "collaborators 16\n", "creation date 2018-08-10 12:06:41+00:00\n", "Name: ocaml-community, dtype: object\n", "\n", "name React Native Community\n", "description Quality code from the React Native Community\n", "url https://github.com/react-native-community/.github\n", "members 40\n", "repositories 72\n", "stars 11743\n", "collaborators 85\n", "creation date 2016-07-03 18:18:20+00:00\n", "Name: react-native-community, dtype: object\n", "\n", "name reasonml-community\n", "description Reason and BuckleScript's community packages\n", "url https://reasonml.github.io/docs/en/community.html\n", "members 11\n", "repositories 31\n", "stars 631\n", "collaborators 26\n", "creation date 2017-01-14 03:09:50+00:00\n", "Name: reasonml-community, dtype: object\n", "\n", "name Electron Userland\n", "description Third party community maintained electron modules\n", "url https://github.com/electron-userland/about#readme\n", "members 9\n", "repositories 30\n", "stars 7498\n", "collaborators 20\n", "creation date 2016-02-27 20:22:27+00:00\n", "Name: electron-userland, dtype: object\n", "\n", "name F# Community Project Incubation Space\n", "description NaN\n", "url http://fsprojects.github.io\n", "members 39\n", "repositories 115\n", "stars 1485\n", "collaborators 120\n", "creation date 2013-11-21 13:40:41+00:00\n", "Name: fsprojects, dtype: object\n", "\n", "name Sous Chefs\n", "description Community of chef cookbook maintainers\n", "url http://sous-chefs.org\n", "members 16\n", "repositories 78\n", "stars 522\n", "collaborators 51\n", "creation date 2015-05-08 20:27:19+00:00\n", "Name: sous-chefs, dtype: object\n", "\n", "name Vox Pupuli\n", "description Modules and tooling maintained by and for the ...\n", "url https://voxpupuli.org\n", "members 59\n", "repositories 175\n", "stars 598\n", "collaborators 12\n", "creation date 2014-09-08 09:22:01+00:00\n", "Name: voxpupuli, dtype: object\n", "\n" ] } ], "source": [ "for org in [\n", " 'coq-community',\n", " 'dlang-community',\n", " 'elm-community',\n", " 'elytra',\n", " 'fluent-plugins-nursery',\n", " 'ocaml-community',\n", " 'react-native-community',\n", " 'reasonml-community',\n", " 'electron-userland',\n", " 'fsprojects',\n", " 'sous-chefs',\n", " 'voxpupuli'\n", "]:\n", " print(values.loc[org][columns + ['creation date']])\n", " print()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Future work: find meta-repositories\n", "\n", "We fetch metrics on all the repositories of a given organization (or just the 100 most...) and we hope to find the meta-repository as an outlier for some metrics." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "query = \"\"\"\n", "query repoMetrics($org: String!) {\n", " organization(login: $org) {\n", " repositories(first: 100, orderBy: {field: STARGAZERS, direction: DESC}) {\n", " nodes {\n", " name\n", " createdAt\n", " diskUsage\n", " issues(first:1) {\n", " totalCount\n", " nodes {\n", " createdAt\n", " comments { totalCount }\n", " }\n", " }\n", " pullRequests(first:1) {\n", " totalCount\n", " nodes {\n", " createdAt\n", " comments { totalCount }\n", " }\n", " }\n", " isFork\n", " forkCount\n", " stargazers { totalCount }\n", " languages {\n", " totalSize\n", " totalCount\n", " }\n", " primaryLanguage { name }\n", " }\n", " }\n", " }\n", "}\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }