{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# StarCraft II Replay Analysis\n", "\n", "This code pattern will guide you through StarCraft II replay analysis.\n", "\n", "> Note: Instructions assume this notebook is running in IBM Watson Studio.\n", "\n", "## Learning goals\n", "\n", "This code pattern covers the following learning goals:\n", "\n", "1. Getting started with Jupyter notebooks in Watson Studio\n", "3. Using IBM Cloud Object Storage to access a replay file\n", "3. Using sc2reader to load a replay into a Python object\n", "4. Examining some of the basic replay information in the result\n", "5. Parsing the contest details into a usable object\n", "6. Visualizing the contest with graphics\n", "7. Storing the processed replay in Cloudant" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup prerequisites\n", "### Install the _sc2reader_, _cloudant_ and _bokeh_ Python packages from PyPI\n", "\n", "* **_sc2reader_** is an open source library for processing StarCraft 2 replay files.\n", "* **_cloudant_** is the Python client for using the Cloudant NoSQL DB.\n", "* **_bokeh_** is a Python interactive visualization library.\n", "* **_seaborn_** is a Python data visualization library based on matplotlib.\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting cloudant==2.10.2\n", " Downloading https://files.pythonhosted.org/packages/3d/01/c2e2829675fcd3f3af93a8a6a39cb6a5606c6cfa76973dacd3fac0a8b8ea/cloudant-2.10.2.tar.gz (57kB)\n", "\u001b[K 100% |████████████████████████████████| 61kB 7.3MB/s eta 0:00:01\n", "\u001b[?25hRequirement not upgraded as not directly required: requests<3.0.0,>=2.7.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cloudant==2.10.2)\n", "Requirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests<3.0.0,>=2.7.0->cloudant==2.10.2)\n", "Requirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests<3.0.0,>=2.7.0->cloudant==2.10.2)\n", "Requirement not upgraded as not directly required: urllib3<1.23,>=1.21.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests<3.0.0,>=2.7.0->cloudant==2.10.2)\n", "Requirement not upgraded as not directly required: certifi>=2017.4.17 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests<3.0.0,>=2.7.0->cloudant==2.10.2)\n", "Building wheels for collected packages: cloudant\n", " Running setup.py bdist_wheel for cloudant ... \u001b[?25ldone\n", "\u001b[?25h Stored in directory: /home/dsxuser/.cache/pip/wheels/dd/0d/0b/6e0ca65193cde5e824aa56cacb4a96c087138464691979edba\n", "Successfully built cloudant\n", "Installing collected packages: cloudant\n", "Successfully installed cloudant-2.10.2\n", "Requirement not upgraded as not directly required: bokeh==0.12.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\n", "Requirement not upgraded as not directly required: six>=1.5.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: PyYAML>=3.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: python-dateutil>=2.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: Jinja2>=2.7 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: numpy>=1.7.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: tornado>=4.3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from bokeh==0.12.10)\n", "Requirement not upgraded as not directly required: MarkupSafe>=0.23 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from Jinja2>=2.7->bokeh==0.12.10)\n", "Collecting seaborn==0.9.0\n", " Downloading https://files.pythonhosted.org/packages/a8/76/220ba4420459d9c4c9c9587c6ce607bf56c25b3d3d2de62056efe482dadc/seaborn-0.9.0-py3-none-any.whl (208kB)\n", "\u001b[K 100% |████████████████████████████████| 215kB 4.3MB/s eta 0:00:01\n", "\u001b[?25hRequirement not upgraded as not directly required: pandas>=0.15.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: scipy>=0.14.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: numpy>=1.9.3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: matplotlib>=1.4.3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas>=0.15.2->seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas>=0.15.2->seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: six>=1.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from matplotlib>=1.4.3->seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: cycler>=0.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from matplotlib>=1.4.3->seaborn==0.9.0)\n", "Requirement not upgraded as not directly required: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from matplotlib>=1.4.3->seaborn==0.9.0)\n", "Installing collected packages: seaborn\n", "Successfully installed seaborn-0.9.0\n", "Collecting sc2reader==1.3.1\n", " Downloading https://files.pythonhosted.org/packages/f3/1d/805429dd1a044623df643369d523ba12b0c27062bb4da7a34f7f451bd703/sc2reader-1.3.1-py3-none-any.whl (281kB)\n", "\u001b[K 100% |████████████████████████████████| 286kB 3.3MB/s eta 0:00:01\n", "\u001b[?25hCollecting mpyq>=0.2.4 (from sc2reader==1.3.1)\n", " Downloading https://files.pythonhosted.org/packages/ff/10/76041d97aa01e4d0f93481942b4faf5652123acdd90fbff4e40bb8d9024c/mpyq-0.2.5.tar.gz\n", "Building wheels for collected packages: mpyq\n", " Running setup.py bdist_wheel for mpyq ... \u001b[?25ldone\n", "\u001b[?25h Stored in directory: /home/dsxuser/.cache/pip/wheels/59/f3/2e/c798fde3edd6431c5c9e66ebebe77d1af338487b77d058cc37\n", "Successfully built mpyq\n", "Installing collected packages: mpyq, sc2reader\n", "Successfully installed mpyq-0.2.5 sc2reader-1.3.1\n" ] } ], "source": [ "# We install the prerequisites using the `!pip install` syntax here.\n", "# In some cases, running pip install from a notebook may require a one-time kernel restart. Check the output for messages.\n", "!pip install --user cloudant==2.10.2\n", "!pip install --user bokeh==0.12.10\n", "!pip install --user seaborn==0.9.0\n", "!pip install --user sc2reader==1.3.1\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# @hidden_cell\n", "credentials_1 = {\n", " \"apikey\": \"redacted\",\n", " \"username\": \"redacted\"\n", "}" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "import sys\n", "import types\n", "import pandas as pd\n", "from ibm_botocore.client import Config\n", "import ibm_boto3\n", "\n", "def __iter__(self): return 0\n", "\n", "# @hidden_cell\n", "# The following code accesses a file in your IBM Cloud Object Storage. It includes your credentials.\n", "# You might want to remove those credentials before you share your notebook.\n", "\n", "# ... credential code redacted ...\n", "\n", "# Your data file was loaded into a botocore.response.StreamingBody object.\n", "# Please read the documentation of ibm_boto3 and pandas to learn more about your possibilities to load the data.\n", "# ibm_boto3 documentation: https://ibm.github.io/ibm-cos-sdk-python/\n", "# pandas documentation: http://pandas.pydata.org/\n", "\n", "streaming_body_1 = streaming_body_2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load the replay\n", "\n", "Load in the replay with sc2reader. We'll use bytes that the inserted code read from our\n", "IBM Cloud Object Storage container." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Replay successfully loaded.\n" ] } ], "source": [ "import sc2reader\n", "from sc2reader.engine.plugins import APMTracker, ContextLoader, SelectionTracker\n", "from sc2reader.events import PlayerStatsEvent, UnitBornEvent, UnitDiedEvent, UnitDoneEvent, UnitTypeChangeEvent, UpgradeCompleteEvent\n", "\n", "# Some extra code here helps catch setup errors.\n", "try:\n", " replay_file = streaming_body_1\n", "except NameError:\n", " print('\\n'\n", " 'SETUP ERROR: Please follow the directions to add a .SC2Replay file and use\\n'\n", " ' \"Insert to code\" to set the streaming_body_1 variable to the resulting bytes.\\n'\n", " ' You may need to rename the data_* variable.')\n", " raise\n", "\n", "replay = sc2reader.load_replay(\n", " replay_file,\n", " engine=sc2reader.engine.GameEngine(plugins=[ContextLoader(), APMTracker(), SelectionTracker()]))\n", "\n", "print(\"Replay successfully loaded.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Print basic replay information\n", "\n", "With the replay added, we can now inspect the object in the notebook. We can easily get information\n", "like date, map name, participants, winner/loser, etc. " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Date: 2016-07-29 14:20:32\n", "Map Name: King Sejong Station LE\n", "Win: Player 1 - Neeb (Protoss)\n", "Loss: Player 2 - ShoWTimE (Protoss)\n" ] } ], "source": [ "print(\"Date: %s\" % replay.date)\n", "print(\"Map Name: \" + replay.map_name)\n", "for player in replay.players:\n", " print(\"%s: %s\" % (player.result, player))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# Establish some unit and building groups\n", "\n", "VESPENE_UNITS = [\"Assimilator\", \"Extractor\", \"Refinery\"]\n", "\n", "SUPPLY_UNITS = [\"Overlord\", \"Overseer\", \"Pylon\", \"SupplyDepot\"]\n", "\n", "WORKER_UNITS = [\"Drone\", \"Probe\", \"SCV\", \"MULE\"]\n", "\n", "BASE_UNITS = [\"CommandCenter\", \"Nexus\", \"Hatchery\", \"Lair\", \"Hive\", \"PlanetaryFortress\", \"OrbitalCommand\"]\n", "\n", "GROUND_UNITS = [\"Barracks\", \"Factory\", \"GhostAcademy\", \"Armory\", \"RoboticsBay\", \"RoboticsFacility\", \"TemplarArchive\",\n", " \"DarkShrine\", \"WarpGate\", \"SpawningPool\", \"RoachWarren\", \"HydraliskDen\", \"BanelingNest\", \"UltraliskCavern\",\n", " \"LurkerDen\", \"InfestationPit\"]\n", "\n", "AIR_UNITS = [\"Starport\", \"FusionCore\", \"RoboticsFacility\", \"Stargate\", \"FleetBeacon\", \"Spire\", \"GreaterSpire\"]\n", "\n", "TECH_UNITS = [\"EngineeringBay\", \"Armory\", \"GhostAcademy\", \"TechLab\", \"FusionCore\", \"Forge\", \"CyberneticsCore\",\n", " \"TwilightCouncil\", \"RoboticsFacility\", \"RoboticsBay\", \"FleetBeacon\", \"TemplarArchive\", \"DarkShrine\",\n", " \"SpawningPool\", \"RoachWarren\", \"HydraliskDen\", \"BanelingNest\", \"UltraliskCavern\", \"LurkerDen\", \"Spire\",\n", " \"GreaterSpire\", \"EvolutionChamber\", \"InfestationPit\"]\n", "\n", "ARMY_UNITS = [\"Marine\", \"Colossus\", \"InfestorTerran\", \"Baneling\", \"Mothership\", \"MothershipCore\", \"Changeling\", \"SiegeTank\", \"Viking\", \"Reaper\",\n", " \"Ghost\", \"Marauder\", \"Thor\", \"Hellion\", \"Hellbat\", \"Cyclone\", \"Liberator\", \"Medivac\", \"Banshee\", \"Raven\", \"Battlecruiser\", \"Nuke\", \"Zealot\",\n", " \"Stalker\", \"HighTemplar\", \"Disruptor\", \"DarkTemplar\", \"Sentry\", \"Phoenix\", \"Carrier\", \"Oracle\", \"VoidRay\", \"Tempest\", \"WarpPrism\", \"Observer\",\n", " \"Immortal\", \"Adept\", \"Zergling\", \"Overlord\", \"Hydralisk\", \"Mutalisk\", \"Ultralisk\", \"Roach\", \"Infestor\", \"Corruptor\",\n", " \"BroodLord\", \"Queen\", \"Overseer\", \"Archon\", \"Broodling\", \"InfestedTerran\", \"Ravager\", \"Viper\", \"SwarmHost\"]\n", "\n", "ARMY_AIR = [\"Mothership\", \"MothershipCore\", \"Viking\", \"Liberator\", \"Medivac\", \"Banshee\", \"Raven\", \"Battlecruiser\",\n", " \"Viper\", \"Mutalisk\", \"Phoenix\", \"Oracle\", \"Carrier\", \"VoidRay\", \"Tempest\", \"Observer\", \"WarpPrism\", \"BroodLord\",\n", " \"Corruptor\", \"Observer\", \"Overseer\"]\n", "\n", "ARMY_GROUND = [k for k in ARMY_UNITS if k not in ARMY_AIR]" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# Establish our event parsers\n", "\n", "def handle_count(caller, event, key, add_value, start_val=0):\n", " if len(caller.players[event.unit.owner.pid][key]) == 0:\n", " caller.players[event.unit.owner.pid][key].append((0, 0))\n", " # Get the last value\n", " last_val = caller.players[event.unit.owner.pid][key][-1][1]\n", " caller.players[event.unit.owner.pid][key].append((event.frame, last_val + add_value))\n", "\n", "\n", "def handle_expansion_events(caller, event):\n", " if type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in BASE_UNITS:\n", " caller.players[event.unit.owner.pid][\"expansion_event\"].append((event.frame, \"+\", unit))\n", " handle_count(caller, event, \"expansion_buildings\", 1, start_val=1)\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in BASE_UNITS:\n", " caller.players[event.unit.owner.pid][\"expansion_event\"].append((event.frame, \"-\", unit))\n", " handle_count(caller, event, \"expansion_buildings\", -1, start_val=1)\n", " elif type(event) is UnitTypeChangeEvent:\n", " if event.unit.name in BASE_UNITS:\n", " caller.players[event.unit.owner.pid][\"expansion_event\"].append((event.frame, \"*\", event.unit.name))\n", "\n", "\n", "def handle_worker_events(caller, event):\n", " if type(event) is PlayerStatsEvent:\n", " caller.players[event.pid][\"workers_active\"].append((event.frame, event.workers_active_count))\n", " elif type(event) is UnitBornEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in WORKER_UNITS:\n", " caller.players[event.control_pid][\"worker_event\"].append((event.frame, \"+\", unit))\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in WORKER_UNITS:\n", " caller.players[event.unit.owner.pid][\"worker_event\"].append((event.frame, \"-\", unit))\n", "\n", "\n", "def handle_supply_events(caller, event):\n", " if type(event) is PlayerStatsEvent:\n", " caller.players[event.pid][\"supply_available\"].append((event.frame, int(event.food_made)))\n", " caller.players[event.pid][\"supply_consumed\"].append((event.frame, int(event.food_used)))\n", " utilization = 0 if event.food_made == 0 else event.food_used / event.food_made\n", " caller.players[event.pid][\"supply_utilization\"].append((event.frame, utilization))\n", " worker_ratio = 0 if event.food_used == 0 else event.workers_active_count / event.food_used\n", " caller.players[event.pid][\"worker_supply_ratio\"].append((event.frame, worker_ratio))\n", " elif type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in SUPPLY_UNITS:\n", " caller.players[event.unit.owner.pid][\"supply_event\"].append((event.frame, \"+\", unit))\n", " elif type(event) is UnitBornEvent:\n", " # Specifically for Overlord\n", " unit = str(event.unit).split()[0]\n", " if unit == \"Overlord\":\n", " caller.players[event.control_pid][\"supply_event\"].append((event.frame, \"+\", unit))\n", " elif type(event) is UnitDiedEvent:\n", " # Buildings/ Overlord/Overseer\n", " unit = str(event.unit).split()[0]\n", " if unit in SUPPLY_UNITS:\n", " caller.players[event.unit.owner.pid][\"supply_event\"].append((event.frame, \"-\", unit))\n", " elif type(event) is UnitTypeChangeEvent:\n", " if event.unit_type_name == \"Overseer\":\n", " caller.players[event.unit.owner.pid][\"supply_event\"].append((event.frame, \"*\", event.unit_type_name))\n", "\n", "\n", "def handle_vespene_events(caller, event):\n", " if type(event) is PlayerStatsEvent:\n", " caller.players[event.pid][\"vespene_available\"].append((event.frame, event.vespene_current))\n", " caller.players[event.pid][\"vespene_collection_rate\"].append((event.frame, event.vespene_collection_rate))\n", " vesp_per_worker = 0 if event.workers_active_count == 0 else event.vespene_collection_rate / event.workers_active_count\n", " caller.players[event.pid][\"vespene_per_worker_rate\"].append((event.frame, vesp_per_worker))\n", " caller.players[event.pid][\"vespene_cost_active_forces\"].append((event.frame, event.vespene_used_active_forces))\n", " caller.players[event.pid][\"vespene_spend\"].append((event.frame, event.vespene_used_current))\n", " caller.players[event.pid][\"vespene_value_current_technology\"].append((event.frame, event.vespene_used_current_technology))\n", " caller.players[event.pid][\"vespene_value_current_army\"].append((event.frame, event.vespene_used_current_army))\n", " caller.players[event.pid][\"vespene_value_current_economic\"].append((event.frame, event.vespene_used_current_economy))\n", " caller.players[event.pid][\"vespene_queued\"].append((event.frame, event.vespene_used_in_progress))\n", " caller.players[event.pid][\"vespene_queued_technology\"].append((event.frame, event.vespene_used_in_progress_technology))\n", " caller.players[event.pid][\"vespene_queued_army\"].append((event.frame, event.vespene_used_in_progress_technology))\n", " caller.players[event.pid][\"vespene_queued_economic\"].append((event.frame, event.vespene_used_in_progress_economy))\n", " elif type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in VESPENE_UNITS:\n", " caller.players[event.unit.owner.pid][\"vespene_event\"].append((event.frame, \"+\", unit))\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in VESPENE_UNITS:\n", " caller.players[event.unit.owner.pid][\"vespene_event\"].append((event.frame, \"-\", unit))\n", "\n", "\n", "def handle_resources_events(caller, event):\n", " if type(event) is PlayerStatsEvent:\n", " caller.players[event.pid][\"mineral_destruction\"].append((event.frame, event.minerals_killed))\n", " caller.players[event.pid][\"mineral_destruction_army\"].append((event.frame, event.minerals_killed_army))\n", " caller.players[event.pid][\"mineral_destruction_economic\"].append((event.frame, event.minerals_killed_economy))\n", " caller.players[event.pid][\"mineral_destruction_technology\"].append((event.frame, event.minerals_killed_technology))\n", " caller.players[event.pid][\"mineral_loss\"].append((event.frame, event.minerals_lost))\n", " caller.players[event.pid][\"mineral_loss_army\"].append((event.frame, event.minerals_lost_army))\n", " caller.players[event.pid][\"mineral_loss_economic\"].append((event.frame, event.minerals_lost_economy))\n", " caller.players[event.pid][\"mineral_loss_technology\"].append((event.frame, event.minerals_lost_technology))\n", "\n", " caller.players[event.pid][\"vespene_destruction\"].append((event.frame, event.vespene_killed))\n", " caller.players[event.pid][\"vespene_destruction_army\"].append((event.frame, event.vespene_killed_army))\n", " caller.players[event.pid][\"vespene_destruction_economic\"].append((event.frame, event.vespene_killed_economy))\n", " caller.players[event.pid][\"vespene_destruction_technology\"].append((event.frame, event.vespene_killed_technology))\n", " caller.players[event.pid][\"vespene_loss\"].append((event.frame, event.vespene_lost))\n", " caller.players[event.pid][\"vespene_loss_army\"].append((event.frame, event.vespene_lost_army))\n", " caller.players[event.pid][\"vespene_loss_economic\"].append((event.frame, event.vespene_lost_economy))\n", " caller.players[event.pid][\"vespene_loss_technology\"].append((event.frame, event.vespene_lost_technology))\n", "\n", "\n", "def handle_ground_events(caller, event):\n", " if type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in GROUND_UNITS:\n", " count_name = \"_\".join([\"building\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"ground_building\"].append((event.frame, \"+\", unit))\n", " handle_count(caller, event, count_name, 1)\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in GROUND_UNITS:\n", " count_name = \"_\".join([\"building\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"ground_building\"].append((event.frame, \"-\", unit))\n", " handle_count(caller, event, count_name, -1)\n", " elif type(event) is UnitTypeChangeEvent:\n", " if event.unit_type_name == \"LurkerDen\":\n", " count_name = \"_\".join([\"building\", event.unit_type_name, \"count\"])\n", " caller.players[event.unit.owner.pid][\"ground_building\"].append((event.frame, \"*\", event.unit_type_name))\n", " handle_count(caller, event, count_name, 1)\n", "\n", "\n", "def handle_air_events(caller, event):\n", " if type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in AIR_UNITS:\n", " count_name = \"_\".join([\"building\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"air_building\"].append((event.frame, \"+\", unit))\n", " handle_count(caller, event, count_name, 1)\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in AIR_UNITS:\n", " count_name = \"_\".join([\"building\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"air_building\"].append((event.frame, \"-\", unit))\n", " handle_count(caller, event, count_name, -1)\n", " elif type(event) is UnitTypeChangeEvent:\n", " if event.unit_type_name == \"GreaterSpire\":\n", " count_name = \"_\".join([\"building\", event.unit_type_name, \"count\"])\n", " caller.players[event.unit.owner.pid][\"air_building\"].append((event.frame, \"*\", event.unit_type_name))\n", " handle_count(caller, event, count_name, 1)\n", "\n", "\n", "def handle_unit_events(caller, event):\n", " if type(event) is UnitBornEvent:\n", " unit = event.unit_type_name\n", " if unit in ARMY_UNITS:\n", " unit_count_name = \"_\".join([\"unit\", unit, \"count\"])\n", " caller.players[event.control_pid][\"army_event\"].append((event.frame, \"+\", unit))\n", " handle_count(caller, event, unit_count_name, 1)\n", " if unit in ARMY_AIR:\n", " handle_count(caller, event, \"army_air\", 1)\n", " elif unit in ARMY_GROUND:\n", " handle_count(caller, event, \"army_ground\", 1)\n", " handle_count(caller, event, \"army_count\", 1)\n", " elif type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in ARMY_UNITS:\n", " unit_count_name = \"_\".join([\"unit\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"army_event\"].append((event.frame, \"+\", unit))\n", " handle_count(caller, event, unit_count_name, 1)\n", " if unit in ARMY_AIR:\n", " handle_count(caller, event, \"army_air\", 1)\n", " elif unit in ARMY_GROUND:\n", " handle_count(caller, event, \"army_air\", 1)\n", " handle_count(caller, event, \"army_count\", 1)\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in ARMY_UNITS:\n", " unit_count_name = \"_\".join([\"unit\", unit, \"count\"])\n", " caller.players[event.unit.owner.pid][\"army_event\"].append((event.frame, \"-\", unit))\n", " if unit in ARMY_AIR:\n", " handle_count(caller, event, \"army_air\", -1)\n", " elif unit in ARMY_GROUND:\n", " handle_count(caller, event, \"army_ground\", -1)\n", " handle_count(caller, event, unit_count_name, -1)\n", " handle_count(caller, event, \"army_count\", -1)\n", " elif type(event) is UnitTypeChangeEvent:\n", " unit = str(event.unit).split()[0]\n", " if event.unit_type_name in ARMY_UNITS:\n", " unit_count_name = \"_\".join([\"unit\", event.unit_type_name, \"count\"])\n", "\n", " caller.players[event.unit.owner.pid][\"army_event\"].append((event.frame, \"*\", unit))\n", "\n", " handle_count(caller, event, unit_count_name, 1)\n", "\n", "\n", "def handle_tech_events(caller, event):\n", " if type(event) is UnitDoneEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in TECH_UNITS:\n", " caller.players[event.unit.owner.pid][\"tech_building\"].append((event.frame, \"+\", unit))\n", " elif type(event) is UnitDiedEvent:\n", " unit = str(event.unit).split()[0]\n", " if unit in TECH_UNITS:\n", " caller.players[event.unit.owner.pid][\"tech_building\"].append((event.frame, \"-\", unit))\n", " elif type(event) is UnitTypeChangeEvent:\n", " if event.unit_type_name in [\"GreaterSpire\", \"LurkerDen\"]:\n", " caller.players[event.unit.owner.pid][\"tech_building\"].append((event.frame, \"*\", event.unit_type_name))\n", "\n", "\n", "def handle_upgrade_events(caller, event):\n", " if type(event) is UpgradeCompleteEvent and event.frame > 0:\n", " if not event.upgrade_type_name.startswith(\"Spray\"):\n", " caller.players[event.pid][\"upgrades\"].append((event.frame, event.upgrade_type_name))\n", "\n", "\n", "def handle_mineral_events(caller, event):\n", " if type(event) is PlayerStatsEvent:\n", " caller.players[event.pid][\"minerals_available\"].append((event.frame, event.minerals_current))\n", " caller.players[event.pid][\"mineral_collection_rate\"].append((event.frame, event.minerals_collection_rate,))\n", " caller.players[event.pid][\"mineral_cost_active_forces\"].append((event.frame, event.minerals_used_active_forces))\n", " mins_per_worker = 0 if event.workers_active_count == 0 else event.minerals_collection_rate / event.workers_active_count\n", " caller.players[event.pid][\"mineral_per_worker_rate\"].append((event.frame, mins_per_worker))\n", " caller.players[event.pid][\"mineral_spend\"].append((event.frame, event.minerals_used_current))\n", " caller.players[event.pid][\"mineral_value_current_technology\"].append((event.frame, event.minerals_used_current_technology))\n", " caller.players[event.pid][\"mineral_value_current_army\"].append((event.frame, event.minerals_used_current_army))\n", " caller.players[event.pid][\"mineral_value_current_economic\"].append((event.frame, event.minerals_used_current_economy))\n", " caller.players[event.pid][\"mineral_queued\"].append((event.frame, event.minerals_used_in_progress))\n", " caller.players[event.pid][\"mineral_queued_technology\"].append((event.frame, event.minerals_used_in_progress_technology))\n", " caller.players[event.pid][\"mineral_queued_army\"].append((event.frame, event.minerals_used_in_progress_army))\n", " caller.players[event.pid][\"mineral_queued_economic\"].append((event.frame, event.minerals_used_in_progress_economy))" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Aggregate all of our event parsers for use by our ReplayData class\n", "\n", "handlers = [handle_expansion_events, handle_worker_events, handle_supply_events, handle_mineral_events,\n", " handle_vespene_events, handle_ground_events, handle_air_events, handle_tech_events, handle_upgrade_events,\n", " handle_unit_events]" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# Below we define our class ReplayData for helping us structure and process our replay files\n", "\n", "class ReplayData:\n", " __parsers__ = handlers\n", "\n", " @classmethod\n", " def parse_replay(cls, replay=None, replay_file=None, file_object=None):\n", " \n", " replay_data = ReplayData(replay_file)\n", " try:\n", " # This is the engine that holds some required plugins for parsing\n", " engine = sc2reader.engine.GameEngine(plugins=[ContextLoader(), APMTracker(), SelectionTracker()])\n", " \n", " if replay:\n", " pass\n", " elif replay_file and not file_object:\n", " # Then we are not using ObjectStorage for accessing replay files\n", " replay = sc2reader.load_replay(replay_file, engine=engine)\n", " elif file_object:\n", " # We are using ObjectStorage to access replay files\n", " replay = sc2reader.load_replay(file_object, engine=engine)\n", " else:\n", " pass\n", " \n", " # Get the number of frames (one frame is 1/16 of a second)\n", " replay_data.frames = replay.frames\n", " # Gets the game mode (if available)\n", " replay_data.game_mode = replay.real_type\n", " # Gets the map hash (if we want to download the map, or do map-based analysis)\n", " replay_data.map_hash = replay.map_hash\n", " \n", " # Use the parsers to get data\n", " for event in replay.events:\n", " for parser in cls.__parsers__:\n", " parser(replay_data, event)\n", " \n", " # Check if there was a winner\n", " if replay.winner is not None:\n", " replay_data.winners = replay.winner.players\n", " replay_data.losers = [p for p in replay.players if p not in replay.winner.players]\n", " else:\n", " replay_data.winners = []\n", " replay_data.losers = []\n", " # Check to see if expansion data is available\n", " print(replay.expansion)\n", " replay_data.expansion = replay.expansion\n", " return replay_data\n", " except:\n", " # print our error and return NoneType object\n", " print_exc()\n", " return None\n", " \n", " def as_dict(self):\n", " return {\n", " \"processed_on\": datetime.utcnow().isoformat(),\n", " \"replay_name\": self.replay,\n", " \"expansion\": self.expansion,\n", " \"frames\": self.frames,\n", " \"mode\": self.game_mode,\n", " \"map\": self.map_hash,\n", " \"matchup\": \"v\".join(sorted([s.detail_data[\"race\"][0].upper() for s in self.winners + self.losers])),\n", " \"winners\": [(s.pid, s.name, s.detail_data['race']) for s in self.winners],\n", " \"losers\": [(s.pid, s.name, s.detail_data['race']) for s in self.losers],\n", " \"stats_names\": [k for k in self.players[1].keys()],\n", " \"stats\": {player: data for player, data in self.players.items()}\n", " }\n", "\n", " def __init__(self, replay):\n", " self.players = defaultdict(lambda: defaultdict(list))\n", " self.replay = replay\n", " self.winners = []\n", " self.losers = []\n", " self.expansion = None" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualize and compare replay events\n", "\n", "The replay file was processed to add event statistics using pandas and\n", "helper functions.\n", "* Bokeh is used to create Nelson rules charts.\n", "* Seaborn is used for box plot charts." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function(root) {\n", " function now() {\n", " return new Date();\n", " }\n", "\n", " var force = true;\n", "\n", " if (typeof (root._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n", " root._bokeh_onload_callbacks = [];\n", " root._bokeh_is_loading = undefined;\n", " }\n", "\n", " var JS_MIME_TYPE = 'application/javascript';\n", " var HTML_MIME_TYPE = 'text/html';\n", " var EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n", " var CLASS_NAME = 'output_bokeh rendered_html';\n", "\n", " /**\n", " * Render data to the DOM node\n", " */\n", " function render(props, node) {\n", " var script = document.createElement(\"script\");\n", " node.appendChild(script);\n", " }\n", "\n", " /**\n", " * Handle when an output is cleared or removed\n", " */\n", " function handleClearOutput(event, handle) {\n", " var cell = handle.cell;\n", "\n", " var id = cell.output_area._bokeh_element_id;\n", " var server_id = cell.output_area._bokeh_server_id;\n", " // Clean up Bokeh references\n", " if (id !== undefined) {\n", " Bokeh.index[id].model.document.clear();\n", " delete Bokeh.index[id];\n", " }\n", "\n", " if (server_id !== undefined) {\n", " // Clean up Bokeh references\n", " var cmd = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n", " cell.notebook.kernel.execute(cmd, {\n", " iopub: {\n", " output: function(msg) {\n", " var element_id = msg.content.text.trim();\n", " Bokeh.index[element_id].model.document.clear();\n", " delete Bokeh.index[element_id];\n", " }\n", " }\n", " });\n", " // Destroy server and session\n", " var cmd = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n", " cell.notebook.kernel.execute(cmd);\n", " }\n", " }\n", "\n", " /**\n", " * Handle when a new output is added\n", " */\n", " function handleAddOutput(event, handle) {\n", " var output_area = handle.output_area;\n", " var output = handle.output;\n", "\n", " // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n", " if ((output.output_type != \"display_data\") || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n", " return\n", " }\n", "\n", " var toinsert = output_area.element.find(`.${CLASS_NAME.split(' ')[0]}`);\n", "\n", " if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n", " toinsert[0].firstChild.textContent = output.data[JS_MIME_TYPE];\n", " // store reference to embed id on output_area\n", " output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n", " }\n", " if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n", " var bk_div = document.createElement(\"div\");\n", " bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n", " var script_attrs = bk_div.children[0].attributes;\n", " for (var i = 0; i < script_attrs.length; i++) {\n", " toinsert[0].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n", " }\n", " // store reference to server id on output_area\n", " output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n", " }\n", " }\n", "\n", " function register_renderer(events, OutputArea) {\n", "\n", " function append_mime(data, metadata, element) {\n", " // create a DOM node to render to\n", " var toinsert = this.create_output_subarea(\n", " metadata,\n", " CLASS_NAME,\n", " EXEC_MIME_TYPE\n", " );\n", " this.keyboard_manager.register_events(toinsert);\n", " // Render to node\n", " var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n", " render(props, toinsert[0]);\n", " element.append(toinsert);\n", " return toinsert\n", " }\n", "\n", " /* Handle when an output is cleared or removed */\n", " events.on('clear_output.CodeCell', handleClearOutput);\n", " events.on('delete.Cell', handleClearOutput);\n", "\n", " /* Handle when a new output is added */\n", " events.on('output_added.OutputArea', handleAddOutput);\n", "\n", " /**\n", " * Register the mime type and append_mime function with output_area\n", " */\n", " OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n", " /* Is output safe? */\n", " safe: true,\n", " /* Index of renderer in `output_area.display_order` */\n", " index: 0\n", " });\n", " }\n", "\n", " // register the mime type if in Jupyter Notebook environment and previously unregistered\n", " if (root.Jupyter !== undefined) {\n", " var events = require('base/js/events');\n", " var OutputArea = require('notebook/js/outputarea').OutputArea;\n", "\n", " if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n", " register_renderer(events, OutputArea);\n", " }\n", " }\n", "\n", " \n", " if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n", " root._bokeh_timeout = Date.now() + 5000;\n", " root._bokeh_failed_load = false;\n", " }\n", "\n", " var NB_LOAD_WARNING = {'data': {'text/html':\n", " \"\\n\"+\n", " \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n", " \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n", " \"
\\n\"+\n", " \"\\n\"+\n",
" \"from bokeh.resources import INLINE\\n\"+\n",
" \"output_notebook(resources=INLINE)\\n\"+\n",
" \"
\\n\"+\n",
" \"\\n\"+\n \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n \"
\\n\"+\n \"\\n\"+\n \"from bokeh.resources import INLINE\\n\"+\n \"output_notebook(resources=INLINE)\\n\"+\n \"
\\n\"+\n \"