{ "metadata": { "name": "", "signature": "sha256:38b59ae8be7fd65d5cce5c6965931cdbf367bff5a149b4d7c4bb4dbc629f67cd" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Preliminaries\n", "\n", "First, we'll load some required libraries and set some global options" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import pandas as pd # our core data analysis toolkit\n", "import numpy as np \n", "import pylab as pl # plotting libraries\n", "from ggplot import *\n", "from pandas import read_json # a function for reading data in JSON files\n", "\n", "# This option shows our plots directly in iPython Notebooks\n", "%matplotlib inline \n", "\n", "# This option gives a more pleasing visual style to our plots\n", "pd.set_option('display.mpl_style', 'default')\n", "\n", "# The location of our playtest data file\n", "filepath = \"2014-05-13 makescape playtest.json\"" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data Analysis\n", "\n", "## Loading and Cleaning\n", "\n", "Below we'll load the data and sort by increasing timestamp." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def loadDataSortedByTimestamp(filepath):\n", " x = read_json(filepath)\n", " x = x.sort(columns='timestamp')\n", " x.index = range(0, len(x))\n", " return(x)\n", "\n", "ms = loadDataSortedByTimestamp(filepath)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 2 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that our data is loaded with the variable `ms` (I chose it as an abbreviation of MakeScape), let's look at it and make sure it's sane. One of the first things I'll do is check the list of columns that our data comes with." ] }, { "cell_type": "code", "collapsed": false, "input": [ "ms.columns" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 3, "text": [ "Index([u'_id', u'ada_base_types', u'adage_version', u'application_name', u'application_version', u'board_mode', u'component_list', u'created_at', u'deviceInfo', u'fish', u'fish_list', u'game', u'game_id', u'key', u'mode_name', u'num_batteries', u'num_leds', u'num_resistors', u'num_timers', u'player_name', u'player_names', u'playspace_id', u'playspace_ids', u'reason', u'resistance', u'session_token', u'timed_out', u'timestamp', u'updated_at', u'user_id', u'virtual_context', u'visability_mode', u'voltage'], dtype='object')" ] } ], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Whoa! That is [alot of columns](http://hyperboleandahalf.blogspot.com/2010/04/alot-is-better-than-you-at-everything.html). 33 columns, to be exact. We can check that by calling Python's function for determining the length of a collection:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "len(ms.columns)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 4, "text": [ "33" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "But we should also check how many rows (in this case, how many distinct events) we have in our dataset. " ] }, { "cell_type": "code", "collapsed": false, "input": [ "len(ms) # returns 8505" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 5, "text": [ "8505" ] } ], "prompt_number": 5 }, { "cell_type": "code", "collapsed": false, "input": [ "columns = ['key', 'timestamp']\n", "ms.head(n=5)[columns]" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", " | key | \n", "timestamp | \n", "
---|---|---|
0 | \n", "ADAGEStartSession | \n", "1398178271860 | \n", "
1 | \n", "ADAGEStartSession | \n", "1398190767768 | \n", "
2 | \n", "ADAGEStartSession | \n", "1398191616469 | \n", "
3 | \n", "ADAGEStartSession | \n", "1398192512628 | \n", "
4 | \n", "ADAGEStartSession | \n", "1398192887546 | \n", "
5 rows \u00d7 2 columns
\n", "\n", " | key | \n", "timestamp | \n", "human-readable-timestamp | \n", "
---|---|---|---|
0 | \n", "ADAGEStartSession | \n", "1398178271860 | \n", "2014-04-22 14:51:11.860000 | \n", "
1 | \n", "ADAGEStartSession | \n", "1398190767768 | \n", "2014-04-22 18:19:27.768000 | \n", "
2 | \n", "ADAGEStartSession | \n", "1398191616469 | \n", "2014-04-22 18:33:36.469000 | \n", "
3 | \n", "ADAGEStartSession | \n", "1398192512628 | \n", "2014-04-22 18:48:32.628000 | \n", "
4 | \n", "ADAGEStartSession | \n", "1398192887546 | \n", "2014-04-22 18:54:47.546000 | \n", "
5 rows \u00d7 3 columns
\n", "\n", " | key | \n", "timestamp | \n", "delta1 | \n", "delta2 | \n", "delta3 | \n", "delta4 | \n", "
---|---|---|---|---|---|---|
2482 | \n", "MakeDisconnectComponent | \n", "1400006485499 | \n", "0 | \n", "0 | \n", "0 | \n", "-2102 | \n", "
2506 | \n", "MakeDisconnectComponent | \n", "1400006510313 | \n", "0 | \n", "0 | \n", "0 | \n", "-1886 | \n", "
2526 | \n", "MakeDisconnectComponent | \n", "1400006521978 | \n", "0 | \n", "0 | \n", "0 | \n", "-3534 | \n", "
2572 | \n", "MakeDisconnectComponent | \n", "1400006562692 | \n", "0 | \n", "0 | \n", "0 | \n", "-4151 | \n", "
2623 | \n", "MakeDisconnectComponent | \n", "1400006622320 | \n", "0 | \n", "0 | \n", "0 | \n", "-1 | \n", "
2646 | \n", "MakeDisconnectComponent | \n", "1400006634386 | \n", "0 | \n", "0 | \n", "0 | \n", "-685 | \n", "
2647 | \n", "MakeDisconnectComponent | \n", "1400006634386 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
2672 | \n", "MakeDisconnectComponent | \n", "1400006645003 | \n", "0 | \n", "0 | \n", "0 | \n", "-1 | \n", "
2958 | \n", "MakeDisconnectComponent | \n", "1400006866421 | \n", "0 | \n", "0 | \n", "0 | \n", "-2335 | \n", "
3018 | \n", "MakeDisconnectComponent | \n", "1400006896668 | \n", "0 | \n", "0 | \n", "0 | \n", "-2534 | \n", "
3200 | \n", "MakeDisconnectComponent | \n", "1400006993361 | \n", "0 | \n", "0 | \n", "0 | \n", "-1301 | \n", "
3351 | \n", "MakeDisconnectComponent | \n", "1400007068806 | \n", "0 | \n", "0 | \n", "0 | \n", "-850 | \n", "
3480 | \n", "MakeDisconnectComponent | \n", "1400007157316 | \n", "0 | \n", "0 | \n", "0 | \n", "-781 | \n", "
3512 | \n", "MakeDisconnectComponent | \n", "1400007196213 | \n", "0 | \n", "0 | \n", "0 | \n", "-1518 | \n", "
3570 | \n", "MakeCaptureFish | \n", "1400007234810 | \n", "0 | \n", "0 | \n", "0 | \n", "-51 | \n", "
3667 | \n", "MakeDisconnectComponent | \n", "1400007308155 | \n", "0 | \n", "0 | \n", "0 | \n", "-3251 | \n", "
3785 | \n", "MakeSnapshot | \n", "1400007410096 | \n", "0 | \n", "0 | \n", "0 | \n", "-1783 | \n", "
4199 | \n", "MakeDisconnectComponent | \n", "1400007951843 | \n", "0 | \n", "0 | \n", "0 | \n", "-768 | \n", "
7955 | \n", "MakeCircuitCreated | \n", "1400083772689 | \n", "0 | \n", "0 | \n", "0 | \n", "-1300 | \n", "
8022 | \n", "MakeDisconnectComponent | \n", "1400083848640 | \n", "0 | \n", "0 | \n", "0 | \n", "-785 | \n", "
8088 | \n", "MakeCaptureFish | \n", "1400083893390 | \n", "0 | \n", "0 | \n", "0 | \n", "-152 | \n", "
8123 | \n", "MakeCaptureFish | \n", "1400083924105 | \n", "0 | \n", "0 | \n", "0 | \n", "-1085 | \n", "
22 rows \u00d7 6 columns
\n", "\n", " | key | \n", "timestamp | \n", "delta1 | \n", "delta2 | \n", "delta3 | \n", "delta4 | \n", "
---|---|---|---|---|---|---|
2470 | \n", "MakeConnectComponent | \n", "1400006472299 | \n", "0 | \n", "-401 | \n", "-307 | \n", "-490 | \n", "
2471 | \n", "MakeConnectComponent | \n", "1400006472730 | \n", "431 | \n", "431 | \n", "832 | \n", "1139 | \n", "
2472 | \n", "MakeCircuitCreated | \n", "1400006472730 | \n", "0 | \n", "-431 | \n", "-862 | \n", "-1694 | \n", "
2473 | \n", "MakeConnectComponent | \n", "1400006472730 | \n", "0 | \n", "0 | \n", "431 | \n", "1293 | \n", "
2474 | \n", "MakeSnapshot | \n", "1400006473382 | \n", "652 | \n", "652 | \n", "652 | \n", "221 | \n", "
2475 | \n", "MakeSpawnFish | \n", "1400006473483 | \n", "101 | \n", "-551 | \n", "-1203 | \n", "-1855 | \n", "
2476 | \n", "MakeSpawnFish | \n", "1400006473484 | \n", "1 | \n", "-100 | \n", "451 | \n", "1654 | \n", "
2477 | \n", "MakeSnapshot | \n", "1400006478397 | \n", "4913 | \n", "4912 | \n", "5012 | \n", "4561 | \n", "
2478 | \n", "MakeSnapshot | \n", "1400006483397 | \n", "5000 | \n", "87 | \n", "-4825 | \n", "-9837 | \n", "
2479 | \n", "MakeCaptureFish | \n", "1400006485499 | \n", "2102 | \n", "-2898 | \n", "-2985 | \n", "1840 | \n", "
2480 | \n", "MakeDisconnectComponent | \n", "1400006485499 | \n", "0 | \n", "-2102 | \n", "796 | \n", "3781 | \n", "
2481 | \n", "MakeDisconnectComponent | \n", "1400006485499 | \n", "0 | \n", "0 | \n", "2102 | \n", "1306 | \n", "
2482 | \n", "MakeDisconnectComponent | \n", "1400006485499 | \n", "0 | \n", "0 | \n", "0 | \n", "-2102 | \n", "
2483 | \n", "MakeDisconnectComponent | \n", "1400006485514 | \n", "15 | \n", "15 | \n", "15 | \n", "15 | \n", "
2484 | \n", "MakeDisconnectComponent | \n", "1400006485514 | \n", "0 | \n", "-15 | \n", "-30 | \n", "-45 | \n", "
15 rows \u00d7 6 columns
\n", "\n", " | timestamp | \n", "key | \n", "
---|---|---|
key | \n", "\n", " | \n", " |
MakeConnectComponent | \n", "2609 | \n", "2609 | \n", "
MakeSnapshot | \n", "2086 | \n", "2086 | \n", "
MakeDisconnectComponent | \n", "943 | \n", "943 | \n", "
MakeAddComponent | \n", "891 | \n", "891 | \n", "
MakeRemoveComponent | \n", "840 | \n", "840 | \n", "
MakeCircuitCreated | \n", "229 | \n", "229 | \n", "
MakeResetBoard | \n", "211 | \n", "211 | \n", "
MakeSpawnFish | \n", "161 | \n", "161 | \n", "
MakeCaptureFish | \n", "136 | \n", "136 | \n", "
MakeEndGame | \n", "100 | \n", "100 | \n", "
MakeSummonBoard | \n", "96 | \n", "96 | \n", "
MakeStartGame | \n", "92 | \n", "92 | \n", "
MakeModeChange | \n", "45 | \n", "45 | \n", "
MakeVisabilityChange | \n", "43 | \n", "43 | \n", "
ADAGEStartSession | \n", "23 | \n", "23 | \n", "
15 rows \u00d7 2 columns
\n", "\n", " | key | \n", "component_list | \n", "
---|---|---|
980 | \n", "MakeDisconnectComponent | \n", "None | \n", "
1154 | \n", "MakeDisconnectComponent | \n", "None | \n", "
1252 | \n", "MakeDisconnectComponent | \n", "None | \n", "
1274 | \n", "MakeDisconnectComponent | \n", "None | \n", "
2958 | \n", "MakeDisconnectComponent | \n", "None | \n", "
3018 | \n", "MakeDisconnectComponent | \n", "None | \n", "
3200 | \n", "MakeDisconnectComponent | \n", "None | \n", "
3262 | \n", "MakeDisconnectComponent | \n", "None | \n", "
3304 | \n", "MakeDisconnectComponent | \n", "None | \n", "
4473 | \n", "MakeDisconnectComponent | \n", "None | \n", "
7228 | \n", "MakeDisconnectComponent | \n", "None | \n", "
11 rows \u00d7 2 columns
\n", "\n", " | key | \n", "timestamp | \n", "
---|---|---|
970 | \n", "MakeConnectComponent | \n", "1400003097166 | \n", "
971 | \n", "MakeDisconnectComponent | \n", "1400003098417 | \n", "
972 | \n", "MakeCaptureFish | \n", "1400003098417 | \n", "
973 | \n", "MakeDisconnectComponent | \n", "1400003098433 | \n", "
974 | \n", "MakeConnectComponent | \n", "1400003098966 | \n", "
975 | \n", "MakeCircuitCreated | \n", "1400003098966 | \n", "
976 | \n", "MakeConnectComponent | \n", "1400003098966 | \n", "
977 | \n", "MakeConnectComponent | \n", "1400003099232 | \n", "
978 | \n", "MakeSpawnFish | \n", "1400003099634 | \n", "
979 | \n", "MakeSnapshot | \n", "1400003100748 | \n", "
980 | \n", "MakeDisconnectComponent | \n", "1400003101299 | \n", "
981 | \n", "MakeCaptureFish | \n", "1400003101299 | \n", "
982 | \n", "MakeDisconnectComponent | \n", "1400003101299 | \n", "
983 | \n", "MakeDisconnectComponent | \n", "1400003101314 | \n", "
984 | \n", "MakeSnapshot | \n", "1400003105748 | \n", "
985 | \n", "MakeConnectComponent | \n", "1400003107948 | \n", "
16 rows \u00d7 2 columns
\n", "\n", " | key | \n", "timestamp | \n", "delta1 | \n", "
---|---|---|---|
980 | \n", "MakeDisconnectComponent | \n", "1400003101299 | \n", "551 | \n", "
981 | \n", "MakeCaptureFish | \n", "1400003101299 | \n", "0 | \n", "
982 | \n", "MakeDisconnectComponent | \n", "1400003101299 | \n", "0 | \n", "
983 | \n", "MakeDisconnectComponent | \n", "1400003101314 | \n", "15 | \n", "
4 rows \u00d7 3 columns
\n", "