{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a Video Game Bot using OpenAI Universe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Let us learn how to build a video game bot which plays car racing game. Our objective is\n",
    "that car has to move forward without getting stuck by any obstacles and hitting other cars."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we import necessary libraries,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gym\n",
    "import universe \n",
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we simulate our car racing environment by make function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env = gym.make('flashgames.NeonRace-v0')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "env.configure(remotes=1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let us create variables for moving the car,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Move left\n",
    "left = [('KeyEvent', 'ArrowUp', True), ('KeyEvent', 'ArrowLeft', True),\n",
    "         ('KeyEvent', 'ArrowRight', False)]\n",
    "\n",
    "# Move right\n",
    "right = [('KeyEvent', 'ArrowUp', True), ('KeyEvent', 'ArrowLeft', False),\n",
    "         ('KeyEvent', 'ArrowRight', True)]\n",
    "\n",
    "# Move forward\n",
    "\n",
    "forward = [('KeyEvent', 'ArrowUp', True), ('KeyEvent', 'ArrowRight', False),\n",
    "            ('KeyEvent', 'ArrowLeft', False), ('KeyEvent', 'n', True)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Followed by, we will initialize some other variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# We use turn variable for deciding whether to turn or not\n",
    "turn = 0\n",
    "\n",
    "# We store all the rewards in rewards list\n",
    "rewards = []\n",
    "\n",
    "# we will use buffer as some kind of threshold\n",
    "buffer_size = 100\n",
    "\n",
    "# We set our initial action has forward i.e our car moves just forward without making any turns\n",
    "action = forward"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Now, let us begin our game agent to play in an infinite loop which continuously performs an action based on interaction with the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "while True:\n",
    "    turn -= 1\n",
    "    \n",
    "    # Let us say initially we take no turn and move forward.\n",
    "    # First, We will check the value of turn, if it is less than 0\n",
    "    # then there is no necessity for turning and we just move forward\n",
    "    \n",
    "    if turn <= 0:\n",
    "        action = forward\n",
    "        turn = 0\n",
    "    \n",
    "    action_n = [action for ob in observation_n]\n",
    "    \n",
    "    # Then we use env.step() to perform an action (moving forward for now) one-time step\n",
    "    \n",
    "    observation_n, reward_n, done_n, info = env.step(action_n)\n",
    "    \n",
    "    # store the rewards in the rewards list\n",
    "    rewards += [reward_n[0]]\n",
    "     \n",
    "    # We will generate some random number and if it is less than 0.5 then we will take right, else\n",
    "    # we will take left and we will store all the rewards obtained by performing each action and\n",
    "    # based on our rewards we will learn which direction is the best over several timesteps. \n",
    "    \n",
    "    if len(rewards) >= buffer_size:\n",
    "        mean = sum(rewards)/len(rewards)\n",
    "        \n",
    "        if mean == 0:\n",
    "            turn = 20\n",
    "            if random.random() < 0.5:\n",
    "                 action = right\n",
    "            else:\n",
    "                action = left\n",
    "        rewards = []\n",
    "        \n",
    "    env.render()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:universe]",
   "language": "python",
   "name": "conda-env-universe-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}