{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Graceful interrupt in batch jobs using pythons yield\n",
    "\n",
    "Imagine we have a program that runs for longer than we'd ideally like.\n",
    "For example we need to update a progress bar or similar. How would we change the program so that we can perform the interrupt making the code any harder to read?\n",
    "\n",
    "Let's start with the basic program:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.07407407407407407, 0.16666666666666666, 0.2962962962962963, 0.46296296296296297]\n"
     ]
    }
   ],
   "source": [
    "def batchprogram(data):\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item + 1\n",
    "\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item * item\n",
    "\n",
    "    total = sum(data)\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item / total\n",
    "\n",
    "    return data\n",
    "\n",
    "\n",
    "print(batchprogram(data=[1,2,3,4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the computational complexity is O(n), this would potentially run for a long time.\n",
    "To return the control to the main process at a minimum cost of complexity, all we have to add is a modest yield:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def batchprogram2(data):\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item + 1\n",
    "        yield\n",
    "\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item * item\n",
    "        yield\n",
    "\n",
    "    total = sum(data)\n",
    "    yield\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item / total\n",
    "        yield\n",
    "\n",
    "    yield data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The change to the code is acceptable: Insert a yield and you're almost done.\n",
    "For the controller to display progress we can use dots:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".............."
     ]
    }
   ],
   "source": [
    "for ix, step in enumerate(batchprogram2(data=[1,2,3,4])):\n",
    "    print(\"\", end=\".\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If I wanted more explicit information about progress, I can add it to the yield:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step 1 - 0/4 complete\n",
      "step 1 - 1/4 complete\n",
      "step 1 - 2/4 complete\n",
      "step 1 - 3/4 complete\n",
      "step 2 - 0/4 complete\n",
      "step 2 - 1/4 complete\n",
      "step 2 - 2/4 complete\n",
      "step 2 - 3/4 complete\n",
      "step 3 - started\n",
      "step 3 - done\n",
      "step 4 - 0/4 complete\n",
      "step 4 - 1/4 complete\n",
      "step 4 - 2/4 complete\n",
      "step 4 - 3/4 complete\n",
      "result:\n",
      "[0.07407407407407407, 0.16666666666666666, 0.2962962962962963, 0.46296296296296297]\n"
     ]
    }
   ],
   "source": [
    "def batchprogram3(data):\n",
    "    n = len(data)\n",
    "\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item + 1\n",
    "        yield f\"step 1 - {ix}/{n} complete\"\n",
    "\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item * item\n",
    "        yield f\"step 2 - {ix}/{n} complete\"\n",
    "\n",
    "    yield f\"step 3 - started\"\n",
    "    total = sum(data)\n",
    "    yield f\"step 3 - done\"\n",
    "\n",
    "    for ix, item in enumerate(data):\n",
    "        data[ix] = item / total\n",
    "        yield f\"step 4 - {ix}/{n} complete\"\n",
    "\n",
    "    yield data\n",
    "\n",
    "\n",
    "for step in batchprogram3(data=[1,2,3,4]):\n",
    "    if isinstance(step, str):\n",
    "        print(step)\n",
    "    else:\n",
    "        data = step\n",
    "\n",
    "print(\"result:\")\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Almost a classic."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "c4df837ac4477c7435bbd736fb9ba2c8df232961013baf2c3c9d338fc856bbbf"
  },
  "kernelspec": {
   "display_name": "Python 3.6.7 64-bit ('python36': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}