{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text Classification\n",
    "\n",
    "**BentoML makes moving trained ML models to production easy:**\n",
    "\n",
    "* Package models trained with **any ML framework** and reproduce them for model serving in production\n",
    "* **Deploy anywhere** for online API serving or offline batch serving\n",
    "* High-Performance API model server with *adaptive micro-batching* support\n",
    "* Central hub for managing models and deployment process via Web UI and APIs\n",
    "* Modular and flexible design making it *adaptable to your infrastrcuture*\n",
    "\n",
    "BentoML is a framework for serving, managing, and deploying machine learning models. It is aiming to bridge the gap between Data Science and DevOps, and enable teams to deliver prediction services in a fast, repeatable, and scalable way.\n",
    "\n",
    "Before reading this example project, be sure to check out the [Getting started guide](https://github.com/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb) to learn about the basic concepts in BentoML.\n",
    "\n",
    "\n",
    "FastText is a library for efficient learning of word representations and sentence classification.\n",
    "\n",
    "The goal of text classification is to assign documents (such as emails, posts, text messages, product reviews, etc...) to one or multiple categories. Such categories can be review scores, spam v.s. non-spam, or the language in which the document was typed. Nowadays, the dominant approach to build such classifiers is machine learning, that is learning classification rules from examples. In order to build such classifiers, we need labeled data, which consists of documents and their corresponding categories (or tags, or labels).\n",
    "\n",
    "As an example, it builds a classifier which automatically classifies stackexchange questions about cooking into one of several possible tags, such as `pot`, `bowl` or `baking`.\n",
    "\n",
    "This example notebook is base on the guide from fasttext: https://fasttext.cc/docs/en/supervised-tutorial.html\n",
    "\n",
    "![Impression](https://www.google-analytics.com/collect?v=1&tid=UA-112879361-3&cid=555&t=event&ec=fasttext&ea=fasttext-text-classification&dt=fasttext-text-classification)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: fasttext in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (0.9.2)\n",
      "Requirement already satisfied: numpy in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from fasttext) (1.16.4)\n",
      "Requirement already satisfied: pybind11>=2.2 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from fasttext) (2.5.0)\n",
      "Requirement already satisfied: setuptools>=0.7.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from fasttext) (45.1.0.post20200119)\n"
     ]
    }
   ],
   "source": [
    "!pip install -q bentoml \"fasttext==0.9.2\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100  446k  100  446k    0     0   466k      0 --:--:-- --:--:-- --:--:--  465k\n",
      "x cooking.stackexchange.id\n",
      "x cooking.stackexchange.txt\n",
      "x readme.txt\n"
     ]
    }
   ],
   "source": [
    "!curl https://dl.fbaipublicfiles.com/fasttext/data/cooking.stackexchange.tar.gz --output cooking.stackexchange.tar.gz && tar xvzf cooking.stackexchange.tar.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__label__sauce __label__cheese How much does potato starch affect a cheese sauce recipe?\r\n",
      "__label__food-safety __label__acidity Dangerous pathogens capable of growing in acidic environments\r\n",
      "__label__cast-iron __label__stove How do I cover up the white spots on my cast iron stove?\r\n",
      "__label__restaurant Michelin Three Star Restaurant; but if the chef is not there\r\n",
      "__label__knife-skills __label__dicing Without knife skills, how can I quickly and accurately dice vegetables?\r\n",
      "__label__storage-method __label__equipment __label__bread What's the purpose of a bread box?\r\n",
      "__label__baking __label__food-safety __label__substitutions __label__peanuts how to seperate peanut oil from roasted peanuts at home?\r\n",
      "__label__chocolate American equivalent for British chocolate terms\r\n",
      "__label__baking __label__oven __label__convection Fan bake vs bake\r\n",
      "__label__sauce __label__storage-lifetime __label__acidity __label__mayonnaise Regulation and balancing of readymade packed mayonnaise and other sauces\r\n"
     ]
    }
   ],
   "source": [
    "!head cooking.stackexchange.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "!head -n 12404 cooking.stackexchange.txt > cooking.train\n",
    "!tail -n 3000 cooking.stackexchange.txt > cooking.valid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fasttext"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = fasttext.train_supervised(input=\"cooking.train\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define and save BentoService"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing text_classification.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile text_classification.py\n",
    "\n",
    "from bentoml import env, artifacts, BentoService, api\n",
    "from bentoml.frameworks.fasttext import FasttextModelArtifact\n",
    "from bentoml.adapters import JsonInput\n",
    "\n",
    "@env(infer_pip_packages=True)\n",
    "@artifacts([FasttextModelArtifact('model')])\n",
    "class FasttextClassification(BentoService):\n",
    "    \n",
    "    @api(input=JsonInput(), batch=True)\n",
    "    def predict(self, json_list):\n",
    "        input = [i['text'] for i in json_list]\n",
    "        result = self.artifacts.model.predict(input)\n",
    "        # return top result\n",
    "        prediction_result = [i[0].replace('__label__', '') for i in result[0]]\n",
    "        return prediction_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:09:52,940] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n",
      "[2020-09-22 12:09:53,839] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.\n",
      "[2020-09-22 12:09:55,087] INFO - Detected non-PyPI-released BentoML installed, copying local BentoML modulefiles to target saved bundle path..\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "warning: no previously-included files matching '*~' found anywhere in distribution\n",
      "warning: no previously-included files matching '*.pyo' found anywhere in distribution\n",
      "warning: no previously-included files matching '.git' found anywhere in distribution\n",
      "warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution\n",
      "warning: no previously-included files matching '__pycache__' found anywhere in distribution\n",
      "no previously-included directories found matching 'e2e_tests'\n",
      "no previously-included directories found matching 'tests'\n",
      "no previously-included directories found matching 'benchmark'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UPDATING BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py\n",
      "set BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py to '0.9.0.pre+3.gcebf2015'\n",
      "[2020-09-22 12:10:00,091] INFO - BentoService bundle 'FasttextClassification:20200922120954_28C4D0' saved to: /Users/bozhaoyu/bentoml/repository/FasttextClassification/20200922120954_28C4D0\n"
     ]
    }
   ],
   "source": [
    "from text_classification import FasttextClassification\n",
    "\n",
    "svc = FasttextClassification()\n",
    "svc.pack('model', model)\n",
    "\n",
    "saved_path = svc.save()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## REST API Model Serving\n",
    "\n",
    "\n",
    "To start a REST API model server with the BentoService saved above, use the bentoml serve command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:11:58,505] INFO - Starting BentoML API server in development mode..\n",
      "[2020-09-22 12:11:58,967] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n",
      "[2020-09-22 12:11:58,981] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 12:11:59,380] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      " * Serving Flask app \"FasttextClassification\" (lazy loading)\n",
      " * Environment: production\n",
      "\u001b[31m   WARNING: This is a development server. Do not use it in a production deployment.\u001b[0m\n",
      "\u001b[2m   Use a production WSGI server instead.\u001b[0m\n",
      " * Debug mode: off\n",
      " * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n",
      "[2020-09-22 12:12:17,344] INFO - {'service_name': 'FasttextClassification', 'service_version': '20200922120954_28C4D0', 'api': 'predict', 'task': {'data': '{\"text\": \"Which baking dish is best to bake a banana bread ?\"}', 'task_id': '81e679d6-5cab-48c5-9ce2-b486cbf7c6c7', 'http_headers': (('Host', 'localhost:5000'), ('User-Agent', 'curl/7.65.3'), ('Accept', '*/*'), ('Content-Type', 'application/json'), ('Content-Length', '62'))}, 'result': {'data': '\"baking\"', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '81e679d6-5cab-48c5-9ce2-b486cbf7c6c7'}\n",
      "127.0.0.1 - - [22/Sep/2020 12:12:17] \"\u001b[37mPOST /predict HTTP/1.1\u001b[0m\" 200 -\n",
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "I0922 12:12:17.346314 4676050368 _internal.py:122] 127.0.0.1 - - [22/Sep/2020 12:12:17] \"\u001b[37mPOST /predict HTTP/1.1\u001b[0m\" 200 -\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "!bentoml serve {saved_path}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running this notebook from Google Colab, you can start the dev server with `--run-with-ngrok` option, to gain acccess to the API endpoint via a public endpoint managed by [ngrok](https://ngrok.com/):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:11:20,296] INFO - Getting latest version FasttextClassification:20200922120954_28C4D0\n",
      "[2020-09-22 12:11:20,297] INFO - Starting BentoML API server in development mode..\n",
      "[2020-09-22 12:11:20,550] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n",
      "[2020-09-22 12:11:20,564] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 12:11:20,902] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      " * Serving Flask app \"FasttextClassification\" (lazy loading)\n",
      " * Environment: production\n",
      "\u001b[31m   WARNING: This is a development server. Do not use it in a production deployment.\u001b[0m\n",
      "\u001b[2m   Use a production WSGI server instead.\u001b[0m\n",
      " * Debug mode: off\n",
      " * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n",
      "Exception in thread Thread-1:\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/local/anaconda3/envs/dev-py3/lib/python3.7/threading.py\", line 917, in _bootstrap_inner\n",
      "    self.run()\n",
      "  File \"/usr/local/anaconda3/envs/dev-py3/lib/python3.7/threading.py\", line 1166, in run\n",
      "    self.function(*self.args, **self.kwargs)\n",
      "  File \"/Users/bozhaoyu/src/bento/bentoml/utils/flask_ngrok.py\", line 90, in start_ngrok\n",
      "    ngrok_address = _run_ngrok(port)\n",
      "  File \"/Users/bozhaoyu/src/bento/bentoml/utils/flask_ngrok.py\", line 39, in _run_ngrok\n",
      "    os.chmod(executable, 0o777)\n",
      "FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/ngrok/ngrok'\n",
      "\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:29] \"\u001b[37mGET / HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /swagger_static/swagger-ui.css HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /swagger_static/swagger-ui-bundle.js HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /docs.json HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[33mGET /favicon.ico HTTP/1.1\u001b[0m\" 404 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[32mGET //favicon.ico HTTP/1.1\u001b[0m\" 308 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET / HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /swagger_static/swagger-ui.css HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /swagger_static/swagger-ui-bundle.js HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:30] \"\u001b[37mGET /docs.json HTTP/1.1\u001b[0m\" 200 -\n",
      "127.0.0.1 - - [22/Sep/2020 12:11:31] \"\u001b[33mGET /favicon.ico HTTP/1.1\u001b[0m\" 404 -\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "!bentoml serve FasttextClassification:latest --run-with-ngrok"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open http://127.0.0.1:5000 to see more information about the REST APIs server in your\n",
    "browser.\n",
    "\n",
    "\n",
    "### Send prediction requeset to the REST API server\n",
    "\n",
    "Navigate to parent directory of the notebook(so you have reference to the `test.jpg` image), and run the following `curl` command to send the image to REST API server and get a prediction result:\n",
    "\n",
    "```\n",
    "curl -X POST \\\n",
    "  http://localhost:5000/predict \\\n",
    "  -H 'Content-Type: application/json' \\\n",
    "  -d '{\"text\": \"Which baking dish is best to bake a banana bread ?\"}'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Containerize model server with Docker\n",
    "\n",
    "\n",
    "One common way of distributing this model API server for production deployment, is via Docker containers. And BentoML provides a convenient way to do that.\n",
    "\n",
    "Note that docker is **not available in Google Colab**. You will need to download and run this notebook locally to try out this containerization with docker feature.\n",
    "\n",
    "If you already have docker configured, simply run the follow command to product a docker container serving the IrisClassifier prediction service created above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:13:07,456] INFO - Getting latest version FasttextClassification:20200922120954_28C4D0\n",
      "\u001b[39mFound Bento: /Users/bozhaoyu/bentoml/repository/FasttextClassification/20200922120954_28C4D0\u001b[0m\n",
      "[2020-09-22 12:13:07,495] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n",
      "[2020-09-22 12:13:07,511] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "\u001b[39mTag not specified, using tag parsed from BentoService: 'fasttextclassification:20200922120954_28C4D0'\u001b[0m\n",
      "Building Docker image fasttextclassification:20200922120954_28C4D0 from FasttextClassification:latest \n",
      "-we in here\n",
      "processed docker file\n",
      "(None, None)\n",
      "root in create archive /Users/bozhaoyu/bentoml/repository/FasttextClassification/20200922120954_28C4D0 ['Dockerfile', 'FasttextClassification', 'FasttextClassification/__init__.py', 'FasttextClassification/__pycache__', 'FasttextClassification/__pycache__/text_classification.cpython-37.pyc', 'FasttextClassification/artifacts', 'FasttextClassification/artifacts/__init__.py', 'FasttextClassification/artifacts/model', 'FasttextClassification/bentoml.yml', 'FasttextClassification/text_classification.py', 'MANIFEST.in', 'README.md', 'bentoml-init.sh', 'bentoml.yml', 'bundled_pip_dependencies', 'bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz', 'docker-entrypoint.sh', 'environment.yml', 'python_version', 'requirements.txt', 'setup.py']\n",
      "about to build\n",
      "about to upgrade params\n",
      "check each param and update\n",
      "if use config proxy\n",
      "if buildargs\n",
      "if shmsize\n",
      "if labels\n",
      "if cache from\n",
      "if target\n",
      "if network_mode\n",
      "if squash\n",
      "if extra hosts is not None\n",
      "if platform is not None\n",
      "if isolcation is not None\n",
      "if context is not None\n",
      "setting auth {'Content-Type': 'application/tar'}\n",
      "\b\\docker build <tempfile._TemporaryFileWrapper object at 0x7ffe4d5b5d68> {'t': 'fasttextclassification:20200922120954_28C4D0', 'remote': None, 'q': False, 'nocache': False, 'rm': False, 'forcerm': False, 'pull': False, 'dockerfile': (None, None)}\n",
      "\b|docker response <Response [200]>\n",
      "context closes\n",
      "print responses\n",
      "\u001b[39mStep 1/15 : FROM bentoml/model-server:0.9.0.pre\u001b[0m\n",
      "\u001b[39m ---> a25066aa8b0e\u001b[0m\n",
      "\u001b[39mStep 2/15 : ARG EXTRA_PIP_INSTALL_ARGS=\u001b[0m\n",
      "\b\\\u001b[39m ---> Running in e09cbc5bfce1\u001b[0m\n",
      "\b-\u001b[39m ---> fc6e47d06522\u001b[0m\n",
      "\u001b[39mStep 3/15 : ENV EXTRA_PIP_INSTALL_ARGS $EXTRA_PIP_INSTALL_ARGS\u001b[0m\n",
      "\b/\u001b[39m ---> Running in 232b8623b0b9\u001b[0m\n",
      "\b|\u001b[39m ---> db8172e98571\u001b[0m\n",
      "\u001b[39mStep 4/15 : COPY environment.yml requirements.txt setup.sh* bentoml-init.sh python_version* /bento/\u001b[0m\n",
      "\b/\u001b[39m ---> c41f172d737b\u001b[0m\n",
      "\u001b[39mStep 5/15 : WORKDIR /bento\u001b[0m\n",
      "\b|\u001b[39m ---> Running in 1e1c28b0fd29\u001b[0m\n",
      "\b\\\u001b[39m ---> 61863d5599df\u001b[0m\n",
      "\u001b[39mStep 6/15 : RUN chmod +x /bento/bentoml-init.sh\u001b[0m\n",
      "\u001b[39m ---> Running in 0bf14a8a8cdf\u001b[0m\n",
      "\b/\u001b[39m ---> 3bba73c15b1a\u001b[0m\n",
      "\u001b[39mStep 7/15 : RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi\u001b[0m\n",
      "\u001b[39m ---> Running in cfa5c292d12e\u001b[0m\n",
      "\b-\u001b[39m\u001b[91m+++ dirname /bento/bentoml-init.sh\n",
      "\u001b[0m\u001b[0m\n",
      "\b/\u001b[39m\u001b[91m++ cd /bento\n",
      "++ pwd -P\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m+ SAVED_BUNDLE_PATH=/bento\n",
      "+ cd /bento\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m+ '[' -f ./setup.sh ']'\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m+ '[' -f ./python_version ']'\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m++ cat ./python_version\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m+ PY_VERSION_SAVED=3.7.3\n",
      "+ DESIRED_PY_VERSION=3.7\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m++ python -c 'import sys; print(f\"{sys.version_info.major}.{sys.version_info.minor}\")'\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39m\u001b[91m+ CURRENT_PY_VERSION=3.7\n",
      "+ [[ 3.7 == \\3\\.\\7 ]]\n",
      "+ echo 'Python Version in docker base image 3.7 matches requirement python=3.7. Skipping.'\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39mPython Version in docker base image 3.7 matches requirement python=3.7. Skipping.\u001b[0m\n",
      "\u001b[39m\u001b[91m+ command -v conda\n",
      "+ echo 'Updating conda base environment with environment.yml'\n",
      "+ conda env update -n base -f ./environment.yml\n",
      "\u001b[0m\u001b[0m\n",
      "\u001b[39mUpdating conda base environment with environment.yml\u001b[0m\n",
      "\b\\\u001b[39mCollecting package metadata (repodata.json): ...working... \u001b[0m\n",
      "\b-\u001b[39mdone\n",
      "Solving environment: ...working... \u001b[0m\n",
      "\b-\u001b[39mdone\u001b[0m\n",
      "\b/\u001b[39m\n",
      "Downloading and Extracting Packages\n",
      "pip-20.2.3           | 1.1 MB    |            |   0% \u001b[0m\n",
      "pip-20.2.3           | 1.1 MB    | 1          |   1% \u001b[0m\n",
      "pip-20.2.3           | 1.1 MB    | #####1     |  51% \u001b[0m\n",
      "pip-20.2.3           | 1.1 MB    | ########## | 100% \u001b[0m\n",
      "pip-20.2.3           | 1.1 MB    | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "python_abi-3.7       | 4 KB      |            |   0% \u001b[0m\n",
      "python_abi-3.7       | 4 KB      | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "python-3.7.9         | 45.3 MB   |            |   0% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   |            |   0% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | 1          |   2% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | 4          |   4% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | 6          |   7% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | 9          |   9% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #2         |  12% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #5         |  15% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #8         |  19% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ##2        |  22% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ##5        |  26% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ##9        |  29% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ###2       |  33% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ###5       |  36% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ###9       |  39% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ####2      |  43% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ####6      |  47% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #####      |  50% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #####4     |  54% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #####8     |  59% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ######2    |  63% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ######6    |  66% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #######    |  70% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #######3   |  74% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #######7   |  78% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ########1  |  81% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ########4  |  84% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ########8  |  88% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #########2 |  92% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | #########6 |  96% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ########## | 100% \u001b[0m\n",
      "python-3.7.9         | 45.3 MB   | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "readline-8.0         | 281 KB    |            |   0% \u001b[0m\n",
      "readline-8.0         | 281 KB    | ########## | 100% \u001b[0m\n",
      "readline-8.0         | 281 KB    | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "ca-certificates-2020 | 145 KB    |            |   0% \u001b[0m\n",
      "ca-certificates-2020 | 145 KB    | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "openssl-1.1.1h       | 2.1 MB    |            |   0% \u001b[0m\n",
      "openssl-1.1.1h       | 2.1 MB    | ###2       |  32% \u001b[0m\n",
      "openssl-1.1.1h       | 2.1 MB    | ########## | 100% \u001b[0m\n",
      "openssl-1.1.1h       | 2.1 MB    | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "certifi-2020.6.20    | 151 KB    |            |   0% \u001b[0m\n",
      "certifi-2020.6.20    | 151 KB    | ########## | 100% \u001b[0m\n",
      "\u001b[39m\n",
      "Preparing transaction: \u001b[0m\n",
      "\u001b[39m...working... \u001b[0m\n",
      "\b\\\u001b[39mdone\u001b[0m\n",
      "\u001b[39mVerifying transaction: ...working... \u001b[0m\n",
      "\b|\u001b[39mdone\u001b[0m\n",
      "\u001b[39mExecuting transaction: ...working... \u001b[0m\n",
      "\b\\\u001b[39mdone\u001b[0m\n",
      "\b/\u001b[39m#\n",
      "# To activate this environment, use\n",
      "#\n",
      "#     $ conda activate base\n",
      "#\n",
      "# To deactivate an active environment, use\n",
      "#\n",
      "#     $ conda deactivate\u001b[0m\n",
      "\b-\u001b[39m\u001b[91m+ pip install -r ./requirements.txt --no-cache-dir\n",
      "\u001b[0m\u001b[0m\n",
      "\b|\u001b[39mRequirement already satisfied: bentoml==0.9.0.pre in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 1)) (0.9.0rc0)\u001b[0m\n",
      "\b/\u001b[39mCollecting fasttext==0.9.2\u001b[0m\n",
      "\b|\u001b[39m  Downloading fasttext-0.9.2.tar.gz (68 kB)\u001b[0m\n",
      "\b-\u001b[39mRequirement already satisfied: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.8.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.27.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.1.11)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.6.2)\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[39mRequirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.24.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: multidict in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.7.6)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2020.6.20)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.19)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.4.3)\u001b[0m\n",
      "\b/\u001b[39mRequirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.3.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.15.87)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.36.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.0.4)\u001b[0m\n",
      "\b|\u001b[39mRequirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (7.1.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.0.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.19.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (8.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.13.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.20.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.7.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: pybind11>=2.2 in /opt/conda/lib/python3.7/site-packages (from fasttext==0.9.2->-r ./requirements.txt (line 2)) (2.5.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: setuptools>=0.7.0 in /opt/conda/lib/python3.7/site-packages (from fasttext==0.9.2->-r ./requirements.txt (line 2)) (49.6.0.post20200814)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: six>=1.5 in /opt/conda/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.7.3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.2.0)\u001b[0m\n",
      "\b\\\u001b[39mRequirement already satisfied: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.5.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.10)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.25.10)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.4.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.11.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.57.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.3.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.18.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.10.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.4.11)\u001b[0m\n",
      "\b-\u001b[39mRequirement already satisfied: typing-extensions>=3.7.4; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.7.4.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.11)\u001b[0m\n",
      "\u001b[39mBuilding wheels for collected packages: fasttext\u001b[0m\n",
      "\u001b[39m  Building wheel for fasttext (setup.py): started\u001b[0m\n",
      "\b-\u001b[39m  Building wheel for fasttext (setup.py): finished with status 'done'\u001b[0m\n",
      "\u001b[39m  Created wheel for fasttext: filename=fasttext-0.9.2-cp37-cp37m-linux_x86_64.whl size=4441002 sha256=c9a69b2133d5b1c05b68f9be59d9e74583689c299224445f20de0dcfc227432a\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-sx8v7vyg/wheels/4e/ca/bf/b020d2be95f7641801a6597a29c8f4f19e38f9c02a345bab9b\u001b[0m\n",
      "\u001b[39mSuccessfully built fasttext\u001b[0m\n",
      "\b/\u001b[39mInstalling collected packages: fasttext\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\\\u001b[39mSuccessfully installed fasttext-0.9.2\u001b[0m\n",
      "\b\\\u001b[39m ---> a90f620bbe4d\u001b[0m\n",
      "\u001b[39mStep 8/15 : COPY . /bento\u001b[0m\n",
      "\b\\\u001b[39m ---> 5d2abfb72a02\u001b[0m\n",
      "\u001b[39mStep 9/15 : RUN if [ -d /bento/bundled_pip_dependencies ]; then pip install -U bundled_pip_dependencies/* ;fi\u001b[0m\n",
      "\u001b[39m ---> Running in 1f2cf07c7029\u001b[0m\n",
      "\b|\u001b[39mProcessing ./bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz\u001b[0m\n",
      "\b|\u001b[39m  Installing build dependencies: started\u001b[0m\n",
      "\b\\\u001b[39m  Installing build dependencies: finished with status 'done'\u001b[0m\n",
      "\u001b[39m  Getting requirements to build wheel: started\u001b[0m\n",
      "\b-\u001b[39m  Getting requirements to build wheel: finished with status 'done'\u001b[0m\n",
      "\u001b[39m    Preparing wheel metadata: started\u001b[0m\n",
      "\b|\u001b[39m    Preparing wheel metadata: finished with status 'done'\u001b[0m\n",
      "\b\\\u001b[39mRequirement already satisfied, skipping upgrade: numpy in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.19.2)\u001b[0m\n",
      "\b-\u001b[39mRequirement already satisfied, skipping upgrade: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.36.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: alembic in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.4.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: gunicorn in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.0.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: aiohttp in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.6.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: psutil in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.7.2)\u001b[0m\n",
      "\b/\u001b[39mRequirement already satisfied, skipping upgrade: prometheus-client in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (7.1.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: tabulate in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.19)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.8.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: configparser in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.0.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: python-json-logger in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.1.11)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: boto3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.15.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: flask in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.1.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: multidict in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.7.6)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.15.87)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.27.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: certifi in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2020.6.20)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: py-zipkin in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.20.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: docker in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.3.1)\u001b[0m\n",
      "\b|\u001b[39mRequirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.24.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: humanfriendly in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (8.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: cerberus in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.13.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: six in /opt/conda/lib/python3.7/site-packages (from sqlalchemy-utils<0.36.8->BentoML==0.9.0rc0+3.gcebf2015) (1.15.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.1.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.0.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: setuptools>=3.0 in /opt/conda/lib/python3.7/site-packages (from gunicorn->BentoML==0.9.0rc0+3.gcebf2015) (49.6.0.post20200814)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.0.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (1.5.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (20.2.0)\u001b[0m\n",
      "\b\\\u001b[39mRequirement already satisfied, skipping upgrade: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.0.4)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->BentoML==0.9.0rc0+3.gcebf2015) (2.4.7)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.10.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.3.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (1.18.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (2.11.2)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.0.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.1.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (0.4.11)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->BentoML==0.9.0rc0+3.gcebf2015) (0.57.0)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (2.10)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (1.25.10)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.1.1)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: typing-extensions>=3.7.4; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.7.4.3)\u001b[0m\n",
      "\u001b[39mRequirement already satisfied, skipping upgrade: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (3.11)\u001b[0m\n",
      "\u001b[39mBuilding wheels for collected packages: BentoML\u001b[0m\n",
      "\u001b[39m  Building wheel for BentoML (PEP 517): started\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b/\u001b[39m  Building wheel for BentoML (PEP 517): finished with status 'done'\u001b[0m\n",
      "\u001b[39m  Created wheel for BentoML: filename=BentoML-0.9.0rc0+3.gcebf2015-py3-none-any.whl size=3064091 sha256=0bd361a45c83e2bb80aa10347199c1fd71684e3a1389eec1208f54b4297c1140\n",
      "  Stored in directory: /root/.cache/pip/wheels/a0/45/41/62152db705af4ff47e7a3d6abf6247986eef4aa1b94a58d3b9\u001b[0m\n",
      "\u001b[39mSuccessfully built BentoML\u001b[0m\n",
      "\b/\u001b[39mInstalling collected packages: BentoML\n",
      "  Attempting uninstall: BentoML\u001b[0m\n",
      "\u001b[39m    Found existing installation: BentoML 0.9.0rc0\u001b[0m\n",
      "\b\\\u001b[39m    Uninstalling BentoML-0.9.0rc0:\u001b[0m\n",
      "\b/\u001b[39m      Successfully uninstalled BentoML-0.9.0rc0\u001b[0m\n",
      "\b\\\u001b[39mSuccessfully installed BentoML-0.9.0rc0+3.gcebf2015\u001b[0m\n",
      "\b/\u001b[39m ---> 860b9bec3b35\u001b[0m\n",
      "\u001b[39mStep 10/15 : ENV PORT 5000\u001b[0m\n",
      "\b|\u001b[39m ---> Running in c2743985d5de\u001b[0m\n",
      "\b\\\u001b[39m ---> c3f39fdc8450\u001b[0m\n",
      "\u001b[39mStep 11/15 : EXPOSE $PORT\u001b[0m\n",
      "\u001b[39m ---> Running in 0733c7dcfea4\u001b[0m\n",
      "\b-\u001b[39m ---> bd7997accbc4\u001b[0m\n",
      "\u001b[39mStep 12/15 : COPY docker-entrypoint.sh /usr/local/bin/\u001b[0m\n",
      "\b|\u001b[39m ---> 71c57d9f3c42\u001b[0m\n",
      "\u001b[39mStep 13/15 : RUN chmod +x /usr/local/bin/docker-entrypoint.sh\u001b[0m\n",
      "\b\\\u001b[39m ---> Running in 8e7c893f2534\u001b[0m\n",
      "\b/\u001b[39m ---> e1c5868e8975\u001b[0m\n",
      "\u001b[39mStep 14/15 : ENTRYPOINT [ \"docker-entrypoint.sh\" ]\u001b[0m\n",
      "\u001b[39m ---> Running in 033db21d262e\u001b[0m\n",
      "\b|\u001b[39m ---> 24858e5a145c\u001b[0m\n",
      "\u001b[39mStep 15/15 : CMD [\"bentoml\", \"serve-gunicorn\", \"/bento\"]\u001b[0m\n",
      "\u001b[39m ---> Running in b1700521fdb7\u001b[0m\n",
      "\b\\\u001b[39m ---> e35ff263b52f\u001b[0m\n",
      "\u001b[39mSuccessfully built e35ff263b52f\u001b[0m\n",
      "\b-\u001b[39mSuccessfully tagged fasttextclassification:20200922120954_28C4D0\u001b[0m\n",
      "\u001b[32mFinished building fasttextclassification:20200922120954_28C4D0 from FasttextClassification:latest\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!bentoml containerize FasttextClassification:latest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 19:17:58,674] INFO - Starting BentoML API server in production mode..\n",
      "[2020-09-22 19:17:59,100] INFO - get_gunicorn_num_of_workers: 3, calculated by cpu count\n",
      "[2020-09-22 19:17:59 +0000] [1] [INFO] Starting gunicorn 20.0.4\n",
      "[2020-09-22 19:17:59 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)\n",
      "[2020-09-22 19:17:59 +0000] [1] [INFO] Using worker: sync\n",
      "[2020-09-22 19:17:59 +0000] [11] [INFO] Booting worker with pid: 11\n",
      "[2020-09-22 19:17:59 +0000] [12] [INFO] Booting worker with pid: 12\n",
      "[2020-09-22 19:17:59 +0000] [13] [INFO] Booting worker with pid: 13\n",
      "[2020-09-22 19:17:59,365] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/bentoml.git@{branch}'\n",
      "[2020-09-22 19:17:59,387] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 19:17:59,387] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.9.\n",
      "[2020-09-22 19:17:59,394] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/bentoml.git@{branch}'\n",
      "[2020-09-22 19:17:59,420] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 19:17:59,422] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.9.\n",
      "[2020-09-22 19:17:59,500] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/bentoml.git@{branch}'\n",
      "[2020-09-22 19:17:59,528] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 19:17:59,529] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.9.\n",
      "[2020-09-22 19:18:08,246] INFO - {'service_name': 'FasttextClassification', 'service_version': '20200922120954_28C4D0', 'api': 'predict', 'task': {'data': '{\"text\": \"Which baking dish is best to bake a banana bread ?\"}', 'task_id': '307d893e-df06-44c9-bba4-1098a6c34cea', 'http_headers': (('Host', 'localhost:5000'), ('User-Agent', 'curl/7.65.3'), ('Accept', '*/*'), ('Content-Type', 'application/json'), ('Content-Length', '62'))}, 'result': {'data': '\"baking\"', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '307d893e-df06-44c9-bba4-1098a6c34cea'}\n",
      "^C\n",
      "[2020-09-22 19:18:15 +0000] [1] [INFO] Handling signal: int\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      "[2020-09-22 19:18:15 +0000] [11] [INFO] Worker exiting (pid: 11)\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      "[2020-09-22 19:18:15 +0000] [13] [INFO] Worker exiting (pid: 13)\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      "[2020-09-22 19:18:15 +0000] [12] [INFO] Worker exiting (pid: 12)\n"
     ]
    }
   ],
   "source": [
    "!docker run --rm -p5000:5000 fasttextclassification:20200922120954_28C4D0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load saved BentoService\n",
    "\n",
    "bentoml.load is the API for loading a BentoML packaged model in python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:18:22,256] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 12:18:22,257] WARNING - Module `text_classification` already loaded, using existing imported module.\n",
      "[2020-09-22 12:18:22,278] WARNING - pip package requirement fasttext already exist\n",
      "['baking']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n"
     ]
    }
   ],
   "source": [
    "from bentoml import load\n",
    "\n",
    "svc = load(saved_path)\n",
    "\n",
    "print(svc.predict([{\"text\": \"which baking dish is the best?\"}]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Launch inference job from CLI\n",
    "\n",
    "BentoML cli supports loading and running a packaged model from CLI. With the DataframeInput adapter, the CLI command supports reading input Dataframe data from CLI argument or local csv or json files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2020-09-22 12:18:42,572] INFO - Getting latest version FasttextClassification:20200922120954_28C4D0\n",
      "[2020-09-22 12:18:42,609] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n",
      "[2020-09-22 12:18:42,623] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015\n",
      "[2020-09-22 12:18:43,064] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.\n",
      "Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.\n",
      "[2020-09-22 12:18:47,361] INFO - {'service_name': 'FasttextClassification', 'service_version': '20200922120954_28C4D0', 'api': 'predict', 'task': {'data': '{\"text\": \"Which baking dish is best to bake a banana bread ?\"}', 'task_id': '54edf965-1d7d-4eb9-929c-c7d0d0e21732', 'cli_args': ('--input {\"text\": \"Which baking dish is best to bake a banana bread ?\"}',)}, 'result': {'data': '\"baking\"', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '54edf965-1d7d-4eb9-929c-c7d0d0e21732'}\n",
      "\"baking\"\n"
     ]
    }
   ],
   "source": [
    "!bentoml run FasttextClassification:latest predict --input '{\"text\": \"Which baking dish is best to bake a banana bread ?\"}'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deployment Options\n",
    "\n",
    "If you are at a small team with limited engineering or DevOps resources, try out automated deployment with BentoML CLI, currently supporting AWS Lambda, AWS SageMaker, and Azure Functions:\n",
    "- [AWS Lambda Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_lambda.html)\n",
    "- [AWS SageMaker Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_sagemaker.html)\n",
    "- [Azure Functions Deployment Guide](https://docs.bentoml.org/en/latest/deployment/azure_functions.html)\n",
    "\n",
    "If the cloud platform you are working with is not on the list above, try out these step-by-step guide on manually deploying BentoML packaged model to cloud platforms:\n",
    "- [AWS ECS Deployment](https://docs.bentoml.org/en/latest/deployment/aws_ecs.html)\n",
    "- [Google Cloud Run Deployment](https://docs.bentoml.org/en/latest/deployment/google_cloud_run.html)\n",
    "- [Azure container instance Deployment](https://docs.bentoml.org/en/latest/deployment/azure_container_instance.html)\n",
    "- [Heroku Deployment](https://docs.bentoml.org/en/latest/deployment/heroku.html)\n",
    "\n",
    "Lastly, if you have a DevOps or ML Engineering team who's operating a Kubernetes or OpenShift cluster, use the following guides as references for implementating your deployment strategy:\n",
    "- [Kubernetes Deployment](https://docs.bentoml.org/en/latest/deployment/kubernetes.html)\n",
    "- [Knative Deployment](https://docs.bentoml.org/en/latest/deployment/knative.html)\n",
    "- [Kubeflow Deployment](https://docs.bentoml.org/en/latest/deployment/kubeflow.html)\n",
    "- [KFServing Deployment](https://docs.bentoml.org/en/latest/deployment/kfserving.html)\n",
    "- [Clipper.ai Deployment Guide](https://docs.bentoml.org/en/latest/deployment/clipper.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}