{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### BentoML Example : Multiple Models with JsonInput\n", "# Titanic Survival Prediction with Xgboost and Lightgbm\n", "\n", "**BentoML makes moving trained ML models to production easy:**\n", "\n", "* Package models trained with **any ML framework** and reproduce them for model serving in production\n", "* **Deploy anywhere** for online API serving or offline batch serving\n", "* High-Performance API model server with *adaptive micro-batching* support\n", "* Central hub for managing models and deployment process via Web UI and APIs\n", "* Modular and flexible design making it *adaptable to your infrastrcuture*\n", "\n", "BentoML is a framework for serving, managing, and deploying machine learning models. It is aiming to bridge the gap between Data Science and DevOps, and enable teams to deliver prediction services in a fast, repeatable, and scalable way.\n", "\n", "Before reading this example project, be sure to check out the [Getting started guide](https://github.com/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb) to learn about the basic concepts in BentoML.\n", "\n", "This is a BentoML Demo Project demonstrating how to package and serve LightBGM model for production using BentoML.\n", "\n", "[BentoML](http://bentoml.ai) is an open source platform for machine learning model serving and deployment.\n", "\n", "In this example, we will use scikit-learn API for both `xgboost` and `lightgbm`. In general, we can use any python model. \n", "\n", "![Impression](https://www.google-analytics.com/collect?v=1&tid=UA-112879361-3&cid=555&t=event&ec=scikit-learn&ea=scikit-learn-multiple-models&dt=scikit-learn-multiple-models)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "\n", "import warnings\n", "\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import bentoml\n", "import lightgbm as lgb\n", "import numpy as np\n", "import pandas as pd\n", "import xgboost as xgb\n", "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Prepare Dataset\n", "download dataset from https://www.kaggle.com/c/titanic/data" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ " % Total % Received % Xferd Average Speed Time Time Time Current\n", " Dload Upload Total Spent Left Speed\n", "\r", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", "100 60302 100 60302 0 0 129k 0 --:--:-- --:--:-- --:--:-- 128k\n", " % Total % Received % Xferd Average Speed Time Time Time Current\n", " Dload Upload Total Spent Left Speed\n", "\r", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", "100 28210 100 28210 0 0 86006 0 --:--:-- --:--:-- --:--:-- 86006\n" ] } ], "source": [ "%%sh\n", "mkdir data\n", "curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/train.csv -o ./data/train.csv\n", "curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/test.csv -o ./data/test.csv" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PassengerIdSurvivedPclassNameSexAgeSibSpParchTicketFareCabinEmbarked
0103Braund, Mr. Owen Harrismale22.010A/5 211717.2500NaNS
1211Cumings, Mrs. John Bradley (Florence Briggs Th...female38.010PC 1759971.2833C85C
2313Heikkinen, Miss. Lainafemale26.000STON/O2. 31012827.9250NaNS
3411Futrelle, Mrs. Jacques Heath (Lily May Peel)female35.01011380353.1000C123S
4503Allen, Mr. William Henrymale35.0003734508.0500NaNS
\n", "
" ], "text/plain": [ " PassengerId Survived Pclass \\\n", "0 1 0 3 \n", "1 2 1 1 \n", "2 3 1 3 \n", "3 4 1 1 \n", "4 5 0 3 \n", "\n", " Name Sex Age SibSp \\\n", "0 Braund, Mr. Owen Harris male 22.0 1 \n", "1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 \n", "2 Heikkinen, Miss. Laina female 26.0 0 \n", "3 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 \n", "4 Allen, Mr. William Henry male 35.0 0 \n", "\n", " Parch Ticket Fare Cabin Embarked \n", "0 0 A/5 21171 7.2500 NaN S \n", "1 0 PC 17599 71.2833 C85 C \n", "2 0 STON/O2. 3101282 7.9250 NaN S \n", "3 0 113803 53.1000 C123 S \n", "4 0 373450 8.0500 NaN S " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df = pd.read_csv(\"./data/train.csv\")\n", "test_df = pd.read_csv(\"./data/test.csv\")\n", "train_df.head()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "y = train_df.pop(\"Survived\")\n", "cols = [\"Pclass\", \"Age\", \"Fare\", \"SibSp\", \"Parch\"]\n", "X_train, X_test, y_train, y_test = train_test_split(\n", " train_df[cols], y, test_size=0.2, random_state=42\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Model Training" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "LGBMClassifier()" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lgb_model = lgb.LGBMClassifier()\n", "lgb_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "xgb_model = xgb.sklearn.XGBRFClassifier()\n", "xgb_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "models = {\"xgb\": xgb_model, \"lgb\": lgb_model}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create BentoService for model serving\n", "We are going to use `JsonInput` and return the data as `JSON` object. `JSON` objects are passed as a **list**." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting multiple_models_titanic_bento_service.py\n" ] } ], "source": [ "%%writefile multiple_models_titanic_bento_service.py\n", "\n", "import json\n", "\n", "import bentoml\n", "import lightgbm as lgb\n", "import pandas as pd\n", "import xgboost as xgb\n", "from bentoml.adapters import JsonInput\n", "from bentoml.frameworks.sklearn import SklearnModelArtifact\n", "\n", "\n", "@bentoml.artifacts([SklearnModelArtifact(\"xgb\"), SklearnModelArtifact(\"lgb\")])\n", "@bentoml.env(\n", " conda_channels=[\"conda-forge\"],\n", " conda_dependencies=[\"lightgbm==2.3.*\", \"pandas==1.0.*\", \"xgboost==1.2.*\"],\n", ")\n", "class TitanicSurvivalPredictionService(bentoml.BentoService):\n", " @bentoml.api(input=JsonInput(), batch=True)\n", " def predict(self, datain):\n", " # datain is a list of a json object.\n", " df = pd.read_json(json.dumps(datain[0]), orient=\"table\")\n", "\n", " data = df[[\"Pclass\", \"Age\", \"Fare\", \"SibSp\", \"Parch\"]]\n", " result = pd.DataFrame()\n", " result[\"xgb_proba\"] = self.artifacts.xgb.predict_proba(data)[:, 1]\n", " result[\"lgb_proba\"] = self.artifacts.lgb.predict_proba(data)[:, 1]\n", " # make sure to return as a list of json\n", " return [result.to_json(orient=\"table\")]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Save BentoML service archive" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-08-25 13:47:58,242] INFO - BentoService bundle 'TitanicSurvivalPredictionService:20200825134757_086746' saved to: /Users/thein/bentoml/repository/TitanicSurvivalPredictionService/20200825134757_086746\n" ] } ], "source": [ "# 1) import the custom BentoService defined above\n", "from multiple_models_titanic_bento_service import TitanicSurvivalPredictionService\n", "\n", "# 2) `pack` it with required artifacts\n", "bento_service = TitanicSurvivalPredictionService()\n", "bento_service.pack(\"xgb\", xgb_model)\n", "bento_service.pack(\"lgb\", lgb_model)\n", "\n", "# 3) save your BentoSerivce\n", "saved_path = bento_service.save()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Containerize model server with Docker\n", "\n", "\n", "One common way of distributing this model API server for production deployment, is via Docker containers. And BentoML provides a convenient way to do that.\n", "\n", "Note that docker is **not available in Google Colab**. You will need to download and run this notebook locally to try out this containerization with docker feature.\n", "\n", "If you already have docker configured, simply run the follow command to product a docker container serving the IrisClassifier prediction service created above:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sending build context to Docker daemon 551.4kB\n", "Step 1/15 : FROM bentoml/model-server:0.8.5\n", " ---> 6639eed59dc6\n", "Step 2/15 : COPY . /bento\n", " ---> 8916c6323930\n", "Step 3/15 : WORKDIR /bento\n", " ---> Running in e8a222116d35\n", "Removing intermediate container e8a222116d35\n", " ---> ba725f7ade92\n", "Step 4/15 : ARG PIP_INDEX_URL=https://pypi.python.org/simple/\n", " ---> Running in e194a9e5087c\n", "Removing intermediate container e194a9e5087c\n", " ---> bfe333f7f373\n", "Step 5/15 : ARG PIP_TRUSTED_HOST=pypi.python.org\n", " ---> Running in dbf5f680e79d\n", "Removing intermediate container dbf5f680e79d\n", " ---> 0c9fc4ae7489\n", "Step 6/15 : ENV PIP_INDEX_URL $PIP_INDEX_URL\n", " ---> Running in cb44a811259f\n", "Removing intermediate container cb44a811259f\n", " ---> 7a6e44b43f45\n", "Step 7/15 : ENV PIP_TRUSTED_HOST $PIP_TRUSTED_HOST\n", " ---> Running in bca323581f0b\n", "Removing intermediate container bca323581f0b\n", " ---> 8f1f8b11a2d2\n", "Step 8/15 : RUN chmod +x /bento/bentoml-init.sh\n", " ---> Running in 3ddc42cb25c2\n", "Removing intermediate container 3ddc42cb25c2\n", " ---> 9b890a10990e\n", "Step 9/15 : RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi\n", " ---> Running in 5c32936f764e\n", "\u001b[91m+++ dirname /bento/bentoml-init.sh\n", "\u001b[0m\u001b[91m++ cd /bento\n", "++ pwd -P\n", "\u001b[0m\u001b[91m+ SAVED_BUNDLE_PATH=/bento\n", "+ cd /bento\n", "+ '[' -f ./setup.sh ']'\n", "\u001b[0m\u001b[91m+ command -v conda\n", "\u001b[0m\u001b[91m+ conda env update -n base -f ./environment.yml\n", "\u001b[0mCollecting package metadata (repodata.json): ...working... done\n", "Solving environment: ...working... done\n", "\n", "Downloading and Extracting Packages\n", "mkl-service-2.3.0 | 218 KB | ########## | 100% \n", "xgboost-1.2.0 | 11 KB | ########## | 100% \n", "intel-openmp-2020.1 | 780 KB | ########## | 100% \n", "pip-20.2.2 | 1.8 MB | ########## | 100% \n", "mkl-2020.1 | 129.0 MB | ########## | 100% \n", "python-dateutil-2.8. | 215 KB | ########## | 100% \n", "pytz-2020.1 | 184 KB | ########## | 100% \n", "scipy-1.5.2 | 14.3 MB | ########## | 100% \n", "certifi-2020.6.20 | 156 KB | ########## | 100% \n", "python_abi-3.7 | 4 KB | ########## | 100% \n", "libxgboost-1.2.0 | 3.1 MB | ########## | 100% \n", "numpy-1.19.1 | 21 KB | ########## | 100% \n", "py-xgboost-1.2.0 | 1.7 MB | ########## | 100% \n", "openssl-1.1.1g | 2.5 MB | ########## | 100% \n", "mkl_random-1.1.1 | 322 KB | ########## | 100% \n", "pandas-1.0.5 | 7.8 MB | ########## | 100% \n", "joblib-0.16.0 | 210 KB | ########## | 100% \n", "scikit-learn-0.23.1 | 5.0 MB | ########## | 100% \n", "_py-xgboost-mutex-2. | 9 KB | ########## | 100% \n", "numpy-base-1.19.1 | 4.1 MB | ########## | 100% \n", "blas-1.0 | 6 KB | ########## | 100% \n", "ca-certificates-2020 | 125 KB | ########## | 100% \n", "threadpoolctl-2.1.0 | 17 KB | ########## | 100% \n", "lightgbm-2.3.0 | 936 KB | ########## | 100% \n", "libgfortran-ng-7.3.0 | 1006 KB | ########## | 100% \n", "mkl_fft-1.1.0 | 143 KB | ########## | 100% \n", "Preparing transaction: ...working... done\n", "Verifying transaction: ...working... done\n", "Executing transaction: ...working... done\n", "#\n", "# To activate this environment, use\n", "#\n", "# $ conda activate base\n", "#\n", "# To deactivate an active environment, use\n", "#\n", "# $ conda deactivate\n", "\n", "\u001b[91m\n", "\n", "==> WARNING: A newer version of conda exists. <==\n", " current version: 4.8.2\n", " latest version: 4.8.4\n", "\n", "Please update conda by running\n", "\n", " $ conda update -n base -c defaults conda\n", "\n", "\n", "\u001b[0m\u001b[91m+ pip install -r ./requirements.txt --no-cache-dir\n", "\u001b[0mLooking in indexes: https://pypi.python.org/simple/\n", "Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 1)) (0.23.1)\n", "Requirement already satisfied: bentoml==0.8.5 in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 2)) (0.8.5)\n", "Requirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r ./requirements.txt (line 1)) (1.19.1)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r ./requirements.txt (line 1)) (2.1.0)\n", "Requirement already satisfied: scipy>=0.19.1 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r ./requirements.txt (line 1)) (1.5.2)\n", "Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r ./requirements.txt (line 1)) (0.16.0)\n", "Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.8.7)\n", "Requirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (5.0.0)\n", "Requirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.3.18)\n", "Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (2020.6.20)\n", "Requirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.3.2)\n", "Requirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.6.2)\n", "Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (20.4)\n", "Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.12.4)\n", "Requirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (4.3.0)\n", "Requirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.27.2)\n", "Collecting python-dateutil<2.8.1,>=2.1\n", " Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)\n", "Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (2.22.0)\n", "Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.8.0)\n", "Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (20.0.4)\n", "Requirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (8.2)\n", "Requirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.1.2)\n", "Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (7.1.2)\n", "Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (5.7.2)\n", "Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.14.39)\n", "Requirement already satisfied: multidict in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (4.7.6)\n", "Requirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.1.11)\n", "Requirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.4.2)\n", "Requirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.20.0)\n", "Requirement already satisfied: sqlalchemy-utils in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.36.8)\n", "Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.15.87)\n", "Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from cerberus->bentoml==0.8.5->-r ./requirements.txt (line 2)) (45.2.0.post20200210)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.5.1)\n", "Requirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.0.1)\n", "Requirement already satisfied: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.0.4)\n", "Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.5->-r ./requirements.txt (line 2)) (19.3.0)\n", "Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.14.0)\n", "Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.8.5->-r ./requirements.txt (line 2)) (2.4.7)\n", "Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.57.0)\n", "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.5->-r ./requirements.txt (line 2)) (2.8)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.25.8)\n", "Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.0.1)\n", "Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.5->-r ./requirements.txt (line 2)) (2.11.2)\n", "Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.1.0)\n", "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.10.0)\n", "Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.3.3)\n", "Requirement already satisfied: botocore<1.18.0,>=1.17.39 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.17.39)\n", "Requirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.1.3)\n", "Requirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.0.4)\n", "Requirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.4.11)\n", "Requirement already satisfied: typing-extensions>=3.7.4; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.7.4.2)\n", "Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->bentoml==0.8.5->-r ./requirements.txt (line 2)) (1.1.1)\n", "Requirement already satisfied: docutils<0.16,>=0.10 in /opt/conda/lib/python3.7/site-packages (from botocore<1.18.0,>=1.17.39->boto3->bentoml==0.8.5->-r ./requirements.txt (line 2)) (0.15.2)\n", "Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.8.5->-r ./requirements.txt (line 2)) (3.11)\n", "Installing collected packages: python-dateutil\n", " Attempting uninstall: python-dateutil\n", " Found existing installation: python-dateutil 2.8.1\n", " Uninstalling python-dateutil-2.8.1:\n", " Successfully uninstalled python-dateutil-2.8.1\n", "Successfully installed python-dateutil-2.8.0\n", "\u001b[91m+ for filename in ./bundled_pip_dependencies/*.tar.gz\n", "+ '[' -e './bundled_pip_dependencies/*.tar.gz' ']'\n", "+ continue\n", "\u001b[0mRemoving intermediate container 5c32936f764e\n", " ---> 089fdb91acb3\n", "Step 10/15 : ENV PORT 5000\n", " ---> Running in f2c62158d05f\n", "Removing intermediate container f2c62158d05f\n", " ---> 0d33795106f8\n", "Step 11/15 : EXPOSE $PORT\n", " ---> Running in b4fb56d2e1c6\n", "Removing intermediate container b4fb56d2e1c6\n", " ---> df14a798ba76\n", "Step 12/15 : COPY docker-entrypoint.sh /usr/local/bin/\n", " ---> 9e24b68d273d\n", "Step 13/15 : RUN chmod +x /usr/local/bin/docker-entrypoint.sh\n", " ---> Running in 0ad73aa6f6f0\n", "Removing intermediate container 0ad73aa6f6f0\n", " ---> c608332dd924\n", "Step 14/15 : ENTRYPOINT [ \"docker-entrypoint.sh\" ]\n", " ---> Running in efee5b711dbf\n", "Removing intermediate container efee5b711dbf\n", " ---> d055e40fe3f9\n", "Step 15/15 : CMD [\"bentoml\", \"serve-gunicorn\", \"/bento\"]\n", " ---> Running in 3e0c629cfb1f\n", "Removing intermediate container 3e0c629cfb1f\n", " ---> fa9a266c2e10\n", "Successfully built fa9a266c2e10\n", "Successfully tagged multi_models_titanic:latest\n" ] } ], "source": [ "!bentoml containerize TitanicSurvivalPredictionService:latest" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# port forward to 7000 \n", "!docker run -d -p 7000:5000 TitanicSurvivalPredictionService --enable-microbatch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load saved BentoService\n", "\n", "bentoml.load is the API for loading a BentoML packaged model in python:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-08-25 13:47:58,780] WARNING - Module `multiple_models_titanic_bento_service` already loaded, using existing imported module.\n" ] } ], "source": [ "import json\n", "\n", "import bentoml\n", "\n", "bento_model = bentoml.load(saved_path)\n", "\n", "print(bento_model.predict(X_tests.to_json(orient='table')))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Deployment Options\n", "\n", "If you are at a small team with limited engineering or DevOps resources, try out automated deployment with BentoML CLI, currently supporting AWS Lambda, AWS SageMaker, and Azure Functions:\n", "- [AWS Lambda Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_lambda.html)\n", "- [AWS SageMaker Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_sagemaker.html)\n", "- [Azure Functions Deployment Guide](https://docs.bentoml.org/en/latest/deployment/azure_functions.html)\n", "\n", "If the cloud platform you are working with is not on the list above, try out these step-by-step guide on manually deploying BentoML packaged model to cloud platforms:\n", "- [AWS ECS Deployment](https://docs.bentoml.org/en/latest/deployment/aws_ecs.html)\n", "- [Google Cloud Run Deployment](https://docs.bentoml.org/en/latest/deployment/google_cloud_run.html)\n", "- [Azure container instance Deployment](https://docs.bentoml.org/en/latest/deployment/azure_container_instance.html)\n", "- [Heroku Deployment](https://docs.bentoml.org/en/latest/deployment/heroku.html)\n", "\n", "Lastly, if you have a DevOps or ML Engineering team who's operating a Kubernetes or OpenShift cluster, use the following guides as references for implementating your deployment strategy:\n", "- [Kubernetes Deployment](https://docs.bentoml.org/en/latest/deployment/kubernetes.html)\n", "- [Knative Deployment](https://docs.bentoml.org/en/latest/deployment/knative.html)\n", "- [Kubeflow Deployment](https://docs.bentoml.org/en/latest/deployment/kubeflow.html)\n", "- [KFServing Deployment](https://docs.bentoml.org/en/latest/deployment/kfserving.html)\n", "- [Clipper.ai Deployment Guide](https://docs.bentoml.org/en/latest/deployment/clipper.html)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 4 }