{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "aZ0Tv8OmUQS0" }, "source": [ "# BentoML Demo: Iris Classifier with custom web UI\n", "\n", "\n", "**BentoML makes moving trained ML models to production easy:**\n", "\n", "* Package models trained with **any ML framework** and reproduce them for model serving in production\n", "* **Deploy anywhere** for online API serving or offline batch serving\n", "* High-Performance API model server with *adaptive micro-batching* support\n", "* Central hub for managing models and deployment process via Web UI and APIs\n", "* Modular and flexible design making it *adaptable to your infrastrcuture*\n", "\n", "BentoML is a framework for serving, managing, and deploying machine learning models. It is aiming to bridge the gap between Data Science and DevOps, and enable teams to deliver prediction services in a fast, repeatable, and scalable way.\n", "\n", "Before reading this example project, be sure to check out the [Getting started guide](https://github.com/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb) to learn about the basic concepts in BentoML.\n", "\n", "\n", "This notebook demonstrates how to use BentoML to __serve a Iris Classification model containing a REST API server with Custom Web UI__.\n", "\n", "![Impression](https://www.google-analytics.com/collect?v=1&tid=UA-112879361-3&cid=555&t=event&ec=scikit-learn&ea=scikit-learn-iris-classifier-web&dt=scikit-learn-iris-classifier-web)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": {}, "colab_type": "code", "id": "vzf20abXU_vz" }, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "\n", "import warnings\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 989 }, "colab_type": "code", "id": "5fANNcHQVC-g", "outputId": "dbedb6bf-97b7-45af-8891-99e67a08d96b" }, "outputs": [], "source": [ "!pip install -q bentoml \"scikit-learn>=0.23.2\"" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "_a2_xA2JtCEo" }, "source": [ "## Create BentoService for model serving" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "colab_type": "code", "id": "ixGEZTxpVXPe", "outputId": "9c80ae20-f8e1-4933-8fe9-8e4498f23dca" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing iris_classifier.py\n" ] } ], "source": [ "%%writefile iris_classifier.py\n", "from bentoml import env, artifacts, api, BentoService\n", "from bentoml.adapters import DataframeInput\n", "from bentoml.frameworks.sklearn import SklearnModelArtifact\n", "\n", "@env(infer_pip_packages=True)\n", "@artifacts([SklearnModelArtifact('model')])\n", "class IrisClassifier(BentoService):\n", "\n", " @api(input=DataframeInput(), batch=True)\n", " def predict(self, df):\n", " # Optional pre-processing, post-processing code goes here\n", " return self.artifacts.model.predict(df)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "colab_type": "code", "id": "LolMzwYrWaqI", "outputId": "cecd7fbe-7b5d-4f85-f94d-9db41fa19e6c" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing main.py\n" ] } ], "source": [ "%%writefile main.py\n", "from sklearn import svm\n", "from sklearn import datasets\n", "\n", "from iris_classifier import IrisClassifier\n", "\n", "if __name__ == \"__main__\":\n", " # Load training data\n", " iris = datasets.load_iris()\n", " X, y = iris.data, iris.target\n", "\n", " # Model Training\n", " clf = svm.SVC(gamma='scale')\n", " clf.fit(X, y)\n", "\n", " # Create a iris classifier service instance\n", " iris_classifier_service = IrisClassifier()\n", "\n", " # Pack the newly trained model artifact\n", " iris_classifier_service.pack('model', clf)\n", "\n", " # Save the prediction service to disk for model serving\n", " saved_path = iris_classifier_service.save()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-09-09 11:50:39,692] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n", "[2020-09-09 11:50:41,005] INFO - Detected non-PyPI-released BentoML installed, copying local BentoML modulefiles to target saved bundle path..\n", "warning: no previously-included files matching '*~' found anywhere in distribution\n", "warning: no previously-included files matching '*.pyo' found anywhere in distribution\n", "warning: no previously-included files matching '.git' found anywhere in distribution\n", "warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution\n", "warning: no previously-included files matching '__pycache__' found anywhere in distribution\n", "no previously-included directories found matching 'e2e_tests'\n", "no previously-included directories found matching 'tests'\n", "no previously-included directories found matching 'benchmark'\n", "UPDATING BentoML-0.8.6+34.g6123b8c6/bentoml/_version.py\n", "set BentoML-0.8.6+34.g6123b8c6/bentoml/_version.py to '0.8.6+34.g6123b8c6'\n", "[2020-09-09 11:50:45,682] INFO - BentoService bundle 'IrisClassifier:20200909115040_9FC7F4' saved to: /Users/bozhaoyu/bentoml/repository/IrisClassifier/20200909115040_9FC7F4\n" ] } ], "source": [ "!python main.py" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "-mEDm1JZtZgp" }, "source": [ "## REST API Model Serving\n", "\n", "\n", "To start a REST API model server with the BentoService saved above, use the bentoml serve command:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 360 }, "colab_type": "code", "id": "PGo6Y_KNWjpY", "outputId": "3d6e17a0-64dd-490f-be7f-2cfe4fbf0d62" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-07-28 16:38:55,465] INFO - Getting latest version IrisClassifier:20200728163753_DE85E9\n", "[2020-07-28 16:38:56,409] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n", "[2020-07-28 16:38:56,422] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.1, but loading from BentoML version 0.8.1+27.g8e155f5.dirty\n", " * Serving Flask app \"IrisClassifier\" (lazy loading)\n", " * Environment: production\n", "\u001b[31m WARNING: This is a development server. Do not use it in a production deployment.\u001b[0m\n", "\u001b[2m Use a production WSGI server instead.\u001b[0m\n", " * Debug mode: off\n", " * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n", "^C\n" ] } ], "source": [ "!bentoml serve IrisClassifier:latest" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you are running this notebook from Google Colab, you can start the dev server with `--run-with-ngrok` option, to gain acccess to the API endpoint via a public endpoint managed by [ngrok](https://ngrok.com/):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!bentoml serveIrisiClassifier:latest --run-with-ngrok" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "qllScrbHtqKl" }, "source": [ "At this point you can test out the Rest API server either by opening http://localhost:5000 in a new tab which will serve the swagger docs:\n", "\n", "![alt text](https://raw.githubusercontent.com/bentoml/gallery/master/scikit-learn/iris-classifier/swagger.png)\n", "\n", "\n", "or by making a curl request in a another terminal window:\n", "\n", "```bash\n", "curl -i \\\n", "--header \"Content-Type: application/json\" \\\n", "--request POST \\\n", "--data '[[1,2,1,2]]' \\\n", "localhost:5000/predict\n", "```" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "_WN9nDgcvcul" }, "source": [ "## Adding Custom Web Static Content\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 68 }, "colab_type": "code", "id": "HO47OfDivier", "outputId": "1a9c658e-7422-4bbd-e75f-38860fec18ef" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " % Total % Received % Xferd Average Speed Time Time Time Current\n", " Dload Upload Total Spent Left Speed\n", "100 197k 100 197k 0 0 200k 0 --:--:-- --:--:-- --:--:-- 200k\n" ] } ], "source": [ "!curl https://raw.githubusercontent.com/bentoml/gallery/master/scikit-learn/iris-classifier/static.tar.xz -o static.tar.xz\n", "!tar --xz -xf static.tar.xz\n", "!rm static.tar.xz" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "rzJ1Wx-fyRcG" }, "source": [ "Here we have a very simple web ui as our static content that bento will serve.\n", "Now we will edit our bento service to point to this static directory.\n", "\n", "Add `@web_static_content('./static')` to `iris_classifier.py`\n", "\n", "**Note**: The path can be both relative or absolute. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "colab_type": "code", "id": "mgakbz1JxMpc", "outputId": "8f8ed48d-4686-430d-e215-1868ce76f0a0" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting iris_classifier.py\n" ] } ], "source": [ "%%writefile iris_classifier.py\n", "from bentoml import env, artifacts, api, BentoService, web_static_content\n", "from bentoml.adapters import DataframeInput\n", "from bentoml.artifact import SklearnModelArtifact\n", "\n", "@env(auto_pip_dependencies=True)\n", "@artifacts([SklearnModelArtifact('model')])\n", "@web_static_content('./static')\n", "class IrisClassifier(BentoService):\n", "\n", " @api(input=DataframeInput(), batch=True)\n", " def test(self, df):\n", " # Optional pre-processing, post-processing code goes here\n", " return self.artifacts.model.predict(df)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-07-30 09:09:28,868] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n", "[2020-07-30 09:09:39,344] INFO - Detect BentoML installed in development model, copying local BentoML module file to target saved bundle path\n", "running sdist\n", "running egg_info\n", "writing BentoML.egg-info/PKG-INFO\n", "writing dependency_links to BentoML.egg-info/dependency_links.txt\n", "writing entry points to BentoML.egg-info/entry_points.txt\n", "writing requirements to BentoML.egg-info/requires.txt\n", "writing top-level names to BentoML.egg-info/top_level.txt\n", "reading manifest file 'BentoML.egg-info/SOURCES.txt'\n", "reading manifest template 'MANIFEST.in'\n", "warning: no previously-included files matching '*~' found anywhere in distribution\n", "warning: no previously-included files matching '*.pyo' found anywhere in distribution\n", "warning: no previously-included files matching '.git' found anywhere in distribution\n", "warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution\n", "warning: no previously-included files matching '__pycache__' found anywhere in distribution\n", "warning: no directories found matching 'bentoml/server/static'\n", "warning: no directories found matching 'bentoml/yatai/web/dist'\n", "no previously-included directories found matching 'e2e_tests'\n", "no previously-included directories found matching 'tests'\n", "no previously-included directories found matching 'benchmark'\n", "writing manifest file 'BentoML.egg-info/SOURCES.txt'\n", "running check\n", "creating BentoML-0.8.3+42.gb8d36b6\n", "creating BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/clipper\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/configuration\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/handlers\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/marshal\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/client\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/__pycache__\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/versions\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/versions/__pycache__\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "creating BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/validator\n", "copying files to BentoML-0.8.3+42.gb8d36b6...\n", "copying LICENSE -> BentoML-0.8.3+42.gb8d36b6\n", "copying MANIFEST.in -> BentoML-0.8.3+42.gb8d36b6\n", "copying README.md -> BentoML-0.8.3+42.gb8d36b6\n", "copying pyproject.toml -> BentoML-0.8.3+42.gb8d36b6\n", "copying setup.cfg -> BentoML-0.8.3+42.gb8d36b6\n", "copying setup.py -> BentoML-0.8.3+42.gb8d36b6\n", "copying versioneer.py -> BentoML-0.8.3+42.gb8d36b6\n", "copying BentoML.egg-info/PKG-INFO -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying BentoML.egg-info/SOURCES.txt -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying BentoML.egg-info/dependency_links.txt -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying BentoML.egg-info/entry_points.txt -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying BentoML.egg-info/requires.txt -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying BentoML.egg-info/top_level.txt -> BentoML-0.8.3+42.gb8d36b6/BentoML.egg-info\n", "copying bentoml/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml\n", "copying bentoml/_version.py -> BentoML-0.8.3+42.gb8d36b6/bentoml\n", "copying bentoml/exceptions.py -> BentoML-0.8.3+42.gb8d36b6/bentoml\n", "copying bentoml/service.py -> BentoML-0.8.3+42.gb8d36b6/bentoml\n", "copying bentoml/service_env.py -> BentoML-0.8.3+42.gb8d36b6/bentoml\n", "copying bentoml/adapters/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/base_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/base_output.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/clipper_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/dataframe_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/dataframe_output.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/default_output.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/fastai_image_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/file_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/image_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/json_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/json_output.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/legacy_image_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/legacy_json_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/multi_image_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/pytorch_tensor_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/tensorflow_tensor_input.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/tensorflow_tensor_output.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/adapters/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/adapters\n", "copying bentoml/artifact/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/fastai2_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/fastai_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/fasttext_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/h2o_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/json_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/keras_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/lightgbm_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/onnx_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/pickle_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/pytorch_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/sklearn_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/spacy_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/text_file_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/tf_savedmodel_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/artifact/xgboost_model_artifact.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/artifact\n", "copying bentoml/cli/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/aws_lambda.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/aws_sagemaker.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/azure_functions.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/bento_management.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/bento_service.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/click_utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/config.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/deployment.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/cli/yatai_service.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/cli\n", "copying bentoml/clipper/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/clipper\n", "copying bentoml/configuration/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration\n", "copying bentoml/configuration/configparser.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration\n", "copying bentoml/configuration/default_bentoml.cfg -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration\n", "copying bentoml/configuration/__pycache__/__init__.cpython-36.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/configuration/__pycache__/__init__.cpython-37.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/configuration/__pycache__/__init__.cpython-38.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/configuration/__pycache__/configparser.cpython-36.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/configuration/__pycache__/configparser.cpython-37.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/configuration/__pycache__/configparser.cpython-38.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/configuration/__pycache__\n", "copying bentoml/handlers/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/handlers\n", "copying bentoml/marshal/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/marshal\n", "copying bentoml/marshal/dispatcher.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/marshal\n", "copying bentoml/marshal/marshal.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/marshal\n", "copying bentoml/marshal/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/marshal\n", "copying bentoml/saved_bundle/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/bentoml-init.sh -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/bundler.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/config.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/docker-entrypoint.sh -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/loader.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/pip_pkg.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/py_module_utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/saved_bundle/templates.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/saved_bundle\n", "copying bentoml/server/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/api_server.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/gunicorn_config.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/gunicorn_server.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/instruments.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/marshal_server.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/open_api.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/trace.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/server/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/server\n", "copying bentoml/utils/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/alg.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/benchmark.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/cloudpickle.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/dataframe_util.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/flask_ngrok.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/hybridmethod.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/lazy_loader.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/log.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/s3.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/tempdir.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/utils/usage_stats.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/utils\n", "copying bentoml/yatai/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/alembic.ini -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/db.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/deployment_utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/status.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/yatai_service.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/yatai_service_impl.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai\n", "copying bentoml/yatai/client/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/client\n", "copying bentoml/yatai/client/bento_repository_api.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/client\n", "copying bentoml/yatai/client/deployment_api.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/client\n", "copying bentoml/yatai/deployment/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment\n", "copying bentoml/yatai/deployment/operator.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment\n", "copying bentoml/yatai/deployment/store.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment\n", "copying bentoml/yatai/deployment/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment\n", "copying bentoml/yatai/deployment/aws_lambda/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "copying bentoml/yatai/deployment/aws_lambda/download_extra_resources.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "copying bentoml/yatai/deployment/aws_lambda/lambda_app.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "copying bentoml/yatai/deployment/aws_lambda/operator.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "copying bentoml/yatai/deployment/aws_lambda/utils.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/aws_lambda\n", "copying bentoml/yatai/deployment/azure_functions/Dockerfile -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/app_init.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/constants.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/host.json -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/local.settings.json -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/operator.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/azure_functions/templates.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/azure_functions\n", "copying bentoml/yatai/deployment/sagemaker/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/deployment/sagemaker/model_server.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/deployment/sagemaker/nginx.conf -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/deployment/sagemaker/operator.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/deployment/sagemaker/serve -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/deployment/sagemaker/wsgi.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/deployment/sagemaker\n", "copying bentoml/yatai/migrations/README -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations\n", "copying bentoml/yatai/migrations/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations\n", "copying bentoml/yatai/migrations/env.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations\n", "copying bentoml/yatai/migrations/script.py.mako -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations\n", "copying bentoml/yatai/migrations/__pycache__/env.cpython-36.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/__pycache__\n", "copying bentoml/yatai/migrations/versions/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/versions\n", "copying bentoml/yatai/migrations/versions/a6b00ae45279_add_last_updated_at_for_deployments.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/versions\n", "copying bentoml/yatai/migrations/versions/__pycache__/a6b00ae45279_add_last_updated_at_for_deployments.cpython-36.pyc -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/migrations/versions/__pycache__\n", "copying bentoml/yatai/proto/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/proto/deployment_pb2.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/proto/repository_pb2.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/proto/status_pb2.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/proto/yatai_service_pb2.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/proto/yatai_service_pb2_grpc.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/proto\n", "copying bentoml/yatai/repository/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/repository/base_repository.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/repository/local_repository.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/repository/metadata_store.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/repository/repository.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/repository/s3_repository.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/repository\n", "copying bentoml/yatai/validator/__init__.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/validator\n", "copying bentoml/yatai/validator/deployment_pb_validator.py -> BentoML-0.8.3+42.gb8d36b6/bentoml/yatai/validator\n", "Writing BentoML-0.8.3+42.gb8d36b6/setup.cfg\n", "UPDATING BentoML-0.8.3+42.gb8d36b6/bentoml/_version.py\n", "set BentoML-0.8.3+42.gb8d36b6/bentoml/_version.py to '0.8.3+42.gb8d36b6'\n", "Creating tar archive\n", "removing 'BentoML-0.8.3+42.gb8d36b6' (and everything under it)\n", "[2020-07-30 09:09:40,124] INFO - BentoService bundle 'IrisClassifier:20200730090929_C02FB7' saved to: /home/bentoml/bentoml/repository/IrisClassifier/20200730090929_C02FB7\n" ] } ], "source": [ "!python main.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 394 }, "colab_type": "code", "id": "rfbaTgKW0D78", "outputId": "558b2660-2a45-4d4e-db66-9a559ace69a9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-07-30 09:09:43,808] INFO - Getting latest version IrisClassifier:20200730090929_C02FB7\n", "[2020-07-30 09:09:43,808] INFO - Starting BentoML API server in development mode..\n", "[2020-07-30 09:09:44,916] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n", "[2020-07-30 09:09:44,949] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.3, but loading from BentoML version 0.8.3+42.gb8d36b6\n", " * Serving Flask app \"IrisClassifier\" (lazy loading)\n", " * Environment: production\n", "\u001b[31m WARNING: This is a development server. Do not use it in a production deployment.\u001b[0m\n", "\u001b[2m Use a production WSGI server instead.\u001b[0m\n", " * Debug mode: off\n", " * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)\n", "127.0.0.1 - - [30/Jul/2020 09:09:52] \"\u001b[37mPOST /test HTTP/1.1\u001b[0m\" 200 -\n", "127.0.0.1 - - [30/Jul/2020 09:09:55] \"\u001b[37mPOST /test HTTP/1.1\u001b[0m\" 200 -\n", "127.0.0.1 - - [30/Jul/2020 09:09:57] \"\u001b[37mPOST /test HTTP/1.1\u001b[0m\" 200 -\n", "127.0.0.1 - - [30/Jul/2020 09:10:00] \"\u001b[37mPOST /test HTTP/1.1\u001b[0m\" 200 -\n" ] } ], "source": [ "!bentoml serve IrisClassifier:latest" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "IIOGBpOZ0GoL" }, "source": [ "Now if you visit http://localhost:5000/, you should be served with a beautiful UI:\n", "\n", "![Custom UI](https://raw.githubusercontent.com/bentoml/gallery/master/scikit-learn/iris-classifier/webui.png)\n", "\n", "It's still possible to access the swagger docs at `/docs`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Containerize model server with Docker\n", "\n", "\n", "One common way of distributing this model API server for production deployment, is via Docker containers. And BentoML provides a convenient way to do that.\n", "\n", "Note that docker is **not available in Google Colab**. You will need to download and run this notebook locally to try out this containerization with docker feature.\n", "\n", "If you already have docker configured, simply run the follow command to product a docker container serving the IrisClassifier prediction service created above:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[2020-09-09 11:52:05,072] INFO - Getting latest version IrisClassifier:20200909115040_9FC7F4\n", "\u001b[39mFound Bento: /Users/bozhaoyu/bentoml/repository/IrisClassifier/20200909115040_9FC7F4\u001b[0m\n", "[2020-09-09 11:52:05,109] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.\n", "[2020-09-09 11:52:05,126] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.6, but loading from BentoML version 0.8.6+34.g6123b8c6\n", "\u001b[39mTag not specified, using tag parsed from BentoService: 'irisclassifier:20200909115040_9FC7F4'\u001b[0m\n", "Building Docker image irisclassifier:20200909115040_9FC7F4 from IrisClassifier:latest \n", "-we in here\n", "processed docker file\n", "(None, None)\n", "root in create archive /Users/bozhaoyu/bentoml/repository/IrisClassifier/20200909115040_9FC7F4 ['Dockerfile', 'IrisClassifier', 'IrisClassifier/__init__.py', 'IrisClassifier/artifacts', 'IrisClassifier/artifacts/__init__.py', 'IrisClassifier/artifacts/model.pkl', 'IrisClassifier/bentoml.yml', 'IrisClassifier/iris_classifier.py', 'MANIFEST.in', 'README.md', 'bentoml-init.sh', 'bentoml.yml', 'bundled_pip_dependencies', 'bundled_pip_dependencies/BentoML-0.8.6+34.g6123b8c6.tar.gz', 'docker-entrypoint.sh', 'environment.yml', 'requirements.txt', 'setup.py']\n", "about to build\n", "about to upgrade params\n", "check each param and update\n", "if use config proxy\n", "if buildargs\n", "if shmsize\n", "if labels\n", "if cache from\n", "if target\n", "if network_mode\n", "if squash\n", "if extra hosts is not None\n", "if platform is not None\n", "if isolcation is not None\n", "if context is not None\n", "setting auth {'Content-Type': 'application/tar'}\n", "\b|docker build {'t': 'irisclassifier:20200909115040_9FC7F4', 'remote': None, 'q': False, 'nocache': False, 'rm': False, 'forcerm': False, 'pull': False, 'dockerfile': (None, None)}\n", "\b-docker response \n", "context closes\n", "print responses\n", "\u001b[39mStep 1/15 : FROM bentoml/model-server:0.8.6\u001b[0m\n", "\u001b[39m ---> 71644b758bed\u001b[0m\n", "\u001b[39mStep 2/15 : COPY . /bento\u001b[0m\n", "\b\\\u001b[39m ---> ac6b4936bb75\u001b[0m\n", "\u001b[39mStep 3/15 : WORKDIR /bento\u001b[0m\n", "\u001b[39m ---> Running in ab9341b14333\u001b[0m\n", "\b-\u001b[39m ---> dc2c710d4a0f\u001b[0m\n", "\u001b[39mStep 4/15 : ARG PIP_INDEX_URL=https://pypi.python.org/simple/\u001b[0m\n", "\b/\u001b[39m ---> Running in f21c5c697580\u001b[0m\n", "\b|\u001b[39m ---> 6e05f460fbc5\u001b[0m\n", "\u001b[39mStep 5/15 : ARG PIP_TRUSTED_HOST=pypi.python.org\u001b[0m\n", "\u001b[39m ---> Running in 900a926e8c53\u001b[0m\n", "\b\\\u001b[39m ---> 3cc575c53975\u001b[0m\n", "\u001b[39mStep 6/15 : ENV PIP_INDEX_URL $PIP_INDEX_URL\u001b[0m\n", "\b-\u001b[39m ---> Running in 3367fdca94cc\u001b[0m\n", "\b/\u001b[39m ---> e20c93c7f376\u001b[0m\n", "\u001b[39mStep 7/15 : ENV PIP_TRUSTED_HOST $PIP_TRUSTED_HOST\u001b[0m\n", "\u001b[39m ---> Running in d14c545e8549\u001b[0m\n", "\b|\u001b[39m ---> b5fcb21c7843\u001b[0m\n", "\u001b[39mStep 8/15 : RUN chmod +x /bento/bentoml-init.sh\u001b[0m\n", "\b\\\u001b[39m ---> Running in 107df63f664e\u001b[0m\n", "\b-\u001b[39m ---> 9073da44fe2d\u001b[0m\n", "\u001b[39mStep 9/15 : RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi\u001b[0m\n", "\u001b[39m ---> Running in e9185848bc40\u001b[0m\n", "\b\\\u001b[39m\u001b[91m+++ dirname /bento/bentoml-init.sh\n", "\u001b[0m\u001b[0m\n", "\u001b[39m\u001b[91m++ cd /bento\n", "++ pwd -P\n", "\u001b[0m\u001b[0m\n", "\u001b[39m\u001b[91m+ SAVED_BUNDLE_PATH=/bento\n", "+ cd /bento\n", "+ '[' -f ./setup.sh ']'\n", "+ command -v conda\n", "+ echo 'Updating conda base environment with environment.yml'\n", "+ conda env update -n base -f ./environment.yml\n", "\u001b[0m\u001b[0m\n", "\u001b[39mUpdating conda base environment with environment.yml\u001b[0m\n", "\b/\u001b[39mCollecting package metadata (repodata.json): ...working... \u001b[0m\n", "\b\\\u001b[39mdone\n", "Solving environment: ...working... \u001b[0m\n", "\b|\u001b[39mdone\u001b[0m\n", "\b\\\u001b[39m\n", "Downloading and Extracting Packages\n", "openssl-1.1.1g | 2.1 MB | | 0% \u001b[0m\n", "openssl-1.1.1g | 2.1 MB | | 1% \u001b[0m\n", "openssl-1.1.1g | 2.1 MB | #9 | 19% \u001b[0m\n", "openssl-1.1.1g | 2.1 MB | ####7 | 47% \u001b[0m\n", "openssl-1.1.1g | 2.1 MB | ######8 | 68% \u001b[0m\n", "openssl-1.1.1g | 2.1 MB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "python_abi-3.7 | 4 KB | | 0% \u001b[0m\n", "python_abi-3.7 | 4 KB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "certifi-2020.6.20 | 151 KB | | 0% \u001b[0m\n", "certifi-2020.6.20 | 151 KB | # | 11% \u001b[0m\n", "certifi-2020.6.20 | 151 KB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "python-3.7.3 | 32.1 MB | | 0% \u001b[0m\n", "python-3.7.3 | 32.1 MB | | 0% \u001b[0m\n", "python-3.7.3 | 32.1 MB | 2 | 3% \u001b[0m\n", "python-3.7.3 | 32.1 MB | 5 | 6% \u001b[0m\n", "python-3.7.3 | 32.1 MB | 9 | 9% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #1 | 12% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #5 | 15% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ## | 21% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ##5 | 25% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ##9 | 30% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ###5 | 35% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ###9 | 40% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ####4 | 45% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ####9 | 50% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #####4 | 55% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ###### | 60% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ######4 | 65% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ######9 | 70% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #######5 | 75% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ######## | 80% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ########6 | 86% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #########1 | 91% \u001b[0m\n", "python-3.7.3 | 32.1 MB | #########7 | 97% \u001b[0m\n", "python-3.7.3 | 32.1 MB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "ca-certificates-2020 | 145 KB | | 0% \u001b[0m\n", "ca-certificates-2020 | 145 KB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "pip-20.2.3 | 1.1 MB | | 0% \u001b[0m\n", "pip-20.2.3 | 1.1 MB | ###9 | 39% \u001b[0m\n", "pip-20.2.3 | 1.1 MB | ########## | 100% \u001b[0m\n", "\u001b[39m\n", "Preparing transaction: ...working... \u001b[0m\n", "\b\\\u001b[39mdone\u001b[0m\n", "\u001b[39mVerifying transaction: ...working... \u001b[0m\n", "\b|\u001b[39mdone\u001b[0m\n", "\u001b[39mExecuting transaction: ...working... \u001b[0m\n", "\b\\\u001b[39mdone\u001b[0m\n", "\b\\\u001b[39m\u001b[91m\n", "\n", "==> WARNING: A newer version of conda exists. <==\n", " current version: 4.8.2\n", " latest version: 4.8.4\n", "\n", "Please update conda by running\n", "\n", " $ conda update -n base -c defaults conda\n", "\n", "\n", "\u001b[0m\u001b[0m\n", "\u001b[39m#\n", "# To activate this environment, use\n", "#\n", "# $ conda activate base\n", "#\n", "# To deactivate an active environment, use\n", "#\n", "# $ conda deactivate\u001b[0m\n", "\b\\\u001b[39m\u001b[91m+ pip install -r ./requirements.txt --no-cache-dir\n", "\u001b[0m\u001b[0m\n", "\b\\\u001b[39mLooking in indexes: https://pypi.python.org/simple/\u001b[0m\n", "\b/\u001b[39mCollecting scikit-learn\u001b[0m\n", "\b|\u001b[39m Downloading scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl (6.8 MB)\u001b[0m\n", "\b-\u001b[39mCollecting pandas\u001b[0m\n", "\b/\u001b[39m Downloading pandas-1.1.2-cp37-cp37m-manylinux1_x86_64.whl (10.5 MB)\u001b[0m\n", "\b\\\u001b[39mRequirement already satisfied: bentoml==0.8.6 in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 3)) (0.8.6)\u001b[0m\n", "\b|\u001b[39mCollecting threadpoolctl>=2.0.0\u001b[0m\n", "\u001b[39m Downloading threadpoolctl-2.1.0-py3-none-any.whl (12 kB)\u001b[0m\n", "\b-\u001b[39mCollecting joblib>=0.11\u001b[0m\n", "\u001b[39m Downloading joblib-0.16.0-py3-none-any.whl (300 kB)\u001b[0m\n", "\b-\u001b[39mCollecting scipy>=0.19.1\u001b[0m\n", "\b/\u001b[39m Downloading scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB)\u001b[0m\n", "\b-\u001b[39mRequirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r ./requirements.txt (line 1)) (1.19.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas->-r ./requirements.txt (line 2)) (2.8.0)\u001b[0m\n", "\b\\\u001b[39mCollecting pytz>=2017.2\u001b[0m\n", "\b-\u001b[39m Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)\u001b[0m\n", "\u001b[39mRequirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.1.11)\u001b[0m\n", "\u001b[39mRequirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (5.7.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.13.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (2020.6.20)\u001b[0m\n", "\u001b[39mRequirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.6.2)\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\b/\u001b[39mRequirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.8.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (2.22.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (5.0.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: sqlalchemy-utils in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.36.8)\u001b[0m\n", "\b|\u001b[39mRequirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.4.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.1.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (7.1.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (20.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.20.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.15.87)\u001b[0m\n", "\u001b[39mRequirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (20.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.3.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.8.7)\u001b[0m\n", "\u001b[39mRequirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.14.48)\u001b[0m\n", "\u001b[39mRequirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.27.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.3.19)\u001b[0m\n", "\u001b[39mRequirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (4.3.1)\u001b[0m\n", "\b\\\u001b[39mRequirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (8.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: multidict in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.6->-r ./requirements.txt (line 3)) (4.7.6)\u001b[0m\n", "\u001b[39mRequirement already satisfied: six>=1.5 in /opt/conda/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas->-r ./requirements.txt (line 2)) (1.14.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.8.6->-r ./requirements.txt (line 3)) (45.2.0.post20200210)\u001b[0m\n", "\u001b[39mRequirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.6->-r ./requirements.txt (line 3)) (20.1.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.0.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.5.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.6->-r ./requirements.txt (line 3)) (2.8)\u001b[0m\n", "\u001b[39mRequirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.25.8)\u001b[0m\n", "\u001b[39mRequirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.1.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.1.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.6->-r ./requirements.txt (line 3)) (2.11.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.0.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.8.6->-r ./requirements.txt (line 3)) (2.4.7)\u001b[0m\n", "\b-\u001b[39mRequirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.4.11)\u001b[0m\n", "\u001b[39mRequirement already satisfied: botocore<1.18.0,>=1.17.48 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.17.48)\u001b[0m\n", "\u001b[39mRequirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.10.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.3.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.57.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied: typing-extensions>=3.7.4; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.7.4.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->bentoml==0.8.6->-r ./requirements.txt (line 3)) (1.1.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.8.6->-r ./requirements.txt (line 3)) (3.11)\u001b[0m\n", "\u001b[39mRequirement already satisfied: docutils<0.16,>=0.10 in /opt/conda/lib/python3.7/site-packages (from botocore<1.18.0,>=1.17.48->boto3->bentoml==0.8.6->-r ./requirements.txt (line 3)) (0.15.2)\u001b[0m\n", "\b/\u001b[39mInstalling collected packages: threadpoolctl, joblib, scipy, scikit-learn, pytz, pandas\u001b[0m\n", "\b-\u001b[39mSuccessfully installed joblib-0.16.0 pandas-1.1.2 pytz-2020.1 scikit-learn-0.23.2 scipy-1.5.2 threadpoolctl-2.1.0\u001b[0m\n", "\b-\u001b[39m\u001b[91m+ for filename in ./bundled_pip_dependencies/*.tar.gz\n", "+ '[' -e ./bundled_pip_dependencies/BentoML-0.8.6+34.g6123b8c6.tar.gz ']'\n", "+ pip install -U ./bundled_pip_dependencies/BentoML-0.8.6+34.g6123b8c6.tar.gz --no-cache-dir\n", "\u001b[0m\u001b[0m\n", "\b\\\u001b[39mLooking in indexes: https://pypi.python.org/simple/\u001b[0m\n", "\u001b[39mProcessing ./bundled_pip_dependencies/BentoML-0.8.6+34.g6123b8c6.tar.gz\u001b[0m\n", "\b-\u001b[39m Installing build dependencies: started\u001b[0m\n", "\b-\u001b[39m Installing build dependencies: finished with status 'done'\n", " Getting requirements to build wheel: started\u001b[0m\n", "\b/\u001b[39m Getting requirements to build wheel: finished with status 'done'\u001b[0m\n", "\u001b[39m Preparing wheel metadata: started\u001b[0m\n", "\b|\u001b[39m Preparing wheel metadata: finished with status 'done'\u001b[0m\n", "\b-\u001b[39mRequirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (20.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: tabulate in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (0.8.7)\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[39mRequirement already satisfied, skipping upgrade: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.27.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (3.13.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: configparser in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (5.0.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: boto3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.14.48)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: aiohttp in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (3.6.2)\u001b[0m\n", "\b/\u001b[39mRequirement already satisfied, skipping upgrade: numpy in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.19.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: prometheus-client in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (0.8.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: psutil in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (5.7.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: alembic in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.4.2)\u001b[0m\n", "\b|\u001b[39mCollecting sqlalchemy-utils<0.36.8\u001b[0m\n", "\b-\u001b[39m Downloading SQLAlchemy-Utils-0.36.7.tar.gz (131 kB)\u001b[0m\n", "\b|\u001b[39mRequirement already satisfied, skipping upgrade: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (0.15.87)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: multidict in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (4.7.6)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: cerberus in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.3.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: flask in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.1.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: python-json-logger in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (0.1.11)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (2.8.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: py-zipkin in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (0.20.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (1.3.19)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (2.22.0)\u001b[0m\n", "\b\\\u001b[39mRequirement already satisfied, skipping upgrade: humanfriendly in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (8.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: docker in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (4.3.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (7.1.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: gunicorn in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (20.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: certifi in /opt/conda/lib/python3.7/site-packages (from BentoML==0.8.6+34.g6123b8c6) (2020.6.20)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: six in /opt/conda/lib/python3.7/site-packages (from packaging->BentoML==0.8.6+34.g6123b8c6) (1.14.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->BentoML==0.8.6+34.g6123b8c6) (2.4.7)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->BentoML==0.8.6+34.g6123b8c6) (45.2.0.post20200210)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.8.6+34.g6123b8c6) (0.10.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.8.6+34.g6123b8c6) (0.3.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: botocore<1.18.0,>=1.17.48 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.8.6+34.g6123b8c6) (1.17.48)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.8.6+34.g6123b8c6) (3.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.8.6+34.g6123b8c6) (1.5.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.8.6+34.g6123b8c6) (3.0.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.8.6+34.g6123b8c6) (20.1.0)\u001b[0m\n", "\b-\u001b[39mRequirement already satisfied, skipping upgrade: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.8.6+34.g6123b8c6) (1.1.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.8.6+34.g6123b8c6) (1.0.4)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.8.6+34.g6123b8c6) (2.11.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.8.6+34.g6123b8c6) (1.1.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.8.6+34.g6123b8c6) (1.0.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->BentoML==0.8.6+34.g6123b8c6) (0.4.11)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.8.6+34.g6123b8c6) (1.25.8)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.8.6+34.g6123b8c6) (2.8)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->BentoML==0.8.6+34.g6123b8c6) (0.57.0)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: docutils<0.16,>=0.10 in /opt/conda/lib/python3.7/site-packages (from botocore<1.18.0,>=1.17.48->boto3->BentoML==0.8.6+34.g6123b8c6) (0.15.2)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: typing-extensions>=3.7.4; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->BentoML==0.8.6+34.g6123b8c6) (3.7.4.3)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->BentoML==0.8.6+34.g6123b8c6) (1.1.1)\u001b[0m\n", "\u001b[39mRequirement already satisfied, skipping upgrade: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->BentoML==0.8.6+34.g6123b8c6) (3.11)\u001b[0m\n", "\u001b[39mBuilding wheels for collected packages: BentoML, sqlalchemy-utils\u001b[0m\n", "\u001b[39m Building wheel for BentoML (PEP 517): started\u001b[0m\n", "\b\\\u001b[39m Building wheel for BentoML (PEP 517): finished with status 'done'\u001b[0m\n", "\b-\u001b[39m Created wheel for BentoML: filename=BentoML-0.8.6+34.g6123b8c6-py3-none-any.whl size=4712455 sha256=e6944330cc4d24e21e20865bf3b08f780a9c2fde3fbc129cbda87c7f36aef89b\u001b[0m\n", "\u001b[39m Stored in directory: /tmp/pip-ephem-wheel-cache-n87zjgsc/wheels/be/7b/58/8207840666d87408400c426e983365fbdcb71015a400a124ea\u001b[0m\n", "\u001b[39m Building wheel for sqlalchemy-utils (setup.py): started\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\b-\u001b[39m Building wheel for sqlalchemy-utils (setup.py): finished with status 'done'\u001b[0m\n", "\u001b[39m Created wheel for sqlalchemy-utils: filename=SQLAlchemy_Utils-0.36.7-py2.py3-none-any.whl size=93226 sha256=2f7ba4b0a4e6875f0b834144d9850d2a7e7e952d19eb97b5225e8fe8849415cb\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-n87zjgsc/wheels/85/a9/d2/3377194742a9ad5eb57ee36f934438c15e1aaa0843dd41c899\u001b[0m\n", "\u001b[39mSuccessfully built BentoML sqlalchemy-utils\u001b[0m\n", "\b|\u001b[39mInstalling collected packages: sqlalchemy-utils, BentoML\n", " Attempting uninstall: sqlalchemy-utils\u001b[0m\n", "\u001b[39m Found existing installation: SQLAlchemy-Utils 0.36.8\u001b[0m\n", "\u001b[39m Uninstalling SQLAlchemy-Utils-0.36.8:\u001b[0m\n", "\b\\\u001b[39m Successfully uninstalled SQLAlchemy-Utils-0.36.8\u001b[0m\n", "\b-\u001b[39m Attempting uninstall: BentoML\u001b[0m\n", "\u001b[39m Found existing installation: BentoML 0.8.6\u001b[0m\n", "\b/\u001b[39m Uninstalling BentoML-0.8.6:\u001b[0m\n", "\b\\\u001b[39m Successfully uninstalled BentoML-0.8.6\u001b[0m\n", "\b/\u001b[39mSuccessfully installed BentoML-0.8.6+34.g6123b8c6 sqlalchemy-utils-0.36.7\u001b[0m\n", "\b|\u001b[39m ---> 0048faff09db\u001b[0m\n", "\u001b[39mStep 10/15 : ENV PORT 5000\u001b[0m\n", "\b\\\u001b[39m ---> Running in 1d42a7a4b733\u001b[0m\n", "\b-\u001b[39m ---> 26ba558b2fd9\u001b[0m\n", "\u001b[39mStep 11/15 : EXPOSE $PORT\u001b[0m\n", "\b|\u001b[39m ---> Running in fafe7833175e\u001b[0m\n", "\b\\\u001b[39m ---> e0a7df8be789\u001b[0m\n", "\u001b[39mStep 12/15 : COPY docker-entrypoint.sh /usr/local/bin/\u001b[0m\n", "\b/\u001b[39m ---> 4d4bf73560df\u001b[0m\n", "\u001b[39mStep 13/15 : RUN chmod +x /usr/local/bin/docker-entrypoint.sh\u001b[0m\n", "\b|\u001b[39m ---> Running in e29127c9dd49\u001b[0m\n", "\b-\u001b[39m ---> f27486f42c1e\u001b[0m\n", "\u001b[39mStep 14/15 : ENTRYPOINT [ \"docker-entrypoint.sh\" ]\u001b[0m\n", "\u001b[39m ---> Running in 03d8e4ff5f2a\u001b[0m\n", "\b\\\u001b[39m ---> 5b46c9c1baeb\u001b[0m\n", "\u001b[39mStep 15/15 : CMD [\"bentoml\", \"serve-gunicorn\", \"/bento\"]\u001b[0m\n", "\u001b[39m ---> Running in ad2385425be6\u001b[0m\n", "\b/\u001b[39m ---> a541a3c92322\u001b[0m\n", "\u001b[39mSuccessfully built a541a3c92322\u001b[0m\n", "\u001b[39mSuccessfully tagged irisclassifier:20200909115040_9FC7F4\u001b[0m\n", "\u001b[32mFinished building irisclassifier:20200909115040_9FC7F4 from IrisClassifier:latest\u001b[0m\n" ] } ], "source": [ "!bentoml containerize IrisClassifier:latest" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!docker run --rm -p 5000:5000 irisclassifier:20200909115040_9FC7F4 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Launch inference job from CLI\n", "\n", "BentoML cli supports loading and running a packaged model from CLI. With the DataframeInput adapter, the CLI command supports reading input Dataframe data from CLI argument or local csv or json files:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!bentoml run IrisClassifier:latest predict --input '[[5, 4, 3, 2]]'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load saved BentoService\n", "\n", "bentoml.load is the API for loading a BentoML packaged model in python:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from bentoml import load\n", "\n", "service = load(saved_path)\n", "\n", "print(service.predict([[5,4,3,2]]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Deployment Options\n", "\n", "If you are at a small team with limited engineering or DevOps resources, try out automated deployment with BentoML CLI, currently supporting AWS Lambda, AWS SageMaker, and Azure Functions:\n", "- [AWS Lambda Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_lambda.html)\n", "- [AWS SageMaker Deployment Guide](https://docs.bentoml.org/en/latest/deployment/aws_sagemaker.html)\n", "- [Azure Functions Deployment Guide](https://docs.bentoml.org/en/latest/deployment/azure_functions.html)\n", "\n", "If the cloud platform you are working with is not on the list above, try out these step-by-step guide on manually deploying BentoML packaged model to cloud platforms:\n", "- [AWS ECS Deployment](https://docs.bentoml.org/en/latest/deployment/aws_ecs.html)\n", "- [Google Cloud Run Deployment](https://docs.bentoml.org/en/latest/deployment/google_cloud_run.html)\n", "- [Azure container instance Deployment](https://docs.bentoml.org/en/latest/deployment/azure_container_instance.html)\n", "- [Heroku Deployment](https://docs.bentoml.org/en/latest/deployment/heroku.html)\n", "\n", "Lastly, if you have a DevOps or ML Engineering team who's operating a Kubernetes or OpenShift cluster, use the following guides as references for implementating your deployment strategy:\n", "- [Kubernetes Deployment](https://docs.bentoml.org/en/latest/deployment/kubernetes.html)\n", "- [Knative Deployment](https://docs.bentoml.org/en/latest/deployment/knative.html)\n", "- [Kubeflow Deployment](https://docs.bentoml.org/en/latest/deployment/kubeflow.html)\n", "- [KFServing Deployment](https://docs.bentoml.org/en/latest/deployment/kfserving.html)\n", "- [Clipper.ai Deployment Guide](https://docs.bentoml.org/en/latest/deployment/clipper.html)\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "BentoML.ipynb", "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 4 }