![\"Web](\"media/ACI_diagram_2.jpg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Pre-requisites ![\"Azure](\"media/experiment.jpg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can now attach our local model to our workspace and experiment."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\n",
" ![\"Azure](\"media/uploaded_model.jpg\")
\n",
"
\n",
"\n",
"\n",
"\n",
" ![\"Azure](\"media/models.jpg\")
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also check that it is programatically accessible"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model:\n",
" --> Name: im_classif_resnet18\n",
" --> ID: im_classif_resnet18:11\n",
" --> Path:azureml-models\\im_classif_resnet18\\11\\im_classif_resnet18.pkl\n"
]
}
],
"source": [
"print(f\"Model:\\n --> Name: {model.name}\\n \\\n",
" --> ID: {model.id}\\n \\\n",
" --> Path:{model._get_model_path_remote(model.name, model.version, ws)}\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['outputs/im_classif_resnet18.pkl']"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"run.get_file_names()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we are also interested in verifying which model we uploaded, we can download it to our local machine"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'im_classif_resnet18.pkl'"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.download(exist_ok=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note: If we ran the cells in both the \"with an experiment\" and \"without experiment\" sections, we got 2 iterations of the same model registered on Azure. This is not a problem as any operation that we perform on the \"model\" object, later on, will be associated with the latest version of the model that we registered. To clean things up, we can go to the portal, select the model we do not want and click the \"Delete\" button. In general, we would register the model using only one of these 2 methods. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We are all done with our model registration, so we can close our run."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"# Close the run\n",
"run.complete()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Access the portal\n",
"run"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.C Scoring script \n",
"For the web service to return predictions on a given input image, we need to provide it with instructions on how to use the model we just registered. These instructions are stored in the scoring script.\n",
"\n",
"This script must contain two required functions, `init()` and `run(input_data)`:\n",
"- In the `init()` function, we typically load the model into a global object. This function is executed only once when the Docker container is started.\n",
"- In the `run(input_data)` function, the model is used to predict a value based on the input data. The input and output of `run` typically use JSON as serialization and de-serialization format but we are not limited to that.\n",
"\n",
"Note: The \"run()\" function here is different from the \"run\" object we created in our experiment\n",
"\n",
"This file must also be stored in the current directory."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"scoring_script = \"score.py\""
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Overwriting score.py\n"
]
}
],
"source": [
"%%writefile $scoring_script\n",
"# Copyright (c) Microsoft. All rights reserved.\n",
"# Licensed under the MIT license.\n",
"\n",
"import os\n",
"import json\n",
"\n",
"from base64 import b64decode\n",
"from io import BytesIO\n",
"\n",
"from azureml.core.model import Model\n",
"from fastai.vision import load_learner, open_image\n",
"\n",
"def init():\n",
" global model\n",
" model_path = Model.get_model_path(model_name='im_classif_resnet18')\n",
" # ! We cannot use the *model_name* variable here otherwise the execution on Azure will fail !\n",
"\n",
" model_dir_path, model_filename = os.path.split(model_path)\n",
" model = load_learner(model_dir_path, model_filename)\n",
"\n",
"\n",
"def run(raw_data):\n",
"\n",
" # Expects raw_data to be a list within a json file\n",
" result = [] \n",
" \n",
" for im_string in json.loads(raw_data)['data']:\n",
" im_bytes = b64decode(im_string)\n",
" try:\n",
" im = open_image(BytesIO(im_bytes))\n",
" pred_class, pred_idx, outputs = model.predict(im)\n",
" result.append({\"label\": str(pred_class), \"probability\": str(outputs[pred_idx].item())})\n",
" except Exception as e:\n",
" result.append({\"label\": str(e), \"probability\": ''})\n",
" return result"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.D Using `azureml.core.environment` to build the docker image and for deployment \n",
"\n",
"Azure Machine Learning provides an easy way to manage environments through [azureml.core.environment](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment?view=azure-ml-py) in the AzureML Python SDK.\n",
"\n",
"`Environments` provide a way to manage software dependencies so that controlled environments are reproducible with minimal manual configuration as you move between local and distributed cloud development environments.\n",
"\n",
"Most importantly, it allows users to encapsulate the various software settings and dependencies in a reproducible manner through the use of containerization and package management tools such as conda, a popular open-source, cross-platform, language-agnostic package management tool.\n",
"\n",
"In order to make predictions on the Azure platform, it is important to create an environment as similar as possible to the one in which the model was trained. Here, we use a fast.ai pretrained model that also requires pytorch and a few other libraries. To re-create this environment, we use a [Docker container](https://www.docker.com/resources/what-container). We configure it via a yaml file that will contain all the conda dependencies needed by the model. This yaml file is a subset of `\n",
"![\"Azure](\"media/docker_images.jpg\")
\n",
"![\"Azure](\"media/deployments.jpg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Notes on web service deployment \n",
"\n",
"As we discussed above, Azure Container Instances tend to be used to develop and test deployments. They are typically configured with CPUs, which usually suffice when the number of requests per second is not too high. When working with several instances, we can configure them further by specifically [allocating CPU resources](https://docs.microsoft.com/en-us/azure/container-instances/container-instances-container-groups#deployment) to each of them.\n",
"\n",
"For production requirements, i.e. when > 100 requests per second are expected, we recommend deploying models to Azure Kubernetes Service (AKS). It is a convenient infrastructure as it manages hosted Kubernetes environments, and makes it easy to deploy and manage containerized applications without container orchestration expertise. It also supports deployments with CPU clusters and deployments with GPU clusters.\n",
"\n",
"We will see an example of this in the [next notebook](22_deployment_on_azure_kubernetes_service.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Clean up \n",
"\n",
"Throughout the notebook, we used a workspace and Azure container instances. To get a sense of the cost we incurred, we can refer to this [calculator](https://azure.microsoft.com/en-us/pricing/calculator/). We can also navigate to the [Cost Management + Billing](https://ms.portal.azure.com/#blade/Microsoft_Azure_Billing/ModernBillingMenuBlade/Overview) pane on the portal, click on our subscription ID, and click on the Cost Analysis tab to check our credit usage.\n",
"\n",
"In order not to incur extra costs, let's delete the resources we no longer need.\n",
"\n",
"Once we have verified that our web service works well on ACI (cf. \"Next steps\" section below), we can delete it. This helps reduce [costs](https://azure.microsoft.com/en-us/pricing/details/container-instances/), since the container group we were paying for no longer exists, and allows us to keep our workspace clean."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# service.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If our goal is to continue using our workspace, we should keep it available. On the contrary, if we plan on no longer using it and its associated resources, we can delete it.\n",
"\n",
"Note: Deleting the workspace will delete all the experiments, outputs, models, Docker images, deployments, etc. that we created in that workspace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ws.delete(delete_dependent_resources=True)\n",
"# This deletes our workspace, the container registry, the account storage, Application Insights and the key vault"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Next steps \n",
"\n",
"In the [next tutorial](22_deployment_on_azure_kubernetes_service.ipynb), we will leverage the same Docker image, and deploy our model on AKS. We will then test both of our web services in the [23_aci_aks_web_service_testing.ipynb](23_aci_aks_web_service_testing.ipynb) notebook."
]
}
],
"metadata": {
"celltoolbar": "Tags",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}