{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Run Inference on Multi-Region S3 Buckets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prerequisites\n", "- terraform\n", "- python\n", "- aws cli (configured)\n", "- bacalhau cli" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%cd multi-region" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Add your regions to regions.md\n", "### Note: The Bootstrap region should be first" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# aws ec2 describe-regions | jq -r '.Regions[].RegionName'\n", "# Bootstrap region must be first\n", "us-east-1\n", "us-west-2\n" ] } ], "source": [ "%cat regions.md" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"us-east-1\": {\n", " \"region\": \"us-east-1\",\n", " \"zone\": \"us-east-1a\",\n", " \"instance_ami\": \"ami-09ad94ebabb1d9472\"\n", " },\n", " \"us-west-1\": {\n", " \"region\": \"us-west-2\",\n", " \"zone\": \"us-west-2a\",\n", " \"instance_ami\": \"ami-09901fdae1bac6fe0\"\n", " }\n", "}\n" ] } ], "source": [ "#Replace the value of the name parameter with the name of your AMI\n", "%python3 get_instance_amis.py --name \"Deep Learning AMI Graviton GPU PyTorch 1.10.0 (Ubuntu 20.04) 20211122\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Replace the Values of \"locations\" with the values outputted from the above command" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"us-east-1\": {\n", " \"region\": \"us-east-1\",\n", " \"zone\": \"us-east-1a\",\n", " \"instance_ami\": \"ami-09ad94ebabb1d9472\"\n", " },\n", " \"us-west-1\": {\n", " \"region\": \"us-west-2\",\n", " \"zone\": \"us-west-2a\",\n", " \"instance_ami\": \"ami-09901fdae1bac6fe0\"\n", " }\n", "}\n", " \"app_tag\": \"bacalhau-edge-inference\",\n", " \"instance_type\": \"g5g.xlarge\",\n", " \"bacalhau_run_file\": \"bacalhau.run\",\n", " \"tailscale_key\": \"\",\n", " \"shelluser\": \"bacalhau-installer\",\n", " \"public_key\": \"~/.ssh/id_rsa.pub\",\n", " \"private_key\": \"~/.ssh/id_rsa\"\n", "}\n" ] } ], "source": [ "%cat ./tf/.env.example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Replace The values of the keys with your own key values and also the value of instance_type if you want" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "cp -r .env.example .env.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Writting the Python Inference Script\n", "\n", "For this example we use the jetson emulator SDK\n", "To install the package run\n", "```\n", " pip install jetson-emulator\n", "```\n", "\n", "In this script we will use the googlenet model to run inference on sample images\n", "```python\n", "import jetson_emulator.inference as inference\n", "import jetson_emulator.utils as utils\n", "\n", "# load the recognition network\n", "net = inference.imageNet(\"googlenet\")\n", "for x in range(1,6):\n", "\t# emulator API to generate sample images for imageNet\n", "\tfilename = net.emulatorGetImageFile() \n", "\timg = utils.loadImage(filename) \n", "\tclass_idx, confidence = net.Classify(img) \n", "\tclass_desc = net.GetClassDesc(class_idx) \n", "\tprint(\"image \"+str(x)+\" is recognized as '{:s}' (class #{:d}) with {:f}% confidence\".\n", "\tformat(class_desc, class_idx, confidence*100))\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Building the container\n", "\n", "You can use a base image of our choice and install the dependencies we need\n", "In this case we will use the balenalib/jetson-tx2-ubuntu-python for simulating a Jetson-TX2 device \n", "```Dockerfile\n", "FROM balenalib/jetson-tx2-ubuntu-python\n", "\n", "RUN pip install jetson-emulator\n", "```\n", "\n", "To Build and Push the container run this command\n", "```\n", " sudo docker buildx build --push --platform linux/amd64,linux/arm64,linux/arm/v7 -t expanso/jetson .\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deploying the infra" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "./bulk-deploy.sh create" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Running the job" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "shellscript" } }, "outputs": [], "source": [ "!bacalhau docker run --concurrency 2 -i https://gist.githubusercontent.com/js-ts/0474dc29b091dc6cf66d9061b2fd5838/raw/bef847a676f1978f5698e38103a355dce00af61e/script.py expanso/jetson -- python inputs/script.py" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Viewing the outputs " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "shellscript" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "image 1 is recognized as 'pole' (class #733) with 16.345388% confidence\n", "image 2 is recognized as 'quill, quill pen' (class #749) with 34.726549% confidence\n", "image 3 is recognized as 'earthstar' (class #995) with 40.020620% confidence\n", "image 4 is recognized as 'garden spider, Aranea diademata' (class #74) with 61.250917% confidence\n", "image 5 is recognized as 'throne' (class #857) with 24.096852% confidence\n" ] } ], "source": [ "!bacalhau logs " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }