{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "7583a486-afd6-42d8-934b-fdb33a6f3362", "metadata": { "tags": [] }, "source": [ "# Using the Edge Impulse Python SDK with TensorFlow and Keras\n", "\n", "" ] }, { "attachments": {}, "cell_type": "markdown", "id": "574b4630-cb17-4bab-8866-ba292e1bc2c8", "metadata": { "tags": [] }, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " View on edgeimpulse.com\n", " \n", " Run in Colab\n", " \n", " View source on GitHub\n", " \n", " Download notebook\n", "
" ] }, { "attachments": {}, "cell_type": "markdown", "id": "867045bb-14c2-41c2-a417-0426470df85b", "metadata": { "tags": [] }, "source": [ "[TensorFlow](https://www.tensorflow.org/) is an open source library for training machine learning models. [Keras](https://keras.io/) is an open source Python library that makes creating neural networks in TensorFlow much easier. We use these two libraries together to very quickly train a model to identify handwritten digits. From there, we use the Edge Impulse Python SDK library to profile the model to see how inference will perform on a target edge device. Then, we use the SDK again to convert our trained model to a C++ library that can be deployed to an edge hardware platform, such as a microcontroller.\n", "\n", "Follow the code below to see how to train a simple machine learning model and deploy it to a C++ library using Edge Impulse.\n", "\n", "To learn more about using the Python SDK, please see: [Edge Impulse Python SDK Overview](https://docs.edgeimpulse.com/docs/edge-impulse-python-sdk/overview)." ] }, { "cell_type": "code", "execution_count": null, "id": "e0de311a-de4c-4fe1-8dd9-e7b2206217d0", "metadata": {}, "outputs": [], "source": [ "# If you have not done so already, install the following dependencies\n", "!python -m pip install tensorflow==2.12.0 edgeimpulse" ] }, { "cell_type": "code", "execution_count": null, "id": "2f85b7c9-e76b-459e-ac37-4b348cbb5906", "metadata": { "tags": [] }, "outputs": [], "source": [ "from tensorflow import keras\n", "import edgeimpulse as ei" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8aef50a1-0a2d-4743-bfc3-de0a9755a87b", "metadata": { "tags": [] }, "source": [ "You will need to obtain an API key from an Edge Impulse project. Log into [edgeimpulse.com](https://edgeimpulse.com/) and create a new project. Open the project, navigate to **Dashboard** and click on the **Keys** tab to view your API keys. Double-click on the API key to highlight it, right-click, and select **Copy**.\n", "\n", "![Copy API key from Edge Impulse project](https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/python-sdk-copy-ei-api-key.png)\n", "\n", "Note that you do not actually need to use the project in the Edge Impulse Studio. We just need the API Key.\n", "\n", "Paste that API key string in the `ei.API_KEY` value in the following cell:" ] }, { "cell_type": "code", "execution_count": null, "id": "3429e02d-5188-4215-97c7-5a50b854b06b", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Settings\n", "ei.API_KEY = \"ei_dae2...\" # Change this to your Edge Impulse API key\n", "labels = [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"]\n", "num_classes = len(labels)\n", "deploy_filename = \"my_model_cpp.zip\"" ] }, { "attachments": {}, "cell_type": "markdown", "id": "dd9078d0-d5a5-4a9f-96f1-0faecb4e2b1c", "metadata": { "tags": [] }, "source": [ "## Train a machine learning model\n", "\n", "We want to create a classifier that can uniquely identify handwritten digits. To start, we will use TensorFlow and Keras to train a very simple convolutional neural network (CNN) on the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, which consists of handwritten digits from 0 to 9." ] }, { "cell_type": "code", "execution_count": null, "id": "0a0abd03-9473-4272-b97d-f59cefa44995", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Load MNIST data\n", "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n", "x_train = keras.utils.normalize(x_train, axis=1)\n", "x_test = keras.utils.normalize(x_test, axis=1)\n", "y_train = keras.utils.to_categorical(y_train, num_classes)\n", "y_test = keras.utils.to_categorical(y_test, num_classes)\n", "input_shape = x_train[0].shape" ] }, { "cell_type": "code", "execution_count": null, "id": "ba42755e-b13c-4e84-a016-4e4fcf4be9f6", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Build the model \n", "model = keras.Sequential([\n", " keras.layers.Flatten(),\n", " keras.layers.Dense(32, activation='relu', input_shape=input_shape),\n", " keras.layers.Dense(num_classes, activation='softmax')\n", "])\n", "\n", "# Compile the model\n", "model.compile(optimizer='adam',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "code", "execution_count": null, "id": "d6ddf625-43c9-40da-9c44-7fbbbea8b572", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Train the model\n", "model.fit(x_train, \n", " y_train, \n", " epochs=5)" ] }, { "cell_type": "code", "execution_count": null, "id": "dc2b0ab6-e2d0-4448-9545-4870e5b2d101", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Evaluate model on test set\n", "score = model.evaluate(x_test, y_test, verbose=0)\n", "print(f\"Test loss: {score[0]}\")\n", "print(f\"Test accuracy: {score[1]}\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "4e04b239-80c6-43e0-87e0-e3a41b8457db", "metadata": { "tags": [] }, "source": [ "## Profile your model\n", "\n", "To start, we need to list the possible target devices we can use for profiling. We need to pick from this list." ] }, { "cell_type": "code", "execution_count": null, "id": "1b622df2-9745-4c93-969f-35de2ff10df6", "metadata": { "tags": [] }, "outputs": [], "source": [ "# List the available profile target devices\n", "ei.model.list_profile_devices()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "eaaaef69-1ba2-4ffd-bbe6-88ca299c8ee7", "metadata": { "tags": [] }, "source": [ "You should see a list printed such as:\n", "\n", "```\n", "['alif-he',\n", " 'alif-hp',\n", " 'arduino-nano-33-ble',\n", " 'arduino-nicla-vision',\n", " 'portenta-h7',\n", " 'brainchip-akd1000',\n", " 'cortex-m4f-80mhz',\n", " 'cortex-m7-216mhz',\n", " ...\n", " 'ti-tda4vm']\n", "```\n", "\n", "A common option is the `cortex-m4f-80mhz`, as this is a relatively low-power microcontroller family. From there, we can use the Edge Impulse Python SDK to generate a profile for your model to ensure it fits on your target hardware and meets your timing requirements." ] }, { "cell_type": "code", "execution_count": null, "id": "3c16e2ab-0c50-483e-8d2e-01c2dd48be23", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Estimate the RAM, ROM, and inference time for our model on the target hardware family\n", "try:\n", " profile = ei.model.profile(model=model,\n", " device='cortex-m4f-80mhz')\n", " print(profile.summary())\n", "except Exception as e:\n", " print(f\"Could not profile: {e}\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "29e4bc88-032a-4d8a-8b3a-4cbf2ec5d778", "metadata": {}, "source": [ "## Deploy your model\n", "\n", "Once you are happy with the performance of the model, you can deploy it to a number of possible hardware targets. To see the available hardware targets, run the following:" ] }, { "cell_type": "code", "execution_count": null, "id": "ee30330d-6ff2-4733-8c37-8d90e3da6d78", "metadata": { "tags": [] }, "outputs": [], "source": [ "# List the available profile target devices\n", "ei.model.list_deployment_targets()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "418ee00a-5714-4cd9-a880-2bdf567d72e5", "metadata": {}, "source": [ "You should see a list printed such as:\n", "\n", "```\n", "['zip',\n", " 'arduino',\n", " 'tinkergen',\n", " 'cubemx',\n", " 'wasm',\n", " ...\n", " 'runner-linux-aarch64-tda4vm']\n", "```\n", "\n", "The most generic target is to download a .zip file that holds a C++ library containing the inference runtime and your trained model, so we choose `'zip'` from the above list. To do that, we first need to create a Classification object which contains our label strings (and other optional information about the model). These strings will be added to the C++ library metadata so you can access them in your edge application.\n", "\n", "Note that instead of writing the raw bytes to a file, you can also specify an `output_directory` argument in the `.deploy()` function. Your deployment file(s) will be downloaded to that directory.\n", "\n", "**Important!** The deployment targets list will change depending on the values provided for `model`, `model_output_type`, and `model_input_type` in the next part. For example, you will not see `openmv` listed once you upload a model (e.g. using `.profile()` or `.deploy()`) if `model_input_type` is not set to `ei.model.input_type.ImageInput()`. If you attempt to deploy to an unavailable target, you will receive the error `Could not deploy: deploy_target: ...`. If `model_input_type` is not provided, it will default to [OtherInput](https://edgeimpulse.github.io/python-sdk/source/edgeimpulse.model.input_type.html#edgeimpulse.model.input_type.OtherInput). See [this page](https://edgeimpulse.github.io/python-sdk/source/edgeimpulse.model.input_type.html) for more information about input types." ] }, { "cell_type": "code", "execution_count": null, "id": "f45a3c73-fbc2-477a-9f8d-04c3e10b1863", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Set model information, such as your list of labels\n", "model_output_type = ei.model.output_type.Classification(labels=labels)\n", "\n", "# Set model input type\n", "model_input_type = ei.model.input_type.OtherInput()\n", "\n", "# Create C++ library with trained model\n", "deploy_bytes = None\n", "try:\n", " \n", " deploy_bytes = ei.model.deploy(model=model,\n", " model_output_type=model_output_type,\n", " model_input_type=model_input_type,\n", " deploy_target='zip')\n", "except Exception as e:\n", " print(f\"Could not deploy: {e}\")\n", " \n", "# Write the downloaded raw bytes to a file\n", "if deploy_bytes:\n", " with open(deploy_filename, 'wb') as f:\n", " f.write(deploy_bytes)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "eddfdb39-215c-4f09-88d6-3e9c75ef3603", "metadata": { "tags": [] }, "source": [ "Your model C++ library should be downloaded as the file *my_model_cpp.zip* in the same directory as this notebook. You are now ready to use your C++ model in your embedded and edge device application! To use the C++ model for local inference, see our documentation [here](https://docs.edgeimpulse.com/docs/deployment/running-your-impulse-locally)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" } }, "nbformat": 4, "nbformat_minor": 5 }