{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "20badcf7-c6a2-476d-95b1-eecb7207f5ac", "metadata": { "tags": [] }, "source": [ "# Using the Edge Impulse Python SDK with Weights & Biases\n", "\n", "" ] }, { "attachments": {}, "cell_type": "markdown", "id": "711a5da2-e3c6-4a4f-831f-126fa5f88943", "metadata": { "tags": [] }, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " View on edgeimpulse.com\n", " \n", " Run in Colab\n", " \n", " View source on GitHub\n", " \n", " Download notebook\n", "
" ] }, { "attachments": {}, "cell_type": "markdown", "id": "7583a486-afd6-42d8-934b-fdb33a6f3362", "metadata": { "tags": [] }, "source": [ "[Weights & Biases](https://wandb.ai/) is an online framework for helping manage machine learning training, data versioning, and experiments. When running experiments for edge-focused ML projects, it can be helpful to see the required memory (RAM and ROM) along with estimated inference times of your model for your target hardware. By viewing these metrics, you can quickly gauge if your model will fit onto your target device!\n", "\n", "Follow the code below to see how to train a simple machine learning model with different hyperparameters and log those values to the Weights & Biases dashboard.\n", "\n", "To learn more about using the Python SDK, please see: [Edge Impulse Python SDK Overview](https://docs.edgeimpulse.com/docs/edge-impulse-python-sdk/overview)" ] }, { "cell_type": "code", "execution_count": null, "id": "e0de311a-de4c-4fe1-8dd9-e7b2206217d0", "metadata": { "tags": [] }, "outputs": [], "source": [ "# If you have not done so already, install the following dependencies\n", "!python -m pip install tensorflow==2.12.0 wandb edgeimpulse" ] }, { "cell_type": "code", "execution_count": null, "id": "2f85b7c9-e76b-459e-ac37-4b348cbb5906", "metadata": { "tags": [] }, "outputs": [], "source": [ "from tensorflow import keras\n", "import wandb\n", "import edgeimpulse as ei" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8aef50a1-0a2d-4743-bfc3-de0a9755a87b", "metadata": { "tags": [] }, "source": [ "You will need to obtain an API key from an Edge Impulse project. Log into [edgeimpulse.com](https://edgeimpulse.com/) and create a new project. Open the project, navigate to **Dashboard** and click on the **Keys** tab to view your API keys. Double-click on the API key to highlight it, right-click, and select **Copy**.\n", "\n", "![Copy API key from Edge Impulse project](https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/python-sdk-copy-ei-api-key.png)\n", "\n", "Note that you do not actually need to use the project in the Edge Impulse Studio. We just need the API Key.\n", "\n", "Paste that API key string in the `ei.API_KEY` value in the following cell:" ] }, { "cell_type": "code", "execution_count": null, "id": "3429e02d-5188-4215-97c7-5a50b854b06b", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Settings %%%RESET API_KEY, project_name\n", "ei.API_KEY = \"ei_dae2...\" # Change this to your Edge Impulse API key\n", "labels = [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"]\n", "num_classes = len(labels)\n", "num_epochs = 5\n", "profile_device = 'cortex-m4f-80mhz' # Run ei.model.list_profile_devices() to see available devices\n", "deploy_filename = \"my_model_cpp.zip\"\n", "\n", "# Define experiment hyperparameters - sweep across number of nodes\n", "project_name = \"nodes-sweep\"\n", "num_nodes_sweep = [8, 16, 32, 64, 128]" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8ac24a48-8c04-433b-93ab-eadc2b6f2715", "metadata": {}, "source": [ "To use Weights and Biases, you will need to create an account on [wandb.ai](https://wandb.ai/home) and call the `wandb.login()` function. This will prompt you to log in to your account. Your credentials should be stored, which allows you to use the `wandb` package in your Python library." ] }, { "cell_type": "code", "execution_count": null, "id": "15145f5e-c0d8-4dae-a596-46667ccd739a", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Log in to Weights and Biases (will open a prompt)\n", "wandb.login()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "dd9078d0-d5a5-4a9f-96f1-0faecb4e2b1c", "metadata": { "tags": [] }, "source": [ "## Gather a dataset\n", "\n", "We want to create a classifier that can uniquely identify handwritten digits. To start, we will use TensorFlow and Keras to train a very simple convolutional neural network (CNN) on the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, which consists of handwritten digits from 0 to 9." ] }, { "cell_type": "code", "execution_count": null, "id": "0a0abd03-9473-4272-b97d-f59cefa44995", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Load MNIST data\n", "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n", "x_train = keras.utils.normalize(x_train, axis=1)\n", "x_test = keras.utils.normalize(x_test, axis=1)\n", "y_train = keras.utils.to_categorical(y_train, num_classes)\n", "y_test = keras.utils.to_categorical(y_test, num_classes)\n", "input_shape = x_train[0].shape" ] }, { "attachments": {}, "cell_type": "markdown", "id": "98f6bf5a-092e-41e3-8f9c-14a27239e5e8", "metadata": {}, "source": [ "## Create an experiment\n", "\n", "We want to vary the hyperparameters in our model and see how it affects the accuracy and predicted RAM, ROM, and inference time on our target platform. To do that, we construct a function that builds a simple model using Keras, trains the model, and computes the accuracy and loss from our holdout test set. We then use the Edge Impulse Python SDK to generate a profile of our model for our target hardware. We log the hyperparameter (number of nodes in the hidden layer), test loss, test accuracy, estimated RAM, estimated ROM, and estimated inference time (ms) to our Weights and Biases console." ] }, { "cell_type": "code", "execution_count": null, "id": "ba42755e-b13c-4e84-a016-4e4fcf4be9f6", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Define experiment - Train and test model, log metrics\n", "def do_experiment(num_nodes):\n", "\n", " # Create W&B project\n", " run = wandb.init(project=project_name,\n", " name=f\"{num_nodes}-nodes\")\n", "\n", " # Build the model (vary number of nodes in the hidden layer)\n", " model = keras.Sequential([\n", " keras.layers.Flatten(),\n", " keras.layers.Dense(num_nodes, activation='relu', input_shape=input_shape),\n", " keras.layers.Dense(num_classes, activation='softmax')\n", " ])\n", "\n", " # Compile the model\n", " model.compile(optimizer='adam',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", "\n", " # Train the model\n", " model.fit(x_train, \n", " y_train, \n", " epochs=num_epochs)\n", " \n", " # Evaluate model\n", " test_loss, test_accuracy = model.evaluate(x_test, y_test)\n", " \n", " # Profile model on target device\n", " try:\n", " profile = ei.model.profile(model=model,\n", " device=profile_device)\n", " except Exception as e:\n", " print(f\"Could not profile: {e}\")\n", "\n", " # Log metrics\n", " if profile.success:\n", " print(\"Profiling successful. Logging.\")\n", " wandb.log({\n", " 'num_nodes': num_nodes,\n", " 'test_loss': test_loss,\n", " 'test_accuracy': test_accuracy,\n", " 'profile_ram': profile.model.profile_info.float32.memory.tflite.ram,\n", " 'profile_rom': profile.model.profile_info.float32.memory.tflite.rom,\n", " 'inference_time_ms': profile.model.profile_info.float32.time_per_inference_ms\n", " })\n", " else:\n", " print(f\"Profiling unsuccessful. Error: {job_resp.error}\")\n", "\n", " # Close run\n", " wandb.finish()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f28652a3-ad96-4791-a8c5-a2692b24e933", "metadata": {}, "source": [ "## Run the experiment\n", "\n", "Now, it's time to run the experiment and log the results in Weights and Biases. Simply call our function and provide a new hyperparameter value for the number of nodes." ] }, { "cell_type": "code", "execution_count": null, "id": "f45a3c73-fbc2-477a-9f8d-04c3e10b1863", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Perform the experiments - check your dashboard in WandB!\n", "for num_nodes in num_nodes_sweep:\n", " do_experiment(num_nodes)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "eddfdb39-215c-4f09-88d6-3e9c75ef3603", "metadata": { "tags": [] }, "source": [ "Head to [wandb.ai](https://wandb.ai/) and log in (if you have not already done so). Under *My projects* on the left, click on the **nodes-sweep** project. You can visualize the results of your experiments with the various charts that Weights & Biases offers. For example, here is a [parallel coordinates plot](https://docs.wandb.ai/guides/app/features/panels/parallel-coordinates) that allows you to quickly visualize the different hyperparameters and metrics (including our new edge profile metrics).\n", "\n", "![Weights and Biases parallel coordinates plot](https://raw.githubusercontent.com/edgeimpulse/notebooks/main/.assets/images/python-sdk-wandb-parallel-plot.png)\n", "\n", "If you would like to deploy your model to your target hardware, the Python SDK can help you with that, too. See our documentation [here](https://docs.edgeimpulse.com/docs/edge-impulse-python-sdk/overview)." ] }, { "attachments": {}, "cell_type": "markdown", "id": "32257794", "metadata": {}, "source": [ "## Deploy Your Model\n", "\n", "Once you are happy with the performance of your model, you can then deploy it to your target hardware. We will assume that 32 nodes in our hidden layer provided the best trade-off of RAM, flash, inference time, and accuracy for our needs. To start, we will retrain the model:" ] }, { "cell_type": "code", "execution_count": null, "id": "dbe79c9d", "metadata": {}, "outputs": [], "source": [ "# Build the model \n", "model = keras.Sequential([\n", " keras.layers.Flatten(),\n", " keras.layers.Dense(32, activation='relu', input_shape=input_shape),\n", " keras.layers.Dense(num_classes, activation='softmax')\n", "])\n", "\n", "\n", "# Compile the model\n", "model.compile(optimizer='adam',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", "\n", "\n", "# Train the model\n", "model.fit(x_train, \n", " y_train, \n", " epochs=5)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "504723b8", "metadata": {}, "source": [ "Next, we should evaluate the model on our holdout test set." ] }, { "cell_type": "code", "execution_count": null, "id": "6f2d198b", "metadata": {}, "outputs": [], "source": [ "# Evaluate model on test set\n", "score = model.evaluate(x_test, y_test, verbose=0)\n", "print(f\"Test loss: {score[0]}\")\n", "print(f\"Test accuracy: {score[1]}\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "99615262", "metadata": {}, "source": [ "From there, we can see the available hardware targets for deployment:" ] }, { "cell_type": "code", "execution_count": null, "id": "763f9425", "metadata": {}, "outputs": [], "source": [ "# List the available profile target devices\n", "ei.model.list_deployment_targets()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "15892a52-96be-4369-89fa-e7da051b6b3c", "metadata": {}, "source": [ "You should see a list printed such as:\n", "\n", "```\n", "['zip',\n", " 'arduino',\n", " 'tinkergen',\n", " 'cubemx',\n", " 'wasm',\n", " ...\n", " 'runner-linux-aarch64-tda4vm']\n", "```\n", "\n", "The most generic target is the .zip file that holds a C++ library containing our trained model and inference runtime. To pass our labels to the C++ library, we create a Classification object, which contains our label strings.\n", "\n", "Note that instead of writing the raw bytes to a file, you can also specify an `output_directory` argument in the .deploy() function. Your deployment file(s) will be downloaded to that directory." ] }, { "cell_type": "code", "execution_count": null, "id": "580a0faf-1f16-40fa-a7d2-03ad575ade60", "metadata": {}, "outputs": [], "source": [ "# Set model information, such as your list of labels\n", "model_output_type = ei.model.output_type.Classification(labels=labels)\n", "\n", "# Create C++ library with trained model\n", "deploy_bytes = None\n", "try:\n", " \n", " deploy_bytes = ei.model.deploy(model=model,\n", " model_output_type=model_output_type,\n", " deploy_target='zip')\n", "except Exception as e:\n", " print(f\"Could not deploy: {e}\")\n", " \n", "# Write the downloaded raw bytes to a file\n", "if deploy_bytes:\n", " with open(deploy_filename, 'wb') as f:\n", " f.write(deploy_bytes)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "74b62b20-3821-4e7c-987f-076e3ecaa23e", "metadata": {}, "source": [ "Your model C++ library should be downloaded as the file *my_model_cpp.zip* in the same directory as this notebook. You are now ready to use your C++ model in your embedded and edge device application! To use the C++ model for local inference, see our documentation [here](https://docs.edgeimpulse.com/docs/deployment/running-your-impulse-locally)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" } }, "nbformat": 4, "nbformat_minor": 5 }