{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "76571396-8f54-40ed-9e81-6c7531e6eaee",
   "metadata": {
    "id": "76571396-8f54-40ed-9e81-6c7531e6eaee"
   },
   "source": [
    "# Efficiently run SetFit Models with 🤗 Optimum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24fd5853-812f-45a4-8a7b-0a0c9a60d0a2",
   "metadata": {
    "id": "24fd5853-812f-45a4-8a7b-0a0c9a60d0a2"
   },
   "source": [
    "[SetFit](https://github.com/huggingface/setfit) is a technique for few-shot text classification that uses contrastive learning to fine-tune Sentence Transformers in domains where little to no labeled data is available. It achieves comparable performance to existing state-of-the-art methods based on large language models, yet requires no prompts and is efficient to train (typically a few seconds on a GPU to minutes on a CPU).\n",
    "\n",
    "In this notebook you'll learn how to further compress SetFit models for faster inference & deployment on GPU using Optimum Onnx."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3b30b35-7875-498f-a771-068132f4084f",
   "metadata": {
    "id": "a3b30b35-7875-498f-a771-068132f4084f",
    "tags": []
   },
   "source": [
    "## 1. Setup development environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc40c7af-1f4f-4324-847c-dc9b7797b60c",
   "metadata": {
    "id": "dc40c7af-1f4f-4324-847c-dc9b7797b60c"
   },
   "source": [
    "Our first step is to install SetFit. Running the following cell will install all the required packages for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "Cu9et-iSaU0i",
   "metadata": {
    "id": "Cu9et-iSaU0i"
   },
   "outputs": [],
   "source": [
    "!pip install setfit accelerate -qqq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffbea843-2e86-4f14-961c-b8895f9de77d",
   "metadata": {
    "id": "ffbea843-2e86-4f14-961c-b8895f9de77d"
   },
   "source": [
    "## 2. Create a performance benchmark\n",
    "\n",
    "Before we train and optimize any models, let's define a performance benchmark that we can use to compare our models. In general, deploying ML models in production environments involves a tradeoff among several constraints:\n",
    "\n",
    "* Model performance: how well does the model perform on a well crafted test set?\n",
    "* Latency: how fast can our model deliver predictions?\n",
    "* Memory: on what cloud instance or device can we store and load our model?\n",
    "\n",
    "The class below defines a simple benchmark that measure each quantity for a given SetFit model and test dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "55756fec-fc22-4590-84d7-2f3df37b9256",
   "metadata": {
    "id": "55756fec-fc22-4590-84d7-2f3df37b9256"
   },
   "outputs": [],
   "source": [
    "!export TOKENIZERS_PARALLELISM=false\n",
    "\n",
    "from pathlib import Path\n",
    "from time import perf_counter\n",
    "\n",
    "import evaluate\n",
    "import numpy as np\n",
    "import torch\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "metric = evaluate.load(\"accuracy\")\n",
    "\n",
    "class PerformanceBenchmark:\n",
    "    def __init__(self, model, dataset, optim_type):\n",
    "        self.model = model\n",
    "        self.dataset = dataset\n",
    "        self.optim_type = optim_type\n",
    "\n",
    "    def compute_accuracy(self):\n",
    "        preds = self.model.predict(self.dataset[\"text\"])\n",
    "        labels = self.dataset[\"label\"]\n",
    "        accuracy = metric.compute(predictions=preds, references=labels)\n",
    "        print(f\"Accuracy on test set - {accuracy['accuracy']:.3f}\")\n",
    "        return accuracy\n",
    "\n",
    "    def compute_size(self):\n",
    "        state_dict = self.model.model_body.state_dict()\n",
    "        tmp_path = Path(\"model.pt\")\n",
    "        torch.save(state_dict, tmp_path)\n",
    "        # Calculate size in megabytes\n",
    "        size_mb = Path(tmp_path).stat().st_size / (1024 * 1024)\n",
    "        # Delete temporary file\n",
    "        tmp_path.unlink()\n",
    "        print(f\"Model size (MB) - {size_mb:.2f}\")\n",
    "        return {\"size_mb\": size_mb}\n",
    "\n",
    "    def time_model(self, query=\"that loves its characters and communicates something rather beautiful about human nature\"):\n",
    "        latencies = []\n",
    "        # Warmup\n",
    "        for _ in range(10):\n",
    "            _ = self.model([query])\n",
    "        # Timed run\n",
    "        for _ in range(100):\n",
    "            start_time = perf_counter()\n",
    "            _ = self.model([query])\n",
    "            latency = perf_counter() - start_time\n",
    "            latencies.append(latency)\n",
    "        # Compute run statistics\n",
    "        time_avg_ms = 1000 * np.mean(latencies)\n",
    "        time_std_ms = 1000 * np.std(latencies)\n",
    "        print(rf\"Average latency (ms) - {time_avg_ms:.2f} +\\- {time_std_ms:.2f}\")\n",
    "        return {\"time_avg_ms\": time_avg_ms, \"time_std_ms\": time_std_ms}\n",
    "\n",
    "    def run_benchmark(self):\n",
    "        metrics = {}\n",
    "        metrics[self.optim_type] = self.compute_size()\n",
    "        metrics[self.optim_type].update(self.compute_accuracy())\n",
    "        metrics[self.optim_type].update(self.time_model())\n",
    "        return metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4QkMcqR0qcBZ",
   "metadata": {
    "id": "4QkMcqR0qcBZ"
   },
   "source": [
    "Beyond that, we'll create a simple function to plot the performances reported by this benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "PurksLh3qcBa",
   "metadata": {
    "id": "PurksLh3qcBa"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "\n",
    "def plot_metrics(perf_metrics):\n",
    "    df = pd.DataFrame.from_dict(perf_metrics, orient=\"index\")\n",
    "\n",
    "    for idx in df.index:\n",
    "        df_opt = df.loc[idx]\n",
    "        plt.errorbar(\n",
    "            df_opt[\"time_avg_ms\"],\n",
    "            df_opt[\"accuracy\"] * 100,\n",
    "            xerr=df_opt[\"time_std_ms\"],\n",
    "            fmt=\"o\",\n",
    "            alpha=0.5,\n",
    "            ms=df_opt[\"size_mb\"] / 15,\n",
    "            label=idx,\n",
    "            capsize=5,\n",
    "            capthick=1,\n",
    "        )\n",
    "\n",
    "    legend = plt.legend(loc=\"lower right\")\n",
    "\n",
    "    plt.ylim(63, 95)\n",
    "    # Use the slowest model to define the x-axis range\n",
    "    xlim = max([metrics[\"time_avg_ms\"] for metrics in perf_metrics.values()]) * 1.2\n",
    "    plt.xlim(0, xlim)\n",
    "    plt.ylabel(\"Accuracy (%)\")\n",
    "    plt.xlabel(\"Average latency with batch_size=1 (ms)\")\n",
    "    plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1402c1ba-aa7f-4b0b-9db5-1e6f0d301e70",
   "metadata": {
    "id": "1402c1ba-aa7f-4b0b-9db5-1e6f0d301e70"
   },
   "source": [
    "## 3. Train/evaluate bge-small SetFit models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a850dfb-fffb-4e03-b468-b1f78d434705",
   "metadata": {
    "id": "5a850dfb-fffb-4e03-b468-b1f78d434705"
   },
   "source": [
    "Before we optimize any models, let's train a few baselines as a point of reference. We'll use the [sst-2](https://huggingface.co/datasets/SetFit/sst2) dataset, which is a collection of sentiment text catagorized into 2 classes: positive, negative\n",
    "\n",
    "Let's start by loading the dataset from the Hub:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7850d846-07c8-48eb-9aa6-2ce1af276ff4",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7850d846-07c8-48eb-9aa6-2ce1af276ff4",
    "outputId": "056eff7c-293f-4fd8-e76e-29d261abff1b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/dkorat/anaconda/envs/dk-setfit-py310/lib/python3.10/site-packages/huggingface_hub/repocard.py:105: UserWarning: Repo card metadata block was not found. Setting CardData to empty.\n",
      "  warnings.warn(\"Repo card metadata block was not found. Setting CardData to empty.\")\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['text', 'label', 'label_text'],\n",
       "        num_rows: 6920\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['text', 'label', 'label_text'],\n",
       "        num_rows: 872\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['text', 'label', 'label_text'],\n",
       "        num_rows: 1821\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "dataset = load_dataset(\"SetFit/sst2\")\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a714cf95-a831-41b7-8f04-ffe0350d4659",
   "metadata": {
    "id": "a714cf95-a831-41b7-8f04-ffe0350d4659"
   },
   "source": [
    "We train a SetFit model with the full dataset. Recall that SetFit excels with few-shot scenario, but this time we are interested to achieve maximum accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a628cbfa-cfcd-4e4f-ab48-b454e5695ac8",
   "metadata": {
    "id": "a628cbfa-cfcd-4e4f-ab48-b454e5695ac8"
   },
   "outputs": [],
   "source": [
    "train_dataset = dataset[\"train\"]\n",
    "test_dataset = dataset[\"validation\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "P8yY_SBbWlv9",
   "metadata": {
    "id": "P8yY_SBbWlv9"
   },
   "source": [
    "Use the following line code to download the [already finetuned model](https://huggingface.co/moshew/bge-small-en-v1.5_setfit-sst2-english) and evaluate. Alternatively, uncomment the code below it to fine-tune the base model from scratch.\n",
    "\n",
    "Note that we perform the evaluations on Google Colab using the free T4 GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "u-w99Y2qW4lU",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u-w99Y2qW4lU",
    "outputId": "57f0b8f7-6dad-4e90-c779-658a7de6e960"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model size (MB) - 127.33\n",
      "Accuracy on test set - 0.884\n",
      "Average latency (ms) - 8.31 +\\- 0.31\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the uploaded model!\n",
    "from setfit import SetFitModel\n",
    "\n",
    "model = SetFitModel.from_pretrained(\"dkorat/bge-small-en-v1.5_setfit-sst2-english\")\n",
    "pb = PerformanceBenchmark(model=model, dataset=test_dataset, optim_type=\"bge-small (PyTorch)\")\n",
    "perf_metrics = pb.run_benchmark()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "52b3bc70-dc3e-4c23-a152-7a149b8b46fe",
   "metadata": {
    "id": "52b3bc70-dc3e-4c23-a152-7a149b8b46fe"
   },
   "outputs": [],
   "source": [
    "# # Fine-tune the base model and Evaluate!\n",
    "# from setfit import SetFitModel, Trainer, TrainingArguments\n",
    "\n",
    "# # Load pretrained model from the Hub\n",
    "# model = SetFitModel.from_pretrained(\n",
    "#    \"BAAI/bge-small-en-v1.5\"\n",
    "# )\n",
    "# args = TrainingArguments(num_iterations=20)\n",
    "\n",
    "# # Create trainer\n",
    "# small_trainer = Trainer(\n",
    "#    model=model, args=args, train_dataset=train_dataset\n",
    "# )\n",
    "# # Train!\n",
    "# small_trainer.train()\n",
    "\n",
    "# # Evaluate!\n",
    "# pb = PerformanceBenchmark(\n",
    "#    model=small_trainer.model, dataset=test_dataset, optim_type=\"bge-small (base)\"\n",
    "# )\n",
    "# perf_metrics = pb.run_benchmark()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82bf3e15-2804-4669-97d6-87e1bbef7223",
   "metadata": {
    "id": "82bf3e15-2804-4669-97d6-87e1bbef7223"
   },
   "source": [
    "Let's plot the results to visualise the performance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "89d0a144-d463-4a61-b78a-861d0d8cd061",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 455
    },
    "id": "89d0a144-d463-4a61-b78a-861d0d8cd061",
    "outputId": "9ba81223-b2d2-4b10-f78d-2691846782a2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAG2CAYAAABlBWwKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIjUlEQVR4nO3deVhUZf8/8PcwwDCsIshWbIK7uOKC9qgliqSmSZpLKeIe5VLqo/ZYlgtpuXzVHs0y3Jc21zIVCpdywQXUUBQkcAHJBQYEBpi5f3/4cx5HFgHBmYPv13Wd62rOuc99PmdmYt6ec59zZEIIASIiIiIJMjF0AURERERVxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSZdAgk5OTg8mTJ8PT0xNKpRKdOnVCbGysbnloaChkMpne1KtXLwNWTERERMbE1JAbHz16NC5cuICNGzfCzc0NmzZtQmBgIBISEvDCCy8AAHr16oXIyEjdOgqFwlDlEhERkZGRGeqhkfn5+bCxscGuXbvQu3dv3fy2bdsiODgY8+bNQ2hoKLKysrBz505DlEhERERGzmBHZIqLi6HRaGBhYaE3X6lU4ujRo7rXMTExcHJygr29PV555RXMmzcPDg4OZfarVquhVqt1r7VaLe7evQsHBwfIZLLq3xEiIiKqdkII5OTkwM3NDSYm5YyEEQYUEBAgunbtKm7cuCGKi4vFxo0bhYmJiWjYsKEQQoitW7eKXbt2iXPnzokdO3aIJk2aiHbt2oni4uIy+/z4448FAE6cOHHixIlTLZiuXbtWbpYw2KklAEhOTkZYWBgOHz4MuVyONm3aoGHDhjh9+jQuXrxYov3Vq1fh4+ODqKgodO/evdQ+Hz8ik52dDQ8PD1y7dg22trY1ti9ERERUfVQqFdzd3ZGVlQU7O7sy2xl0sK+Pjw8OHTqE+/fvQ6VSwdXVFW+++Sbq169favv69evD0dERSUlJZQYZhUJR6oBgW1tbBhkiIiKJedKwEKO4j4yVlRVcXV1x79497N+/H/369Su13fXr13Hnzh24uro+4wqJiIjIGBn0iMz+/fshhECjRo2QlJSEadOmoXHjxhg5ciRyc3PxySefICQkBC4uLkhOTsb06dPh6+uLoKAgQ5ZNRERERsKgR2Sys7MRHh6Oxo0bY/jw4XjppZewf/9+mJmZQS6X49y5c3jttdfQsGFDjBo1Cm3btsWRI0d4LxkiIiICYMD7yDwrKpUKdnZ2yM7O5hgZIiIiiajo77dRjJEhIiIiqgoGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsgwaZnJwcTJ48GZ6enlAqlejUqRNiY2N1y4UQ+Oijj+Dq6gqlUonAwEBcuXLFgBUTERGRMTFokBk9ejQOHjyIjRs34vz58+jZsycCAwNx48YNAMCiRYuwfPlyrF69GidOnICVlRWCgoJQUFBgyLKJiIjISMiEEMIQG87Pz4eNjQ127dqF3r176+a3bdsWwcHBmDt3Ltzc3PDBBx9g6tSpAIDs7Gw4Oztj3bp1GDx4cIW2o1KpYGdnh+zsbNja2tbIvhAREVH1qujvt8GOyBQXF0Oj0cDCwkJvvlKpxNGjR5GSkoKMjAwEBgbqltnZ2aFDhw44duxYmf2q1WqoVCq9iYiIiGongwUZGxsbBAQEYO7cubh58yY0Gg02bdqEY8eOIT09HRkZGQAAZ2dnvfWcnZ11y0oTEREBOzs73eTu7l6j+0FERESGY9AxMhs3boQQAi+88AIUCgWWL1+OIUOGwMSk6mXNnDkT2dnZuunatWvVWDEREREZE4MGGR8fHxw6dAi5ubm4du0aTp48iaKiItSvXx8uLi4AgFu3bumtc+vWLd2y0igUCtja2upNREREVDsZxX1krKys4Orqinv37mH//v3o168fvL294eLigujoaF07lUqFEydOICAgwIDVEhERkbEwNeTG9+/fDyEEGjVqhKSkJEybNg2NGzfGyJEjIZPJMHnyZMybNw8NGjSAt7c3Zs+eDTc3N/Tv39+QZRMREZGRMGiQyc7OxsyZM3H9+nXUrVsXISEhmD9/PszMzAAA06dPx/379zF27FhkZWXhpZdewq+//lriSiciIiJ6PhnsPjLPCu8jQ0REJD1Gfx8ZIiIioqfFIENERESSxSBDREREksUgQ0RERJLFIENERPSIXHUxjiXfQa66uNLrFmu0yC/UoFijfebbfl4Z9PJrIiIiY5OnLsbxq3fgU88K1oon/0xqtQJXb9/H2bR7OHc9G8VaLUxNTNDiRTu09rBHfUcrmJjIamTbxCBDRERUZepiDXbH3cQfSbdRUKSFnaUZzOUmKNRoEX0xE38k3UZnX0e81soNClO5ocutlRhkiIiIqkCrFdgddxPRFzPhbKuAp4O53nIXWwtk5RUi+mImACCkzYsVPjJDFccxMkRERFVw9fZ9/JF0G862CtSxNC+1TR1LczjbKvBH0m2k3Ln/jCt8PvCIDBER0WM0WoFCjRaFxWUP2o39+y7uqzV40d4SGm3ZN8m3sTBDenYBYlPuwt3estztFlZxkPDzjEGGiIjoMbF/30VBkQZWZQy41WoFDl3+BxohoCooemJ/uepibD2Zhhv38ss9vXRfXVzmNql0PLVERERUSRohoBUCclnFxrzIZTJotAKa2v14Q4Ng7CMiInpMO6+6GNzeHU42FqUuL9ZokZ1fhMJiLZxtS2/zqFuqAihMTfDuy74wlZd9DCEzpwDfn7pe5bqfRwwyREREj5GbyGAuN4G5aemhw9zUBK3c6yDq4i241VE+sb+cgmJ0aOoEyyecNjIvJ+RQ6fiOERERVUFrD3sozeTIyisst11WXiEszEzQ2sP+GVX2fGGQISIiqoL6jlbo7OuIWyp1mWEmK68Qt1RqdPZ1hLeD1TOu8PnAU0tERERVYGIiw2ut3AAAfyTdxi2VWu/Ovtl5RbAwM0H3Jk54rZUbb4ZXQxhkiIiIqkhhKkdImxfRxtMeZ9Pu4fz1bBRptLAwNUG7pk5o7WEPb4eKP2uJKo9BhoiI6BGWClN0rO/wxIG5D5mYyOBTzxo+9azRv9ULKNIImMll5V6dVF3bJgYZIiIiPdYKUwT4OFRpXVO5CZ7m2ZBPs+3nFQf7EhERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERkWQxyBAREZFkGTTIaDQazJ49G97e3lAqlfDx8cHcuXMhhNC1CQ0NhUwm05t69eplwKqJiIjIWJgacuMLFy7EqlWrsH79ejRr1gynTp3CyJEjYWdnh4kTJ+ra9erVC5GRkbrXCoXCEOUSERGRkTFokPnzzz/Rr18/9O7dGwDg5eWFrVu34uTJk3rtFAoFXFxcDFEiERERGTGDnlrq1KkToqOjcfnyZQBAfHw8jh49iuDgYL12MTExcHJyQqNGjTBhwgTcuXOnzD7VajVUKpXeRERERLWTQY/IzJgxAyqVCo0bN4ZcLodGo8H8+fMxbNgwXZtevXphwIAB8Pb2RnJyMmbNmoXg4GAcO3YMcrm8RJ8RERH45JNPnuVuEBERkYHIxKMja5+xbdu2Ydq0afj888/RrFkzxMXFYfLkyViyZAlGjBhR6jpXr16Fj48PoqKi0L179xLL1Wo11Gq17rVKpYK7uzuys7Nha2tbY/tCRERE1UelUsHOzu6Jv98GPSIzbdo0zJgxA4MHDwYA+Pn5ITU1FREREWUGmfr168PR0RFJSUmlBhmFQsHBwERERM8Jg46RycvLg4mJfglyuRxarbbMda5fv447d+7A1dW1pssjIiIiI2fQIzJ9+/bF/Pnz4eHhgWbNmuHs2bNYsmQJwsLCAAC5ubn45JNPEBISAhcXFyQnJ2P69Onw9fVFUFCQIUsnIiIiI2DQMTI5OTmYPXs2duzYgczMTLi5uWHIkCH46KOPYG5ujvz8fPTv3x9nz55FVlYW3Nzc0LNnT8ydOxfOzs4V2kZFz7ERERGR8ajo77dBg8yzwCBDREQkPRX9/eazloiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyGGSIiIhIshhkiIiISLIYZIiIiEiyTCvTWKvV4tChQzhy5AhSU1ORl5eHevXqoXXr1ggMDIS7u3tN1UlERERUQoWOyOTn52PevHlwd3fHq6++in379iErKwtyuRxJSUn4+OOP4e3tjVdffRXHjx+v6ZqJiIiIAFTwiEzDhg0REBCAr7/+Gj169ICZmVmJNqmpqdiyZQsGDx6MDz/8EGPGjKn2YomIiIgeJRNCiCc1unjxIpo0aVKhDouKipCWlgYfH5+nLq46qFQq2NnZITs7G7a2toYuh4iIiCqgor/fFTq1VNEQAwBmZmZGE2KIiIiodqvUYN9HFRcX46uvvkJMTAw0Gg06d+6M8PBwWFhYVGd9RERERGWqcpCZOHEiLl++jAEDBqCoqAgbNmzAqVOnsHXr1uqsj4iIiKhMFQ4yO3bswOuvv657feDAASQmJkIulwMAgoKC0LFjx+qvkIiIiKgMFb4h3rfffov+/fvj5s2bAIA2bdpg/Pjx+PXXX7Fnzx5Mnz4d7dq1q7FCiYiIiB5X4SCzZ88eDBkyBN26dcOKFSuwZs0a2Nra4sMPP8Ts2bPh7u6OLVu21GStRERERHoqdPn1o7KysjB9+nTEx8dj9erVaN26dU3VVi14+TUREZH0VOvl14+qU6cO1qxZg88//xzDhw/HtGnTUFBQ8FTFEhEREVVFhYNMWloaBg0aBD8/PwwbNgwNGjTA6dOnYWlpiZYtW2Lfvn01WScRERFRCRU+tdStWze4uLggNDQU+/fvR3JyMnbv3g3gwZ1/x40bBxcXF3z33Xc1WnBl8dQSERGR9FT097vCl1+fOnUK8fHx8PHxQVBQELy9vXXLmjRpgsOHD2PNmjVPVzURERFRJVQ4yLRt2xYfffQRRowYgaioKPj5+ZVoM3bs2GotjoiIiKg8FR4js2HDBqjVakyZMgU3btzAV199VZN1ERERET1RhY/IeHp64ocffqjJWoiIiIgqpUJHZO7fv1+pTivbnoiIiKgqKhRkfH198dlnnyE9Pb3MNkIIHDx4EMHBwVi+fHm1FUhERERUlgqdWoqJicGsWbMwZ84ctGzZEv7+/nBzc4OFhQXu3buHhIQEHDt2DKamppg5cybGjRtX03UTERERVe4RBWlpafj+++9x5MgRpKamIj8/H46OjmjdujWCgoIQHBysexq2seB9ZIiIiKSnor/flX7WktQwyBAREUlPjT1riYiIiMhYMMgQERGRZDHIEBERkWQxyBAREZFkMcgQERGRZFU6yHh5eeHTTz9FWlraU29co9Fg9uzZ8Pb2hlKphI+PD+bOnYtHL6QSQuCjjz6Cq6srlEolAgMDceXKlafeNhEREUlfpYPM5MmT8dNPP6F+/fro0aMHtm3bBrVaXaWNL1y4EKtWrcLKlStx8eJFLFy4EIsWLcKKFSt0bRYtWoTly5dj9erVOHHiBKysrBAUFISCgoIqbZOIiIhqjyrfR+bMmTNYt24dtm7dCo1Gg6FDhyIsLAxt2rSpcB99+vSBs7Mz1q5dq5sXEhICpVKJTZs2QQgBNzc3fPDBB5g6dSoAIDs7G87Ozli3bh0GDx78xG3wPjJERETSU+P3kWnTpg2WL1+Omzdv4uOPP8Y333yDdu3aoVWrVvj2229RkXzUqVMnREdH4/LlywCA+Ph4HD16FMHBwQCAlJQUZGRkIDAwULeOnZ0dOnTogGPHjpXap1qthkql0puIiIiodqrQs5ZKU1RUhB07diAyMhIHDx5Ex44dMWrUKFy/fh2zZs1CVFQUtmzZUm4fM2bMgEqlQuPGjSGXy6HRaDB//nwMGzYMAJCRkQEAcHZ21lvP2dlZt+xxERER+OSTT6q6W0RERCQhlQ4yZ86cQWRkJLZu3QoTExMMHz4cS5cuRePGjXVtXn/9dbRr1+6JfX333XfYvHkztmzZgmbNmiEuLg6TJ0+Gm5sbRowYUdnSAAAzZ87E+++/r3utUqng7u5epb6IiIjIuFU6yLRr1w49evTAqlWr0L9/f5iZmZVo4+3tXaHxK9OmTcOMGTN0bf38/JCamoqIiAiMGDECLi4uAIBbt27B1dVVt96tW7fQqlWrUvtUKBRQKBSV3S0iIiKSoEoHmatXr8LT07PcNlZWVoiMjHxiX3l5eTAx0R+mI5fLodVqATwIRC4uLoiOjtYFF5VKhRMnTmDChAmVLZ2IiIhqmUoHmczMTGRkZKBDhw5680+cOAG5XA5/f/8K99W3b1/Mnz8fHh4eaNasGc6ePYslS5YgLCwMACCTyTB58mTMmzcPDRo0gLe3N2bPng03Nzf079+/sqUTERFRLVPpq5bCw8Nx7dq1EvNv3LiB8PDwSvW1YsUKvPHGG3jnnXfQpEkTTJ06FePGjcPcuXN1baZPn4733nsPY8eORbt27ZCbm4tff/0VFhYWlS2diIiIaplK30fG2toa586dQ/369fXmp6SkoEWLFsjJyanWAp8W7yNDREQkPTV2HxmFQoFbt26VmJ+eng5T0ypfzU1ERERUaZUOMj179sTMmTORnZ2tm5eVlYVZs2ahR48e1VocERERUXkqfQjliy++QJcuXeDp6YnWrVsDAOLi4uDs7IyNGzdWe4FEREREZal0kHnhhRdw7tw5bN68GfHx8VAqlRg5ciSGDBlS6j1liIiIiGpKlQa1WFlZYezYsdVdCxEREVGlVHl0bkJCAtLS0lBYWKg3/7XXXnvqooiIiIgqokp39n399ddx/vx5yGQy3VOuZTIZAECj0VRvhURERERlqPRVS5MmTYK3tzcyMzNhaWmJv/76C4cPH4a/vz9iYmJqoEQiIiKi0lX6iMyxY8fw22+/wdHRESYmJjAxMcFLL72EiIgITJw4EWfPnq2JOomIiIhKqPQRGY1GAxsbGwCAo6Mjbt68CQDw9PREYmJi9VZHREREVI5KH5Fp3rw54uPj4e3tjQ4dOmDRokUwNzfHmjVrSjy2gIiIiKgmVTrI/Oc//8H9+/cBAJ9++in69OmDf/3rX3BwcMD27durvUAiIiKislT6oZGluXv3Luzt7XVXLhkTPjSSiIhIemrkoZFFRUUwNTXFhQsX9ObXrVvXKEMMERER1W6VCjJmZmbw8PDgvWKIiIjIKFT6qqUPP/wQs2bNwt27d2uiHiIiIqIKq/Rg35UrVyIpKQlubm7w9PSElZWV3vIzZ85UW3FERERE5al0kOnfv38NlEFERERUedVy1ZIx41VLRERE0lMjVy0RERERGZNKn1oyMTEp91JrXtFEREREz0qlg8yOHTv0XhcVFeHs2bNYv349Pvnkk2orjIiIiOhJqm2MzJYtW7B9+3bs2rWrOrqrNhwjQ0REJD3PfIxMx44dER0dXV3dERERET1RtQSZ/Px8LF++HC+88EJ1dEdERERUIZUeI/P4wyGFEMjJyYGlpSU2bdpUrcURERERlafSQWbp0qV6QcbExAT16tVDhw4dYG9vX63FEREREZWn0kEmNDS0BsogIiIiqrxKj5GJjIzE999/X2L+999/j/Xr11dLUUREREQVUekgExERAUdHxxLznZycsGDBgmopioiIiKgiKh1k0tLS4O3tXWK+p6cn0tLSqqUoIiIiooqodJBxcnLCuXPnSsyPj4+Hg4NDtRRFREREVBGVDjJDhgzBxIkT8fvvv0Oj0UCj0eC3337DpEmTMHjw4JqokYiIiKhUlb5qae7cufj777/RvXt3mJo+WF2r1WL48OEcI0NERETPVJWftXTlyhXExcVBqVTCz88Pnp6e1V1bteCzloiIiKSnor/flT4i81CDBg3QoEGDqq5ORERE9NQqPUYmJCQECxcuLDF/0aJFGDhwYLUURURERFQRlQ4yhw8fxquvvlpifnBwMA4fPlwtRRERERFVRKWDTG5uLszNzUvMNzMzg0qlqpaiiIiIiCqi0kHGz88P27dvLzF/27ZtaNq0abUURURERFQRlR7sO3v2bAwYMADJycl45ZVXAADR0dHYunVrqc9gIiIiIqoplQ4yffv2xc6dO7FgwQL88MMPUCqVaNGiBaKiotC1a9eaqJGIiIioVFW+j0xpLly4gObNm1dXd9WC95EhIiKSnor+fld6jMzjcnJysGbNGrRv3x4tW7Z82u6IiIiIKqzKQebw4cMYPnw4XF1d8cUXX+CVV17B8ePHq7M2IiIionJVaoxMRkYG1q1bh7Vr10KlUmHQoEFQq9XYuXMnr1giIiKiZ67CR2T69u2LRo0a4dy5c1i2bBlu3ryJFStW1GRtREREROWqcJDZt28fRo0ahU8++QS9e/eGXC5/6o17eXlBJpOVmMLDwwEA3bp1K7Fs/PjxT71dIiIiqh0qHGSOHj2KnJwctG3bFh06dMDKlStx+/btp9p4bGws0tPTddPBgwcBQO+ZTWPGjNFrs2jRoqfaJhEREdUeFQ4yHTt2xNdff4309HSMGzcO27Ztg5ubG7RaLQ4ePIicnJxKb7xevXpwcXHRTXv37oWPj4/e/WgsLS312vASaiIiInqo0lctWVlZISwsDEePHsX58+fxwQcf4LPPPoOTkxNee+21KhdSWFiITZs2ISwsDDKZTDd/8+bNcHR0RPPmzTFz5kzk5eWV249arYZKpdKbiIiIqHZ6qvvINGrUCIsWLcL169exdevWpypk586dyMrKQmhoqG7e0KFDsWnTJvz++++YOXMmNm7ciLfeeqvcfiIiImBnZ6eb3N3dn6ouIiIiMl7VemffpxEUFARzc3Ps2bOnzDa//fYbunfvjqSkJPj4+JTaRq1WQ61W616rVCq4u7vzzr5EREQSUtE7+1b6WUs1ITU1FVFRUfjpp5/KbdehQwcAKDfIKBQKKBSKaq+RiIiIjM9TP6KgOkRGRsLJyQm9e/cut11cXBwAwNXV9RlURURERMbO4EdktFotIiMjMWLECJia/q+c5ORkbNmyBa+++iocHBxw7tw5TJkyBV26dEGLFi0MWDEREREZC4MHmaioKKSlpSEsLExvvrm5OaKiorBs2TLcv38f7u7uCAkJwX/+8x8DVUpERETGxmgG+9aUig4WIiIiIuNR0d9voxgjQ0RERFQVDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQYNMl5eXpDJZCWm8PBwAEBBQQHCw8Ph4OAAa2trhISE4NatW4YsmYiIiIyIQYNMbGws0tPTddPBgwcBAAMHDgQATJkyBXv27MH333+PQ4cO4ebNmxgwYIAhSyYiIiIjIhNCCEMX8dDkyZOxd+9eXLlyBSqVCvXq1cOWLVvwxhtvAAAuXbqEJk2a4NixY+jYsWOF+lSpVLCzs0N2djZsbW1rsnwiIiKqJhX9/TaaMTKFhYXYtGkTwsLCIJPJcPr0aRQVFSEwMFDXpnHjxvDw8MCxY8fK7EetVkOlUulNREREVDsZTZDZuXMnsrKyEBoaCgDIyMiAubk56tSpo9fO2dkZGRkZZfYTEREBOzs73eTu7l6DVRMREZEhGU2QWbt2LYKDg+Hm5vZU/cycORPZ2dm66dq1a9VUIRERERkbU0MXAACpqamIiorCTz/9pJvn4uKCwsJCZGVl6R2VuXXrFlxcXMrsS6FQQKFQ1GS5REREZCSM4ohMZGQknJyc0Lt3b928tm3bwszMDNHR0bp5iYmJSEtLQ0BAgCHKJCIiIiNj8CMyWq0WkZGRGDFiBExN/1eOnZ0dRo0ahffffx9169aFra0t3nvvPQQEBFT4iiUiIiKq3QweZKKiopCWloawsLASy5YuXQoTExOEhIRArVYjKCgI//3vfw1QJRERERkjo7qPTE3gfWSIiIikR3L3kSEiIiKqLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikixTQxdARESlE0KguLgYGo3G0KUQVTu5XA5TU1PIZLKn6odBhojICBUWFiI9PR15eXmGLoWoxlhaWsLV1RXm5uZV7oNBhojIyGi1WqSkpEAul8PNzQ3m5uZP/a9WImMihEBhYSH++ecfpKSkoEGDBjAxqdpoFwYZIiIjU1hYCK1WC3d3d1haWhq6HKIaoVQqYWZmhtTUVBQWFsLCwqJK/XCwLxGRkarqv1CJpKI6vuP8v4SIqBbIVRfjWPId5KqLSywr1miRX6hBsUZbqfWIpMDgQebGjRt466234ODgAKVSCT8/P5w6dUq3PDQ0FDKZTG/q1auXASsmIjI+eepiHL96B3n/P5BotQJJmbn4/tQ1fLInAZ/u/Quf7EnA96euISkzF1qtKHU9IqkxaJC5d+8eOnfuDDMzM+zbtw8JCQlYvHgx7O3t9dr16tUL6enpumnr1q0GqpiIyPipizX48cx1rPztCqIvZqJQo4WpiQkKNVpEX8zEyt+u4Mcz16Eurv7Lurt164bJkydXe7/Gat26dahTp47u9Zw5c9CqVasnrjd79myMHTu25gqrYY/vd2lmzJiB9957r8ZrMehg34ULF8Ld3R2RkZG6ed7e3iXaKRQKuLi4VKhPtVoNtVqte61SqZ6+UCIiidAKgd1xNxF9MRPOtgp4Ouhf1upia4GsvEJEX8wEAPyrgaMhynyuZWRk4P/+7/9w/vx53bzQ0FCsX78eAGBmZgYPDw8MHz4cs2bNgqlp+T/V69atw8iRI8ttk5KSAi8vr6euvTKmTp2K+vXrY8qUKahfv36NbcegR2R2794Nf39/DBw4EE5OTmjdujW+/vrrEu1iYmLg5OSERo0aYcKECbhz506ZfUZERMDOzk43ubu71+QuEBEZlbQ7efgj6TacbRWoY1n6vTnqWJrD2VaBP5Ju49pd3qfmWfvmm2/QqVMneHp66s1/ePbhypUr+OCDDzBnzhx8/vnnT+zvzTff1DtrERAQgDFjxujNq8xvYWFhYaX3qTSOjo4ICgrCqlWrqqW/shg0yFy9ehWrVq1CgwYNsH//fkyYMAETJ07UpVLgwQe7YcMGREdHY+HChTh06BCCg4PLvNPlzJkzkZ2drZuuXbv2rHaHiMigNFqBuGtZuK/WwMbCDBqtKHOysTBDXqEGZ69lVXsdxcXFePfdd2FnZwdHR0fMnj0bQgjd8vT0dPTu3RtKpRLe3t7YsmULvLy8sGzZMl2brKwsjB49GvXq1YOtrS1eeeUVxMfHl7vdmJgYtG/fHlZWVqhTpw46d+6M1NRUAP875fPtt9/Cw8MD1tbWeOedd6DRaLBo0SK4uLjAyckJ8+fP1+tzyZIl8PPzg5WVFdzd3fHOO+8gNzf3qd6fbdu2oW/fviXmPzz74OnpiQkTJiAwMBC7d+/G/fv3YWtrix9++EGv/c6dO2FlZYXi4mK4uLjoJnNzc1haWupeFxYWYsCAAbC2toatrS0GDRqEW7du6fp5+N5888038Pb21l0GnZWVhXHjxsHZ2RkWFhZo3rw59u7dq1fD/v370aRJE1hbW+uC2KP69u2Lbdu2PdX79SQGPbWk1Wrh7++PBQsWAABat26NCxcuYPXq1RgxYgQAYPDgwbr2fn5+aNGiBXx8fBATE4Pu3buX6FOhUEChUDybHSAiMiInU+4gp6AYJiYyqAqKntg+V12M62fz0L2xc7XWsX79eowaNQonT57EqVOnMHbsWHh4eGDMmDEAgOHDh+P27duIiYmBmZkZ3n//fWRmZur1MXDgQCiVSuzbtw92dnb46quv0L17d1y+fBl169Ytsc3i4mL0798fY8aMwdatW1FYWIiTJ0/q3UgwOTkZ+/btw6+//ork5GS88cYbuHr1Kho2bIhDhw7hzz//RFhYGAIDA9GhQwcADy4PXr58Oby9vXH16lW88847mD59Ov773/9W6b25e/cuEhIS4O/v/8S2SqUSd+7cgZWVFQYPHozIyEi88cYbuuUPX9vY2JTZh1arRb9+/WBtbY1Dhw6huLgY4eHhePPNNxETE6Nrl5SUhB9//BE//fQT5HI5tFotgoODkZOTg02bNsHHxwcJCQmQy+W6dfLy8vDFF19g48aNMDExwVtvvYWpU6di8+bNujbt27fH9evX8ffff9fYqS2DBhlXV1c0bdpUb16TJk3w448/lrlO/fr14ejoiKSkpFKDDBHR80orHoyRMZNV7GC7XCZDoUYLzSNHS6qDu7s7li5dCplMhkaNGuH8+fNYunQpxowZg0uXLiEqKgqxsbG6H/NvvvkGDRo00K1/9OhRnDx5EpmZmbp/mH7xxRfYuXMnfvjhh1IHyapUKmRnZ6NPnz7w8fEB8OD35FFarRbffvstbGxs0LRpU7z88stITEzEL7/8AhMTEzRq1AgLFy7E77//rgsyjw5c9vLywrx58zB+/PgqB5m0tDQIIeDm5lZmGyEEoqOjsX//ft1g2dGjR6NTp05IT0+Hq6srMjMz8csvvyAqKqrc7UVHR+P8+fNISUnRnV7asGEDmjVrhtjYWLRr1w7Ag9NJGzZsQL169QAABw4cwMmTJ3Hx4kU0bNgQAEqMcykqKsLq1at17/e7776LTz/9VK/Nw/1MTU2tnUGmc+fOSExM1Jt3+fLlEucNH3X9+nXcuXMHrq6uNV0eEZGktPeqiwxVPuQmJnC2ffJdUm+pCqAVAvJqfvxBx44d9Y6EBAQEYPHixdBoNEhMTISpqSnatGmjW+7r66t3tWp8fDxyc3Ph4OCg129+fj6Sk5ORlpam94/gWbNmYdasWQgNDUVQUBB69OiBwMBADBo0SO+3wsvLS+/ohbOzM+Ryud5N2ZydnfWODkVFRSEiIgKXLl2CSqVCcXExCgoKkJeXV6W7Lufn5wNAqXex3bt3L6ytrVFUVAStVouhQ4dizpw5AB4c2WjWrBnWr1+PGTNmYNOmTfD09ESXLl3K3d7Fixfh7u6uN0amadOmqFOnDi5evKgLMp6enroQAwBxcXF48cUXdSGmNJaWlroQA0AXsB6lVCoBoEafGWbQMTJTpkzB8ePHsWDBAiQlJWHLli1Ys2YNwsPDAQC5ubmYNm0ajh8/jr///hvR0dHo168ffH19ERQUZMjSiYiMjpmpCZq52UFVUAS5ieyJU05BMZq52cLExLie45SbmwtXV1fExcXpTYmJiZg2bRrc3Nz05o8fPx7Ag1Mtx44dQ6dOnbB9+3Y0bNgQx48f1/VrZmamtx2ZTFbqPK32wY0D//77b/Tp0wctWrTAjz/+iNOnT+PLL78EUPUBsY6OD64Su3fvXollL7/8MuLi4nDlyhXk5+dj/fr1sLKy0i0fPXo01q1bp9vXkSNHVtszuB7dDvC/AFKe0t478djRvbt37wKAXkiqbgYNMu3atcOOHTuwdetWNG/eHHPnzsWyZcswbNgwAA8e8X3u3Dm89tpraNiwIUaNGoW2bdviyJEjHAdDRFQKvxfsoDSTIyuv/B/arLxCWJiZwO8Fu2qv4cSJE3qvjx8/jgYNGkAul6NRo0YoLi7G2bNndcuTkpL0ftjbtGmDjIwMmJqawtfXV29ydHQsMf/RMTOtW7fGzJkz8eeff6J58+bYsmVLlffj9OnT0Gq1WLx4MTp27IiGDRvi5s2bVe4PAHx8fGBra4uEhIQSy6ysrODr6wsPD49SL7l+6623kJqaiuXLlyMhIUE3lrQ8TZo0wbVr1/QufElISEBWVlaJoR2PatGiBa5fv47Lly9XcM9Kd+HCBZiZmaFZs2ZP1U95DP7QyD59+qBPnz6lLlMqldi/f/8zroiISLo8HCzR2ddRd5+Y0i7BzsorxC2VGt2bOMG9riWAsm9pURVpaWl4//33MW7cOJw5cwYrVqzA4sWLAQCNGzdGYGAgxo4di1WrVsHMzAwffPABlEql7uhCYGAgAgIC0L9/fyxatEgXIH7++We8/vrrpQ6UTUlJwZo1a/Daa6/Bzc0NiYmJuHLlCoYPH17l/fD19UVRURFWrFiBvn374o8//sDq1aur3B/wYPBwYGAgjh49iv79+1dqXXt7ewwYMADTpk1Dz5498eKLLz5xncDAQPj5+WHYsGFYtmwZiouL8c4776Br167lDjju2rUrunTpgpCQECxZsgS+vr64dOlSpe+uf+TIEfzrX/+q0BGeqjL4IwqIiKj6mMhkeK2VG7o3cUJ2fhESM3KQoSrA3fuFyFAVIDEjB9n5RejexAmvtXKDSTWPjwEeXJWUn5+P9u3bIzw8HJMmTdIboLthwwY4OzujS5cueP311zFmzBjY2Njoxo3IZDL88ssv6NKlC0aOHImGDRti8ODBSE1NhbNz6VdYWVpa4tKlSwgJCUHDhg0xduxYhIeHY9y4cVXej5YtW2LJkiVYuHAhmjdvjs2bNyMiIqLK/T00evRobNu2TXcKqzJGjRqFwsJChIWFVai9TCbDrl27YG9vjy5duiAwMBD169fH9u3bn7jujz/+iHbt2mHIkCFo2rQppk+fXuatT8qybds23dVqNUUmHj+hVcuoVCrY2dkhOzsbtra2hi6HiOiJCgoKkJKSondPjyfJVBVg84k0DOvgASdbC2i1Ail37uNs2j2cv56NIo0WZnIT+L1oh9Ye9vB2sIKJiazEeoZw/fp1uLu7Iyoq6rm4GlUIgQ4dOmDKlCkYMmRIpdbduHEjpkyZgps3b8LcvPQbHhqLffv24YMPPsC5c+fKvDtxed/1iv5+G/zUEhERPT1LhSk61neApeLBn3UTExl86lnDp541+rd6AUUaATO5DKZyk3LXexZ+++035Obmws/PD+np6Zg+fTq8vLyeeAVObSGTybBmzRq9RxQ8SV5eHtLT0/HZZ59h3LhxRh9iAOD+/fuIjIx84iMWnhaDDBFRLWCtMEWAj0Opy0zlJjCVl7qo3PVqSlFREWbNmoWrV6/CxsYGnTp1wubNm0tcBVObtWrVqkIPl3xo0aJFmD9/Prp06YKZM2fWXGHV6NGb99UknloiIjIyVTm1RCRF1XFqiYN9iYiISLIYZIiIjFQtP2BOVC3fcQYZIiIj83CsSE3e1p3IGDz8jj/N+CgO9iUiMjJyuRx16tTRPbfG0tKy2m5FT2QMhBDIy8tDZmYm6tSpo/dU7cpikCEiMkIuLi4AUOIhfES1SZ06dXTf9apikCEiMkIymQyurq5wcnJCUVGRocshqnZmZmZPdSTmIQYZIiIjJpfLq+WPPVFtxcG+REREJFkMMkRERCRZDDJEREQkWbV+jMzDm+2oVCoDV0JEREQV9fB3+0k3zav1QebOnTsAAHd3dwNXQkRERJWVk5MDOzu7MpfX+iBTt25dAEBaWlq5bwTVPJVKBXd3d1y7do0P8DQC/DyMBz8L48LPwzgIIZCTkwM3N7dy29X6IGNi8mAYkJ2dHb+QRsLW1pafhRHh52E8+FkYF34ehleRAxAc7EtERESSxSBDREREklXrg4xCocDHH38MhUJh6FKee/wsjAs/D+PBz8K48POQFpl40nVNREREREaq1h+RISIiotqLQYaIiIgki0GGiIiIJItBhoiIiCSrVgeZL7/8El5eXrCwsECHDh1w8uRJQ5f0XIqIiEC7du1gY2MDJycn9O/fH4mJiYYuiwB89tlnkMlkmDx5sqFLeW7duHEDb731FhwcHKBUKuHn54dTp04ZuqznjkajwezZs+Ht7Q2lUgkfHx/MnTv3ic/5IcOrtUFm+/bteP/99/Hxxx/jzJkzaNmyJYKCgpCZmWno0p47hw4dQnh4OI4fP46DBw+iqKgIPXv2xP379w1d2nMtNjYWX331FVq0aGHoUp5b9+7dQ+fOnWFmZoZ9+/YhISEBixcvhr29vaFLe+4sXLgQq1atwsqVK3Hx4kUsXLgQixYtwooVKwxdGj1Brb38ukOHDmjXrh1WrlwJANBqtXB3d8d7772HGTNmGLi659s///wDJycnHDp0CF26dDF0Oc+l3NxctGnTBv/9738xb948tGrVCsuWLTN0Wc+dGTNm4I8//sCRI0cMXcpzr0+fPnB2dsbatWt180JCQqBUKrFp0yYDVkZPUiuPyBQWFuL06dMIDAzUzTMxMUFgYCCOHTtmwMoIALKzswH874Ge9OyFh4ejd+/eev+P0LO3e/du+Pv7Y+DAgXByckLr1q3x9ddfG7qs51KnTp0QHR2Ny5cvAwDi4+Nx9OhRBAcHG7gyepJa+dDI27dvQ6PRwNnZWW++s7MzLl26ZKCqCHhwZGzy5Mno3Lkzmjdvbuhynkvbtm3DmTNnEBsba+hSnntXr17FqlWr8P7772PWrFmIjY3FxIkTYW5ujhEjRhi6vOfKjBkzoFKp0LhxY8jlcmg0GsyfPx/Dhg0zdGn0BLUyyJDxCg8Px4ULF3D06FFDl/JcunbtGiZNmoSDBw/CwsLC0OU897RaLfz9/bFgwQIAQOvWrXHhwgWsXr2aQeYZ++6777B582Zs2bIFzZo1Q1xcHCZPngw3Nzd+FkauVgYZR0dHyOVy3Lp1S2/+rVu34OLiYqCq6N1338XevXtx+PBhvPjii4Yu57l0+vRpZGZmok2bNrp5Go0Ghw8fxsqVK6FWqyGXyw1Y4fPF1dUVTZs21ZvXpEkT/Pjjjwaq6Pk1bdo0zJgxA4MHDwYA+Pn5ITU1FREREQwyRq5WjpExNzdH27ZtER0drZun1WoRHR2NgIAAA1b2fBJC4N1338WOHTvw22+/wdvb29AlPbe6d++O8+fPIy4uTjf5+/tj2LBhiIuLY4h5xjp37lziVgSXL1+Gp6engSp6fuXl5cHERP8nUS6XQ6vVGqgiqqhaeUQGAN5//32MGDEC/v7+aN++PZYtW4b79+9j5MiRhi7tuRMeHo4tW7Zg165dsLGxQUZGBgDAzs4OSqXSwNU9X2xsbEqMTbKysoKDgwPHLBnAlClT0KlTJyxYsACDBg3CyZMnsWbNGqxZs8bQpT13+vbti/nz58PDwwPNmjXD2bNnsWTJEoSFhRm6NHoSUYutWLFCeHh4CHNzc9G+fXtx/PhxQ5f0XAJQ6hQZGWno0kgI0bVrVzFp0iRDl/Hc2rNnj2jevLlQKBSicePGYs2aNYYu6bmkUqnEpEmThIeHh7CwsBD169cXH374oVCr1YYujZ6g1t5HhoiIiGq/WjlGhoiIiJ4PDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREbi77//hkwmQ1xcnKFLMbiYmBjIZDJkZWWV287LywvLli2rVN/dunXD5MmTq1xbVdXk51vR98vYFBYWwtfXF3/++WeNbWP16tXo27dvjfVPhscgQ0bt2LFjkMvl6N27t6FLMUqhoaHo37+/ocuodp06dUJ6ejrs7OwAAOvWrUOdOnUMW9QjjC04PP5+PWs//fQTevbsCQcHh0qFtdWrV8Pb2xudOnWqsdrCwsJw5swZHDlypMa2QYbFIENGbe3atXjvvfdw+PBh3Lx5s0a3JYRAcXFxjW6DKsbc3BwuLi6QyWSGLkUSDP1+3b9/Hy+99BIWLlxY4XWEEFi5ciVGjRpVg5U9eG+GDh2K5cuX1+h2yHAYZMho5ebmYvv27ZgwYQJ69+6NdevW6ZYNHToUb775pl77oqIiODo6YsOGDQAArVaLiIgIeHt7Q6lUomXLlvjhhx907R/+q3rfvn1o27YtFAoFjh49iuTkZPTr1w/Ozs6wtrZGu3btEBUVpbet9PR09O7dG0qlEt7e3tiyZUuJ0xxZWVkYPXo06tWrB1tbW7zyyiuIj4+v8P5rNBqMGjVKV3+jRo3wf//3f7rlc+bMwfr167Fr1y7IZDLIZDLExMQAAK5du4ZBgwahTp06qFu3Lvr164e///5bt+7DIzlffPEFXF1d4eDggPDwcBQVFenaqNVq/Pvf/4a7uzsUCgV8fX2xdu1aCCHg6+uLL774Qq/euLg4yGQyJCUlldiXCxcuwMTEBP/88w8A4O7duzAxMcHgwYN1bebNm4eXXnpJ77PJyspCTEwMRo4ciezsbN1+zpkzR7deXl4ewsLCYGNjAw8Pjwo9Obq4uBjvvvsu7Ozs4OjoiNmzZ+PRx85t3LgR/v7+sLGxgYuLC4YOHYrMzEwAD04RvfzyywAAe3t7yGQyhIaGAnjwnVu0aBF8fX2hUCjg4eGB+fPn62376tWrePnll2FpaYmWLVvi2LFjT6wXAFJTU9G3b1/Y29vDysoKzZo1wy+//FLi/QIenD57+F49Oj38Djztd/Nxb7/9Nj766CMEBgZWeJ3Tp08jOTlZ72jrw9Nv3333Hf71r39BqVSiXbt2uHz5MmJjY+Hv7w9ra2sEBwfrvksP9799+/awsrJCnTp10LlzZ6SmpuqW9+3bF7t370Z+fn6V95GMmCGfWElUnrVr1wp/f38hxIMnBPv4+AitViuEEGLv3r1CqVSKnJwcXfs9e/YIpVIpVCqVEEKIefPmicaNG4tff/1VJCcni8jISKFQKERMTIwQQojff/9dABAtWrQQBw4cEElJSeLOnTsiLi5OrF69Wpw/f15cvnxZ/Oc//xEWFhYiNTVVt63AwEDRqlUrcfz4cXH69GnRtWtXoVQqxdKlS/Xa9O3bV8TGxorLly+LDz74QDg4OIg7d+6Uur8pKSkCgDh79qwQQojCwkLx0UcfidjYWHH16lWxadMmYWlpKbZv3y6EECInJ0cMGjRI9OrVS6Snp4v09HShVqtFYWGhaNKkiQgLCxPnzp0TCQkJYujQoaJRo0a6J/mOGDFC2NraivHjx4uLFy+KPXv2CEtLS70nLw8aNEi4u7uLn376SSQnJ4uoqCixbds2IYQQ8+fPF02bNtWrf+LEiaJLly6l7ptWqxWOjo7i+++/F0IIsXPnTuHo6ChcXFz03q8PP/xQ77O5d++eUKvVYtmyZcLW1la3nw8/d09PT1G3bl3x5ZdfiitXroiIiAhhYmIiLl26VGodQjx42re1tbWYNGmSuHTpku59fXTf165dK3755ReRnJwsjh07JgICAkRwcLAQQoji4mLx448/CgAiMTFRpKeni6ysLCGEENOnTxf29vZi3bp1IikpSRw5ckR8/fXXep9v48aNxd69e0ViYqJ44403hKenpygqKiqz3od69+4tevToIc6dOyeSk5PFnj17xKFDh0q8X0IIcefOHd17lZ6eLgYMGCAaNWok8vLydO91ed/Nw4cPCysrq3KnTZs2lajx8e9weZYsWSIaN25c6voP/79NSEgQHTt2FG3bthXdunUTR48eFWfOnBG+vr5i/PjxQgghioqKhJ2dnZg6dapISkoSCQkJYt26dXr/v96/f1+YmJiI33///Yl1kfQwyJDR6tSpk1i2bJkQ4sEfK0dHR90fooevN2zYoGs/ZMgQ8eabbwohhCgoKBCWlpbizz//1Otz1KhRYsiQIUKI//3x37lz5xNradasmVixYoUQQoiLFy8KACI2Nla3/MqVKwKALsgcOXJE2NraioKCAr1+fHx8xFdffVXqNiryIxAeHi5CQkJ0r0eMGCH69eun12bjxo2iUaNGutAnhBBqtVoolUqxf/9+3Xqenp6iuLhY12bgwIG69y8xMVEAEAcPHiy1jhs3bgi5XC5OnDghhHgQuhwdHcW6devKrH3AgAEiPDxcCCHE5MmTxbRp04S9vb24ePGiKCwsFJaWluLAgQNCiJI/zJGRkcLOzq5En56enuKtt97SvdZqtcLJyUmsWrWqzDq6du0qmjRpovf+/Pvf/xZNmjQpc53Y2FgBQBegHq9PCCFUKpVQKBS64PK4h5/vN998o5v3119/CQDi4sWLZW77IT8/PzFnzpxSl5VWz0NLliwRderUEYmJiUKIin038/LyxJUrV8qdHv6DobR9rEiQmTRpknjllVdKXf/R92jr1q0CgIiOjtbNi4iIEI0aNRJCPAhtAHT/QCnLw4BJtY/pMzz4Q1RhiYmJOHnyJHbs2AEAMDU1xZtvvom1a9eiW7duMDU1xaBBg7B582a8/fbbuH//Pnbt2oVt27YBAJKSkpCXl4cePXro9VtYWIjWrVvrzfP399d7nZubizlz5uDnn39Geno6iouLkZ+fj7S0NF1tpqamaNOmjW4dX19f2Nvb617Hx8cjNzcXDg4Oen3n5+cjOTm5wu/Dl19+iW+//RZpaWnIz89HYWEhWrVqVe468fHxSEpKgo2Njd78goICvW03a9YMcrlc99rV1RXnz58H8OA0kVwuR9euXUvdhpubG3r37o1vv/0W7du3x549e6BWqzFw4MAy6+ratavutM+hQ4ewYMECXL58GTExMbh79y6KiorQuXPncvetNC1atND9t0wmg4uLi+40UFk6duyoN54kICAAixcvhkajgVwux+nTpzFnzhzEx8fj3r170Gq1AIC0tDQ0bdq01D4vXrwItVqN7t27V7heV1dXAEBmZiYaN25c7noTJ07EhAkTcODAAQQGBiIkJESvr9Ls27cPM2bMwJ49e9CwYUMAFftuKpVK+Pr6ltv308rPz4eFhUWpyx7dL2dnZwCAn5+f3ryHn3HdunURGhqKoKAg9OjRA4GBgRg0aJDuvX1IqVQiLy+vuneDjACDDBmltWvXori4GG5ubrp5QggoFAqsXLkSdnZ2GDZsGLp27YrMzEwcPHgQSqUSvXr1AvAgjADAzz//jBdeeEGvb4VCoffayspK7/XUqVNx8OBBfPHFF/D19YVSqcQbb7yBwsLCCtefm5sLV1dX3ZiVR1X06ptt27Zh6tSpWLx4MQICAmBjY4PPP/8cJ06ceOK227Zti82bN5dYVq9ePd1/m5mZ6S2TyWS6H2ylUvnE+kaPHo23334bS5cuRWRkJN58801YWlqW2f7hZc9XrlxBQkICXnrpJVy6dAkxMTG4d+8e/P39y12/LOXtR1Xcv38fQUFBCAoKwubNm1GvXj2kpaUhKCio3O9ARd6zx+t9GKYqUu/o0aMRFBSEn3/+GQcOHEBERAQWL16M9957r9T2CQkJGDx4MD777DP07NlTN78i380jR44gODi43Hq++uorDBs27Il1l8XR0VEXnB9X2nv0+LxH37PIyEhMnDgRv/76K7Zv347//Oc/OHjwIDp27Khrc/fuXb3vP9UeDDJkdIqLi7FhwwYsXrxY7w8wAPTv3x9bt27F+PHj0alTJ7i7u2P79u3Yt28fBg4cqPtj17RpUygUCqSlpZV5VKEsf/zxB0JDQ/H6668DePCH/9GBso0aNUJxcTHOnj2Ltm3bAnhwBOjevXu6Nm3atEFGRgZMTU3h5eVVhXfhQR2dOnXCO++8o5v3+NEcc3NzaDQavXlt2rTB9u3b4eTkBFtb2ypt28/PD1qtFocOHSpzAOerr74KKysrrFq1Cr/++isOHz78xD7t7e0xb948tGrVCtbW1ujWrRsWLlyIe/fuoVu3bmWuW9p+Po3Hw+Dx48fRoEEDyOVyXLp0CXfu3MFnn30Gd3d3AMCpU6dK1ANAr6YGDRpAqVQiOjoao0ePrrZaH+Xu7o7x48dj/PjxmDlzJr7++utSg8zt27fRt29fhISEYMqUKXrLKvLd9Pf3f+Il1A+PlFRV69atsWrVKgghquVqq9atW6N169aYOXMmAgICsGXLFl2QSU5ORkFBQYmjsVQ78KolMjp79+7FvXv3MGrUKDRv3lxvCgkJwdq1a3Vthw4ditWrV+PgwYN6/zq0sbHB1KlTMWXKFKxfvx7Jyck4c+YMVqxYgfXr15e7/QYNGuCnn35CXFwc4uPjMXToUL1//TVu3BiBgYEYO3YsTp48ibNnz2Ls2LFQKpW6P8iBgYEICAhA//79ceDAAfz999/4888/8eGHH5b4USyvjlOnTmH//v24fPkyZs+ejdjYWL02Xl5eOHfuHBITE3H79m0UFRVh2LBhcHR0RL9+/XDkyBGkpKQgJiYGEydOxPXr1yu0bS8vL4wYMQJhYWHYuXOnro/vvvtO10YulyM0NBQzZ85EgwYNEBAQUG6fMpkMXbp0webNm3WhpUWLFlCr1YiOji43cHp5eSE3NxfR0dG4ffv2U58iSEtLw/vvv4/ExERs3boVK1aswKRJkwAAHh4eMDc3x4oVK3D16lXs3r0bc+fO1Vvf09MTMpkMe/fuxT///IPc3FxYWFjg3//+N6ZPn44NGzYgOTkZx48f1/u+Po3Jkydj//79SElJwZkzZ/D777+jSZMmpbYNCQmBpaUl5syZg4yMDN2k0Wgq9N18eGqpvOnRU5d3795FXFwcEhISADw4/RoXF4eMjIwy9+fll19Gbm4u/vrrr6d6X1JSUjBz5kwcO3YMqampOHDgAK5cuaL33hw5cgT169eHj4/PU22LjJShB+kQPa5Pnz7i1VdfLXXZiRMnBAARHx8vhBAiISFBABCenp56gzeFeDDwc9myZaJRo0bCzMxM1KtXTwQFBZV5pcdDKSkp4uWXXxZKpVK4u7uLlStXiq5du4pJkybp2ty8eVMEBwcLhUIhPD09xZYtW4STk5NYvXq1ro1KpRLvvfeecHNzE2ZmZsLd3V0MGzZMpKWllbpvjw+ULCgoEKGhocLOzk7UqVNHTJgwQcyYMUO0bNlSt05mZqbo0aOHsLa2FgB0g6HT09PF8OHDhaOjo1AoFKJ+/fpizJgxIjs7WwhR+iDhSZMmia5du+pe5+fniylTpghXV1dhbm4ufH19xbfffqu3TnJysgAgFi1aVOo+PW7p0qUCgNi3b59uXr9+/YSpqaneFWilfTbjx48XDg4OAoD4+OOPhRAPBvs+eqWYEEK0bNlSt7w0Xbt2Fe+8844YP368sLW1Ffb29mLWrFl6358tW7YILy8voVAoREBAgNi9e3eJQayffvqpcHFxETKZTIwYMUIIIYRGoxHz5s0Tnp6ewszMTHh4eIgFCxYIIUofCHvv3j29z6087777rvDx8REKhULUq1dPvP322+L27dulvl8ASp1SUlKEEJX/bj5JZGRkqdsr73MQ4sGVcTNmzNC9Lu09Ku278Ojg74yMDNG/f3/d99TT01N89NFHQqPR6Nr37NlTREREVGnfyPjJhHjk5glEVCXXr1+Hu7s7oqKinjjYszY5cuQIunfvjmvXrj31qQZ6/pw7dw49evRAcnIyrK2ta2Qbf/31F1555RVcvnzZYHc+pprFIENUBb/99htyc3Ph5+eH9PR0TJ8+HTdu3MDly5dLDD6tjdRqNf755x+MGDECLi4upQ4sJqqIdevWoW3btnpXJVWnqKgoaDQaBAUF1Uj/ZHgcI0NUBUVFRZg1axaaNWuG119/HfXq1UNMTMxzEWIAYOvWrfD09ERWVhYWLVpk6HJqheDgYFhbW5c6LViwwNDl1ZjQ0NAaCzHAg/FqDDG1G4/IEBEZgRs3bpR5C/26deuibt26z7giImlgkCEiIiLJ4qklIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikqz/BzTCzrTrZ7uGAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics(perf_metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AiPUhOCNWRny",
   "metadata": {
    "id": "AiPUhOCNWRny"
   },
   "source": [
    "## 4. Compressing with Optimum ONNX and CUDAExecutionProvider\n",
    "\n",
    "We'll be using Optimum's ONNX Runtime support with `CUDAExecutionProvider` [because it's fast while also supporting dynamic shapes](https://github.com/huggingface/optimum-benchmark/tree/main/examples/fast-mteb#notes)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "NEnwnsEQWRn8",
   "metadata": {
    "cellView": "form",
    "id": "NEnwnsEQWRn8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    }
   ],
   "source": [
    "!pip install optimum[onnxruntime-gpu] -qqq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HjeZkCtSqcBe",
   "metadata": {
    "id": "HjeZkCtSqcBe"
   },
   "source": [
    "Optimum makes it extremely easy to export a model to ONNX and apply SOTA graph optimizations / kernel fusions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11ee6e82-97d3-4aa7-9181-12c5860c1a92",
   "metadata": {},
   "source": [
    "We use the `\"CUDAExecutionProvider\"` provider to export the SentenceTransformer to ONNX. Feel free to also experiment with other providers, such as `\"TensorrtExecutionProvider\"` and `\"CPUExecutionProvider\"`. The former may be even faster than `\"CUDAExecutionProvider\"`, but requires more installation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "hPqEcDi8WRn8",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hPqEcDi8WRn8",
    "outputId": "0e1202d8-aa84-422c-f10f-6bb0b43d1ef8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Framework not specified. Using pt to export the model.\n",
      "Using the export variant default. Available variants are:\n",
      "    - default: The default ONNX variant.\n",
      "Using framework PyTorch: 2.2.0+cu121\n",
      "Overriding 1 configuration item(s)\n",
      "\t- use_cache -> False\n",
      "\u001b[1;31m2024-02-22 03:05:55.640032039 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647539, index: 3, mask: {4, 60, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.640032719 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647541, index: 5, mask: {6, 62, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.640041583 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647542, index: 6, mask: {7, 63, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.640079415 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647540, index: 4, mask: {5, 61, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.640163101 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647543, index: 7, mask: {8, 64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.654549780 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647544, index: 8, mask: {9, 65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.654591315 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647545, index: 9, mask: {10, 66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662593133 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647546, index: 10, mask: {11, 67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662621937 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647575, index: 39, mask: {40, 96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662624961 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647551, index: 15, mask: {16, 72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662651809 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647583, index: 47, mask: {48, 104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662663533 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647576, index: 40, mask: {41, 97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662668827 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647553, index: 17, mask: {18, 74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662699111 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647577, index: 41, mask: {42, 98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662701054 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647556, index: 20, mask: {21, 77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.666578993 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647586, index: 50, mask: {51, 107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.666581759 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647554, index: 18, mask: {19, 75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662651812 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647555, index: 19, mask: {20, 76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670581414 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647587, index: 51, mask: {52, 108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670595708 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647547, index: 11, mask: {12, 68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670600042 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647578, index: 42, mask: {43, 99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670586137 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647567, index: 31, mask: {32, 88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670603445 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647561, index: 25, mask: {26, 82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670609320 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647559, index: 23, mask: {24, 80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670617441 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647557, index: 21, mask: {22, 78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.670641238 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647568, index: 32, mask: {33, 89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662608333 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647582, index: 46, mask: {47, 103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.674571380 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647569, index: 33, mask: {34, 90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.674575700 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647579, index: 43, mask: {44, 100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.674602518 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647549, index: 13, mask: {14, 70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662608541 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647548, index: 12, mask: {13, 69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678552261 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647580, index: 44, mask: {45, 101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678598758 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647552, index: 16, mask: {17, 73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678600947 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647570, index: 34, mask: {35, 91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678641069 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647563, index: 27, mask: {28, 84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.674576614 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647590, index: 54, mask: {55, 111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.682551559 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647573, index: 37, mask: {38, 94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.682591291 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647581, index: 45, mask: {46, 102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.662625165 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647550, index: 14, mask: {15, 71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.686555708 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647574, index: 38, mask: {39, 95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678641452 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647566, index: 30, mask: {31, 87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678601571 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647562, index: 26, mask: {27, 83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.686594340 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647584, index: 48, mask: {49, 105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.686603504 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647572, index: 36, mask: {37, 93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678600949 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647565, index: 29, mask: {30, 86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.690554475 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647585, index: 49, mask: {50, 106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.690556754 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647558, index: 22, mask: {23, 79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.678601569 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647560, index: 24, mask: {25, 81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.686576196 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647571, index: 35, mask: {36, 92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.686585628 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647564, index: 28, mask: {29, 85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.694547648 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647588, index: 52, mask: {53, 109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:55.698549871 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647589, index: 53, mask: {54, 110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.523815838 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647594, index: 3, mask: {4, 60, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.523841730 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647595, index: 4, mask: {5, 61, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.523868246 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647596, index: 5, mask: {6, 62, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.523892346 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647597, index: 6, mask: {7, 63, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.523927330 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647598, index: 7, mask: {8, 64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.538554841 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647601, index: 10, mask: {11, 67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.538558983 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647603, index: 12, mask: {13, 69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.538607912 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647621, index: 30, mask: {31, 87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.538558896 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647599, index: 8, mask: {9, 65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542555994 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647604, index: 13, mask: {14, 70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542595574 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647609, index: 18, mask: {19, 75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542555154 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647602, index: 11, mask: {12, 68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542645202 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647607, index: 16, mask: {17, 73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542556757 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647608, index: 17, mask: {18, 74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.546560849 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647605, index: 14, mask: {15, 71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.546560845 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647622, index: 31, mask: {32, 88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.550547098 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647610, index: 19, mask: {20, 76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.550551045 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647623, index: 32, mask: {33, 89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.550585865 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647615, index: 24, mask: {25, 81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly s/home/dkorat/anaconda/envs/dk-setfit-py310/lib/python3.10/site-packages/optimum/onnxruntime/configuration.py:770: FutureWarning: disable_embed_layer_norm will be deprecated soon, use disable_embed_layer_norm_fusion instead, disable_embed_layer_norm_fusion is set to True.\n",
      "  warnings.warn(\n",
      "o the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.542596833 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647606, index: 15, mask: {16, 72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.554546108 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647611, index: 20, mask: {21, 77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.554550822 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647616, index: 25, mask: {26, 82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.550550783 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647614, index: 23, mask: {24, 80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.554603672 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647617, index: 26, mask: {27, 83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.558543885 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647625, index: 34, mask: {35, 91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.558544825 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647640, index: 49, mask: {50, 106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.558578185 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647613, index: 22, mask: {23, 79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.554551136 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647624, index: 33, mask: {34, 90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562543634 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647600, index: 9, mask: {10, 66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562545137 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647641, index: 50, mask: {51, 107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.558545673 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647612, index: 21, mask: {22, 78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562576578 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647633, index: 42, mask: {43, 99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562588967 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647642, index: 51, mask: {52, 108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562547925 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647618, index: 27, mask: {28, 84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.566544582 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647626, index: 35, mask: {36, 92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.566577959 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647635, index: 44, mask: {45, 101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.562545195 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647632, index: 41, mask: {42, 98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.570543359 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647619, index: 28, mask: {29, 85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.570544277 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647627, index: 36, mask: {37, 93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.570547003 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647636, index: 45, mask: {46, 102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.574463616 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647643, index: 52, mask: {53, 109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.574541957 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647620, index: 29, mask: {30, 86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.574545822 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647637, index: 46, mask: {47, 103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.574564214 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647629, index: 38, mask: {39, 95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.574546839 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647628, index: 37, mask: {38, 94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.578543180 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647644, index: 53, mask: {54, 110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.578544793 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647638, index: 47, mask: {48, 104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.578561211 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647645, index: 54, mask: {55, 111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.566546186 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647634, index: 43, mask: {44, 100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.582545304 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647630, index: 39, mask: {40, 96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.582545850 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647639, index: 48, mask: {49, 105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:05:57.586544037 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647631, index: 40, mask: {41, 97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:05:58.146991559 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:05:58.147024351 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.\u001b[m\n",
      "Optimizing model...\n",
      "\u001b[1;31m2024-02-22 03:06:00.127222014 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647655, index: 4, mask: {5, 61, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.127310758 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647654, index: 3, mask: {4, 60, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.127296334 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647657, index: 6, mask: {7, 63, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.127327029 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647656, index: 5, mask: {6, 62, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.134569885 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647677, index: 26, mask: {27, 83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.134645620 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647675, index: 24, mask: {25, 81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.134651216 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647685, index: 34, mask: {35, 91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.134737848 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647673, index: 22, mask: {23, 79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.137722356 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647683, index: 32, mask: {33, 89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.138586661 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647676, index: 25, mask: {26, 82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.138591680 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647686, index: 35, mask: {36, 92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.141944708 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647674, index: 23, mask: {24, 80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142623307 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647679, index: 28, mask: {29, 85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142632763 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647687, index: 36, mask: {37, 93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142639429 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647684, index: 33, mask: {34, 90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142636954 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647696, index: 45, mask: {46, 102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142711153 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647682, index: 31, mask: {32, 88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.134650976 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647672, index: 21, mask: {22, 78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.142639933 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647658, index: 7, mask: {8, 64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.146586161 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647691, index: 40, mask: {41, 97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.146615035 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647689, index: 38, mask: {39, 95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.146617398 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647698, index: 47, mask: {48, 104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.150549811 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647680, index: 29, mask: {30, 86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.150601359 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647700, index: 49, mask: {50, 106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.150603264 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647694, index: 43, mask: {44, 100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.150653550 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647701, index: 50, mask: {51, 107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.152675179 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647692, index: 41, mask: {42, 98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.154578342 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647695, index: 44, mask: {45, 101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.157717236 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647693, index: 42, mask: {43, 99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.158568412 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647702, index: 51, mask: {52, 108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.158605717 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647666, index: 15, mask: {16, 72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.158570322 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647665, index: 14, mask: {15, 71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.162569291 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647668, index: 17, mask: {18, 74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.162599847 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647671, index: 20, mask: {21, 77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.162569512 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647681, index: 30, mask: {31, 87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.162569316 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647663, index: 12, mask: {13, 69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.166556487 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647662, index: 11, mask: {12, 68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.166578492 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647667, index: 16, mask: {17, 73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.166556871 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647669, index: 18, mask: {19, 75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.170552776 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647661, index: 10, mask: {11, 67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.172874787 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647703, index: 52, mask: {53, 109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.174555696 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647659, index: 8, mask: {9, 65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.177928083 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647704, index: 53, mask: {54, 110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.182548167 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647678, index: 27, mask: {28, 84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.182554152 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647705, index: 54, mask: {55, 111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.184640698 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647697, index: 46, mask: {47, 103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.186548487 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647670, index: 19, mask: {20, 76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.190546592 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647688, index: 37, mask: {38, 94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.193092508 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647664, index: 13, mask: {14, 70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.194572935 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647690, index: 39, mask: {40, 96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.194572887 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647699, index: 48, mask: {49, 105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:00.246590877 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647660, index: 9, mask: {10, 66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:06:00.630152883 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:06:00.630163616 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.\u001b[m\n",
      "Configuration saved in bge_auto_opt_O4/ort_config.json\n",
      "Optimized model saved at: bge_auto_opt_O4 (external data format: False; saved all tensor to one file: True)\n",
      "\u001b[1;31m2024-02-22 03:06:05.371144433 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647709, index: 3, mask: {4, 60, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.371246032 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647713, index: 7, mask: {8, 64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.371181424 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647710, index: 4, mask: {5, 61, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.371237552 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647712, index: 6, mask: {7, 63, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.371182624 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647711, index: 5, mask: {6, 62, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.371293315 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647714, index: 8, mask: {9, 65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.386549981 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647717, index: 11, mask: {12, 68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.386552378 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647716, index: 10, mask: {11, 67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.386590894 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647720, index: 14, mask: {15, 71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.390558515 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647732, index: 26, mask: {27, 83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.390561896 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647715, index: 9, mask: {10, 66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.390601512 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647725, index: 19, mask: {20, 76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.386557014 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647719, index: 13, mask: {14, 70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.394546642 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647726, index: 20, mask: {21, 77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.398547907 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647733, index: 27, mask: {28, 84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.401612692 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647727, index: 21, mask: {22, 78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.402544574 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647734, index: 28, mask: {29, 85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.402545711 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647728, index: 22, mask: {23, 79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406544967 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647729, index: 23, mask: {24, 80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406568184 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647743, index: 37, mask: {38, 94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406586415 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647752, index: 46, mask: {47, 103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406549655 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647750, index: 44, mask: {45, 101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.410548750 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647721, index: 15, mask: {16, 72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.410566583 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647745, index: 39, mask: {40, 96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406548056 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647735, index: 29, mask: {30, 86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414544594 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647723, index: 17, mask: {18, 74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414546668 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647730, index: 24, mask: {25, 81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414579851 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647754, index: 48, mask: {49, 105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414582609 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647758, index: 52, mask: {53, 109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.410550854 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647744, index: 38, mask: {39, 95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.406568810 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647751, index: 45, mask: {46, 102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.418543156 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647724, index: 18, mask: {19, 75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.418546305 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647731, index: 25, mask: {26, 82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414547886 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647753, index: 47, mask: {48, 104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.422542426 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647738, index: 32, mask: {33, 89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.422545189 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647736, index: 30, mask: {31, 87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.422563774 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647760, index: 54, mask: {55, 111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.426543681 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647739, index: 33, mask: {34, 90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.426545475 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647737, index: 31, mask: {32, 88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.426562517 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647740, index: 34, mask: {35, 91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.426569949 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647755, index: 49, mask: {50, 106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.422545735 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647759, index: 53, mask: {54, 110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.430543964 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647746, index: 40, mask: {41, 97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.430545003 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647756, index: 50, mask: {51, 107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.430574982 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647742, index: 36, mask: {37, 93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.431462738 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647757, index: 51, mask: {52, 108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.430544590 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647741, index: 35, mask: {36, 92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.414546691 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647722, index: 16, mask: {17, 73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.434545507 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647747, index: 41, mask: {42, 98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.434551690 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647718, index: 12, mask: {13, 69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:06:05.435870770 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.\u001b[m\n",
      "\u001b[0;93m2024-02-22 03:06:05.435879901 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.438546913 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647749, index: 43, mask: {44, 100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n",
      "\u001b[1;31m2024-02-22 03:06:05.442544715 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 3647748, index: 42, mask: {43, 99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.\u001b[m\n"
     ]
    }
   ],
   "source": [
    "from optimum.onnxruntime import ORTModelForFeatureExtraction, AutoOptimizationConfig, ORTOptimizer\n",
    "\n",
    "# Load a PyTorch model and export it to the ONNX format\n",
    "ort_model = ORTModelForFeatureExtraction.from_pretrained(\n",
    "    \"dkorat/bge-small-en-v1.5_setfit-sst2-english\",\n",
    "    export=True,\n",
    "    provider=\"CUDAExecutionProvider\",\n",
    ")\n",
    "\n",
    "# Create the optimizer\n",
    "optimizer = ORTOptimizer.from_pretrained(ort_model)\n",
    "\n",
    "# Optimize using the appropriate optimization strategy\n",
    "opt_model_path = optimizer.optimize(save_dir=\"bge_auto_opt_O4\", optimization_config=AutoOptimizationConfig.O4())\n",
    "\n",
    "# Load the optimized ONNX model\n",
    "ort_model = ORTModelForFeatureExtraction.from_pretrained(opt_model_path, provider=\"CUDAExecutionProvider\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "IYkxQOTRqcBe",
   "metadata": {
    "id": "IYkxQOTRqcBe"
   },
   "source": [
    "We may see some errors, but these are not ones to be concerned about. We'll see later that it does not affect the model performance.\n",
    "\n",
    "First of all, we'll create a subclass of our performance benchmark to also allow benchmarking ONNX models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8hvfl3xvlnEs",
   "metadata": {
    "id": "8hvfl3xvlnEs"
   },
   "outputs": [],
   "source": [
    "class OnnxPerformanceBenchmark(PerformanceBenchmark):\n",
    "    def __init__(self, *args, model_path, **kwargs):\n",
    "        super().__init__(*args, **kwargs)\n",
    "        self.model_path = model_path\n",
    "\n",
    "    def compute_size(self):\n",
    "        size_mb = Path(self.model_path).stat().st_size / (1024 * 1024)\n",
    "        print(f\"Model size (MB) - {size_mb:.2f}\")\n",
    "        return {\"size_mb\": size_mb}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Odn2lSPJqcBf",
   "metadata": {
    "id": "Odn2lSPJqcBf"
   },
   "source": [
    "And let's make a class that uses the tokenizer, ONNX Runtime (ORT) model and a model head."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "enaQpBF9WRn9",
   "metadata": {
    "id": "enaQpBF9WRn9"
   },
   "outputs": [],
   "source": [
    "from setfit.exporters.utils import mean_pooling\n",
    "\n",
    "\n",
    "class OnnxSetFitModel:\n",
    "    def __init__(self, ort_model, tokenizer, model_head):\n",
    "        self.ort_model = ort_model\n",
    "        self.tokenizer = tokenizer\n",
    "        self.model_head = model_head\n",
    "\n",
    "    def predict(self, inputs):\n",
    "        encoded_inputs = self.tokenizer(\n",
    "            inputs, padding=True, truncation=True, return_tensors=\"pt\"\n",
    "        ).to(self.ort_model.device)\n",
    "\n",
    "        outputs = self.ort_model(**encoded_inputs)\n",
    "        embeddings = mean_pooling(\n",
    "            outputs[\"last_hidden_state\"], encoded_inputs[\"attention_mask\"]\n",
    "        )\n",
    "        return self.model_head.predict(embeddings)\n",
    "\n",
    "    def __call__(self, inputs):\n",
    "        return self.predict(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "N1TDdcOkqcBh",
   "metadata": {
    "id": "N1TDdcOkqcBh"
   },
   "source": [
    "We can initialize this model like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "qRviEk2WWRn9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qRviEk2WWRn9",
    "outputId": "33f010a8-376e-4f0c-b21b-97fe25bf1a81"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 0], device='cuda:0')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "\n",
    "# Load the optimized ONNX model\n",
    "tokenizer = AutoTokenizer.from_pretrained(opt_model_path, model_max_length=512)\n",
    "onnx_setfit_model = OnnxSetFitModel(ort_model, tokenizer, model.model_head)\n",
    "\n",
    "# Perform inference\n",
    "onnx_setfit_model(test_dataset[\"text\"][:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3DPl1ZpYqcBh",
   "metadata": {
    "id": "3DPl1ZpYqcBh"
   },
   "source": [
    "Time to benchmark this ONNX model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "O8jpZ3gdWRn9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O8jpZ3gdWRn9",
    "outputId": "8d31c81a-67e4-4074-cf35-9f56d6dcdd20"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model size (MB) - 63.39\n",
      "Accuracy on test set - 0.882\n",
      "Average latency (ms) - 1.54 +\\- 0.10\n"
     ]
    }
   ],
   "source": [
    "pb = OnnxPerformanceBenchmark(\n",
    "    onnx_setfit_model,\n",
    "    test_dataset,\n",
    "    \"bge-small (optimum ONNX)\",\n",
    "    model_path=\"bge_auto_opt_O4/model_optimized.onnx\",\n",
    ")\n",
    "perf_metrics.update(pb.run_benchmark())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "tpjtxQQlZQPa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 455
    },
    "id": "tpjtxQQlZQPa",
    "outputId": "01efad97-4780-4c47-f10f-3afa7e819d15"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAG2CAYAAABlBWwKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABXVUlEQVR4nO3deVwV9f4/8NfhAIfDqrJTbIIiKi64gl1XFE1NizSXUlzTyDU10TRLjbRcrtbVLMV9adPScoPCpVBxwSUUBRFcQHOBA7Kf8/n9wc/z9cgiIHgYeT0fj3ngmfnMzHvmHDkvZj4zIxNCCBARERFJkIG+CyAiIiKqLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLL0GmczMTEyePBmurq5QKpXw9/dHTEyMdnpwcDBkMpnO0LNnTz1WTERERDWJoT5XPnr0aFy4cAGbNm2Ck5MTNm/ejICAAMTFxeGll14CAPTs2RPh4eHaeRQKhb7KJSIiohpGpq+HRubk5MDCwgK//PILevfurR3fqlUr9OrVCwsWLEBwcDDS09Oxa9cufZRIRERENZzejsgUFhZCrVbDxMREZ7xSqcTRo0e1r6OiomBnZ4e6deuia9euWLBgAaytrUtdbl5eHvLy8rSvNRoN7t+/D2tra8hksqrfECIiIqpyQghkZmbCyckJBgZl9IQReuTn5yc6deokbt68KQoLC8WmTZuEgYGBaNiwoRBCiG3btolffvlFnDt3TuzcuVN4e3uLNm3aiMLCwlKX+fHHHwsAHDhw4MCBA4cXYLh+/XqZWUJvp5YAIDExESNHjsThw4chl8vh6+uLhg0b4tSpU7h48WKx9levXoWHhwciIiLQrVu3Epf55BGZjIwMuLi44Pr167C0tKy2bSEiIqKqo1Kp4OzsjPT0dFhZWZXaTq+dfT08PHDo0CE8fPgQKpUKjo6OeOutt1C/fv0S29evXx82NjZISEgoNcgoFIoSOwRbWloyyBAREUnM07qF1Ij7yJiZmcHR0REPHjzA/v370a9fvxLb3bhxA/fu3YOjo+NzrpCIiIhqIr0ekdm/fz+EEPDy8kJCQgKmT5+ORo0aYcSIEcjKysInn3yCoKAgODg4IDExETNmzICnpycCAwP1WTYRERHVEHo9IpORkYGQkBA0atQIw4YNwyuvvIL9+/fDyMgIcrkc586dw2uvvYaGDRti1KhRaNWqFY4cOcJ7yRAREREAPd5H5nlRqVSwsrJCRkYG+8gQERFJRHm/v2tEHxkiIiKiymCQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIslikCEiIiLJYpAhIiIiyWKQISIiIsnSa5DJzMzE5MmT4erqCqVSCX9/f8TExGinCyEwd+5cODo6QqlUIiAgAFeuXNFjxURERFST6DXIjB49GgcPHsSmTZtw/vx59OjRAwEBAbh58yYAYPHixVixYgVWr16N48ePw8zMDIGBgcjNzdVn2URERFRDyIQQQh8rzsnJgYWFBX755Rf07t1bO75Vq1bo1asX5s+fDycnJ3zwwQeYNm0aACAjIwP29vZYv349Bg0aVK71qFQqWFlZISMjA5aWltWyLURERFS1yvv9rbcjMoWFhVCr1TAxMdEZr1QqcfToUSQlJSEtLQ0BAQHaaVZWVmjXrh2io6NLXW5eXh5UKpXOQERERC8mvQUZCwsL+Pn5Yf78+bh16xbUajU2b96M6OhopKamIi0tDQBgb2+vM5+9vb12WknCwsJgZWWlHZydnat1O4iIiEh/9NpHZtOmTRBC4KWXXoJCocCKFSswePBgGBhUvqzQ0FBkZGRoh+vXr1dhxURERFST6DXIeHh44NChQ8jKysL169dx4sQJFBQUoH79+nBwcAAA3L59W2ee27dva6eVRKFQwNLSUmcgIiKiF1ONuI+MmZkZHB0d8eDBA+zfvx/9+vWDu7s7HBwcEBkZqW2nUqlw/Phx+Pn56bFaIiIiqikM9bny/fv3QwgBLy8vJCQkYPr06WjUqBFGjBgBmUyGyZMnY8GCBWjQoAHc3d0xZ84cODk5oX///vosm4iIiGoIvQaZjIwMhIaG4saNG6hXrx6CgoKwcOFCGBkZAQBmzJiBhw8fYuzYsUhPT8crr7yCffv2FbvSiYiIiGonvd1H5nnhfWSIiIikp8bfR4aIiIjoWTHIEBERkWQxyBAREZFkMcgQERGRZDHIEBERPSYrrxDRifeQlVdY4XkL1Rrk5KtRqNY893XXVnq9/JqIiKimyc4rxLGr9+BhawZzxdO/JjUagat3H+JMygOcu5GBQo0GhgYGaPayFVq61EV9GzMYGMiqZd3EIENERFRpeYVq/Bp7C38l3EVugQZWpkYwlhsgX61B5MU7+CvhLjp42uC1Fk5QGMr1Xe4LiUGmNsvLBG7FAk4tAIXFs7cjIqpFNBqBX2NvIfLiHdhbKuBqbawz3cHSBOnZ+Yi8eAcAEOT7crmPzFD5sY9MbZb/ELh2tOhnVbQjIqpFrt59iL8S7sLeUoE6psYltqljagx7SwX+SriLpHv8HVodeESGiIjoCWqNQL5ag/zC0jvtxly7j4d5arxc1xRqTek3ybcwMUJqRi5iku7Dua5pmevNr2Qn4dqMQYZ0qQuBgoeAkRkg58eDiGqnmGv3kVughlkpHW41GoFDl/+FWgiocgueurysvEJsO5GCmw9yyjy99DCvsNR1Usm4t6iIRgMkHwUS/gCy7wGm1oBnV8D1FX1XRkRU46iFgEYIyGXl6/Mil8mg1giohYAB2E+mKjHI1HYaNVCYD1yNAs5sBGRywKQOkHEdOLkOUKsBaw99V0lE9Fy1cauHQW2dYWdhUuL0QrUGGTkFyC/UwN6y5DaPu63KhcLQAO938YShvPTuqXcyc/HDyRuVrrs2YpCp7VL+LurEe+s0kKcClHWBzNSiaTkPgEOfA06+gAmfHE5EtYfcQAZjuQGMDUsOHcaGBmjhXAcRF2/DqY7yqcvLzC1Eu8Z2MH3KaSPjMkIOlYx7jAB1AVCQDRg+8VeFoQlQkFM0nYiIdLR0qQulkRzp2flltkvPzoeJkQFautR9TpXVLjwiU9u5+AMthgCnlIDqFlDX/f+mPUgCLJ2AViOAczv0VyMRUQ1U38YMHTxttPeJKekS7PTsfNxW5aGbtx3crc2ed4m1AoNMbWcgB4xNgQbdgVPrgfQUwMQKyM0AIIAGPYqmExGRDgMDGV5r4QQA+CvhLm6r8nTu7JuRXQATIwN087bDay2ceDO8asIgQ0VcXwEgAxIigZx7RUdiPLsBrh2Ah3f0XR0RUY2kMJQjyPdl+LrWxZmUBzh/IwMFag1MDA3QprEdWrrUhbt1+Z+1RBXHIENFDAwA9/8ALn68jwwR1WqmCkO0r2/91I65jxgYyOBhaw4PW3P0b/ESCtQCRnJZmVcnVdW6iUGGniQ3BORW+q6CiEhvzBWG8POwrtS8hnIDPMuzIZ9l3bUVr1oiIiIiyWKQqc2MzQC3V4p+VkU7IiKi54ynlmozhUVRv5iqakdERPSc8YgMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUmWXoOMWq3GnDlz4O7uDqVSCQ8PD8yfPx9CCG2b4OBgyGQynaFnz556rJqIiIhqCkN9rnzRokVYtWoVNmzYgCZNmuDkyZMYMWIErKysMHHiRG27nj17Ijw8XPtaoVDoo1wiIiKqYfQaZP7++2/069cPvXv3BgC4ublh27ZtOHHihE47hUIBBwcHfZRIRERENZheTy35+/sjMjISly9fBgCcPXsWR48eRa9evXTaRUVFwc7ODl5eXhg/fjzu3btX6jLz8vKgUql0BiIiInox6fWIzMyZM6FSqdCoUSPI5XKo1WosXLgQQ4cO1bbp2bMn3njjDbi7uyMxMRGzZs1Cr169EB0dDblcXmyZYWFh+OSTT57nZhAREZGeyMTjPWufs+3bt2P69On44osv0KRJE8TGxmLy5MlYunQphg8fXuI8V69ehYeHByIiItCtW7di0/Py8pCXl6d9rVKp4OzsjIyMDFhaWlbbthAREVHVUalUsLKyeur3t16PyEyfPh0zZ87EoEGDAAA+Pj5ITk5GWFhYqUGmfv36sLGxQUJCQolBRqFQsDMwERFRLaHXPjLZ2dkwMNAtQS6XQ6PRlDrPjRs3cO/ePTg6OlZ3eURERFTD6fWITN++fbFw4UK4uLigSZMmOHPmDJYuXYqRI0cCALKysvDJJ58gKCgIDg4OSExMxIwZM+Dp6YnAwEB9lk5EREQ1gF77yGRmZmLOnDnYuXMn7ty5AycnJwwePBhz586FsbExcnJy0L9/f5w5cwbp6elwcnJCjx49MH/+fNjb25drHeU9x0ZEREQ1R3m/v/UaZJ4HBhkiIiLpKe/3N5+1RERERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJLFIENERESSxSBDREREksUgQ0RERJJlWJHGGo0Ghw4dwpEjR5CcnIzs7GzY2tqiZcuWCAgIgLOzc3XVSURERFRMuY7I5OTkYMGCBXB2dsarr76KvXv3Ij09HXK5HAkJCfj444/h7u6OV199FceOHavumomIiIgAlPOITMOGDeHn54dvv/0W3bt3h5GRUbE2ycnJ2Lp1KwYNGoTZs2djzJgxVV4sERER0eNkQgjxtEYXL16Et7d3uRZYUFCAlJQUeHh4PHNxVUGlUsHKygoZGRmwtLTUdzlERERUDuX9/i7XqaXyhhgAMDIyqjEhhoiIiF5sFers+7jCwkJ88803iIqKglqtRocOHRASEgITE5OqrI+IiIioVJUOMhMnTsTly5fxxhtvoKCgABs3bsTJkyexbdu2qqyPiIiIqFTlDjI7d+7E66+/rn194MABxMfHQy6XAwACAwPRvn37qq+QiIiIqBTlviHeunXr0L9/f9y6dQsA4Ovri3HjxmHfvn3YvXs3ZsyYgTZt2lRboURERERPKneQ2b17NwYPHozOnTtj5cqVWLNmDSwtLTF79mzMmTMHzs7O2Lp1a3XWSkRERKSjXJdfPy49PR0zZszA2bNnsXr1arRs2bK6aqsSvPyaiIhIeqr08uvH1alTB2vWrMEXX3yBYcOGYfr06cjNzX2mYomIiIgqo9xBJiUlBQMHDoSPjw+GDh2KBg0a4NSpUzA1NUXz5s2xd+/e6qyTiIiIqJhyn1rq3LkzHBwcEBwcjP379yMxMRG//vorgKI7/7777rtwcHDA999/X60FVxRPLREREUlPeb+/y3359cmTJ3H27Fl4eHggMDAQ7u7u2mne3t44fPgw1qxZ82xVExEREVVAuYNMq1atMHfuXAwfPhwRERHw8fEp1mbs2LFVWhwRERFRWcrdR2bjxo3Iy8vDlClTcPPmTXzzzTfVWRcRERHRU5X7iIyrqyt+/PHH6qyFiIiIqELKdUTm4cOHFVpoRdsTERERVUa5goynpyc+//xzpKamltpGCIGDBw+iV69eWLFiRZUVSERERFSacp1aioqKwqxZszBv3jw0b94crVu3hpOTE0xMTPDgwQPExcUhOjoahoaGCA0NxbvvvlvddRMRERFV7BEFKSkp+OGHH3DkyBEkJycjJycHNjY2aNmyJQIDA9GrVy/t07BrCt5HhoiISHrK+/1d4WctSQ2DDBERkfRU27OWiIiIiGoKBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSrAoHGTc3N3z66adISUl55pWr1WrMmTMH7u7uUCqV8PDwwPz58/H4hVRCCMydOxeOjo5QKpUICAjAlStXnnndREREJH0VDjKTJ0/Gzz//jPr166N79+7Yvn078vLyKrXyRYsWYdWqVfjqq69w8eJFLFq0CIsXL8bKlSu1bRYvXowVK1Zg9erVOH78OMzMzBAYGIjc3NxKrZOIiIheHJW+j8zp06exfv16bNu2DWq1GkOGDMHIkSPh6+tb7mX06dMH9vb2WLt2rXZcUFAQlEolNm/eDCEEnJyc8MEHH2DatGkAgIyMDNjb22P9+vUYNGjQU9fB+8gQERFJT7XfR8bX1xcrVqzArVu38PHHH+O7775DmzZt0KJFC6xbtw7lyUf+/v6IjIzE5cuXAQBnz57F0aNH0atXLwBAUlIS0tLSEBAQoJ3HysoK7dq1Q3R0dInLzMvLg0ql0hmIiIjoxVSuZy2VpKCgADt37kR4eDgOHjyI9u3bY9SoUbhx4wZmzZqFiIgIbN26tcxlzJw5EyqVCo0aNYJcLodarcbChQsxdOhQAEBaWhoAwN7eXmc+e3t77bQnhYWF4ZNPPqnsZhEREZGEVDjInD59GuHh4di2bRsMDAwwbNgwLFu2DI0aNdK2ef3119GmTZunLuv777/Hli1bsHXrVjRp0gSxsbGYPHkynJycMHz48IqWBgAIDQ3F1KlTta9VKhWcnZ0rtSwiIiKq2SocZNq0aYPu3btj1apV6N+/P4yMjIq1cXd3L1f/lenTp2PmzJnatj4+PkhOTkZYWBiGDx8OBwcHAMDt27fh6Oione/27dto0aJFictUKBRQKBQV3SwiIiKSoAoHmatXr8LV1bXMNmZmZggPD3/qsrKzs2FgoNtNRy6XQ6PRACgKRA4ODoiMjNQGF5VKhePHj2P8+PEVLZ2IiIheMBUOMnfu3EFaWhratWunM/748eOQy+Vo3bp1uZfVt29fLFy4EC4uLmjSpAnOnDmDpUuXYuTIkQAAmUyGyZMnY8GCBWjQoAHc3d0xZ84cODk5oX///hUtnYiIiF4wFb5qKSQkBNevXy82/ubNmwgJCanQslauXIk333wT7733Hry9vTFt2jS8++67mD9/vrbNjBkzMGHCBIwdOxZt2rRBVlYW9u3bBxMTk4qWTkRERC+YCt9HxtzcHOfOnUP9+vV1xiclJaFZs2bIzMys0gKfFe8jQ0REJD3Vdh8ZhUKB27dvFxufmpoKQ8NKX81NREREVGEVDjI9evRAaGgoMjIytOPS09Mxa9YsdO/evUqLIyIiIipLhQ+hfPnll+jYsSNcXV3RsmVLAEBsbCzs7e2xadOmKi+QiIiIqDQVDjIvvfQSzp07hy1btuDs2bNQKpUYMWIEBg8eXOI9ZYiIiIiqS6U6tZiZmWHs2LFVXQsRERFRhVS6d25cXBxSUlKQn5+vM/6111575qKIiIiIyqNSd/Z9/fXXcf78echkMu1TrmUyGQBArVZXbYVEREREpajwVUuTJk2Cu7s77ty5A1NTU/zzzz84fPgwWrdujaioqGookYiIiKhkFT4iEx0djT/++AM2NjYwMDCAgYEBXnnlFYSFhWHixIk4c+ZMddRJREREVEyFj8io1WpYWFgAAGxsbHDr1i0AgKurK+Lj46u2OiIiIqIyVPiITNOmTXH27Fm4u7ujXbt2WLx4MYyNjbFmzZpijy0gIiIiqk4VDjIfffQRHj58CAD49NNP0adPH/znP/+BtbU1duzYUeUFEhEREZWmwg+NLMn9+/dRt25d7ZVLNQkfGklERCQ91fLQyIKCAhgaGuLChQs64+vVq1cjQwwRERG92CoUZIyMjODi4sJ7xRAREVGNUOGrlmbPno1Zs2bh/v371VEPERERUblVuLPvV199hYSEBDg5OcHV1RVmZmY600+fPl1lxRERERGVpcJBpn///tVQBhEREVHFVclVSzUZr1oiIiKSnmq5aomIiIioJqnwqSUDA4MyL7XmFU1ERET0vFQ4yOzcuVPndUFBAc6cOYMNGzbgk08+qbLCiIiIiJ6myvrIbN26FTt27MAvv/xSFYurMuwjQ0REJD3PvY9M+/btERkZWVWLIyIiInqqKgkyOTk5WLFiBV566aWqWBwRERFRuVS4j8yTD4cUQiAzMxOmpqbYvHlzlRZHREREVJYKB5lly5bpBBkDAwPY2tqiXbt2qFu3bpUWR0RERFSWCgeZ4ODgaiiDiIiIqOIq3EcmPDwcP/zwQ7HxP/zwAzZs2FAlRRERERGVR4WDTFhYGGxsbIqNt7Ozw2effVYlRRERERGVR4WDTEpKCtzd3YuNd3V1RUpKSpUURURERFQeFQ4ydnZ2OHfuXLHxZ8+ehbW1dZUURURERFQeFQ4ygwcPxsSJE/Hnn39CrVZDrVbjjz/+wKRJkzBo0KDqqJGIiIioRBW+amn+/Pm4du0aunXrBkPDotk1Gg2GDRvGPjJERET0XFX6WUtXrlxBbGwslEolfHx84OrqWtW1VQk+a4mIiEh6yvv9XeEjMo80aNAADRo0qOzsRERERM+swn1kgoKCsGjRomLjFy9ejAEDBlRJUURERETlUeEgc/jwYbz66qvFxvfq1QuHDx+ukqKIiIiIyqPCQSYrKwvGxsbFxhsZGUGlUlVJUURERETlUeEg4+Pjgx07dhQbv337djRu3LhKiiIiIiIqjwp39p0zZw7eeOMNJCYmomvXrgCAyMhIbNu2rcRnMBERERFVlwoHmb59+2LXrl347LPP8OOPP0KpVKJZs2aIiIhAp06dqqNGIiIiohJV+j4yJblw4QKaNm1aVYurEryPDBERkfSU9/u7wn1knpSZmYk1a9agbdu2aN68+bMujoiIiKjcKh1kDh8+jGHDhsHR0RFffvklunbtimPHjlVlbURERERlqlAfmbS0NKxfvx5r166FSqXCwIEDkZeXh127dvGKJSIiInruyn1Epm/fvvDy8sK5c+ewfPly3Lp1CytXrqzO2oiIiIjKVO4gs3fvXowaNQqffPIJevfuDblc/swrd3Nzg0wmKzaEhIQAADp37lxs2rhx4555vURERPRiKHeQOXr0KDIzM9GqVSu0a9cOX331Fe7evftMK4+JiUFqaqp2OHjwIADoPLNpzJgxOm0WL178TOskIiKiF0e5g0z79u3x7bffIjU1Fe+++y62b98OJycnaDQaHDx4EJmZmRVeua2tLRwcHLTDnj174OHhoXM/GlNTU502vISaiIiIHqnwVUtmZmYYOXIkjh49ivPnz+ODDz7A559/Djs7O7z22muVLiQ/Px+bN2/GyJEjIZPJtOO3bNkCGxsbNG3aFKGhocjOzi5zOXl5eVCpVDoDERERvZie6T4yXl5eWLx4MW7cuIFt27Y9UyG7du1Ceno6goODteOGDBmCzZs3488//0RoaCg2bdqEt99+u8zlhIWFwcrKSjs4Ozs/U11ERERUc1XpnX2fRWBgIIyNjbF79+5S2/zxxx/o1q0bEhIS4OHhUWKbvLw85OXlaV+rVCo4Ozvzzr5EREQSUt47+1b4WUvVITk5GREREfj555/LbNeuXTsAKDPIKBQKKBSKKq+RiIiIap5nfkRBVQgPD4ednR169+5dZrvY2FgAgKOj43OoioiIiGo6vR+R0Wg0CA8Px/Dhw2Fo+H/lJCYmYuvWrXj11VdhbW2Nc+fOYcqUKejYsSOaNWumx4qJiIioptB7kImIiEBKSgpGjhypM97Y2BgRERFYvnw5Hj58CGdnZwQFBeGjjz7SU6VERERU09SYzr7VpbydhYiIiKjmKO/3d43oI0NERERUGQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFkMMkRERCRZDDJEREQkWQwyREREJFl6DTJubm6QyWTFhpCQEABAbm4uQkJCYG1tDXNzcwQFBeH27dv6LJmIiIhqEL0GmZiYGKSmpmqHgwcPAgAGDBgAAJgyZQp2796NH374AYcOHcKtW7fwxhtv6LNkIiIiqkFkQgih7yIemTx5Mvbs2YMrV65ApVLB1tYWW7duxZtvvgkAuHTpEry9vREdHY327duXa5kqlQpWVlbIyMiApaVldZZPREREVaS83981po9Mfn4+Nm/ejJEjR0Imk+HUqVMoKChAQECAtk2jRo3g4uKC6OjoUpeTl5cHlUqlMxAREdGLqcYEmV27diE9PR3BwcEAgLS0NBgbG6NOnTo67ezt7ZGWllbqcsLCwmBlZaUdnJ2dq7FqIiIi0qcaE2TWrl2LXr16wcnJ6ZmWExoaioyMDO1w/fr1KqqQiIiIahpDfRcAAMnJyYiIiMDPP/+sHefg4ID8/Hykp6frHJW5ffs2HBwcSl2WQqGAQqGoznKJiIiohqgRR2TCw8NhZ2eH3r17a8e1atUKRkZGiIyM1I6Lj49HSkoK/Pz89FEmERER1TB6PyKj0WgQHh6O4cOHw9Dw/8qxsrLCqFGjMHXqVNSrVw+WlpaYMGEC/Pz8yn3FEhEREb3Y9B5kIiIikJKSgpEjRxabtmzZMhgYGCAoKAh5eXkIDAzE//73Pz1USURERDVRjbqPTHXgfWSIiIikR3L3kSEiIiKqKAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikiwGGSIiIpIsBhkiIiKSLAYZIiIikixDfRdARPSs1Go1CgoK9F0GEVWAXC6HoaEhZDLZMy2HQYaIJC0rKws3btyAEELfpRBRBZmamsLR0RHGxsaVXgaDDBFJllqtxo0bN2BqagpbW9tn/suOiJ4PIQTy8/Px77//IikpCQ0aNICBQeV6uzDIEJFkFRQUQAgBW1tbKJVKfZdDRBWgVCphZGSE5ORk5Ofnw8TEpFLLYWdfIpI8HokhkqbKHoXRWUYV1EFEJAlZeYWITryHrLzCYtMK1Rrk5KtRqNZUaD4i0i+eWiKiWiM7rxDHrt6Dh60ZzBWG0GgErt59iDMpD3DuRgYKNRoYGhig2ctWaOlSF/VtzGBgICs2HxHVHDwiQ0S1Ul6hGj+dvoGv/riCyIt3kK8uCjH5ag0iL97BV39cwU+nbyCvUF3l6+7cuTMmT55c5cutqdavX486depoX8+bNw8tWrR46nxz5szB2LFjq6+wavbkdpdk5syZmDBhwvMp6AXFIENEtY5GCPwaewuRF+/ASmkELwcLOFiaoJ6ZMRwsTeDlYAErpREiL97Br7G3oOGl3c9dWloa/vvf/2L27NnaccHBwZDJZJDJZDA2Noanpyc+/fRTFBY+/ZTf+vXrtfOWNly7dq0at6hk06ZNw4YNG3D16tXnvu4XBYMMEdU6Kfey8VfCXdhbKlDHtOT7V9QxNYa9pQJ/JdzF9fvZz7lC+u677+Dv7w9XV1ed8T179kRqaiquXLmCDz74APPmzcMXX3zx1OW99dZbSE1N1Q5+fn4YM2aMzjhnZ+dy15efn1/hbSqJjY0NAgMDsWrVqipZXm3EIENEtYpaIxB7PR0P89SwMDGCWiNKHSxMjJCdr8aZ6+lVXkdhYSHef/99WFlZwcbGBnPmzNG5qV9qaip69+4NpVIJd3d3bN26FW5ubli+fLm2TXp6OkaPHg1bW1tYWlqia9euOHv2bJnrjYqKQtu2bWFmZoY6deqgQ4cOSE5OBvB/p3zWrVsHFxcXmJub47333oNarcbixYvh4OAAOzs7LFy4UGeZS5cuhY+PD8zMzODs7Iz33nsPWVlZz7R/tm/fjr59+xYbr1Ao4ODgAFdXV4wfPx4BAQH49ddf8fDhQ1haWuLHH3/Uab9r1y6YmZmhsLAQDg4O2sHY2Bimpqba1/n5+XjjjTdgbm4OS0tLDBw4ELdv39Yu59G++e677+Du7q69VDg9PR3vvvsu7O3tYWJigqZNm2LPnj06Nezfvx/e3t4wNzfXBrHH9e3bF9u3b3+m/VWbsdcaEdUqJ5LuITO3EAYGMqhyn/5Yg6y8Qtw4k41ujeyrtI4NGzZg1KhROHHiBE6ePImxY8fCxcUFY8aMAQAMGzYMd+/eRVRUFIyMjDB16lTcuXNHZxkDBgyAUqnE3r17YWVlhW+++QbdunXD5cuXUa9evWLrLCwsRP/+/TFmzBhs27YN+fn5OHHihM7l64mJidi7dy/27duHxMREvPnmm7h69SoaNmyIQ4cO4e+//8bIkSMREBCAdu3aASi6hHbFihVwd3fH1atX8d5772HGjBn43//+V6l9c//+fcTFxaF169ZPbatUKnHv3j2YmZlh0KBBCA8Px5tvvqmd/ui1hYVFqcvQaDTo168fzM3NcejQIRQWFiIkJARvvfUWoqKitO0SEhLw008/4eeff4ZcLodGo0GvXr2QmZmJzZs3w8PDA3FxcZDL5dp5srOz8eWXX2LTpk0wMDDA22+/jWnTpmHLli3aNm3btsWNGzdw7do1uLm5VWxnEYMMEdUuGlHUR8ZIVr4D0nKZDPlqDdRV3E/G2dkZy5Ytg0wmg5eXF86fP49ly5ZhzJgxuHTpEiIiIhATE6P9Mv/uu+/QoEED7fxHjx7FiRMncOfOHSgUCgDAl19+iV27duHHH38ssZOsSqVCRkYG+vTpAw8PDwCAt7e3ThuNRoN169bBwsICjRs3RpcuXRAfH4/ff/8dBgYG8PLywqJFi/Dnn39qg8zjHZfd3NywYMECjBs3rtJBJiUlBUIIODk5ldpGCIHIyEjs379f21l29OjR8Pf3R2pqKhwdHXHnzh38/vvviIiIKHN9kZGROH/+PJKSkrSnlzZu3IgmTZogJiYGbdq0AVB0Omnjxo2wtbUFABw4cAAnTpzAxYsX0bBhQwBA/fr1dZZdUFCA1atXa/f3+++/j08//VSnzaPtTE5OZpCpBAYZIqpV2rrVQ5oqB3IDA9hbPv1OordVudAIAXkV33Svffv2OkdC/Pz8sGTJEqjVasTHx8PQ0BC+vr7a6Z6enqhbt6729dmzZ5GVlQVra2ud5ebk5CAxMREpKSlo3LixdvysWbMwa9YsBAcHIzAwEN27d0dAQAAGDhwIR0dHbTs3Nzedoxf29vaQy+U6Ny6zt7fXOToUERGBsLAwXLp0CSqVCoWFhcjNzUV2djZMTU0rvG9ycnIAoMQ7ve7Zswfm5uYoKCiARqPBkCFDMG/ePABFRzaaNGmCDRs2YObMmdi8eTNcXV3RsWPHMtd38eJFODs76/SRady4MerUqYOLFy9qg4yrq6s2xABAbGwsXn75ZW2IKYmpqak2xADQBqzHPbordXY2+2JVBvvIEFGtYmRogCZOVlDlFkBuIHvqkJlbiCZOljAwqFl3D87KyoKjoyNiY2N1hvj4eEyfPh1OTk4648eNGweg6FRLdHQ0/P39sWPHDjRs2BDHjh3TLtfIyEhnPTKZrMRxGk3RjQOvXbuGPn36oFmzZvjpp59w6tQpfP311wAq3yHWxsYGAPDgwYNi07p06YLY2FhcuXIFOTk52LBhA8zMzLTTR48ejfXr12u3dcSIEVV25+fH1wOgXI/FKGnfPfmA0/v37wOATkii8mOQIaJax+clKyiN5EjPLvuLNj07HyZGBvB5yarKazh+/LjO62PHjqFBgwaQy+Xw8vJCYWEhzpw5o52ekJCg88Xu6+uLtLQ0GBoawtPTU2ewsbEpNv7xPjMtW7ZEaGgo/v77bzRt2hRbt26t9HacOnUKGo0GS5YsQfv27dGwYUPcunWr0ssDAA8PD1haWiIuLq7YNDMzM3h6esLFxQWGhsVPKrz99ttITk7GihUrEBcXh+HDhz91fd7e3rh+/TquX7+uHRcXF4f09HSdo1pPatasGW7cuIHLly+Xc8tKduHCBRgZGaFJkybPtJzaikGGiGodF2tTdPC0wW1VXqlhJj07H7dVeejgaQPnehU/PfI0KSkpmDp1KuLj47Ft2zasXLkSkyZNAgA0atQIAQEBGDt2LE6cOIEzZ85g7NixUCqV2qMLAQEB8PPzQ//+/XHgwAFcu3YNf//9N2bPno2TJ0+WuM6kpCSEhoYiOjoaycnJOHDgAK5cuVKsn0xFeHp6oqCgACtXrsTVq1exadMmrF69utLLA4o6DwcEBODo0aMVnrdu3bp44403MH36dPTo0QMvv/zyU+cJCAiAj48Phg4ditOnT+PEiRMYNmwYOnXqVGaH406dOqFjx44ICgrCwYMHkZSUpO0oXRFHjhzBf/7zHz74tJIYZIio1jGQyfBaCyd087ZDRk4B4tMykabKxf2H+UhT5SI+LRMZOQXo5m2H11o4waAaHko5bNgw5OTkoG3btggJCcGkSZN0Ouhu3LgR9vb26NixI15//XWMGTMGFhYW2n4jMpkMv//+Ozp27IgRI0agYcOGGDRoEJKTk2FvX/IVVqamprh06RKCgoLQsGFDjB07FiEhIXj33XcrvR3NmzfH0qVLsWjRIjRt2hRbtmxBWFhYpZf3yOjRo7F9+3btKayKGDVqFPLz8zFy5MhytZfJZPjll19Qt25ddOzYEQEBAahfvz527Njx1Hl/+ukntGnTBoMHD0bjxo0xY8YMqNUVuxv09u3btVerUcXJxJMn614wKpUKVlZWyMjIgKWlpb7LIaIqlJubi6SkJJ37epTljioXW46nYGg7F9hZmkCjEUi6V/SspfM3MlCg1sBIbgCf//+sJXfromctPTmfPty4cQPOzs6IiIhAt27d9FLD8ySEQLt27TBlyhQMHjy4QvNu2rQJU6ZMwa1bt2BsXPIND2uKvXv34oMPPsC5c+dKPFX2oivr/3B5v79r314jolrLVGGI9vWtYfr/H/xoYCCDh605PGzN0b/FSyhQCxjJZTCUG5Q53/Pwxx9/ICsrCz4+PkhNTcWMGTPg5ub21CtwXhQymQxr1qzB+fPnyz1PdnY2UlNT8fnnn+Pdd9+t8SEGAB4+fIjw8PBaGWKqCvccEdUa5gpD+HlYlzjNUG4AQ3mJk8qcr7oUFBRg1qxZuHr1KiwsLODv748tW7YUuwrmRdaiRYtyPVzykcWLF2PhwoXo2LEjQkNDq6+wKvT4zfuocnhqiYgkq6KnloioZqmKU0vs7EtERESSxSBDREREksUgQ0RERJLFIENERESSxSBDRLVHXiaQdKTo5/OYj4iqnd6DzM2bN/H222/D2toaSqUSPj4+OrfXDg4Ohkwm0xl69uypx4qJSLLyHwLXjhb9fB7zEVG102uQefDgATp06AAjIyPs3bsXcXFxWLJkic6j6gGgZ8+eSE1N1Q7btm3TU8VERM+uc+fOmDx5sr7LeG7Wr1+POnXqaF/PmzevXPeHmTNnjs5jG6qTm5sbli9f/lzWVVsMGjQIS5Ysqfb16DXILFq0CM7OzggPD0fbtm3h7u6OHj16wMPDQ6edQqGAg4ODdngy6DwuLy8PKpVKZyAieip1IZCbUfST9C4tLQ3//e9/MXv27Cpd7pOh6pGYmJjnFpqqyj///IOBAwfC1tYWCoUCDRs2xNy5c5Gdna3Tzs3NDTKZDMeOHdMZP3nyZHTu3Fn7et68eZDJZBg3bpxOu9jYWMhkMly7dg0A8Pvvv8PY2BinT5/WabdkyRLY2NggLS0NAPDRRx9h4cKFyMjIqKItLpleg8yvv/6K1q1bY8CAAbCzs0PLli3x7bffFmsXFRUFOzs7eHl5Yfz48bh3716pywwLC4OVlZV2cHZ2rs5NICKp02iApMPAHwuA/R8V/Uw6XDSe9Oa7776Dv78/XF1dn8v6bG1tYWpa9U85ry7Hjh1Du3btkJ+fj99++w2XL1/GwoULsX79enTv3h35+bpPdTcxMcGHH3741OWamJhg7dq1uHLlSqltXn31VQwbNgzDhg1DXl4eACAuLg4fffQRvv76azg4OAAAmjZtCg8PD2zevPkZtvTp9Bpkrl69ilWrVqFBgwbYv38/xo8fj4kTJ2LDhg3aNj179sTGjRsRGRmJRYsW4dChQ+jVq1epTxcNDQ1FRkaGdrh+/frz2hwikgKNGijM/7/hahRwch2QcR2QGxf9PLkOuHpIt10VKywsxPvvvw8rKyvY2Nhgzpw5ePxG66mpqejduzeUSiXc3d2xdevWYqc/0tPTMXr0aNja2sLS0hJdu3bF2bNny1xvVFQU2rZtCzMzM9SpUwcdOnRAcnIygP875bNu3Tq4uLjA3Nwc7733HtRqNRYvXgwHBwfY2dlh4cKFOstcunQpfHx8YGZmBmdnZ7z33nvIysp6pv2zfft29O3bV2dcXl4eJk6cCDs7O5iYmOCVV15BTEyMzrbJZDL89ttvaNasGUxMTNC+fXtcuHBBO33EiBHIyMjQ9rmcN28egOKnlmQyGb755hv06dMHpqam8Pb2RnR0NBISEtC5c2eYmZnB398fiYmJ2nmCg4PRv39/nZqfPOrRuXNnTJgwAZMnT0bdunVhb2+Pb7/9Fg8fPsSIESNgYWEBT09P7N27t9R9I4TAqFGj4O3tjZ9//hlt27aFq6srBgwYgN27dyM6OhrLli3TmWfs2LE4duwYfv/99zL3u5eXF7p06fLUI2HLli1DVlYWPv74YxQWFmL48OHo27cv3nrrLZ12ffv2xfbt28tc1rPSa5DRaDTw9fXFZ599hpYtW2Ls2LEYM2YMVq9erW0zaNAgvPbaa/Dx8UH//v2xZ88exMTEICoqqsRlKhQKWFpa6gxERFopfwPHvwGOLAEOfwEcWgT8Gw9kpgL/Xvz/P+OBQ58XTT+ypKh9FduwYQMMDQ1x4sQJ/Pe//8XSpUvx3XffaacPGzYMt27dQlRUFH766SesWbMGd+7c0VnGgAEDcOfOHezduxenTp2Cr68vunXrhvv375e4zsLCQvTv3x+dOnXCuXPnEB0djbFjx0Imk2nbJCYmYu/evdi3bx+2bduGtWvXonfv3rhx4wYOHTqERYsW4aOPPsLx48e18xgYGGDFihX4559/sGHDBvzxxx+YMWNGpffN/fv3ERcXh9atW+uMnzFjBn766Sds2LABp0+fhqenJwIDA4tt7/Tp07FkyRLExMTA1tYWffv2RUFBAfz9/bF8+XJYWlpq+1xOmzat1Drmz5+PYcOGITY2Fo0aNcKQIUPw7rvvIjQ0FCdPnoQQAu+//36Ft2/Dhg2wsbHBiRMnMGHCBIwfPx4DBgyAv78/Tp8+jR49euCdd94pdorokdjYWMTFxWHq1KkwMND9Gm/evDkCAgKK9SV1d3fHuHHjEBoaCs1TjjZ+/vnn+Omnn3QuvHmShYUF1q1bhyVLlmDo0KG4fv06Vq1aVaxd27ZtceLECe2Rm+qg1yDj6OiIxo0b64zz9vZGSkpKqfPUr18fNjY2SEhIqO7yiOhFp84HCrIBwyee02RoAhTkFE2vJs7Ozli2bBm8vLwwdOhQTJgwQftX9KVLlxAREYFvv/0W7dq1g6+vL7777jvk5ORo5z969ChOnDiBH374Aa1bt0aDBg3w5Zdfok6dOvjxxx9LXKdKpUJGRgb69OkDDw8PeHt7Y/jw4XBxcdG20Wg0WLduHRo3boy+ffuiS5cuiI+Px/Lly+Hl5YURI0bAy8sLf/75p3aeyZMno0uXLnBzc0PXrl2xYMECfP/995XeNykpKRBCwMnJSTvu4cOHWLVqFb744gv06tULjRs3xrfffgulUom1a9fqzP/xxx+je/fu8PHxwYYNG3D79m3s3LkTxsbGsLKygkwm0/a5NDc3L7WOESNGYODAgWjYsCE+/PBDXLt2DUOHDkVgYCC8vb0xadKkUv+oLkvz5s3x0UcfoUGDBggNDYWJiQlsbGwwZswYNGjQAHPnzsW9e/dw7ty5Eue/fPkygKLvy5J4e3tr2zzuo48+QlJSErZs2VJmfb6+vhg4cOBTT0V17doVb775Jr7//nusWLEC1tbFH6zq5OSE/Px8bb+Z6qDXp1936NAB8fHxOuMuX75c5jnRGzdu4N69e3B0dKzu8ojoReTiD/gOAywcAE0hIDSA6hZQ1/3/2jxIAiydgE4fAgaGQGYaEFv2L/+Kat++vc6RED8/PyxZsgRqtRrx8fEwNDSEr6+vdrqnp6fOhQ5nz55FVlZWsS+PnJwcJCYmIiUlRecPxVmzZmHWrFkIDg5GYGAgunfvjoCAAAwcOFDn96mbmxssLCy0r+3t7SGXy3X+8re3t9c5OhQREYGwsDBcunQJKpUKhYWFyM3NRXZ2dqX6nTwKbI8/RDAxMREFBQXo0KGDdpyRkRHatm2Lixcv6szv5+en/Xe9evXg5eVVrE15NGvWTPtve3t7AICPj4/OuNzcXKhUqgod/X98uXK5HNbW1sWWC6DYEbgnVfSZz7a2tpg2bRrmzp1b7BTQkxYsWABvb28cOHAAdnZ2Jba5efMm9u3bB1NTUxw5cgQDBw4s1kapVAJAqUeXqoJej8hMmTIFx44dw2effYaEhARs3boVa9asQUhICAAgKysL06dPx7Fjx3Dt2jVERkaiX79+2sOJREQVZiAHDI2LBmNToEF3AAJITym6aik9peh1gx5F0x+1rWGysrLg6OiI2NhYnSE+Ph7Tp0+Hk5OTzvhHV6KEh4cjOjoa/v7+2LFjBxo2bKhzNYuRkZHOemQyWYnjHp2euHbtGvr06YNmzZrhp59+wqlTp/D1118DQLEOp+VlY2MDoOgWHfr0+HY/Cp0ljXu0LwwMDIqFi4KCgjKX+2g5ZS33SQ0bNgSAUsPZxYsXtW2eNHXqVOTk5OB///tfidMf8fDwwJgxYzBz5sxSA9OYMWPQqlUr7NmzB6tWrcKhQ4eKtXl02s/W1rbM9T0LvQaZNm3aYOfOndi2bRuaNm2K+fPnY/ny5Rg6dCiAoqR67tw5vPbaa2jYsCFGjRqFVq1a4ciRI1AoFPosnYheFK6vAK1GFB2BUecW/Ww1AnDt8PR5n8HjfUyAoqtQGjRoALlcDi8vLxQWFuLMmTPa6QkJCTpf7L6+vkhLS4OhoSE8PT11Bhsbm2Lj69Wrp523ZcuWCA0Nxd9//42mTZti69atld6OU6dOQaPRYMmSJWjfvj0aNmyIW7duVXp5QNGXqKWlJeLi4nTGGRsb46+//tKOKygoQExMTLEuCo8HswcPHuDy5cva0zDGxsalXizyrGxtbZGamqozLjY2tsrX06JFCzRq1AjLli0rFnbOnj2LiIgIDB48uMR5zc3NMWfOHCxcuBCZmWXfqXru3Lm4fPlyiZ11v/vuOxw9ehRr165Fly5dMH78eIwcORIPH+reNPLChQt4+eWXteG0Ouj9zr59+vTB+fPnkZubi4sXL2LMmDHaaUqlEvv378edO3eQn5+Pa9euYc2aNdrDbkREz8zAAHD/D9D1I6DHgqKf7v8pGl+NUlJSMHXqVMTHx2Pbtm1YuXIlJk2aBABo1KgRAgICMHbsWJw4cQJnzpzB2LFjoVQqtX+tBwQEwM/PD/3798eBAwdw7do1/P3335g9e3apnTSTkpIQGhqK6OhoJCcn48CBA7hy5UqpfS3Kw9PTEwUFBVi5ciWuXr2KTZs26VywURkGBgYICAjA0aNHtePMzMwwfvx4TJ8+Hfv27UNcXBzGjBmD7OxsjBo1Smf+Tz/9FJGRkbhw4QKCg4NhY2OjvZrIzc0NWVlZiIyMxN27d6v0lEfXrl1x8uRJbNy4EVeuXMHHH3+svWKqKslkMqxduxZxcXEICgrCiRMnkJKSgh9++AF9+/aFn59fmTdcHDt2LKysrJ4aYO3t7TF16lSsWLFCZ3xycjKmTp2KL7/8UtsVZNGiRZDJZJg5c6ZO2yNHjqBHjx6V29By0nuQISKqEeSGgIlV0c/nYNiwYcjJyUHbtm0REhKCSZMm6dyQbePGjbC3t0fHjh3x+uuvY8yYMbCwsND2G5HJZPj999/RsWNHjBgxAg0bNsSgQYOQnJxc6h97pqamuHTpEoKCgtCwYUOMHTsWISEhePfddyu9Hc2bN8fSpUuxaNEiNG3aFFu2bEFYWFill/fI6NGjsX37dp0jDp9//jmCgoLwzjvvwNfXFwkJCdi/f3+xm6R+/vnnmDRpElq1aoW0tDTs3r0bxsZFpwf9/f0xbtw4vPXWW7C1tcXixYufudZHAgMDMWfOHMyYMQNt2rRBZmYmhg0bVmXLf5y/vz+OHTsGuVyOXr16wdPTE6GhoRg+fDgOHjxY5lkLIyMjzJ8/H7m5uU9dz7Rp03Q6RD+69NvPz0/n82pqaor169frnGLKzc3Frl27dA5QVAeZqGhvIYlRqVSwsrJCRkYGL8UmesHk5uYiKSkJ7u7uOh1DS5WZBpwMB1qPKOrsW16Vna8K3bhxA87OzoiIiEC3bt30UsPzJIRAu3btMGXKlFJPkzwpKioKXbp0wYMHD0q8ey89X6tWrcLOnTtx4MCBUtuU9X+4vN/fer1qiYjouTI2A9xeKfr5POZ7Bn/88QeysrLg4+OD1NRUzJgxA25ubujYseNzq0GfZDIZ1qxZg/Pnz+u7FKokIyMjrFy5strXwyBDRLWHwqKo/8vzmu8ZFBQUYNasWbh69SosLCzg7++PLVu2FLvi5UXWokWLcj1ckmqm0aNHP5f1MMgQEdVAgYGBvM1EBXXu3LnC91Yh6WNnXyIiIpIsBhkikjz+FU4kTVXxf5dBhogkSy6XA6j8HWSJSL8e3cfnWfp+sY8MEUmWoaEhTE1N8e+//8LIyKjYk4CJqGYSQiA7Oxt37txBnTp1tH+UVAaDDBFJlkwmg6OjI5KSkpCcnKzvcoiogurUqQMHh2e7NxODDBFJmrGxMRo0aMDTS0QSY2Rk9ExHYh5hkCEiyTMwMCjfnX2J6IXDE8pEREQkWQwyREREJFkMMkRERCRZL3wfmUc321GpVHquhIiIiMrr0ff2026a98IHmXv37gEAnJ2d9VwJERERVVRmZiasrKxKnf7CB5l69eoBAFJSUsrcEVT9VCoVnJ2dcf36dVhaWuq7nFqP70fNwfeiZuH7UTMIIZCZmQknJ6cy273wQebRnT6trKz4gawhLC0t+V7UIHw/ag6+FzUL3w/9K88BCHb2JSIiIslikCEiIiLJeuGDjEKhwMcffwyFQqHvUmo9vhc1C9+PmoPvRc3C90NaZOJp1zURERER1VAv/BEZIiIienExyBAREZFkMcgQERGRZDHIEBERkWS90EHm66+/hpubG0xMTNCuXTucOHFC3yXVSmFhYWjTpg0sLCxgZ2eH/v37Iz4+Xt9lEYDPP/8cMpkMkydP1ncptdbNmzfx9ttvw9raGkqlEj4+Pjh58qS+y6p11Go15syZA3d3dyiVSnh4eGD+/PlPfc4P6d8LG2R27NiBqVOn4uOPP8bp06fRvHlzBAYG4s6dO/ourdY5dOgQQkJCcOzYMRw8eBAFBQXo0aMHHj58qO/SarWYmBh88803aNasmb5LqbUePHiADh06wMjICHv37kVcXByWLFmCunXr6ru0WmfRokVYtWoVvvrqK1y8eBGLFi3C4sWLsXLlSn2XRk/xwl5+3a5dO7Rp0wZfffUVAECj0cDZ2RkTJkzAzJkz9Vxd7fbvv//Czs4Ohw4dQseOHfVdTq2UlZUFX19f/O9//8OCBQvQokULLF++XN9l1TozZ87EX3/9hSNHjui7lFqvT58+sLe3x9q1a7XjgoKCoFQqsXnzZj1WRk/zQh6Ryc/Px6lTpxAQEKAdZ2BggICAAERHR+uxMgKAjIwMAP/3QE96/kJCQtC7d2+d/yP0/P36669o3bo1BgwYADs7O7Rs2RLffvutvsuqlfz9/REZGYnLly8DAM6ePYujR4+iV69eeq6MnuaFfGjk3bt3oVarYW9vrzPe3t4ely5d0lNVBBQdGZs8eTI6dOiApk2b6rucWmn79u04ffo0YmJi9F1KrXf16lWsWrUKU6dOxaxZsxATE4OJEyfC2NgYw4cP13d5tcrMmTOhUqnQqFEjyOVyqNVqLFy4EEOHDtV3afQUL2SQoZorJCQEFy5cwNGjR/VdSq10/fp1TJo0CQcPHoSJiYm+y6n1NBoNWrdujc8++wwA0LJlS1y4cAGrV69mkHnOvv/+e2zZsgVbt25FkyZNEBsbi8mTJ8PJyYnvRQ33QgYZGxsbyOVy3L59W2f87du34eDgoKeq6P3338eePXtw+PBhvPzyy/oup1Y6deoU7ty5A19fX+04tVqNw4cP46uvvkJeXh7kcrkeK6xdHB0d0bhxY51x3t7e+Omnn/RUUe01ffp0zJw5E4MGDQIA+Pj4IDk5GWFhYQwyNdwL2UfG2NgYrVq1QmRkpHacRqNBZGQk/Pz89FhZ7SSEwPvvv4+dO3fijz/+gLu7u75LqrW6deuG8+fPIzY2Vju0bt0aQ4cORWxsLEPMc9ahQ4dityK4fPkyXF1d9VRR7ZWdnQ0DA92vRLlcDo1Go6eKqLxeyCMyADB16lQMHz4crVu3Rtu2bbF8+XI8fPgQI0aM0HdptU5ISAi2bt2KX375BRYWFkhLSwMAWFlZQalU6rm62sXCwqJY3yQzMzNYW1uzz5IeTJkyBf7+/vjss88wcOBAnDhxAmvWrMGaNWv0XVqt07dvXyxcuBAuLi5o0qQJzpw5g6VLl2LkyJH6Lo2eRrzAVq5cKVxcXISxsbFo27atOHbsmL5LqpUAlDiEh4fruzQSQnTq1ElMmjRJ32XUWrt37xZNmzYVCoVCNGrUSKxZs0bfJdVKKpVKTJo0Sbi4uAgTExNRv359MXv2bJGXl6fv0ugpXtj7yBAREdGL74XsI0NERES1A4MMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDBEREUkWgwwRERFJFoMMERERSRaDDFENce3aNchkMsTGxuq7FL2LioqCTCZDenp6me3c3NywfPnyCi27c+fOmDx5cqVrq6zqfH/Lu79qmvz8fHh6euLvv/+utnWsXr0affv2rbblk/4xyFCNFh0dDblcjt69e+u7lBopODgY/fv313cZVc7f3x+pqamwsrICAKxfvx516tTRb1GPqWnB4cn99bz9/PPP6NGjB6ytrSsU1lavXg13d3f4+/tXW20jR47E6dOnceTIkWpbB+kXgwzVaGvXrsWECRNw+PBh3Lp1q1rXJYRAYWFhta6DysfY2BgODg6QyWT6LkUS9L2/Hj58iFdeeQWLFi0q9zxCCHz11VcYNWpUNVZWtG+GDBmCFStWVOt6SH8YZKjGysrKwo4dOzB+/Hj07t0b69ev104bMmQI3nrrLZ32BQUFsLGxwcaNGwEAGo0GYWFhcHd3h1KpRPPmzfHjjz9q2z/6q3rv3r1o1aoVFAoFjh49isTERPTr1w/29vYwNzdHmzZtEBERobOu1NRU9O7dG0qlEu7u7ti6dWux0xzp6ekYPXo0bG1tYWlpia5du+Ls2bPl3n61Wo1Ro0Zp6/fy8sJ///tf7fR58+Zhw4YN+OWXXyCTySCTyRAVFQUAuH79OgYOHIg6deqgXr166NevH65du6ad99GRnC+//BKOjo6wtrZGSEgICgoKtG3y8vLw4YcfwtnZGQqFAp6enli7di2EEPD09MSXX36pU29sbCxkMhkSEhKKbcuFCxdgYGCAf//9FwBw//59GBgYYNCgQdo2CxYswCuvvKLz3qSnpyMqKgojRoxARkaGdjvnzZunnS87OxsjR46EhYUFXFxcyvXk6MLCQrz//vuwsrKCjY0N5syZg8cfO7dp0ya0bt0aFhYWcHBwwJAhQ3Dnzh0ARaeIunTpAgCoW7cuZDIZgoODARR95hYvXgxPT08oFAq4uLhg4cKFOuu+evUqunTpAlNTUzRv3hzR0dFPrRcAkpOT0bdvX9StWxdmZmZo0qQJfv/992L7Cyg6ffZoXz0+PPoMPOtn80nvvPMO5s6di4CAgHLPc+rUKSQmJuocbX10+u3777/Hf/7zHyiVSrRp0waXL19GTEwMWrduDXNzc/Tq1Uv7WXq0/W3btoWZmRnq1KmDDh06IDk5WTu9b9+++PXXX5GTk1PpbaQaTJ9PrCQqy9q1a0Xr1q2FEEVPCPbw8BAajUYIIcSePXuEUqkUmZmZ2va7d+8WSqVSqFQqIYQQCxYsEI0aNRL79u0TiYmJIjw8XCgUChEVFSWEEOLPP/8UAESzZs3EgQMHREJCgrh3756IjY0Vq1evFufPnxeXL18WH330kTAxMRHJycnadQUEBIgWLVqIY8eOiVOnTolOnToJpVIpli1bptOmb9++IiYmRly+fFl88MEHwtraWty7d6/E7U1KShIAxJkzZ4QQQuTn54u5c+eKmJgYcfXqVbF582ZhamoqduzYIYQQIjMzUwwcOFD07NlTpKamitTUVJGXlyfy8/OFt7e3GDlypDh37pyIi4sTQ4YMEV5eXton+Q4fPlxYWlqKcePGiYsXL4rdu3cLU1NTnScvDxw4UDg7O4uff/5ZJCYmioiICLF9+3YhhBALFy4UjRs31ql/4sSJomPHjiVum0ajETY2NuKHH34QQgixa9cuYWNjIxwcHHT21+zZs3XemwcPHoi8vDyxfPlyYWlpqd3OR++7q6urqFevnvj666/FlStXRFhYmDAwMBCXLl0qsQ4hip72bW5uLiZNmiQuXbqk3a+Pb/vatWvF77//LhITE0V0dLTw8/MTvXr1EkIIUVhYKH766ScBQMTHx4vU1FSRnp4uhBBixowZom7dumL9+vUiISFBHDlyRHz77bc672+jRo3Enj17RHx8vHjzzTeFq6urKCgoKLXeR3r37i26d+8uzp07JxITE8Xu3bvFoUOHiu0vIYS4d++edl+lpqaKN954Q3h5eYns7Gztvi7rs3n48GFhZmZW5rB58+ZiNT75GS7L0qVLRaNGjUqc/9H/27i4ONG+fXvRqlUr0blzZ3H06FFx+vRp4enpKcaNGyeEEKKgoEBYWVmJadOmiYSEBBEXFyfWr1+v8//14cOHwsDAQPz5559PrYukh0GGaix/f3+xfPlyIUTRLysbGxvtL6JHrzdu3KhtP3jwYPHWW28JIYTIzc0Vpqam4u+//9ZZ5qhRo8TgwYOFEP/3y3/Xrl1PraVJkyZi5cqVQgghLl68KACImJgY7fQrV64IANogc+TIEWFpaSlyc3N1luPh4SG++eabEtdRni+BkJAQERQUpH09fPhw0a9fP502mzZtEl5eXtrQJ4QQeXl5QqlUiv3792vnc3V1FYWFhdo2AwYM0O6/+Ph4AUAcPHiwxDpu3rwp5HK5OH78uBCiKHTZ2NiI9evXl1r7G2+8IUJCQoQQQkyePFlMnz5d1K1bV1y8eFHk5+cLU1NTceDAASFE8S/m8PBwYWVlVWyZrq6u4u2339a+1mg0ws7OTqxatarUOjp16iS8vb119s+HH34ovL29S50nJiZGANAGqCfrE0IIlUolFAqFNrg86dH7+91332nH/fPPPwKAuHjxYqnrfsTHx0fMmzevxGkl1fPI0qVLRZ06dUR8fLwQonyfzezsbHHlypUyh0d/MJS0jeUJMpMmTRJdu3Ytcf7H99G2bdsEABEZGakdFxYWJry8vIQQRaENgPYPlNI8Cpj04jF8jgd/iMotPj4eJ06cwM6dOwEAhoaGeOutt7B27Vp07twZhoaGGDhwILZs2YJ33nkHDx8+xC+//ILt27cDABISEpCdnY3u3bvrLDc/Px8tW7bUGde6dWud11lZWZg3bx5+++03pKamorCwEDk5OUhJSdHWZmhoCF9fX+08np6eqFu3rvb12bNnkZWVBWtra51l5+TkIDExsdz74euvv8a6deuQkpKCnJwc5Ofno0WLFmXOc/bsWSQkJMDCwkJnfG5urs66mzRpArlcrn3t6OiI8+fPAyg6TSSXy9GpU6cS1+Hk5ITevXtj3bp1aNu2LXbv3o28vDwMGDCg1Lo6deqkPe1z6NAhfPbZZ7h8+TKioqJw//59FBQUoEOHDmVuW0maNWum/bdMJoODg4P2NFBp2rdvr9OfxM/PD0uWLIFarYZcLsepU6cwb948nD17Fg8ePIBGowEApKSkoHHjxiUu8+LFi8jLy0O3bt3KXa+joyMA4M6dO2jUqFGZ802cOBHjx4/HgQMHEBAQgKCgIJ1llWTv3r2YOXMmdu/ejYYNGwIo32dTqVTC09OzzGU/q5ycHJiYmJQ47fHtsre3BwD4+PjojHv0HterVw/BwcEIDAxE9+7dERAQgIEDB2r37SNKpRLZ2dlVvRlUAzDIUI20du1aFBYWwsnJSTtOCAGFQoGvvvoKVlZWGDp0KDp16oQ7d+7g4MGDUCqV6NmzJ4CiMAIAv/32G1566SWdZSsUCp3XZmZmOq+nTZuGgwcP4ssvv4SnpyeUSiXefPNN5Ofnl7v+rKwsODo6avusPK68V99s374d06ZNw5IlS+Dn5wcLCwt88cUXOH78+FPX3apVK2zZsqXYNFtbW+2/jYyMdKbJZDLtF7ZSqXxqfaNHj8Y777yDZcuWITw8HG+99RZMTU1Lbf/osucrV64gLi4Or7zyCi5duoSoqCg8ePAArVu3LnP+0pS1HZXx8OFDBAYGIjAwEFu2bIGtrS1SUlIQGBhY5megPPvsyXofhany1Dt69GgEBgbit99+w4EDBxAWFoYlS5ZgwoQJJbaPi4vDoEGD8Pnnn6NHjx7a8eX5bB45cgS9evUqs55vvvkGQ4cOfWrdpbGxsdEG5yeVtI+eHPf4PgsPD8fEiROxb98+7NixAx999BEOHjyI9u3ba9vcv39f5/NPLw4GGapxCgsLsXHjRixZskTnFzAA9O/fH9u2bcO4cePg7+8PZ2dn7NixA3v37sWAAQO0v+waN24MhUKBlJSUUo8qlOavv/5CcHAwXn/9dQBFv/gf7yjr5eWFwsJCnDlzBq1atQJQdATowYMH2ja+vr5IS0uDoaEh3NzcKrEXiurw9/fHe++9px335NEcY2NjqNVqnXG+vr7YsWMH7OzsYGlpWal1+/j4QKPR4NChQ6V24Hz11VdhZmaGVatWYd++fTh8+PBTl1m3bl0sWLAALVq0gLm5OTp37oxFixbhwYMH6Ny5c6nzlrSdz+LJMHjs2DE0aNAAcrkcly5dwr179/D555/D2dkZAHDy5Mli9QDQqalBgwZQKpWIjIzE6NGjq6zWxzk7O2PcuHEYN24cQkND8e2335YYZO7evYu+ffsiKCgIU6ZM0ZlWns9m69atn3oJ9aMjJZXVsmVLrFq1CkKIKrnaqmXLlmjZsiVCQ0Ph5+eHrVu3aoNMYmIicnNzix2NpRcDr1qiGmfPnj148OABRo0ahaZNm+oMQUFBWLt2rbbtkCFDsHr1ahw8eFDnr0MLCwtMmzYNU6ZMwYYNG5CYmIjTp09j5cqV2LBhQ5nrb9CgAX7++WfExsbi7NmzGDJkiM5ff40aNUJAQADGjh2LEydO4MyZMxg7diyUSqX2F3JAQAD8/PzQv39/HDhwANeuXcPff/+N2bNnF/tSLKuOkydPYv/+/bh8+TLmzJmDmJgYnTZubm44d+4c4uPjcffuXRQUFGDo0KGwsbFBv379cOTIESQlJSEqKgoTJ07EjRs3yrVuNzc3DB8+HCNHjsSuXbu0y/j++++1beRyOYKDgxEaGooGDRrAz8+vzGXKZDJ07NgRW7Zs0YaWZs2aIS8vD5GRkWUGTjc3N2RlZSEyMhJ379595lMEKSkpmDp1KuLj47Ft2zasXLkSkyZNAgC4uLjA2NgYK1euxNWrV/Hrr79i/vz5OvO7urpCJpNhz549+Pfff5GVlQUTExN8+OGHmDFjBjZu3IjExEQcO3ZM5/P6LCZPnoz9+/cjKSkJp0+fxp9//glvb+8S2wYFBcHU1BTz5s1DWlqadlCr1eX6bD46tVTW8Pipy/v37yM2NhZxcXEAik6/xsbGIi0trdTt6dKlC7KysvDPP/88035JSkpCaGgooqOjkZycjAMHDuDKlSs6++bIkSOoX78+PDw8nmldVEPpu5MO0ZP69OkjXn311RKnHT9+XAAQZ8+eFUIIERcXJwAIV1dXnc6bQhR1/Fy+fLnw8vISRkZGwtbWVgQGBpZ6pccjSUlJokuXLkKpVApnZ2fx1VdfiU6dOolJkyZp29y6dUv06tVLKBQK4erqKrZu3Srs7OzE6tWrtW1UKpWYMGGCcHJyEkZGRsLZ2VkMHTpUpKSklLhtT3aUzM3NFcHBwcLKykrUqVNHjB8/XsycOVM0b95cO8+dO3dE9+7dhbm5uQCg7Qydmpoqhg0bJmxsbIRCoRD169cXY8aMERkZGUKIkjsJT5o0SXTq1En7OicnR0yZMkU4OjoKY2Nj4enpKdatW6czT2JiogAgFi9eXOI2PWnZsmUCgNi7d692XL9+/YShoaHOFWglvTfjxo0T1tbWAoD4+OOPhRBFnX0fv1JMCCGaN2+unV6STp06iffee0+MGzdOWFpairp164pZs2bpfH62bt0q3NzchEKhEH5+fuLXX38t1on1008/FQ4ODkImk4nhw4cLIYRQq9ViwYIFwtXVVRgZGQkXFxfx2WefCSFK7gj74MEDnfetLO+//77w8PAQCoVC2NrainfeeUfcvXu3xP0FoMQhKSlJCFHxz+bThIeHl7i+st4HIYqujJs5c6b2dUn7qKTPwuOdv9PS0kT//v21n1NXV1cxd+5coVarte179OghwsLCKrVtVPPJhHjs5glEVCk3btyAs7MzIiIintrZ80Vy5MgRdOvWDdevX3/mUw1U+5w7dw7du3dHYmIizM3Nq2Ud//zzD7p27YrLly/r7c7HVL0YZIgq4Y8//kBWVhZ8fHyQmpqKGTNm4ObNm7h8+XKxzqcvory8PPz7778YPnw4HBwcSuxYTFQe69evR6tWrXSuSqpKERERUKvVCAwMrJblk/6xjwxRJRQUFGDWrFlo0qQJXn/9ddja2iIqKqpWhBgA2LZtG1xdXZGeno7Fixfru5wXQq9evWBubl7i8Nlnn+m7vGoTHBxcbSEGKOqvxhDzYuMRGSKiGuDmzZul3kK/Xr16qFev3nOuiEgaGGSIiIhIsnhqiYiIiCSLQYaIiIgki0GGiIiIJItBhoiIiCSLQYaIiIgki0GGiIiIJItBhoiIiCTr/wGsJYkN+VYfXwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics(perf_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "a0b2ed13-0661-4c4d-8a10-e3e280f1407a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Speedup: 5.41x\n"
     ]
    }
   ],
   "source": [
    "print(f\"Speedup: {perf_metrics['bge-small (PyTorch)']['time_avg_ms'] / perf_metrics['bge-small (optimum ONNX)']['time_avg_ms']:.2f}x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gvdggvIbvowO",
   "metadata": {
    "id": "gvdggvIbvowO"
   },
   "source": [
    "By applying ONNX, we were able to improve the latency from 8.31ms per sample to 1.54ms per sample, for a speedup of 5.41x!\n",
    "\n",
    "For further improvements, we recommend increasing the inference batch size, as this may also heavily improve the throughput. For example, setting the batch size to 128 reduces the latency further down to 0.3ms, and down to 0.2ms at a batch size of 2048."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}