{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "k4bVoXnmx6Nz" }, "source": [ "This notebook is adapted from the Tensorflow\n", "research notebook\n", "[hosted on colab](https://colab.research.google.com/github/tensorflow/models/blob/master/research/object_detection/colab_tutorials/eager_few_shot_od_training_tf2_colab.ipynb#scrollTo=nyHoF4mUrv5-)\n", "and\n", "[shared (r) at github](https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/eager_few_shot_od_training_tf2_colab.ipynb)\n", "\n", "This notebook adds to the tutorial an option for android lawn statue images and tests on a YouTube video.\n", "\n", "The model configuration used by the original duckies tutorial is a TPU trained model.\n", "Other pretrained models to explore and more information:\n", "\n", "* https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/install.html#tensorflow-object-detection-api-installation\n", "* https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/auto_examples/object_detection_camera.html#sphx-glr-auto-examples-object-detection-camera-py\n", "* https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_training_and_evaluation.md\n" ] }, { "cell_type": "markdown", "metadata": { "id": "rOvvWAVTkMR7" }, "source": [ "# Eager Few Shot Object Detection Colab\n", "\n", "Welcome to the Eager Few Shot Object Detection Colab --- in this colab we demonstrate fine tuning of a (TF2 friendly) RetinaNet architecture on very few examples of a novel class after initializing from a pre-trained COCO checkpoint.\n", "Training runs in eager mode.\n", "\n", "Estimated time to run through this colab (with GPU): < 5 minutes." ] }, { "cell_type": "markdown", "metadata": { "id": "vPs64QA1Zdov" }, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LBZ9VWZZFUCT" }, "outputs": [], "source": [ "#!pip install -U --pre tensorflow==\"2.2.0\"" ] }, { "cell_type": "markdown", "source": [ "## Choose dataname by uncommenting one of the following:" ], "metadata": { "id": "CIRY3RTYg-HL" } }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "DV1wlZUZ2sAA" }, "outputs": [], "source": [ "'''\n", "duckies:\n", " this is adapted from the original tensorflow notebook (referenced above)\n", "\n", "statues:\n", " training data: 7 images, containing a mix of 8 android mascots\n", " test data: a short YouTube video of mavy android mascots\n", "\n", "gingerbread_man:\n", " training data: 3 images containing the gingerbread man mascot\n", " test data: 2 images containing the gingerbread man mascot\n", "\n", "gingerbread_man_2:\n", " training data: 5 images containing the gingerbread man mascot\n", " test data: a short YouTube video of mavy android mascots\n", "'''\n", "#dataname = \"duckies\"\n", "dataname = \"statues\"\n", "#dataname = \"gingerbread_man\"\n", "#dataname = \"gingerbread_man_2\"\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "oi28cqGGFWnY" }, "outputs": [], "source": [ "import os\n", "import pathlib\n", "import math\n", "\n", "# Clone the tensorflow models repository if it doesn't already exist\n", "if \"models\" in pathlib.Path.cwd().parts:\n", " while \"models\" in pathlib.Path.cwd().parts:\n", " os.chdir('..')\n", "elif not pathlib.Path('models').exists():\n", " !git clone --depth 1 https://github.com/tensorflow/models" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "NwdsBdGhFanc" }, "outputs": [], "source": [ "# Install the Object Detection API\n", "#\n", "# if you are running a notebook in an environment missing protoc through protobuf,\n", "# there are several ways to install protoc. one is to download a binary and put\n", "# it in your path (add that binary path to $PATH in .bash_profile or other shell init file)\n", "# https://github.com/protocolbuffers/protobuf/releases\n", "# if you're installing on macos, you may need to follow these directions to give the app permission to run\n", "# https://support.apple.com/guide/mac-help/apple-cant-check-app-for-malicious-software-mchleab3a043/mac\n", "%%bash\n", "cd models/research/\n", "protoc object_detection/protos/*.proto --python_out=.\n", "cp object_detection/packages/tf2/setup.py .\n", "python -m pip install ." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "uZcqD4NLdnf4" }, "outputs": [], "source": [ "import matplotlib\n", "import matplotlib.pyplot as plt\n", "\n", "import os\n", "import random\n", "import io\n", "import imageio\n", "import glob\n", "import scipy.misc\n", "import numpy as np\n", "from six import BytesIO\n", "from PIL import Image, ImageDraw, ImageFont\n", "from IPython.display import display, Javascript\n", "from IPython.display import Image as IPyImage\n", "\n", "import tensorflow as tf\n", "\n", "from object_detection.utils import label_map_util\n", "from object_detection.utils import config_util\n", "from object_detection.utils import visualization_utils as viz_utils\n", "from object_detection.utils import colab_utils\n", "from object_detection.builders import model_builder\n", "\n", "%matplotlib inline\n", "\n", "FIGSIZE = (8, 6)\n", "THRESH = 0.5" ] }, { "cell_type": "markdown", "metadata": { "id": "IogyryF2lFBL" }, "source": [ "# Utilities" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "-y9R0Xllefec" }, "outputs": [], "source": [ "def load_image_into_numpy_array(path : str):\n", " \"\"\"Load an image from file into a numpy array.\n", "\n", " Puts image into numpy array to feed into tensorflow graph.\n", " Note that by convention we put it into a numpy array with shape\n", " (height, width, channels), where channels=3 for RGB.\n", "\n", " Args:\n", " path: a file path.\n", "\n", " Returns:\n", " uint8 numpy array with shape (img_height, img_width, 3)\n", " \"\"\"\n", " img_data = tf.io.gfile.GFile(path, 'rb').read()\n", " image = Image.open(BytesIO(img_data))\n", " (im_width, im_height) = image.size\n", " return np.array(image.getdata()).reshape(\n", " (im_height, im_width, 3)).astype(np.uint8)\n", "\n", "def plot_detections(image_np : np.array,\n", " boxes,\n", " classes,\n", " scores,\n", " category_index,\n", " figsize=FIGSIZE, thresh=THRESH,\n", " image_name=None):\n", " \"\"\"Wrapper function to visualize detections.\n", "\n", " Args:\n", " image_np: uint8 numpy array with shape (img_height, img_width, 3)\n", " boxes: a numpy array of shape [N, 4]\n", " classes: a numpy array of shape [N]. Note that class indices are 1-based,\n", " and match the keys in the label map.\n", " scores: a numpy array of shape [N] or None. If scores=None, then\n", " this function assumes that the boxes to be plotted are groundtruth\n", " boxes and plot all boxes as black with no classes or scores.\n", " category_index: a dict containing category dictionaries (each holding\n", " category index `id` and category name `name`) keyed by category indices.\n", " figsize: size for the figure.\n", " image_name: a name for the image file.\n", " \"\"\"\n", " #print(f'classes={classes}')\n", " #print(f'category_index={category_index}')\n", " #print(f'image_np.shape={image_np.shape}')\n", " image_np_with_annotations = image_np.copy()\n", " viz_utils.visualize_boxes_and_labels_on_image_array(\n", " image_np_with_annotations,\n", " boxes,\n", " classes,\n", " scores,\n", " category_index,\n", " use_normalized_coordinates=True,\n", " max_boxes_to_draw=25,\n", " min_score_thresh=THRESH)\n", " if image_name:\n", " plt.imsave(image_name, image_np_with_annotations)\n", " else:\n", " plt.imshow(image_np_with_annotations)\n", " # consider import google.colab.patches import cv2_imshow\n" ] }, { "cell_type": "markdown", "metadata": { "id": "sSaXL28TZfk1" }, "source": [ "# Load Rubber Ducky or android statues or gingerbread man data\n", "\n", "We will start with some toy (literally) data consisting of 5 images of a rubber\n", "ducky. Note that the [coco](https://cocodataset.org/#explore) dataset contains a number of animals, but notably, it does *not* contain rubber duckies (or even ducks for that matter), so this is a novel class." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "SQy3ND7EpFQM" }, "outputs": [], "source": [ "# Load images and visualize\n", "train_images_np = []\n", "test_images_np = [] #this will be empty for video tests\n", "\n", "if dataname == \"duckies\" :\n", " train_image_dir = 'models/research/object_detection/test_images/ducky/train/'\n", " for i in range(1, 6):\n", " image_path = os.path.join(train_image_dir, 'robertducky' + str(i) + '.jpg')\n", " train_images_np.append(load_image_into_numpy_array(image_path))\n", "\n", " num_classes = 1\n", " duck_class_id = 0\n", " class_mapping = {duck_class_id:'rubber_ducky'}\n", " category_index = {i: {'id':i, 'name':cls}\n", " for i, cls in class_mapping.items()}\n", " gt_boxes = [\n", " np.array([[0.436, 0.591, 0.629, 0.712]], dtype=np.float32),\n", " np.array([[0.539, 0.583, 0.73, 0.71]], dtype=np.float32),\n", " np.array([[0.464, 0.414, 0.626, 0.548]], dtype=np.float32),\n", " np.array([[0.313, 0.308, 0.648, 0.526]], dtype=np.float32),\n", " np.array([[0.256, 0.444, 0.484, 0.629]], dtype=np.float32)\n", " ]\n", " class_ids = [[duck_class_id], [duck_class_id], [duck_class_id], [duck_class_id], [duck_class_id]]\n", "\n", " class_ids_test = []\n", " gt_boxes_test = [] #empty as we don't have bounding boxes.\n", " test_image_dir = 'models/research/object_detection/test_images/ducky/test/'\n", " for i in range(1, 50):\n", " image_path = os.path.join(test_image_dir, 'out' + str(i) + '.jpg')\n", " #test_images_np.append(np.expand_dims(\n", " # load_image_into_numpy_array(image_path), axis=0))\n", " test_images_np.append(load_image_into_numpy_array(image_path))\n", " class_ids_test.append([duck_class_id])\n", "\n", "else:\n", " #statues or gingerbread_man or gingerbread_man_2\n", "\n", " # define bounding boxes and load images\n", "\n", " # gt_boxes.shape=(batch, num_classes_in_image, 4)\n", " # [1,6,4]\n", " # [1,4,4]\n", " # [1,4,4]\n", " # [1,6,4]\n", " # [1,2,4]\n", " # [1,3,4]\n", " # [1,2,4]\n", " '''\n", " android statues 01 is from:\n", " https://www.flickr.com/photos/67287915@N00/8570385915\n", " android statues 02 is from:\n", " https://www.flickr.com/photos/quinnanya/5847206255\n", " android statues 03 and 04 are from:\n", " https://github.com/nking/curvature-scale-space-corners-and-transformations.git\n", " android statues 05 is from:\n", " https://commons.wikimedia.org/wiki/File:IceCream_Sandwich_%287791561448%29.jpg\n", " android statues 06 is from:\n", " https://upload.wikimedia.org/wikipedia/commons/thumb/3/3c/Sculpture_for_Android_Donut_at_Google_Mountain_View.jpg/320px-Sculpture_for_Android_Donut_at_Google_Mountain_View.jpg\n", " android statues 07 is from:\n", " https://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Android_Jelly_Bean_Lawn_Statue_%2812757851595%29.jpg/320px-Android_Jelly_Bean_Lawn_Statue_%2812757851595%29.jpg\n", "\n", " ytop xleft ybottom xright w h\n", " android 1 (tr) w=640 h=279\n", " 0 cupcake 134, 18, 191, 70,\n", " 1 euclair 176, 101, 244, 229,\n", " 2 icecream 129, 125, 188, 175,\n", " 3 gingerbread_man 122, 212, 193, 262,\n", " 4 icecream_sandwich 101, 280, 213, 365,\n", " 5 honey_comb 128, 391, 190, 455,\n", "\n", " android 2 (tr) w=640, h=427\n", " 0 cupcake 125, 128, 213, 212\n", " 1 euclair 186, 61, 358, 207,\n", " 2 icecream 105, 274, 233, 376\n", " 3 gingerbread_man 59, 493, 272, 594\n", "\n", " android 3 (te) w=1280, h=960\n", " 1 euclair 396, 278, 467, 497,\n", " 3 gingerbread_man 234, 57, 525, 303,\n", " 4 icecream_sandwich 227, 501, 494, 740,\n", " (obscured)\n", " 5 honeycomb 279, 702, 447, 788,\n", " 6 kitkat 228, 494, 590, 714,\n", " 7 jellybean 255, 932, 477, 1046,\n", "\n", " android 4 (tr) w=1280, h=960\n", " 0 cupcake 309, 311, 615, 605,\n", " 1 euclair 456, 998, 507, 1131,\n", " 2 icecream 329, 671, 575, 848,\n", " 3 gingerbread_man 318, 875, 552, 1041,\n", "\n", " android 5 (te) w=450, h=600\n", " 3 gingerbread_man 210, 107, 413, 177,\n", " 4 icecream_sandwich 161, 152, 507, 480,\n", "\n", " android 6 (tr) w=320, h=181\n", " 0 cupcake 33, 0, 121, 41,\n", " 6 kitkat 44, 239, 97, 261,\n", " 8 donut 0, 58, 180, 244,\n", "\n", " android 7 (tr) w=320, h=213\n", " 7 jellybean 15, 151, 197, 274,\n", " 8 donut 64, 80, 110, 112,\n", " ytop xleft ybottom xright\n", " '''\n", "\n", " def make_bb(ytop, xleft, ybottom, xright, w, h) :\n", " return [ytop/h, xleft/w, ybottom/h, xright/w]\n", "\n", " class_mapping = {0:'cupcake', 1:'euclair', 2:'icecream', 3:'gingerbread_man',\n", " 4:'icecream_sandwich', 5:'honey_comb', 6:'kitkat',\n", " 7:'jellybean', 8:'donut'}\n", "\n", " class_ids_01 = [0, 1, 2, 3, 4, 5]\n", " bb_01 = [make_bb(134, 18, 191, 70, 640, 279),\n", " make_bb(176, 101, 244, 229, 640, 279),\n", " make_bb(129, 125, 188, 175, 640, 279),\n", " make_bb(122, 212, 193, 262, 640, 279),\n", " make_bb(101, 280, 213, 365, 640, 279),\n", " make_bb(128, 391, 190, 455, 640, 279),\n", " ]\n", "\n", " class_ids_02 = [0, 1, 2, 3]\n", " bb_02 = [make_bb(125, 128, 213, 212, 640, 427),\n", " make_bb(186, 61, 358, 207, 640, 427),\n", " make_bb(105, 274, 233, 376, 640, 427),\n", " make_bb(59, 493, 272, 594, 640, 427),\n", " ]\n", "\n", " class_ids_03 = [1, 3, 4, 5, 6, 7]\n", " bb_03 = [make_bb(396, 278, 467, 497, 1280, 960),\n", " make_bb(234, 57, 525, 303, 1280, 960),\n", " make_bb(227, 501, 494, 740, 1280, 960),\n", " make_bb(279, 702, 447, 788, 1280, 960),\n", " make_bb(228, 494, 590, 714, 1280, 960),\n", " make_bb(255, 932, 477, 1046, 1280, 960),\n", " ]\n", "\n", " class_ids_04 = [0,1,2,3]\n", " bb_04 = [make_bb(309, 311, 615, 605, 1280, 960),\n", " make_bb(456, 998, 507, 1131, 1280, 960),\n", " make_bb(329, 671, 575, 848, 1280, 960),\n", " make_bb(318, 875, 552, 1041, 1280, 960),\n", " ]\n", "\n", " class_ids_05 = [3, 4]\n", " bb_05 = [make_bb(210, 107, 413, 177, 450, 600),\n", " make_bb(161, 152, 507, 480, 450, 600),\n", " ]\n", "\n", " class_ids_06 = [0, 6, 8]\n", " bb_06 = [make_bb(33, 0, 121, 41, 320, 181),\n", " make_bb(44, 239, 97, 261, 320, 181),\n", " make_bb(0, 58, 180, 244, 320, 181)\n", " ]\n", "\n", " class_ids_07 = [7, 8]\n", " bb_07 = [make_bb(15, 151, 197, 274, 320, 213),\n", " make_bb(64, 80, 110, 112, 320, 213)\n", " ]\n", "\n", " import tempfile\n", " from tempfile import TemporaryDirectory\n", "\n", " remote_url = \"https://raw.githubusercontent.com/nking/curvature-scale-space-corners-and-transformations/master/testresources/\"\n", "\n", " data_dir = tempfile.mkdtemp()\n", " print(f'temp_dataset_dir={data_dir}')\n", "\n", " # download and store images\n", " for i in range(1, 8):\n", " file_name = 'android_statues_0' + str(i) + '.jpg'\n", " req_url = remote_url + file_name\n", " !wget {req_url} -P {data_dir}\n", " image_path = os.path.join(data_dir, file_name)\n", " # statues train is all images, 1 - 7\n", " # gingerbread_man train is images 1,2,4, test is 3,5\n", " # gingerbread_man_2 train is images 1-5\n", " if dataname == \"statues\":\n", " train_images_np.append(load_image_into_numpy_array(image_path))\n", " elif dataname == \"gingerbread_man\":\n", " if i == 1 or i == 2 or i == 4:\n", " train_images_np.append(load_image_into_numpy_array(image_path))\n", " elif i == 3 or i == 5:\n", " test_images_np.append(load_image_into_numpy_array(image_path))\n", " elif dataname == \"gingerbread_man_2\":\n", " if i < 6:\n", " train_images_np.append(load_image_into_numpy_array(image_path))\n", "\n", " if dataname == \"statues\":\n", " gt_boxes = [\n", " np.array(bb_01, dtype=np.float32),\n", " np.array(bb_02, dtype=np.float32),\n", " np.array(bb_03, dtype=np.float32),\n", " np.array(bb_04, dtype=np.float32),\n", " np.array(bb_05, dtype=np.float32),\n", " np.array(bb_06, dtype=np.float32),\n", " np.array(bb_07, dtype=np.float32)\n", " ]\n", " class_ids = [class_ids_01, class_ids_02, class_ids_03, class_ids_04,\n", " class_ids_05, class_ids_06, class_ids_07]\n", " category_index = {i: {'id':i, 'name':cls}\n", " for i, cls in class_mapping.items()}\n", " # test is video\n", " elif dataname == \"gingerbread_man_2\":\n", " gt_boxes = [\n", " np.array([bb_01[3]], dtype=np.float32),\n", " np.array([bb_02[3]], dtype=np.float32),\n", " np.array([bb_03[3]], dtype=np.float32),\n", " np.array([bb_04[3]], dtype=np.float32),\n", " np.array([bb_05[3]], dtype=np.float32),\n", " ]\n", " class_ids = [[0], [0], [0], [0], [0]]\n", " category_index = {0: {'id': 0, 'name': 'gingerbread_man'}}\n", " # test is video\n", " elif dataname == \"gingerbread_man\":\n", " # gingerbread_man\n", " gt_boxes = [\n", " np.array([bb_01[3]], dtype=np.float32),\n", " np.array([bb_02[3]], dtype=np.float32),\n", " np.array([bb_04[3]], dtype=np.float32),\n", " ]\n", " class_ids = [[0], [0], [0]]\n", " gt_boxes_test = [\n", " np.array([bb_03[1]], dtype=np.float32),\n", " np.array([bb_05[0]], dtype=np.float32),\n", " ]\n", " class_ids_test = [[0], [0]]\n", " category_index = {0: {'id': 0, 'name': 'gingerbread_man'}}\n", "\n", " num_classes = len(category_index)\n", "\n", "print(f'num_classes={num_classes}')\n", "print(f'category_index={category_index}')\n", "print(f'len(gt_boxes)={len(gt_boxes)}')\n", "print(f'class_ids={class_ids}')\n", "\n", "plt.rcParams['axes.grid'] = False\n", "plt.rcParams['xtick.labelsize'] = False\n", "plt.rcParams['ytick.labelsize'] = False\n", "plt.rcParams['xtick.top'] = False\n", "plt.rcParams['xtick.bottom'] = False\n", "plt.rcParams['ytick.left'] = False\n", "plt.rcParams['ytick.right'] = False\n", "plt.rcParams['figure.figsize'] = [FIGSIZE[0], FIGSIZE[1]]\n" ] }, { "cell_type": "markdown", "metadata": { "id": "cbKXmQoxcUgE" }, "source": [ "# Annotate images with bounding boxes\n", "\n", "In this cell you will annotate the rubber duckies --- draw a box around the rubber ducky in each image; click `next image` to go to the next image and `submit` when there are no more images.\n", "\n", "If you'd like to skip the manual annotation step, we totally understand. In this case, simply skip this cell and run the next cell instead, where we've prepopulated the groundtruth with pre-annotated bounding boxes.\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "-nEDRoUEcUgL" }, "outputs": [], "source": [ "# nice tool, but can only label one object per image\n", "#gt_boxes = []\n", "#colab_utils.annotate(train_images_np, box_storage_pointer=gt_boxes)" ] }, { "cell_type": "markdown", "metadata": { "id": "wTP9AFqecUgS" }, "source": [ "# In case you didn't want to label...\n", "\n", "Run this cell only if you didn't annotate anything above and\n", "would prefer to just use our preannotated boxes. Don't forget\n", "to uncomment." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "wIAT6ZUmdHOC" }, "outputs": [], "source": [ "# moved this to a higher cell and skipped the interactive labelling\n", "# gt_boxes = [\n", "# np.array([[0.436, 0.591, 0.629, 0.712]], dtype=np.float32),\n", "# np.array([[0.539, 0.583, 0.73, 0.71]], dtype=np.float32),\n", "# np.array([[0.464, 0.414, 0.626, 0.548]], dtype=np.float32),\n", "# np.array([[0.313, 0.308, 0.648, 0.526]], dtype=np.float32),\n", "# np.array([[0.256, 0.444, 0.484, 0.629]], dtype=np.float32)\n", "# ]" ] }, { "cell_type": "markdown", "metadata": { "id": "Dqb_yjAo3cO_" }, "source": [ "# Prepare data for training\n", "\n", "Below we add the class annotations (for simplicity, we assume a single class in this colab; though it should be straightforward to extend this to handle multiple classes). We also convert everything to the format that the training\n", "loop below expects (e.g., everything converted to tensors, classes converted to one-hot representations, etc.)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "HWBqFVMcweF-" }, "outputs": [], "source": [ "\n", "\n", "# Convert class labels to one-hot; convert everything to tensors.\n", "train_image_tensors = []\n", "gt_classes_one_hot_tensors = []\n", "gt_box_tensors = []\n", "for idx, train_image_np in enumerate(train_images_np):\n", " gt_box_np = gt_boxes[idx]\n", " n = gt_box_np.shape[0]\n", " box_label_idxs = class_ids[idx] #TODO check on this\n", " train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(\n", " train_image_np, dtype=tf.float32), axis=0))\n", " gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))\n", "\n", " #print(f'{n} labeled boxes. num_classes={num_classes}')\n", " print(f'gt_box_tensors={gt_box_tensors}')\n", "\n", " #zero_indexed_groundtruth_classes = tf.convert_to_tensor(\n", " # np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32) - label_id_offset)\n", " cls_arr = np.array(box_label_idxs, dtype=np.int32)\n", " zero_indexed_groundtruth_classes = tf.convert_to_tensor(cls_arr)\n", " #print(f'zero_indexed_groundtruth_classes={zero_indexed_groundtruth_classes}')\n", "\n", " one_hot_per_box_tensor = tf.one_hot(\n", " indices=zero_indexed_groundtruth_classes, depth=num_classes)\n", " gt_classes_one_hot_tensors.append(one_hot_per_box_tensor)\n", "\n", " print(f'gt_classes_one_hot_tensors={gt_classes_one_hot_tensors}')\n", "\n", "print('Done prepping data.')" ] }, { "cell_type": "markdown", "metadata": { "id": "b3_Z3mJWN9KJ" }, "source": [ "# Let's just visualize the rubber duckies (or statues) as a sanity check\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "YBD6l-E4N71y" }, "outputs": [], "source": [ "nt = len(train_images_np)\n", "ncols = 3\n", "nrows = int(math.ceil(nt/ncols))\n", "plt_h = int(nrows * FIGSIZE[1]/2)\n", "fig = plt.figure(figsize=(FIGSIZE[0], plt_h))\n", "\n", "for idx in range(nt):\n", " n = gt_boxes[idx].shape[0]\n", "\n", " fake_pred_classes = np.array(class_ids[idx])\n", " fake_pred_scores = np.ones(shape=[n], dtype=np.float32) # give boxes a score of 100%\n", "\n", " print(f'[{idx}] box labels = {fake_pred_classes}')\n", " print(f' scores={fake_pred_scores}')\n", " print(f'gt_boxes[{idx}]={gt_boxes[idx]}')\n", "\n", " ax = plt.subplot(nrows, ncols, idx+1)\n", " ax.set_title('train ' + str(idx))\n", " plot_detections(train_images_np[idx], gt_boxes[idx],\n", " fake_pred_classes, fake_pred_scores, category_index)\n", "\n", "has_test_bounding_boxes = False\n", "try:\n", " has_test_bounding_boxes = len(gt_boxes_test) == len(test_images_np)\n", "except NameError:\n", " pass\n", "\n", "if has_test_bounding_boxes:\n", " nt = len(test_images_np)\n", " ncols = 3\n", " nrows = int(math.ceil(nt/ncols))\n", " for idx in range(len(test_images_np)):\n", " n = gt_boxes_test[idx].shape[0]\n", "\n", " fake_pred_classes = np.array(class_ids_test[idx])\n", " fake_pred_scores = np.ones(shape=[n], dtype=np.float32) # give boxes a score of 100%\n", "\n", " #print(f'[{idx}] box labels = {fake_pred_classes}')\n", " #print(f' scores={fake_pred_scores}')\n", " #print(f'gt_boxes_test[{idx}]={gt_boxes_test[idx]}')\n", "\n", " ax = plt.subplot(nrows, ncols, idx+1)\n", " ax.set_title('test ' + str(idx))\n", " plot_detections(test_images_np[idx], gt_boxes_test[idx],\n", " fake_pred_classes, fake_pred_scores, category_index)\n", "else :\n", " # a peek at subset of test images. will do noting for empty test array\n", " j = 1\n", " nt = len(test_images_np)\n", " ncols = 3\n", " nrows = int(math.ceil((nt/10.)/ncols))\n", " for idx in range(0, nt, 10):\n", " ax = plt.subplot(nrows, ncols, j)\n", " ax.set_title('test ' + str(idx))\n", " plt.imshow(test_images_np[idx])\n", " j += 1\n" ] }, { "cell_type": "markdown", "metadata": { "id": "ghDAsqfoZvPh" }, "source": [ "# Create model and restore weights for all but last layer\n", "\n", "In this cell we build a single stage detection architecture (RetinaNet) and restore all but the classification layer at the top (which will be automatically randomly initialized).\n", "\n", "For simplicity, we have hardcoded a number of things in this colab for the specific RetinaNet architecture at hand (including assuming that the image size will always be 640x640), however it is not difficult to generalize to other model configurations." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "9J16r3NChD-7" }, "outputs": [], "source": [ "# Download the checkpoint and put it into models/research/object_detection/test_data/\n", "\n", "#TODO: consider pretrained SSD models trained on architecture runnable on desktop\n", "# https://pytorch.org/hub/nvidia_deeplearningexamples_ssd/ if not jetson...\n", "# https://docs.nvidia.com/metropolis/TLT/tlt-user-guide/text/object_detection/ssd.html\n", "# https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md\n", "\n", "#consider checking for existence of models/research/object_detection/test_data/checkpoint\n", "# if not pathlib.Path('checkpoint').exists():\n", "## however, the download is less than 10 sec on collab.\n", "\n", "!wget http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.tar.gz\n", "!tar -xf ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.tar.gz\n", "!mv ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/checkpoint models/research/object_detection/test_data/\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "RyT4BUbaMeG-" }, "outputs": [], "source": [ "tf.keras.backend.clear_session()\n", "\n", "#https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/auto_examples/object_detection_camera.html#sphx-glr-auto-examples-object-detection-camera-py\n", "\n", "print('Building model and restoring weights for fine-tuning...', flush=True)\n", "pipeline_config = 'models/research/object_detection/configs/tf2/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.config'\n", "checkpoint_path = 'models/research/object_detection/test_data/checkpoint/ckpt-0'\n", "\n", "# Lists the checkpoint keys and shapes of variables in a checkpoint.\n", "# Returns: List of tuples (key, shape).\n", "#cp_list = tf.compat.v2.train.list_variables(checkpoint_path)\n", "#for cp_key, cp_shape in cp_list:\n", "# print(f'checkpoint key={cp_key}')\n", "\n", "# Load pipeline config and build a detection model.\n", "#\n", "# Since we are working off of a COCO architecture which predicts 90\n", "# class slots by default, we override the `num_classes` field here to be just\n", "# one (for our new rubber ducky class).\n", "configs = config_util.get_configs_from_pipeline_file(pipeline_config)\n", "'''\n", "config key=model\n", "config key=train_config\n", "config key=train_input_config\n", "config key=eval_config\n", "config key=eval_input_configs\n", "config key=eval_input_config\n", "'''\n", "\n", "'''\n", "configs[model] = ssd {\n", " num_classes: 90\n", " image_resizer {\n", " fixed_shape_resizer {\n", " height: 640\n", " width: 640\n", " }\n", " }\n", " feature_extractor {\n", " type: \"ssd_resnet50_v1_fpn_keras\"\n", " depth_multiplier: 1.0\n", " min_depth: 16\n", " conv_hyperparams {\n", " regularizer {\n", " l2_regularizer {\n", " weight: 0.00039999998989515007\n", " }\n", " }\n", " initializer {\n", " truncated_normal_initializer {\n", " mean: 0.0\n", " stddev: 0.029999999329447746\n", " }\n", " }\n", " activation: RELU_6\n", " batch_norm {\n", " decay: 0.996999979019165\n", " scale: true\n", " epsilon: 0.0010000000474974513\n", " }\n", " }\n", " override_base_feature_extractor_hyperparams: true\n", " #FPN is Feature Pyramid Network. FPN constructions uses features maps starting from fpn_min_levelupto the fpn_max_level.\n", " fpn {\n", " min_level: 3\n", " max_level: 7\n", " }\n", " }\n", " box_coder {\n", " faster_rcnn_box_coder {\n", " y_scale: 10.0\n", " x_scale: 10.0\n", " height_scale: 5.0\n", " width_scale: 5.0\n", " }\n", " }\n", " matcher {\n", " argmax_matcher {\n", " matched_threshold: 0.5\n", " unmatched_threshold: 0.5\n", " ignore_thresholds: false\n", " negatives_lower_than_unmatched: true\n", " force_match_for_each_row: true\n", " use_matmul_gather: true\n", " }\n", " }\n", " similarity_calculator {\n", " iou_similarity {\n", " }\n", " }\n", " box_predictor {\n", " weight_shared_convolutional_box_predictor {\n", " conv_hyperparams {\n", " regularizer {\n", " l2_regularizer {\n", " weight: 0.00039999998989515007\n", " }\n", " }\n", " initializer {\n", " random_normal_initializer {\n", " mean: 0.0\n", " stddev: 0.009999999776482582\n", " }\n", " }\n", " activation: RELU_6\n", " batch_norm {\n", " decay: 0.996999979019165\n", " scale: true\n", " epsilon: 0.0010000000474974513\n", " }\n", " }\n", " depth: 256\n", " num_layers_before_predictor: 4\n", " kernel_size: 3\n", " class_prediction_bias_init: -4.599999904632568\n", " }\n", " }\n", " anchor_generator {\n", " multiscale_anchor_generator {\n", " min_level: 3\n", " max_level: 7\n", " anchor_scale: 4.0\n", " aspect_ratios: 1.0\n", " aspect_ratios: 2.0\n", " aspect_ratios: 0.5\n", " scales_per_octave: 2\n", " }\n", " }\n", " post_processing {\n", " batch_non_max_suppression {\n", " score_threshold: 9.99999993922529e-09\n", " iou_threshold: 0.6000000238418579\n", " max_detections_per_class: 100\n", " max_total_detections: 100\n", " }\n", " score_converter: SIGMOID\n", " }\n", " normalize_loss_by_num_matches: true\n", " loss {\n", " localization_loss {\n", " weighted_smooth_l1 {\n", " }\n", " }\n", " classification_loss {\n", " # sigmoid for multi-label classification\n", " weighted_sigmoid_focal {\n", " gamma: 2.0\n", " alpha: 0.25\n", " }\n", " }\n", " classification_weight: 1.0\n", " localization_weight: 1.0\n", " }\n", " encode_background_as_zeros: true\n", " normalize_loc_loss_by_codesize: true\n", " inplace_batchnorm_update: true\n", " freeze_batchnorm: false\n", "}\n", "'''\n", "model_config = configs['model']\n", "model_config.ssd.num_classes = num_classes\n", "model_config.ssd.freeze_batchnorm = True\n", "\n", "#TODO: deprecated, so update to use createModel (Context context, String modelPath, Model.Options options)\n", "detection_model = model_builder.build(model_config=model_config, is_training=True)\n", "\n", "trainable_variables = detection_model.trainable_variables\n", "for i in range(len(trainable_variables) // 2):\n", " print(f'trainable variable[{i}] = {trainable_variables[i]}')\n", " #trainable_variables[i].trainable = False\n", "\n", "# Set up object-based checkpoint restore\n", "# RetinaNet has two prediction `heads`\n", "# --- one for classification,\n", "# --- the other for box regression.\n", "# We will restore the box regression head but initialize the classification head\n", "# from scratch (we show the omission below by commenting out the line that\n", "# we would add if we wanted to restore both heads)\n", "fake_box_predictor = tf.compat.v2.train.Checkpoint(\n", " _base_tower_layers_for_heads=detection_model._box_predictor._base_tower_layers_for_heads,\n", " # to restore the classification weights requires us to use\n", " # configs['model'].ssd.num_classes = 90 to match the checkpoint\n", " # else uncommenting the next line leads to ValueError from shape in WeightSharedConvolutionalClassHead\n", " #_prediction_heads=detection_model._box_predictor._prediction_heads,\n", " # (i.e., _prediction_heads includes the classification head that we *will not* restore)\n", " _box_prediction_head=detection_model._box_predictor._box_prediction_head,\n", " )\n", "\n", "# fake_model loads the pre-trained weights. the feature extractor is used by our classification training\n", "fake_model = tf.compat.v2.train.Checkpoint(\n", " _feature_extractor = detection_model._feature_extractor,\n", " _box_predictor = fake_box_predictor)\n", "ckpt = tf.compat.v2.train.Checkpoint(model=fake_model)\n", "ckpt.restore(checkpoint_path).expect_partial()\n", "\n", "# Run model through a dummy image so that variables are created\n", "# arg is a rank 4 image tensor: [1, height, width, channels]\n", "image, shapes = detection_model.preprocess(tf.zeros([1, 640, 640, 3]))\n", "# implies we are using batch_size=1; argument features shape is[batch_size, height, width, channels]\n", "\n", "#print(f'fake shapes from preprocess: \\n {shapes}') #[[640 640 3]]\n", "\n", "prediction_dict = detection_model.predict(image, shapes)\n", "\n", "#print(f'fake prediction from predict: \\n {prediction_dict}')\n", "# {'preprocessed_inputs'\n", "#'feature_maps'\n", "#'anchors'\n", "#'final_anchors'\n", "#'box_encodings'\n", "#'class_predictions_with_background'\n", "\n", "#class_predictions_with_background: A float tensor of shape\n", "# [batch_size, 1, num_class_slots] representing the class predictions for\n", "# the proposals.\n", "\n", "_ = detection_model.postprocess(prediction_dict, shapes)\n", "\n", "#print(f'fake postprocess = {_}')\n", "#fake postprocess = {'detection_boxes'\n", "#'detection_scores' # a tensor of shape=(1, 100). for 100 coco classes I think. follow up on this\n", "#'detection_classes': 1-1=0 so best scores are very small numbers. min_iou=0 => 1\n", " # (1-0.99) should be resolvable as an integer so will use fctr >= 100\n", " fctr = 1000.\n", " for i in range(m):\n", " for j in range(n):\n", " iou_i_j = intersection_over_union(gt_b[i], detected_b[j])\n", " costs[i][j] = round(fctr*(1. - iou_i_j))\n", "\n", " solver = pywraplp.Solver.CreateSolver(\"SCIP\")\n", "\n", " if not solver:\n", " print(f'no solver found')\n", " exit()\n", "\n", " num_workers = m\n", " num_tasks = n\n", "\n", " # Variables\n", " # x[i, j] is an array of 0-1 variables, which will be 1\n", " # if worker i is assigned to task j.\n", " x = {}\n", " for i in range(num_workers):\n", " for j in range(num_tasks):\n", " x[i, j] = solver.IntVar(0, 1, \"\")\n", "\n", " # Constraints\n", " # Each worker is assigned to at most 1 task.\n", " for i in range(num_workers):\n", " solver.Add(solver.Sum([x[i, j] for j in range(num_tasks)]) <= 1)\n", "\n", " # Each task is assigned to exactly one worker.\n", " for j in range(num_tasks):\n", " solver.Add(solver.Sum([x[i, j] for i in range(num_workers)]) == 1)\n", "\n", " # Objective\n", " objective_terms = []\n", " for i in range(num_workers):\n", " for j in range(num_tasks):\n", " objective_terms.append(costs[i][j] * x[i, j])\n", " solver.Minimize(solver.Sum(objective_terms))\n", "\n", " # Solve\n", " status = solver.Solve()\n", "\n", " # Print solution.\n", " if status == pywraplp.Solver.OPTIMAL or status == pywraplp.Solver.FEASIBLE:\n", " print(f\"Total cost = {solver.Objective().Value()}\\n\")\n", " matched = {}\n", " for i in range(num_workers):\n", " for j in range(num_tasks):\n", " # Test if x[i,j] is 1 (with tolerance for floating point arithmetic).\n", " if x[i, j].solution_value() > 0.5:\n", " if swap:\n", " matched[j] = i\n", " print(f\"Worker {j} assigned to task {i}.\" + f\" Cost: {costs[i][j]}\")\n", " else:\n", " matched[i] = j\n", " print(f\"Worker {i} assigned to task {j}.\" + f\" Cost: {costs[i][j]}\")\n", " return matched\n", " else:\n", " print(\"No solution found.\")\n", " return None\n", "\n", "def classification_loss(matched_idxs : dict, gt_class_ids : list,\n", " detection_class_ids : list, detection_scores : list):\n", " '''\n", " for use if want to calculate the classification losses based up the\n", " bipartite unbalanced weighted matching by bounding boxes. Not completely\n", " consistent to use this method because we use the detection scores and not what\n", " went into making them.\n", " '''\n", " # MCCE: -sum_over_k_classes(y_k * log(p_k))\n", " # where y_k is the indicator function. it is 1 when y_pred matches y_true, else is 0.\n", " # the p_k are the scores from the predictions. they have already been\n", " # converted to probabilities using softmax and so from_logits=False in use\n", " # of tensorflow's SparseCategoricalCrossEntropy.\n", "\n", " loss = 1E-7\n", " for idx1, idx2 in matched_idxs.items():\n", " c1 = gt_class_ids[idx1]\n", " c2 = detection_class_ids[idx2]\n", " loss += -1.*math.log(c2*c1)\n", " loss /= (len(matched_idxs))\n", " return loss\n", "\n", "# if wanted to do the mAP analysis:\n", "# use a range of thresholds of iou to decide about the class labeling:\n", "# thresh 0.5 to 0.95 with d=0.05.\n", "# - calc confusion matrix\n", "# - calc precision and recall\n", "# - calc area under precision recall curve\n", "# use iou as PR_AUC = integral from 0 to 1 of precision(rev) * d(rec)\n", "# - measure the average precision for each class, AP_i\n", "# - mAP = (1/N) * sum over i (AP_i) where N is the number of classes\n", "\n", "'''\n", "Got this error from installing ortools, but the protobuf version conflict did\n", "not prevent use of the solver:\n", "\n", "ERROR: pip's dependency resolver does not currently take into account all the\n", "packages that are installed. This behaviour is the source of the following\n", "dependency conflicts.\n", "tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3,\n", "but you have protobuf 4.25.0 which is incompatible.\n", "apache-beam 2.51.0 requires protobuf!=4.0.*,!=4.21.*,!=4.22.0,!=4.23.*,!=4.24.0,!=4.24.1,!=4.24.2,<4.25.0,>=3.20.3,\n", "but you have protobuf 4.25.0 which is incompatible.\n", "'''\n", "!pip install ortools\n", "import ortools\n", "from ortools.linear_solver import pywraplp\n", "\n", "def calc_losses(gt_b : list, gt_class_ids: list, detected_b : list,\n", " detected_class_ids : list, detected_scores : list) :\n", " '''\n", " given the ground truth bounding boxes and the detected bounding boxes,\n", " and their respective classification indexes,\n", " find the best matches and\n", " return the matched indexes, ious as localication loss, classification loss.\n", " TODO: consider adding mAP too.\n", "\n", " Note: It is not completely consistent to use this method because we use the\n", " detection scores and not what went into making them.\n", "\n", " Args:\n", " gt_b (list) : list of ground truth bounding boxes. each bounding box is\n", " a list of floating point values of length 4\n", " gt_class_ids (list) : list of ground truth class ids\n", " detected_b (list) : list of detected bounding boxes\n", " detected_class_ids (list) : list of detected class ids\n", " detection_scores (list) : list of detected scores for the bounding boxes and class ids.\n", "\n", " Returns: a dictionary with keys 'localization_loss', 'classification_loss'\n", " '''\n", "\n", " matched_idxs = calc_matched_indexes(gt_b, detected_b)\n", "\n", " if matched_idxs is None:\n", " return None\n", "\n", " classification_loss = classification_loss(matched_idxs, gt_class_ids,\n", " detected_class_ids, detected_scores)\n", "\n", " localization_loss = 0.\n", " for idx1, idx2 in matched_idxs.items():\n", " iou = intersection_over_union(gt_b[idx1], detected_b[idx2])\n", " localization_loss += (1. - iou)\n", "\n", " localization_loss /= (len(matched_idxs))\n", "\n", " return {'localization_loss': localization_loss, 'classification_loss': classification_loss}\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "WHlXL1x_Z3tc" }, "source": [ "# Load any test video and run inference with new model!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "WcE6OwrHQJya" }, "outputs": [], "source": [ "# @title\n", "\n", "# @tf.function is used to tell TensorFlow that a function should be compiled\n", "# into a graph that can be executed by TensorFlow's optimized runtime\n", "\n", "# Again, uncomment this decorator if you want to run inference eagerly\n", "@tf.function\n", "def detect(input_tensor : tf.Tensor):\n", " \"\"\"Run detection on an input image.\n", "\n", " Args:\n", " input_tensor: A [1, height, width, 3] Tensor of type tf.float32.\n", " Note that height and width can be anything since the image will be\n", " immediately resized according to the needs of the model within this\n", " function.\n", "\n", " Returns:\n", " A dict containing 3 Tensors (`detection_boxes`, `detection_classes`,\n", " and `detection_scores`).\n", " \"\"\"\n", " preprocessed_image, shapes = detection_model.preprocess(input_tensor)\n", " # model.predict returns Numpy array(s) of predictions.\n", " prediction_dict = detection_model.predict(preprocessed_image, shapes)\n", " detections = detection_model.postprocess(prediction_dict, shapes)\n", " return detections, prediction_dict\n", "\n", "# Note that the first frame will trigger tracing of the tf.function, which will\n", "# take some time, after which inference should be fast.\n", "\n", "if dataname == \"duckies\":\n", " # there are 50, so plot only 7\n", " for i in range(0, len(test_images_np), 7):\n", "\n", " input_tensor = tf.convert_to_tensor(np.expand_dims(test_images_np[i], axis=0), dtype=tf.float32)\n", " detections, prediction_dict = detect(input_tensor)\n", "\n", " #print(f\"detections['detection_boxes']={detections['detection_boxes']}\")\n", " #print(f\"detections['detection_classes']={detections['detection_classes']}\")\n", " #print(f\"detections['detection_scores']={detections['detection_scores']}\")\n", "\n", " print(f'({i}) next plot_detections')\n", " plot_detections(\n", " test_images_np[i],\n", " detections['detection_boxes'][0].numpy(),\n", " detections['detection_classes'][0].numpy().astype(np.uint32),\n", " detections['detection_scores'][0].numpy(),\n", " category_index, figsize=FIGSIZE, thresh=THRESH\n", " #, image_name=\"gif_frame_\" + ('%02d' % i) + \".jpg\"\n", " )\n", "\n", "elif dataname == \"gingerbread_man\":\n", " for i in range(len(test_images_np)):\n", " input_tensor = tf.convert_to_tensor(np.expand_dims(test_images_np[i], axis=0), dtype=tf.float32)\n", " detections, prediction_dict = detect(input_tensor)\n", "\n", " print(f\"detections['detection_boxes']={detections['detection_boxes']}\")\n", " print(f\"detections['detection_classes']={detections['detection_classes']}\")\n", " print(f\"detections['detection_scores']={detections['detection_scores']}\")\n", "\n", " bb = gt_boxes_test[i].tolist()\n", " labels = class_ids_test[i]\n", " pred = detections['detection_boxes'][0].numpy().tolist()\n", " plot_detections(\n", " test_images_np[i],\n", " detections['detection_boxes'][0].numpy(),\n", " detections['detection_classes'][0].numpy().astype(np.uint32),\n", " detections['detection_scores'][0].numpy(),\n", " category_index, figsize=FIGSIZE, thresh=THRESH)\n", " # find matching bounding boxes\n", " #breakpoint()\n", " matched_idxs = calc_matched_indexes(bb, pred)\n", " print(f'matched_idxs={matched_idxs}')\n", "else:\n", " #android statues video as test\n", " # video title: \"Google HQ Android Statues\"\n", " # creator: Polsky Morillo\n", " video_url = \"https://www.youtube.com/watch?v=BRKLw_16Lac\"\n", "\n", " !pip install opencv-python\n", " !pip install --upgrade yt-dlp\n", " !apt -y install ffmpeg lame\n", "\n", " import imutils\n", " import cv2\n", "\n", " def get_frame(time, frame_count, filepath) :\n", " vid_cap.set(cv2.CAP_PROP_POS_MSEC, time)\n", " # youtube frame rate options: 24 to 60 frames/sec\n", " frame_det, frame = vid_cap.read()\n", " print(f'read correctly={frame_det}')\n", " if frame_det:\n", " frame = imutils.resize(frame, width=640) #smaller width of 128?\n", " print(f'writing to {filepath}')\n", " return cv2.imwrite(filepath, frame) #return True if written\n", " else:\n", " return None\n", "\n", " stream_uri = os.path.join(data_dir, \"android_statues.mp4\")\n", "\n", " #https://pypi.org/project/yt-dlp/\n", " #Video Format Options\n", " # --check-formats\n", "\n", " #!yt-dlp $video_url --list-formats\n", " '''\n", " [info] Available formats for BRKLw_16Lac:\n", " ID EXT RESOLUTION FPS CH │ FILESIZE TBR PROTO │ VCODEC VBR ACODEC ABR ASR MORE INFO\n", " ────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n", " sb3 mhtml 48x27 3 │ mhtml │ images storyboard\n", " sb2 mhtml 79x45 1 │ mhtml │ images storyboard\n", " sb1 mhtml 159x90 1 │ mhtml │ images storyboard\n", " sb0 mhtml 319x180 1 │ mhtml │ images storyboard\n", " 233 mp4 audio only │ m3u8 │ audio only unknown Default\n", " 234 mp4 audio only │ m3u8 │ audio only unknown Default\n", " 599 m4a audio only 2 │ 110.08KiB 31k https │ audio only mp4a.40.5 31k 22k ultralow, m4a_dash\n", " 600 webm audio only 2 │ 13.08KiB 4k https │ audio only opus 4k 48k ultralow, webm_dash\n", " 139 m4a audio only 2 │ 174.06KiB 49k https │ audio only mp4a.40.5 49k 22k low, m4a_dash\n", " 249 webm audio only 2 │ 13.08KiB 4k https │ audio only opus 4k 48k low, webm_dash\n", " 250 webm audio only 2 │ 13.08KiB 4k https │ audio only opus 4k 48k low, webm_dash\n", " 140 m4a audio only 2 │ 459.75KiB 130k https │ audio only mp4a.40.2 130k 44k medium, m4a_dash\n", " 251 webm audio only 2 │ 13.08KiB 4k https │ audio only opus 4k 48k medium, webm_dash\n", " 17 3gp 176x144 7 1 │ 248.81KiB 70k https │ mp4v.20.3 mp4a.40.2 22k 144p\n", " 597 mp4 256x144 15 │ 113.67KiB 32k https │ avc1.4d400b 32k video only 144p, mp4_dash\n", " 602 mp4 256x144 15 │ ~375.83KiB 104k m3u8 │ vp09.00.10.08 104k video only\n", " 598 webm 256x144 15 │ 141.00KiB 40k https │ vp9 40k video only 144p, webm_dash\n", " 269 mp4 256x144 30 │ ~617.65KiB 170k m3u8 │ avc1.4D400C 170k video only\n", " 160 mp4 256x144 30 │ 393.15KiB 111k https │ avc1.4D400C 111k video only 144p, mp4_dash\n", " 603 mp4 256x144 30 │ ~668.17KiB 184k m3u8 │ vp09.00.11.08 184k video only\n", " 278 webm 256x144 30 │ 362.32KiB 103k https │ vp09.00.11.08 103k video only 144p, webm_dash\n", " 229 mp4 426x240 30 │ ~ 1.13MiB 318k m3u8 │ avc1.4D4015 318k video only\n", " 133 mp4 426x240 30 │ 871.24KiB 247k https │ avc1.4D4015 247k video only 240p, mp4_dash\n", " 604 mp4 426x240 30 │ ~ 1.23MiB 347k m3u8 │ vp09.00.20.08 347k video only\n", " 242 webm 426x240 30 │ 803.84KiB 228k https │ vp09.00.20.08 228k video only 240p, webm_dash\n", " 230 mp4 638x360 30 │ ~ 2.83MiB 800k m3u8 │ avc1.4D401E 800k video only\n", " 134 mp4 638x360 30 │ 2.18MiB 631k https │ avc1.4D401E 631k video only 360p, mp4_dash\n", " 18 mp4 638x360 30 2 │ 2.55MiB 735k https │ avc1.42001E mp4a.40.2 44k 360p\n", " 605 mp4 638x360 30 │ ~ 2.37MiB 668k m3u8 │ vp09.00.21.08 668k video only\n", " 243 webm 638x360 30 │ 1.47MiB 426k https │ vp09.00.21.08 426k video only 360p, webm_dash\n", " '''\n", "\n", " if (not os.path.exists(stream_uri)):\n", " print(f'downloading youtube file')\n", " # default is ffmpeg, can choose mp4 with -f\n", " !yt-dlp $video_url -vU -f mp4 -o $stream_uri\n", " '''\n", " [youtube] Extracting URL: https://www.youtube.com/watch?v=BRKLw_16Lac\n", " [youtube] BRKLw_16Lac: Downloading webpage\n", " [youtube] BRKLw_16Lac: Downloading ios player API JSON\n", " [youtube] BRKLw_16Lac: Downloading android player API JSON\n", " [youtube] BRKLw_16Lac: Downloading m3u8 information\n", " ERROR: [youtube] BRKLw_16Lac: Requested format is not available.\n", " Use --list-formats for a list of available formats\n", " ...\n", " [debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto\n", " [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id\n", " [info] BRKLw_16Lac: Downloading 1 format(s): 18\n", " [debug] Invoking http downloader on \"https://rr2---sn-npoldn7s.googlevideo.com/videoplayback?expire=1700180177&ei=cVxWZb78DIOZ3LUPq7WkIA&ip=34.124.190.55&id=o-ABYYepNgKahbwzodzQLY1cI5jo4X2FgV_7-a4v57a-2k&itag=18&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=-v&mm=31%2C26&mn=sn-npoldn7s%2Csn-a5m7lnl6&ms=au%2Conr&mv=m&mvi=2&pl=23&initcwndbps=5193750&spc=UWF9f8kgIgBkhETrsm2hHAQAVuedUjg&vprv=1&svpuc=1&mime=video%2Fmp4&gir=yes&clen=2669045&ratebypass=yes&dur=29.048&lmt=1694608020841733&mt=1700157908&fvip=4&fexp=24007246&beids=24350018&c=ANDROID&txp=5318224&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ANLwegAwRgIhAMAqjIs-alzT8ZigFN1dr-txOKvx54pQ93O3f2h0X-d-AiEAquZ2bo8ZKdBYRtyUYOJlmRozRkF1WZPREMBT3gIDy5k%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AM8Gb2swRQIgVn2fb2Xc4YPnMR9JpFyQ84cHuGtLFXT9AvwsOQcYgdICIQCWxhVd3Sc9iRaR4zkRYwr-dFiy0IOAXRBC9BpyoPB2Ng%3D%3D\"\n", " [download] Destination: /tmp/tmphwbs6im8/android_statues.mp4\n", " [download] 100% of 2.55MiB in 00:00:01 at 1.44MiB/s\n", " '''\n", " else:\n", " print(f'have already downloaded YouTube mp4 file')\n", "\n", " # the video is 35 sec long\n", " vid_cap = cv2.VideoCapture(stream_uri)\n", " if not vid_cap.isOpened():\n", " print(f'video capture is not opened yet. try now:')\n", " vid_cap.open(video_url)\n", " print(f'vid_cap.isOpened()={vid_cap.isOpened()}')\n", "\n", " #vid_cap.set(cv2.CV_CAP_PROP_FRAME_WIDTH, 640)\n", "\n", " fps = vid_cap.get(cv2.CAP_PROP_FPS)\n", " print(f'fps={fps}')\n", " print(f'CAP_PROP_FRAME_COUNT={vid_cap.get(cv2.CAP_PROP_FRAME_COUNT)}')\n", " num_frames = int(vid_cap.get(cv2.CAP_PROP_FRAME_COUNT))\n", " if (fps > 0):\n", " end_time = int(num_frames/fps)*1000\n", " else:\n", " end_time = 35*1000\n", " print(f'end_time={end_time}')\n", "\n", " pred_all = []\n", " scores_all = []\n", "\n", " start_time = 0\n", " frame_rate = 2000 #in millisec = 2 sec\n", " frame_count = 1\n", " for time in range(start_time, end_time, frame_rate):\n", " time = round(time, 2)\n", " filepath = os.path.join(data_dir, str(frame_count) + '.jpg')\n", " result = get_frame(time, frame_count, filepath)\n", " frame_count += 1\n", " if result is not None:\n", " img = load_image_into_numpy_array(filepath)\n", " input_tensor = tf.convert_to_tensor(np.expand_dims(img, axis=0), dtype=tf.float32)\n", " detections, prediction_dict = detect(input_tensor)\n", " dc = detections['detection_classes'][0].numpy().astype(np.uint32)\n", " ds = detections['detection_scores'][0].numpy()\n", " plot_detections(\n", " img,\n", " detections['detection_boxes'][0].numpy(),\n", " dc, ds,\n", " category_index, figsize=(15, 20), thresh=THRESH\n", " #, image_name=filepath\n", " )\n", " plt.show()\n", " # useful for making confusion matrix, prec, recall, mAP, etc:\n", " p = []\n", " s = []\n", " for ii in range(len(ds)):\n", " if ds[ii] >= THRESH:\n", " p.append(dc[ii])\n", " s.append(ds[ii])\n", " print(f\"pred_{frame_count-1}={p}\")\n", " print(f\"score_{frame_count-1}=\",[\"{0:0.2f}\".format(ii) for ii in s])\n", " pred_all.extend(p)\n", " scores_all.extend(s)\n", " else:\n", " print(f'get_frame failed')\n", "\n", " vid_cap.release()\n", " cv2.destroyAllWindows()\n", "\n", " print(f'the id to class mapping={class_mapping}')\n", " print(f\"pred_all= {pred_all}\")\n", " print(\"scores_all=\",[\"{0:0.2f}\".format(ii) for ii in scores_all])\n", "\n", "plt.show() #plt.show() should be called only once per python session\n", "print('done')" ] }, { "cell_type": "markdown", "metadata": { "id": "AYePKT31_7ms" }, "source": [ "# **Misc Notes**\n", "\n", "---\n", "Variations tried for android lawn statues:\n", "* (1) For a train set being android statues 01, 02, and 04, and a test set being android statues 03 and 05:\n", "There were 100 detections for each of the test images.\n", "The train set includes classes 0 through 5, inclusive, but not 6 or 7. The detector did not report any classes as 6 or 7, as expected because those were not in the train set.\n", "> android statues 03 test had 3 scores above 0.5. Of those with scores >= 0.5, only the bounding box for detection[1] matched the expected bounding box and classification. To look at details further, I used an unbalanced weighted bipartite matching for the bounding boxes where the cost was 1-iou between 2 boxes. IOU is intersection over union a.k.a. Jaccard index. The matches by IOU showed matched_idxs={1: 1, 5: 14, 0: 21, 2: 22, 4: 45, 3: 66} which correspond to classes 3:3, 7:5, 1:1, ... The 3rd match is correct too, but has score 0.117.\n", "> android statues 05 test. Looks like non-max supression failed for the first 2 detection\n", "bounding boxes. predicted 1st class correctly, but with wrong bounding box.\n", "> * If I truly wanted to keep the training set this small, then a way to improve the detections might be to\n", "reduce the number of classes as that would improve the scores for the expected detections. e.g. Could reduce the classes to gingerbread man only, as the duckies example does. (In progress...)\n", "> Increasing the training set number of images and using a video to test, that is, put all my old train and test images into train, and then use a short video from YouTube to test. (done, can see this now when dataname == \"statues\")\n", "\n", "* (2) For a train set being android statues 01, 02, 03, 04, and 05, and a test set being several frames from a short YouTube video of the android statues:\n", "> The detector sometimes identified non-statue objects in the image as statues, such as a truck or a bench.\n", "> * We are currently using transfer learning as a pre-trained model built from ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.config.\n", "We use the feature extractor and bounding box detector from the pretrained model and re-train for classification (which uses feature extractor).\n", "To identify the non-android statue objects we could use the object detector without retraining (it identifies COCO class objects). We note those\n", "identifications and scores.\n", "And then identify android statues with our re-trained model. The identified android statue bounding boxes can be compared to the identified COCO bounding boxes and one chosen over the other.\n", "(note: COCO has 80 classes (640x480).\n", "PascalVOC has 20 classes ().\n", "imagenet1K has 1000 classes (~470x~390).\n", "open-image has 600 classes.)\n", "> * I added train image(s) for the android donut, lolli-pop and jelly-bean to improve the training.\n", "\n", "* (3) For a train set being android statues 01, 02, 03, 04, 05, 06, and 07 and a test set being several frames from a short YouTube video of the android statues:\n", "> precision, recall, f1 score and accuracy were calculated as a function of score threshold and a threshold of 0.5 was found to result in a macro averaged precision, recall, f1 score and accuracy > 0.72. For that reason, THRESH above has been set to 0.5 now.\n", "Here are the statistics derived from THRESH 0.2 to 1.0, which can also be found [here](https://github.com/nking/curvature-scale-space-corners-and-transformations/raw/master/doc/statues_transfer_learning_object_detection.pdf)\n", "```\n", "class id class name #of images w/ class in it\n", "0 cupcake : 4\n", "1 euclair: 4\n", "2 icecream: 3\n", "3 gingerbread_man: 5\n", "4 icecream_sandwich: 3\n", "5 honeycomb: 2\n", "6 kitkat: 2\n", "7 jellybean: 2\n", "8 donut: 2\n", "```\n", "```\n", "14 test frames were taken from a YouTube video.\n", "```\n", "ground_truth_labels_all=[3, 9, 2, 2, 3, 9, 5, 2, 2, 2, 2, 2, 3, 2, 8, 8, 2, 8, 8, 3, 8, 9, 9, 3, 6, 3, 5, 8, 9, 8, 0, 0, 9, 4, 9, 9, 9, 9, 8, 0, 0, 4, 0, 4, 4, 9, 9, 0, 9, 0, 9, 0, 3, 0, 3, 7, 1, 6, 9, 1, 9, 6]\n", "```\n", "predicted_labels_all= [3, 7, 0, 0, 3, 7, 5, 1, 0, 2, 0, 2, 3, 1, 8, 8, 2, 1, 8, 3, 1, 4, 7, 5, 4, 3, 5, 8, 7, 1, 0, 0, 0, 7, 0, 0, 7, 0, 0, 7, 7, 7, 0, 8, 7, 5, 0, 0, 7, 0, 4, 7, 3, 7, 8, 2, 1, 7, 1, 1, 7, 6]\n", "```\n", "scores_all= ['0.29', '0.25', '0.55', '0.45', '0.39', '0.33', '0.32', '0.32', '0.30', '0.29', ‘0.25', '0.25', '0.53', '0.38', '0.29', '0.62', '0.32', '0.31', '0.66', '0.45', '0.43', '0.29', '0.28', '0.27', '0.32', '0.63', '0.47', '0.44', '0.42', '0.32', '0.29', '0.28', '0.26', '0.52', '0.48', '0.45', '0.37', '0.33', '0.30', '0.28', '0.28', '0.44', '0.34', '0.39', '0.36', '0.35', '0.33', '0.31', '0.31', '0.28', '0.27', '0.27', '0.61', '0.48', '0.35', '0.28', '0.35', '0.28', '0.50', '0.47', '0.38', '0.35']\n", "```\n", "\n", "![statues.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAHQCAYAAAB9bRT0AAABhWlDQ1BJQ0MgcHJvZmlsZQAAKJF9kTtIw1AUhv+mSn1UHMwg4pChOlkQFXHUKhShQqgVWnUwuekLmjQkKS6OgmvBwcdi1cHFWVcHV0EQfIC4ujgpukiJ5yaFFrEeuNyP/57/595zAaFWYrrdMQ7ohmMl4zEpnVmVQq/ohYgBBNCtMNuck+UE2tbXPfVR3UV5Vvu+P6tPy9oMCEjEs8y0HOIN4ulNx+S8TyyygqIRnxOPWXRB4keuqz6/cc57LPBM0Uol54lFYinfwmoLs4KlE08RRzTdoHwh7bPGeYuzXqqwxj35C8NZY2WZ67SGEcciliBDgooKiijBQZR2gxQbSTqPtfEPeX6ZXCq5imDkWEAZOhTPD/4Hv2dr5yYn/KRwDOh8cd2PESC0C9Srrvt97Lr1EyD4DFwZTX+5Bsx8kl5tapEjoH8buLhuauoecLkDDD6ZiqV4UpCWkMsB72f0TRlg4BboWfPn1jjH6QOQolklboCDQ2A0T9nrbd7d1Tq3f3sa8/sBCB5yfLlrH9MAAAAGYktHRAD/AP8A/6C9p5MAAAAJcEhZcwAADzoAAA86AZc528IAAAAHdElNRQfnCxQTAikB2peEAAAgAElEQVR42uzdd3RU1d7G8WdmkklPCElIIBCQ3hSkg5dyUQSU6gW9ioUuTURUXqyISr9SRBABAS8qiCLSRekgKIJKkw4hoSQkJCE9mZlz3j+UXCOgpIiBfD9rZSlzZs7M/KY9Z+999raYpmkKAAAAhcZKCQAAAAhYAAAABCwAAAACFgAAAAhYAAAABCwAAAACFgAAAAhYAAAABCwAAAACFgAAAAhYAAAABCwAAAACFgAAAAELAAAABCwAAAACFgAAAAELAAAABCwAAAACFgAAAAELAAAABCwAAAACFgAAAAELAAAABCwAAAACFgAAAAELAACAgAUAAAACFgAAAAELAACAgAUAAAACFgAAAAELAACAgAUAAAACFgAAAAELAACAgAUAAAACFgAAAAELAACAgAUAAEDAAgAAAAELAACAgAUAAEDAAgAAAAELAACAgAUAAEDAAgAAAAELAACAgAUAAEDAAgAAAAELAACAgAUAAEDAAgAAIGABAACAgAUAAEDAAgAAIGABAACAgAUAAEDAAgAAIGABAACAgAUAAEDAAgAAIGABAAAQsAAAAEDAAgAAIGABAAAQsAAAAEDAAgAAIGABAADcgtwoAf5OhmHINE0KAaBIslqtslgsFAIELNxc9u7dq23btsnNjbcigKLDNE05HA79+9//VlhYGAUBAQs3l7NnzyowMFANGzbkKBFAkWEYhhYtWqSsrCyKAQIWbj4Wi0URERGqXr06xQBQpAJWWFiYrFaGKiN/eOcAAAAUMlqwUCQ5nU45nc4i97gsFovc3d05qgUAELBwc0lJSVFiYqLc3d2L3LgswzBkGIZCQkLk4eHBiwUAIGCh6HO5XIqPj1epUqXk5eVVJB/jxYsXlZycrJCQEF4wAAABC0Xf5XmxPDw8imw3nJeXl1JSUmSaJmc+AgCuioEkQB4RqgAAf4YWLEBSVlaWDMOQy+WSxWKRl5cXA9kBAAQs3NgwcuTIEcXFxalatWoqW7Zsru0ul0snT55UdHS0wsLCVK1aNdlstr/s8VxeaacgDUsffvihvvvuO9ntdiUnJ2vIkCFq1KgRLzYAIF84REeeJSQkaNWqVVq6dKm2bdt2xfajR49q5MiRunjxokaPHq2ffvop38EpK0tKT7/23+nTpubOdenIEfMPr5eZ+b8gdjWZmZmSpEmTJqlfv3764IMPeKEBAPlGCxbyLCwsTM8//7xWrlyp7OzsK7avW7dOjz76qDp37iy73a6PPvpI9evXz1fAenOWS3O/unYy8rBKbqY0doVLmca193X/nRa9O8omd/erb7dYLKpevbq8vLxUvnx5xcTEyDAMugmBwjw4Oxut4wf3qnzF6gooFZrroMdikaw2mySLDKdTpq7cZpFFLpfzioOly2uZ/n6bxSLZbG6Ki47UmagTqnpHfZUIK80LAQIWiqbLk21eq9svNjZWDRs2lNVqVUREhGJjY3OFlZSUFJ04cUKmaerEiROqW7fuNe5H6tXVqq73XP1xOJ3Shq2Glm42dU8Dizq3tepaU1N5e0p/1EtpmqZOnjyp9PR0RUdHKzg4mHAFFCKX06nJr4zUvA8/Vt+HmuqH83alZ/3vqCg0wKrR3cso2M9NfWad1qWM/yWlQB+LJjwSriA/dz2zIEpRCa5fPreSSnhb9fYT4QrwcdP/fXRGh887ZPl1W0lfq6Y/UVZTlx7RRyt3a9ATPfXSe/N5MUDAwk36pnJzk8v1yxegw+GQzWbLdebdpUuX9P3338vpdOrw4cN/GLAqRvzxwKoaFa1qWc9UteoWBQUV7Oy+6OhoTZgwQUePHlWfPn14IYFCZJimtny7Qecd0vEj59WmS1/Z3D1ztnvaLfIu6y+bp1Vt2yXJ6fpfwLK7W+RV1l9WT5ta3J2UK5i5u1tkDw+QxW5Vs5ZJqpVu5LqdJdhbx0/u0Lks6fSxA7wQIGCh6DJNM2cpG6fTKYfDIavVqtTUVPn5+aly5cqKjIxU48aNdfDgQdWoUSNXwCpbtqz69esn0zS1evVqmX80OOpP+PlZ1Oyugk+bYLFY1K5dOz355JMyTZPWK6CQZaWmKSYlW/273qdnRrykao2bXvk5/PW/TzbTNbf1anrtbY82ufq2VyfcplZrvlCHx/vxQoCAhaIrKSlJU6dO1a5du2QYhqKiovT4449rzJgxmjhxotq1a6dJkyZp69at8vHx0YgRI64ZaorKnFKXl+UhWAF/jXUL3lNGcoqeenm0qt1Z/w8/+3/0rZCfbXXuvld3/PMeWfh8g4CFoiwgIEDPPPNMTjegm5ubfH19NWHCBPn6+srX11evv/66MjMzZbfb5ePjU+Sf06OPPsoLC/xFDKdTu06cVO0KlRUSFHTDD6wsFossf+FUMQABC4XCarWqRIkSV1zu7++f8//e3t7y9va+aZ5TUV33ELgVpMfHyj32Z7Xr2knBZcpSEBCwAAAoKLvjkv6vTapcEdVlu9ZcKcAthg5pAMBfxpWdrQPfbNeZVA+51WxBQVBs0IIFAPjLZGZmaOqsOZLF0Kz+IRQExQYtWMDvFGTaCAC5JcXHa/eZeLVq1Ul2dw8KgmKDFixA0qZNm3ThwgXFxcXp9ttvV8uWLSkKUAh+2LhWabFR+ke3rnLzIGCh+KAFC0WaaRgyXa5r/hmpqXIcPCgjKekPr2f+OqXEtRw8eFALFy5UkyZNVK1aNQoPFALD6dL+U6dVqWINBfoFUBAUK7RgoUiHq8zlS+Q8vP/a10lLk+vwz7JWqiLrVaaOyDmSCI+Q9yN9ZHG7+lvearWqefPmatCgAYUHCklWfIyqp+1XyP0tVLJ0OAUBAQsoCiyS5OMrS0iZq4crp1Ou49/ITI6RccYua0QlWa7RBWHxL/HL4oZ/ICCAI2ygUH9gXCm6u3KcMoLay+bOzw0IWEDRYLXK8577rr3dNOVs3U7pny+VZ9u2st9eR/qDpTCKyrI8QHGxaeVaJR53qdODD1MMELCAouTP1g5zr1JV/s+N+GUZjAIEqMaNGxPAgEK2dOWnSk6R/hUYSDFAwAJurgRmuea4qryoX78+tQQK0cXoaG05GKl+j/WWlXUAUQxxFiEAoNAd2v+TkqPOq26DO2S18lMDAhYAAAViOF2KOhuj22rVVKUKVSgICFgAABSUKz1ZFSM/1UsPNle5mrdTEBRLjMECABQqMytVNcKOK7NMJ6ZnQLFFCxbwK4fDoezsbBmGQTGA/IYrw9CaTz7VyE9OyVL3bgqCYotDC0BSdHS0XnjhBd1xxx2qU6eOzp07p5CQEN13330M0AXyErBMU98eOqioc/6yenhTEBRb/HKg2MvOztbmzZuVkpKiVq1ayel0KjExUatXr6Y1C8ij2JMntG/fYd3fb6QCgkIpCAhYQBE+Jr72n2lKRrZkGn98PZnX3LvD4dDp06eVnp6uyMhI3XvvvercuTMTjwL5cPzwPv38/Q7VvKOm3L08KQiKLboIUYRzlSkj+YyUnXaNK1hkZl2QK2qJrKU7yOpb4dpBys1D1hLlJcuVxxQ+Pj5q0qSJYmNj9eCDD/561yb1B/L6kXUZij57Tj6lAlS5QmUKAgIWUES/ruU8/KlcZzZdc7vpTJKc6VLUblncS+pajbLWoNvl0WL0VQMWgMJhODLVIGWr5jxaT+E1a1EQELCAoskie70nZdbtfY1vc6ecZzbKdewT2SLaya1iZ8lmv8aubJLl+t7uqampSkxMVGZmpi5evKiSJUvK3d2dlwP4E+lJifJ0j1PFO7qKHnYQsIAim68skruP/uh72r3SA3Ir11oWdz/J5pHvu7Lb7fL19ZUkLV++XAsXLlRUVJSGDx+uF198UbVqcTQO/JmNny7S4iXHNPmjjhQDBCxKgJua1U0Wz+AC76ZFixZq0aKFJKlHjx7q0aMHtQXyaMeBnxR7yVceXkzPADAgBQBQYMnxcfp+z89q0KqVAkoGURAQsCgBAKDAASs1VWejjqlelQqy2e0UBAQsSgAAKAjD5VLM3r26vXI5NWzRjoIABCwAQIE5s1Rm33RN6FRJFWpVpx4AAQsAUFCXLsTpUOJJuVdsLYuNKU0AAhYAoMB2b9+iwfMidancnbK6EbAAAhYAoEBMw9TRs2dVutRtCgjg7EHgMubBQtH/Ar/KuoCXF2J2OBxKTk6Wr6+v7L+euXR5229v92cLNx86dEjvvvuuEhIS1KhRI/Xq1Ut+fn4UH/gTcVGR2vL5MjW+v4NKV6pKQQACFm4GKSkpGjNmTK7LgoODNWjQILlcLo0ePVq7d+9WtWrVFBgYqODgYA0fPlzZ2dmaPHmykpOT5XK59OyzzyosLOya91OyZEkNGjRInp6emjlzpr7//nu1bt2aFwD4ExcT4/XNzu/VsWsnuTE9A0DAws3B6XTq8OHDuS4LCwuTYRhyOBzavn27vvvuO0VGRqpWrVoqW7asTNOUYRg6fvy4EhIS5HA4lJWV9Yf3ExMTo/nz5ys5OVknTpxQzZo1KT5wHdKPH1TrRtI9nR/605ZigIAFFBGBgYH6/PPPr7jcYrHI29tbM2fO1Jw5c/TQQw/prrvuks1my9k2d+7cXNe/FsMwtGjRIrVp00atWrXS3Llz5XQ6KT5wHSombdfUfzdTUNUqFAMgYOFmYrVe/VwMi8WiunXravLkybLb7bLZbNd1u6vtv3r16lqxYoXOnDmjjRs3qkOHDhQe+BMJZ85oZ2SkKt3eTQxvBwhYuMXCl5eXV4H3061bN91xxx0yDEN33323PD09KS7wJ37auUUD3tqgxauGUwyAgAVcydfXV/Xq1aMQwHUyTVNHos8qqHQ5lQkpTUGA3zcAUAIAQF4ZLqe2ff6pGjWtq9vuqEtBAAIWAKCgnA6ndu/frfqVqkucPQgQsHBzuNrkokWFy+XidHQU78+nYShm3QoN6xSue7s9SEGAq2AMForWG9LNTR4eHkpMTJS/v3+RCzIul0uXLl2Sn58fIQvFOWHJ6/QGPVi3igLKMP4KIGChyLNYLAoLC1NcXJwuXLhQJB+jr6+v/P39ebFQbCWcj9HsH06obet2qufHBA0AAQs3BZvNptDQ0CLbTWixWGi9QrF26uhBzV+2TQ079paNKU0AAhYKj2EYcjqdslqtObOnX2aaplwulwzDkM1mu2ICUEIMcBN/9p1OHT56Qla/IFWsWI2CAAQsFBaHw6Fly5Zp9erVCg0N1eDBg1W+fPmc7efPn9f06dMVExOjevXqqU+fPvL29qZwwC0gPTVZG5Z8pFoVyyqiRi0KAlwDZxEiz/bu3aulS5fq9ddfV7NmzTRnzhwZhpGzfe7cubrzzjs1YcIEJScna+XKlRQNuFUCVnqmfog+q3b1GsnNjWN0gICFQmGapk6ePKl27dopIiJC9erVU1RUVK6AlZqaqpCQEAUFBSkgIEDr16+/Yh8ulyunGxHATfQdsH+r5vYI0IPPjZCNgAVcE58O5DlgpaenKyjolzOH3NzcZJpmrgHpHTp00Lvvvqt169bp5MmTV4zBOn36tObMmaPs7GydPHlSTz/9NIUFborPv2TG/6hyIWEKKBnEBKMAAQuFxWq1qkSJEoqKipIkZWVlXTHIvUWLFrrrrruUlpamNWvWaO/evbn2Ua5cOb344ouSpC+//JKiAjeJpJjzGvnRN2rZrKF6enpREOCPfi8pAfKqVq1a2rhxo7Zv367FixerQYMGio+P16xZs5SZmakTJ05oz5492rNnj1auXKmePXvmur3NZpOPj498fHzk6elZpGdtB/A/F+JjteO73XL3CpIlH2cHA8UJLVjIs0qVKumFF17Qli1bVK5cOd13332SpODg4JzpFbZt2yabzabnnntOVapUoWjALeDIj3tlDwhVkxb3UgyAgIXCZrVa1bBhQzVs2DDX5d26dZMkVaxYUc8++yyFAm4xa+ZNUa2KZVSh7p0UA/iz30pKAAD4M860dJ2I26uG9ZrJauWnA/gztGABAP5U4s71mt2rhkq0uU9Wxl8Bf4rDEADAHzJchi4lRMnmHia7TygFAQhYAICCSoo5p3FzlmrhuTvkGVGVggAELABAQV1Mitfa9ZsVHhQkm91OQQACFgCgIEzD0P5vv9d5SY1bMT0DQMACABSYYZraufVrtfCVylZlTjuAgAUAKLCsxESV8kzWE8++KE9PbwoCXCemaQAAXDtg7d+m3rcnyOueB+Tu6UlBgOtECxYA4KpMw9Cx86d05KJNplcJCgIQsAAABZWVkaF3P1ytuT94yKfcbRQEIGABAAoq2+HQvmOHdOdtVSSWxwEIWACAgvt+5QpdTHPq7kcHUAyAgAUAKAxff7VMlUr6KywinGIAecRZhACAK2QnJqpumVTVqd5RPr7+FAQgYAEAChywzp9Q+4qX5Fajvjy8mf8KyCu6CAEAuRgul/YdP60lP7iUGViTggAELABAQaUnJ+m/b0/W5vNB8qlUjYIABCwAQEGlpKTqq907VK9CJbm5u1MQgIAFACiovZs36dQlqX3PfnJzI2ABBCwAQIEdPXVALcr5KyQkSLJYKAiQD4V+FqFpmjIMQzabrUg/cYfDIXeavgEgl6z4ODX0i9VtQ/rKPyiEggD5VGgtWJs2bVJ6erpSUlI0bdq0IvlkV61apaysLEnS888/r7S0NN4BAPAb2XGnVMk8oOY1azA9A1AUAtbWrVuVlpYm0zQVGxsrSXI6nVdc7/eXXe06l1vBrnmE9WtIutrtpF9ap65m7dq1yszMlCSdPHlSpmnm/Pu3XC4X7wwAxdJ3+45q5raLclZoQjGAAiiULsJt27bpiy++0KlTp/Sf//xH+/fv19NPP624uDg9/PDD6tixo/bu3atRo0apatWq8vb21iuvvKJXX31Vdrtdu3fv1vPPP68WLVpozZo1WrRokXx8fHTXXXfpsccey7mfS5cu6YknnlDTpk21atUqzZs3TwkJCfriiy+0ZcsWvf322xozZoxq1qyp5ORkTZs2TdZfFyj97LPPtGHDBj3zzDOaN2+eHA6HevTooUaNGmnnzp367LPPtGfPHk2ZMkXh4eGqVKmShg4dyjsEQLGyavEHOn7WXSVr1KAYwN8dsJo3b64uXbpo0KBBcnd3l6enp6ZNm6bs7Gz17NlTbdu21YgRIzR79mz5+PhowYIFWrZsmXbu3Kk5c+Zo5MiR2r59u1JSUjR58mQtXrxYTqdTzzzzjNq0aaOwsDBJv7RMvfjii6pdu7ZKlSqlqKgo+fn5SfqlBa13794aM2aMSpUqpbVr12r58uXq2rWrJKlbt27avHmzxowZI+mXVqp58+YpKChICxcu1LfffqtXXnlFn376qaxWq8aPH6/Dhw+revXqvEsAFAvply7pm0M/64E2HWUt4uNogWIRsC673EVXuXJlSZLdbldaWprS0tKUkJCg2bNn5wx+Dw8P16xZs/TBBx/ohx9+0MCBA3X27FmlpaXp7bffliRVqVLlivt4++23ValSJcXHx6tcuXKSpNtvv10Oh0MxMTH673//K7vdLklq0KDBNR+jp6dnzvW8vb116tQppaamasaMGbJYLPL19ZWFs2cAFCO7v1qjE8fO6a4ZD1AMoKgELIvFoqysLHl4eFyxzd/fX+XKldPw4cNVokQJrVmzRjExMdq1a5deeeUVpaenq02bNtq6dauCgoL08ssvy2q1auLEibnO9Fu6dKl69eqlli1b6sMPP8wZp2W32+Xh4aFKlSpp0KBBCg8P1+7du68YX2WxWJSdnX3Vx1+5cmWVKlVKI0eOlKenp+bNm3fV5wIAt6rIuFjVrRSqMoElKAZQQIU2yL1p06YaMGCAHA5HTsuQJHl5eclms2nKlCnq27evnnrqKe3fv18dO3aU1WrV4MGDNWzYML355pvy9PTUCy+8oIcffliDBg1SRESESpYsmbOvNm3aaMKECRoyZIiys7M1ceJE2Ww22Ww2Wa1WjR49Ws8884yGDh2qpUuXqlmzZrkeY+PGjdW7d++cx3W5hcpqtcrDw0OTJ09Wjx49NGTIEDkcDkVERPAOAVAsOFKS1cp+QFOf7aayVWtREKCALOblPrMbwDRNZWVlydPTM+eyrKws2Ww2ubn9rzHN5XLJ5XLlCmqXOZ1OmaYpd3f3a7aYZWRkyMvLK1+P0TAMORwOWq9ukNWrV8vX11ctW7akGMDfKCM2Sqfn3qvg6n0V/K/nin09DMPQrFmz1LFjx5zhKEBe3NCZ3C0WS65wJUkeHh65wpUk2Wy2q4YrSXJzc8vpNrxWCMpvuJL+15oFAMUmTLhc2rX9Oz390REll29BQYCbLWABAIoel8up1Us/VmaajwJorQEIWACAgku5mKifzsarfafH5ePjS0GA4hawHA6Hli9frsTERF45ACgkh3ZtU+TP+9TiX/+Sp68fBQGKW8Bau3atpF/mrQIAFI5LKbEqFeit0BKBFAMoJIU2D9aOHTu0d+9e+fr66sEHH5SHh4cOHDigb775RhaLRW3btlXZsmW1fv16JSQkKCkpSa1bt9a3334rp9OpBx54QIGBgVq3bp2OHz+uiIgI3X///TlL3Zw+fVqff/65qlevrjZt2kiSli9frjNnzig8PFydO3fW6tWrFRwcrAMHDqhv3768ugDwJxwpl1Q/e7++eK6lStRgegagsBRKC9aFCxf0+uuvq1OnTipbtqzmzp0rSRo4cKDuvfde1alTR2+88YZcLpfatWunOnXqqEKFCurfv78aNWqk6tWra9KkSdq8ebOOHDmiTp06KSYmRjt37sy5j5IlS6pixYqqU6eO3N3dNW/ePMXExKhDhw5KTk7We++9p9WrV2v58uVXncEdAHClrPRUnTz6g9Kc5eXGGdRA0QpYJUuWVNWqVTVu3Dj5+vqqZ8+ekqRFixbp/PnzWrFiRc78VT169FDNmjVVtWpV3X333apevboaNWqks2fPat26dTmLNycmJmratGk59+Hn55ezCLPVatXmzZvVqVMnlS9fXu3bt9e2bduUmJiowYMHq27duryyAHAdvtu0Vb0XR0l3PUIxgEJUKF2ELpdLb7zxhux2uz755BONHTtW77zzjgYMGKCPPvpIFStW1CuvvCJJOWsRWiyWnO6/y4KCgtS5c2dVqlRJLpdLKSkpV70/i8UiLy8vORwOSb9MCHd5RncbC5QCwHXbvHaxSvt5K7jibRQDKESF0oKVkpKibt266dixYwoNDVWJEiWUnp4uSYqJidGsWbN04cKFnEB0LXfddZcmTZqkQ4cOadSoUTp+/PjVH7TVqvbt2+uNN97Q/v379dJLL+mBBx6Qnx9nvwDA9UpLSNS2w2d0Z+PWstvpHgQKU6EtlZOYmKiTJ0/K399fERER8vDwUHR0tM6fP6+KFSvq/Pnzqlq1qpKTkxUSEqLs7GylpqaqZMmSMk1T8fHxCgkJUUxMjE6dOqXbbrtNoaGhOesFSlJSUpK8vLxyZlqPiYnRiRMnVLlyZYWGhio+Pl4BAQG5FohG0cZSOcDf5+i+n9SuVVO9OWaMHhk4nIL8BkvloKAK7SzCwMBA1a9fP9dl5cqVy3ljBgcHS5JCQkIkSXa7PWchZ4vFknN5WFiYwsLCrnofJUrkXuH999e9fB8AgD9mmqZsaYl6oGlZ3VHtdgoCFNWABQC4iQJWVoa8d83X6/9uKq+WrSgIUMhYKgcAiqH09DRt3POxolLKSRZ+CoBbKmB9/fXXMgxDWVlZysrKkiR99dVXf3r9v0pSUpJ++OGHAu3j8uN3OBxasmSJTp8+rfXr1+dpH8nJybo8NG7jxo05tQGAwrJ740Y9utCl+ErNZSFgAUU/YP12zPyfjZ8fNWqUXC6XtmzZoj179kj6ZeD6tfZ3+fp/tP/fX/Znj+G32y9duqRVq1YVaP+XH//WrVvl7u6u8PBwnT9/Pk91e+2115ScnCxJio2Nzdl2redSSOcp5El8fLz279+vEydOKDs7O9c2l8ulyMhI7du3TydPnvzTs0cB3FimaWrf4QNqFCCViwiXfnMyEYDCUWhjsGbOnCmr1arTp0+rS5cu2rNnj86dO6cuXbqoQYMGmj59uvr27SsvLy9t27ZNpUuXliSlpaVp4cKFstvtCg4OVlxcnCQpIyND06ZNU1pampo1a6b27dvn3FdaWlrOtrp166p79+7Kzs7We++9p5iYGEVEROjJJ5/Uli1btH79etntdg0aNEhBQUG5HvP69eu1adMmeXh4qF+/frm2ffnll9q+fbusVqv69Omj0qVLX9f+4+LiFBsbqzlz5igwMFDNmjXLeU4JCQmaMWOGHA6H7rnnHrVo0UI///yzPvnkExmGoa5du+rcuXPasmWLxo0bp/HjxyspKUmGYWjPnj1atmyZLBaLHnnkEdWoUUPz58+Xp6en9u/fr4iICA0YMOCGvGliY2P16quv6s4779SPP/6odu3aqUuXLjlnfO7YsUPz589X+/bt9eWXX6pTp07q3LkznzagiLh0PkY/7jmgNr2GKbRsBQoC/EVHMoVCknnkyBHz+PHj5h133GHu37/fPH36tNmhQwfTNE2zY8eOZnJysmmapvnZZ5+Zu3btMps2bWpmZGSYCxcuNJcuXWpmZGSYtWvXNg3DMPv06WPu2bPHjIuLM1999VXz4MGDZtOmTc3s7Gxz6NCh5u7du82LFy+a999/v5mVlWVu377dnDRpkpmQkGDOnj3bXLFihXnPPfeYsbGx5pEjR8zXX3891+ONi4szu3TpYsbExJiHDh0y+/TpY0ZGRpqjR482s7OzzUcffdSMi4szt23bZo4dO/a691+7dm0zOzvbnD17trlmzRozKyvLrFWrlmmaptmhQwfz0KFDZmxsrNm3b18zOjra/Mc//mGeO3fOPHnypNmmTRszPj7efPzxx82DBw+apmmaXbt2Nc+ePWs2aNDAjI6ONk+dOmXWr1/fdDgc5oABA6txvrUAACAASURBVMwVK1aYcXFx5jvvvGPu3r3bvBG+/PJL8+WXXzYzMzPN/fv3m/369TMdDkfO9gULFpizZs0yMzMzzffee88cN27cNfe1atUqc/PmzSaAG2fvji1mFbvMBTOnUYxrcLlc5owZM8yoqCiKgXwptBashg0bqmrVqkpPT1fFihVVo0YN2Wy2nNaba7HZbAoKCpKfn588PT1lsViUnJys5ORk1apVSx4eHho9enSu27z66qvau3ev5s6dq9WrVys7O1u1atXSlClTFB8fr/bt26tx48Y6ffq0+vfvr169eunJJ5/MtY8dO3ZowIABCg0NVWhoqObOnavTp09Lktzd3fXcc89p48aNWrdunSpUqHDd+7dYLHJ3d1dwcLBCQ0Nlt9tlsVgUGxurMmXKqGrVqrJarZozZ44kacGCBdqzZ4+2bt2q9PR0lShRQn5+fipTpkzO/jZs2KBhw4apbNmykqQHHnhAhw8fls1mU7169RQcHKwGDRpcV1dkYXQtxMbGql69erLb7QoMDFRGRkaubsomTZroxRdf1M6dO5WZmamXXnop1z7S09MVFRUl0zQVHR2tmjVrcqQD3CCG4dLGFZ/rWLb0zepV6tjpAZUML0thgEJWaGOwfr/szdVc7kL6/Tir3267vK/MzMycH+19+/YpMjJS0i+DxwcOHKjSpUtrxIgR6tWrlywWi7KysvThhx9q5MiR+uabb/Txxx+rS5cu+vzzz+Xn56fmzZvnuj9vb29duHAhJzQsX74853EcOHBA77//vrp06aIXXnhBFotFmZmZedr/77m5uSkxMTHn39988412796tIUOGqEWLFho9enTOvGC/r0dYWJiio6Nz/h0dHa3AwMArrme5QeMorFarnE5nTu1+f9+7d+/OeU4tW7bUF198kev2cXFx2rBhg9avX68ff/yRTyFwA8UdP6bTP2+RJB3a+bXSUi9RFKAoB6xrDT6/fHnDhg21ZMkSLV26VO+//36ubUFBQVq0aJHOnTsnp9MpPz8/tWnTRuPGjdP27ds1YsQIhYSE5Fzf09NT0dHRmj9/vg4ePKhjx47p4sWL6tWrl06fPq3U1FQFBARo4MCB2rJli9LT09W0adNcj7dp06ZavHixtmzZokmTJuUs7WOapiwWi6Kjo7V3715NnTpVp06d0oULF65r/78PHpcvCwoKUu3atTVlyhRt2LBBkydPVsmSJZWYmKijR49q7NixOnPmjFJTU1WqVCn997///fVo01D9+vV14MABffLJJ1qwYIFsNpvCwsLyPKC/MFgsFpUtW1bbtm1TZmamLly4oICAALlcrpyzH48dO6aqVauqTJkyqlGjRk44viwiIkIDBw7U4MGD1bFjx79lkD5QHDkdDk2dOllHT1/Q630e0uiZH6lMpWoUBvgrfi8La6mctLQ0+fj4yDRNZWRkyNvbO9flTqdTkZGR8vf3l5+fn9zc3JSdnS0fHx8ZhqGMjAx5eHgoKysr57KEhAQlJycrPDxcHh4eOfvKyMhQZGRkzlI6mZmZCgsLU1pamiIjI1W2bFkFBAQoOztbZ8+elYeHh0JDQ+XmlrtHNCMjQ1FRUQoJCVFAQIAkKTs7W15eXrp48aISEhJUrlw5nTt3ThUqVFB6evqf7v/yY8zOzpbVas11mcvlUnx8vDIzM1W6dGnZ7XZdunRJZ8+eVfny5ZWQkKDg4GBZrVY5HA75+voqPT09Z2Hrc+fOyWaz5XQ9pqeny9PTM+f6pmnKbrf/5W+aS5cuadq0aYqNjVVWVpZ69uypChUqaNq0aXrttdd08OBBzZw5U0FBQUpJSdEjjzyiVq1aXXVfLJUD3BiGy6Vtn36iXkMH6v+GDlPv51+Uzd39unofimW9WCoHRSVgoXjJyMhQWlqa3Nzc5OfnJ9M0lZaWJn9/f5mmqeTkZDmdTrm5ucnf3/+aX+IELODGOHP8qB7t1kURpUM0df4ilQwrQ1EIWPgLsVQO8sXLy0teXl65LrvcCmixWK5YNxLA33hAlJysj2ZM1/mzMfrPW9NUMrQ0RQEIWACAgnAe26Vmnl+p9IhnVLd5SyYWBQhYAICCiDt1Spd2zFDNyvXVuNtQud2AcZoAWOwZAG5ZpmFo8cL39fKH38mo1VP2X7vxARCwAAD5Clemflj3lV4ZPUb1mnRSUMO7KQpAwAIAFET82dN66+2Jat28mXoMfU5Wm42iADcQY7AA4Bbjcji0YdEH+mLdJq1askhlbqtIUYAbjBYsALjFpO3fqduNtZo/6ln9o0MXWZhMFLjhaMECgFtI+qVLOrd7ocoEBanyAyNk9/SkKMDfgMMaALhVmKYWT5mgfm+t1PnQHrKXDKEmAAELAFAQh/bs0oS5c9Tin+1U8e7OsjChKEDAAgDkX0LMeb0z5nWFB4epZ98h8vTxoSjA34gxWABwC/hu7TLN/GKN1ixaqMr161MQ4G9GCxYA3ORSD/2gmqnLtGpUH7Voez9dgwABCwBQEFlpadqzepY83Qy1fuJZ+QQGUhSAgAUAyC/DMLTy/TnqOfYD7XN7QF631aAoAAELAFAQMZEnNW3OTN13Xxc17fYwBQEIWACAgkhLTtbU117VpZQU9X9yoHwDS1IUgIAFAMgv0zD084ZlWrV+pV547gXVatKMogBFDNM0AMBNxhF3VhXjluuDge1U4+EecnO3UxSgiKEFCwBuIs7sbG18d5TSMmNV9V+vyLckXYMAAQsAkG+mYWjHhq80YPoifWvcJ//qtSXmvAIIWACA/Is9HalxI5/RXQ3v0j+7PCqLla9woKhiDBYA3ARcTqc+nDdbW/Yd1zcz5yqkQnmKAhRhHP4AQBFnmqYit6/V1mUzNGnUy6pZtwFFAYo4WrAAoIjLjo9VyeMfacyjLRT+xAB5+PhQFICABQDIL5fDoaXvTlV55yHV7DhdgaXDKQpwE6CLEACKKNM0dfr4Ub0+6z1tzmihgLpMKAoQsAAABZJ0IU7De/VQ5QoV1bPfEFnd6XQAbhZ8WgGgCMrOyNDSBbP13U979cGHn6h05SoUBbiJ0IIFAEXQpRP79MPKd/Rk3z5q3radrMx5BdxUaMECgCLGcSlZtp1T9UaPWnLr/Ia8/PwpCnCT4ZAIAIoQ0zS1fPEH2vzTHtnveEr+pcMoCkDAAgAUJFyd/PFHjR07WjtT68mjzj2ysNYgQMACAORfesolvfXWeHl4emnQyFdk9/WlKMBNijFYAFAEuBwOrf34Y33+2WeaMe1tRVSpRlGAmxgtWABQBKSeOqSYnbP0SNcOavvvR2Rz4/gXuJnxCQaAv1lWSoqyvl+gh5oGSa3GyycgkKIANzlasADgb7bzq9X6fO3nskb0UHC1GgxsBwhYAICCOH/shIYNH6Kf0mvIp/lDhCuAgAUAKIik+FhNGf+yXA5T/YY+L08/P4oCELAAAPmVnXJJc8a9rknzFsvfzV0BPt4UBSBgAQDyy8zKUuLK6cqOXiFJSkuIlcOZTWGAWwhnEQLADXTx/Dktfn2A2lWJ1iNtH1OZ+p6qVKWGqjZsSnEAAhYAIC8Ml0uHvv9eb77xgo78fES3NxqhfzzypHp5eFEcgIAF/MI0TblcLlksFtlsttw/JIYhwzBy/m21WmW10huN4vx5MfTt2tV69dWXle50aOLY/6hZt+6yurtTHICABfwvQK1Zs0ZffPGFwsLCNGTIEIWFheVsnzNnjr7//ntJUlZWlpo0aaLBgwdTOBTXD4zOrf9cLzz7tLz9fbX4oxUqW7WyrL87MAFwa6FZAXm2b98+ffDBB3r++edVq1YtzZ49O1eLVY8ePTR+/Hi9+uqrkqSmTRlbguIpMzlZZ9e+L7/TY/Tm4//UrI+/ULnqVQlXQDFACxbyxDRNHT9+XG3btlXVqlXl5eWltWvXyjCMnG5AX19f+fr66uTJk7Lb7apWrdoV+7j899tgBtxKLpyO1Nzp03Ro20KNGdhfjfoOlEdoOQoDELCAqweslJQUhYaG/vIGcnOTYRgyTfOK665Zs0aNGzeWl1fuQbzR0dFasmSJnE6nDh06pD59+lBY3FKfkdioSI17/ll9tm6FXntxjEp27i+PQNYXBAhYwDVYrVYFBATo7NmzkiSHwyE3N7crlvdIT0/XmjVr9NZbb10xwL1MmTLq37+/TNPUunXrKCpupXSlC9GR6nd3dUU5Smjm9HfV4ZHesrnRJQgUu99LSoC8qlmzptatW6ddu3Zp2bJlqlOnjhISErRw4UJlZWVJkvbs2aPatWurcuXKV6Z6Nzf5+/vL399fPj4+V239Am42htOp1L3bFLeonwz/2zR9+gx1eKQX4QoopmjBQp5VqVJFw4cP1/Lly1WuXDl169ZNkuR0OnOmb0hLS1P37t3l5nbttxiL2uJW4cjK0pr/zleF+EUqX7a85s4Zq9C69RnMDhCwgOtns9nUvHlzNW/ePNflvXr1yvn/du3aUSgUC5lpqVr41jiNfXu6enX5p0b0HK8SpctQGKCYo4sQAPIpKe6CZowbrcGjxmpwv94aNmGePAlXAEQLFgDkS/rpE5ox/kXNWbJGM0a/rIeHDpdvCc4UBEDAAoA8M1wuxfz0vTz3z1TbMudUe8Jk3f9ET7mx7A2A36CLEACuk8vp1O5NX6n3Ew8q7ny0anSZpE69+xCuABCwACA/nFnZ+uK/c9W3b0/5evrIq/NUedduLAsLmQMgYAFA3qVevKh3x4/RiBf/Ty0b3qW3l6xQuRp3MNUIgGtiDBYA/IHsC2eUuGq6vlr6tvo+9qSeHPa8SoaHUxgABCwAyI+zJ47p7OLXVKPUHs2Z8K5KtnpI9t+trQkABCwAuA6mYejInj16blh/ZSUn6sO3pyi0eQdZ3BjMDuD6MAYLAH7DkZWpjYs/Vq/ej8vhsmrUm1MU2qoL4QoAAQsA8sPISteORbP19IhnVb5sKU2fOUd3dewsMZgdQB7RRQgAkoxsp9JWv6OIxA/0ZIeWevT1dxRYqhSFAZAvtGABKN5MU8d+2K3ujatoxuIpsgZ2VP//zCFcASBgAUB+pSQl6e1pU/T5T5Ga+XmMzpasJw/fAAoDoEDoIgRQbDmysvTZvBla+OHH+kdZD91er7XKV61NYQAQsAAgX0xTmxfO1biJb+mZR7tr+LRZsts95OHtQ20AELAAIO/ZylTKz3tVOX2ZRj1+jzqMnCm/EiUpDIBCwxgsAMVO1OGD2rLgWfn6hKjr0PEKCAqmKAAKFS1YAIqVC9GRGjfqBZ04eEK1ur6qkLIVKQoAAhYA5Fd6UpKmj3pJSz9dpcWffKQKTZoziSiAvwRdhACKBZfDqQ+mT9WCZZ9p2ozJavWvB2W18hUIgIAFAPliulxK/W6VYn/8QIP79Ne/nnhSNhsN+AD+OnzDALi1w5Vp6OL+HXL7eaL6deiggPtfkoePN4UB8JeiBQvALe3Iru/Uv3cvbTqQrRKth8o3NIyiACBgAUB+nT91Sm+Ofk2xDun2Pu/Ju3xligLghqCLEMAtKTM1VTP+86Z+OPiTpr01XZVuv1MWzhgEcIPQggXglmNkZ2nrx7M0d/58jRj8jO7u+i9ZOGMQwA1ECxaAWytcOZ2KW/+Z7jQ+1JI3RqhBv4Gy2mwUBsANxSEdgFuGaRjatmqlPv/0LZkezdSs73Py9g+gMAAIWACQX6cOHdC4SW9ozVGL7PeMlFsAawwC+HvQRQjglpAUF6uRA59QYkyi5n+4SCXKRVAUAAQsAMiv1LgLmjS4n34+eUZTJ09RjcaNKAqAvxVdhABuas70NKXvXKLK/vv15uhRatX1IVmsDGoH8PeiBQvATcs0DB3esFRBpxaoS4eh8mvbV27u7hQGwN+OFiwAN61vv1ypxwcP0YpjVeR775Ny8/KkKACKBFqwANx0TNPU0e++04hnhql61Vrq8PQouXuzgDOAooMWLAA3nehjx/TyyyNlcXfppXGTFV6lOkUBUKTQggXgpuJKTdKRz6fq+Ml9mvHux6pRrzFFAUDAAoD8cjocurh0nBqG/KDF46eoyt33yGqjIR5A0cM3E4Cbgsvh0NIZ0zXlk4+V4tVOVbr8W1Y3jhEBELAAIH/hyunS1lUr9X9vvCxF/FMhHZ+R1e5BYQAQsAAgv4789L1ee/0VtWjaSkNHvCZPPxZwBlC00b4OoEhLiTqpcf83VM4LZ/Xy7AUqXbEiRQFAwAKA/MpOSlTGlql6or5TgS8sUpUGDWShLAAIWACQP+kpKVo6fZKaee5U0w7Pybt5e1lIVwBuEozBQr6kpqbq3Llzio+Pl9PpzLXNNE2lp6crJiZG58+fV0ZGBgVDnrgcDn29/DO9OGWathtd5NHoAcIVgJsKLVjIs7i4OL355psKDw/X0aNH1bVrV913332y/PoLmJCQoLFjx6p06dIKCAhQkyZNdPvtt1M4XLeNSxfp6YG99UinB9WlV1+5ebLGIICbCy1YyLO9e/fK09NTTz31lIYMGaKVK1fKMIyc7V9++aV8fX1Vq1Yt3XnnnapSpQpFw3XbumyJXh31mv7Z8p96fuJbCigVSlEAELBwazNNU+fPn1fjxo3l6empkJAQpaWl5QpY+/fv148//qjo6Gi999572rRpU659ZGZmKioqSpGRkbpw4UJOyxeK+3vL0I/rVmjY4P76+WSkHv73YwoOL0thANyU6CJEvoPWH2177LHH1L17d9WuXVuzZ89W+/btc7afPXtWn332mVwulw4fPqxKlSpRUN5POnvokHYuf18/xl6ShyG5DBeFAUDAQvFgsVgUHh6uVatW6b777lNcXJz8/PxkGIZSUlLk6+urKlWqKC0tTdnZ2UpLS5O3t3eufVSsWFHPPfecJGnNmjV/GNZQPMLVoZ3f6vnnh6m0V5pGP95dgaXKqHmHBygOAAIWio+6devq66+/1muvvaaEhAQ99NBDiouL0/Tp0zVq1Ci1bdtWb775po4dO6akpCQ98cQTV4Q0m80mSbJa6aUu5ulK+7du1cgXRyglPUVvjJ2p2xs3kdXNJpubO/UBcPM2SJg0HyAf0tPTlZaWJjc3N/n7+8s0TaWmpsrf318Wi0VpaWnKyMiQu7u7AgICrjnOavXq1fL19VXLli0panHLVoahE7u/18DBA+QwsvXOO3NVq3FjWQjdKAIMw9CsWbPUsWNHlStXjoIgz2jBQr54e3tf0fVXokSJnP/39fWVr68vhcI1Je3/QWkbX1XZAGn4mPdVq1FjTngAQMACgPzat/FrWfdMVsUgL02aPk8lq9UhXAG4pdAWD+CGMQ1D365eqaHDntZH316Qxz1jFVS9LmPxABCwACA/DJdLP23dpGHPPK2AEl7qO3qB7LfVpOUKAAELAPLH1A9frVXfnk+oVGhJvT3nQ1WqxfJJAAhYAJBvSTvWyrVrvBrXLKdpc+Yromo1iYYrALcwBrkD+MuYpqkdK5fKvm+qqoUHa8x7UxRYtoJEtyCAWxwtWAD+mnBlGPpmzXL1e6i7Pv1B8mr/lgLL3Ua4AkDAAoD8hqutKz9T30ceUJO7W2vo+Dmyh1ekMAAIWACQL4ahzZ/M07BhT6tJ42Z6feoMla1aQxYGXQEoRhiDBaDQmC6X0r5brhLH3lW7Rnfo2SnvK7hMWQoDgIAFAPnhcrm0ffXnCjn4msqE1NeoGRPlGRxGYQAUS3QRAigw0zC0etECde/8oBYfrCa/zm8QrgAQsAAgv1xOpz6b/66eHT5YD3frrMFv/kfeZcpTGADFGl2EAPIfrrKztX3Zhxr5f8PUoXVHvfLWVAWXi6AwAAhYlABAfsNV5q41qh47XYO6/1u9X52gwNJlKAwAELAA5Ed2Rqa+WDBTpaPmqUr1Dho0epi8SjHmCgAuYwwWgLyFq/QMfTjrbb0w+jVtu9REJTo8R7gCAAIWgPxKT07Rormz9MLL/6dHOj+kwa+NlWdQMIUBAAIWgPxwpGdo/Ydz1fPp4Xq6/1A9/8YYBZQqRWEA4CoYgwXgT2WlpChj2weqkb5Q818bqc5PPi1/whUAELAA5E9aUpLmvzVe1TPXqEHzJ/Rwy37yCPCnMADwB+giBHBNKfEXNW/SRP3nvXd10N5W3v/sT7gCgOtACxaAa4SreL07fpxeemuypo95XY8NHiq7nx+FAQACFoD8uBh5TC8PG6LZq77W608P0qMDn5JPQACFAQACFoC8Mg1DF89EasesF7R0/dcyDKlCuXD5lCBcAQABC0CeuZxO7f9up94cOVSdqqarXfUKSs62qn6zFrJYLBQIAAhYAPIartbOn6cXx72hMqGlVe+xmXpgUh2ZkvxKMpEoABCwAOSJMzNDsyeM0uSZc3XPP5tr5MtjFFGzhqxWG8UBAAIWgDwxTWWci5Jj+xxFfbdAvR76l4aMGqcAlr4BAAIWgPxkK1OHd27X3mXT1bHSQQ0bOk2B/+ggD1+mYQAAAhaAPHM5nPrx2+0a/Hh3lQr00z/+/aHK1GkkqxtfBwBQWJjJHShGUi5e1MfvTNPD3buocuXamjBrocrWa0a4AoBCxrcqUEw44s9rx7wJenHCLD3a7VE99ewIlalalcIAAAELQF6Zhqnk+Hg5lw5TXb8T+nTiWN358AB5eHtTHAD4i9BFCNzCDJdLO1et0MOd2+mLH0/IXutFNe45lHAFAAQsAPmRlZGupXNmaMCQgfLwdFe9hyeqRLNOsthouAYAAhaAPMtMStT8aRM17Lln9c+Gt+vdBYt1Z6vWsjCYHQBuCL5tgVtMxvkzSvtyqoz9S/TMwKc09I2xsnt6UhgAIGAByCvD5dLB77Zr9YwXNai5qUcGvCWPO+4lXAHA34AuQuAW4MjM0tolizR4UH9tPHBRyXVeU0DTrvIKCKA4AEDAApBXaUmJ+mDCGA17dpgiQsto1sJPVLbpvYy3AoC/Ed/AwE3MkRCnA/8dp5fGTNGIEc+r58BhCgovQ2EAgIAFID9O/PijPL6fqkqeJ/XxtIlq2qO/vP3pEgSAooAuQuAm48zO1tbPP1Hvxx7RJzuOyu2OkWo94FnCFQAQsHArME3zD7dd/kPhyUhN1cqP5qlvv94KKx2k+5+aphJN75PFwkcZAIoSugiRZ4ZhaMOGDVq9erVKly6t3r17KyQkJGf7pk2btGTJErm7u8vDw0OjR4+WN0uzFFhmUpLeHvOaZs5/Xw+1aKUR78xScHg5CgMARRCHvcizgwcPatasWerZs6eCg4P1/vvvyzCMnO0XL15U7dq1NXLkSA0fPlyezMNUMKapjKijSlk2QtUz1+nFZ57Ty3MWEK4AoAijBQt5/K03dfToUd17772qW7euAgMD9corr8gwDFmtv+R1d3d3rV69WsePH1eTJk3UuXPnXCHrt12Hee1CdDoc+mbFMu3avl5172qjwLBwSZLNKt0W6i27m1XHz6cq2/m/21gsUuUwb7nZrDoVm6ZMx2/u0yJVLe0jN6tFkXHpSs8yct2uerivrBYpMi5de3d+q6gTP6txm07yK1FSVcr4yM1m1ekL6UrNdOW6XaVQb3nabTodl66UDFeu51Ax1Fuedqui4zOu2HZbqLe83K06m5ihS2m/bPtx2wbt2rZGz7fOUuuHXpNn/U5y9/LizQgABCzcSgErOTlZ4eHhMk1T7u7uMgwjV1Bq1qyZ6tSpI8MwNHHiRIWEhKh169Y528+cOaOvv/5aLpdL+/btU/fu3a/7/uOiT2n0gIe0KV7S1Dm5th34T4BKB/qoZZ9zSvrd7fZNDFBoCW89MeK8fvjdxqNTg1TS10NPjz+n9cdzb0t4P1w2m0UvvHVGy/f/euG0X+73/Lth8vdx18gJ0Vp+MPftDk0MUvnSPho5Pkpf/G7brtG+qlw2QKPeOatPfsy9bdsrHqpRPkhj557Tgm9zb6tVa6Cevqs7460AgICFW43VapW/v7/OnTsnSXI4HLLZbLJYLDnXCQkJyRmT1ahRI+3evTtXwLocuH7b6nW9PL18VOuu+xS9bo0e791f3foMkNVqk9UihQZ6yc1m1e69aXL+ryFKFotUOtBbVqtFnzVNV7bzt61mpkoH+chqsWhOkwxlOXK3YNmDfGSxSKPKHJXX2DH6fsc3Gvrs82rb/RH5BPvKYrXq7frpmpBt5HqcpQO9JDebptVL1/jfbQst4Sm7u00Tm2Ro9G+2mZLCSnjK7mbTG80yNTLLpfPHj+g/E8bq6IH/b+/+Y6OuDz+Ov+6ud73eXX+wtlJoS3+3cOWHEEE2oRBmigENOjIGka5ZNhzUTKMYEFyGbjNZY/2FmWIEBBJ0GRpRkSndDwtbRgbIjwiF2h90cpS2lP6i195d7/PZH4b7Up1+kS2Oa5+P/9q76937fvV5n8/n3u9jyi2Y+NmZLDwPAYDAwrBTVFSkX/ziF5o+fbr+8Y9/qLi4WF1dXdq/f78WLlyoEydOKCEhQYFAQAcOHFBZWdnQSHI6lZ2dLdM0derUqa+1m3DUmHStr3pW5RUrVXDzDCWmjv5CcOSlf/nlczK+/LTsrzjt5rG5qhybrtZzZ1U8s0SuhKTI9Y77iuu73tOufCWgcMokZeZmq7ujXZNmz5PFytYrACCwMCxNmDBBq1at0tatW5WZmany8nKFw2H5fD4ZhqHW1lbt2LFDdrtdixcv1qxZs/7t37FYLEO2fF2rMfmFGpNf+I2O2WKxaFzxZI0rnvyNX2/etOk86QAgylhMJirC/9B7770nj8ejOXPmcGcAuGEYhqFNmzbprrvuUmYm39jF18f+BgAAAAILAACAwAIAACCwAAAAQGABAAAQWAAAAAQWAAAACCxEh+uZbBQAgBsZM7njf8owDPl8PjU2Nn6tJXMsFotGyhy5VwKU8TLe4TLeaBiraZpqa2uTYRi8UYPAQvTJzMzU7t271djYeM2XGRwc1L59+zR//nzZbLZhfx+1tLSora1NkydPHhFb+2pra2W325Wfnz8il/HMkgAADt1JREFUPmD8/e9/l9fr1ahRo4b9eAcHB7Vnzx7dddddUfHaNQxDcXFxvFGDwEL0mTJlytcOh0AgoNbWVj366KOy2+3D/j46duyYTp48qWXLlsk6AhZ73r17t1wul0pLS4f9WMPhsJ5//nndc889ysnJGfbjDQQCqq+vj5rXrmmaI+I1BwILw9D1LPh85fzXu1h0NN5HkmS1WkfMeEfSYzvSxhtNr12ODwWBhZH1pI2J0Q9+8IMRsXtQkjIyMkbUp+ji4mI5HI4RMVar1apvf/vbSklJGTGv3RUrVoyY1y5G+AYEc6QcWQkAAPBNfYDiLgAAAPjvYhchbmimaWpgYEDhcFgxMTGKjY0d8rX2QCCgcDgsSYqNjVVMTHQ/pQ3D0MDAgAzDkMPhGLKr7Or7wmKxDIvxhsNhDQwMfOXjNzg4qEAgIKfTGfW7lkKhkILBoCwWi5xO55Bdv6FQSIFAIPJzXFxc1I83EAgoFArJZrPJ6XR+4ZimYDCoUCgkScPi8QUILESNuro6VVZWKiUlRf39/Vq3bp3Gjh0rSfL7/XrmmWcUCATU29ur3NxcVVRURPU3C//2t79px44dSkhIUGxsrNauXavExMTIeLdv3662tjZ1dXVp0qRJuvfee+V0OqM2Nt5++239+c9/liR5vV6tWrVqyD9ZwzD05ptv6sUXX9RLL70kr9cbtY9tb2+vNm3apPPnz6unp0ff+973tGDBgkh0fPjhh6qsrNScOXNktVp13333KTU1NWrH297erieffFJWq1Xd3d1as2aNCgsLI+O9ePGiNm7cqHA4LLvdrrKyMuXl5fGmBwIL+Cb84Q9/0Lx587R48WJt2bJFe/bs0X333Rf5hF9RUSG3262BgQH95Cc/0d13362srKyoHe+2bdtUVlamW265RevWrdORI0c0b968yHjLy8sVExOj9vZ2rV69WgsXLlRaWlpUjrWrq0vbt2/Xc889p/j4eD3wwANatGiRMjMzI+c5e/aszpw5o/z8/KifiLOurk6NjY2qqqpSe3u7fv3rX2vWrFmRgA6Hw8rJydGiRYuUmpqq5OTkqB2raZo6fPiw0tPTVVFRoY8++ki/+93v9Oijjyo2NlaStGvXLiUlJamkpERJSUlDHndgOOAYLNzQWzjq6+s1bdo0xcXFqaioSMeOHfu/J6/VquTkZDmdTvl8PsXExET1P6WBgQG1tbWpsLBQHo9HXq9XTU1NQ8ZrmqZ27typF154QYsXL47q8XZ2diovL09paWlKSkpSdna2enp6Iqd3d3fr6aef1tKlS6N6S84Vra2t8nq9crvduummm2SxWCK7RyUpOTlZeXl5+vDDD7Vu3TqdPXs2qgOrtbVVxcXFcrvdyszMVG1tbWR3oCS9++67qqmp0f79+/XYY4/pk08+4U0PBBbwTb1JXz3R35ctsdHS0qKNGzfqoYceksfjierxXhnn1UF1NafTqZkzZ2rmzJk6cODAkCCJxvFePR/S1cfnGIahDz74QBkZGert7VVbW5vq6uqGBEk0jverptuYOnWqHnroIa1cuVIlJSV69913o/r1axjGkPF+/rns9/t177336v7779f3v/991dTU8KYHAgv4JjgcDo0bNy7yybe5uVlFRUUKBAIaGBiQaZqqr6/XunXrtHTpUk2bNi2qxxsXF6eEhAT985//jMx4nZ6erv7+foVCIRmGoXA4rKKiIpWWlqqlpUUdHR1RO97ExEQ1NTWps7NTfX198vl8crvd6uvr0+DgoLKzs5WcnKxDhw7J5/PpxIkTCgaDUTvelJQU1dfXKxgMqre3V+FwWDabTZcvX5ZpmgqHw7JarbJarbLZbEO29kQbi8Wi1NRUNTQ0KBgMqr29XVlZWTIMQ36/X5I0Y8aMIeMFhhvb448//jh3A25UbrdbmzdvVktLiw4dOqQVK1bor3/9q06cOKGcnBytWbNGiYmJ8ng8am5uVmpqqlwuV9SONzY2Vr///e/V0NAgn8+nH/7wh9q8ebPsdrtcLpe2bt2q06dP6/3339fo0aP13e9+N2oPcnc4HOrr69O+fft08OBBjRo1SnfeeafWr1+vwsJCTZw4UdOmTdPUqVNVW1urpUuXRvVxOm63WwcPHtTx48e1Z88ezZ49W2lpadqwYYPmzp2r6upq/eUvf9GJEydUU1OjsrIyjR49OmoDy+12a+fOnWpqatLevXu1YsUKNTY26q233tKsWbOUkZGhHTt26Pz586qurtaSJUs0ZswY3vQwbDDRKG5ohmGot7dXoVBIDodD8fHx6uvrk2macrlc6uzsHLJrLT4+PnIQbTQKh8Pq7u6WYRhyuVyKi4tTd3e3YmNj5XA4IqdZLBZ5PJ6oHqukyNYc0zQjj92lS5fk8XiGTFHR1dUll8sV9TO8+/1++f1+WSwWJSQkSPrs24VJSUny+/1DpqzweDxRv1RLT0+PgsGgbDabEhMTNTAwoGAwqKSkJJmmqe7ubg0ODsputys+Pp51/0BgAQAA4MvxcQEAAIDAAgAAILAAAAAILAC4kVxZd9IwDEnSxx9/rFdeeSXy83/i5MmT2rZt2xfmaXr55ZdVW1v7/17eMAy98MILqqurG/L7zs5Ovfjii0PWFwRAYAHANcePYRhDAuVafndlIlnDMCKhdOV3n4+dgYEBPfHEE6qvr5dpmrp48aI++uijf3sdV//Na7kdFy9e1LFjx75wvjNnzqijo+MLt+fzlzcMQ4cOHVJnZ+eQ0wOBgA4fPvxfiUAA0Ye1CAFct9bWVm3ZskVtbW3Ky8tTRUWFLl26pG3btsnn86mgoEArV67UqVOn9PrrrysQCGjx4sW69dZb9cc//lGnT59We3u7br31VuXm5uq1115TMBjUggULNHfu3Mg0BZ988ol27dqls2fPasaMGZo0aZI6Ozv19NNPq6urS8uWLdPkyZP1/PPPy26369y5cyorK9O5c+dUXV0tl8ul8vJyZWVl6Z133tH+/fvlcDj0ox/9SNJny/JUVVWpq6tLS5Ys0dSpU2WapmpqavTOO+8oKytL5eXlcjqd2rdvn6qrq5WYmKjly5crOzs7cn/09/frjTfe0JEjR5Sbm6twOMyTBBih2IIF4LqEw2HV1NQoPT1d69evV2lpqUzT1C9/+UulpqZqzZo1uu222+T3+/WrX/1KpaWl+vGPf6ynnnpKPp9PLS0tOnr0qMrKyjRx4kRVVlZq+fLlWrVqlbZu3apPP/00cl15eXkqLS3VI488op/+9Key2Wxqbm7WggUL9J3vfEebN2+WJB09elTnz5/Xgw8+qP7+fu3du1f333+/7rjjDq1fv161tbXaunWrHn74YVVUVETWODx69KgWLlyoOXPmaPv27QqHw5HZ1VetWqULFy7o6NGjqq2t1ZYtW/Szn/1MU6ZM0bPPPhu5jaZp6uDBgzp8+LBWr14th8MRmbUcAIEFANf25mG1qrCwUNXV1Xr11Vcjk2g2NDTo9ttv19ixY3XzzTfL5/PJ4/GopKREEyZMUH5+vhoaGmSz2TR37lwVFRXpwoULOnnypH7zm9/oiSeeUH9/v86dOzfkuhwOh9xut+Li4iRJEyZMkNfrVWFhYWTJoJiYGJWUlCgtLU0NDQ06cuSINmzYoN/+9rcKhUJyuVyaMmWKqqqqdOTIkcjEljNmzJDX69X48ePV09MTWTdw9uzZysnJ0cyZM/Xpp5/q+PHjmjp1qnJzczV79my1tLRE1oM0TVOnT5/WkiVLlJmZqYULF0b9xKgArh+7CAFcF4vFouLiYlVWVur48ePasGGDqqqqZJqmLl++HFk70eFwKBAIyO/3y263q7+/P7Kckd1ul/TZsjnZ2dl68skn5XQ6ZRhGZKbzK9dls9kiu9wsFoucTqcsFsuQRbGtVmskwGJiYjR//nytWLFCVqtVg4ODSklJ0dq1a3XmzBnt27dPly5dUk5OTuT2fH5B8SuBdOXybrc7so7g5cuXZbFYhkSUy+VSR0eHwuGwenp6OP4KILAA4OsZHBxUTU2NOjs7Zbfb5Xa75fF4tHz5cr300kuaP3++WltbVVZWpuzsbG3cuFEZGRkKBoMaP368zpw5E/lbXq9X+fn52rVrlyZMmKDGxkbdfffdGjt2bCR00tPT9ac//UkWi+WawmXWrFnatWuX3nvvPd100006deqUFi1apAMHDigrK0vBYFDx8fHXvByNaZq67bbbtHv3br3xxhv6+OOPNWfOnMhakFarVdOnT9fPf/5zGYah6upqsVAGMII/hLJUDoDrYZqm2tra1NzcLNM0NWbMGI0bN06hUEinT59WX1+fEhMTI7vd6uvrNTg4qLy8PCUnJ8vn88kwDI0bN06SdPnyZdXV1SkYDCohIUEFBQWRLVzSZ9/2a2hoUFxcnMaMGaOOjg4VFRXJ7/erublZXq9Xp06d0ujRo5WcnCxJkdtnGIZSUlKUlpampqYm9fb2yu12q6CgQH6/Xx0dHSooKFB/f7+am5s1fvx41dbWKi0tTd/61rd0/vx5GYah9PR0XbhwQc3NzYqNjVVhYaFcLpdOnjyp9PR0JSYmqqmpSW1tbUpOTlYoFNL48eNls9l4wgAEFgAAAP4THOQOAABAYAEAABBYAAAABBYAAAAILAAAAAILAACAwAIAAACBBQAAQGABAAAQWAAAACCwAAAACCwAAAACCwAAgMACAAAAgQUAAEBgAQAAEFgAAAAgsAAAAAgsAAAAAgsAAAAEFgAAAIEFAABAYAEAAIDAAgAAILAAAAAILAAAAAILAAAABBYAAACBBQAAQGABAACAwAIAACCwAAAACCwAAAAQWAAAAAQWAAAAgQUAAAACCwAAgMACAAAgsAAAAAgsAAAAEFgAAAAEFgAAAIEFAAAAAgsAAIDAAgAAILAAAABAYAEAABBYAAAABBYAAAAILAAAAAILAACAwAIAACCwAAAAQGABAAAQWAAAAAQWAAAACCwAAAACCwAAgMACAAAAgQUAAEBgAQAAEFgAAAAEFgAAAAgsAAAAAgsAAIDAAgAAAIEFAABAYAEAABBYAAAAILAAAAAILAAAAAILAAAAV/sXlgBsbQbH9iMAAAAASUVORK5CYII=)\n", "---" ] }, { "cell_type": "markdown", "source": [ "\n", "# *In Conclusion* next steps could be:\n", "* add more training data\n", "* try other pre-trained models followed by training for classification of android statues.\n", "* try other variants of transfer-learning with object detection models.\n", "* for the test frames from the video, could consider applying HMM inference to the best scoring identifications for “smoothing” inference where missing data. can use “filtering” queries for the next observations… can use particle filtering for similar reasons. **can build a better training set by adding the inferred missing identifications (bounding boxes and new ground truth labels) to the existing training dataset and then get a new test dataset.** consider whether using optical flow for a small number of frames after good identifications would be a fast addition to the boot-strapping of bounding boxes between frames. consider multi-object tracking models." ], "metadata": { "id": "v7LxXMtKWfCb" } } ], "metadata": { "accelerator": "TPU", "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }