{ "cells": [ { "cell_type": "markdown", "id": "265833e1-2fd8-4699-b2e4-6ca0a311365e", "metadata": {}, "source": [ "# ZDO 2022 - Pig Leg Surgery\n", "\n", "Navrhněte a naprogramujte aplikaci pro detekci chirurgických nástrojů v čase\n", "Vstupem jsou cesty k videím z pohledu shora během šití prasečí nožičky\n", "Cílem je určit `[x, y]` pozice obarvených hrotů chirurgických nástrojů pro každý snímek\n", "Výstup (bude sloužit pro vyhodnocení na našich datech):\n", "json pro každou sekvenci obrázků bude ve formátu: \n", "\n", "```python\n", "annotation={\n", " \"filename\": [\"vid1.mp4\", \"vid1.mp4\", \"vid1.mp4\"...], # pth.parts[-1]\n", " \"frame_id\": [0,1,1,...],\n", " \"object_id\": [0,0,1],\n", " \"x_px\": [110, 110, 300], # x pozice obarvených hrotů v pixelech\n", " \"y_px\": [50, 50, 400], # y pozice obarvených hrotů v pixelech\n", " \"annotation_timestamp\": [],\n", "}\n", "```\n", "\n", "\n", "vykreslení výstupů do videa:\n", "\n", "centry hrotů vykreslit buď viditelným bodem nebo ohraničujícím boxem rozumné velikostí (např. 100x100 px)\n", "* každý objekt označen jinou barvou\n", "* Primární úloha: tracking needle holderu\n", "* Bonusová úloha: tracking ostatních nástrojů\n", " " ] }, { "cell_type": "markdown", "id": "a2245442-6d2d-4458-add0-e2812e7e8ddc", "metadata": {}, "source": [ "# Annotation tool CVAT\n", "\n", "\n", "http://147.228.124.52:8080/\n", "\n", "Popis práce s CVAT naleznete na [stránkách loňského zadání](https://nbviewer.org/github/mjirik/ZDO/blob/master/ZDOsem2021.ipynb)\n", "\n", "Z něj si stáhnete anotce ve formátu CVAT for videos" ] }, { "cell_type": "markdown", "id": "fe8a594e-5c0c-41fe-ac46-f3c63618ca77", "metadata": {}, "source": [ "# Práce s videem a anotacemi\n", "\n", "Základy práce s videem lze načerpat [zde](https://nbviewer.org/github/mjirik/ZDO/blob/master/ZDOsem2020.ipynb)\n", "\n", "Převod anotací z formátu CVAT for videos " ] }, { "cell_type": "code", "execution_count": 35, "id": "0cc64cba-95e7-4636-bd93-6b76a4db39de", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import lxml\n", "from lxml import etree\n", "\n", "pth = Path(\"sem/2022/annotations.xml\")\n", "# root = etree.fromstring(xml, base_url=pth)\n", "\n", "annotation={\n", " \"filename\": [], # pth.parts[-1]\n", " \"frame_id\": [],\n", " \"object_id\": [],\n", " \"x_px\": [], # x pozice obarvených hrotů v pixelech\n", " \"y_px\": [], # y pozice obarvených hrotů v pixelech\n", " \"annotation_timestamp\": [],\n", "}\n", "\n", "\n", "tree = etree.parse(pth)\n", "\n", "updated = tree.xpath(\"//updated\")[0].text # date of last change in CVAT\n", "\n", "for track in tree.xpath('track'):\n", " for point in track.xpath(\"points\"):\n", " pts = point.get(\"points\").split(\",\")\n", " x, y = pts\n", " annotation[\"filename\"].append(str(pth))\n", " annotation[\"object_id\"].append(track.get(\"id\"))\n", " annotation[\"x_px\"].append(x)\n", " annotation[\"y_px\"].append(y)\n", " annotation[\"frame_id\"].append(point.get(\"frame\"))\n", " annotation[\"annotation_timestamp\"].append(updated)\n", " \n", " # print(track.find('./@id'))\n", " \n" ] }, { "cell_type": "code", "execution_count": 37, "id": "089c1d58-d5a1-4987-8d12-487deabb98ba", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filenameframe_idobject_idx_pxy_pxannotation_timestamp
0sem\\2022\\annotations.xml4703814.911522.372022-03-11 07:39:11.353911+00:00
1sem\\2022\\annotations.xml4803757.171485.892022-03-11 07:39:11.353911+00:00
2sem\\2022\\annotations.xml4903699.431449.412022-03-11 07:39:11.353911+00:00
3sem\\2022\\annotations.xml5003641.681412.942022-03-11 07:39:11.353911+00:00
4sem\\2022\\annotations.xml5103583.941376.462022-03-11 07:39:11.353911+00:00
.....................
950sem\\2022\\annotations.xml49611389.90204.802022-03-11 07:39:11.353911+00:00
951sem\\2022\\annotations.xml49711389.90204.802022-03-11 07:39:11.353911+00:00
952sem\\2022\\annotations.xml49811389.90204.802022-03-11 07:39:11.353911+00:00
953sem\\2022\\annotations.xml49911389.90204.802022-03-11 07:39:11.353911+00:00
954sem\\2022\\annotations.xml50011389.90204.802022-03-11 07:39:11.353911+00:00
\n", "

955 rows × 6 columns

\n", "
" ], "text/plain": [ " filename frame_id object_id x_px y_px \\\n", "0 sem\\2022\\annotations.xml 47 0 3814.91 1522.37 \n", "1 sem\\2022\\annotations.xml 48 0 3757.17 1485.89 \n", "2 sem\\2022\\annotations.xml 49 0 3699.43 1449.41 \n", "3 sem\\2022\\annotations.xml 50 0 3641.68 1412.94 \n", "4 sem\\2022\\annotations.xml 51 0 3583.94 1376.46 \n", ".. ... ... ... ... ... \n", "950 sem\\2022\\annotations.xml 496 1 1389.90 204.80 \n", "951 sem\\2022\\annotations.xml 497 1 1389.90 204.80 \n", "952 sem\\2022\\annotations.xml 498 1 1389.90 204.80 \n", "953 sem\\2022\\annotations.xml 499 1 1389.90 204.80 \n", "954 sem\\2022\\annotations.xml 500 1 1389.90 204.80 \n", "\n", " annotation_timestamp \n", "0 2022-03-11 07:39:11.353911+00:00 \n", "1 2022-03-11 07:39:11.353911+00:00 \n", "2 2022-03-11 07:39:11.353911+00:00 \n", "3 2022-03-11 07:39:11.353911+00:00 \n", "4 2022-03-11 07:39:11.353911+00:00 \n", ".. ... \n", "950 2022-03-11 07:39:11.353911+00:00 \n", "951 2022-03-11 07:39:11.353911+00:00 \n", "952 2022-03-11 07:39:11.353911+00:00 \n", "953 2022-03-11 07:39:11.353911+00:00 \n", "954 2022-03-11 07:39:11.353911+00:00 \n", "\n", "[955 rows x 6 columns]" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "pd.DataFrame(annotation)" ] }, { "cell_type": "markdown", "id": "a5b08f30-8aef-400f-9d4f-2fecc670d66a", "metadata": {}, "source": [ "# Test data\n", "\n", "* UWB Google Apps account required.\n", "* [Downlowad here](https://drive.google.com/drive/folders/1ll1_w7UR_oCm9Z8Iu1uCq-_YXxd4_IG6?usp=sharing)" ] }, { "cell_type": "markdown", "id": "67267a58-021d-4375-99ca-fe39c8ab4fec", "metadata": {}, "source": [ "# Odevzdání\n", "\n", "* ve formě git repozitáře\n", "* dokumentace na gitu (jména autorů ideálně v readme.md)\n", "* spustitelný automatický test - viz [šablona, kterou můžete využít](https://github.com/mjirik/ZDO2022)" ] }, { "cell_type": "code", "execution_count": 8, "id": "bc9d443f-3cde-47cd-a14d-386caa26932f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[WindowsPath('../scaffanweb/deploy/etc/supervisor/conf.d'),\n", " WindowsPath('../skelet3d/src/compile.m'),\n", " WindowsPath('../skelet3d/src/skelet3d.m')]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from pathlib import Path\n", "types = ('*.MP4', '*.mkv') # the tuple of file types\n", "files_grabbed = []\n", "for files in types:\n", " files_grabbed.extend(glob.glob(files))\n", "list(Path(\"..\").glob(\"**/*.[md|png]\"))" ] }, { "cell_type": "code", "execution_count": 13, "id": "5f3eac58-00ea-4e95-9e1d-92197bee9469", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "WindowsPath('Metody přesné lokalizace hran.xml')" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lst = list(Path(\".\").glob(\"*.pdf\"))\n", "pt = lst[0] \n", "pt.with_suffix(\".xml\")" ] }, { "cell_type": "code", "execution_count": null, "id": "9ba36e8d-e40a-4cc8-929b-14a171f4b301", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.10" } }, "nbformat": 4, "nbformat_minor": 5 }