{ "cells": [ { "cell_type": "markdown", "id": "7be10adc-8677-4a69-87f6-489b10d7a989", "metadata": { "tags": [] }, "source": [ "# Metadata collection with ZnTrack\n", "\n", "ZnTrack allows for the collection of some metadata.\n", "One example is measuring the execution time of Nodes or even methods inside the Nodes easily.\n", "This can be achieved by using the `@TimeIt` decorator which is shown in the following example." ] }, { "cell_type": "code", "execution_count": 1, "id": "f0676d7e-7803-488c-9891-d500d55aa47f", "metadata": {}, "outputs": [], "source": [ "import zntrack\n", "from time import sleep\n", "\n", "zntrack.config.nb_name = \"05_metadata.ipynb\"" ] }, { "cell_type": "code", "execution_count": 2, "id": "355f3eb2-b42b-43d6-aa18-29dfa579b742", "metadata": { "nbsphinx": "hidden", "tags": [] }, "outputs": [], "source": [ "from zntrack.utils import cwd_temp_dir\n", "\n", "temp_dir = cwd_temp_dir()" ] }, { "cell_type": "code", "execution_count": 3, "id": "c2ea08fe-7113-421e-b6b3-71b4e048c724", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initialized empty Git repository in /tmp/tmp2n5usz7x/.git/\n", "Initialized DVC repository.\n", "\n", "You can now commit the changes to git.\n", "\n", "+---------------------------------------------------------------------+\n", "| |\n", "| DVC has enabled anonymous aggregate usage analytics. |\n", "| Read the analytics documentation (and how to opt-out) here: |\n", "| |\n", "| |\n", "+---------------------------------------------------------------------+\n", "\n", "What's next?\n", "------------\n", "- Check out the documentation: \n", "- Get help and share ideas: \n", "- Star us on GitHub: \n" ] } ], "source": [ "!git init\n", "!dvc init" ] }, { "cell_type": "code", "execution_count": 4, "id": "bf980834-b0d5-4be4-ac26-d653736645f6", "metadata": {}, "outputs": [], "source": [ "class SleepNode(zntrack.Node):\n", " metadata = zntrack.zn.metrics()\n", "\n", " @zntrack.tools.timeit(\"metadata\")\n", " def run(self):\n", " self.sleep_1s()\n", " self.sleep_2s()\n", "\n", " @zntrack.tools.timeit(\"metadata\")\n", " def sleep_1s(self):\n", " sleep(1)\n", "\n", " def sleep_2s(self):\n", " sleep(2)" ] }, { "cell_type": "code", "execution_count": 5, "id": "e4e13211-1462-44d4-939a-9a96a057969e", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "DeprecationWarning for write_graph: Building a graph is now done using 'with zntrack.Project() as project: ...' (Deprecated since 0.6.0)\n", "Running DVC command: 'stage add --name SleepNode --force ...'\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Creating 'dvc.yaml'\n", "Adding stage 'SleepNode' in 'dvc.yaml'\n", "\n", "To track the changes with git, run:\n", "\n", "\tgit add dvc.yaml nodes/SleepNode/.gitignore\n", "\n", "To enable auto staging, run:\n", "\n", "\tdvc config core.autostage true\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Jupyter support is an experimental feature! Please save your notebook before running this command!\n", "Submit issues to https://github.com/zincware/ZnTrack.\n", "[NbConvertApp] Converting notebook 05_metadata.ipynb to script\n", "/data/fzills/miniconda3/envs/zntrack/lib/python3.10/site-packages/nbformat/__init__.py:93: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.\n", " validate(nb)\n", "[NbConvertApp] Writing 1732 bytes to 05_metadata.py\n", "Running DVC command: 'repro SleepNode'\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Running stage 'SleepNode':\n", "> zntrack run src.SleepNode.SleepNode --name SleepNode\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Could not load field metadata for node SleepNode.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Generating lock file 'dvc.lock'\n", "Updating lock file 'dvc.lock'\n", "\n", "To track the changes with git, run:\n", "\n", "\tgit add dvc.lock\n", "\n", "To enable auto staging, run:\n", "\n", "\tdvc config core.autostage true\n", "Use `dvc push` to send your updates to remote storage.\n" ] } ], "source": [ "with zntrack.Project() as project:\n", " node = SleepNode()\n", "\n", "project.run()" ] }, { "cell_type": "code", "execution_count": 6, "id": "d87965c0-b9c9-40cc-9b63-dc9047c42915", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Path run sleep_1s \n", "nodes/SleepNode/metadata.json 3.1168 1.00106\n" ] } ], "source": [ "!dvc metrics show" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8d5fdb4c-ae21-47d9-8c2f-e1c89d48a1da", "metadata": {}, "source": [ "We can also time a single function multiple times, using the following example:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "class SleepNodeMulti(zntrack.Node):\n", " metadata = zntrack.zn.metrics()\n", "\n", " @zntrack.tools.timeit(\"metadata\")\n", " def run(self):\n", " self.sleep(1)\n", " self.sleep(2)\n", "\n", " @zntrack.tools.timeit(\"metadata\")\n", " def sleep(self, time):\n", " sleep(time)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "DeprecationWarning for write_graph: Building a graph is now done using 'with zntrack.Project() as project: ...' (Deprecated since 0.6.0)\n", "Running DVC command: 'stage add --name SleepNodeMulti --force ...'\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Adding stage 'SleepNodeMulti' in 'dvc.yaml'\n", "\n", "To track the changes with git, run:\n", "\n", "\tgit add nodes/SleepNodeMulti/.gitignore dvc.yaml\n", "\n", "To enable auto staging, run:\n", "\n", "\tdvc config core.autostage true\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[NbConvertApp] Converting notebook 05_metadata.ipynb to script\n", "/data/fzills/miniconda3/envs/zntrack/lib/python3.10/site-packages/nbformat/__init__.py:93: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.\n", " validate(nb)\n", "[NbConvertApp] Writing 1732 bytes to 05_metadata.py\n", "Running DVC command: 'repro SleepNodeMulti'\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Running stage 'SleepNodeMulti':\n", "> zntrack run src.SleepNodeMulti.SleepNodeMulti --name SleepNodeMulti\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Could not load field metadata for node SleepNodeMulti.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Updating lock file 'dvc.lock'\n", "\n", "To track the changes with git, run:\n", "\n", "\tgit add dvc.lock\n", "\n", "To enable auto staging, run:\n", "\n", "\tdvc config core.autostage true\n", "Use `dvc push` to send your updates to remote storage.\n" ] } ], "source": [ "with zntrack.Project() as project:\n", " node = SleepNodeMulti()\n", "\n", "project.run()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Path run sleep.mean sleep.std sleep_1s\n", "nodes/SleepNode/metadata.json 3.1168 - - 1.00106\n", "nodes/SleepNodeMulti/metadata.json 3.1003 1.50156 0.50054 -\n" ] } ], "source": [ "!dvc metrics show" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "One can also access the metrics directly within Python. This is possible, because they are just another `zn.metrics` which is automatically added when using one of the given metadata decorators." ] }, { "cell_type": "code", "execution_count": 10, "id": "6b6d4e62-f2db-4550-b89c-9012c3387cbf", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'sleep': {'values': [1.0010173320770264, 2.0021069049835205],\n", " 'mean': 1.5015621185302734,\n", " 'std': 0.5005447864532471},\n", " 'run': 3.1002955436706543}" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "SleepNodeMulti.from_rev().metadata" ] }, { "cell_type": "code", "execution_count": 11, "id": "35ed65a1-9590-43f8-8aa9-244fd187f823", "metadata": { "nbsphinx": "hidden", "tags": [] }, "outputs": [], "source": [ "temp_dir.cleanup()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.9" } }, "nbformat": 4, "nbformat_minor": 5 }