{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "f7374c5c", "metadata": { "scrolled": true }, "outputs": [], "source": [ "import dynamo as dyn\n", "import time\n", "\n", "from dynamo import LoggerManager, main_tqdm, main_critical,main_warning, main_info, Logger\n", "from dynamo.dynamo_logger import main_tqdm, main_critical,main_warning, main_info, Logger\n", "import dynamo.tools" ] }, { "cell_type": "markdown", "id": "98daaa2b", "metadata": {}, "source": [ "## Introduction to dynamo logger utilities for developers\n", "\n" ] }, { "cell_type": "markdown", "id": "9d185c43", "metadata": {}, "source": [ "### \"main_\\*\" utility functions\n", "Function names started with \"main_\" in dynamo_logger.py are utility functions for directly writing information in the main logger. We design these functions so that developers can avoid tedious Logger class creation. Under most circumstances , developers only need to use main_\\* series functions. We build our logger upon Python's logging library and our main logger occupies \"dynamo\" namespace in Python's underlying logging library. Devlopers can create logger with different namespaces with Logger and LoggerManager class we will introduce later. " ] }, { "cell_type": "markdown", "id": "deb9ce59", "metadata": {}, "source": [ "#### main_tqdm\n", "A handy utility function to allow developers to time loops in TQDM library style." ] }, { "cell_type": "code", "execution_count": 2, "id": "d1da9721", "metadata": { "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----> [using TQDM style logging] in progress: 100.0000%\n", "|-----> [using TQDM style logging] finished [1.0725s]\n", "|-----------> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [using LoggerManager's progress_logger] finished [1.0570s]\n" ] } ], "source": [ "for i in main_tqdm(range(1, 11), desc=\"using TQDM style logging\"):\n", " time.sleep(0.1)\n", "for i in LoggerManager.progress_logger(range(1, 11), progress_name=\"using LoggerManager's progress_logger\", indent_level=2):\n", " time.sleep(0.1)" ] }, { "cell_type": "markdown", "id": "f872cb64", "metadata": {}, "source": [ "#### main_info, main_warning, main_critical" ] }, { "cell_type": "code", "execution_count": 3, "id": "5ef5a631", "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----> someInfoMessage\n", "|> someInfoMessage\n", "|-----------? someWarningMessage\n", "|-----------------!! someCriticalMessage\n", "|-----------!! someERRORMessage\n" ] } ], "source": [ "main_info(\"someInfoMessage\")\n", "main_info(\"someInfoMessage\", indent_level=0)\n", "main_warning(\"someWarningMessage\", indent_level=2)\n", "main_critical(\"someCriticalMessage\", indent_level=3)\n", "main_critical(\"someERRORMessage\", indent_level=2)" ] }, { "cell_type": "markdown", "id": "6e3e9d82", "metadata": {}, "source": [ "### Log nested loops\n", "To log nested loops, developers should pass different loggers to main_tqdm's logger argument, because each logger has its own small timer variable and thus the single logger cannot handle this case." ] }, { "cell_type": "code", "execution_count": 4, "id": "4a23b4ec", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3206s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3169s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3154s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3216s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3150s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3158s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3160s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3141s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3175s]\n", "|-----------> [[Inner] using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----------> [[Inner] using LoggerManager's progress_logger] finished [0.3152s]\n", "|\n", "|-----> [[Outer] using LoggerManager's progress_logger] finished [3.2337s]\n" ] } ], "source": [ "for i in main_tqdm(range(1, 11), desc=\"[Outer] using LoggerManager's progress_logger\", logger=LoggerManager.get_main_logger()):\n", " for i in main_tqdm(range(1, 4), desc=\"[Inner] using LoggerManager's progress_logger\", logger=LoggerManager.gen_logger(\"someLoggerSpace\"), indent_level=2):\n", " time.sleep(0.1)" ] }, { "cell_type": "markdown", "id": "f994bca6", "metadata": {}, "source": [ "### LoggerManager class\n", "We adopt Factory design pattern and create LoggerManager to create and manage important loggers." ] }, { "cell_type": "markdown", "id": "b72fc454", "metadata": {}, "source": [ "#### LoggerManager's main_logger() getter\n", "This getter function returns the main logger dynamo logger. The statements below basically do the same thing as the code block above with \"main_\\*\" functions.\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "47452db4", "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----> someInfoMessage\n", "|> someInfoMessage\n", "|-----------? someWarningMessage\n", "|-----------------!! someCriticalMessage\n", "|-----------!! someERRORMessage\n" ] } ], "source": [ "main_logger = LoggerManager.get_main_logger()\n", "test_logger = main_logger\n", "test_logger.info(\"someInfoMessage\")\n", "test_logger.info(\"someInfoMessage\", indent_level=0)\n", "test_logger.warning(\"someWarningMessage\", indent_level=2)\n", "test_logger.critical(\"someCriticalMessage\", indent_level=3)\n", "test_logger.critical(\"someERRORMessage\", indent_level=2)" ] }, { "cell_type": "markdown", "id": "ff84c3f3", "metadata": {}, "source": [ "### Logger class" ] }, { "cell_type": "markdown", "id": "b1efa6f4", "metadata": {}, "source": [ "Developers can create loggers with different namespace and set their own logging levels. For example, some important functions may have their own logger namespace. Each logger instance has an internal timer to record some code blocks' lapsed time. **log_time()** is for logging the current time and save the lapsed time amount from the last **log_time()** call. The Logger constructor calls log_time." ] }, { "cell_type": "code", "execution_count": 6, "id": "c70552d2", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----> in progress: 90.0000%\n", "|-----> [pytest simple progress logger test] finished [1.0609s]\n" ] } ], "source": [ "test_logger = Logger(\"someLoggerSpace\")\n", "total = 10\n", "test_logger.log_time()\n", "for i in range(total):\n", " # test_logger.report_progress(i / total * 100)\n", " test_logger.report_progress(count=i, total=total)\n", " time.sleep(0.1)\n", "test_logger.finish_progress(progress_name=\"pytest simple progress logger test\")" ] }, { "cell_type": "markdown", "id": "e3d4c900", "metadata": {}, "source": [ "**Known issue** main_tqdm currently does not support nested loops as mentioned above. This can be avoided by using two different loggers for now." ] }, { "cell_type": "code", "execution_count": 7, "id": "e4dc50bb", "metadata": { "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3182s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3198s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3225s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3133s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3161s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3154s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3138s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3186s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3188s]\n", "|-----> [using LoggerManager's progress_logger] in progress: 100.0000%\n", "|-----> [using LoggerManager's progress_logger] finished [0.3209s]\n", "|\n", "|-----> [using LoggerManager's progress_logger] finished [0.0045s]\n" ] } ], "source": [ "for i in main_tqdm(range(1, 11), desc=\"using LoggerManager's progress_logger\"):\n", " for i in main_tqdm(range(1, 4), desc=\"using LoggerManager's progress_logger\"):\n", " time.sleep(0.1)" ] }, { "cell_type": "markdown", "id": "5703eeb8", "metadata": {}, "source": [ "As the above results show, the final result for the outer loop is incorrect." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" } }, "nbformat": 4, "nbformat_minor": 5 }