{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Functional Interface\n",
"\n",
"pytask offers a functional interface to users who want more flexibility than is given by a command line interface. It even allows you to run pytask from a Python interpreter or a Jupyter notebook like this article here.\n",
"\n",
"\n",
"Let's see how it works!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"from typing import Annotated\n",
"\n",
"import pytask\n",
"from pytask import task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is a small workflow where two tasks create two text files and the third task merges both of them into one file.\n",
"\n",
"One important bit to note here is that the second task is created from a lambda function. So, you can use dynamically defined functions to create tasks.\n",
"\n",
"It also shows how easy it is to wrap any third-party function where you have no control over the signature, but you can still easily wrap them with pytask."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def task_create_first_file() -> Annotated[str, Path(\"first.txt\")]:\n",
" return \"Hello, \"\n",
"\n",
"\n",
"task_create_second_file = task(\n",
" name=\"task_create_second_file\", produces=Path(\"second.txt\")\n",
")(lambda *x: \"World!\")\n",
"\n",
"\n",
"def task_merge_files(\n",
" first: Path = Path(\"first.txt\"), second: Path = Path(\"second.txt\")\n",
") -> Annotated[str, Path(\"hello_world.txt\")]:\n",
" return first.read_text() + second.read_text()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let us execute this little workflow."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"────────────────────────────────────────────── Start pytask session ───────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[39m────────────────────────────────────────────── \u001b[0mStart pytask session\u001b[39m ───────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Platform: darwin -- Python 3.12.12, pytask 0.6.0, pluggy 1.6.0\n",
"\n"
],
"text/plain": [
"Platform: darwin -- Python 3.12.12, pytask 0.6.0, pluggy 1.6.0\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Root: /Users/tobiasr/git/pytask\n",
"\n"
],
"text/plain": [
"Root: \u001b[35m/Users/tobiasr/git/\u001b[0m\u001b[95mpytask\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n"
],
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Collected 3 tasks.\n",
"\n"
],
"text/plain": [
"Collected 3 tasks.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
],
"text/plain": [
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c58c8439b3e24cbd9bd96afd98ed0b06",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Output()"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n"
],
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"╭─────────────────────────┬─────────╮\n",
"│ Task │ Outcome │\n",
"├─────────────────────────┼─────────┤\n",
"│ task_create_first_file │ . │\n",
"│ task_create_second_file │ . │\n",
"│ task_merge_files │ . │\n",
"╰─────────────────────────┴─────────╯\n",
"\n"
],
"text/plain": [
"╭─────────────────────────┬─────────╮\n",
"│\u001b[1m \u001b[0m\u001b[1mTask \u001b[0m\u001b[1m \u001b[0m│\u001b[1m \u001b[0m\u001b[1mOutcome\u001b[0m\u001b[1m \u001b[0m│\n",
"├─────────────────────────┼─────────┤\n",
"│ \u001b]8;id=73604;file://None\u001b\\task_create_first_file \u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"│ \u001b]8;id=609946;file://None\u001b\\task_create_second_file\u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"│ \u001b]8;id=378329;file://None\u001b\\task_merge_files \u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"╰─────────────────────────┴─────────╯\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
],
"text/plain": [
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[2m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"╭─────────── Summary ────────────╮\n",
"│ 3 Collected tasks │\n",
"│ 3 Succeeded (100.0%) │\n",
"╰────────────────────────────────╯\n",
"\n"
],
"text/plain": [
"\u001b[38;2;19;124;57m╭─\u001b[0m\u001b[38;2;19;124;57m──────────\u001b[0m\u001b[1;38;2;242;242;242m Summary \u001b[0m\u001b[38;2;19;124;57m───────────\u001b[0m\u001b[38;2;19;124;57m─╮\u001b[0m\n",
"\u001b[38;2;19;124;57m│\u001b[0m \u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m3\u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242mCollected tasks\u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m \u001b[38;2;19;124;57m│\u001b[0m\n",
"\u001b[38;2;19;124;57m│\u001b[0m \u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m3\u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57mSucceeded \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m(100.0%)\u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m \u001b[38;2;19;124;57m│\u001b[0m\n",
"\u001b[38;2;19;124;57m╰────────────────────────────────╯\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"──────────────────────────────────────────── Succeeded in 0.03 seconds ────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[38;2;19;124;57m──────────────────────────────────────────── \u001b[0m\u001b[38;2;19;124;57mSucceeded in 0.03 seconds\u001b[0m\u001b[38;2;19;124;57m ────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"session = pytask.build(\n",
" tasks=[task_create_first_file, task_merge_files, task_create_second_file]\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The information on the executed workflow can be found in the `session`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Session(config={'pm': , 'markers': {'filterwarnings': 'Add a filter for a warning to a task.', 'persist': 'Prevent execution of a task if all products exist and even if something has changed (dependencies, source file, products). This decorator might be useful for expensive tasks where only the formatting of the file has changed. The state of the files which have changed will also be remembered and another run will skip the task with success.', 'skip': 'Skip a task and all its dependent tasks.', 'skip_ancestor_failed': 'Internal decorator applied to tasks if any of its preceding tasks failed.', 'skip_unchanged': 'Internal decorator applied to tasks which have already been executed and have not been changed.', 'skipif': 'Skip a task and all its dependent tasks if a condition is met.', 'task': 'Mark a function as a task regardless of its name. Or mark tasks which are repeated in a loop. See this tutorial for more information: [link https://bit.ly/3DWrXS3]https://bit.ly/3DWrXS3[/].', 'try_first': 'Try to execute a task a early as possible.', 'try_last': 'Try to execute a task a late as possible.'}, 'config': None, 'database_url': sqlite:////Users/tobiasr/git/pytask/.pytask/pytask.sqlite3, 'editor_url_scheme': 'file', 'export': <_ExportFormats.NO: 'no'>, 'hook_module': None, 'ignore': ['.codecov.yml', '.gitignore', '.pre-commit-config.yaml', '.readthedocs.yml', '.readthedocs.yaml', 'readthedocs.yml', 'readthedocs.yaml', 'environment.yml', 'pyproject.toml', 'setup.cfg', 'tox.ini', '.git/*', '.venv/*', '.pixi/*', '*.egg-info/*', '.ipynb_checkpoints/*', '.mypy_cache/*', '.nox/*', '.tox/*', '_build/*', '__pycache__/*', 'build/*', 'dist/*', 'pytest_cache/*'], 'paths': [], 'layout': 'dot', 'output_path': 'dag.pdf', 'rank_direction': <_RankDirection.TB: 'TB'>, 'expression': '', 'marker_expression': '', 'nodes': False, 'strict_markers': False, 'directories': False, 'exclude': [None, '.git/*', '/Users/tobiasr/git/pytask/.pytask/*'], 'mode': <_CleanMode.DRY_RUN: 'dry-run'>, 'quiet': False, 'capture': , 'debug_pytask': False, 'disable_warnings': False, 'dry_run': False, 'explain': False, 'force': False, 'max_failures': inf, 'n_entries_in_table': 15, 'pdb': False, 'pdbcls': None, 's': False, 'show_capture': , 'show_errors_immediately': False, 'show_locals': False, 'show_traceback': True, 'sort_table': True, 'trace': False, 'verbose': 1, 'stop_after_first_failure': False, 'check_casing_of_paths': True, 'pdb_cls': '', 'tasks': [, , at 0x1116de160>], 'task_files': ('task_*.py',), 'command': 'build', 'root': PosixPath('/Users/tobiasr/git/pytask'), 'filterwarnings': []}, collection_reports=[CollectionReport(outcome=, node=TaskWithoutPath(name='task_merge_files', function=, depends_on={'first': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={}), 'second': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/hello_world.txt'), name='pytask/docs/source/how_to_guides/hello_world.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e5b74aa3-9f13-4f44-b1a3-81ebb5338ae3'), 'after': [], 'is_generator': False, 'duration': (1762291140.39473, 1762291140.395363)}), exc_info=None), CollectionReport(outcome=, node=TaskWithoutPath(name='task_create_first_file', function=, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('ced0baa8-1f08-43dc-b681-88bdc41a2a50'), 'after': [], 'is_generator': False, 'duration': (1762291140.380946, 1762291140.381288)}), exc_info=None), CollectionReport(outcome=, node=TaskWithoutPath(name='task_create_second_file', function= at 0x1116de160>, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e45f1eec-32c3-4a93-bac4-92efa48f3b3f'), 'after': [], 'is_generator': False, 'duration': (1762291140.388247, 1762291140.388503)}), exc_info=None)], dag=, hook=, tasks=[TaskWithoutPath(name='task_merge_files', function=, depends_on={'first': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={}), 'second': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/hello_world.txt'), name='pytask/docs/source/how_to_guides/hello_world.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e5b74aa3-9f13-4f44-b1a3-81ebb5338ae3'), 'after': [], 'is_generator': False, 'duration': (1762291140.39473, 1762291140.395363)}), TaskWithoutPath(name='task_create_first_file', function=, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('ced0baa8-1f08-43dc-b681-88bdc41a2a50'), 'after': [], 'is_generator': False, 'duration': (1762291140.380946, 1762291140.381288)}), TaskWithoutPath(name='task_create_second_file', function= at 0x1116de160>, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e45f1eec-32c3-4a93-bac4-92efa48f3b3f'), 'after': [], 'is_generator': False, 'duration': (1762291140.388247, 1762291140.388503)})], dag_report=None, execution_reports=[ExecutionReport(task=TaskWithoutPath(name='task_create_first_file', function=, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('ced0baa8-1f08-43dc-b681-88bdc41a2a50'), 'after': [], 'is_generator': False, 'duration': (1762291140.380946, 1762291140.381288)}), outcome=, exc_info=None, sections=[]), ExecutionReport(task=TaskWithoutPath(name='task_create_second_file', function= at 0x1116de160>, depends_on={}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e45f1eec-32c3-4a93-bac4-92efa48f3b3f'), 'after': [], 'is_generator': False, 'duration': (1762291140.388247, 1762291140.388503)}), outcome=, exc_info=None, sections=[]), ExecutionReport(task=TaskWithoutPath(name='task_merge_files', function=, depends_on={'first': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/first.txt'), name='pytask/docs/source/how_to_guides/first.txt', attributes={}), 'second': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/second.txt'), name='pytask/docs/source/how_to_guides/second.txt', attributes={})}, produces={'return': PathNode(path=PosixPath('/Users/tobiasr/git/pytask/docs/source/how_to_guides/hello_world.txt'), name='pytask/docs/source/how_to_guides/hello_world.txt', attributes={})}, markers=[Mark(name='task', args=(), kwargs={})], report_sections=[], attributes={'collection_id': UUID('e5b74aa3-9f13-4f44-b1a3-81ebb5338ae3'), 'after': [], 'is_generator': False, 'duration': (1762291140.39473, 1762291140.395363)}), outcome=, exc_info=None, sections=[])], exit_code=, collection_start=1762291140.371644, collection_end=1762291140.374173, execution_start=1762291140.374835, execution_end=1762291140.402872, n_tasks_failed=0, scheduler=TopologicalSorter(dag=, priorities={'45a637ca3cc7aa973d4b315cc1bef02217b79918357fd35c6fa61f4e2d2f9948': 0, '82d6a7ce01a2a50d5d4bd5081d662df92b8c500fbc172f94fb026c9d1d4ebc4a': 0, '2a06f358fc8e621754c133af76f5ac1b3e8ad5172b5803823cb264b30ea5d829': 0}, _nodes_processing=set(), _nodes_done={'2a06f358fc8e621754c133af76f5ac1b3e8ad5172b5803823cb264b30ea5d829', '82d6a7ce01a2a50d5d4bd5081d662df92b8c500fbc172f94fb026c9d1d4ebc4a', '45a637ca3cc7aa973d4b315cc1bef02217b79918357fd35c6fa61f4e2d2f9948'}), should_stop=False, warnings=[])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"session"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Repeated Tasks\n",
"\n",
"You can also create multiple tasks with the same function by repeating the task in a loop. (Because we are collecting the tasks ourselves in a list, we don't necessarily need the `@task` decorator, but you can still use it.) This is useful when you want to run the same operation with different parameters."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"from pytask import Product\n",
"\n",
"tasks = []\n",
"for i in range(3):\n",
"\n",
" def create_file(\n",
" value: int = i * 100, path: Annotated[Path, Product] = Path(f\"output_{i}.txt\")\n",
" ) -> None:\n",
" path.write_text(f\"Result: {value}\")\n",
"\n",
" tasks.append(create_file)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"────────────────────────────────────────────── Start pytask session ───────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[39m────────────────────────────────────────────── \u001b[0mStart pytask session\u001b[39m ───────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Platform: darwin -- Python 3.12.12, pytask 0.6.0, pluggy 1.6.0\n",
"\n"
],
"text/plain": [
"Platform: darwin -- Python 3.12.12, pytask 0.6.0, pluggy 1.6.0\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Root: /Users/tobiasr/git/pytask\n",
"\n"
],
"text/plain": [
"Root: \u001b[35m/Users/tobiasr/git/\u001b[0m\u001b[95mpytask\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n"
],
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"Collected 3 tasks.\n",
"\n"
],
"text/plain": [
"Collected 3 tasks.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
],
"text/plain": [
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "859e802368444863ba691ca1474c1163",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Output()"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n"
],
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"╭────────────────────────┬─────────╮\n",
"│ Task │ Outcome │\n",
"├────────────────────────┼─────────┤\n",
"│ create_file[0-path0] │ . │\n",
"│ create_file[100-path1] │ . │\n",
"│ create_file[200-path2] │ . │\n",
"╰────────────────────────┴─────────╯\n",
"\n"
],
"text/plain": [
"╭────────────────────────┬─────────╮\n",
"│\u001b[1m \u001b[0m\u001b[1mTask \u001b[0m\u001b[1m \u001b[0m│\u001b[1m \u001b[0m\u001b[1mOutcome\u001b[0m\u001b[1m \u001b[0m│\n",
"├────────────────────────┼─────────┤\n",
"│ \u001b]8;id=133063;file://None\u001b\\create_file[0-path0] \u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"│ \u001b]8;id=167889;file://None\u001b\\create_file[100-path1]\u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"│ \u001b]8;id=365664;file://None\u001b\\create_file[200-path2]\u001b]8;;\u001b\\ │ \u001b[38;2;19;124;57m. \u001b[0m │\n",
"╰────────────────────────┴─────────╯\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
],
"text/plain": [
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[2m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"╭─────────── Summary ────────────╮\n",
"│ 3 Collected tasks │\n",
"│ 3 Succeeded (100.0%) │\n",
"╰────────────────────────────────╯\n",
"\n"
],
"text/plain": [
"\u001b[38;2;19;124;57m╭─\u001b[0m\u001b[38;2;19;124;57m──────────\u001b[0m\u001b[1;38;2;242;242;242m Summary \u001b[0m\u001b[38;2;19;124;57m───────────\u001b[0m\u001b[38;2;19;124;57m─╮\u001b[0m\n",
"\u001b[38;2;19;124;57m│\u001b[0m \u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m3\u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242mCollected tasks\u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m\u001b[38;2;242;242;242m \u001b[0m \u001b[38;2;19;124;57m│\u001b[0m\n",
"\u001b[38;2;19;124;57m│\u001b[0m \u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m3\u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57mSucceeded \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m(100.0%)\u001b[0m\u001b[38;2;242;242;242;48;2;19;124;57m \u001b[0m \u001b[38;2;19;124;57m│\u001b[0m\n",
"\u001b[38;2;19;124;57m╰────────────────────────────────╯\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"──────────────────────────────────────────── Succeeded in 0.03 seconds ────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[38;2;19;124;57m──────────────────────────────────────────── \u001b[0m\u001b[38;2;19;124;57mSucceeded in 0.03 seconds\u001b[0m\u001b[38;2;19;124;57m ────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"session = pytask.build(tasks=tasks)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# Cleanup\n",
"for i in range(3):\n",
" Path(f\"output_{i}.txt\").unlink()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuring the build\n",
"\n",
"To configure the build, {func}`pytask.build` has many more options that are the same that you find on the commandline."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[31mSignature:\u001b[39m\n",
"pytask.build(\n",
" *,\n",
" capture: \u001b[33m\"Literal['fd', 'no', 'sys', 'tee-sys'] | CaptureMethod\"\u001b[39m = ,\n",
" check_casing_of_paths: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mTrue\u001b[39;00m,\n",
" config: \u001b[33m'Path | None'\u001b[39m = \u001b[38;5;28;01mNone\u001b[39;00m,\n",
" database_url: \u001b[33m'str'\u001b[39m = \u001b[33m''\u001b[39m,\n",
" debug_pytask: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" disable_warnings: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" dry_run: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" editor_url_scheme: \u001b[33m\"Literal['no_link', 'file', 'vscode', 'pycharm'] | str\"\u001b[39m = \u001b[33m'file'\u001b[39m,\n",
" explain: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" expression: \u001b[33m'str'\u001b[39m = \u001b[33m''\u001b[39m,\n",
" force: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" ignore: \u001b[33m'Iterable[str]'\u001b[39m = (),\n",
" marker_expression: \u001b[33m'str'\u001b[39m = \u001b[33m''\u001b[39m,\n",
" max_failures: \u001b[33m'float'\u001b[39m = inf,\n",
" n_entries_in_table: \u001b[33m'int'\u001b[39m = \u001b[32m15\u001b[39m,\n",
" paths: \u001b[33m'Path | Iterable[Path]'\u001b[39m = (),\n",
" pdb: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" pdb_cls: \u001b[33m'str'\u001b[39m = \u001b[33m''\u001b[39m,\n",
" s: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" show_capture: \u001b[33m\"Literal['no', 'stdout', 'stderr', 'all'] | ShowCapture\"\u001b[39m = ,\n",
" show_errors_immediately: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" show_locals: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" show_traceback: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mTrue\u001b[39;00m,\n",
" sort_table: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mTrue\u001b[39;00m,\n",
" stop_after_first_failure: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" strict_markers: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" tasks: \u001b[33m'Callable[..., Any] | PTask | Iterable[Callable[..., Any] | PTask]'\u001b[39m = (),\n",
" task_files: \u001b[33m'Iterable[str]'\u001b[39m = (\u001b[33m'task_*.py'\u001b[39m,),\n",
" trace: \u001b[33m'bool'\u001b[39m = \u001b[38;5;28;01mFalse\u001b[39;00m,\n",
" verbose: \u001b[33m'int'\u001b[39m = \u001b[32m1\u001b[39m,\n",
" **kwargs: \u001b[33m'Any'\u001b[39m,\n",
") -> \u001b[33m'Session'\u001b[39m\n",
"\u001b[31mDocstring:\u001b[39m\n",
"Run pytask.\n",
"\n",
"This is the main command to run pytask which usually receives kwargs from the\n",
"command line interface. It can also be used to run pytask interactively. Pass\n",
"configuration in a dictionary.\n",
"\n",
"Parameters\n",
"----------\n",
"capture\n",
" The capture method for stdout and stderr.\n",
"check_casing_of_paths\n",
" Whether errors should be raised when file names have different casings.\n",
"config\n",
" A path to the configuration file.\n",
"database_url\n",
" An URL to the database that tracks the status of tasks.\n",
"debug_pytask\n",
" Whether debug information should be shown.\n",
"disable_warnings\n",
" Whether warnings should be disabled and not displayed.\n",
"dry_run\n",
" Whether a dry-run should be performed that shows which tasks need to be rerun.\n",
"editor_url_scheme\n",
" An url scheme that allows to click on task names, node names and filenames and\n",
" jump right into you preferred editor to the right line.\n",
"explain\n",
" Explain why tasks need to be executed by showing what changed.\n",
"expression\n",
" Same as ``-k`` on the command line. Select tasks via expressions on task ids.\n",
"force\n",
" Run tasks even though they would be skipped since nothing has changed.\n",
"ignore\n",
" A pattern to ignore files or directories. Refer to ``pathlib.Path.match`` for\n",
" more info.\n",
"marker_expression\n",
" Same as ``-m`` on the command line. Select tasks via marker expressions.\n",
"max_failures\n",
" Stop after some failures.\n",
"n_entries_in_table\n",
" How many entries to display in the table during the execution. Tasks which are\n",
" running are always displayed.\n",
"paths\n",
" A path or collection of paths where pytask looks for the configuration and\n",
" tasks.\n",
"pdb\n",
" Start the interactive debugger on errors.\n",
"pdb_cls\n",
" Start a custom debugger on errors. For example:\n",
" ``--pdbcls=IPython.terminal.debugger:TerminalPdb``\n",
"s\n",
" Shortcut for ``capture=\"no\"``.\n",
"show_capture\n",
" Choose which captured output should be shown for failed tasks.\n",
"show_errors_immediately\n",
" Show errors with tracebacks as soon as the task fails.\n",
"show_locals\n",
" Show local variables in tracebacks.\n",
"show_traceback\n",
" Choose whether tracebacks should be displayed or not.\n",
"sort_table\n",
" Sort the table of tasks at the end of the execution.\n",
"stop_after_first_failure\n",
" Stop after the first failure.\n",
"strict_markers\n",
" Raise errors for unknown markers.\n",
"tasks\n",
" A task or a collection of tasks which can be callables or instances following\n",
" {class}`~pytask.PTask`.\n",
"task_files\n",
" A pattern to describe modules that contain tasks.\n",
"trace\n",
" Enter debugger in the beginning of each task.\n",
"verbose\n",
" Make pytask verbose (>= 0) or quiet (= 0).\n",
"\n",
"Returns\n",
"-------\n",
"session : pytask.Session\n",
" The session captures all the information of the current run.\n",
"\u001b[31mFile:\u001b[39m ~/git/pytask/src/_pytask/build.py\n",
"\u001b[31mType:\u001b[39m function"
]
}
],
"source": [
"pytask.build?"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"# Cleanup\n",
"for name in (\"first.txt\", \"second.txt\", \"hello_world.txt\"):\n",
" Path(name).unlink()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
}