What’s Digdag?¶
Digdag is a simple tool that helps you to build, run, schedule, and monitor complex pipelines of tasks. It handles dependency resolution so that tasks run in order or in parallel.
Digdag fits simple replacement of cron, IT operations automation, data analytics batch jobs, machine learning pipelines, and many more by using Directed Acyclic Graphs (DAG) as the infrastructure.
Learn more¶
> “Architecture“
Table of Contents¶
- Getting started
- Architecture
- Workflow definition
- Scheduling workflow
- Operators
- call>: Calls another workflow
- require>: Depends on another workflow
- py>: Python scripts
- rb>: Ruby scripts
- sh>: Shell scripts
- loop>: Repeat tasks
- for_each>: Repeat tasks
- if>: Conditional execution
- fail>: make the workflow failed
- td>: Treasure Data queries
- td_run>: Treasure Data saved queries
- td_load>: Treasure Data bulk loading
- td_ddl>: Treasure Data operations
- td_table_export>: Treasure Data table export to S3
- pg>: PostgreSQL operations
- mail>: Sending email
- embulk>: Embulk data transfer
- Command reference
- Language API - Python
- Language API - Ruby
- Release Notes
- Release 0.8.8
- Release 0.8.7
- Release 0.8.6
- Release 0.8.5
- Release 0.8.4
- Release 0.8.3
- Release 0.8.2
- Release 0.8.1
- Release 0.8.0
- Release 0.7.1
- Release 0.7.0
- Release 0.6.1
- Release 0.6.0
- Release 0.5.9
- Release 0.5.8
- Release 0.5.7
- Release 0.5.6
- Release 0.5.5
- Release 0.5.4
- Release 0.5.3
- Release 0.5.2
- Release 0.5.1
- Release 0.5.0
- Release 0.4.3
- Release 0.4.2
- Release 0.4.1
- Release 0.4.0
- Release 0.3.6
- Release 0.3.5
- Release 0.3.4
- Release 0.3.3
- Release 0.3.2
- Release 0.3.1
- Release 0.3.0
- Release 0.2.8
- Release 0.2.7
- Release 0.2.6
- Release 0.2.5
- Release 0.2.4
- Release 0.2.3
- Release 0.2.2
- Release 0.2.1
- Release 0.2.0
License¶
Digdag is open-source software licensed under Apache License, Version 2.0.
See NOTICE for details.