---
warning: DO NOT CHANGE THIS MANUALLY, THIS IS GENERATED BY https://github.com/duckdb/community-extensions repository, check README there
title: duck_block_utils
excerpt: |
DuckDB Community Extensions
Build, transform, validate, and extract content from structured documents using the duck_block type
extension:
name: duck_block_utils
description: Build, transform, validate, and extract content from structured documents using the duck_block type
version: 4.0.0
language: C++
build: cmake
license: MIT
maintainers:
- teaguesterling
repo:
github: teaguesterling/duckdb_duck_block_utils
# andium (DuckDB v1.4.5 track) intentionally left at the v1.2.1 commit. The tree
# this release is built from targets DuckDB v1.5.5 and carries no v1.4.x support,
# so advancing this pin would point a v1.4.5 build at a tree that cannot build for
# it. v1.4.5 users should move to the v1.5.x track.
andium: 125662df9e5450105dc9b7957ad955cb53d7beec
ref: 65f03cb7f776549a68ae9efba0b5a6d6adc259d3
# variegata pins the v1.5.6 track. scripts/build.py sends a v1.5.6 build here when
# present and to `ref` otherwise, and unlike ref_next it has no skip branch -- so
# once `ref` is 2.0-only a v1.5.6 build would fail rather than be skipped. Equal to
# `ref` while both lines build; advance only to a commit v1.5.6 can still build.
variegata: 65f03cb7f776549a68ae9efba0b5a6d6adc259d3
ref_next: 65f03cb7f776549a68ae9efba0b5a6d6adc259d3
docs:
hello_world: |
-- Build a document programmatically
SELECT duck_blocks_assemble([
duck_block_heading(1, 'Hello World'),
duck_block_paragraph([
duck_block_text('This is '),
duck_block_bold('important'),
duck_block_text(' content.')
]),
duck_block_code('sql', 'SELECT * FROM documents')
]);
-- Render it as plain text
SELECT duck_blocks_to_text(duck_blocks_assemble([
duck_block_heading(1, 'Hello World'),
duck_block_paragraph('Some prose.')
]));
-- Works with the markdown extension for reading and writing files
LOAD markdown;
LOAD duck_block_utils;
-- Table of contents from a markdown file
SELECT * FROM duck_blocks_toc_rows(
(SELECT list(b) FROM read_markdown_blocks('README.md') b)
);
-- Validate document structure
SELECT duck_blocks_validate(
(SELECT list(b) FROM read_markdown_blocks('doc.md') b)
);
extended_description: |
The Duck Block Utils extension provides tools for building, transforming, validating, and
extracting content from structured documents. All functions use the `duck_block` type - a
unified representation for document elements that works across formats.
**Documentation:** [duck-block-utils.readthedocs.io](https://duck-block-utils.readthedocs.io)
## 4.0.0 moves the Pandoc converter out
Nine names and eleven overloads left this extension for
[duckdb_panduck](https://github.com/teaguesterling/duckdb_panduck), with no
alias path: `pandoc_ast_to_blocks`, `read_pandoc_ast`, `write_pandoc_ast`,
`duck_blocks_to_pandoc_ast`, `duck_blocks_to_pandoc_blocks`, `pandoc_ast`,
`pandoc_inlines_to_db_inlines`, `pandoc_inlines_to_text` and
`duck_blocks_inlines_to_pandoc`. This extension is utility macros over the
`duck_block` type; the Pandoc AST is panduck's subject.
**Upgrade both sides.** One extension must own each name, and `pandoc_ast`
has an identical signature in both, so loading 3.5.1 together with panduck
0.7.0 makes the second `LOAD` fail. Every other combination is fine.
No spec change: `duck_block_spec_version()` is unchanged at spec 1.4.
## 2.0.0 is a breaking rename
Every `db_*` function was renamed to `duck_block_*` (builders) or `duck_blocks_*`
(operations on a list). 54 names present in 1.6.1 are gone and there is no alias
path, so 1.6.1 queries must be updated. The mapping is mechanical:
```sql
db_heading(1, 'Title') -> duck_block_heading(1, 'Title')
db_assemble(blocks) -> duck_blocks_assemble(blocks)
db_blocks_to_text(blocks) -> duck_blocks_to_text(blocks)
db_page -> page_break
```
## The duck_block Type
A unified struct for block-level, inline and metadata-value elements:
```sql
STRUCT(
kind VARCHAR, -- 'block', 'inline' or 'value'
element_type VARCHAR, -- 'heading', 'paragraph', 'bold', 'link', etc.
content VARCHAR, -- text, iff the element has a single text child
level INTEGER, -- depth in a depth-first ordering; top is 1 (0 is the
-- optional explicit document root)
encoding VARCHAR, -- 'text', 'json', 'yaml', 'html', 'xml',
-- 'latex', 'markdown', 'toml'
attributes MAP(VARCHAR, VARCHAR), -- element-specific metadata
element_order INTEGER -- position in the document
)
```
`level` is structural depth, never a heading rank - a heading's rank lives in
`attributes['heading_level']`. `content` is populated if and only if an element has
a single text child; a heading additionally keeps a flattened title alongside its
rich children.
## Builders
All builder functions return `LIST(duck_block)` so they compose by nesting:
```sql
SELECT duck_block_paragraph([
duck_block_text('Click '),
duck_block_link('https://example.com', 'here'),
duck_block_text(' for more.')
]);
```
| Function | Description |
|----------|-------------|
| `duck_block_heading(level, content)` | Heading (h1-h6) |
| `duck_block_paragraph(content)` | Paragraph |
| `duck_block_code(language, content)` | Fenced code block |
| `duck_block_blockquote(content)` | Block quote |
| `duck_block_list_block(ordered, items[])` | Ordered or unordered list |
| `duck_block_hr()` | Horizontal rule |
| `duck_block_image(src, alt, title)` | Image |
| `duck_block_metadata(content)` | Metadata blob (YAML frontmatter) |
| `duck_block_raw(format, content)` | Raw content block |
| `duck_block_text(content)` | Plain text |
| `duck_block_bold(content)` | Bold |
| `duck_block_italic(content)` | Italic |
| `duck_block_link(href, content)` | Hyperlink |
| `duck_block_inline_code(content)` | Inline code |
| `duck_block_math(content)` | Math |
| `duck_block_strikethrough(content)` | Strikethrough |
| `duck_block_superscript(content)` / `duck_block_subscript(content)` | Super/subscript |
## Assembly
| Function | Description |
|----------|-------------|
| `duck_blocks_assemble(blocks[])` | Combine blocks into a document |
| `duck_blocks_document(blocks[])` | Alias for assemble |
| `duck_block_section(level, title, children[])` | Section with a heading |
| `duck_blocks_concat(a, b)` | Concatenate block lists |
| `duck_blocks_rebase_levels(blocks, offset)` | Shift heading levels |
## Query and extraction
| Function | Description |
|----------|-------------|
| `duck_blocks_to_text(blocks)` | Plain text |
| `duck_blocks_headings(blocks)` | Heading hierarchy |
| `duck_blocks_toc_rows(blocks)` | Table of contents, one row per entry |
| `duck_blocks_get_section(blocks, pattern)` | One section by title or id |
| `duck_blocks_sections_like(blocks, term)` | Sections whose text matches |
| `duck_blocks_page_rows(blocks)` | Page boundaries, separate from the outline |
| `duck_blocks_get_pages(blocks, first, last)` | A page range |
| `duck_blocks_links(blocks)` | Every link |
| `duck_blocks_stats(blocks)` / `duck_blocks_structure(blocks)` | Document shape |
| `duck_blocks_diff(before, after)` | ADDED / REMOVED / MOVED between two versions |
## Validation
| Function | Description |
|----------|-------------|
| `duck_blocks_validate(blocks)` | Spec conformance: valid, plus errors |
| `duck_blocks_repair(blocks)` | Deterministic fixes for the list-level rules: wrap orphans, rebase, renumber |
| `duck_block_implicit_parent(type, kind)` | The wrapper a fragment of that element gets, from the spec table |
| `duck_block_is_body(kind, element_type)` | Per-row half of the body rule: block or inline, and not the metadata blob |
| `duck_blocks_body(blocks)` | The body rows only: no value or metadata subtree, element_order kept |
| `duck_blocks_lint(blocks)` | Advisory conformance warnings |
| `duck_blocks_quality(blocks)` | Document quality, which is NOT conformance |
| `duck_block_spec_version()` | The spec version this build implements |
## Rendering
| Function | Description |
|----------|-------------|
| `duck_blocks_render_ansi(blocks, width)` | Terminal rendering with themes |
## Pandoc integration moved to panduck in 4.0.0
The Pandoc converter now lives in
[duckdb_panduck](https://github.com/teaguesterling/duckdb_panduck), under the
same names. `LOAD panduck;` and the call sites are unchanged.
```sql
LOAD panduck;
SELECT pandoc_ast_to_blocks(ast_json);
```
## Ecosystem
- **[duckdb_markdown](https://github.com/teaguesterling/duckdb_markdown)** - `read_markdown_blocks()`, and `COPY ... TO ... (FORMAT markdown, MARKDOWN_MODE duck_block)`
- **[duckdb_webbed](https://github.com/teaguesterling/duckdb_webbed)** - HTML parsing and generation
```sql
-- Read markdown, append to it, write it back
LOAD markdown;
LOAD duck_block_utils;
COPY (
SELECT unnest(duck_blocks_concat(
(SELECT list(b) FROM read_markdown_blocks('doc.md') b),
duck_blocks_assemble([
duck_block_hr(),
duck_block_paragraph('Generated by DuckDB')
])
)) AS block
) TO 'output.md' (FORMAT markdown, MARKDOWN_MODE duck_block);
```
extension_star_count: 6
extension_star_count_pretty: 6
extension_download_count: 1057
extension_download_count_pretty: 1.1k
image: '/images/community_extensions/social_preview/preview_community_extension_duck_block_utils.png'
layout: community_extension_doc
---
### Compatibility Matrix
This table covers all DuckDB releases that have not reached [end-of-life](https://duckdb.org/release_calendar), including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
| :--- | :---: | :---: | :---: | :---: | :---: |
| Linux AMD64 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Linux ARM64 | ✅ | ✅ | ✅ | ✅ | ✅ |
| macOS AMD64 | ✅ | ✅ | ✅ | ✅ | ✅ |
| macOS ARM64 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Windows AMD64 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Windows ARM64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| WebAssembly | ✅ | ✅ | ✅ | ✅ | ✅ |
### Installing and Loading
```sql
INSTALL {{ page.extension.name }} FROM community;
LOAD {{ page.extension.name }};
```
{% if page.docs.hello_world %}
### Example
```sql
{{ page.docs.hello_world }}```
{% endif %}
{% if page.docs.extended_description %}
### About {{ page.extension.name }}
{{ page.docs.extended_description }}
{% endif %}
### Added Functions
| function_name | function_type | description | comment | examples |
|---------------------------------|---------------|--------------------------------------------------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------------------------|
| duck_block | scalar | Construct a duck_block struct. | NULL | [duck_block('paragraph', 'Hello world'), duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1)] |
| duck_block_aliases | pragma | NULL | NULL | |
| duck_block_attr | scalar | Get an attribute value by key from a duck_block struct. | NULL | [duck_block_attr(duck_block('heading', 'Title', 1, 'text', MAP {'id': 'title'}, 1), 'id')] |
| duck_block_blockquote | scalar | NULL | NULL | |
| duck_block_bold | scalar | NULL | NULL | |
| duck_block_cite | scalar | NULL | NULL | |
| duck_block_code | scalar | NULL | NULL | |
| duck_block_content | scalar | Get the content field of a duck_block struct. | NULL | [duck_block_content(duck_block('paragraph', 'Hello'))] |
| duck_block_div | scalar | NULL | NULL | |
| duck_block_doc_macros | pragma | NULL | NULL | |
| duck_block_encoding | scalar | Get the encoding field of a duck_block struct. | NULL | [duck_block_encoding(duck_block('paragraph', 'Hello'))] |
| duck_block_encoding_names | scalar | Return the list of valid duck_block encoding names. | NULL | [duck_block_encoding_names()] |
| duck_block_ensure_extension | scalar | Ensure specified extension is loaded. | NULL | [duck_block_ensure_extension('json')] |
| duck_block_heading | scalar | NULL | NULL | |
| duck_block_hr | scalar | Build a horizontal rule block (V2). | NULL | [duck_block_hr()] |
| duck_block_image | scalar | NULL | NULL | |
| duck_block_implicit_parent | scalar | Return the implicit parent element type for an element type. | NULL | [duck_block_implicit_parent('block', 'paragraph')] |
| duck_block_inline_code | scalar | Build an inline code element from text (V2). | NULL | [duck_block_inline_code('foo()')] |
| duck_block_inline_image | scalar | NULL | NULL | |
| duck_block_is_body | scalar | Return true if the element kind and type represent body content. | NULL | [duck_block_is_body('block', 'paragraph')] |
| duck_block_italic | scalar | NULL | NULL | |
| duck_block_kind_names | scalar | Return the list of valid duck_block kind names. | NULL | [duck_block_kind_names()] |
| duck_block_level | scalar | Get the level field of a duck_block struct. | NULL | [duck_block_level(duck_block('paragraph', 'Hello'))] |
| duck_block_linebreak | scalar | Build a hard line break inline element (V2). | NULL | [duck_block_linebreak()] |
| duck_block_link | scalar | NULL | NULL | |
| duck_block_list | scalar | NULL | NULL | |
| duck_block_list_block | scalar | NULL | NULL | |
| duck_block_list_item | scalar | NULL | NULL | |
| duck_block_math | scalar | NULL | NULL | |
| duck_block_metadata | scalar | Build a document metadata block from YAML string (V2). | NULL | [duck_block_metadata('title: Document
author: Me')] |
| duck_block_note | scalar | NULL | NULL | |
| duck_block_order | scalar | Get the element_order field of a duck_block struct. | NULL | [duck_block_order(duck_block('paragraph', 'Hello'))] |
| duck_block_paragraph | scalar | NULL | NULL | |
| duck_block_plain | scalar | NULL | NULL | |
| duck_block_quoted | scalar | NULL | NULL | |
| duck_block_raw | scalar | NULL | NULL | |
| duck_block_raw_inline | scalar | NULL | NULL | |
| duck_block_render | pragma | NULL | NULL | |
| duck_block_section | scalar | NULL | NULL | |
| duck_block_set_content | scalar | Return a copy of the duck_block struct with content updated. | NULL | [duck_block_set_content(duck_block('paragraph', 'Hello'), 'World')] |
| duck_block_set_level | scalar | Return a copy of the duck_block struct with level updated. | NULL | [duck_block_set_level(duck_block('paragraph', 'Hello'), 2)] |
| duck_block_set_order | scalar | Return a copy of the duck_block struct with element_order updated. | NULL | [duck_block_set_order(duck_block('paragraph', 'Hello'), 2)] |
| duck_block_smallcaps | scalar | NULL | NULL | |
| duck_block_softbreak | scalar | Build a soft line break inline element (V2). | NULL | [duck_block_softbreak()] |
| duck_block_space | scalar | Build a whitespace space inline element (V2). | NULL | [duck_block_space()] |
| duck_block_span | scalar | NULL | NULL | |
| duck_block_spec_version | scalar | Return the current duck_blocks specification version. | NULL | [duck_block_spec_version()] |
| duck_block_strikethrough | scalar | NULL | NULL | |
| duck_block_subscript | scalar | NULL | NULL | |
| duck_block_superscript | scalar | NULL | NULL | |
| duck_block_terminal_width | scalar | Detect current terminal width in columns. | NULL | [duck_block_terminal_width()] |
| duck_block_text | scalar | Build a text inline element (V2). | NULL | [duck_block_text('Hello world')] |
| duck_block_type | scalar | Get the element_type field of a duck_block struct. | NULL | [duck_block_type(duck_block('paragraph', 'Hello'))] |
| duck_block_type_names | scalar | Return the list of valid duck_block element type names. | NULL | [duck_block_type_names()] |
| duck_block_underline | scalar | NULL | NULL | |
| duck_block_valid | scalar | Check if a duck_block struct is valid according to the duck_blocks spec. | NULL | [duck_block_valid(duck_block('paragraph', 'Hello'))] |
| duck_blocks_assemble | scalar | NULL | NULL | |
| duck_blocks_body | scalar | Extract body blocks excluding non-body/metadata elements. | NULL | [duck_blocks_body(blocks)] |
| duck_blocks_code_blocks | scalar | Extract code blocks from a block list. | NULL | [duck_blocks_code_blocks(blocks)] |
| duck_blocks_code_blocks_structs | scalar | Extract code blocks as list of structs. | NULL | [duck_blocks_code_blocks_structs(blocks)] |
| duck_blocks_concat | scalar | Concatenate two block lists. | NULL | [duck_blocks_concat(blocks1, blocks2)] |
| duck_blocks_diff | table_macro | NULL | NULL | |
| duck_blocks_document | scalar | NULL | NULL | |
| duck_blocks_exclude | scalar | Exclude specified element types from a list of duck_blocks. | NULL | [duck_blocks_exclude(blocks, ['raw', 'metadata'])] |
| duck_blocks_filter | scalar | Filter a list of duck_blocks to only include specified element types. | NULL | [duck_blocks_filter(blocks, ['heading', 'paragraph'])] |
| duck_blocks_get_pages | macro | NULL | NULL | |
| duck_blocks_get_pages_text | macro | NULL | NULL | |
| duck_blocks_get_section | macro | NULL | NULL | |
| duck_blocks_get_section_text | macro | NULL | NULL | |
| duck_blocks_headings | scalar | Extract heading blocks from a block list. | NULL | [duck_blocks_headings(blocks)] |
| duck_blocks_headings_structs | scalar | Extract headings as list of structs. | NULL | [duck_blocks_headings_structs(blocks)] |
| duck_blocks_links | scalar | Extract links from a block list. | NULL | [duck_blocks_links(blocks)] |
| duck_blocks_links_structs | scalar | Extract links as list of structs. | NULL | [duck_blocks_links_structs(blocks)] |
| duck_blocks_lint | scalar | Lint a block list for structural style and issues. | NULL | [duck_blocks_lint(blocks)] |
| duck_blocks_merge | scalar | Merge two lists of duck_blocks and renumber element orders. | NULL | [duck_blocks_merge(blocks1, blocks2)] |
| duck_blocks_normalize | scalar | Normalize block hierarchy by collapsing lone plain blocks into parent. | NULL | [duck_blocks_normalize(blocks)] |
| duck_blocks_page_rows | table_macro | NULL | NULL | |
| duck_blocks_quality | table_macro | NULL | NULL | |
| duck_blocks_rebase_levels | scalar | Rebase hierarchy levels of a list of blocks. | NULL | [duck_blocks_rebase_levels(blocks, 1)] |
| duck_blocks_render_ansi | scalar | NULL | NULL | |
| duck_blocks_reorder | scalar | Renumber element_order sequentially from 1 for a list of duck_blocks. | NULL | [duck_blocks_reorder(blocks)] |
| duck_blocks_repair | scalar | Repair and normalize block structure and levels. | NULL | [duck_blocks_repair(blocks)] |
| duck_blocks_sections_like | table_macro | NULL | NULL | |
| duck_blocks_sections_like_text | table_macro | NULL | NULL | |
| duck_blocks_slice | scalar | Slice a list of duck_blocks by 1-based index range. | NULL | [duck_blocks_slice(blocks, 1, 5)] |
| duck_blocks_stamp | scalar | Stamp duck_blocks list with current spec version metadata. | NULL | [duck_blocks_stamp(blocks)] |
| duck_blocks_stats | scalar | Compute summary statistics of elements in block list. | NULL | [duck_blocks_stats(blocks)] |
| duck_blocks_structure | scalar | Analyze structural composition of a block list. | NULL | [duck_blocks_structure(blocks)] |
| duck_blocks_to_match_text | macro | NULL | NULL | |
| duck_blocks_to_text | scalar | NULL | NULL | |
| duck_blocks_toc | scalar | Extract table of contents as bullet list blocks. | NULL | [duck_blocks_toc(blocks)] |
| duck_blocks_toc_rows | table_macro | NULL | NULL | |
| duck_blocks_toc_structs | scalar | Extract table of contents as list of structs. | NULL | [duck_blocks_toc_structs(blocks)] |
| duck_blocks_validate | scalar | Validate structural correctness of block list. | NULL | [duck_blocks_validate(blocks)] |
| duck_blocks_version | scalar | Return the stamped spec version of a duck_blocks list. | NULL | [duck_blocks_version(blocks)] |
| to_duck_block | scalar | Convert a compatible struct to a duck_block struct. | NULL | [to_duck_block({'element_type': 'paragraph', 'content': 'Hello', 'level': 1, 'encoding': 'text', 'attributes': MAP {}, 'element_order': 1})] |
### Overloaded Functions
This extension does not add any function overloads.
### Added Types
| type_name | type_size | logical_type | type_category | internal |
|----------------|----------:|--------------|---------------|----------|
| duck_block | 0 | STRUCT | COMPOSITE | true |
| duck_block_ext | 0 | STRUCT | COMPOSITE | true |
### Added Settings
| name | description | input_type | scope | aliases | typed_value |
|----------------|---------------------------------------------------------|------------|--------|---------|-------------|
| temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |