---
warning: DO NOT CHANGE THIS MANUALLY, THIS IS GENERATED BY https://github.com/duckdb/community-extensions repository, check README there
title: opendal
excerpt: |
  DuckDB Community Extensions
  Access S3, local filesystem, and other storage services through Apache OpenDAL

extension:
  name: opendal
  description: Access S3, local filesystem, and other storage services through Apache OpenDAL
  version: "0.1.0"
  language: C++ & Rust
  build: cmake
  license: MIT
  maintainers:
    - chitralverma
  requires_toolchains: rust
  # Exclusions:
  # - WASM: extension-ci-tools pins rust-toolchain@1.86 which is below opendal's MSRV (1.91). See https://github.com/duckdb/extension-ci-tools/issues/385
  # - MinGW: Rtools MinGW is missing libgcc_eh.a which is required by rustc for the gnu target. Rely on standard MSVC instead.
  excluded_platforms: "wasm_mvp;wasm_eh;wasm_threads;windows_amd64_rtools;windows_amd64_mingw"

repo:
  github: chitralverma/duckdb-opendal
  ref: 6e998efad81618458518fbc061f1797354f13c56

docs:
  hello_world: |
    -- Install and load the extension
    INSTALL opendal FROM community;
    LOAD opendal;

    -- Create a secret for local filesystem access
    CREATE SECRET my_fs (TYPE fs, SCOPE 'fs://', config MAP{'root': '.'});

    -- Write a Parquet file through OpenDAL
    COPY (SELECT range AS id, range * 2 AS doubled FROM range(100))
    TO 'fs:///output/data.parquet' (FORMAT parquet);

    -- Read it back
    SELECT count(*), sum(doubled) FROM read_parquet('fs:///output/data.parquet');

    -- List files
    SELECT * FROM opendal_ls('fs:///output/');

    -- Stat a file
    SELECT name, metadata.content_length FROM opendal_stat('fs:///output/data.parquet');
  extended_description: |
    The `opendal` extension integrates [Apache OpenDAL](https://opendal.apache.org/)
    into DuckDB as a virtual filesystem, enabling transparent read and write access
    to multiple storage services through a unified SQL interface.

    ### Supported Storage Services

    - **`fs://`** / **`file://`** — Local filesystem
    - **`s3://`** — Amazon S3 and S3-compatible stores (MinIO, R2, etc.)
    - **`memory://`** — In-memory storage (useful for testing)
    - more services coming soon

    ### Key Features

    - **Read & write** Parquet, CSV, and JSON files through any supported service
    - **Glob patterns** work transparently (`read_parquet('s3://bucket/**/*.parquet')`)
    - **Table functions** for storage operations:
      - `opendal_stat()` — file metadata
      - `opendal_ls()` — directory listing with optional recursion
      - `opendal_glob()` — pattern-based file discovery
      - `opendal_du()` — disk usage summaries
      - `opendal_copy()` — cross-service and same-service file copy
    - **Scoped secrets** with per-service configuration via DuckDB's secret manager
    - **Configurable layers** — retry, timeout, I/O concurrency, and caching
      (in-memory via foyer, optional on-disk tier)
    - **Global settings** for default I/O, retry, timeout, and cache configuration
    - **Coexistence** with DuckDB's native `httpfs` extension via
      `opendal_override_native_filesystems` setting
    - **DuckDB FileSystem logging** integration for structured I/O tracing

    ### Configuration Example

    ```sql
    CREATE SECRET warehouse (
        TYPE s3,
        SCOPE 's3://warehouse',
        config MAP{
            'access_key_id': '...',
            'secret_access_key': '...',
            'region': 'us-east-1',
            'endpoint': 'http://localhost:9000'
        },
        io_config MAP{'read.concurrent': '4', 'write.chunk_size': '8 MiB'},
        retry_config MAP{'max_times': '5', 'jitter': 'true'},
        cache_config MAP{'memory_size': '256 MiB'}
    );

    SELECT * FROM read_parquet('s3://warehouse/data/*.parquet');
    ```

    For full documentation, see the
    [project repository](https://github.com/chitralverma/duckdb-opendal).

extension_star_count: 2
extension_star_count_pretty: 2
extension_download_count: 728
extension_download_count_pretty: 728
image: '/images/community_extensions/social_preview/preview_community_extension_opendal.png'
layout: community_extension_doc
---
### Compatibility Matrix

This table covers all DuckDB releases that have not reached [end-of-life](https://duckdb.org/release_calendar), including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

<div class="extension_compatibility_table"></div>

| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
| :--- | :---: | :---: | :---: | :---: | :---: |
| Linux AMD64 | ❌ | ❌ | ✅ | ❌ | ❌ |
| Linux ARM64 | ❌ | ❌ | ✅ | ❌ | ❌ |
| macOS AMD64 | ❌ | ❌ | ✅ | ❌ | ❌ |
| macOS ARM64 | ❌ | ❌ | ✅ | ❌ | ❌ |
| Windows AMD64 | ❌ | ❌ | ✅ | ❌ | ❌ |
| Windows ARM64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| WebAssembly | ❌ | ❌ | ❌ | ❌ | ❌ |

### Installing and Loading
```sql
INSTALL {{ page.extension.name }} FROM community;
LOAD {{ page.extension.name }};
```

{% if page.docs.hello_world %}
### Example
```sql
{{ page.docs.hello_world }}```
{% endif %}

{% if page.docs.extended_description %}
### About {{ page.extension.name }}
{{ page.docs.extended_description }}
{% endif %}
### Added Functions

<div class="extension_functions_table"></div>

|      function_name      | function_type | description | comment | examples |
|-------------------------|---------------|-------------|---------|----------|
| opendal_copy            | table         | NULL        | NULL    |          |
| opendal_du              | table         | NULL        | NULL    |          |
| opendal_glob            | table         | NULL        | NULL    |          |
| opendal_is_manually_set | scalar        | NULL        | NULL    |          |
| opendal_ls              | table         | NULL        | NULL    |          |
| opendal_stat            | table         | NULL        | NULL    |          |
| opendal_version         | scalar        | NULL        | NULL    |          |

### Overloaded Functions

<div class="extension_functions_table"></div>

This extension does not add any function overloads.

### Added Types

<div class="extension_types_table"></div>

This extension does not add any types.

### Added Settings

<div class="extension_settings_table"></div>

|                name                 |                                                                             description                                                                             |      input_type       | scope  | aliases |
|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------|--------|---------|
| opendal_cache_config                | Global OpenDAL data-cache options.                                                                                                                                  | MAP(VARCHAR, VARCHAR) | GLOBAL | []      |
| opendal_io_config                   | Global OpenDAL reader/writer options.                                                                                                                               | MAP(VARCHAR, VARCHAR) | GLOBAL | []      |
| opendal_override_native_filesystems | Comma-separated schemes (e.g. 's3,gcs') for which the OpenDAL filesystem overrides native extensions in DuckDB's filesystem dispatch. Empty disables all overrides. | VARCHAR               | GLOBAL | []      |
| opendal_retry_config                | Global OpenDAL retry-layer options.                                                                                                                                 | MAP(VARCHAR, VARCHAR) | GLOBAL | []      |
| opendal_timeout_config              | Global OpenDAL timeout-layer options.                                                                                                                               | MAP(VARCHAR, VARCHAR) | GLOBAL | []      |