---
warning: DO NOT CHANGE THIS MANUALLY, THIS IS GENERATED BY https://github.com/duckdb/community-extensions repository, check README there
title: tidbfs
excerpt: |
  DuckDB Community Extensions
  Read and write analytics files on TiDB Filesystem through its HTTP API, with experimental native DuckDB database storage.

extension:
  name: tidbfs
  description: Read and write analytics files on TiDB Filesystem through its HTTP API, with experimental native DuckDB database storage.
  version: 0.4.0
  language: C++
  build: cmake
  license: MIT
  maintainers:
    - qiffang
  # Initial release: Linux amd64 (glibc) and macOS arm64 only.
  excluded_platforms: 'linux_arm64;linux_amd64_musl;linux_arm64_musl;osx_amd64;windows_amd64;windows_arm64;windows_amd64_mingw;windows_amd64_rtools;wasm_mvp;wasm_eh;wasm_threads'
  opt_in_platforms: 'osx_arm64'
  requires_toolchains: 'vcpkg;python3'
  vcpkg_commit: cd61e1e26a038e82d6550a3ebbe0fbbfe7da78e3
repo:
  github: drive9-ai/duckdb
  ref: 797d2c6250592d2b84716ebf43198b39bdfd3241
docs:
  docs_url: https://github.com/drive9-ai/duckdb#readme
  hello_world: |
    INSTALL tidbfs FROM community;
    LOAD tidbfs;

    -- Use your TiDB Filesystem data-plane endpoint and API key.
    CREATE TEMPORARY SECRET fs (
        TYPE tidbfs,
        ENDPOINT 'https://YOUR_FILESYSTEM_ENDPOINT',
        TOKEN 'YOUR_FILESYSTEM_API_KEY',
        SCOPE 'tidbfs:///'
    );

    COPY (SELECT i AS id, i * 10 AS value FROM range(100) t(i))
    TO 'tidbfs:///example.parquet' (FORMAT PARQUET);

    SELECT count(*), sum(value) FROM read_parquet('tidbfs:///example.parquet');
  extended_description: |
    `tidbfs` is a native C++ filesystem extension for TiDB Filesystem. It calls
    the existing HTTP data plane directly, without FUSE or a Go runtime.
    Its initial community release targets DuckDB 2.0.0 on Linux amd64 (glibc)
    and macOS arm64. OpenSSL and libcurl are statically linked into the artifact.

    The `tidbfs:///` scheme supports Parquet, CSV and JSON reads, globbing,
    Hive partition pruning, `COPY TO`, partitioned output, and database
    `EXPORT`/`IMPORT`. Reads use conditional HTTP Range requests and parallel
    reads. Writes stage locally and publish a complete file using the server's
    revision checks and multipart upload APIs. They require local scratch
    space; multi-file operations are not atomic transactions.

    Experimental `tidbfs+db:///` storage can place native DuckDB database and
    WAL files on the same filesystem using the public extent API and C++ SDK.
    It preserves the existing sessions, advisory file locks and persistence
    contract. All writers must cooperate with these locks; this is not a
    replacement for DuckDB's concurrency rules or a claim of arbitrary
    multi-process writer support. Native mode requires an unscoped filesystem
    token and a server with the documented extent/session APIs.

    See the [compatibility matrix](https://github.com/drive9-ai/duckdb/blob/main/docs/compatibility.md)
    for the supported server capabilities, recovery tests and remaining limits.
    The [publication guide](https://github.com/drive9-ai/duckdb/blob/797d2c6250592d2b84716ebf43198b39bdfd3241/docs/distribution.md)
    documents the official build and artifact-level regression tests.

extension_star_count: 1
extension_star_count_pretty: 1
extension_download_count: 5
extension_download_count_pretty: 5
image: '/images/community_extensions/social_preview/preview_community_extension_tidbfs.png'
layout: community_extension_doc
---
### Compatibility Matrix

This table covers all DuckDB releases that have not reached [end-of-life](https://duckdb.org/release_calendar), including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

<div class="extension_compatibility_table"></div>

| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
| :--- | :---: | :---: | :---: | :---: | :---: |
| Linux AMD64 | ❌ | ❌ | ❌ | ❌ | ✅ |
| Linux ARM64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| macOS AMD64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| macOS ARM64 | ❌ | ❌ | ❌ | ❌ | ✅ |
| Windows AMD64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| Windows ARM64 | ❌ | ❌ | ❌ | ❌ | ❌ |
| WebAssembly | ❌ | ❌ | ❌ | ❌ | ❌ |

### Installing and Loading
```sql
INSTALL {{ page.extension.name }} FROM community;
LOAD {{ page.extension.name }};
```

{% if page.docs.hello_world %}
### Example
```sql
{{ page.docs.hello_world }}```
{% endif %}

{% if page.docs.extended_description %}
### About {{ page.extension.name }}
{{ page.docs.extended_description }}
{% endif %}
### Added Functions

<div class="extension_functions_table"></div>

This extension does not add any functions.

### Overloaded Functions

<div class="extension_functions_table"></div>

This extension does not add any function overloads.

### Added Types

<div class="extension_types_table"></div>

This extension does not add any types.

### Added Settings

<div class="extension_settings_table"></div>

|        name         |                                      description                                       | input_type | scope  | aliases | typed_value |
|---------------------|----------------------------------------------------------------------------------------|------------|--------|---------|-------------|
| temp_directory      | Set the directory to which temporary files are written.                                | VARCHAR    | GLOBAL | []      | post.db.tmp |
| tidbfs_heartbeat_ms | Extent session heartbeat in milliseconds; set before first extent access (expiry = 5x) | BIGINT     | GLOBAL | []      | 12000       |
| tidbfs_read_ahead   | Bytes to fetch ahead on small reads (0 disables prefetch, max 1 MiB)                   | BIGINT     | GLOBAL | []      | 65536       |