--- warning: DO NOT CHANGE THIS MANUALLY, THIS IS GENERATED BY https://github.com/duckdb/community-extensions repository, check README there title: duckdb_object_storage excerpt: | DuckDB Community Extensions Store writable DuckDB databases in local or S3-compatible object storage through SlateDB extension: name: duckdb_object_storage description: Store writable DuckDB databases in local or S3-compatible object storage through SlateDB version: 0.2.0 language: Rust & C++ build: cmake license: MIT excluded_platforms: "wasm_mvp;wasm_eh;wasm_threads;windows_amd64;windows_amd64_mingw;windows_amd64_rtools;windows_arm64;windows_arm64_mingw" requires_toolchains: rust maintainers: - dentiny repo: github: dentiny/duckdb-object-storage ref: 56d9c7b928836cfaffa6790c77b7640d50102fe3 docs: hello_world: | INSTALL duckdb_object_storage FROM community; LOAD duckdb_object_storage; ATTACH 'duckdb_objfs://database.db' AS object_db; CREATE TABLE object_db.items(i INTEGER); INSERT INTO object_db.items VALUES (1), (2); CHECKPOINT object_db; SELECT * FROM object_db.items; extended_description: | `duckdb_object_storage` provides a DuckDB virtual filesystem backed by SlateDB. A native DuckDB database, including the files required for WAL recovery, can be stored through the `duckdb_objfs://` URI scheme. The default backend stores SlateDB objects on the local filesystem: ```sql SET duckdb_objfs_root = '/var/lib/duckdb-objfs'; ATTACH 'duckdb_objfs://analytics.db' AS analytics; ``` An in-memory backend is available for testing: ```sql SET duckdb_objfs_backend = 'memory'; ATTACH 'duckdb_objfs://temporary.db' AS temporary; ``` For S3 and S3-compatible services, credentials and endpoint options are resolved through DuckDB's Secret Manager: ```sql INSTALL cache_httpfs FROM community; LOAD cache_httpfs; CREATE SECRET duckdb_objfs_s3 ( TYPE S3, PROVIDER credential_chain, REGION 'us-east-1', SCOPE 's3://my-bucket/duckdb-data' ); SET duckdb_objfs_backend = 's3'; SET duckdb_objfs_bucket = 'my-bucket'; SET duckdb_objfs_root = 'duckdb-data'; ATTACH 'duckdb_objfs://analytics.db' AS analytics; ``` The extension supports configurable in-memory and persistent local SST caches, cache statistics, OpenDAL I/O statistics, read-only attachments, and multiple read-only processes alongside a single writer. extension_star_count: 93 extension_star_count_pretty: 93 extension_download_count: 794 extension_download_count_pretty: 794 image: '/images/community_extensions/social_preview/preview_community_extension_duckdb_object_storage.png' layout: community_extension_doc --- ### Compatibility Matrix This table covers all DuckDB releases that have not reached [end-of-life](https://duckdb.org/release_calendar), including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 | | :--- | :---: | :---: | :---: | :---: | :---: | | Linux AMD64 | ❌ | ❌ | ❌ | ✅ | ✅ | | Linux ARM64 | ❌ | ❌ | ❌ | ✅ | ✅ | | macOS AMD64 | ❌ | ❌ | ❌ | ✅ | ✅ | | macOS ARM64 | ❌ | ❌ | ❌ | ✅ | ✅ | | Windows AMD64 | ❌ | ❌ | ❌ | ❌ | ❌ | | Windows ARM64 | ❌ | ❌ | ❌ | ❌ | ❌ | | WebAssembly | ❌ | ❌ | ❌ | ❌ | ❌ | ### Installing and Loading ```sql INSTALL {{ page.extension.name }} FROM community; LOAD {{ page.extension.name }}; ``` {% if page.docs.hello_world %} ### Example ```sql {{ page.docs.hello_world }}``` {% endif %} {% if page.docs.extended_description %} ### About {{ page.extension.name }} {{ page.docs.extended_description }} {% endif %} ### Added Functions | function_name | function_type | description | comment | examples | |-------------------------------------|---------------|-------------|---------|----------| | duckdb_objfs_attach_mapped_database | pragma | NULL | NULL | | | duckdb_objfs_cache_stats | table | NULL | NULL | | | duckdb_objfs_io_stats | table | NULL | NULL | | ### Overloaded Functions This extension does not add any function overloads. ### Added Types This extension does not add any types. ### Added Settings | name | description | input_type | scope | aliases | typed_value | |---------------------------------------------|------------------------------------------------------------|------------|--------|---------|-------------| | duckdb_objfs_backend | Storage backend: s3, local, or memory | VARCHAR | GLOBAL | [] | local | | duckdb_objfs_bucket | S3 bucket used by the DuckDB object filesystem | VARCHAR | GLOBAL | [] | NULL | | duckdb_objfs_cache_shards | In-memory cache shard count; zero selects the CPU count | UBIGINT | GLOBAL | [] | 0 | | duckdb_objfs_memory_cache_size | Foyer data-block cache capacity in bytes | UBIGINT | GLOBAL | [] | 536870912 | | duckdb_objfs_metadata_cache_size | Foyer SST metadata cache capacity in bytes | UBIGINT | GLOBAL | [] | 134217728 | | duckdb_objfs_persistent_cache_on_compaction | Populate the persistent cache from compaction output | BOOLEAN | GLOBAL | [] | false | | duckdb_objfs_persistent_cache_on_flush | Populate the persistent cache from memtable flush output | BOOLEAN | GLOBAL | [] | false | | duckdb_objfs_persistent_cache_part_size | Persistent SST cache part size in bytes | UBIGINT | GLOBAL | [] | 4194304 | | duckdb_objfs_persistent_cache_path | Local path for the persistent SST cache; empty disables it | VARCHAR | GLOBAL | [] | | | duckdb_objfs_persistent_cache_size | Persistent SST cache capacity in bytes | UBIGINT | GLOBAL | [] | 17179869184 | | duckdb_objfs_root | Local directory or S3 object prefix used by the filesystem | VARCHAR | GLOBAL | [] | NULL | | temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |