---
layout: docu
title: Logging
---

DuckDB implements a logging mechanism that provides users with detailed information about events such as query execution,
performance metrics and system events.

## Basics

The DuckDB logging mechanism can be enabled or disabled using a special function, `enable_logging`. Logs are stored in a special view
named `duckdb_logs`, which can be queried like any standard table.

Example:

```sql
CALL enable_logging();
-- Run some queries...
SELECT * FROM duckdb_logs;
```

To disable logging, run

```sql
CALL disable_logging();
```

To clear the current log, run

```sql
CALL truncate_duckdb_logs();
```

## Log Level

DuckDB supports different logging levels that control the verbosity of the logs:

* `ERROR`: Only logs error messages
* `WARN`: Logs warnings and errors
* `INFO`: Logs general information, warnings and errors (default)
* `DEBUG`: Logs detailed debugging information
* `TRACE`: Logs very detailed tracing information

The log level can be set using:

```sql
CALL enable_logging(level = 'debug');
```

## Warnings

DuckDB emits warnings, for example when you use deprecated syntax or a deprecated setting, through the logging system at the `WARN` level. In the CLI, warnings are printed by default. To capture them from any client, or to inspect them after the fact, enable logging and query the `duckdb_logs` view, filtering on the log level:

```sql
CALL enable_logging(level = 'warn');
-- Run some queries that may emit warnings...
SELECT timestamp, message
FROM duckdb_logs
WHERE level = 'WARN';
```

The default log level is `INFO`, which already includes warnings, so calling `enable_logging()` without a `level` also captures warnings. Setting `level = 'warn'` excludes lower-severity `INFO` messages.

## Log Types

In DuckDB, log messages can have an associated log type. Log types allow two main things:

* Fine-grained control over log message generation
* Support for structured logging

### Logging-Specific Types

To log only messages of a specific type:

```sql
CALL enable_logging('HTTP');
```

The above function will automatically set the correct log level and will add the `HTTP` type to the `enabled_log_types` settings. This ensures
only log messages of the 'HTTP' type will be written to the log.

To enable multiple log types, simply pass:

```sql
CALL enable_logging(['HTTP', 'QueryLog']);
```

### Structured Logging

Some log types like `HTTP` will have an associated message schema. To make DuckDB automatically parse the message, use the `duckdb_logs_parsed()` macro. For example:

```sql
SELECT request.headers FROM duckdb_logs_parsed('HTTP');
```

To view the schema of each structure log type simply run:

```sql
DESCRIBE FROM duckdb_logs_parsed('HTTP');
```

### List of Available Log Types

This is a (non-exhaustive) list of the available log types in DuckDB.

| Log Type           | Description                                              | Structured |
|--------------------|----------------------------------------------------------|------------|
| `QueryLog`         | Logs which queries are executed in DuckDB                | No         |
| `FileSystem`       | Logs all FileSystem interaction with DuckDB's Filesystem | Yes        |
| `HTTP`             | Logs all HTTP traffic from DuckDB's internal HTTP client | Yes        |
| `PhysicalOperator` | Logs events emitted by physical operators during query execution | Yes |
| `Metrics`          | Logs profiling metrics collected during query execution  | Yes        |

The structured log types expose the following schemas, which you can inspect at any time with
`DESCRIBE FROM duckdb_logs_parsed(⟨log_type⟩)`{:.language-sql .highlight}:

<div class="monospace_table"></div>

| Log Type           | Schema |
|--------------------|--------|
| `FileSystem`       | `fs VARCHAR`, `path VARCHAR`, `op VARCHAR`, `bytes BIGINT`, `pos BIGINT` |
| `HTTP`             | `request STRUCT(type, url, start_time, duration_ms, headers MAP)`, `response STRUCT(status, reason, headers MAP)` |
| `PhysicalOperator` | `operator_type VARCHAR`, `parameters MAP(VARCHAR, VARCHAR)`, `class VARCHAR`, `event VARCHAR`, `info MAP(VARCHAR, VARCHAR)` |
| `Metrics`          | `metric VARCHAR`, `value VARCHAR` |

## Log Storages

By default, DuckDB logs to an in-memory log storage (`memory`). DuckDB supports different types of log storage. Currently,
the following log storage types are implemented in core DuckDB.

| Log storage | Description                                               |
|-------------|-----------------------------------------------------------|
| `memory`    | (default) Log to an in-memory buffer                      |
| `stdout`    | Log to the stdout of the current process (in CSV format)  |
| `file`      | Log to (a) csv file(s)                                    |


Note that the `duckdb_logs` view is automatically updated to target the currently active log storage. This means that switching
the log storage may influence what is returned by the `duckdb_logs` function.

### Logging to stdout

```sql
CALL enable_logging(storage = 'stdout');
```

### Logging to File 

```sql
CALL enable_logging(storage = 'file', storage_config = {'path': 'path/to/store/logs'});
```

or using the equivalent shorthand:

```sql
CALL enable_logging(storage_path = 'path/to/store/logs');
```

## Advanced Usage

### Normalized vs. Denormalized Logging

DuckDB's log storages can log in two ways: normalized vs. denormalized.

In denormalized logging, the log context information is appended directly to each log entry, while in normalized logging
the log entries are stored separately with context_ids referencing the context information.

| Log storage | Normalized   |
|-------------|--------------|
| `memory`    | yes          |
| `file`      | configurable |
| `stdout`    | no           |

For file storage, you can switch between normalized and denormalized by providing a path ending in `.csv` (for normalized)
or without `.csv `(for denormalized). For file logging, denormalized is generally recommended since this increases performance 
and reduces the total size of the logs. To configure normalization of `file` log storage:

```sql
-- normalized: creates `/tmp/duckdb_log_contexts.csv` and `/tmp/duckdb_log_entries.csv`
CALL enable_logging(storage_path = '/tmp');
-- denormalized: creates `/tmp/logs.csv`
CALL enable_logging(storage_path = '/tmp/logs.csv');
```

Note that the difference between normalized and denormalized is typically hidden from users through the 'duckdb_logs' function,
which automatically joins normalized tables into a single unified result. To illustrate, both configurations above will be
queryable using `FROM duckdb_logs;` and will produce identical results.

### Buffer Size

The log storage in DuckDB implements a buffering mechanism to optimize logging performance. This implementation
introduces a potential delay between message logging and storage writing. This delay can obscure the actual message writing time,
which is particularly problematic when debugging crashes, as messages generated immediately before a crash might not be
written. To address this, the buffer size can be configured as follows:

```sql
CALL enable_logging(storage_config = {'buffer_size': 0});
```

or using the equivalent shorthand:

```sql
CALL enable_logging(storage_buffer_size = 0);
```

Note that the default buffer size is different for different log storages:

| Log storage | Default buffer size           |
|-------------|-------------------------------|
| `memory`    | `STANDARD_VECTOR_SIZE` (2048) |
| `file`      | `STANDARD_VECTOR_SIZE` (2048) |
| `stdout`    | Disabled (0)                  |

So for example, if you want to increase your `stdout` logging performance, simply enable buffering to greatly (>10x) speed up 
your logging:

```sql
CALL enable_logging(storage = 'stdout', storage_buffer_size = 2048);
```

Or imagine you are debugging a crash in DuckDB and you want to use the `file` logger to understand what's going on.
Simply disable the buffering using:

```sql
CALL enable_logging(storage_path = '/tmp/mylogs', storage_buffer_size = 0);
```

### Syntactic Sugar

DuckDB contains some syntactic sugar to make common paths easier. For example, the following statements are all equal:

```sql
-- regular invocation 
CALL enable_logging(storage = 'file', storage_config = {'path': 'path/to/store/logs'});
-- using shorthand for common path storage config param 
CALL enable_logging(storage = 'file', storage_path = 'path/to/store/logs');
-- omitting `storage = 'file'` -> is implied from presence of `storage_config`
CALL enable_logging(storage_config = {'path': 'path/to/store/logs'});
```