### Watch all release presentations and videos at [ClickHouse Theater](https://presentations.clickhouse.com/) and [YouTube Playlist](https://www.youtube.com/playlist?list=PL0Z2YDlm0b3jAlSy1JxyP8zluvXaN3nxU). ### Table of Contents **[ClickHouse release v26.10, 2026-10-22](#2610)**
**[ClickHouse release v26.9, 2026-09-21](#269)**
**[ClickHouse release v26.8 LTS, 2026-08-27](#268)**
**[ClickHouse release v26.7, 2026-07-22](#267)**
**[ClickHouse release v26.6, 2026-06-25](#266)**
**[ClickHouse release v26.5, 2026-05-21](#265)**
**[ClickHouse release v26.4, 2026-04-30](#264)**
**[ClickHouse release v26.3 LTS, 2026-03-26](#263)**
**[ClickHouse release v26.2, 2026-02-26](#262)**
**[ClickHouse release v26.1, 2026-01-29](#261)**
**[Changelog for 2025](https://clickhouse.com/docs/whats-new/changelog/2025/)**
**[Changelog for 2024](https://clickhouse.com/docs/whats-new/changelog/2024/)**
**[Changelog for 2023](https://clickhouse.com/docs/whats-new/changelog/2023/)**
**[Changelog for 2022](https://clickhouse.com/docs/whats-new/changelog/2022/)**
**[Changelog for 2021](https://clickhouse.com/docs/whats-new/changelog/2021/)**
**[Changelog for 2020](https://clickhouse.com/docs/whats-new/changelog/2020/)**
**[Changelog for 2019](https://clickhouse.com/docs/whats-new/changelog/2019/)**
**[Changelog for 2018](https://clickhouse.com/docs/whats-new/changelog/2018/)**
**[Changelog for 2017](https://clickhouse.com/docs/whats-new/changelog/2017/)**
# 2026 Changelog ### ClickHouse release 26.10, 2026-10-22 #### Backward Incompatible Change * The query analysis that preceded the analyzer is removed. It has not been selectable since 26.9, when `enable_analyzer` became obsolete; now a server no longer uses it for a query another server sends either, so a cluster whose initiator is older than 26.10 gets the analyzer on this server regardless of what the initiator asks for. Such a mixed cluster can see `NOT_FOUND_COLUMN_IN_BLOCK` on the old initiator, because the two analyses name the result columns differently; set `enable_analyzer = 1` on every server before upgrading any of them. The `use_analyzer_for_mutations` server setting is removed, and `parallel_replicas_only_with_analyzer` is obsolete. [#120311](https://github.com/ClickHouse/ClickHouse/pull/120311) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The predicate push-down that the query analysis preceding the analyzer performed on the AST is removed, and `enable_optimize_predicate_expression` is now obsolete and has no effect. Push-down of a predicate into a subquery is performed by the analyzer as a query plan optimization and is not affected; push-down into a distributed subquery still happens on the AST level and is controlled by `allow_push_predicate_ast_for_distributed_subqueries`. The visible difference is that `EXPLAIN SYNTAX` of a query that is not a `SELECT`, for example `EXPLAIN SYNTAX INSERT INTO ... SELECT ...`, no longer prints a predicate copied into a subquery, a non-aggregate `HAVING` conjunct moved to `WHERE`, or the right-hand table of a `JOIN` wrapped into a subquery. [#121465](https://github.com/ClickHouse/ClickHouse/pull/121465) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `system.metric_log` now uses the `bucketed` schema by default: all profile events and current metrics are stored in a single `metrics` column of type `Map(Enum16(...), Int64)` with the bucketed `Map` serialization and a constant number of 128 buckets, so the table has a few columns instead of thousands, zero values are not stored, and reading one metric reads only one of the 128 buckets. Every metric is also exposed as an `ALIAS` column, so queries such as `SELECT ProfileEvent_Query FROM system.metric_log` continue to work unchanged. On upgrade the existing table is renamed to `metric_log_0`, as with any other change of the structure of a system log table, and the built-in dashboards read `merge('system', '^metric_log')`, so they keep showing the data collected before the upgrade. Add `wide` to the `` section of the server configuration to keep the previous schema. [#118318](https://github.com/ClickHouse/ClickHouse/pull/118318) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124016](https://github.com/ClickHouse/ClickHouse/pull/124016) ([Raúl Marín](https://github.com/Algunenano)). * `clickhouse-server` can now apply a `seccomp` system call filter at startup. The `seccomp` setting controls refused system calls: `log` only records them, while `trap`, `kill`, and `errno` enforce the policy; `disabled` installs no filter. The `config.xml` shipped with the server packages and the Docker image now sets `seccomp` to `trap`, so a server that runs in a sandbox which refuses the `seccomp` system call must switch it to `log` or `disabled` before the upgrade. Extended-attribute probes used by some hostname-resolution libraries are rejected with `ENOSYS` instead of the configured seccomp action, so server startup still succeeds under the default `trap` mode. You might need to disable seccomp when using executable user-defined functions. [#119545](https://github.com/ClickHouse/ClickHouse/pull/119545) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122577](https://github.com/ClickHouse/ClickHouse/pull/122577) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124856](https://github.com/ClickHouse/ClickHouse/pull/124856) ([Azat Khuzhin](https://github.com/azat)). * Settings constraints and `allow_feature_tier` are now enforced for users, roles and settings profiles defined by SQL. A statement is refused if it changes a restricted setting of some user indirectly (such as `GRANT`, `DROP ROLE`, `DROP SETTINGS PROFILE` or `CREATE USER OR REPLACE`), sets a value outside its author's constraints, or removes a setting its author is constrained on. A user defined by SQL whose stored settings, roles or assigned settings profiles violate the constraints or `allow_feature_tier` can no longer log in. The `compatibility` setting no longer applies values that settings constraints or `allow_feature_tier` refuse; such settings keep their current value. Login checks use the user's resolved constraints, preserving bounds inherited from the default profile. Closes [#124966](https://github.com/ClickHouse/ClickHouse/issues/124966). [#119472](https://github.com/ClickHouse/ClickHouse/pull/119472) ([Raúl Marín](https://github.com/Algunenano)). [#125582](https://github.com/ClickHouse/ClickHouse/pull/125582) ([Raúl Marín](https://github.com/Algunenano)). * Overriding a key that is stored in a named collection now requires `SHOW NAMED COLLECTIONS SECRETS` on that collection. This also applies to overrides through an alias (including flat `kafka_*` settings over nested `kafka.*` keys), through a `SETTINGS` clause of a table engine or table function, and through keys in a dictionary source (checked when the dictionary is created). Resolving a relative `url` stored in a named collection against `s3_base` or `url_base` also counts as an override of `url`. Adding a key that the collection does not store remains allowed, and keys marked `NOT OVERRIDABLE` cannot be overridden. The `allow_named_collection_override_by_default` setting is obsolete. Existing tables whose `SETTINGS` or alias arguments override a `NOT OVERRIDABLE` key cannot be loaded after the upgrade, and the server does not start: before upgrading, recreate such tables without the override, or make the key overridable. [#123133](https://github.com/ClickHouse/ClickHouse/pull/123133) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * `max_bytes_before_external_join` and `max_bytes_ratio_before_external_join` are now the spill threshold for every hash-based `join_algorithm`, including `grace_hash`, which previously ignored them and spilled on `max_rows_in_join` / `max_bytes_in_join` instead. Those two settings are a hard cap again for every algorithm: they no longer cause spilling, and a join that spills no longer exceeds them. All join algorithms remain available; `join_algorithm = 'grace_hash'` now needs a non-zero spill threshold, and is skipped in favour of the next algorithm in the list when it has none. Set `legacy_join_size_limits_trigger_spilling = 1` (or `compatibility` to 26.8 or older) to restore the previous behavior. [#117356](https://github.com/ClickHouse/ClickHouse/pull/117356) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#119335](https://github.com/ClickHouse/ClickHouse/pull/119335) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `arrayCount` now returns `UInt64` instead of `UInt32`, so it stays exact for arrays with more than 4294967295 matching elements. Set `array_count_legacy_uint32_result = 1` (or `compatibility = '26.9'`) to restore the previous result type. During a rolling upgrade, enable `array_count_legacy_uint32_result` on upgraded servers for the users under which shard-side queries execute, so distributed queries initiated by not-yet-upgraded servers keep the previous behavior on upgraded shards; remove it after the upgrade is complete. In addition, `length(arrayFilter(func, arr))` is now rewritten to `arrayCount(func, arr)`, which is up to 1.29x faster on long arrays, controlled by `optimize_rewrite_array_filter_length_to_array_count`. [#113023](https://github.com/ClickHouse/ClickHouse/pull/113023) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `accurateCast` from `String` or `FixedString` to a `Nullable` target now throws the error its non-`Nullable` target already raised (`CANNOT_PARSE_TEXT`, `TOO_LARGE_STRING_SIZE`, ...) instead of returning `NULL`, and no longer throws for a row the source marked `NULL`. This affects every use of `accurateCast`: `DeltaLake` writes stop committing an unrepresentable value as `NULL`, `dictGetOrDefault` rejects a default its `Nullable` attribute cannot hold and now converts only the defaults it uses, `IN` under `transform_null_in = 1` no longer matches an unparsable key against a `NULL` in the set, and `cast_ipv4_ipv6_default_on_conversion_error` yields the type's default for an unparsable address. `CAST`, `accurateCastOrNull`, `accurateCastOrDefault`, and ordinary `INSERT` are unchanged, as is what a non-text source does with a value it cannot represent; use `accurateCastOrNull` for the previous behavior. Closes [#120524](https://github.com/ClickHouse/ClickHouse/issues/120524). [#120674](https://github.com/ClickHouse/ClickHouse/pull/120674) ([Groene AI](https://github.com/groeneai)). * `SELECT ... FINAL` from `SummingMergeTree` no longer removes the rows whose summed columns are all zero, unlike background merges. Before, this removal looked only at the columns the query read, so results depended on them: for example, `SELECT count() FROM table FINAL` could return fewer rows than `SELECT * FROM table FINAL`. Background merges and `OPTIMIZE TABLE ... FINAL` still remove such rows, so filter out the all-zero rows explicitly if a query relied on their removal. [#115032](https://github.com/ClickHouse/ClickHouse/pull/115032) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A read of a `Distributed` or `Remote` table with `skip_unavailable_shards = 1` now fails with `ALL_CONNECTION_TRIES_FAILED` when every shard was skipped before returning any data, instead of returning an empty result, matching what already happens when the only shard is local. Previously an aggregate without `GROUP BY` silently returned zero rows instead of one. The error fails the whole query, so a query that combines such a read with other data, for example a `UNION ALL` with local rows, no longer returns those rows. When a shard is itself a `Distributed` table whose shards were all skipped, the outer query now fails even if another shard answered; `skip_unavailable_shards_mode = 'unavailable_or_exception_before_processing'` keeps the partial result in that case. No setting restores the previous empty result. Closes [#115646](https://github.com/ClickHouse/ClickHouse/issues/115646). [#115775](https://github.com/ClickHouse/ClickHouse/pull/115775) ([Groene AI](https://github.com/groeneai)). * A `MATERIALIZED` column computed from a column with a `TTL` is now recomputed when the `TTL` resets that column, both during a merge and in `ALTER TABLE ... MATERIALIZE TTL`. Previously it kept the pre-expiry value forever. If the dependent expression is not defined on the expired column's default (for example `intDiv(100, x)` when `x` expires to `0`), the `TTL` merge now throws instead of writing the stale value; make the expression total on the default (`intDivOrZero`, `if(x = 0, ...)`) if such a table must keep merging. Closes [#117310](https://github.com/ClickHouse/ClickHouse/issues/117310). [#117815](https://github.com/ClickHouse/ClickHouse/pull/117815) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * With `input_format_null_as_default = 0`, a JSON `null` for a non-`Nullable` numeric column now raises an exception instead of being silently stored as `0` (or `nan`), consistent with other types and formats. Keep `input_format_null_as_default` at its default `1` to insert the column default instead. [#122137](https://github.com/ClickHouse/ClickHouse/pull/122137) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... ADD COLUMN` and `MODIFY COLUMN` now apply the `data_type_default_nullable` setting the same way `CREATE TABLE` does: with the setting enabled, a column type written without an explicit `NULL` or `NOT NULL` modifier becomes `Nullable`, so the rows that existed before `ADD COLUMN` read `NULL`, `ADD COLUMN x Int32 DEFAULT NULL` is accepted, and `MODIFY COLUMN x Int64` keeps a `Nullable` column `Nullable` instead of failing. As in `CREATE TABLE`, an `Array`, `Map`, `LowCardinality`, or another type that cannot be inside `Nullable` then requires `NOT NULL`, and so does a `MODIFY COLUMN` of a sorting key column. To keep the previous result of such an `ALTER`, write `NOT NULL` explicitly or run it with `data_type_default_nullable = 0`. Closes [#124504](https://github.com/ClickHouse/ClickHouse/issues/124504). [#124597](https://github.com/ClickHouse/ClickHouse/pull/124597) ([Groene AI](https://github.com/groeneai)). * `arrayJoin` is now lowered to an `ARRAY JOIN` step by default (setting `query_plan_lower_array_join_function`). The expansion streams through the pipeline instead of being materialized inside the projection, filters on the joined elements are applied before the rows are multiplied, and queries with several `arrayJoin` calls no longer build the whole cross product in memory. As a consequence, a non-deterministic function such as `rand` or `generateUUIDv4` next to `arrayJoin` in the same `SELECT` is now evaluated once per output row, as with the `ARRAY JOIN` clause, instead of once per source row. Set `arrayjoin_nondeterministic_functions_before_expansion = 1` (or `compatibility` to 26.9 or older) to keep the previous behavior. [#117882](https://github.com/ClickHouse/ClickHouse/pull/117882) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#120390](https://github.com/ClickHouse/ClickHouse/pull/120390) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#122240](https://github.com/ClickHouse/ClickHouse/pull/122240) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * A function that depends on blocks or row order, such as `runningDifference`, `neighbor` or `rowNumberInBlock`, next to the `arrayJoin` function now sees the blocks of the expansion, like with the `ARRAY JOIN` clause. Set `legacy_array_join_function_nondeterministic_evaluation = 1` to get the previous behavior. [#123143](https://github.com/ClickHouse/ClickHouse/pull/123143) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * A row policy on a `Distributed` table (or another table that processes the query itself) read through a `Merge` table now makes queries to the `Merge` table fail with `ILLEGAL_PREWHERE` for the users the policy applies to, like queries to that table itself. Before, the `Merge` table applied such a policy after the table had already sorted, limited and computed `extremes`, so queries could return wrong rows and show values of hidden rows in `extremes`. Define the row policy on the underlying tables of the remote servers and on the local child tables instead. This also fixes `NOT_FOUND_COLUMN_IN_BLOCK` on every query over a `Merge` table whose child has a row policy when the `Merge` table also reads a `Distributed` table. [#125765](https://github.com/ClickHouse/ClickHouse/pull/125765) ([Groene AI](https://github.com/groeneai)). * `clickhouse-local --ignore-error` now reports a statement that failed at execution instead of skipping it silently: previously it printed no message and no exception code, so a failing multi-statement script looked exactly like a passing one, while `clickhouse-client --ignore-error` reported every error all along. `clickhouse-client` also ignored the option entirely when it was combined with `--interactive` and `--query`/`--queries-file` on a terminal, and now applies it to those queries too. The exit code of `clickhouse-client --ignore-error` changes accordingly: it used to be the code of whichever error the last statement of the batch happened to hit, which made the result depend on the position of the failing statement rather than on the batch as a whole, and it is now a success code, matching what `clickhouse-local --ignore-error` has always returned. Scripts that relied on the old exit code to detect failures should drop `--ignore-error` so that the run stops and fails at the first error, or check the reported errors on `stderr`. [#119572](https://github.com/ClickHouse/ClickHouse/pull/119572) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * `ATTACH TABLE ... FROM ''` now requires `READ ON FILE` and `WRITE ON FILE` on the source directory in `user_files`, in addition to `CREATE TABLE`, because the query reads the directory and then moves it out of `user_files` on success. Previously a user could attach and read staged data without those `FILE` grants. Grant the corresponding `FILE` privileges to users that rely on this form. [#120582](https://github.com/ClickHouse/ClickHouse/pull/120582) ([Alexander Gololobov](https://github.com/davenger)). * A `MySQL` dictionary source created with a `CREATE DICTIONARY` query can no longer enable `enable_local_infile`, at the source or at one of its `REPLICA` entries; loading such a dictionary fails with `BAD_ARGUMENTS`, which with the default `dictionaries_lazy_load` is on first use rather than at `CREATE`. A `REPLICA` that does not set the key inherits the source's value, as the connection pool does. To keep the option, define the dictionary in a server configuration file; `enable_local_infile = 0`, and a `SOURCE(MYSQL(NAME ...))` reference to a named collection where the key was never read, are unaffected. [#120972](https://github.com/ClickHouse/ClickHouse/pull/120972) ([Groene AI](https://github.com/groeneai)). * The definitions of disks in `storage_configuration` and in `disk` are validated: an element that nothing reads is reported as `UNKNOWN_ELEMENT_IN_CONFIG` instead of being silently ignored, which finds typos and options that do not exist for this disk type, such as `path` in an `S3` disk. If a server does not start after the upgrade because of an element it does not know, remove that element from the configuration, or set `1`. [#118440](https://github.com/ClickHouse/ClickHouse/pull/118440) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `system.*_log` tables can now be configured uniformly via `system_tables.*`. In `system.opentelemetry_span_log`, `finish_date` is renamed to `event_date` and kept as an alias. Aliases are now supported in configured `PARTITION BY`, `ORDER BY`, `TTL`, `PRIMARY KEY`, and `SAMPLE BY` expressions, so custom definitions can continue to use `finish_date`. Some tables now fall back to the default TTL when none is configured, so set `ttl` explicitly if that matters. [#121563](https://github.com/ClickHouse/ClickHouse/pull/121563) ([Azat Khuzhin](https://github.com/azat)). [#125564](https://github.com/ClickHouse/ClickHouse/pull/125564) ([Azat Khuzhin](https://github.com/azat)). * The setting `validate_mutation_query` is deprecated and ignored: mutation queries (`ALTER TABLE ... UPDATE`, `ALTER TABLE ... DELETE`, `DELETE FROM`) are now always validated before being accepted. It was enabled by default since 24.11, and disabling it did not avoid the failure but only deferred it to the background, where the mutation got stuck and required manual intervention. Setting it to `0` no longer lets an invalid mutation be accepted: it is rejected when submitted. [#121419](https://github.com/ClickHouse/ClickHouse/pull/121419) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Settings `query_plan_read_in_order`, `query_plan_optimize_prewhere`, and `query_plan_aggregation_in_order` are obsolete and no longer have any effect. Use `optimize_read_in_order`, `optimize_aggregation_in_order`, and `optimize_move_to_prewhere` instead. [#121390](https://github.com/ClickHouse/ClickHouse/pull/121390) ([Anton Popov](https://github.com/CurtizJ)). * The setting `use_compact_format_in_distributed_parts_names` is obsolete. The non-compact directory format for the async `INSERT` queue of a `Distributed` table has been removed. The setting was `1` by default since version 21.1. Before upgrading, set it to `1` and let the queue drain. [#120186](https://github.com/ClickHouse/ClickHouse/pull/120186) ([Alexander Tokmakov](https://github.com/tavplubix)). * The `MergeTree` settings `use_minimalistic_part_header_in_zookeeper` and `use_minimalistic_checksums_in_zookeeper` are obsolete and have no effect: new data parts of `ReplicatedMergeTree` tables are always stored in ZooKeeper as a single node with the compact header, the default since 20.1. An explicit `0` in table metadata or in the `merge_tree` configuration is ignored, and such tables and configurations still load. Part nodes written in the old format by earlier versions are still read. Closes [#122744](https://github.com/ClickHouse/ClickHouse/issues/122744). [#122790](https://github.com/ClickHouse/ClickHouse/pull/122790) ([Groene AI](https://github.com/groeneai)). * Settings `s3_max_connections` and the `S3` disk's `max_connections` key are obsolete. They have had no effect since 21.4, when the per-endpoint connection pool of the `S3` client they bounded was removed, but were still documented as limiting the number of connections per server. Both are still accepted and do nothing. To bound the connections to object storage, use the server settings `disk_connections_soft_limit`, `disk_connections_store_limit`, `disk_connections_warn_limit`, and `disk_connections_hard_limit` and their `storage_connections_*` and `http_connections_*` counterparts; to bound the request rate, use `s3_max_get_rps` and `s3_max_put_rps`. [#119550](https://github.com/ClickHouse/ClickHouse/pull/119550) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the zero-argument `year` function. Queries that used `year()` must switch to `toYear(today())` instead. [#120268](https://github.com/ClickHouse/ClickHouse/pull/120268) ([Robert Schulze](https://github.com/rschu1ze)). * Removed the `system.codecs.is_experimental` column. Queries that filtered or selected it should use `tier = 'Experimental'` instead. [#118523](https://github.com/ClickHouse/ClickHouse/pull/118523) ([Raufs Dunamalijevs](https://github.com/rienath)). * Removed the experimental function `detectTonality`. Queries that use it now fail with `UNKNOWN_FUNCTION`. [#123327](https://github.com/ClickHouse/ClickHouse/pull/123327) ([Groene AI](https://github.com/groeneai)). * Update the `zxc` library to v0.14.0, which changes the compressed block format of the experimental `ZXC` codec. Data written with `CODEC(ZXC)` by earlier versions cannot be read by the new version and fails with `CANNOT_DECOMPRESS`. If you used the experimental `ZXC` codec, re-encode the affected columns to a different codec before upgrading, or re-insert the data after the upgrade. [#119097](https://github.com/ClickHouse/ClickHouse/pull/119097) ([Actuele AI](https://github.com/actueleai)). * An aggregate function state that serializes to zero bytes cannot be read back, so the two parameterizations that produced one are now rejected: a `-Resample` combinator over an empty range (`begin >= end`, such as `countResample(10, 5, 1)`) throws `ARGUMENT_OUT_OF_BOUND`, and `covarSampMatrix`, `covarPopMatrix`, and `corrMatrix` with no arguments throw `NUMBER_OF_ARGUMENTS_DOESNT_MATCH`. Both returned an empty array before. No serialized format changes, but a table with such a column no longer loads. Before upgrading, drop the column where the engine supports `DROP COLUMN`, and the whole table where it does not (`Log`, `TinyLog`, `StripeLog`, `Set`, `Join`, `File`). [#119728](https://github.com/ClickHouse/ClickHouse/pull/119728) ([Groene AI](https://github.com/groeneai)). * A data type naming the state of a function that only works as a window function, such as `AggregateFunction(rank)`, is now rejected with `BAD_ARGUMENTS`: such a state has no serialization, so the column could never hold a value. This applies wherever a type is named, including inside `Array`, `Map`, `Tuple`, and another state's argument list. A table that already stores such a column will no longer attach after the upgrade; drop or rewrite the column before upgrading. [#122148](https://github.com/ClickHouse/ClickHouse/pull/122148) ([Groene AI](https://github.com/groeneai)). * `rankState` and every other `-State` of a function that only works as a window function, such as `denseRankStateIf` or `initializeAggregation('lagInFrameState', x)`, are now rejected with `BAD_ARGUMENTS` when the query is analyzed, also where the state is never computed (`toTypeName`, `EXPLAIN`, a column the outer query drops), and so is such a type in a binary-encoded `Native` header. Computing such a state always failed, but a view or `CREATE TABLE ... AS SELECT` could still store its type. A table whose engine expressions build such a state, for example in a sorting key, partition key, skip index, or `TTL`, no longer attaches. [#122463](https://github.com/ClickHouse/ClickHouse/pull/122463) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE` and full-definition `ATTACH TABLE` now refuse a `CONSTRAINT` name declared more than once, and `ALTER TABLE ... ADD CONSTRAINT` / `MODIFY CONSTRAINT` now refuse an expression containing `arrayJoin` even when the command would install nothing. A table already stored with a repeated constraint name still attaches and keeps enforcing each of its declarations; give each constraint a unique name in new definitions. [#121235](https://github.com/ClickHouse/ClickHouse/pull/121235) ([Groene AI](https://github.com/groeneai)). * The `Remote`, `RemoteSecure`, and `Cluster` database engines now reject an empty remote database name with `BAD_ARGUMENTS` instead of creating a database that lists and reads no tables. A server that holds such a database, created by an earlier version, does not start after the upgrade, so drop these databases before upgrading. Closes [#122616](https://github.com/ClickHouse/ClickHouse/issues/122616). [#122675](https://github.com/ClickHouse/ClickHouse/pull/122675) ([Groene AI](https://github.com/groeneai)). * Connections to MySQL (the `mysql` table function, the `MySQL` table and database engines, and `MySQL` dictionaries) that set `ssl_ca` or `ssl_ca_pem` now verify the server certificate chain and host name, after updating `mariadb-connector-c` to v3.4.10. Previously the CA was not enforced, so a connection to a server whose certificate is signed by another CA, or whose certificate does not list the host name or IP address used to connect, succeeded and now fails with `self-signed certificate in certificate chain` or `Hostname verification failed`. Before upgrading, make sure the configured CA signs the MySQL server certificate and that the certificate lists the host name or IP address ClickHouse connects to. [#119510](https://github.com/ClickHouse/ClickHouse/pull/119510) ([Actuele AI](https://github.com/actueleai)). #### New Feature * Sets built for `IN` with a subquery or a table now spill to disk by default once query memory usage exceeds half the available memory under the applicable server or user memory limit, calculated at the start of execution (`max_bytes_ratio_before_external_set = 0.5`). The `max_bytes_before_external_set` setting can set an absolute threshold (`max_memory_usage` is not taken into account by the ratio, so use it to spill before that limit is reached), and setting both thresholds to `0` keeps sets in memory. This feature is now generally available. [#123860](https://github.com/ClickHouse/ClickHouse/pull/123860) ([Nihal Z. Miaji](https://github.com/nihalzp)). [#124907](https://github.com/ClickHouse/ClickHouse/pull/124907) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Support parts of the Trino SQL dialect in regular ClickHouse SQL: `TRY_CAST(x AS T)`, the `ARRAY[...]` array constructor, the type names `ARRAY(T)` and `MAP(K, V)`, and aliases for Trino functions such as `starts_with`, `slice`, `regexp_like`, `day_of_week`, `url_extract_host`, `bitwise_and`, `arbitrary`, and `bool_and`. [#124763](https://github.com/ClickHouse/ClickHouse/pull/124763) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implemented a generic block nested loop join algorithm. `JOIN` now supports an arbitrary `ON` condition with no equality between the two tables, executing it as a block nested loop join, as slow as a `CROSS JOIN` with a filter, instead of failing with `INVALID_JOIN_ON_EXPRESSION`. The new setting `allow_block_nested_loop_join` turns it on/off. [#123642](https://github.com/ClickHouse/ClickHouse/pull/123642) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123792](https://github.com/ClickHouse/ClickHouse/pull/123792) ([Groene AI](https://github.com/groeneai)). * Added the function-composition operator `f | g` and the argument placeholders `_`, `_1`, `_2`, ...: for example, `arrayMap(plus(_, 1) | multiply(_, 2), [1, 2, 3])` returns `[4, 6, 8]`. A composition can be used wherever a function is expected, including as an argument of a higher-order function. The operands can be lambdas, function names, nested compositions, or expressions with placeholders, which also work on their own, for example `arrayMap(_1 * 2 + 1, [1, 2, 3])`. The composition is resolved at query analysis time into an ordinary lambda. Note that `|` here is not bitwise OR; use `bitOr` for that. [#116029](https://github.com/ClickHouse/ClickHouse/pull/116029) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The query cache can now store its entries on disk, backed by a preconfigured filesystem cache: this gives the entries more space, makes them survive server restarts, and, with a distributed filesystem cache, provides a way to share them between servers. Select the filesystem cache with the new setting `query_cache_on_disk_cache_name`; see also the new settings `query_cache_on_disk_codec`, `enable_writes_to_query_cache_on_disk`, and `enable_reads_from_query_cache_on_disk`. Closes [#115361](https://github.com/ClickHouse/ClickHouse/issues/115361). [#115382](https://github.com/ClickHouse/ClickHouse/pull/115382) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a notebook mode to the Web UI (`/play`). After a query is run, a `[+]` button under the result adds further cells, each either a query with its own result or a block of Markdown text, turning the tab into a notebook in the style of Jupyter or Observable. Cells can be run, reordered, inserted and deleted individually; the query editor collapses to a single line when it is not focused, results are bounded and scroll inside themselves, and the whole notebook is restored on reload. Notebook mode is per tab, so a tab that has only one cell looks and behaves exactly as before. [#115378](https://github.com/ClickHouse/ClickHouse/pull/115378) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added support for reading and writing data in the `SQLite` format. [#104510](https://github.com/ClickHouse/ClickHouse/pull/104510) ([Yue Ni](https://github.com/niyue)). * Added the `Cluster` database engine, which provides real-time access to the tables of a configured cluster database and forwards `SELECT` and `INSERT` queries to it. [#114417](https://github.com/ClickHouse/ClickHouse/pull/114417) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `Overlay` database engine: `CREATE DATABASE db ENGINE = Overlay(db1, db2, ...)` exposes the union of the tables of existing databases, each table working as an `Alias` table to the table of the first source database that has it. [#121878](https://github.com/ClickHouse/ClickHouse/pull/121878) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `CREATE DATABASE` and `ATTACH DATABASE` now accept a `SETTINGS` clause without an explicit `ENGINE` clause, defaulting to `Atomic`, so a database can define its metadata disk inline the same way a single table does. A single `SETTINGS` clause can now also mix database-engine settings with ordinary query settings. [#118423](https://github.com/ClickHouse/ClickHouse/pull/118423) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `MergeTree` tables in `Ordinary` databases can now be converted to `ReplicatedMergeTree` with `ATTACH TABLE ... AS REPLICATED` or the `convert_to_replicated` flag. [#118757](https://github.com/ClickHouse/ClickHouse/pull/118757) ([Kirill](https://github.com/kirillgarbar)). * Added query-aware scheduling in workloads: the new workload setting `scheduler` selects how queries of a workload are scheduled — `fifo` (default), `fair` (weighted fair queueing with per-query weights that can be lowered as a query runs), `las` (least-attained-service), or `priority` (by the query setting `workload_priority`). The leaf-node `type` in `system.scheduler` is now `request_queue` instead of `fifo`. [#116800](https://github.com/ClickHouse/ClickHouse/pull/116800) ([Sergei Trifonov](https://github.com/serxa)). * Queries can now spill to disk under memory pressure based on limits set per workload. The `max_memory_before_spill` and `max_memory_to_spill_ratio` options of `WORKLOAD` define a soft memory limit that triggers spilling for `ORDER BY`, `GROUP BY`, `IN`, `DISTINCT`, and `JOIN` with `grace_hash`. [#119443](https://github.com/ClickHouse/ClickHouse/pull/119443) ([Azat Khuzhin](https://github.com/azat)). * Quotas can now limit the number of rows written by `INSERT` queries with the new `written_rows` resource, for example `CREATE QUOTA q KEYED BY ip_address FOR INTERVAL 1 minute MAX written_rows = 100000, query_inserts = 250 TO user`. Until now only the written bytes could be limited (`written_bytes`), which is awkward when the natural unit of the workload is rows. The current and maximum values are exposed as `written_rows` and `max_written_rows` in `system.quota_usage`, `system.quotas_usage`, and `system.quota_limits`, and the resource can also be set in `users.xml` as ``. [#118272](https://github.com/ClickHouse/ClickHouse/pull/118272) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the settings `max_temporary_tables`, `max_temporary_table_memory_usage`, `max_temporary_table_size_bytes_compressed`, and `max_temporary_table_size_bytes_uncompressed` to limit the number of temporary tables in a session, the memory usage of `Memory` temporary tables, and the size of `MergeTree` temporary tables. [#122165](https://github.com/ClickHouse/ClickHouse/pull/122165) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the SQL-standard `SIMILAR TO` pattern-matching predicate. [#101514](https://github.com/ClickHouse/ClickHouse/pull/101514) ([Zhao Zheguang](https://github.com/zheguang)). * Allow `MATERIALIZED` common table expressions inside recursive `WITH`. A materialized helper CTE is evaluated once and every recursive step reads the snapshot, instead of the subquery being re-evaluated on each iteration. The recursive CTE itself still cannot be materialized. [#123452](https://github.com/ClickHouse/ClickHouse/pull/123452) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Query parameters can now be used for primary `USER` and `ROLE` names in `CREATE`, `ALTER`, and `DROP`, and for role and grantee names in `GRANT` and `REVOKE`, for example `GRANT {role:Identifier} TO {user:Identifier}`. [#109696](https://github.com/ClickHouse/ClickHouse/pull/109696) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)). * `SET DEFAULT ROLE` now accepts an `ON CLUSTER` clause, in either position: `SET DEFAULT ROLE role1 TO user ON CLUSTER my_cluster` and `SET DEFAULT ROLE ON CLUSTER my_cluster role1 TO user`. `CURRENT_USER` is resolved on the initiator before the query is distributed. `SET ROLE` only affects the current session and still takes no `ON CLUSTER` clause. [#118429](https://github.com/ClickHouse/ClickHouse/pull/118429) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `CREATE OR REPLACE NAMED COLLECTION ...` to replace an existing named collection entirely. [#121569](https://github.com/ClickHouse/ClickHouse/pull/121569) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)). * The `postgresql` table function and the `PostgreSQL` table engine can now connect to another ClickHouse server over the PostgreSQL protocol. The `query(...)` variant still needs an explicit column list. Added the PostgreSQL-compatibility functions `format_type` and `current_setting`. Also fixed cancellation over the PostgreSQL protocol: a cancel request never cancelled anything. [#112940](https://github.com/ClickHouse/ClickHouse/pull/112940) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Disks with the `plain_rewritable` metadata (`s3_plain_rewritable`, and `azure`/`local` object storages with `metadata_type = plain_rewritable`) support hard links if configured with the new `enable_hard_links` setting, which enables `ALTER` operations and mutations on `MergeTree` tables stored on them. Mutations and partition operations share the unchanged blobs between parts instead of copying them. The setting is off by default, because a disk that has hard links cannot be read by older servers. [#110939](https://github.com/ClickHouse/ClickHouse/pull/110939) ([yanglongwei](https://github.com/ylw510)). [#119304](https://github.com/ClickHouse/ClickHouse/pull/119304) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `EXCEPT DATA FROM TABLE` / `EXCEPT DATA FROM TABLES` clauses to `BACKUP` queries, so a table's definition can be backed up without its rows, for any table engine that supports backups. Formatting now quotes keyword table names correctly in `EXCEPT DATA FROM TABLES`, so `BACKUP ... ON CLUSTER` round-trips through the formatter. Closes [#111827](https://github.com/ClickHouse/ClickHouse/issues/111827). [#111895](https://github.com/ClickHouse/ClickHouse/pull/111895) ([Aditya Kumar](https://github.com/adityaksolves)). [#122468](https://github.com/ClickHouse/ClickHouse/pull/122468) ([David Meng](https://github.com/davidmenggx)). * Added five columns to `system.query_log` describing what a query really executed: `used_number_of_joins`, `used_join_algorithms`, `used_join_kinds`, `used_join_strictness`, and `spilled_to_disk`. This makes it possible to find problematic `JOIN` patterns and queries that fall back to external memory across a fleet without reproducing each query. A spilling `SELECT DISTINCT` now records `distinct` in `spilled_to_disk` instead of `[]`, and a spilling block nested loop join records `join`. [#113833](https://github.com/ClickHouse/ClickHouse/pull/113833) ([Matheus Nerone](https://github.com/Manerone)). [#121830](https://github.com/ClickHouse/ClickHouse/pull/121830) ([Groene AI](https://github.com/groeneai)). [#124681](https://github.com/ClickHouse/ClickHouse/pull/124681) ([David Meng](https://github.com/davidmenggx)). * Added table function `traceView` that renders an OpenTelemetry trace from `system.opentelemetry_span_log` as a call tree with a per-span timeline bar, for debugging where the time of a traced, for example distributed, query was spent. It has a `host` column with the `hostname` of the server that recorded each span, so the spans of the initiator, remote shards, replicas, and stateless workers can be distinguished and filtered with `WHERE host = ...`. [#116325](https://github.com/ClickHouse/ClickHouse/pull/116325) ([Diego Gomes Tomé](https://github.com/diegomestre2)). [#122169](https://github.com/ClickHouse/ClickHouse/pull/122169) ([Diego Gomes Tomé](https://github.com/diegomestre2)). [#123528](https://github.com/ClickHouse/ClickHouse/pull/123528) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * `clickhouse-client --login` can now authenticate against arbitrary OAuth/OIDC identity providers via endpoint discovery, with explicit endpoint and scope overrides and improved RFC 8628 device-code handling. On Linux desktops it now also passes the client's environment to `xdg-open` when opening the identity-provider login page, so the system browser is detected correctly. Closes [#120127](https://github.com/ClickHouse/ClickHouse/issues/120127). [#110718](https://github.com/ClickHouse/ClickHouse/pull/110718) ([yanglongwei](https://github.com/ylw510)). [#121337](https://github.com/ClickHouse/ClickHouse/pull/121337) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Added interactive client commands `/dialect`, `/lang`, and `/language` to `clickhouse-client` and `clickhouse-local`, so you can switch the active SQL dialect before parsing, including when the current dialect cannot express a `SET` query, for example to leave `kusto` or `promql`. The current non-default dialect is now shown in the prompt, and `SET dialect = ...` keeps the prompt in sync. [#114568](https://github.com/ClickHouse/ClickHouse/pull/114568) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `chdig` [v26.10.1](https://github.com/azat/chdig/releases/tag/v26.10.1): ZooKeeper browser, per-host summary, and better filters. [#124100](https://github.com/ClickHouse/ClickHouse/pull/124100) ([Azat Khuzhin](https://github.com/azat)). * Support reading the `Parquet` `VARIANT` type (unshredded, as written for example by Spark 4.0) into `Dynamic`, with object values read as `JSON`. A variant column can also be read directly as `JSON` if all its values are objects. Shredded variants are not supported yet. [#117106](https://github.com/ClickHouse/ClickHouse/pull/117106) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Support the `variant` type of `Iceberg`. Based on the `Parquet` variant reader in [#117106](https://github.com/ClickHouse/ClickHouse/pull/117106). [#117198](https://github.com/ClickHouse/ClickHouse/pull/117198) ([Konstantin Vedernikov](https://github.com/scanhex12)). [#125458](https://github.com/ClickHouse/ClickHouse/pull/125458) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support the `Parquet` `FILE` logical type (a reference to bytes stored inline or in an external file): it is read as a named `Tuple`, and can be written from a `Tuple` with the corresponding element names with the new setting `output_format_parquet_write_file_logical_type`. [#125132](https://github.com/ClickHouse/ClickHouse/pull/125132) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Lazy type hints for the `JSON` type are now generally available under the `enable_json_lazy_type_hints` setting. The setting remains disabled by default. [#122711](https://github.com/ClickHouse/ClickHouse/pull/122711) ([Pavel Kruglov](https://github.com/Avogar)). * Add a new input format `Freeform`, which does schema inference for arbitrary tabular text files. Closes [#34006](https://github.com/ClickHouse/ClickHouse/issues/34006). [#43449](https://github.com/ClickHouse/ClickHouse/pull/43449) ([Tri Dao](https://github.com/dmtri35)). * The `generateRandom` table function and the `GenerateRandom` table engine now support `JSON`, `Dynamic`, `Variant`, `BFloat16`, `Time`, and `Time64`. Generated `JSON` documents have a realistic schema derived from the seed. The new settings `null_ratio`, `max_json_depth`, and `max_json_keys_per_object` control the share of nulls and absent keys, the nesting depth, and the number of keys per object. Generation respects memory limits. `CREATE TABLE ... AS generateRandom(...)` also keeps the `SETTINGS` argument in the stored definition. [#107966](https://github.com/ClickHouse/ClickHouse/pull/107966) ([apst41](https://github.com/apst41)). [#124772](https://github.com/ClickHouse/ClickHouse/pull/124772) ([Vadim Lopatin](https://github.com/buggins)). [#123247](https://github.com/ClickHouse/ClickHouse/pull/123247) ([Raúl Marín](https://github.com/Algunenano)). * Add the `obfuscate` table function for generating anonymized query results. [#115004](https://github.com/ClickHouse/ClickHouse/pull/115004) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Text indexes with the `keyValuePairs` tokenizer now answer `map['key'] IN ('value1', 'value2')` directly from the index, as they already did for `map['key'] = 'value'`. Added the function `mapContainsKeyValue(map, key, value)`, which returns whether a map contains an entry with the given key and value; in contrast to `map[key] = value`, all entries of a repeated key are considered, and such an index evaluates the function directly. The `mergeTreeTextIndex` table function returns the decoded parts of each token of such an index in the columns `token_key` and `token_value`. [#120244](https://github.com/ClickHouse/ClickHouse/pull/120244) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#120010](https://github.com/ClickHouse/ClickHouse/pull/120010) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#120183](https://github.com/ClickHouse/ClickHouse/pull/120183) ([Anton Popov](https://github.com/CurtizJ)). * The `mergeTreeTextIndex` table function now returns posting lists and token positions for a text index in the new virtual columns `row_id` and `positions`. If one of them is selected, the function returns a row per posting. The `positions` column is available for indexes created with `support_phrase_search = 1`. [#125536](https://github.com/ClickHouse/ClickHouse/pull/125536) ([Anton Popov](https://github.com/CurtizJ)). * Added `mapRemove`, which removes all entries with a specified key from a map, and `mapEntries`, which returns a map's entries as an array of key-value tuples. [#123322](https://github.com/ClickHouse/ClickHouse/pull/123322) ([Minh Vu](https://github.com/fallintoplace)). [#123344](https://github.com/ClickHouse/ClickHouse/pull/123344) ([Minh Vu](https://github.com/fallintoplace)). * The `-Map` combinator can now be used with non-`Map` columns (the key is passed as the last argument, e.g. `sumMap(value, key)`) and supports aggregate functions with multiple arguments, e.g. `argMaxMap(arg, val, key)`. [#123449](https://github.com/ClickHouse/ClickHouse/pull/123449) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The aggregate functions `avg`, `sum`, `sumWithOverflow`, and `sumKahan` now support the `Interval` data types. The result is an interval of the same unit. [#121458](https://github.com/ClickHouse/ClickHouse/pull/121458) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#121600](https://github.com/ClickHouse/ClickHouse/pull/121600) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The linear regression aggregate functions can now also be called under ClickHouse-style names: `regrSlope`, `regrIntercept`, `regrR2`, `regrCount`, `regrAvgX`, `regrAvgY`, `regrSXX`, `regrSYY`, and `regrSXY`. [#119796](https://github.com/ClickHouse/ClickHouse/pull/119796) ([mosya415](https://github.com/mosya415)). * Added the `arrayMinIndex` and `arrayMaxIndex` array functions. [#120097](https://github.com/ClickHouse/ClickHouse/pull/120097) ([Minh Vu](https://github.com/fallintoplace)). [#123686](https://github.com/ClickHouse/ClickHouse/pull/123686) ([Minh Vu](https://github.com/fallintoplace)). * Added the `arrayRemoveAt` array function. [#120445](https://github.com/ClickHouse/ClickHouse/pull/120445) ([Minh Vu](https://github.com/fallintoplace)). * Added higher-order array functions to pack bits: `arrayPackBitsToUInt64`, `arrayPackBitsToString`, `arrayPackBitsToFixedString`, `arrayPackBitGroupsToUInt64`, `arrayPackBitGroupsToString`, and `arrayPackBitGroupsToFixedString`. [#123478](https://github.com/ClickHouse/ClickHouse/pull/123478) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added SQL functions `normalizedQueryHashUnordered` and `normalizedQueryHashUnorderedOrNull`. Unlike `normalizedQueryHash`, they ignore the order of expression lists such as the `SELECT` list, `GROUP BY` keys, and function arguments, so logically equivalent queries can collapse to one pattern in `query_log`; the `OrNull` variant returns `NULL` instead of throwing when the input is not valid ClickHouse SQL in the current session. [#116345](https://github.com/ClickHouse/ClickHouse/pull/116345) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Add functions `encodeQuotedPrintable` and `decodeQuotedPrintable` for Quoted-Printable encoding (RFC 2045), commonly used in email. [#125454](https://github.com/ClickHouse/ClickHouse/pull/125454) ([Jay Pokale](https://github.com/JayPokale)). * Added the `sql_compatible_division_for_integers` setting, disabled by default, to make `/` and `divide` return integer quotients truncated toward zero when both operands are integers. It supports integer types up to 256 bits and checks division by zero and result overflow; division involving `Decimal` or floating-point operands is unchanged. [#124490](https://github.com/ClickHouse/ClickHouse/pull/124490) ([Ilya Yatsishin](https://github.com/qoega)). * Added `SYSTEM RESET FILELOG [db.]table [FILE 'file_name' [OFFSET n | TO END]]`: a `FileLog` table reads all its files again from the beginning, or one file from the beginning, from a given offset, or only from data appended after the command, without recreating the table. Requires the new `SYSTEM RESET FILELOG` privilege. [#123545](https://github.com/ClickHouse/ClickHouse/pull/123545) ([Groene AI](https://github.com/groeneai)). * The `FileLog` table engine now accepts globs (`*`, `?`, `{a,b}`, `{N..M}`) in the file name of its path, for example `FileLog('/var/lib/clickhouse/user_files/my_app/*.log', 'JSONEachRow')`, and reads only the matching files of that directory, so the compressed files `logrotate` leaves next to the log are not read. A file that is already read keeps being read after it is renamed to a non-matching name, such as `app.log` rotated to `app.log.1`. [#123590](https://github.com/ClickHouse/ClickHouse/pull/123590) ([Groene AI](https://github.com/groeneai)). * Added the `flat_namespaces` setting to the `DataLakeCatalog` database engine. Set it for `Iceberg` REST catalogs that have only single-level namespaces and reject the `parent` query parameter of the list-namespaces endpoint, such as Apache Polaris federated to AWS Glue: only top-level namespaces are then listed and sub-namespaces are not requested. [#117329](https://github.com/ClickHouse/ClickHouse/pull/117329) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added the ClickHouse Keeper `ListWithOptions` operation for filtered direct or recursive listing with optional stats, data, limits, and shuffled results. [#117521](https://github.com/ClickHouse/ClickHouse/pull/117521) ([Antonio Andelic](https://github.com/antonio2368)). * Added the `MergeTree` setting `min_unreserved_disk_space_for_merge`: background merge selection subtracts the specified amount from the unreserved disk space, so large merges stop being selected before they can take the last free space and starve inserts. Unlike `keep_free_space_bytes`, the protected space remains usable by ClickHouse itself. [#115723](https://github.com/ClickHouse/ClickHouse/pull/115723) ([Valery Petrov](https://github.com/valerypetrov)). * You can now disable reading through the `executable` table function and from `Executable`/`ExecutablePool` tables with the new `allow_executable_tables` setting, for a deployment that must not run arbitrary server-side scripts. It defaults to `true`, so existing behavior is unchanged. [#120713](https://github.com/ClickHouse/ClickHouse/pull/120713) ([Hanzi Jiang](https://github.com/HanziJiang)). * Added server setting `ai_function_allow_sql_named_collections`. When disabled, AI functions accept only named collections defined in the server configuration file. [#124999](https://github.com/ClickHouse/ClickHouse/pull/124999) ([George Larionov](https://github.com/george-larionov)). * Added `` and `` settings to restrict which TLS 1.3 cipher suites a TLS port negotiates. Without them, `cipherList` controls only TLS 1.2 and below. An absent or empty value keeps the OpenSSL defaults. Closes [#119929](https://github.com/ClickHouse/ClickHouse/issues/119929). [#120480](https://github.com/ClickHouse/ClickHouse/pull/120480) ([Groene AI](https://github.com/groeneai)). * You can now set a custom syslog program name for ClickHouse logs using `logger.syslog.programname`, making it easier to distinguish multiple ClickHouse instances in syslog. The default syslog identifier is now command-specific: remote syslog previously sent `-` as the RFC 5424 `APP-NAME` and now sends the command name (for example `clickhouse-server`), and the multi-purpose binary started as `clickhouse server` or `clickhouse keeper` now uses `clickhouse-server` / `clickhouse-keeper` instead of `clickhouse` for local syslog. [#98789](https://github.com/ClickHouse/ClickHouse/pull/98789) ([François Martin](https://github.com/martinfrancois)). * Added the `Copy` event and the `source_bucket` and `source_remote_path` columns to `system.blob_storage_log` for server-side object copies (S3 `CopyObject` and `UploadPartCopy`, Azure Blob Storage copies). Azure backups, restores, and `AzureQueue` moves now write to `system.blob_storage_log` too. Closes [#123163](https://github.com/ClickHouse/ClickHouse/issues/123163). [#123178](https://github.com/ClickHouse/ClickHouse/pull/123178) ([Groene AI](https://github.com/groeneai)). * Added an `is_lazy` column to `system.dictionaries`. It shows whether each dictionary loads on first use, taking into account both its `dictionary_lazy_load` setting and the server setting `dictionaries_lazy_load`. This makes it possible to tell a lazy dictionary that has not been used yet from an eager one that has not been loaded yet; both show `NOT_LOADED`. [#124260](https://github.com/ClickHouse/ClickHouse/pull/124260) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Added the `replica_group` column to `system.clusters`. For clusters that belong to a `Replicated` database, it shows the replica group of each node as published from the server configuration `replica_group_name`, or its alias `role`. [#120537](https://github.com/ClickHouse/ClickHouse/pull/120537) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Added `SYSTEM DISABLE ALL FAILPOINTS`, which disables every fail point at once and resumes any thread blocked on a pauseable one. [#118877](https://github.com/ClickHouse/ClickHouse/pull/118877) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Experimental Feature * Added an experimental columns cache that keeps deserialized columns of wide `MergeTree` parts in memory, one entry per stripe of granules of a column. Compared with the uncompressed cache, this saves time on deserialization and copying. It is enabled by the experimental setting `use_columns_cache`, with `enable_reads_from_columns_cache` and `enable_writes_to_columns_cache` as beta sub-settings, and it applies to tables in databases that assign UUIDs (`Atomic` and `Replicated`). The cache gives its memory back to queries when the server is short of it. [#96844](https://github.com/ClickHouse/ClickHouse/pull/96844) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added experimental `LATERAL JOIN` support, allowing correlated subqueries in the `FROM` clause. This enables the standard SQL pattern for "top N per group" and per-row aggregation queries. Controlled by the `allow_experimental_lateral_join` setting. [#97833](https://github.com/ClickHouse/ClickHouse/pull/97833) ([Dimitri König](https://github.com/dimitri-koenig)). * Added an experimental `logsql` dialect: LogsQL, the log query language of VictoriaLogs, can now be used to query ClickHouse tables with logs. Enable it with `SET enable_logsql_dialect = 1, logsql_table = '', dialect = 'logsql'` (plus `logsql_database`, `logsql_time_column`, and `logsql_message_column` as needed). LogsQL queries are translated at parse time into ordinary `SELECT` queries, so text filters expect `String`-backed log columns, while the `math` pipe applies raw arithmetic and needs numeric operand columns (`String` field values are not coerced to numbers). [#113207](https://github.com/ClickHouse/ClickHouse/pull/113207) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123891](https://github.com/ClickHouse/ClickHouse/pull/123891) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the beta `PCO` compression codec for fixed-width numeric columns, based on the [pcodec](https://github.com/pcodec/pcodec) library. Enable it with `enable_pco_codec = 1`. [#110613](https://github.com/ClickHouse/ClickHouse/pull/110613) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new implementation of the `unity` catalog type in the `DataLakeCatalog` database engine. It serves both `DeltaLake` and `Iceberg` tables from a single Unity Catalog, detecting the format of each table and routing `Iceberg` tables through the Unity Catalog `Iceberg` REST endpoint. Enable it with the database setting `use_unity_catalog_v2`, or set the session setting of the same name before `CREATE DATABASE`. Switch an existing database with `ALTER DATABASE ... MODIFY SETTING use_unity_catalog_v2 = 1`. [#115879](https://github.com/ClickHouse/ClickHouse/pull/115879) ([Asya Shneerson](https://github.com/asya-ch)). * Added an experimental embedded `Iceberg` REST catalog server with ClickHouse HTTP authentication. It supports `GET /v1/config`, listing, creating, and checking the existence of namespaces, stored either in memory or in `Keeper`, and listing, creating, loading, checking the existence of, and dropping tables, with their metadata files stored in `S3`. The catalog can also serve TLS via `secure_port`, using the certificate and key in `openSSL.server`, and correctly handles `/` in table and namespace names. Closes [#125762](https://github.com/ClickHouse/ClickHouse/issues/125762). [#118924](https://github.com/ClickHouse/ClickHouse/pull/118924) ([Asya Shneerson](https://github.com/asya-ch)). [#123000](https://github.com/ClickHouse/ClickHouse/pull/123000) ([Asya Shneerson](https://github.com/asya-ch)). [#123331](https://github.com/ClickHouse/ClickHouse/pull/123331) ([Asya Shneerson](https://github.com/asya-ch)). [#124978](https://github.com/ClickHouse/ClickHouse/pull/124978) ([Asya Shneerson](https://github.com/asya-ch)). [#125791](https://github.com/ClickHouse/ClickHouse/pull/125791) ([Groene AI](https://github.com/groeneai)). * Added experimental XGBoost dictionaries: the `XGBOOST` layout trains a gradient-boosted model from the source table at load time, and `predictXGBoost` runs inference. Enable it with `enable_xgboost`. Available only in builds that include the XGBoost contrib library. [#109999](https://github.com/ClickHouse/ClickHouse/pull/109999) ([Matheus Nerone](https://github.com/Manerone)). * Added `ipcryptEncrypt`, `ipcryptDecrypt`, `ipcryptPrefixEncrypt` and `ipcryptPrefixDecrypt` functions for format-preserving encryption of IP addresses following the [IPCrypt specification](https://ipcrypt-std.github.io/). The deterministic mode produces consistent output for deduplication, while the prefix-preserving mode maintains subnet relationships. Enable with `SET enable_ipcrypt = 1`. Contributed by Frank Denis. [#123680](https://github.com/ClickHouse/ClickHouse/pull/123680) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added read support for `Iceberg` v3 deletion vectors stored in `Puffin` files. [#110781](https://github.com/ClickHouse/ClickHouse/pull/110781) ([linjiayu1025-collab](https://github.com/linjiayu1025-collab)). * Added experimental support for `ALTER TABLE ... DROP PARTITION` on standalone `Iceberg` tables and `Iceberg` REST catalogs. On standalone tables, the operation currently requires `Iceberg` format-version 2 with a single non-evolved partition spec. Old date-transform spellings in the partition expression are now evaluated with the `Iceberg` transform semantics too. [#111219](https://github.com/ClickHouse/ClickHouse/pull/111219) ([Den Kalantaevskii](https://github.com/Diskein)). [#123273](https://github.com/ClickHouse/ClickHouse/pull/123273) ([Groene AI](https://github.com/groeneai)). [#122269](https://github.com/ClickHouse/ClickHouse/pull/122269) ([Den Kalantaevskii](https://github.com/Diskein)). * `parallel_replicas_plan_based` can now distribute queries with an `IN (subquery)` predicate instead of falling back to single-node execution. Closes [#111876](https://github.com/ClickHouse/ClickHouse/issues/111876). [#114086](https://github.com/ClickHouse/ClickHouse/pull/114086) ([Igor Nikonov](https://github.com/devcrafter)). * With `parallel_replicas_plan_based` enabled, distributed `INSERT ... SELECT` again uses distributed execution when `parallel_distributed_insert_select = 2`, instead of running the `SELECT` on the initiator alone. Closes [#118276](https://github.com/ClickHouse/ClickHouse/issues/118276). [#121097](https://github.com/ClickHouse/ClickHouse/pull/121097) ([Igor Nikonov](https://github.com/devcrafter)). * Distributed query plans built with `make_distributed_plan` now compress exchanged data with `network_compression_method`; for `ZSTD`, they also use `network_zstd_compression_level` instead of the server's default codec settings. [#120333](https://github.com/ClickHouse/ClickHouse/pull/120333) ([Alexander Gololobov](https://github.com/davenger)). * Queries executed with the experimental `make_distributed_plan` setting now propagate the OpenTelemetry trace context to stateless workers via the `traceparent` and `tracestate` HTTP headers of the task dispatch request. Each dispatched task produces a `StatelessWorkerClient::sendTask` span on the initiator plus `InterserverIOHTTPHandler` and `DistributedPlanTask::execute` spans on the worker, with the task id, initial query id, plan hash, and exchange streams as attributes, so the whole distributed plan is visible as one trace in `system.opentelemetry_span_log`. [#119010](https://github.com/ClickHouse/ClickHouse/pull/119010) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Join runtime filters can now travel across distributed query plan stages with the experimental setting `distributed_plan_join_runtime_filters` (default off). Build-side tasks send partial filters to probe-side scans before their shuffle exchange, reducing the number of rows crossing the network. A filter that is lost, late, or never built cannot change results. [#114822](https://github.com/ClickHouse/ClickHouse/pull/114822) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * Distributed index analysis is now beta. [#123577](https://github.com/ClickHouse/ClickHouse/pull/123577) ([Azat Khuzhin](https://github.com/azat)). * Added the experimental setting `enable_join_seal_gated_reading` (default off). When enabled together with `enable_join_runtime_filters` and the probe-side join key is a prefix of the table's primary key, the probe side of a hash `JOIN` starts reading only after the build side completes, and the completed runtime filter prunes whole mark ranges by the primary key before read tasks are created, instead of only filtering rows after they were read. [#113512](https://github.com/ClickHouse/ClickHouse/pull/113512) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Replaced the experimental settings `query_plan_optimize_join_order_use_conflict_detector_a` and `query_plan_optimize_join_order_use_conflict_detector_c` with `query_plan_optimize_join_order_conflict_detector`, which takes `a`, `c`, or `''` (the default). [#120630](https://github.com/ClickHouse/ClickHouse/pull/120630) ([Fisnik Kastrati](https://github.com/fkastrati)). * Adaptive codec selection (`enable_adaptive_codec_selection`) also tries each specialized codec followed by the table's default codec, and makes merges of `Float32` and `Float64` columns faster. [#121630](https://github.com/ClickHouse/ClickHouse/pull/121630) ([Raufs Dunamalijevs](https://github.com/rienath)). [#122248](https://github.com/ClickHouse/ClickHouse/pull/122248) ([Raufs Dunamalijevs](https://github.com/rienath)). * Added the setting `json_extract_named_tuples_as_objects` (default off): when enabled, the `JSONExtract` functions fill named tuples from JSON objects only. By default a JSON array fills a named tuple positionally, so which element lands in which field depends on the order the fields were declared in. [#122637](https://github.com/ClickHouse/ClickHouse/pull/122637) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Experimental `UNIQUE KEY` SST probes now batch lookups with `MultiGet`, so absent keys can be filtered by the SST bloom filter. More unsupported `UNIQUE KEY` operations are now rejected early. [#119997](https://github.com/ClickHouse/ClickHouse/pull/119997) ([johnjing](https://github.com/JingYanchao)). [#123002](https://github.com/ClickHouse/ClickHouse/pull/123002) ([Murphy](https://github.com/murphy-4o)). * The experimental reader executor (`use_reader_executor = 1`) now plans filesystem cache reads ahead over a window set by `reader_executor_plan_look_ahead` instead of re-checking the cache on every read window. It also no longer fails spuriously with `CANNOT_READ_ALL_DATA` when a concurrent download advances the filesystem cache during a cold read. [#115816](https://github.com/ClickHouse/ClickHouse/pull/115816) ([Sema Checherinda](https://github.com/CheSema)). [#122214](https://github.com/ClickHouse/ClickHouse/pull/122214) ([Sema Checherinda](https://github.com/CheSema)). * Streaming queries now prefer a `commit_order` projection when one is available for the current read round. [#120420](https://github.com/ClickHouse/ClickHouse/pull/120420) ([Mikhail Artemenko](https://github.com/Michicosun)). * PromQL again supports the range functions `present_over_time`, `absent_over_time`, `quantile_over_time`, and `predict_linear`, and now also supports `stddev_over_time`, `stdvar_over_time`, `first_over_time`, `ts_of_first_over_time`, `ts_of_last_over_time`, and `mad_over_time`. [#112355](https://github.com/ClickHouse/ClickHouse/pull/112355) ([Valery Petrov](https://github.com/valerypetrov)). [#120352](https://github.com/ClickHouse/ClickHouse/pull/120352) ([Vitaly Baranov](https://github.com/vitlibar)). [#121578](https://github.com/ClickHouse/ClickHouse/pull/121578) ([Vitaly Baranov](https://github.com/vitlibar)). [#121800](https://github.com/ClickHouse/ClickHouse/pull/121800) ([Vitaly Baranov](https://github.com/vitlibar)). * Added PromQL functions `min_of` and `max_of` to return the smaller or larger of two scalar values. [#121250](https://github.com/ClickHouse/ClickHouse/pull/121250) ([Minh Vu](https://github.com/fallintoplace)). * PromQL now implements `timestamp` for both direct selectors and general expressions, matching Prometheus 3.5.0 semantics. [#112356](https://github.com/ClickHouse/ClickHouse/pull/112356) ([Valery Petrov](https://github.com/valerypetrov)). * Support standalone PromQL `step` and `range` scalar functions. The PromQL parser now also accepts bare calls to `start`, `end`, `sort_by_label`, `sort_by_label_desc`, `double_exponential_smoothing`, and `limit_ratio`, so these not yet implemented functions are rejected after parsing instead of failing with a parse error. [#121118](https://github.com/ClickHouse/ClickHouse/pull/121118) ([Minh Vu](https://github.com/fallintoplace)). [#123589](https://github.com/ClickHouse/ClickHouse/pull/123589) ([James Cunningham](https://github.com/JTCunning)). * `prometheusQuery` now always returns `Float64` values and `DateTime64` timestamps with at least millisecond precision. The time zone of the returned timestamps comes from its `DateTime` or `DateTime64` arguments when they agree, and otherwise from the table. [#120285](https://github.com/ClickHouse/ClickHouse/pull/120285) ([Vitaly Baranov](https://github.com/vitlibar)). * PromQL now treats Prometheus stale markers as absent in instant selectors and omits them from range selectors. [#121098](https://github.com/ClickHouse/ClickHouse/pull/121098) ([Minh Vu](https://github.com/fallintoplace)). * PromQL now rejects queries that use the same label in both `on(...)` and `group_left(...)` or `group_right(...)`. [#114376](https://github.com/ClickHouse/ClickHouse/pull/114376) ([Minh Vu](https://github.com/fallintoplace)). * PromQL now rejects vector selectors whose matchers can all match the empty string. [#121099](https://github.com/ClickHouse/ClickHouse/pull/121099) ([Minh Vu](https://github.com/fallintoplace)). * PromQL's `stddev` and `stdvar` now return `NaN` instead of `0`. [#122743](https://github.com/ClickHouse/ClickHouse/pull/122743) ([James Cunningham](https://github.com/JTCunning)). * PromQL now returns `NaN` from date functions when the sample is not finite. [#124300](https://github.com/ClickHouse/ClickHouse/pull/124300) ([James Cunningham](https://github.com/JTCunning)). * PromQL no longer fails on empty joins. [#124362](https://github.com/ClickHouse/ClickHouse/pull/124362) ([James Cunningham](https://github.com/JTCunning)). * Aligned PromQL string and comment parsing with Prometheus: quoted strings cannot contain a line feed, and a line comment can end at the end of the query without a space after `#`. Closes [#117812](https://github.com/ClickHouse/ClickHouse/issues/117812). [#124584](https://github.com/ClickHouse/ClickHouse/pull/124584) ([Vitaly Baranov](https://github.com/vitlibar)). * PromQL range functions over scalars now handle windows with no samples without failing on `NULL`, including when used in nested range functions. [#124586](https://github.com/ClickHouse/ClickHouse/pull/124586) ([Vitaly Baranov](https://github.com/vitlibar)). * Added deduplication caches for `TimeSeries` metric families and tags, and fixed stale cache entries when the pending rows are full. [#121915](https://github.com/ClickHouse/ClickHouse/pull/121915) ([Vitaly Baranov](https://github.com/vitlibar)). [#122771](https://github.com/ClickHouse/ClickHouse/pull/122771) ([Vitaly Baranov](https://github.com/vitlibar)). * The `tags` inner table of the `TimeSeries` table engine is split into a `tags` table without `min_time` and `max_time` and a separate table of time ranges. [#121924](https://github.com/ClickHouse/ClickHouse/pull/121924) ([Vitaly Baranov](https://github.com/vitlibar)). * The inner tables of the `TimeSeries` table engine now use the outer column name `metric_family` instead of `metric_family_name`. [#119226](https://github.com/ClickHouse/ClickHouse/pull/119226) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed `ALIAS_REQUIRED` for a join with a `VIEW` written without an alias under the experimental setting `analyzer_inline_views = 1`: the inlined subquery now keeps the view's name as a table expression name, so it needs no alias and stays usable as a column qualifier, with or without the database name. [#118401](https://github.com/ClickHouse/ClickHouse/pull/118401) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading subcolumns of the experimental `Quantized` codec (`vec.quantized`, `vec.product_quantization_codebook`) through a `Merge` table no longer breaks when another child table of the `Merge` lacks the base column. [#123895](https://github.com/ClickHouse/ClickHouse/pull/123895) ([Groene AI](https://github.com/groeneai)). #### Performance Improvement * Large hash tables used for `GROUP BY`, `uniqExact`, and `JOIN` now use up to two times less memory, and aggregation with many keys is faster on many threads. [#125001](https://github.com/ClickHouse/ClickHouse/pull/125001) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The query condition cache now serves conditions that involve the current time, e.g. `WHERE time >= now() - INTERVAL 10 DAY`, when reading from `MergeTree` tables, by deriving a deterministic condition from them with the current-time constant rounded onto a time grid. Closes [#115504](https://github.com/ClickHouse/ClickHouse/issues/115504). [#115522](https://github.com/ClickHouse/ClickHouse/pull/115522) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enabled the setting `apply_string_filters_during_scan` by default. Substring search conditions on `String` columns (`LIKE`, `position`, `startsWith`, `endsWith`) from `PREWHERE` are pushed down into the column scan, and non-matching values are read as empty strings without copying their data. This makes such queries faster and lowers their memory usage when reading from `MergeTree` tables and from `Parquet` pages whose string values use the `PLAIN`, `DELTA_LENGTH_BYTE_ARRAY`, `DELTA_BYTE_ARRAY`, or dictionary encodings, including reads using the uncompressed cache on object storage. [#115848](https://github.com/ClickHouse/ClickHouse/pull/115848) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124503](https://github.com/ClickHouse/ClickHouse/pull/124503) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#125079](https://github.com/ClickHouse/ClickHouse/pull/125079) ([Groene AI](https://github.com/groeneai)). [#124507](https://github.com/ClickHouse/ClickHouse/pull/124507) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries `GROUP BY key LIMIT n` and `GROUP BY key ORDER BY key LIMIT n` over `MergeTree` tables now read much less data: the aggregation publishes the boundary of its top-K heap to the reading step, which drops rows beyond it before reading the other columns and can skip whole granules through the primary key or a `minmax` skip index. Controlled by the new setting `enable_group_by_top_k_dynamic_filtering`. The same primary-key granule skipping now also applies to `ORDER BY column LIMIT n` when the column is in the primary key. [#121145](https://github.com/ClickHouse/ClickHouse/pull/121145) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries with `ORDER BY LIMIT n` over `Parquet` files now use TopN dynamic filtering with `use_top_k_dynamic_filtering`: rows that cannot enter the top-K are filtered inside the reader before the remaining columns are read, and whole row groups are skipped when their min/max statistics prove they cannot contribute. This works through the `file` and `url` table functions, the `File` and `URL` engines, object storage (`s3`, `Azure`, `HDFS`), and data lake tables such as `Iceberg`. Floating-point sort columns are excluded for now. [#115798](https://github.com/ClickHouse/ClickHouse/pull/115798) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122109](https://github.com/ClickHouse/ClickHouse/pull/122109) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124861](https://github.com/ClickHouse/ClickHouse/pull/124861) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `query_plan_join_shard_by_pk_ranges` is enabled by default. A `full_sorting_merge` or `parallel_full_sorting_merge` `JOIN` of two `MergeTree` tables whose join keys contain a prefix of the primary key of both tables now runs as independent streaming merge joins over primary-key ranges on all threads instead of a single-threaded merge join, several times faster and with several times less memory in our tests. Hash joins are affected only when both join runtime filters and spilling to disk are disabled. Set `query_plan_join_shard_by_pk_ranges = 0` to restore the previous behavior. [#117856](https://github.com/ClickHouse/ClickHouse/pull/117856) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119925](https://github.com/ClickHouse/ClickHouse/pull/119925) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enabled the text-index postings cache by default with `use_text_index_postings_cache`. On a 1B-row table, a four-token phrase query that previously recomputed posting lists and phrase-search results within one query went from 19 s to 0.12 s. The shared cache is bounded by `text_index_postings_cache_size` (2 GiB by default). [#120677](https://github.com/ClickHouse/ClickHouse/pull/120677) ([Nikita Fomichev](https://github.com/fm4v)). [#123228](https://github.com/ClickHouse/ClickHouse/pull/123228) ([Anton Popov](https://github.com/CurtizJ)). * With `optimize_aggregation_in_order = 1`, or with the default `optimize_distinct_in_order = 1`, a `GROUP BY` or `DISTINCT` whose keys extend past the table's sorting key no longer re-packs the whole block's grouping key once per run of equal sorting-key values. That cost was quadratic in `max_block_size` once the sorting prefix is near-unique, so `GROUP BY a, b` on a table sorted by `a` is up to 130x faster. Closes [#45386](https://github.com/ClickHouse/ClickHouse/issues/45386). [#120259](https://github.com/ClickHouse/ClickHouse/pull/120259) ([Groene AI](https://github.com/groeneai)). * Blocks sent back by remote replicas are now marshalled in parallel (`enable_parallel_blocks_marshalling`) when `parallel_replicas_plan_based` is enabled. [#121642](https://github.com/ClickHouse/ClickHouse/pull/121642) ([Igor Nikonov](https://github.com/devcrafter)). * `GROUP BY` with a `Nullable` key now uses the adaptive aggregator, except when that key is the only one and is a number of up to 8 bytes, a `String`, or a `FixedString`, or when any key is `LowCardinality`. Schema inference makes such keys common in CSV, JSON, and `Parquet` files; high-cardinality aggregations over them become up to 2x faster with up to 2.2x less memory. [#124868](https://github.com/ClickHouse/ClickHouse/pull/124868) ([Groene AI](https://github.com/groeneai)). * Queries over `Nullable` columns now use the same optimizations as over non-`Nullable` ones: `length`, `empty`, `notEmpty`, and comparisons with an empty string on `Nullable(String)` read the `size` subcolumn, `sum(x + c)` is rewritten to `sum(x) + c * count(x)`, and `uniqExact` over `Nullable` is merged in parallel. Some queries become several times faster. [#125011](https://github.com/ClickHouse/ClickHouse/pull/125011) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The trivial `GROUP BY ... LIMIT n` optimization (setting `optimize_trivial_group_by_limit_query`) now also applies to queries with aggregate functions in the projection when the `GROUP BY` top-K optimization does not apply, for example above `query_plan_max_limit_for_top_k_optimization`: once any aggregation thread exceeds `n + offset` distinct keys, all threads are restricted to a single shared set of kept keys, so the hash tables stay small while the aggregate values of the returned keys remain exact, with up to 7-9x speedups. The optimization no longer fires for queries with `DISTINCT`, `QUALIFY`, `LIMIT AFTER`, `LIMIT UNTIL`, window functions, `arrayJoin`, or `exact_rows_before_limit`, where it could return wrong or inexact results. [#114690](https://github.com/ClickHouse/ClickHouse/pull/114690) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * When `max_threads` is not set explicitly, a query is planned with the fair share of CPU slots of the concurrency control instead of all CPU cores. This increases the throughput and lowers the memory usage under many concurrent queries, while a query on an idle server still uses all cores. It can be disabled with the setting `use_concurrency_control_for_max_threads`. [#125085](https://github.com/ClickHouse/ClickHouse/pull/125085) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A single-level `GROUP BY` result smaller than `max_block_size` is now split into several blocks when it was aggregated by several threads, so the steps after it, such as the probe of a following join, run in parallel instead of on one thread. Previously such queries could be several times slower, most visibly under `compatibility` older than 26.8, which disables `enable_parallel_single_level_merge` and `enable_adaptive_aggregator`. The result of a `GROUP BY` on a single 8- or 16-bit integer key is split the same way. [#121711](https://github.com/ClickHouse/ClickHouse/pull/121711) ([Pratham Nayak](https://github.com/protomn)). * Restored the batched hash-and-prefetch insert path for `uniq`, `uniqCombined`, and `uniqCombined64` in ordinary `GROUP BY` queries, where they had silently fallen back to a per-row loop and were up to 46% slower. Closes [#120531](https://github.com/ClickHouse/ClickHouse/issues/120531). [#120578](https://github.com/ClickHouse/ClickHouse/pull/120578) ([Groene AI](https://github.com/groeneai)). * Faster `GROUP BY` over keys that have to be serialized, such as several columns that include a `String`. [#122857](https://github.com/ClickHouse/ClickHouse/pull/122857) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * `GROUP BY` over multiple `LowCardinality` keys now serializes the keys from the per-block dictionaries instead of copying them into full columns first, reducing CPU time and memory usage of aggregation queries over `LowCardinality` dimensions. [#121795](https://github.com/ClickHouse/ClickHouse/pull/121795) ([artemkulyk](https://github.com/artemkulyk)). * Speed up `GROUP BY` over several `LowCardinality` keys when the same key tuples repeat, by caching aggregate states per tuple of dictionary positions in the serialized aggregation method. [#121998](https://github.com/ClickHouse/ClickHouse/pull/121998) ([artemkulyk](https://github.com/artemkulyk)). * `GROUP BY` over a single `String` column with only `count` is faster when the grouping state does not fit in the CPU cache, especially with long strings. [#123655](https://github.com/ClickHouse/ClickHouse/pull/123655) ([Groene AI](https://github.com/groeneai)). * `GROUP BY` queries with many groups that end in `ORDER BY count() ... LIMIT n` and use no other aggregate function are slightly faster when they group by a `String` column or by several columns. [#124277](https://github.com/ClickHouse/ClickHouse/pull/124277) ([Groene AI](https://github.com/groeneai)). * Share the `GROUP BY` top-K skip boundary between aggregation threads. [#116150](https://github.com/ClickHouse/ClickHouse/pull/116150) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Allow using fused `GROUP BY` top-K with a serialized query plan. [#116163](https://github.com/ClickHouse/ClickHouse/pull/116163) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Added a query-plan optimization, controlled by the new setting `query_plan_aggregation_having_prefilter`, for `HAVING count() ` over a `GROUP BY`. While a two-level bucket of the aggregation result is converted to chunks, a group whose `count()` cannot satisfy the bound is now skipped before its key columns are materialized, instead of being materialized and then discarded by the filter above. This speeds up "groups above a threshold" queries over a high-cardinality `GROUP BY` with wide keys, where the discarded keys are most of the conversion. A skipped group is neither filtered nor finalized, so a `HAVING` conjunct written before the bound, and a sibling aggregate's finalization, stop being evaluated on the groups the bound rejects: a query that raised an exception from one of those can now return rows instead. Setting `query_plan_aggregation_having_prefilter = 0`, or `compatibility` to a version below `26.10`, keeps the previous behavior. [#120510](https://github.com/ClickHouse/ClickHouse/pull/120510) ([Groene AI](https://github.com/groeneai)). * `optimize_aggregators_of_group_by_keys` now works for `LowCardinality` `GROUP BY` keys, so a `HAVING` predicate such as `HAVING min(s) = 'x'` is pushed down to storage and can use skip indexes, as for a plain key. It also eliminates `anyHeavy`, `quantileExact`, `quantileExactLow`, `quantileExactHigh`, `groupBitAnd`, and `groupBitOr` of a `GROUP BY` key. [#110059](https://github.com/ClickHouse/ClickHouse/pull/110059) ([Groene AI](https://github.com/groeneai)). * `-If` aggregate functions with one `Nullable` argument no longer allocate a buffer the size of the whole block for every group with `optimize_aggregation_in_order` or for every window frame. With small groups or frames this makes them 1.7 to 3.1 times faster on x86_64. [#122505](https://github.com/ClickHouse/ClickHouse/pull/122505) ([Groene AI](https://github.com/groeneai)). * Speed up `-ForEach` aggregate functions such as `sumForEach` and `avgForEach`: rows are processed in batches instead of calling `add` for each row, and an array of two or more elements is passed to the nested function in one call instead of one call per element. Grouped multi-argument `-If` aggregate functions with a `Nullable` argument or condition are also processed in batches. [#121137](https://github.com/ClickHouse/ClickHouse/pull/121137) ([Valery Petrov](https://github.com/valerypetrov)). [#124207](https://github.com/ClickHouse/ClickHouse/pull/124207) ([Groene AI](https://github.com/groeneai)). * Speed up serialization of `AggregateFunction` state columns with a fixed-size state, such as `sum`, `count`, `avg`, `min`, `max`, and `any`, and their `-If`, `-Array`, `-Merge`, and `Nullable` variants. This affects writes to `AggregatingMergeTree`, the `Native` format, distributed queries, and spilling aggregate states to disk. [#113962](https://github.com/ClickHouse/ClickHouse/pull/113962) ([Manuel](https://github.com/raimannma)). * Speed up reading and merging of `uniq` aggregate function states, such as `uniqMerge` over an `AggregatingMergeTree`, by 10-35% depending on state sizes and merge shape. [#111493](https://github.com/ClickHouse/ClickHouse/pull/111493) ([Manuel](https://github.com/raimannma)). * Faster merging of `uniqCombined` and `uniqCombined64` states with many distinct values into empty states, e.g. in `-Merge` combinators and `AggregatingMergeTree`: up to 6x faster. [#124389](https://github.com/ClickHouse/ClickHouse/pull/124389) ([Manuel](https://github.com/raimannma)). * Reduce memory usage and improve performance of external `DISTINCT` by avoiding spills for duplicate-heavy input, writing smaller spill blocks, budgeting tail spilling before the final merge, preserving constant payloads, combining chunks before sorting, and retaining fewer rows while restoring order with `LIMIT`. [#121772](https://github.com/ClickHouse/ClickHouse/pull/121772) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Faster sorted-stream processing on a multi-column key whose leading column has long runs, in `LIMIT BY`, negative `LIMIT BY`, `DISTINCT` in order, `full_sorting_merge` and `partial_merge` `JOIN`, and `ReplacingMergeTree` `FINAL` without vertical `FINAL`: the run of a key prefix is now found once instead of again for every group in it. The query from [#123700](https://github.com/ClickHouse/ClickHouse/issues/123700) is about 3.5x faster. [#123808](https://github.com/ClickHouse/ClickHouse/pull/123808) ([Groene AI](https://github.com/groeneai)). * Short queries with many keyed aggregations or scalar subqueries over large constant-folded literals are faster because hash-table-stats cache keys no longer hash whole variable-size constant values. [#122738](https://github.com/ClickHouse/ClickHouse/pull/122738) ([Nikita Taranov](https://github.com/nickitat)). * Reduce memory usage of some window functions by precomputing peer-group boundaries instead of keeping earlier blocks in memory. [#123737](https://github.com/ClickHouse/ClickHouse/pull/123737) ([Mikhail Artemenko](https://github.com/Michicosun)). * A filter on the `PARTITION BY` columns of a window function is no longer pushed below the window (this reverts [#110114](https://github.com/ClickHouse/ClickHouse/pull/110114)). The push-down made queries up to 8 times slower when the window reads a `JOIN` and the filter removes every row, for example `IN` over an empty subquery, because the join built its right side before the filter could stop the query. Closes [#123875](https://github.com/ClickHouse/ClickHouse/issues/123875). [#125463](https://github.com/ClickHouse/ClickHouse/pull/125463) ([Groene AI](https://github.com/groeneai)). * `hash` and `parallel_hash` joins now copy right-side output columns directly from the stored blocks instead of calling the stored column object once per row. This speeds up every column type, including `String`, `Array`, `Tuple`, `Nullable`, `Variant`, `Map`, `LowCardinality`, `JSON`, `Dynamic`, `AggregateFunction`, and `QBit`. [#117460](https://github.com/ClickHouse/ClickHouse/pull/117460) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * Speed up hash join output construction in the row-major layout: output reconstruction is up to 2x faster for joins with many rows per key and about 1.5x for wide or `Nullable` payloads. The row-major layout is now also skipped for payloads too narrow to benefit from it. [#121981](https://github.com/ClickHouse/ClickHouse/pull/121981) ([Hechem Selmi](https://github.com/m-selmi)). * Hash joins with many matches per key are faster when they output a `LowCardinality` column whose blocks have different dictionaries (several parts, `FINAL`, or a `LowCardinality` value computed per block): short ranges from another dictionary are now inserted key by key instead of through temporary columns. Such joins run 4-9x faster, at the 26.6 level or better. [#123900](https://github.com/ClickHouse/ClickHouse/pull/123900) ([Groene AI](https://github.com/groeneai)). * `RIGHT` and `FULL` joins using `hash`, `parallel_hash`, or `grace_hash` skip the search for unmatched right rows when every right key has matched. With 50 million right keys, that search was about 10% of query time for `hash` (356 ms of 3.6 s) and about 15% for `parallel_hash` (106 ms of 0.7 s). [#124775](https://github.com/ClickHouse/ClickHouse/pull/124775) ([Groene AI](https://github.com/groeneai)). * A `full_sorting_merge` join now advertises that its result is ordered by the join key, so a sort above it, the pre-sort of the next merge join in a chain or an `ORDER BY`, merges the already sorted streams instead of sorting from scratch. Joining several tables on the same key with `full_sorting_merge` no longer performs one full sort per join. Also fixed the interplay with `query_plan_join_shard_by_pk_ranges`, which could otherwise fail with a logical error or return wrong results for such a chain. [#117786](https://github.com/ClickHouse/ClickHouse/pull/117786) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A join step that has a condition of its own now also joins on equalities implied through other tables, so for `a.x = b.x AND b.x = c.x`, joining `a` with `c` also uses `a.x = c.x`. With `query_plan_optimize_join_order_algorithm = 'dpsize'`, this makes TPC-H Q02 join its correlated subquery to `partsupp` on the part key instead of on the supply cost alone. Closes [#122396](https://github.com/ClickHouse/ClickHouse/issues/122396). [#122445](https://github.com/ClickHouse/ClickHouse/pull/122445) ([Groene AI](https://github.com/groeneai)). * A `WHERE` condition that no row satisfies is pushed to every `JOIN` input again, including the non-preserved side of an `OUTER JOIN` and a side with no column used above the join, so that input is no longer read and hash-built for nothing. For example, a `UNION ALL` branch whose constant column fails the outer `WHERE` is skipped as in 26.6. Closes [#123702](https://github.com/ClickHouse/ClickHouse/issues/123702). Closes [#123878](https://github.com/ClickHouse/ClickHouse/issues/123878). [#123829](https://github.com/ClickHouse/ClickHouse/pull/123829) ([Groene AI](https://github.com/groeneai)). * Constant comparison, logical, and null (`IS NULL` / `IS NOT NULL`) filters over materialized constants are now folded and pushed down, reducing unnecessary reads in `WHERE`, `PREWHERE`, and `UNION` branches. Always-true `WHERE` filters are replaced with expressions so join reordering still sees all joined tables as one graph. [#105987](https://github.com/ClickHouse/ClickHouse/pull/105987) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#124056](https://github.com/ClickHouse/ClickHouse/pull/124056) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `OUTER JOIN` queries can now be faster: when a `WHERE` clause or another `JOIN` later in the query would discard the unmatched rows anyway, ClickHouse converts the join to an `INNER JOIN`. The new setting `query_plan_convert_outer_join_to_inner_join_transitively` controls this, and only takes effect when `query_plan_convert_outer_join_to_inner_join` is enabled. [#120253](https://github.com/ClickHouse/ClickHouse/pull/120253) ([Hechem Selmi](https://github.com/m-selmi)). * Restored filter push-down over `ANY INNER JOIN`, so primary-key pruning works again and such queries no longer read much more data than necessary. Because `ANY` keeps an arbitrary matching row, `SELECT ... ANY INNER JOIN ... WHERE ...` can now return a different matching row than before. [#120765](https://github.com/ClickHouse/ClickHouse/pull/120765) ([Kirill Shokhin](https://github.com/kirillshokhin)). * `optimize_and_compare_chain` again derives transitive conditions through a `DateTime` or lower-scale `DateTime64` constant compared with a `DateTime64` expression, as it did before 26.8. For example, `t1.ts < toDateTime('2026-09-16 00:00:00') AND t1.ts >= t2.ts` adds `t2.ts < toDateTime('2026-09-16 00:00:00')`, so the other side of a join can be pruned by its partition and primary key instead of being read in full. Closes [#123706](https://github.com/ClickHouse/ClickHouse/issues/123706). [#123796](https://github.com/ClickHouse/ClickHouse/pull/123796) ([Groene AI](https://github.com/groeneai)). * Improve join order selection when column statistics are missing: an equi join without NDV statistics is now treated more like a foreign-key-to-primary-key join instead of a cross product, and the greedy join-order algorithm prefers equi-connected pairs over pairs linked only by non-equi predicates. [#120176](https://github.com/ClickHouse/ClickHouse/pull/120176) ([Vladimir Cherkasov](https://github.com/vdimir)). [#121075](https://github.com/ClickHouse/ClickHouse/pull/121075) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improves `JOIN` cardinality estimation by propagating join-key distinct-value estimates according to join semantics. Outer joins retain the preserved-side estimate, anti joins keep their input estimates, and semi joins narrow only the output side, preventing underestimation that could lead to suboptimal query plans. [#122305](https://github.com/ClickHouse/ClickHouse/pull/122305) ([Christoph Viebig](https://github.com/cv4g)). * `IEJoin` now treats opposite-direction conditions on unrelated columns as independent conditions instead of as a band, which avoids choosing a join-key pair that passes thousands of times more rows to the residual condition. [#120213](https://github.com/ClickHouse/ClickHouse/pull/120213) ([Vladimir Cherkasov](https://github.com/vdimir)). * Function `firstNonDefault`, which computes the `USING` columns of `FULL JOIN`, now copies whole ranges of rows instead of making two virtual calls per row. Queries that use a `USING` column of a `FULL JOIN` after the join, for example in `WHERE`, are faster. [#123302](https://github.com/ClickHouse/ClickHouse/pull/123302) ([Groene AI](https://github.com/groeneai)). * Enabled `enable_join_runtime_filters_index_analysis` by default, so join runtime filters can prune granules during the probe phase. [#121859](https://github.com/ClickHouse/ClickHouse/pull/121859) ([Robert Schulze](https://github.com/rschu1ze)). * Join runtime filters now run after the query's own `PREWHERE` conditions also for tables without column statistics, as they already did with statistics. Before, a runtime filter on a well-compressed join key could run first and read that key for every row, so enabling `enable_join_runtime_filters` could make a query read several times more data than without it. [#122452](https://github.com/ClickHouse/ClickHouse/pull/122452) ([Groene AI](https://github.com/groeneai)). * The granule pruning of `enable_join_runtime_filters_index_analysis` now also applies when the left side of a `JOIN` is read from a normal projection. The setting description states that the pruning does not happen with parallel replicas, a distributed query plan, or `FINAL`. [#119041](https://github.com/ClickHouse/ClickHouse/pull/119041) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The query condition cache now records the static `WHERE` conditions of a join whose `PREWHERE` also holds a join runtime filter, so a repeated join skips the granules those conditions exclude instead of reading them again. This helps whenever the runtime filter cannot skip those granules through an index itself, for example when no primary key or skip index covers the join key, or when `enable_join_runtime_filters_index_analysis` is disabled. [#122380](https://github.com/ClickHouse/ClickHouse/pull/122380) ([Groene AI](https://github.com/groeneai)). * Repeated insertion of empty and single-element `Array` values into in-memory columns is now faster. This happens, for example, during joins. [#117321](https://github.com/ClickHouse/ClickHouse/pull/117321) ([Minh Vu](https://github.com/fallintoplace)). * With `use_top_k_dynamic_filtering = 1` (the default), a `SELECT ... ORDER BY ... LIMIT n` could read far more data than with the setting off: the internal `__topKFilter` took over the read's `PREWHERE` slot and displaced the query's own condition, and an explicit `PREWHERE` disabled the feature outright. The threshold filter now shares `PREWHERE` with the other conditions and takes the first slot. It is also no longer installed on a read that a sorting projection already delivers in `ORDER BY` order, where it forced a full scan. [#117968](https://github.com/ClickHouse/ClickHouse/pull/117968) ([Groene AI](https://github.com/groeneai)). * Granules fully filtered by the dynamic threshold of an `ORDER BY ... LIMIT n` read are now recorded in the query condition cache, so repeat runs of such queries skip them at the mark-selection stage instead of re-reading and re-filtering most of the table. On the ClickBench Q24/Q26 shapes (`SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime LIMIT 10`), a warm run drops from about 95 million rows / 12000 granules read to about 1.3 million rows / 160 granules, and from about 15-30 ms to about 8 ms on a 96-core machine. This now also works for `ORDER BY ... LIMIT n` queries over `Parquet` files of `File` tables, with or without a `WHERE` clause: a repeated query reads only the row groups which can hold rows of its result. [#114659](https://github.com/ClickHouse/ClickHouse/pull/114659) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122064](https://github.com/ClickHouse/ClickHouse/pull/122064) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve performance of `ORDER BY ... LIMIT` queries that use minmax-index top-K pruning (`use_skip_indexes_for_top_k`) on large tables with many granules. [#116512](https://github.com/ClickHouse/ClickHouse/pull/116512) ([Anton Popov](https://github.com/CurtizJ)). * Queries with `FINAL` that sort in reverse order of the sorting key, such as `SELECT ... FROM t FINAL ORDER BY key DESC LIMIT n`, now read in order through `Merge` tables over `ReplacingMergeTree` tables too, and the `topKThroughJoin` optimization defers to reverse-order reads below a `LEFT JOIN` when the table supports them. This extends [#115345](https://github.com/ClickHouse/ClickHouse/pull/115345). [#118008](https://github.com/ClickHouse/ClickHouse/pull/118008) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a performance regression in `ORDER BY` merge sort: queries with a sort followed by `LIMIT` no longer do redundant per-output-block work in result-block reservation. [#108179](https://github.com/ClickHouse/ClickHouse/pull/108179) ([Groene AI](https://github.com/groeneai)). * Setting `optimize_move_to_prewhere_if_final` is now enabled by default, so queries with `FINAL` benefit from the automatic `PREWHERE` optimization. Only deterministic conditions over the sorting key are moved, and query results are not affected. [#121350](https://github.com/ClickHouse/ClickHouse/pull/121350) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * With short-circuit evaluation, non-throwing arguments of `and` and `or` are now ordered by measured cost and selectivity, so cheap filters can avoid evaluating expensive expressions on excluded rows. Controlled by `short_circuit_function_evaluation_reorder_arguments`, enabled by default. [#122508](https://github.com/ClickHouse/ClickHouse/pull/122508) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a performance regression in short-circuit evaluation of `and` and `or` by avoiding expansion of intermediate results that are only used to update the logical mask. [#120520](https://github.com/ClickHouse/ClickHouse/pull/120520) ([LB7666](https://github.com/acking-you)). * On a table with no applicable data skipping index, a query no longer stores the same predicate under two query condition cache keys. This halves the number of `O(marks per part)` cache entries such a query allocates, and removes one cache lookup per part. Queries with `distributed_index_analysis` enabled keep both keys. [#116259](https://github.com/ClickHouse/ClickHouse/pull/116259) ([Groene AI](https://github.com/groeneai)). * `SELECT ... FINAL` with a selective skip-index plan is much faster on tables with many granules: the `use_skip_indexes_if_final_exact_mode` recovery pass now iterates ranges instead of individual granules. [#121799](https://github.com/ClickHouse/ClickHouse/pull/121799) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Improve primary index analysis in queries that filter both by the primary key and by other columns, for example `WHERE timestamp BETWEEN ... AND level = 'error'` where `level` is not in the primary key. [#123535](https://github.com/ClickHouse/ClickHouse/pull/123535) ([Anton Popov](https://github.com/CurtizJ)). * `IS NULL` and `IS NOT NULL` predicates can now prune `MergeTree` parts using the `NULL` row count tracked by `basic` statistics on `Nullable` columns. [#116697](https://github.com/ClickHouse/ClickHouse/pull/116697) ([zoomxi](https://github.com/zoomxi)). * A `set` skip index on a `Nullable` column again skips granules whose stored set contains `NULL` but not the searched value for `IN` and `=` conditions, restoring the pruning lost in 26.8. Closes [#123705](https://github.com/ClickHouse/ClickHouse/issues/123705). [#123778](https://github.com/ClickHouse/ClickHouse/pull/123778) ([Groene AI](https://github.com/groeneai)). * Skip indexes (`text`, `bloom_filter`, `tokenbf_v1`, `ngrambf_v1`, `minmax`) and the primary key are now used when the indexed column or the searched constant is wrapped in a conversion that does not change its value, such as `toLowCardinality`, `toNullable`, or a `CAST` to `Nullable` or `LowCardinality`. For example, `m['key'] = 'value'` on `Map(String, LowCardinality(String))` and `hasToken(s, toLowCardinality('x'))` no longer scan all granules. `bloom_filter` and text indexes also work when a `Nullable` indexed column is cast to the non-`Nullable` type, such as `CAST(x, 'String')` or `json.path::String`. [#120220](https://github.com/ClickHouse/ClickHouse/pull/120220) ([Anton Popov](https://github.com/CurtizJ)). [#123239](https://github.com/ClickHouse/ClickHouse/pull/123239) ([Anton Popov](https://github.com/CurtizJ)). [#124220](https://github.com/ClickHouse/ClickHouse/pull/124220) ([Anton Popov](https://github.com/CurtizJ)). * Speed up planning of queries with skip indexes on tables with many parts and pending patch parts of lightweight updates. [#124931](https://github.com/ClickHouse/ClickHouse/pull/124931) ([Anton Popov](https://github.com/CurtizJ)). * Speed up queries with multiple `PREWHERE` conditions on AMD Zen 1 and Zen 2 CPUs. [#120283](https://github.com/ClickHouse/ClickHouse/pull/120283) ([Manuel](https://github.com/raimannma)). * Analysis of a query over a `Merge` table no longer traverses every child table for each query-tree node when `optimize_functions_to_subcolumns` is enabled. The answer is now reused across the query tree, so analysis time grows with the query size plus the number of child tables instead of their product. Closes [#113202](https://github.com/ClickHouse/ClickHouse/issues/113202). [#113367](https://github.com/ClickHouse/ClickHouse/pull/113367) ([Groene AI](https://github.com/groeneai)). * Queries to the `merge` table function and creation of `Merge` tables are faster when the source tables have many columns: combining the source columns no longer takes quadratic time in the number of columns. [#124115](https://github.com/ClickHouse/ClickHouse/pull/124115) ([Groene AI](https://github.com/groeneai)). * Faster analysis of queries to tables with many `ALIAS` columns, such as `system.metric_log`: the expression of an `ALIAS` column is analyzed only if the query uses that column. [#123926](https://github.com/ClickHouse/ClickHouse/pull/123926) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduce aggregate projection analysis overhead for tables with many subcolumns. [#110979](https://github.com/ClickHouse/ClickHouse/pull/110979) ([Rory Shanks](https://github.com/rorylshanks)). * A condition such as `name IN ('a', 'b')` on the system tables that enumerate tables (`system.tables`, `system.columns`, `system.parts`, `system.replicas`, and others) now narrows the enumeration instead of walking every table of every database, the same way an equality condition already did. This is most visible for databases whose table list comes from a remote system, such as `PostgreSQL`, `SQLite`, and the DataLake catalogs. [#117589](https://github.com/ClickHouse/ClickHouse/pull/117589) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `external_storage_push_down_limit` setting to reduce data read from external databases for safe `LIMIT` queries. Also fixed wrong results when `LIMIT` plus `OFFSET` does not fit in `UInt64`: `numbers`, `system.numbers`, `generate_series`, `primes`, `system.primes`, and the `PostgreSQL` table engine returned no rows for the common "no limit" idiom `LIMIT 18446744073709551615 OFFSET n`, and a `LIMIT` over a `JOIN` with `numbers`-like sources returned too few rows. [#115238](https://github.com/ClickHouse/ClickHouse/pull/115238) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118131](https://github.com/ClickHouse/ClickHouse/pull/118131) ([Groene AI](https://github.com/groeneai)). * Push filters into the query fragment executed by parallel replicas so replicas can filter rows and prune data before sending results to the initiator. This also allows join runtime filters to reach the relevant reads. Controlled by `parallel_replicas_filter_pushdown`, now enabled by default; `SET parallel_replicas_filter_pushdown = 0` restores the behavior before 26.10. [#117373](https://github.com/ClickHouse/ClickHouse/pull/117373) ([Nikita Taranov](https://github.com/nickitat)). * With parallel replicas, `optimize_aggregation_in_order` no longer disables projections: aggregate and normal projections, and `_minmax_count_projection` for `count()` with a partition predicate, are used as with `optimize_aggregation_in_order = 0`. Closes [#123061](https://github.com/ClickHouse/ClickHouse/issues/123061). [#123098](https://github.com/ClickHouse/ClickHouse/pull/123098) ([Groene AI](https://github.com/groeneai)). * Parallel replicas now apply `enable_join_runtime_filters_index_analysis` to prune probe-side granules locally on each replica, reducing `JOIN` read work. [#123116](https://github.com/ClickHouse/ClickHouse/pull/123116) ([Nikita Taranov](https://github.com/nickitat)). * Automatic parallel replicas no longer distributes a query when most of its reading would be repeated in full by every replica rather than split between them. The limit is set by the new setting `automatic_parallel_replicas_max_duplicated_read_ratio` (default `0.5`; `1` disables it). [#123332](https://github.com/ClickHouse/ClickHouse/pull/123332) ([Nikita Taranov](https://github.com/nickitat)). * Automatic parallel replicas for multi-read queries now reuse local plan index analysis in the parallel-replicas plan, reducing extra reads when parallel replicas are enabled. [#115153](https://github.com/ClickHouse/ClickHouse/pull/115153) ([Nikita Taranov](https://github.com/nickitat)). * Improve reuse of collected query statistics by automatic parallel replicas for equivalent local and distributed plans. [#116781](https://github.com/ClickHouse/ClickHouse/pull/116781) ([Nikita Taranov](https://github.com/nickitat)). * Automatic parallel replicas now account correctly for the size of aggregated results on more aggregation paths, so the cost model no longer treats shipping aggregate states as free and is less likely to choose parallel replicas when a query is cheaper locally. [#119499](https://github.com/ClickHouse/ClickHouse/pull/119499) ([Nikita Taranov](https://github.com/nickitat)). * Automatic parallel replicas now collect planning statistics for the table read that is actually split between replicas, including after join reordering and with lazy materialization, improving the decision to distribute a query. [#122906](https://github.com/ClickHouse/ClickHouse/pull/122906) ([Nikita Taranov](https://github.com/nickitat)). * Avoid building the parallel replicas candidate plan when cached statistics already show the query should stay on a single replica. [#124983](https://github.com/ClickHouse/ClickHouse/pull/124983) ([Nikita Taranov](https://github.com/nickitat)). * Added quick checks to skip automatic parallel replicas optimization for queries unsupported by parallel replicas, including keyless aggregations whose aggregate state owns memory and therefore never produces the probe statistics the optimization needs. [#120861](https://github.com/ClickHouse/ClickHouse/pull/120861) ([Nikita Taranov](https://github.com/nickitat)). [#125470](https://github.com/ClickHouse/ClickHouse/pull/125470) ([Nikita Taranov](https://github.com/nickitat)). * Automatic parallel replicas (`automatic_parallel_replicas_mode`) can now be applied to queries with `ORDER BY ... WITH FILL`. [#123929](https://github.com/ClickHouse/ClickHouse/pull/123929) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries that search for several tokens at once with a text index, such as `hasAllTokens`, are now faster. The automatic choice of algorithm for intersecting posting lists is improved too, and is controlled by the new setting `text_index_postings_intersection_algorithm`. [#121038](https://github.com/ClickHouse/ClickHouse/pull/121038) ([Anton Popov](https://github.com/CurtizJ)). * Queries that use a text index read large posting lists faster and with fewer disk operations. The improvement is most noticeable for searches for frequent tokens in big tables. [#121040](https://github.com/ClickHouse/ClickHouse/pull/121040) ([Anton Popov](https://github.com/CurtizJ)). * A query that searches a text index and also selects a column no index covers no longer reads that column for every row of every granule the index selected. On a 1M-row table with a 1055-byte payload and 64 matching rows, `SELECT ts, payload FROM t WHERE hasToken(message, 'needle')` now reads 578 KiB instead of 532 MiB. [#120517](https://github.com/ClickHouse/ClickHouse/pull/120517) ([Groene AI](https://github.com/groeneai)). * A query that reads a `text` index no longer asks the filesystem for each index substream's on-disk name and size once per part; both are now taken from the part's checksums, which already hold them. On an 8M-row table split into 400 parts, `SELECT count() FROM logs WHERE hasToken(message, 'needle')` drops from a median 12 ms to 9 ms, with 42% less system time. [#120875](https://github.com/ClickHouse/ClickHouse/pull/120875) ([Groene AI](https://github.com/groeneai)). * Speed up queries that filter by a text index together with other `PREWHERE` conditions, such as the dynamic filter of `ORDER BY ... LIMIT` queries. [#124880](https://github.com/ClickHouse/ClickHouse/pull/124880) ([Anton Popov](https://github.com/CurtizJ)). * Speed up `LIKE`, `ILIKE`, `startsWith`, and `endsWith` evaluated by a text-index dictionary scan (`use_text_index_like_evaluation_by_dictionary_scan`): the scan now seeks the sorted dictionary to the block range an anchored pattern can match, and tests a block's tokens for the pattern's mandatory literal in one pass instead of running the matcher on every token. [#120495](https://github.com/ClickHouse/ClickHouse/pull/120495) ([Groene AI](https://github.com/groeneai)). * Repeated broad `LIKE` and `ILIKE` queries over a text index no longer rescan the index dictionary of every part only to abandon the scan again: the decision is cached (setting `use_text_index_pattern_bypass_cache`). [#113572](https://github.com/ClickHouse/ClickHouse/pull/113572) ([Rory Shanks](https://github.com/rorylshanks)). * A text index with the `splitByNonAlpha` or `array` tokenizer now answers single-needle `multiSearchAny`, `multiSearchAnyUTF8`, `multiSearchAnyCaseInsensitive`, and `multiSearchAnyCaseInsensitiveUTF8` with the same dictionary scan that `LIKE '%needle%'` uses, so a needle inside a word, such as `PointerExc` in `NullPointerException`, no longer reads the whole column. [#124520](https://github.com/ClickHouse/ClickHouse/pull/124520) ([Groene AI](https://github.com/groeneai)). * `multiSearchAny` over a map element can now use a `mapValues(...)` ngram or token text index instead of being rejected by index analysis. [#121787](https://github.com/ClickHouse/ClickHouse/pull/121787) ([Groene AI](https://github.com/groeneai)). * Optimize `SELECT count()` queries answered from a text index with a compressed posting list codec (`bitpacking`, `pfor`). [#123294](https://github.com/ClickHouse/ClickHouse/pull/123294) ([Anton Popov](https://github.com/CurtizJ)). * Reduced the CPU overhead of reading and intersecting text index posting lists in lazy apply mode. [#123786](https://github.com/ClickHouse/ClickHouse/pull/123786) ([Anton Popov](https://github.com/CurtizJ)). [#123873](https://github.com/ClickHouse/ClickHouse/pull/123873) ([Anton Popov](https://github.com/CurtizJ)). * Merges of tables with a `text` index are now faster. Pages of a source part's row ids that hold consecutive values are stored as a first value plus a count, so they are neither bit-packed nor decoded per posting row id and need no storage for packed values. [#120982](https://github.com/ClickHouse/ClickHouse/pull/120982) ([Groene AI](https://github.com/groeneai)). * Merging positional text indexes is faster: position lists are now merged segment by segment instead of being materialized and sorted in a temporary `roaringish` structure. [#121976](https://github.com/ClickHouse/ClickHouse/pull/121976) ([Elmi Ahmadov](https://github.com/ahmadov)). * Added the `pfor` codec for text-index posting lists. It improves compression ratio and decoding speed. Enable it per index with `posting_list_codec = 'pfor'`, or for all text indexes of a table with the `text_index_posting_list_codec` setting. [#118123](https://github.com/ClickHouse/ClickHouse/pull/118123) ([Elmi Ahmadov](https://github.com/ahmadov)). * Building text indexes during `INSERT` is now faster, especially for indexes with the `bitpacking` codec. [#113856](https://github.com/ClickHouse/ClickHouse/pull/113856) ([Anton Popov](https://github.com/CurtizJ)). * Building a text index over a `LowCardinality(String)` or `Array(LowCardinality(String))` column is faster: each distinct value is tokenized once per batch of rows instead of at every occurrence. `MATERIALIZE INDEX` over a column with 50 distinct values takes about 33% less time. [#122409](https://github.com/ClickHouse/ClickHouse/pull/122409) ([Groene AI](https://github.com/groeneai)). * Improve performance of the `ngrams` tokenizer. This speeds up `tokens(s, 'ngrams', N)`, `ngrams(s, N)`, `hasAnyTokens`, `hasAllTokens`, and `hasPhrase` with the `ngrams` tokenizer, as well as inserts into tables with an `ngrambf_v1` or `text(tokenizer = ngrams(N))` index. `tokens(URL, 'ngrams', 3)` over ClickBench URLs takes 26% (aarch64) to 36% (x86_64) less time. [#122328](https://github.com/ClickHouse/ClickHouse/pull/122328) ([Groene AI](https://github.com/groeneai)). * Reading columns and subcolumns of `MergeTree` tables no longer rebuilds the serialization of a column for every reader: the serialization the data part already holds is reused. The gain is largest for `JSON` columns with many typed paths, where the whole set of path serializations used to be rebuilt per read task and per filter step. [#120632](https://github.com/ClickHouse/ClickHouse/pull/120632) ([Pavel Kruglov](https://github.com/Avogar)). * Reduce per-block overhead when reading `MergeTree` parts with small blocks: column-stream names and substream-cache keys are no longer rebuilt for every block, and reads skip patch-part handling when there are none, virtual-column name lookups, and some small-column reallocations. This makes queries with a low `max_block_size` faster. [#124816](https://github.com/ClickHouse/ClickHouse/pull/124816) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124817](https://github.com/ClickHouse/ClickHouse/pull/124817) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduce CPU overhead of reads from `MergeTree` tables by not formatting a tracing span name for every block. [#124820](https://github.com/ClickHouse/ClickHouse/pull/124820) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading many columns from Compact parts is up to 2% faster by avoiding per-call copies of column metadata. [#122840](https://github.com/ClickHouse/ClickHouse/pull/122840) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Restore parallel prefix deserialization when reading an `Object` or `JSON` column from a Wide part: marks loading no longer queues behind one mutex. Closes [#114136](https://github.com/ClickHouse/ClickHouse/issues/114136). [#114252](https://github.com/ClickHouse/ClickHouse/pull/114252) ([Groene AI](https://github.com/groeneai)). * Queries that read subcolumns of `JSON` columns with many typed paths are faster: resolving a subcolumn by name visits only the typed paths that can match it. For a `JSON` column with 500 typed paths, reading subcolumns from 96 parts takes about half the time. [#122029](https://github.com/ClickHouse/ClickHouse/pull/122029) ([Rory Shanks](https://github.com/rorylshanks)). * Reduce serialization construction when reading `JSON` subcolumns from `MergeTree` parts by reusing safe cached subcolumn serializations. [#110957](https://github.com/ClickHouse/ClickHouse/pull/110957) ([Rory Shanks](https://github.com/rorylshanks)). * Improve nested `JSON` path read performance with `map_with_buckets` by reusing thread-local serializations for `Dynamic` values. [#113344](https://github.com/ClickHouse/ClickHouse/pull/113344) ([Rory Shanks](https://github.com/rorylshanks)). * Writing `JSON` columns with many paths in shared data is faster: the writer no longer rebuilds the statistics of the column for every block. For merges of parts with small granules, this saves up to 13% of the merge time. [#121987](https://github.com/ClickHouse/ClickHouse/pull/121987) ([Rory Shanks](https://github.com/rorylshanks)). * Make constructing a `JSON` data type that uses `SKIP REGEXP` cheaper. The patterns are now compiled once per type instead of once for every declared path, so a column declaring 183 typed paths and 6 `SKIP REGEXP` entries performs 6 regular expression compilations where it previously performed 1098. This matters most for wide `JSON` schemas, where the patterns are often long alternations that are expensive to compile. [#122941](https://github.com/ClickHouse/ClickHouse/pull/122941) ([arc1999](https://github.com/arc1999)). * Read only the matching `Map` subcolumn for safe `LIKE` predicates over `Map` values. [#119809](https://github.com/ClickHouse/ClickHouse/pull/119809) ([Minh Vu](https://github.com/fallintoplace)). * Improve `SELECT` performance by avoiding lock contention when getting mutation snapshots with many concurrent background fetches. [#102382](https://github.com/ClickHouse/ClickHouse/pull/102382) ([c-end](https://github.com/c-end)). * Support `PREWHERE` (including the automatic move of `WHERE` conditions by `optimize_move_to_prewhere`) for `Memory` tables: only the columns of the conditions are read at first, and the remaining columns are read only for the blocks where some rows pass, and only for the passing rows. This is especially beneficial for tables with `SETTINGS compress = true`, because for a selective condition most columns are never decompressed. Additionally, `SELECT count() FROM table` on a `Memory` table is now served from metadata, and `system.columns` shows real per-column sizes for `Memory` tables. [#116248](https://github.com/ClickHouse/ClickHouse/pull/116248) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `PREWHERE` over a `Memory` table is evaluated in multiple steps, like for `MergeTree` (`enable_multiple_prewhere_read_steps`): the columns of each next condition are read only for the rows that passed the previous ones. For a table with `compress = true`, a condition on a wide column next to a selective one no longer decompresses that column for every block, which makes such queries up to several times faster. [#122604](https://github.com/ClickHouse/ClickHouse/pull/122604) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries with `ORDER BY ... LIMIT n` over `Memory` tables use TopN dynamic filtering: rows that cannot get into the result are dropped while reading, before the other columns are read or decompressed with `compress = true`. [#123787](https://github.com/ClickHouse/ClickHouse/pull/123787) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries whose only aggregate functions are `min` and `max` of columns of `Parquet` files read with the `file` table function or the `File` engine, without `GROUP BY` or filters (for example ClickBench Q7), now take the values from the column chunk statistics instead of reading the data. Controlled by the new setting `optimize_min_max_from_files`. [#124201](https://github.com/ClickHouse/ClickHouse/pull/124201) ([Groene AI](https://github.com/groeneai)). * `ORDER BY ... LIMIT` over a single `Parquet` file reads first the row groups whose min/max statistics are best for the result, so the top-K threshold skips the other row groups: `SELECT URL FROM file('hits.parquet') ORDER BY UserID DESC LIMIT 10` skips 207 of its 226 row groups at 4 threads. New profile event `ParquetTopKSkippedRowGroups`. [#123639](https://github.com/ClickHouse/ClickHouse/pull/123639) ([Groene AI](https://github.com/groeneai)). * Row group pruning by dictionaries in the `Parquet` reader is faster for an equality or a short `IN` list on a column with large dictionaries, such as `WHERE UserID = ...` over ClickBench `hits.parquet`: the hashes of the dictionary values are no longer sorted unless a column chunk is checked against many values. [#123579](https://github.com/ClickHouse/ClickHouse/pull/123579) ([Groene AI](https://github.com/groeneai)). * `Parquet` readers now check a column chunk's bloom filter before reading its dictionary page. A row group that the bloom filter rules out no longer reads the much larger dictionary page to reach the same conclusion, so `input_format_parquet_dictionary_filter_push_down` costs nothing on files that carry both, while pruning as much as before. [#120907](https://github.com/ClickHouse/ClickHouse/pull/120907) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `Parquet` footer-size reads are now tuned by storage type: remote and object-storage reads use more optimistic footer-size estimates to fetch the footer in one read instead of two, reducing footer-read latency, while local-file reads again use a 64 KiB tail so short queries over many local `Parquet` files read less data. [#115686](https://github.com/ClickHouse/ClickHouse/pull/115686) ([UnamedRus](https://github.com/UnamedRus)). [#124883](https://github.com/ClickHouse/ClickHouse/pull/124883) ([Groene AI](https://github.com/groeneai)). * Avoid a redundant object-storage GET request when reading random-access formats from object storage. The generic from-start read-ahead is no longer issued for files larger than a single read buffer, where it could not reach the footer. Also, `input_format_allow_seeks = 0` is now honored by the `Parquet` reader, which previously sought regardless of the setting: with seeks disabled the whole object is read sequentially instead, which can raise memory usage for queries that set it. [#115681](https://github.com/ClickHouse/ClickHouse/pull/115681) ([UnamedRus](https://github.com/UnamedRus)). * Speed up writing `Parquet` bloom filters: instead of deduplicating the value hashes in a hash set to determine the filter size, the writer now builds a filter sized for the number of values and folds it down to the size implied by `output_format_parquet_bloom_filter_bits_per_value`. This removes the worst-case slowdown on data that is adversarial for the hash set, and lowers the worst-case memory usage. [#119230](https://github.com/ClickHouse/ClickHouse/pull/119230) ([Armin Braun](https://github.com/original-brownbear)). * Parse some GeoParquet metadata lazily. [#122693](https://github.com/ClickHouse/ClickHouse/pull/122693) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Snappy decompression on AArch64 now uses NEON for short-offset copies, as upstream snappy does by default, which makes reading Snappy-compressed `Parquet` with repetitive string columns up to ~9% faster on ARM servers. The ARMv8.0 compatibility build (`NO_ARMV81_OR_HIGHER`) is unchanged. [#124608](https://github.com/ClickHouse/ClickHouse/pull/124608) ([Groene AI](https://github.com/groeneai)). * Reading `CSV` with `input_format_csv_trim_whitespaces = 0` no longer prevents the parser from scaling across CPU cores. Each field used to copy a `shared_ptr` shared by all parsing threads, turning its reference counter into a contention point. On a 96-core machine, reading 200 million rows became 4.6 times faster and used 23 times less CPU. [#117251](https://github.com/ClickHouse/ClickHouse/pull/117251) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `Iceberg` partition pruning now uses the partition value of a `day`, `month`, `year`, or `hour` transform for a filter that wraps the partition source column in a monotonic function, such as `WHERE toDate(ts) = '2024-01-20'`, including for partition values from before 1970. Closes [#103433](https://github.com/ClickHouse/ClickHouse/issues/103433). [#118606](https://github.com/ClickHouse/ClickHouse/pull/118606) ([Konstantin Vedernikov](https://github.com/scanhex12)). [#119302](https://github.com/ClickHouse/ClickHouse/pull/119302) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Faster planning of queries to `Iceberg` tables with many manifest files: when a manifest file header has the `schema-id` key and that schema is already known from `metadata.json`, the schema copy stored in the manifest header is no longer parsed and compared with it. ClickHouse now also writes the `schema-id` key into the headers of manifest files it creates. [#122574](https://github.com/ClickHouse/ClickHouse/pull/122574) ([Konstantin Vedernikov](https://github.com/scanhex12)). * An `INSERT` into an `Iceberg` table that loses the commit race now reuses its manifests on retry and rewrites only the manifest list and the metadata file, instead of regenerating all manifests. [#122096](https://github.com/ClickHouse/ClickHouse/pull/122096) ([Asya Shneerson](https://github.com/asya-ch)). * Restore the summary-based trivial `count()` optimization for `Iceberg` tables when the snapshot summary contains a safe row count. The optimization is skipped when equality deletes are present, because their recorded count is not the number of deleted rows. [#123267](https://github.com/ClickHouse/ClickHouse/pull/123267) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `Iceberg` min/max hyperrectangles are now built only for columns the filter uses. [#122681](https://github.com/ClickHouse/ClickHouse/pull/122681) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Restore min/max pruning of `Iceberg` data files for a `Decimal(38, 38)` column. A manifest bound of magnitude above roughly 0.7 could not be widened inside `Int128` and was dropped, leaving the column without a min/max condition; it is now saturated at the largest magnitude the precision allows, which is still outside every value the file can hold. [#119712](https://github.com/ClickHouse/ClickHouse/pull/119712) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries to `system.iceberg_history` with filters on `table` now avoid listing unrelated data lake catalog namespaces. [#113867](https://github.com/ClickHouse/ClickHouse/pull/113867) ([Den Kalantaevskii](https://github.com/Diskein)). * Improve HTTP upload and download efficiency for `S3`, `Azure`, the `url` table function and engine, dictionaries, and data lake catalogs. Uploads use 1 MiB chunks instead of 8 KiB chunks, reducing the number of system calls for a 15 MB body from 1817 to 15 and avoiding redundant copies. Reads from `Azure` also avoid an intermediate 8 KiB buffer. [#116222](https://github.com/ClickHouse/ClickHouse/pull/116222) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading from object storage with `remote_filesystem_read_method = 'read'` now keeps full-size network reads. Before, a short read at the end of one range capped every later read of the same file, which could make a scan hundreds of times slower, for example a compact part on an encrypted `S3` disk without the filesystem cache. [#123777](https://github.com/ClickHouse/ClickHouse/pull/123777) ([Groene AI](https://github.com/groeneai)). * Reads that are served from the OS page cache no longer consume the budget of the local read bandwidth throttler (`max_local_read_bandwidth_for_server` and `max_local_read_bandwidth`), so a query over cached data is not slowed down to the configured disk bandwidth. This works with the default `local_filesystem_read_method = 'pread_threadpool'`, with `pread`, and for the files of the filesystem cache; the `read` method cannot detect cached reads and is accounted as before. [#118495](https://github.com/ClickHouse/ClickHouse/pull/118495) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Populating the filesystem cache on a cache miss is faster: the writer now reserves space ahead instead of taking the cache lock on every write. [#123255](https://github.com/ClickHouse/ClickHouse/pull/123255) ([Kseniia Sumarokova](https://github.com/kssenii)). * Loading the metadata of a large `s3_plain_rewritable` disk is several times faster. Previously it made one `ListObjects` request per directory and enumerated the `__meta` directory by a single paginated listing, so a disk with tens of thousands of directories needed tens of thousands of sequential-ish requests. The listings are now split by key prefix and run in parallel, and the files of all directories are collected by a few listings of the whole disk. For a disk with 36 thousand directories holding half a million objects, the load time dropped from 116 to 37 seconds. [#121412](https://github.com/ClickHouse/ClickHouse/pull/121412) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Metadata transactions of unrelated tables on a `plain_rewritable` disk no longer wait for each other: the disk-wide lock used to be held across all object storage requests of a commit, so one long `MOVE PARTITION`, `DROP TABLE`, or part removal blocked every merge, insert, and `CREATE TABLE` on the same disk. The lock is now taken per affected path, and the in-memory metadata is updated by replaying the recorded changes. Also fixes a permanent leak of objects when the server was killed during `DROP TABLE` (or another removal) on a `plain_rewritable` disk: the leftovers of such a removal are now marked and deleted when the disk metadata is loaded on the next start. [#118490](https://github.com/ClickHouse/ClickHouse/pull/118490) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removing data parts from a disk with `metadata_type = plain_rewritable` (cleanup of outdated parts, `TRUNCATE`, `DROP PARTITION`, `DROP TABLE`) no longer copies every file of the part to a temporary object before deleting it, and no longer holds the disk metadata lock for those copies. The part directory is removed with one batched delete request. Closes [#123275](https://github.com/ClickHouse/ClickHouse/issues/123275). [#123303](https://github.com/ClickHouse/ClickHouse/pull/123303) ([Groene AI](https://github.com/groeneai)). * Reduce metadata update cost and lock contention for `s3_plain_rewritable` disks with many parts by sharing unchanged directory entries between snapshots. [#119625](https://github.com/ClickHouse/ClickHouse/pull/119625) ([Rory Shanks](https://github.com/rorylshanks)). * Avoid object reads for unchanged directories when a plain-rewritable disk refreshes its metadata. [#119760](https://github.com/ClickHouse/ClickHouse/pull/119760) ([Rory Shanks](https://github.com/rorylshanks)). * `OPTIMIZE TABLE ... FINAL` on a non-replicated `MergeTree` table now assigns and runs the merges for all partitions at once instead of processing them one partition at a time. [#109004](https://github.com/ClickHouse/ClickHouse/pull/109004) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `TTL DELETE` merges can now use the vertical merge algorithm on `ReplacingMergeTree`, `CollapsingMergeTree`, and `VersionedCollapsingMergeTree`, where `vertical_merge_optimize_ttl_delete` previously applied only to plain `MergeTree`. [#119886](https://github.com/ClickHouse/ClickHouse/pull/119886) ([Murphy](https://github.com/murphy-4o)). * Allow vertical merges to produce `Wide` parts with `Packed` storage instead of forcing the horizontal merge algorithm. [#125760](https://github.com/ClickHouse/ClickHouse/pull/125760) ([alesapin](https://github.com/alesapin)). * Improved merge performance for `SummingMergeTree` tables by skipping unnecessary aggregate-column materialization. [#120852](https://github.com/ClickHouse/ClickHouse/pull/120852) ([Eduard Karacharov](https://github.com/korowa)). * Many queued lightweight deletes with an `IN` condition that reach the same part in one mutation are now applied together, instead of evaluating each `IN` predicate over the whole part separately. Controlled by the new `MergeTree` setting `enable_row_mask_update_coalescing`. [#122718](https://github.com/ClickHouse/ClickHouse/pull/122718) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Optimize the insert backpressure path in `MergeTree` by short-circuiting the parts-count check. [#114754](https://github.com/ClickHouse/ClickHouse/pull/114754) ([johnjing](https://github.com/JingYanchao)). * Reduced merge-selection work when all storage volumes allow merges. [#119782](https://github.com/ClickHouse/ClickHouse/pull/119782) ([Rory Shanks](https://github.com/rorylshanks)). * Reduce merge-selection CPU overhead for tables with column TTLs by avoiding copies of TTL descriptions. [#119644](https://github.com/ClickHouse/ClickHouse/pull/119644) ([Rory Shanks](https://github.com/rorylshanks)). * Speed up decompression of codec chains. The first codec now reads the compressed block in place and the last codec writes the result in place, unless that codec requires a padded buffer (`LZ4`, `ZXC`). [#120043](https://github.com/ClickHouse/ClickHouse/pull/120043) ([Raufs Dunamalijevs](https://github.com/rienath)). * Faster decompression of `T64`-encoded columns. [#120414](https://github.com/ClickHouse/ClickHouse/pull/120414) ([Raufs Dunamalijevs](https://github.com/rienath)). [#122442](https://github.com/ClickHouse/ClickHouse/pull/122442) ([Raufs Dunamalijevs](https://github.com/rienath)). * A query with `arrayJoin` and a small `LIMIT` again shrinks the read block size instead of reading whole blocks on every thread for a few output rows, fixing a regression from 26.6. Closes [#119193](https://github.com/ClickHouse/ClickHouse/issues/119193). [#120214](https://github.com/ClickHouse/ClickHouse/pull/120214) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Chained `ARRAY JOIN`s no longer materialize the whole block of a lazily replicated array column before expanding it, which cuts their peak memory by several times. [#121397](https://github.com/ClickHouse/ClickHouse/pull/121397) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Avoid copying whole columns when slicing them, including in `ARRAY JOIN`. [#118788](https://github.com/ClickHouse/ClickHouse/pull/118788) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Improve `ARRAY JOIN` performance for queries with wide columns that are not expanded, by avoiding unnecessary copies and preserving lazy replication through filtering and sorting. [#124727](https://github.com/ClickHouse/ClickHouse/pull/124727) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Reduced the peak memory usage of `RESTORE` and of incremental `BACKUP`: the metadata file of a backup is now parsed while it is read, instead of being read into memory whole first. For backups of millions of files this saves gigabytes. [#125712](https://github.com/ClickHouse/ClickHouse/pull/125712) ([Julia Kartseva](https://github.com/jkartseva)). * Faster startup of `clickhouse-local` and other tools. Time zone lookup tables are built 2.4 to 5.6 times faster and use half the memory (2.3 MiB instead of 4.6 MiB per loaded time zone). The `clickhouse` binary also starts faster when it is not in the page cache, with a 1.40x speedup measured for `clickhouse local`. [#125665](https://github.com/ClickHouse/ClickHouse/pull/125665) ([Raúl Marín](https://github.com/Algunenano)). [#125672](https://github.com/ClickHouse/ClickHouse/pull/125672) ([Raúl Marín](https://github.com/Algunenano)). * A full scan of a complex-key `hashed`, `sparse_hashed`, `hashed_array`, or `range_hashed` dictionary (`SELECT ... FROM ` or the `dictionary` table function) is faster: each reading stream now deserializes only the keys of its own block instead of one thread deserializing all keys before reading starts. A scan with `LIMIT` deserializes only the blocks it reads. A full scan also no longer keeps all deserialized keys in memory at once. [#123883](https://github.com/ClickHouse/ClickHouse/pull/123883) ([Groene AI](https://github.com/groeneai)). * Added the `SIEVE` cache eviction policy for in-memory caches such as the mark cache and the uncompressed cache. Enable it with the corresponding `*_cache_policy` server settings, for example `mark_cache_policy` or `uncompressed_cache_policy`. It is not available for the filesystem cache. [#123435](https://github.com/ClickHouse/ClickHouse/pull/123435) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The symbolized stack trace of a failed query is now built only when the error message will be logged or the trace will be stored in `system.query_log`. Previously each failing `clickhouse-local` invocation built the symbol index of the whole binary to produce a string that was then discarded. [#120660](https://github.com/ClickHouse/ClickHouse/pull/120660) ([Groene AI](https://github.com/groeneai)). * Reduces the CPU overhead of the workload scheduler thread by updating the `system.scheduler` introspection metrics more cheaply. A saturated scheduler can now serve about 15% more resource requests per second. [#121958](https://github.com/ClickHouse/ClickHouse/pull/121958) ([Sergei Trifonov](https://github.com/serxa)). * PromQL now copies label matchers across binary operators for the labels the operator matches series by, so `b / on(job) a{job="x"}` reads only the matching series of `b`. As in Prometheus, duplicate left series of a one-to-one match in a group the right side does not have no longer fail the query. The new setting `promql_push_down_label_matchers` turns this off. [#122972](https://github.com/ClickHouse/ClickHouse/pull/122972) ([Valery Petrov](https://github.com/valerypetrov)). [#125267](https://github.com/ClickHouse/ClickHouse/pull/125267) ([Groene AI](https://github.com/groeneai)). * Accelerate exact PromQL label matching on `TimeSeries` tables with a text index on the `tags` map. [#120083](https://github.com/ClickHouse/ClickHouse/pull/120083) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Use more compact codecs for generated `timestamp` and `value` columns in the samples and recent samples tables of the `TimeSeries` engine: `Delta`, `T64`, and `ZSTD(3)` for `timestamp`, and `ALP` plus `ZSTD(3)` for `value`. [#120454](https://github.com/ClickHouse/ClickHouse/pull/120454) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Reduce memory usage of the `timeSeries*ToGrid` aggregate functions. [#124773](https://github.com/ClickHouse/ClickHouse/pull/124773) ([Valery Petrov](https://github.com/valerypetrov)). * Filtering and sorting are now faster: the helpers that build filter masks, count filter bytes, and check for an identity permutation use AVX2 or AVX-512 on x86 instead of SSE2, and are vectorized on ARM where they were not. [#121403](https://github.com/ClickHouse/ClickHouse/pull/121403) ([Raúl Marín](https://github.com/Algunenano)). * Faster filtering of numeric columns on ARM (up to 1.28x) and on AMD Zen 4 (up to 2.8x), speeding up queries with `WHERE` and multi-step `PREWHERE`. [#122893](https://github.com/ClickHouse/ClickHouse/pull/122893) ([Raúl Marín](https://github.com/Algunenano)). * Filters on numeric columns that keep few rows are faster. [#124024](https://github.com/ClickHouse/ClickHouse/pull/124024) ([Raúl Marín](https://github.com/Algunenano)). * Filtering numeric columns no longer over-allocates the result buffer (by up to 2x) when the size of the result is known in advance. Closes [#123546](https://github.com/ClickHouse/ClickHouse/issues/123546). [#123576](https://github.com/ClickHouse/ClickHouse/pull/123576) ([Raúl Marín](https://github.com/Algunenano)). * Vectorized comparisons of `UUID`, `IPv4`, and `IPv6` columns with columns or constants of the same type, which makes such comparisons several times faster. Comparisons of `FixedString(16)` values are now branchless too, making them faster when it is unpredictable whether the values are equal. [#120193](https://github.com/ClickHouse/ClickHouse/pull/120193) ([Manuel](https://github.com/raimannma)). * Comparisons `<`, `>`, `<=`, and `>=` of `IPv6` and `FixedString(16)` values are 1.4 to 1.8 times faster on CPUs with AVX-512. [#125151](https://github.com/ClickHouse/ClickHouse/pull/125151) ([Groene AI](https://github.com/groeneai)). * `LIKE`, `ILIKE`, `match`, and their variants are faster when the pattern is not a constant, up to 5 times with 8 threads. [#122764](https://github.com/ClickHouse/ClickHouse/pull/122764) ([Groene AI](https://github.com/groeneai)). [#122843](https://github.com/ClickHouse/ClickHouse/pull/122843) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Faster `multiMatchAny` and related functions with non-constant needles when `reject_expensive_hyperscan_regexps` is disabled. [#122842](https://github.com/ClickHouse/ClickHouse/pull/122842) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * `CAST` of strings to `Nullable(Bool)` (also `accurateCastOrNull(s, 'Bool')` and `toBool` of a `Nullable(String)`) is up to 7 times faster with 8 threads, and `arrayIntersect`, `arrayUnion`, `arraySymmetricDifference`, and `arrayJaccardIndex` over non-numeric elements such as `DateTime64`, `UUID`, or `Tuple` are up to 18 times faster with 8 threads: they no longer allocate memory and update profile events for every row. `multiMatchAny` and the other multi-regexp functions with non-constant needles now add their cache hit and miss counts once per block. [#122920](https://github.com/ClickHouse/ClickHouse/pull/122920) ([Groene AI](https://github.com/groeneai)). * Speed up extracting a value from a `Map` with `LowCardinality` keys. The key is now resolved to its dictionary position once instead of being compared against every entry of a row, so the cost of `m[key]` no longer grows with the position of the key in the row. Also fix extracting a value from a `Map` whose key is `NaN`, or holds a `NULL` inside a composite key: such a key used to compare equal to every other key. [#120453](https://github.com/ClickHouse/ClickHouse/pull/120453) ([Pavel Kruglov](https://github.com/Avogar)). * `L2DistanceTransposed`, `cosineDistanceTransposed`, `dotProductTransposed`, and their `...Quantized` variants with precision `1` are now computed directly from the packed sign bit plane with per-byte lookup tables, 2 to 3 times faster than before. The new setting `qbit_one_bit_symmetric_distance` additionally reduces the reference vector to its signs and derives the result from the Hamming distance between the two sign vectors, computed with XOR and popcount, which is up to 10 times faster. [#120594](https://github.com/ClickHouse/ClickHouse/pull/120594) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up `randomHadamardTransform` and vector quantization for vector dimensions of the form 2^k × 12 or 2^k × 20, such as 768 or 1536, on x86 with AVX2: 1.9x to 2.5x higher throughput and 50% to 60% lower median latency. [#120874](https://github.com/ClickHouse/ClickHouse/pull/120874) ([Jingxin (Philip) Li](https://github.com/philip-jingxin)). * `hasAll` and `hasAny` on integer arrays use SIMD on both x86 and ARM, including `Nullable` arrays and `hasAny`, which previously used a scalar loop. `hasAny` is up to 15x faster, and `hasAll` becomes several times faster on ARM. [#123420](https://github.com/ClickHouse/ClickHouse/pull/123420) ([Groene AI](https://github.com/groeneai)). * `hasAny` over `Array(String)` looks each element up in a hash set, instead of comparing it with every string, when its second argument is a constant array of four or more strings of up to 256 bytes and a block has enough rows to pay for building the set. With 32 strings this uses 87% less CPU. Closes [#122333](https://github.com/ClickHouse/ClickHouse/issues/122333). [#122507](https://github.com/ClickHouse/ClickHouse/pull/122507) ([Groene AI](https://github.com/groeneai)). * Vectorize `least` and `greatest` for `Nullable` numeric arguments and for three or more numeric arguments, and `clamp` for numeric arguments. These cases were previously evaluated row-by-row and are now 3-7 times faster. [#109783](https://github.com/ClickHouse/ClickHouse/pull/109783) ([Manuel](https://github.com/raimannma)). * Speed up `round`, `roundBankers`, `floor`, `ceil`, and `trunc` with a scale argument on `Float32` and `Float64` columns by using AVX2 on x86 instead of SSE4.1, and speed up rounding of integers narrower than 64 bits on ARM by vectorizing it. [#122032](https://github.com/ClickHouse/ClickHouse/pull/122032) ([Raúl Marín](https://github.com/Algunenano)). * `modulo` and `moduloOrNull` of two unsigned integers of different widths are faster again, as before 26.9, when the wider one is `UInt128` (about 3 to 4 times, for example `UInt128 % UInt16`) or the dividend is `UInt256` (1.6 to 2 times). [#123170](https://github.com/ClickHouse/ClickHouse/pull/123170) ([Groene AI](https://github.com/groeneai)). * `exp10` is up to 2x faster by splitting its argument with `trunc` instead of the `modf` library call, with bitwise-identical results. [#122263](https://github.com/ClickHouse/ClickHouse/pull/122263) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Speed up `entropy` and multi-argument `uniqExact` on short strings by optimizing `SipHash` for short inputs. Hash values are unchanged. [#123658](https://github.com/ClickHouse/ClickHouse/pull/123658) ([Groene AI](https://github.com/groeneai)). * `sumMap`, `minMap`, `maxMap`, `sumMapWithOverflow`, and `sumMapFiltered` over arrays are up to 2.3 times faster for integer, `Enum`, `Date`, `Date32`, and `DateTime` keys. [#124340](https://github.com/ClickHouse/ClickHouse/pull/124340) ([Groene AI](https://github.com/groeneai)). * `h3PolygonToCells` and `h3PolygonToCellsWithContainment` are several times faster on polygons much smaller than a cell (9x on the query from the issue). This brings `h3PolygonToCells` back to the speed it had on such polygons before 26.8.12 and 26.9. Closes [#123877](https://github.com/ClickHouse/ClickHouse/issues/123877). [#123911](https://github.com/ClickHouse/ClickHouse/pull/123911) ([Groene AI](https://github.com/groeneai)). * `isValidUTF8` validates a `String` or `FixedString` column with one `simdutf` call over the whole column instead of one call per row, which makes it up to 2.8 times faster on short strings. [#123835](https://github.com/ClickHouse/ClickHouse/pull/123835) ([Groene AI](https://github.com/groeneai)). * Speed up `lowerUTF8` and `upperUTF8` on text whose non-ASCII characters are mostly two-byte UTF-8 (`U+0080` to `U+07FF`, e.g. Latin-1, Greek, Cyrillic): the column is processed in one pass, these characters are mapped from a table built from ICU, and ICU is called only for the rest of a row that contains other characters. [#123867](https://github.com/ClickHouse/ClickHouse/pull/123867) ([Groene AI](https://github.com/groeneai)). * `lower`, `upper`, and the ASCII path of `lowerUTF8` and `upperUTF8` are now faster by using AVX2 or AVX-512 depending on the CPU. [#121374](https://github.com/ClickHouse/ClickHouse/pull/121374) ([Manuel](https://github.com/raimannma)). * Speed up `lengthUTF8` and other `UTF-8` functions on long strings by counting code points 64 bytes at a time, on both x86 and ARM. [#122049](https://github.com/ClickHouse/ClickHouse/pull/122049) ([Raúl Marín](https://github.com/Algunenano)). * Fixed ARM performance regressions in `indexOf` with a constant needle and in text output formats over non-ASCII strings. Closes [#123338](https://github.com/ClickHouse/ClickHouse/issues/123338). Closes [#123346](https://github.com/ClickHouse/ClickHouse/issues/123346). [#123377](https://github.com/ClickHouse/ClickHouse/pull/123377) ([Raúl Marín](https://github.com/Algunenano)). * UTF-8 string functions such as `substringUTF8` are faster on long ASCII strings on CPUs with AVX-512. [#120088](https://github.com/ClickHouse/ClickHouse/pull/120088) ([Minh Vu](https://github.com/fallintoplace)). * Improved performance of functions `extractKeyValuePairs` and `extractKeyValuePairsWithEscaping`. [#119361](https://github.com/ClickHouse/ClickHouse/pull/119361) ([Anton Popov](https://github.com/CurtizJ)). * Function `mortonEncode` now resolves the type of each argument and the ratios of a constant range mask once per block instead of once per row, which makes it up to 2x faster. [#123354](https://github.com/ClickHouse/ClickHouse/pull/123354) ([Groene AI](https://github.com/groeneai)). * Speed up `mortonDecode` on AArch64 by decoding with compile-time shift-and-mask sequences instead of a lookup table. The output is unchanged on every architecture. [#116205](https://github.com/ClickHouse/ClickHouse/pull/116205) ([Groene AI](https://github.com/groeneai)). * `encrypt` and `aes_encrypt_mysql` are faster on values shorter than 16 bytes in the `aes-*-ecb` and `aes-*-cbc` modes (up to 2x for `encrypt` with `aes-128-ecb`). [#124103](https://github.com/ClickHouse/ClickHouse/pull/124103) ([Groene AI](https://github.com/groeneai)). * Speed up `hex` and `unhex` with optimized `AVX` and `AVX2` implementations when they are available. [#123511](https://github.com/ClickHouse/ClickHouse/pull/123511) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Faster `toMillisecond`, `toMicrosecond`, and `toNanosecond` on `DateTime64`. [#122852](https://github.com/ClickHouse/ClickHouse/pull/122852) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Speed up `reverse` for `String` and `FixedString` values. [#119462](https://github.com/ClickHouse/ClickHouse/pull/119462) ([Minh Vu](https://github.com/fallintoplace)). * `arrayDistinct` is faster: it now does a single hash-table lookup per element instead of two, and `Decimal` arrays use a specialized fast path. [#113966](https://github.com/ClickHouse/ClickHouse/pull/113966) ([Manuel](https://github.com/raimannma)). [#121646](https://github.com/ClickHouse/ClickHouse/pull/121646) ([Minh Vu](https://github.com/fallintoplace)). * Specialize `has` and `indexOf` searches in non-nullable numeric arrays when the needle is constant. [#119090](https://github.com/ClickHouse/ClickHouse/pull/119090) ([Minh Vu](https://github.com/fallintoplace)). * `arrayFilter` is faster when the lambda keeps all or none of the elements of a block. [#119816](https://github.com/ClickHouse/ClickHouse/pull/119816) ([Minh Vu](https://github.com/fallintoplace)). * Improve performance of `bitmapAndCardinality`, `bitmapOrCardinality`, `bitmapXorCardinality`, `bitmapAndnotCardinality`, and `bitmapHasAny` for `UInt64` and `Int64` bitmaps, which previously had to materialize the intersection to compute the result. On AArch64, `bitmapAndCardinality`, `bitmapOrCardinality`, `bitmapXorCardinality`, and `bitmapAndnotCardinality` no longer hit a rare code-layout-dependent slow path. [#120615](https://github.com/ClickHouse/ClickHouse/pull/120615) ([Ashish kumar choubey](https://github.com/aschoube)). [#123675](https://github.com/ClickHouse/ClickHouse/pull/123675) ([Groene AI](https://github.com/groeneai)). * Restored the existing `Decimal` fast paths in `arrayCompact`. [#121245](https://github.com/ClickHouse/ClickHouse/pull/121245) ([Minh Vu](https://github.com/fallintoplace)). * `uniqUpTo` now stops hashing after it exceeds its threshold. [#121247](https://github.com/ClickHouse/ClickHouse/pull/121247) ([Minh Vu](https://github.com/fallintoplace)). * Restored the `FixedString` fast path for `arrayEnumerateUniq` and `arrayEnumerateDense`. [#121338](https://github.com/ClickHouse/ClickHouse/pull/121338) ([Minh Vu](https://github.com/fallintoplace)). * `arrayReverse` is now faster for `Decimal` and wide integer arrays. [#121342](https://github.com/ClickHouse/ClickHouse/pull/121342) ([Minh Vu](https://github.com/fallintoplace)). [#121644](https://github.com/ClickHouse/ClickHouse/pull/121644) ([Minh Vu](https://github.com/fallintoplace)). * Avoid copying `String` columns for `substring(s, 1)`. [#119820](https://github.com/ClickHouse/ClickHouse/pull/119820) ([Minh Vu](https://github.com/fallintoplace)). * Skip KMP preprocessing for impossible `hasSubstr` searches. [#119824](https://github.com/ClickHouse/ClickHouse/pull/119824) ([Minh Vu](https://github.com/fallintoplace)). * Speed up `arrayMin` and `arrayMax` for `Decimal` and `DateTime64` arrays. [#123507](https://github.com/ClickHouse/ClickHouse/pull/123507) ([Minh Vu](https://github.com/fallintoplace)). * Speed up `arrayUniq` for `Decimal32`, `Decimal64`, and `Decimal128` arrays. [#122121](https://github.com/ClickHouse/ClickHouse/pull/122121) ([Minh Vu](https://github.com/fallintoplace)). * Allow a single condition to constrain multiple key columns during index analysis. For example, if the primary key is `(toYYYYMMDD(dt), toDate(dt), dt)` and the query has `WHERE dt >= toDateTime('2026-01-10 00:00:00')`, all three key columns are now used for pruning, instead of only one of them as before. This behavior is controlled by the setting `analyze_index_with_multiple_key_columns_per_condition` (enabled by default). [#96225](https://github.com/ClickHouse/ClickHouse/pull/96225) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Left-side columns absent from a hash join's output (for example the join key in `SELECT count() ... LEFT JOIN ...`) are now pruned when only a spill threshold is set, as with the default `max_bytes_ratio_before_external_join`, instead of being carried through the pipeline. [#108191](https://github.com/ClickHouse/ClickHouse/pull/108191) ([Groene AI](https://github.com/groeneai)). * Joins on null-safe comparison (`<=>`) of nullable keys use fixed-width hash keys instead of a serialized tuple. [#120596](https://github.com/ClickHouse/ClickHouse/pull/120596) ([Manuel](https://github.com/raimannma)). * Fixed a quadratic slowdown of `JOIN` with `join_algorithm = 'hash'` when `ON` has a non-equality condition or several `OR` conditions. [#121209](https://github.com/ClickHouse/ClickHouse/pull/121209) ([nt](https://github.com/namtran1812)). * Faster parsing of floating-point numbers from text formats. Update `fast_float` to v8.3.2. [#121383](https://github.com/ClickHouse/ClickHouse/pull/121383) ([Actuele AI](https://github.com/actueleai)). * Query plan optimizations can now use the sort order reported by any table engine, not only `MergeTree`, avoiding unnecessary sorting. [#121858](https://github.com/ClickHouse/ClickHouse/pull/121858) ([Ivan Zhukov](https://github.com/zhkvia)). * Slightly reduce the per-block overhead of aggregation by not copying the columns of the block when preparing the aggregate function instructions. [#124819](https://github.com/ClickHouse/ClickHouse/pull/124819) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `GROUP BY` by several keys, which requires the serialization of keys, serializes the keys of a block in chunks that fit in the CPU cache instead of all at once. This makes such queries faster when many threads are used, for example, ClickBench Q18 is 37% faster on a 96-core machine. [#124955](https://github.com/ClickHouse/ClickHouse/pull/124955) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the setting `cluster_function_wait_for_replica_ms` (default `0`, disabled), which lets cluster table functions wait for the replica assigned a file to connect before giving that file to another replica. This improves file distribution when replicas start at different times. Work stealing now takes files from the active replica with the most remaining bytes without scanning all unprocessed files. [#125611](https://github.com/ClickHouse/ClickHouse/pull/125611) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The `SLRU` filesystem cache policy now moves a partially downloaded file segment to the protected queue on a cache hit, as it does for a fully downloaded one. Before, a scan could evict the data of narrow reads from the probationary queue. [#125657](https://github.com/ClickHouse/ClickHouse/pull/125657) ([Sema Checherinda](https://github.com/CheSema)). #### Improvement * Lightweight updates (the `UPDATE` statement and patch parts) are now generally available instead of Beta. The `enable_lightweight_update` setting is now in the production feature tier. [#125719](https://github.com/ClickHouse/ClickHouse/pull/125719) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Cluster discovery is no longer experimental: clusters with a `discovery` section in `remote_servers` work without enabling `allow_experimental_cluster_discovery`. The setting is now obsolete. [#123917](https://github.com/ClickHouse/ClickHouse/pull/123917) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The Kusto (KQL) and Trino query dialects were promoted from experimental to beta. They are still enabled with the `allow_experimental_kusto_dialect` and `enable_trino_dialect` settings. [#124750](https://github.com/ClickHouse/ClickHouse/pull/124750) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Tables and databases can now be addressed through URL paths in the HTTP interface by default: `curl http://localhost:8123/my_db/my_table.csv` now works without extra configuration. The settings `http_allow_path_requests`, `http_allow_database_as_path`, `http_allow_table_as_file`, and `http_allow_filters_as_path` are enabled by default; `http_allow_filters_as_unrecognized_url_parameters` stays disabled. [#121143](https://github.com/ClickHouse/ClickHouse/pull/121143) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `all_...` union tables over the system log tables (`system.all_query_log`, `system.all_text_log`, and so on) are now created by default: the default server configuration enables the `create_union_system_log_tables` section with `merge_rotated_tables`, so queries over the rotated versions of a log no longer have to remember constructs like `merge(system, '^query_log')`. The `cluster` setting is shown as a commented-out example, because the table definitions would reference the cluster and require it to exist. [#117943](https://github.com/ClickHouse/ClickHouse/pull/117943) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In a `DataLakeCatalog` database with `allow_experimental_iceberg_compaction = 1`, only `Iceberg` tables are now cached. `DeltaLake` and `Paimon` tables no longer keep the first schema they saw, so `SELECT *` and `DESCRIBE` pick up added columns without recreating the database or restarting the server. Closes [#122548](https://github.com/ClickHouse/ClickHouse/issues/122548). [#122568](https://github.com/ClickHouse/ClickHouse/pull/122568) ([Groene AI](https://github.com/groeneai)). * The background memory worker now corrects the global memory tracker from real memory usage on every tick (server setting `memory_worker_correct_memory_tracker`, enabled by default). Previously the tracker was a pure counter that was re-baselined only at startup and when it went negative, so upward accounting drift was permanent: servers were observed refusing every query with `MEMORY_LIMIT_EXCEEDED`, reporting more than 100 GiB in use while actually using 7 GiB, until restart. The drift itself remains visible as the difference between the `MemoryTrackingUncorrected` and `MemoryTracking` metrics. [#118094](https://github.com/ClickHouse/ClickHouse/pull/118094) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The AI function per-query quotas are now disabled by default: `ai_function_max_input_tokens_per_query`, `ai_function_max_output_tokens_per_query`, and `ai_function_max_api_calls_per_query` default to `0`, meaning no limit. To keep the previous caps, set the three settings explicitly or use `compatibility = '26.9'`. [#121424](https://github.com/ClickHouse/ClickHouse/pull/121424) ([George Larionov](https://github.com/george-larionov)). * Named `Tuple` columns in `Pretty` formats are now displayed as subcolumns, with the element names shown in additional header and footer lines. Controlled by the new setting `output_format_pretty_named_tuples_as_subcolumns` (enabled by default). A named tuple rendered inside one cell, for example nested in an `Array` or when the setting is disabled, is still displayed according to `output_format_pretty_named_tuples_as_json`. [#78927](https://github.com/ClickHouse/ClickHouse/pull/78927) ([Engel Danila](https://github.com/aaaengel)). * Show the query result in `Pretty` formats as soon as it arrives. Previously a query that returns a few rows early and then keeps reading, such as `SELECT DISTINCT` over a large table, displayed nothing until it finished or was cancelled. A later fix prevents a background write error after the query has finished from surfacing as a late `LOGICAL_ERROR`. [#117005](https://github.com/ClickHouse/ClickHouse/pull/117005) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#125028](https://github.com/ClickHouse/ClickHouse/pull/125028) ([Groene AI](https://github.com/groeneai)). * Display non-printable control characters as Unicode "Control Pictures" in the `Vertical` and `Pretty*` output formats by default, so they are no longer silently swallowed and no longer deform the table. `TAB`, the line feed, and `ESC` are excluded, because a terminal interprets them rather than swallowing them. The new setting `output_format_pretty_display_control_characters` can be disabled to restore raw-byte output. [#101774](https://github.com/ClickHouse/ClickHouse/pull/101774) ([Yash ](https://github.com/Onyx2406)). * Expired authentication methods are now removed from a user automatically. Every `ALTER USER` drops the authentication methods whose `VALID UNTIL` deadline has already passed and keeps all the others, so short-lived credentials can be rotated indefinitely with `ALTER USER ... ADD IDENTIFIED WITH ... VALID UNTIL ...` without the expired ones accumulating against `max_authentication_methods_per_user`. A method added by the same statement is not affected, and a user whose every method is expired keeps them rather than being left with no way to authenticate. [#118880](https://github.com/ClickHouse/ClickHouse/pull/118880) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * On `ReplicatedMergeTree`, the sync value `3` (`mutations_sync`, `lightweight_deletes_sync`, `alter_sync`) now waits for the active replicas, as documented, instead of all of them, so an inactive replica no longer causes `UNFINISHED`. This also affects `OPTIMIZE`, `TRUNCATE`, `DROP`/`DETACH PARTITION`, `REPLACE PARTITION`, and `MOVE PARTITION TO TABLE`. [#120592](https://github.com/ClickHouse/ClickHouse/pull/120592) ([Nikolay Degterinsky](https://github.com/evillique)). * A number written directly as the first argument of `CAST` is now read with the target type when that type reads it more precisely than the number itself carries it, such as a `Decimal` or an integer wider than 64 bits. `Nullable`, `Array`, and `LowCardinality` of those types take the same exact path. [#116043](https://github.com/ClickHouse/ClickHouse/pull/116043) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `TTL ... GROUP BY (a, b, c)` is now accepted and means the same as `TTL ... GROUP BY a, b, c`, matching how `ORDER BY (a, b, c)` works. Previously the parenthesized form was rejected with `TTL Expression GROUP BY key should be a prefix of primary key`. [#118426](https://github.com/ClickHouse/ClickHouse/pull/118426) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `sequenceMatch`, `sequenceCount`, and `sequenceMatchEvents` now accept a `DateTime64` timestamp instead of failing. Events are ordered at the column's full precision, so two events within the same second are no longer ambiguous. [#118427](https://github.com/ClickHouse/ClickHouse/pull/118427) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `argMin` and `argMax` now consistently return the value from the first row among rows with the same minimum or maximum within a block, matching `argMinIf` and `argMaxIf`. [#122892](https://github.com/ClickHouse/ClickHouse/pull/122892) ([Raúl Marín](https://github.com/Algunenano)). * Added the setting `analyzer_compatibility_allow_cte_redefinition`. When enabled, a CTE name may be defined more than once in a single `WITH` clause; a later definition shadows the earlier one as the query analysis before 24.3 did. `MATERIALIZED` CTEs and CTEs in `WITH RECURSIVE` cannot be redefined. [#121623](https://github.com/ClickHouse/ClickHouse/pull/121623) ([Dmitry Novik](https://github.com/novikd)). * Added settings `semi_join_include_columns_from_both_sides` and `anti_join_include_columns_from_both_sides` (enabled by default) for `SEMI` and `ANTI` joins in the analyzer. When enabled, behavior is unchanged: columns from both sides are accessible after the join. Disable either setting to enforce SQL-standard semantics, where only the preserved side's columns are visible after the join. [#96662](https://github.com/ClickHouse/ClickHouse/pull/96662) ([Zhang Yifan](https://github.com/zhangyifan27)). * The setting `aggregate_function_input_format` with the values `value` and `array` now works in every input format, including `RowBinary`, `Native`, and `Parquet`, and for `AggregateFunction` nested in `Array`, `Tuple`, and `Map`. In the JSON formats the value is now a JSON value, for example `{"x": [1, 2, 3]}`, not a string containing it. [#121120](https://github.com/ClickHouse/ClickHouse/pull/121120) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Compression codec names and their keyword arguments are now matched case-insensitively, so `CODEC(zstd)` works like `CODEC(ZSTD)` and `ALP(std)` like `ALP(STD)`. Previously a codec whose name is mixed-case, such as `Delta`, `DoubleDelta`, `Gorilla` or `Quantized`, could not be used in the `default_compression_codec`, `marks_compression_codec` and `primary_key_compression_codec` settings, nor in the `` section of the server configuration, because those paths upper-cased the codec name. [#119313](https://github.com/ClickHouse/ClickHouse/pull/119313) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added aliases `allow_suspicious_indexes`, `ignore_data_skipping_indexes`, `force_data_skipping_indexes`, and `secondary_indexes_enable_bulk_filtering` for the settings spelled with `indices`, so both plural forms are accepted. [#122565](https://github.com/ClickHouse/ClickHouse/pull/122565) ([Mohammed Sohail](https://github.com/msohail22)). * `APPLY (f, 'prefix')` now keeps the table qualifier in generated column names. [#122702](https://github.com/ClickHouse/ClickHouse/pull/122702) ([Groene AI](https://github.com/groeneai)). * Async insert now supports eligible single-block local `INSERT ... SELECT` queries, with synchronous fallback for larger, multi-block, empty, transactional, or non-parallel-quorum inserts. [#111663](https://github.com/ClickHouse/ClickHouse/pull/111663) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Added the opt-in `optimize_row_order_if_no_order_by` setting for `MergeTree` tables with an empty sorting key (`ORDER BY ()` / `ORDER BY tuple()`). When enabled, it applies row-order optimization automatically to inserts into such tables. [#104591](https://github.com/ClickHouse/ClickHouse/pull/104591) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `WHATIF` can now estimate hypothetical projections without reading the whole table: past `max_rows_to_scan` rows it estimates from a sample of granules and reports the resulting range in `marks_span`, `max_rows_to_read` lowers that budget, and it follows `(prefer|force)_optimize_projection`. It now also uses the real projection-optimizer weighing logic, so its verdicts, reasons, and pruned ranges track the optimizer more closely. [#118233](https://github.com/ClickHouse/ClickHouse/pull/118233) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#121766](https://github.com/ClickHouse/ClickHouse/pull/121766) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#122060](https://github.com/ClickHouse/ClickHouse/pull/122060) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#123540](https://github.com/ClickHouse/ClickHouse/pull/123540) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#123587](https://github.com/ClickHouse/ClickHouse/pull/123587) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Added the setting `parallel_replicas_for_queries_with_multiple_tables` (default `true`) to control whether parallel replicas are used for queries joining multiple tables. When disabled, parallel replicas are not used for queries with `JOIN`, where the non-leftmost side is read in full on every replica. The setting does not affect a `UNION` query without a `JOIN`, where each branch is an independent single-table read, nor `ARRAY JOIN`. [#76595](https://github.com/ClickHouse/ClickHouse/pull/76595) ([Igor Nikonov](https://github.com/devcrafter)). * `max_joined_block_size_rows` and `max_joined_block_size_bytes` now also bound the result blocks of `parallel_hash` and `grace_hash` joins: squashing of the join output no longer merges blocks past these limits. Closes [#124198](https://github.com/ClickHouse/ClickHouse/issues/124198). [#124304](https://github.com/ClickHouse/ClickHouse/pull/124304) ([Groene AI](https://github.com/groeneai)). * Added the setting `query_plan_filter_push_down_below_limit_by` (default `1`), controlling whether filters on `LIMIT BY` key columns are pushed below the `LIMIT BY` step. Setting it to `0` restores the plan shape used before 26.7, for queries where the pushed predicate makes index analysis costlier than the scan it saves. [#121675](https://github.com/ClickHouse/ClickHouse/pull/121675) ([Groene AI](https://github.com/groeneai)). * Added the `use_aggregation_memory_tracker` setting to choose how `GROUP BY` measures memory when deciding on two-level aggregation. `compatibility` below `26.9` restores the measurement used before 26.7. [#124439](https://github.com/ClickHouse/ClickHouse/pull/124439) ([Hechem Selmi](https://github.com/m-selmi)). * `system.query_log` and `system.processes` now report the metrics of each task of a distributed query plan (`make_distributed_plan = 1`) in a row of its own: a task is a secondary query with its own `query_id`, correlated to the statement that started it by `initial_query_id`. The initiator's row reports its own work instead of the volume its tasks read. [#120949](https://github.com/ClickHouse/ClickHouse/pull/120949) ([Groene AI](https://github.com/groeneai)). * `system.query_log` now records the databases, tables, and columns read by a replica that executes a query plan shipped to it, which happens with `parallel_replicas_plan_based`. [#122657](https://github.com/ClickHouse/ClickHouse/pull/122657) ([Igor Nikonov](https://github.com/devcrafter)). * Queries with `send_logs_level` now also forward text logs from stateless `make_distributed_plan` workers through the coordinator, so worker-side exception context reaches the client. [#118603](https://github.com/ClickHouse/ClickHouse/pull/118603) ([Antonio Filipovic](https://github.com/antoniofilipovic)). * The setting `parallel_replicas_insert_select_local_pipeline` is obsolete. It was a workaround for a performance problem fixed in 25.5, and it duplicated `parallel_replicas_local_plan`, which already decides whether the initiator reads its own share of a distributed `INSERT SELECT`. Use `parallel_replicas_local_plan = 0` to leave all the reading to the remote replicas. [#121653](https://github.com/ClickHouse/ClickHouse/pull/121653) ([Igor Nikonov](https://github.com/devcrafter)). * Distributed `SELECT` queries now emit one OpenTelemetry span per remote shard read, with attributes such as `clickhouse.cluster`, `clickhouse.shard_num`, `clickhouse.processed_stage`, `clickhouse.query_id`, `clickhouse.initial_query_id`, and `clickhouse.target_host`, so slow or failed shards can be identified directly from `system.opentelemetry_span_log`. [#115784](https://github.com/ClickHouse/ClickHouse/pull/115784) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Reads with parallel replicas are now covered by OpenTelemetry spans: a summary span with the totals of the coordination, one span per announcement and per read request handled by the coordinator, and `clickhouse.replica_num` on the per-replica spans, so the work distribution between replicas can be inspected in `system.opentelemetry_span_log`. [#118570](https://github.com/ClickHouse/ClickHouse/pull/118570) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Added profile events for network activity: `DNSRequests`, `DNSRequestMicroseconds`, and `DNSRequestError`; `DNSReverseRequests`, `DNSReverseRequestMicroseconds`, and `DNSReverseError`; `TLSHandshakes`, `TLSHandshakeMicroseconds`, and `TLSHandshakeErrors`; `TLSServerHandshakes`, `TLSServerHandshakeMicroseconds`, and `TLSServerHandshakeErrors`; and `HTTPServerConnectionsErrors`. DNS now exposes timing as well as error counts, and TLS handshakes are now observable. [#116136](https://github.com/ClickHouse/ClickHouse/pull/116136) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `AsyncInsertFlush` profile event. Compare it with `AsyncInsertQuery` to see how many queries are batched into one asynchronous `INSERT` flush. [#120187](https://github.com/ClickHouse/ClickHouse/pull/120187) ([Sema Checherinda](https://github.com/CheSema)). * Add the `RuntimeFilterLookupsBeforeBuildFinished` profile event, counting blocks that a JOIN runtime filter passed through unfiltered because its build side had not finished yet (for example with `join_overflow_mode = 'break'` and a finite `max_rows_in_join` / `max_bytes_in_join`). [#107108](https://github.com/ClickHouse/ClickHouse/pull/107108) ([Groene AI](https://github.com/groeneai)). * Added `last_error_symbols` and `last_error_lines` to `system.errors` and `system.error_log`. Closes [#74561](https://github.com/ClickHouse/ClickHouse/issues/74561). [#100374](https://github.com/ClickHouse/ClickHouse/pull/100374) ([Narasimha Pakeer](https://github.com/npakeer)). * `system.processors_profile_log` now has entries for queries that fail with an exception, and new columns `exception_code` and `exception`. [#121503](https://github.com/ClickHouse/ClickHouse/pull/121503) ([Aleksandr Efimov](https://github.com/alexandrefimov)). * `DROP TABLE` and `DROP DATABASE` now write a `RemovePart` event to `system.part_log` for every part of the dropped table, as merges, `DROP PARTITION`, and `TRUNCATE` already do. [#122093](https://github.com/ClickHouse/ClickHouse/pull/122093) ([Garrett Thomas](https://github.com/garrettthomaskth)). * Added the server setting `min_allocation_size_to_log_stack_trace`. When a single charge to the global memory tracker reaches it, the stack trace is written to the server log and to `system.trace_log` with trace type `MemoryLargeAllocation`. This helps attribute a global tracked total that has grown far beyond real memory usage, a state in which the server refuses every allocation while far below its limit. Default `0` keeps it off. [#119527](https://github.com/ClickHouse/ClickHouse/pull/119527) ([Groene AI](https://github.com/groeneai)). * Show throttler and workload IO scheduler wait time in the client progress bar (`ThrottlerSleepMicroseconds`, `SchedulerIOReadWaitMicroseconds` and `SchedulerIOWriteWaitMicroseconds` are summed into the `waited` figure, and the stalled sections of the bar are colored yellow). [#118536](https://github.com/ClickHouse/ClickHouse/pull/118536) ([Azat Khuzhin](https://github.com/azat)). * `EXPLAIN ANALYZE time=1` shows more information about time and parallelism of every step. [#114786](https://github.com/ClickHouse/ClickHouse/pull/114786) ([Kirill Kopnev](https://github.com/Fgrtue)). * Peak-memory reporting under `EXPLAIN ANALYZE` now accounts for aggregation correctly. [#123324](https://github.com/ClickHouse/ClickHouse/pull/123324) ([Azat Khuzhin](https://github.com/azat)). * Removed nine unneeded `ProfileEvents`: the eight never-incremented `DistributedIndexAnalysisParts`, `ParallelReplicasDeniedRequests`, `WasmMemoryAllocated`, `S3QueueSetFileProcessingMicroseconds`, `S3QueueSetFileProcessedMicroseconds`, `S3QueueSetFileFailedMicroseconds`, `FileSegmentRemoveMicroseconds`, and `ExternalDataSourceLocalCacheReadBytes`, plus the duplicate `CachedReadBufferPredownloadedBytes`, which always matched `CachedReadBufferPredownloadedFromSourceBytes`. This reduces per-query memory and bookkeeping overhead; queries and dashboards should use `CachedReadBufferPredownloadedFromSourceBytes`. [#123040](https://github.com/ClickHouse/ClickHouse/pull/123040) ([Raúl Marín](https://github.com/Algunenano)). [#123042](https://github.com/ClickHouse/ClickHouse/pull/123042) ([Raúl Marín](https://github.com/Algunenano)). * Killing a query that is loading skip-index marks from object storage now stops the read promptly: the remote read path checks query cancellation before each buffer fill, instead of reading on for minutes after the query was already cancelled. [#113436](https://github.com/ClickHouse/ClickHouse/pull/113436) ([Groene AI](https://github.com/groeneai)). * Reduced cancellation latency for `GROUP BY ... WITH TOTALS` queries: `KILL QUERY` now interrupts totals accumulation within a single chunk instead of waiting for all aggregate states to be merged. [#119470](https://github.com/ClickHouse/ClickHouse/pull/119470) ([Roman Vasin](https://github.com/rvasin)). * `KILL QUERY` and Ctrl+C in `clickhouse-client` now cancel long-running functions inside `WHERE` and `HAVING` filters instead of waiting for the function to finish. [#112705](https://github.com/ClickHouse/ClickHouse/pull/112705) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `KILL MUTATION`, `SYSTEM STOP MERGES`, `DETACH`/`DROP` of a table, `REPLACE`/`ATTACH PARTITION ... FROM` and server shutdown now stop a running mutation within a fraction of a second when the mutation's predicate has an `IN (subquery)` whose subquery works without reading new data (for example a cross join). Previously they waited until the whole set was built, and `REPLACE`/`ATTACH PARTITION ... FROM` gave up with `TIMEOUT_EXCEEDED` after 120 seconds. `KILL MUTATION` now also stops a mutation that is already writing its new part. [#124122](https://github.com/ClickHouse/ClickHouse/pull/124122) ([Groene AI](https://github.com/groeneai)). * `KILL QUERY` and `max_execution_time` (with the default `timeout_overflow_mode = 'throw'`) now interrupt writing the `Parquet` format in the middle of a block that is split into many row groups (for example with a small `output_format_parquet_row_group_size`), instead of only after the whole block is written. [#125489](https://github.com/ClickHouse/ClickHouse/pull/125489) ([Groene AI](https://github.com/groeneai)). * `DROP TABLE`, `DETACH TABLE`, server shutdown, and `ALTER TABLE ... MODIFY SETTING refresh_statistics_interval` no longer wait for a background refresh of the statistics cache of a `MergeTree` table to read every part. On object storage with thousands of parts this wait could take minutes per table and delay a restart. Closes [#125136](https://github.com/ClickHouse/ClickHouse/issues/125136). [#125165](https://github.com/ClickHouse/ClickHouse/pull/125165) ([Groene AI](https://github.com/groeneai)). * `DROP TABLE` and other operations that cancel merges and mutations no longer wait for an in-progress `vector_similarity` index build to finish. [#123062](https://github.com/ClickHouse/ClickHouse/pull/123062) ([Raúl Marín](https://github.com/Algunenano)). [#123739](https://github.com/ClickHouse/ClickHouse/pull/123739) ([Raúl Marín](https://github.com/Algunenano)). * `KILL QUERY WHERE query_id = ''` now cancels the caller's own query without requiring `SELECT` on `system.processes`. For a caller holding neither that grant nor `KILL QUERY`, an id that is not running as them cancels nothing and returns no rows. Any other `WHERE` condition keeps requiring the privileges it requires today. [#117999](https://github.com/ClickHouse/ClickHouse/pull/117999) ([Groene AI](https://github.com/groeneai)). * Access to `system.jemalloc_profile_text`, `system.jemalloc_sampled_allocations`, `system.jemalloc_bins`, `system.jemalloc_arena_bins`, and `system.jemalloc_stats` now requires `SYSTEM_JEMALLOC`; the heap profiles flushed by `system.jemalloc_profile_text` and `system.jemalloc_sampled_allocations` are also removed after being returned. [#121622](https://github.com/ClickHouse/ClickHouse/pull/121622) ([Azat Khuzhin](https://github.com/azat)). [#121634](https://github.com/ClickHouse/ClickHouse/pull/121634) ([Azat Khuzhin](https://github.com/azat)). * Added the `access_control_improvements.readonly_restricts_set_role` server setting, `false` by default. When enabled, `SET ROLE` is refused with `READONLY` in a session or query whose `readonly` setting is not zero. Previously `SET ROLE` was the only identity-changing statement that `readonly` did not refuse. [#118021](https://github.com/ClickHouse/ClickHouse/pull/118021) ([Groene AI](https://github.com/groeneai)). * Secret arguments of table functions, table engines, database engines, backup engines, and dictionary sources are now declared next to each feature and are masked consistently in `SHOW CREATE`, system tables, query logs, and `EXPLAIN AST`, instead of relying on a central hardcoded parser list. [#123364](https://github.com/ClickHouse/ClickHouse/pull/123364) ([Raúl Marín](https://github.com/Algunenano)). * The `PostgreSQL`, `MaterializedPostgreSQL`, `DataLakeCatalog`, `Cluster`, `Remote`, and `HDFS` database engines key the exemption for the server's replay of its own stored metadata on the metadata loader instead of on the query being internal. A `HDFS` database restored with `RESTORE DATABASE` is now checked against `remote_url_allow_hosts`. [#120280](https://github.com/ClickHouse/ClickHouse/pull/120280) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow `remote_url_allow_hosts` to match the bucket name of an `S3` URL. [#124453](https://github.com/ClickHouse/ClickHouse/pull/124453) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Support proxy credentials in the `http_proxy` and `https_proxy` environment variables. Previously the userinfo part of the proxy URI was discarded, so ClickHouse could not use an authenticating proxy. Proxy credentials are sent only to the proxy, rather than to the target server inside a `CONNECT` tunnel. [#123069](https://github.com/ClickHouse/ClickHouse/pull/123069) ([Akanshaak](https://github.com/Akanshaak)). [#124396](https://github.com/ClickHouse/ClickHouse/pull/124396) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Interserver authentication failures on a listener that uses the PROXY protocol now record the client address from the PROXY header in `system.session_log` when `auth_use_forwarded_address` is enabled, instead of the proxy address. [#125326](https://github.com/ClickHouse/ClickHouse/pull/125326) ([Groene AI](https://github.com/groeneai)). * Added the `tcp_with_proxy_allow_interserver_mode` server setting, which allows disabling the native protocol's interserver mode on `tcp_with_proxy_port`. [#123188](https://github.com/ClickHouse/ClickHouse/pull/123188) ([Pradeep Chhetri](https://github.com/chhetripradeep)). * Reject unexpected packets in the native protocol before deserializing their payload, avoiding unnecessary processing of data types and aggregate function states before returning `UNEXPECTED_PACKET_FROM_CLIENT`. [#119368](https://github.com/ClickHouse/ClickHouse/pull/119368) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the setting `kill_throw_if_noop`, enabled by default, which makes local `KILL QUERY` and `KILL MUTATION` statements throw `NO_QUERY_TO_KILL` when their selection produces no eligible target. `ON CLUSTER` execution remains tolerant of empty matches. Set `kill_throw_if_noop = false` to preserve the previous behavior. Related [#83348](https://github.com/ClickHouse/ClickHouse/issues/83348). [#85023](https://github.com/ClickHouse/ClickHouse/pull/85023) ([zwy991114](https://github.com/zwy991114)). * Merges and queries on `S3` disks no longer hold one HTTP connection per open stream for their whole duration once connections are scarce. Previously this let a few merges of tables with many substreams, for example a `JSON` column, saturate `disk_connections_hard_limit`, stall each other, and fail every other request to the disks with `HTTP_CONNECTION_LIMIT_REACHED`. Once the `Disk` connection group is at or above `disk_connections_soft_limit`, reads from `S3` disks fetch each buffer fill with a separate request and return the connection to the pool between fills; the new profile event `ReadBufferFromS3RequestsCut` counts such requests. [#119730](https://github.com/ClickHouse/ClickHouse/pull/119730) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124701](https://github.com/ClickHouse/ClickHouse/pull/124701) ([Groene AI](https://github.com/groeneai)). * The `filesystem_cache_boundary_alignment` setting can now be changed per query. For disk reads, `0` means to use the cache configuration's `boundary_alignment`; for non-disk reads, `0` still disables alignment. [#118791](https://github.com/ClickHouse/ClickHouse/pull/118791) ([Kseniia Sumarokova](https://github.com/kssenii)). * Added the filesystem cache setting `use_real_disk_size` (disabled by default), which accounts for cache reservations in filesystem allocation units instead of raw bytes. It makes `FilesystemCacheSize`, eviction-size metrics, and `current_size` in `system.filesystem_cache_settings` approximate the disk footprint. The accounting is block-aligned rather than exact and can include reserve-ahead bytes of in-flight reads. [#79490](https://github.com/ClickHouse/ClickHouse/pull/79490) ([codeworse](https://github.com/codeworse)). * Custom HTTP headers configured for S3 are now handled case-insensitively by name, so a header spelled `X-Amz-Meta-Foo` reaches Google Cloud Storage translated to `x-goog-meta-foo`, as `x-amz-meta-foo` already did. [#120014](https://github.com/ClickHouse/ClickHouse/pull/120014) ([Sema Checherinda](https://github.com/CheSema)). * Better error messages for S3 request retries: the HTTP response code is no longer printed as `18446744073709551615` when the request received no response at all, and the misleading message about a possible time skew is no longer printed on every retry. [#118439](https://github.com/ClickHouse/ClickHouse/pull/118439) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow `DESCRIBE FILESYSTEM CACHE` on the introspection port. [#115894](https://github.com/ClickHouse/ClickHouse/pull/115894) ([Groene AI](https://github.com/groeneai)). * Files on encrypted disks now support positional reads, so a table with a `UNIQUE KEY` on an encrypted disk can be loaded again after a server restart. [#119991](https://github.com/ClickHouse/ClickHouse/pull/119991) ([johnjing](https://github.com/JingYanchao)). * The `url` table function and the `URL` table engine now expand glob patterns lazily instead of materializing every address before the query starts. `glob_expansion_max_elements` now limits how many addresses a query actually generates rather than how large the pattern is, so a query that stops early - for example under a `LIMIT` - can use a pattern that describes far more addresses than the limit. [#115680](https://github.com/ClickHouse/ClickHouse/pull/115680) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `Iceberg` manifest writing now uses sequence-number inheritance: `ADDED` entries have `NULL` `sequence_number` and `file_sequence_number` unless compaction assigns an explicit sequence number. [#120031](https://github.com/ClickHouse/ClickHouse/pull/120031) ([Asya Shneerson](https://github.com/asya-ch)). * `Iceberg` manifest statistics are now written for `boolean` and `fixed` columns. [#121548](https://github.com/ClickHouse/ClickHouse/pull/121548) ([Konstantin Vedernikov](https://github.com/scanhex12)). * When a table is created through a catalog, `CREATE TABLE catalog.table ENGINE = IcebergS3` can now omit explicit credentials and reuse the catalog credentials. [#114477](https://github.com/ClickHouse/ClickHouse/pull/114477) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `ALTER TABLE ... EXECUTE expire_snapshots` now works for `Iceberg` tables in transactional catalogs such as Iceberg REST and Unity. [#123248](https://github.com/ClickHouse/ClickHouse/pull/123248) ([alesapin](https://github.com/alesapin)). * A `DataLakeCatalog` database whose catalog cannot be built no longer reports its tolerated table-listing failure as an error of the query that enumerated it: the skipped database is now logged at `debug` instead of `error`, so a query such as `SELECT ... FROM system.tables` or `system.iceberg_history` no longer sends a server-side exception to a client whose `send_logs_level` includes `error`. Set `show_data_lake_catalogs_in_system_tables = 1` to have the real exception raised instead. [#121475](https://github.com/ClickHouse/ClickHouse/pull/121475) ([Groene AI](https://github.com/groeneai)). * When a `DeltaLake` write commit fails, ClickHouse now always reports the commit error rather than an error raised while removing the data files the failed `INSERT` had already uploaded, and every removal that fails during that cleanup is logged with the path it was left on. Previously the first failing removal replaced the commit error in the response, so the reason nothing was written was lost and no path was logged. Closes [#122023](https://github.com/ClickHouse/ClickHouse/issues/122023). [#122123](https://github.com/ClickHouse/ClickHouse/pull/122123) ([Groene AI](https://github.com/groeneai)). * Reading an `Iceberg` table whose position delete file has no `pos` or `file_path` column now fails with `ICEBERG_SPECIFICATION_VIOLATION` naming the file and the missing column, instead of a `LOGICAL_ERROR` (missing `pos`) or a misleading `UNKNOWN_IDENTIFIER` (missing `file_path`). [#123583](https://github.com/ClickHouse/ClickHouse/pull/123583) ([Groene AI](https://github.com/groeneai)). * `Paimon` `BOOLEAN` columns are now read as `Bool` instead of `Int8`, matching `Iceberg` and `DeltaLake`. `DESCRIBE` and `toTypeName` now report `Bool`, and text formats render the values as `false` and `true` instead of `0` and `1`. [#119404](https://github.com/ClickHouse/ClickHouse/pull/119404) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Avro files now include per-column size statistics in their metadata. [#121392](https://github.com/ClickHouse/ClickHouse/pull/121392) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The `Parquet` reader now rejects a column chunk whose metadata has a negative `num_values`, `total_uncompressed_size`, or `unencoded_byte_array_data_bytes` with `INCORRECT_DATA` instead of failing later with a misleading memory-allocation error. It also caps memory estimates derived from these fields at the reader's watermarks and ignores pre-allocations too large to ever succeed, so absurdly large values no longer cause an internal error. Closes [#124224](https://github.com/ClickHouse/ClickHouse/issues/124224). [#124330](https://github.com/ClickHouse/ClickHouse/pull/124330) ([Groene AI](https://github.com/groeneai)). * Bumped the bundled `ORC` library so that an `ORC` file declaring a compression codec the reader does not implement is rejected with an error naming the codec, instead of an internal error that debug and sanitizer builds treat as a failed assertion. [#115889](https://github.com/ClickHouse/ClickHouse/pull/115889) ([Groene AI](https://github.com/groeneai)). * `INSERT ... VALUES` over a stream that fails mid-row now reports that read failure directly, instead of retrying the failed read while recovering from a parse error. This affects an HTTP request body that is shorter than its declared `Content-Length` and is streamed to the input format. [#121916](https://github.com/ClickHouse/ClickHouse/pull/121916) ([Groene AI](https://github.com/groeneai)). * ClickHouse Keeper no longer drops the client session when the outgoing request queue stays full for longer than `operation_timeout_ms`. The enqueue wait is now three times `session_timeout_ms`, so a short burst of backpressure no longer invalidates the session's ephemeral nodes and watches. [#121559](https://github.com/ClickHouse/ClickHouse/pull/121559) ([Kseniia Sumarokova](https://github.com/kssenii)). * ClickHouse Keeper now reports orphaned nodes in a snapshot as `CORRUPTED_DATA` instead of an opaque internal error. With `keeper_server.remove_orphaned_nodes_on_startup = true` and `keeper_server.digest_enabled = false`, it can also remove those nodes during startup so the snapshot still loads. [#95761](https://github.com/ClickHouse/ClickHouse/pull/95761) ([Seva Potapov](https://github.com/seva-potapov)). * The Keeper `rcfg` command now honors its `retry` count per action, stops once `max_total_wait_time_ms` is exhausted, and rejects a negative `retry` before running any action. [#122758](https://github.com/ClickHouse/ClickHouse/pull/122758) ([Groene AI](https://github.com/groeneai)). * Updated `NuRaft` to fix handling of abandoned peers when a removed server rejoins. [#120736](https://github.com/ClickHouse/ClickHouse/pull/120736) ([Antonio Andelic](https://github.com/antonio2368)). * Added the setting `get_zookeeper_lock_acquire_timeout_ms` so Keeper-backed queries such as reads from `system.zookeeper`, `zookeeperSessionUptime`, and `SYSTEM RECONNECT ZOOKEEPER` can fail fast while another thread is reconnecting Keeper. [#94836](https://github.com/ClickHouse/ClickHouse/pull/94836) ([c-end](https://github.com/c-end)). * Activating a replica whose `active` or `is_active` node in `Keeper` is still held by another session is now reported as `REPLICA_ALREADY_EXISTS` instead of `LOGICAL_ERROR`, so it no longer aborts the server in debug and sanitizer builds. The same applies when that node is rewritten while it is being removed. [#114408](https://github.com/ClickHouse/ClickHouse/pull/114408) ([Groene AI](https://github.com/groeneai)). * The ZooKeeper client, the Graphite metrics writer, the Redis source, `system.zookeeper_info`, the distributed query exchange, and host names in `listen_host` and `bind_host` now resolve through the ClickHouse DNS cache, so these lookups are cached, visible in `system.dns_cache` and subject to `dns_allow_resolve_names_to_ipv4` and `dns_allow_resolve_names_to_ipv6`. Previously they resolved names on their own, bypassing the cache. [#120831](https://github.com/ClickHouse/ClickHouse/pull/120831) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `http_x_clickhouse_format_overrides_output_format` setting (default `true`). When disabled, the `X-ClickHouse-Format` HTTP header only sets `default_format`, as it did before 26.8, instead of overriding a query's `FORMAT` clause via `output_format`. [#121132](https://github.com/ClickHouse/ClickHouse/pull/121132) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The Prometheus HTTP API accepts `RFC3339` times with `Z` or a `+hh:mm` offset and fractional seconds in `time`, `start`, and `end`, as Prometheus does, so clients like vmalert work. String time arguments of `prometheusQuery`, `timeSeriesSelector`, and the `timeSeries*ToGrid` functions accept `RFC3339` too. [#122808](https://github.com/ClickHouse/ClickHouse/pull/122808) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL regex matchers that list plain literals, such as `{job=~"api|web"}`, now use the primary key and return exactly the listed values, as in Prometheus. Before, only the first alternative was anchored at the start and only the last at the end, so such a matcher also returned values like `api-server` or `myweb`. [#122796](https://github.com/ClickHouse/ClickHouse/pull/122796) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL instant queries now return series in a deterministic order: `topk` and `bottomk` at the root are sorted by value, as Prometheus returns them, and other instant queries are sorted by their labels, like range queries, so repeated requests list the series in the same order. [#122347](https://github.com/ClickHouse/ClickHouse/pull/122347) ([James Cunningham](https://github.com/JTCunning)). [#122976](https://github.com/ClickHouse/ClickHouse/pull/122976) ([Valery Petrov](https://github.com/valerypetrov)). * `/api/v1/format_query` and PromQL serialization now print durations in Prometheus units, so `sum(rate(x[5m]))` formats as itself rather than as `sum(rate(x[300]))`. [#120939](https://github.com/ClickHouse/ClickHouse/pull/120939) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL now validates label names in `label_join` and `label_replace`. [#118190](https://github.com/ClickHouse/ClickHouse/pull/118190) ([Minh Vu](https://github.com/fallintoplace)). * A Prometheus remote-write or remote-read request whose body cannot be decompressed is now answered with `400` instead of `500`, and the error names the size of the block and the uncompressed length it declares. [#120938](https://github.com/ClickHouse/ClickHouse/pull/120938) ([Valery Petrov](https://github.com/valerypetrov)). * Added the `merge_selector_min_age_to_disable_right_tail_heuristic` `MergeTree` setting. It lets the `Simple` and `StochasticSimple` merge selectors include small parts at the right end of a range in merge candidates once every part in the range is old enough. [#122638](https://github.com/ClickHouse/ClickHouse/pull/122638) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `MergeTree` settings `merge_selector_small_parts_min_count`, `merge_selector_small_parts_threshold`, and `merge_selector_small_parts_max_age` to reduce the number of merge operations under rapid small-part insertion by requiring a minimum batch size before merging small fresh parts. [#100436](https://github.com/ClickHouse/ClickHouse/pull/100436) ([Amos Bird](https://github.com/amosbird)). * Calculate projections using the query analyzer and reject missing projection columns rather than writing incomplete projection parts. Row policies and `additional_table_filters` for `system.one` no longer affect projection calculation. [#118375](https://github.com/ClickHouse/ClickHouse/pull/118375) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... ADD COLUMN ..., MODIFY ORDER BY ...` now rejects a new sorting key that uses a subcolumn of the added `DEFAULT` or `MATERIALIZED` column, and its error message names whether the expression kind is `DEFAULT` or `MATERIALIZED` instead of always saying `default expression`. [#121873](https://github.com/ClickHouse/ClickHouse/pull/121873) ([GON](https://github.com/gonappuccino)). * An unresolved name in a column `DEFAULT`, `MATERIALIZED`, or `ALIAS` expression is now reported by the name the user wrote, instead of an internal `_tmp_alter` alias that appears nowhere in the query and also broke the "Maybe you meant" hint. A sorting, primary, or partition key that contains a nullable column or a constant now names that column or constant. [#116871](https://github.com/ClickHouse/ClickHouse/pull/116871) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: save and load the list of tabs, their queries, and query parameters as a Markdown file. The tab bar is now always shown, and the connection settings always live behind the key button in its corner. [#121234](https://github.com/ClickHouse/ClickHouse/pull/121234) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: added a `Format` button to the query editor, which formats the query with `formatQuery`. [#121652](https://github.com/ClickHouse/ClickHouse/pull/121652) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122477](https://github.com/ClickHouse/ClickHouse/pull/122477) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: clicking a cell of a `JSON` column pretty-prints the value with two-space indentation and syntax highlighting. [#118640](https://github.com/ClickHouse/ClickHouse/pull/118640) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The graph view of the Web UI for `EXPLAIN ... graph = 1` now follows the light/dark theme instead of always drawing a white canvas with black lines, and its labels use a sans-serif font instead of the Graphviz default. Rendering a graph also works when `play.html` is opened as a local file. [#116208](https://github.com/ClickHouse/ClickHouse/pull/116208) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, the digits of a long number in the result table are now grouped by a thin gap, as in `1 234 567`, instead of an underline. The gap is an empty element, so selecting or copying the number still yields the digits alone. [#119839](https://github.com/ClickHouse/ClickHouse/pull/119839) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Show a checkmark on the Play UI result-table copy button for 1.2 s after a cell value is copied to the clipboard. [#120617](https://github.com/ClickHouse/ClickHouse/pull/120617) ([yanglongwei](https://github.com/ylw510)). * In the Web UI, the browser window title of a renamed query tab is now exactly the tab's name. [#122067](https://github.com/ClickHouse/ClickHouse/pull/122067) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: when a login last succeeded with an empty password, a stale password autofilled by the browser's password manager for that server and user is no longer used. [#118638](https://github.com/ClickHouse/ClickHouse/pull/118638) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The schema visualizer now supports hiding and showing table categories by clicking their legend labels, with a Hide all / Show all control. [#124963](https://github.com/ClickHouse/ClickHouse/pull/124963) ([Mark Needham](https://github.com/mneedham)). * The command-line client no longer shows as-you-type hints on a query recalled from the history or otherwise displayed at once, so pressing Up/Down keeps navigating the history until the line is edited. [#115065](https://github.com/ClickHouse/ClickHouse/pull/115065) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The progress bar in `clickhouse-client` and `clickhouse-local` no longer flickers when the query result is redirected to a pipe or a file. [#123549](https://github.com/ClickHouse/ClickHouse/pull/123549) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A bare table name can now use the archive path syntax in `clickhouse-local`, so `SELECT * FROM 'archive.tar.zst::data.native'` reads a file inside an archive, like `SELECT * FROM file('archive.tar.zst::data.native')`. The same applies to tables of a `URL` database with a `file://` target and of a `Filesystem` database. [#118170](https://github.com/ClickHouse/ClickHouse/pull/118170) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Clean up client output when a web terminal or SSH session is closed during a running query, avoiding spurious write-buffer errors. [#125558](https://github.com/ClickHouse/ClickHouse/pull/125558) ([Groene AI](https://github.com/groeneai)). * A musl-built ClickHouse binary is now self-contained enough to run in a `FROM scratch` container image with no extra files: CA certificates are embedded, `localhost` resolves to loopback even without `/etc/hosts`, and DNS no longer falls back to the unreachable localhost when `/etc/resolv.conf` is missing. [#115389](https://github.com/ClickHouse/ClickHouse/pull/115389) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Hedged requests are now supported on macOS. Previously `use_hedged_requests` was silently ignored on macOS and distributed queries always used non-hedged connections. [#119915](https://github.com/ClickHouse/ClickHouse/pull/119915) ([Raúl Marín](https://github.com/Algunenano)). * Determine the number of CPUs from the kernel's sysfs lists instead of the libc, so per-CPU structures and `auto` thread counts do not depend on the libc's interpretation of the affinity mask. [#118912](https://github.com/ClickHouse/ClickHouse/pull/118912) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Workloads created without a parent workload are now allowed; they become children of an implicit root workload, so several workload trees can coexist. [#118578](https://github.com/ClickHouse/ClickHouse/pull/118578) ([Sergei Trifonov](https://github.com/serxa)). * `RANDOMIZE FOR` is now applied on each replica of a refreshable materialized view independently, so a random replica performs each refresh instead of the same one every time. Replicas no longer agree on `next_refresh_time`, so the value reported by `system.view_refreshes` on one replica is the time that replica would refresh at, not necessarily when the refresh happens. Closes [#78410](https://github.com/ClickHouse/ClickHouse/issues/78410). [#112979](https://github.com/ClickHouse/ClickHouse/pull/112979) ([Groene AI](https://github.com/groeneai)). * `FileLog` now reads a file again from the beginning if, at the next read, it is shorter than the offset already read from it (for example after `logrotate` with `copytruncate`), instead of failing every read of the table with `CANNOT_READ_ALL_DATA` until the file grows back to its old size. [#123390](https://github.com/ClickHouse/ClickHouse/pull/123390) ([Groene AI](https://github.com/groeneai)). * A relative path in the `FileLog` table engine is now resolved against `user_files_path`, like in the `file` table function, so `FileLog('nginx/', 'JSONEachRow')` reads `/nginx/`. A relative path that already points inside `user_files_path` from the server's working directory keeps its meaning. [#123397](https://github.com/ClickHouse/ClickHouse/pull/123397) ([Groene AI](https://github.com/groeneai)). * Connections from the `MongoDB` table engine, table function, and dictionary source now identify themselves to the MongoDB server with the ClickHouse name and version in the driver handshake metadata. MongoDB operators can tell ClickHouse connections apart from other clients in server logs, `currentOp` output, and monitoring tools. [#122205](https://github.com/ClickHouse/ClickHouse/pull/122205) ([bisht2050](https://github.com/bisht2050)). * Lookups of `YTsaurus` dictionaries can now be throttled. [#100044](https://github.com/ClickHouse/ClickHouse/pull/100044) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * AI functions using the `openai`, `huggingface`, or `tei` provider send the query ID and the initial query ID in the `X-ClickHouse-Query-Id` and `X-ClickHouse-Initial-Query-Id` HTTP headers. [#123622](https://github.com/ClickHouse/ClickHouse/pull/123622) ([George Larionov](https://github.com/george-larionov)). * `aiFilter` now requests structured JSON Schema output. [#121526](https://github.com/ClickHouse/ClickHouse/pull/121526) ([Andrew Wilkins](https://github.com/axw)). * WebAssembly UDF calls with `ABI BUFFERED_V1` now measure the whole input batch instead of one serialization per row, so usage is priced by the stream the guest actually receives for formats whose cost is not additive across rows. [#118989](https://github.com/ClickHouse/ClickHouse/pull/118989) ([Vasily Chekalkin](https://github.com/bacek)). * Allow the `ON CLUSTER` clause for `RENAME TABLE` and `EXCHANGE TABLES` on a `Replicated` or Cloud `Shared` database when the cluster name matches the database's automatic cluster name. The clause is treated as a no-op and the database engine propagates the DDL as usual, which lets scripts written for non-`Replicated` databases be ported without rewriting `ON CLUSTER` clauses. [#106192](https://github.com/ClickHouse/ClickHouse/pull/106192) ([Groene AI](https://github.com/groeneai)). * Stored HTTP handler queries no longer break when a request supplies `max_parser_depth`, `max_parser_backtracks`, or `max_query_size`, and the `CREATE HANDLER`, `ALTER HANDLER`, and `DROP HANDLER` documentation is now published. [#114883](https://github.com/ClickHouse/ClickHouse/pull/114883) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve memory tracking for query sources, including recursive CTEs and external data sources, so their container allocations count toward memory limits. [#124952](https://github.com/ClickHouse/ClickHouse/pull/124952) ([Nikita Taranov](https://github.com/nickitat)). * A subquery with its own `SETTINGS` clause now plans `GROUP BY` using the settings after constraints and `max_threads` clamping are applied, instead of reapplying the clause as written. This prevents nested `SETTINGS max_threads = ...` from bypassing settings constraints or expanding the aggregation pipeline to an excessive number of streams. [#123156](https://github.com/ClickHouse/ClickHouse/pull/123156) ([Groene AI](https://github.com/groeneai)). * Improved the error message of `parseDateTime`. [#120489](https://github.com/ClickHouse/ClickHouse/pull/120489) ([Robert Schulze](https://github.com/rschu1ze)). * Restoring an invalid data part from backup now reports `BACKUP_DAMAGED` instead of `CORRUPTED_DATA`. [#113085](https://github.com/ClickHouse/ClickHouse/pull/113085) ([Alexander Tokmakov](https://github.com/tavplubix)). * Deserializing an aggregate function state that declares an oversized element count now reports `TOO_LARGE_ARRAY_SIZE` and names both the offending count and the limit, instead of a bare `POCO_EXCEPTION` carrying no numbers. [#121381](https://github.com/ClickHouse/ClickHouse/pull/121381) ([Groene AI](https://github.com/groeneai)). * A setting given a value of the wrong type or out of range now names the setting and the value in the error message. [#116882](https://github.com/ClickHouse/ClickHouse/pull/116882) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * When a dictionary with a `PostgreSQL` source and a custom `QUERY` returns its columns in an order that does not match the dictionary structure, the error now names the destination dictionary column and the column order the query must return, instead of only reporting that a value could not be parsed. [#114212](https://github.com/ClickHouse/ClickHouse/pull/114212) ([Groene AI](https://github.com/groeneai)). * A `PostgreSQL` dictionary whose `QUERY` returns fewer columns than the dictionary structure now fails to load with a `NUMBER_OF_COLUMNS_DOESNT_MATCH` error that names the column order the `QUERY` must use, instead of a `LOGICAL_ERROR`. A `QUERY` that returns too many columns gets the same column-order hint. [#125397](https://github.com/ClickHouse/ClickHouse/pull/125397) ([Groene AI](https://github.com/groeneai)). * A column matcher (`*`, `t.*`, `COLUMNS('...')`) inside `additional_table_filters`, `additional_result_filter` or a row policy is now rejected with a clear `BAD_ARGUMENTS` diagnostic instead of the obscure `Unqualified matcher * cannot be resolved. There are no table sources`. `CREATE` / `ALTER ROW POLICY` with a matcher now fails immediately. A matcher inside a subquery of such a filter, for example `x IN (SELECT * FROM allowed)`, keeps working. [#119250](https://github.com/ClickHouse/ClickHouse/pull/119250) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A matcher (`*`, `t.*`, `COLUMNS(...)`) used as a whole `LIMIT AFTER` or `LIMIT UNTIL` boundary is now rejected with `UNEXPECTED_EXPRESSION`, as in `WHERE`, instead of failing with `std::out_of_range` or silently using only its first column. [#123309](https://github.com/ClickHouse/ClickHouse/pull/123309) ([Groene AI](https://github.com/groeneai)). * `CREATE FUNCTION` now rejects a SQL user-defined function whose lambda arguments cannot be evaluated, instead of storing a function that later fails with `BAD_ARGUMENTS` on every call. Closes [#121197](https://github.com/ClickHouse/ClickHouse/issues/121197). [#121370](https://github.com/ClickHouse/ClickHouse/pull/121370) ([Groene AI](https://github.com/groeneai)). * A correlated subquery in `HAVING` of a `WITH TOTALS` query that also uses `max_rows_to_group_by` with `group_by_overflow_mode = 'any'` (and `totals_mode` other than `after_having_exclusive`) now fails with `NOT_IMPLEMENTED` instead of a logical error. [#122601](https://github.com/ClickHouse/ClickHouse/pull/122601) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `mergeTreeAnalyzeIndexes` and `mergeTreeAnalyzeIndexesUUID` table functions now reject a `vector_search_index_analysis` limit above `max_limit_for_vector_search_queries`, the bound under which the plan optimization rewrites a vector search query. A larger limit previously made the vector similarity index reserve memory proportional to it and exhaust the server's memory. [#122154](https://github.com/ClickHouse/ClickHouse/pull/122154) ([Groene AI](https://github.com/groeneai)). * An unconditioned `CROSS JOIN` is no longer reported as `INNER` in `EXPLAIN` and in the join columns of `system.query_log` when the `dpsub` join order algorithm runs with a conflict detector enabled. Query results were not affected. [#122236](https://github.com/ClickHouse/ClickHouse/pull/122236) ([Fisnik Kastrati](https://github.com/fkastrati)). * Formatting a query where an `IN` is the left operand of a tuple element (`(x IN (1)).1`) inside a multi-argument function call is now stable across a format-parse-format round trip. [#124325](https://github.com/ClickHouse/ClickHouse/pull/124325) ([Groene AI](https://github.com/groeneai)). * Exchanging `-Merge` aggregate states with older servers now stays compatible: the state version of a `-Merge` function follows the state version of the function it merges only for peers that announce the new protocol revision `54494`. [#123070](https://github.com/ClickHouse/ClickHouse/pull/123070) ([Groene AI](https://github.com/groeneai)). * An aggregate function parameter whose value has no unambiguous bare literal form, such as a `Decimal`, a wide integer, a `UUID`, an `IPv4` or `IPv6`, or a non-negative `Int64`, is now always printed with an explicit `::Type` suffix, and inside an array or tuple parameter the suffix goes on the elements. Such a parameter therefore reads back as the same value for every family, instead of only for the ones that had opted in. The typed form also travels over the `Native` protocol, so an insert into a column whose parameter was spelled without the type is now rejected there too, as it already was locally. [#121725](https://github.com/ClickHouse/ClickHouse/pull/121725) ([Groene AI](https://github.com/groeneai)). * Diagnostics about slowly acquired locks are now logged at `Information` level instead of `Warning`, so they no longer reach clients that use `send_logs_level = 'warning'` while still being written to the server log with a stack trace. [#123797](https://github.com/ClickHouse/ClickHouse/pull/123797) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Adaptive aggregation now logs less noisily by reducing the severity of its messages. [#123312](https://github.com/ClickHouse/ClickHouse/pull/123312) ([Azat Khuzhin](https://github.com/azat)). * `kqlRangeCount` now rejects the `Int256` minimum span with step `-1` instead of incorrectly reporting an empty range. [#117212](https://github.com/ClickHouse/ClickHouse/pull/117212) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A distributed query whose shards plan aggregation with the experimental `enable_cascades_optimizer` no longer hits the logical error `GroupingAggregatedTransform pushed bucket N twice` on the initiator. [#119814](https://github.com/ClickHouse/ClickHouse/pull/119814) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Local execution of distributed query plans (`distributed_plan_execute_locally = 1`) now materializes constant columns in its in-memory exchange like serialized exchanges do, fixing a query that could fail with a `LOGICAL_ERROR` when it sorted by a constant expression. [#124097](https://github.com/ClickHouse/ClickHouse/pull/124097) ([Groene AI](https://github.com/groeneai)). * The internal function `__bitSwapLastTwo` now raises the same `BAD_ARGUMENTS` error for unsupported argument types when the expression is JIT-compiled as when it is interpreted. [#117613](https://github.com/ClickHouse/ClickHouse/pull/117613) ([Groene AI](https://github.com/groeneai)). * Fix a UBSan failure during parallel parsing of `TabSeparated` input with an empty buffer. [#122636](https://github.com/ClickHouse/ClickHouse/pull/122636) ([Ray Winkelman](https://github.com/rwinkelman)). * The `ErrorCode` registry is now sparse, so ClickHouse can reserve gaps between error codes without preallocating counters for them. [#111796](https://github.com/ClickHouse/ClickHouse/pull/111796) ([Maxim Orlovsky](https://github.com/UberDever)). * Fixed a signed integer overflow when a negative or huge `lock_acquire_timeout` value is used as the timeout for scheduling a background task. [#117898](https://github.com/ClickHouse/ClickHouse/pull/117898) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Cancelled marks-loading background tasks are no longer counted as errors in `system.errors` and `system.error_log`. Previously `ASYNC_LOAD_CANCELED` was recorded during normal operation whenever a reader was dropped before its asynchronously loaded marks were needed. [#118928](https://github.com/ClickHouse/ClickHouse/pull/118928) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... EXECUTE expire_snapshots` for `Iceberg` tables now counts a file as deleted only when its removal succeeds, reports the removals that return an error in a new `failed_deletions_count` metric like `remove_orphan_files`, and counts a file that several expired manifests list only once. Closes [#125332](https://github.com/ClickHouse/ClickHouse/issues/125332). [#125379](https://github.com/ClickHouse/ClickHouse/pull/125379) ([Groene AI](https://github.com/groeneai)). * Reading an archive that libarchive fails to open without giving a reason (for example a `.tar.gz` with a truncated gzip trailer) now throws `CANNOT_UNPACK_ARCHIVE` instead of `STD_EXCEPTION` (`fmt::format_error: string pointer is null`). Closes [#125620](https://github.com/ClickHouse/ClickHouse/issues/125620). [#125650](https://github.com/ClickHouse/ClickHouse/pull/125650) ([Groene AI](https://github.com/groeneai)). * `RESTORE` from a tar backup with a non-empty `password` setting now fails with `NOT_IMPLEMENTED`, like `BACKUP` to a tar archive, instead of `LOGICAL_ERROR`. Closes [#125648](https://github.com/ClickHouse/ClickHouse/issues/125648). [#125666](https://github.com/ClickHouse/ClickHouse/pull/125666) ([Groene AI](https://github.com/groeneai)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fixed silent data loss when writing to an S3 key that already holds an object, for example `INSERT INTO FUNCTION s3(...)` with `s3_truncate_on_insert = 1`. Closes [#114348](https://github.com/ClickHouse/ClickHouse/issues/114348). [#118829](https://github.com/ClickHouse/ClickHouse/pull/118829) ([Sema Checherinda](https://github.com/CheSema)). * Fix possible data loss when a part's `checksums.txt` is repaired (recalculated and written on load, or by `CHECK TABLE`) and the server loses power afterwards. The metadata rewrite is now durable and atomic (write to a temporary file, fsync, atomically rename), and an empty `checksums.txt` is now recalculated on load instead of detaching the whole part as broken. [#111414](https://github.com/ClickHouse/ClickHouse/pull/111414) ([Groene AI](https://github.com/groeneai)). * Fix a part being detached as broken, losing its rows, when `columns.txt` or `metadata_version.txt` survives a power loss at length zero. Such a file is now treated like an absent one, which the load path already handles safely. [#119582](https://github.com/ClickHouse/ClickHouse/pull/119582) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix data loss when loading a part with a broken projection: a marks file damaged by an interrupted write could be accepted as an empty part instead of being detached as broken. [#119586](https://github.com/ClickHouse/ClickHouse/pull/119586) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed replica recovery of a `Replicated` database deleting the data of a `KeeperMap` table, for example after the server was killed during an `ALTER` of that table. A `KeeperMap` table that still exists in Keeper is now kept or renamed with its data, instead of being dropped and re-created empty. Closes [#111957](https://github.com/ClickHouse/ClickHouse/issues/111957). [#122917](https://github.com/ClickHouse/ClickHouse/pull/122917) ([Groene AI](https://github.com/groeneai)). * `DROP DATABASE` of a `Replicated` database now drops a materialized view with `ENGINE = TimeSeries` instead of failing with `It's not initial query` and leaving the database half-dropped. [#123663](https://github.com/ClickHouse/ClickHouse/pull/123663) ([Groene AI](https://github.com/groeneai)). * Fixed data loss when a single `ALTER TABLE` renamed a `Nested` member, or another column with a dot in its name, and modified the renamed column. Such an `ALTER` is now rejected with `NOT_IMPLEMENTED`, as it already was for other columns. [#122794](https://github.com/ClickHouse/ClickHouse/pull/122794) ([Groene AI](https://github.com/groeneai)). * Fixed merges deleting all rows of a part whose rows TTL was never calculated, although none of its rows had expired: a part attached with `ATTACH PARTITION ... FROM` from a table without that rows TTL, or written before `MODIFY TTL` with `materialize_ttl_after_modify = 0` added it, for example after `MODIFY COLUMN ... REMOVE TTL`. The rows were lost in background TTL merges and in merges with fully expired parts. Closes [#121124](https://github.com/ClickHouse/ClickHouse/issues/121124). [#123068](https://github.com/ClickHouse/ClickHouse/pull/123068) ([Groene AI](https://github.com/groeneai)). * Fixed a race in `MergeTree` where an `ALTER TABLE ... UPDATE` or `DELETE` mutation submitted concurrently with another mutation could be silently skipped on some parts while being reported as done. [#124167](https://github.com/ClickHouse/ClickHouse/pull/124167) ([Murphy](https://github.com/murphy-4o)). * A mutation on a `Replicated*` table whose predicate contains a lambda can no longer prune to too few partitions when a lambda parameter name matches an `ALIAS` or `EPHEMERAL` column definition. Previously some matching rows were left unmutated while the mutation still finished successfully. [#121193](https://github.com/ClickHouse/ClickHouse/pull/121193) ([Groene AI](https://github.com/groeneai)). * Fix background merges with TTL silently stopping for the whole server. A merge with TTL that was selected and then dropped before it ran — for example because its table was dropped right after the expired rows were inserted — leaked one of the `max_number_of_merges_with_ttl_in_pool` slots, which are shared by all tables. After two such merges no table would delete expired rows any more until the server was restarted. [#121236](https://github.com/ClickHouse/ClickHouse/pull/121236) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Make the asynchronous startup of a `ReplicatedMergeTree` table robust against a concurrent `DETACH`, which could previously hang the startup forever or leave background tasks running on a table that was already shut down. [#121672](https://github.com/ClickHouse/ClickHouse/pull/121672) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix corrupted shared data of nested `JSON` objects after a failed attempt to parse a value during insert. It could lead to errors like `Unexpected version of JSON type binary encoding` or `Unknown type code` when reading the data after merges or when serializing it in binary formats. [#123898](https://github.com/ClickHouse/ClickHouse/pull/123898) ([Rory Shanks](https://github.com/rorylshanks)). * The server no longer terminates when a transient error such as `MEMORY_LIMIT_EXCEEDED`, a network error, or concurrent `SYSTEM RESTART REPLICA` happens while outdated data parts are loading in the background. Such errors are now retried later instead of aborting the server. [#117981](https://github.com/ClickHouse/ClickHouse/pull/117981) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed silent row loss when a `MergeTree` table with a projection is read with a `WHERE` in which an `AND` is used with negative polarity, under a `NOT`, as an operand of a comparison, or as the branch condition of `if`/`multiIf`/`CASE`, and one of that `AND`'s conjuncts references a column the projection does not store. [#120286](https://github.com/ClickHouse/ClickHouse/pull/120286) ([Groene AI](https://github.com/groeneai)). * Fixed some `SELECT` queries silently returning fewer rows when the table has a projection that exists only in some of its parts, for example one added with `ALTER TABLE ... ADD PROJECTION` after data was inserted, and `query_plan_remove_unused_columns` is `0`, which `compatibility` up to `25.11` sets. [#124022](https://github.com/ClickHouse/ClickHouse/pull/124022) ([Groene AI](https://github.com/groeneai)). * Fixed wrong query results with skip indexes on `SummingMergeTree`, `AggregatingMergeTree`, `CoalescingMergeTree`, and `GraphiteMergeTree` tables: with `optimize_on_insert`, the default, an `INSERT` built every skip index whose expression is not a plain column from the rows before they were merged, so queries using the index could skip granules with matching rows. `SummingMergeTree` also no longer keeps rows whose summed columns total zero because of such an index. [#123291](https://github.com/ClickHouse/ClickHouse/pull/123291) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when a `WHERE` clause mixing `AND` and `OR` was evaluated with skip indexes (`use_skip_indexes_for_disjunctions`, enabled by default). If a `minmax` or `set` index could not evaluate one atom of the predicate for a granule, the per-atom verdicts recorded for that granule were shifted, so one atom's verdict was applied to another and granules holding matching rows were skipped. [#124303](https://github.com/ClickHouse/ClickHouse/pull/124303) ([Groene AI](https://github.com/groeneai)). * Fixed a query silently returning fewer rows after an unrelated `JOIN` had run: a join runtime filter moved into a read's `PREWHERE` made the read record a false "no matching rows" verdict in the query condition cache. [#119823](https://github.com/ClickHouse/ClickHouse/pull/119823) ([Groene AI](https://github.com/groeneai)). * Fixed the query condition cache serving one session's granule verdicts to another when the two differ only in a setting a conversion function captured, such as `precise_float_parsing`, which silently dropped matching rows from the second session's result. [#119168](https://github.com/ClickHouse/ClickHouse/pull/119168) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `LEFT`, `RIGHT`, `FULL`, and `ANY` outer `JOIN` silently dropping non-matched rows when the `WHERE` filter reads a `Date32` or `Enum` column of the non-preserved side. Closes [#121115](https://github.com/ClickHouse/ClickHouse/issues/121115). [#121181](https://github.com/ClickHouse/ClickHouse/pull/121181) ([Groene AI](https://github.com/groeneai)). * Fixed a join runtime filter silently dropping rows whose join key is a float. When the build side had a single distinct value the filter replaced set membership with `equals`, but a hash join compares float keys bitwise, so `NaN` did not match itself and `0.0` matched `-0.0`. [#119720](https://github.com/ClickHouse/ClickHouse/pull/119720) ([Groene AI](https://github.com/groeneai)). * Fix wrong values after lazy materialization expands a sparse column during a `JOIN`: rows could receive values from other rows. [#124442](https://github.com/ClickHouse/ClickHouse/pull/124442) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * `ALTER TABLE ... EXECUTE expire_snapshots` on an `Iceberg` table now always keeps the head snapshot of every branch, even when the table property `history.expire.min-snapshots-to-keep` is `0`. Previously such a table could lose all its data files. Closes [#124333](https://github.com/ClickHouse/ClickHouse/issues/124333). [#125365](https://github.com/ClickHouse/ClickHouse/pull/125365) ([Groene AI](https://github.com/groeneai)). * Fixed an `ssl_certificate` authentication bypass where an embedded NUL byte in a certificate's Common Name (for example `john\0.evil.com`) was truncated server-side to `john`, letting a holder of such a certificate authenticate as a different user. [#117403](https://github.com/ClickHouse/ClickHouse/pull/117403) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed one failed write to a non-replicated `MergeTree` table's deduplication log (with `non_replicated_deduplication_window` greater than zero) making every following `INSERT` into that table fail with `Cannot write to canceled buffer` until the server was restarted. The log now replaces a writer that a failed flush left unusable, no longer finalizes such a writer while rotating or shutting down, and no longer stops removing its outdated files, which could also make `ALTER TABLE ... MODIFY SETTING non_replicated_deduplication_window` fail. [#118350](https://github.com/ClickHouse/ClickHouse/pull/118350) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results after `ALTER TABLE ... MODIFY COLUMN AFTER ` on a column used in `PARTITION BY`. Closes [#92834](https://github.com/ClickHouse/ClickHouse/issues/92834). [#120862](https://github.com/ClickHouse/ClickHouse/pull/120862) ([Groene AI](https://github.com/groeneai)). * Fixed a merge or a mutation making a table unreadable (`Stream ... is not found`) when a column that a part does not hold is dropped and its name is also the name of a subcolumn of another column, for example `a.size0` next to an `Array` column `a`. [#118246](https://github.com/ClickHouse/ClickHouse/pull/118246) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong rows returned by `ORDER BY ... LIMIT` over a `MergeTree` table read in the order of a function of its sorting key, for example `ORDER BY toStartOfMonth(date) DESC, i`, when the sorting-key column uses sparse serialization. [#109424](https://github.com/ClickHouse/ClickHouse/pull/109424) ([Groene AI](https://github.com/groeneai)). * `ALTER TABLE ... DROP PARTITION` and other partition commands with an invalid or out-of-range `Date` or `DateTime` literal, such as `'2024-02-30'` or `'9999-01-01'`, no longer silently act on a different partition. They now raise an error. [#122419](https://github.com/ClickHouse/ClickHouse/pull/122419) ([DeviousCardi](https://github.com/DeviousCardi)). * Fixed wrong query results and unsorted parts in `MergeTree` tables whose sorting, primary or partition key reads a subcolumn of a column (for example ``PRIMARY KEY cityHash64(`c1.1`)`` or `ORDER BY t.a`) when a column `TTL` or an `UPDATE` changes that column. A column `TTL` on such a column is now refused as on any other key column, `UPDATE` of a column that a `MATERIALIZED` key column of this kind is computed from is refused, and TTL merges of a table already stored with such a `TTL` fail instead of writing unsorted parts. [#123628](https://github.com/ClickHouse/ClickHouse/pull/123628) ([Groene AI](https://github.com/groeneai)). * After a canary OOM, only the merges and mutations that are already running are cancelled. Newly scheduled ones now run as usual instead of all later merges and mutations staying cancelled until server restart. Closes [#121748](https://github.com/ClickHouse/ClickHouse/issues/121748). [#123870](https://github.com/ClickHouse/ClickHouse/pull/123870) ([Javier G. Sogo](https://github.com/jgsogo)). * Fixed a spurious `Cannot schedule a task: no free thread ... (CANNOT_SCHEDULE_TASK)` error that rejected a valid `SELECT` over a `MergeTree` table when it had more parts to analyse than the query had threads and their analysis took longer than `lock_acquire_timeout`. Index analysis now runs one pool job per thread instead of one per part, which also makes it faster for queries over many parts. [#116822](https://github.com/ClickHouse/ClickHouse/pull/116822) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and `CANNOT_READ_ALL_DATA` or `INCORRECT_DATA` errors when reading `MergeTree` parts in packed storage (`min_bytes_for_full_part_storage`), for example with parallel replicas, with a patch part of a lightweight update or delete, or when reading a `JSON` subcolumn. [#116920](https://github.com/ClickHouse/ClickHouse/pull/116920) ([Groene AI](https://github.com/groeneai)). * Fixed `ILLEGAL_COLUMN` (`Illegal column ... of first argument of function to...`) raised by every query touching a column that was dropped and re-added with a different type, for as long as a part still carried the old column. The re-added column now reads as its default, as it does once the drop's mutation rewrites the part. Queries served from a projection no longer return the old values of such a column. [#118585](https://github.com/ClickHouse/ClickHouse/pull/118585) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed reading stale data, or a `Sizes of nested column and null map of Nullable column are not equal` / `Got empty stream for SerializationLowCardinality keys` error, from a column that was turned into `ALIAS` or `EPHEMERAL` (or dropped while its partition was detached) and later made physical again, when a mutation that rewrites only some columns (for example `MATERIALIZE INDEX`) ran on a Wide part in between. [#124516](https://github.com/ClickHouse/ClickHouse/pull/124516) ([Groene AI](https://github.com/groeneai)). * Fixed a use-after-free (on some versions a crash with `Pure virtual function called!`) on `INSERT`, merge or `SELECT ... FINAL` of a `SummingMergeTree` table with a `Nested` column whose name ends in `Map` and whose element type has `LowCardinality` inside an `Array` or `Map`, for example `Array(LowCardinality(String))`. [#125533](https://github.com/ClickHouse/ClickHouse/pull/125533) ([Groene AI](https://github.com/groeneai)). * Fix a `MergeTree` table with a rows `TTL` and a `text` index never dropping its expired parts: the `TTLDrop` merge failed with `Text index transform for index ... not found` and retried forever. [#113385](https://github.com/ClickHouse/ClickHouse/pull/113385) ([Groene AI](https://github.com/groeneai)). * Fixed dropping whole parts with TTL (`ttl_only_drop_parts = 1`) still reading all data from large, fully expired parts. [#108626](https://github.com/ClickHouse/ClickHouse/pull/108626) ([Michael Jarrett](https://github.com/EmeraldShift)). * Fixed a server crash on `CREATE TABLE` or `ALTER TABLE ... MODIFY TTL` when a `TTL` expression uses the result of `arrayMap` with a constant lambda, such as `toString(arrayMap(x -> CAST(1, 'Dynamic'), arr))`. [#122868](https://github.com/ClickHouse/ClickHouse/pull/122868) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` (`Not found column _row_exists in block`) that permanently blocked all further mutations of a `MergeTree` table after a lightweight `DELETE` run with `lightweight_delete_mode = 'lightweight_update'` was followed by a heavyweight `ALTER TABLE ... UPDATE` and then by another mutation rewriting all columns. Closes [#115570](https://github.com/ClickHouse/ClickHouse/issues/115570). [#115665](https://github.com/ClickHouse/ClickHouse/pull/115665) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER DELETE` and `ALTER UPDATE` silently skipping rows when the predicate references an `ALIAS` column whose definition contains a non-deterministic function such as `now`. [#117273](https://github.com/ClickHouse/ClickHouse/pull/117273) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a patch part whose `source_parts.dat` lost its content to a zeroed block loading as an empty index, which silently unapplied an acknowledged lightweight `UPDATE` and then let background cleanup delete the only copy of it. Such a part is now detached as broken. [#119585](https://github.com/ClickHouse/ClickHouse/pull/119585) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALTER TABLE ... DELETE` and `ALTER TABLE ... UPDATE` skipping a whole partition when the parts the initiating replica holds in it are all empty: rows another replica had acknowledged in that partition survived the mutation on every replica. [#119149](https://github.com/ClickHouse/ClickHouse/pull/119149) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a pending lightweight `DELETE` read with `apply_mutations_on_fly = 1` evaluating its predicate over patch parts created after it, which made rows disappear from the result and reappear after the mutation materialized. Closes [#118049](https://github.com/ClickHouse/ClickHouse/issues/118049). [#118919](https://github.com/ClickHouse/ClickHouse/pull/118919) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119306](https://github.com/ClickHouse/ClickHouse/pull/119306) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed lightweight `UPDATE` for parts adopted from another table via `REPLACE PARTITION` or `ATTACH PARTITION FROM`: after such a part was cloned again (by a mutation that does not modify it, by a clone instead of a fetch of a replicated mutation, or by a move between disks), applying patch parts could silently drop updated rows in release builds and raise the `current_version != existing_version` exception in debug and sanitizer builds. [#115979](https://github.com/ClickHouse/ClickHouse/pull/115979) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed lightweight `UPDATE ... ON CLUSTER` on `ReplicatedMergeTree` tables running on every replica of a shard instead of once per shard, which applied assignments such as `SET v = v + 1` once per replica. A plain `UPDATE` in a `Replicated` database with more than one shard had the same problem. [#122393](https://github.com/ClickHouse/ClickHouse/pull/122393) ([Pratham Nayak](https://github.com/protomn)). * Fixed a concurrent heavyweight `ALTER TABLE ... UPDATE`/`DELETE` silently dropping an acknowledged lightweight `UPDATE` on a plain `MergeTree` table. [#119155](https://github.com/ClickHouse/ClickHouse/pull/119155) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed mutations such as `ALTER TABLE ... UPDATE` failing with `CANNOT_READ_ALL_DATA` on every attempt, and never completing, when they read a column together with its `LowCardinality` subcolumn in a Wide part: a `Dynamic` column and its `LowCardinality(String)` subcolumn, or a `Map` with `map_serialization_version = 'with_buckets'` and its `keys` or `values` subcolumn. The problem appeared in 26.6. [#123160](https://github.com/ClickHouse/ClickHouse/pull/123160) ([Groene AI](https://github.com/groeneai)). * Fixed an acknowledged lightweight `UPDATE` being invisible on `ReplicatedMergeTree` when it targets a column whose `RENAME COLUMN` mutation has not materialized yet. [#119144](https://github.com/ClickHouse/ClickHouse/pull/119144) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a lightweight `UPDATE` made before `ALTER TABLE ... RENAME COLUMN` on a `ReplicatedMergeTree` table being lost when a merge ran before the rename's mutation (for example with `apply_patches_on_merge = 0`): queries returned the old value, and the mutation then dropped the update from disk. [#124653](https://github.com/ClickHouse/ClickHouse/pull/124653) ([Groene AI](https://github.com/groeneai)). * Fixed a wrong result of a mutation of a `MergeTree` table when several pending mutations are executed as a single task and a patch part, created by a lightweight `UPDATE`, has a version in between: the patch was applied before the commands that precede it, so they were evaluated over data they must not see. [#119127](https://github.com/ClickHouse/ClickHouse/pull/119127) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ReplicatedMergeTree` TTL processing stalling indefinitely when a TTL merge that drops fully expired parts was larger than half the free disk space. Such a merge deletes every row and therefore needs no room for its source parts, but it was postponed on their size, and the postponed entry kept occupying the only `max_replicated_merges_with_ttl_in_queue` slot, so no TTL merge ran and no space was ever freed. It is now admitted when the table's only TTL is an unconditional rows TTL. Closes [#119775](https://github.com/ClickHouse/ClickHouse/issues/119775). [#119983](https://github.com/ClickHouse/ClickHouse/pull/119983) ([Groene AI](https://github.com/groeneai)). * Fixed `ATTACH PARTITION` on `ReplicatedMergeTree` failing to rename a broken part in `detached/` aside, so every later `ATTACH PARTITION` for that partition kept failing on it. [#116734](https://github.com/ClickHouse/ClickHouse/pull/116734) ([Groene AI](https://github.com/groeneai)). * Fixed the replication queue never draining when a part is present locally but its node in `ZooKeeper` is missing: the `GET_PART` entry for such a part re-downloaded it and failed with `DUPLICATE_DATA_PART` on every retry. [#119189](https://github.com/ClickHouse/ClickHouse/pull/119189) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Writing the empty covering part of a replicated table now happens before the covered parts are removed from the working set, so an exception during the operation no longer leaves the table with unexpected data parts and readonly after reload. [#122610](https://github.com/ClickHouse/ClickHouse/pull/122610) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Background Keeper users now retry a `get_zookeeper_lock_acquire_timeout_ms` timeout like other Keeper errors, so a `ReplicatedMergeTree` table attached during a transient Keeper outage no longer stays read-only. [#124327](https://github.com/ClickHouse/ClickHouse/pull/124327) ([Groene AI](https://github.com/groeneai)). * A table read by a scalar subquery inside a `CONSTRAINT` is now a loading dependency of the table that declares the constraint. Previously such a table could be dropped, after which the server did not start, because attaching the dependent table executes the subquery. [#114923](https://github.com/ClickHouse/ClickHouse/pull/114923) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server startup failure after creating a table with a `quantile*` state column whose level is a `Decimal`, a wide integer, a `UUID`, an `IPv4`, or an `IPv6`. Such a level was persisted into the metadata as a quoted string, which reparsed as a `String` and was rejected, so metadata loading failed with `CANNOT_CONVERT_TYPE` and the server did not start. The same `CREATE` also failed outright on a `Replicated` database. [#119810](https://github.com/ClickHouse/ClickHouse/pull/119810) ([Groene AI](https://github.com/groeneai)). * A table whose stored definition carries a column `TTL` inherited from its source table now loads again on engines that do not support `TTL` (`Merge`, `Buffer`, `Distributed`, `Remote`). Previously such a table could not be loaded, which failed the load of the whole database, and with `async_load_databases = 0` the server refused to start. The `TTL` stays in the definition and has no effect; `ALTER TABLE ... MODIFY COLUMN ... REMOVE TTL` removes it. [#122273](https://github.com/ClickHouse/ClickHouse/pull/122273) ([Mohammed Sohail](https://github.com/msohail22)). * Fix a `Remote` or `Cluster` database without a local shard making `SELECT count()` through the `merge` table function return `0`, and making the server fail to start when a dictionary reads one of its tables. Closes [#122612](https://github.com/ClickHouse/ClickHouse/issues/122612). [#122661](https://github.com/ClickHouse/ClickHouse/pull/122661) ([Groene AI](https://github.com/groeneai)). * Fix `Part ... intersects previous part ...` when attaching a `MergeTree` table after `REPLACE PARTITION`, if parts outdated by an earlier `DROP PART` were still on disk. The server could fail to start because of it. [#120980](https://github.com/ClickHouse/ClickHouse/pull/120980) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Part ... intersects next part ...` when loading a `MergeTree` table after `REPLACE PARTITION` into it ran concurrently with `MOVE PARTITION` from it. The server could fail to start because of it. [#123780](https://github.com/ClickHouse/ClickHouse/pull/123780) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `S3`, `HDFS`, `AzureBlobStorage`, and `URL` tables over a named collection with `format = 'auto' NOT OVERRIDABLE` failing to load with `Override not allowed for 'format'`, which could stop the server from starting. The format that ClickHouse inferred and stored at `CREATE` time is now accepted when the table is loaded again; a format written by the user over the locked value is still refused. [#124215](https://github.com/ClickHouse/ClickHouse/pull/124215) ([Groene AI](https://github.com/groeneai)). * Restored compatibility with Azure Blob Storage endpoints that omit the `ETag` header. [#120419](https://github.com/ClickHouse/ClickHouse/pull/120419) ([alesapin](https://github.com/alesapin)). * Fix `Part ... intersects previous part ...` when loading a `MergeTree` table after a `TRUNCATE`, `DROP PARTITION` or `MOVE PARTITION TO TABLE` failed to commit (e.g. because of a memory limit or a disk error) and a merge ran before the restart. The server could fail to start because of it. [#122375](https://github.com/ClickHouse/ClickHouse/pull/122375) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A `PROJECTION` that could not be analyzed when the table was loaded at server startup is no longer deleted from the table's persisted metadata by the next `ALTER`, which also orphaned its already-materialized data with no error at any step. While such a declaration is present, an `ALTER` that rewrites the table metadata is now refused (`ALTER TABLE ... DROP PROJECTION` and `CLEAR PROJECTION` still work), so it survives until the cause is removed or the user drops it. [#118378](https://github.com/ClickHouse/ClickHouse/pull/118378) ([Groene AI](https://github.com/groeneai)). * Fixed non-deterministic conditions being moved to `PREWHERE` in `SELECT ... FINAL` when `optimize_move_to_prewhere` is enabled. [#109538](https://github.com/ClickHouse/ClickHouse/pull/109538) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixes wrong results with `query_plan_optimize_lazy_final` when the primary key is nullable and holds a `NULL`: the `NULL`-keyed row was silently missing from the result of a `FINAL` query. [#112857](https://github.com/ClickHouse/ClickHouse/pull/112857) ([Groene AI](https://github.com/groeneai)). * Fixed rows silently dropped by index analysis when a `DateTime64` column is compared with a constant of another scale and the sorting or partition key contains a `CAST` of that column, such as `ORDER BY d::String`. [#118688](https://github.com/ClickHouse/ClickHouse/pull/118688) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix rows silently dropped by partition, primary-key, and skip-index pruning, and by `Iceberg` partition pruning, when a string constant is compared with a `DateTime` or `DateTime64` column whose type name carries no time zone and the effective `session_timezone` differs from the one that column's type was built with. [#120114](https://github.com/ClickHouse/ClickHouse/pull/120114) ([Groene AI](https://github.com/groeneai)). * Fixed a `minmax` skip index, a `set` skip index, part-level minmax pruning, and statistics-based part pruning dropping rows that hold a `NaN` when the query negates a float comparison, for example `WHERE NOT (val BETWEEN 0 AND 3)`. Closes [#106533](https://github.com/ClickHouse/ClickHouse/issues/106533). Closes [#106948](https://github.com/ClickHouse/ClickHouse/issues/106948). [#107074](https://github.com/ClickHouse/ClickHouse/pull/107074) ([Groene AI](https://github.com/groeneai)). * Fixed silently missing rows for a range predicate on `toDayOfWeek(key, mode)` with the Sunday-first modes `2` and `3` over a date primary key. [#117683](https://github.com/ClickHouse/ClickHouse/pull/117683) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ngrambf_v1`, `tokenbf_v1`, and `sparse_grams` skip indexes silently dropping matching rows when an `IPv6` column is compared with a string literal, as in `WHERE ip = '2001:db8::'`. [#113157](https://github.com/ClickHouse/ClickHouse/pull/113157) ([Groene AI](https://github.com/groeneai)). * Fixed a `set` skip index whose expression has type `Dynamic` or `Variant`, which includes an undeclared `JSON` path such as `j.a`, silently returning no rows for `!=` and `IS NULL`. [#120780](https://github.com/ClickHouse/ClickHouse/pull/120780) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when a query converts a `Date` or `Date32` column to `DateTime` and the column's values leave the range of day numbers that fits into 32-bit seconds. Key analysis, statistics part pruning, and reading in sorting-key order assumed the conversion preserves order, so a filter could skip parts holding matching rows or answer `count` from metadata without filtering, and `ORDER BY` could return rows out of order. Affects `toDateTime`, `toDateTime32`, `CAST(... AS DateTime)`, and `accurateCast`. [#117031](https://github.com/ClickHouse/ClickHouse/pull/117031) ([Groene AI](https://github.com/groeneai)). * Fix rows silently dropped by primary-key pruning when `toDayOfWeek` is applied to a `DateTime` column whose time zone differs from the session one, or over a `Date32`/`DateTime64` key that leaves the `Date` range. `toDayOfWeek` with a Sunday-first `week_mode` can use the primary key again. [#118655](https://github.com/ClickHouse/ClickHouse/pull/118655) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed rows silently missing from a query over a table whose primary key is a `Tuple` with a `Nullable` element holding `NULL`: primary-key analysis dropped whole granules of matching rows. [#118945](https://github.com/ClickHouse/ClickHouse/pull/118945) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `JSONAllPaths` skip index silently pruning every granule for a filter on a subcolumn inside a declared typed path of a `JSON` column (for example `json.a.b` for `JSON(a JSON)` or `JSON(a Tuple(b Int64))`). Such a filter no longer uses the index. [#119321](https://github.com/ClickHouse/ClickHouse/pull/119321) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix rows silently dropped by primary key and partition pruning when a `MergeTree` key expression applies a timezone-observing function such as `toYYYYMM`, `toString` or a `CAST` to `String` to a `DateTime`/`DateTime64` column and a compared constant declares a different timezone. Both a scalar comparison constant and an `IN`-set element are affected, including when the differing timezone sits under an `Array`, `Map`, `Tuple`, `Nullable`, `LowCardinality` or `Variant` wrapper that only one side carries, when a custom type name such as `SimpleAggregateFunction` wraps that composite on one side only, and when a sibling leaf differs only by a custom type name (`Bool` against `UInt8`). Also fix the same loss when a compared `Tuple` and the key column's disagree on explicit element names, which `toJSONString` renders as an object against an array; no timezone difference is needed. [#113781](https://github.com/ClickHouse/ClickHouse/pull/113781) ([Groene AI](https://github.com/groeneai)). * Fixed rows silently dropped, or counted without being filtered, when a `DateTime64` constant of a finer scale than the column is compared with `!=`, `NOT IN`, or `IN`: index analysis truncated the constant and treated the truncated value as the exact point the query asked for, pruning the granule or partition that held a matching row. [#118957](https://github.com/ClickHouse/ClickHouse/pull/118957) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed index analysis for a condition that wraps the primary key in a widening integer cast such as `toInt64` and then applies another monotonic function such as `negate` or `abs`. Such a condition could silently drop matching rows, or fail with `LOGICAL_ERROR: abs's argument does not match the expected data type`. Present since 26.5. [#116699](https://github.com/ClickHouse/ClickHouse/pull/116699) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from primary-key analysis for predicates such as `toDate(dt64_key) >= ...` and `toDateTime(date32_key) >= ...`: at the default `date_time_overflow_behavior = 'ignore'` the conversion wraps for values the result type cannot hold, so matching rows silently disappeared and `count()` could overcount. [#117394](https://github.com/ClickHouse/ClickHouse/pull/117394) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed partition pruning skipping a partition whose rows match a filter like `NOT (c <=> f(partition_key))` when `f` returns `NULL` for that partition's key value (for example `moduloOrNull` or `nullIf`). Such queries returned fewer rows than with `use_partition_pruning = 0`. [#124041](https://github.com/ClickHouse/ClickHouse/pull/124041) ([Groene AI](https://github.com/groeneai)). * Fixed `use_skip_indexes = 0` also disabling part pruning by the partition-key min-max index since 26.5, which made such queries read every part of the table. [#123403](https://github.com/ClickHouse/ClickHouse/pull/123403) ([Ashish Kumar Choubey](https://github.com/aschoube)). * Fixed missing rows in queries mixing `AND` and `OR` when a `minmax` or `set` skip index over an expression with `mortonEncode` or `hilbertEncode` is combined with other skip indexes (`use_skip_indexes_for_disjunctions`, enabled by default). Closes [#124452](https://github.com/ClickHouse/ClickHouse/issues/124452). [#124784](https://github.com/ClickHouse/ClickHouse/pull/124784) ([Groene AI](https://github.com/groeneai)). * Fix a query that compared a `Dynamic` or `JSON` column with a constant, such as `d = '2024-01-05'` or `d = toDate('2024-01-05')`, and returned fewer rows when the table was ordered by a transform of that column, such as `ORDER BY d::String`. Such a comparison no longer uses the primary key; compare the key expression itself, as in `d::String = '2024-01-05'`, to keep using it. [#124476](https://github.com/ClickHouse/ClickHouse/pull/124476) ([Ashish Kumar Choubey](https://github.com/aschoube)). * Fixed a `text` index dropping matching rows when a `FixedString` is involved in the comparison. [#118972](https://github.com/ClickHouse/ClickHouse/pull/118972) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results of `hasPhrase`, `hasToken`, `hasAnyTokens`, and `hasAllTokens` over a text index with a `postprocessor` or `preprocessor` when direct read from the text index could not answer the function, for example `NOT hasPhrase` with a phrase whose tokens are all removed by a stop-word postprocessor. [#125118](https://github.com/ClickHouse/ClickHouse/pull/125118) ([Groene AI](https://github.com/groeneai)). * Fixed text-index search functions returning wrong results, including silently dropped rows, on a column whose text index declares a preprocessor. [#117774](https://github.com/ClickHouse/ClickHouse/pull/117774) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed wrong results after a mutation of a wide data part that does not store a column added by a metadata-only `ALTER TABLE ... ADD COLUMN`: the mutation wrote a column statistic for that column, so statistics-based part pruning could drop a part holding matching rows and `min` and `max` could be answered with the pre-`ALTER` default. [#121936](https://github.com/ClickHouse/ClickHouse/pull/121936) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results for `min`, `max`, and filtered `count` on a `MergeTree` column that carries a column `TTL`. When the `TTL` emptied the column, the part kept a statistic for it and queries were answered from that stale statistic instead of from the rows the part stores. [#122058](https://github.com/ClickHouse/ClickHouse/pull/122058) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from `use_query_condition_cache_for_top_k`: `SELECT ... SAMPLE ... ORDER BY ... LIMIT n` no longer reuses cache entries written by the same query without `SAMPLE`, and Top-K query condition cache entries are no longer written for reads with on-the-fly mutations or patch parts. Closes [#122564](https://github.com/ClickHouse/ClickHouse/issues/122564). [#122538](https://github.com/ClickHouse/ClickHouse/pull/122538) ([Nikita Fomichev](https://github.com/fm4v)). [#122617](https://github.com/ClickHouse/ClickHouse/pull/122617) ([Nikita Fomichev](https://github.com/fm4v)). * Fix an exception `NOT_FOUND_COLUMN_IN_BLOCK` and, in other queries, wrong results when a lambda argument has the same name as an identifier of an expression that is bound to an alias outside of that lambda, for example `SELECT number + 1 AS n, arrayMap(number -> number + n, [1, 2]) FROM numbers(2)`. [#115544](https://github.com/ClickHouse/ClickHouse/pull/115544) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results when a `GROUP BY`, `DISTINCT`, `LIMIT BY`, or window `PARTITION BY` key was `plus` or `minus` of a partition column with a non-integer constant. [#120182](https://github.com/ClickHouse/ClickHouse/pull/120182) ([Groene AI](https://github.com/groeneai)). * Fixed a regression in 26.8 where columns of a parameterized view could not be qualified by the view name (`SELECT pv.col FROM pv(p = 1)` failed with `UNKNOWN_IDENTIFIER`) and a parameterized view used as a `JOIN` operand required an explicit alias. Closes [#119837](https://github.com/ClickHouse/ClickHouse/issues/119837). [#112481](https://github.com/ClickHouse/ClickHouse/pull/112481) ([Groene AI](https://github.com/groeneai)). [#120698](https://github.com/ClickHouse/ClickHouse/pull/120698) ([Dmitry Novik](https://github.com/novikd)). * Fixed `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN` and `CANNOT_PARSE_TEXT` when a `PREWHERE` guard predicate and a throwing expression, such as a `CAST` or `toUInt64(substring(...))`, end up in the same read step: either because they read subcolumns of the same column, for example a `JSON` or `Map` column, or because a JOIN runtime filter's key computation was merged into the guarded subquery. Such a query used to fail because the throwing expression was evaluated on rows the guard had rejected. [#111954](https://github.com/ClickHouse/ClickHouse/pull/111954) ([Groene AI](https://github.com/groeneai)). * Fix wrong window function results when a `QUALIFY` condition, or a comparison derived from it by `optimize_and_compare_chain`, turns into an `indexHint` that prunes rows below the window. Closes [#121177](https://github.com/ClickHouse/ClickHouse/issues/121177). [#122975](https://github.com/ClickHouse/ClickHouse/pull/122975) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed wrong aggregate values with `optimize_aggregation_in_order_limit` when the `ORDER BY` is a strict prefix of the `GROUP BY` key and a group's rows span multiple parts. Closes [#116849](https://github.com/ClickHouse/ClickHouse/issues/116849). [#117573](https://github.com/ClickHouse/ClickHouse/pull/117573) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed wrong results of `sum`, `min`, `max`, and `avg` over arithmetic with a `Decimal` operand and a constant, for example `min` greater than `max`, a changed `sum` of a division, or a missing `DECIMAL_OVERFLOW`: `optimize_arithmetic_operations_in_aggregate_functions` no longer moves such arithmetic out of the aggregate. [#117779](https://github.com/ClickHouse/ClickHouse/pull/117779) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `IfOrNullIf` and `IfOrDefaultIf` ignoring the outer condition when used without `GROUP BY` keys or as a window function with non-`Nullable` arguments. [#122908](https://github.com/ClickHouse/ClickHouse/pull/122908) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a possible hang of a distributed `GROUP BY` query with `uniqExact` and `distributed_aggregation_memory_efficient = 0` when merging large states of two-level aggregation. [#123352](https://github.com/ClickHouse/ClickHouse/pull/123352) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed missing rows of an empty grouping set `()` in `GROUP BY GROUPING SETS` queries without aggregate functions. For example, `SELECT number FROM numbers(3) GROUP BY GROUPING SETS ((number), ())` returned 3 rows instead of 4, and `GROUPING SETS (())` returned one row per input row. Closes [#61461](https://github.com/ClickHouse/ClickHouse/issues/61461). [#123539](https://github.com/ClickHouse/ClickHouse/pull/123539) ([Groene AI](https://github.com/groeneai)). * The `uniq` aggregate function returned garbage (for example `18446743978444128518`) for cardinalities above roughly thirty billion, because its state kept only 32 bits of each hash and the correction formula overflowed. The state now additionally keeps a small sample of full 64-bit hashes (state version `1`) which the estimate switches to at about two billion elements: `SELECT uniq(number) FROM numbers_mt(100000000000)` returns `99360630405` (−0.64%), and the estimate stays accurate (about 1%) up to trillions. Below two billion nothing changes: the estimates, the memory usage, and the state format are exactly as before, and states written by older versions remain readable and merge correctly in mixed-version clusters. [#115393](https://github.com/ClickHouse/ClickHouse/pull/115393) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results for `uniq*`, `GROUP BY`, `LIMIT n BY`, `ORDER BY` after `GROUP BY`, and window `PARTITION BY` over `toString` of a `DateTime`/`DateTime64` in a time zone with a daylight-saving transition, and over `toFixedString` of a `String`. Both functions claimed to be injective while mapping distinct arguments onto the same result, and the default-on optimizations that take that claim as a proof then eliminated the wrapper. [#117238](https://github.com/ClickHouse/ClickHouse/pull/117238) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `optimize_rewrite_aggregate_function_with_if` (enabled by default) turning a short-circuit-guarded aggregate such as `sum(if(number = 0, 0, intDiv(42, number)))` into a query failure: the rewritten `sumIf(intDiv(42, number), number != 0)` evaluates its argument for every row, so the guarded expression ran on the rows the condition excluded and threw. The rewrite is now refused when the lifted branch relies on short-circuit evaluation. Closes [#85784](https://github.com/ClickHouse/ClickHouse/issues/85784). [#118697](https://github.com/ClickHouse/ClickHouse/pull/118697) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and errors of `min`, `max`, `sum`, and `avg` over an arithmetic operation with a constant with `optimize_arithmetic_operations_in_aggregate_functions` enabled (the default): on an `Array` or `Tuple` operand (`min(arr * -1)`), on an `IPv4` or `IPv6` operand (`sum(ip * 1)` threw `ILLEGAL_TYPE_OF_ARGUMENT`), for `avg` of a date, time, or interval (`avg(t + INTERVAL 500 MILLISECOND)`), for multiplication or division by zero, infinity, or `NULL` (`min(x / -0.0)`), and for a day or longer added to a `DateTime` or a `Time64` across a DST change or a month end. Closes [#122647](https://github.com/ClickHouse/ClickHouse/issues/122647). [#123908](https://github.com/ClickHouse/ClickHouse/pull/123908) ([Groene AI](https://github.com/groeneai)). * Fixed `hasAll`, `hasAny`, `hasSubstr`, `startsWith`, and `endsWith` returning wrong results on `Array(Nullable(T))`. A `NULL` element was compared as whatever value was physically stored beneath it, so these functions disagreed with `has` on identical input: `hasAll([1, 2, NULL, 4], [0])` returned `1` while `has([1, 2, NULL, 4], 0)` returned `0`. Closes [#68310](https://github.com/ClickHouse/ClickHouse/issues/68310). [#116239](https://github.com/ClickHouse/ClickHouse/pull/116239) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from JIT-compiled conversions between integers and `Decimal`. With the default `compile_expressions = 1`, a conversion the destination could not hold silently wrapped instead of raising `DECIMAL_OVERFLOW` once the expression crossed `min_count_to_compile_expression`. Closes [#118958](https://github.com/ClickHouse/ClickHouse/issues/118958). [#118994](https://github.com/ClickHouse/ClickHouse/pull/118994) ([Groene AI](https://github.com/groeneai)). * Fixed `if`, `multiIf`, and `CASE` returning a wrapped value instead of raising `DECIMAL_OVERFLOW` once JIT-compiled, when a `Decimal`, `DateTime64`, or `Time64` branch is lifted to a larger scale (with default settings, after the query had run a few times). [#123186](https://github.com/ClickHouse/ClickHouse/pull/123186) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of `extract` and `replaceRegexpAll` (and its alias `regexp_replace`) under the default-enabled `optimize_rewrite_regexp_functions`: `extract` could return a different occurrence when a leading greedy `^.*` was stripped, or a truncated result for a pattern ending with `.*$` whose only parentheses are an inline flag group, such as `(?i)b.*$`; `replaceRegexpAll` could silently lose replacements when a pattern that can match more than once (with an inline `(?m)` flag, or a tail that can match the empty string, such as `o*$`) was rewritten to `replaceRegexpOne`; and both mishandled a trailing `.*$` or `$` that is literal text inside an unterminated `\Q`. [#117228](https://github.com/ClickHouse/ClickHouse/pull/117228) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#117227](https://github.com/ClickHouse/ClickHouse/pull/117227) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118361](https://github.com/ClickHouse/ClickHouse/pull/118361) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed rows silently missing from the result of `match` with a pattern that has a zero-allowing quantifier behind an inline flag group or an empty `\Q\E` quote. [#118351](https://github.com/ClickHouse/ClickHouse/pull/118351) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the best-effort date/time parsers reading one byte past the end of the value. The parsed result of a row of a `String` column could depend on the text of the following row: a valid value was rejected as `NULL` or zero, or `addMonths` and the rest of the add-interval family returned date and time fields taken from the neighbouring row. [#122246](https://github.com/ClickHouse/ClickHouse/pull/122246) ([Groene AI](https://github.com/groeneai)). * Fixed `toStartOfMinute`, `toStartOfHour`, `toStartOfDay`, `toStartOfFiveMinutes`, `toStartOfTenMinutes`, `toStartOfFifteenMinutes`, `timeSlot`, `toTimeWithFixedDate`, `toStartOfInterval` and `dateTrunc` returning a value one second later, possibly in the next interval, for a `DateTime64` before 1970 with a fractional part. For example, `toStartOfMinute(toDateTime64('1969-12-31 23:58:59.500', 3))` returned `23:59:00`. Also fixed `toStartOfInterval` with an origin and a `MONTH`, `QUARTER` or `YEAR` interval returning a value after its argument when the argument was earlier in the day than the origin, and rounding a bucket start clipped to the end of a shorter month to the previous bucket. [#122138](https://github.com/ClickHouse/ClickHouse/pull/122138) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed accepting a column whose two serialization streams map to the same file name (for example ``Nullable(JSON(`null` Int64))`` or ``Tuple(`a` Tuple(`b` UInt64), `a.b` UInt64)``). Such a table was created successfully, but its parts were written with both streams in one file and could not be read, merged, or checked. Creating such a column now fails with `BAD_ARGUMENTS`, as a collision between two different columns already did. Existing tables still attach, with the collision logged. [#115632](https://github.com/ClickHouse/ClickHouse/pull/115632) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a valid `JOIN ON` query rejected with `TYPE_MISMATCH` by `optimize_redundant_comparisons`, which is enabled by default since 26.7. It affects a join expression whose `AND` has a `Nothing`-typed operand, for example a join key compared against an `ARRAY JOIN []` alias. [#117476](https://github.com/ClickHouse/ClickHouse/pull/117476) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results, or an `ILLEGAL_COLUMN` error, of a `JOIN` whose condition equates both a column and an expression of that column (for example `ON t1.k = t2.k AND intDiv(t1.k, 2) = intDiv(t2.k, 2)`) when a filter on that expression is pushed down to the other side of the join: the pushed-down copy applied the expression twice. Closes [#122613](https://github.com/ClickHouse/ClickHouse/issues/122613). [#122686](https://github.com/ClickHouse/ClickHouse/pull/122686) ([Groene AI](https://github.com/groeneai)). * Fixed `SAMPLE` on the first table of a `JOIN` being applied to the other joined tables too, which returned too few rows, or failed with `SAMPLING_NOT_SUPPORTED` when a joined table had no sampling key. [#124767](https://github.com/ClickHouse/ClickHouse/pull/124767) ([awss](https://github.com/awss1i)). * Fixed missing rows from a `Distributed` or `remote` query whose `WHERE` has a `NOT` over an `AND` containing a conjunct that cannot be converted to an AST, such as a lambda or `dictGet`: the pushed-down predicate became stronger than the original one. [#117620](https://github.com/ClickHouse/ClickHouse/pull/117620) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results when a query selects an `ALIAS` column together with a window function with `enable_analyzer = 1`, over a `Distributed` table (or `remote`) with two or more shards, or over a table read with parallel replicas. Values could be returned in the wrong columns with no error, or the query could fail with `NUMBER_OF_COLUMNS_DOESNT_MATCH` or `CANNOT_PARSE_DATETIME`. [#116498](https://github.com/ClickHouse/ClickHouse/pull/116498) ([Groene AI](https://github.com/groeneai)). * Fixed a distributed `SELECT` silently returning a partial result when `connection_pool_max_wait_ms` is set to a finite value and `skip_unavailable_shards = 1`. A shard whose replica the initiator had no free connection slot for was reported as an unavailable shard and dropped, so a two-shard query could return one shard's rows with exit code `0`. Such a shard now fails the query with `ALL_CONNECTION_TRIES_FAILED`, the same error the default `skip_unavailable_shards = 0` already produces. Closes [#120832](https://github.com/ClickHouse/ClickHouse/issues/120832). [#120969](https://github.com/ClickHouse/ClickHouse/pull/120969) ([Groene AI](https://github.com/groeneai)). * Fixed asynchronous TLS connections occasionally failing with an unrelated OpenSSL error left by another connection on the same OS thread. This could terminate distributed queries with a misleading network exception. [#116692](https://github.com/ClickHouse/ClickHouse/pull/116692) ([Dmitriy Borisenko](https://github.com/otselnik)). * Fixed a query over a `Distributed` table silently returning too few rows when its `WHERE` calls `joinGet`: the condition was sent to the shards, and each one answered the call against its own local `Join` table rather than the initiator's. `joinGet` is now treated as non-deterministic and stays on the initiator. [#121822](https://github.com/ClickHouse/ClickHouse/pull/121822) ([Nikita Taranov](https://github.com/nickitat)). * Fix `THERE_IS_NO_COLUMN` for a `Distributed` query that has an `IN` over the sharding key outside its `WHERE`, in the `SELECT` projection, in `GROUP BY`, in `ORDER BY`, or in `LIMIT BY`. The default-enabled `optimize_skip_unused_shards_rewrite_in` rewrote that `IN` per shard, which changed the result column's name, so the initiator could not bind the column the shard returned. [#117252](https://github.com/ClickHouse/ClickHouse/pull/117252) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed cancelling a distributed, `s3Cluster` or parallel-replicas `SELECT` taking up to `receive_timeout` (300 s by default) to take effect, and never reaching some of the remote sources, when one of them was draining the packets left over after a `LIMIT` was satisfied. [#118696](https://github.com/ClickHouse/ClickHouse/pull/118696) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort (`vector[] index out of bounds`) when reading a `Distributed` table with `prefer_global_in_and_join = 1` and an `IN` expression that the analyzer shares between two parent expressions. The local to `GLOBAL IN` rewrite now rewrites the `IN` function in place, so every reference to a shared node is promoted. [#120681](https://github.com/ClickHouse/ClickHouse/pull/120681) ([Groene AI](https://github.com/groeneai)). * Fixed `INSERT INTO SELECT ... FROM url(...)`, and the same shape with `s3` or `azureBlobStorage`, with parallel replicas enabled inserting the data once per replica. Closes [#120485](https://github.com/ClickHouse/ClickHouse/issues/120485). [#120488](https://github.com/ClickHouse/ClickHouse/pull/120488) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `INSERT INTO SELECT ... FROM url(...)` (and `s3`, `azureBlobStorage`, ...) with parallel replicas enabled inserting the data once per shard of the destination cluster: the query forwarded to the shards now names the `*Cluster` table function with the cluster of the `Distributed` table, so every shard reads only its share of the files. Such an `INSERT` also no longer writes into the shards' local tables when the destination has a deterministic sharding key, because the rows a cluster table function hands to a shard cannot satisfy that key and the resulting placement made `optimize_skip_unused_shards` return partial results; it falls back to the ordinary `INSERT ... SELECT` instead, which places the rows through the `Distributed` sink. [#121070](https://github.com/ClickHouse/ClickHouse/pull/121070) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed duplicated rows returned by `ANY RIGHT JOIN` with `any_join_distinct_right_table_keys = 1` when parallel replicas are enabled. [#123683](https://github.com/ClickHouse/ClickHouse/pull/123683) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE MATERIALIZED VIEW ... POPULATE` and `CREATE TABLE ... AS SELECT` in a `Replicated` database with parallel replicas enabled: each replica sent its population to other replicas, so its rows landed in other replicas' tables (duplicated there, missing on the replica itself), or the query failed with `UNKNOWN_TABLE`. The population now ignores the query's parallel replicas and `parallel_distributed_insert_select` settings, so each replica fills only its own table. Closes [#124446](https://github.com/ClickHouse/ClickHouse/issues/124446). [#124477](https://github.com/ClickHouse/ClickHouse/pull/124477) ([Groene AI](https://github.com/groeneai)). * Fix `S3` client refresh overwriting static authentication settings and breaking subsequent queries. [#120433](https://github.com/ClickHouse/ClickHouse/pull/120433) ([alesapin](https://github.com/alesapin)). * Fix a server abort when an Azure Blob Storage endpoint reports an asynchronous blob copy as completed without the optional `x-ms-copy-source` or `x-ms-copy-status-description` header. [#120935](https://github.com/ClickHouse/ClickHouse/pull/120935) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reads from Azure Blob Storage now verify that the returned range starts at the requested offset, so an endpoint or proxy that ignores the request `Range` fails the read with `HTTP_RANGE_NOT_SATISFIABLE` instead of delivering the wrong bytes. [#120955](https://github.com/ClickHouse/ClickHouse/pull/120955) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A bounded read from Azure Blob Storage whose response ends before the requested right bound is now reopened and, if the endpoint keeps answering short, fails with `UNEXPECTED_END_OF_FILE` instead of silently returning a truncated file. [#120963](https://github.com/ClickHouse/ClickHouse/pull/120963) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a buffer over-read on the sequential read path from Azure Blob Storage: an endpoint that answers a ranged request with more data than was requested could deliver bytes from outside the requested range to the caller. The end of a read is now decided by the locally set right bound, not by the `Content-Length` of the response, and a read is pinned to the listed generation of the blob with `If-Match`, so an object replaced during a read fails with `AZURE_OBJECT_CHANGED_DURING_READ` instead of being truncated (new setting `azure_validate_etag_on_read`, enabled by default). [#120910](https://github.com/ClickHouse/ClickHouse/pull/120910) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a disk with `object_storage_type = 'local'` and a relative `path` removing its own root directory once the last blob in it was deleted. Anything else stored in that directory was lost with it; in particular, a disk whose `path` pointed at the directory of its own `local` metadata became unusable. [#119119](https://github.com/ClickHouse/ClickHouse/pull/119119) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `S3` requests that keep failing with a retryable error being attempted `(s3_retry_attempts + 1)^2` times instead of `s3_retry_attempts + 1`. Fixed an error that a plain-HTTP `S3` endpoint sends before reading the whole request body (for example `NoSuchUpload` for an expired multipart upload) being retried as a `Broken pipe` network error instead of being reported. Closes [#124326](https://github.com/ClickHouse/ClickHouse/issues/124326). [#124557](https://github.com/ClickHouse/ClickHouse/pull/124557) ([Groene AI](https://github.com/groeneai)). * Fixed a heap use-after-free in the `s3`, `azureBlobStorage`, `hdfs`, and similar object-storage table functions and engines when an exception such as `MEMORY_LIMIT_EXCEEDED` was thrown while creating the reader of a file in a row input format such as `CSV`. [#125090](https://github.com/ClickHouse/ClickHouse/pull/125090) ([Groene AI](https://github.com/groeneai)). * Fixed exceptions such as `UNKNOWN_CODEC` when reading `MergeTree` tables through the userspace page cache (`use_page_cache_for_disks_without_file_cache = 1` with `remote_filesystem_read_method = 'read'`, or `use_page_cache_for_local_disks = 1`): files kept inside packed archives (statistics, skip indices, parts in packed storage) were read from the wrong offset. Closes [#122157](https://github.com/ClickHouse/ClickHouse/issues/122157). [#122949](https://github.com/ClickHouse/ClickHouse/pull/122949) ([Groene AI](https://github.com/groeneai)). * Fix Parquet `time32`/`time64` columns being read as `DateTime64` and then silently shifted by `session_timezone` when inserted into a `Time64` target column. Parquet `TIME` logical types now map directly to `Time64`, and the missing `Time64 -> Time` cast is added. [#106019](https://github.com/ClickHouse/ClickHouse/pull/106019) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `Parquet` reader bug where a filter on an integer column read with a type hint that reorders the file's statistics (for example `UInt32` over a plain `INT32` column, `Enum8` over a `UINT_8` one, or `IPv4` over a `UINT_64` one) could silently return fewer rows than the file holds. Such statistics are no longer used for row group or page pruning, and a `DATE` value outside the requested type's range now raises the same error with pruning as without it. Closes [#118376](https://github.com/ClickHouse/ClickHouse/issues/118376). [#121033](https://github.com/ClickHouse/ClickHouse/pull/121033) ([Groene AI](https://github.com/groeneai)). * Fixed an infinite loop in the Parquet reader on a ZSTD or Brotli compressed page that decompresses to fewer bytes than its header declares. Such a query could not be stopped by `max_execution_time` or `KILL QUERY`; it now fails with `INCORRECT_DATA`. Closes [#123274](https://github.com/ClickHouse/ClickHouse/issues/123274). [#123298](https://github.com/ClickHouse/ClickHouse/pull/123298) ([Groene AI](https://github.com/groeneai)). * Fix `Unexpected number of rows in column subchunk` (`LOGICAL_ERROR`) when filtering a Parquet file whose `Bool` column is RLE-encoded (pyarrow's default for data page v2). The filter is now applied while decoding RLE-encoded and dictionary-encoded pages, which also makes `PREWHERE` over dictionary-encoded columns of Parquet files without a page index faster (ClickBench q31 1.92 s to 1.21 s). Closes [#123483](https://github.com/ClickHouse/ClickHouse/issues/123483). [#123618](https://github.com/ClickHouse/ClickHouse/pull/123618) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when reading a `Parquet` file with a structure hint whose type differs from the stored one. Row-group and page pruning, bloom filters, and dictionary filters now hash and interpret the query constant in the stored physical type rather than the hinted one, which fixes cases such as narrowing, signedness flips, `Enum`, `Date`/`Date32`/`Time`, length-mismatched `String`/`FixedString`, and `Bool` over an integer column. Reading a `UINT_8` column with a `Bool` hint now also treats every nonzero value as `true`, so `x = true`, `x IN (true)`, and `WHERE x` agree. [#123864](https://github.com/ClickHouse/ClickHouse/pull/123864) ([Groene AI](https://github.com/groeneai)). [#121077](https://github.com/ClickHouse/ClickHouse/pull/121077) ([Groene AI](https://github.com/groeneai)). [#124285](https://github.com/ClickHouse/ClickHouse/pull/124285) ([Groene AI](https://github.com/groeneai)). * Fixed `Parquet` output failing with `Capacity error: array cannot contain more than 2147483646 bytes` or `Uncompressed page is too big` when a batch of rows held more than 2 GiB of `String` or `FixedString` data. A batch is now bounded by bytes as well as by rows, which also lowers the peak memory of the write. [#122482](https://github.com/ClickHouse/ClickHouse/pull/122482) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `SELECT ... FORMAT Arrow` failing with `Method getDataAt is not supported` for `JSON`, `Dynamic`, and `QBit` columns, and silently exporting heap addresses instead of the aggregate state for `AggregateFunction` columns. Added the `output_format_arrow_unsupported_types` setting (`throw` / `text` / `binary`, default `binary`), which supersedes `output_format_arrow_unsupported_types_as_binary`. Such columns can now also be read back into their original type, which previously failed or silently corrupted the value. `INSERT INTO ... ENGINE = ArrowFlight` now follows this setting too; it previously rejected such columns unconditionally, so an insert that used to fail may now write an opaque column. Set `output_format_arrow_unsupported_types = 'throw'` to keep rejecting it. [#115250](https://github.com/ClickHouse/ClickHouse/pull/115250) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix `O(n^2)` decompression of gzip/zlib/deflate streams whose single DEFLATE block spans the whole stream, the shape zlib-ng produces at compression level 1 (the default of the official .NET SDK): a 24 MiB gzip HTTP body that took 11 s to ingest in 26.7 now takes 0.12 s, identical to a multi-block body, with memory bounded by the buffer size instead of the block size. [#114059](https://github.com/ClickHouse/ClickHouse/pull/114059) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a segmentation fault or `std::length_error` when reading `TSV` formats with `input_format_tsv_crlf_end_of_line = 1` if an escape sequence ended exactly at the end of a read buffer. [#123861](https://github.com/ClickHouse/ClickHouse/pull/123861) ([Groene AI](https://github.com/groeneai)). * Fixed lost rows when a query over a `MySQL`, `PostgreSQL`, `SQLite`, or ODBC/JDBC table compares a column, string, number, or `Decimal`, with an `Enum` constant, also inside a tuple, an `IN` list, `Nullable`, `Variant`, or `Dynamic`, at any depth. The filter sent to the external database used the `Enum`'s number, while ClickHouse compares a string column with the `Enum`'s name, and `PostgreSQL` rejected such a query. An `IN` list or row of `Variant` or `Decimal` constants is no longer sent as one quoted string. [#123843](https://github.com/ClickHouse/ClickHouse/pull/123843) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of a negated `IN` with a `NULL` in the list, e.g. `x NOT IN (1, NULL)` or `NOT (x IN (1, NULL))`, over `MySQL`, `PostgreSQL`, `SQLite`, and `ODBC` tables: the filter pushed to the external database now leaves the `NULL` members out of the list, the way ClickHouse evaluates the set. [#124339](https://github.com/ClickHouse/ClickHouse/pull/124339) ([Groene AI](https://github.com/groeneai)). * Fixed silent data loss when an `EmbeddedRocksDB` table is renamed. `RENAME TABLE` now moves the table's `RocksDB` directory to the table's new location, so the rows are still there after a server restart. Previously the directory was left behind, and after a restart the table opened a fresh empty `RocksDB` while the real rows stayed orphaned under the old path. [#112224](https://github.com/ClickHouse/ClickHouse/pull/112224) ([Groene AI](https://github.com/groeneai)). * `optimize_inverse_dictionary_lookup` (enabled by default) no longer rewrites `dictGet(dict, attr, key) = const` when a `Variant`, `Dynamic`, or `JSON` appears in the type of `key` or in the dictionary's declared key type. The rewritten predicate is not equivalent there, so the query could fail with `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN`, `ILLEGAL_TYPE_OF_ARGUMENT`, or `NO_COMMON_TYPE`, or return wrong results. [#120888](https://github.com/ClickHouse/ClickHouse/pull/120888) ([Groene AI](https://github.com/groeneai)). * Fixed SQL injection through identifiers in the `PostgreSQL` integration: relation, schema, column, publication, and replication-slot names used by the `PostgreSQL` table engine, the `postgresql` table function, `PostgreSQL` dictionary sources, and `MaterializedPostgreSQL` were escaped with backslashes, which PostgreSQL does not honour inside a quoted identifier, so a remote name containing a double quote could end the identifier and let the rest execute on the PostgreSQL server. Relations and columns whose names contain a double quote or a backslash can now be read and replicated, including those listed in `materialized_postgresql_tables_list`; an empty element in that setting is now rejected. [#119216](https://github.com/ClickHouse/ClickHouse/pull/119216) ([Groene AI](https://github.com/groeneai)). [#119103](https://github.com/ClickHouse/ClickHouse/pull/119103) ([Groene AI](https://github.com/groeneai)). * Fixed `PARALLEL WITH`, `EXECUTE AS`, and the `QueryRunner` table engine skipping the access checks of the `CREATE` statements they execute, which allowed creating tables and databases without the corresponding privileges and defeated the `readonly` and `allow_ddl` settings. [#119848](https://github.com/ClickHouse/ClickHouse/pull/119848) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an `HDFS` database serving a table from its internal cache without the `READ ON HDFS` grant. The grant and the `remote_url_allow_hosts` filter were checked only while a table name was resolved for the first time, so once any user holding the grant had read a name, every user with `SELECT` on the database was served the contents of the file, and that table kept being served after its host was removed from the allow-list. `EXISTS TABLE`, which requires only `SHOW TABLES`, answered from the same cache and so reported which names other users had resolved. Both checks now run before the cache is consulted, and the existence probe no longer consults it. [#119523](https://github.com/ClickHouse/ClickHouse/pull/119523) ([Groene AI](https://github.com/groeneai)). * Fixed filesystem containment checks accepting a path with an embedded NUL byte. A `DeltaLakeLocal` or `IcebergLocal` table could be pointed outside of `user_files` this way. [#119846](https://github.com/ClickHouse/ClickHouse/pull/119846) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed several engines disclosing an `Alias` table's target schema to a user who holds no privilege on that target. Reading the metadata of an `Alias` through any of them now requires table-level `SHOW COLUMNS` on its target, which is what a `DESCRIBE` of the target already requires. [#117572](https://github.com/ClickHouse/ClickHouse/pull/117572) ([Groene AI](https://github.com/groeneai)). * `remote(, 'db', 'table', 'user', 'password')`, and the same form of `remoteSecure` and of the `Remote` and `RemoteSecure` table engines, no longer write the password into `system.query_log`, `system.text_log`, and the server log in cleartext. [#121474](https://github.com/ClickHouse/ClickHouse/pull/121474) ([Groene AI](https://github.com/groeneai)). * Fixed SQL injection through identifiers when inserting into a table with the `SQLite` engine. It is now also possible to insert into a SQLite table whose name, or column name, legitimately contains a double quote or a backslash. [#117421](https://github.com/ClickHouse/ClickHouse/pull/117421) ([Groene AI](https://github.com/groeneai)). * Fixed credential masking for `nats_url`, `rabbitmq_address`, the `jdbc` and `odbc` connection string, and the `format_avro_schema_registry_url`, `url_base`, and `s3_base` settings in `SHOW CREATE`, `system.tables`, and the query text in `system.query_log`. Closes [#121257](https://github.com/ClickHouse/ClickHouse/issues/121257). [#121425](https://github.com/ClickHouse/ClickHouse/pull/121425) ([Groene AI](https://github.com/groeneai)). * Fixed a `Backup` database engine printing the credentials of its destination in plaintext. `SHOW CREATE DATABASE`, `system.databases.engine_full`, `system.query_log`, and the messages that reject a malformed locator no longer expose an `AzureBlobStorage` `account_key`, a connection string, or a presigned URL regardless of `format_display_secrets_in_show_and_select`. [#117429](https://github.com/ClickHouse/ClickHouse/pull/117429) ([Groene AI](https://github.com/groeneai)). * `KILL TRANSACTION ... ON CLUSTER`, `KILL PART_MOVE_TO_SHARD ... ON CLUSTER`, `SYSTEM STOP THREAD FUZZER ON CLUSTER`, `SYSTEM START THREAD FUZZER ON CLUSTER`, and `SYSTEM RESET COVERAGE ON CLUSTER` no longer run without checking any privilege. Each now requires what its non-`ON CLUSTER` spelling has always required. [#118360](https://github.com/ClickHouse/ClickHouse/pull/118360) ([Groene AI](https://github.com/groeneai)). * Fixed metadata disclosure through a chain of `Alias` tables. `DESCRIBE`, `hasColumnInTable`, `system.columns`, `system.tables`, `system.projections`, `system.constraints`, and `system.data_skipping_indices` now require the privilege on every table an `Alias` resolves through, as reading its rows already did. [#118967](https://github.com/ClickHouse/ClickHouse/pull/118967) ([Groene AI](https://github.com/groeneai)). * Fixed a bug where `REVOKE GRANT OPTION FOR` was accepted but then omitted from `SHOW GRANTS`, from `system.grants`, and from the persisted access metadata whenever another revoked database, table, column, or named-collection name shared a prefix with it. The grant option therefore came back after a server restart, and the user could pass it on with `GRANT CURRENT GRANTS`. [#122078](https://github.com/ClickHouse/ClickHouse/pull/122078) ([Groene AI](https://github.com/groeneai)). * Fixed settings constraints from a settings profile not being enforced against the other setting assignments in the same statement. `SET profile = 'p', = ` could override a `CONST`, `MIN`/`MAX`, or `readonly` constraint that profile `p` installs. The same escape existed through an in-query `SETTINGS` clause, HTTP URL parameters, and Arrow Flight session options. [#120376](https://github.com/ClickHouse/ClickHouse/pull/120376) ([Raúl Marín](https://github.com/Algunenano)). * Fixed row policies on `Memory` tables being ignored when a read is executed from a serialized query plan, such as a distributed plan or a parallel-replicas plan. Since `Memory` gained `PREWHERE` support, such reads could return rows the policy excludes. [#122503](https://github.com/ClickHouse/ClickHouse/pull/122503) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an integer underflow in the `MySQL` protocol handshake handler that could cause the server to buffer an unbounded amount of data from an unauthenticated client. [#115776](https://github.com/ClickHouse/ClickHouse/pull/115776) ([Dmitrii Tishkin](https://github.com/dmi03)). * Statistics-based part pruning is now disabled for tables with row policies. [#122035](https://github.com/ClickHouse/ClickHouse/pull/122035) ([alesapin](https://github.com/alesapin)). * A row policy on a `Set`-engine table was accepted but never applied: the right side of `IN` read the prebuilt set directly and returned the rows the policy hides. Such a query now fails with `ACCESS_DENIED`, because the `Set` engine has no read path that could filter those rows, and `IN set_table` now requires `SELECT` on all columns of the table. On `Replicated` databases, `ALTER ... DELETE`, `ALTER ... UPDATE`, `DELETE`, and `UPDATE` now run the initiator's checks as the submitting user, so the same row-policy and access checks apply there too; with `ON CLUSTER`, run the query on a host that has the right-hand `Set` table, or enable `distributed_ddl_use_initial_user_and_roles` on every host and set `distributed_ddl_entry_format_version` to at least `8`. Queries that used a `Set` table covered by a database-wide row policy ran unfiltered before and now throw. [#120312](https://github.com/ClickHouse/ClickHouse/pull/120312) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). [#122141](https://github.com/ClickHouse/ClickHouse/pull/122141) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the `Kafka` and `NATS` table engines ignoring `remote_url_allow_hosts`: broker addresses from `kafka_broker_list`, `nats_url`, and `nats_server_list` are now checked against the remote host filter. A forbidden address is rejected at `CREATE TABLE` with `UNACCEPTABLE_URL`, as the `RabbitMQ` engine already does. [#121825](https://github.com/ClickHouse/ClickHouse/pull/121825) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * HTTP headers in `HTTP` dictionary sources are now hidden in `SHOW CREATE DICTIONARY`, `system.tables`, and query logs, like passwords. They can still be shown with `display_secrets_in_show_and_select`. [#122673](https://github.com/ClickHouse/ClickHouse/pull/122673) ([Matheus Nerone](https://github.com/Manerone)). * Fixed secret settings of `S3Queue` and `AzureQueue` tables written with the legacy `s3queue_` prefix, such as `s3queue_after_processing_move_secret_access_key`, being shown in cleartext in `SHOW CREATE TABLE`, `system.tables`, `system.query_log`, and the server log. They are now hidden as they are without the prefix. [#123060](https://github.com/ClickHouse/ClickHouse/pull/123060) ([Groene AI](https://github.com/groeneai)). * Column-level `SELECT` privileges are now enforced for columns used only inside `indexHint` and for `ALIAS` columns referenced in `PREWHERE`. As in `WHERE`, referencing an `ALIAS` column in `PREWHERE` requires the grant on the alias name; a grant on its source columns is no longer sufficient. A metadata-only `ALTER` that changes a subcolumn read by a `MATERIALIZED` column through an `ALIAS` is now refused like a direct reference. [#120516](https://github.com/ClickHouse/ClickHouse/pull/120516) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fixed a row policy bypass: a direct `JOIN` with an `EmbeddedRocksDB`, `KeeperMap`, or `Redis` table in the legacy analyzer (`enable_analyzer = 0`) ignored the row policy of the right table and returned hidden rows, and `joinGet` on a `Join` table ignored its row policy in both analyzers. Such a table is now joined through its query plan, which keeps the filter, and `JOIN` or `joinGet` against a `Join` table under a row policy throws `ACCESS_DENIED`. [#121398](https://github.com/ClickHouse/ClickHouse/pull/121398) ([Vladimir Cherkasov](https://github.com/vdimir)). * `CREATE OR REPLACE` in a `Replicated` database or with `ON CLUSTER` now requires the drop privilege for the kind of the table it replaces, so a user without `DROP DICTIONARY` can no longer replace a dictionary. [#122272](https://github.com/ClickHouse/ClickHouse/pull/122272) ([Nikolay Degterinsky](https://github.com/evillique)). * A view with `SQL SECURITY DEFINER` or `SQL SECURITY NONE` that can hide rows is now read as a whole and is opaque to the optimizations of the query that reads it; previously an expression of the outer query could be evaluated on the hidden rows and reveal them, for example through an exception message. `EXPLAIN` shows the plan of such a view only to users who could have created it or who may display secrets, and seeing the plan of such a sealed view now also requires the `CREATE VIEW` privilege. [#122330](https://github.com/ClickHouse/ClickHouse/pull/122330) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123325](https://github.com/ClickHouse/ClickHouse/pull/123325) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a credential written in cleartext to `system.query_log` and the server log when a `key = value` argument precedes the positional arguments of the `mongodb`, `redis`, `remote`, `arrowFlight`, and `ytsaurus` table functions or the `MongoDB`, `Redis`, `Remote`, `ArrowFlight`, `ExternalDistributed`, and `YTsaurus` table engines. A positional argument after a named one is now always hidden, and the value of a named-collection override whose key is written as an expression is hidden for every such engine. [#122770](https://github.com/ClickHouse/ClickHouse/pull/122770) ([Raúl Marín](https://github.com/Algunenano)). * Rewriting the body of a view with `ALTER TABLE ... MODIFY QUERY` now needs the same grants as creating it with its SQL security: `SET DEFINER` on the definer of a `SQL SECURITY DEFINER` view, or `ALLOW SQL SECURITY NONE` for a `SQL SECURITY NONE` view. An `ON CLUSTER MODIFY QUERY` fails with `UNKNOWN_TABLE` unless it runs on a host that has the view. [#121629](https://github.com/ClickHouse/ClickHouse/pull/121629) ([Sema Checherinda](https://github.com/CheSema)). * Fixed `GRANT ... ON db*.table` being silently rewritten to `db.table*`, which could cause incorrect privilege grants. The pattern `db*.table` is now rejected as a syntax error. `CREATE` / `ALTER` / `DROP ROW POLICY`, `SHOW POLICIES`, and `SHOW CREATE POLICY` now also reject prefix wildcards (`db*.*`, `db.table*`, `table*`), which were previously silently narrowed to the exact name (`db.*`, `db.table`, `table`). [#108063](https://github.com/ClickHouse/ClickHouse/pull/108063) ([Aditya Kumar](https://github.com/adityaksolves)). * Update `mongo-c-driver` to 2.5.6 and `mongo-cxx-driver` to r4.6.0, fixing an out-of-bounds write during MongoDB connection monitoring reconnects (CVE-2026-96746). [#119346](https://github.com/ClickHouse/ClickHouse/pull/119346) ([Actuele AI](https://github.com/actueleai)). [#124282](https://github.com/ClickHouse/ClickHouse/pull/124282) ([Actuele AI](https://github.com/actueleai)). [#124648](https://github.com/ClickHouse/ClickHouse/pull/124648) ([Actuele AI](https://github.com/actueleai)). * Fixed access entities (users, roles, row policies, and others) of the `local_directory` access storage being lost after a restart, or a dropped row policy making every query fail with `FILE_DOESNT_EXIST`. This happened to entities created or dropped within a minute after `SYSTEM RELOAD USERS` if the server was then killed, or at any time after `SYSTEM RELOAD USERS` failed to write the storage's `.list` files. [#125058](https://github.com/ClickHouse/ClickHouse/pull/125058) ([Groene AI](https://github.com/groeneai)). * The web terminal and the SSH interface no longer reset the settings of the user (from settings profiles, roles, and `CREATE USER ... SETTINGS`) to default values. Previously, restrictions such as `readonly`, resource limits, and `additional_table_filters` were not applied in these interfaces. [#125515](https://github.com/ClickHouse/ClickHouse/pull/125515) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Update `expat` to 2.9.0, fixing a denial of service through a crafted backup manifest during `RESTORE` (CVE-2026-66046) and the other 2.8.3–2.9.0 security fixes. [#124255](https://github.com/ClickHouse/ClickHouse/pull/124255) ([Actuele AI](https://github.com/actueleai)). * Fixed failures during pipeline expansion, such as external sorting or the merge stage of a `GROUP BY`, when the memory limit is exceeded. The query now reports the original exception instead of an inconsistent-pipeline error, a segmentation fault, or a read of freed memory. Closes [#125251](https://github.com/ClickHouse/ClickHouse/issues/125251). [#121502](https://github.com/ClickHouse/ClickHouse/pull/121502) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#125283](https://github.com/ClickHouse/ClickHouse/pull/125283) ([Groene AI](https://github.com/groeneai)). * Fixed a stored query combining `UNION` with a parenthesized `INTERSECT`/`EXCEPT` group losing its parentheses when serialized, so `X UNION ALL (Y EXCEPT Z)` was read back as `(X UNION ALL Y) EXCEPT Z`. Views returned wrong results after a server restart, and a materialized view could end up with metadata the server refuses to load, which prevented it from starting. [#111931](https://github.com/ClickHouse/ClickHouse/pull/111931) ([Groene AI](https://github.com/groeneai)). * Fixed the `compatibility` setting: `compatibility = '26.8'` no longer reverts five defaults that 26.8 shipped or forces `MergeTree` `patch_parts_version` back to `v1`, and `compatibility = '26.1'` no longer disables `use_skip_indexes_on_data_read`, which 26.1 shipped enabled. [#118666](https://github.com/ClickHouse/ClickHouse/pull/118666) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `compatibility = '25.11'` incorrectly resolving `query_plan_max_limit_for_lazy_materialization` to `10` instead of `100`. [#122768](https://github.com/ClickHouse/ClickHouse/pull/122768) ([Raúl Marín](https://github.com/Algunenano)). * Setting the obsolete `enable_analyzer` (`allow_experimental_analyzer`) to `0` is accepted again and replaced with `1`, instead of failing with `SETTING_CONSTRAINT_VIOLATION` or `INCORRECT_QUERY`. Queries, sessions, settings profiles, and client configurations that still set it keep working after an upgrade, always with the analyzer, including configurations that spell the value as a `Float64` literal. [#122401](https://github.com/ClickHouse/ClickHouse/pull/122401) ([Nikita Fomichev](https://github.com/fm4v)). [#122763](https://github.com/ClickHouse/ClickHouse/pull/122763) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a server crash on AArch64 that could happen when the query profiler sampled a thread while it was connecting to a remote server, for example with `remote` or `remoteSecure`. Capturing stack traces, used for exceptions and by the query profiler, is also several times faster. [#122834](https://github.com/ClickHouse/ClickHouse/pull/122834) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `listen_host` `::` on hosts without a global IPv6 address, such as a container on an IPv4-only Docker network: it failed with a DNS error, so the official Docker image listened on IPv4 only and `localhost` over `::1` was refused. The server now listens on the dual-stack wildcard there. gRPC and Arrow Flight use one listener when `::` is the only wildcard, and `host_regexp` / `HOST REGEXP` / `HOST LIKE` restrictions match IPv4 clients of a dual-stack listener. [#122785](https://github.com/ClickHouse/ClickHouse/pull/122785) ([Groene AI](https://github.com/groeneai)). * Fixed an issue where pressing Ctrl-R and then Ctrl-Z in the web terminal or the SSH interface suspended the whole server process. [#125512](https://github.com/ClickHouse/ClickHouse/pull/125512) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `TOO_LARGE_STRING_SIZE` (`Too large sizes of FixedString to deserialize`) when reading a correctly written part whose `Array(FixedString(N))` column holds more than 1 GiB of elements in one read span. Such a span is now read in bounded steps instead of being rejected, so the affected parts can be read and merged again without being rewritten. Closes [#116772](https://github.com/ClickHouse/ClickHouse/issues/116772). [#117385](https://github.com/ClickHouse/ClickHouse/pull/117385) ([Groene AI](https://github.com/groeneai)). * Fixed `CollapsingMergeTree` returning wrong rows when a row's `sign` is neither `1` nor `-1`. [#119900](https://github.com/ClickHouse/ClickHouse/pull/119900) ([Murphy](https://github.com/murphy-4o)). * Fixed a table that exceeds `max_table_size_rows` never being able to come back under the limit: every merge threw `TABLE_SIZE_LIMIT_EXCEEDED`, so the table could not be compacted and its part count grew until inserts started failing. [#117229](https://github.com/ClickHouse/ClickHouse/pull/117229) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a shutdown stall where a background merge or mutation left in a `MergeTree` background executor's queue kept holding its table lock, so any table lock taken later during shutdown waited for the full `lock_acquire_timeout` and then failed with `DEADLOCK_AVOIDED`. [#113653](https://github.com/ClickHouse/ClickHouse/pull/113653) ([Groene AI](https://github.com/groeneai)). * Fixed `count()` returning the number of default values instead of the number of `NULL`s when `optimize_trivial_count_with_sparsity_filter` is used on a column widened to `Nullable` by a metadata-only `ALTER TABLE ... MODIFY COLUMN`. [#120245](https://github.com/ClickHouse/ClickHouse/pull/120245) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `MergeTree` tables with `non_replicated_deduplication_window` on object storage failing to load their deduplication log after being detached and attached back without inserts in between, and fixed the deduplication log possibly dropping its active file. [#121670](https://github.com/ClickHouse/ClickHouse/pull/121670) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALTER TABLE ... COMMENT COLUMN` and `ALTER TABLE ... RENAME COLUMN` making reads of an `ALIAS` column's subcolumns, for example `arr.size0`, fail with `NO_SUCH_COLUMN_IN_TABLE` until the table was reloaded. Closes [#121421](https://github.com/ClickHouse/ClickHouse/issues/121421). [#121449](https://github.com/ClickHouse/ClickHouse/pull/121449) ([Groene AI](https://github.com/groeneai)). * Fixed `CANNOT_READ_ALL_DATA` (`while reading prefix of column ...`) during query planning when a `MergeTree` table has an active empty Wide part, left by a `TTL` drop or by a mutation that deleted all rows of a part, and the query filters on a subcolumn, such as a key of a `Map(LowCardinality(String), String)` column. [#122756](https://github.com/ClickHouse/ClickHouse/pull/122756) ([Groene AI](https://github.com/groeneai)). * Fixed `ATTACH PART` failing with `CORRUPTED_DATA` for a Compact part whose `default_compression_codec.txt` is missing, for example a part copied or restored without that file. [#119114](https://github.com/ClickHouse/ClickHouse/pull/119114) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` (`Sizes of nested column and null map of Nullable column are not equal after deserialization`) that made a `SELECT` of a `Nested` member fail with `Code: 49` until the server was restarted, after an `ALTER TABLE ... MODIFY COLUMN` on that member had been killed and a later mutation hardlinked the column into a new wide part. [#121561](https://github.com/ClickHouse/ClickHouse/pull/121561) ([Groene AI](https://github.com/groeneai)). * Wide parts no longer split a fixed-width value across two compressed blocks when `max_compress_block_size` is not a multiple of the value size. This improves the compression ratio of type-specific codecs on such tables and fixes `CANNOT_COMPRESS` from `ALP`. [#122841](https://github.com/ClickHouse/ClickHouse/pull/122841) ([Raufs Dunamalijevs](https://github.com/rienath)). * After `ALTER TABLE ... RENAME COLUMN`, the subcolumns of the renamed column, for example `c0.null` or `a.size0`, stayed resolvable under the old name. Reading them through a `Buffer` or `Merge` table, or using them in `OPTIMIZE ... DEDUPLICATE BY`, read uninitialized memory, and once a new column reused the old name they returned wrong values instead of throwing `UNKNOWN_IDENTIFIER`. [#122527](https://github.com/ClickHouse/ClickHouse/pull/122527) ([Groene AI](https://github.com/groeneai)). * Fixed excessive memory usage when loading a data part with a corrupted `columns_substreams.txt`; such a part is now detached as broken. [#124157](https://github.com/ClickHouse/ClickHouse/pull/124157) ([Murphy](https://github.com/murphy-4o)). * Fixed a `LOGICAL_ERROR` and silent data corruption when a column is named into the `skp_idx_` prefix so that its on-disk stream base equals a skip index's, making both share one marks file. Such a write is now rejected with `INCORRECT_FILE_NAME` naming both owners. [#113093](https://github.com/ClickHouse/ClickHouse/pull/113093) ([Groene AI](https://github.com/groeneai)). * Fixed reads of a renamed `Nested` leaf column returning default values from wide parts while the `RENAME COLUMN` mutation was still pending. [#119134](https://github.com/ClickHouse/ClickHouse/pull/119134) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CANNOT_COMPRESS` on merge or insert for a `Map` column with `with_buckets` serialization and `CODEC(T64)`. [#124695](https://github.com/ClickHouse/ClickHouse/pull/124695) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fixed a column added by `ALTER TABLE ... ADD COLUMN ... DEFAULT ` being filled with the type default instead of the evaluated expression when it is read through a single-conjunct explicit `PREWHERE` on the expression's dependency. [#117533](https://github.com/ClickHouse/ClickHouse/pull/117533) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the order of rows in a data part written by a merge or a mutation with `TTL ... GROUP BY ... SET` when the `SET` clause assigns a new value to a column the sorting key is calculated from. Closes [#108514](https://github.com/ClickHouse/ClickHouse/issues/108514). [#119863](https://github.com/ClickHouse/ClickHouse/pull/119863) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when merging a table that has a rows TTL together with a move (`TO DISK`, `TO VOLUME`) or recompression (`RECOMPRESS`) TTL over a different column. [#121536](https://github.com/ClickHouse/ClickHouse/pull/121536) ([Murphy](https://github.com/murphy-4o)). * Fixed `INSERT` into a table with a constant `TTL ... DELETE WHERE` whose condition is not exactly `UInt8`, for example `c > 10` over a `Nullable` column: a part where no row matched the condition was marked with the delete TTL and later rewritten by a needless TTL merge, and a condition that cannot be a filter, such as an `Array` column, was accepted instead of being rejected as in the other TTL code paths. [#122516](https://github.com/ClickHouse/ClickHouse/pull/122516) ([Groene AI](https://github.com/groeneai)). * Fixed `INSERT` into a table whose `TTL` expression changes its type under a session setting, for example a `CAST` of a `Nullable` column under `cast_keep_nullable = 1`, which failed with `LOGICAL_ERROR` `Unexpected type ... of result TTL column`. Under such a setting, `ALTER TABLE ... MODIFY TTL` could also store a `TTL` that `CREATE TABLE` rejects, after which the table could not be attached again. A stored `TTL` is now analyzed with the server settings in `INSERT` and `ALTER`, as in `CREATE TABLE`. [#122973](https://github.com/ClickHouse/ClickHouse/pull/122973) ([Groene AI](https://github.com/groeneai)). * Fixed an unsorted part with a primary index that did not match the data after a `TTL GROUP BY ... SET` merge that rewrites the column a subcolumn sorting key (such as `ORDER BY n.a`) is taken from. Queries with a condition on the sorting key could return wrong results. [#122373](https://github.com/ClickHouse/ClickHouse/pull/122373) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALTER TABLE ... MODIFY TTL` accepting a TTL whose subquery reads a table function (for example `numbers` or `remote`), a view over one, a parameterized view, a temporary table, or the table itself. After such an `ALTER` the table could not be loaded again, so a server with synchronous table loading did not start. Now the `ALTER` fails. [#123561](https://github.com/ClickHouse/ClickHouse/pull/123561) ([Groene AI](https://github.com/groeneai)). * Fixed standalone `UPDATE ... SET` substituting a `WITH` expression alias into a nested subquery that cannot resolve it. Where that name is a table name, the statement read the alias instead of the table, or silently succeeded where the equivalent `SELECT` raises `UNKNOWN_IDENTIFIER`. [#120026](https://github.com/ClickHouse/ClickHouse/pull/120026) ([Groene AI](https://github.com/groeneai)). * `allow_non_metadata_alters = 0` now also rejects the `ALTER` commands that reach a `MergeTree` table as explicit mutations: `UPDATE`, `DELETE WHERE`, the five `MATERIALIZE` forms, `APPLY DELETED MASK`, `APPLY PATCHES`, and `REWRITE PARTS`. [#115287](https://github.com/ClickHouse/ClickHouse/pull/115287) ([Groene AI](https://github.com/groeneai)). * Fixed `Cannot convert NULL value to non-Nullable type` in `ALTER TABLE ... UPDATE` with a scalar subquery when the subquery result is `Nullable` and the updated column is not. [#119926](https://github.com/ClickHouse/ClickHouse/pull/119926) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... DROP COLUMN` and `ALTER TABLE ... CLEAR COLUMN` are now refused when they would leave a part with no columns at all, instead of wedging the mutation queue with a logical error or leaving an unreadable part. [#121104](https://github.com/ClickHouse/ClickHouse/pull/121104) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a mutation such as `ALTER TABLE ... MODIFY COLUMN` or `RENAME COLUMN` committing a data part with no columns when the part held only columns the table no longer has (for example a partition detached before `DROP COLUMN` and re-attached after it), after which every read of the table failed with a logical error. Such a mutation now fails with `NO_SUCH_COLUMN_IN_TABLE`, like reads of that part, and the part and its data are kept. [#122400](https://github.com/ClickHouse/ClickHouse/pull/122400) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER TABLE ... UPDATE` and `MATERIALIZE COLUMN` on a `Memory` table after `MODIFY COLUMN` of the updated column: later reads failed with `ILLEGAL_COLUMN` or returned wrong values, and `BACKUP` failed. [#123043](https://github.com/ClickHouse/ClickHouse/pull/123043) ([Groene AI](https://github.com/groeneai)). * Fix `Found patch part ... that intersects mutation with version ...` on lightweight updates in a table created with `CREATE TABLE ... CLONE AS` or `ATTACH PARTITION ALL FROM` a table with lightweight updates. [#123620](https://github.com/ClickHouse/ClickHouse/pull/123620) ([Nikolay Degterinsky](https://github.com/evillique)). * A synchronous mutation (`mutations_sync`) on a `MergeTree` table no longer reports success when the table is detached before the mutation runs; it fails with `UNFINISHED`, and the mutation continues after the table is attached again. [#124161](https://github.com/ClickHouse/ClickHouse/pull/124161) ([Murphy](https://github.com/murphy-4o)). * `DELETE`, `UPDATE` and `ALTER ... DELETE/UPDATE` no longer refuse a `MergeTree` table of a database with `lazy_load_tables` that has not been accessed yet, and `SYSTEM DROP REPLICA ... FROM ZKPATH` no longer skips such a table when checking that the replica being dropped is not a live local one. [#118152](https://github.com/ClickHouse/ClickHouse/pull/118152) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `ReplicatedMergeTree` table whose metadata carries `table_readonly = 1` being unusable: its backup could not be restored, it could not be re-attached after `DETACH`, and `ALTER TABLE ... MODIFY SETTING table_readonly = 0` was refused. Converting a table with that setting to replicated is no longer done, by either the `convert_to_replicated` flag or `ATTACH TABLE ... AS REPLICATED`. [#119224](https://github.com/ClickHouse/ClickHouse/pull/119224) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `CHECK TABLE` on a `ReplicatedMergeTree` replica whose Keeper session was lost now fails with `ABORTED` instead of returning `0`, which means that the check found broken data. Nothing was read in that case, so the healthy table was certified as broken. [#119381](https://github.com/ClickHouse/ClickHouse/pull/119381) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the automatic `MergeTree` to `ReplicatedMergeTree` conversion silently not finishing for a table in a database with `lazy_load_tables`, which left the table read-only with no metadata in ZooKeeper and the conversion flag in place. [#118135](https://github.com/ClickHouse/ClickHouse/pull/118135) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an integer overflow in the DDL log retention check of `Replicated` databases that could cause premature removal of DDL log entries and replicas being incorrectly declared lost for a very large `logs_to_keep`. `logs_to_keep` is now limited to `2147483647`, so replicas on older releases also stay safe during a rolling upgrade: out-of-range values are rejected on `CREATE` and clamped, with a warning, when replayed from existing metadata, server config, or Keeper, and existing databases that hold a larger value in Keeper are fixed automatically when a replica on the new release starts. [#118103](https://github.com/ClickHouse/ClickHouse/pull/118103) ([Kirill Shokhin](https://github.com/kirillshokhin)). [#123105](https://github.com/ClickHouse/ClickHouse/pull/123105) ([Kirill Shokhin](https://github.com/kirillshokhin)). * `ALTER DATABASE ... MODIFY COMMENT` on a database with `ENGINE = Backup` no longer writes a definition the server cannot read back, which made it refuse to start; metadata already written in that form now loads too. Backward incompatibility: a backup taken by an older server of a database with `ENGINE = Backup` stores the locator quoted into a string literal, and such a definition is no longer accepted from a `RESTORE`. [#118674](https://github.com/ClickHouse/ClickHouse/pull/118674) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Tables in a database with `lazy_load_tables` were missing from `system.parts`, `system.replicas` and related system tables, and refused by replica commands, after a restart. [#121332](https://github.com/ClickHouse/ClickHouse/pull/121332) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed the server failing to start with `TABLE_ALREADY_EXISTS` or `Mapping for table with UUID=... already exists` after it was killed during `CREATE OR REPLACE TABLE`, `REPLACE TABLE`, or `CREATE TABLE ... AS SELECT` in an `Atomic` database whose metadata is on a `plain_rewritable` disk (`database_disk`). [#123902](https://github.com/ClickHouse/ClickHouse/pull/123902) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123832](https://github.com/ClickHouse/ClickHouse/pull/123832) ([Groene AI](https://github.com/groeneai)). * Fixed `EXISTS TABLE` returning `0` for a table that is still being loaded asynchronously after server startup. [#125729](https://github.com/ClickHouse/ClickHouse/pull/125729) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `SELECT ... FINAL` returning a superseded row when the partition expression distinguishes floating-point values that the `FINAL` merge comparator treats as equal, such as `-0.0` versus `0.0` or different `NaN` payloads. `enable_automatic_decision_for_merging_across_partitions_for_final` no longer engages for such a key. [#117482](https://github.com/ClickHouse/ClickHouse/pull/117482) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Row policies over floating-point sorting key columns are now applied after `FINAL`, with `apply_row_policy_after_final` enabled, and such columns are excluded from index analysis of deferred filters. Previously a policy condition distinguishing `-0.0` from `0.0` or `NaN` payloads could filter out the winning row version before the merge and return stale data. [#121782](https://github.com/ClickHouse/ClickHouse/pull/121782) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `CREATE TABLE` and `ALTER TABLE ... ADD INDEX` now reject a skip index whose expression cannot be evaluated, such as `INDEX i c0 = c1 TYPE set(0)` over `String` and `Int8`, and an `ALTER TABLE ... MODIFY COLUMN` that would leave an existing index in that state is rejected too. Such a declaration used to be accepted, after which every `INSERT` and merge failed with `NO_COMMON_TYPE`, and `MATERIALIZE INDEX` retried that error forever. Closes [#69597](https://github.com/ClickHouse/ClickHouse/issues/69597). [#118373](https://github.com/ClickHouse/ClickHouse/pull/118373) ([Groene AI](https://github.com/groeneai)). * Fixed `SELECT count()` returning more rows than the filter accepts when a `MergeTree` primary key is a `Tuple` holding a floating point element and the key data contains a `nan`. For example, with `ORDER BY t` over `t Tuple(Float64, Int32)`, `SELECT count() FROM t WHERE t >= (5., 3)` returned `6` while `SELECT * FROM t WHERE t >= (5., 3)` returned `2` rows. A `nan` nested in a tuple is not visible in the granule bounds, so primary key analysis reported a granule of non-matching rows as wholly matching and the exact-count optimization counted its rows without evaluating the filter. Closes [#117819](https://github.com/ClickHouse/ClickHouse/issues/117819). [#118683](https://github.com/ClickHouse/ClickHouse/pull/118683) ([Groene AI](https://github.com/groeneai)). * Fixed `WHERE f IN (nan)` silently losing the `NaN` rows of a floating-point column: statistics-based part pruning and the `minmax` skip index use ranges that exclude `NaN`, so a part or granule holding `NaN` was pruned even though `IN` matches `NaN`. [#117453](https://github.com/ClickHouse/ClickHouse/pull/117453) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results when a `String` primary key column is compared against a `FixedString` constant wrapped in `Variant` or `Dynamic`. [#114353](https://github.com/ClickHouse/ClickHouse/pull/114353) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results for `f = 0.0` and `f != 0.0` over a float column when the primary key is an injective transform of it (for example `ORDER BY toString(f)`): the index treated the transformed constant as exact, although the transform distinguishes `-0.0` from `+0.0` while the comparison does not. [#117722](https://github.com/ClickHouse/ClickHouse/pull/117722) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Comparing an `Enum` column with a string literal that is not one of its elements no longer throws `UNKNOWN_ELEMENT_OF_ENUM` when `validate_enum_literals_in_operators` is disabled (the default) and the column is part of the primary key, the partition key, or a `bloom_filter` index. `!=` with such a literal now returns true for every row instead of throwing. Closes [#122235](https://github.com/ClickHouse/ClickHouse/issues/122235), [#111545](https://github.com/ClickHouse/ClickHouse/issues/111545). [#122672](https://github.com/ClickHouse/ClickHouse/pull/122672) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed an exception (such as `Division by zero`) thrown from index analysis when a monotonic function chain over a key column cannot be evaluated on a key range endpoint, for example `SELECT count() FROM t WHERE p != 1 AND intDiv(1, p - 1) > 0` with `use_skip_indexes = 0`. Also fixed a possible segmentation fault caused by a poisoned entry in the index analysis function result cache. [#117355](https://github.com/ClickHouse/ClickHouse/pull/117355) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `bloom_filter` skip index on `mapValues` or `mapKeys` that dropped rows for `m['k'] IN (...)` when the map values are an `Enum` with negative values and the set contains the name of the value `0`. [#124232](https://github.com/ClickHouse/ClickHouse/pull/124232) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a wrong order (and wrong rows under `LIMIT`) from the read-in-order optimization when the `ORDER BY` expression maps a stored `inf` to `NaN`, as in `ORDER BY x / inf`. [#118720](https://github.com/ClickHouse/ClickHouse/pull/118720) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed primary key, partition key, and skip index analysis that could skip or over-count rows, and `ORDER BY` optimization that could return rows in the wrong order, for conditions on `timezoneOffset`, `toTimeWithFixedDate`, `toYYYYMMDDhhmmss`, `toHour`, and `toMinute` over a `DateTime` or `DateTime64` column when the key range spans a change of the UTC offset, for example a daylight saving time change in `America/New_York` or the 30-minute change in `Australia/Lord_Howe`. [#124474](https://github.com/ClickHouse/ClickHouse/pull/124474) ([Groene AI](https://github.com/groeneai)). [#124530](https://github.com/ClickHouse/ClickHouse/pull/124530) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of `ORDER BY` with `optimize_read_in_order`, `GROUP BY` with `optimize_aggregation_in_order`, and `LIMIT BY` with the in-order optimizations when a `String` sorting-key column is compared with a `FixedString` constant, also one held by a `Variant` or `Dynamic` constant (for example `WHERE s = toFixedString('a', 2)` or `WHERE s = CAST(toFixedString('a', 2) AS Dynamic)`), or when a `Variant` or `Dynamic` expression, such as a JSON path, is compared with a constant (for example `WHERE j.a = 1` over `Int64` and `Float64` values). Such a comparison matches several values that sort differently, but the optimization treated them as one value and skipped sorting by them. [#123677](https://github.com/ClickHouse/ClickHouse/pull/123677) ([Ashish Kumar Choubey](https://github.com/aschoube)). [#125378](https://github.com/ClickHouse/ClickHouse/pull/125378) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Fixed `count()` counting rows that the filter rejects when a whole-tuple comparison against a `Tuple` primary key involves a `NULL` or a `NaN` nested in the constant, or a `NaN` nested in the key data. [#118944](https://github.com/ClickHouse/ClickHouse/pull/118944) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong result order (and wrong rows under `LIMIT`) from the read-in-order optimization when `ORDER BY` is a monotonically decreasing function of a `Float` sorting key containing `NaN`, e.g. `ORDER BY negate(x)`. The opposite request for the same shape, `ORDER BY negate(x) ASC NULLS FIRST`, is now read in order instead of being sorted. [#118469](https://github.com/ClickHouse/ClickHouse/pull/118469) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed repeated reading of text index posting lists in multi-threaded queries when the global text index caches are disabled. [#120844](https://github.com/ClickHouse/ClickHouse/pull/120844) ([Anton Popov](https://github.com/CurtizJ)). * The `text`, `tokenbf_v1`, `ngrambf_v1`, and `sparse_grams` skip indexes are now used for `IN` when `transform_null_in = 1` and the set contains no `NULL`. Previously such a query read every granule, and one using `force_data_skipping_indices` failed with `INDEX_NOT_USED`. [#121375](https://github.com/ClickHouse/ClickHouse/pull/121375) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixed wrong results and the exception `Column ... already added for reading` for a `FINAL` query with `query_plan_optimize_lazy_final = 1` that reads a `text` index directly, when the `ReplacingMergeTree` sorting key is `Nullable`, floating-point, or mixes `ASC` and `DESC` columns. [#122534](https://github.com/ClickHouse/ClickHouse/pull/122534) ([Groene AI](https://github.com/groeneai)). * On a column whose `text` index has a postprocessor, `hasPhrase` rejoined the index tokens with a space before matching, so it returned wrong results unless that space matched how the tokenizer splits. Allowing `hasPhrase` to accept an `Array(String)` removes joining tokens completely. [#120859](https://github.com/ClickHouse/ClickHouse/pull/120859) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed a `text` index with `tokenizer = 'japanese'` returning fewer rows than a full scan: over a `LowCardinality` column every second row of a run of equal values was not indexed, and two search terms of the same byte length in one query (for example `hasAnyTokens(s, 'エンジン', 'japanese') AND hasAnyTokens(s, '形態解析', 'japanese')`) matched no rows. Indexes built before this fix keep the missing rows until `ALTER TABLE ... MATERIALIZE INDEX` is run. [#123389](https://github.com/ClickHouse/ClickHouse/pull/123389) ([Groene AI](https://github.com/groeneai)). * Fixed a query failing with an exception like `Cannot parse UInt64 from String` on a table with a text index on `mapKeys` or `JSONAllPaths`, when the predicate cannot be evaluated on the value of a missing map key or JSON path. [#124612](https://github.com/ClickHouse/ClickHouse/pull/124612) ([Anton Popov](https://github.com/CurtizJ)). * Fixed `count()` with a text index search function such as `hasToken`, `hasAnyTokens`, or `hasAllTokens` in `WHERE` and a constant `GROUP BY` key returning a row with `0` instead of no rows when nothing matches. [#124593](https://github.com/ClickHouse/ClickHouse/pull/124593) ([Groene AI](https://github.com/groeneai)). * Fix `hasAnyTokens` with a `String` needle on a text index with the `sparseGrams` tokenizer returning fewer rows than the equivalent `Array` needle. Grams covered by a longer gram of the needle were dropped, which is only valid for `hasAllTokens`. [#124613](https://github.com/ClickHouse/ClickHouse/pull/124613) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixed slow planning of queries with a text index on tables with many parts when other skip indices are stored in the packed `skp_idx.packed` archive. Checking for an absent optional text-index file no longer calls `stat` on disk for every part. [#125381](https://github.com/ClickHouse/ClickHouse/pull/125381) ([Anton Popov](https://github.com/CurtizJ)). * Fix `ALTER TABLE ... MATERIALIZE STATISTICS` / `DROP STATISTICS` failing forever and wedging the table's mutation queue when a server profile sets `allow_statistics = 0` and the statement enables it per-session, the same opt-in `ADD STATISTICS` honours. Also close a hole where `MATERIALIZE STATISTICS` ignored `allow_statistics = 0` altogether under `validate_mutation_query = 0`. [#117351](https://github.com/ClickHouse/ClickHouse/pull/117351) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a mutation silently dropping one of two swapped columns' statistics when the part carried a composed `RENAME COLUMN` pair that exchanged their names. [#118977](https://github.com/ClickHouse/ClickHouse/pull/118977) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * When the query cache is unable to store results (with `enable_writes_to_query_cache = 0`, in `clickhouse-local`, or with a zero limit in the `query_cache` server configuration), queries with `use_query_cache = 1` no longer fail with `QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS`, `QUERY_CACHE_USED_WITH_SYSTEM_TABLE` or `QUERY_CACHE_USED_WITH_NON_THROW_OVERFLOW_MODE`: these checks protect against caching wrong results, which cannot happen then. When a `query_cache` limit is reconfigured to 0, the existing entries are now dropped instead of lingering. [#123604](https://github.com/ClickHouse/ClickHouse/pull/123604) ([Konstantin Bogdanov](https://github.com/thevar1able)). * `LIMIT [n] AFTER start_expr UNTIL end_expr` now checks `end_expr` only after `start_expr` first matches. Previously a match of `end_expr` anywhere before the first `start_expr` incorrectly made the result empty. [#120514](https://github.com/ClickHouse/ClickHouse/pull/120514) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed `SELECT DISTINCT` and `LIMIT BY` over a floating point column returning a different number of rows depending on whether the in-order variant of the operator was used: `-0.0` and `0.0`, and distinct `NaN` payloads, are no longer merged by the in-order variant. [#118198](https://github.com/ClickHouse/ClickHouse/pull/118198) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a higher-order function with a non-deterministic lambda body, such as `arrayMap(i -> rand64(i), ...)`, being evaluated once per source row instead of once per output row after `ARRAY JOIN` when lazy columns replication is enabled. [#120796](https://github.com/ClickHouse/ClickHouse/pull/120796) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a performance regression where a query whose `WHERE` requires the same expression to equal two different constants stopped being recognized as always false when `compatibility` was set below `26.7`, or when `optimize_redundant_comparisons` was disabled explicitly. Closes [#121007](https://github.com/ClickHouse/ClickHouse/issues/121007). Closes [#121012](https://github.com/ClickHouse/ClickHouse/issues/121012). [#121065](https://github.com/ClickHouse/ClickHouse/pull/121065) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_AN_AGGREGATE` being thrown for a `GROUP BY` key referenced from a window function when `group_by_use_nulls` is enabled together with `ROLLUP`, `CUBE`, or `GROUPING SETS`, as in `SELECT rank() OVER (PARTITION BY k) FROM t GROUP BY k WITH ROLLUP`. [#118173](https://github.com/ClickHouse/ClickHouse/pull/118173) ([Groene AI](https://github.com/groeneai)). * Fixed `AMBIGUOUS_IDENTIFIER` for a column qualified with the alias of a table expression when another table of the same query is named like that alias. Such a qualifier now refers to the alias, as it does in the old analyzer, both in a join that reported the ambiguity and in a two-table join that resolved it to the other side. [#119082](https://github.com/ClickHouse/ClickHouse/pull/119082) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when a query tree pass rewrites a window function's aggregate in place. A query whose only reference to a `Nullable` column was a windowed `count`, such as `SELECT count(n) OVER () FROM t`, was rejected at default settings because `optimize_functions_to_subcolumns` rewrote it into `sum(not(n.null))` and lost the node's window kind. `optimize_uniq_to_count` corrupted `uniq(n) OVER ()` over a `SELECT DISTINCT` subquery the same way. Closes [#119635](https://github.com/ClickHouse/ClickHouse/issues/119635). [#119681](https://github.com/ClickHouse/ClickHouse/pull/119681) ([Groene AI](https://github.com/groeneai)). * Fix a wrong result from `optimize_injective_functions_in_group_by` on the two-argument form `toString(x, timezone)`. `GROUP BY toString(number, NULL)` could be rewritten to `GROUP BY number` and produce one group per row instead of one. [#117197](https://github.com/ClickHouse/ClickHouse/pull/117197) ([Ilya Yatsishin](https://github.com/qoega)). * Fixed a server crash in `CREATE TABLE ... AS
` when an argument of the table function is a scalar subquery with a `JOIN` and `enable_join_runtime_filters_index_analysis` is enabled in the default settings profile. [#122795](https://github.com/ClickHouse/ClickHouse/pull/122795) ([Groene AI](https://github.com/groeneai)). * Fixed table names not being qualified with the database when a `WITH` name declared elsewhere in the same query happened to match them: a view whose `UNION` branch referenced a table with the name of a `WITH RECURSIVE` CTE of another branch could not be read from another database, and `ALTER TABLE ... UPDATE` with a common table expression referenced from a subquery failed with `UNKNOWN_DATABASE` or read a table of the session database. [#114908](https://github.com/ClickHouse/ClickHouse/pull/114908) ([Groene AI](https://github.com/groeneai)). * Fixed `IN` against a table-backed set (`ENGINE = Set`), or with `transform_null_in = 1`, throwing `CANNOT_CONVERT_TYPE` for probe values outside the set key type's range instead of evaluating them to false, which also made the query succeed or fail depending on how much the plan pruned. [#117606](https://github.com/ClickHouse/ClickHouse/pull/117606) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash and wrong results with `convert_query_to_cnf = 1` when `WHERE` or `HAVING` references the same `SELECT` alias more than once. [#123414](https://github.com/ClickHouse/ClickHouse/pull/123414) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results, `Unexpected return type from tupleElement` errors and `NOT_FOUND_COLUMN_IN_BLOCK` errors when a `Nested` column is read through `ARRAY JOIN` with a numeric `tupleElement` index (for example `tupleElement(n, 3)`) while another subcolumn is unused, and the same expression is reused by `ORDER BY ALL`, `GROUP BY ALL` or an alias. [#123647](https://github.com/ClickHouse/ClickHouse/pull/123647) ([Groene AI](https://github.com/groeneai)). * Fix `UNKNOWN_IDENTIFIER` in the analyzer when a table is referenced as `database.table` in `FROM` or `JOIN` and another table expression in the same query uses that database name as its alias, for example `FROM db.t1 AS db INNER JOIN db.t2 AS t ON ...`. Closes [#122239](https://github.com/ClickHouse/ClickHouse/issues/122239). [#124262](https://github.com/ClickHouse/ClickHouse/pull/124262) ([Dmitry Novik](https://github.com/novikd)). * Fix a wrong result, a leaked totals row, and a `LOGICAL_ERROR` exception with `rewrite_in_to_join` when the subquery of `IN` uses `WITH TOTALS`. [#124096](https://github.com/ClickHouse/ClickHouse/pull/124096) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Unexpected return type from __topKFilter` logical error raised by `ORDER BY ... LIMIT` over a `MergeTree` table with more than one part when the sort column contains a `Nullable` element, for example `Tuple(UInt64, Nullable(UInt64))`. `use_top_k_dynamic_filtering` is enabled by default, so no opt-in setting was required to hit this. [#113406](https://github.com/ClickHouse/ClickHouse/pull/113406) ([Groene AI](https://github.com/groeneai)). * Reject a duplicate column alias name in a CTE or table expression, which previously could return an incorrect result instead of failing. Closes [#114427](https://github.com/ClickHouse/ClickHouse/issues/114427). [#114505](https://github.com/ClickHouse/ClickHouse/pull/114505) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `APPLY lambda(tuple(x), ...)` followed by an operator, as in `SELECT * APPLY lambda(tuple(x), toString(x)) > '0'`, which failed with `UNKNOWN_IDENTIFIER`. A view using `APPLY (lambda(...))` followed by an operator also stopped working after a server restart. [#124142](https://github.com/ClickHouse/ClickHouse/pull/124142) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `Equal values are not contiguous within the range assumed to be sorted` (and potential wrong `DISTINCT` results) when `optimize_distinct_in_order` is enabled and an `ORDER BY` over an `arrayJoin`-derived column is followed by another `ARRAY JOIN` that reproduces a column of the same name. [#110778](https://github.com/ClickHouse/ClickHouse/pull/110778) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and a `LOGICAL_ERROR` (`Cannot capture column ... because it has incompatible type`) when a matcher such as `b.*`, `*`, `COLUMNS('...')`, or `b.* EXCEPT c` appears inside a lambda body. A matcher expanded inside a lambda kept its pre-`JOIN` column types, so `join_use_nulls = 1` did not turn columns from the `NULL` side of a `LEFT`/`FULL`/`RIGHT JOIN` into `Nullable` (unmatched rows produced a value computed over defaults instead of `NULL`), and `group_by_use_nulls = 1` with `WITH ROLLUP` / `WITH CUBE` / `GROUP BY GROUPING SETS` failed with `NOT_AN_AGGREGATE`. A matcher expanded inside a lambda now carries the same types it would have if the columns were written out explicitly, and a lambda inside `PREWHERE` is no longer promoted to `Nullable` by `join_use_nulls`. Also fixed wrong results for a matcher inside a lambda when `analyzer_compatibility_join_using_top_level_identifier = 1` makes a `JOIN ... USING` key resolve to a `SELECT` projection alias: the matcher selected the merged `USING` key instead of the right table's column. [#111993](https://github.com/ClickHouse/ClickHouse/pull/111993) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and `NOT_FOUND_COLUMN_IN_BLOCK` for three-argument `multiIf` and `if`-chain `GROUP BY` keys with `ROLLUP`, `CUBE`, or `GROUPING SETS` and `group_by_use_nulls = 1`. Also fixed constant-condition keys reused through aliases with `enable_identifier_resolve_cache`, lost grouping sets when a key collapses to another key, and a `Bad cast` exception for a correlated subquery reading that column. Closes [#122404](https://github.com/ClickHouse/ClickHouse/issues/122404). Part of [#103419](https://github.com/ClickHouse/ClickHouse/issues/103419). [#123065](https://github.com/ClickHouse/ClickHouse/pull/123065) ([Groene AI](https://github.com/groeneai)). [#122512](https://github.com/ClickHouse/ClickHouse/pull/122512) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of `ORDER BY`, `LIMIT BY`, and `GROUP BY` with `optimize_aggregation_in_order` over a function of the sorting key with a `NULL` constant argument, such as `toString(dt, NULL)` or `toString(dt, {tz:Nullable(String)})` with a `NULL` parameter. Such a function returns `NULL` for every row, but reading in order treated it as strictly monotonic and kept the next sorting key column in table order. [#125042](https://github.com/ClickHouse/ClickHouse/pull/125042) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash, wrong results, or a rejected query in `ORDER BY ... WITH FILL ... INTERPOLATE` when one `INTERPOLATE` expression has to be converted to its target type and another item reads the same column or literal, for example `INTERPOLATE (a AS b, b AS a)` with targets of different types. [#125129](https://github.com/ClickHouse/ClickHouse/pull/125129) ([Groene AI](https://github.com/groeneai)). * A `WITH` list whose elements refer to the preceding ones, or a chain of common table expressions that read the previous one more than once, now fails with `TOO_BIG_AST` past `max_expanded_ast_elements` instead of exhausting the server's memory in `EXPLAIN AST optimize = 1`, `CREATE VIEW`, a full `ATTACH VIEW ... AS SELECT` and `ALTER TABLE ... MODIFY QUERY`. [#122362](https://github.com/ClickHouse/ClickHouse/pull/122362) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix excessive memory usage for queries with `ARRAY JOIN` and filters on both rows and array elements: the row filter is now applied before expanding the array. [#125577](https://github.com/ClickHouse/ClickHouse/pull/125577) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed `ORDER BY ... LIMIT` over a column of an `Interval` type (or `Nullable`, `LowCardinality`, `Tuple` or `Array` of one) failing with `TYPE_MISMATCH` when `use_top_k_dynamic_filtering` is enabled, which is the default. Closes [#125197](https://github.com/ClickHouse/ClickHouse/issues/125197). [#125221](https://github.com/ClickHouse/ClickHouse/pull/125221) ([Groene AI](https://github.com/groeneai)). * Fixed a heap-use-after-free when a correlated subquery reading a `Merge` table was planned inside `EXPLAIN`, a subquery in `FROM`, a view, or an `IN` subquery. [#122475](https://github.com/ClickHouse/ClickHouse/pull/122475) ([Groene AI](https://github.com/groeneai)). * Fix a correlated scalar subquery used as the whole `WHERE`, `HAVING`, or `QUALIFY` condition, e.g. `WHERE (SELECT count() > 0 FROM t AS i WHERE i.k = o.k)`, being rejected with `ILLEGAL_AGGREGATION` or `NOT_AN_AGGREGATE`, or failing with `NOT_IMPLEMENTED` (`NOT_FOUND_COLUMN_IN_BLOCK` on older versions) when the outer column it uses is not read elsewhere in the query. Closes [#124499](https://github.com/ClickHouse/ClickHouse/issues/124499). [#124594](https://github.com/ClickHouse/ClickHouse/pull/124594) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_IMPLEMENTED` (`None of the algorithms enabled by the 'join_algorithm' setting ... can execute this SEMI/ANTI JOIN`) for a correlated `EXISTS`/`NOT EXISTS` subquery, for an `IN (subquery)` under `rewrite_in_to_join`, or for an `ANY LEFT JOIN` of an `ENGINE = Join` table under an always-false `WHERE`, when `join_algorithm` is set to `full_sorting_merge` or `partial_merge`. [#118315](https://github.com/ClickHouse/ClickHouse/pull/118315) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Comparing two deeply nested `Array`, `Tuple`, `Map`, or `Object` values doubled the work for every nesting level, because the ordering asked each equal element both whether it was less and whether it was greater. A single-value aggregate such as `min`, `max`, `argMin`, `groupArraySorted`, or an `-ArgMin` combinator over a 30-level nested argument therefore ran for minutes on a few hundred rows, and could not be interrupted by `max_execution_time`. Such a comparison now visits each element once. [#116455](https://github.com/ClickHouse/ClickHouse/pull/116455) ([Groene AI](https://github.com/groeneai)). * Fixed a server double free when finalizing an aggregate function with the `-State` combinator nested under a combinator that transfers several substates in one call, such as `-Resample`, `-ForEach`, `-Map`, or `-Tuple`, if an exception is thrown part-way through the transfer. [#111287](https://github.com/ClickHouse/ClickHouse/pull/111287) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from `quantileExactWeightedInterpolated`, `quantileInterpolatedWeighted`, and their `quantiles*` forms over integer-backed types. [#114571](https://github.com/ClickHouse/ClickHouse/pull/114571) ([Groene AI](https://github.com/groeneai)). * Fixed `topK`, `topKWeighted`, `approx_top_k`, `approx_top_count`, `approx_top_sum`, `uniqUpTo`, `uniqCombined`, `uniqCombined64`, `exponentialMovingAverage`, `stochasticLinearRegression`, `stochasticLogisticRegression`, and `groupNumericIndexedVector` state columns with a `Decimal` or wide-integer parameter. The parameter was persisted as a quoted string that reparsed as `String`, so after a restart the table was unreadable, uninsertable, and undroppable. [#121491](https://github.com/ClickHouse/ClickHouse/pull/121491) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort when the `flameGraph` aggregate function was constructed on a thread without a query context, for example while attaching a stored table whose column type names `AggregateFunction(flameGraph, ...)`. [#121894](https://github.com/ClickHouse/ClickHouse/pull/121894) ([Groene AI](https://github.com/groeneai)). * Fixed `runningAccumulate` returning an accumulation computed over only part of the rows when it reads a state column under `if`, `multiIf`, `and`, or `or` at the default `short_circuit_function_evaluation = 'enable'`. Inside a boolean predicate this also changed which rows passed the filter. An argument computed inline by another lazily evaluated function, whether it is the state or the grouping key, is still filtered. Closes [#117121](https://github.com/ClickHouse/ClickHouse/issues/117121). [#117126](https://github.com/ClickHouse/ClickHouse/pull/117126) ([Groene AI](https://github.com/groeneai)). * Fixed `arrayIntersect` returning a `NULL` that is present only in its first argument, which made the result depend on the order of the arguments. `NULL` is now in the result only when every argument array contains one. [#118433](https://github.com/ClickHouse/ClickHouse/pull/118433) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `arrayUniq`, `arrayEnumerateUniq`, `arrayEnumerateDense`, and `arrayJaccardIndex` now distinguish `NULL` from the nested type's default value for nullable arrays that use generic hashing. [#120676](https://github.com/ClickHouse/ClickHouse/pull/120676) ([xiaohuanlin](https://github.com/xiaohuanlin)). * Fixed an out-of-bounds heap write in `stem`. The output buffer now grows when the stemmed word is longer than the estimate from the input. [#121402](https://github.com/ClickHouse/ClickHouse/pull/121402) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix array and map search functions returning a match for a `NULL` `String` needle when an empty string is present. [#118741](https://github.com/ClickHouse/ClickHouse/pull/118741) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `least` and `greatest` returning wrapped-around values for a `LowCardinality` numeric argument of mixed signedness, `intDiv`/`%`/`positiveModulo` throwing `Division by zero` on NULL rows of a `LowCardinality(Nullable(...))` denominator, and `fromModifiedJulianDay`, `runningConcurrency`, `hasPhrase`, `formatRow`, `now`, and `CAST` rejecting valid `LowCardinality` arguments. [#117450](https://github.com/ClickHouse/ClickHouse/pull/117450) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122418](https://github.com/ClickHouse/ClickHouse/pull/122418) ([Groene AI](https://github.com/groeneai)). * Fix `has`, `notHas`, `indexOf`, `countEqual`, and `mapContains` silently finding nothing in a constant array or map with `Enum` keys searched for the name of an enum value, e.g. `mapContains(map('a', 1)::Map(Enum8('a' = 1), UInt8), 'a')` returned `0`. They now compare by the name of the enum value, consistently with `equals` and with the same functions over a non-constant argument. Also fix indexing a `Map` with `Enum` keys by a numeric value that is not in the `Enum` throwing `UNKNOWN_ELEMENT_OF_ENUM` during query analysis instead of returning the default value, and an exception in the analysis of a `bloom_filter` index over `mapKeys` of a `Map` with `Enum` keys. Also fix a `bloom_filter` index over an `Array(String)` or `Array(FixedString)` column wrongly pruning the granules that match `hasAny` or `hasAll` with a constant `Array(Enum)` argument. [#116093](https://github.com/ClickHouse/ClickHouse/pull/116093) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed comparison of tuples with an untyped `NULL` element, such as `(1, 2) != (3, NULL)` or `(1, 2) != toNullable((3, NULL))`: the result was `NULL` (or `0` for a `Nullable` tuple) even when another element decides it. It is now the same as with a typed `NULL`: `0` or `1` when another element decides the comparison, `NULL` otherwise. A filter and its negation no longer both drop a row, e.g. with `WHERE NOT (_partition_value > toNullable((..., NULL)))`. [#123896](https://github.com/ClickHouse/ClickHouse/pull/123896) ([Groene AI](https://github.com/groeneai)). * Fixed `pointInPolygon` with a constant polygon returning wrong results for some points of ordinary polygons, for example `0` for the point `(0.76, 0.01)` inside the polygon `[(1, 0), (0, 1), (-1, 0), (0, -1)]`, or `1` for a point inside a small hole. The results now match those for a non-constant polygon. [#123767](https://github.com/ClickHouse/ClickHouse/pull/123767) ([Groene AI](https://github.com/groeneai)). * Arithmetic between a `Decimal` and an integer that does not fit the native width of the decimal result now raises `DECIMAL_OVERFLOW` instead of silently wrapping the integer operand. [#118721](https://github.com/ClickHouse/ClickHouse/pull/118721) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results of `pointInPolygon` for polygons with huge coordinates. With a constant array polygon, a filter on a table whose primary key covers the point columns could also return fewer rows than a full scan. `pointInPolygon` now throws `BAD_ARGUMENTS` where it would compute with a polygon coordinate larger than `1e150` in absolute value. [#123740](https://github.com/ClickHouse/ClickHouse/pull/123740) ([Groene AI](https://github.com/groeneai)). * Fixed `date_time_overflow_behavior` being ignored while parsing `Date` and `DateTime`. Closes [#113151](https://github.com/ClickHouse/ClickHouse/issues/113151). [#116099](https://github.com/ClickHouse/ClickHouse/pull/116099) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `toStartOfInterval` no longer widens `Date` and `DateTime` inputs to `Date32` and `DateTime64` when `enable_extended_results_for_datetime_functions = 1`; only `Date32` and `DateTime64` inputs keep the extended result types, matching the `toStartOfX` functions. This also removes the saturated result (`2106-02-07 06:28:15`) that `toStartOfInterval(, INTERVAL ... DAY)` returned under that setting for dates after `2106-02-07`. [#119806](https://github.com/ClickHouse/ClickHouse/pull/119806) ([Groene AI](https://github.com/groeneai)). [#121809](https://github.com/ClickHouse/ClickHouse/pull/121809) ([Kirill Kopnev](https://github.com/Fgrtue)). * Fixed `age('week', ...)` sometimes being off by one week when the dates fall on different weekdays. [#122633](https://github.com/ClickHouse/ClickHouse/pull/122633) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Fixed wrong results when an `Enum8` or `Enum16` value is converted to `DateTime` or `Date`. A negative member wrapped instead of saturating like the `Int8` or `Int16` it is stored as, so `toDateTime` of an enum returned a far-future timestamp and primary-key analysis discarded granules that did contain matching rows. [#118039](https://github.com/ClickHouse/ClickHouse/pull/118039) ([Groene AI](https://github.com/groeneai)). * Fixed `toDateTime64`, `toDateTime` with a scale and `toTime64` throwing `CANNOT_PARSE_DATETIME` for a `Nullable(String)` or `Nullable(FixedString)` argument that contains `NULL`; such rows now return `NULL`. For `toDateTime64` this was a regression in 26.5, where `cast_string_to_date_time_mode` became `best_effort` by default. [#123311](https://github.com/ClickHouse/ClickHouse/pull/123311) ([Groene AI](https://github.com/groeneai)). * Fixed `dateDiff` and `age` with unit `day` returning a result off by 65536 (for example `-65535` instead of `1`) when an argument is a `DateTime` on the local day 1969-12-31 in a time zone west of UTC, such as `toDateTime(0)` in `America/New_York`. If a table stores such a value in a sorting key, partition key, skip index, projection or `MATERIALIZED`/`DEFAULT` column, rebuild it. [#124555](https://github.com/ClickHouse/ClickHouse/pull/124555) ([Groene AI](https://github.com/groeneai)). * Setting `date_time_overflow_behavior` is now honored by `CAST` and the conversion functions for numeric conversions to `Date`, `Date32`, `DateTime`, `Time`, `DateTime64`, and `Time64`. For `DateTime64` and `Time64`, the same now applies to the `INSERT ... VALUES` expression fallback and the `values` table function, and for `DateTime64` sort keys to numeric `WITH FILL` bounds. Previously an out-of-range value was silently clamped even in the `throw` mode. `accurateCast`, `accurateCastOrNull`, and `accurateCastOrDefault` no longer clamp an unrepresentable value into `DateTime64` or `Time64`, and saturation now reaches the extreme representable tick of the target rather than the last whole second. [#117667](https://github.com/ClickHouse/ClickHouse/pull/117667) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a constant of `Variant` type reached through an `Array`, `Tuple`, or `Map` being mis-serialized to a secondary server, so a query that returned a result on a single node failed with `CANNOT_CONVERT_TYPE` through parallel replicas, `remote`, or `Distributed`. A `String` member of a multi-member `Variant` constant, scalar or nested, arrived there as whichever member its text parsed as, and an empty `Map` whose key or value is a `Variant` was rejected outright. [#117831](https://github.com/ClickHouse/ClickHouse/pull/117831) ([Groene AI](https://github.com/groeneai)). * Fixed `LOGICAL_ERROR` when reading a `sizeN` subcolumn of a shared-data `JSON` path, for example a `size1` subcolumn over data of type `Array(Array(Nullable(Int64)))`. It affected parts written with the `advanced` shared data serialization version, which is the default for merged parts. [#120039](https://github.com/ClickHouse/ClickHouse/pull/120039) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed wrong results when reading the `null` subcolumn of a `Nullable` field of a `Variant`, `Dynamic`, or `JSON` element, for example ``value.`Tuple(a Nullable(UInt32), b String)`.a.null``. On rows holding a different alternative the subcolumn returned `0` although the field itself was `NULL`, so `optimize_functions_to_subcolumns` (enabled by default) rewriting `IS NULL` to that subcolumn silently dropped matching rows. Closes [#119935](https://github.com/ClickHouse/ClickHouse/issues/119935). [#120099](https://github.com/ClickHouse/ClickHouse/pull/120099) ([Groene AI](https://github.com/groeneai)). * Fix `JSONHas` and `JSONExtractRaw` returning an absent result for a typed `JSON` path whose literal value is `NULL` but which contains a non-empty sub-object when `type_json_skip_null_typed_paths` is enabled. [#120321](https://github.com/ClickHouse/ClickHouse/pull/120321) ([JasonLi-cn](https://github.com/JasonLi-cn)). * Fixed `max_memory_usage` and `max_server_memory_usage` not being respected when reading a `JSON` column with the `FLATTENED` serialization version from the `Native` format. [#117730](https://github.com/ClickHouse/ClickHouse/pull/117730) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed reading flattened `Nested` members next to a `JSON` member in `Wide` parts. [#122519](https://github.com/ClickHouse/ClickHouse/pull/122519) ([Mohammed Sohail](https://github.com/msohail22)). * Allow the `JSON` type and the `JSON*` functions to parse documents whose `String` values hold bytes that are not valid UTF-8, which previously failed with `INCORRECT_DATA`. As a side effect, `isValidJSON` and the `JSONExtract*` family now accept such documents instead of reporting them invalid. [#119122](https://github.com/ClickHouse/ClickHouse/pull/119122) ([Groene AI](https://github.com/groeneai)). * Fix stack exhaustion when reading a deeply nested `Dynamic` or `JSON` value as a `Field` or printing it in a text format. Both paths now raise `TOO_DEEP_RECURSION` instead of terminating the server. [#121441](https://github.com/ClickHouse/ClickHouse/pull/121441) ([Groene AI](https://github.com/groeneai)). * Fix reading bucketed runtime `Map` values from `JSON` columns using `advanced` shared-data serialization. [#118577](https://github.com/ClickHouse/ClickHouse/pull/118577) ([Rory Shanks](https://github.com/rorylshanks)). * Fixed a `Dynamic` constant losing the type of its value when a query is sent to other servers (`Distributed`, `remote`, parallel replicas): a constant holding `Enum8('7' = 3)` arrived as the number `3`, so `IN` and `NOT IN` matched the wrong rows and `=` failed with `NO_COMMON_TYPE`, and a `UInt64`, `IPv4`, or `Map` value arrived as a different type. Closes [#119745](https://github.com/ClickHouse/ClickHouse/issues/119745). [#118988](https://github.com/ClickHouse/ClickHouse/pull/118988) ([Groene AI](https://github.com/groeneai)). * Fixed a crash on a query with a deeply nested `Dynamic` or `JSON` constant. [#120885](https://github.com/ClickHouse/ClickHouse/pull/120885) ([Pavel Kruglov](https://github.com/Avogar)). * Fix inconsistent results and missing type errors when `optimize_functions_to_subcolumns` rewrites a `JSON` path accessed with `tupleElement` or `arrayElement` using an unsupported key type. Such expressions now behave the same with and without the optimization. [#125471](https://github.com/ClickHouse/ClickHouse/pull/125471) ([Groene AI](https://github.com/groeneai)). * Fixed `TYPE_MISMATCH` when a `LowCardinality` join key is also selected from the NULL-extended side of a `LEFT`, `FULL`, or `ASOF LEFT` join with `join_use_nulls = 1` and `join_algorithm = 'full_sorting_merge'` or `'parallel_full_sorting_merge'`. Closes [#120424](https://github.com/ClickHouse/ClickHouse/issues/120424). [#120469](https://github.com/ClickHouse/ClickHouse/pull/120469) ([Groene AI](https://github.com/groeneai)). * Fixed `ILLEGAL_PREWHERE` on sharded `GLOBAL RIGHT JOIN` queries that used `PREWHERE` on the left table. The filter is applied to the broadcast side before the join instead of being sent to a `Memory` temporary table that does not support `PREWHERE`. `PREWHERE` on a remote table in a join is also preserved when that table is read through a subquery wrap. [#119780](https://github.com/ClickHouse/ClickHouse/pull/119780) ([Anton Ivashkin](https://github.com/ianton-ru)). * Fixed a `Pipeline stuck` exception when a query combined a merge join using `join_algorithm = 'parallel_full_sorting_merge'` with a step that glues independent streams together, such as reading from a `Merge` table, or a `UNION ALL` narrowed by `max_streams_for_union_step`. Such a query now runs instead of failing. [#121227](https://github.com/ClickHouse/ClickHouse/pull/121227) ([Groene AI](https://github.com/groeneai)). * Fixed a query failing with an error such as `CANNOT_READ_ARRAY_FROM_TEXT` when a subquery's `SELECT` list computes an expression that the same subquery's `WHERE` guards against, and that subquery is the probe side of a `JOIN` for which a runtime join filter is built (`enable_join_runtime_filters = 1`, enabled by default). The guarded expression was evaluated on rows that the `WHERE` had already removed. [#122143](https://github.com/ClickHouse/ClickHouse/pull/122143) ([Groene AI](https://github.com/groeneai)). * Fixed lost matches on a `Dynamic` join key when outer joins are reordered with a join order conflict detector enabled (`query_plan_optimize_join_order_conflict_detector`) and `join_use_nulls = 1`. [#123113](https://github.com/ClickHouse/ClickHouse/pull/123113) ([Fisnik Kastrati](https://github.com/fkastrati)). * Fix `NOT_IMPLEMENTED` error for a `LEFT JOIN` over a dictionary or other key-value storage under `join_algorithm = 'direct'`. [#123491](https://github.com/ClickHouse/ClickHouse/pull/123491) ([Hechem Selmi](https://github.com/m-selmi)). * Fix exponential planning time in the transitive outer-to-inner JOIN conversion when the expression DAG has shared nodes, such as a column used twice (x + x) across nested subqueries. The analysis now visits each node once instead of every path. [#123509](https://github.com/ClickHouse/ClickHouse/pull/123509) ([Hechem Selmi](https://github.com/m-selmi)). * Fix a performance regression where the join runtime filter was skipped for joins that combine an `=` key with a null-safe `<=>` (`IS NOT DISTINCT FROM`) key once hash table statistics were collected. [#122618](https://github.com/ClickHouse/ClickHouse/pull/122618) ([Manuel](https://github.com/raimannma)). * Fixed `indexHint` over the non-preserved side of an OUTER join silently returning extra rows, and `join_algorithm = 'direct'` failing with `NOT_IMPLEMENTED` when a column-free conjunct such as `materialize(1)` is added to the `WHERE` clause. [#117569](https://github.com/ClickHouse/ClickHouse/pull/117569) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `NOT_IMPLEMENTED` for an outer `JOIN` with `join_algorithm = 'full_sorting_merge'` when `optimize_and_compare_chain` derived a condition on the preserved side of the join from the `ON` clause. [#124233](https://github.com/ClickHouse/ClickHouse/pull/124233) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `UNKNOWN_IDENTIFIER` errors in `INSERT ... SELECT` from a parameterized view when `parallel_distributed_insert_select` is enabled. The view is now analyzed with the analyzer instead of the legacy interpreter. [#119495](https://github.com/ClickHouse/ClickHouse/pull/119495) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a query over a `Distributed` table (or `cluster`/`remote`/`clusterAllReplicas` with an inner `view`) spanning more than one shard that references no column of the table anywhere in the query, such as `SELECT count(*) OVER () FROM distributed_table`. Such a query silently returned no rows, and returned `LOGICAL_ERROR` `Invalid number of columns in chunk pushed to OutputPort` when the table produced a constant. The fix covers the analyzer; `enable_analyzer = 0` is fixed separately by [#113955](https://github.com/ClickHouse/ClickHouse/issues/113955). [#113469](https://github.com/ClickHouse/ClickHouse/pull/113469) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash when a scalar or `IN` subquery inside a standalone expression, such as a `CHECK` constraint, a `TTL` expression, or a row policy, reads a `Distributed` table or the `remote` table function. [#121418](https://github.com/ClickHouse/ClickHouse/pull/121418) ([Groene AI](https://github.com/groeneai)). [#121639](https://github.com/ClickHouse/ClickHouse/pull/121639) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` for `grouping` over a `Distributed` table with `optimize_skip_unused_shards = 1`: a constant that a query tree pass builds is now named on a shard exactly as the initiator names it. [#119273](https://github.com/ClickHouse/ClickHouse/pull/119273) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Refuse a query with `ORDER BY` over a `Distributed` table whose declared column type differs from the shard table's type in a way that does not preserve the order, for example `Int8` declared over a `String` column. The shards sorted by their own type and the initiator merged the streams as if they were sorted by the declared type, which returned rows in the wrong order, cut the wrong rows with `LIMIT`, and duplicated values under `DISTINCT`. The check runs when a shard is the local server, where the shard table's types are known, and only for the columns the query sorts by. Exact widenings (`Float32` to `Float64`, `Date` to `Date32`, `DateTime` to `DateTime64`, a `Decimal` with no smaller scale and no fewer integer digits, `FixedString` to `String`) are now recognized as order-preserving by both this check and the one of the `Merge` engine. [#118445](https://github.com/ClickHouse/ClickHouse/pull/118445) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed queries over a `Distributed` table when `optimize_skip_unused_shards` skips every shard and the query uses `distributed_group_by_no_merge` or reads the table through another `Distributed` table. Such queries failed with `NUMBER_OF_COLUMNS_DOESNT_MATCH`, `CANNOT_CONVERT_TYPE`, or `std::out_of_range` (with `ALIAS` columns) instead of returning an empty result or the aggregate of an empty set. [#122982](https://github.com/ClickHouse/ClickHouse/pull/122982) ([Groene AI](https://github.com/groeneai)). * Fixed queries that go through another server (`remote`, `Distributed`, parallel replicas, `GLOBAL IN`, and `QueryRunner` jobs sent to a cluster) failing with `SYNTAX_ERROR`, `UNKNOWN_CODEC`, or `CORRUPTED_DATA` when `output_format_native_encode_types_in_binary_format` or `input_format_native_decode_types_in_binary_format` is enabled. The blocks of these queries now carry type names on the wire, and a shard with this change also works with an initiator without it. `clickhouse-client --query_kind secondary_query` follows the same rule. [#123649](https://github.com/ClickHouse/ClickHouse/pull/123649) ([Groene AI](https://github.com/groeneai)). [#124716](https://github.com/ClickHouse/ClickHouse/pull/124716) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` (or `THERE_IS_NO_COLUMN`) when a query sent to other servers (a `Distributed` table, `remote`, parallel replicas) has an `IN` over a parameterized view called without its database name inside an expression the servers return, for example `countIf(x IN (SELECT y FROM pv(p = 1)))`, a `GROUP BY` key, or a `SELECT` item. Also fixed `NOT_AN_AGGREGATE` when one query spells the same parameterized view call both with and without its database name. [#123662](https://github.com/ClickHouse/ClickHouse/pull/123662) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when reading a subcolumn (`Array` `.size0`, a `Tuple` element, `Map` `.keys`/`.values`, `Nullable` `.null`, a `JSON` path, or a `Dynamic` typed subcolumn) through a `VIEW` over a `Distributed` table. [#119293](https://github.com/ClickHouse/ClickHouse/pull/119293) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results of aggregations, `LIMIT`, `DISTINCT`, `LIMIT BY` and window functions over a `Buffer` table whose destination is a `Distributed` table that a single shard finishes (one shard, `distributed_group_by_no_merge`, or a `GROUP BY` by the sharding key): the rows still in the buffer and the rows of the destination were processed separately and both results were returned. [#124981](https://github.com/ClickHouse/ClickHouse/pull/124981) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a condition outside a distributed subquery with `QUALIFY` was pushed into it and evaluated before the window functions `QUALIFY` filters by. [#125491](https://github.com/ClickHouse/ClickHouse/pull/125491) ([Nikita Taranov](https://github.com/nickitat)). * Fixed `THERE_IS_NO_COLUMN` and `NOT_FOUND_COLUMN_IN_BLOCK` errors in queries over `remote` or with parallel replicas that compare tuples where one pair of elements is constant and false, such as `(n, 1) = (n, 2)`, or where `NULL` is compared with a column, such as `(n, s) = (NULL, s)`. [#125195](https://github.com/ClickHouse/ClickHouse/pull/125195) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results with `parallel_replicas_plan_based = 1` when the plan fragment sent to the replicas has no output columns, for example `SELECT count() FROM (SELECT 1 FROM t LIMIT 500)`. Such a query returned `0`, or an empty result set, instead of the correct value. [#115871](https://github.com/ClickHouse/ClickHouse/pull/115871) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and a `LOGICAL_ERROR` when a query reads with parallel replicas inside a distributed sub-query whose `cluster_for_parallel_replicas` numbers its shards differently from the cluster the query is distributed over: a `VIEW` carrying its own `SETTINGS cluster_for_parallel_replicas`, a read through `clusterAllReplicas` of that same cluster, a cluster built by cluster discovery or by a `Replicated` database when the two servers see a different set of shards under one name, or a plain `remote_servers` cluster whose per-server copies number the shards differently while a configuration change rolls out. The shard number shipped by the initiator was applied to the other numbering and selected the wrong shard. Parallel replicas are now declined for such a read. [#113589](https://github.com/ClickHouse/ClickHouse/pull/113589) ([Groene AI](https://github.com/groeneai)). [#124724](https://github.com/ClickHouse/ClickHouse/pull/124724) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed `SELECT count()` returning a value multiplied by the number of replicas when executed inside a transaction with parallel replicas enabled. Trivial count optimization now is not rejected inside a transaction, so it is executed solely on the initiator. Followers no longer apply the trivial count optimization during query planning, which previously caused each of them to independently answer with the total row count read from the table's metadata whenever the query reached them. [#116365](https://github.com/ClickHouse/ClickHouse/pull/116365) ([Kirill Shokhin](https://github.com/kirillshokhin)). * Fixed `max_bytes_to_read`, `max_bytes_to_read_leaf`, and read-speed and timeout limits being ignored for reads executed from a query plan shipped to another node, which happens with `parallel_replicas_plan_based` and `make_distributed_plan`. [#119958](https://github.com/ClickHouse/ClickHouse/pull/119958) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed `LOGICAL_ERROR` `Cannot find a matching set in the map of sets from single-replica plan`, and silently lost parallel replicas, when automatic parallel replicas were enabled and an `IN` subquery carried its own `SETTINGS` clause naming `force_primary_key` or `automatic_parallel_replicas_mode`. [#120799](https://github.com/ClickHouse/ClickHouse/pull/120799) ([Groene AI](https://github.com/groeneai)). * Fix `UNKNOWN_FUNCTION` on a shard or a replica for a higher-order function whose lambda body is an `AND` of comparisons when `serialize_query_plan = 1` or `parallel_replicas_plan_based = 1`: a JIT-compiled lambda body was serialized into the plan. [#119332](https://github.com/ClickHouse/ClickHouse/pull/119332) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix missing total rows in progress (and therefore a non-working progress bar) for queries with parallel replicas when `parallel_replicas_local_plan` is enabled (the default). [#123928](https://github.com/ClickHouse/ClickHouse/pull/123928) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed duplicate filter push-down in plan-based parallel replicas. [#117731](https://github.com/ClickHouse/ClickHouse/pull/117731) ([Nikita Taranov](https://github.com/nickitat)). * Fixed wrong results and `UNKNOWN_IDENTIFIER` errors when a query reads a `MergeTree` table after reading one of its projections with the `mergeTreeProjection` table function, for example in a `JOIN` or `UNION ALL` of the two. With parallel replicas such a query could also fail with `Got read request from replica N for unknown stream`. Closes [#102182](https://github.com/ClickHouse/ClickHouse/issues/102182). [#123915](https://github.com/ClickHouse/ClickHouse/pull/123915) ([Groene AI](https://github.com/groeneai)). * `FINAL` on table functions now disables parallel replicas the same way as `FINAL` on tables, so a `TimeSeries` table function such as `timeSeriesTags` no longer returns duplicate rows under parallel replicas. [#122492](https://github.com/ClickHouse/ClickHouse/pull/122492) ([Groene AI](https://github.com/groeneai)). * Fixed parallel replicas not being used when reading a `Distributed` table whose underlying table is another `Distributed` table and every shard of the outer cluster has a single replica. [#123566](https://github.com/ClickHouse/ClickHouse/pull/123566) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed `TOO_MANY_ROWS` being raised for a query that stays within its `max_rows_to_read` budget when `parallel_replicas_min_number_of_rows_per_replica` is set. The estimate sizing the replica set enforced the limit against whole granules at planning time. [#121269](https://github.com/ClickHouse/ClickHouse/pull/121269) ([Groene AI](https://github.com/groeneai)). * Fix a `NOT_IMPLEMENTED` exception for `JOIN` queries with `IN` subqueries when using plan-based parallel replicas. [#121508](https://github.com/ClickHouse/ClickHouse/pull/121508) ([Manuel](https://github.com/raimannma)). * Parallel replicas now honor a `SETTINGS` clause that changes table eligibility when it is written inside a subquery or a view body. Such a clause was ignored, so a query over a view whose body sets `parallel_replicas_for_non_replicated_merge_tree = 0`, or sets `parallel_replicas_allow_view_over_mergetree = 0` and reads a further view, returned `max_parallel_replicas` times too many rows, and a `UNION` over a view overriding `parallel_replicas_allow_view_over_mergetree` per subquery could throw `Can't determine table for parallel replicas`. [#110020](https://github.com/ClickHouse/ClickHouse/pull/110020) ([Groene AI](https://github.com/groeneai)). * A query that reads a `MATERIALIZED` CTE more than once, for example inside an `IN` subquery, now works with parallel replicas when the initiator also reads locally (`parallel_replicas_local_plan = 1`). The other replicas no longer fail with `UNKNOWN_TABLE` on the CTE's temporary table. Closes [#112642](https://github.com/ClickHouse/ClickHouse/issues/112642). [#123121](https://github.com/ClickHouse/ClickHouse/pull/123121) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` "Block structure mismatch in QueryPipeline stream" when selecting a computed column through the `merge` table function or the `Merge` engine. If constant folding folded that column for one matched table and not another, the query failed. [#115971](https://github.com/ClickHouse/ClickHouse/pull/115971) ([Groene AI](https://github.com/groeneai)). * Fix the exception `No set is registered for key` for a `SELECT` that reads a `Distributed` table through a `Merge` table or the `merge` table function and whose `GROUP BY`, `DISTINCT`, or `LIMIT BY` key contains an `IN`-family function. [#117817](https://github.com/ClickHouse/ClickHouse/pull/117817) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when reading a subcolumn of a `Merge` table column whose type differs between the source tables. Such a subcolumn exists only in the common type `Merge` derives, so it was filled with the default value for every row instead of being read. Most visibly, `variantElement` over a `Merge` table returned `NULL` for every row unless the call sat in the top-level projection next to another use of the whole column. [#121462](https://github.com/ClickHouse/ClickHouse/pull/121462) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server crash when a `SELECT` with `serialize_query_plan = 1` read a `Merge` table through a table that forwards its reads, such as `Alias`, `Buffer` with a destination, or a `MATERIALIZED VIEW` with a `TO` target. Closes [#119899](https://github.com/ClickHouse/ClickHouse/issues/119899). [#119387](https://github.com/ClickHouse/ClickHouse/pull/119387) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when a `Merge` table (or the `merge` table function) reads a subcolumn of a column that one of its child tables does not have: for that child, `x.null` of a missing `Nullable` column `x` was read as 0, so `x IS NULL` returned no rows from it. Element subcolumns of `Tuple`, `Variant` and `JSON` columns and the sizes of a missing flattened `Nested` member were affected the same way. Closes [#122523](https://github.com/ClickHouse/ClickHouse/issues/122523). [#123633](https://github.com/ClickHouse/ClickHouse/pull/123633) ([Groene AI](https://github.com/groeneai)). * Fixed a `Merge` table returning the value of another column, or failing with a `CAST` error, for a column that one of its child tables does not have, when that child has a row policy or an `ALIAS` column. Such a column is now filled with its default value. [#124674](https://github.com/ClickHouse/ClickHouse/pull/124674) ([Groene AI](https://github.com/groeneai)). * Fixed `merge` and `Merge` tables with a `Distributed` table among them returning the same `GROUP BY` group several times, each copy with a part of its aggregates, when the aggregation is two-level and a table lacks a grouping column or has it under another type than the common one. [#124720](https://github.com/ClickHouse/ClickHouse/pull/124720) ([Groene AI](https://github.com/groeneai)). * Fixed a use-after-free when a query reads through an `Alias` table whose target is dropped concurrently. [#120098](https://github.com/ClickHouse/ClickHouse/pull/120098) ([Groene AI](https://github.com/groeneai)). * Fix `CREATE MATERIALIZED VIEW ... POPULATE AS SELECT count() FROM ` counting rows inserted during the population twice when `materialized_views_populate_atomically` is enabled. [#123182](https://github.com/ClickHouse/ClickHouse/pull/123182) ([Murphy](https://github.com/murphy-4o)). * Fixed a materialized view over a system log table such as `system.query_log` silently stopping to receive rows after the server renamed the table to `_N` because its definition changed (a server upgrade, or a change of `` or `` in the config). Closes [#124177](https://github.com/ClickHouse/ClickHouse/issues/124177). [#124217](https://github.com/ClickHouse/ClickHouse/pull/124217) ([Groene AI](https://github.com/groeneai)). * Fixes wrong results when a materialized view reads an ordinary view over a table with a row policy that contains a subquery (`x IN (SELECT ...)` or a scalar subquery). The policy's subquery now reads the real source table instead of the block being inserted into the materialized view. [#124875](https://github.com/ClickHouse/ClickHouse/pull/124875) ([Aditya Kumar](https://github.com/adityaksolves)). * `SYSTEM WAIT VIEW` on a refreshable materialized view in a `Replicated` or `Shared` database no longer returns `Ok.` while a refresh requested on another replica has not run yet. Requested refreshes are now published in `Keeper`. [#120623](https://github.com/ClickHouse/ClickHouse/pull/120623) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Fixed a segmentation fault when a refreshable materialized view is started up and shut down at the same time, for example when a failed `ATTACH DATABASE` rolls the database back, or when the server shuts down while tables are still loading asynchronously. [#121037](https://github.com/ClickHouse/ClickHouse/pull/121037) ([Groene AI](https://github.com/groeneai)). * Fixed a refreshable materialized view with `REFRESH DEPENDS ON` (or `REFRESH AFTER ... DEPENDS ON`) and no `REFRESH EVERY` re-running its refresh without limit after a failure used up `refresh_retries`. It now waits for its dependencies' next refresh. [#123369](https://github.com/ClickHouse/ClickHouse/pull/123369) ([Ashish Kumar Choubey](https://github.com/aschoube)). * Fixed `plain_rewritable` disks leaving object storage and the in-memory filesystem in disagreement when a failed metadata transaction could not be reversed completely. Transient failures during reversal are now retried until object storage accepts the operation, and rollback runs for all exception types. Missing blobs are reported as `CORRUPTED_DATA`, and replacing a file with a missing source no longer deletes the existing target. The disagreement previously surfaced only after a restart, when a directory could appear under a name the server never committed. [#119235](https://github.com/ClickHouse/ClickHouse/pull/119235) ([Sema Checherinda](https://github.com/CheSema)). [#122696](https://github.com/ClickHouse/ClickHouse/pull/122696) ([Sema Checherinda](https://github.com/CheSema)). * Fixed a query reading from `file`, `s3`, `url`, `azure`, or `hdfs` with `session_timezone` set getting the `DateTime` schema inferred by an earlier session with a different `session_timezone`, which formatted and computed the values in that other session's time zone. [#117811](https://github.com/ClickHouse/ClickHouse/pull/117811) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed unnecessary object-storage reads, and ``-level exception messages such as `Code: 499 ... The specified key does not exist ... (S3_ERROR)`, while destroying a data part on an object-storage disk when its skip indices are packed into `skp_idx.packed`. Evicting the part's index marks from the mark cache no longer inspects the part's on-disk layout, so it performs no I/O. [#113454](https://github.com/ClickHouse/ClickHouse/pull/113454) ([Groene AI](https://github.com/groeneai)). * Fixed hive partition columns of glob-path object storage tables being permanently disabled when the first query over the table ran while the path matched no files. [#120308](https://github.com/ClickHouse/ClickHouse/pull/120308) ([Zach Rippas](https://github.com/zatchet)). * Fixed unbounded memory and CPU usage when a path passed to `file`, `s3`, `azureBlobStorage`, or `hdfs` contains many `{a,b,c}` glob groups or long `{N..M}` ranges. Such a path is now rejected with an error. [#118467](https://github.com/ClickHouse/ClickHouse/pull/118467) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `copyObjectToAnotherObjectStorage` copying an object twice when the source and destination are the same object storage, which doubled the data transferred by a copy between two disks backed by that storage. [#120953](https://github.com/ClickHouse/ClickHouse/pull/120953) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `UNKNOWN_IDENTIFIER` for a `DEFAULT` column that depends on a column used by a row policy, in tables with the `File`, `URL`, `S3`, and other object-storage engines. [#122714](https://github.com/ClickHouse/ClickHouse/pull/122714) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix object storage tables with `partition_strategy = 'hive'` being silently empty for formats whose name is not the file extension real files carry. The read glob was built from the lowercased format name (`**.jsoneachrow`, `**.csvwithnames`), so pointing a `JSONEachRow` table at a lake of `.json` / `.jsonl` files — or a `CSVWithNames` table at `.csv` files — read zero rows with no error or warning. The glob is now built from every file extension registered for the format (and for its `WithNames`/`WithNamesAndTypes` base format), plus additionally registered read extensions: `JSONEachRow` matches `.json`/`.jsonl`/`.ndjson` files and `TabSeparated` matches `.tsv`. Files written by ClickHouse itself keep matching. [#120319](https://github.com/ClickHouse/ClickHouse/pull/120319) ([Zach Rippas](https://github.com/zatchet)). * Fixed reads of Hive-partitioned files (`.../key=9/...`) that need no column of the data file, such as `count()` with `optimize_count_from_files = 0` or `SELECT key, _row_number`, returning no rows (`Native` threw an exception) when a partition column was smaller than the data columns. The wrong `0` was also cached and returned by later `count()` queries. The same reads of a structure that lists only partition columns returned no rows or threw an exception for most formats. Affects `file`, `url`, `s3` and other object storages. [#123381](https://github.com/ClickHouse/ClickHouse/pull/123381) ([Groene AI](https://github.com/groeneai)). * A query over an object storage table (e.g. `S3`) no longer waits for another query that is resolving the table's hive partitioning on its first use, so `KILL QUERY` and `max_execution_time` apply to it while the endpoint is slow or unreachable. [#123532](https://github.com/ClickHouse/ClickHouse/pull/123532) ([Groene AI](https://github.com/groeneai)). * Fixed `BAD_ARGUMENTS` (`Expected positional arguments to go before key-value arguments`) for a valid `S3` argument list that combines `headers(...)` with a key-value argument, in either order, and for the engines and table functions sharing that parser (`Iceberg`, `DeltaLake`, `Hudi`, `Paimon`, `GCS`, `S3Queue`). With `file_like_engine_default_partition_strategy = 'wildcard'` the server persisted such a list itself, so a created table could no longer be attached and the server refused to start; affected tables load again. [#119070](https://github.com/ClickHouse/ClickHouse/pull/119070) ([Groene AI](https://github.com/groeneai)). * Fix the logical error `Attempt to read beyond right offset` when the right bound of a read from Azure Blob Storage is lowered below bytes the reader has already buffered. [#120957](https://github.com/ClickHouse/ClickHouse/pull/120957) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `WHERE` condition or a row policy on a Hive partition column (`use_hive_partitioning = 1`) returning too few rows when the `Parquet` data file also has a column with that name and other values: `Parquet` row-group, page, and dictionary pruning used the file's values instead of the path's. The same applied to a virtual column such as `_file` when the data file has a physical column with that name. Affects `file`, `url`, `s3`, and other object storages. [#124991](https://github.com/ClickHouse/ClickHouse/pull/124991) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an exception when a `WHERE` condition on the `_file`, `_path`, or `_part` virtual column guards a branch with `if`, `and`, `or`, or `multiIf`, for example `WHERE _file LIKE 'events_%' AND toDate(substring(_file, 8, 10)) >= '2024-01-01'`. Pruning files and parts by virtual columns now respects `short_circuit_function_evaluation` instead of evaluating the excluded branch for every file or part name. The same path now also rebuilds copied `indexHint` expressions and reduced `and` expressions with the right arguments, avoiding argument-count exceptions during lazy evaluation. [#125033](https://github.com/ClickHouse/ClickHouse/pull/125033) ([Groene AI](https://github.com/groeneai)). [#125150](https://github.com/ClickHouse/ClickHouse/pull/125150) ([Groene AI](https://github.com/groeneai)). [#125126](https://github.com/ClickHouse/ClickHouse/pull/125126) ([Groene AI](https://github.com/groeneai)). * Fixed loading, short `ATTACH`, and `UNDROP TABLE` for existing SQL-defined `S3` disks whose endpoints are absent from `remote_url_allow_hosts`; new full-definition `CREATE` and `ATTACH` still reject disallowed endpoints. [#124930](https://github.com/ClickHouse/ClickHouse/pull/124930) ([Alexey Bakharew](https://github.com/alexbakharew)). * Fixed filesystem cache accounting when the directory for a new cache key cannot be created, for example because the cache disk is out of space or inodes. The failed reservation stayed counted in the cache until the query released the file segment. [#122211](https://github.com/ClickHouse/ClickHouse/pull/122211) ([Sema Checherinda](https://github.com/CheSema)). * Fixed `Parquet` writing size statistics twice for a column chunk that falls back from dictionary encoding, which made `unencoded_byte_array_data_bytes` and the level histograms too large. The `ParquetMetadata` format now also reports `size_statistics` for each column chunk. [#120027](https://github.com/ClickHouse/ClickHouse/pull/120027) ([Erez Rokah](https://github.com/erezrokah)). * Fix two `JSON` object-size-limit bugs: bytes before the opening bracket are now counted toward the limit, and valid data is no longer rejected just because earlier rows in the same block already used the shared buffer. [#115795](https://github.com/ClickHouse/ClickHouse/pull/115795) ([Pavel Kruglov](https://github.com/Avogar)). * Schema inference for the `Freeform` format no longer exhausts memory on a wide row of strings: the search for the row structure is bounded by the setting `input_format_freeform_max_search_steps` (default `4096`, `0` means unlimited) and now also stops at `max_memory_usage`, `max_execution_time`, and `KILL QUERY`. [#122447](https://github.com/ClickHouse/ClickHouse/pull/122447) ([Mohammed Sohail](https://github.com/msohail22)). [#123238](https://github.com/ClickHouse/ClickHouse/pull/123238) ([Groene AI](https://github.com/groeneai)). * Fix a crash when reading an ORC file whose root type is not a struct (for example, a file written with a `bigint` or `array` root type). Such files are now rejected with an `INCORRECT_DATA` error. Closes [#123305](https://github.com/ClickHouse/ClickHouse/issues/123305). [#123337](https://github.com/ClickHouse/ClickHouse/pull/123337) ([Groene AI](https://github.com/groeneai)). * Fixed a possible out of bounds access when reading an ORC file. [#123464](https://github.com/ClickHouse/ClickHouse/pull/123464) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix reading Avro files with a union whose named branches (records, enums, fixed) have the same structure. Before the fix, such files failed with `The number of (non-null) union types in Avro record (N) does not match the number of types in destination Variant type (M)`, even with the inferred schema. [#123236](https://github.com/ClickHouse/ClickHouse/pull/123236) ([Dmitry Kovalev](https://github.com/dk-github)). * Avro schema parsing no longer overflows the stack on a deep chain of named types. [#124053](https://github.com/ClickHouse/ClickHouse/pull/124053) ([Groene AI](https://github.com/groeneai)). * Fixed a crash (stack overflow) on an Avro schema with a long chain of named type references, in schema inference (`DESC`, or `SELECT` without an explicit structure, for `Avro` and `AvroConfluent`), in reads into a `Variant` column and in Iceberg manifest reading. Such a schema now fails with an error instead. [#124644](https://github.com/ClickHouse/ClickHouse/pull/124644) ([Groene AI](https://github.com/groeneai)). * Fixes a crash when reading an ORC file whose metadata has fewer stripe statistics than the file has stripes (for example, after a corrupt `metadata_length`) with `input_format_orc_filter_push_down` enabled. The `ORC` input format now reads such files without stripe-level filtering. [#124315](https://github.com/ClickHouse/ClickHouse/pull/124315) ([Groene AI](https://github.com/groeneai)). * Fixed a crash (stack overflow) when reading an Avro file with an explicit structure that does not read a field nested through a long chain of named types, or a field of a recursive type holding deeply nested data such as a long linked list. Such a file now fails with `TOO_DEEP_RECURSION` instead. Closes [#124925](https://github.com/ClickHouse/ClickHouse/issues/124925). [#124964](https://github.com/ClickHouse/ClickHouse/pull/124964) ([Groene AI](https://github.com/groeneai)). * Fixed a stack overflow crash on a deeply nested `CapnProto` schema that the nesting depth check missed: a comment or a byte order mark between `import` and its path, an escape sequence in the import path, or a backtick text line. A backtick text line with many brackets is no longer rejected as nested too deeply. [#125487](https://github.com/ClickHouse/ClickHouse/pull/125487) ([Groene AI](https://github.com/groeneai)). * Fixed a crash (stack overflow) on a `CapnProto` schema with a long chain of type expression suffixes (`Int32.a.a...`, `Int32()()...`) or of declarations that refer to each other (constants, aliases, struct fields, annotations, imports). Such a schema is now rejected with `The CapnProto schema is nested too deeply`. Closes [#125782](https://github.com/ClickHouse/ClickHouse/issues/125782). [#125900](https://github.com/ClickHouse/ClickHouse/pull/125900) ([Groene AI](https://github.com/groeneai)). * Fixed a data race on the file list of a `File` engine table. Two concurrent `INSERT`s with `engine_file_allow_create_multiple_files = 1` could pick the same file name, which made `SELECT count()` over-report, rejected a valid `INSERT` with `CANNOT_APPEND_TO_FILE`, and could corrupt the heap. Closes [#121517](https://github.com/ClickHouse/ClickHouse/issues/121517). [#121598](https://github.com/ClickHouse/ClickHouse/pull/121598) ([Groene AI](https://github.com/groeneai)). * Fixed `count()` over `file('*.csv', 'CSVWithNames')` with `optimize_count_from_files = 1` over-counting a CSV file by one row in rare cases. [#120552](https://github.com/ClickHouse/ClickHouse/pull/120552) ([Knut-Aage-Hofseth](https://github.com/Knut-Aage-Hofseth)). * Fixed concurrent `INSERT`s and `SELECT`s on a table with the `File` engine stalling for `lock_acquire_timeout` (120 seconds by default) and then failing with `TIMEOUT_EXCEEDED`. A single `SELECT` can open several readers on one `File` table, and an `INSERT` arriving between two of them made the query wait for itself. Closes [#115931](https://github.com/ClickHouse/ClickHouse/issues/115931). [#121779](https://github.com/ClickHouse/ClickHouse/pull/121779) ([Groene AI](https://github.com/groeneai)). * Fixed `TinyLog` returning fewer rows than the table contains when one of the data files being read ends before the others. An `Array`, `Map`, or `Nested` column contributes an elements file that holds one entry per element rather than per row, so it runs out early whenever the trailing rows hold empty containers, and a column holding only empty arrays contributes an elements file of zero bytes. The read stopped at the first data file by name to reach its end, cutting the result at a block boundary. [#120307](https://github.com/ClickHouse/ClickHouse/pull/120307) ([Groene AI](https://github.com/groeneai)). * Fixed `BACKUP` and `RESTORE` queries with `SETTINGS = DEFAULT`: such a query either failed to parse or was formatted into text that could not be parsed back, and the other settings of the same clause were discarded. Repeated `async` items now use the last value, `async = '1'` no longer fails after the operation has started, and `compression` is rejected in the `SETTINGS` clause as in other queries. [#112978](https://github.com/ClickHouse/ClickHouse/pull/112978) ([Groene AI](https://github.com/groeneai)). * Fixed `BACKUP` writing a `.backup` manifest that is not well-formed XML when a value in it contained `&`, `<`, or `>`, for example `BACKUP ... SETTINGS id = 'a&b'`, or an incremental backup whose `base_backup` path contained one of those characters. The backup reported `BACKUP_CREATED`, but every later `RESTORE` from it failed with `SAXParseException: Invalid token`, leaving the data unreachable. Such values are now escaped. Backups already written with an unescaped value remain unreadable and have to be taken again. [#118898](https://github.com/ClickHouse/ClickHouse/pull/118898) ([Julia Kartseva](https://github.com/jkartseva)). * Fixed `RESTORE` of `system.functions` failing, and aborting the whole restore, when the backup holds a SQL user-defined function whose definition the version that created it accepted but `CREATE FUNCTION` now rejects, for example `CREATE FUNCTION f AS lambda(identity(x), x)` created on 25.5 or earlier. [#122912](https://github.com/ClickHouse/ClickHouse/pull/122912) ([Groene AI](https://github.com/groeneai)). * Fixes `BACKUP ... ON CLUSTER` creating a backup that cannot be restored when a host names its data files differently from the initiator: with `data_file_name_generator = 'checksum'` and a `data_file_name_prefix_length` other than the hosts' `backups.data_file_name_prefix_length` (3 by default), or when a host's config sets `backups.data_file_name_generator = checksum` and the initiator uses the default generator. Backups already created this way still cannot be restored. [#122624](https://github.com/ClickHouse/ClickHouse/pull/122624) ([Groene AI](https://github.com/groeneai)). * Fixed `RESTORE` of a lightweight snapshot from `Azure Blob Storage` when the source disk endpoint includes a key prefix below the container name. [#122697](https://github.com/ClickHouse/ClickHouse/pull/122697) ([Garrett Thomas](https://github.com/garrettthomaskth)). * Fixed column comments of a `ReplicatedMergeTree` table being lost in a backup when they were changed after the last metadata `ALTER`. [#121309](https://github.com/ClickHouse/ClickHouse/pull/121309) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `BACKUP` / `RESTORE ... ON CLUSTER` hanging forever, unaffected by `KILL QUERY` and blocking server shutdown, when `compatibility` is set to `24.10` or older and a host fails on a `Keeper` error before it starts. Such a `compatibility` value no longer sets `backup_restore_finish_timeout_after_error_sec`, `backup_restore_failure_after_host_disconnected_for_seconds`, `backup_restore_keeper_max_retries_while_initializing`, and `backup_restore_keeper_max_retries_while_handling_error` to `0`, a combination no release ever used. [#123752](https://github.com/ClickHouse/ClickHouse/pull/123752) ([Groene AI](https://github.com/groeneai)). * Fixed `BACKUP` failing with `A concurrent backup writing to the same destination ... detected` when nothing else was writing there. The lock file is claimed with a conditional write, and a retry of a request that had already been applied made the backup mistake its own lock for someone else's. [#116376](https://github.com/ClickHouse/ClickHouse/pull/116376) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `BACKUP` and `RESTORE` to `S3` now retry requests that fail with `ExpiredToken` after refreshing credentials. The retry time limit is 120 seconds by default and can be changed with `backup_s3_expired_token_retry_timeout_ms`. [#123195](https://github.com/ClickHouse/ClickHouse/pull/123195) ([Nikolai Ovchinnikov](https://github.com/kolsha)). * `BACKUP ... TO File(...)` / `Disk(...)` now fsyncs the backup data files, the `.backup` manifest and the containing directories before reporting `BACKUP_CREATED`, so an acknowledged backup to local storage survives power loss. The guarantee covers `backups.allowed_path` and everything the backup writes below it. On a `Disk(...)` over object storage that keeps its metadata in local files, those metadata files are fsynced instead, since the backup is reached only through them. Controlled by the new backup setting `fsync_backup_files` (default `true`). `S3(...)`/`AzureBlobStorage(...)` destinations are unaffected. [#111394](https://github.com/ClickHouse/ClickHouse/pull/111394) ([Groene AI](https://github.com/groeneai)). * `Iceberg` tables whose manifest file headers carry a schema that conflicts with the schema registered for the same `schema-id` in `metadata.json` are now readable: the `metadata.json` schema is preferred and the divergent manifest header copy is ignored with a warning, no matter which of the two copies is read first, matching the behavior of other query engines such as Spark, Trino, PyIceberg, and DuckDB. Controlled by the new setting `iceberg_tolerate_conflicting_manifest_schemas` (default `true`); `compatibility` below `26.9` restores the previous strict behavior, which failed such queries with `ICEBERG_SPECIFICATION_VIOLATION`. [#119743](https://github.com/ClickHouse/ClickHouse/pull/119743) ([Melvyn Peignon](https://github.com/melvynator)). * Fixed `Iceberg` manifest statistics recording the wrong file count for added files. [#120858](https://github.com/ClickHouse/ClickHouse/pull/120858) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed `Iceberg` `INSERT` writing no `lower_bounds` or `upper_bounds` at all into the manifest entry when the table has a column whose type cannot be serialized into a statistics bound, or a column that is entirely `NULL`. Such a field is now omitted individually and the bounds of every other column are still written, so min/max file pruning keeps working. Closes [#112706](https://github.com/ClickHouse/ClickHouse/issues/112706). [#112934](https://github.com/ClickHouse/ClickHouse/pull/112934) ([Groene AI](https://github.com/groeneai)). * Fixed `Iceberg` date partition transforms being computed in the server time zone instead of UTC. [#118583](https://github.com/ClickHouse/ClickHouse/pull/118583) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed `SHOW TABLES` and `system.tables` failing on a database with the `DataLakeCatalog` engine over an `Iceberg` REST catalog when an unrelated namespace was dropped by another client while the listing was in flight. [#113740](https://github.com/ClickHouse/ClickHouse/pull/113740) ([Groene AI](https://github.com/groeneai)). * Fixed validation of `Iceberg` manifest-list entries during writes. [#122829](https://github.com/ClickHouse/ClickHouse/pull/122829) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed metadata of empty `Iceberg` tables created by ClickHouse, so that Spark and other `Iceberg` Java-based tools can open them before the first insert and evolve their partition spec. [#122077](https://github.com/ClickHouse/ClickHouse/pull/122077) ([Asya Shneerson](https://github.com/asya-ch)). * Prevent loss of newer committed `Iceberg` data by rejecting `OPTIMIZE` when `iceberg_metadata_file_path`, `iceberg_snapshot_id`, or `iceberg_timestamp_ms` is explicitly set. [#122683](https://github.com/ClickHouse/ClickHouse/pull/122683) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed reading an Iceberg table partitioned by `DateTime64` when a data file does not store the partition column: the query failed with `DECIMAL_OVERFLOW`. Decimal partition values stored in a manifest in their raw form are decoded as well, and `system.iceberg_files.partition` now shows partition values in a readable form instead of raw bytes. [#117090](https://github.com/ClickHouse/ClickHouse/pull/117090) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CREATE TABLE` and `DROP TABLE` in Iceberg REST catalog databases: the catalog now writes the initial metadata file, `ORDER BY` and table properties are sent to the catalog, multi-level namespaces are created correctly, and `DROP TABLE` uses the correct endpoint. [#123631](https://github.com/ClickHouse/ClickHouse/pull/123631) ([Asya Shneerson](https://github.com/asya-ch)). * Fix `SELECT count()` over `Iceberg` tables returning too few rows after compaction leaves dangling position-delete files. Closes [#123494](https://github.com/ClickHouse/ClickHouse/issues/123494). [#123851](https://github.com/ClickHouse/ClickHouse/pull/123851) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed reading `Iceberg` tables through the `DataLakeCatalog` database engine when the catalog reports a column type in a spelling the Iceberg type parser did not know: `varbyte` (Glue's name for the Iceberg `binary` type), `list<...>` (the Iceberg spelling of `array<...>`, also written by Glue), and `timestamp with local time zone` (Hive 3's name for the Iceberg `timestamptz` type). Such tables were rejected with `Unknown Iceberg type`. Closes [#121850](https://github.com/ClickHouse/ClickHouse/issues/121850). Closes [#87716](https://github.com/ClickHouse/ClickHouse/issues/87716). [#121974](https://github.com/ClickHouse/ClickHouse/pull/121974) ([Groene AI](https://github.com/groeneai)). * The `iceberg_metadata_staleness_ms` setting is now respected when parsing `Iceberg` metadata, so `Iceberg` table functions and `ON CLUSTER` workers no longer list the whole `metadata/` directory on every query. [#124113](https://github.com/ClickHouse/ClickHouse/pull/124113) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix a server abort when reading an `Iceberg` table whose sort order names a transform ClickHouse cannot parse, e.g. `bucket[xyz]`; ClickHouse now ignores that sort-order hint and keeps the table readable. [#121687](https://github.com/ClickHouse/ClickHouse/pull/121687) ([Ilias](https://github.com/iliasabk)). * Fixed `OPTIMIZE TABLE` for OSS Iceberg tables so compaction no longer removes `version-hint.text` and breaks hint-based reads. [#122833](https://github.com/ClickHouse/ClickHouse/pull/122833) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed a crash (SIGSEGV) when reading a malformed deletion vector with the `Puffin` input format, for example one whose roaring run container has zero runs. Structurally invalid roaring bitmaps are now rejected with an error instead of crashing or returning wrong deleted-row positions. Closes [#124688](https://github.com/ClickHouse/ClickHouse/issues/124688). [#124735](https://github.com/ClickHouse/ClickHouse/pull/124735) ([Groene AI](https://github.com/groeneai)). * Fixed an `INSERT` into a `Parquet` `Iceberg` table failing with `Logical error: 'Iceberg data file has 1 columns while its schema has 2 fields'` when `ALTER TABLE ... ADD COLUMN` committed while the `INSERT` was running. [#121821](https://github.com/ClickHouse/ClickHouse/pull/121821) ([Groene AI](https://github.com/groeneai)). * Fixed `Iceberg` v3 virtual columns `_row_id` and `_last_updated_sequence_number`: filters on them no longer return no rows for a data file rewritten by copy-on-write when the manifest has statistics for the table columns but not for the row-lineage columns, and rows of files written before the table was upgraded to format version 3 no longer return `NULL` for these columns. Closes [#124271](https://github.com/ClickHouse/ClickHouse/issues/124271). [#124591](https://github.com/ClickHouse/ClickHouse/pull/124591) ([Groene AI](https://github.com/groeneai)). [#124598](https://github.com/ClickHouse/ClickHouse/pull/124598) ([Groene AI](https://github.com/groeneai)). * Fixed `OPTIMIZE TABLE ... MANIFEST` failing with `BAD_ARGUMENTS` on an `Iceberg` table whose current snapshot summary omits an optional `total-records`, `total-files-size`, or `total-data-files` metric, which made manifest compaction permanently unavailable on tables written by engines that omit those metrics. Closes [#117000](https://github.com/ClickHouse/ClickHouse/issues/117000). [#117012](https://github.com/ClickHouse/ClickHouse/pull/117012) ([Groene AI](https://github.com/groeneai)). * Fixed `Iceberg` min/max pruning skipping a data file that holds matching rows when the file's `uuid` bound was written by an external engine such as Spark, Trino, or PyIceberg. Such a bound now prunes only when its lower and upper bounds are equal. This fixes the `Iceberg` manifest bounds only: the `uuid` statistics of the `Parquet` data file itself are a separate defect, so a query at default settings can still miss rows. Closes [#118366](https://github.com/ClickHouse/ClickHouse/issues/118366). [#118380](https://github.com/ClickHouse/ClickHouse/pull/118380) ([Groene AI](https://github.com/groeneai)). * Fixed `Iceberg` writes storing ClickHouse unsigned integers in a narrower `Iceberg` integer type than the source. [#118596](https://github.com/ClickHouse/ClickHouse/pull/118596) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `ALTER TABLE ... EXECUTE remove_orphan_files` on `Iceberg` tables now rejects an `older_than` that is less than `iceberg_orphan_files_min_retention_seconds` (default 1 day) before now, so it no longer deletes the files of a write that started within that window and has not committed yet. Setting `iceberg_orphan_files_min_retention_seconds` to `0` restores the previous behavior. Closes [#125331](https://github.com/ClickHouse/ClickHouse/issues/125331). [#125376](https://github.com/ClickHouse/ClickHouse/pull/125376) ([Groene AI](https://github.com/groeneai)). * Fixed `DROP TABLE` in a `DataLakeCatalog` database with an `Iceberg` REST catalog ignoring `iceberg_delete_data_on_drop` from the server's default profile: ClickHouse always sent `purgeRequested=False`, so the catalog never deleted the table's data, and Microsoft OneLake rejected the drop with HTTP 405. ClickHouse now sends `purgeRequested=true` or `purgeRequested=false` (lowercase, as the `Iceberg` REST spec defines), chosen by that setting; a session-level `SET` does not reach this drop. Closes [#125410](https://github.com/ClickHouse/ClickHouse/issues/125410). [#125469](https://github.com/ClickHouse/ClickHouse/pull/125469) ([Groene AI](https://github.com/groeneai)). * Fixed `add.modificationTime` in Delta Lake commit logs being written in seconds instead of the milliseconds the Delta protocol requires, so a data file written by ClickHouse appeared to Spark, `delta-rs`, and `delta-kernel-rs` as last modified in 1970. Closes [#120523](https://github.com/ClickHouse/ClickHouse/issues/120523). [#120564](https://github.com/ClickHouse/ClickHouse/pull/120564) ([Groene AI](https://github.com/groeneai)). * Fixed wrong values and a `LOGICAL_ERROR` when reading a partitioned `DeltaLake` table with `allow_delta_kernel_rs = 0` and column types that differ from the ones in `_delta_log`. [#121544](https://github.com/ClickHouse/ClickHouse/pull/121544) ([Raúl Marín](https://github.com/Algunenano)). * Fix reading Delta Lake tables with column mapping where a field name with a dot, like `a.b`, equals the path of a nested field (the field `b` of a struct `a`). Such a field returned the other field's values or `NULL`, `SELECT *` failed with `COLUMN_QUERIED_MORE_THAN_ONCE`, and a partitioned table failed with a `LOGICAL_ERROR`. [#123102](https://github.com/ClickHouse/ClickHouse/pull/123102) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a query reads a struct column of a `DeltaLake` table with column mapping together with one of its fields, such as `SELECT s, s.x`, `SELECT *, s.x`, or `SELECT s ... WHERE s.x = 1`. Closes [#123811](https://github.com/ClickHouse/ClickHouse/issues/123811). [#123852](https://github.com/ClickHouse/ClickHouse/pull/123852) ([Groene AI](https://github.com/groeneai)). * Fixed `INSERT` into `DeltaLake` tables through a Unity `DataLakeCatalog`: the catalog now requests write credentials instead of reusing read-only ones. [#123543](https://github.com/ClickHouse/ClickHouse/pull/123543) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix DeltaLake statistics pruning for filters that compare a `Date32` column through `CAST`, `toDate`, or `toDate32`, including a wrong-results case where `1 >= d` was treated as `d >= 1`. [#119184](https://github.com/ClickHouse/ClickHouse/pull/119184) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed `INSERT` into a `DeltaLake` table with a `timestamp_ntz` column writing the `Parquet` data file with `isAdjustedToUTC=true`, which made the committed table unreadable by Spark and other Delta engines. Such a column is now annotated `TIMESTAMP(isAdjustedToUTC=false)`, as the Delta protocol requires. Closes [#120653](https://github.com/ClickHouse/ClickHouse/issues/120653). [#120863](https://github.com/ClickHouse/ClickHouse/pull/120863) ([Groene AI](https://github.com/groeneai)). * Fixed `SHOW TABLES`, `system.tables` scans, and table resolution hanging indefinitely against a `paimon_rest` catalog whose listings paginate. [#115534](https://github.com/ClickHouse/ClickHouse/pull/115534) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fixed data lake tables created on top of a server disk (`SETTINGS disk = '...'`): they no longer overwrite the disk's object storage settings with the table's or the query's settings, and they follow the disk's configuration on `SYSTEM RELOAD CONFIG` instead. [#117398](https://github.com/ClickHouse/ClickHouse/pull/117398) ([alesapin](https://github.com/alesapin)). * Reading an Apache Paimon primary-key table now throws `NOT_IMPLEMENTED` instead of silently returning wrong results. Merge-on-read is not implemented, so the reader returned the raw union of the snapshot's data files, including the row versions superseded by later upserts. Closes [#113360](https://github.com/ClickHouse/ClickHouse/issues/113360). [#113460](https://github.com/ClickHouse/ClickHouse/pull/113460) ([Groene AI](https://github.com/groeneai)). * Fixed `SHOW CREATE DATABASE` and `BACKUP` of a `DataLakeCatalog` database omitting its `COMMENT`, and `ALTER DATABASE ... MODIFY SETTING` erasing that comment from the stored metadata, so that it was lost after a restart. [#122942](https://github.com/ClickHouse/ClickHouse/pull/122942) ([Groene AI](https://github.com/groeneai)). * Reject `Paimon` manifest and data paths that escape the table directory. A crafted table could previously read files outside its own directory through an absolute path or `..` components in metadata paths. [#123099](https://github.com/ClickHouse/ClickHouse/pull/123099) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed a data race between concurrent queries on one `Hudi` table: the list of the table's data files was filled on first use without synchronization. [#119137](https://github.com/ClickHouse/ClickHouse/pull/119137) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the Paimon incremental read silently duplicating and dropping data. Advancing the consumed-snapshot watermark in Keeper is now fenced by the `processing_lock` and monotonic, so a read that lost the lock can no longer commit over another consumer's progress. A cursor left ahead of the warehouse, and a snapshot that fails to load for any reason other than Paimon expiration, now raise an error instead of being silently skipped. [#117630](https://github.com/ClickHouse/ClickHouse/pull/117630) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fixed two ways a table with the `Redis` engine mishandled an `MGET` reply. Reading the whole table stopped at the first key without a string value and silently dropped every key scanned after it, so one key of another Redis type was enough to truncate the result. A reply carrying a different number of elements than keys requested could terminate the server; such a reply is now rejected with `INTERNAL_REDIS_ERROR`. [#120230](https://github.com/ClickHouse/ClickHouse/pull/120230) ([Groene AI](https://github.com/groeneai)). * Fixed a broker-side queue leak when dropping a `RabbitMQ` table. `DROP TABLE` asked the broker to delete the table's queues without waiting for its own consumer channels to close, so the broker could still see a registered consumer and refuse the deletion with `PRECONDITION_FAILED`. [#116050](https://github.com/ClickHouse/ClickHouse/pull/116050) ([Groene AI](https://github.com/groeneai)). * Fixed spurious `NATS` connection failures when the vendored `nats-io` client gets `EINTR` while waiting for a socket, and picked up the upstream JetStream pull-subscription deadlock fix in the vendored client. [#120202](https://github.com/ClickHouse/ClickHouse/pull/120202) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `arrowFlight` and the `ArrowFlight` table engine hanging forever when the remote Flight server accepts a request and never answers it. Requests are now bounded by the new `arrow_flight_request_timeout_sec` setting (300 seconds by default, `0` disables the bound), and a stalled read honors `KILL QUERY` and `max_execution_time`. [#120231](https://github.com/ClickHouse/ClickHouse/pull/120231) ([Groene AI](https://github.com/groeneai)). * Fixed `SELECT` queries on an `EmbeddedRocksDB` table with a composite `PRIMARY KEY` returning fewer rows than match the predicate when the predicate combines `IN` sets with `OR` and one set's element type cannot be converted to the key type. Such a set no longer narrows the candidate keys. [#122261](https://github.com/ClickHouse/ClickHouse/pull/122261) ([Groene AI](https://github.com/groeneai)). * Fixed a `FileLog` table that stopped reading all of its files forever after a single read error (such as a line that failed to parse or a partially written last line): every later poll reported `Another select query is running on this table` and every direct `SELECT` failed with `CANNOT_SELECT` until the server was restarted. Related to [#123368](https://github.com/ClickHouse/ClickHouse/issues/123368). [#123395](https://github.com/ClickHouse/ClickHouse/pull/123395) ([Groene AI](https://github.com/groeneai)). * Fixed a `NATS` engine table reading from a JetStream stream (`nats_stream`) silently stopping to consume messages after the NATS server restarted or the connection was re-established. [#115343](https://github.com/ClickHouse/ClickHouse/pull/115343) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119301](https://github.com/ClickHouse/ClickHouse/pull/119301) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a rare server crash (`SIGSEGV` in `mongoc_log`) when a MongoDB dictionary source or `MongoDB` table was created while another thread created the first MongoDB connection in the process. Messages logged by the MongoDB driver now go to the server log. [#124235](https://github.com/ClickHouse/ClickHouse/pull/124235) ([Vladimir Cherkasov](https://github.com/vdimir)). * `RENAME DATABASE` is now refused when the database holds a loaded `S3Queue` or `AzureQueue` table whose `keeper_path` expanded the `{database}` macro when the table was loaded. Previously the rename succeeded silently, the table bound to a different `Keeper` path at the next server start and reprocessed every file it had already consumed. Not checked: detached tables, tables not yet materialized under `lazy_load_tables = 1`, and a `keeper_path` whose macros a config reload redefined after the table was loaded. [#120103](https://github.com/ClickHouse/ClickHouse/pull/120103) ([Groene AI](https://github.com/groeneai)). * `SELECT count()` and `system.tables.total_rows` of a `Join` table that keeps every row of a key, `ALL`, or `ANY` / `SEMI` / `ANTI` with `RIGHT`, returned the number of distinct keys instead of the number of rows. [#124099](https://github.com/ClickHouse/ClickHouse/pull/124099) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Enum8` and `Enum16` columns of `PostgreSQL` and `MySQL` tables, `Nullable(Enum8)` and `Nullable(Enum16)` columns of `PostgreSQL` tables, and `Enum` range columns of a `PostgreSQL` dictionary source. Values are now returned as the declared `Enum` type, parsed by element name or by number, instead of the raw remote text typed as `String` or an `Unexpected value N in enum` error. A remote value that is not an element of the enum is now rejected with an error. [#125198](https://github.com/ClickHouse/ClickHouse/pull/125198) ([Groene AI](https://github.com/groeneai)). * Fixes `RabbitMQ` tables with `rabbitmq_num_consumers > 1` that insert into materialized views with `parallel_view_processing = 0` (the default) falling behind: the consumers were read one after another, all but the first inserted a single message per flush interval, and their messages waited many intervals (up to about 28 s at 100 msg/s with a 1 s interval). They are now read in parallel. [#125106](https://github.com/ClickHouse/ClickHouse/pull/125106) ([Sven Eliasson](https://github.com/comino)). * Fixed a server crash (segmentation fault) in a direct `SELECT` from a `NATS` table while the table replaces a connection that the NATS client library closed for good, for example after the broker rejected rotated credentials. Closes [#120199](https://github.com/ClickHouse/ClickHouse/issues/120199). [#125494](https://github.com/ClickHouse/ClickHouse/pull/125494) ([Groene AI](https://github.com/groeneai)). * Fixed a heap-use-after-free when an `EmbeddedRocksDB` table with a composite `PRIMARY KEY` was filtered by a tuple equality whose tuple has a `LowCardinality` element, e.g. `WHERE (k, id) = (SELECT (name, id) FROM t LIMIT 1)`. Closes [#120527](https://github.com/ClickHouse/ClickHouse/issues/120527). [#125509](https://github.com/ClickHouse/ClickHouse/pull/125509) ([Groene AI](https://github.com/groeneai)). * Fixed the Arrow Flight interface returning garbage values for a `MergeTree` column stored in sparse serialization, which is the default for a column that holds mostly default values. Affected `DoGet` with a SQL ticket, `GetFlightInfo` and `PollFlightInfo`. [#125553](https://github.com/ClickHouse/ClickHouse/pull/125553) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash when a `RabbitMQ` table with the `headers` exchange type had a routing key without `=`, or when `rabbitmq_routing_key_list` was empty. Routing keys without `=` for the `headers` exchange type are now rejected with an exception. The problem was found and reported by [@dstesk](https://github.com/dstesk). [#125661](https://github.com/ClickHouse/ClickHouse/pull/125661) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `StorageKafka2` now counts only live replicas when distributing partition locks, so a replica that lost its `Keeper` session no longer leaves some partitions unconsumed. [#120111](https://github.com/ClickHouse/ClickHouse/pull/120111) ([johnjing](https://github.com/JingYanchao)). * Fixed memory growth in `Kafka` tables with offsets stored in `Keeper` when partitions move between consumers: prefetched messages for released partitions are now freed rather than retained until restart. Closes [#125077](https://github.com/ClickHouse/ClickHouse/issues/125077). [#125076](https://github.com/ClickHouse/ClickHouse/pull/125076) ([Aparajita Pandey](https://github.com/aparajita31pandey)). * Fixed spurious `TLS/SSL error: Interrupted system call` failures in `mysql`, the `MySQL` table engine, the `MySQL` database engine, and `MySQL` dictionaries by updating `mariadb-connector-c` to v3.4.10, with a patch that clears the OpenSSL error queue before every TLS operation. [#120526](https://github.com/ClickHouse/ClickHouse/pull/120526) ([Groene AI](https://github.com/groeneai)). * Fixed a durability bug in the `MySQL` and `PostgreSQL` database engines. The marker file recording whether a table is detached or dropped was created and removed without fsyncing the metadata directory, so a power loss could silently revert an acknowledged `DETACH TABLE ... PERMANENTLY`, `DROP TABLE`, or `ATTACH TABLE` and change the table's visibility on restart. It is now fsynced when `fsync_metadata` is enabled (the default). A failed `ATTACH TABLE` no longer leaves the table half-attached, which made a retry report `TABLE_ALREADY_EXISTS`. [#112170](https://github.com/ClickHouse/ClickHouse/pull/112170) ([Groene AI](https://github.com/groeneai)). * Using the `PostgreSQL` engine, inserts now quote table and column names so PostgreSQL does not fold them to lowercase. The destination table can therefore use any identifier casing. [#120435](https://github.com/ClickHouse/ClickHouse/pull/120435) ([Thomas Sileghem](https://github.com/mastilver)). * Fixed `KILL QUERY` and `max_execution_time` having no effect on a query reading from `PostgreSQL` when the remote host stops sending data, and `DROP` of a refreshable materialized view whose refresh reads from `PostgreSQL` hanging for the same reason. [#118995](https://github.com/ClickHouse/ClickHouse/pull/118995) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `PostgreSQL` connections losing `send_timeout` after the handshake. [#123094](https://github.com/ClickHouse/ClickHouse/pull/123094) ([Sema Checherinda](https://github.com/CheSema)). * HTTP dictionary sources (`SOURCE(HTTP(...))`) now respect the `max_http_get_redirects` setting, so dictionaries can be created from URLs that redirect. Previously such dictionaries always failed with `TOO_MANY_REDIRECTS`. [#101768](https://github.com/ClickHouse/ClickHouse/pull/101768) ([Yash ](https://github.com/Onyx2406)). * Fix `optimize_inverse_dictionary_lookup` rewriting `dictGet` comparisons incorrectly: complex-key dictionaries with a single key column called with a `tuple`-wrapped, possibly `Nullable`, key expression failed with `ILLEGAL_TYPE_OF_ARGUMENT`; key expressions that rely on `dictGet`'s implicit key type conversion, e.g. a `String` expression for a `UUID` key, failed with `NO_COMMON_TYPE`, and lossy ones silently returned results where `dictGet` throws; a match on a dictionary key containing `NULL` produced `NULL` instead of `false`; a single matching floating-point `NaN` or zero key produced wrong results; and mismatched key shapes were folded to a result instead of the `dictGet` error. Key expressions that would need a conversion are no longer rewritten and keep the `dictGet` lookup, except a signed integer probe of a simple-key dictionary, which is converted to the `UInt64` lookup type the way `dictGet` converts it. [#114865](https://github.com/ClickHouse/ClickHouse/pull/114865) ([Artur Beglaryan](https://github.com/artbeglaryan)). * Fixed a dictionary with a local `CLICKHOUSE` source that stopped updating with `ACCESS_DENIED` and kept serving stale data after a new role was granted to its source user and the old role was dropped. Every query of such a source now uses the current default roles of the user, like a new session of that user. Closes [#122663](https://github.com/ClickHouse/ClickHouse/issues/122663). [#124051](https://github.com/ClickHouse/ClickHouse/pull/124051) ([Groene AI](https://github.com/groeneai)). * Fixed ClickHouse Keeper terminating when a client sent a `multi` request with no subrequests. Such a request was written to the Keeper changelog before it failed, so the termination repeated on every restart. [#119847](https://github.com/ClickHouse/ClickHouse/pull/119847) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed ClickHouse Keeper discarding a `world:anyone` ACL entry with all permissions when the request contained other entries. Such a node was stored more restrictively than requested and could end up repairable only with a `super` credential; with `` configured and no ``, a `ReplicatedMergeTree` under the configured path could not be created at all. These nodes now get the requested permissions, which grants every session full access to them, as ZooKeeper does. Closes [#120177](https://github.com/ClickHouse/ClickHouse/issues/120177). [#120265](https://github.com/ClickHouse/ClickHouse/pull/120265) ([Groene AI](https://github.com/groeneai)). * Fixed `Keeper` persistent recursive watch events to report the path of the changed node, matching ZooKeeper behavior. Previously, they reported the path where the watch was registered. [#121333](https://github.com/ClickHouse/ClickHouse/pull/121333) ([Kirill](https://github.com/kirillgarbar)). * Fix a segmentation fault in `ClickHouse Keeper` when preallocating space for a changelog file fails because the disk is full. [#122337](https://github.com/ClickHouse/ClickHouse/pull/122337) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * ClickHouse Keeper no longer sends the same watch event to one session several times when more than one of the session's watches matches a change, for example persistent recursive watches on `/a` and `/a/b` for a `create /a/b/c`, or a single persistent watch on a removed node. Like ZooKeeper, Keeper now sends each event to a session once. Closes [#122291](https://github.com/ClickHouse/ClickHouse/issues/122291). [#122706](https://github.com/ClickHouse/ClickHouse/pull/122706) ([Groene AI](https://github.com/groeneai)). * Fixed a Keeper durability gap: with `force_sync` enabled, the directory entry of a new changelog segment, or of a segment continued after a restart, was never fsynced, so acknowledged writes in that segment could be lost after a power loss on filesystems where a file fsync does not persist the file's name. Keeper now fsyncs the log directory before the first entries of such a segment are reported durable, and stops if that fails. [#122755](https://github.com/ClickHouse/ClickHouse/pull/122755) ([Groene AI](https://github.com/groeneai)). * Fixed Keeper silently giving a new session to a ZooKeeper client, for example the Java client or `kazoo`, that reconnects and asks to continue its session. Such a client now gets `SessionExpired`, ZooKeeper's answer for a session that cannot be continued, instead of losing its ephemeral nodes later without notice. Keeper still does not restore sessions. Closes [#116690](https://github.com/ClickHouse/ClickHouse/issues/116690). [#122767](https://github.com/ClickHouse/ClickHouse/pull/122767) ([Groene AI](https://github.com/groeneai)). * Fixed the Keeper HTTP storage API (`/api/v1/storage`) storing a truncated value and reporting success when a `POST` or `PUT` body was larger than `max_request_size`. Such requests are now rejected with `413`, and the node is left unchanged. Closes [#122286](https://github.com/ClickHouse/ClickHouse/issues/122286). [#122928](https://github.com/ClickHouse/ClickHouse/pull/122928) ([Groene AI](https://github.com/groeneai)). * Fix Keeper recovery with `remove_orphaned_nodes_on_startup` leaving an unrepaired snapshot on disk. Persist the repaired snapshot so followers can install it successfully and the recovered node can restart with orphan recovery disabled. [#123206](https://github.com/ClickHouse/ClickHouse/pull/123206) ([Seva Potapov](https://github.com/seva-potapov)). * Fixed a Keeper node that restarted after a crash sometimes giving a new client the session ID of a client that had connected to it before the restart, after which the new client received that client's responses. [#124307](https://github.com/ClickHouse/ClickHouse/pull/124307) ([Groene AI](https://github.com/groeneai)). * ClickHouse now rejects relative paths in `checksums.txt`. [#120356](https://github.com/ClickHouse/ClickHouse/pull/120356) ([alesapin](https://github.com/alesapin)). * Encryption codecs (`AES_128_GCM_SIV`, `AES_256_GCM_SIV`) now also cover the structural substreams of a column. [#120636](https://github.com/ClickHouse/ClickHouse/pull/120636) ([Raufs Dunamalijevs](https://github.com/rienath)). * Rejected a compressed block whose codec stores data verbatim, such as `NONE` or `Quantized`, when its declared uncompressed size does not match the body length. Previously that passthrough read path accepted such a block while every other decompression path rejected it. [#114344](https://github.com/ClickHouse/ClickHouse/pull/114344) ([Groene AI](https://github.com/groeneai)). * Reject invalid HTTP header names supplied to SQL-facing HTTP clients (the `url` table function and engine, HTTP dictionaries, `S3`/object storage, and `DataLake`/REST catalogs). A header name must be a valid RFC 9110 token (letters, digits and ``!#$%&'*+-.^_`|~``); any other byte, `:`, whitespace, control characters, or separators such as `/` and `=`, is rejected. Such names could previously bypass the `` restriction. [#118830](https://github.com/ClickHouse/ClickHouse/pull/118830) ([Sergei Trifonov](https://github.com/serxa)). * Fixed `ALTER QUOTA` silently not taking effect when a quota is changed to `KEYED BY client_key`: a user whose quota context was already cached kept the previous, more permissive limits, and the failed refresh also stopped every other cached context from picking up unrelated quota changes. [#120240](https://github.com/ClickHouse/ClickHouse/pull/120240) ([Groene AI](https://github.com/groeneai)). * Kafka producer and consumer trace logs now show secret `librdkafka` properties, such as `sasl.password`, as `[HIDDEN]` instead of their cleartext values. [#119244](https://github.com/ClickHouse/ClickHouse/pull/119244) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * `EXPLAIN AST` no longer prints secret function arguments, such as the key of `encrypt`, in cleartext, including with `optimize = 1` for the body of a view the user may only `SELECT` from. [#120313](https://github.com/ClickHouse/ClickHouse/pull/120313) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * ClickHouse now masks credentials in `system.query_log` even for invalid URLs of external table engines. [#122057](https://github.com/ClickHouse/ClickHouse/pull/122057) ([alesapin](https://github.com/alesapin)). * The value of `role_session_name` is now hidden in `query_log`. [#122423](https://github.com/ClickHouse/ClickHouse/pull/122423) ([Alexey Bakharew](https://github.com/alexbakharew)). * `BACKUP ... TO S3(...)` and `RESTORE ... FROM S3(...)` now enforce `remote_url_allow_hosts` on the initial `S3` URL, instead of checking only redirect targets. Endpoints that the `s3` table function would reject now fail with `UNACCEPTABLE_URL` here as well. [#123114](https://github.com/ClickHouse/ClickHouse/pull/123114) ([Konstantin Vedernikov](https://github.com/scanhex12)). * A password or presigned signature in a URL is no longer printed in clear by the `ConfluentSchemaRegistry` trace log, by the `URL ... is not allowed in configuration file, see ` error, or by the `Unsupported scheme in URI` error. These now show `scheme://[HIDDEN]@host` and `X-Amz-Signature=[HIDDEN]`, as `system.query_log` already does for `format_avro_schema_registry_url`. [#123226](https://github.com/ClickHouse/ClickHouse/pull/123226) ([Groene AI](https://github.com/groeneai)). * Fixed `SYSTEM SYNC TRANSACTION LOG` (local, non-`ON CLUSTER` form) not checking the `SYSTEM_SYNC_TRANSACTION_LOG` privilege. The `ON CLUSTER` form already required it, so a user without the grant could run the local form when experimental transactions were enabled. [#112441](https://github.com/ClickHouse/ClickHouse/pull/112441) ([Azhar Izzannada E](https://github.com/azharizz)). * The values of MongoDB connection options that carry a secret (`tlsCertificateKeyFilePassword`, `sslClientCertificateKeyPassword`, `AWS_SESSION_TOKEN` in `authMechanismProperties`) are now hidden in `SHOW CREATE`, `system.query_log` and `system.dictionaries.source` for the `MongoDB` table engine, the `mongodb` table function and the MongoDB dictionary source, `system.dictionaries.source` no longer shows the MongoDB password, and a named-collection `uri` override after another override is no longer shown as written. [#123240](https://github.com/ClickHouse/ClickHouse/pull/123240) ([Groene AI](https://github.com/groeneai)). * A `SELECT` row policy on the `mergeTreeProjection` table function (`_table_function.mergeTreeProjection`) is now also applied to a read shipped to another server as a serialized query plan (`serialize_query_plan = 1`, with `process_query_plan_packet` enabled there), as it already is for a local read or with `serialize_query_plan = 0`. [#123609](https://github.com/ClickHouse/ClickHouse/pull/123609) ([Groene AI](https://github.com/groeneai)). * `SYSTEM DROP DATABASE REPLICA` without a database name is now denied for a user without the `SYSTEM DROP REPLICA` privilege, instead of silently doing nothing and reporting success. This makes it consistent with `SYSTEM DROP REPLICA`. [#117964](https://github.com/ClickHouse/ClickHouse/pull/117964) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... MODIFY SETTING disk = disk(...)` could register a disk defined in SQL whose local path is outside `custom_local_disks_base_directory`, and create its directory, bypassing the check that `CREATE TABLE` applies. Now it is rejected the same way, and a statement rejected this way leaves no disk and no directory behind. [#119850](https://github.com/ClickHouse/ClickHouse/pull/119850) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALTER TABLE ... ATTACH PARTITION ... FROM ...` and `CREATE TABLE ... CLONE AS ...` requiring the `ALTER DELETE` privilege on the destination table. These statements only add data, so `INSERT` is now enough. `REPLACE PARTITION`, which does drop the destination data, still requires `ALTER DELETE`. A session temporary table used as the source of `ATTACH` or `REPLACE PARTITION ... FROM` also no longer wrongly requires `SELECT` on a same-named table of the current database. [#117936](https://github.com/ClickHouse/ClickHouse/pull/117936) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the server terminating when an argument of a `disk(...)` setting value is a key without a value, such as `disk(equals(name))`. A malformed `disk(...)` argument no longer causes the whole query, including its other secrets, to be logged unmasked. [#123847](https://github.com/ClickHouse/ClickHouse/pull/123847) ([Groene AI](https://github.com/groeneai)). * Mask secrets in the queries that a cluster-mode `QueryRunner` table writes to `system.query_log` as well as in queries executed as startup scripts. [#118978](https://github.com/ClickHouse/ClickHouse/pull/118978) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * A `RESTORE TABLE ... AS ` of a backup whose engine arguments were stripped now validates the Keeper path it derives from `default_replica_path`. Previously the `{database}` and `{table}` substitutions were expanded against the restored name without being checked, so a name containing `/` resolved into another table's Keeper subtree and left an inactive replica there. [#121546](https://github.com/ClickHouse/ClickHouse/pull/121546) ([Groene AI](https://github.com/groeneai)). * Hide secrets in the `system.mutations` table. [#122717](https://github.com/ClickHouse/ClickHouse/pull/122717) ([alesapin](https://github.com/alesapin)). * Creating a `FileLog` table now requires the `READ` and `WRITE` privileges, as for a `File` table. [#125701](https://github.com/ClickHouse/ClickHouse/pull/125701) ([Robert Schulze](https://github.com/rschu1ze)). * Fixed an out-of-bounds read when reading a `Variant` column from a truncated `Native` stream using compact discriminators serialization. [#120252](https://github.com/ClickHouse/ClickHouse/pull/120252) ([Raufs Dunamalijevs](https://github.com/rienath)). * Reject malformed PostgreSQL startup messages whose parameter data exceeds the declared payload size. [#112780](https://github.com/ClickHouse/ClickHouse/pull/112780) ([Minh Vu](https://github.com/fallintoplace)). * Reading settings from the Native protocol no longer sizes a string from the length a peer declares before the payload arrives, so a Query packet or a server hello can no longer turn a few bytes on the wire into a large memory allocation. This covers settings and query parameters, on the server and in the client. [#118612](https://github.com/ClickHouse/ClickHouse/pull/118612) ([Groene AI](https://github.com/groeneai)). [#120931](https://github.com/ClickHouse/ClickHouse/pull/120931) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash when a client sends a block over the native TCP protocol that selects the internal `Detached` serialization kind for a column. [#117699](https://github.com/ClickHouse/ClickHouse/pull/117699) ([Pavel Kruglov](https://github.com/Avogar)). * `COPY` options are no longer silently ignored on the PostgreSQL wire protocol: `WITH (FORMAT csv)` and the legacy `WITH CSV` are now applied, unsupported options are reported instead of skipped, and TSV header auto-detection no longer drops the first data row of a `COPY ... FROM STDIN`. [#119376](https://github.com/ClickHouse/ClickHouse/pull/119376) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Bound the header column count in `RowBinary` formats to avoid excessive memory allocation from a crafted header. [#120282](https://github.com/ClickHouse/ClickHouse/pull/120282) ([George Larionov](https://github.com/george-larionov)). * Reject malformed `Native` blocks or corrupted parts that could make ClickHouse read past the end of a `JSON` or `Dynamic` column, for example because of duplicate types or paths, a truncated stream, or a part-only serialization version sent over the wire. [#117765](https://github.com/ClickHouse/ClickHouse/pull/117765) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed `handshake_timeout_milliseconds` not bounding a connection that stops sending during the handshake, which held a thread for `receive_timeout` (300 seconds by default) instead. The setting now also covers the MySQL and PostgreSQL handshake phases and the TLS negotiation; previously a peer that dribbled bytes during a TLS handshake, or stayed silent on the PostgreSQL port, could hold a thread indefinitely. HTTP header reading keeps its own `http_headers_read_timeout` budget, now enforced by the same mechanism. On `tcp_port_secure`, ClickHouse no longer writes an exception to a client whose TLS handshake has already timed out, so such a peer no longer holds a handler thread for twice the configured budget. [#121547](https://github.com/ClickHouse/ClickHouse/pull/121547) ([Sema Checherinda](https://github.com/CheSema)). [#123101](https://github.com/ClickHouse/ClickHouse/pull/123101) ([Sema Checherinda](https://github.com/CheSema)). * Reject malformed binary input with out-of-range VarInt, integer, or enum values, including settings, client info, query plans, distributed task descriptions, and text indexes, instead of silently truncating them. [#113944](https://github.com/ClickHouse/ClickHouse/pull/113944) ([sunliqiang](https://github.com/Pyolar)). [#123242](https://github.com/ClickHouse/ClickHouse/pull/123242) ([Sergei Trifonov](https://github.com/serxa)). * The collections of the `Native` protocol, the settings and query parameters of a query, the client information, the granted roles, and the table-status lists, are now bounded, so a small packet can no longer make the server allocate memory for data that has not arrived. [#117015](https://github.com/ClickHouse/ClickHouse/pull/117015) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Code: 102. UNEXPECTED_PACKET_FROM_SERVER (... got Progress)` aborting an `INSERT` that requests a table structure from the server, when a subquery is evaluated while the server analyses the query. The `Progress` and `ProfileEvents` packets sent in that window are now drained instead of rejected, both by the client and by the initiator of an `INSERT` into a `Distributed` or `remote` table, where rejecting them also hid the shard's own error. Affects `INSERT ... SELECT`, `input` inserts, and `INSERT ... VALUES` with `async_insert = 0`. [#118946](https://github.com/ClickHouse/ClickHouse/pull/118946) ([Groene AI](https://github.com/groeneai)). * Fixed the PostgreSQL wire protocol rejecting a simple query that consists only of comments, such as the `-- ping` that the Go driver `pgx` v5 sends from `Ping`, with `Empty query`. Such a query is now answered with `EmptyQueryResponse`, as in PostgreSQL. Closes [#84085](https://github.com/ClickHouse/ClickHouse/issues/84085). [#125161](https://github.com/ClickHouse/ClickHouse/pull/125161) ([Groene AI](https://github.com/groeneai)). * Fixed `http_max_multipart_form_data_size` silently truncating oversized multipart data instead of failing the query, and made it span a form's external-table parts instead of resetting per part. [#120036](https://github.com/ClickHouse/ClickHouse/pull/120036) ([Sema Checherinda](https://github.com/CheSema)). * `KILL QUERY`, query timeouts, and client disconnects now stop a query that reads over HTTP, for example over the `url` table function, while it is retrying a request, instead of waiting until all retry attempts are exhausted. [#112930](https://github.com/ClickHouse/ClickHouse/pull/112930) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed gRPC queries with transport compression (`gzip`, `deflate`) crashing the server, as they could in official builds since 26.4. Closes [#115121](https://github.com/ClickHouse/ClickHouse/issues/115121). [#122668](https://github.com/ClickHouse/ClickHouse/pull/122668) ([Groene AI](https://github.com/groeneai)). * Prevented asynchronous inserts that only need to buffer data from being blocked by another insert waiting for a flush worker while holding their shared queue lock. `SYSTEM FLUSH ASYNC INSERT QUEUE` now also waits for batches removed from the queue that are still waiting for a worker. [#118352](https://github.com/ClickHouse/ClickHouse/pull/118352) ([RJ](https://github.com/VrtxOmega)). * Fixed asynchronous `INSERT` under a quota `KEYED BY client_key`: the flush ran without the client's quota key, so it failed with `QUOTA_REQUIRES_CLIENT_KEY` (or charged the wrong bucket). The quota key is now carried through the async batch and restored on flush, and background quota recomputes no longer log a spurious `QUOTA_REQUIRES_CLIENT_KEY` error. [#122296](https://github.com/ClickHouse/ClickHouse/pull/122296) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fixed `max_memory_usage_for_user` being exceeded by memory the user no longer holds. Memory that a query allocates but does not free itself, such as the data of in-memory table engines, cache entries and data waiting in the asynchronous insert queue, is now accounted to whoever owns it, and whatever a query leaves behind is settled when it ends. `max_memory_usage_for_user` is also applied when a query hands over memory it allocated before it had a user. [#113647](https://github.com/ClickHouse/ClickHouse/pull/113647) ([Raúl Marín](https://github.com/Algunenano)). * Queries that send data over pooled connections, such as `INSERT` into a `Distributed` table, are no longer charged for connection buffers that stay in the pool after they finish, which could count against `max_memory_usage_for_user` while another query of the same user was running. [#125573](https://github.com/ClickHouse/ClickHouse/pull/125573) ([Raúl Marín](https://github.com/Algunenano)). * Fix a possible hang of a query under workload CPU scheduling with `cpu_slot_preemption` enabled: when all its threads were preempted, the query could keep CPU slots it had already used and wait forever for one of them. [#123640](https://github.com/ClickHouse/ClickHouse/pull/123640) ([Sergei Trifonov](https://github.com/serxa)). * Fixed vector search returning too few rows for a query with `arrayJoin` below `ORDER BY LIMIT`. A row policy now counts as an additional filter for vector search, so `vector_search_filter_strategy = 'prefilter'` is honored and the index fetch multiplier is applied for a query filtered by a row policy. Fixed `NOT_FOUND_COLUMN_IN_BLOCK` for a non-rescoring vector search query when a `WHERE` filter or a row policy reads the vector column. [#114074](https://github.com/ClickHouse/ClickHouse/pull/114074) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a vector search returning fewer rows than its `LIMIT`, and missing true nearest neighbors, after a lightweight `DELETE` (or with a delete mutation applied on the fly): the vector index is no longer used for a part while it still holds deleted rows. [#119371](https://github.com/ClickHouse/ClickHouse/pull/119371) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `CREATE TABLE` now rejects a `SETTINGS` name that is neither a setting of the table engine nor a query setting for the `File`, `URL`, `S3` (with `COSN`, `OSS`, `GCS`), `AzureBlobStorage`, `HDFS`, `Log`, `TinyLog`, and `StripeLog` engines, with `UNKNOWN_SETTING`, as `MergeTree` already did. The `name = DEFAULT` and `param_name = value` spellings are rejected too, except where the whole clause is dropped before any engine sees it. Such a name used to be accepted, stored in the table metadata, and never read, so a typo like `input_format_csv_delimiter` for `format_csv_delimiter` silently did nothing. A table whose stored definition already names a non-setting still loads. Closes [#120705](https://github.com/ClickHouse/ClickHouse/issues/120705). [#120898](https://github.com/ClickHouse/ClickHouse/pull/120898) ([Groene AI](https://github.com/groeneai)). * `arrayJoin` in a projection or in a `Distributed` sharding key is now rejected at `CREATE`/`ALTER` time. It changes the number of rows, which made every insert into a table with such a projection fail with an exception, and misrouted or failed inserts into such a `Distributed` table. [#117791](https://github.com/ClickHouse/ClickHouse/pull/117791) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALTER TABLE ... MODIFY ORDER BY` accepting a key expression over a column that the same `ALTER` retypes through a subcolumn, for example `MODIFY COLUMN x Tuple(a UInt32, b UInt32), MODIFY ORDER BY (k, x.a)`, or renames, for example `RENAME COLUMN x TO y, MODIFY ORDER BY (k, y)`. Existing parts are not sorted by such a key, so queries that rely on the sort order could return wrong results. Such an `ALTER` is now refused with `BAD_ARGUMENTS`. Closes [#122685](https://github.com/ClickHouse/ClickHouse/issues/122685). [#122765](https://github.com/ClickHouse/ClickHouse/pull/122765) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER TABLE ... MODIFY QUERY` deleting the column comments of a materialized view with a `TO` table. A column that the new query still produces now keeps its comment. [#122775](https://github.com/ClickHouse/ClickHouse/pull/122775) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER TABLE COMMENT COLUMN '...'` and comment-only `ALTER TABLE MODIFY COLUMN COMMENT '...'` returning success and storing nothing on a materialized view with an inner table, which left no supported way to change a column comment on such a view after `CREATE`. [#119469](https://github.com/ClickHouse/ClickHouse/pull/119469) ([Groene AI](https://github.com/groeneai)). [#123033](https://github.com/ClickHouse/ClickHouse/pull/123033) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix `DROP NAMED COLLECTION` succeeding while a `MongoDB` table uses the collection, after which the server could not attach the table on restart. The drop is now refused with `NAMED_COLLECTION_IS_USED`, as for other table engines. [#123636](https://github.com/ClickHouse/ClickHouse/pull/123636) ([Groene AI](https://github.com/groeneai)). * A sharding key of a `Distributed` table, a `Remote` table or the `remote`/`cluster` table functions that contains `IN` with a subquery or with a non-`Set` table is now rejected when the table is defined. The set of such an `IN` was never built, so every `INSERT` failed with a `Not-ready Set` logical error and `optimize_skip_unused_shards` could prune to the wrong shard and return wrong results. A table defined earlier with such a key still loads, but an `INSERT` that needs the key fails with `BAD_ARGUMENTS` and shard pruning does not use it. Pruning by a key with `IN` over a `Set` table, which can change after rows are placed, now needs `allow_nondeterministic_optimize_skip_unused_shards`, as for `joinGet`. [#124646](https://github.com/ClickHouse/ClickHouse/pull/124646) ([Groene AI](https://github.com/groeneai)). * Reject a `RENAME DATABASE` that would make the table dependency graph cyclic, for example one that leaves an `Alias` table referring to itself. Such a rename put a cycle into the server-wide table dependency graph, after which every unrelated `CREATE` that adds a dependency edge failed with `INFINITE_LOOP` and the renamed database stopped being listable after a restart. [#117253](https://github.com/ClickHouse/ClickHouse/pull/117253) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed formatting of a function named `any` with a single subquery argument: ``SELECT 1 = `any`((SELECT 1))`` lost its backticks, and because bare `ANY` before a subquery is the SQL quantifier, formatting the query changed its meaning to `IN`. This affected round-trips such as stored view definitions and `SHOW CREATE`. [#120175](https://github.com/ClickHouse/ClickHouse/pull/120175) ([Groene AI](https://github.com/groeneai)). * Fixed parsing of a column named `from` inside parentheses, such as `WHERE (from IN ('a'))`, which since 26.8 was read as a subquery starting with the `FROM` clause and failed with `Unknown table function IN`, `Unknown table expression identifier`, or a syntax error. [#120950](https://github.com/ClickHouse/ClickHouse/pull/120950) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Preserve quoting for identifiers named `cube`, `rollup`, and `recursive` so formatted queries can be parsed back correctly. [#121896](https://github.com/ClickHouse/ClickHouse/pull/121896) ([Jaysurya Ray](https://github.com/Jaysurya85)). * Fix `SET = DEFAULT` ignoring an active `compatibility` setting: the setting now gets the value of that version, unless the settings constraints forbid it, and `SET compatibility = DEFAULT` reverts the settings it changed. Closes [#116883](https://github.com/ClickHouse/ClickHouse/issues/116883). [#116933](https://github.com/ClickHouse/ClickHouse/pull/116933) ([Groene AI](https://github.com/groeneai)). * Fixed direct config substitutions via `from_env` failing when XML-1.0-legal values contain XML special characters like `&`, `<`, `>`, or the `]]>` sequence; existing `` XML-subtree substitutions are unchanged. An explicit `` substitution can read a YAML subtree wherever the generic `` form is used, including inside a leaf element, so it must not be used for a literal secret or setting. Without the explicit `yaml="true"` attribute, a non-`<` value is literal text, but direct non-`<` leaf `from_zk` substitutions are recommended for exact-byte secrets or settings, because surrounding whitespace of an `` becomes part of a leaf value. Values beginning with `<` continue to be interpreted as XML fragments for backward compatibility. Upgrade note: a direct `from_env` value or a non-`<` leaf `from_zk` value that was previously XML-entity-encoded, for example `a&b`, now resolves to its exact literal bytes and must be stored raw as `a&b` instead. [#101773](https://github.com/ClickHouse/ClickHouse/pull/101773) ([Yash ](https://github.com/Onyx2406)). * Fixed the server refusing to start with `UNKNOWN_ELEMENT_IN_CONFIG` when the definition files of executable user-defined functions or of their drivers are placed in `config.d` (for example with `config.d/*_function.xml`), which also merges their `` or `` elements into the server config. Closes [#123308](https://github.com/ClickHouse/ClickHouse/issues/123308). [#123586](https://github.com/ClickHouse/ClickHouse/pull/123586) ([Groene AI](https://github.com/groeneai)). * Queries over `system.parts`, `system.parts_columns`, `system.projection_parts`, `system.projection_parts_columns`, and `system.dropped_tables_parts` now respect cancellation and `max_execution_time` while the result is being built. [#113909](https://github.com/ClickHouse/ClickHouse/pull/113909) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a data race on the exception message for a cancelled query: threads no longer append context concurrently to one shared exception object. [#119077](https://github.com/ClickHouse/ClickHouse/pull/119077) ([Groene AI](https://github.com/groeneai)). * Fixed unbounded memory growth from time zone names that were not names of the time zone database. A time zone name now has to be one that `system.time_zones` lists, or a fixed UTC offset spelled `Fixed/UTC±HH:MM:SS`. [#120288](https://github.com/ClickHouse/ClickHouse/pull/120288) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server abort during transaction rollback: internal exception-reporting helpers could call `std::terminate` while no exception was being handled. [#122891](https://github.com/ClickHouse/ClickHouse/pull/122891) ([Murphy](https://github.com/murphy-4o)). * Fixed a crash when synchronous logging (`logger.async = 0`) had no log sink that accepted a message. [#123336](https://github.com/ClickHouse/ClickHouse/pull/123336) ([Groene AI](https://github.com/groeneai)). * Fixed a permanent hang of the server on macOS, where threads that format a time (for example while signing `S3` requests, reporting an exceeded quota or reloading a dictionary) could wait forever on the C library's internal time zone lock while the query profiler was enabled. [#125369](https://github.com/ClickHouse/ClickHouse/pull/125369) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` on a table that has a physical column named `_part` shadowing the virtual column, for queries such as `SELECT count() FROM t WHERE isNull(_part > 0)` and `SELECT count() FROM t WHERE _partition_id = 'all'`. [#117539](https://github.com/ClickHouse/ClickHouse/pull/117539) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The statistics cache is now refreshed for read-only `MergeTree` tables. [#119559](https://github.com/ClickHouse/ClickHouse/pull/119559) ([Rory Shanks](https://github.com/rorylshanks)). * A `MergeTree` table with `table_disk = 1` mounts the parts already present at the disk root. If it declares a sorting key those parts were not written with, reading it raised `LOGICAL_ERROR` and aborted the server in builds with assertions enabled. It now reports `INCORRECT_DATA` and names the offending part. [#119295](https://github.com/ClickHouse/ClickHouse/pull/119295) ([Groene AI](https://github.com/groeneai)). * Fix a refresh of a table whose disks are all read-only (`SYSTEM RESTART DISK`, or the background `refresh_parts_interval` task) failing with a `LOGICAL_ERROR` and hiding committed parts when a rolled-back part covers them. Follow-up to [#100992](https://github.com/ClickHouse/ClickHouse/issues/100992), which fixed the same case on the startup part-loading path. [#111167](https://github.com/ClickHouse/ClickHouse/pull/111167) ([Groene AI](https://github.com/groeneai)). * `SYSTEM SYNC MERGES` no longer returns while the part covering the scheduled merges is still uncommitted: it now waits until a new transaction can see that part. [#123396](https://github.com/ClickHouse/ClickHouse/pull/123396) ([Murphy](https://github.com/murphy-4o)). * Reject empty part names in `CHECK TABLE`. [#123625](https://github.com/ClickHouse/ClickHouse/pull/123625) ([Minh Vu](https://github.com/fallintoplace)). * Partition commands on a `DateTime64` key accept a literal with `.` as the date separator again, such as `'2024.02.19 00:00:00.500'`. [#122995](https://github.com/ClickHouse/ClickHouse/pull/122995) ([DeviousCardi](https://github.com/DeviousCardi)). * A `CHECK TABLE` running while a part writes `txn_version.txt` no longer fails with `Unexpected file txn_version.txt.tmp in data part`. [#124158](https://github.com/ClickHouse/ClickHouse/pull/124158) ([Murphy](https://github.com/murphy-4o)). * Fixed `ALTER TABLE ... MATERIALIZE TTL` failing with `UNKNOWN_IDENTIFIER` and being retried forever on a table with `ttl_only_drop_parts = 1` when a part still holds a column that the table dropped while that part's partition was detached. Mutating such a partition after cloning it into another table with `ATTACH PARTITION FROM` no longer fails with a logical error either. [#115962](https://github.com/ClickHouse/ClickHouse/pull/115962) ([Groene AI](https://github.com/groeneai)). * Fixed a member of a `Nested` added by `ALTER TABLE ... ADD COLUMN` to a `Memory` table being read as empty arrays for the rows inserted before the `ALTER`, unless another member of the same `Nested` was read at the same time. It now gets the array sizes of the other members, like in `MergeTree`. This fixes wrong results of `SELECT`, `WHERE`, `PREWHERE`, `ARRAY JOIN`, and row policies over such a column, and `SIZES_OF_ARRAYS_DONT_MATCH` errors. [#122991](https://github.com/ClickHouse/ClickHouse/pull/122991) ([Groene AI](https://github.com/groeneai)). * Fixed `SELECT` from `system.part_moves_between_shards` and `KILL PART_MOVE_TO_SHARD` failing with `Code: 999 ... No node, path .../part_moves_shard` when any replicated table on the server is attached without its Keeper tree. [#122132](https://github.com/ClickHouse/ClickHouse/pull/122132) ([Groene AI](https://github.com/groeneai)). * Fixed an exception (`BAD_GET` or `NOT_IMPLEMENTED`) when a filter on a table with a `_part_offset` projection had a `Float` or `Nullable(Nothing)` condition over a column the projection does not store. [#124568](https://github.com/ClickHouse/ClickHouse/pull/124568) ([Groene AI](https://github.com/groeneai)). * Fixed a skip index recording the wrong expression type when `ALTER TABLE` ran in a session whose settings change how that expression resolves. `toString(v)` over a `Dynamic` column is `Nullable(String)` under `cast_keep_nullable` and `String` at the server baseline, so such an `ALTER` stored `Nullable(String)` while the writers resolved `String`, and the next index rebuild failed an internal type assertion. [#119799](https://github.com/ClickHouse/ClickHouse/pull/119799) ([Groene AI](https://github.com/groeneai)). * Fixed `hasPhrase` rejecting `FixedString` input for some tokenizers, such as `icu` and `splitByRegexp('\W+')`. Closes [#123426](https://github.com/ClickHouse/ClickHouse/issues/123426). [#123585](https://github.com/ClickHouse/ClickHouse/pull/123585) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed statistics about the number of selected rows in trivial `count` queries answered from a text index. [#125380](https://github.com/ClickHouse/ClickHouse/pull/125380) ([Anton Popov](https://github.com/CurtizJ)). * `ALTER TABLE ... RESET SETTING auto_statistics_types` now rebuilds the automatic column statistics from the default types immediately. Previously the table kept the statistics of the old setting value until the server was restarted, so `system.columns` and the statistics written into new parts disagreed with the table definition. [#123599](https://github.com/ClickHouse/ClickHouse/pull/123599) ([Rion Bora](https://github.com/GoldenEmperor1177)). * Fix `basic` statistics of parts written by 26.6 or 26.7: after `ALTER TABLE ... UPDATE` or a lightweight `DELETE`, non-`Nullable` columns reported a default-value count of 0, which made selectivity estimates wrong. Query results were not affected. To repair parts that are already affected, run `ALTER TABLE ... MATERIALIZE STATISTICS ALL`. [#123536](https://github.com/ClickHouse/ClickHouse/pull/123536) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Fixed `indexHint` with a `Nothing`-typed argument, such as `indexHint(assumeNotNull(materialize(NULL)))`, which threw `Cannot create non-empty column with type Nothing` instead of only narrowing the read. [#117635](https://github.com/ClickHouse/ClickHouse/pull/117635) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `arrayReduce` over a non-deterministic aggregate such as an unseeded `groupArraySample` being treated as deterministic, which pushed a `HAVING` predicate below the aggregation and collapsed per-row draws into a single one. [#118971](https://github.com/ClickHouse/ClickHouse/pull/118971) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an internal error when two `*` matchers whose column transformers differ in kind at the same position are compared, for example two lambdas sharing an alias with bodies `* EXCEPT (a)` and `* REPLACE (1 AS a)`. Such a query now reports `MULTIPLE_EXPRESSIONS_FOR_ALIAS`. [#120234](https://github.com/ClickHouse/ClickHouse/pull/120234) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Can't set alias of ... of QualifiedAsterisk`) when a column transformer that assigns a name, such as `APPLY` with a name prefix or `REPLACE`, is applied to an expression that expands to an asterisk or a `COLUMNS` matcher. Such queries now return a clear `BAD_ARGUMENTS` exception. [#109223](https://github.com/ClickHouse/ClickHouse/pull/109223) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` for `LIMIT BY` on an injective function of a `GROUP BY` key, such as `GROUP BY toString(c0) LIMIT 1 BY toString(c0)`. [#120070](https://github.com/ClickHouse/ClickHouse/pull/120070) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `LOGICAL_ERROR: Incorrect table expression` when `EXPLAIN AST optimize = 1` or `EXPLAIN SYNTAX` reads from the experimental table function `eval`. [#121272](https://github.com/ClickHouse/ClickHouse/pull/121272) ([Groene AI](https://github.com/groeneai)). * A non-finite `FROM`, `TO`, `STEP`, or `STALENESS` value in `ORDER BY ... WITH FILL` (`nan`, `inf`, or `-inf`) is now rejected with `INVALID_WITH_FILL_EXPRESSION`. Previously such a value either made the query generate fill rows endlessly (`TO nan`, `TO inf` ascending, `TO -inf` descending, and the same values for `STALENESS`) or produced a wrong result: a non-finite `FROM` emitted the bound itself as a row and dropped the fill rows that `TO` asked for, while a non-finite `STEP`, or a `TO` bound the fill runs away from, returned the rows unfilled. Pass a finite value instead, or omit it. [#118434](https://github.com/ClickHouse/ClickHouse/pull/118434) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed exception `Trying to compare constants of incompatible types` for a query with an `IN` list over a column of a table that has a `CONSTRAINT ... ASSUME`, when both `convert_query_to_cnf` and `optimize_using_constraints` are enabled. [#118278](https://github.com/ClickHouse/ClickHouse/pull/118278) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `EXPLAIN AST` no longer throws when a `Backup` locator name looks like a `COLUMNS` matcher. [#122870](https://github.com/ClickHouse/ClickHouse/pull/122870) ([Groene AI](https://github.com/groeneai)). * `ORDER BY ... WITH FILL ... INTERPOLATE` of the fill column itself is now rejected over a `Distributed` table as well, with the same exception a local table already reported, instead of returning rows whose fill column was overwritten by the interpolated expression. [#118304](https://github.com/ClickHouse/ClickHouse/pull/118304) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a query over the `filesystem` table function combines `indexHint` with an ordinary condition on the same column. [#118671](https://github.com/ClickHouse/ClickHouse/pull/118671) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results from `optimize_using_constraints` with a `CONSTRAINT ... ASSUME NOT (...)`: a negated comparison could be added to the constraints graph as if it were positive, making predicates such as `a < b` appear always true and `a >= b` always false. [#121255](https://github.com/ClickHouse/ClickHouse/pull/121255) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A `HAVING` clause referencing a `SELECT` alias whose quoted name contains a dot, as in ``SELECT sum(t1.c1) AS `t1.c1` FROM t1 AS t1 HAVING `t1.c1` > 0``, is no longer rejected with `NOT_AN_AGGREGATE`. The identifier resolution cache conflated that quoted one-part name with the qualified `t1`.`c1`, which could also return a wrong result instead of an error when both spellings met in one query. Default settings were affected from 26.6, where `enable_identifier_resolve_cache` became enabled by default. [#122017](https://github.com/ClickHouse/ClickHouse/pull/122017) ([Groene AI](https://github.com/groeneai)). * The analyzer now rejects `arrayJoin` in `INTERPOLATE` and in `HAVING` with `WITH TOTALS`. Before, `INTERPOLATE` used only the first array element, and `HAVING` failed only during execution. [#123538](https://github.com/ClickHouse/ClickHouse/pull/123538) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when a comparison of single-element tuples whose elements are tuples, such as `tuple((a, b)) = tuple((c, d))`, is part of an alias used in `GROUP BY` and also in `ORDER BY`, `HAVING`, or `WHERE`. Such comparisons are now rewritten completely, so `WHERE tuple(tuple(x)) = tuple(tuple(y))` over a `JOIN` no longer returns rows where `x` and `y` are both `NULL`, the same as `WHERE tuple(x) = tuple(y)`. [#124546](https://github.com/ClickHouse/ClickHouse/pull/124546) ([Groene AI](https://github.com/groeneai)). * Fixed the exception `Not-ready Set is passed as the second argument for function 'in'` when the `additional_result_filter` setting contains an `IN` subquery. Such a query now applies the filter instead of failing. [#117719](https://github.com/ClickHouse/ClickHouse/pull/117719) ([Groene AI](https://github.com/groeneai)). * A nested correlated subquery that cannot be decorrelated now reports `NOT_IMPLEMENTED` instead of the internal error `Cannot find column ... in actions DAG input`. [#118317](https://github.com/ClickHouse/ClickHouse/pull/118317) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_IMPLEMENTED` for a correlated subquery in the `SELECT` list of a query that has a `WHERE`, when the correlated column is not also selected, and `NOT_FOUND_COLUMN_IN_BLOCK` for a correlated subquery in `HAVING` of a query with `WITH TOTALS`. [#118402](https://github.com/ClickHouse/ClickHouse/pull/118402) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` for a correlated subquery that reads a tuple element of an outer column by position (`o.p.1`), and for the same shape on the left of `IN (subquery)` under `rewrite_in_to_join`. [#118287](https://github.com/ClickHouse/ClickHouse/pull/118287) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Cannot evaluate correlated scalar subquery` for a query that uses one correlated scalar subquery twice in the same expression, for example in both branches of `if(1 = 1, ..., ...)`. [#118321](https://github.com/ClickHouse/ClickHouse/pull/118321) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an out-of-bounds read when finalizing a crafted `analysisOfVariance` aggregate state whose internal arrays have mismatched lengths. [#107860](https://github.com/ClickHouse/ClickHouse/pull/107860) ([uwezkhan](https://github.com/uwezkhan)). * Fixed an out-of-bounds read of the parsed pattern in `sequenceMatch`, `sequenceCount`, and `sequenceMatchEvents`. When the trailing part of a pattern consisted only of actions that a zero-length match satisfies (`.*` and `(?t...)` conditions), the matcher read one element past the end of its action list, so `sequenceMatch` and `sequenceCount` could return `0` for a pattern that does match. [#119871](https://github.com/ClickHouse/ClickHouse/pull/119871) ([Groene AI](https://github.com/groeneai)). * Fix an out-of-bounds read when deserializing a `uniqTheta` state shorter than 24 bytes, for instance from `CAST( AS AggregateFunction(uniqTheta, ...))`. [#119598](https://github.com/ClickHouse/ClickHouse/pull/119598) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `regr_r2` returning `1` instead of `nan` when the data contains a NaN or an infinity. `regr_sxx` and `regr_syy` returned `0` in the same case. [#123249](https://github.com/ClickHouse/ClickHouse/pull/123249) ([mosya415](https://github.com/mosya415)). * Reject malformed serialized `groupBitmap` states with `INCORRECT_DATA` instead of failing later during bitmap operations. [#124049](https://github.com/ClickHouse/ClickHouse/pull/124049) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed wrong results in the `largestTriangleThreeBuckets` aggregate function for `DateTime64` arguments close to the type's upper bound: the maximum `DateTime64(9)` value was returned as `1677-09-21 00:12:43.145224192`. Such values are now saturated instead of wrapping, and a deserialized state whose `Date`, `Date32`, `DateTime` or `DateTime64` coordinates are out of range or fractional is rejected with an error. [#120284](https://github.com/ClickHouse/ClickHouse/pull/120284) ([Groene AI](https://github.com/groeneai)). * Fix a wrong result, or an exception in debug builds, when `optimize_aggregators_of_group_by_keys` eliminated `any`, `min` or `max` of a `GROUPING SETS` key with `group_by_use_nulls`. [#125575](https://github.com/ClickHouse/ClickHouse/pull/125575) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed several URL-parsing bugs in `domainRFC` and `portRFC` for bracketed IPv6 hosts: an embedded IPv4 tail (for example `[::ffff:192.168.0.1]`) was rejected, non-IPv6 bracket contents and garbage following the closing bracket were silently accepted, and userinfo state could leak into host validation. Also fixed `domainRFC` and `portRFC` resolving the wrong host when a URL's userinfo contains more than one `@`, rejected a host starting with a literal dot in `domain` and `domainRFC`, and fixed `firstSignificantSubdomainRFC` and `cutToFirstSignificantSubdomainRFC` returning a bogus subdomain for a bracketed IPv6 host instead of an empty result. [#118979](https://github.com/ClickHouse/ClickHouse/pull/118979) ([Luís Lizardo](https://github.com/lizardoluis)). * Fixed `IPv4StringToNum`, `IPv6StringToNum`, `isIPAddressInRange`, `parseDateTime`, `parseDateTime64`, `parseDateTimeInJodaSyntax`, `parseTimeDelta`, `parseReadableSize`, `toModifiedJulianDay`, `JSONMergePatch`, `stringToH3`, and the H3 index functions throwing an exception on entirely valid `LowCardinality` data. [#117359](https://github.com/ClickHouse/ClickHouse/pull/117359) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the exception `Cannot convert out of range floating point value to integer type` when a logical function (`and`, `or`, `xor`) receives a constant floating point argument outside `{0, 1}`, such as `SELECT 1 OR 2.0`. [#117965](https://github.com/ClickHouse/ClickHouse/pull/117965) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `timeSeriesRange` and `timeSeriesFromGrid` now throw `ARGUMENT_OUT_OF_BOUND` when the requested number of points would exceed `function_range_max_elements_in_block`, matching the behavior of `range`. Previously an unbounded request would run for tens of seconds and allocate hundreds of gigabytes before hitting the memory limit. [#121056](https://github.com/ClickHouse/ClickHouse/pull/121056) ([Ilya Yatsishin](https://github.com/qoega)). * `bitShiftLeft`, `bitShiftRight`, `bitRotateLeft`, and `bitRotateRight` no longer answer, once the expression is compiled, for the arguments their interpreted implementation rejects: a big-integer or negative shift amount. [#118716](https://github.com/ClickHouse/ClickHouse/pull/118716) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `mapPopulateSeries` keeping the wrong value when duplicate keys are present. [#119832](https://github.com/ClickHouse/ClickHouse/pull/119832) ([Minh Vu](https://github.com/fallintoplace)). * Fix `format` handling of escaped braces around replacement fields. [#120704](https://github.com/ClickHouse/ClickHouse/pull/120704) ([Minh Vu](https://github.com/fallintoplace)). * `jaroSimilarity` and `jaroWinklerSimilarity` now return `0` when one input is empty. [#119710](https://github.com/ClickHouse/ClickHouse/pull/119710) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `plus` and `minus` over an array of dates and an array of tuples holding `NULL`, for example `toString([toDateTime64(1, 3)] + [(2, NULL)])`, and the exception `ColumnTuple cannot have ColumnConst as its element` when adding a constant interval to a non-constant interval of another unit, for example `toIntervalDay(number) + INTERVAL 1 HOUR`. [#122451](https://github.com/ClickHouse/ClickHouse/pull/122451) ([Groene AI](https://github.com/groeneai)). * Fixed `timeSeriesCopyTags` and `timeSeriesCopyTag` when both group arguments are constant. They returned a single row regardless of the number of input rows, so such a call over several rows failed with `Invalid number of rows in Chunk` (`LOGICAL_ERROR`), and a single-row query printed an extra row in `clickhouse-client`. [#122990](https://github.com/ClickHouse/ClickHouse/pull/122990) ([Groene AI](https://github.com/groeneai)). * Fixed `if` and `CASE` over tuples failing with `ColumnTuple cannot have ColumnConst as its element` when an element is `NULL` in both branches, for example `if(number = 0, (1, NULL), (number, NULL))`. [#123268](https://github.com/ClickHouse/ClickHouse/pull/123268) ([Groene AI](https://github.com/groeneai)). * Fixed `geometryIntersectCartesian` and `geometryIntersectSpherical` returning an arbitrary result (usually `1`) for an empty `LineString`, or a `MultiLineString` with an empty element, against a `Ring`, `Polygon`, or `MultiPolygon`. For such a `MultiLineString`, `polygonsDistanceCartesian` and `polygonsDistanceSpherical` could return `0`. [#124101](https://github.com/ClickHouse/ClickHouse/pull/124101) ([Groene AI](https://github.com/groeneai)). * Fixed an infinite, uncancellable loop in functions `hop`, `hopStart` and `hopEnd` when subtracting the hop interval does not move the window end, for example `hopEnd(toDateTime(0, 'UTC'), toIntervalQuarter(413844), toIntervalQuarter(413844))`, where the date saturates at the calendar limit. Such arguments now throw the `Time overflow` error. [#124570](https://github.com/ClickHouse/ClickHouse/pull/124570) ([Groene AI](https://github.com/groeneai)). * Fixed the result type of `randConstant`, `rand`, `rand32`, `rand64`, `randCanonical`, `generateUUIDv4`, `generateUUIDv7` and `generateULID` when their optional ignored argument has type `Nothing`: they declared `Nothing` rather than their documented type. `randConstant` with more than one argument also raised `LOGICAL_ERROR` during query analysis instead of reporting the wrong argument count. [#116072](https://github.com/ClickHouse/ClickHouse/pull/116072) ([Groene AI](https://github.com/groeneai)). * `KILL QUERY` and `max_execution_time` (including `timeout_overflow_mode = 'break'`) now stop AI functions in the middle of a block, instead of only after all requests for the block have finished. [#125698](https://github.com/ClickHouse/ClickHouse/pull/125698) ([George Larionov](https://github.com/george-larionov)). * Fixed the exception `Provided time zone must be non-empty` when the time zone argument of `formatDateTime`, `formatDateTimeInJodaSyntax`, `fromUnixTimestamp`, or `fromUnixTimestampInJodaSyntax` is a `LowCardinality` column. [#117006](https://github.com/ClickHouse/ClickHouse/pull/117006) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `toWeek` and `toYearWeek` with mode `8` returning week `135` for `Date32` and `DateTime64` values in the year 1900 and earlier. The wrong values also made the primary key analysis drop matching rows for predicates on `toYearWeek(key, 8)`. [#117694](https://github.com/ClickHouse/ClickHouse/pull/117694) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * An out-of-range integer constant in an `IN` set for a `DateTime64` or `Time64` column no longer wraps to a negative timestamp and matches an unrelated row. [#118656](https://github.com/ClickHouse/ClickHouse/pull/118656) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `makeDate` and `makeDate32` with two arguments to accept day 366 in leap years. [#119363](https://github.com/ClickHouse/ClickHouse/pull/119363) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `CAST` of an empty container to a container of `Variant` or `Dynamic`, which was rejected with `Cannot convert type Nothing to Variant(...)` or `Variant type should have at least one nested type`. This also fixes a query returning an empty `Array(Variant(...))` failing once part of it runs on a secondary server, where the folded constant is re-rendered as a `_CAST` of an empty container. [#118038](https://github.com/ClickHouse/ClickHouse/pull/118038) ([Groene AI](https://github.com/groeneai)). * Fixed `TYPE_MISMATCH` (`CAST AS Array can only be performed between same-dimensional array types`) when casting an `Array(Map(...))` to an array whose element type contains the `Map` inside a `Variant`, or to `Array(Dynamic)`, `Array(JSON)`, or `Array(String)`. [#120948](https://github.com/ClickHouse/ClickHouse/pull/120948) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `arraySlice` and `arrayElement` on a `QBit` column replicated by `ARRAY JOIN` or a `JOIN`: debug builds raised `LOGICAL_ERROR`, while release builds could return wrong results. [#120224](https://github.com/ClickHouse/ClickHouse/pull/120224) ([Groene AI](https://github.com/groeneai)). * Fixed a function over a `Dynamic` column reporting `LOGICAL_ERROR` when an unrelated exception, such as `MEMORY_LIMIT_EXCEEDED`, was raised while converting the nested result to the declared result type. Such exceptions now keep their own error code, message, and stack instead of being reported as an internal error. [#121280](https://github.com/ClickHouse/ClickHouse/pull/121280) ([Groene AI](https://github.com/groeneai)). * The `JSON` type and the `JSON*` functions now also accept `String` values holding bytes that are not valid UTF-8 on builds where simdjson uses its scalar `fallback` backend (`s390x`, `riscv64` and `loongarch64` without RVV / LSX, `amd64compat` on CPUs without SSE4.2 or PCLMULQDQ), which previously failed with `INCORRECT_DATA`. [#123945](https://github.com/ClickHouse/ClickHouse/pull/123945) ([Groene AI](https://github.com/groeneai)). * Fixed an exception when a function applied to a `Dynamic` column returns a result of the `Nothing` type, for example `arrayElement` on an empty array literal. Such queries now return `NULL` for every row instead of failing with `std::out_of_range` or `Variant type should have at least one nested type`. [#115982](https://github.com/ClickHouse/ClickHouse/pull/115982) ([Groene AI](https://github.com/groeneai)). * Fixed `CAST AS QBit can only be performed from String, Array or another QBit` (and `Unsupported types for IN`) errors when a query with a `QBit` constant, such as `WHERE q = CAST([1, 2], 'QBit(Float32, 2)')` or `q IN (...)`, runs over `remote`, a `Distributed` table, or parallel replicas. [#125038](https://github.com/ClickHouse/ClickHouse/pull/125038) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR: Column identifier ... is already registered` when `analyzer_compatibility_join_using_top_level_identifier = 1` and a `JOIN ... USING` key resolves to an `ALIAS` column of a table function such as `merge` or `remote`. `NATURAL JOIN` now also matches `ALIAS` columns of a table function, as it already did for a table, and no longer takes a non-selectable `EPHEMERAL` column as a join key on either side; with no other shared name it behaves as a `CROSS JOIN` rather than failing with `Code: 16`. [#118734](https://github.com/ClickHouse/ClickHouse/pull/118734) ([Groene AI](https://github.com/groeneai)). * With `analyzer_compatibility_join_using_top_level_identifier = 1`, a `JOIN USING` identifier of the outermost `JOIN` is now also resolved from an alias defined in the `WITH` clause or in another clause of the query, not only from the `SELECT` list, and duplicated aliases with identical definitions no longer prevent the resolution; an inner `JOIN` of a query with several joins resolves it from its left table, as the old analyzer did. This fixes `UNKNOWN_IDENTIFIER` for such queries and a `USING clause expected identifier` error when an inner `USING` join is placed below an aliased outermost `USING` key. Closes [#121401](https://github.com/ClickHouse/ClickHouse/issues/121401). [#121975](https://github.com/ClickHouse/ClickHouse/pull/121975) ([Vladimir Cherkasov](https://github.com/vdimir)). [#123103](https://github.com/ClickHouse/ClickHouse/pull/123103) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a null dereference while formatting the `LOGICAL_ERROR` for a `USING` column with no expression; the query now reports the column name instead. Closes [#123292](https://github.com/ClickHouse/ClickHouse/issues/123292). [#123296](https://github.com/ClickHouse/ClickHouse/pull/123296) ([Raúl Marín](https://github.com/Algunenano)). * Fix the AST predicate push-down into distributed subqueries treating non-deterministic `EXECUTABLE` and `WASM` user-defined functions as deterministic. It pushed a predicate over such a function's result to the shard, which evaluated the function again. [#119865](https://github.com/ClickHouse/ClickHouse/pull/119865) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` (`Initiator received more initial requests than there are replicas`, or `Duplicate announcement received for replica number N`) when a `mergeTreeAnalyzeIndexes` predicate contained a correlated subquery and parallel replicas were enabled. [#119860](https://github.com/ClickHouse/ClickHouse/pull/119860) ([Groene AI](https://github.com/groeneai)). * Parallel replicas are kept enabled for cluster engines when `automatic_parallel_replicas_mode` is set. Also, from now on, with `enable_parallel_replicas = 2` parallel replicas are forced and the automatic mode is not applied. [#122306](https://github.com/ClickHouse/ClickHouse/pull/122306) ([Nikita Taranov](https://github.com/nickitat)). * Fixed `NO_SUCH_DATA_PART` in a `make_distributed_plan` bucketed read when a worker's own index analysis prunes a part that the coordinator selected, for example with `use_index_for_in_with_subqueries = 0` and an `IN (subquery)` predicate. Such parts are now skipped instead of producing a false "part ... selected by the coordinator is not available on this replica" error. Added `DistributedPlanWorkerPartsReceived`, `DistributedPlanWorkerPartsScanned`, and `DistributedPlanWorkerPartsPruned` profile events. [#115514](https://github.com/ClickHouse/ClickHouse/pull/115514) ([Groene AI](https://github.com/groeneai)). * A serialized query plan carrying `max_block_size = 0` is now rejected when it is read, instead of constructing steps with a zero block size that could make the receiving server hang. [#120606](https://github.com/ClickHouse/ClickHouse/pull/120606) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed a case where `make_distributed_plan` fell back to local execution for the outer query but still built inner `IN` subquery sets as distributed plans inside views, scalar subqueries, `GLOBAL IN` subqueries, `Distributed` shard plans, materialized CTEs, and `TimeSeries` reads. [#119986](https://github.com/ClickHouse/ClickHouse/pull/119986) ([Antonio Filipovic](https://github.com/antoniofilipovic)). * `make_distributed_plan` queries that resolve dictionary functions, `joinGet`, or `MergeTree` column defaults only on the initiator now fall back to local execution, avoiding worker-side `Dictionary (...) not found` failures and similar unresolved-reference errors. [#121198](https://github.com/ClickHouse/ClickHouse/pull/121198) ([Antonio Filipovic](https://github.com/antoniofilipovic)). [#121567](https://github.com/ClickHouse/ClickHouse/pull/121567) ([Antonio Filipovic](https://github.com/antoniofilipovic)). * A cancelled or failed distributed query with `make_distributed_plan = 1` no longer waits for the handshake timeout of a streaming exchange whose producer does not answer. Closes [#122395](https://github.com/ClickHouse/ClickHouse/issues/122395). [#122422](https://github.com/ClickHouse/ClickHouse/pull/122422) ([Groene AI](https://github.com/groeneai)). * Fixed distributed query plans losing a `LIMIT` early-stop signal on an in-memory exchange when a port closed concurrently. Such queries could keep reading whole inputs or wait for exchange streams after the `LIMIT` was already satisfied. [#122453](https://github.com/ClickHouse/ClickHouse/pull/122453) ([Groene AI](https://github.com/groeneai)). * Fixed the `LOGICAL_ERROR` `Reading from materialized CTE '...' before its materialization completed - DelayedPortsProcessor gate is missing in the query plan` on a table read with more than one stream, raised by a lightweight `UPDATE` or `DELETE` whose predicate passes an `IN` subquery that defines a `MATERIALIZED` CTE to a function such as `ignore`, and by a lightweight `UPDATE`/`DELETE` or a `SELECT` when an `IN` or other set subquery defines a `MATERIALIZED` CTE that two other `MATERIALIZED` CTEs of that subquery read. [#119715](https://github.com/ClickHouse/ClickHouse/pull/119715) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#121010](https://github.com/ClickHouse/ClickHouse/pull/121010) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when a predicate on a `Merge` table's `_table` or `_database` virtual column is wrapped in `indexHint`. [#117612](https://github.com/ClickHouse/ClickHouse/pull/117612) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix ``Column `__array_join_exp_1` not found in table`` for `ARRAY JOIN` with a filter on the array-joined column over a `Merge` table whose child is a `Distributed` table. [#119307](https://github.com/ClickHouse/ClickHouse/pull/119307) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` `Unexpected return type` when reading a `Merge` table whose underlying table is another `Merge` table with different column types, with `optimize_prewhere_after_pushdown = 1` and `move_all_conditions_to_prewhere = 0`. [#124037](https://github.com/ClickHouse/ClickHouse/pull/124037) ([Groene AI](https://github.com/groeneai)). * Fixed `TRUNCATE TABLE` failing with `DEADLOCK_AVOIDED`, leaving the data in place, while the table is being read concurrently, for a `MergeTree` table in a database with `lazy_load_tables = 1` and for an `Alias` table whose target is a `MergeTree` table. [#119105](https://github.com/ClickHouse/ClickHouse/pull/119105) ([Groene AI](https://github.com/groeneai)). [#121703](https://github.com/ClickHouse/ClickHouse/pull/121703) ([Groene AI](https://github.com/groeneai)). [#113634](https://github.com/ClickHouse/ClickHouse/pull/113634) ([Groene AI](https://github.com/groeneai)). * `TRUNCATE ALL TABLES FROM` and `TRUNCATE TABLES FROM ... LIKE ...` now skip `Alias` tables, which hold no data of their own. Previously such a statement could fail with `LOGICAL_ERROR`, truncate a target in another database, or fail when the alias target was missing. [#121134](https://github.com/ClickHouse/ClickHouse/pull/121134) ([Groene AI](https://github.com/groeneai)). * Fix `SHOW CREATE TABLE`, `system.tables.create_table_query`, and metadata files of `Alias` tables including a column list, which is invalid for `ENGINE = Alias` and goes stale when the target table changes. [#115141](https://github.com/ClickHouse/ClickHouse/pull/115141) ([Enric Calabuig](https://github.com/eclbg)). * Fixed a logical error when a non-`APPEND` refreshable materialized view was created in a non-`Atomic`/`Replicated` database via a full-definition `ATTACH`, or via a `RESTORE` of a view that had a UUID into such a database. `CREATE` already rejected this combination; `ATTACH` and `RESTORE` bypassed the check and produced a broken view with no UUID. Backups of views that never had a UUID stay restorable, and refresh now fails cleanly instead of aborting in debug and sanitizer builds. [#111928](https://github.com/ClickHouse/ClickHouse/pull/111928) ([Groene AI](https://github.com/groeneai)). * Fixed `SELECT count()` never terminating, or returning an invented row count, on an `Avro` file with a corrupted block header. The bundled `ClickHouse/avro` now rejects a block row count larger than its payload can hold wherever it can derive a lower bound for the schema, so `SELECT count()` over `snappy`-compressed `Avro` files still uses the block header row count instead of decoding every row. [#116156](https://github.com/ClickHouse/ClickHouse/pull/116156) ([Groene AI](https://github.com/groeneai)). [#121211](https://github.com/ClickHouse/ClickHouse/pull/121211) ([Groene AI](https://github.com/groeneai)). * Fix `rows_read` and `rows_before_limit_at_least` in the statistics of the `JSON` and `XML` output formats being reported as 0 or too small when a query with `LIMIT` reads from remote servers (parallel replicas or the `remote` table function), because the output format could be finalized before the remaining progress was drained from the connections. Also fix a query broken off by `timeout_overflow_mode = 'break'` answering with a truncated response - or with no response body at all - instead of a complete document with the statistics, and, over the native protocol, dropping the rows that had already been produced when the time limit was checked. [#118000](https://github.com/ClickHouse/ClickHouse/pull/118000) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CANNOT_SEEK_THROUGH_FILE` when reading a subset of columns from an `ArrowStream` through a pipe, for example with `clickhouse local` reading stdin. [#125689](https://github.com/ClickHouse/ClickHouse/pull/125689) ([Mishalski](https://github.com/wywywywycloud)). * Fixed `FileLog` on macOS re-reading a file from offset `0` when it is renamed during a directory rescan. [#122448](https://github.com/ClickHouse/ClickHouse/pull/122448) ([Groene AI](https://github.com/groeneai)). * Fixed `FileLog` silently skipping the first rows of a file moved into the watched directory from elsewhere (for example a staging file renamed into place) when that file got the inode of a file just deleted from the directory. Such a file is now read from the beginning. [#123593](https://github.com/ClickHouse/ClickHouse/pull/123593) ([Groene AI](https://github.com/groeneai)). * A `FileLog` file that cannot be opened (removed, a symlink whose target was removed, not readable by the server) no longer stops the table: the other files keep being consumed, and it is skipped, logged once and retried until it can be opened while the table is loaded, unless the directory reports that it was removed, renamed or replaced. With the default `handle_error_mode`, a record that cannot be parsed is now logged and skipped while the table streams into materialized views, instead of failing the whole insert together with the rows read from the other files; a direct `SELECT` still throws. [#123595](https://github.com/ClickHouse/ClickHouse/pull/123595) ([Groene AI](https://github.com/groeneai)). * Fixed the warning about a setting duplicated in the definition of an `S3Queue` or `AzureQueue` table, which said that the first value is used while the table uses the last one. [#124916](https://github.com/ClickHouse/ClickHouse/pull/124916) ([Vadim Lopatin](https://github.com/buggins)). * Fix the `KafkaConsumersWithAssignment` metric drifting negative: it was decremented twice for every consumer-group rebalance. [#113919](https://github.com/ClickHouse/ClickHouse/pull/113919) ([kalavt](https://github.com/kalavt)). * Fixed the histogram metric `keeper_read_wait_for_write_time_milliseconds` and the OpenTelemetry span `keeper.read.wait_for_write` always reading zero and never being emitted when `use_new_dispatcher` is enabled, which is the default. [#115252](https://github.com/ClickHouse/ClickHouse/pull/115252) ([Groene AI](https://github.com/groeneai)). * `TestKeeper` no longer treats nodes that only share a name prefix as descendants: `removeRecursive` of `/q` keeps `/q2`, and `listRecursive` and `listWithOptions` of `/q` return its children when a sibling such as `/q-x` exists. [#122769](https://github.com/ClickHouse/ClickHouse/pull/122769) ([Groene AI](https://github.com/groeneai)). * Fixed Keeper `ProfileEvents` `KeeperPacketsSent`, `KeeperPacketsReceived`, `KeeperRequestTotal`, `KeeperRequestTotalWithSubrequests`, and `KeeperLatency` being counted twice. After upgrading, these counters grow at half the previous rate. [#122811](https://github.com/ClickHouse/ClickHouse/pull/122811) ([myeongjun](https://github.com/myeongjjun)). * Fix unbounded recursion when decoding a deeply nested value inside a binary-encoded type, for example an `AggregateFunction` parameter in a `Dynamic` value read from `RowBinary` or `Native`. [#122667](https://github.com/ClickHouse/ClickHouse/pull/122667) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed `SHOW COLUMNS` failing with `Cannot parse string 'Decimal' as Int8` (`CANNOT_PARSE_TEXT`) for `Nullable(Decimal(P, S))` columns when the connection is made through the MySQL wire protocol. [#75694](https://github.com/ClickHouse/ClickHouse/pull/75694) ([ultrabig](https://github.com/ultrabig)). * Fix a logical error `Cannot write to canceled buffer` in the PostgreSQL wire protocol handler when a client disconnects while the server is still sending the result of a query. [#118369](https://github.com/ClickHouse/ClickHouse/pull/118369) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a signed overflow in the workload throttler that made `system.scheduler.throttling_us` report a negative value for a workload whose `max_bytes_per_second`, `max_queries_per_second`, or `max_cpus` is small enough to postpone a request for longer than a year. Such a postponement is now saturated at one year. [#113697](https://github.com/ClickHouse/ClickHouse/pull/113697) ([Groene AI](https://github.com/groeneai)). * Fixed the `throughput` column of `system.scheduler` reporting a small fraction of the real value (about 0.05x initially and 0.3x after 4 seconds) for about 20 seconds after a scheduling node is created, for example right after server start or `CREATE WORKLOAD`. The client's CPU usage indicator, which uses the same `EventRateMeter`, was affected the same way on its first samples. Closes [#121962](https://github.com/ClickHouse/ClickHouse/issues/121962). [#122006](https://github.com/ClickHouse/ClickHouse/pull/122006) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `Cannot add step Limit to QueryPlan because it has incompatible header with root step Sorting` when a vector search query with `vector_search_with_rescoring = 0` was executed with the experimental `make_distributed_plan` setting. [#114457](https://github.com/ClickHouse/ClickHouse/pull/114457) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `assignCentroid(vec, 'dict')` with an unqualified dictionary name over a `Distributed` table, `cluster`/`remote`, or parallel replicas. Remote shards and replicas resolved the name in their own `default` database, so the query failed with `Dictionary (dict) not found` or silently used a dictionary with the same name from `default`. The name is now resolved in the current database of the initiator, as for `dictGet`. [#123711](https://github.com/ClickHouse/ClickHouse/pull/123711) ([Groene AI](https://github.com/groeneai)). * Fixed `UNDROP TABLE` failing with `LOGICAL_ERROR` (`Failed to get metadata path from database`) instead of `UNKNOWN_TABLE` when the target database engine stores no table metadata on disk, for example a `Memory` database that reused the name of a dropped `Atomic` one, or the default database of `clickhouse-local`. [#120945](https://github.com/ClickHouse/ClickHouse/pull/120945) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER TABLE ... MODIFY SETTING name = DEFAULT` skipping the checks an engine makes on a setting reset. [#119475](https://github.com/ClickHouse/ClickHouse/pull/119475) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `TRUNCATE ALL TABLES FROM ` failing with `NOT_IMPLEMENTED` for the whole database when it held a table whose engine stores no rows of its own, such as `Merge`, `Null`, `GenerateRandom`, `Executable`, `ExecutablePool`, `FuzzQuery`, `FuzzJSON`, or `URL`. Such tables are now skipped, as views and dictionaries already were. [#121248](https://github.com/ClickHouse/ClickHouse/pull/121248) ([Groene AI](https://github.com/groeneai)). * A table created without a column list over a source that carries a per-column `TTL` no longer inherits that `TTL` when its own engine does not support one, such as `Merge`, `Buffer`, `Distributed`, or `Remote`. [#121335](https://github.com/ClickHouse/ClickHouse/pull/121335) ([Groene AI](https://github.com/groeneai)). * Fixed the logical error `Cannot find column in ColumnsDescription` when a single `ALTER` statement on a materialized view applies `COMMENT COLUMN` or `MODIFY COLUMN ... COMMENT` after a `MODIFY QUERY` whose new query no longer produces that column. Such a statement now reports the missing column instead of aborting in debug and sanitizer builds. [#121469](https://github.com/ClickHouse/ClickHouse/pull/121469) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` `Cannot capture column N because it has incompatible type` when a materialized view with a lambda over an `Array(Tuple(...))` column receives a block while the source table's column type is being changed by `ALTER TABLE ... MODIFY COLUMN`. [#122345](https://github.com/ClickHouse/ClickHouse/pull/122345) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#123514](https://github.com/ClickHouse/ClickHouse/pull/123514) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#124740](https://github.com/ClickHouse/ClickHouse/pull/124740) ([Groene AI](https://github.com/groeneai)). * In a `Memory` database, the name of a detached table, view or dictionary is now reserved, as in `Atomic` databases. A statement that would create a new object under it (`CREATE`, `CREATE OR REPLACE [MATERIALIZED] VIEW`, or `ATTACH` with a full definition) fails with `TABLE_ALREADY_EXISTS`, `IF NOT EXISTS` makes it a no-op, and a short `ATTACH` restores the detached object. Previously these statements were accepted, but `SHOW CREATE TABLE` and the table dependencies kept describing the detached object, and a later `ALTER` of such a view failed with a `LOGICAL_ERROR` exception. [#122670](https://github.com/ClickHouse/ClickHouse/pull/122670) ([Groene AI](https://github.com/groeneai)). * Fixed a window definition that places a frame before the parent window name, as in `WINDOW x AS (ROWS UNBOUNDED PRECEDING w)`. That order is now a syntax error, while a parent window named after a frame keyword, as in `WINDOW x AS (rows)`, now parses. [#120156](https://github.com/ClickHouse/ClickHouse/pull/120156) ([Groene AI](https://github.com/groeneai)). * Formatting `EXPLAIN AST () FORMAT ...` or `... SETTINGS ...` now keeps the parentheses for `INSERT`, `DELETE`, `CREATE`, a nested non-`SELECT` `EXPLAIN`, other non-`SELECT` queries, and a `SELECT` or `EXPLAIN` with output options of its own, and `EXPLAIN SYNTAX SYSTEM ... FORMAT ...` is no longer formatted with parentheses that do not parse, so the formatted query parses back with the same output options. [#123189](https://github.com/ClickHouse/ClickHouse/pull/123189) ([Groene AI](https://github.com/groeneai)). * Fixed formatting of PostgreSQL `COPY` queries so table names, columns, direction, format, and header options are preserved. [#123601](https://github.com/ClickHouse/ClickHouse/pull/123601) ([Minh Vu](https://github.com/fallintoplace)). * Keep empty `LIKE` and `ILIKE` patterns in `SHOW` statements, including `SHOW FUNCTIONS`. [#123624](https://github.com/ClickHouse/ClickHouse/pull/123624) ([Minh Vu](https://github.com/fallintoplace)). [#123626](https://github.com/ClickHouse/ClickHouse/pull/123626) ([Minh Vu](https://github.com/fallintoplace)). * A one-argument `tuple` in the `ORDER BY` of a projection, such as `ORDER BY tuple(c0)`, is now kept when the definition is formatted, so the formatted query parses back to the same definition. [#123162](https://github.com/ClickHouse/ClickHouse/pull/123162) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed formatting of the tuple element access operator when its index carries an alias. `SELECT ((1,2) AS t).(1 AS a)` was formatted as `SELECT ((1, 2) AS t).(1) AS a`, which re-parses with the alias attached to the whole `tupleElement` instead of to the index. This changed the column names of a query passed through `formatQuery` or sent to a remote shard, and raised `Inconsistent AST formatting` on builds with assertions enabled. [#120217](https://github.com/ClickHouse/ClickHouse/pull/120217) ([Groene AI](https://github.com/groeneai)). * Fixed `formatQuery` and `formatQueryFromJSON` dropping `FULL` from `SHOW FULL DATABASES` and the other supported `SHOW FULL` forms. `SHOW FULL DICTIONARIES` no longer selects a non-existent `engine` column, and builds with assertions enabled no longer fail with `Inconsistent AST formatting`. Closes [#123967](https://github.com/ClickHouse/ClickHouse/issues/123967). [#124036](https://github.com/ClickHouse/ClickHouse/pull/124036) ([Groene AI](https://github.com/groeneai)). * Fixed a crash and a `LOGICAL_ERROR` when a `lambda` function reached query analysis without the `lambda(tuple(...), body)` shape the SQL parser always produces. An AST deserialized from JSON (`dialect = clickhouse_json`) can no longer mark an arbitrary function as a lambda, and `CREATE FUNCTION f AS g(tuple(x), x)` is now rejected with `BAD_ARGUMENTS` instead of failing when `f` is called. Closes [#120380](https://github.com/ClickHouse/ClickHouse/issues/120380). [#120483](https://github.com/ClickHouse/ClickHouse/pull/120483) ([Groene AI](https://github.com/groeneai)). * Fixed server termination on `CREATE TABLE`/`ALTER TABLE` when a constraint expression restored through the `clickhouse_json` dialect held a function without an `arguments` list. Such a payload now fails with `BAD_ARGUMENTS` while the AST is deserialized, so shapes that used to be accepted and fail later at `INSERT` (for example `notEquals` with no argument list) are rejected at parse time inside that dialect. [#120507](https://github.com/ClickHouse/ClickHouse/pull/120507) ([Groene AI](https://github.com/groeneai)). * Fixed `formatQueryFromJSON` accepting enumerator names that name no value (an enumeration bound, or an unset default), such as `{"type":"SystemQuery","query_type":"UNKNOWN"}`, which raised `LOGICAL_ERROR: Unknown SYSTEM command` instead of reporting bad input. Such names are now rejected with `BAD_ARGUMENTS`. Closes [#120741](https://github.com/ClickHouse/ClickHouse/issues/120741). [#120886](https://github.com/ClickHouse/ClickHouse/pull/120886) ([Groene AI](https://github.com/groeneai)). * Fixed two segfaults in `formatQueryFromJSON` when a node omits a child that formatting dereferences unconditionally. Closes [#120739](https://github.com/ClickHouse/ClickHouse/issues/120739). [#120876](https://github.com/ClickHouse/ClickHouse/pull/120876) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash when a `clickhouse_json` payload restores an expression holding a function with no `arguments` list, a shape the SQL parser cannot produce. Such a payload is now rejected with `BAD_ARGUMENTS`, as is an empty `TTL` list, which was persisted as metadata that no longer loaded. [#120882](https://github.com/ClickHouse/ClickHouse/pull/120882) ([Groene AI](https://github.com/groeneai)). * Fixed `formatQueryFromJSON` raising `LOGICAL_ERROR: Unknown element in AST: Foreign Key` for a payload of type `ForeignKeyDeclaration`. That node type has no formatter and no query can produce one, so it is now rejected with `BAD_ARGUMENTS`. Closes [#120738](https://github.com/ClickHouse/ClickHouse/issues/120738). [#120915](https://github.com/ClickHouse/ClickHouse/pull/120915) ([Groene AI](https://github.com/groeneai)). * Fixed stack exhaustion when deserializing an AST from JSON through `formatQueryFromJSON` or the `clickhouse_json` dialect. Such payloads now fail with `TOO_DEEP_RECURSION`. [#121693](https://github.com/ClickHouse/ClickHouse/pull/121693) ([Groene AI](https://github.com/groeneai)). * Fixed a silently corrupted query when a node restored from the AST JSON format (`formatQueryFromJSON`, `dialect = 'clickhouse_json'`) carried a shape the parser cannot produce, such as a bare select query argument or an expression list where an argument was expected. Such a node is now rejected with `BAD_ARGUMENTS` instead of formatting into SQL that parses back as a different query. [#120725](https://github.com/ClickHouse/ClickHouse/pull/120725) ([Groene AI](https://github.com/groeneai)). * Fixed stack exhaustion in `parseQueryToJSON` when serializing a query nested deeper than the serializer could follow, reachable by raising `max_parser_depth` above its default. Such a query now fails with `TOO_DEEP_RECURSION` instead of terminating the server. [#121856](https://github.com/ClickHouse/ClickHouse/pull/121856) ([Groene AI](https://github.com/groeneai)). * A `CREATE DICTIONARY` sent in the AST JSON format (`dialect = 'clickhouse_json'`) is now rejected with `BAD_ARGUMENTS` when the brackets of a key-value argument do not match its value, a bracketed list holds something other than key-value pairs, the `SOURCE` has no brackets, the primary key is empty or not a list of identifiers, a `DEFAULT` is not a literal, or the attributes or the definition are missing or empty, and so is any other query that carries them. Such queries could crash the server or prevent it from starting. [#122981](https://github.com/ClickHouse/ClickHouse/pull/122981) ([Groene AI](https://github.com/groeneai)). * Fix a server crash (SIGSEGV) when a `CREATE TABLE ... PROJECTION` / `ALTER TABLE ... ADD PROJECTION` statement is restored from a `clickhouse_json` AST whose projection `query` subtree contains a function node without an `arguments` list. The malformed payload is now rejected with `BAD_ARGUMENTS` during AST JSON deserialization. [#121113](https://github.com/ClickHouse/ClickHouse/pull/121113) ([Ilias](https://github.com/iliasabk)). * A `clickhouse_json` payload that restores a function without an `arguments` list into an expression slot read by the pre-analyzer stack is now rejected with `BAD_ARGUMENTS` while it is being deserialized. Previously such a payload was accepted and could reach a visitor that dereferences the missing list, terminating the server. [#121119](https://github.com/ClickHouse/ClickHouse/pull/121119) ([Groene AI](https://github.com/groeneai)). * Fixed `parseSize` and `parseTime` logic to correctly distinguish between size thresholds and time-based rotation schedules such as `12:00`. Fixes [#103108](https://github.com/ClickHouse/ClickHouse/issues/103108) and [#100910](https://github.com/ClickHouse/ClickHouse/issues/100910). [#103127](https://github.com/ClickHouse/ClickHouse/pull/103127) ([Jianmei Zhang](https://github.com/zhangjmruc)). * A configuration file can be written in XML or in YAML, but only `config.xml` was looked up as the default configuration file. Now `clickhouse-server`, `clickhouse-keeper`, `clickhouse-local`, `clickhouse-keeper-client` and `clickhouse-disks` also find `config.yaml` and `config.yml` (`keeper_config.yaml` and `keeper_config.yml` for Keeper). Previously, `clickhouse-server` silently started with the configuration embedded into the binary when only a `config.yaml` was present. [#118422](https://github.com/ClickHouse/ClickHouse/pull/118422) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse install` (and therefore `apt`/`dpkg` package upgrades) no longer prompts interactively to set up a password for the `default` user when that user has been explicitly removed from configuration via `` in `users.d`. Previously the installer only checked whether the `default` user already had a password configured, missing the case where the user itself does not exist in the merged configuration at all, so it kept asking for a password on every upgrade even though there was no user to set it for. [#110438](https://github.com/ClickHouse/ClickHouse/pull/110438) ([Eduardo Gómez](https://github.com/edugomz)). * Fixed the per-query `MemoryAllocatedWithoutCheck` and `MemoryAllocatedWithoutCheckBytes` values in `system.query_log`. A thread's untracked memory balance is flushed when the thread is attached to a query, and that flush was reported as the query's, so memory allocated before the query existed was charged to it. [#116505](https://github.com/ClickHouse/ClickHouse/pull/116505) ([Groene AI](https://github.com/groeneai)). * Fixed `AsynchronousInsertQueueSize` and `AsynchronousInsertQueueBytes` in `system.metrics` growing permanently after `SYSTEM FLUSH ASYNC INSERT QUEUE`. [#120215](https://github.com/ClickHouse/ClickHouse/pull/120215) ([Sema Checherinda](https://github.com/CheSema)). * Fixed a trailing NUL byte in the `notation` column of `system.unicode`, which made `length` return a wrong value and comparisons such as `notation = 'U+0041'` never match. [#122119](https://github.com/ClickHouse/ClickHouse/pull/122119) ([David Meng](https://github.com/davidmenggx)). * Fixed `CGroup*` asynchronous metrics being silently missing on cgroup v2 systems that run ClickHouse in a nested cgroup, such as a typical `systemd` service. Closes [#103968](https://github.com/ClickHouse/ClickHouse/issues/103968). [#122918](https://github.com/ClickHouse/ClickHouse/pull/122918) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `NO_COMMON_TYPE` for `use_variant_as_common_type = 0` in the jemalloc fragmentation dashboard. [#121621](https://github.com/ClickHouse/ClickHouse/pull/121621) ([Azat Khuzhin](https://github.com/azat)). * Fixed `TYPE_MISMATCH` being counted in `system.errors` on every execution of a `SELECT` comparing a `DateTime64` column against a `DateTime`, `Date` or `Date32` constant inside `AND`. A date or time constant with a part the column type cannot hold is no longer pushed down to MongoDB as a truncated bound. [#121078](https://github.com/ClickHouse/ClickHouse/pull/121078) ([Navneet Kumar](https://github.com/navneet83)). * Fixed reading `system.parts`, `system.parts_columns` and `system.projection_parts` for a table whose `PARTITION BY` contains a `modulo` with an unsigned left and a signed right operand: the query aborted with `Bad get: has Int128, requested UInt128 (BAD_GET)` when the partition value was wider than 8 bytes, and otherwise showed a reinterpreted value that `ALTER TABLE ... DROP PARTITION` rejects. The same value is rendered while a merge or mutation is scheduled, while a `system.part_log` row is written, and while a partition is pruned, so on such a table merges failed to start, `part_log` rows were lost, and a filtered read could abort during planning. [#119333](https://github.com/ClickHouse/ClickHouse/pull/119333) ([Groene AI](https://github.com/groeneai)). * Fix `system.server_settings` reporting the async logger queue size under the misspelled key `logger.async_queye_max_size` instead of `logger.async_queue_max_size`. [#123578](https://github.com/ClickHouse/ClickHouse/pull/123578) ([Raúl Marín](https://github.com/Algunenano)). * Fix `SELECT` from `system.stack_trace` failing with `Cannot read from file /proc//status ... No such process` when a thread exits while the table is being scanned. [#123805](https://github.com/ClickHouse/ClickHouse/pull/123805) ([Groene AI](https://github.com/groeneai)). * Fix `system.jemalloc_sampled_allocations` and the collapsed output of `system.jemalloc_profile_text` on sampled allocations whose backtrace jemalloc could not unwind: the allocation is now reported with `trace = []` or under `[unknown]` instead of failing with `CANNOT_PARSE_TEXT` or being omitted. [#123818](https://github.com/ClickHouse/ClickHouse/pull/123818) ([Groene AI](https://github.com/groeneai)). * Fix per-protocol asynchronous metrics such as `TCPThreads` and `HTTPThreads` not being reported for listeners configured through the `` section. Such a listener is now reported under the same metric as the equivalent built-in port. Metrics are also now reported for the `tcp_with_proxy` and `tcp_ssh` listeners, which previously had none. [#112977](https://github.com/ClickHouse/ClickHouse/pull/112977) ([Chester](https://github.com/superdiaodiao)). * Fix a deadlock when an obsolete union system log table is replaced with synchronous drops. [#124610](https://github.com/ClickHouse/ClickHouse/pull/124610) ([Azat Khuzhin](https://github.com/azat)). * Fixed the `max_local_read_bandwidth_for_server`, `max_local_write_bandwidth_for_server`, `max_remote_read_network_bandwidth_for_server` and `max_remote_write_network_bandwidth_for_server` rows of `system.server_settings` reporting the reading session's own bandwidth limit instead of the server-wide one. `getServerSetting` on those names is fixed the same way. [#119556](https://github.com/ClickHouse/ClickHouse/pull/119556) ([Groene AI](https://github.com/groeneai)). * Fix tables of the `clickhouse-local` default database becoming invisible (`UNKNOWN_TABLE`) when the same `--path` is later used with another `default_database`. [#123124](https://github.com/ClickHouse/ClickHouse/pull/123124) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed missing crash reports of `clickhouse-local` when neither `stdin` nor `stderr` is a terminal: a fatal signal printed nothing, and `--client_logs_file` was not written. [#115102](https://github.com/ClickHouse/ClickHouse/pull/115102) ([Groene AI](https://github.com/groeneai)). * Fixed `clickhouse-local` hanging forever on `DROP TABLE|VIEW|DATABASE ... SYNC` when `--path` is set and no metadata is loaded from it, for example a new directory on released versions or `--only-system-tables`. [#123862](https://github.com/ClickHouse/ClickHouse/pull/123862) ([Groene AI](https://github.com/groeneai)). * The wasm64 `clickhouse local` now starts with 2 GiB of linear memory, so it no longer fails at startup with `memory access out of bounds` when Node.js races a `memory.grow` across worker threads. Closes [#124067](https://github.com/ClickHouse/ClickHouse/issues/124067). [#123944](https://github.com/ClickHouse/ClickHouse/pull/123944) ([Groene AI](https://github.com/groeneai)). * Fixed `SET profile` having no effect on subsequent queries in `clickhouse-local`. Profile settings such as `readonly` and `join_use_nulls` now apply to subsequent queries again. Closes [#123916](https://github.com/ClickHouse/ClickHouse/issues/123916). [#125171](https://github.com/ClickHouse/ClickHouse/pull/125171) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `promql_evaluation_time` silently rounding the requested instant, which could make an instant query return a different sample. The setting also now accepts an integer Unix timestamp over the native protocol, which it previously refused. [#120942](https://github.com/ClickHouse/ClickHouse/pull/120942) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL queries with regex label matchers (`=~`, `!~`) again work under `--dialect promql`. [#115073](https://github.com/ClickHouse/ClickHouse/pull/115073) ([Valery Petrov](https://github.com/valerypetrov)). * Allow PromQL `# ` and `#!` line comments to end at EOF without a trailing newline. [#113201](https://github.com/ClickHouse/ClickHouse/pull/113201) ([Minh Vu](https://github.com/fallintoplace)). * Fix PromQL duplicate-series detection to check one step at a time, so series that share tags but have values at different steps are now merged instead of failing with `Multiple series have the same tags`. Binary operators check for duplicates after matching series, so unmatched series no longer cause this exception. [#124086](https://github.com/ClickHouse/ClickHouse/pull/124086) ([Vitaly Baranov](https://github.com/vitlibar)). [#124879](https://github.com/ClickHouse/ClickHouse/pull/124879) ([Vitaly Baranov](https://github.com/vitlibar)). * PromQL: fixed expressions over an instant vector that is known to be empty, such as `(0 < clamp(up, 1, -1)) or vector(7)`, which failed in range queries with `Number of values doesn't match number of steps` or returned samples at `1970-01-01`. [#123879](https://github.com/ClickHouse/ClickHouse/pull/123879) ([Valery Petrov](https://github.com/valerypetrov)). * Fixed `SELECT count()` returning the whole-table row count for a streaming read (`FROM ... STREAM`), ignoring the `CURSOR` bound. The implicit min-max/count projection answered the query from part metadata instead of the streaming reader. [#117188](https://github.com/ClickHouse/ClickHouse/pull/117188) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash (`std::terminate` with `write failed`) in the embedded SSH client when the client's terminal was closed before the line reader was destroyed. [#120764](https://github.com/ClickHouse/ClickHouse/pull/120764) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Code: 1001 STD_EXCEPTION` (`filesystem error: ... File name too long`) when a query names a table or dictionary longer than the filesystem's `NAME_MAX` in `clickhouse local`, or in a server holding a `Filesystem` or `URL` database. Such a name is now reported as missing rather than as a filesystem error. Closes [#121304](https://github.com/ClickHouse/ClickHouse/issues/121304). [#121471](https://github.com/ClickHouse/ClickHouse/pull/121471) ([Groene AI](https://github.com/groeneai)). * A malformed argument list of an optimization of the `mergeTreeAnalyzeIndexes` table function is now reported as `BAD_ARGUMENTS` instead of a logical error or a raw `std::exception`. [#118644](https://github.com/ClickHouse/ClickHouse/pull/118644) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `max_execution_time` not being enforced in `clickhouse-local` for a query blocked inside a non-interruptible read. `clickhouse-local` did not start the watchdog that turns a time limit expiry into a query cancellation, so a query reading from an unresponsive `s3` endpoint ran to the end of the S3 retry budget (`s3_retry_attempts` defaults to 500, which takes minutes) instead of failing with `TIMEOUT_EXCEEDED` at its limit. [#120933](https://github.com/ClickHouse/ClickHouse/pull/120933) ([Groene AI](https://github.com/groeneai)). * Fixed distributed queries hanging at 100% CPU on macOS when `max_threads = 1`. [#122905](https://github.com/ClickHouse/ClickHouse/pull/122905) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a server abort on macOS when an HTTP client resets the connection during header reading. [#123074](https://github.com/ClickHouse/ClickHouse/pull/123074) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Unexpected return type from materialize`, a server abort in debug and sanitizer builds) when reading a `Buffer` table whose destination declares a column with a different but convertible type and a forwarded filter references that column: with a row policy and the implicit `WHERE` to `PREWHERE` move, and under `query_plan_optimize_lazy_final`. The same cause made such a read fail with `NOT_FOUND_COLUMN_IN_BLOCK` when a text index on that column was read directly and the predicate appeared in both `PREWHERE` and `WHERE`, fail with `TYPE_MISMATCH` over a `URL(Parquet)` destination, abort with `Bad cast from ColumnMap to ColumnArray` when a row policy and an explicit `PREWHERE` both referenced it, or fail with `AMBIGUOUS_COLUMN_NAME` when several `PREWHERE` conditions referenced the converted column. [#113900](https://github.com/ClickHouse/ClickHouse/pull/113900) ([Groene AI](https://github.com/groeneai)). [#124632](https://github.com/ClickHouse/ClickHouse/pull/124632) ([Groene AI](https://github.com/groeneai)). * Fixed `SYSTEM DROP DNS CACHE` leaving entries in the DNS cache when it ran at the same time as the periodic DNS cache update. [#123699](https://github.com/ClickHouse/ClickHouse/pull/123699) ([Groene AI](https://github.com/groeneai)). * Fix `UNSUPPORTED_METHOD` error for queries using `IN`/`NOT IN`/`GLOBAL IN` with a subquery inside `AND`/`OR` when `enable_function_early_short_circuit` is enabled. [#116037](https://github.com/ClickHouse/ClickHouse/pull/116037) ([fhw12345](https://github.com/fhw12345)). * Fixed a crash caused by a stack overflow when a deeply nested JSON document was parsed into a `JSON` column or passed to the `JSONExtract` family of functions. Such documents are now rejected with the `TOO_DEEP_RECURSION` error. [#120887](https://github.com/ClickHouse/ClickHouse/pull/120887) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed the `LOGICAL_ERROR` `Not-ready Set is passed as the second argument for function 'in'` for a query with `ORDER BY ... WITH FILL ... INTERPOLATE` whose `INTERPOLATE` expression contains `IN` with a subquery or a table that is not used anywhere else in the query. [#124518](https://github.com/ClickHouse/ClickHouse/pull/124518) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of a `WHERE` equality between `Tuple` values with `Nullable` elements, or between `Variant` values, over a comma, `CROSS` or `INNER` join: the pair whose equality is `NULL`, such as `tuple(NULL) = tuple(NULL)`, was returned. With `allow_dynamic_type_in_join_keys = 0`, a comma or `CROSS` join with a `WHERE` equality on a `Dynamic` value no longer returns no rows or fails with `ILLEGAL_COLUMN`. [#124854](https://github.com/ClickHouse/ClickHouse/pull/124854) ([Groene AI](https://github.com/groeneai)). * Fixed secret arguments being exposed in `system.query_log` and the server log when a table function or function was called with the wrong letter case, such as `S3` instead of `s3`, `MySQL` instead of `mysql`, or `ENCRYPT` instead of `encrypt`. Such calls are still rejected. [#124862](https://github.com/ClickHouse/ClickHouse/pull/124862) ([Groene AI](https://github.com/groeneai)). * Fixed tables of a `MySQL` database ignoring the database's `connection_auto_close` setting: with `connection_auto_close = 0`, every read still closed its connection and the next read reconnected. A table attached again with `ATTACH TABLE` also lost the database's settings. `INSERT` into any `MySQL` table ignored the setting and always kept its connection; with the default `connection_auto_close = 1` it now closes it after the query, as reads do. Closes [#124749](https://github.com/ClickHouse/ClickHouse/issues/124749). [#124884](https://github.com/ClickHouse/ClickHouse/pull/124884) ([Groene AI](https://github.com/groeneai)). * Hide the password of a URL in `SHOW CREATE TABLE`, `system.tables` and `system.query_log` only within the URL's authority: a `@` in the query string no longer rewrites the shown host, and a password containing `@` is hidden whole. [#124885](https://github.com/ClickHouse/ClickHouse/pull/124885) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `merge` and `Merge` tables over a `Distributed` or `remote` table rejecting valid queries on a column that some of the tables lack: aggregates over `Decimal` and `Map` columns failed with errors such as `THERE_IS_NO_COLUMN`, and an integer column of this kind in `GROUP BY`, `ORDER BY` or `LIMIT BY` failed with `Positional argument number 0 is out of bounds`. [#124976](https://github.com/ClickHouse/ClickHouse/pull/124976) ([Groene AI](https://github.com/groeneai)). * Fixed trace samples in `system.trace_log` losing their `query_id` while a thread attaches to or detaches from a query, so memory profiling samples remain attributable to the query. [#125176](https://github.com/ClickHouse/ClickHouse/pull/125176) ([Groene AI](https://github.com/groeneai)). * Fixed wrong window function results in the totals row of a `WITH TOTALS` query when an `arrayJoin` expands the totals into several rows: those rows were not sorted by the window's `PARTITION BY` / `ORDER BY` keys. [#125324](https://github.com/ClickHouse/ClickHouse/pull/125324) ([Groene AI](https://github.com/groeneai)). * Fixed primary key and skip index analysis that could skip or over-count rows for conditions on date and time functions such as `toHour`, `toDayOfMonth`, `timezoneOffset` or `toTimeWithFixedDate` over a `Nullable(DateTime)` or `LowCardinality(Nullable(DateTime))` key whose time zone differs from the session time zone. [#125327](https://github.com/ClickHouse/ClickHouse/pull/125327) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results of `ORDER BY`, `DISTINCT`, `LIMIT BY` and aggregation in order over a `Merge` table or the `merge` table function when it reads an `Iceberg` table with a sort order: the rows came back unsorted or duplicated. [#125383](https://github.com/ClickHouse/ClickHouse/pull/125383) ([Groene AI](https://github.com/groeneai)). * Fixed `count()` with a `WHERE` condition on the sorting key returning too many rows on a `MergeTree` table with a normal projection: rows counted from the primary key by the exact count optimization were counted again when the projection was read. Closes [#125183](https://github.com/ClickHouse/ClickHouse/issues/125183). [#125412](https://github.com/ClickHouse/ClickHouse/pull/125412) ([Groene AI](https://github.com/groeneai)). * Fixed a possible segmentation fault when reading an `Avro` `map` into a `Map` column (for example `Map(DateTime, String)`) with the `AvroConfluent` format, including `Kafka` tables. Closes [#125457](https://github.com/ClickHouse/ClickHouse/issues/125457). [#125505](https://github.com/ClickHouse/ClickHouse/pull/125505) ([Groene AI](https://github.com/groeneai)). * Fix a server crash on `CREATE DICTIONARY` without a `SOURCE` clause when `restore_replace_external_dictionary_source_to_null` is enabled. Such a query now fails with `INCORRECT_DICTIONARY_DEFINITION`, as it does with the setting disabled. With the same setting, a local `CLICKHOUSE` source that uses a constant function such as `DB currentDatabase()` is no longer rejected. [#125659](https://github.com/ClickHouse/ClickHouse/pull/125659) ([Groene AI](https://github.com/groeneai)). * Fixed the `Arrow` and `ArrowStream` input formats rejecting valid dictionary-encoded columns whose dictionary contains repeated values. Closes [#125139](https://github.com/ClickHouse/ClickHouse/issues/125139). [#125677](https://github.com/ClickHouse/ClickHouse/pull/125677) ([Jay Pokale](https://github.com/JayPokale)). * Fixed the Arrow Flight interface writing the text of each query, including secrets such as `IDENTIFIED BY` passwords and storage keys, to the server log and `system.text_log`. The gRPC interface did the same for the first 64 characters of each query. Now queries from these interfaces are logged only with their secrets hidden. [#125716](https://github.com/ClickHouse/ClickHouse/pull/125716) ([Groene AI](https://github.com/groeneai)). * Fix a server crash on macOS where an idle thread pool worker aborted with `condition_variable wait failed: Invalid argument`. The `EINVAL` that Darwin's `pthread_cond_wait` returns when the kernel fails the wait is now treated as a spurious wakeup. Closes [#84761](https://github.com/ClickHouse/ClickHouse/issues/84761). [#125721](https://github.com/ClickHouse/ClickHouse/pull/125721) ([Groene AI](https://github.com/groeneai)). #### Build/Testing/Packaging Improvement * Harden official builds by compiling with `-fstack-protector-strong` and `-fstack-clash-protection`, and by linking with `-Wl,-z,separate-code`. [#119039](https://github.com/ClickHouse/ClickHouse/pull/119039) ([Raúl Marín](https://github.com/Algunenano)). [#123073](https://github.com/ClickHouse/ClickHouse/pull/123073) ([Raúl Marín](https://github.com/Algunenano)). * The musl build of ClickHouse is now a static position-independent executable, so it gets address space layout randomization like the glibc build. [#125669](https://github.com/ClickHouse/ClickHouse/pull/125669) ([Raúl Marín](https://github.com/Algunenano)). * Added an optional packaging mode `SYMBOL_SECTIONS=gnu_debugdata` that replaces `.symtab` and `.strtab` in release binaries with compressed `.gnu_debugdata` (MiniDebugInfo), reducing symbol data from about 148 MiB to about 10 MiB in the official build. Stack traces and symbol access from gdb, lldb, elfutils, and perf remain available. [#115322](https://github.com/ClickHouse/ClickHouse/pull/115322) ([Andrew Grigorev](https://github.com/ei-grad)). * Update `cctz` to 2026d. Time zone names with the `file:` prefix are now rejected. [#121325](https://github.com/ClickHouse/ClickHouse/pull/121325) ([Actuele AI](https://github.com/actueleai)). Update `cctz` to 2026e: `America/Winnipeg`, `America/Rainy_River`, and `Canada/Central` now stay at UTC−5 from 2026-11-01, reflecting Manitoba's removal of the DST fall-back. [#124249](https://github.com/ClickHouse/ClickHouse/pull/124249) ([Actuele AI](https://github.com/actueleai)). * Update `OpenSSL` to 3.5.9. [#124278](https://github.com/ClickHouse/ClickHouse/pull/124278) ([Actuele AI](https://github.com/actueleai)). * Update `aws-sdk-cpp` to 1.11.907, the AWS CRT libraries to 0.43.8, and `aws-c-common` to 1.0.2. [#125080](https://github.com/ClickHouse/ClickHouse/pull/125080) ([Groene AI](https://github.com/groeneai)). * The `delta-kernel-rs` library no longer carries its own copy of OpenSSL, so the binary always links the OpenSSL built from `contrib/openssl` with the current build flags instead of an older copy kept by the Rust build cache. [#123889](https://github.com/ClickHouse/ClickHouse/pull/123889) ([Groene AI](https://github.com/groeneai)). * Rust crates are now built in parallel with each other and with the C++ code, which reduces the build time. [#124645](https://github.com/ClickHouse/ClickHouse/pull/124645) ([Antonio Andelic](https://github.com/antonio2368)). * Builds of the experimental GPU engine are faster: `nvcc` compilations are now cached with `ccache`, and unused PTX plus CUDA line info are no longer generated. [#120421](https://github.com/ClickHouse/ClickHouse/pull/120421) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed building with `-DNO_ARMV81_OR_HIGHER=1` on AArch64: the compatibility profile no longer disables `NEON` for Rust, so `ring`-based components (`chdig`, `delta-kernel-rs`) compile and stay enabled. This unblocks building for ARMv8.2-A CPUs that lack `rcpc`, such as HiSilicon Kunpeng 920. [#107440](https://github.com/ClickHouse/ClickHouse/pull/107440) ([Tiaonmmn](https://github.com/Tiaonmmn)). * Compile code linked into the position-independent `clickhouse` executable with `-fPIE` instead of `-fPIC`, avoiding unnecessary symbol indirection. Apply the flag to the C++ runtime libraries from `contrib` too, so builds with compilers that do not default to PIE link successfully, including Fedora's `clang` on `aarch64` and `x86_64`. [#125094](https://github.com/ClickHouse/ClickHouse/pull/125094) ([oleg-milanov](https://github.com/oleg-milanov)). [#125488](https://github.com/ClickHouse/ClickHouse/pull/125488) ([Groene AI](https://github.com/groeneai)). * Allow building with XRay instrumentation under musl. [#118914](https://github.com/ClickHouse/ClickHouse/pull/118914) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed a missing CMake link dependency that could make `src/Functions/array/distanceTransposed.cpp` fail with `simsimd/simsimd.h file not found` on a from-scratch build. [#120504](https://github.com/ClickHouse/ClickHouse/pull/120504) ([Craig Ringer](https://github.com/ringerc)). * Reduce linker warnings in Rust/wasm builds: remove `--icf=safe` warnings about `SHT_LLVM_ADDRSIG` in the stripped Rust libraries, and apply the `__jit_debug_register_code` workaround only to the `clickhouse` binary and only when `wasmtime` is linked. [#122904](https://github.com/ClickHouse/ClickHouse/pull/122904) ([Raúl Marín](https://github.com/Algunenano)). [#123287](https://github.com/ClickHouse/ClickHouse/pull/123287) ([Azat Khuzhin](https://github.com/azat)). * Update `google-protobuf` to v36.1. `google-cloud-cpp` protobuf sources are now regenerated when `protoc` changes, so incremental builds do not fail after such an update. [#118926](https://github.com/ClickHouse/ClickHouse/pull/118926) ([Actuele AI](https://github.com/actueleai)). [#123603](https://github.com/ClickHouse/ClickHouse/pull/123603) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Update `rocksdb` to v11.8.1. [#118929](https://github.com/ClickHouse/ClickHouse/pull/118929) ([Actuele AI](https://github.com/actueleai)). * Update `abseil-cpp` to 20260817.0. [#118897](https://github.com/ClickHouse/ClickHouse/pull/118897) ([Actuele AI](https://github.com/actueleai)). * Update `PostgreSQL` to 18.6. [#124281](https://github.com/ClickHouse/ClickHouse/pull/124281) ([Actuele AI](https://github.com/actueleai)). * Update `thrift` to v0.25.0. [#124280](https://github.com/ClickHouse/ClickHouse/pull/124280) ([Actuele AI](https://github.com/actueleai)). * Update `cppkafka` to v0.4.1. [#119468](https://github.com/ClickHouse/ClickHouse/pull/119468) ([Actuele AI](https://github.com/actueleai)). * Update `h3` to v4.5.0. [#119328](https://github.com/ClickHouse/ClickHouse/pull/119328) ([Actuele AI](https://github.com/actueleai)). * Update `flatbuffers` to v25.12.19. [#119100](https://github.com/ClickHouse/ClickHouse/pull/119100) ([Actuele AI](https://github.com/actueleai)). * Update `croaring` to v5.2.2. [#119464](https://github.com/ClickHouse/ClickHouse/pull/119464) ([Actuele AI](https://github.com/actueleai)). * Update `simdutf` to v9.1.1. [#119324](https://github.com/ClickHouse/ClickHouse/pull/119324) ([Actuele AI](https://github.com/actueleai)). * Update `double-conversion` to v3.4.0. [#119095](https://github.com/ClickHouse/ClickHouse/pull/119095) ([Actuele AI](https://github.com/actueleai)). * Update `libcpuid` to v0.8.2. [#121382](https://github.com/ClickHouse/ClickHouse/pull/121382) ([Actuele AI](https://github.com/actueleai)). * Update `liburing` to 2.15. [#118895](https://github.com/ClickHouse/ClickHouse/pull/118895) ([Actuele AI](https://github.com/actueleai)). * Update `libuv` to v1.52.1. [#119099](https://github.com/ClickHouse/ClickHouse/pull/119099) ([Actuele AI](https://github.com/actueleai)). * Update `libcotp` to v4.2.2. [#122000](https://github.com/ClickHouse/ClickHouse/pull/122000) ([Actuele AI](https://github.com/actueleai)). * Update `vectorscan` to 5.4.13. [#125774](https://github.com/ClickHouse/ClickHouse/pull/125774) ([Actuele AI](https://github.com/actueleai)). * Update `snappy` to 1.2.2. [#119325](https://github.com/ClickHouse/ClickHouse/pull/119325) ([Actuele AI](https://github.com/actueleai)). * Update `brotli` to v1.2.0. [#119096](https://github.com/ClickHouse/ClickHouse/pull/119096) ([Actuele AI](https://github.com/actueleai)). * Update `libdeflate` to v1.26. [#119349](https://github.com/ClickHouse/ClickHouse/pull/119349) ([Actuele AI](https://github.com/actueleai)). * Update `xxHash` to v0.8.4. [#124231](https://github.com/ClickHouse/ClickHouse/pull/124231) ([Actuele AI](https://github.com/actueleai)). * Update `google-benchmark` to v1.9.5. [#118893](https://github.com/ClickHouse/ClickHouse/pull/118893) ([Actuele AI](https://github.com/actueleai)). * `MemoryWorker` now corrects `MemoryTracker` using allocated memory from ASan, TSan, and MSan instead of resident memory in sanitizer builds when cgroups are available. [#119956](https://github.com/ClickHouse/ClickHouse/pull/119956) ([Konstantin Morozov](https://github.com/k-morozov)). * Add support for cross-compiling ClickHouse for illumos. Closes [#23777](https://github.com/ClickHouse/ClickHouse/issues/23777). [#125073](https://github.com/ClickHouse/ClickHouse/pull/125073) ([Alexey Milovidov](https://github.com/alexey-milovidov)). ### ClickHouse release 26.9, 2026-09-21. [Presentation](https://presentations.clickhouse.com/2026-release-26.9/), [Video](https://www.youtube.com/watch?v=wLrTlU5LLWI) #### Backward Incompatible Change * The analyzer can no longer be disabled: the `enable_analyzer` setting (and its old name `allow_experimental_analyzer`) is obsolete, an attempt to set it to `0` is rejected, and the `compatibility` setting no longer reverts it. The analyzer has been the default since 24.3, and the query analysis used before it is no longer supported. To compare the behaviour or the performance of a query with the old query analysis, use a ClickHouse version older than 26.9. [#118629](https://github.com/ClickHouse/ClickHouse/pull/118629) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The default compression method is changed from `LZ4` to `ZSTD(3)`. Client/server and server/server network communication switches to `ZSTD(3)` uniformly. For `MergeTree` column data the built-in default is now size-aware: `LZ4` for parts below 100 MB and `ZSTD(3)` at or above it, judged by the part size known at write time (the size of the source parts for a merge or a mutation). The `StripeLog` data stream, the persistent files of the `Set` and `Join` engines, and `MergeTree` auxiliary files such as `checksums.txt` switch to `ZSTD(3)` uniformly. HTTP `compress=1` now uses the `network_compression_method` setting too; the default is still `ZSTD(3)`, and `compatibility` set to `26.8` or earlier restores `LZ4` over HTTP as well. This improves compression ratios and reduces storage and network usage out of the box, at a modest increase in CPU usage. To restore the previous behavior: for network compression, use the `compatibility` setting or set `network_compression_method = 'LZ4'`; for `MergeTree` column data, set `CODEC(LZ4)` on the column or table, or set the server `` default to `LZ4`; for the `Log` and `TinyLog` engines, set a column `CODEC(LZ4)`. The remaining streams always use the built-in default and have no runtime rollback; reading stays correct because every compressed frame is self-describing. [#108786](https://github.com/ClickHouse/ClickHouse/pull/108786) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119825](https://github.com/ClickHouse/ClickHouse/pull/119825) ([Alexey Milovidov](https://github.com/alexey-milovidov)). This change is backward compatible, it is listed here for your convenience. * Removed the CatBoost integration. The `catboostEvaluate` function, the `system.models` table and the `SYSTEM RELOAD MODEL(S)` queries are no longer available. Queries, views and grants referring to them have to be migrated before the upgrade: in particular, revoke `SYSTEM RELOAD MODEL` from all users and roles beforehand, because an access entity whose grants still mention the removed privilege cannot be parsed on startup. Also let any in-flight `SYSTEM RELOAD MODEL(S) ON CLUSTER` entries drain from the distributed DDL queue before the upgrade: upgraded hosts can no longer parse such entries and will finish them with an error status instead of executing them. The same applies to persisted table metadata: a table whose column `DEFAULT`/`MATERIALIZED`/`ALIAS` expressions, secondary indices, constraints, projections, partition/order/sample keys, or TTL expressions still reference `catboostEvaluate` cannot be loaded after the upgrade (`UNKNOWN_FUNCTION`), so drop or rewrite such definitions beforehand. The revoke also has to cover every other carrier of serialized grants: a `GRANT SYSTEM RELOAD MODEL` line in the `grants` section of `users.xml` must be removed as well, because it can no longer be parsed and the server then refuses to load the users configuration at startup; on clusters with replicated (Keeper-backed) access storage, revoke the privilege before upgrading any replica, because with the default `access_control_improvements.throw_on_invalid_replicated_access_entities = 0` an upgraded replica that reads a user or role still carrying `SYSTEM RELOAD MODEL` from ZooKeeper cannot parse it and silently drops that entity from its in-memory copy (set `throw_on_invalid_replicated_access_entities = 1` to fail loudly instead); and access backups (`BACKUP ... ACCESS`) taken while the grant was still present cannot be restored with `RESTORE ... ACCESS` on the new version, so re-create such backups after the revoke. [#109710](https://github.com/ClickHouse/ClickHouse/pull/109710) ([Robert Schulze](https://github.com/rschu1ze)). [#116586](https://github.com/ClickHouse/ClickHouse/pull/116586) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the deprecated `read_resource` and `write_resource` disk options from `storage_configuration`. Attach a scheduler resource to a disk with `CREATE RESOURCE name (READ DISK disk, WRITE DISK disk)` instead. [#115286](https://github.com/ClickHouse/ClickHouse/pull/115286) ([Sergei Trifonov](https://github.com/serxa)). * Removed the `WasmEdge` engine for WebAssembly UDFs. `wasmtime`, which has always been the default, is now the only supported value of the `webassembly_udf_engine` server setting. A server configured with `webassembly_udf_engine = 'wasmedge'` will not start; remove that setting or set it to `wasmtime`. On binaries where `wasmtime` is unavailable, including MemorySanitizer, `arm_v80compat`, and `riscv64`, WebAssembly UDFs are not available at all: the server still starts, but `system.webassembly_modules` is absent and any attempt to create or use a WebAssembly UDF fails with `SUPPORT_IS_DISABLED`. [#115316](https://github.com/ClickHouse/ClickHouse/pull/115316) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The experimental `WINDOW VIEW` feature is removed, together with the `WATCH` statement and the `windowID` function. It has been non-functional since version 24.12 (inserts into the source table were silently not delivered to window views) and was never supported by the analyzer, which is enabled by default since 24.3. The settings `allow_experimental_window_view`, `window_view_clean_interval`, `window_view_heartbeat_interval`, and `wait_for_window_view_fire_signal_timeout` are kept as obsolete no-ops. The time-window functions `tumble`, `tumbleStart`, `tumbleEnd`, `hop`, `hopStart`, and `hopEnd` are kept. If you still have window views defined, drop them before upgrading: a server that finds a window view in its metadata fails to start (`Cannot parse definition from metadata file ... (SYNTAX_ERROR)`), the same way the removal of the `Lazy` database engine behaves. Similar functionality will be reimplemented on top of streaming queries (issue [#114738](https://github.com/ClickHouse/ClickHouse/issues/114738)). [#114747](https://github.com/ClickHouse/ClickHouse/pull/114747) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the sharded `GROUP BY` aggregation (`enable_sharding_aggregator`, disabled by default) in favor of the adaptive aggregation (`enable_adaptive_aggregator`, enabled by default), which targets the same high-cardinality workloads without the weaknesses on skewed or low-cardinality keys and with support for external aggregation. The `enable_sharding_aggregator` setting is kept as obsolete: setting it is still accepted and has no effect. [#116532](https://github.com/ClickHouse/ClickHouse/pull/116532) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed wrong results from `runningConcurrency` under `LEFT JOIN` and `ANY JOIN`, and under lazy evaluation in `if`, `multiIf`, `and`, and `or`, by treating it as non-deterministic. Queries using it may be slower. A sorting key, partition key, or text index `preprocessor`/`postprocessor` containing `runningConcurrency` is now rejected, and a table created earlier with one will not load after upgrading: remove or replace it before the upgrade. [#114403](https://github.com/ClickHouse/ClickHouse/pull/114403) ([Groene AI](https://github.com/groeneai)). * `BACKUP ... TO Disk(...)`, `RESTORE ... FROM Disk(...)`, and `CREATE DATABASE ... ENGINE = Backup(..., Disk(...))` now require `SOURCES` grants for the target `Disk(...)` location: `WRITE ON DISK` to write and `READ ON DISK` to read. Previously a user with `BACKUP` but without `SOURCES` could still use any disk listed in `backups.allowed_disk`; such users now need the corresponding `DISK` grant, while users already granted `SOURCES` are unaffected. [#117406](https://github.com/ClickHouse/ClickHouse/pull/117406) ([Groene AI](https://github.com/groeneai)). * Compression codecs are now gated by dedicated per-codec settings whose declared tier defines the codec maturity. The `ALP` codec is promoted to beta through `enable_alp_codec`, and `allow_experimental_codecs` is now obsolete: it no longer enables any codec, so use the per-codec `enable__codec` settings instead. [#116337](https://github.com/ClickHouse/ClickHouse/pull/116337) ([Raufs Dunamalijevs](https://github.com/rienath)). [#117545](https://github.com/ClickHouse/ClickHouse/pull/117545) ([Raufs Dunamalijevs](https://github.com/rienath)). * Reading a `File` source with `rename_files_after_processing` set now requires the `WRITE ON FILE` grant on top of `READ ON FILE` (on the initiator for `fileCluster`): the setting renames the files a `SELECT` has read, which is a write to the source, and it used to happen for anyone who could read them. A query relying on it now needs `GRANT WRITE ON FILE`. A per-query renaming rule is also no longer cached by a `Filesystem` database, where it could make an unrelated later query rename files, or be dropped silently. [#117895](https://github.com/ClickHouse/ClickHouse/pull/117895) ([Groene AI](https://github.com/groeneai)). * The `interface` and `http_method` columns of `system.query_log`, `system.query_thread_log` and `system.processes` are now `Enum8` instead of `UInt8`, over the value set `system.session_log.interface` already uses, plus `Unknown` for an unrecognized interface. Comparing to a number keeps working (`WHERE interface = 1`) and comparing to a name now works too (`WHERE interface = 'TCP'`), but a plain `SELECT` renders the name, and arithmetic on the column (`interface + 0`) is no longer accepted. The default-enabled `system.user_query_log` and the opt-in `all_` union tables copy these types. As with any system log schema change, the existing log tables are renamed with the first free suffix (`query_log_0`, `query_log_1`, ...) on upgrade; their rows read as names after `ALTER TABLE system.query_log_0 MODIFY COLUMN interface Enum8(...)`, and the same for `http_method`, preceded by `MODIFY SETTING table_readonly = 0` if that table was frozen. [#118136](https://github.com/ClickHouse/ClickHouse/pull/118136) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE ... AS mergeTreeIndex(...)` is no longer accepted, nor the same form over `mergeTreeProjection`, `mergeTreeAnalyzeIndexes`, `mergeTreeAnalyzeIndexesUUID`, `mergeTreeTextIndex` and `mergeTreeCodecBlockCounts`. The storage these functions return holds its source table's storage object, so once such a persisted table had been read, `DROP TABLE SYNC` never returned, the source stayed in `system.dropped_tables` forever and its data was never reclaimed from disk. Reading these functions directly, as in `SELECT ... FROM mergeTreeIndex(...)`, is unaffected. An existing table of this shape keeps loading at server startup, but `ATTACH TABLE`, `RESTORE` and `Replicated` database replica recovery now refuse it: list them with `SELECT database, name FROM system.tables WHERE engine = 'Proxy'`, drop the ones over these six functions, and read the functions directly instead. [#118969](https://github.com/ClickHouse/ClickHouse/pull/118969) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE ... AS timeSeriesSamples(...)` is no longer accepted, nor the same form over `timeSeriesData`, `timeSeriesTags`, or `timeSeriesMetrics`. Reading those functions directly is unaffected. Existing tables of this shape still load at server startup, but `ATTACH TABLE`, `RESTORE`, and `Replicated` replica recovery now refuse them, because once such a persisted proxy table had been read, `DROP TABLE ` dropped its live target and destroyed its rows, and later reads could fail with `QueryPlan was not initialized`. Find candidates with `SELECT database, name, create_table_query FROM system.tables WHERE engine = 'Proxy'` and keep only the ones whose outer call is one of these four functions; a `view(...)` wrapper is still allowed. Drop such a table on a freshly started server, before anything reads it, so the drop removes only the proxy and leaves the `TimeSeries` data intact. Do not use `DETACH TABLE` instead, because a detached definition can no longer be `ATTACH`ed. If metadata loading of such a table already fails after a restart, move the metadata file out of `store/` and restart: until then, unfiltered reads of `system.tables` can fail with `ASYNC_LOAD_WAIT_FAILED`. [#119786](https://github.com/ClickHouse/ClickHouse/pull/119786) ([Groene AI](https://github.com/groeneai)). * Removed the `s3_disable_checksum` setting. It existed to skip a second read of the source data for the checksum of an S3 upload; the checksum is now computed during the single read, so there is nothing left to disable. [#119496](https://github.com/ClickHouse/ClickHouse/pull/119496) ([Azat Khuzhin](https://github.com/azat)). * `Nullable(Tuple(...))` is now generally available and enabled by default. `Nullable(Tuple(...))` columns can be created without any setting, schema inference of nullable structs in the `Parquet`, `Arrow`, `ORC`, `Avro` and `JSON` formats returns `Nullable(Tuple(...))`, and a `Tuple` subcolumn extracted from a `Variant`, `Dynamic`, `JSON` or `Nullable(Tuple)` column is `Nullable(Tuple(...))` and `NULL` where the value is missing, instead of a tuple of default values. The previous behavior is available with `enable_nullable_tuple_type = 0` and `allow_nullable_tuple_in_extracted_subcolumns = 0`; the subcolumn setting is read at server startup, so it has to be set in the default profile. Upgrading existing tables whose partition keys, sorting keys, TTL expressions, or skip indexes depend on a tuple subcolumn extracted from a `Dynamic`, `Variant` or `Nullable(Tuple)` column or from a `JSON` path not declared as `Tuple(...)` is unsafe with the new default: tables may fail to attach, queries may fail, or data parts may be detached during startup. Before the first startup after upgrading, set `allow_nullable_tuple_in_extracted_subcolumns = 0` in the default profile, or preserve the previous behavior through `compatibility` in that profile. Changing the setting after parts have been detached does not restore them automatically; those parts must also be reattached. [#119222](https://github.com/ClickHouse/ClickHouse/pull/119222) ([Nihal Z. Miaji](https://github.com/nihalzp)). * New setting `validate_group_by_all_key_types` (default `true`) gates the key-type validation that `GROUP BY ALL` applies to the grouping keys it expands the `SELECT` expressions into. Since 26.7 it rejects a `Variant` or `Dynamic` grouping key, for example an untyped JSON subpath, that earlier versions accepted with the analyzer enabled (the default), with no way to keep such a query running across an upgrade. Setting `compatibility` to a version before 26.7, or `validate_group_by_all_key_types = 0`, restores the earlier behavior. An explicit `GROUP BY` is unaffected and still rejects such a key, as it did before 26.7. Closes [#119852](https://github.com/ClickHouse/ClickHouse/issues/119852). [#119868](https://github.com/ClickHouse/ClickHouse/pull/119868) ([Groene AI](https://github.com/groeneai)). #### New Feature * `DISTINCT` can now spill data to disk, like external aggregation and external sort: the new settings `max_bytes_before_external_distinct` and `max_bytes_ratio_before_external_distinct` set the threshold; by default, spilling starts once query memory usage exceeds half the available memory under the applicable server or user memory limits, or as soon as the next hash table allocation would exceed it. [#116569](https://github.com/ClickHouse/ClickHouse/pull/116569) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Refreshable materialized views can now refresh incrementally with `REFRESH ... APPEND INCREMENTAL`, appending only the rows committed to the source table since the previous refresh instead of re-reading the whole source. [#114152](https://github.com/ClickHouse/ClickHouse/pull/114152) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). An incremental refresh into an `Iceberg` target is now exactly-once: the refresh cursor is committed atomically inside the `Iceberg` snapshot summary. [#114823](https://github.com/ClickHouse/ClickHouse/pull/114823) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * New `LIMIT` forms that select rows by boundary conditions instead of by position, following the stream order (use `ORDER BY` to define it): `LIMIT n AFTER cond` returns up to `n` rows starting at the first row where `cond` is true (`SELECT number FROM numbers(10) ORDER BY number LIMIT 3 AFTER number >= 5` returns `5, 6, 7`); `LIMIT UNTIL cond` returns the rows before the first row where `cond` is true (`LIMIT UNTIL number >= 3` returns `0, 1, 2`); `LIMIT AFTER cond1 UNTIL cond2` returns the rows between the two (`LIMIT AFTER number >= 2 UNTIL number >= 6` returns `2, 3, 4, 5`); `LIMIT n AFTER cond ALL` opens a range of `n` rows at every matching row and returns their union without duplicates (`LIMIT 2 AFTER number IN (2, 6) ALL` returns `2, 3, 6, 7`). [#99508](https://github.com/ClickHouse/ClickHouse/pull/99508) ([Zakhar Kravchuk](https://github.com/zakr600)). * Added the `CREATE TOKEN` statement, which generates a random secret, adds it to the current user as an additional authentication method and returns it together with its expiration deadline: `CREATE TOKEN VALID FOR INTERVAL 30 DAY GRANTS (SELECT ON db.table)`. It is a convenient way to create tokens for applications: a token is tied to the user, can be limited in time and in privileges, and stops working when the user is deleted. Without an explicit `VALID UNTIL` or `VALID FOR` clause a token lives for `create_token_default_ttl_seconds`, 30 minutes by default. The statement requires the new `CREATE TOKEN` privilege, which also authorizes the equivalent `ALTER USER ADD IDENTIFIED ...` statement. [#116957](https://github.com/ClickHouse/ClickHouse/pull/116957) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Authentication methods in `CREATE USER` and `ALTER USER ... ADD IDENTIFIED` support a `GRANTS (SELECT ON db.table, ...)` clause which limits the access rights of sessions authenticated with that method to the intersection with the listed grants. This allows using additional credentials as tokens for applications: `ALTER USER vasya ADD IDENTIFIED WITH password BY '...' VALID UNTIL '2026-12-31' GRANTS (SELECT ON db.table)`. Closes [#109117](https://github.com/ClickHouse/ClickHouse/issues/109117). [#110144](https://github.com/ClickHouse/ClickHouse/pull/110144) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the SQL standard linear regression aggregate functions `regr_slope`, `regr_intercept`, `regr_r2`, `regr_count`, `regr_avgx`, `regr_avgy`, `regr_sxx`, `regr_syy` and `regr_sxy`. [#119347](https://github.com/ClickHouse/ClickHouse/pull/119347) ([mosya415](https://github.com/mosya415)). * Added `MergeTree` settings `max_table_size_rows`, `max_table_size_bytes_compressed`, and `max_table_size_bytes_uncompressed`, which limit the total size of a table. The limits are checked at the start of `INSERT` and when new parts are committed, including the results of background merges and mutations, but not on replicated fetches. Closes [#115063](https://github.com/ClickHouse/ClickHouse/issues/115063). [#115069](https://github.com/ClickHouse/ClickHouse/pull/115069) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `max_tables` database setting to limit the number of tables in `Atomic` and `Ordinary` databases. Every table-like object counts toward the limit, including tables, views, and dictionaries. [#115202](https://github.com/ClickHouse/ClickHouse/pull/115202) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added bracket syntax for `JSON` subcolumns, such as `json['key1']['key2']`; the parser translates it to nested `arrayElement` calls. [#107039](https://github.com/ClickHouse/ClickHouse/pull/107039) ([Pavel Kruglov](https://github.com/Avogar)). * Added the `system.statements` system table, which exposes the documentation of SQL statements such as their name, syntax, description, and examples. [#115341](https://github.com/ClickHouse/ClickHouse/pull/115341) ([Robert Schulze](https://github.com/rschu1ze)). * Added virtual columns for the `Iceberg` v3 manifest fields `first_row_id` and `last_seq_num`. [#115603](https://github.com/ClickHouse/ClickHouse/pull/115603) ([Konstantin Vedernikov](https://github.com/scanhex12)). The same fields are now also written when inserting into an `Iceberg` v3 table. [#116171](https://github.com/ClickHouse/ClickHouse/pull/116171) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added the opt-in `MergeTree` setting `skip_empty_columns_on_insert`, which avoids storing columns that contain only their type-default value. It requires `serialization_info_version = 'with_missing_columns'`; during rolling upgrades with servers that cannot read the new part metadata, keep the older serialization version. [#98472](https://github.com/ClickHouse/ClickHouse/pull/98472) ([Amos Bird](https://github.com/amosbird)). * Added support for `+` and `-` between `DateTime` and `Time` data types. [#82980](https://github.com/ClickHouse/ClickHouse/pull/82980) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Added the `parseISO8601Duration` function, which parses ISO 8601 duration strings such as `PT1H30M` into seconds. [#116993](https://github.com/ClickHouse/ClickHouse/pull/116993) ([Navneet Kumar](https://github.com/navneet83)). * Added the `type_json_skip_null_typed_paths` setting for `JSON`. When it is enabled, typed paths in `JSON` columns that have `NULL` values are treated as absent, matching the behavior of dynamic paths. Closes [#99191](https://github.com/ClickHouse/ClickHouse/issues/99191). [#106959](https://github.com/ClickHouse/ClickHouse/pull/106959) ([Pavel Kruglov](https://github.com/Avogar)). * Added the `arrayFlattenedLength` function, which returns the total number of elements in a multidimensional array as if it was flattened with `arrayFlatten`. Unlike `cardinality` and `length`, which count only the elements of the outermost array, it counts elements at every nesting level, matching PostgreSQL's `cardinality`. [#113696](https://github.com/ClickHouse/ClickHouse/pull/113696) ([David Meng](https://github.com/davidmenggx)). * `clickhouse-client` now accepts `--ssh-key-file` without a value: the key is looked up the same way as `ssh` does it, from `~/.ssh/config`, the default identity files such as `~/.ssh/id_ed25519`, and the keys held by `ssh-agent`. Authentication with a key held by `ssh-agent` is now supported without passing its path or passphrase explicitly. [#115072](https://github.com/ClickHouse/ClickHouse/pull/115072) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `splitByRegexp` tokenizer, used in text indexes and the `tokens` function, now accepts an optional `extract` argument. With `splitByRegexp(re, 1)`, `re` no longer acts as a separator: each match contributes at most one token, the first capture group of the match or the whole match if `re` has no capture group. [#116127](https://github.com/ClickHouse/ClickHouse/pull/116127) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Added the `system.session_query_ids` system table, which contains the query ids of the queries executed in the current session, in execution order. It makes it easy to find "the queries I just ran" in `system.query_log` without assigning `query_id` client-side or tagging queries with `log_comment`, which is especially useful for testing and benchmarking. The history size is bounded per session by the new `session_query_ids_history_size` setting (default `1000`, `0` disables recording), and `TRUNCATE TABLE system.session_query_ids` clears the current session's history. [#110536](https://github.com/ClickHouse/ClickHouse/pull/110536) ([Vladimir Cherkasov](https://github.com/vdimir)). * Added a zero-argument form of `year` that returns the current year, mirroring `now` and `today`: `year()` is equivalent to `toYear(today())`. `year` is now a function in its own right rather than a case-insensitive alias of `toYear`, and is reported non-deterministic, so `year()` is excluded from features that require a deterministic function: the query result cache, mutations of `Replicated*` tables, on-the-fly mutation application (`apply_mutations_on_fly`), mutation partition pruning, and the trivial-view pushdown to `Distributed` tables; use `toYear()` where determinism is required. Results, result types, index analysis and projection selection for `year()` are unchanged. [#109097](https://github.com/ClickHouse/ClickHouse/pull/109097) ([Groene AI](https://github.com/groeneai)). * Added a memory fragmentation profiler for jemalloc. The new `system.jemalloc_sampled_allocations` table lists the sampled allocations that are currently alive, with their backtrace, age and size class; together with `system.jemalloc_arena_bins` it shows which code holds the allocations that keep fragmented slabs alive, and the same data is shown in a new "Fragmentation" tab of the jemalloc page of the server's web UI. [#115456](https://github.com/ClickHouse/ClickHouse/pull/115456) ([Azat Khuzhin](https://github.com/azat)). * Support `BACKUP` and `RESTORE` for `WORKLOAD` and `RESOURCE` entities. SQL-defined `CREATE WORKLOAD` and `CREATE RESOURCE` definitions are now included when backing up `system.workloads` and `system.resources` and are recreated on restore, including `ON CLUSTER`. [#108431](https://github.com/ClickHouse/ClickHouse/pull/108431) ([Sergei Trifonov](https://github.com/serxa)). * Added the `workload_admission_timeout_ms` query setting that bounds how long a query waits to acquire its workload query slot and memory reservation before failing. `0` (the default) keeps the previous unbounded wait. [#118852](https://github.com/ClickHouse/ClickHouse/pull/118852) ([Sergei Trifonov](https://github.com/serxa)). * Added `mode = 'exclusive'` to the `S3Queue` engine: file processing is tracked only in the memory of the server, with no coordination through Keeper, for high-throughput ingestion on a single server, such as from a local `minio` instance. [#110552](https://github.com/ClickHouse/ClickHouse/pull/110552) ([Ivan Tkatchev](https://github.com/ivan-tkatchev)). * Added the `nats_ca_file`, `nats_client_cert_file` and `nats_client_key_file` settings to the `NATS` engine, for verifying a broker certificate signed by a private CA and for client-certificate authentication. The feature was first proposed in [#69396](https://github.com/ClickHouse/ClickHouse/pull/69396) by [sidhmads](https://github.com/sidhmads). [#116117](https://github.com/ClickHouse/ClickHouse/pull/116117) ([Nikolay Degterinsky](https://github.com/evillique)). * Added a four-letter-word command to `clickhouse-keeper` for the backpressure that the leader applies to slow Raft members. [#117728](https://github.com/ClickHouse/ClickHouse/pull/117728) ([Kseniia Sumarokova](https://github.com/kssenii)). * Added the `keyValuePairs` text index tokenizer, which indexes a `Map(String, String)` column directly and answers `map['key'] = 'value'` from the index without reading the map column. [#115579](https://github.com/ClickHouse/ClickHouse/pull/115579) ([Jimmy Aguilar Mena](https://github.com/Ergus)). #### Experimental Feature * Added Prometheus HTTP API support for the `/api/v1/metadata`, `/api/v1/labels`, and `/api/v1/label//values` endpoints on `TimeSeries` tables. The `/api/v1/labels` endpoint returns the sorted unique label names of the matched time series, and `/api/v1/label//values` returns the sorted unique values of one label; both support the optional `match[]`, `start`, `end`, and `limit` parameters. [#116518](https://github.com/ClickHouse/ClickHouse/pull/116518) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). [#116731](https://github.com/ClickHouse/ClickHouse/pull/116731) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). [#117160](https://github.com/ClickHouse/ClickHouse/pull/117160) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Added the `/api/v1/format_query` endpoint of the Prometheus HTTP API. It parses a PromQL expression and returns its canonical form. [#117159](https://github.com/ClickHouse/ClickHouse/pull/117159) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Added an embedded SQL Console UI at `/ui` on the HTTP server. This is the same console that is available in ClickHouse Cloud - not to be confused with the Web UI. [#100406](https://github.com/ClickHouse/ClickHouse/pull/100406) ([Luis Neves](https://github.com/lneves12)). * Reimplemented the experimental `KQL` (Kusto) dialect with a dedicated lexer and parser instead of translating it to SQL text and reparsing it. This fixes expression-injection in string operators such as `contains` and `has`, several results that disagreed with Kusto such as `7 / 2`, `substring` with a negative start, and `bin`, and functions that were registered but did nothing. Unsupported syntax is now rejected instead of being mistranslated. [#112932](https://github.com/ClickHouse/ClickHouse/pull/112932) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added an experimental `trino` value of the `dialect` setting (gated by `enable_trino_dialect`): queries written in Trino SQL are translated to ClickHouse SQL, including Trino-specific syntax (`ARRAY[...]` literals, `TRY_CAST`, `UNNEST`, `ROW` constructors and types, `VALUES` tables, `OFFSET` before `LIMIT`, `FETCH`) and a mapping of several hundred Trino function names and argument conventions to their ClickHouse equivalents. Bitwise shifts over untyped integer literals now match Trino semantics. [#115383](https://github.com/ClickHouse/ClickHouse/pull/115383) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119430](https://github.com/ClickHouse/ClickHouse/pull/119430) ([Ethan Lin](https://github.com/linsen458-spec)). * Added the `hierarchicalKMeans` and `assignCentroid` functions, building blocks for IVF-style vector search. [#112309](https://github.com/ClickHouse/ClickHouse/pull/112309) ([Shankar Iyer](https://github.com/shankar-iyer)). * Added `SELECT` support for `TimeSeries` tables. [#115622](https://github.com/ClickHouse/ClickHouse/pull/115622) ([Vitaly Baranov](https://github.com/vitlibar)). * `TimeSeries` tables now derive their inner engine family from `default_table_engine`, require all inner tables to use the same replication family, and always pin `recent_samples_ttl_seconds` into `SETTINGS` for `RECENT SAMPLES` targets. [#115922](https://github.com/ClickHouse/ClickHouse/pull/115922) ([Vitaly Baranov](https://github.com/vitlibar)). * Added PromQL support for `absent` and `count_values`, including range queries and `by`/`without` aggregation modifiers. [#112795](https://github.com/ClickHouse/ClickHouse/pull/112795) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL now supports the range functions `sum_over_time`, `avg_over_time`, `count_over_time`, `present_over_time`, `absent_over_time`, `quantile_over_time`, and `predict_linear`. [#112353](https://github.com/ClickHouse/ClickHouse/pull/112353) ([Valery Petrov](https://github.com/valerypetrov)). [#112842](https://github.com/ClickHouse/ClickHouse/pull/112842) ([Valery Petrov](https://github.com/valerypetrov)). * The `TimeSeries` table engine now supports dictionary-encoded series identifiers: the `id` column can be declared with a `LowCardinality` type, and new tables use `Tuple(UInt64, LowCardinality(UUID))` by default while existing tables keep their declared `id` type. The previous layout remains available through `TAGS INNER COLUMNS (id Tuple(UInt64, UUID))`, and the new layout gives a 1.16x geometric-mean PromQL speedup in the benchmark. [#117033](https://github.com/ClickHouse/ClickHouse/pull/117033) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * `timeSeriesGroupArray` is now supported in `SimpleAggregateFunction`, and the `timeSeries*ToGrid` functions now also accept the samples as a single `Array(Tuple(timestamp, value))` argument. [#117340](https://github.com/ClickHouse/ClickHouse/pull/117340) ([Vitaly Baranov](https://github.com/vitlibar)). [#117341](https://github.com/ClickHouse/ClickHouse/pull/117341) ([Vitaly Baranov](https://github.com/vitlibar)). * `TimeSeries` tables now expose the outer samples column as `samples` instead of `time_series`. `INSERT INTO ts_new SELECT * FROM ts_old` still works because the column order is unchanged. [#119225](https://github.com/ClickHouse/ClickHouse/pull/119225) ([Vitaly Baranov](https://github.com/vitlibar)). The `METRICS` target is renamed to `METRIC FAMILIES`, which is what that inner table actually holds; `METRICS` keeps working as an alias. [#119227](https://github.com/ClickHouse/ClickHouse/pull/119227) ([Vitaly Baranov](https://github.com/vitlibar)). * Distributed query plans (`make_distributed_plan`) now stop idle upstream stages promptly after a satisfied `LIMIT`, instead of continuing to read and compute data that nobody consumes. They also no longer stall when the initiator pipeline has only one execution thread, because waiting for distributed plan stages no longer occupies that thread. Plans with many nodes now honor the query's `max_threads` in worker fragments, so they no longer exhaust the server thread pool or outlive `max_execution_time`; `use_concurrency_control` is disabled automatically for these queries because it can starve worker tasks while they wait on exchange sockets. Plans with many finished stages also no longer pay up to 100 ms of idle waiting per stage while draining completed stages, which speeds up multi-join queries several times in some cases. The data sent between nodes is now also serialized on all threads of a task instead of one thread per destination, removing a bottleneck when a query runs on a small number of workers. [#115690](https://github.com/ClickHouse/ClickHouse/pull/115690) ([Alexander Gololobov](https://github.com/davenger)). [#116017](https://github.com/ClickHouse/ClickHouse/pull/116017) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#116300](https://github.com/ClickHouse/ClickHouse/pull/116300) ([Groene AI](https://github.com/groeneai)). [#116878](https://github.com/ClickHouse/ClickHouse/pull/116878) ([Alexander Gololobov](https://github.com/davenger)). [#117328](https://github.com/ClickHouse/ClickHouse/pull/117328) ([Alexander Gololobov](https://github.com/davenger)). [#119067](https://github.com/ClickHouse/ClickHouse/pull/119067) ([Alexander Gololobov](https://github.com/davenger)). * `parallel_replicas_plan_based` can now collect runtime statistics and automatically enable parallel replicas for supported queries. [#115788](https://github.com/ClickHouse/ClickHouse/pull/115788) ([Igor Nikonov](https://github.com/devcrafter)). * A `Merge` table can now be read with parallel replicas: reads from the underlying `MergeTree` tables are coordinated across replicas, and aggregations and joins above them are distributed too. Enabled by the `parallel_replicas_allow_merge_tables` setting together with `parallel_replicas_plan_based`. [#115001](https://github.com/ClickHouse/ClickHouse/pull/115001) ([Igor Nikonov](https://github.com/devcrafter)). * Added experimental support for the `silk` fiber runtime, enabled with the `enable_silk_runtime` server setting. When it is enabled, subsystems that support it can run jobs on fibers instead of occupying an operating-system thread while waiting for I/O. [#112667](https://github.com/ClickHouse/ClickHouse/pull/112667) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Adaptive codec selection (`enable_adaptive_codec_selection`) for `MergeTree` now also applies to columns without a specialized candidate codec: when no type-specific codec candidate exists, each block is written with whichever of the table's default codec and `NONE` compresses it smaller, so incompressible data is no longer stored larger than raw. For `Float32` and `Float64` columns it now also considers the `ALP` codec. [#115639](https://github.com/ClickHouse/ClickHouse/pull/115639) ([Raufs Dunamalijevs](https://github.com/rienath)). [#117553](https://github.com/ClickHouse/ClickHouse/pull/117553) ([Raufs Dunamalijevs](https://github.com/rienath)). * Added aggregation pushdown below `JOIN` (eager aggregation) to the experimental Cascades optimizer as a cost-based alternative: a partial aggregation is performed on one join input before the join and merged above it, or, when the join provably neither duplicates nor null-extends the aggregated side and the join keys are a subset of the `GROUP BY` keys, the aggregation is pushed below the join entirely. Controlled by the new setting `cascades_aggregation_pushdown` (enabled by default; takes effect only with `enable_cascades_optimizer = 1` and `make_distributed_plan = 1`). [#115354](https://github.com/ClickHouse/ClickHouse/pull/115354) ([Dmitry Novik](https://github.com/novikd)). * Queries with `INTERSECT` and `EXCEPT` now work under the experimental `make_distributed_plan` setting. [#116563](https://github.com/ClickHouse/ClickHouse/pull/116563) ([Alexander Gololobov](https://github.com/davenger)). When `make_distributed_plan` is enabled and no distributed plan can be built, the query falls back to local execution. [#116487](https://github.com/ClickHouse/ClickHouse/pull/116487) ([Antonio Filipovic](https://github.com/antoniofilipovic)). A failing `make_distributed_plan = 1` query reports the error of the task that failed, not the closed exchange connection that failure caused; worker task failures reach the client with the worker's error code. [#118390](https://github.com/ClickHouse/ClickHouse/pull/118390) ([Groene AI](https://github.com/groeneai)). * Fixed `Column '__text_index_...' not found in table` and `Query builder not found for text search query` when a query using a text index read with `parallel_replicas_plan_based` enabled. [#116363](https://github.com/ClickHouse/ClickHouse/pull/116363) ([Igor Nikonov](https://github.com/devcrafter)). * `parallel_replicas_plan_based` now supports `RIGHT JOIN` and `ORDER BY ... WITH FILL`; when the analyzer is disabled, the query runs locally instead of falling back to query-based parallel replicas, because plan-based parallel replicas are not supported there. [#113515](https://github.com/ClickHouse/ClickHouse/pull/113515) ([Groene AI](https://github.com/groeneai)). [#116135](https://github.com/ClickHouse/ClickHouse/pull/116135) ([Igor Nikonov](https://github.com/devcrafter)). [#118781](https://github.com/ClickHouse/ClickHouse/pull/118781) ([Igor Nikonov](https://github.com/devcrafter)). * Adaptive codec selection (`enable_adaptive_codec_selection`) now applies to every substream of a column, such as `Array` and `String` sizes and null maps, instead of the main data stream only. [#117348](https://github.com/ClickHouse/ClickHouse/pull/117348) ([Raufs Dunamalijevs](https://github.com/rienath)). * WebAssembly UDFs now split an input block that would not fit into the guest's linear memory into smaller batches, sized by the input's measured serialized size. The new setting `webassembly_udf_input_split_memory_ratio` controls how much of the linear memory one call's input may occupy. [#116552](https://github.com/ClickHouse/ClickHouse/pull/116552) ([Vasily Chekalkin](https://github.com/bacek)). A UDF using the `BUFFERED_V1` ABI that returns a result buffer with a non-zero size but a data pointer at linear-memory offset `0` is now rejected instead of unrelated memory being read. [#116548](https://github.com/ClickHouse/ClickHouse/pull/116548) ([Vasily Chekalkin](https://github.com/bacek)). * The experimental `ReaderExecutor` now shrinks its read window and block size under memory pressure, configured via `reader_executor_memory_pressure_{elevated,high,critical}_level_pct`; its default read window is now 8 MiB. [#115635](https://github.com/ClickHouse/ClickHouse/pull/115635) ([Sema Checherinda](https://github.com/CheSema)). * PromQL: implemented the `max_over_time` and `min_over_time` functions. [#112354](https://github.com/ClickHouse/ClickHouse/pull/112354) ([Valery Petrov](https://github.com/valerypetrov)). PromQL now also parses legacy octal numeric literals with Prometheus-compatible semantics. [#113586](https://github.com/ClickHouse/ClickHouse/pull/113586) ([Minh Vu](https://github.com/fallintoplace)). `sgn` now preserves `NaN` and negative zero. [#113747](https://github.com/ClickHouse/ClickHouse/pull/113747) ([Minh Vu](https://github.com/fallintoplace)). `stddev` and `stdvar` no longer return `NaN` for large finite values whose variance is representable. [#114250](https://github.com/ClickHouse/ClickHouse/pull/114250) ([Minh Vu](https://github.com/fallintoplace)). * PromQL no longer counts consecutive `NaN` samples as changes. [#113744](https://github.com/ClickHouse/ClickHouse/pull/113744) ([Minh Vu](https://github.com/fallintoplace)). * `CREATE TABLE ... ENGINE = DeltaLake(...)` can now create a new `DeltaLake` table, writing the initial commit through `delta-kernel-rs`, attach to an existing `_delta_log`, or register an existing table into a Unity catalog. It is gated behind the new `allow_delta_lake_create_table` setting; creating a partitioned table (`PARTITION BY`) is not supported yet. Writes to a `DeltaLake` table now cast values to the Delta write-schema type with an accurate cast that throws when a value does not fit the target type, for example `300` written into a Delta `byte`, instead of silently truncating; the new `delta_lake_accurate_write_cast` setting, enabled by default, controls this, and `compatibility` below `26.9` restores the previous truncating behavior. [#106011](https://github.com/ClickHouse/ClickHouse/pull/106011) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). #### Performance Improvement * Reduced memory usage and query-plan optimization time for keyed aggregations over large constant-folded literals by hashing the aggregation hash-table-statistics cache key while it is serialized instead of accumulating a full copy in memory first. For an 80 MB literal, peak memory drops by about 250 MB and optimization time by about 2.5x. [#115847](https://github.com/ClickHouse/ClickHouse/pull/115847) ([Groene AI](https://github.com/groeneai)). * Added the `query_plan_aggregation_bucket_top_k` setting (enabled by default) to control the optimization that keeps only each two-level aggregation bucket's best `n` groups when a final aggregation feeds `ORDER BY ... LIMIT n`. The optimization is now visible in `EXPLAIN` output. [#116124](https://github.com/ClickHouse/ClickHouse/pull/116124) ([Nihal Z. Miaji](https://github.com/nihalzp)). * A `WHERE` equality is now merged into the `JOIN` condition even when its operands have different types but a common supertype, such as `Int32` and `Nullable(Int32)`. This lets more `CROSS JOIN`s be rewritten to `INNER JOIN`s. [#112630](https://github.com/ClickHouse/ClickHouse/pull/112630) ([Hechem Selmi](https://github.com/m-selmi)). * Improved performance of merges of text indexes. [#114525](https://github.com/ClickHouse/ClickHouse/pull/114525) ([Anton Popov](https://github.com/CurtizJ)). * Reduce CPU usage of the post-filter used by `IN` and `NOT IN` predicates. [#111690](https://github.com/ClickHouse/ClickHouse/pull/111690) ([Elmi Ahmadov](https://github.com/ahmadov)). * The `arrayJoin` function syntax is now planned as the `ARRAY JOIN` operator internally, so it can use the same `ARRAY JOIN` optimizations. A filter on a column produced by `ARRAY JOIN` is pushed into the array before it is expanded, so elements that do not match are never turned into rows and other columns are not replicated for them, which speeds up selective `ARRAY JOIN ... WHERE` queries over wide rows. A follow-up reduces the fused path's per-block overhead by avoiding per-window filter copies and computing survivor offsets only when eager replication needs them. Controlled by the `query_plan_fuse_filter_into_array_join` setting. [#115474](https://github.com/ClickHouse/ClickHouse/pull/115474) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#116910](https://github.com/ClickHouse/ClickHouse/pull/116910) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#117660](https://github.com/ClickHouse/ClickHouse/pull/117660) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a performance regression where a query combining a text index filter with `ORDER BY ... LIMIT` stopped answering the filter from the index and read the whole text column instead. [#115586](https://github.com/ClickHouse/ClickHouse/pull/115586) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Speed up query pipeline construction for queries with large expressions or many streams by computing each plan step's output header once instead of once per stream. For a ClickBench Q29-like query on a 96-core machine, this removes about a third of the hot query time. [#115677](https://github.com/ClickHouse/ClickHouse/pull/115677) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up building the hash table of an `ANY`, `SEMI`, or `ANTI JOIN` when the right side repeats the same join keys many times. [#115820](https://github.com/ClickHouse/ClickHouse/pull/115820) ([Nikita Taranov](https://github.com/nickitat)). * Speed up random data generation. `rand`, `rand64`, `randConstant`, `randomString`, `randomFixedString`, `generateUUIDv4`, `generateUUIDv7`, and `dateTimeToUUIDv7` now generate bytes with ARS (reduced-round AES in counter mode) wherever the CPU has AES instructions: on AArch64 chosen at build time, on x86-64 chosen at run time when VAES is present. Targets and CPUs without those instructions keep the previous generator unchanged. A follow-up reduced the fixed setup cost on x86-64, which is noticeable when blocks are small. [#116133](https://github.com/ClickHouse/ClickHouse/pull/116133) ([Groene AI](https://github.com/groeneai)). [#116158](https://github.com/ClickHouse/ClickHouse/pull/116158) ([Raúl Marín](https://github.com/Algunenano)). * Fix a CPU regression for queries that issue many independent small reads against the same `MergeTree` part when the table has `LowCardinality` columns. Such queries on tables with a small `index_granularity` were up to several times slower since 26.6. [#116134](https://github.com/ClickHouse/ClickHouse/pull/116134) ([Groene AI](https://github.com/groeneai)). * Speed up `GROUP BY` without aggregates by inlining some functions and by starting prefetching at the same cardinality as `GROUP BY` with aggregates, which helps when the hash table stops fitting in the L2 cache. [#115840](https://github.com/ClickHouse/ClickHouse/pull/115840) ([Nikita Taranov](https://github.com/nickitat)). [#116267](https://github.com/ClickHouse/ClickHouse/pull/116267) ([Nikita Taranov](https://github.com/nickitat)). * Use the `bloom_filter` skip index for `IN` predicates on a directly indexed column when `transform_null_in = 1` and the `IN` set contains no `NULL` values. Previously the index was skipped for such queries, forcing a full scan. [#111329](https://github.com/ClickHouse/ClickHouse/pull/111329) ([Groene AI](https://github.com/groeneai)). * The adaptive `GROUP BY` aggregation (`enable_adaptive_aggregator`) now falls back to the ordinary algorithm when rare keys or aggregate arguments are too wide to copy profitably, so aggregations over heavy string columns run at the ordinary algorithm's speed and memory usage. Adds `adaptive_aggregator_freeze_threshold_bytes` to bound each thread's frozen table in bytes. [#115755](https://github.com/ClickHouse/ClickHouse/pull/115755) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Lightweight `UPDATE` queries now write patch parts that carry the table's sort-key columns, so applying a patch no longer has to hold the whole patch part in memory. [#116007](https://github.com/ClickHouse/ClickHouse/pull/116007) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added an alternative row-major payload layout for hash joins to speed up join output reconstruction. Controlled by `enable_hash_join_row_store` (default `true`) and `min_rows_ratio_for_hash_join_row_store` (default `5`). A probe that emits no right-side columns, such as `SELECT count()`, no longer publishes a fake zero match count into the row-store cache, so a later full-output run can still use the row-store optimization. Closes [#117260](https://github.com/ClickHouse/ClickHouse/issues/117260). [#104884](https://github.com/ClickHouse/ClickHouse/pull/104884) ([Hechem Selmi](https://github.com/m-selmi)). [#117410](https://github.com/ClickHouse/ClickHouse/pull/117410) ([Groene AI](https://github.com/groeneai)). * The `IEJoin` algorithm now chooses which two `JOIN ON` inequality conditions to use as its key conditions by their estimated selectivity from the column min/max statistics, instead of taking the first two in syntax order. The remaining conditions are evaluated per candidate pair, so picking the most selective pair can speed up such joins by orders of magnitude. [#115790](https://github.com/ClickHouse/ClickHouse/pull/115790) ([Vladimir Cherkasov](https://github.com/vdimir)). * A `bloom_filter` skip index on an `Array` column is now used for `arrayJoin(column) IN (set)`, `arrayJoin(column) GLOBAL IN (set)`, and `arrayJoin(column) = const`, the same way it was already used for `hasAny(column, set)` and `has(column, const)`. Previously these `arrayJoin` forms fell back to a full scan. [#109536](https://github.com/ClickHouse/ClickHouse/pull/109536) ([Groene AI](https://github.com/groeneai)). * Reduce cancellation latency for distributed queries that use parallel replicas by not waiting for a replica's initial announcement after the query is already cancelled. [#114862](https://github.com/ClickHouse/ClickHouse/pull/114862) ([Nikita Taranov](https://github.com/nickitat)). * Speed up reading `system.tables` by not copying the query context for every row. [#115867](https://github.com/ClickHouse/ClickHouse/pull/115867) ([Nikolay Degterinsky](https://github.com/evillique)). * Speed up `Array(LowCardinality(T))` aggregations by preserving dictionary identity. [#115961](https://github.com/ClickHouse/ClickHouse/pull/115961) ([Ivan Babrou](https://github.com/bobrik)). * Improved the performance of the `roundDuration` and `roundAge` functions on AArch64 by rewriting their comparison cascades into a branchless form that the compiler can auto-vectorize. Results are unchanged. [#116164](https://github.com/ClickHouse/ClickHouse/pull/116164) ([Groene AI](https://github.com/groeneai)). * Avoid extremely slow `BACKUP` of a `Memory` or `KeeperMap` table when `max_compress_block_size` is set to a small value by sizing the temporary file stream with `temporary_files_buffer_size`. [#116233](https://github.com/ClickHouse/ClickHouse/pull/116233) ([Groene AI](https://github.com/groeneai)). * Speed up server shutdown for databases with many tables by shutting tables down in parallel. Controlled by the new `database_catalog_shutdown_table_concurrency` setting; `0` means the number of CPU cores. [#104969](https://github.com/ClickHouse/ClickHouse/pull/104969) ([Jayme Bird](https://github.com/jaymebrd)). * `arrayElement` and `arraySlice` now read lazily replicated arrays (produced by `ARRAY JOIN` and `JOIN` when `enable_lazy_columns_replication` is enabled) directly, without materializing them. This reduces memory usage and speeds up queries that index or slice a large array. [#112304](https://github.com/ClickHouse/ClickHouse/pull/112304) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * `A UNION DISTINCT B` now deduplicates each branch in parallel before merging them, the same way its `SELECT DISTINCT * FROM (A UNION ALL B)` rewrite already did. Previously the query funneled every row through a single thread, so it did not scale with `max_threads`. [#115173](https://github.com/ClickHouse/ClickHouse/pull/115173) ([Groene AI](https://github.com/groeneai)). * Speed up regular-expression searches for simple anchored literals, including prefix, suffix, exact-literal, and escaped-literal matches. [#116554](https://github.com/ClickHouse/ClickHouse/pull/116554) ([Elmi Ahmadov](https://github.com/ahmadov)). [#117331](https://github.com/ClickHouse/ClickHouse/pull/117331) ([Elmi Ahmadov](https://github.com/ahmadov)). * Reduce CPU and memory overhead of `TTL` deletions during merges: blocks are now filtered in one vectorized pass instead of being rebuilt column by column, and a block with nothing to delete is passed through untouched instead of being copied. [#116623](https://github.com/ClickHouse/ClickHouse/pull/116623) ([Shaohua Wang](https://github.com/tiandiwonder)). * Optimize one-argument date conversions on `Date` and `DateTime` columns. [#114267](https://github.com/ClickHouse/ClickHouse/pull/114267) ([Ivan Babrou](https://github.com/bobrik)). * `clickhouse-local` now attaches the `system` and `information_schema` tables on first access instead of at startup. [#115489](https://github.com/ClickHouse/ClickHouse/pull/115489) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Speed up `GRANT`, `REVOKE`, and `SHOW GRANTS` for users with many grants by avoiding a deep copy of the access-rights tree when it is only read. [#115696](https://github.com/ClickHouse/ClickHouse/pull/115696) ([miao tang](https://github.com/a819721810)). * Reduce memory usage when inserting into a wide `MergeTree` part that writes many more streams than it has columns, for example a table with a few bucketed `Map` columns. The `min_columns_to_activate_adaptive_write_buffer` setting is now compared against the number of streams the part writes, since one write buffer is allocated per stream. [#115759](https://github.com/ClickHouse/ClickHouse/pull/115759) ([Groene AI](https://github.com/groeneai)). * Fixed a performance regression since 26.6: a query filtering a `Nullable` column with `IN (subquery)` read the whole table instead of skipping the read when the subquery returned no rows. Closes [#116450](https://github.com/ClickHouse/ClickHouse/issues/116450). [#116538](https://github.com/ClickHouse/ClickHouse/pull/116538) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed quadratic complexity of `groupConcat` over a `FixedString` argument. The whole argument column was converted to `String` once per row, so a query over 400,000 rows took 45 seconds instead of 0.1 seconds and could overrun its `max_execution_time`, which is only checked between blocks. [#116795](https://github.com/ClickHouse/ClickHouse/pull/116795) ([Groene AI](https://github.com/groeneai)). * Push a `WHERE` condition on a `USING` column through a `RIGHT JOIN` so the left `MergeTree` input can use it as a primary-key index condition, even when the join keys have different types. Applies with `enable_analyzer = 1`. [#103478](https://github.com/ClickHouse/ClickHouse/pull/103478) ([Manuel](https://github.com/raimannma)). * Reduce memory usage of Parquet dictionary-based row-group pruning (`input_format_parquet_dictionary_filter_push_down`) several times by hashing integer dictionaries directly and keeping the value set as a sorted vector instead of a hash table. [#115744](https://github.com/ClickHouse/ClickHouse/pull/115744) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading dictionary-encoded `Parquet` data is now faster: dictionary indexes are decoded and resolved to values in one fused pass, turning runs of a repeated value into bulk fills instead of per-row gathers, for about 20% faster scans of dictionary-encoded numeric columns. [#115819](https://github.com/ClickHouse/ClickHouse/pull/115819) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up some `PREWHERE` queries with text indexes by placing text-index lookups and other zero-size-column predicates before expensive predicates on the indexed column. [#116546](https://github.com/ClickHouse/ClickHouse/pull/116546) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Speed up the `numbers` table function, sorting, and other bulk index fills on AArch64 by improving code generation of their sequential-fill loops. Results and x86-64/macOS code generation are unchanged. [#116616](https://github.com/ClickHouse/ClickHouse/pull/116616) ([Groene AI](https://github.com/groeneai)). * Speed up `groupUniqArray` for 8-bit types (`UInt8`, `Int8`, `Enum8`, `Bool`) by using a 256-bit bitmap instead of a hash set. [#116880](https://github.com/ClickHouse/ClickHouse/pull/116880) ([Manuel](https://github.com/raimannma)). * Writing rows in JSON output formats that emit object keys, including `JSONEachRow` and its aliases, `PrettyJSONEachRow`, `JSON`, `JSONStrings`, `JSONStringsEachRow`, `JSONObjectEachRow`, and `GeoJSON`, is now faster because each field name is no longer copied into a temporary string and re-measured for every row. Compact formats that emit no keys are unchanged. [#117082](https://github.com/ClickHouse/ClickHouse/pull/117082) ([Groene AI](https://github.com/groeneai)). * Fixed a performance regression introduced in 26.4 when writing `JSON` and `Dynamic` columns in text formats, including `Dynamic(JSON)` text output. [#115852](https://github.com/ClickHouse/ClickHouse/pull/115852) ([Pavel Kruglov](https://github.com/Avogar)). [#117655](https://github.com/ClickHouse/ClickHouse/pull/117655) ([Nikita Fomichev](https://github.com/fm4v)). * Speed up repeated insertion of the same `LowCardinality` value into an in-memory column, for example during joins. [#115020](https://github.com/ClickHouse/ClickHouse/pull/115020) ([Minh Vu](https://github.com/fallintoplace)). * Correlated subqueries with equality on columns that differ only in nullability, or on native-number columns with a least common type, are now decorrelated via equivalent-expression substitution instead of falling back to a `CROSS JOIN` with a post-join filter. This avoids severe memory growth on such queries. Closes [#113407](https://github.com/ClickHouse/ClickHouse/issues/113407). [#115493](https://github.com/ClickHouse/ClickHouse/pull/115493) ([Dmitry Novik](https://github.com/novikd)). * Speed up division and modulo of `Int128`, `UInt128`, and `Decimal128` values that fit in 64 bits. `Decimal128` rounding with `round`, `floor`, `ceil`, `trunc`, and `roundBankers` is 1.5x faster on x86-64, and gains more on architectures that have no 128-bit divide instruction. Results are unchanged. [#116199](https://github.com/ClickHouse/ClickHouse/pull/116199) ([Groene AI](https://github.com/groeneai)). * Functions with several SIMD implementations, including `MD5`, `SHA1`, hashing functions, `greatCircleDistance`, and `geoDistance`, now choose one from the CPU's capabilities instead of benchmarking them at runtime. `MD5` becomes 2-3.7x faster. The `function_implementation` setting is obsolete and has no effect. [#116501](https://github.com/ClickHouse/ClickHouse/pull/116501) ([Raúl Marín](https://github.com/Algunenano)). * Speed up `dotProductTransposed` on `QBit` columns with floating-point element types on hosts where SimSIMD has no vectorized dot kernel, which for `Float64` means AArch64 and x86 without AVX-512. The reduction now uses independent partial sums instead of a single accumulator, so its summation order changes. Closes [#116784](https://github.com/ClickHouse/ClickHouse/issues/116784). [#116816](https://github.com/ClickHouse/ClickHouse/pull/116816) ([Groene AI](https://github.com/groeneai)). * Reduce jemalloc fragmentation by lowering `lg_extent_max_active_fit` to its default value `6`. [#116836](https://github.com/ClickHouse/ClickHouse/pull/116836) ([Azat Khuzhin](https://github.com/azat)). * Speed up `formatDateTime` with non-constant time zone arguments by caching resolved time zones locally. [#116851](https://github.com/ClickHouse/ClickHouse/pull/116851) ([Robert Schulze](https://github.com/rschu1ze)). * PromQL selector scans over `TimeSeries` tables are now parallelized, which speeds up reads of the narrow, decode-heavy samples table columns. [#116932](https://github.com/ClickHouse/ClickHouse/pull/116932) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Speed up conversion of `TimeSeries` tag arrays by reserving the final output size up front and by building extracted tag values and `NULL` maps in one pass. [#115620](https://github.com/ClickHouse/ClickHouse/pull/115620) ([Minh Vu](https://github.com/fallintoplace)). [#115687](https://github.com/ClickHouse/ClickHouse/pull/115687) ([Minh Vu](https://github.com/fallintoplace)). * Speed up `timeSeriesGroupArray` when the input samples are already sorted by timestamp. [#117703](https://github.com/ClickHouse/ClickHouse/pull/117703) ([Vitaly Baranov](https://github.com/vitlibar)). * The `Native` protocol now batches end-of-query control messages into a single socket write by default, and `sync` now flushes them immediately. This removes one round trip of latency per query for clients on high-latency connections that idle between queries. [#111034](https://github.com/ClickHouse/ClickHouse/pull/111034) ([Kaviraj Kanagaraj](https://github.com/kavirajk)). * Phrase search in text indexes now stores positions in packed blocks and reads only the blocks that cover candidate rows, reducing position-data reads and memory usage. [#111550](https://github.com/ClickHouse/ClickHouse/pull/111550) ([Elmi Ahmadov](https://github.com/ahmadov)). * Appending many default values to `Array` and `String` columns now pre-sizes the offsets array, reducing reallocations and peak memory usage. [#113577](https://github.com/ClickHouse/ClickHouse/pull/113577) ([Groene AI](https://github.com/groeneai)). * The preliminary per-stream `DISTINCT` now abandons deduplication when the input is almost entirely unique, freeing its hash tables because the final `DISTINCT` still deduplicates the output. This roughly halves memory usage and speeds up `SELECT DISTINCT` on unique data. Controlled by `allow_preliminary_distinct_abandoning` and observable via `DistinctTransformsAbandonedDeduplication`. [#116508](https://github.com/ClickHouse/ClickHouse/pull/116508) ([Nihal Z. Miaji](https://github.com/nihalzp)). * PromQL instant-vector responses now reuse the formatted evaluation timestamp within each result block. [#115567](https://github.com/ClickHouse/ClickHouse/pull/115567) ([Minh Vu](https://github.com/fallintoplace)). * Speed up `TimeSeries` reads by writing sampling keys directly into the result column. [#115683](https://github.com/ClickHouse/ClickHouse/pull/115683) ([Minh Vu](https://github.com/fallintoplace)). * Reduce lock contention and per-lookup allocations during serialization by sharding the object pool. [#116869](https://github.com/ClickHouse/ClickHouse/pull/116869) ([Manuel](https://github.com/raimannma)). * `LIKE ... ESCAPE 'c'` and `ILIKE ... ESCAPE 'c'` can now use skip indexes instead of falling back to a full scan. `TYPE text` indexes prune granules for both; `LIKE ... ESCAPE 'c'` can also use the `ngrambf_v1`, `tokenbf_v1`, and `sparse_grams` indexes, and `NOT LIKE ... ESCAPE 'c'` can use `bloom_filter` skip indexes and the primary key when applicable, matching plain `NOT LIKE`. `TYPE text` indexes still do not support `NOT LIKE`. [#105848](https://github.com/ClickHouse/ClickHouse/pull/105848) ([Groene AI](https://github.com/groeneai)). * Distributed `ORDER BY` query plans can now use `read in order` when the query order matches the table's sorting key, and projections can reuse that base-table in-order read too. [#115244](https://github.com/ClickHouse/ClickHouse/pull/115244) ([Shankar Iyer](https://github.com/shankar-iyer)). [#116854](https://github.com/ClickHouse/ClickHouse/pull/116854) ([Mikhail Artemenko](https://github.com/Michicosun)). * Slightly speed up `INSERT` into `JSON` columns by making typed-path default insertion cheaper. [#107359](https://github.com/ClickHouse/ClickHouse/pull/107359) ([Pavel Kruglov](https://github.com/Avogar)). * Added the `query_plan_lift_predicate_across_join` optimization, which copies equi-join filter conjuncts across `MergeTree` join inputs for better primary-key pruning. It speeds up some queries by 70-80x. [#108377](https://github.com/ClickHouse/ClickHouse/pull/108377) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Automatic parallel replicas now share prepared sets with the probe plan instead of executing them twice. [#114231](https://github.com/ClickHouse/ClickHouse/pull/114231) ([Nikita Taranov](https://github.com/nickitat)). * Supported `LIKE` and `ILIKE` patterns can now use text indexes. [#114728](https://github.com/ClickHouse/ClickHouse/pull/114728) ([Elmi Ahmadov](https://github.com/ahmadov)). * Improve generic `arrayReverse` performance by reserving space for all nested elements. [#115011](https://github.com/ClickHouse/ClickHouse/pull/115011) ([Minh Vu](https://github.com/fallintoplace)). * `LEFT ANTI JOIN` and `LEFT SEMI JOIN`, when no column of the right table is selected, now build a hash table that stores the keys alone and release the right-hand blocks as they are built. This makes them up to 1.5x faster and can halve memory usage. Controlled by the new setting `enable_join_key_only_hash_tables`, which is enabled by default. [#115301](https://github.com/ClickHouse/ClickHouse/pull/115301) ([Nikita Taranov](https://github.com/nickitat)). * Speed up `timeSeriesFromGrid` for dense rows by bulk-copying values. [#115518](https://github.com/ClickHouse/ClickHouse/pull/115518) ([Minh Vu](https://github.com/fallintoplace)). * Answer `min`, `max`, and `count` aggregations without `GROUP BY` and filters directly from per-part column statistics, collected automatically for suitable columns by the `auto_statistics_types` `MergeTree` setting, reading no data for the parts that have statistics materialized. This speeds up ClickBench Q6 (`SELECT MIN(EventDate), MAX(EventDate) FROM hits`) from tens of milliseconds to a few milliseconds. Controlled by the new setting `use_statistics_for_min_max_aggregation`, which is enabled by default. [#115738](https://github.com/ClickHouse/ClickHouse/pull/115738) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * On AArch64, short runtime-sized copies in hot string, join, and part-reading paths now stay inline instead of being rewritten to a `memcpy` call. [#117353](https://github.com/ClickHouse/ClickHouse/pull/117353) ([Groene AI](https://github.com/groeneai)). * Improve performance of `bitmapAndCardinality`, `bitmapOrCardinality`, `bitmapXorCardinality`, `bitmapAndnotCardinality`, and `bitmapHasAny` by computing the result directly instead of first materializing the intersection bitmap. [#117478](https://github.com/ClickHouse/ClickHouse/pull/117478) ([Anand Kr Shaw](https://github.com/anandheritage)). * Reduce memory usage of reading a `TimeSeries` table directly with SQL (`SELECT ... FROM ts_table`). The samples are now aggregated in sorting-key order instead of being buffered in a hash table, so peak memory no longer grows with the size of the scanned window, and `LIMIT` is pushed into the read. [#117726](https://github.com/ClickHouse/ClickHouse/pull/117726) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Queries with join-side disjunction predicates can now push the per-side filters down to `PREWHERE` on `MergeTree` reads, reducing unnecessary column reads. [#117760](https://github.com/ClickHouse/ClickHouse/pull/117760) ([Nikita Taranov](https://github.com/nickitat)). * Reduce `clickhouse-keeper` request latency for concurrent `RemoveRecursive` requests when many uncommitted nodes exist. [#117778](https://github.com/ClickHouse/ClickHouse/pull/117778) ([Alexander Gololobov](https://github.com/davenger)). * Improve performance of `hasToken`, `hasAnyTokens`, and `hasAllTokens` queries with text indexes in lazy posting-list apply mode. [#117905](https://github.com/ClickHouse/ClickHouse/pull/117905) ([Anton Popov](https://github.com/CurtizJ)). * Speed up k-way merges: merging sorted streams with multi-column or collated sorting keys uses a sorted array of cursors instead of a heap (7-8% faster), merges of non-intersecting parts detect the whole cursor as one batch in O(1) comparisons, and runs of duplicate keys inside a part are skipped, which makes `SELECT ... FINAL` from `ReplacingMergeTree` use up to 40% less CPU and the insert-time deduplication of `optimize_on_insert` about three times faster when there are many duplicates. [#115869](https://github.com/ClickHouse/ClickHouse/pull/115869) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading columns with many substreams, such as `JSON`, from object storage is much faster and makes far fewer requests: the marks of all streams of a `MergeTree` part are now loaded asynchronously and in parallel by default (setting `load_marks_asynchronously`). [#118477](https://github.com/ClickHouse/ClickHouse/pull/118477) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Marks of streams with an empty data file, such as the `SharedVariant` substreams of `JSON` paths with values of a single type, are not loaded at all, which removes about a third of the requests. [#118478](https://github.com/ClickHouse/ClickHouse/pull/118478) ([Alexey Milovidov](https://github.com/alexey-milovidov)). The metadata streams of `JSON` and `Dynamic` columns are no longer read twice when prefetching. [#118479](https://github.com/ClickHouse/ClickHouse/pull/118479) ([Alexey Milovidov](https://github.com/alexey-milovidov)). A read of a file inside a packed part is bounded to the file's slice, so it does not transfer the rest of the archive and reuses HTTP connections. [#118483](https://github.com/ClickHouse/ClickHouse/pull/118483) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved performance of reading individual paths and subcolumns from `JSON` columns that use the `advanced` shared data serialization. Each path and substream now starts in its own compressed block, so a selective read no longer decompresses data belonging to other paths. [#118154](https://github.com/ClickHouse/ClickHouse/pull/118154) ([Pavel Kruglov](https://github.com/Avogar)). * The join reordering optimizer uses the CD-C and CD-A conflict detectors from the SIGMOD'13 paper "On the Correct and Complete Enumeration of the Core Search Space", so joins of any kind can be reordered, including `ANTI` and `SEMI` joins, not only `INNER` joins. For seven relations, CD-C explores about 12 times more valid plans than before. [#117677](https://github.com/ClickHouse/ClickHouse/pull/117677) ([Fisnik Kastrati](https://github.com/fkastrati)). * Decode `Iceberg` data manifest files concurrently while the list of data files for a query is produced. The setting `iceberg_manifest_decode_concurrency` (default `4`) bounds how many manifests, delete and data alike, are decoded at a time; it replaces `iceberg_delete_manifest_decode_concurrency`, which is kept as an alias. [#116723](https://github.com/ClickHouse/ClickHouse/pull/116723) ([Aaron Harlap](https://github.com/aaron276h)). * Use the `lower_bound` and `upper_bound` statistics of the `Iceberg` manifest list to prune data files. Closes [#104567](https://github.com/ClickHouse/ClickHouse/issues/104567). [#115817](https://github.com/ClickHouse/ClickHouse/pull/115817) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Reduce peak memory of the `JSONExtract*` functions on the rapidjson code path (`allow_simdjson = 0`): the parser's DOM allocator is rewound between rows, so the DOM is bounded by one document instead of growing with `max_block_size`. A 100-level nested document uses 84% less memory and is 30% faster, a 200-key object 61% less and 21% faster. [#117618](https://github.com/ClickHouse/ClickHouse/pull/117618) ([Groene AI](https://github.com/groeneai)). * Faster writing of `String` values in the JSON output formats (`JSONEachRow` and its variants, `JSON`, `JSONObjectEachRow`, `PrettyJSONEachRow` and `GeoJSON`) and in `JSONExtractRaw`: a run of bytes that needs no escaping is copied in one block instead of one byte at a time. Values in which most bytes need escaping become slower. The output is unchanged. [#117755](https://github.com/ClickHouse/ClickHouse/pull/117755) ([Groene AI](https://github.com/groeneai)). * `addDays`, `addWeeks`, `subtractDays` and `subtractWeeks` over a `DateTime` column are up to 12% faster in time zones with UTC-offset changes: a per-row range check that the argument type makes impossible to fail is no longer executed. [#118219](https://github.com/ClickHouse/ClickHouse/pull/118219) ([Groene AI](https://github.com/groeneai)). * `toHour` and `toMinute` no longer walk the calendar lookup table in time zones where the result is pure arithmetic: `toHour` in every zone whose UTC offset does not change after 1970 (including `UTC` and `Asia/Kolkata`), and `toMinute` in zones that keep a constant minute-of-hour, such as `Asia/Kolkata`, `Pacific/Chatham` and `America/St_Johns`. Results are unchanged. [#117562](https://github.com/ClickHouse/ClickHouse/pull/117562) ([Groene AI](https://github.com/groeneai)). * Reduce the per-query cost of applying the `compatibility` setting and of logging changed settings into `system.query_log`. A query on a service that sets an old `compatibility` value can run more than twice as fast. [#118114](https://github.com/ClickHouse/ClickHouse/pull/118114) ([Raúl Marín](https://github.com/Algunenano)). * The bound on the cost of estimating the selectivity of `col IN (...)` from column statistics, controlled by `statistics_max_set_size_for_exact_selectivity_estimation`, is now also applied when `compatibility` is set to 26.6 or an earlier version. [#118095](https://github.com/ClickHouse/ClickHouse/pull/118095) ([Nikita Fomichev](https://github.com/fm4v)). * Automatic parallel replicas now skip their planning overhead entirely when no read step of the query is estimated to read at least `automatic_parallel_replicas_min_bytes_per_replica` times the number of replicas. [#116521](https://github.com/ClickHouse/ClickHouse/pull/116521) ([Nikita Taranov](https://github.com/nickitat)). * Join runtime filters now reach the local probe-side plan under parallel replicas when `parallel_replicas_filter_pushdown` is enabled, so grouped probe-side queries filter rows before local aggregation instead of after it. [#118650](https://github.com/ClickHouse/ClickHouse/pull/118650) ([Nikita Taranov](https://github.com/nickitat)). The join runtime filter no longer builds a Bloom filter that the build-side key count predicts to be too dense to be useful. [#118336](https://github.com/ClickHouse/ClickHouse/pull/118336) ([Manuel](https://github.com/raimannma)). * The bucket-local top-K optimization (`query_plan_aggregation_bucket_top_k`) no longer materializes every group's key for a statistic that only automatic parallel replicas consume, so `GROUP BY ... ORDER BY count() DESC LIMIT n` over high-cardinality keys spends less CPU in the final aggregation. [#118374](https://github.com/ClickHouse/ClickHouse/pull/118374) ([Groene AI](https://github.com/groeneai)). * Skip analyzing the implicit `minmax_count` projection during query planning when the query aggregates with a function that projection cannot provide. Previously every aggregating query over a `MergeTree` table interpreted the projection's definition while optimizing the plan, even for `sum`, `avg`, `uniq` and every other aggregate it can never serve. [#118723](https://github.com/ClickHouse/ClickHouse/pull/118723) ([Groene AI](https://github.com/groeneai)). * A condition on an `ARRAY JOIN` element, such as `ARRAY JOIN tags AS t WHERE t IN (...)`, is now used for skip index analysis the same way `arrayJoin(tags) IN (...)` already is. [#118582](https://github.com/ClickHouse/ClickHouse/pull/118582) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). A `WHERE` condition that compares an `ARRAY JOIN` element with other columns of the row is applied before the expansion, so filtered-out elements are never expanded. [#118904](https://github.com/ClickHouse/ClickHouse/pull/118904) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `ARRAY JOIN` of a `Nested` column inside a subquery, CTE, `UNION` branch or view now reads only the referenced subcolumns, like a top-level `ARRAY JOIN` does. Fixed an exception `Tuple doesn't have element with name` when a second `ARRAY JOIN` referenced a subcolumn of a `Nested` column joined by the first one. [#119055](https://github.com/ClickHouse/ClickHouse/pull/119055) ([Vladimir Cherkasov](https://github.com/vdimir)). * A `bloom_filter` skip index is no longer selected for a condition built only from `!=` or `NOT IN`. A bloom filter can only prove that a value is absent from a granule, so such a condition never skipped a granule while the whole index was still read. `force_data_skipping_indices` now reports `INDEX_NOT_USED` for it. [#116544](https://github.com/ClickHouse/ClickHouse/pull/116544) ([Groene AI](https://github.com/groeneai)). * Text indexes on `mapKeys` and `JSONAllPaths` can now prune `IN` predicates even when the right-hand-side set exceeded `use_index_for_in_with_subqueries_max_values` and was built without stored elements. [#118128](https://github.com/ClickHouse/ClickHouse/pull/118128) ([Nikita Taranov](https://github.com/nickitat)). A text index built with the `array` tokenizer can now serve arbitrary `LIKE` and `ILIKE` patterns by scanning its dictionary. [#115649](https://github.com/ClickHouse/ClickHouse/pull/115649) ([Elmi Ahmadov](https://github.com/ahmadov)). * Speed up primary key index analysis for conditions with a large `IN` set by reusing per-thread buffers instead of allocating on every call. [#110937](https://github.com/ClickHouse/ClickHouse/pull/110937) ([Manuel](https://github.com/raimannma)). Avoid a redundant `LowCardinality` type unwrap per mark when evaluating `IN` conditions on the primary key. [#117054](https://github.com/ClickHouse/ClickHouse/pull/117054) ([Manuel](https://github.com/raimannma)). * Improved performance of aggregate functions that combine a null map with an `-If` condition (`sum`, `avg`, `min`/`max`/`argMin`/`argMax` with `-If`, functions with two or more `Nullable` arguments such as `corr` and `quantileExactWeighted`, and the `timeSeries*` functions with `-If`) when a block is consumed in increasing sub-ranges, as `optimize_aggregation_in_order` and window frames do. [#118025](https://github.com/ClickHouse/ClickHouse/pull/118025) ([Groene AI](https://github.com/groeneai)). * Hashing a value that has no contiguous in-memory representation (`JSON`, `Variant`, `AggregateFunction`) no longer allocates and copies a buffer per row. `xxHash64` over a `Variant` column is about 40% faster, `sipHash64` over a `JSON` column about 5%. Counting distinct aggregate states with `uniqExact` is about 33% faster for 200-byte states; `uniq`, `DISTINCT` and `arrayDistinct` hash states through the same path. [#119322](https://github.com/ClickHouse/ClickHouse/pull/119322) ([Groene AI](https://github.com/groeneai)). * Speed up compression and decompression with the `T64` codec on AArch64 and on x86-64 hosts without AVX-512. The byte and bit transposes now use portable vector shuffles instead of a per-byte scalar loop. Compressed output is unchanged. [#116206](https://github.com/ClickHouse/ClickHouse/pull/116206) ([Groene AI](https://github.com/groeneai)). Faster `DoubleDelta` decompression. [#118787](https://github.com/ClickHouse/ClickHouse/pull/118787) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Improve `LIMIT BY` performance by stopping input reads after a constant grouping key reaches its limit. [#118770](https://github.com/ClickHouse/ClickHouse/pull/118770) ([Minh Vu](https://github.com/fallintoplace)). * Squash small chunks before writing the intermediate table during recursive CTE evaluation. Deep recursions no longer accumulate one tiny `Memory` block per produced chunk, which degraded per-step reads of the working table. [#107068](https://github.com/ClickHouse/ClickHouse/pull/107068) ([Zach Naimon](https://github.com/ArctypeZach)). * Speed up `jaroSimilarity` and `jaroWinklerSimilarity` using AVX2. [#104457](https://github.com/ClickHouse/ClickHouse/pull/104457) ([Manuel](https://github.com/raimannma)). Speed up `isValidASCII` by using SIMD kernels from simdutf. [#119062](https://github.com/ClickHouse/ClickHouse/pull/119062) ([Konstantin Bogdanov](https://github.com/thevar1able)). Faster `base58Encode` and `base58Decode`. [#117619](https://github.com/ClickHouse/ClickHouse/pull/117619) ([Groene AI](https://github.com/groeneai)). Improve the performance of string functions in byte mode by avoiding unnecessary ASCII scans. [#115003](https://github.com/ClickHouse/ClickHouse/pull/115003) ([Minh Vu](https://github.com/fallintoplace)). * Speed up `mortonEncode` and `mortonDecode` by resolving each input column and the constant range mask once per call instead of once per row. Results are unchanged. [#117727](https://github.com/ClickHouse/ClickHouse/pull/117727) ([Groene AI](https://github.com/groeneai)). * Speed up the fast Walsh-Hadamard transform used by `randomHadamardTransform` and by vector quantization with AVX2 (about 28% higher throughput). [#119064](https://github.com/ClickHouse/ClickHouse/pull/119064) ([Jingxin (Philip) Li](https://github.com/philip-jingxin)). * Faster `cbrt`, `fmod`, `fma` and `hypot`: in release builds the slower implementations from Rust's `compiler_builtins` were shadowing the optimized `llvm-libc` ones, and `hypot` now uses the libc implementation. [#118910](https://github.com/ClickHouse/ClickHouse/pull/118910) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Reduced the time and peak memory of `s2GetNeighbors` over many rows by reserving its result array once per block instead of once per row. Peak memory at four million rows in a single block drops by about 44%, and the gain grows with `max_block_size`. [#115919](https://github.com/ClickHouse/ClickHouse/pull/115919) ([Groene AI](https://github.com/groeneai)). * Avoid materializing constant arguments in generic `least` and `greatest` evaluation. [#119108](https://github.com/ClickHouse/ClickHouse/pull/119108) ([Minh Vu](https://github.com/fallintoplace)). * Faster materialization of lazily replicated sparse columns when consecutive output rows refer to the same source row. [#118237](https://github.com/ClickHouse/ClickHouse/pull/118237) ([Minh Vu](https://github.com/fallintoplace)). * Skip rebuilding column TTL blocks when no row in the block is expired. [#119109](https://github.com/ClickHouse/ClickHouse/pull/119109) ([Minh Vu](https://github.com/fallintoplace)). * Faster `LEFT ANTI` joins with the `ie_join` algorithm when there is no residual condition: empty bitmap scans are skipped. [#118584](https://github.com/ClickHouse/ClickHouse/pull/118584) ([Minh Vu](https://github.com/fallintoplace)). * PromQL: a math binary operator applied to two aggregations of the same expression with the same grouping, such as `sum(x) - max(x)` or `sum by (job) (rate(x[5m])) / count by (job) (rate(x[5m]))`, now evaluates the argument once and computes both aggregations in a single `GROUP BY`, instead of scanning the argument twice and joining the two results by group. [#118307](https://github.com/ClickHouse/ClickHouse/pull/118307) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). Use the constant empty group for aggregations with an empty `by()` modifier. [#118132](https://github.com/ClickHouse/ClickHouse/pull/118132) ([Minh Vu](https://github.com/fallintoplace)). Improved the performance of the `deriv` function. [#118987](https://github.com/ClickHouse/ClickHouse/pull/118987) ([Vitaly Baranov](https://github.com/vitlibar)). * A PromQL selector matching a whole metric on a `TimeSeries` table no longer re-evaluates its primary-key range on `id` for every row it reads; measured 23.7% faster on a 105M-row whole-metric query. [#119092](https://github.com/ClickHouse/ClickHouse/pull/119092) ([Groene AI](https://github.com/groeneai)). * Faster `timeSeries*ToGrid` aggregate functions: per-sample timestamp validation is skipped when the sorted bucket endpoints already prove the range. [#118189](https://github.com/ClickHouse/ClickHouse/pull/118189) ([Minh Vu](https://github.com/fallintoplace)). * Reduced contention on the global logger mutex in the `MergeTree` read path, which was noticeable on servers with many cores and queries touching many parts. [#117498](https://github.com/ClickHouse/ClickHouse/pull/117498) ([Manuel](https://github.com/raimannma)). Partition values are no longer formatted for a trace-level log message when trace logging is disabled. [#119605](https://github.com/ClickHouse/ClickHouse/pull/119605) ([Rory Shanks](https://github.com/rorylshanks)). * Push the `disk_name` filter of `system.remote_data_paths` into the disk traversal, so a query that names the disks it needs no longer walks every object storage disk on the server. [#118400](https://github.com/ClickHouse/ClickHouse/pull/118400) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up `clickhouse-client` startup with many options. [#118386](https://github.com/ClickHouse/ClickHouse/pull/118386) ([Ivan Babrou](https://github.com/bobrik)). * Fewer allocations when reporting `ProfileEvents` snapshots. [#118305](https://github.com/ClickHouse/ClickHouse/pull/118305) ([Ilya Yatsishin](https://github.com/qoega)). * Speed up reading subcolumns from wide parts and from compact parts with substream marks by avoiding unnecessary full-column serializations. [#111020](https://github.com/ClickHouse/ClickHouse/pull/111020) ([Rory Shanks](https://github.com/rorylshanks)). * Speed up aggregation batches with non-null states. [#117009](https://github.com/ClickHouse/ClickHouse/pull/117009) ([Ivan Babrou](https://github.com/bobrik)). * Speed up flushing `system.opentelemetry_span_log` by writing spans directly into its columns instead of boxing each value into a `Field`. A burst of spans no longer makes `SYSTEM FLUSH LOGS opentelemetry_span_log` hit its timeout. [#117481](https://github.com/ClickHouse/ClickHouse/pull/117481) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Avoid join-order regressions when the selectivity estimator cannot evaluate every predicate in a `WHERE` condition: unaffected predicates are still used to estimate cardinality instead of falling back to full-table estimates. [#118137](https://github.com/ClickHouse/ClickHouse/pull/118137) ([Nikita Taranov](https://github.com/nickitat)). * Improved performance of some array functions: generic `arrayFirst` and `arrayLast` reserve result rows, `arrayExcept` no longer copies the input when the exclusion array is empty, single-argument `arrayConcat` no longer rebuilds the result, and `arraySlice(arr, 1)` no longer rebuilds the result array. [#118188](https://github.com/ClickHouse/ClickHouse/pull/118188) ([Minh Vu](https://github.com/fallintoplace)). [#119665](https://github.com/ClickHouse/ClickHouse/pull/119665) ([Minh Vu](https://github.com/fallintoplace)). [#119697](https://github.com/ClickHouse/ClickHouse/pull/119697) ([Minh Vu](https://github.com/fallintoplace)). [#119817](https://github.com/ClickHouse/ClickHouse/pull/119817) ([Minh Vu](https://github.com/fallintoplace)). * Reuse HTTP connections to object storage when a read range ends at the end of the object. Streams reading the same `MergeTree` part no longer open a new connection for each range. [#118480](https://github.com/ClickHouse/ClickHouse/pull/118480) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `SELECT count()` can use the trivial count optimization on tables with lightweight-update patches that only change values, avoiding unnecessary table and patch reads. [#118882](https://github.com/ClickHouse/ClickHouse/pull/118882) ([Anton Popov](https://github.com/CurtizJ)). * Improve `Map` subcolumn pruning: `mapContainsValue` can read only the values subcolumn, `mapKeys`/`mapValues` filters can read only the matching keys or values subcolumn even when the full `Map` is also selected, and `has`/`notHas` over a `Map` can read only the keys subcolumn when it is safe to do so. [#119172](https://github.com/ClickHouse/ClickHouse/pull/119172) ([Minh Vu](https://github.com/fallintoplace)). [#119261](https://github.com/ClickHouse/ClickHouse/pull/119261) ([Minh Vu](https://github.com/fallintoplace)). [#119190](https://github.com/ClickHouse/ClickHouse/pull/119190) ([Minh Vu](https://github.com/fallintoplace)). * The `advanced` shared-data serialization for `JSON` now sizes compressed blocks by `min_compress_block_size` instead of flushing per path and substream, avoiding large part-size regressions on tables with many shared-data paths. [#119285](https://github.com/ClickHouse/ClickHouse/pull/119285) ([Pavel Kruglov](https://github.com/Avogar)). * Reduce decompression overhead for chained compression codecs by constructing codec-description ASTs only when requested. [#119490](https://github.com/ClickHouse/ClickHouse/pull/119490) ([Raufs Dunamalijevs](https://github.com/rienath)). * Accelerate `countBytesInFilter` and `countBytesInFilterWithNull` on AArch64 using direct NEON vector register accumulation, improving filter counting throughput by up to 15x. [#119555](https://github.com/ClickHouse/ClickHouse/pull/119555) ([Joshua Dorst](https://github.com/JoshyDo)). * Partition pruning now runs before loading column statistics and before calculating skip-index ordering costs for `PREWHERE` optimization. Query-plan `PREWHERE` column sizes and statistics are now also estimated from the parts left after pruning, retaining the existing byte-cost estimate when those parts have no per-column measurements. [#119558](https://github.com/ClickHouse/ClickHouse/pull/119558) ([Rory Shanks](https://github.com/rorylshanks)). [#119606](https://github.com/ClickHouse/ClickHouse/pull/119606) ([Rory Shanks](https://github.com/rorylshanks)). [#119607](https://github.com/ClickHouse/ClickHouse/pull/119607) ([Rory Shanks](https://github.com/rorylshanks)). * Speed up single-argument `uniq`, `uniqCombined`, and `uniqCombined64` on common scalar types when aggregating many interleaved groups. [#119594](https://github.com/ClickHouse/ClickHouse/pull/119594) ([Manuel](https://github.com/raimannma)). * UTF-8 validation is now vectorized across ClickHouse, not only inside the `isValidUTF8` SQL function on x86. Text index conditions, Prometheus query parsing, format escaping and output, `detectLanguage`, and `isValidUTF8` itself now validate long strings much faster, including on AArch64. Closes [#116074](https://github.com/ClickHouse/ClickHouse/issues/116074). [#116204](https://github.com/ClickHouse/ClickHouse/pull/116204) ([Groene AI](https://github.com/groeneai)). * Improved the performance of the `splitByNonAlpha` and `splitByString` tokenizers. [#119434](https://github.com/ClickHouse/ClickHouse/pull/119434) ([Anton Popov](https://github.com/CurtizJ)). * Speed up `lowerUTF8` and `upperUTF8` for mixed columns with mostly ASCII rows. [#119790](https://github.com/ClickHouse/ClickHouse/pull/119790) ([Minh Vu](https://github.com/fallintoplace)). * Speed up sorting of wide tables (`ORDER BY` over many columns) by skipping an unnecessary per-row check in the merge step when no column can be replicated. This removes overhead that was noticeable on ARM. [#110627](https://github.com/ClickHouse/ClickHouse/pull/110627) ([Groene AI](https://github.com/groeneai)). * Speed up query analysis for large parameterized views by avoiding repeated hashing of the substituted view query during query-tree comparisons. Closes [#118736](https://github.com/ClickHouse/ClickHouse/issues/118736). [#119517](https://github.com/ClickHouse/ClickHouse/pull/119517) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Improvement * AI functions (`aiGenerate`, `aiClassify`, `aiFilter`, `aiExtract`, `aiTranslate`, `aiRedact`, `aiEmbed`, `aiSimilarity`) are now beta and no longer require the `allow_experimental_ai_functions` setting, which is now obsolete. The `ai_function_*` settings moved to the beta tier, and `ai_function_max_retries` now defaults to `1` so that a single transient provider error does not fail the query. [#116227](https://github.com/ClickHouse/ClickHouse/pull/116227) ([George Larionov](https://github.com/george-larionov)). * Added dimensional metrics for `S3Queue`. [#115422](https://github.com/ClickHouse/ClickHouse/pull/115422) ([Bharat Nallan](https://github.com/bharatnc)). * Very large `max_streams_to_max_threads_ratio` values on `Merge` table reads are now validated correctly: stream-count calculations that overflow `size_t` or exceed the maximum source count are rejected with `PARAMETER_OUT_OF_BOUND` instead of hitting undefined behavior. [#113382](https://github.com/ClickHouse/ClickHouse/pull/113382) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The Web UI no longer shows sort and filter controls in column headers when the first page of a result contains only a single row and those controls would only re-run the same query. Results that are already sorted or filtered, or truncated at the display limit, still show them. [#116181](https://github.com/ClickHouse/ClickHouse/pull/116181) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A trace started by sampling (`opentelemetry_start_trace_probability`) is now written back into the query's client info, so remote and distributed secondary queries and DDL entries join the same trace instead of starting disjoint ones. [#115619](https://github.com/ClickHouse/ClickHouse/pull/115619) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * The `Empty query` syntax error now says which text was empty when the text is not the query itself but a fragment parsed on its own, such as the value of the `parallel_replicas_custom_key` or `additional_result_filter` setting: `Empty query (parallel replicas custom key)`. The error for custom-key filtering requested without `parallel_replicas_custom_key` now names the settings involved. [#115959](https://github.com/ClickHouse/ClickHouse/pull/115959) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Using `join_algorithm = 'direct'` against a `Buffer` table now returns the ordinary unsupported-storage error instead of `Cannot clone ReadFromPreparedSource plan step` (`NOT_IMPLEMENTED`). [#115309](https://github.com/ClickHouse/ClickHouse/pull/115309) ([Groene AI](https://github.com/groeneai)). * `INSERT INTO ... SELECT` from a cluster table function such as `s3Cluster` with `parallel_distributed_insert_select` now expands aliases in the `WHERE` condition before pruning files to read, so alias-based filters participate in pruning and no longer write a stack trace to the server log. [#115932](https://github.com/ClickHouse/ClickHouse/pull/115932) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Wrapped object-storage disks now account reads and writes to the workload of the underlying object storage, so workload scheduling uses the intended limits and priorities. [#116288](https://github.com/ClickHouse/ClickHouse/pull/116288) ([Mikhail Artemenko](https://github.com/Michicosun)). * A server started without a config file now declares the same single-node `default` cluster as the packaged config, so `cluster('default', ...)` and `clusterAllReplicas('default', ...)` work instead of failing with `CLUSTER_DOESNT_EXIST`. [#116382](https://github.com/ClickHouse/ClickHouse/pull/116382) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `/` commands of the client (`/help`, `/man`, `/clear`) are now offered by the line editor: typing `/` at the beginning of the input lists them as hints, and `Tab` completes the command being typed. A mistake in a command name is now reported with similar commands suggested, instead of being parsed as SQL. [#115039](https://github.com/ClickHouse/ClickHouse/pull/115039) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support `Decimal` types when writing into `Iceberg` tables, including decimal partition columns and decimal min/max statistics. [#115832](https://github.com/ClickHouse/ClickHouse/pull/115832) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Web UI: the query text selection stays visible after the editor loses focus, so it is possible to see what the `Run selected` button will run. [#116201](https://github.com/ClickHouse/ClickHouse/pull/116201) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ReplicatedMergeTree` mutations now automatically prune affected partitions from the `WHERE` condition when possible. Mutation queries also now accept multiple values in the `IN PARTITION` clause. [#110968](https://github.com/ClickHouse/ClickHouse/pull/110968) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow `JOIN` on keys that have no common supertype, such as `UInt64` and `Int64`. Previously such queries were rejected with `There is no supertype for types UInt64, Int64`. Keys are now converted to the type of the values they have in common, and a value outside that range does not match anything. [#112951](https://github.com/ClickHouse/ClickHouse/pull/112951) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduce the volume of logs from the parallel replicas coordinator: it no longer prints every part and all its mark ranges on each announcement, read request, and response. [#115833](https://github.com/ClickHouse/ClickHouse/pull/115833) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `Maybe you meant` hint for an unknown column, function, or table is now printed before the query instead of after it, so it stays visible for long queries. When a column cannot be resolved from a table expression with an alias, the message also names the table behind the alias. [#116809](https://github.com/ClickHouse/ClickHouse/pull/116809) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Comparing a number with an empty or truncated string literal, such as `WHERE numeric_column <> ''`, now reports `Cannot convert string '' to type Int32` instead of `Attempt to read after eof`. A correlated subquery returning several columns where one value is expected now also reports that directly. [#116812](https://github.com/ClickHouse/ClickHouse/pull/116812) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A `Values` row whose values are fine but whose `,` or `)` is missing or misplaced now reports the missing delimiter and the column it should follow, instead of claiming that the last value could not be parsed as its own type. [#116877](https://github.com/ClickHouse/ClickHouse/pull/116877) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * TLS CA certificates configured with `openSSL.server.caConfig` / `openSSL.client.caConfig` and `caConfig` of composable `protocols` are now reloaded on file changes and on `SYSTEM RELOAD CONFIG`, allowing CA rotation without restart for server, client, and Keeper connections. [#117387](https://github.com/ClickHouse/ClickHouse/pull/117387) ([James](https://github.com/sanjams2)). * Bare `UNION` now tells you to write `UNION ALL` or `UNION DISTINCT`, or to set `union_default_mode`, instead of naming an internal AST class. The `EXCEPT` and `INTERSECT` variants are reworded the same way. [#116815](https://github.com/ClickHouse/ClickHouse/pull/116815) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * An aggregate function used in `WHERE` or `PREWHERE`, often through an alias such as `SELECT count() AS c FROM t WHERE c > 1`, is now told to use `HAVING` instead. Values that cannot be parsed as `Date` no longer claim to be too short when that is not the reason; the error now names the expected format. [#116817](https://github.com/ClickHouse/ClickHouse/pull/116817) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Invalid calls to the internal `viewExplain` table function now return `BAD_ARGUMENTS` instead of analyzing the invalid expression and potentially failing with an unrelated error such as `UNKNOWN_TABLE`. [#108384](https://github.com/ClickHouse/ClickHouse/pull/108384) ([Christoph Viebig](https://github.com/cv4g)). * Unexpected `jemalloc` warnings are no longer suppressed, and the server now reports disabled per-CPU arenas in `system.warnings`. [#116899](https://github.com/ClickHouse/ClickHouse/pull/116899) ([Azat Khuzhin](https://github.com/azat)). * Enable reading `ReplacingMergeTree` data in reverse order with `FINAL`. [#115345](https://github.com/ClickHouse/ClickHouse/pull/115345) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `snapshot_zstd_compression_level` Keeper coordination setting to configure the `ZSTD` compression level used for snapshots. It defaults to `3`, preserving the existing behavior. [#116866](https://github.com/ClickHouse/ClickHouse/pull/116866) ([Antonio Andelic](https://github.com/antonio2368)). * A typo in the key, index, or TTL expression of an `ALTER TABLE` now suggests the column that was probably meant, the way `SELECT` already does. Previously it printed a list of available columns that, for a `MergeTree` table, was dominated by virtual columns and truncated before the real ones. [#116915](https://github.com/ClickHouse/ClickHouse/pull/116915) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Aggregate functions that take parameters and are wrapped with the `-Array` combinator now preserve those parameters correctly, so expressions such as `argMaxArray(2)` and state types such as `AggregateFunction(groupArrayMovingSum(42), UInt64)` no longer drop them. In debug and sanitizer builds this also fixes an internal assertion. [#110143](https://github.com/ClickHouse/ClickHouse/pull/110143) ([Groene AI](https://github.com/groeneai)). * `rseq` is now enabled correctly on x86-64 builds that use link-time optimization; previously it was silently disabled. [#117700](https://github.com/ClickHouse/ClickHouse/pull/117700) ([Azat Khuzhin](https://github.com/azat)). * Take a column's `CODEC` into account when estimating how many bytes a query reads from compact parts. The estimate compressed a sample with the default codec (`LZ4`) no matter how the data is really stored, so a column stored with a stronger codec was overestimated, which inflates `effective_max_reading_threads` and can make automatic parallel replicas distribute a query that is cheaper to run on a single node. [#114535](https://github.com/ClickHouse/ClickHouse/pull/114535) ([Nikita Taranov](https://github.com/nickitat)). * `Iceberg` tables stored on `S3` can now read data files from a different bucket than the table metadata. [#116771](https://github.com/ClickHouse/ClickHouse/pull/116771) ([Konstantin Vedernikov](https://github.com/scanhex12)). * An aggregate function with the `-If` combinator whose condition has the wrong type, or is missing, now says that the last argument is the condition and must be `UInt8`, instead of only reporting an illegal type for the last argument. An `ASOF` join with no inequality predicate now says that one is needed in addition to the equality predicates, instead of ending the message in `, in .`. [#116916](https://github.com/ClickHouse/ClickHouse/pull/116916) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-local` now exposes `system.warnings`, and ClickHouse now checks `rseq` compatibility at startup. [#117717](https://github.com/ClickHouse/ClickHouse/pull/117717) ([Azat Khuzhin](https://github.com/azat)). * The `TimeSeries` table engine, the PromQL dialect and the `timeSeries*` aggregate functions are moved from the experimental tier to the private preview tier. Settings `allow_experimental_time_series_table` and `allow_experimental_time_series_aggregate_functions` are renamed to `enable_time_series_table` and `enable_time_series_aggregate_functions`; the old names remain as aliases. [#118124](https://github.com/ClickHouse/ClickHouse/pull/118124) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * `TimeSeries` tables now store an explicit schema version in the `version` engine setting. The PromQL execution layer supports only a declared range of versions and rejects queries over tables outside that range with an instructive error. [#111204](https://github.com/ClickHouse/ClickHouse/pull/111204) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). A `TimeSeries` table definition copied with `CREATE TABLE ... AS` now gets the settings of the new table, and the new `id_type` setting lets `CREATE TABLE ... AS src` copy only `src`'s definition without reading its target tables. [#118517](https://github.com/ClickHouse/ClickHouse/pull/118517) ([Vitaly Baranov](https://github.com/vitlibar)). [#119613](https://github.com/ClickHouse/ClickHouse/pull/119613) ([Vitaly Baranov](https://github.com/vitlibar)). * Reads from `TimeSeries` tables now respect the `join_algorithm` and the partition-wise aggregation settings of the query instead of overriding them, and the internal join of samples and tags supports `full_sorting_merge`. [#118864](https://github.com/ClickHouse/ClickHouse/pull/118864) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). The `promql_evaluation_time` setting defaults to `auto` regardless of the `compatibility` setting. [#119202](https://github.com/ClickHouse/ClickHouse/pull/119202) ([Raúl Marín](https://github.com/Algunenano)). * Join runtime filter settings are now production-tier (previously experimental). [#117383](https://github.com/ClickHouse/ClickHouse/pull/117383) ([Robert Schulze](https://github.com/rschu1ze)). * Improved syntax error messages. An unclosed bracket is now reported at the place where the parser stopped, which is next to the actual mistake, and every unclosed bracket is listed with its own position; previously the outermost bracket was blamed, which in a large query can be thousands of characters away from the mistake. A mistyped keyword is now named explicitly (`Maybe you meant: SELECT.`) instead of only being one item in a list of dozens of alternatives. [#116806](https://github.com/ClickHouse/ClickHouse/pull/116806) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `psql`-style `\d` of the interactive client now describes a table when a table name is given (`\d hits`), like it does in `psql`; without an argument it still lists the tables. [#118281](https://github.com/ClickHouse/ClickHouse/pull/118281) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-local` now merges the `config.d` and `conf.d` directories of the current directory into its configuration even if there is no main configuration file, so a directory with a `config.d` no longer additionally requires an empty `config.xml`. [#118410](https://github.com/ClickHouse/ClickHouse/pull/118410) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `clickhouse_version` and `system_processor` columns to the system log tables. [#117896](https://github.com/ClickHouse/ClickHouse/pull/117896) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * `EXPLAIN ANALYZE` now shows more information about the estimation for joins. [#114792](https://github.com/ClickHouse/ClickHouse/pull/114792) ([Kirill Kopnev](https://github.com/Fgrtue)). * Automatic parallel replicas now support queries with `LIMIT` and `OFFSET`. [#117402](https://github.com/ClickHouse/ClickHouse/pull/117402) ([Nikita Taranov](https://github.com/nickitat)). * Object storage writes (`INSERT` into `s3`, `azure` and other object storage tables) now support parallel replicas. [#119214](https://github.com/ClickHouse/ClickHouse/pull/119214) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added the `min_partition_age_to_force_merge_seconds` `MergeTree` setting: when every part in a partition is at least this old, the `Simple` and `StochasticSimple` merge selectors compact that partition's parts within the `max_bytes_to_merge_at_max_space_in_pool` limit. [#115547](https://github.com/ClickHouse/ClickHouse/pull/115547) ([Valery Petrov](https://github.com/valerypetrov)). * `TRUNCATE TABLE` on a `ReplicatedMergeTree` table no longer blocks concurrent `DROP`, `RENAME` and other DDL queries on the same table while the data is being removed. [#114672](https://github.com/ClickHouse/ClickHouse/pull/114672) ([Nikolay Degterinsky](https://github.com/evillique)). * Allow `ALTER TABLE ... MODIFY SETTING storage_policy` for `MergeTree` tables when newly added disks contain only a matching `format_version.txt` and an empty `detached/` directory under the table path. [#107736](https://github.com/ClickHouse/ClickHouse/pull/107736) ([Maxim Orlovsky](https://github.com/UberDever)). * Added the `s3_upload_checksum_algorithm` setting to choose the checksum sent with `S3` uploads (`MD5`, `CRC32` or `SHA256`). The default is empty and keeps the current behavior. Use `CRC32` or `SHA256` on FIPS builds, where `MD5` is unavailable and uploads would otherwise carry no checksum, which buckets with Object Lock reject. [#107318](https://github.com/ClickHouse/ClickHouse/pull/107318) ([Sebastian Vercruysse](https://github.com/sebver)). * Partition field summaries are written into the manifest list of `Iceberg` tables. [#118796](https://github.com/ClickHouse/ClickHouse/pull/118796) ([Konstantin Vedernikov](https://github.com/scanhex12)). Concurrent `INSERT`s into an `Iceberg` table through a catalog no longer back off on a commit conflict and retry immediately. [#119238](https://github.com/ClickHouse/ClickHouse/pull/119238) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Log when an object storage listing returns an empty page while reporting that more objects follow. This is legal and is handled correctly, but it has twice been the trigger for silently losing objects and nothing recorded that it occurred. [#119065](https://github.com/ClickHouse/ClickHouse/pull/119065) ([Sema Checherinda](https://github.com/CheSema)). * A warning is now logged when a CTE is declared as `MATERIALIZED` but the setting `enable_materialized_cte` is disabled, in which case the keyword is ignored and the CTE is inlined at each reference. Previously the keyword was silently ignored. [#118032](https://github.com/ClickHouse/ClickHouse/pull/118032) ([Dmitry Novik](https://github.com/novikd)). * An unknown profile event name in the `trace_profile_events_list` setting now gives a readable error message with hints instead of `std::out_of_range: unordered_map::at: key not found`. [#116049](https://github.com/ClickHouse/ClickHouse/pull/116049) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The dictionary loader now logs `never` instead of a timestamp in the year 294247 when an object has no automatic update scheduled (for example a dictionary with `LIFETIME(0)`), which previously looked like a scheduling bug next to a failed-update message. [#118443](https://github.com/ClickHouse/ClickHouse/pull/118443) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Correct the error message for an unsupported attribute in an `ip_trie` dictionary: it claimed that `Array` attributes are unsupported, while in fact they are supported and only `Nullable` attributes are rejected. [#118492](https://github.com/ClickHouse/ClickHouse/pull/118492) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `MEMORY_LIMIT_EXCEEDED` is reported promptly for `parallel_hash` joins whose hash-table construction uses deferred memory tracking. [#116477](https://github.com/ClickHouse/ClickHouse/pull/116477) ([Konstantin Morozov](https://github.com/k-morozov)). * Less verbose logging of `INSERT`s into `MergeTree` and `ReplicatedMergeTree` tables. [#116486](https://github.com/ClickHouse/ClickHouse/pull/116486) ([Azat Khuzhin](https://github.com/azat)). * The hash tables behind the TLD lists (`firstSignificantSubdomainCustom` and similar functions), the naive Bayes classifier and text indexes use a simpler, packed string representation. [#117175](https://github.com/ClickHouse/ClickHouse/pull/117175) ([Surya Teja](https://github.com/sleepingeight)). * ClickHouse Keeper now counts the per-entry allocation overhead in `latest_logs_cache_size_threshold`, `KeeperLatestLogsCacheSize`, and `lgif`'s `latest_logs_cache_size`, so the configured limit tracks cache memory rather than only serialized payload bytes. [#119045](https://github.com/ClickHouse/ClickHouse/pull/119045) ([Nikita Taranov](https://github.com/nickitat)). The `latest_logs_cache_entry_count_threshold` setting is effective again and limits the in-memory latest-log cache by entry count independently of `latest_logs_cache_size_threshold`. [#119151](https://github.com/ClickHouse/ClickHouse/pull/119151) ([Antonio Andelic](https://github.com/antonio2368)). * Show the progress percentage again for queries reading from the `file` table function and `INFILE`. [#119213](https://github.com/ClickHouse/ClickHouse/pull/119213) ([Raúl Marín](https://github.com/Algunenano)). * Warn when `CLEAR COLUMN` leaves an `EPHEMERAL`-derived `MATERIALIZED` column stale. [#115371](https://github.com/ClickHouse/ClickHouse/pull/115371) ([Shaohua Wang](https://github.com/tiandiwonder)). * Reduced cancellation latency for `INTERSECT` and `EXCEPT` queries: `KILL QUERY` now interrupts the hash-table build and filter loops within a single chunk instead of waiting for the whole chunk. [#118046](https://github.com/ClickHouse/ClickHouse/pull/118046) ([Roman Vasin](https://github.com/rvasin)). * The `compatibility` setting can now keep selected new defaults in effect: settings changes marked `StartUsingNew` are no longer rolled back to the previous default when `compatibility` points to an older version. [#114190](https://github.com/ClickHouse/ClickHouse/pull/114190) ([Mikhail Artemenko](https://github.com/Michicosun)). * Added write support for `DeltaLake` tables on `Azure` storage. [#116698](https://github.com/ClickHouse/ClickHouse/pull/116698) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Added aliases without the `allow_experimental_` prefix for existing settings whose tier already marks them as experimental or beta; the old names continue to work. [#119056](https://github.com/ClickHouse/ClickHouse/pull/119056) ([Robert Schulze](https://github.com/rschu1ze)). * Bump the bundled `libssh` from `0.12.0` to `0.12.2`. [#119262](https://github.com/ClickHouse/ClickHouse/pull/119262) ([Robert Schulze](https://github.com/rschu1ze)). * Function `replicate` now returns a constant column when all its arguments are constant, like `arrayWithConstant`. [#111069](https://github.com/ClickHouse/ClickHouse/pull/111069) ([sakshichitnis27](https://github.com/sakshichitnis27)). * The `PROJECTION_NOT_USED` and `INCORRECT_DATA` errors of `force_optimize_projection` and `force_optimize_projection_name` now list every projection the optimizer considered and why each was rejected. [#119624](https://github.com/ClickHouse/ClickHouse/pull/119624) ([Mikhail Artemenko](https://github.com/Michicosun)). #### Bug Fix (user-visible misbehavior in an official stable release) * `ALTER TABLE ATTACH PARTITION FROM` is no longer replicated in `Replicated` databases. [#94835](https://github.com/ClickHouse/ClickHouse/pull/94835) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * Invalid Azure upload-size settings (for example `azure_min_upload_part_size = 0` or `azure_max_blocks_in_multipart_upload = 0`) are now rejected early with a clear error instead of triggering an internal exception (`second_size > 0`) deep in the write path. This applies to both query settings and Azure disk configuration. [#103394](https://github.com/ClickHouse/ClickHouse/pull/103394) ([Statxc](https://github.com/statxc)). * Fixed wrong results for comma or `CROSS JOIN` queries whose `WHERE` clause compares columns from the two sides through a non-deterministic expression such as `rand`. Such queries could return rows that contradict their own `WHERE` clause because rewriting the join to `INNER JOIN` re-evaluated the expression per row on each side. [#112938](https://github.com/ClickHouse/ClickHouse/pull/112938) ([Groene AI](https://github.com/groeneai)). * `S3Queue` and `AzureQueue` tables whose metadata handle is gone no longer fail with `Logical error: 'Files metadata is empty'` when read, altered, or flushed; they now report `TABLE_IS_DROPPED` instead. `SYSTEM FLUSH OBJECT STORAGE QUEUE` on such a table no longer terminates the server. [#113695](https://github.com/ClickHouse/ClickHouse/pull/113695) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` when filtering an `Iceberg` column that `ALTER TABLE ... MODIFY COLUMN` made `Nullable`. Closes [#85029](https://github.com/ClickHouse/ClickHouse/issues/85029). [#114521](https://github.com/ClickHouse/ClickHouse/pull/114521) ([Groene AI](https://github.com/groeneai)). * PromQL grouping modifiers now support quoted label names. [#114545](https://github.com/ClickHouse/ClickHouse/pull/114545) ([Minh Vu](https://github.com/fallintoplace)). * Fixed reading a table while a mutation is in progress: a `MATERIALIZED` column could return a stale or incorrectly recomputed value, and selecting it together with an updated column could fail with an error. [#114561](https://github.com/ClickHouse/ClickHouse/pull/114561) ([Shaohua Wang](https://github.com/tiandiwonder)). [#115372](https://github.com/ClickHouse/ClickHouse/pull/115372) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed an RBAC bypass in the `Alias` table engine that allowed users without privileges on the target table to reveal its schema, row count, size, and existence. [#114596](https://github.com/ClickHouse/ClickHouse/pull/114596) ([Kai Zhu](https://github.com/nauu)). * Fixed a column that was renamed, dropped, and added again in one `ALTER` returning the dropped column's values instead of its own default while the mutation was still pending. [#114601](https://github.com/ClickHouse/ClickHouse/pull/114601) ([Shaohua Wang](https://github.com/tiandiwonder)). * `ANY RIGHT JOIN` and `SEMI RIGHT JOIN` with several `OR`-ed conditions in the `ON` clause no longer duplicate some right-table rows and lose others. A written `ANY LEFT JOIN` was affected too, because the planner may swap the tables and execute it as `ANY RIGHT JOIN`. [#114676](https://github.com/ClickHouse/ClickHouse/pull/114676) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix invalid backslash escape handling in text, `tokenbf_v1`, and `ngrambf_v1` indexes. [#115634](https://github.com/ClickHouse/ClickHouse/pull/115634) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix crashes of the `Parquet`, `ORC`, and `Iceberg` readers on malformed input: a `DELTA_BYTE_ARRAY` page with an empty length stream, a `DELTA_BYTE_ARRAY` page of a `Decimal` column read under a filter, an out-of-range enum in an `Iceberg` manifest file, and an oversized allocation in the ORC library. [#115703](https://github.com/ClickHouse/ClickHouse/pull/115703) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject invalid names of projections and indexes. [#115824](https://github.com/ClickHouse/ClickHouse/pull/115824) ([Alexander Tokmakov](https://github.com/tavplubix)). * A `KeeperMap` table with unparsable metadata no longer prevents server startup; it is now quarantined instead. Closes [#115881](https://github.com/ClickHouse/ClickHouse/issues/115881). [#115941](https://github.com/ClickHouse/ClickHouse/pull/115941) ([Groene AI](https://github.com/groeneai)). * Fixed `LOGICAL_ERROR: Index with name auto_minmax_index_ already exists` when a single `ALTER TABLE` statement combined `RENAME COLUMN` with another command, such as `MODIFY SETTING`, on a table with implicit minmax indices (`add_minmax_index_for_numeric_columns` and friends). Renaming a plain column also no longer leaves a following `DROP COLUMN` failing with `UNKNOWN_IDENTIFIER`, nor a following `ADD COLUMN` without its implicit index. [#116063](https://github.com/ClickHouse/ClickHouse/pull/116063) ([Groene AI](https://github.com/groeneai)). * Fixed a crash of stateless workers with object storage. [#116067](https://github.com/ClickHouse/ClickHouse/pull/116067) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed distributed queries that filter or compute on `Decimal`, `DateTime64`, or `Time64` constants: pushed-down constants could fail with `CANNOT_PARSE_DATETIME`, lose precision, or change their `Dynamic` subtype when sent to remote shards. [#95055](https://github.com/ClickHouse/ClickHouse/pull/95055) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fixed `JSON`, `Object`, and `Dynamic` columns hashing logically equal values differently depending on their physical layout: how `JSON`/`Object` paths were split between dynamic subcolumns and shared data, or whether a `Dynamic` value sat in a typed or shared variant. Hash-based joins on such a key silently missed matches, and a `grace_hash` spill could raise a `LOGICAL_ERROR` (`Invalid state transition`). A `JSON` key needed no non-default setting; `Dynamic` also required `allow_dynamic_type_in_join_keys`. [#109450](https://github.com/ClickHouse/ClickHouse/pull/109450) ([Groene AI](https://github.com/groeneai)). * Setting a Keeper coordination timeout such as `operation_timeout_ms`, `session_timeout_ms`, or `startup_timeout` to a very large value no longer overflows and expires the wait immediately. Builds with the undefined behavior sanitizer no longer abort. [#112088](https://github.com/ClickHouse/ClickHouse/pull/112088) ([Groene AI](https://github.com/groeneai)). * Fixed a `bloom_filter` skip index dropping granules that contain matching rows when a `String` or `FixedString` column is compared for equality against a constant that is wider than the indexed type, so queries such as `SELECT count() FROM t WHERE s = toFixedString('abc', 5)` no longer silently return too few rows. [#112639](https://github.com/ClickHouse/ClickHouse/pull/112639) ([Groene AI](https://github.com/groeneai)). * Fixed a large `max_concurrent_threads_ratio_to_cores` value in `CREATE WORKLOAD` producing an unspecified concurrent-thread limit. The ratio is now clamped before narrowing, so such a value means "no limit". [#113206](https://github.com/ClickHouse/ClickHouse/pull/113206) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when plan-based parallel replicas shipped a read pinned by `select_sequential_consistency`. Follower replicas rebuilt the read without the pinned block-number boundary and returned rows the insert quorum never confirmed. Such a read is now executed locally. [#114263](https://github.com/ClickHouse/ClickHouse/pull/114263) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Parquet` files whose offset index does not start at row 0. Such a file could make the native `Parquet` reader return wrong rows, or report `LOGICAL_ERROR` instead of `INCORRECT_DATA`. The offset index is now validated when it is read. [#114530](https://github.com/ClickHouse/ClickHouse/pull/114530) ([Groene AI](https://github.com/groeneai)). * Fix query condition cache poisoning by `JOIN` queries with `query_plan_join_shard_by_pk_ranges = 1`, which made later queries with the same `WHERE` condition return too few rows. [#114827](https://github.com/ClickHouse/ClickHouse/pull/114827) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed AI functions under-reporting provider usage in `system.query_log` and in per-query quotas: counters accumulated before a failed request were dropped, and tokens for a response the provider billed for but that failed validation were never counted. [#114905](https://github.com/ClickHouse/ClickHouse/pull/114905) ([George Larionov](https://github.com/george-larionov)). * Fixed `Coordination mode mismatch for stream
: got Default, expected WithOrder` when a parameterized view is used as an `IN` subquery of an ordered query with parallel replicas enabled. A parameterized view used this way no longer plans its inner read with parallel replicas coordination on a replica. [#114916](https://github.com/ClickHouse/ClickHouse/pull/114916) ([Groene AI](https://github.com/groeneai)). * Fix asynchronous inserts into `JSON` columns ignoring the `max_dynamic_subcolumns_in_json_type_parsing` setting. Previously, an asynchronous insert (`async_insert = 1`) could store more dynamic paths than the setting allows, giving a different column layout than a synchronous insert with the same settings. The cap is now preserved when asynchronous inserts aggregate parsed data. [#115329](https://github.com/ClickHouse/ClickHouse/pull/115329) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Validate the discriminator of deserialized `Variant` values, so malformed data is rejected instead of being read out of bounds. [#115443](https://github.com/ClickHouse/ClickHouse/pull/115443) ([Azat Khuzhin](https://github.com/azat)). * Fix `UNKNOWN_IDENTIFIER` when creating a materialized view whose query uses an expression alias as the right argument of `IN`. Closes [#94894](https://github.com/ClickHouse/ClickHouse/issues/94894). [#115651](https://github.com/ClickHouse/ClickHouse/pull/115651) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `RESTORE ... AS ` leaving the `REFRESH ... DEPENDS ON` list of a refreshable materialized view pointing at the original database. The restored view refreshed in response to a table it no longer read, ignored refreshes of its own parent, and stopped refreshing entirely once the original database was dropped. [#115689](https://github.com/ClickHouse/ClickHouse/pull/115689) ([Groene AI](https://github.com/groeneai)). * Validate the deserialized states of `stochasticLinearRegression`, `stochasticLogisticRegression`, `groupUniqArray`, `groupArrayIntersect`, and `topK`, and of the `Resample` combinator, so malformed data is rejected instead of being read out of bounds. [#115701](https://github.com/ClickHouse/ClickHouse/pull/115701) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Mask the `aws_external_id` setting of the `DataLakeCatalog` database engine in `SHOW CREATE DATABASE` and `system.databases.engine_full`. It was previously printed in plaintext, unlike the other AWS credential settings. Closes [#114999](https://github.com/ClickHouse/ClickHouse/issues/114999). [#115791](https://github.com/ClickHouse/ClickHouse/pull/115791) ([Groene AI](https://github.com/groeneai)). * Fixed `SELECT count()` never terminating, or returning an invented row count, on a corrupted `MsgPack` or `ProtobufList` file. The `optimize_count_from_files` fast path counted rows without requiring the read position to advance, so a two-byte `ProtobufList` file produced billions of rows and never returned. Such input is now rejected with the same error the ordinary read path gives. [#115884](https://github.com/ClickHouse/ClickHouse/pull/115884) ([Groene AI](https://github.com/groeneai)). * Fixed `ATTACH TABLE ... AS REPLICATED` and the `convert_to_replicated` flag file accepting a conversion whose ZooKeeper path is not a safe path. On a server whose `default_replica_path` substitutes `{database}` or `{table}`, a name containing `/` used to resolve to a path inside another table's subtree, letting the converted table register a replica under a table it has no privileges on. Such a conversion is now rejected with `BAD_ARGUMENTS`, the same way an equivalent `CREATE` already was. Closes [#115967](https://github.com/ClickHouse/ClickHouse/issues/115967). [#115991](https://github.com/ClickHouse/ClickHouse/pull/115991) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when joining against a view or `view` whose projected column has the same name as a column of one of its own inner relations, for example `SELECT expr(x.c0) AS c0 FROM a AS x, b AS y`. The join order optimizer merged such an expression into the flattened join graph and then applied it twice, so the query silently returned skewed values. When the expression also changed the column type, the same cause produced `AMBIGUOUS_COLUMN_NAME` at plan time instead. Closes [#116036](https://github.com/ClickHouse/ClickHouse/issues/116036). [#116152](https://github.com/ClickHouse/ClickHouse/pull/116152) ([Groene AI](https://github.com/groeneai)). * Listing `tlsv1_3` in `` (or the `client` equivalent) now actually disables TLS 1.3. Previously the token was dropped with no error or log line and the server kept negotiating TLS 1.3, so an operator who disabled it believed TLS 1.3 was off while it was not. Closes [#79876](https://github.com/ClickHouse/ClickHouse/issues/79876). [#116170](https://github.com/ClickHouse/ClickHouse/pull/116170) ([Groene AI](https://github.com/groeneai)). * Fixed `Invalid number of columns in chunk pushed to OutputPort` (`LOGICAL_ERROR`) when a `SELECT ... FROM t STREAM ... ORDER BY ... LIMIT n` query read a table with a `PROJECTION`. Lazy materialization is no longer applied to a `STREAM` read. [#116187](https://github.com/ClickHouse/ClickHouse/pull/116187) ([Groene AI](https://github.com/groeneai)). * Fixed `system.detached_tables` failing with `NOT_IMPLEMENTED` when the server has databases such as `Dictionary`, `Filesystem`, `HDFS`, `Overlay`, or `S3`. Unsupported database engines are now skipped during iteration. Closes [#104868](https://github.com/ClickHouse/ClickHouse/issues/104868). [#107943](https://github.com/ClickHouse/ClickHouse/pull/107943) ([Aditya Kumar](https://github.com/adityaksolves)). * Skip the Query Condition Cache for object-storage files whose `ETag` is not a strong content identifier, such as `HDFS`. Previously a same-second, same-size overwrite could reuse stale skip marks and silently drop matching rows. Closes [#112016](https://github.com/ClickHouse/ClickHouse/issues/112016). [#112051](https://github.com/ClickHouse/ClickHouse/pull/112051) ([Zhang Yifan](https://github.com/zhangyifan27)). * Fixed text index functions `hasToken`, `hasAnyTokens`, `hasAllTokens`, and `hasPhrase` outside `WHERE`/`PREWHERE`: they now apply the index's tokenizer and pre/postprocessing in the `SELECT` list, aggregate arguments, and with `use_skip_indexes = 0`, so results match the index path. Closes [#108808](https://github.com/ClickHouse/ClickHouse/issues/108808). [#113154](https://github.com/ClickHouse/ClickHouse/pull/113154) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix wrong results for a query that calls the same table function twice with arguments that differ only in a part of the query that was not taken into account by the AST hash, such as `UNION ALL` against `UNION DISTINCT`, `INTERSECT` against `EXCEPT`, `LIMIT` against `OFFSET`, or the bounds of a window frame. The second call reused the result of the first one. [#114171](https://github.com/ClickHouse/ClickHouse/pull/114171) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `THERE_IS_NO_COLUMN` / `NOT_FOUND_COLUMN_IN_BLOCK` when a filter that runs after `FINAL` is a plain column reference. This affected an explicit `PREWHERE b` under `FINAL` and, with default settings, a row policy whose expression is a bare column. [#114578](https://github.com/ClickHouse/ClickHouse/pull/114578) ([Groene AI](https://github.com/groeneai)). * Fix loading old `S3` tables with globbed paths and `PARTITION BY` after upgrading from versions before 26.6. [#115277](https://github.com/ClickHouse/ClickHouse/pull/115277) ([Bharat Nallan](https://github.com/bharatnc)). * Fix `Iceberg` identity-partitioned reads: partition values are now taken from partition metadata instead of returned as `NULL`. Closes [#110216](https://github.com/ClickHouse/ClickHouse/issues/110216). [#116129](https://github.com/ClickHouse/ClickHouse/pull/116129) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `KeeperMap` now writes single-expression primary keys without redundant outer parentheses, so metadata created by newer servers remains readable by ClickHouse 26.4 during supported rolling upgrades and rollbacks. [#116369](https://github.com/ClickHouse/ClickHouse/pull/116369) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fixed `Kafka` tables using `kafka_sasl_mechanism = 'AWS_MSK_IAM'` failing authentication once an hour with `SASL authentication error: Access denied`. The `OAUTHBEARER` token lifetime advertised to `librdkafka` is now capped by the remaining validity of the AWS credentials that signed the token, so the next token refresh is always scheduled before those credentials expire. Successful token refreshes are now logged, and unhelpful ClickHouse-side stack traces are omitted from broker-originated SASL errors in `system.kafka_consumers`. [#109125](https://github.com/ClickHouse/ClickHouse/pull/109125) ([kalavt](https://github.com/kalavt)). * Fixed silent truncation of `DateTime64` and `Time64` values with a scale above 6 when `date_time_64_output_format_cut_trailing_zeros_align_to_groups_of_thousands` is enabled. The printed fraction was capped at 6 digits, so a `DateTime64(9)` value of `00:00:00.000000700` rendered as `00:00:00.000000` even though the stored value was not zero. The documented output range of the setting widens from `[0, 3, 6]` to `[0, 3, 6, 9]`. [#112854](https://github.com/ClickHouse/ClickHouse/pull/112854) ([Groene AI](https://github.com/groeneai)). * Fixed the `dpsub` and `dpsize` join-order optimizations (`query_plan_optimize_join_order_algorithm = 'dpsub'` / `'dpsize'`) silently dropping single-table filter conditions from a `JOIN ... ON` clause, which could return extra rows. [#114629](https://github.com/ClickHouse/ClickHouse/pull/114629) ([Fisnik Kastrati](https://github.com/fkastrati)). [#118126](https://github.com/ClickHouse/ClickHouse/pull/118126) ([Fisnik Kastrati](https://github.com/fkastrati)). * `ZooKeeper` multi requests now reject unsupported and mixed read/write operations. [#115768](https://github.com/ClickHouse/ClickHouse/pull/115768) ([Antonio Andelic](https://github.com/antonio2368)). * You can now read `Iceberg` v2 tables whose manifests omit the optional `snapshot_id` column, such as tables managed by BigQuery; reading one previously failed with `Cannot find column snapshot_id`. A manifest that omits `sequence_number` is also no longer read as sequence number `0`. Both values are inherited from the manifest list, as the `Iceberg` spec requires. [#116040](https://github.com/ClickHouse/ClickHouse/pull/116040) ([Takayuki Enomoto](https://github.com/tkykenmt)). * Fixed `_path` path normalization for globbed files and object-storage reads, so `WHERE _path = ''` no longer misses rows because of redundant or leading separators. Globbed reads through symlinks and `..` names also no longer read or, with `rename_files_after_processing`, rename the wrong file. [#116244](https://github.com/ClickHouse/ClickHouse/pull/116244) ([Groene AI](https://github.com/groeneai)). [#116252](https://github.com/ClickHouse/ClickHouse/pull/116252) ([Groene AI](https://github.com/groeneai)). * Fixed wrong window function results for `PARTITION BY` over a floating-point key (also `JSON`, `Dynamic`, and these types nested in `Nullable`, `LowCardinality`, `Array`, `Tuple`, `Map`, or `Variant`) with `max_threads > 1`: values that compare as equal, such as `-0.` and `0.`, could hash into different threads, and one logical partition got several window frames. [#116315](https://github.com/ClickHouse/ClickHouse/pull/116315) ([Alexander Gololobov](https://github.com/davenger)). * A `Native`-protocol client can now forward an unknown setting to the server by name. Previously, when a client sent such a setting as a custom setting, the server left the real setting unchanged and reported it at its default value in the settings received with the query. [#116317](https://github.com/ClickHouse/ClickHouse/pull/116317) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed `h3Line` returning uninitialized memory in the tail of the result array when a line between two valid H3 indices cannot be drawn because it crosses pentagon distortion. Such a call now throws `INCORRECT_DATA`. [#116583](https://github.com/ClickHouse/ClickHouse/pull/116583) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE ... AS
` no longer accepts a `PRIMARY KEY` in its column list. Previously, such a statement wrote table metadata that the server could not parse back on the next start, so the database failed to load with a syntax error. [#116645](https://github.com/ClickHouse/ClickHouse/pull/116645) ([Groene AI](https://github.com/groeneai)). * Fixed wrong row counts and dropped rows when a `DateTime64` primary-key column is filtered through `toStartOfDay`, `toRelativeSecondNum`, `toRelativeMinuteNum`, `toRelativeHourNum`, `toRelativeDayNum`, `toRelativeWeekNum`, `toMonthNumSinceEpoch`, or `toYearNumSinceEpoch` with values outside the `UInt32`-seconds range (before 1970 or beyond 2106). These functions claimed to be always monotonic to the primary index, but their standard-precision results wrapped for out-of-range `DateTime64`, breaking primary-key pruning. [#110344](https://github.com/ClickHouse/ClickHouse/pull/110344) ([Groene AI](https://github.com/groeneai)). * Fix regressions around `use_legacy_to_time`: `MergeTree` key expressions using `toTime` that were created with the legacy semantics now work again, and `CREATE TABLE ... AS SELECT`, `CREATE TABLE ... AS
`, and `CREATE MATERIALIZED VIEW` no longer fail or silently flatten `Nested` columns when `use_legacy_to_time = 1`. Closes [#117001](https://github.com/ClickHouse/ClickHouse/issues/117001). [#110958](https://github.com/ClickHouse/ClickHouse/pull/110958) ([Groene AI](https://github.com/groeneai)). [#116953](https://github.com/ClickHouse/ClickHouse/pull/116953) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `OPTIMIZE TABLE ... MANIFEST` on `Iceberg` tables: tables whose metadata file omits the spec-optional top-level `refs` object no longer fail with `Poco::NullPointerException`, and an experimental `Iceberg` table whose rows had all been deleted no longer crashes. Closes [#112082](https://github.com/ClickHouse/ClickHouse/issues/112082). [#112163](https://github.com/ClickHouse/ClickHouse/pull/112163) ([Groene AI](https://github.com/groeneai)). [#111241](https://github.com/ClickHouse/ClickHouse/pull/111241) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix a server crash during a `JOIN` that spills to disk. When an allocation failed while the join rehashed its buckets, the in-memory join was left either holding released data or absent altogether, and another build thread could dereference it. [#113410](https://github.com/ClickHouse/ClickHouse/pull/113410) ([Groene AI](https://github.com/groeneai)). * Fix wrong statistics-part pruning for negated floating-point predicates with `NaN`. [#113507](https://github.com/ClickHouse/ClickHouse/pull/113507) ([Han Fei](https://github.com/hanfei1991)). * Fix a spurious `PreconditionFailed` (`Code: 499`) error when creating or committing to an `Iceberg` table. If the object store drops the response to the metadata write after the object was written, the retry of that conditional write hits its own object and the query fails even though the write succeeded. Also stop a conditional multipart completion from reporting success over an object it did not write, silently losing the write it was meant to reject. [#113994](https://github.com/ClickHouse/ClickHouse/pull/113994) ([Groene AI](https://github.com/groeneai)). * Fixed lightweight `UPDATE` on `ReplicatedMergeTree` being rejected with `TIMEOUT_EXCEEDED` after a few seconds under concurrent updates, long before the configured `lock_acquire_timeout` elapsed. The Keeper lock acquisition loop was bounded by a fixed number of tries instead of by the timeout, so raising `lock_acquire_timeout` had no effect. [#114125](https://github.com/ClickHouse/ClickHouse/pull/114125) ([Groene AI](https://github.com/groeneai)). * Fix `ORDER BY ... LIMIT` queries on `Tuple()` columns raising a `NOT_IMPLEMENTED` exception when top-K dynamic filtering is enabled. Direct empty tuples now skip the optimization, while supported nested composite types continue to use top-K filtering. [#114182](https://github.com/ClickHouse/ClickHouse/pull/114182) ([Langning Zhang](https://github.com/Boulea7)). * Fix `Iceberg` tables partitioned by `toRelativeDayNum` or `toRelativeHourNum` publishing the non-canonical transform names `days` and `hours` in metadata. ClickHouse now writes the spec-defined singular names `day` and `hour`, so external readers such as `pyiceberg` can parse the partition spec. [#114864](https://github.com/ClickHouse/ClickHouse/pull/114864) ([yanglongwei](https://github.com/ylw510)). * Fix the server failing to start when `listen_host` contains both `::` and `0.0.0.0` and `arrowflight_port` is configured, and fix two gRPC listeners silently sharing one port in the same configuration. `listen_try` now also covers the gRPC and Arrow Flight protocols, which bind their listening socket when they are started. [#115074](https://github.com/ClickHouse/ClickHouse/pull/115074) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a case where expensive ZooKeeper exception logging could delay session finalization and leave pending requests unresolved. [#115396](https://github.com/ClickHouse/ClickHouse/pull/115396) ([Antonio Andelic](https://github.com/antonio2368)). * An invalid `http_handlers` configuration is now reported with its own error instead of being wrapped into `NETWORK_ERROR "Listen [host]:port failed"`. Previously, when `listen_try` was in effect, the error was only logged as a warning about `listen_host` and the server started without serving HTTP at all. The same applies to the Prometheus listener on `prometheus.port`, to the Keeper control endpoints on `keeper_server.http_control`, and to `SYSTEM START LISTEN HTTP` in `clickhouse-local`, which blamed `listen_host` and `http_port`. [#115548](https://github.com/ClickHouse/ClickHouse/pull/115548) ([Groene AI](https://github.com/groeneai)). * Fixed zero-precision `DateTime64` and `Time64` text conversions returning `DateTime` or `Time` instead of `DateTime64(0)` or `Time64(0)`: `toDateTime64OrNull`, `toDateTime64OrZero`, `parseDateTime64BestEffort`, `parseDateTime64BestEffortOrNull`, `parseDateTime64BestEffortOrZero`, `parseDateTime64BestEffortUS`, `parseDateTime64BestEffortUSOrNull`, `parseDateTime64BestEffortUSOrZero`, `toTime64OrZero`, and `toTime64OrNull`. This previously prevented conversion of dates before `1970-01-01` and made the `Time64` functions silently lose values that `toTime64` parses. [#115618](https://github.com/ClickHouse/ClickHouse/pull/115618) ([Mark Needham](https://github.com/mneedham)). [#116979](https://github.com/ClickHouse/ClickHouse/pull/116979) ([Navneet Kumar](https://github.com/navneet83)). * Fixed a server startup failure after creating an `Alias` table whose target did not exist yet and later became an `Alias` table itself. Such a table was accepted by `CREATE`, but rejected while loading metadata, which aborted startup with `BAD_ARGUMENTS`. The `Alias` target restrictions are now applied only to freshly supplied definitions. [#116153](https://github.com/ClickHouse/ClickHouse/pull/116153) ([Groene AI](https://github.com/groeneai)). * Fix `SHOW CREATE TABLE` failing instead of describing the table when the table's storage object cannot be opened, for example a `DataLakeCatalog` table the catalog reports as unreadable. The `Alias` target-access check now consults the engine name of the create query instead of opening the table. [#116231](https://github.com/ClickHouse/ClickHouse/pull/116231) ([Groene AI](https://github.com/groeneai)). * Fix `ProfileEvents['SelectedRows']` and `ProfileEvents['SelectedBytes']` being counted twice for reads from file, object-storage, data lake, queue, and executable table engines and table functions, and for executable dictionaries and UDFs. They now agree with `read_rows` and `read_bytes`, as does the `Read:` line of `EXPLAIN ANALYZE`. [#116427](https://github.com/ClickHouse/ClickHouse/pull/116427) ([Groene AI](https://github.com/groeneai)). [#116679](https://github.com/ClickHouse/ClickHouse/pull/116679) ([Groene AI](https://github.com/groeneai)). * Fixed `executable_pool` dictionaries, `executable_pool` user-defined functions, and `ExecutablePool` tables waiting ten times longer than the configured `max_command_execution_time` when the process pool is exhausted, while the resulting error still reported the configured value. Closes [#116467](https://github.com/ClickHouse/ClickHouse/issues/116467). [#116560](https://github.com/ClickHouse/ClickHouse/pull/116560) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash when reading an `Iceberg` table whose manifest list declares a very large `manifest_length`. The declared value was used as the metadata cache entry weight, overflowing the cache accounting and terminating the server. The size measured while reading the manifest file is now used instead. [#116564](https://github.com/ClickHouse/ClickHouse/pull/116564) ([Groene AI](https://github.com/groeneai)). * Fixed a hang when a query names a database that does not exist, either through the `database` setting (for example `SELECT 1 SETTINGS database = 'no_such_db'`) or through the `database` field of a gRPC request. Such a query now fails with `UNKNOWN_DATABASE` instead of blocking its connection forever; while it was hung it was also invisible to `system.processes` and to `KILL QUERY`. [#116582](https://github.com/ClickHouse/ClickHouse/pull/116582) ([Groene AI](https://github.com/groeneai)). * Fixed a use-after-free where a subscriber of workload entity changes, such as an object-storage disk, could be destroyed while its own change callback was still running. Destroying a workload entity subscription now waits for an in-flight callback and stops new ones from starting. Reachable from a `CREATE TABLE` with an inline `disk(...)` definition whose startup fails, concurrently with `CREATE RESOURCE` or `DROP RESOURCE` from another session. [#116607](https://github.com/ClickHouse/ClickHouse/pull/116607) ([Groene AI](https://github.com/groeneai)). * Fixed an exception when reading an `Iceberg` table whose manifest marks a delete file as an equality delete without listing any equality field id. Such an entry is now rejected with `ICEBERG_SPECIFICATION_VIOLATION` instead of an untyped `std::out_of_range`. [#116617](https://github.com/ClickHouse/ClickHouse/pull/116617) ([Groene AI](https://github.com/groeneai)). * Fixed the `file` table function reading and writing outside `user_files_path` when a `..` reached the path behind a directory symlink leaving `user_files_path`: through brace expansion in a glob pattern, or through a `..` partition id in `INSERT INTO FUNCTION file(...) PARTITION BY`. [#116767](https://github.com/ClickHouse/ClickHouse/pull/116767) ([Groene AI](https://github.com/groeneai)). * Fixed corrupted or crafted `Gorilla`-compressed data being decoded to wrong values instead of being rejected. Such data is now rejected with `CANNOT_DECOMPRESS`. [#116820](https://github.com/ClickHouse/ClickHouse/pull/116820) ([Groene AI](https://github.com/groeneai)). * Fixed the `loop` table function disclosing the column names and types of an `Alias` table's target to a user who holds no privilege on that target: a query that read no rows received them in its response header. `loop` now requires table-level `SELECT` on the target, matching what it already required of the table name it is given. [#116873](https://github.com/ClickHouse/ClickHouse/pull/116873) ([Groene AI](https://github.com/groeneai)). * Fixed incorrect results or an exception for `-State` aggregates combined with `-OrDefault` or `-OrFill` under `WITH TOTALS`, `ROLLUP`, or `CUBE`. [#99980](https://github.com/ClickHouse/ClickHouse/pull/99980) ([Ilya Yatsishin](https://github.com/qoega)). * Fixed `Iceberg` `INSERT` writing manifest-wide aggregate column statistics into every `data_file` entry when a partition produced more than one data file. Each entry's `null_value_counts`, `column_sizes`, `lower_bounds`, and `upper_bounds` now describe only the data file it points at. Previously an entry could report more nulls than the file had rows, so a spec-conforming external reader could prune files that contained matching rows. [#112414](https://github.com/ClickHouse/ClickHouse/pull/112414) ([Groene AI](https://github.com/groeneai)). * Fix a server abort while committing a mutated part when its column or secondary-index sizes had not been computed yet. [#112691](https://github.com/ClickHouse/ClickHouse/pull/112691) ([Groene AI](https://github.com/groeneai)). * Fix reading corrupted `ArrowStream` files. A malformed per-buffer uncompressed-length prefix in a compressed Arrow IPC record batch could reach the allocator as an enormous allocation size and produce `LOGICAL_ERROR`; it is now rejected as `INCORRECT_DATA`. [#113006](https://github.com/ClickHouse/ClickHouse/pull/113006) ([Groene AI](https://github.com/groeneai)). * Fix wrong results for `GROUP BY` whose only key is an injective function of constants, for example `SELECT 1 FROM t GROUP BY materialize(NULL)`. Such a key could be removed, leaving the query with no `GROUP BY` and no aggregation, so it returned one row per input row instead of one row per group. Over a `Merge` table that mixes a local and a remote child, the same cause could also raise the exception `Chunk info was not set for chunk in GroupingAggregatedTransform`. [#114981](https://github.com/ClickHouse/ClickHouse/pull/114981) ([Groene AI](https://github.com/groeneai)). * Fixed the exception `Not-ready Set is passed as the second argument for function 'in'` in three cases: when selecting from a `Merge` table (or the `merge` table function) whose underlying table has a row policy containing an `IN` subquery, when a query with `partial_result_on_first_cancel = 1` was cancelled while building the set for `IN (subquery)`, and when a correlated subquery whose body contains `IN (subquery)` runs with `correlated_subqueries_use_in_memory_buffer = 0`, which `make_distributed_plan = 1` also sets. [#114058](https://github.com/ClickHouse/ClickHouse/pull/114058) ([Groene AI](https://github.com/groeneai)). [#115083](https://github.com/ClickHouse/ClickHouse/pull/115083) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#115868](https://github.com/ClickHouse/ClickHouse/pull/115868) ([Groene AI](https://github.com/groeneai)). * Fix authentication over the PostgreSQL wire protocol for users with several authentication methods. Users who also had an `ssh_key` method, or another method with no password, were rejected with `Invalid user or password` even when the correct password for `scram_sha256_password` was supplied, and the wrong SCRAM salt was sent when `scram_sha256_password` was not the last method. [#115084](https://github.com/ClickHouse/ClickHouse/pull/115084) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `THERE_IS_NO_COLUMN` (`Cannot find column in source stream`) for `SELECT DISTINCT expr(k) ... ORDER BY k` on a table with an aggregate projection grouped on `k`. Closes [#112168](https://github.com/ClickHouse/ClickHouse/issues/112168). [#115147](https://github.com/ClickHouse/ClickHouse/pull/115147) ([Groene AI](https://github.com/groeneai)). * A `text` index is now used for `hasAnyTokens`, `hasAllTokens`, and `hasPhrase` called with an explicit tokenizer argument when that argument denotes the same tokenizer as the index. Previously the three-argument form always fell back to a full scan. Closes [#115275](https://github.com/ClickHouse/ClickHouse/issues/115275). [#115348](https://github.com/ClickHouse/ClickHouse/pull/115348) ([Groene AI](https://github.com/groeneai)). * Fix an exception in a window function over a `Tuple` with a sparse element, for example `groupConcat(t) OVER ()`. [#115705](https://github.com/ClickHouse/ClickHouse/pull/115705) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Conversion between named tuples with disjoint element names is now positional instead of silently filling the result with default values, and aliases that only give names to tuple elements can now repeat without error, for example `SELECT tuple(1 AS x), tuple(2 AS x)`. [#115717](https://github.com/ClickHouse/ClickHouse/pull/115717) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Cancel in-flight merges, mutations, and part fetches during graceful server shutdown instead of waiting for the running step to finish. Previously a single slow merge step or a large part download could delay shutdown for its whole duration. [#115733](https://github.com/ClickHouse/ClickHouse/pull/115733) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `WHERE _table = ...` or `WHERE _database = ...` condition over a `Merge` table silently returning no rows when the matching child table reads its data from other tables, for example a `Distributed`, `Merge`, `Buffer`, or `Alias` child. Such children are now always read, and the condition filters their rows. [#116371](https://github.com/ClickHouse/ClickHouse/pull/116371) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix corrupted output when `framing_output_format` is used together with `http_write_exception_in_output_format = 1` (the default for `compatibility` below 25.11). Data packets after the first one repeated the bytes of the first packet padded with zeros, and the query failed with `std::length_error`. This affected the Web UI, which reported a JSON parse error after the first few rows. [#116620](https://github.com/ClickHouse/ClickHouse/pull/116620) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix links between settings, functions, and section headings on the built-in `/docs` page. References to other settings or functions did nothing when clicked, and `#` heading anchors put a corrupted section into the URL. [#116635](https://github.com/ClickHouse/ClickHouse/pull/116635) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an exception when reading from a table after `DETACH` and `ATTACH` of a partition with patch parts created by lightweight updates. [#116695](https://github.com/ClickHouse/ClickHouse/pull/116695) ([Anton Popov](https://github.com/CurtizJ)). * Fixed silently wrong results when reading a `Parquet` file whose offset index disagrees with the page data on an `Array`, `Map`, or nested column. Such a page is now rejected with `INCORRECT_DATA` instead of returning another page's values under the requested row numbers. [#116788](https://github.com/ClickHouse/ClickHouse/pull/116788) ([Groene AI](https://github.com/groeneai)). * Fix background cleanup of unused `store/` directories on encrypted disks. ClickHouse can now revoke permissions and remove these orphaned directories instead of failing during cleanup. [#116919](https://github.com/ClickHouse/ClickHouse/pull/116919) ([Ivan Shelestov](https://github.com/Shelestov7)). * Fix `Native`-protocol memory allocation from truncated input: server-declared sizes no longer make the client allocate memory for data that has not arrived; exception strings are read as they arrive, the decompressed block size is bounded, and the maximum number of rows in a `Native` block is now one billion. [#117011](https://github.com/ClickHouse/ClickHouse/pull/117011) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the exception `max_rows > 0` (and possible incorrect merge results in release builds) in `MergingSortedTransform` when an input delivers a trailing empty chunk together with end-of-stream, for example under `optimize_read_in_order`. [#117048](https://github.com/ClickHouse/ClickHouse/pull/117048) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Internal queries derived from an existing `QueryPlan` now preserve execution limits and shared resources instead of losing settings such as `max_threads` and concurrency control. [#109722](https://github.com/ClickHouse/ClickHouse/pull/109722) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when an `Enum8` or `Enum16` constant is compared with a `String` or `FixedString` column. The constant was converted to the enum's underlying number instead of its name, so primary-key and `PARTITION BY` pruning, the `bloom_filter` skip index, `IN` and `NOT IN` sets, and the `values` table function could silently use the wrong value. Matching rows could be dropped, `NOT IN` could keep rows it should exclude, and `IN` could disagree with `=` on the same constant. [#112647](https://github.com/ClickHouse/ClickHouse/pull/112647) ([Groene AI](https://github.com/groeneai)). * Fix wrong `ORDER BY` and `DISTINCT` results, and `CANNOT_CONVERT_TYPE` for aggregate functions, when reading a `Merge` table whose declared column type does not preserve the sort order of a source table's type, for example `UInt64` over a source holding negative `Int64`. [#113242](https://github.com/ClickHouse/ClickHouse/pull/113242) ([Groene AI](https://github.com/groeneai)). * Fix `min` and `max` returning a wrong value when the implicit `_minmax_count_projection` answered them from a part-level bound that is not a real row value: `NULL` in a `Nullable` sorting key, `NaN` in a floating-point key, or a `Tuple` bound assembled component-wise. Such a query now reads rows instead of using the projection. [#115151](https://github.com/ClickHouse/ClickHouse/pull/115151) ([Groene AI](https://github.com/groeneai)). * Fixed a settings-constraint bypass: a `SETTINGS` clause inside a subquery, a CTE, or a view's inner query was applied without checking the settings constraints, so it could leave `readonly` mode or override a `CONST`, `MIN`, or `MAX` constraint such as `additional_table_filters`, breaking row-level isolation. Nested clauses are now clamped to the constraints instead. [#115397](https://github.com/ClickHouse/ClickHouse/pull/115397) ([Itamar Tempelhof](https://github.com/tempel-it)). * Fix `enable_adaptive_aggregator` with `max_rows_to_group_by` in `throw` mode: adaptive aggregation no longer silently falls back to the regular algorithm, and the merged total is now checked against the limit in parallel two-level aggregation, including bucket Top-K and spill-to-disk paths. [#116451](https://github.com/ClickHouse/ClickHouse/pull/116451) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed wrong results and an exception in primary-key analysis for `intDiv` and `divide` over a `Decimal` key with an integer constant divisor that does not fit the decimal's native width. Such a query could return no rows at all instead of the matching ones. [#116608](https://github.com/ClickHouse/ClickHouse/pull/116608) ([Groene AI](https://github.com/groeneai)). * Column names such as `inf`, `infinity`, `nan`, `true`, and `false` are now quoted when formatted, so a sorting key that refers to them no longer writes metadata the server cannot parse on the next start. Previously the table, and with it the whole database, failed to load. [#116802](https://github.com/ClickHouse/ClickHouse/pull/116802) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a server crash when creating a table whose columns form a long chain of `DEFAULT` expressions, such as `c1 DEFAULT c0 + 1, c2 DEFAULT c1 + 1, ...`. Planning such a statement no longer exhausts the thread stack. [#116978](https://github.com/ClickHouse/ClickHouse/pull/116978) ([Groene AI](https://github.com/groeneai)). * Fixed a `ReplicatedMergeTree` replication queue getting stuck forever after a new replica was added by cloning an existing one. The queue could contain two `ALTER_METADATA` entries for the same `ALTER`, so subsequent `ALTER`s waited indefinitely and `SYSTEM SYNC REPLICA` timed out. [#117107](https://github.com/ClickHouse/ClickHouse/pull/117107) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Detect a column-count mismatch between a tuple and a subquery in the `IN` clause during analysis, preventing constant folding from silently hiding the error. [#97540](https://github.com/ClickHouse/ClickHouse/pull/97540) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` (`Temporary part ... already added`) when `OPTIMIZE ... DRY RUN` runs concurrently with a real merge, or with another `DRY RUN`, on the same parts. [#110916](https://github.com/ClickHouse/ClickHouse/pull/110916) ([Groene AI](https://github.com/groeneai)). * Fixed a server startup crash when the background updater for dictionaries and executable user-defined functions applied an empty configuration before the first configuration repository was registered. [#113238](https://github.com/ClickHouse/ClickHouse/pull/113238) ([Groene AI](https://github.com/groeneai)). * Fixed a data race and a use-after-free on the metadata of `DeltaLake` and `Iceberg` tables. Concurrent queries reading `total_rows` or `total_bytes` from `system.tables` for the same table could crash the server. Closes [#113265](https://github.com/ClickHouse/ClickHouse/issues/113265). [#113427](https://github.com/ClickHouse/ClickHouse/pull/113427) ([Groene AI](https://github.com/groeneai)). * Fixed `arrayROCAUC` and `arrayAUCPR`, and their aliases `arrayAUC` and `arrayPRAUC`, returning a different result for the same data depending on the order of the array elements when a score was `NaN`. A `NaN` score now makes the result `NaN`. Closes [#104763](https://github.com/ClickHouse/ClickHouse/issues/104763). [#115101](https://github.com/ClickHouse/ClickHouse/pull/115101) ([Groene AI](https://github.com/groeneai)). * Fix `RESTORE` of an incremental `S3` backup chain whose locator authenticates by assuming a role with `extra_credentials(role_arn = ...)`. The role identifiers are now kept in the `` locator of the backup metadata, so each backup in the chain can reopen its base; previously chains deeper than one level could not be restored at all. Chains already written without them can still be restored with `use_same_s3_credentials_for_base_backup = 1`, except when the locator is a named collection. [#116223](https://github.com/ClickHouse/ClickHouse/pull/116223) ([Julia Kartseva](https://github.com/jkartseva)). * Fixed missed insert deduplication when a block reaches the sink with sparse column representation, for example `INSERT ... SELECT` from a part with sparse serialization. Such an insert did not deduplicate against the same data inserted seconds ago from unmerged parts. [#116287](https://github.com/ClickHouse/ClickHouse/pull/116287) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Functions `h3PolygonToCells` and `h3PolygonToCellsWithContainment` now check for query timeouts and `KILL QUERY` while covering a polygon, and reject vertices whose longitude is outside `-180..180` or latitude outside `-90..90` degrees. Previously a polygon with such a vertex could keep a thread busy for tens of minutes, ignoring both. Closes [#116195](https://github.com/ClickHouse/ClickHouse/issues/116195). [#116335](https://github.com/ClickHouse/ClickHouse/pull/116335) ([Raúl Marín](https://github.com/Algunenano)). * Fixed the exception `... to have 0 rows, but it has 1: while executing function materialize` when a dictionary with `LAYOUT(DIRECT())` is read through the `merge` table function. Closes [#116269](https://github.com/ClickHouse/ClickHouse/issues/116269). [#116378](https://github.com/ClickHouse/ClickHouse/pull/116378) ([Groene AI](https://github.com/groeneai)). * Fix a `bloom_filter` skip index returning fewer rows for `IN` on an `Array` column when the set contains an empty array, for example `WHERE x IN ([])`. Closes [#116431](https://github.com/ClickHouse/ClickHouse/issues/116431). [#116449](https://github.com/ClickHouse/ClickHouse/pull/116449) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from search functions (`hasPhrase`, `hasToken`, `hasAnyTokens`, `hasAllTokens`) on a text index whose `preprocessor` or `postprocessor` uses `IN` or `NOT IN`, such as `if(token IN ('the'), '', token)`. The transform was silently ignored whenever the predicate was evaluated on rows instead of read from the index, so matching rows could be missing. Closes [#116476](https://github.com/ClickHouse/ClickHouse/issues/116476). [#116578](https://github.com/ClickHouse/ClickHouse/pull/116578) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed a 404 (`S3_ERROR`) when reading an `Iceberg` table through a path that is a parent of the table directory, with the metadata document named by the `iceberg_metadata_file_path` setting. Writes and maintenance operations on such a table are now refused instead of acting outside the table. Closes [#116985](https://github.com/ClickHouse/ClickHouse/issues/116985). [#117037](https://github.com/ClickHouse/ClickHouse/pull/117037) ([Groene AI](https://github.com/groeneai)). * Fixed incorrect results from `sum(if(cond, 0, 1))` when `cond` has type `LowCardinality(Nullable(...))` and `optimize_rewrite_sum_if_to_count_if` is enabled, which is the default. Rows where the condition was `NULL` were not counted. Closes [#116938](https://github.com/ClickHouse/ClickHouse/issues/116938). [#117046](https://github.com/ClickHouse/ClickHouse/pull/117046) ([Pratham Nayak](https://github.com/protomn)). * Fixed `clickhouse-client` and `clickhouse-local` not emitting the `--chime` alert for a long-running query that ends with an error. The threshold was checked against the query time last reported by the server, which is not refreshed when a query fails, so it could be arbitrarily behind the real duration of the query. It is now checked against the time the client actually waited. [#117179](https://github.com/ClickHouse/ClickHouse/pull/117179) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server crash when executing `ALTER TABLE ... MATERIALIZE TTL` on a table that is not a `MergeTree` but whose metadata carries a TTL. This can affect `Alias` tables whose target has a TTL and `MaterializedView` tables that declare a TTL on one of their own columns. Closes [#117327](https://github.com/ClickHouse/ClickHouse/issues/117327). [#117181](https://github.com/ClickHouse/ClickHouse/pull/117181) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` in KQL `timespan` parameter handling for `LowCardinality` interval arguments. [#117162](https://github.com/ClickHouse/ClickHouse/pull/117162) ([Groene AI](https://github.com/groeneai)). * Fix `temporary_files_buffer_size` and `format_avro_schema_registry_*` accepting out-of-range values that then made every subsequent query in the session fail with no way to recover. Such values are now bounded when set. Closes [#113782](https://github.com/ClickHouse/ClickHouse/issues/113782). [#113787](https://github.com/ClickHouse/ClickHouse/pull/113787) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix `allow_feature_tier` rejecting `merge_tree_`-prefixed settings in settings profiles as unknown and rejecting `CREATE TABLE` because of `MergeTree` settings applied by `compatibility` or the `merge_tree` config section. Also fix constraints on a `merge_tree_`-prefixed setting being bypassed when the setting is written through its alias, and a statement that rewrites such a setting to its current value being refused by a `CONST` constraint. [#113156](https://github.com/ClickHouse/ClickHouse/pull/113156) ([Raúl Marín](https://github.com/Algunenano)). * Fix a server crash and out-of-bounds write when inserting into a column with a long chain of compression codecs, for example `CODEC(FPC, FPC, ..., FPC)`. Such chains are now rejected with `CANNOT_COMPRESS`; chains longer than 255 codecs, which previously could write unreadable parts, are rejected up front with `BAD_ARGUMENTS` unless `allow_suspicious_codecs` is set. [#113877](https://github.com/ClickHouse/ClickHouse/pull/113877) ([Groene AI](https://github.com/groeneai)). * Fix `OpenSSL` `cipherList` being silently ignored when its value is written on its own line, as XML or YAML auto-formatters do. The configured value is now trimmed before it reaches `OpenSSL`, and an invalid cipher list is reported instead of discarded, so the server no longer keeps `OpenSSL`'s default cipher list while appearing to honor the configured restriction. Closes [#115808](https://github.com/ClickHouse/ClickHouse/issues/115808). [#116053](https://github.com/ClickHouse/ClickHouse/pull/116053) ([Groene AI](https://github.com/groeneai)). * Fix a server crash in the `ReplicatedMergeTree` cleanup thread when `max_replicated_logs_to_keep = 0`. That value is now rejected, matching the documented domain of the setting. [#117147](https://github.com/ClickHouse/ClickHouse/pull/117147) ([Groene AI](https://github.com/groeneai)). * Fix path traversal in `RESTORE`: backup metadata entries whose `` is not already normalized are now rejected with `INSECURE_PATH` instead of being restored outside the part directory while `RESTORE` reports success. [#117371](https://github.com/ClickHouse/ClickHouse/pull/117371) ([Groene AI](https://github.com/groeneai)). * Fixed memory leaks of abandoned nested aggregate states. [#113926](https://github.com/ClickHouse/ClickHouse/pull/113926) ([Groene AI](https://github.com/groeneai)). * Preserve row counts above the signed 32-bit range on the `PostgreSQL` wire protocol. [#114397](https://github.com/ClickHouse/ClickHouse/pull/114397) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `ILLEGAL_COLUMN` for `ARRAY JOIN` queries filtered by an empty `IN` set, including an empty list and an empty `Set` table. Closes [#116018](https://github.com/ClickHouse/ClickHouse/issues/116018). [#116057](https://github.com/ClickHouse/ClickHouse/pull/116057) ([Groene AI](https://github.com/groeneai)). * Fixed `Alias` tables ignoring row policies defined on their target tables. An `Alias` table now combines the row policies of both the alias and its target table. [#116290](https://github.com/ClickHouse/ClickHouse/pull/116290) ([Kai Zhu](https://github.com/nauu)). * Fixed concurrent writes of the same `format_schema_source = 'string'` or `'query'` schema being rejected with `CANNOT_OPEN_FILE` or `ATOMIC_RENAME_FAIL`. [#116502](https://github.com/ClickHouse/ClickHouse/pull/116502) ([Groene AI](https://github.com/groeneai)). * Fixed `moduloOrNull`, `intDivOrNull`, `divideOrNull`, and `positiveModuloOrNull` returning the raw division result instead of `NULL` when dividing by a constant zero inside a subquery whose result is read or persisted by an outer query. Closes [#116500](https://github.com/ClickHouse/ClickHouse/issues/116500). [#116594](https://github.com/ClickHouse/ClickHouse/pull/116594) ([Groene AI](https://github.com/groeneai)). * Fixed `Iceberg` partition pruning when a manifest's partition tuple contains fields ClickHouse cannot model. Such tables no longer fail with `STD_EXCEPTION`, and a manifest whose tuple length disagrees with its own spec is now rejected with `ICEBERG_SPECIFICATION_VIOLATION` instead of returning deleted rows. [#116615](https://github.com/ClickHouse/ClickHouse/pull/116615) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from `sumKahan` and `sumKahanIf` when the aggregated batch does not start at the first row of the block, including window frames, parallel `GROUP BY`, and `optimize_aggregation_in_order = 1`. [#116990](https://github.com/ClickHouse/ClickHouse/pull/116990) ([Groene AI](https://github.com/groeneai)). * Fixed `GenerateRandom` reads with a trivial `LIMIT` returning no rows because the source-count reduction could overflow to zero. Absurdly large requested stream counts are now also rejected with `PARAMETER_OUT_OF_BOUND`. [#117171](https://github.com/ClickHouse/ClickHouse/pull/117171) ([Groene AI](https://github.com/groeneai)). * Fixed out-of-bounds access in `timeSeriesGroupArray`. [#117404](https://github.com/ClickHouse/ClickHouse/pull/117404) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed `GROUP BY` emitting a group from a zero-row block, returning a row with `count() = 0`. Closes [#116830](https://github.com/ClickHouse/ClickHouse/issues/116830). [#117503](https://github.com/ClickHouse/ClickHouse/pull/117503) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `arrayIntersect` with mixed `Nullable` and not-`Nullable` arguments. Such calls could fail or return spurious default-valued elements for some data types; `arrayUnion` and `arraySymmetricDifference` are unaffected. [#117577](https://github.com/ClickHouse/ClickHouse/pull/117577) ([Groene AI](https://github.com/groeneai)). * `ATTACH PARTITION FROM`, `REPLACE PARTITION`, and `MOVE PARTITION TO TABLE` no longer accept a non-adaptive part whose `index_granularity` differs from the destination table's. Previously such a part could return wrong row counts in release builds. Closes [#117524](https://github.com/ClickHouse/ClickHouse/issues/117524). [#117594](https://github.com/ClickHouse/ClickHouse/pull/117594) ([Groene AI](https://github.com/groeneai)). * Fixed `max_joined_block_size_rows = 0` (documented as unlimited) making a hash join emit one left row per output block instead of honoring `max_joined_block_size_bytes` when a left row has multiple right matches. Closes [#102007](https://github.com/ClickHouse/ClickHouse/issues/102007). [#117604](https://github.com/ClickHouse/ClickHouse/pull/117604) ([Groene AI](https://github.com/groeneai)). * Fixed `hasColumnInTable` disclosing whether a column exists on the target of an `Alias` table when the user lacks `SHOW COLUMNS` on the target. The function now requires that privilege on the target too. [#117621](https://github.com/ClickHouse/ClickHouse/pull/117621) ([Groene AI](https://github.com/groeneai)). * Fix silent data loss when a TTL expression applies a large interval directly to a `Date` or `DateTime` column by widening the arithmetic away from 16/32-bit overflow. [#101793](https://github.com/ClickHouse/ClickHouse/pull/101793) ([BoloniniD](https://github.com/BoloniniD)). * Fix `Cannot insert field with type ...` when inserting a row that omits an `AggregateFunction` column whose argument types contain `LowCardinality`. `defaultValueOfTypeName` is fixed for the same type names. [#102403](https://github.com/ClickHouse/ClickHouse/pull/102403) ([Enric Calabuig](https://github.com/eclbg)). * Fixed a privilege-escalation gap where the `executable` table function did not enforce the `TABLE ENGINE ON Executable` grant. With `table_engines_require_grant` enabled, a user denied `CREATE TABLE ... ENGINE = Executable` could still run the same server-side script via `SELECT ... FROM executable(...)`. It now requires `GRANT TABLE ENGINE ON Executable`, matching `CREATE TABLE`. [#109331](https://github.com/ClickHouse/ClickHouse/pull/109331) ([Groene AI](https://github.com/groeneai)). * Fix silently wrong `GROUP BY` results with `optimize_aggregation_in_order = 1`. Groups were merged into one another and aggregate values were wrong when the matched sorting-key prefix contained a reverse (`DESC`) column, and, independently of reverse keys, when a group-by key was a negative monotonic function of a `Float` sorting-key column around the `NaN` boundary. This affected a single `MergeTree` table and a `Merge` table over such tables. For a `Merge` table whose underlying tables read the matched prefix in opposite physical directions, aggregation in order is now declined because a single stream direction cannot describe both. [#112200](https://github.com/ClickHouse/ClickHouse/pull/112200) ([Groene AI](https://github.com/groeneai)). * Fix `_path` / `_file` filtering for object storage, `file`, and `url` reads. A `GLOBAL IN` subquery over `_path` or `_file` of a path without globs no longer throws `Not-ready Set is passed as the second argument for function 'globalIn'`; an explicitly named archive member is filtered before the archive is opened; `Hive` partition columns are inferred for an explicitly named key that does not exist; and `urlCluster` / `fileCluster` now prune a filtered-out path on the initiator instead of handing it to a replica. [#112968](https://github.com/ClickHouse/ClickHouse/pull/112968) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `FileLog` tables no longer silently skip files that appear after `CREATE TABLE`. The directory watch ran as a `BackgroundSchedulePool` task but never returned, so it held its slot of the per-task-type budget forever; once `background_schedule_pool_size` times `background_schedule_pool_max_parallel_tasks_per_type_ratio` tables existed, further tables never watched their directory at all and `SELECT` returned fewer rows instead of erroring. The watch now runs on its own thread. [#113897](https://github.com/ClickHouse/ClickHouse/pull/113897) ([Groene AI](https://github.com/groeneai)). * A query queued on a full distributed connection pool now responds to `KILL QUERY`, and to `max_execution_time` at the default `timeout_overflow_mode = 'throw'`, instead of waiting until another query happens to release a connection, and a positive `connection_pool_max_wait_ms` now actually expires. Previously the wait ended only on success at every value of the setting, so the configured timeout was ignored, `max_execution_time` was reported long after its deadline, and the query could not be killed. The wait applies per replica, so an expired one is a soft failure that fails over to the remaining replicas and only then reports `ALL_CONNECTION_TRIES_FAILED`. [#113979](https://github.com/ClickHouse/ClickHouse/pull/113979) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a filter on a `JSON` subcolumn that mixes a literal prefix with a quoted path segment is analyzed against a `JSONAllPaths` skip index. The subcolumn name was misparsed, so all granules were skipped and the query returned fewer rows than it should. [#114873](https://github.com/ClickHouse/ClickHouse/pull/114873) ([Pavel Kruglov](https://github.com/Avogar)). * Fix wrong results when a `SELECT` with `select_sequential_consistency = 1` on a `ReplicatedMergeTree` table runs with classic (task-based) parallel replicas. Every replica performed a complete local read clamped to the quorum boundary instead of a coordinator-assigned slice, so rows were returned once per participating replica. [#114945](https://github.com/ClickHouse/ClickHouse/pull/114945) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE DICTIONARY` failing with `Incorrect type of ID column: must be UInt64, but it is LowCardinality` when the simple primary key is declared as `LowCardinality(UInt64)` and `dictionary_validate_primary_key_type` is enabled. Closes [#104539](https://github.com/ClickHouse/ClickHouse/issues/104539). [#115139](https://github.com/ClickHouse/ClickHouse/pull/115139) ([Groene AI](https://github.com/groeneai)). * Fix a hang in workload memory reservations when an increase races with an in-flight decrease. [#116085](https://github.com/ClickHouse/ClickHouse/pull/116085) ([Azat Khuzhin](https://github.com/azat)). * A `Kafka` table with `kafka_keeper_path` set (`StorageKafka2`) no longer skips messages it had already consumed when a streaming cycle fails part-way through; previously those messages were never delivered to the materialized views and a later cycle committed an offset past them. As a consequence, with the default `kafka_handle_error_mode` an unparseable message now blocks the table instead of being silently skipped, matching `StorageKafka`. `kafka_handle_error_mode = 'stream'`, `kafka_handle_error_mode = 'dead_letter_queue'`, and `kafka_skip_broken_messages` remain the ways to skip such a message. [#116485](https://github.com/ClickHouse/ClickHouse/pull/116485) ([Groene AI](https://github.com/groeneai)). * Fixed the `Unexpected return type from materialize` logical error when reading a column that has a `text` index through the index after a metadata-only or not-yet-materialized `ALTER MODIFY COLUMN` (including lazy `JSON` type hints) that requires a read-time type conversion. [#116549](https://github.com/ClickHouse/ClickHouse/pull/116549) ([Pavel Kruglov](https://github.com/Avogar)). * A table created with `CREATE TABLE ... AS
(...)` is no longer resolved to answer a `SYSTEM STOP`-family lock query or a `system.tables` size query about it. Previously those statements contacted the endpoint of every such table, so one unreachable endpoint made them fail or stall. Until such a table is first accessed it now reports `NULL` sizes and an empty `data_paths`, and `DROP TABLE ... IF EMPTY` removes it instead of reporting `TABLE_NOT_EMPTY`. [#116774](https://github.com/ClickHouse/ClickHouse/pull/116774) ([Groene AI](https://github.com/groeneai)). * Fixes wrong results when a `minmax` skip index, `ORDER BY`, or `PARTITION BY` is built over an `Array`, `Tuple`, or `Map` holding a `Bool`, such as `Array(Tuple(Float64, Bool))` or `Map(String, Bool)`, and when a `minmax` index serves an `ORDER BY ... LIMIT` over a `Bool` column. Previously, rows that the predicate accepts could be dropped, because the stored bound and the value compared against it represented the boolean differently. [#116897](https://github.com/ClickHouse/ClickHouse/pull/116897) ([Groene AI](https://github.com/groeneai)). * Fixed an external `GROUP BY` exceeding `max_memory_usage` when `enable_adaptive_aggregator` is on. Intermediate data the adaptive aggregator held and could not release kept a thread on the baseline algorithm flushing its own hash table on every block, so each block wrote a temporary file and the merge then allocated a reader per file. That data is now written out first. [#117326](https://github.com/ClickHouse/ClickHouse/pull/117326) ([Groene AI](https://github.com/groeneai)). * Malformed `Native` blocks can no longer overflow the allocation of aggregate-function states during deserialization and cause out-of-bounds writes. [#117370](https://github.com/ClickHouse/ClickHouse/pull/117370) ([Raúl Marín](https://github.com/Algunenano)). * Fixed the byte width of the `Iceberg` manifest `lower_bounds` and `upper_bounds` written for `int` and `date` columns: they were 8 bytes wide where the `Iceberg` specification requires 4. Spec-conformant readers such as `pyiceberg` failed with `unpack requires a buffer of 4 bytes` while planning any scan carrying a filter on such a column. [#117571](https://github.com/ClickHouse/ClickHouse/pull/117571) ([Groene AI](https://github.com/groeneai)). * Remote-write async insert flush timeouts are now handled correctly: they return retryable `503 Service Unavailable` instead of `408 Request Timeout`, so Prometheus-compatible senders can retry them, and `wait_for_async_insert_timeout` is now clamped instead of overflowing for large magnitudes. [#117632](https://github.com/ClickHouse/ClickHouse/pull/117632) ([Vitaly Baranov](https://github.com/vitlibar)). [#118040](https://github.com/ClickHouse/ClickHouse/pull/118040) ([Groene AI](https://github.com/groeneai)). * Fixed `base58Encode` and `base58Decode` of 32- and 64-byte values on big-endian builds such as `s390x`. The fixed-size codec byte-swapped every 4-byte word unconditionally, so on a big-endian host `base58Encode` returned a string that disagrees with the base58 specification and with every other architecture, and `base58Decode` rejected well-formed input when given a 32- or 64-byte size hint. Little-endian builds are bit-identical to before. Strings previously produced by a big-endian build will change, because they were wrong. [#117649](https://github.com/ClickHouse/ClickHouse/pull/117649) ([Groene AI](https://github.com/groeneai)). * Fixed `format_schema_source = 'query'` running the schema query with full privileges when used from a background task, such as a streaming engine consumer (`Kafka`, `NATS`, `RabbitMQ`, `ObjectStorageQueue`) or a `Buffer` background flush. The schema query is now rejected there; use `format_schema_source = 'file'` or `format_schema_source = 'string'` instead, or pre-populate the schema cache. [#117737](https://github.com/ClickHouse/ClickHouse/pull/117737) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `absent` and `scalar` PromQL queries returning no result when `empty_result_for_aggregation_by_empty_set` is enabled. [#117816](https://github.com/ClickHouse/ClickHouse/pull/117816) ([Valery Petrov](https://github.com/valerypetrov)). * A `bloom_filter` index over an `Array(Nullable(T))` column is now rejected at `CREATE` / `ALTER` time. It used to be accepted and then made every insert, merge, and mutation of the table fail, with `DROP INDEX` refused until the failed mutation was killed. [#117899](https://github.com/ClickHouse/ClickHouse/pull/117899) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `GROUPING` function rejecting constant aliases that are valid `GROUP BY` keys. [#97504](https://github.com/ClickHouse/ClickHouse/pull/97504) ([Héctor Pablos](https://github.com/hctpbl)). * Fixed SQL injection in the `PostgreSQL` wire protocol emulation. Prepared-statement parameters, in both the extended-query `Parse` / `Bind` / `Execute` path used by `psycopg2`, JDBC, `node-postgres`, Npgsql, and others, and the simple-query `EXECUTE` path, were substituted into the executed SQL by raw string splicing, so a crafted parameter could break out of its value position and inject SQL. Parameters are now bound as data. This also fixes string parameters that previously failed with `UNKNOWN_IDENTIFIER` when a client sent them unquoted. [#108469](https://github.com/ClickHouse/ClickHouse/pull/108469) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`Part ... intersects previous part ... It is a bug or a result of manual intervention`) that makes a `MergeTree` table impossible to load: attaching it, and every later attempt to load it, fails with that exception, and debug and sanitizer builds abort. A merge could span a block gap holding an outdated part whose mutation version is higher than the merge result's, producing a pair of parts on disk that neither contain nor are disjoint from each other. Closes [#113864](https://github.com/ClickHouse/ClickHouse/issues/113864). [#113941](https://github.com/ClickHouse/ClickHouse/pull/113941) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR: Next task callback is not set for query with id` exception when `parallel_replicas_plan_based` is enabled and the query reads a system log table. [#114022](https://github.com/ClickHouse/ClickHouse/pull/114022) ([Groene AI](https://github.com/groeneai)). * Fix `files_read` and `bytes_read` in `system.backups` and `system.backup_log` being permanently undercounted after a successful `RESTORE`, because a concurrent restore task holding an early snapshot could overwrite a larger count published by another task. [#114304](https://github.com/ClickHouse/ClickHouse/pull/114304) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a non-boolean condition, such as a bare integer column in `WHERE`, is left alone after the other conditions are merged into the `JOIN` condition or pushed down. Rows whose condition value was a non-zero multiple of 256 were skipped. [#114533](https://github.com/ClickHouse/ClickHouse/pull/114533) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a single `ALTER` with two `MODIFY COLUMN` subcommands on the same column silently corrupting that column's default: a default set by one subcommand was dropped, and a `REMOVE DEFAULT` / `MATERIALIZED` / `ALIAS` was undone, whenever a following subcommand restated the column type. [#114807](https://github.com/ClickHouse/ClickHouse/pull/114807) ([Groene AI](https://github.com/groeneai)). * You can now access nested properties on a `Nullable(JSON)` value, for example `col::Nullable(JSON).Id`. Previously this failed with `First argument for function tupleElement cannot be Nullable(JSON)`, which made it impossible to combine a cast to `Nullable(JSON)` with property access on the result. Outer `NULL` rows now propagate through extracted paths using the existing subcolumn semantics: paths whose type can represent `NULL` return `NULL`, while types such as `Array` and `Map` return their default values. [#114863](https://github.com/ClickHouse/ClickHouse/pull/114863) ([Luís Lizardo](https://github.com/lizardoluis)). * Fix an integer overflow in the deserialization of `String` values in the binary formats, and bound the recursion of the binary deserialization of `Dynamic` and `JSON` values. [#115702](https://github.com/ClickHouse/ClickHouse/pull/115702) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `PostgreSQL` compatibility now handles `pg_catalog` qualifiers case-insensitively like PostgreSQL does, and the synthetic OIDs exposed through `pg_namespace` and `pg_class` are now unique, so `psql` commands such as `\d` no longer duplicate tables or list them under the wrong schema. [#115960](https://github.com/ClickHouse/ClickHouse/pull/115960) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed data loss in `MergeTree` when a merge runs while an `ALTER TABLE ... RENAME COLUMN` is still pending. The merge applied the rename but did not record it, so the mutation applied it a second time and read the renamed column as missing, filling it with defaults. Columns without a default expression, such as `Dynamic`, lost their values. [#115981](https://github.com/ClickHouse/ClickHouse/pull/115981) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a query over an `Iceberg` table returning fewer rows than the table holds when a manifest declares a lower bound above the upper bound for a filtered column. Such a pair describes an empty range, so min/max pruning dropped the data file holding the matching rows; those bounds are now ignored instead. Closes [#116499](https://github.com/ClickHouse/ClickHouse/issues/116499). [#116604](https://github.com/ClickHouse/ClickHouse/pull/116604) ([Groene AI](https://github.com/groeneai)). * Fixed two `Binary Viewer` (`/binary`) bugs: the image was not centered on load if the URL contained parameters such as `user`, and the connection parameters (`host`, `user`) were lost from the URL after moving the view. The `password` parameter is no longer kept in the address bar or browser history. [#117065](https://github.com/ClickHouse/ClickHouse/pull/117065) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a large performance regression, present since 26.5, for a query that filters on a column with an equality and orders by a later column of the same sorting key. Read-in-order analysis did not look through an `ALIAS` node on the filter column, so the filtered column was not recognized as constant and the read was not done in order, forcing a full sort of every matching row before the `LIMIT` applied. This reaches a table with a normal `PROJECTION`, whose rewrite wraps the query filter in such an alias, and a table-level `ALIAS` column used as a predicate. [#116854](https://github.com/ClickHouse/ClickHouse/issues/116854) separately made a base table that can serve the `ORDER BY` itself preferred; this fixes the case where only the projection can. [#116835](https://github.com/ClickHouse/ClickHouse/pull/116835) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when reading the `.null` subcolumn of a column that went through a metadata-only `T` to `Nullable(T)` `ALTER` and then had a `RENAME COLUMN` still pending: `IS NULL`, `IS NOT NULL`, `count()`, and an explicit `x.null` read reported rows holding real data as `NULL`. [#116901](https://github.com/ClickHouse/ClickHouse/pull/116901) ([Groene AI](https://github.com/groeneai)). * Fixed a stack overflow when reading hadoop-snappy data containing many blocks that decode to nothing. [#116994](https://github.com/ClickHouse/ClickHouse/pull/116994) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The handshake response of the `MySQL` wire protocol is now bounded, so a client can no longer make the server grow memory during the handshake. The `mysql_require_secure_transport` server setting is now enforced from the actual state of the connection instead of a capability bit advertised by the client. [#117003](https://github.com/ClickHouse/ClickHouse/pull/117003) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix background `Distributed` inserts that deleted unsent spool files after a split batch got a transient error. ClickHouse now keeps the files and retries the batch. [#117034](https://github.com/ClickHouse/ClickHouse/pull/117034) ([Rory Shanks](https://github.com/rorylshanks)). * Fix wrong results for `GROUP BY` and `uniq*` over `MACNumToString`, `unhex`, `unbin`, and `UUIDStringToNum`. All four declared themselves injective, so the default-enabled `optimize_injective_functions_in_group_by` and `optimize_injective_functions_inside_uniq` grouped and counted by the raw argument: arguments that map to the same result came back as separate groups and were counted separately by `uniqExact`. [#117210](https://github.com/ClickHouse/ClickHouse/pull/117210) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results from the default-enabled `optimize_uniq_to_count`. `uniq(x)` over a `SELECT DISTINCT x` or `GROUP BY x` subquery reported one distinct value too many when the column contained a `NULL`, and counted the extra super-aggregate rows of `WITH ROLLUP` / `WITH CUBE` as additional distinct values. [#117222](https://github.com/ClickHouse/ClickHouse/pull/117222) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `NO_SUCH_COLUMN_IN_TABLE` for every query using a text index while a mutation is pending with `apply_mutations_on_fly = 1`. Until the mutation materialized, `hasToken`, `hasAnyTokens`, and `hasAllTokens` on the indexed column could not be executed at all. [#117277](https://github.com/ClickHouse/ClickHouse/pull/117277) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Several aggregate function deserializers allocated memory from a size declared in the serialized state before reading the payload, so a state of a few bytes could request gigabytes. They now read only what the buffer already holds. [#117536](https://github.com/ClickHouse/ClickHouse/pull/117536) ([Groene AI](https://github.com/groeneai)). * `getServerSetting`, `getMergeTreeSetting`, and `getMacro` returned the values of `system.server_settings`, `system.merge_tree_settings`, and `system.macros` to any user, bypassing the `SELECT` grant that reading those tables requires. They now require the same grant. [#117551](https://github.com/ClickHouse/ClickHouse/pull/117551) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a bug where the `analyzer_inline_views` setting could bypass the `SELECT` access check on a view, letting a user without a grant on the view read its contents. [#117555](https://github.com/ClickHouse/ClickHouse/pull/117555) ([Dmitry Novik](https://github.com/novikd)). * `ENGINE = EmbeddedRocksDB(ttl, rocksdb_dir, read_only)` with an explicit `rocksdb_dir` now requires the `FILE` source grant that `file` requires for the same directory: `READ ON FILE` to create such a table, plus `WRITE ON FILE` when `read_only = 0`. Previously it required no source grant, so a user holding only `CREATE TABLE` in a database of their own could point a table at another table's directory and read its rows. `ON CLUSTER` and `Replicated`-database DDL are not covered: the replay runs with no user. The argument-less form is unaffected. Backward incompatible for anyone already using an explicit `rocksdb_dir` without them. [#117428](https://github.com/ClickHouse/ClickHouse/pull/117428) ([Groene AI](https://github.com/groeneai)). [#117665](https://github.com/ClickHouse/ClickHouse/pull/117665) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix `ALTER` and `CREATE TABLE ... AS` dropping the other settings of a `TimeSeries` table. [#117693](https://github.com/ClickHouse/ClickHouse/pull/117693) ([Vitaly Baranov](https://github.com/vitlibar)). [#117839](https://github.com/ClickHouse/ClickHouse/pull/117839) ([Vitaly Baranov](https://github.com/vitlibar)). [#118119](https://github.com/ClickHouse/ClickHouse/pull/118119) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results for suffix `LIKE` patterns with escape sequences when `optimize_rewrite_like_perfect_affix` is enabled. Such patterns are now rewritten to `endsWith` only when the suffix is an exact literal. [#117712](https://github.com/ClickHouse/ClickHouse/pull/117712) ([Alex Prabhat Bara](https://github.com/alexprabhat99)). * Fix a dictionary with a `Redis` source and an `Int256` key column silently returning the default value from `dictGet` for keys that exist in `Redis`. Wide integer key columns are now refused with a clear error instead of being admitted and never matching. [#117735](https://github.com/ClickHouse/ClickHouse/pull/117735) ([Pratham Nayak](https://github.com/protomn)). * Fixed excessive memory usage when reading from `File`, `URL`, object storage, `GenerateRandom`, or any other storage that uses the generic read path with `max_streams_to_max_threads_ratio` above 1. The number of output ports created after a read is now bounded by `max_threads`, so a `SELECT` over a tiny table no longer allocates tens of gigabytes while the pipeline is still being built. [#117772](https://github.com/ClickHouse/ClickHouse/pull/117772) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ARRAY JOIN` producing rows in incorrect order in some cases. [#117918](https://github.com/ClickHouse/ClickHouse/pull/117918) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix a potential out-of-bounds write in `formatDateTimeInJodaSyntax` when called with a time zone name longer than 32 characters. [#117925](https://github.com/ClickHouse/ClickHouse/pull/117925) ([Robert Schulze](https://github.com/rschu1ze)). * Fix `EXPLAIN QUERY TREE` and `EXPLAIN SYNTAX` exposing column names and types of tables the user has no privileges for: the analyzer now checks that the user may see at least one column of the table at the moment a table identifier is resolved. The check also makes the error kind independent of whether the queried column exists and stops recording column names the user may not see into `missing_privileges` of the query log. Closes [#78938](https://github.com/ClickHouse/ClickHouse/issues/78938). [#117979](https://github.com/ClickHouse/ClickHouse/pull/117979) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed Keeper aborting when memory usage stays near the limit: moving finished changelogs and snapshots to object storage was refused, so the local log disk filled up. [#117983](https://github.com/ClickHouse/ClickHouse/pull/117983) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `Kafka` / `RabbitMQ` / `NATS` / `FileLog` / `S3Queue` / `AzureQueue` table in a database with `lazy_load_tables` silently stopping to feed its materialized views after a server restart. [#118009](https://github.com/ClickHouse/ClickHouse/pull/118009) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed cancellation of a synchronous distributed `INSERT` (`distributed_foreground_insert`) waiting for that `INSERT` to finish instead of interrupting it. `KILL QUERY` against such an insert, a client disconnect, and server shutdown could block for as long as a shard was slow. [#118013](https://github.com/ClickHouse/ClickHouse/pull/118013) ([Groene AI](https://github.com/groeneai)). * Fixed a `LIMIT` being pushed into `numbers()` when the query's `WHERE` referenced an `arrayJoin` through a `WITH` alias, which could make the source stop before the expansion produced enough rows. [#118045](https://github.com/ClickHouse/ClickHouse/pull/118045) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `Iceberg` schema evolution for additional nested `Array`, `Map`, and `Struct` combinations whose inner fields are added, removed, or reordered. [#118127](https://github.com/ClickHouse/ClickHouse/pull/118127) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed incremental backups to Azure Blob Storage storing the beginning of each file instead of its tail. Restoring such a backup silently dropped all the data written after the base backup. [#114975](https://github.com/ClickHouse/ClickHouse/pull/114975) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fixed potential data loss when an `INSERT` into `ReplicatedMergeTree` was cancelled or timed out while ClickHouse was recovering an uncertain Keeper commit. [#111196](https://github.com/ClickHouse/ClickHouse/pull/111196) ([kakao-tars-case](https://github.com/kakao-tars-case)). * Fixed object storage listing silently skipping objects when S3 returns an empty page together with a continuation token. This could make a `plain_rewritable` disk appear empty and hide table parts, and could make data lake tables read a stale snapshot. [#118615](https://github.com/ClickHouse/ClickHouse/pull/118615) ([Sema Checherinda](https://github.com/CheSema)). * Fixed a SQL injection in the PostgreSQL metadata queries. The `schema` of the `PostgreSQL` database engine, the requested column names and the publication name were formatted into SQL literals without escaping, so a value containing a quote could execute arbitrary SQL on the connected PostgreSQL server as the role ClickHouse connects with. Reaching it through `schema` needed only `CREATE DATABASE` plus use of a named collection whose `schema` is overridable. [#111516](https://github.com/ClickHouse/ClickHouse/pull/111516) ([Shaohua Wang](https://github.com/tiandiwonder)). * Hide the credentials that a setting value can carry, such as the password in `format_avro_schema_registry_url` or `url_base`, everywhere a setting value is displayed or logged. `system.query_log`, `system.processes`, `system.settings`, `system.settings_profile_elements`, `system.session_log`, `system.distributed_ddl_queue`, `SHOW CREATE USER` and `SHOW CREATE SETTINGS PROFILE` previously showed them in plaintext to any user who could read them. [#117523](https://github.com/ClickHouse/ClickHouse/pull/117523) ([Raúl Marín](https://github.com/Algunenano)). `EXPLAIN` now respects `format_display_secrets_in_show_and_select` as well. [#118129](https://github.com/ClickHouse/ClickHouse/pull/118129) ([Kirill Kopnev](https://github.com/Fgrtue)). * `CREATE USER OR REPLACE` and the `OR REPLACE` form of `CREATE ROLE`, `CREATE QUOTA`, `CREATE SETTINGS PROFILE` and `CREATE ROW POLICY` now require the corresponding `DROP` privilege in addition to `CREATE`, because they overwrite an existing entity. Previously `CREATE USER` alone was enough to reset the password of any other user, including a privileged one. [#117960](https://github.com/ClickHouse/ClickHouse/pull/117960) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CREATE OR REPLACE TABLE`/`VIEW`/`DICTIONARY` and `REPLACE TABLE` requiring privileges on the internal temporary table they create (`_tmp_replace_*`). Its name is random, so no grant could cover it and the query failed with `ACCESS_DENIED` unless the user held wildcard grants on the whole database. The required privileges are now checked against the user-visible names only: replacing an object needs the `CREATE` and `DROP` grants of its own kind, plus `INSERT` on it when the statement populates it. [#117967](https://github.com/ClickHouse/ClickHouse/pull/117967) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `TRUNCATE DATABASE ... ON CLUSTER` and `TRUNCATE [ALL] TABLES FROM ... ON CLUSTER` ran without checking any privilege: a user holding only `SELECT` and `CLUSTER` could destroy every table in a database, and `readonly = 1` did not prevent it. Both now require `DROP DATABASE`, the privilege the non-cluster spellings have always required. [#118155](https://github.com/ClickHouse/ClickHouse/pull/118155) ([Groene AI](https://github.com/groeneai)). * Fixed a privilege escalation in implicit access-rights computation: holding any single child privilege of the `ALTER TABLE` group (for example `ALTER MODIFY COMMENT`) on a view implicitly conferred the entire `ALTER VIEW` group, so such a user could run `ALTER TABLE MODIFY QUERY` and rewrite the view's query. `ALTER VIEW` is now implied only by the complete `ALTER TABLE` grant. A deployment that relied on a partial `ALTER TABLE` grant to alter views must now grant `ALTER VIEW` (or one of its members) or the full `ALTER TABLE` explicitly. [#118194](https://github.com/ClickHouse/ClickHouse/pull/118194) ([Groene AI](https://github.com/groeneai)). * Expressions supplied through the `additional_table_filters` and `parallel_replicas_custom_key` settings are now subject to the column-level `SELECT` check, and are no longer injected into the body of `SQL SECURITY DEFINER` or `NONE` views. Previously a user could evaluate predicates over columns they were not granted. [#118063](https://github.com/ClickHouse/ClickHouse/pull/118063) ([Dmitry Novik](https://github.com/novikd)). * Forbid `CREATE TABLE ... AS viewIfPermitted(...)`, including `viewIfPermitted` nested in another table function such as `remote`, which now fails with `BAD_ARGUMENTS`, as it already does for `eval` and the `*Cluster` table functions. The function chooses between its `SELECT` query and the `ELSE` function according to the current user, but a persisted table is resolved with no user and therefore full access: reading one reported the source table's column names and types to a user with no grant on it, and the `ELSE` fallback was unreachable. Tables created before this change keep loading; drop them to close the disclosure. [#116885](https://github.com/ClickHouse/ClickHouse/pull/116885) ([Groene AI](https://github.com/groeneai)). * Fixed a `Filesystem` database serving a table from its internal cache without the `READ ON FILE` grant. The grant was checked only while a table name was resolved for the first time, so once any user holding it had read a name, every user with `SELECT` on the database was served the contents of the file. `EXISTS TABLE`, which requires only `SHOW TABLES`, answered from that cache and probed the directory without the grant too. Both now require it, and a user without it is refused before the existence of the file is observed. [#119029](https://github.com/ClickHouse/ClickHouse/pull/119029) ([Groene AI](https://github.com/groeneai)). * Apply `remote_url_allow_hosts` before SQL-supplied S3 and Azure endpoints send schema, metadata, or container requests. [#114357](https://github.com/ClickHouse/ClickHouse/pull/114357) ([Rory Shanks](https://github.com/rorylshanks)). * Fixed the server deserializing the payload of a `Data` packet that arrives on a connection before any query. In interserver mode the connection is not authenticated until the `Query` packet is processed, so such a packet was rejected only after its Native block had been read, which let an unauthenticated peer choose the column type the server constructed. The packet is now rejected before its payload is touched. [#118588](https://github.com/ClickHouse/ClickHouse/pull/118588) ([Groene AI](https://github.com/groeneai)). * Each code for a time-based one-time password is now accepted at most once, as required by RFC 6238: reusing an already used code, or a code for an earlier time step, is rejected. A user with `time_based_one_time_password` can successfully authenticate at most once per `period`. [#117563](https://github.com/ClickHouse/ClickHouse/pull/117563) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed wrong results when `join_algorithm = 'parallel_hash'` is combined with a sorted consumer such as `optimize_aggregation_in_order`, `optimize_distinct_in_order` or `LIMIT BY`. With several join slots and a single-level hash map the join does not keep the order of the left table, so the read-in-order-through-join optimization must not advertise the left sort order in that case. [#109225](https://github.com/ClickHouse/ClickHouse/pull/109225) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Join is supported only for pipelines with one output port`) raised by a merge `JOIN` when its two sides reached the join with a different number of output ports. [#109393](https://github.com/ClickHouse/ClickHouse/pull/109393) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a doubly negated expression is compared with a constant, such as `WHERE (NOT NOT c) > -0.5`. Index analysis cancelled the two `NOT`s and compared the column itself with the constant, pruning parts, granules and partitions that do match. [#116381](https://github.com/ClickHouse/ClickHouse/pull/116381) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results after `ALTER TABLE ... MODIFY ORDER BY` on a table with a descending sorting key column (`ORDER BY a DESC`). The `ALTER` silently dropped the `DESC` direction while existing parts stayed physically sorted the old way, so primary key index analysis pruned the wrong granules and queries returned missing rows. Such an `ALTER` is now rejected. [#116473](https://github.com/ClickHouse/ClickHouse/pull/116473) ([Groene AI](https://github.com/groeneai)). * Fix the `Invalid status NotActive for associated output` logical error that could happen for queries with a high `max_threads`, for example a `GROUPING SETS` aggregation over a remote table. [#116478](https://github.com/ClickHouse/ClickHouse/pull/116478) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed reading uninitialized memory as literal data when decompressing a malformed `LZ4`-compressed block. [#116992](https://github.com/ClickHouse/ClickHouse/pull/116992) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix silently missing rows for a predicate on `toMillisecond`, `toMicrosecond` or `toNanosecond` of a `DateTime64` primary key. All three claimed to be monotonic everywhere, although they restart at zero every second, so primary-key analysis treated the predicate as one contiguous key range and pruned granules that contain matching rows. [#117246](https://github.com/ClickHouse/ClickHouse/pull/117246) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong row counts from the default-enabled `query_plan_merge_filter_into_join_condition` when an equality conjunct of the `WHERE` filter contains a non-deterministic function such as `rand()`. The conjunct was moved into the join condition, where it is evaluated once per join input row instead of once per output row, and cloned into the build-side runtime filter as a second evaluation site. [#117247](https://github.com/ClickHouse/ClickHouse/pull/117247) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the default-enabled `query_plan_merge_expression_into_join` returning rows that violate the query's own `JOIN ON` condition when a subquery computes a non-deterministic expression that is then used as a join key. The merged expression was computed twice from the raw inputs, drawing independently for the match and for the output column. [#117250](https://github.com/ClickHouse/ClickHouse/pull/117250) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the default-enabled `use_join_disjunctions_push_down` silently dropping rows when the filter above a join hides a non-deterministic function inside a lambda body, as in `arrayExists(z -> rand(z) % 2 = 0, [a])`. The extracted pre-filter was pushed below the join while the original filter stayed on top, so the lambda was drawn twice per row and only rows passing both draws survived. [#112700](https://github.com/ClickHouse/ClickHouse/pull/112700) ([Groene AI](https://github.com/groeneai)). * Fixed a filter containing a non-deterministic function being moved below `ARRAY JOIN`, which evaluated it once per source row instead of once per expanded row and made whole arrays pass or fail together. [#117756](https://github.com/ClickHouse/ClickHouse/pull/117756) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Fixed `rand`, `generateUUIDv4` and other non-deterministic functions being evaluated before an `arrayJoin` replicates the rows, which silently duplicated their values across the expanded rows. [#118963](https://github.com/ClickHouse/ClickHouse/pull/118963) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject a `CHECK` constraint whose expression contains `arrayJoin`. Such a constraint was accepted and then read past the end of the inserted block, surfacing as a nonsensical `Array of size 18446744073709551613 is too large`, or an abort in a debug build, or blamed a violation on the wrong row. [#117267](https://github.com/ClickHouse/ClickHouse/pull/117267) ([Alexey Milovidov](https://github.com/alexey-milovidov)). `arrayJoin` in a table constraint is now also rejected when the definition is copied by `CREATE TABLE ... AS src` or `CLONE AS src`, stated by a full-definition `ATTACH`, or declared as `ASSUME`. A constraint containing it that is already stored in the metadata of a table is no longer used by the query-time constraint optimizer, which could remove a `WHERE` that the constraint did not really imply. [#117799](https://github.com/ClickHouse/ClickHouse/pull/117799) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject a TTL expression that contains `arrayJoin`. Such a TTL was accepted and then silently deleted rows whose own TTL was far in the future, and read past the end of the expression column when an array was empty. [#117266](https://github.com/ClickHouse/ClickHouse/pull/117266) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a silent result change for a query over a trivial view over a `Distributed` table whose `WHERE` or view-keyed `additional_table_filters` contains a bare `IN some_table` predicate. The default-enabled `optimize_trivial_view_pushdown_to_distributed` shipped that predicate to the shards, where it was evaluated against each shard's local table of that name instead of the initiator's, so it disagreed with the `IN (SELECT ...)` spelling of the same predicate. [#117278](https://github.com/ClickHouse/ClickHouse/pull/117278) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `query_plan_aggregation_bucket_top_k` silently dropping result rows when an `arrayJoin` sits between the aggregation and the `ORDER BY ... LIMIT`. [#117464](https://github.com/ClickHouse/ClickHouse/pull/117464) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `SELECT ... FINAL` with a `WHERE` on the partition key expression returning a superseded row when the partition key is computed from a floating-point sorting-key column: partition pruning dropped the partition holding the deduplication winner. [#117509](https://github.com/ClickHouse/ClickHouse/pull/117509) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `mutations_sync = 3`, `lightweight_deletes_sync = 3` and `alter_sync = 3` performing no wait at all on `ReplicatedMergeTree`, so a mutation or `ALTER` returned as if it had synchronised. As documented, the value now behaves the same as `2` and waits for all replicas. This also affects `OPTIMIZE`, `TRUNCATE`, `DROP`/`DETACH PARTITION`, `REPLACE PARTITION` and `MOVE PARTITION TO TABLE`, which take the same `alter_sync` wait path. Statements that previously returned immediately now wait, and can raise `UNFINISHED` when a replica is inactive; set the value to `0`, `1` or `2` to choose a different wait explicitly. [#117517](https://github.com/ClickHouse/ClickHouse/pull/117517) ([Groene AI](https://github.com/groeneai)). * Fixed the logical error `Cannot add ZooKeeper operation because query is executed` when running `TRUNCATE TABLE` on a `TimeSeries` table in a `Replicated` database. [#117902](https://github.com/ClickHouse/ClickHouse/pull/117902) ([Raúl Marín](https://github.com/Algunenano)). * Fix a lightweight `UPDATE` silently not applied to rows of a data part when the patch part is written in the `v1` format (`patch_parts_version = 'v1'`) and covers rows of several data parts. One row was lost when a data part's patch rows began in a later granule of the patch part, which made the result depend on `merge_tree_min_read_task_size` and `max_block_size`. All updates for a data part were lost, on disk as well as at read time, when the patch part had no final mark (`index_granularity_bytes = 0`) and that part's patch rows began in the patch part's last granule. [#117962](https://github.com/ClickHouse/ClickHouse/pull/117962) ([Groene AI](https://github.com/groeneai)). * Fix wrong results and a `NOT_IMPLEMENTED` exception when a lightweight update's patch part is applied on top of a pending `ALTER TABLE ... UPDATE col = ` with `apply_mutations_on_fly = 1`. The constant-folded column was not materialized before the patch was written into it, so the patch was either silently discarded or the read failed. [#118545](https://github.com/ClickHouse/ClickHouse/pull/118545) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `system.tables.total_rows`/`total_bytes` overcounting a non-replicated `MergeTree` table while a patch part from a lightweight `UPDATE` is alive, and `SELECT count()` returning that inflated number with `apply_patch_parts = 0`. [#117463](https://github.com/ClickHouse/ClickHouse/pull/117463) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a query with a `hasToken`-family predicate on a `text` index silently returning no rows when the query also read a column modified by a pending lightweight update. [#118496](https://github.com/ClickHouse/ClickHouse/pull/118496) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix excessive patch-part reads for lightweight updates on tables with redundant parentheses in sorting-key expressions, such as `ORDER BY (k)`. [#118878](https://github.com/ClickHouse/ClickHouse/pull/118878) ([Anton Popov](https://github.com/CurtizJ)). * Fixed the `LOGICAL_ERROR` `Reading from materialized CTE '...' before its materialization completed` raised by a lightweight `UPDATE`, or a lightweight `DELETE` in `lightweight_update_force` mode, whose predicate contains an `IN` subquery that defines a materialized CTE, when the set is built at run time. [#119489](https://github.com/ClickHouse/ClickHouse/pull/119489) ([Dmitry Novik](https://github.com/novikd)). * Fixed a retried `INSERT SELECT` over a join not being deduplicated when the two attempts differed only in whether a column was carried as a lazily replicated column. [#118004](https://github.com/ClickHouse/ClickHouse/pull/118004) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a logical error `Cannot write to finalized buffer` (an abort in debug and sanitizer builds) in the non-replicated `MergeTree` deduplication log when opening the next log file during rotation failed, for example after a transient I/O error on the deduplication log's disk. Such a failure no longer fails the insert that triggered the rotation, so a retry of that insert is not wrongly deduplicated. [#118089](https://github.com/ClickHouse/ClickHouse/pull/118089) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix silent data loss on upgrade in `Distributed` background inserts with `background_insert_split_batch_on_failure`: batch recovery no longer deletes a spool file that was never sent when a later file of the same batch was quarantined into `broken/` by a previous server version. [#118211](https://github.com/ClickHouse/ClickHouse/pull/118211) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `SELECT count()` returning too many rows for a negated comparison (`!=`, `NOT IN`, `NOT LIKE`) over a `Nullable` primary key column: the exact-count optimization counted the NULL-key rows that the `WHERE` condition rejects. [#118214](https://github.com/ClickHouse/ClickHouse/pull/118214) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `AMBIGUOUS_COLUMN_NAME` on every read of a table that has a column named like a subcolumn of another column (for example `a.size0` next to an `Array` column `a`) after `ALTER TABLE ... MODIFY COLUMN` changed that column's type. [#118228](https://github.com/ClickHouse/ClickHouse/pull/118228) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix unbounded memory growth from time zone names of the form `Fixed/UTC±HH:MM:SS`. Every distinct name permanently cost ~4.6 MiB that the server never gave back, and 172801 of them were accepted. Only offsets that a time zone can have are accepted now: a whole number of quarters of an hour, no further from UTC than 14 hours. [#118286](https://github.com/ClickHouse/ClickHouse/pull/118286) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `MULTIPLE_EXPRESSIONS_FOR_ALIAS` under `rewrite_in_to_join` for a query that repeats one identical aliased `IN (subquery)` expression more than once. [#118292](https://github.com/ClickHouse/ClickHouse/pull/118292) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A table created with `CREATE TABLE ... AS
(...)` is no longer resolved to answer a `system.columns` size query about it. Previously selecting `data_compressed_bytes`, `data_uncompressed_bytes` or `marks_bytes` contacted the endpoint of every such table, so one unreachable endpoint made the whole scan fail. Until such a table is first accessed it now reports zero sizes. [#118302](https://github.com/ClickHouse/ClickHouse/pull/118302) ([Groene AI](https://github.com/groeneai)). * Fixed an `Inconsistent AST formatting` logical error for an aliased `IN` expression whose left operand also has an alias when it is nested inside a function-argument list, e.g. `SELECT ifNull(sum((x AS a1) IN [y] AS c3 = TRUE), 0)`. [#110229](https://github.com/ClickHouse/ClickHouse/pull/110229) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (server abort in debug/sanitizer builds) when a constant-folded comparison appears in the `SELECT` list together with `group_by_use_nulls = 1` and `GROUP BY GROUPING SETS`. [#110703](https://github.com/ClickHouse/ClickHouse/pull/110703) ([Groene AI](https://github.com/groeneai)). Fixed a `Bad cast` error, and wrong results in release builds, when an aggregate reads a derived table that uses `GROUP BY ALL` with `ROLLUP` or `CUBE` under `group_by_use_nulls = 1`. [#110921](https://github.com/ClickHouse/ClickHouse/pull/110921) ([Groene AI](https://github.com/groeneai)). Fixed a return type mismatch error when `group_by_use_nulls` is used with `GLOBAL IN` in a `GROUP BY` key. [#116980](https://github.com/ClickHouse/ClickHouse/pull/116980) ([George Larionov](https://github.com/george-larionov)). * Fixed a `LOGICAL_ERROR` ("query tree node does not have valid source node") when a `Merge` table wrapping a `Distributed` table is joined and a bare column of the joined-away side is used as a predicate. [#111327](https://github.com/ClickHouse/ClickHouse/pull/111327) ([Groene AI](https://github.com/groeneai)). * Fixes a `LOGICAL_ERROR` (`Column identifier ... is already registered`), a `MULTIPLE_EXPRESSIONS_FOR_ALIAS` error and a `NOT_FOUND_COLUMN_IN_BLOCK` error for a query whose `IN` / `NOT IN` / `GLOBAL IN` right operand names a table expression that also appears in the `FROM` section, for example `SELECT l.x FROM l JOIN (SELECT x FROM r) AS r ON l.x = r.x WHERE l.x IN r`. Also makes `ORDER BY WITH FILL ... INTERPOLATE ( ...)` inside a CTE or any other copied subquery raise `INVALID_WITH_FILL_EXPRESSION`, as the same query already does when it is not copied: the check was silently skipped for the copy, and such a query returned rows instead of the error. [#112219](https://github.com/ClickHouse/ClickHouse/pull/112219) ([Groene AI](https://github.com/groeneai)). * Fixes `NOT_FOUND_COLUMN_IN_BLOCK` when `query_plan_join_shard_by_pk_ranges = 1` is combined with `join_algorithm = 'full_sorting_merge'` and a join input is filtered on a sorting-key column that is not the join key and is not selected. Closes [#112402](https://github.com/ClickHouse/ClickHouse/issues/112402). [#112664](https://github.com/ClickHouse/ClickHouse/pull/112664) ([Groene AI](https://github.com/groeneai)). Fixed rows being silently dropped from a join when `query_plan_join_shard_by_pk_ranges` is enabled and the primary key is a floating-point column containing `NaN`. [#118918](https://github.com/ClickHouse/ClickHouse/pull/118918) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Code: 476` (`Can not use WITH TIES without ORDER BY`) raised by a remote shard, and the logical error `LIMIT WITH TIES without ORDER BY` raised on the initiator, when selecting from a `Merge` table over a `Distributed` table with a `JOIN` and `ORDER BY ... LIMIT n WITH TIES`. The child query kept its `LIMIT` clause after its `ORDER BY` was removed. [#113594](https://github.com/ClickHouse/ClickHouse/pull/113594) ([Groene AI](https://github.com/groeneai)). * Fixed `LOGICAL_ERROR: There are no events in epoll` when a distributed query with `use_hedged_requests` is cancelled or completes early (for example because of a `LIMIT`) while a replacement replica is still pending. Closes [#113981](https://github.com/ClickHouse/ClickHouse/issues/113981). [#114065](https://github.com/ClickHouse/ClickHouse/pull/114065) ([Groene AI](https://github.com/groeneai)). * Fixed `... PARTITION ` statements against a table whose `PARTITION BY` contains `modulo` with an unsigned left operand and a signed right operand, such as `PARTITION BY (37528 % c0)` with `c0 Int32`. Such a statement raised `LOGICAL_ERROR`, or silently affected no partition. A value outside the range of the partition key type is now rejected with `ARGUMENT_OUT_OF_BOUND` instead of silently matching nothing. [#116606](https://github.com/ClickHouse/ClickHouse/pull/116606) ([Groene AI](https://github.com/groeneai)). * Fix `BAD_ARGUMENTS` for a query that combines several `quantile` calls where at least one level is written as an integer literal, for example `SELECT quantile(1)(x), quantile(0.5)(x)`. The default-enabled `optimize_syntax_fuse_functions` rejected the integer level while the aggregate function itself accepts it, so the query failed although each call is valid on its own. [#117211](https://github.com/ClickHouse/ClickHouse/pull/117211) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results for `GROUP BY`, `LIMIT BY` and `uniq*` over `bech32Encode`, `dictGetHierarchy` and `reverseUTF8`: the three functions declared themselves injective, so the default-enabled `optimize_injective_functions_in_group_by` and `optimize_injective_functions_inside_uniq` grouped and counted by the raw argument, although distinct inputs can produce the same result. [#117239](https://github.com/ClickHouse/ClickHouse/pull/117239) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#117357](https://github.com/ClickHouse/ClickHouse/pull/117357) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118432](https://github.com/ClickHouse/ClickHouse/pull/118432) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `has`, `indexOf` and `countEqual` over an array of numbers returning a match for a needle that `equals` does not consider equal: `has(arr, -1)` matched an `Array(UInt64)` element equal to the `UInt64` maximum, and `has(arr, 16777217)` matched an `Array(Float32)` element equal to `16777216`. The constant-array path already answered correctly, so the same haystack gave different results depending on whether it was a constant or a materialized column. The same phantom matches were returned for an `Array(LowCardinality(T))` column, which also matched a needle its element type cannot represent outside the numeric domain and reported every element equal to the default value as a match for a `NULL` needle. In the other direction, a needle counting a different unit than the element, such as a `DateTime` searched in an `Array(Date)`, was found nowhere although `equals` considers the pair equal; such a pair is now compared in the type the two meet in. [#117313](https://github.com/ClickHouse/ClickHouse/pull/117313) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `uniqTheta` and its variants silently returning a value that is too low when reading an `AggregateFunction(uniqTheta, ...)` state that another Apache DataSketches implementation had written in the compressed Theta sketch form. [#117422](https://github.com/ClickHouse/ClickHouse/pull/117422) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed undefined behavior and an empty result of a `SELECT ... ORDER BY LIMIT n OFFSET m` over a table with a `vector_similarity` index when `n + m`, or `LIMIT` multiplied by `vector_search_index_fetch_multiplier`, overflows `UInt64`; such a query now skips the index and runs the exact search. [#117713](https://github.com/ClickHouse/ClickHouse/pull/117713) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#117892](https://github.com/ClickHouse/ClickHouse/pull/117892) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Invalid number of columns in chunk pushed to OutputPort`) for a vector search over a column with a `Quantized(...)` codec when `vector_search_use_quantized_codes` and the experimental `make_distributed_plan` were both enabled. [#118850](https://github.com/ClickHouse/ClickHouse/pull/118850) ([Groene AI](https://github.com/groeneai)). * Fixed an external `GROUP BY` with `enable_adaptive_aggregator` exceeding `max_memory_usage` or failing with `MEMORY_LIMIT_EXCEEDED` inside the aggregator's own memory valve: the working set of the memory-pressure sweeps was sized from constants unrelated to `max_bytes_before_external_group_by`, and records staged before the aggregator thawed its tables stayed resident until the merge, so a thawed thread wrote a temporary file per block. Both are now bounded by that threshold. [#117880](https://github.com/ClickHouse/ClickHouse/pull/117880) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118474](https://github.com/ClickHouse/ClickHouse/pull/118474) ([Groene AI](https://github.com/groeneai)). * Fixed a comment `ALTER` on a `KeeperMap` table in a `Replicated` database being applied only to the local replica. Keeper kept the pre-`ALTER` `CREATE` statement and digest, so other replicas never saw the change and the next start declared the replica lost, reverting the `ALTER` and, if no other table shared its `KeeperMap` path, dropping its data. [#118011](https://github.com/ClickHouse/ClickHouse/pull/118011) ([Groene AI](https://github.com/groeneai)). * Fixes `bloom_filter` skip indexes on big-endian platforms (s390x), where the index payload was never written or read, so a query using such an index returned no rows for data that is present. [#118164](https://github.com/ClickHouse/ClickHouse/pull/118164) ([Vadim Ilves](https://github.com/vadimilves)). * Fixed `arrayIntersect` returning a spurious `NULL` when it is missing from one of the input arrays. [#118312](https://github.com/ClickHouse/ClickHouse/pull/118312) ([Minh Vu](https://github.com/fallintoplace)). Fixed `arrayIntersect`, `arrayUnion` and `arraySymmetricDifference` mishandling a `Nullable` argument whose elements do not fit the common element type: `arrayIntersect([toNullable(1)], [toNullable(257)])` returned `[1]`, and `arrayUnion([1], [toNullable(257)])` failed with `ILLEGAL_COLUMN` instead of returning `[1,257]`. [#118485](https://github.com/ClickHouse/ClickHouse/pull/118485) ([Groene AI](https://github.com/groeneai)). * Fixed a `UInt64` constant that does not fit `UInt32` matching an unrelated row of a `DateTime` column in `IN` (and in an `OR` chain of equalities, which is rewritten into `IN`): the constant was truncated modulo 2^32 instead of being excluded from the set. [#118367](https://github.com/ClickHouse/ClickHouse/pull/118367) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `CORRUPTED_DATA` ("Cannot get value from Set") for a `FINAL` query with `ORDER BY ... WITH FILL INTERPOLATE` whose filter is an `IN` set moved into `PREWHERE`: the set placeholder column is no longer kept alive past the filter. [#118420](https://github.com/ClickHouse/ClickHouse/pull/118420) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `lcm` returning a negative result for `Int128` and `Int256` arguments when exactly one of them is negative, e.g. `lcm(toInt128(-6), toInt128(4))` returned `-12` instead of `12`. [#118431](https://github.com/ClickHouse/ClickHouse/pull/118431) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `text`, `tokenbf_v1` and `ngrambf_v1` skip indexes not being used when the searched constant has type `LowCardinality(String)` (e.g. `hasToken(s, toLowCardinality('rare'))` or `s = toLowCardinality('rare')`), which caused a silent full scan. Such predicates now prune granules like their plain `String` counterparts. [#110017](https://github.com/ClickHouse/ClickHouse/pull/110017) ([Groene AI](https://github.com/groeneai)). * Fix a `text` or `ngrambf_v1` index silently dropping matching rows when the indexed column is compared for equality against a `FixedString` value, or searched with `hasAny` or `hasAll` over `FixedString` needles. Happens at default settings, since `use_skip_indexes` and `enable_full_text_index` are both on. [#117314](https://github.com/ClickHouse/ClickHouse/pull/117314) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix rows silently dropped by an `ngrambf_v1` index for a `LIKE` pattern with a run of escaped backslashes before a wildcard. [#118673](https://github.com/ClickHouse/ClickHouse/pull/118673) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `bloom_filter` skip index dropping granules that hold matching rows for `hasAny`/`hasAll` against a constant array whose elements are of a different date/time type than the indexed column, and `TYPE_MISMATCH` for a `Date32` element. [#118956](https://github.com/ClickHouse/ClickHouse/pull/118956) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results when a table declares both a `Map` column `m` and a column named exactly `` `m.key_` `` (dots are legal in column names), or has that name claimed by a subcolumn such as a `Tuple` element or a typed JSON path. Skip-index analysis read the name as the map's key subcolumn and pruned every granule whose map lacks ``, so a `mapKeys` or `mapValues` index (`ngrambf_v1`, `tokenbf_v1`, `sparse_grams`, `bloom_filter`, `text`) dropped rows that matched the predicate. [#119457](https://github.com/ClickHouse/ClickHouse/pull/119457) ([Groene AI](https://github.com/groeneai)). * Fixes incorrect primary-key pruning for `has` and `notHas` when array element types differ from the key type, such as `String` literals against `Enum` or `FixedString` keys. [#115294](https://github.com/ClickHouse/ClickHouse/pull/115294) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed rows being silently dropped by partition pruning, primary key analysis and `minmax` skip indexes when a `NaN` is involved: a comparison constant that holds a `NaN`, possibly nested in a tuple or appearing only after conversion to the key type, as in `NOT (f < 'nan')`. [#116763](https://github.com/ClickHouse/ClickHouse/pull/116763) ([Groene AI](https://github.com/groeneai)). A string constant such as `'nan'` compared with a floating point column under an injective key transform such as `ORDER BY toString(f)`, so `f != 'nan'` skipped the granule holding the `NaN` row. [#117501](https://github.com/ClickHouse/ClickHouse/pull/117501) ([Groene AI](https://github.com/groeneai)). A `Float` primary key holding `±inf` where the predicate's arithmetic maps that endpoint to `NaN`, e.g. `WHERE k / inf = 0`. [#118468](https://github.com/ClickHouse/ClickHouse/pull/118468) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results when `indexHint` is given an argument that index analysis cannot read as a condition. `indexHint(256)` was narrowed to `CAST(256, 'UInt8')` = 0 and discarded every part, so the query silently returned an empty result; a wide integer such as `indexHint(toUInt256(256))` did the same, and a `String`, `Decimal` or `NULL` argument threw. Such an argument now contributes no filter, consistently for the primary key, `minmax`, `set` and `bloom_filter` analysis. Closes [#111684](https://github.com/ClickHouse/ClickHouse/issues/111684). [#114048](https://github.com/ClickHouse/ClickHouse/pull/114048) ([Groene AI](https://github.com/groeneai)). Also fixed a bare key column of a type with no boolean reading (`UInt256`, `Int128`, `BFloat16`, or their `LowCardinality` forms) being read as `column != 0` inside `indexHint`, `NOT indexHint(x)` failing with `ILLEGAL_TYPE_OF_ARGUMENT`, and a negated `indexHint` in a disjunction keeping its condition un-inverted, so `count()` could report rows that no row of the table matches. [#119209](https://github.com/ClickHouse/ClickHouse/pull/119209) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from `optimize_functions_to_subcolumns`. A `Tuple` element or a `JSON` path named like an automatic subcolumn could be mistaken for it, so for example `length(c.a)` on ``Tuple(`a.size` UInt64, `a` String)`` returned the element instead of the length; `empty`, `notEmpty`, `isNull`, `isNotNull`, `count`, `mapKeys`, `mapValues`, `mapContains`, and `m['key']` were affected the same way. Separately, `length` on an `Array` or `Map` inside a `Nullable` (for example ``Nullable(JSON(`a` Array(Int64)))``) returned `NULL` instead of `0` for a `NULL` row, `tupleElement` on a member of a `Nullable(Tuple(...))` column could fail with `LOGICAL_ERROR`, and a function is now rewritten only to a subcolumn of the same root column, so a physical column such as `` `j.m` `` is no longer confused with the `m` path of `JSON` column `j`. [#115777](https://github.com/ClickHouse/ClickHouse/pull/115777) ([Pavel Kruglov](https://github.com/Avogar)). [#119150](https://github.com/ClickHouse/ClickHouse/pull/119150) ([Groene AI](https://github.com/groeneai)). [#119631](https://github.com/ClickHouse/ClickHouse/pull/119631) ([Groene AI](https://github.com/groeneai)). * Declaring `STATISTICS(...)` on an `ALIAS` or `EPHEMERAL` column is now rejected when the table is created or altered, instead of being accepted and making every subsequent `INSERT` fail with `NOT_FOUND_COLUMN_IN_BLOCK`. Closes [#114791](https://github.com/ClickHouse/ClickHouse/issues/114791). [#115882](https://github.com/ClickHouse/ClickHouse/pull/115882) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE TABLE` accepting a subcolumn as a `UNIQUE KEY` element, of either a user column (for example `` UNIQUE KEY (`c.null`) ``) or a virtual one (`` UNIQUE KEY (`_partition_value.1`) ``), which produced a table that could never be inserted into, failing every `INSERT` with `NOT_FOUND_COLUMN_IN_BLOCK`. Such a definition is now rejected with `BAD_ARGUMENTS`. Closes [#114470](https://github.com/ClickHouse/ClickHouse/issues/114470). [#115854](https://github.com/ClickHouse/ClickHouse/pull/115854) ([Groene AI](https://github.com/groeneai)). * Fixed an in-process Keeper client (`KeeperOverDispatcher`) reporting a failed `multi` transaction as successful, because a rolled-back batch's aggregate error was left `ZOK`. [#117135](https://github.com/ClickHouse/ClickHouse/pull/117135) ([Sergei Trifonov](https://github.com/serxa)). * Fixed OOM when reading a `Parquet` file with an invalid `num_children` field in the schema. [#117797](https://github.com/ClickHouse/ClickHouse/pull/117797) ([Michael Kolupaev](https://github.com/al13n321)). * Fixed a `Parquet` reader bug where a filter on a `UUID` column could silently return fewer rows than the file holds, or reject a valid file with `INCORRECT_DATA: Statistics have min_value > max_value`. Parquet sorts `uuid` statistics by big-endian bytes while ClickHouse sorts `UUID` by its second half, so those statistics are no longer used for row group or page pruning. Closes [#118371](https://github.com/ClickHouse/ClickHouse/issues/118371). [#118387](https://github.com/ClickHouse/ClickHouse/pull/118387) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when reading a `Parquet` file with a nullable struct column without an explicit schema. Schema inference pushed the nullability down to the tuple elements and returned `Tuple(Nullable(...), ...)`, which cannot hold a struct-level NULL, so a NULL struct was read as a tuple of NULLs. Such a column is now inferred as `Nullable(Tuple(...))` when `allow_experimental_nullable_tuple_type` is enabled. That setting was also added to the `Parquet` schema inference cache key, so a schema cached under one of its values is no longer reused under the other. [#115653](https://github.com/ClickHouse/ClickHouse/pull/115653) ([Groene AI](https://github.com/groeneai)). * Fix reading `Arrow` and `ArrowStream` files containing values hidden by NULLs or unused nested ranges, which could cause validation errors or expose undefined values. Fix nullable tuple conversions and destination defaults, dictionary decoding under requested types, and NULL dictionary entries for complex types. Accept compatible shared dictionaries with differing field annotations and reject replacement dictionaries in `Arrow` files. Validate nested row counts before allocation and read bufferless batches incrementally without materializing their entire declared row count. [#110531](https://github.com/ClickHouse/ClickHouse/pull/110531) ([Nihal Z. Miaji](https://github.com/nihalzp)). Fix reading a flattened `Tuple` subcolumn (e.g. `` `s.v` ``) from an `Arrow`/`ArrowStream` file whose column is a nullable struct returning column defaults instead of the actual data when `allow_experimental_nullable_tuple_type` is enabled. Also fix `ORC` reads resolving a column or struct field name to the first case-folded match under `input_format_orc_case_insensitive_column_matching`, which returned another field's data when two names differ only by case. [#109741](https://github.com/ClickHouse/ClickHouse/pull/109741) ([Groene AI](https://github.com/groeneai)). * Fix a crash and wrong results when reading the `DWARF` format with column types other than the ones the format produces, for example `SELECT * FROM file('a.elf', DWARF, 'tag String')`. Requesting a column that does not exist in the format now reports a proper error instead of `std::out_of_range`. [#118794](https://github.com/ClickHouse/ClickHouse/pull/118794) ([Raúl Marín](https://github.com/Algunenano)). * Fix data silently dropped when reading a hadoop-snappy stream whose block has several subblocks and the input is refilled in the middle of that block, which happens when the file is read over a pipe or HTTP. [#118705](https://github.com/ClickHouse/ClickHouse/pull/118705) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed two defects in the legacy binary settings format, used for native-protocol clients declaring a revision below 54429. Signed integer settings (for example `max_partitions_to_read` or `iceberg_snapshot_id`) were silently ignored, and were reported as changed while holding their default, so they overrode the table-level value with that default. Settings required to be greater than zero (for example `max_block_size`) were not validated when read from that format. [#118610](https://github.com/ClickHouse/ClickHouse/pull/118610) ([Groene AI](https://github.com/groeneai)). * Fixed a query over an `Iceberg` table returning rows that had been deleted, when a position-delete manifest declares a lower bound above the upper bound for the reserved `file_path` column. Such a pair describes an empty range, so the delete file was skipped for every data file it covers; those bounds are now ignored instead. Closes [#118113](https://github.com/ClickHouse/ClickHouse/issues/118113). [#118181](https://github.com/ClickHouse/ClickHouse/pull/118181) ([Groene AI](https://github.com/groeneai)). * Fixed reading an `Iceberg` table failing with `ICEBERG_SPECIFICATION_VIOLATION` after a column was dropped while its data is still present in the data files. Closes [#113324](https://github.com/ClickHouse/ClickHouse/issues/113324). [#116488](https://github.com/ClickHouse/ClickHouse/pull/116488) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed `write_full_path_in_iceberg_metadata` writing an unresolvable `local://` URI, with a doubled root slash, into the metadata of an `IcebergLocal` table, so that no external Iceberg reader could open it. A local table location is now spelled `file:///path`, and a backend without a namespace no longer contributes an empty authority segment. [#117828](https://github.com/ClickHouse/ClickHouse/pull/117828) ([Groene AI](https://github.com/groeneai)). * Fixed writing `Iceberg` statistics on top of Azure storage. [#119369](https://github.com/ClickHouse/ClickHouse/pull/119369) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `ATTACH TABLE` of a data lake table no longer reads anything from object storage, and such tables no longer report row statistics in system tables. [#117584](https://github.com/ClickHouse/ClickHouse/pull/117584) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Reading a table backed by local object storage (`paimonLocal`, `icebergLocal`, `deltaLakeLocal`, or a disk with `object_storage_type: local`) whose metadata references a file that is missing on disk now fails with a `FILE_DOESNT_EXIST` exception naming the file, instead of a raw `std::filesystem` error surfaced as the opaque `STD_EXCEPTION` (code 1001). [#115542](https://github.com/ClickHouse/ClickHouse/pull/115542) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fixed a `SELECT` from a table with `paimon_incremental_read = 1` failing with `REPLICA_IS_ALREADY_ACTIVE` ("Failed to activate Paimon replica after Keeper reconnection") on the first read after the server's Keeper session was lost and re-established. [#118457](https://github.com/ClickHouse/ClickHouse/pull/118457) ([Groene AI](https://github.com/groeneai)). * Fixed a spurious `Code: 76 CANNOT_OPEN_FILE` error (`Parent directory doesn't exist`) on any write to an `hdfs` disk. Removing a blob deleted its emptied object-key prefix directory, which could race a concurrent write that had just created the same directory and not yet opened its object. Emptied prefix directories are now kept. [#118654](https://github.com/ClickHouse/ClickHouse/pull/118654) ([Groene AI](https://github.com/groeneai)). * Fixed a server refusing to start after `remote_url_allow_hosts` was tightened while a database with the `HDFS` engine existed: the host check ran during metadata loading and took every other database down with it. [#118964](https://github.com/ClickHouse/ClickHouse/pull/118964) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server that refused to start after the directory backing a `Filesystem` database had been removed. Such a database is now loaded without tables and can be inspected and dropped, while a `CREATE DATABASE` or `ATTACH DATABASE` issued by a user is still rejected when the path does not exist. [#118684](https://github.com/ClickHouse/ClickHouse/pull/118684) ([Groene AI](https://github.com/groeneai)). * With `enable_analyzer = 1` (the default), fixed `INCOMPATIBLE_TYPE_OF_JOIN` and `NOT_FOUND_COLUMN_IN_BLOCK` errors when an `ALL INNER JOIN` against a table with the `Join` table engine carried an `ON` condition referencing only that table, such as `ON t.key = j.key AND j.value > 100`. The condition is now applied after the join, which is what the equivalent `WHERE` spelling already did. Closes [#116541](https://github.com/ClickHouse/ClickHouse/issues/116541). [#116612](https://github.com/ClickHouse/ClickHouse/pull/116612) ([Groene AI](https://github.com/groeneai)). * Fixed uneven partition distribution in the `Kafka` engine with `kafka_keeper_path` (`StorageKafka2`) when `kafka_num_consumers` is greater than 1. A node running N consumers could claim N times its fair share of the topic's partitions and hold them indefinitely, leaving other replicas idle. [#116843](https://github.com/ClickHouse/ClickHouse/pull/116843) ([Aparajita Pandey](https://github.com/aparajita31pandey)). * `clickhouse-local` now preserves detailed `Poco::Exception` messages from local query execution instead of reporting them as generic `STD_EXCEPTION` errors. This includes errors returned by the `mysql` table function. [#116884](https://github.com/ClickHouse/ClickHouse/pull/116884) ([Mark Needham](https://github.com/mneedham)). * Fixed `HashJoin` right-side memory accounting when a memory limit interrupts a join build. A block could be stored without being counted, so the join under-reported its size to the memory limits and the spilling thresholds, and a `Join` engine table held the memory of a failed `INSERT` for the rest of its lifetime. [#116991](https://github.com/ClickHouse/ClickHouse/pull/116991) ([Groene AI](https://github.com/groeneai)). * Fix wrong aggregate values from the default-enabled `optimize_rewrite_aggregate_function_with_if` when the constant is in the first branch of the `if` and the condition can be NULL, for example `sum(if(c, 0, x))` or `avg(if(c, NULL, x))`. Rows whose condition was NULL were silently dropped from the aggregate. [#117221](https://github.com/ClickHouse/ClickHouse/pull/117221) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a signed integer overflow when the effective table lock timeout, the smaller of `lock_acquire_timeout` and a non-zero `max_execution_time`, was above roughly 9.2e9 seconds or a large negative value. Reading or writing a `Log`, `TinyLog`, `StripeLog` or `File` table built the lock deadline from it without bounding it first, so the wait was no longer bounded by the setting. [#117974](https://github.com/ClickHouse/ClickHouse/pull/117974) ([Groene AI](https://github.com/groeneai)). * `divideDecimal` no longer returns `0` instead of the correct value when the dividend reduces to a single digit after being rescaled to the result scale, for example `divideDecimal(toDecimal256(5, 0), toDecimal256(1, 0), 0)` or `divideDecimal(toDecimal32(5.6789, 4), toDecimal32(2, 0), 0)`. The bug was present in every release since 22.12, so any stored result of such a division is wrong and should be recomputed. [#118346](https://github.com/ClickHouse/ClickHouse/pull/118346) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `ClickHouseDictionarySource table cannot be dictionary table` being raised for a dictionary whose source is a table on another ClickHouse server that happens to share the dictionary's database and table name. The check against a dictionary using itself as its own source now applies only when the source is the local server. [#118441](https://github.com/ClickHouse/ClickHouse/pull/118441) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `arrayElementOrNull` over an array of tuples, and `arrayElement` over an array of `Nullable` tuples, returning a tuple of default values instead of `NULL` for a constant `UInt64` index close to the maximum of `UInt64`, such as `toUInt64(18446744073709551615)`. [#118447](https://github.com/ClickHouse/ClickHouse/pull/118447) ([Minh Vu](https://github.com/fallintoplace)). * Fixed a query parameter being reported as unset (`UNKNOWN_QUERY_PARAMETER`) when it is the entire `INTERPOLATE` expression, as in `ORDER BY x WITH FILL INTERPOLATE (value AS {p:Int64})`, while the same parameter nested in a larger expression worked. Closes [#118458](https://github.com/ClickHouse/ClickHouse/issues/118458). [#118568](https://github.com/ClickHouse/ClickHouse/pull/118568) ([Groene AI](https://github.com/groeneai)). The same for a parameter that is the entire predicate of a lightweight `DELETE`, as in `DELETE FROM t WHERE {p:UInt8}`. [#118667](https://github.com/ClickHouse/ClickHouse/pull/118667) ([Groene AI](https://github.com/groeneai)). * Fix partition pruning and primary key filtering being silently disabled when `parallel_replicas_min_number_of_rows_per_replica` is set inside a subquery or CTE instead of at top query level. [#116529](https://github.com/ClickHouse/ClickHouse/pull/116529) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed wrong results and a `LOGICAL_ERROR` for `SELECT ... FINAL` under `make_distributed_plan = 1` when a `PREWHERE` or a row policy is deferred until after `FINAL` (`apply_prewhere_after_final`, or the default `apply_row_policy_after_final`). [#116626](https://github.com/ClickHouse/ClickHouse/pull/116626) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from JIT-compiled conversions of a floating point value to an integer or `Decimal` type when the value does not fit the destination: a comparison against the converted value could be evaluated against the pre-conversion value, so the same query could flip between repeated executions. Such conversions are now evaluated by the interpreter, which also means they raise `CANNOT_CONVERT_TYPE` for `NaN` or `Inf` and `DECIMAL_OVERFLOW` on `Decimal` overflow where an arbitrary value was previously returned. Closes [#117442](https://github.com/ClickHouse/ClickHouse/issues/117442). [#117590](https://github.com/ClickHouse/ClickHouse/pull/117590) ([Groene AI](https://github.com/groeneai)). Fixed wrong results from a JIT-compiled `CAST` of a `Nullable` value to a non-`Nullable` type: converting a NULL must raise `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN`, but the compiled expression returned the nested value instead, so the same query silently started answering with wrong data once `min_count_to_compile_expression` was crossed. Such a conversion is no longer compiled. [#119079](https://github.com/ClickHouse/ClickHouse/pull/119079) ([Groene AI](https://github.com/groeneai)). * A simple `Query` in the PostgreSQL wire protocol now destroys the unnamed prepared statement and the unnamed portal, as PostgreSQL does, so a later `Bind` or `Execute` on them is rejected instead of silently reusing stale extended-query state. [#118210](https://github.com/ClickHouse/ClickHouse/pull/118210) ([Groene AI](https://github.com/groeneai)). * `portRFC` now returns the explicit port of a URL whose host is an IPv6 address in brackets, such as `http://[2001:db8::1]:8080/`, instead of the default port. [#118632](https://github.com/ClickHouse/ClickHouse/pull/118632) ([Alexey Milovidov](https://github.com/alexey-milovidov)). `port` and `portRFC` now return the explicit port for valid URLs with a query string or fragment after the port. [#118482](https://github.com/ClickHouse/ClickHouse/pull/118482) ([Minh Vu](https://github.com/fallintoplace)). * `fuzzBits` and `generateULID` now return an independently generated value per row when all of their arguments are constants, instead of one value repeated for every row. [#118643](https://github.com/ClickHouse/ClickHouse/pull/118643) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `EmbeddedRocksDB`, `KeeperMap` and `Redis` returning no rows for `WHERE key = `, such as a `Date` key compared with a `DateTime` literal. [#118689](https://github.com/ClickHouse/ClickHouse/pull/118689) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the row numbers shown in the Web UI for a page other than the first one: they restarted at 1 on every page instead of counting from the page's position in the whole result. [#116614](https://github.com/ClickHouse/ClickHouse/pull/116614) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix delayed ClickHouse Keeper shutdown when idle client sessions are blocked in socket polling. [#118014](https://github.com/ClickHouse/ClickHouse/pull/118014) ([Antonio Andelic](https://github.com/antonio2368)). Fix graceful shutdown of `clickhouse-keeper` hanging for up to 300 seconds when a client connection on `keeper_server.tcp_port_secure` is stalled in a TLS read. [#119450](https://github.com/ClickHouse/ClickHouse/pull/119450) ([Groene AI](https://github.com/groeneai)). * Fixed a Keeper crash when a member that was removed from the cluster is added back without its process being restarted. Such a member could also replay already-applied entries. [#118740](https://github.com/ClickHouse/ClickHouse/pull/118740) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix handling of regular expressions ending with an unescaped backslash in `match`, `extractAll`, `countMatches` and `splitByRegexp`. Such invalid patterns are no longer incorrectly treated as trivial substring matches. [#118229](https://github.com/ClickHouse/ClickHouse/pull/118229) ([Alex Prabhat Bara](https://github.com/alexprabhat99)). * Fixed `match` and `countMatches` returning wrong results for a pattern whose character class starts with a literal `]` (`[]a]b`) or contains a literal `[` next to an alternation (`abc[[]|b`): the required-substring prefilter demanded a substring the pattern does not require. The same defect made `optimize_rewrite_regexp_functions` turn such a `replaceRegexpAll` into `replaceRegexpOne`. [#118342](https://github.com/ClickHouse/ClickHouse/pull/118342) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `startsWith*` array functions returning suffix matches for some variants. [#118672](https://github.com/ClickHouse/ClickHouse/pull/118672) ([Minh Vu](https://github.com/fallintoplace)). * Fixed an out-of-bounds access that could happen with the `getURLHostRFC` function. [#119026](https://github.com/ClickHouse/ClickHouse/pull/119026) ([Groene AI](https://github.com/groeneai)). * Fix the HTTP connection pool comparing `disk_connections_store_limit`, `storage_connections_store_limit`, and `http_connections_store_limit` against the total number of connections in the group instead of the number of stored connections. A server whose concurrency exceeded the limit cached no connections at all, opened a new TCP connection for every request to object storage, and could exhaust the ephemeral port range of the host. The limit now bounds only the idle connections kept for reuse. [#119078](https://github.com/ClickHouse/ClickHouse/pull/119078) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the hash-family join algorithms treating a non-NULL `Nullable(Tuple(...))` key that contains a NULL element as a NULL key, which made the result of a join depend on the `join_algorithm` setting. `IN` with a subquery over such a key was affected in the same way. [#119081](https://github.com/ClickHouse/ClickHouse/pull/119081) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results when `analyzer_compatibility_join_using_top_level_identifier = 1` and a `JOIN ... USING` key resolves to a `SELECT`-list alias whose name is a virtual column of the left table expression, such as `SELECT upper(t._part) AS _part FROM t JOIN r USING (_part)`. The raw virtual column value was silently used as the join key instead of the aliased expression. [#119102](https://github.com/ClickHouse/ClickHouse/pull/119102) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Got read request from replica N for unknown stream
`) when `parallel_replicas_plan_based` is enabled and a join is distributed across replicas. [#118084](https://github.com/ClickHouse/ClickHouse/pull/118084) ([Groene AI](https://github.com/groeneai)). * Fixes a case where `date_time_overflow_behavior = 'saturate'` clamped an out-of-range value to the whole second and discarded the sub-second component. Closes [#113152](https://github.com/ClickHouse/ClickHouse/issues/113152). [#116122](https://github.com/ClickHouse/ClickHouse/pull/116122) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `system.dictionaries.bytes_allocated` misreporting memory for `HASHED_ARRAY` dictionaries: the key containers contributed their element count instead of their buffer size in bytes, so the reported figure came out roughly 2-5x lower than the memory the dictionary actually held, and the bit-packed per-attribute null masks were counted in elements, which over-reported that term by 8x. [#118641](https://github.com/ClickHouse/ClickHouse/pull/118641) ([George Viamontes](https://github.com/gviamont)). Fixed the same figure over-reporting memory for nullable `RANGE_HASHED` dictionaries, whose bit-packed nullable mask was counted one byte per flag. [#118675](https://github.com/ClickHouse/ClickHouse/pull/118675) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `max_execution_time_leaf` and `timeout_overflow_mode_leaf` being ignored for queries executed with parallel replicas (including `INSERT SELECT` with `parallel_distributed_insert_select = 2`). [#69540](https://github.com/ClickHouse/ClickHouse/pull/69540) ([Igor Nikonov](https://github.com/devcrafter)). * Fix `CREATE TABLE ... CLONE AS SELECT ...` and `CREATE TABLE ... CLONE AS ` silently ignoring the `CLONE` keyword and creating a plain `CREATE ... AS SELECT` instead. These shapes are now rejected with `BAD_ARGUMENTS`. [#105004](https://github.com/ClickHouse/ClickHouse/pull/105004) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix a logical error (a server abort in debug builds) when a non-transactional `DROP PARTITION`, `DETACH PARTITION`, `DROP PART`, `REPLACE PARTITION FROM` or `MOVE PARTITION TO TABLE` covered a part whose creating transaction had not committed yet. Such an operation now fails with `SERIALIZATION_ERROR` and can be retried once the creating transaction finishes; it no longer removes or replaces any part of the affected batch. [#105116](https://github.com/ClickHouse/ClickHouse/pull/105116) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix a logical error when a correlated `MATERIALIZED` CTE is referenced more than once inside a `JOIN`. Such a CTE is now rejected at analysis time with `UNSUPPORTED_METHOD`, consistently with the single-reference and `IN`-subquery paths. [#108883](https://github.com/ClickHouse/ClickHouse/pull/108883) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `Invalid cast from Nothing to native type` (an abort in debug/sanitizer builds) when `compile_expressions = 1` compiled an expression with a `Nullable(Nothing)` operand, e.g. `and(Nullable(Nothing), )`. [#111508](https://github.com/ClickHouse/ClickHouse/pull/111508) ([Groene AI](https://github.com/groeneai)). * Fix wrong results with `enable_parallel_replicas = 1` when a join tree contains a join other than the leftmost one that cannot be applied independently per replica: a strictness other than `ALL` outside a `LEFT` join, for example `t1 INNER JOIN t2 ON t1.c = t2.c ANY INNER JOIN t3 ON 1`, which returned the row once per replica instead of once; a `FULL` join, which emitted every row twice; or a `PASTE` join, which pairs rows by position. Parallel replicas are now disabled for such a join tree, as they already are for the equivalent two-table join. [#112317](https://github.com/ClickHouse/ClickHouse/pull/112317) ([Groene AI](https://github.com/groeneai)). * Fix a segfault and silently wrong results when reading a `sizeN` subcolumn of a nested `Array(JSON)` path. Closes [#113686](https://github.com/ClickHouse/ClickHouse/issues/113686). [#113752](https://github.com/ClickHouse/ClickHouse/pull/113752) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and a server abort when a `JOIN ON` condition evaluated during an `ie_join` join contains `arrayJoin`. Such a condition is now rejected with `INVALID_JOIN_ON_EXPRESSION`. Previously `SEMI`/`ANTI`/`LEFT`/`RIGHT`/`FULL` joins using the `ie_join` algorithm could return duplicated or missing rows, or abort the server. [#114325](https://github.com/ClickHouse/ClickHouse/pull/114325) ([Groene AI](https://github.com/groeneai)). * Fix reading a `Log` or `TinyLog` table whose array or map elements are missing while their sizes are present: such a table returned rows with elements that are not there and made the server read out of bounds. It is now reported as `INCORRECT_DATA`. [#115528](https://github.com/ClickHouse/ClickHouse/pull/115528) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a segmentation fault when reading in order from a `Merge` table over `MergeTree` tables whose sorting key is shorter than the key prefix chosen for the query. [#116364](https://github.com/ClickHouse/ClickHouse/pull/116364) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR: GroupingAggregatedTransform pushed bucket N twice` when `make_distributed_plan` picks the `Shuffle` aggregation strategy while the aggregation must produce results in bucket order (for example with `distributed_aggregation_memory_efficient`). It is now planned as a partial aggregation plus a merge. [#116446](https://github.com/ClickHouse/ClickHouse/pull/116446) ([Groene AI](https://github.com/groeneai)). * Fix wrong row order when sorting a `LowCardinality` column of a floating-point type whose dictionary holds two entries that compare equal, such as `-0.0` and `+0.0`. A multi-column `ORDER BY` did not apply the remaining keys to those rows, and the `MergeTree` writer's stable sort did not keep them in insertion order, changing which row `ReplacingMergeTree` keeps. [#116870](https://github.com/ClickHouse/ClickHouse/pull/116870) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from the default-on `optimize_time_filter_with_preimage` for `toYear`/`toYYYYMM` predicates at the column type's maximum (`Date` at 2149, `DateTime` at 2106), a `DECIMAL_OVERFLOW` exception for the same predicates on a `DateTime64(9)` column at 2262, and wrong results for the same predicates on `DateTime`/`DateTime64` columns in time zones whose year or month boundary is not local midnight (`America/Lima`, `America/Managua`). [#117451](https://github.com/ClickHouse/ClickHouse/pull/117451) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_IMPLEMENTED` ("Cannot decorrelate query, because 'DelayedCreatingSets' step is not supported") for a correlated subquery whose `WHERE` builds a set, for example an `IN (subquery)`, or a `dictGet(...) >= const` comparison that `optimize_inverse_dictionary_lookup` rewrites into one. [#118412](https://github.com/ClickHouse/ClickHouse/pull/118412) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a race between `SYSTEM RESTORE DATABASE REPLICA` and the DDL worker of a `Replicated` database. The restore re-initialised the replica while the previous DDL worker was still running its own startup recovery, so the recovery could mark the wrong range of replicated-DDL log entries as finished, or move local tables to `_broken_replicated_tables`. The worker is now stopped before the restore touches Keeper. [#118481](https://github.com/ClickHouse/ClickHouse/pull/118481) ([Groene AI](https://github.com/groeneai)). * Fixed `WITH RECURSIVE` losing its recursiveness in the second and later branches of a `UNION`, `INTERSECT` or `EXCEPT`. When a later branch had no `WITH` clause of its own, the first branch's `WITH` list was copied into it without the `RECURSIVE` marker, so `WITH RECURSIVE src AS (SELECT 1 AS id UNION ALL SELECT id + 1 FROM src WHERE id < 3) SELECT sum(id) FROM src UNION ALL SELECT sum(id) FROM src` was rejected with `UNKNOWN_TABLE`, and returned a wrong value for the copied branch when an enclosing CTE of the same name existed. `CREATE VIEW` also stored the copied branch without `RECURSIVE`. [#118510](https://github.com/ClickHouse/ClickHouse/pull/118510) ([Groene AI](https://github.com/groeneai)). * Fixed a `GLOBAL RIGHT JOIN` over `Distributed` tables returning unmatched rows of the right table once per shard: the original left table is now broadcast and the preserved right side stays sharded. Closes [#95728](https://github.com/ClickHouse/ClickHouse/issues/95728). [#118611](https://github.com/ClickHouse/ClickHouse/pull/118611) ([Anton Ivashkin](https://github.com/ianton-ru)). * Fix an endless `ORDER BY ... WITH FILL` when the `STEP` is too small to change the value it is added to. [#118633](https://github.com/ClickHouse/ClickHouse/pull/118633) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `Invalid local IN function name globalIn` error for a query with `IN` over a `UNION` of `Distributed` tables when `distributed_product_mode = 'global'` or `prefer_global_in_and_join` is enabled. [#118639](https://github.com/ClickHouse/ClickHouse/pull/118639) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `toLastDayOfMonth` and `toLastDayOfWeek` of a `Date` argument now saturate at the `Date` maximum (`2149-06-06`) instead of wrapping around to 1970. The wrapped result also silently dropped matching rows through primary-key pruning. [#118642](https://github.com/ClickHouse/ClickHouse/pull/118642) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash and a `LOGICAL_ERROR` in the `mergeTreeIndex` table function when reading a key column whose `ALTER TABLE ... MODIFY COLUMN` between `T` and `LowCardinality(T)` has an unmaterialized mutation. The key column is now emitted in the type the current metadata declares. [#118709](https://github.com/ClickHouse/ClickHouse/pull/118709) ([Groene AI](https://github.com/groeneai)). * Fix `SYNTAX_ERROR` when loading views, materialized views, or projections whose `SELECT` list starts with a column named `TOP`. [#118754](https://github.com/ClickHouse/ClickHouse/pull/118754) ([Kirill](https://github.com/kirillgarbar)). * Fixed `optimize_or_like_chain` returning extra rows for `OR` chains of `LIKE`/`ILIKE` patterns that do not end in `%`: a value followed by a trailing newline wrongly matched the filter. [#118766](https://github.com/ClickHouse/ClickHouse/pull/118766) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixed a `LOGICAL_ERROR` ("Expected CommonSubplanReferenceStep to reference CommonSubplanStep") raised at query planning time when `query_plan_convert_join_to_in = 1` was used with a correlated aggregating scalar subquery. The optimization now keeps the `JOIN` instead of rewriting it when either join input carries a common subplan node. [#118797](https://github.com/ClickHouse/ClickHouse/pull/118797) ([Groene AI](https://github.com/groeneai)). * Fix a wrong result for `SELECT * APPLY (x -> f(x))` over `JOIN ... USING (a, b)`: every `USING` column was named after the lambda parameter (`f(x)`) instead of the matched column, so two output columns shared one name, and wrapping the query in `SELECT * FROM (...)` returned one `USING` column's values under the other column's position; a type-returning lambda such as `toTypeName` failed with `AMBIGUOUS_COLUMN_NAME` instead. The same happened for `COLUMNS(a, b) APPLY (x -> f(x))`, and for two or more `ARRAY JOIN` columns. The lambda form of `APPLY` now names these columns like `APPLY f` and a plain `*` already did. [#118825](https://github.com/ClickHouse/ClickHouse/pull/118825) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash that could happen when a `WORKLOAD` setting limiting concurrency (e.g. `max_concurrent_queries` or `max_bytes_inflight`) was lowered below the number of currently in-flight resource requests while the workload was active. [#118836](https://github.com/ClickHouse/ClickHouse/pull/118836) ([Sergei Trifonov](https://github.com/serxa)). * Fixed a logical error (`Method == not implemented for AST`) raised by a `SET` of a custom setting that already held an AST value such as `disk(type = 's3', ...)`. Assigning such a setting a second time in one session, or clamping a nested `SETTINGS` clause against it, reached a comparison that was never implemented. [#118993](https://github.com/ClickHouse/ClickHouse/pull/118993) ([Groene AI](https://github.com/groeneai)). * Fixed a broken part in `detached/` never being renamed with the `broken_` prefix by the `ATTACH_PART` executor, which left it occupying a valid part name and failing every later attach attempt. [#118999](https://github.com/ClickHouse/ClickHouse/pull/118999) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `TOO_DEEP_SUBQUERIES` being thrown for `GLOBAL IN` and `GLOBAL JOIN` subqueries in two analyzer cases: when `max_subquery_depth` is set, the depth is now counted in nested subqueries instead of query tree nodes; and when a `UNION` is on the path to the `GLOBAL` subquery, `UNION` nodes no longer count toward that depth. Closes [#79345](https://github.com/ClickHouse/ClickHouse/issues/79345). Closes [#119623](https://github.com/ClickHouse/ClickHouse/issues/119623). [#119033](https://github.com/ClickHouse/ClickHouse/pull/119033) ([Vladimir Cherkasov](https://github.com/vdimir)). [#119650](https://github.com/ClickHouse/ClickHouse/pull/119650) ([Groene AI](https://github.com/groeneai)). * With `prefer_column_name_to_alias = 1`, an unqualified column name that is ambiguous between joined tables now resolves to the alias with the same name, as the old analyzer did, instead of throwing `AMBIGUOUS_IDENTIFIER`. [#119192](https://github.com/ClickHouse/ClickHouse/pull/119192) ([Vladimir Cherkasov](https://github.com/vdimir)). [#119906](https://github.com/ClickHouse/ClickHouse/pull/119906) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix a wrong-order merge (logical error `Virtual row does not cover sort column` in debug builds) when reading in order with virtual rows and `ORDER BY` contains constants, e.g. `SELECT DISTINCT a, b FROM t ORDER BY 'x', a, 'd' COLLATE 'cs', b` on a table with key `(a, b)`. [#119195](https://github.com/ClickHouse/ClickHouse/pull/119195) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed wrong results when a table's sorting key expression resolves to a different result type in the query than in the table, for example a key of `CAST(json.b, 'String')` read with `cast_keep_nullable = 1`. Read-in-order treated the two as interchangeable, so rows could come back out of order, `optimize_aggregation_in_order` could split groups and report wrong aggregates, and `read_in_order_use_virtual_row = 1` could fail with `LOGICAL_ERROR: Virtual row has different type`. [#119385](https://github.com/ClickHouse/ClickHouse/pull/119385) ([Groene AI](https://github.com/groeneai)). * Fixed `BACKUP` of a `Memory` table reading an unrelated file named `tmp_sizes_json` from the server's working directory: the `sizes.json` shipped inside the backup is now built in memory, so such a file can no longer make the backup fail or add foreign entries to it. [#119330](https://github.com/ClickHouse/ClickHouse/pull/119330) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a `MergeTree` skip index uses `mortonEncode` or `hilbertEncode` directly and a query filters on one of the curve arguments. [#119355](https://github.com/ClickHouse/ClickHouse/pull/119355) ([Minh Vu](https://github.com/fallintoplace)). * Fixed an `ALTER` on a view or materialized view with `SQL SECURITY DEFINER` deleting the auto-created `:definer` account when the `ALTER` did not change the definer. Every later `SELECT` from the view, and every `INSERT` into a materialized view's source table, then failed permanently with `There is no user ':definer' in 'user directories'` (`UNKNOWN_USER`). [#119357](https://github.com/ClickHouse/ClickHouse/pull/119357) ([Groene AI](https://github.com/groeneai)). * Fixed a materialized view whose `SELECT` contains a `UNION`, `INTERSECT` or `EXCEPT` in a subquery or CTE breaking after `DETACH TABLE` + `ATTACH TABLE`, or after `RESTORE`: every subsequent `INSERT` into its source table failed with `UNION mode UNION_DEFAULT must be normalized`, and under `enable_analyzer = 0` it was silently evaluated as a plain union, returning wrong results. `ATTACH TABLE` (short form) and `RESTORE` now re-run set-operation normalization on the stored definition. [#119365](https://github.com/ClickHouse/ClickHouse/pull/119365) ([Groene AI](https://github.com/groeneai)). * Fix an `ALIAS` column that references another column (`b UInt64 ALIAS a`) reading back as the default value of its type when the table is read through a `Merge` table over a `Distributed` table with a remote shard. [#119561](https://github.com/ClickHouse/ClickHouse/pull/119561) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `arrayShiftLeft` and `arrayShiftRight` when an explicit `NULL` default is used with a nullable array. [#119626](https://github.com/ClickHouse/ClickHouse/pull/119626) ([Minh Vu](https://github.com/fallintoplace)). * Fix `ALTER TABLE ... MODIFY SETTING name = DEFAULT` being ignored. Closes [#49613](https://github.com/ClickHouse/ClickHouse/issues/49613). [#118526](https://github.com/ClickHouse/ClickHouse/pull/118526) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix `arrayNormalizedGini` rejecting valid rows with different array sizes in the same block. [#118631](https://github.com/ClickHouse/ClickHouse/pull/118631) ([Minh Vu](https://github.com/fallintoplace)). * Fix `substringIndex` and `substringIndexUTF8` rejecting `UInt64` counts above `Int64::max()` instead of interpreting them as negative counts. [#118645](https://github.com/ClickHouse/ClickHouse/pull/118645) ([Minh Vu](https://github.com/fallintoplace)). * Fix a logical error for an asynchronous `INSERT` combined with a non-parallel quorum (`insert_quorum` greater than one or `'auto'` together with `insert_quorum_parallel = 0`) when the data is sent as blocks over the native protocol. Such a query is now rejected with `UNSUPPORTED_PARAMETER`, as it already was when the data was inlined in the query text. [#118968](https://github.com/ClickHouse/ClickHouse/pull/118968) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a graceful shutdown silently discarding the rows of a `Buffer` table whose destination lives in another database or is another `Buffer` table. [#118985](https://github.com/ClickHouse/ClickHouse/pull/118985) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#116898](https://github.com/ClickHouse/ClickHouse/pull/116898) ([Jayme Bird](https://github.com/jaymebrd)). * Fix a use-after-free in the gRPC server when a query with a streaming input finishes while a read of the next message is still in flight. [#115707](https://github.com/ClickHouse/ClickHouse/pull/115707) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Custom object metadata written to Google Cloud Storage over the S3 API without HMAC credentials, which is how ClickHouse Cloud on GCP accesses object storage, now takes effect: the headers that GCS spells with the `x-goog-` prefix are translated for every request, not only for server-side copies. Previously GCS discarded the `x-amz-` spelling silently, so the write succeeded and the metadata was not applied. [#119042](https://github.com/ClickHouse/ClickHouse/pull/119042) ([Sema Checherinda](https://github.com/CheSema)). * Fixed a crash when reading a subcolumn of a `Variant`, `Dynamic`, or `JSON` column whose own type is `Nullable` or `LowCardinality(Nullable)` from a `MergeTree` table, for example ``SELECT value.`Tuple(a Nullable(UInt32), b String)`.a`` over a `Dynamic` column. Previously such a read hit a `LOGICAL_ERROR` in debug builds and crashed in release builds. [#112338](https://github.com/ClickHouse/ClickHouse/pull/112338) ([Groene AI](https://github.com/groeneai)). * With `shutdown_wait_backups_and_restores = 0`, a `BACKUP` or `RESTORE` whose I/O did not return could keep the server from terminating on a single `SIGTERM`. That wait is now bounded by `shutdown_wait_unfinished`, and new backups and restores are refused once shutdown has begun. [#115377](https://github.com/ClickHouse/ClickHouse/pull/115377) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when query plan optimizations treat non-deterministic expressions as deterministic, including lambdas and WebAssembly UDFs, which could silently drop rows. [#117248](https://github.com/ClickHouse/ClickHouse/pull/117248) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#117249](https://github.com/ClickHouse/ClickHouse/pull/117249) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Column identifier ... is already registered` (`LOGICAL_ERROR`) for a correlated subquery over a `VIEW` on a `Distributed` table. Such a query is now refused with `Correlated subqueries are not supported with remote tables`, the same as a correlated subquery over the `Distributed` table itself. The same applies to parameterized views, `Merge` tables over such views, and materialized views whose target is such a view. [#117335](https://github.com/ClickHouse/ClickHouse/pull/117335) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `transform` and the simple `CASE expr WHEN` form returning the default value instead of the matching one when the expression is a `Nullable(Enum)` column. Only the member whose numeric value was `0` used to match. Closes [#117206](https://github.com/ClickHouse/ClickHouse/issues/117206). [#117520](https://github.com/ClickHouse/ClickHouse/pull/117520) ([Groene AI](https://github.com/groeneai)). * Fix `SYSTEM RELOAD DICTIONARY ON CLUSTER` and `SYSTEM UNLOAD DICTIONARY ON CLUSTER` so normal distributed DDL clusters resolve the bare dictionary name against the initiator's current database before queuing the task, matching existing table-like `ON CLUSTER` DDL behavior. Closes [#114322](https://github.com/ClickHouse/ClickHouse/issues/114322). [#117596](https://github.com/ClickHouse/ClickHouse/pull/117596) ([Alexander Yoon](https://github.com/XanderYoon)). * Fixed silently missing rows for predicates on `toUnixTimestamp` of a `Date` or `Date32` primary key or column with statistics: the conversion wraps around `UInt32` for dates after 2106-02-07, but index analysis assumed it was monotonic everywhere. [#117709](https://github.com/ClickHouse/ClickHouse/pull/117709) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a query getting stuck when the server fails to start a thread for a parallel task, for example when a thread pool is exhausted. [#117781](https://github.com/ClickHouse/ClickHouse/pull/117781) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results for a distributed query whose folded constant contains a `DateTime` value. The constant is now sent to secondary servers as its raw Unix timestamp wherever the receiving side re-applies the literal's declared type, so DST fall-back overlaps no longer reinterpret it as another instant. [#117837](https://github.com/ClickHouse/ClickHouse/pull/117837) ([Groene AI](https://github.com/groeneai)). * Fixed queries failing with `AUTHENTICATION_FAILED` after `RENAME DATABASE` on a `Replicated` database that uses `cluster_secret`. The cached cluster object kept the old database name, which participates in the interserver handshake. [#117874](https://github.com/ClickHouse/ClickHouse/pull/117874) ([Kirill Shokhin](https://github.com/kirillshokhin)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when `transform_null_in = 1` and a distributed or parallel-replicas query aggregates over a predicate that an analyzer pass rewrote into `IN`, such as `SELECT sumIf(id, has([1, 2, 3], id)) FROM t`. [#117897](https://github.com/ClickHouse/ClickHouse/pull/117897) ([Groene AI](https://github.com/groeneai)). * Fixed `THERE_IS_NO_COLUMN` (`Cannot find column ... in source stream`) when a query shipped as a serialized query plan (`serialize_query_plan = 1`) reads a `Merge` table that spans a local table and a `Distributed` table. [#117969](https://github.com/ClickHouse/ClickHouse/pull/117969) ([Groene AI](https://github.com/groeneai)). * Fixed a `SELECT` returning fewer rows than it should when a `JSON` subcolumn is compared with an `Enum` constant whose label is the empty string, and the table has a `JSONAllPaths` skip index (`bloom_filter`, `tokenbf_v1`, `ngrambf_v1`, or `sparse_grams`). [#117975](https://github.com/ClickHouse/ClickHouse/pull/117975) ([Groene AI](https://github.com/groeneai)). * An `Enum` constant wrapped in `Variant` or `Dynamic` and compared against a `String` or `FixedString` column no longer uses the enum's underlying number instead of its name, which previously made `IN`, `NOT IN`, and key analysis use the wrong value. [#117998](https://github.com/ClickHouse/ClickHouse/pull/117998) ([Groene AI](https://github.com/groeneai)). * Fixed a `Replicated` database being unable to recover a lost replica if it held a `Remote` or `RemoteSecure` table whose target is a table function such as `merge(...)` that no longer resolves. [#118221](https://github.com/ClickHouse/ClickHouse/pull/118221) ([Groene AI](https://github.com/groeneai)). * Fixed `Not found column ... in block` for a query that combines a constant `ORDER BY` key, `LIMIT BY`, and `LIMIT ... WITH TIES`. [#118222](https://github.com/ClickHouse/ClickHouse/pull/118222) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed two overflow cases where `memory_usage_overcommit_max_wait_microseconds` lost the wait it was asked for and timed out immediately instead of waiting for memory to be freed. [#118227](https://github.com/ClickHouse/ClickHouse/pull/118227) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE VIEW` being refused with `SUSPICIOUS_TYPE_FOR_LOW_CARDINALITY` or `ILLEGAL_COLUMN` when the view's inferred column type is gated by settings such as `allow_suspicious_low_cardinality_types` or `enable_time_time64_type`. Views store no data, so those storage-type checks no longer block them. [#118298](https://github.com/ClickHouse/ClickHouse/pull/118298) ([Groene AI](https://github.com/groeneai)). * Fixed `Not-ready Set is passed as the second argument for function 'globalIn'` when a `PREWHERE` or row-level filter expression containing an `IN` subquery is evaluated by a query planned to the `FetchColumns` stage. [#118359](https://github.com/ClickHouse/ClickHouse/pull/118359) ([Groene AI](https://github.com/groeneai)). * `SELECT count() FROM ` now works for the `Set` engine, using the table's stored row count instead of failing with `Method read is not supported by storage Set`. Reading regular columns from a `Set` table is still unsupported. [#118491](https://github.com/ClickHouse/ClickHouse/pull/118491) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `sequenceMatchEvents` returning an event chain that the pattern does not authorize, for example one that skips a `(?t...)` time gate between two of the returned events. The returned chain now always matches a prefix of the pattern. Closes [#118120](https://github.com/ClickHouse/ClickHouse/issues/118120). [#118497](https://github.com/ClickHouse/ClickHouse/pull/118497) ([Groene AI](https://github.com/groeneai)). * Fixed `toSecond`, `toStartOfMinute`, `toStartOfHour`, `toStartOfInterval`, and the other `toStartOf*` functions returning wrong values for pre-1970 `DateTime64` values in time zones whose historical offset had a sub-minute component, such as `Europe/Amsterdam` before 1937 or `Asia/Kolkata` before 1906. [#118501](https://github.com/ClickHouse/ClickHouse/pull/118501) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix array aggregation functions returning non-default values for empty arrays when the lambda result is constant. [#118636](https://github.com/ClickHouse/ClickHouse/pull/118636) ([Minh Vu](https://github.com/fallintoplace)). * Conversions of 128- and 256-bit integers to `DateTime`, `DateTime64`, and `Time64` now saturate like the 64-bit conversions instead of wrapping or storing an out-of-range value. This also fixes silent row loss through primary-key pruning. [#118717](https://github.com/ClickHouse/ClickHouse/pull/118717) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a possible heap buffer overflow on malformed blob-storage responses. [#119054](https://github.com/ClickHouse/ClickHouse/pull/119054) ([Arsen Muk](https://github.com/arsenmuk)). * Preserve input sorting for `groupArrayInsertAt`, `stochasticLinearRegression`, and `stochasticLogisticRegression`, so an `ORDER BY` feeding them is not optimized away. [#119142](https://github.com/ClickHouse/ClickHouse/pull/119142) ([Ivan Babrou](https://github.com/bobrik)). * Fixed a crash and a `LOGICAL_ERROR` for statements that write `trim(BOTH '' FROM x)` where a function is expected, such as in a column's `STATISTICS(...)` or `CODEC(...)` list, `INDEX ... TYPE`, `ALTER TABLE ... ADD STATISTICS`, a table function, a named collection, or `SETTINGS`. [#119317](https://github.com/ClickHouse/ClickHouse/pull/119317) ([Groene AI](https://github.com/groeneai)). * Fix `CHECK TABLE` permanently reporting `Found unexpected projection directories` after a mutation when the part carries a projection that the table has since dropped. [#119348](https://github.com/ClickHouse/ClickHouse/pull/119348) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a wedged mutation queue after `ATTACH PARTITION` of a partition that was detached before a `DROP COLUMN`: mutations of a Compact part carrying the dropped column no longer fail forever with `UNKNOWN_IDENTIFIER`. [#119354](https://github.com/ClickHouse/ClickHouse/pull/119354) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a bug where unfiltered `TRUNCATE ALL TABLES FROM` queries permanently disabled merges on retained `MergeTree` tables, causing later `OPTIMIZE TABLE ... FINAL` queries to fail with `ABORTED`. [#119425](https://github.com/ClickHouse/ClickHouse/pull/119425) ([Paul Annesley](https://github.com/pda)). * Fix `Unexpected value 0 in enum` thrown by string functions such as `LIKE`, `match`, and `position` when applied to a `Nullable(Enum)` value that is `NULL`, including `NULL`s produced by a `LEFT JOIN` with `join_use_nulls = 1`. [#119513](https://github.com/ClickHouse/ClickHouse/pull/119513) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `arrayJaccardIndex` when its input arrays contain duplicate elements. [#119560](https://github.com/ClickHouse/ClickHouse/pull/119560) ([Minh Vu](https://github.com/fallintoplace)). * `modulo`, `positiveModulo`, `moduloOrZero`, and `moduloOrNull` no longer return wrong results when one operand is a signed integer and the other is an unsigned integer at least as wide. [#119616](https://github.com/ClickHouse/ClickHouse/pull/119616) ([Serhiy Bzhezytskyy](https://github.com/serhiy-bzhezytskyy)). * A `Filesystem` database no longer keeps pointing at a removed directory, which previously made the server refuse to start on the next restart. [#119618](https://github.com/ClickHouse/ClickHouse/pull/119618) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a standalone `UPDATE ... SET` reading the wrong table when a mutation expression names a table that a same-named `WITH` element hides but the subquery does not see because its own `SETTINGS` clause disables `enable_global_with_statement`. [#119630](https://github.com/ClickHouse/ClickHouse/pull/119630) ([Groene AI](https://github.com/groeneai)). * Fixed a stack buffer overflow when reading a `BIT` column from MySQL through the `mysql` table function, the `MySQL` table and database engines, or MySQL dictionaries. [#119680](https://github.com/ClickHouse/ClickHouse/pull/119680) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `sparseGrams` ignoring `max_ngram_length` when `min_cutoff_length` is specified. [#119713](https://github.com/ClickHouse/ClickHouse/pull/119713) ([Minh Vu](https://github.com/fallintoplace)). * Fix `BACKUP` and `RESTORE` of a `Log` or `TinyLog` table when one column occupies zero bytes on disk while the table has rows, such as a column of `countResample` states over an empty range. Previously the restored table could come back empty or fail with `FILE_DOESNT_EXIST`. [#119722](https://github.com/ClickHouse/ClickHouse/pull/119722) ([Groene AI](https://github.com/groeneai)). * Fix a race between `ALTER TABLE` and `RENAME TABLE` / `EXCHANGE TABLES`. [#103044](https://github.com/ClickHouse/ClickHouse/pull/103044) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix `BACKUP` of `EmbeddedRocksDB` tables. Previously only the table metadata was stored in the backup and the actual data was silently dropped, so `RESTORE` produced an empty table. The backup now includes all key-value pairs and restores them correctly. [#109327](https://github.com/ClickHouse/ClickHouse/pull/109327) ([Groene AI](https://github.com/groeneai)). * Fix `toStartOfWeek` and `toLastDayOfWeek` with a Sunday-first week mode returning a wrapped day number for a `Date32` value outside the representable calendar, for example after `toDate32('1970-01-01') + INTERVAL 2147483647 DAY`. The week boundary is now computed at the calendar boundary, consistently with the Monday-first week modes. [#111923](https://github.com/ClickHouse/ClickHouse/pull/111923) ([Groene AI](https://github.com/groeneai)). * Fixed `mergeTreeCodecBlockCounts` disclosing whether a source table exists, and even its engine type, to a user without any privilege on that table. Resolving the table function, for example with `DESCRIBE`, now checks access before consulting the catalog. [#116647](https://github.com/ClickHouse/ClickHouse/pull/116647) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Messages of the PostgreSQL wire protocol are now parsed within their declared size, so a malformed message can no longer make the server allocate memory for a payload that never arrives. [#116998](https://github.com/ClickHouse/ClickHouse/pull/116998) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `CREATE TABLE dst AS src` copying the schema of an `ENGINE = Alias` table's target for a user who holds `SHOW COLUMNS` on the alias but no privilege on its target. The statement now requires table-level `SHOW COLUMNS` on the target, matching `DESCRIBE` of that target. [#117702](https://github.com/ClickHouse/ClickHouse/pull/117702) ([Groene AI](https://github.com/groeneai)). * Fixed the query condition cache serving a "no matching rows" verdict to a query that differs from the cached one only in a setting that changes how a function evaluates, such as the `formatdatetime_*` / `parsedatetime_*` settings or `function_locate_has_mysql_compatible_argument_order`, which silently dropped all matching rows. [#117803](https://github.com/ClickHouse/ClickHouse/pull/117803) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `CAST AS Tuple can only be performed between tuple types or from String` error, and silently wrong results for a `Tuple(String, ...)` key, when a `MergeTree` sorting, partition or skip-index key wraps a single `Tuple`-typed column in a deterministic function (for example `ORDER BY toString(k)`) and the query filters that column with `k IN (tuple(...))`, `k NOT IN (tuple(...))` or a multi-column subquery set. [#118899](https://github.com/ClickHouse/ClickHouse/pull/118899) ([Groene AI](https://github.com/groeneai)). * Fix queries that use a higher-order function together with `ARRAY JOIN`: a lambda that captures nothing is no longer moved across the `ARRAY JOIN` step away from the function that applies it. [#119186](https://github.com/ClickHouse/ClickHouse/pull/119186) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `TYPE_MISMATCH` in the selectivity estimator when a predicate on a `DateTime64` column uses an unconvertible string literal. Closes [#119126](https://github.com/ClickHouse/ClickHouse/issues/119126). [#119265](https://github.com/ClickHouse/ClickHouse/pull/119265) ([Han Fei](https://github.com/hanfei1991)). * Fix unstable ordering of equal sorting keys during inserts, which could cause `CollapsingMergeTree` to retain stale rows when cancellation and replacement rows were inserted together with `optimize_on_insert` enabled, and `ReplacingMergeTree` to retain the wrong row when versions were equal. [#119474](https://github.com/ClickHouse/ClickHouse/pull/119474) ([Anton Kovalenko](https://github.com/antonkovalenko)). * Fixed cleartext credentials in `system.asynchronous_insert_log.query`. Secrets of table functions such as `s3`, `url` or `mysql` are now masked as `[HIDDEN]`, the same way `system.query_log` masks them. [#119487](https://github.com/ClickHouse/ClickHouse/pull/119487) ([Sema Checherinda](https://github.com/CheSema)). * Fix a crash when reading a table with active lightweight updates whose sorting key contains a column that is missing in the data part and has a `DEFAULT` expression. [#119540](https://github.com/ClickHouse/ClickHouse/pull/119540) ([Groene AI](https://github.com/groeneai)). * Fixed Keeper replaying its local changelog tail one entry at a time after a restart when the changelog is stored on object storage. The commit read-ahead used to discard its own prefetched entries whenever it missed the serve deadline, so on storage slower than that deadline it could never catch up and every entry cost two object reads. The default of `log_readahead_commit_window_bytes` is now 16 MiB instead of 500 MiB, since it bounds a prefetch window rather than a cache. [#119627](https://github.com/ClickHouse/ClickHouse/pull/119627) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix unbounded memory growth from time zone names of the form `libc:`. These are internal interfaces of the `cctz` library that ClickHouse never intended to expose: any suffix was accepted, each distinct name permanently cost ~4.6 MiB that the server never gave back, and the time zone silently behaved as UTC instead of the name being reported as unknown. They are rejected now. [#119634](https://github.com/ClickHouse/ClickHouse/pull/119634) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect pruning for space-filling-curve indexes with `UInt64` arguments. [#119663](https://github.com/ClickHouse/ClickHouse/pull/119663) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `sum(x + 1) OVER (...)` returning a single row holding the overall total instead of the windowed result. `optimize_arithmetic_operations_in_aggregate_functions`, which is enabled by default, rewrote the windowed `sum` into `sum(x) + 1 * count(x)`, an expression over plain aggregates that cannot carry the `OVER` clause, so the window definition was discarded. Both `enable_analyzer = 1` (the default) and `enable_analyzer = 0` were affected. [#119699](https://github.com/ClickHouse/ClickHouse/pull/119699) ([Groene AI](https://github.com/groeneai)). * Fix `substringUTF8` returning too many code points when a negative offset starts before the beginning of a string. [#119840](https://github.com/ClickHouse/ClickHouse/pull/119840) ([Minh Vu](https://github.com/fallintoplace)). * Fix `URLHierarchy`, `URLPathHierarchy`, and `URLHash` failing to recognize valid URL schemes containing `a`, `z`, `0`, or `9`. [#119844](https://github.com/ClickHouse/ClickHouse/pull/119844) ([Minh Vu](https://github.com/fallintoplace)). * Fix `mapSubtract` subtracting duplicate keys in its first argument. [#119845](https://github.com/ClickHouse/ClickHouse/pull/119845) ([Minh Vu](https://github.com/fallintoplace)). * Fixed the server not validating the schema of the second and subsequent data blocks that a client sends for an external table over the `Native` protocol. Such a block could make the data be interpreted as a type other than the one it was sent as. [#119849](https://github.com/ClickHouse/ClickHouse/pull/119849) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed AST formatting of the `APPLY` column transformer: a column-name prefix containing a single quote or a backslash, or a function name that needs back-quoting, was emitted without escaping, so `formatQuery` returned a query that either could not be parsed or parsed back to a different query. [#119851](https://github.com/ClickHouse/ClickHouse/pull/119851) ([Groene AI](https://github.com/groeneai)). * Fixed a crash of the server caused by a table with `ENGINE = NATS` whose credentials the broker started to reject, for example after a password rotation or on an expired token. Such a table now recreates its connection and resumes consuming once the credentials are accepted again, instead of staying silently idle. [#119853](https://github.com/ClickHouse/ClickHouse/pull/119853) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `timeSeriesTagsToMap` treating `FixedString` tag names and values differently between the array form and the separate-argument form. `FixedString` padding is now trimmed consistently, so the same logical tags produce the same map and lookups such as `mapContains(..., 'a')` work in both forms. [#119859](https://github.com/ClickHouse/ClickHouse/pull/119859) ([Minh Vu](https://github.com/fallintoplace)). * Fixed wrong results when `join_algorithm = 'direct'` reads the right-hand table of a join from a `MergeTree` table and plan-based parallel replicas (`parallel_replicas_plan_based = 1`) are enabled. Such a join could silently return only part of the rows that should have matched, with no error reported. The lookup read that the join performs is no longer distributed across replicas. [#119861](https://github.com/ClickHouse/ClickHouse/pull/119861) ([Groene AI](https://github.com/groeneai)). * Fixed data loss where a `MergeTree` part whose `columns.txt` was left empty (zero bytes) by an interrupted metadata rewrite was detached as broken on load, discarding all of its rows. For a wide part the column list is now rebuilt from the part's own `columns_substreams.txt` and the table metadata, as it already was for an absent `columns.txt`; when the rebuilt list cannot be validated, the part is left untouched for recovery instead. [#111597](https://github.com/ClickHouse/ClickHouse/pull/111597) ([Groene AI](https://github.com/groeneai)). * Integers in the `TSV`, `TSVRaw`, `TSKV` and `CustomSeparated` formats can now be zero-padded (`007`) or carry a redundant leading `+` (`+7`), spellings that previously failed to parse but that `CSV` already accepted. The same reader serves other conversions from text to an integer, so `WHERE i = '007'` on a numeric column, query parameters and `CAST('[007]' AS Array(Int64))` accept them too. A field that is a lone `+` remains an error, now reported as `CANNOT_PARSE_NUMBER` on every one of these routes instead of a route-specific code. Schema inference is unchanged: `007` still infers as `String`. [#113910](https://github.com/ClickHouse/ClickHouse/pull/113910) ([Groene AI](https://github.com/groeneai)). * Fixed permanent data loss in `ALTER TABLE ... EXECUTE remove_orphan_files` on `Iceberg` tables with `iceberg_use_version_hint = 1`. When `metadata/version-hint.text` named an older version than the newest committed metadata file, every object of the newer committed snapshot, including its data files, was deleted as an orphan. Reachability is now computed from the authoritative current metadata, and a metadata listing this command cannot order unambiguously is refused instead of resolved by listing order. [#114324](https://github.com/ClickHouse/ClickHouse/pull/114324) ([Groene AI](https://github.com/groeneai)). * Fix buffer overflows and a server abort on malformed responses from external servers: an Azure Blob Storage endpoint that returns more data than was requested or omits the `ETag` header, and a `MySQL` server that returns an oversized `BIT` value. `Azure` reads, copies and deletes, and `S3` copies made by backups and by `S3Queue` post-processing, are now pinned to the object generation they selected, so an object overwritten in place is reported instead of silently mixing two generations. Also fix the decoding of a `MySQL` `BIT` value shorter than 8 bytes on big-endian hosts. [#115706](https://github.com/ClickHouse/ClickHouse/pull/115706) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash inside the crash handler itself when a fatal signal arrives during startup of `clickhouse-server`, `clickhouse-keeper` or `clickhouse-local`. The handler dereferenced the global context, which does not exist yet at that point, so right after logging the stack trace it took a second fault, hung for about five minutes, and never sent the configured crash report. [#116245](https://github.com/ClickHouse/ClickHouse/pull/116245) ([Groene AI](https://github.com/groeneai)). * Fixed `CLEAR COLUMN` silently leaving a `MATERIALIZED` column derived from the cleared column at its old value. This happened whenever the mutation had already built a stage before the recompute was appended: another pending mutation coalescing with the `CLEAR` into one pass, which needs no special settings, or an enabled `enable_block_number_column` / `enable_block_offset_column`. The wrong value was permanent; later merges did not repair it. [#117296](https://github.com/ClickHouse/ClickHouse/pull/117296) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `ILIKE` on a `text` index returning rows the predicate does not match, and skipping rows it does match, when the pattern or the data contains a non-ASCII code point that case-folds onto an ASCII letter. Affected indexes with a `lowerUTF8`/`upperUTF8` preprocessor and patterns whose needle contains `k`. [#117354](https://github.com/ClickHouse/ClickHouse/pull/117354) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixed wrong results in five cases where a `WHERE` predicate on a join key was pushed to the other side of a `JOIN` whose two keys have different types. Such a key is no longer substituted when the keys' common type is a float, because `-0.0` and `+0.0` are join-equal but differ bit for bit; nor when it is `Dynamic` or `JSON`, whose runtime contents the static type does not describe; nor when the key is unstable within the query or changes the row count; nor in a conjunct that reads a column's representation rather than its value, as `isConstant` and `toColumnTypeName` do, including when such a read or an unstable function is hidden inside a lambda body. Separately, with `enable_analyzer = 1`, a predicate on the wider of two differently typed join keys is now also used as an index condition on the other table of a plain `INNER JOIN`, so it is read through its primary key instead of in full; previously only `ANY INNER JOIN`, `LEFT JOIN` and `RIGHT JOIN` got this. [#117973](https://github.com/ClickHouse/ClickHouse/pull/117973) ([Groene AI](https://github.com/groeneai)). Also fixed a `LOGICAL_ERROR` (`Unexpected return type from _CAST`) when such a predicate was pushed down to a side that `join_use_nulls` widens to `Nullable`. [#120109](https://github.com/ClickHouse/ClickHouse/pull/120109) ([Groene AI](https://github.com/groeneai)). * Fix rows silently dropped by index analysis for `has(array_of_named_tuples, key)` when the tuple fields are declared in another order and the key is a function of the column. [#118706](https://github.com/ClickHouse/ClickHouse/pull/118706) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed ClickHouse Keeper returning `ZNOAUTH` for every request on a node whose ACL list is empty but is stored under a nonzero internal ACL id, which happens for Keeper data converted from a ZooKeeper snapshot or written by a very old Keeper. Such nodes are unrestricted again, as they were before 26.6. Closes [#117603](https://github.com/ClickHouse/ClickHouse/issues/117603). [#118965](https://github.com/ClickHouse/ClickHouse/pull/118965) ([Groene AI](https://github.com/groeneai)). * Fixed `YYYYMMDDhhmmssToDateTime64` returning incorrect fractional seconds for timestamps before 1970-01-01. [#119240](https://github.com/ClickHouse/ClickHouse/pull/119240) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a crash when a disk is configured with `metadata_type = web` or `metadata_type = web_index` on top of a non-web object storage. Such a configuration is now rejected with `INVALID_CONFIG_PARAMETER` instead of interpreting an unrelated object storage as a web one. Closes [#119798](https://github.com/ClickHouse/ClickHouse/issues/119798). [#119818](https://github.com/ClickHouse/ClickHouse/pull/119818) ([Groene AI](https://github.com/groeneai)). * `Iceberg` tables recovered from stored metadata no longer enable background compaction or old-file cleanup unless those settings are explicitly stored in the table definition. [#119977](https://github.com/ClickHouse/ClickHouse/pull/119977) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed a `NATS` table whose settings all come from a named collection (`ENGINE = NATS(collection)` with no `SETTINGS` clause) being written to its metadata file with a dangling `SETTINGS` keyword. The stored definition could not be parsed again, so the table failed to load and aborted the load of its whole database, making unrelated tables in it inaccessible. Closes [#119120](https://github.com/ClickHouse/ClickHouse/issues/119120). [#120025](https://github.com/ClickHouse/ClickHouse/pull/120025) ([Groene AI](https://github.com/groeneai)). * Fixed non-idempotent formatting of a function named `SOME`: the name was printed unquoted, so the server read its own output back as the `SOME` array quantifier. `formatQuery('SELECT 1 < SOME(a, b -> 1)')` returned a query that re-parses to `arrayExists`, and a user-defined function whose body compared against `` `some`(...) `` was persisted without the quotes and then failed to load after a restart. [#120096](https://github.com/ClickHouse/ClickHouse/pull/120096) ([Groene AI](https://github.com/groeneai)). * Fix reading a column that has two streams whose names differ only by escaping, for example a `Tuple` element named `size0` inside an `Array`, or a typed `JSON` path named like a substream of a sibling path. Such columns returned wrong values, failed with `CANNOT_READ_ALL_DATA` or another exception, and made the table unmergeable. [#115670](https://github.com/ClickHouse/ClickHouse/pull/115670) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `least`, `greatest`, `bitShiftRight`, `midpoint`, and `avg2` returning wrong results when a JIT-compiled expression has an `Int128` result. With the default `compile_expressions = 1`, a negative `Int128` result could be treated as unsigned after a few executions. [#118363](https://github.com/ClickHouse/ClickHouse/pull/118363) ([Groene AI](https://github.com/groeneai)). * Fix `ttl_only_drop_parts` preventing expired columns from ever being cleared. A column `TTL` requires rewriting the part, so with this setting enabled an expired column could be kept forever instead of being cleared. Column-`TTL` rewrites are now scheduled separately and still run when `ttl_only_drop_parts` is enabled. [#118428](https://github.com/ClickHouse/ClickHouse/pull/118428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `mergeTreeTextIndex` bypassing row policies: reading the text index is now denied whenever a row policy applies to the table, regardless of which columns the policy uses. [#119032](https://github.com/ClickHouse/ClickHouse/pull/119032) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix `force_primary_key` incorrectly rejecting some queries with `INDEX_NOT_USED` when `optimize_extract_common_expressions` (enabled by default) simplifies the filter to a trivial primary-key condition wrapped in an implicit cast. [#119083](https://github.com/ClickHouse/ClickHouse/pull/119083) ([kasimtj](https://github.com/kasimtj)). * Fix `groupNumericIndexedVector` returning a value that is too large when an index repeats in the input, and returning different results depending on `max_threads`. Also fix `numericIndexedVectorPointwiseMultiply` and `numericIndexedVectorPointwiseDivide` dropping zero-valued indices, and `numericIndexedVectorPointwiseDivide` by a vector of all ones keeping the value of an index the divisor is missing instead of zeroing it. A `groupNumericIndexedVectorState` already stored in a table is not repaired by the upgrade and has to be recomputed. [#119564](https://github.com/ClickHouse/ClickHouse/pull/119564) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `isNull` and `IS NULL` now return `1` for every `NULL` row, including when the column's NULL mask holds a byte other than `0` or `1`, which the `Native` format permits and which `if` can produce from its condition column. Previously such a row could contribute its raw byte, so `sum(isNull(e))` returned `30` where `20` is correct. `if(cond, x, NULL)` with a `Nullable` `x` no longer returns `NULL` for a row whose condition byte was neither `0` nor `1`. If an older server stored a value computed from `isNull` on such a column (sorting key, skip index, partition key, projection, `MATERIALIZED` or `DEFAULT` column), rebuild it. [#119785](https://github.com/ClickHouse/ClickHouse/pull/119785) ([Groene AI](https://github.com/groeneai)). * Preserve explicit time zones in `timeSeriesRange` `DateTime64` results. [#119858](https://github.com/ClickHouse/ClickHouse/pull/119858) ([Minh Vu](https://github.com/fallintoplace)). #### Build/Testing/Packaging Improvement * Added a CMake file for compiling ClickHouse with the `nvcc` compiler. Related: [#116807](https://github.com/ClickHouse/ClickHouse/issues/116807). [#117734](https://github.com/ClickHouse/ClickHouse/pull/117734) ([Konstantin Vedernikov](https://github.com/scanhex12)). Added the `cudf` contrib library. [#117869](https://github.com/ClickHouse/ClickHouse/pull/117869) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added a build option to select 4 KiB or 16 KiB jemalloc page size on AArch64 instead of the default 64 KiB. [#115424](https://github.com/ClickHouse/ClickHouse/pull/115424) ([Azat Khuzhin](https://github.com/azat)). * Fixed several portability issues on `illumos`, including missing `termios.h`, register-macro conflicts, and floating-point environment casts. [#117363](https://github.com/ClickHouse/ClickHouse/pull/117363) ([Joshua Carp](https://github.com/jmcarp)). * ClickHouse is now built as a position independent executable on Linux amd64 and aarch64 with glibc, including the release builds, so the text segment is subject to ASLR. Addresses in the bare stack trace of a fatal log are now file offsets relative to the object that contains them, annotated with that object for frames outside the main executable, so they can be resolved with `addr2line -e ` regardless of where the binary was loaded. For the same reason `system.errors.last_error_trace`, `system.error_log.last_error_trace` and `system.crash_log.trace` now store a frame of the main executable as its file offset, keeping runtime addresses for frames in other objects. [#117449](https://github.com/ClickHouse/ClickHouse/pull/117449) ([Groene AI](https://github.com/groeneai)). [#119284](https://github.com/ClickHouse/ClickHouse/pull/119284) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed compilation of `benchmark_statistics` with `ENABLE_BENCHMARKS=ON`. [#116292](https://github.com/ClickHouse/ClickHouse/pull/116292) ([Groene AI](https://github.com/groeneai)). * Use `cctz` tzdata 2026c. [#116539](https://github.com/ClickHouse/ClickHouse/pull/116539) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Restored the bundled `aws-c-http` library to v0.11.0 after it had been accidentally reverted to an older submodule revision. [#116911](https://github.com/ClickHouse/ClickHouse/pull/116911) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix building `ArrowFlight` protobuf files with the CMake `Unix Makefiles` generator by creating the `protoc` output directory before invoking `protoc`. [#116879](https://github.com/ClickHouse/ClickHouse/pull/116879) ([Alexey Bakharew](https://github.com/alexbakharew)). * Reduced compile time and object file sizes by removing redundant template instantiations in comparison functions, binary arithmetic, `arrayDistance`, hashed dictionaries, and `argMin`/`argMax`. [#117540](https://github.com/ClickHouse/ClickHouse/pull/117540) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix building `unit_tests_dbms` when ClickHouse is configured with `-DENABLE_LIBRARIES=OFF` (and `-DENABLE_RUST=OFF`). [#105684](https://github.com/ClickHouse/ClickHouse/pull/105684) ([Ivan Babrou](https://github.com/bobrik)). * Harden official builds by enabling full RELRO: the global offset table is now mapped read-only after startup. [#119037](https://github.com/ClickHouse/ClickHouse/pull/119037) ([Raúl Marín](https://github.com/Algunenano)). * Cross-compilation sysroots updated: FreeBSD builds now target FreeBSD 14 (13 is end-of-life), riscv64 and loongarch64 builds use current Debian glibc and kernel headers. Unused files were removed from the sysroot submodule. [#119049](https://github.com/ClickHouse/ClickHouse/pull/119049) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed startup of the shell-based Docker server images when an absolute `filesystem_caches` path is a missing directory below a directory the server does not own. The entrypoint now prepares these cache directories with the same ownership handling as `storage_configuration` disk paths. [#118842](https://github.com/ClickHouse/ClickHouse/pull/118842) ([thien-ch](https://github.com/thien-ch)). * Update `openssl` to 3.5.8. [#118927](https://github.com/ClickHouse/ClickHouse/pull/118927) ([Actuele AI](https://github.com/actueleai)). * Update `xxHash` to v0.8.3. [#117597](https://github.com/ClickHouse/ClickHouse/pull/117597) ([Actuele AI](https://github.com/actueleai)). * Update `libxml2` to 2.15.4. [#118649](https://github.com/ClickHouse/ClickHouse/pull/118649) ([Actuele AI](https://github.com/actueleai)). * Update `libdivide` to v5.4.0. [#118660](https://github.com/ClickHouse/ClickHouse/pull/118660) ([Actuele AI](https://github.com/actueleai)). * Update `Jieba-CPP` to v1.0.0. [#118889](https://github.com/ClickHouse/ClickHouse/pull/118889) ([Actuele AI](https://github.com/actueleai)). * Update `minizip-ng` to 4.2.2. [#118886](https://github.com/ClickHouse/ClickHouse/pull/118886) ([Actuele AI](https://github.com/actueleai)). * Update `miniselect` to 0.4.0. [#118887](https://github.com/ClickHouse/ClickHouse/pull/118887) ([Actuele AI](https://github.com/actueleai)). * Update `openldap` to 2.6.15. [#118888](https://github.com/ClickHouse/ClickHouse/pull/118888) ([Actuele AI](https://github.com/actueleai)). [#119359](https://github.com/ClickHouse/ClickHouse/pull/119359) ([Actuele AI](https://github.com/actueleai)). * Update `sparsehash-c11` to v2.11.1. [#118891](https://github.com/ClickHouse/ClickHouse/pull/118891) ([Actuele AI](https://github.com/actueleai)). * Bump `croaring` from v4.5.1 to v5.1.1. Serialized bitmaps are unchanged on little-endian platforms; on big-endian platforms the on-disk layout of `groupBitmap` states changes to the portable little-endian format. [#118917](https://github.com/ClickHouse/ClickHouse/pull/118917) ([Actuele AI](https://github.com/actueleai)). * Update `xz` to v5.8.4. [#119360](https://github.com/ClickHouse/ClickHouse/pull/119360) ([Actuele AI](https://github.com/actueleai)). * Update `librdkafka` to v2.15.1. [#119467](https://github.com/ClickHouse/ClickHouse/pull/119467) ([Actuele AI](https://github.com/actueleai)). * Update `simdjson` to v4.6.11. [#119508](https://github.com/ClickHouse/ClickHouse/pull/119508) ([Actuele AI](https://github.com/actueleai)). * Fixed building ClickHouse with `clang` 23, which emitted a `lifetime-safety` diagnostic that was suppressed only for `clang` 24 and later. [#118005](https://github.com/ClickHouse/ClickHouse/pull/118005) ([Navneet Kumar](https://github.com/navneet83)). [#119211](https://github.com/ClickHouse/ClickHouse/pull/119211) ([mosya415](https://github.com/mosya415)). ### ClickHouse release 26.8 LTS, 2026-08-27. [Presentation](https://presentations.clickhouse.com/2026-release-26.8/), [Video](https://www.youtube.com/watch?v=5A9gRYE0v2M) #### Backward Incompatible Change * The `X-ClickHouse-Format` HTTP request header now explicitly selects the format of the response: it is an alias for the `output_format` setting, so it overrides the `FORMAT` clause of the query and the path extension. In previous versions, it only set the default when the output format was unspecified, like the `default_format` parameter. This provides an "out-of-band" option to request data in a desired format. It never changes how the request body of an `INSERT` is parsed - use `input_format` or `format` for that. [#105249](https://github.com/ClickHouse/ClickHouse/pull/105249) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Unquoted JSON numbers for `DateTime` and `DateTime64` columns in `JSONEachRow` and similar formats are now interpreted as Unix timestamps with optional sub-second precision, consistent with `Values`, `CAST`, and `toDateTime64`. Previously, a fractional number such as `1703363853.035` was rejected, and a bare integer such as `1703363853` was read as the raw scaled value of `DateTime64`, producing a `1970-...` timestamp. Quoted strings and ClickHouse's own JSON output are unaffected. [#108091](https://github.com/ClickHouse/ClickHouse/pull/108091) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The default value of `max_insert_threads` changed from `1` to `auto`, which resolves to the number of CPU cores available to the server. This parallelizes `INSERT SELECT` by default, and `max_insert_threads` can now also parallelize the writing side of a plain `INSERT` when the destination write path can safely fan out. This can change the number of parts created by such queries and the order of inserted rows. To restore the previous behavior, set `max_insert_threads` to `1`, or set `compatibility` to a version below `26.8`. [#109000](https://github.com/ClickHouse/ClickHouse/pull/109000) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#109006](https://github.com/ClickHouse/ClickHouse/pull/109006) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `PostgreSQL` and `MaterializedPostgreSQL` database engines now respect `remote_url_allow_hosts`, like the `PostgreSQL` table engine, the `postgresql` table function, and DDL-created dictionaries already do. With `remote_url_allow_hosts` configured, `CREATE DATABASE` and user-issued `ATTACH DATABASE` now reject `PostgreSQL` and `MaterializedPostgreSQL` databases that point at disallowed hosts with `UNACCEPTABLE_URL`. Existing databases still load at server startup. [#111135](https://github.com/ClickHouse/ClickHouse/pull/111135) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The experimental `ALP(STD)` codec now performs `Float32` scaling arithmetic in `Float64`. This improves compression ratios and eliminates exception-heavy compression of decimal data, but `Float32` values written by earlier versions may now decode 1 ULP differently. [#111627](https://github.com/ClickHouse/ClickHouse/pull/111627) ([Raufs Dunamalijevs](https://github.com/rienath)). * Removed the `library` dictionary source. `SOURCE(LIBRARY(...))` now fails with `UNKNOWN_ELEMENT_IN_CONFIG`, and the `dictionaries_lib_path` server setting is obsolete and has no effect. [#111980](https://github.com/ClickHouse/ClickHouse/pull/111980) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the Apache Arrow library-based reader and writer for the `Arrow` and `ArrowStream` formats. The native ClickHouse implementation, which has been the default since 26.7, is now the only one. The settings `input_format_arrow_use_native_reader` and `output_format_arrow_use_native_writer` are obsolete: they are still accepted, but have no effect, so a query that set them to `0` to force the Apache Arrow implementation now uses the native one. [#111996](https://github.com/ClickHouse/ClickHouse/pull/111996) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject timespan setting (milliseconds/seconds) values that overflow Int64 microseconds. [#112757](https://github.com/ClickHouse/ClickHouse/pull/112757) ([Azat Khuzhin](https://github.com/azat)). This might be an incompatibility only when you used absurdly large values that overflowed. * TLS credentials of a `MySQL` source (`ssl_ca`, `ssl_cert`, `ssl_key`) can no longer be specified as file paths from SQL, such as in `CREATE NAMED COLLECTION`, query arguments, or `CREATE DICTIONARY`, because the server opens those files with its own privileges. Paths remain supported in the server configuration file. Elsewhere, pass the certificate or key contents in the new `ssl_ca_pem`, `ssl_cert_pem`, and `ssl_key_pem` parameters, which are masked in logs and in `SHOW` queries like passwords are. [#112070](https://github.com/ClickHouse/ClickHouse/pull/112070) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Privilege checks for the `SYSTEM ... CACHE ON CLUSTER` commands now use each command's own privilege instead of the `SYSTEM DROP CACHE` privilege group. This lets holders of a single granular cache privilege run its matching command, and prevents a holder of the group from running `SYSTEM SYNC FILESYSTEM CACHE ON CLUSTER` without `SYSTEM SYNC FILESYSTEM CACHE`. [#114042](https://github.com/ClickHouse/ClickHouse/pull/114042) ([Groene AI](https://github.com/groeneai)). * For views whose body is a plain `SELECT` over a single `Distributed` table, the whole outer query is now pushed to the shards (new setting `optimize_trivial_view_pushdown_to_distributed`, enabled by default). This changes observable behaviour for such views: `FINAL` and `SAMPLE` written on the view reference are now propagated to the shard-local table instead of being ignored, and `extremes` is not reported on single-shard clusters. Set `optimize_trivial_view_pushdown_to_distributed = 0` to restore the previous behaviour. [#101791](https://github.com/ClickHouse/ClickHouse/pull/101791) ([simonmichal](https://github.com/simonmichal)). * Lightweight `UPDATE` patch parts now use a new v2 on-disk format sorted by (`sorting_key..., _block_number, _block_offset`) and applied with a new merging algorithm. Peak memory is bounded by the largest equal-sort-key run instead of the full patch, and updates that cross merge boundaries no longer fall back to in-memory Join apply. Old-format patch parts remain readable. During a rolling upgrade from a version before 26.8, keep `patch_parts_version = 'v1'` or use the `compatibility` setting until all replicas are upgraded. [#103182](https://github.com/ClickHouse/ClickHouse/pull/103182) ([Anton Popov](https://github.com/CurtizJ)). * Added the `VALID FOR ` clause to `CREATE USER` and `ALTER USER` as a shorthand for `VALID UNTIL`. The expiration deadline is computed as the current time plus the given interval at query execution time and stored in the `VALID UNTIL` form. The `valid_until` column of the `system.users` table now has the type `Array(DateTime64(0))` instead of `Array(DateTime)`, so that deadlines beyond the year 2106 are represented exactly; tooling that reads this column should handle the new type. [#110171](https://github.com/ClickHouse/ClickHouse/pull/110171) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `EXPLAIN SYNTAX` now returns the reformatted query as a single `String` record (with embedded newlines) instead of one record per line, so its output is a recoverable single row that is directly usable (for example, `SELECT count() FROM (EXPLAIN SYNTAX ...)` returns `1`). This is controlled by the new `single_record` option, which defaults to `1`; set `single_record = 0` to restore the historical one-record-per-line output. Other `EXPLAIN` kinds (`PLAN`/`PIPELINE`/`AST`) keep their per-line tree output. [#110479](https://github.com/ClickHouse/ClickHouse/pull/110479) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Extended the supported range of `Date32` from `[1900-01-01, 2299-12-31]` to `[0000-01-01, 9999-12-31]`, matching `DateTime64`. Parsing and conversions now accept the extended range instead of silently clamping to the old boundaries. Backward compatibility notes: in the numeric conversion `toDate32(N)`, values in `[120530, 2932896]` are now interpreted as day numbers (dates from `2300-01-01` to `9999-12-31`) instead of Unix timestamps in early 1970, matching the rule that a number that fits into the day-number range is a day number; numbers below the day number of `0000-01-01` and timestamps after `9999-12-31` saturate to the new boundaries. [#111534](https://github.com/ClickHouse/ClickHouse/pull/111534) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `arrayIntersect` and `arraySymmetricDifference` no longer treat a value repeated inside a single argument as if it appeared in several arguments. Queries relying on the previous behavior can now return different results: `arrayIntersect([1], [2], [1, 1])` returns `[]` instead of `[1]`, `arrayIntersect([1, 2], [2], [1, 1, 2])` returns `[2]` instead of `[1, 2]`, and `arraySymmetricDifference([1], [2], [1, 1])` returns `[2, 1]` instead of `[2]`. For `arraySymmetricDifference` two arguments are already enough: `arraySymmetricDifference([1], [2, 2])` returns `[2, 1]` instead of `[1]`. A value is now counted for an argument only when it was present in every argument before it, so the result contains exactly the values present in all of the arguments. As part of the same change, `arrayIntersect` builds its hash table from the smallest argument rather than from all of them, which makes it up to 1.85x faster and use a third less memory when the arguments differ a lot in size. `arrayUnion` is not affected. [#113021](https://github.com/ClickHouse/ClickHouse/pull/113021) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `disable_insertion_and_mutation` now prevents background consumption from `Kafka`, `RabbitMQ`, and `NATS` tables while still allowing direct writes to external storage. Gated `Kafka2`, `NATS`, and `RabbitMQ` tables do not initialize consumers for direct `SELECT`. The recently introduced `message_queue_disable_insertion` setting now requires a server restart. [#113660](https://github.com/ClickHouse/ClickHouse/pull/113660) ([Christoph Wurm](https://github.com/cwurm)). * A window `PARTITION BY` or `ORDER BY` over an `AggregateFunction` column is now rejected with `ILLEGAL_COLUMN`, as top-level `ORDER BY` over such a column already was. Previously such a query was accepted by at least one analyzer, and window `PARTITION BY` partitioned differently depending on `max_threads`. The refusal also covers a state nested in `Array`, `Tuple`, `Map`, `Variant` or `SimpleAggregateFunction`. A `SimpleAggregateFunction` over an ordinary type, `QBit`, and `GROUP BY` and `DISTINCT` over a state, are unaffected. [#113878](https://github.com/ClickHouse/ClickHouse/pull/113878) ([Groene AI](https://github.com/groeneai)). * The `include_from` setting no longer defaults to `/etc/metrika.xml`. Previously that file was used for configuration substitutions whenever it existed, even though nothing in the ClickHouse configuration referred to it. If you relied on that implicit substitutions file, add `/etc/metrika.xml` explicitly to each affected config file; separately loaded `users.xml` and XML dictionary configs need their own `include_from` element. [#114161](https://github.com/ClickHouse/ClickHouse/pull/114161) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `NATS` table engine accepts credentials inline in the new `nats_credentials` setting (the same payload as a `.creds` file), and no longer accepts `nats_credential_file` from SQL: the path is a reference to a file on the server filesystem, which the server opens with its own privileges, so it can only be specified in a named collection defined in the server configuration file, or as `nats.credential_file` in the server configuration itself. A query may replace such a configured path with inline `nats_credentials`, unless the operator pinned it with ``. Tables created before this restriction keep working after an upgrade. [#114644](https://github.com/ClickHouse/ClickHouse/pull/114644) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Asynchronous metrics can now have the `Map` data type, and the per-CPU-core and per-device metrics were converted to single key-value metrics: `OSUserTimeCPU0`, `OSUserTimeCPU1`, ... became a single `OSUserTimeCPU` metric with a map from the CPU core number to the value (and similarly for the other `OS*TimeCPU*` metrics, `CPUFrequencyMHz_*`, `Temperature*`, `EDAC*`, `Block*_*`, `Network(Receive|Send)*_*`, `Disk*_*`, `*BlobsQueueEstimate`, and `AsyncLogging*QueueSize`). `system.asynchronous_metrics` has a new `key_values Map(LowCardinality(String), Float64)` column (the `value` column is `NaN` for such metrics), `system.asynchronous_metric_log` logs them as one row per key using a new `key` column, the Prometheus endpoint exports them with a label (e.g. `ClickHouseAsyncMetrics_BlockReadBytes{device="sda"}`), and the Graphite `MetricsTransmitter` sends them as `..`. If your monitoring reads the old metric names, set the new `asynchronous_metrics_key_values_mode` server setting to `legacy_names` to publish every key as a separate scalar metric under its previous name again, or to `both` to publish both forms at the same time while the monitoring is being migrated; the setting is applied by `SYSTEM RELOAD CONFIG`, without a restart. [#115333](https://github.com/ClickHouse/ClickHouse/pull/115333) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#116791](https://github.com/ClickHouse/ClickHouse/pull/116791) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### New Feature * Added `CREATE HANDLER`, `ALTER HANDLER` and `DROP HANDLER` statements to define custom HTTP handlers from SQL, persisted in a local or Keeper storage. Added the `currentHandler` and `currentRequestURL` functions, the `system.handlers` table, and the `http_handler_name` / `http_request_url` columns in `system.query_log`. [#106231](https://github.com/ClickHouse/ClickHouse/pull/106231) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a way to access tables and databases via URL paths in the HTTP interface (e.g. \`/database/table.format.gz?filter=a>0\`), plus new settings (\`http_allow_database_as_path\`, \`http_allow_table_as_file\`, \`http_allow_filters_as_path\`, \`http_allow_filters_as_unrecognized_url_parameters\`, \`select\`, \`order\`, \`sort\`, \`filter\`, \`page\`, \`compression\`, \`format\`, \`input_format\`, \`output_format\`, \`default_format\`, \`database\`) that compose with one another and with the existing \`query\` URL parameter. [#105249](https://github.com/ClickHouse/ClickHouse/pull/105249) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add framing formats, selected by the new setting `framing_output_format`: they multiplex different response parts of the query in a single HTTP response stream — chunks of data, totals and extremes, progress packets, profile events, server logs, and exceptions. Implemented framing formats: `None` (default, everything works as before), `EventStream` (HTTP server-sent events), `JSONEachPacketBase64`, and `JSONEachPacketString` (a JSON object per packet with base64-encoded or string data). [#110127](https://github.com/ClickHouse/ClickHouse/pull/110127) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `default_session_user` server setting: the user to authenticate as when a client connects without specifying a user name (previously hardcoded to `default`; the native, MySQL, and PostgreSQL protocols now accept an empty user name instead of rejecting it, and Arrow Flight honors the setting instead of its own hardcoded `default`). Composable protocol endpoints in the `protocols` section can override it per endpoint with a `default_session_user` key, so different listening ports can serve different anonymous users. Setting it to an empty string prohibits connections without a user name. Interserver connections never use the default session user. [#110179](https://github.com/ClickHouse/ClickHouse/pull/110179) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support pipe operators in SQL queries: `FROM t |> WHERE x > 1 |> AGGREGATE count() AS c GROUP BY y |> ORDER BY c DESC |> LIMIT 10`, similar to the pipe syntax of GoogleSQL. Any `SELECT` query can be followed by a chain of `|>` operators, and each operator wraps the query before it into a subquery, so the resulting AST is the same as for the equivalent query with nested subqueries. In a query that starts with the `FROM` clause, the `SELECT` clause is now optional and defaults to `SELECT *`. [#111151](https://github.com/ClickHouse/ClickHouse/pull/111151) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `parseQueryToJSON` and `formatQueryFromJSON` functions to serialize and deserialize ASTs as JSON, and an experimental `clickhouse_json` dialect enabled by `enable_json_ast_dialect`. [#100412](https://github.com/ClickHouse/ClickHouse/pull/100412) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#113480](https://github.com/ClickHouse/ClickHouse/pull/113480) ([Nikita Fomichev](https://github.com/fm4v)). * New setting `run_query_in_background`. The server accepts the query, immediately returns an empty result, and runs it to completion regardless of what happens to the connection. The result is discarded. Track the query by its `query_id` in `system.processes` and `system.query_log`. Intended for long queries like `INSERT ... SELECT`, `CREATE TABLE … AS SELECT`, or `CREATE MATERIALIZED VIEW … POPULATE` that must not die with a dropped connection. [#112816](https://github.com/ClickHouse/ClickHouse/pull/112816) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Added a new system table, `system.user_query_log`, which shows every user their own query log records without requiring access to the query log table. The original implementation and the idea belong to Yue Ni (@niyue), [#104462](https://github.com/ClickHouse/ClickHouse/issues/104462). [#110156](https://github.com/ClickHouse/ClickHouse/pull/110156) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implement the remaining items of the URL unification (issue [#59617](https://github.com/ClickHouse/ClickHouse/issues/59617)). The new `s3_base` setting resolves relative URLs in the `s3` table function and the `S3` table engine (and the functions sharing their configuration: `s3Cluster`, `gcs`, `oss`), similarly to `url_base`. The new `URL` database engine treats table names as URLs, resolves them against an optional base URL, and dispatches by the URL scheme, so files, web and object storage are handled uniformly. It replaces `Filesystem` in the default (`Overlay`) database of `clickhouse-local` with a `file://` base URL, so plain file names keep working, and you can also write `SELECT * FROM 'https://example.com/data.csv'`. [#111512](https://github.com/ClickHouse/ClickHouse/pull/111512) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `bigquery` table function and the `BigQuery` table engine for reading from and writing to Google BigQuery tables, including public datasets. The table structure is inferred from the BigQuery table schema. Authentication supports an OAuth access token, a service account key in `JSON` format, and an OAuth client with a refresh token. [#110166](https://github.com/ClickHouse/ClickHouse/pull/110166) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Plain `CREATE MATERIALIZED VIEW ... POPULATE` is now locally atomic: the view is subscribed to new inserts of the source table and the existing data is snapshotted together under a brief exclusive lock on the source, so rows inserted through the same server concurrently with the population are no longer missed or duplicated (new setting `materialized_views_populate_atomically`, enabled by default). The guarantee covers the local insert path only; inserts arriving on another replica or through a distributed write path are outside the cut, and it requires a source that can provide a pinned snapshot (the `MergeTree` family and `Memory`); other sources, as well as `CREATE OR REPLACE` / `REPLACE` and views created in `Replicated` databases, keep the legacy non-atomic population. `POPULATE` can now also be used together with `TO` to backfill the target table. [#108715](https://github.com/ClickHouse/ClickHouse/pull/108715) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Keeper can store data on disk now, in a custom LSM tree. It has similar performance to the previous storage. Use coordnation setting `use_lsmt_storage = true` to enable, `storage_memory_only = false` to store files on disk, add `data_storage_path` or `data_storage_disk` to config to specify where to put the files. To write to S3, point `data_storage_disk` to a disk of type `s3_plain`. [#113903](https://github.com/ClickHouse/ClickHouse/pull/113903) ([Michael Kolupaev](https://github.com/al13n321)). * The "S3 tables" engine catalog for data lakes, the same as [#103220](https://github.com/ClickHouse/ClickHouse/pull/103220) but with a working `INSERT`. [#113505](https://github.com/ClickHouse/ClickHouse/pull/113505) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Add support for Snowflake Horizon. You can now query Iceberg table in Iceberg behind the horizon catalog. You can also write to the Iceberg table via the catalog. [#114547](https://github.com/ClickHouse/ClickHouse/pull/114547) ([Melvyn Peignon](https://github.com/melvynator)). * Added support for the `Puffin` file format. It can be used with the `file`, `url`, `s3`, and similar table functions. [#103936](https://github.com/ClickHouse/ClickHouse/pull/103936) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Support the `GROUPS` frame mode for window functions (SQL:2011), e.g. `any(price) OVER (PARTITION BY symbol ORDER BY ts GROUPS BETWEEN CURRENT ROW AND 1 FOLLOWING)`. In a `GROUPS` frame the boundaries count whole peer groups — sets of rows that are equal on the `ORDER BY` key — so `N PRECEDING`/`N FOLLOWING` mean `N` peer groups before/after the current row's peer group, rather than physical rows (`ROWS`) or `ORDER BY` value distances (`RANGE`). [#108653](https://github.com/ClickHouse/ClickHouse/pull/108653) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Added the `mergeTreeCodecBlockCounts` table function, which reports for each part, column, and substream of a `MergeTree` table how many compressed blocks use each codec. [#109623](https://github.com/ClickHouse/ClickHouse/pull/109623) ([Raufs Dunamalijevs](https://github.com/rienath)). * Added the `MultiPoint` geo data type, stored as `Array(Point)`, and included it in the `Geometry` type. [#109951](https://github.com/ClickHouse/ClickHouse/pull/109951) ([David Meng](https://github.com/davidmenggx)). * Added the `_etag` virtual column for `HDFS` storage. [#108255](https://github.com/ClickHouse/ClickHouse/pull/108255) ([Zhang Yifan](https://github.com/zhangyifan27)). * The native `ORC` reader and output format now support `uniontype`, mapping it to the ClickHouse `Variant` type. Previously, reading such columns failed with `Unsupported ORC type`, and writing them failed with `ILLEGAL_COLUMN`. [#110078](https://github.com/ClickHouse/ClickHouse/pull/110078) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#110085](https://github.com/ClickHouse/ClickHouse/pull/110085) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a `japanese` tokenizer for text indexes and the `tokens`, `hasAnyTokens`, and `hasAllTokens` functions, using the MeCab morphological analyzer. The dictionary is loaded at runtime from a location set in the server configuration and verified against a SHA-256 checksum before use. [#111420](https://github.com/ClickHouse/ClickHouse/pull/111420) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Added a `chinese` tokenizer for the `tokens` function and `MergeTree` text indexes. It segments Chinese text into words using a dictionary and a Hidden Markov Model (the algorithm follows [jieba](https://github.com/fxsjy/jieba)), with `coarse_grained` (default) and `fine_grained` granularities. Continues [#80174](https://github.com/ClickHouse/ClickHouse/issues/80174). [#89945](https://github.com/ClickHouse/ClickHouse/pull/89945) ([Amos Bird](https://github.com/amosbird)). * Added a new `icu` tokenizer for text indexes and the `tokens`, `hasAnyTokens`, and `hasAllTokens` functions, providing locale-aware Unicode word segmentation (including dictionary-based segmentation for languages such as Chinese, Japanese, and Thai). [#109940](https://github.com/ClickHouse/ClickHouse/pull/109940) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Added a `splitByRegexp` tokenizer for text indexes and the `tokens` function. It splits the input into tokens using a regular expression as the separator, for example `tokenizer = splitByRegexp('[^\p{L}\p{N}#+]+')`, which allows preserving tokens containing special characters such as `C++` or `C#` that other tokenizers would break apart. [#110002](https://github.com/ClickHouse/ClickHouse/pull/110002) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * The Keeper HTTP dashboard now includes a Cluster tab that shows Raft membership as a topology graph with health colors, lag, and priority, so you can see which nodes are healthy, lagging, or unreachable and open the leader's dashboard directly. [#111655](https://github.com/ClickHouse/ClickHouse/pull/111655) ([yanglongwei](https://github.com/ylw510)). * A new server configuration section, `create_union_system_log_tables`, requests the creation and automatic maintenance of `all_...` tables (e.g. `system.all_query_log`) that query the union of a system log table, its rotated versions and/or the same tables across all replicas of a cluster. The `remote`, `remoteSecure`, `cluster` and `clusterAllReplicas` table functions now accept a trailing `SETTINGS` clause with the settings of the `Distributed` storage they create, e.g. `clusterAllReplicas('default', system.query_log, SETTINGS skip_unavailable_shards = 1)`. [#112670](https://github.com/ClickHouse/ClickHouse/pull/112670) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `nats_credentials` setting to the NATS table engine, allowing users to specify NATS credentials inline as a string (matching the payload of a `.creds` file). [#110733](https://github.com/ClickHouse/ClickHouse/pull/110733) ([addshore](https://github.com/addshore)). * Added the sort-based `IEJoin` algorithm for joins whose `ON` clause has two inequality comparisons (`<`, `<=`, `>`, `>=`) between the joined tables, enabled by adding `ie_join` to the `join_algorithm` setting. Supported kinds are `ALL INNER/LEFT/RIGHT/FULL JOIN` and `SEMI`/`ANTI LEFT/RIGHT JOIN`. Previously such queries were executed as a `CROSS JOIN` with a filter (`INNER` only), which is much slower on large tables. [#109920](https://github.com/ClickHouse/ClickHouse/pull/109920) ([Vladimir Cherkasov](https://github.com/vdimir)). * Added the aggregate function `gini`, which computes the Gini coefficient of finite, non-negative numeric values, including grouped and distributed aggregations. [#112280](https://github.com/ClickHouse/ClickHouse/pull/112280) ([Amirreza Akhondi](https://github.com/amirreza1307)). [#114643](https://github.com/ClickHouse/ClickHouse/pull/114643) ([Groene AI](https://github.com/groeneai)). * Added the `HiveText` output format, which writes Apache Hive `LazySimpleSerDe` text. Fields are separated by `\x01`; rows by the `format_hive_text_rows_delimiter` setting (default `\n`); and nested `Array`, `Map`, and `Tuple` values use Hive's separator list. The output targets Hive's default `LazySimpleSerDe` and is not symmetric with ClickHouse's `HiveText` input for nested values or custom row delimiters. [#107582](https://github.com/ClickHouse/ClickHouse/pull/107582) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added support for constant labels on the Prometheus metrics endpoint: a new `` section inside `` adds the configured labels to every exposed metric, which helps distinguish multiple clusters scraped by the same Prometheus. Label values support standard config substitutions such as `from_env`. Closes [#85969](https://github.com/ClickHouse/ClickHouse/issues/85969). [#111672](https://github.com/ClickHouse/ClickHouse/pull/111672) ([Valery Petrov](https://github.com/valerypetrov)). * Added the `always_fetch_mutated_part` setting. When enabled, a replica can download mutated parts from another replica instead of executing mutations locally. [#113020](https://github.com/ClickHouse/ClickHouse/pull/113020) ([Rory Shanks](https://github.com/rorylshanks)). * Added the aggregate function `mergedJSONPatch` for RFC 7396-style merge-patch aggregation over `JSON` values, including state-merge support and `AggregatingMergeTree` usage. Users can aggregate a `JSON` column using a timestamp or another ordering column as the sort key to compute the latest state. [#108349](https://github.com/ClickHouse/ClickHouse/pull/108349) ([larryluogit](https://github.com/larryluogit)). * Added the `aiSimilarity` function, which computes the semantic similarity between two texts using an embedding model. It returns a `Nullable(Float32)` in `[-1, 1]`, where `1` means the texts are identical, or `NULL` if an operand is `NULL`/empty or its embedding failed. [#110777](https://github.com/ClickHouse/ClickHouse/pull/110777) ([David Meng](https://github.com/davidmenggx)). * Add `restore_table_data`, `restore_access_entities`, and `restore_functions` restore settings for granular control over what gets restored. These can override `structure_only` for individual categories — e.g. `structure_only=true, restore_access_entities=true` restores table definitions and access entities without table data. [#102402](https://github.com/ClickHouse/ClickHouse/pull/102402) ([Nikita Fomichev](https://github.com/fm4v)). * The array subscript operator supports an array of integers as the index: `arr[indexes]` returns the elements at all of the given positions, equivalently to `arrayMap(i -> arr[i], indexes)`. [#108371](https://github.com/ClickHouse/ClickHouse/pull/108371) ([folly](https://github.com/ucasfl)). * Add function `notHas`, the negation of `has` for arrays, maps, and JSON. When the haystack is a constant array, `notHas(constant_array, x)` is rewritten to `x NOT IN constant_array` by `optimize_rewrite_has_to_in` (enabled by default), so it executes via a set lookup and can prune by the primary key index like `NOT IN`. [#109926](https://github.com/ClickHouse/ClickHouse/pull/109926) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Functions `arrayElement` (the `vec[n]` operator) and `arraySlice` now work for the `QBit` data type: `qbit[n]` returns the n-th vector element at full precision, and `arraySlice(qbit, offset, length)` returns a projection to a subset of dimensions. Both read only the bit planes of the stride groups they need, and slices aligned to stride-group boundaries reuse the stored streams without copying. This closes [#109942](https://github.com/ClickHouse/ClickHouse/issues/109942). [#109953](https://github.com/ClickHouse/ClickHouse/pull/109953) ([Utkal Singh](https://github.com/Utkal059)). * Support animated PNG in the `PNG` output format. A `t` column turns the result of a query into an animation, with the time scale of `t` set by `output_format_image_time_multiplier_seconds` and `output_format_image_time_divisor_seconds`, and `output_format_image_streaming_animation` to write the frames out as the query produces them instead of buffering them in memory. [#112846](https://github.com/ClickHouse/ClickHouse/pull/112846) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * New function `aiRedact` that detects and redacts personally identifiable information (PII) in text using an LLM provider. Specify the categories to redact (e.g. `['email', 'name']`) or pass an empty array to use a default set of PII categories. Matched values are replaced with a token (`[REDACTED]` by default). [#110464](https://github.com/ClickHouse/ClickHouse/pull/110464) ([David Meng](https://github.com/davidmenggx)). * Add `aiFilter` function that evaluates a natural-language condition against text with an LLM and returns `UInt8` for use in `WHERE`, `PREWHERE`, and `JOIN ... ON`. [#110594](https://github.com/ClickHouse/ClickHouse/pull/110594) ([yanglongwei](https://github.com/ylw510)). * Support TLS/SSL connections to PostgreSQL for the `PostgreSQL` table engine, the `postgresql` table function, the `PostgreSQL` and `MaterializedPostgreSQL` database engines, and `PostgreSQL` dictionaries: `sslmode` plus the certificates and the key, given either as literal contents (`sslrootcert_pem`, `sslcert_pem`, `sslkey_pem`; masked like passwords) or as paths (`sslrootcert`, `sslcert`, `sslkey`; accepted only from a named collection defined in the server configuration file). [#110615](https://github.com/ClickHouse/ClickHouse/pull/110615) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * ClickHouse server now has an introspection port. This is a native protocol TCP listener that starts before the server begins attaching tables and stops only after the tables' detach completes. During these windows, an operator can connect to it with `clickhouse client` and run queries such as `SHOW PROCESSLIST`, `SELECT * FROM system.stack_trace`, or `SYSTEM INSTRUMENT ADD 'QueryMetricLog::startQuery' SLEEP ENTRY 0.5`. Additionally, the default value of the `shutdown_wait_unfinished` server setting is increased from 5 to 120 seconds. The previous default was smaller than the connection poll interval, so the server could force exit in the middle of shutdown instead of waiting for outstanding connections to finish. [#110838](https://github.com/ClickHouse/ClickHouse/pull/110838) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Introduce emulated ZooKeeper metrics in Keeper `mntr` command: `zk_leader_uptime`, `zk_sum_election_time`, `zk_cnt_election_time`, `zk_sum_leader_unavailable_time`, `zk_cnt_leader_unavailable_time`; Also introduce related leader-oriented metrics for Keeper only: `KeeperLastLeaderElectionTime`, `KeeperLastLeaderUnavailableTime`. [#113334](https://github.com/ClickHouse/ClickHouse/pull/113334) ([Maxim Orlovsky](https://github.com/UberDever)). * Support ALTER TABLE ... MODIFY PROJECTION to change projection-level settings (e.g. index_granularity) of an existing projection without rebuilding it. The new settings apply lazily via merges. [#113343](https://github.com/ClickHouse/ClickHouse/pull/113343) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Added a `finish_time` column to `system.mutations` that records when a mutation was completed. Unfinished mutations and mutations whose completion time is unknown report zero. [#113474](https://github.com/ClickHouse/ClickHouse/pull/113474) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Added a new `private preview` feature tier between `experimental` and `beta`, so feature tiers are now ordered `experimental < private preview < beta < production`. The server setting `allow_feature_tier` gains a level: `0` allows all tiers, `1` excludes experimental, `2` additionally excludes private preview, and `3` allows production settings only. Configurations that previously pinned `allow_feature_tier = 2` to allow production settings only must now use `3`. Closes [#113481](https://github.com/ClickHouse/ClickHouse/issues/113481). [#113581](https://github.com/ClickHouse/ClickHouse/pull/113581) ([Groene AI](https://github.com/groeneai)). * The built-in Web SQL UI (`/play`) can now sort a result by its columns, filter it by their values, and page through it, without editing the query. Sorting arrows and a filter input appear in every column header, a selected cell offers the comparisons that fit its value, and a pager with an adjustable page size appears under a result that does not fit on one screen. It is all performed by the server, using the `order`, `filter`, `limit` and `page` query-construction settings, so the whole result is shaped rather than the rows that happen to be displayed; the chosen sort, filters and page travel in the page URL, so a shared link reproduces the result. [#115540](https://github.com/ClickHouse/ClickHouse/pull/115540) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `chdig` [v26.8.1](https://github.com/azat/chdig/releases/tag/v26.8.1): ratatui, tmux panes, tmuxinator sessions, sharing logs with colors, [feature tour](https://github.com/azat/chdig/blob/v26.8.1/Documentation/Features.md). [#115216](https://github.com/ClickHouse/ClickHouse/pull/115216) ([Azat Khuzhin](https://github.com/azat)). #### Experimental Feature * Added an experimental Cascades cost-based optimizer for distributed query plans, enabled by `enable_cascades_optimizer = 1` together with `make_distributed_plan = 1`. It chooses between shuffle, broadcast, replicated, and local join strategies, two-phase, shuffle, and local aggregation, two-stage distributed top-N, and parallel and replicated reads by estimated cost, inserting exchange operators as needed. [#86353](https://github.com/ClickHouse/ClickHouse/pull/86353) ([Alexander Gololobov](https://github.com/davenger)). * Added an experimental `MergeTree` setting `enable_adaptive_codec_selection`. When enabled, merges and mutations pick the smallest-output codec per block for columns that use the default codec. [#111834](https://github.com/ClickHouse/ClickHouse/pull/111834) ([Raufs Dunamalijevs](https://github.com/rienath)). * Adaptive codec selection now compresses with each candidate codec at most once per block instead of measuring all candidates and then recompressing the winner. [#113511](https://github.com/ClickHouse/ClickHouse/pull/113511) ([Raufs Dunamalijevs](https://github.com/rienath)). * The `clickhouse` binary builds for WebAssembly (`wasm64`, via Emscripten), and `clickhouse local` runs queries, including `MergeTree` tables, under Node.js ≥ 24 and in the browser. A new `Build (wasm64)` CI job pins the build and the Node.js execution path (the browser path is not covered by CI yet) and provides the WebAssembly module as an artifact. [#113404](https://github.com/ClickHouse/ClickHouse/pull/113404) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `parallel_replicas_plan_based` now chooses the local/remote boundary by optimizing the query plan, can parallelize queries over views with a `UNION` without `parallel_replicas_allow_view_over_mergetree`, falls back to the local plan for `IN (subquery)` queries instead of failing, and now also distributes eligible `JOIN`s (`INNER`, `LEFT`, `RIGHT`, including `SEMI`/`ANTI`). [#111063](https://github.com/ClickHouse/ClickHouse/pull/111063) ([Igor Nikonov](https://github.com/devcrafter)). [#112268](https://github.com/ClickHouse/ClickHouse/pull/112268) ([Igor Nikonov](https://github.com/devcrafter)). [#112443](https://github.com/ClickHouse/ClickHouse/pull/112443) ([Igor Nikonov](https://github.com/devcrafter)). * Distributed query plans (`make_distributed_plan`) now also support queries whose source is known at planning time to produce no rows, such as reads from an empty `MergeTree` table. [#111941](https://github.com/ClickHouse/ClickHouse/pull/111941) ([Groene AI](https://github.com/groeneai)). * Added the `use_projection_index_in_read_pools` setting, disabled by default. When enabled, ranges fully filtered out by a projection or skip index are dropped inside `MergeTree` read pools before read tasks are created for them, instead of being skipped granule by granule during reading. This avoids creating read tasks and setting up readers for data that will not be read. [#110291](https://github.com/ClickHouse/ClickHouse/pull/110291) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). [#112296](https://github.com/ClickHouse/ClickHouse/pull/112296) ([Anton Popov](https://github.com/CurtizJ)). * PromQL: fixed operations on instant vectors without tags, e.g. `vector(1) + vector(2)`, `topk(1, vector(1))`, `label_replace(vector(1), ...)` and binary operators with `on()`, which failed with a type error. [#111872](https://github.com/ClickHouse/ClickHouse/pull/111872) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL: implemented the `changes` and `resets` functions. [#112352](https://github.com/ClickHouse/ClickHouse/pull/112352) ([Valery Petrov](https://github.com/valerypetrov)). * Report the error that cancelled a distributed query (`make_distributed_plan = 1`) instead of a bare `QUERY_WAS_CANCELLED`. A failing worker task's real error was previously visible only in the server log. [#112590](https://github.com/ClickHouse/ClickHouse/pull/112590) ([Alexander Gololobov](https://github.com/davenger)). * When `make_distributed_plan = 1`, ClickHouse now disables features that distributed query plans do not support yet instead of letting them fail later. Closes [#109476](https://github.com/ClickHouse/ClickHouse/issues/109476). [#112463](https://github.com/ClickHouse/ClickHouse/pull/112463) ([alesapin](https://github.com/alesapin)). * Distributed query plans (`make_distributed_plan`) now use lazy materialization for `ORDER BY ... LIMIT k` queries. [#113111](https://github.com/ClickHouse/ClickHouse/pull/113111) ([Shankar Iyer](https://github.com/shankar-iyer)). * Support non-partitioned window functions (`OVER (ORDER BY ...)`) in distributed query plans (`make_distributed_plan`). Previously any query with a window function failed because `WindowStep` could not be serialized for remote execution; windows with `PARTITION BY` are still rejected for now. [#109802](https://github.com/ClickHouse/ClickHouse/pull/109802) ([Vighnesh Pathrikar](https://github.com/VighneshPath)). * Added the setting `use_query_condition_cache_for_top_k`, which gates the query condition cache for `ORDER BY ... LIMIT n` (TopK) queries. [#111492](https://github.com/ClickHouse/ClickHouse/pull/111492) ([Alexey Milovidov](https://github.com/alexey-milovidov)). The [query condition cache](/operations/query-condition-cache) is now enabled by default for queries that use the `ORDER BY LIMIT n` (TopK) optimization. It can be turned off again with the setting `use_query_condition_cache_for_top_k`. [#114539](https://github.com/ClickHouse/ClickHouse/pull/114539) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added static partition-to-shard affinity for `StorageKafka2` via the `kafka_partition_shard_num` and `kafka_shard_count` settings, allowing multiple ClickHouse shards to deterministically consume disjoint subsets of Kafka partitions. [#108886](https://github.com/ClickHouse/ClickHouse/pull/108886) ([johnjing](https://github.com/JingYanchao)). * Harden the experimental AI functions: deny insecure (`http`) endpoints to remote hosts by default (new setting `ai_function_allow_insecure_endpoint`), bound outbound API calls per query by default (`ai_function_max_api_calls_per_query` now defaults to `1000`), and sanitize provider error responses written to the logs. [#111286](https://github.com/ClickHouse/ClickHouse/pull/111286) ([George Larionov](https://github.com/george-larionov)). * PromQL: `quantile` with a quantile level outside [0, 1] now returns -Inf/+Inf/NaN like Prometheus instead of throwing an error, and `histogram_quantile` now ignores input series whose `le` label is missing or unparsable, like Prometheus. [#111871](https://github.com/ClickHouse/ClickHouse/pull/111871) ([Valery Petrov](https://github.com/valerypetrov)). * Allow multi-component identifiers in TimeSeries tables. [#112799](https://github.com/ClickHouse/ClickHouse/pull/112799) ([Vitaly Baranov](https://github.com/vitlibar)). * Added `TimeSeries` settings `samples_index_granularity` and `tags_inner_granularity` to control the index granularity of inner tables. [#113075](https://github.com/ClickHouse/ClickHouse/pull/113075) ([Vitaly Baranov](https://github.com/vitlibar)). * PromQL: added the `deriv` function and fixed a bug where its backing aggregate function returned values 1000x too small with millisecond-resolution timestamps. [#112360](https://github.com/ClickHouse/ClickHouse/pull/112360) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL: reject invalid cross-delimiter escapes in strings and invalid Unicode surrogate code points. [#113587](https://github.com/ClickHouse/ClickHouse/pull/113587) ([Minh Vu](https://github.com/fallintoplace)). [#113203](https://github.com/ClickHouse/ClickHouse/pull/113203) ([Minh Vu](https://github.com/fallintoplace)). * PromQL: preserve finite values when using modulo with an infinite divisor. [#113746](https://github.com/ClickHouse/ClickHouse/pull/113746) ([Minh Vu](https://github.com/fallintoplace)). * PromQL: date/time functions `hour`, `minute`, `month`, `year`, `day_of_week`, `day_of_month`, `day_of_year`, and `days_in_month` can now be called without arguments, defaulting to `vector(time())` as in Prometheus. [#111869](https://github.com/ClickHouse/ClickHouse/pull/111869) ([Valery Petrov](https://github.com/valerypetrov)). * PromQL now supports the Prometheus HTTP API `lookback_delta` parameter for instant and range queries. [#113971](https://github.com/ClickHouse/ClickHouse/pull/113971) ([Minh Vu](https://github.com/fallintoplace)). * Distributed query plans (`make_distributed_plan`) now support `IN (subquery)` without rewriting it to a `JOIN`: the set is built once on the initiator and its values are shipped with the worker tasks. The forced `rewrite_in_to_join` override is removed; the rewrite remains available as an explicit setting. [#113826](https://github.com/ClickHouse/ClickHouse/pull/113826) ([Alexander Gololobov](https://github.com/davenger)). * Streaming queries support watermarks. [#106169](https://github.com/ClickHouse/ClickHouse/pull/106169) ([Mikhail Artemenko](https://github.com/Michicosun)). * Added a read-through filesystem cache to the experimental `ReaderExecutor` read path (`use_reader_executor`, disabled by default). [#110029](https://github.com/ClickHouse/ClickHouse/pull/110029) ([Sema Checherinda](https://github.com/CheSema)). * PromQL: implemented functions `clamp`, `clamp_min`, `clamp_max`, and `round`. [#111870](https://github.com/ClickHouse/ClickHouse/pull/111870) ([Valery Petrov](https://github.com/valerypetrov)). * Correctly collect insert stats and queries through the Prometheus remote-write endpoint. [#112825](https://github.com/ClickHouse/ClickHouse/pull/112825) ([James Cunningham](https://github.com/JTCunning)). * PromQL aggregation operators now accept mixed letter casing, matching Prometheus. [#114076](https://github.com/ClickHouse/ClickHouse/pull/114076) ([Minh Vu](https://github.com/fallintoplace)). * TimeSeries: store all tags in the `tags` column. [#114300](https://github.com/ClickHouse/ClickHouse/pull/114300) ([Vitaly Baranov](https://github.com/vitlibar)). * Apply the read-in-order optimization to `ORDER BY` queries when `parallel_replicas_plan_based` is enabled: the sort is now shipped to the replicas and merged on the initiator, so each replica reads in the sorting key order. Also fixes `Unknown function __topKFilter` for `ORDER BY` on a non-primary-key column with this setting enabled. [#114315](https://github.com/ClickHouse/ClickHouse/pull/114315) ([Igor Nikonov](https://github.com/devcrafter)). * Reject literal line feeds in ordinary quoted PromQL strings. [#114381](https://github.com/ClickHouse/ClickHouse/pull/114381) ([Minh Vu](https://github.com/fallintoplace)). * Fix duplicated `GROUP BY` keys in the partial aggregation strategy of `make_distributed_plan`, and allow this strategy for aggregations over a distributed read. [#114523](https://github.com/ClickHouse/ClickHouse/pull/114523) ([Alexander Gololobov](https://github.com/davenger)). * Throw exception on violating the allowlist of hypothetical index types for WHATIF. [#114646](https://github.com/ClickHouse/ClickHouse/pull/114646) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Restores quantized sub-column read for vector search (introduced in 26.7). [#114763](https://github.com/ClickHouse/ClickHouse/pull/114763) ([Shankar Iyer](https://github.com/shankar-iyer)). * The experimental `TimeSeries` table engine no longer applies the `Gorilla` codec to auto-created `value` columns of the samples inner table; they now default to `CODEC(ZSTD(1))`. Explicitly declared columns keep whatever codecs the user wrote. The `timestamp` column default is unchanged (`CODEC(DoubleDelta, ZSTD(1))`). [#114790](https://github.com/ClickHouse/ClickHouse/pull/114790) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * A distributed plan query (`make_distributed_plan`) now stops its upstream stages when a `LIMIT` is satisfied, instead of computing the full result and discarding it. [#114806](https://github.com/ClickHouse/ClickHouse/pull/114806) ([Alexander Gololobov](https://github.com/davenger)). * Added a userspace page cache tier to the experimental `ReaderExecutor` read path (`use_reader_executor`, off by default). On a miss the executor now populates the userspace page cache and serves subsequent reads from it, zero-copy. [#114890](https://github.com/ClickHouse/ClickHouse/pull/114890) ([Sema Checherinda](https://github.com/CheSema)). * Support `ROLLUP`, `CUBE`, `GROUPING SETS` and the `grouping` function in queries executed with `make_distributed_plan = 1`. [#114950](https://github.com/ClickHouse/ClickHouse/pull/114950) ([Alexander Gololobov](https://github.com/davenger)). * Fix the error `Illegal type Decimal(18, 0) of argument of function toIntervalNanosecond` when a PromQL query uses `offset` inside a range selector, e.g. `rate(m[2m] offset 5m)`. [#114997](https://github.com/ClickHouse/ClickHouse/pull/114997) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * A very large `distributed_plan_workers_num` no longer makes the server abort with a `std::length_error` when the experimental `make_distributed_plan` and `enable_cascades_optimizer` are enabled. Such a node count is now rejected with `INVALID_SETTING_VALUE`. [#115398](https://github.com/ClickHouse/ClickHouse/pull/115398) ([Groene AI](https://github.com/groeneai)). * Support `GROUPING SETS` aggregation in distributed plans built by the Cascades optimizer (`make_distributed_plan = 1, enable_cascades_optimizer = 1`). [#115426](https://github.com/ClickHouse/ClickHouse/pull/115426) ([Alexander Gololobov](https://github.com/davenger)). * `TimeSeries` tables can now keep a "recent samples" table: when the new engine setting `recent_samples_ttl_seconds` is non-zero, the engine creates a TTL'd copy of the samples table (partitioned by `recent_samples_partition_by`, by day by default) written on every insert, and PromQL queries whose time range fits in the TTL window automatically read from it, making alert/recording rules and short-range dashboards several times faster on large tables. The preference can be disabled with the new query-level setting `time_series_prefer_recent_samples_table`. [#115441](https://github.com/ClickHouse/ClickHouse/pull/115441) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Implement SELECT query from TimeSeries. [#115622](https://github.com/ClickHouse/ClickHouse/pull/115622) ([Vitaly Baranov](https://github.com/vitlibar)). * Implement the `/api/v1/series` endpoint of the Prometheus HTTP API for `TimeSeries` tables: `match[]` series selectors (repeated values are a union), an optional `start`/`end` time range, and an optional `limit` parameter. [#115676](https://github.com/ClickHouse/ClickHouse/pull/115676) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Distributed query plans (`make_distributed_plan`) now stop idle upstream stages promptly after a satisfied `LIMIT`: an idle exchange sink reads `NoMoreDataNeeded` as soon as it arrives, and an idle exchange source notices its closed output and forwards the stop upstream. Previously such stages kept reading and computing data that nobody consumes. [#115690](https://github.com/ClickHouse/ClickHouse/pull/115690) ([Alexander Gololobov](https://github.com/davenger)). * Automatic parallel replicas can now collect runtime statistics and enable parallel replicas for supported queries when `parallel_replicas_plan_based` is enabled. [#115788](https://github.com/ClickHouse/ClickHouse/pull/115788) ([Igor Nikonov](https://github.com/devcrafter)). * Register an external recent samples table of a TimeSeries table as a referential dependency. [#115944](https://github.com/ClickHouse/ClickHouse/pull/115944) ([Groene AI](https://github.com/groeneai)). #### Performance Improvement * New adaptive algorithm for parallel `GROUP BY` (controlled via setting `enable_adaptive_aggregator`, enabled by default): each thread aggregates into its own hash table until it holds `adaptive_aggregator_freeze_threshold` keys and then freezes it, so frequent keys keep updating the small cache-resident tables with no coordination, while rare keys are routed by their hash into per-bucket backlogs and aggregated exactly once, inside the bucket-parallel merge. [#111459](https://github.com/ClickHouse/ClickHouse/pull/111459) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Reduced memory usage of `MergeTree` tables with many data parts by sharing the schema-derived per-part metadata (column list, column descriptions, serializations, and columns substreams) across parts of a table instead of copying it into every part. The effect is largest for wide tables with many small parts. [#109816](https://github.com/ClickHouse/ClickHouse/pull/109816) ([Raúl Marín](https://github.com/Algunenano)). * Optimize `GROUP BY ... ORDER BY ... LIMIT` and `GROUP BY ... LIMIT` queries by maintaining a bounded heap during aggregation to prune groups that cannot appear in the result, significantly reducing memory usage and execution time for high-cardinality grouping. Controlled by the `enable_group_by_top_k_optimization` setting. Based on the initial implementation by [Dmitriy Terenichev](https://github.com/Dmitry909) ([#78553](https://github.com/ClickHouse/ClickHouse/pull/78553)). [#96630](https://github.com/ClickHouse/ClickHouse/pull/96630) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Reduce hash-table cell size for single-`String` aggregation keys, improving string-heavy `GROUP BY` workloads. The new `enable_packed_string_keys_in_aggregation` setting is enabled by default; disabling it, or setting `compatibility` below `26.8`, restores the legacy `StringHashTable` method, which can still be faster for very low-cardinality aggregation with keys longer than 11 bytes. [#110573](https://github.com/ClickHouse/ClickHouse/pull/110573) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). [#112089](https://github.com/ClickHouse/ClickHouse/pull/112089) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * Parallelize the final merge of single-level aggregation hash tables with the new `enable_parallel_single_level_merge` setting, enabled by default. This speeds up `GROUP BY` queries whose per-thread hash tables stay below the two-level threshold but whose merge dominates the runtime. [#110395](https://github.com/ClickHouse/ClickHouse/pull/110395) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Improve performance of `DISTINCT` when the input stream is sorted by a prefix of the distinct columns, for example `SELECT DISTINCT a, b FROM table ORDER BY a`. Such queries run up to 2.4 times faster. [#110170](https://github.com/ClickHouse/ClickHouse/pull/110170) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up and reduce the memory usage of `DISTINCT` queries on partitioned `MergeTree` tables by keeping each partition's rows within a single stream, so each stream's preliminary `DISTINCT` works on a disjoint set of keys, instead of the same set of keys appearing in multiple preliminary `DISTINCT`s. This applies when the partition expression is a deterministic function of the `DISTINCT` columns, and is gated by a cost heuristic on partition count and skew (which can be toggled by setting `force_distinct_partitions_independently`, disabled by default). The same per-partition disjointness is propagated through `Expression`, `Filter` and `ARRAY JOIN` steps to the final `DISTINCT`, `LIMIT BY` and `GROUP BY` consumers so they can skip their merge too. Controlled by the new settings `allow_distinct_partitions_independently` (enabled by default) and `max_number_of_partitions_for_independent_distinct` to tune the heuristic. [#108326](https://github.com/ClickHouse/ClickHouse/pull/108326) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Optimized functions `uniq`, `uniqExact`, and `uniqHLL12` in aggregation without key. [#110150](https://github.com/ClickHouse/ClickHouse/pull/110150) ([Anton Popov](https://github.com/CurtizJ)). * The `Parquet` V3 reader can now skip whole row groups using the dictionary page, in addition to min/max statistics and bloom filters, for equality and `IN` conditions when a column chunk is fully dictionary-encoded. Controlled by the new `input_format_parquet_dictionary_filter_push_down` setting. [#106952](https://github.com/ClickHouse/ClickHouse/pull/106952) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#110567](https://github.com/ClickHouse/ClickHouse/pull/110567) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Lazy materialization for reading `Parquet` files from object storage, including `Iceberg` tables: for `ORDER BY ... LIMIT n` queries, the columns that are not needed for sorting and filtering are read only for the `n` rows that survive the `LIMIT`. On a 200 MB `Parquet` file on `S3`, `SELECT k, s ... ORDER BY k DESC LIMIT 10` reads 3.3 MB instead of 171 MB (51x less I/O, 8x faster). Controlled by the new setting `query_plan_optimize_lazy_materialization_for_object_storage` (enabled by default, and also requires `query_plan_optimize_lazy_materialization`). [#110970](https://github.com/ClickHouse/ClickHouse/pull/110970) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Lazy materialization for `ORDER BY ... LIMIT n` queries ([#110970](https://github.com/ClickHouse/ClickHouse/issues/110970)) now also applies to local Parquet files read with the `file` table function and the `File` table engine: the columns that are not needed for sorting and filtering are read only for the `n` rows that survive the `LIMIT`. The second read of a surviving file fails close with the new `FILE_CHANGED_DURING_READ` error if the file was modified between the two passes. Controlled by the new setting `query_plan_optimize_lazy_materialization_for_file` (enabled by default). Also fixes lazy materialization for object storage failing with `Not found column or subcolumn ... in block` when a requested subcolumn (e.g. of a `JSON` column) is deferred. [#114262](https://github.com/ClickHouse/ClickHouse/pull/114262) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Column statistics are now materialized on `INSERT` by default when the table's current active size plus the written block size is at most the new `materialize_statistics_on_insert_max_table_size` setting (default 25 GiB); the check is per written block, so a first bulk load into an empty table may still materialize statistics for each written block. This gives the cost-based join optimizer accurate estimates for freshly-loaded dimension tables and avoids pathological join orders (for example, TPC-H Q5 and Q8 no longer time out at scale factor 40), while large established fact tables keep materializing statistics during merges. [#109454](https://github.com/ClickHouse/ClickHouse/pull/109454) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up window functions on partitioned `MergeTree` tables by reading each partition through its own stream and evaluating the windows per partition, skipping the hash scatter that ordinarily reshuffles every row across threads before the window sort. This applies when the table partition expression is a deterministic function of the window `PARTITION BY` columns and is gated by the same cost heuristic as per-partition `GROUP BY` / `DISTINCT` (enough partitions relative to `max_threads`, no dominant partition). Controlled by the new settings `allow_window_partitions_independently` (enabled by default), `force_window_partitions_independently`, and `max_number_of_partitions_for_independent_window`. [#114783](https://github.com/ClickHouse/ClickHouse/pull/114783) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Reduce memory usage when writing packed parts by streaming the final archive directly into the destination file instead of holding a second full copy in memory during part commit. [#111995](https://github.com/ClickHouse/ClickHouse/pull/111995) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new `join_algorithm` value `parallel_full_sorting_merge`: for hash-compatible equality joins, it shards a full sorting merge join by the hash of the join keys into independent per-shard merge joins running on all threads. It keeps the low, streaming memory usage of a merge join while parallelizing it (in a benchmark, ~2.4x faster and ~3.3x less memory than `parallel_hash`). `ASOF` joins fall back to a single `full_sorting_merge`; hash-incompatible key types (floating-point, `JSON`, `Object`, `Dynamic`) skip only the hash-scatter rewrite and can still be sharded at the source by primary-key ranges when `query_plan_join_shard_by_pk_ranges` is enabled. The result is not ordered. [#109005](https://github.com/ClickHouse/ClickHouse/pull/109005) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add GeoParquet spatial pruning at row-group and page levels to the `Parquet` reader. Row-group pruning skips entire row groups whose bounding box does not overlap the query geometry. Page-level pruning uses the `covering.bbox` column index to skip irrelevant pages within row groups, and spatial predicate pushdown now also applies during `Parquet` row reading. [#104435](https://github.com/ClickHouse/ClickHouse/pull/104435) ([Vasily Chekalkin](https://github.com/bacek)). * Parquet filter push-down (row group pruning and column index) supports more type conversions: `DateTime64(x) <-> DateTime64(y)`, `Decimal(x) <-> Decimal(y)`, `Float32 <-> Float64`. [#110457](https://github.com/ClickHouse/ClickHouse/pull/110457) ([Michael Kolupaev](https://github.com/al13n321)). * Reduce duplicated read requests on remote disks when mark ranges are fragmented, for example by primary or skip indexes. [#113584](https://github.com/ClickHouse/ClickHouse/pull/113584) ([Anton Popov](https://github.com/CurtizJ)). * Speed up execution of `IN` over clustered (e.g. sorted by primary key) columns by reusing the previous row's result for equal consecutive rows. [#114853](https://github.com/ClickHouse/ClickHouse/pull/114853) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * An aggregation without `GROUP BY` keys no longer fans its single-row output out to `max_threads` streams, so queries whose plan continues after such an aggregation get a much smaller pipeline (fewer processors to create, schedule and profile) without losing any parallelism. [#115170](https://github.com/ClickHouse/ClickHouse/pull/115170) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Keeper now reads changelog entries with less lock contention: catch-up reads plan under a shared lock and do disk I/O lock-free, and both follower catch-up and the commit path can pre-decode entries in a background thread pool, reducing redundant disk reads and CPU decoding on the leader. [#108473](https://github.com/ClickHouse/ClickHouse/pull/108473) ([Antonio Andelic](https://github.com/antonio2368)). * Speed up Keeper startup by reading multiple changelog files concurrently instead of serially, controlled by new settings `log_startup_read_max_streams` and `log_startup_read_buffer_size`. [#109881](https://github.com/ClickHouse/ClickHouse/pull/109881) ([Antonio Andelic](https://github.com/antonio2368)). * Add a new `bucketed` schema type for `system.metric_log`, which stores all metrics in a single `Map(Enum16(...), Int64)` column using the bucketed `Map` serialization with 128 buckets, plus a per-metric `ALIAS` column for compatibility: the table consists of a few columns instead of thousands, zero values are not stored, and reading a single metric reads only one of the 128 buckets. In addition, a `Map` with `Enum` keys can now be indexed by the string name of the enum value, e.g. `map['name']`. [#115380](https://github.com/ClickHouse/ClickHouse/pull/115380) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a query plan optimization that pushes volume-reducing functions (`length`, `lengthUTF8`, `empty`, `notEmpty`) below the `Sorting` and `Filter` steps, replacing the wide `String` / `FixedString` argument with the fixed-size result, so it is neither buffered by a sort nor copied by a filter. Controlled by the new setting `query_plan_push_down_volume_reducing_functions`, enabled by default. [#106199](https://github.com/ClickHouse/ClickHouse/pull/106199) ([Peng](https://github.com/fastio)). * Enable `read_in_order_use_virtual_row` by default. When reading in order of the primary key (e.g. `ORDER BY primary_key LIMIT n`) over a table with many parts, only the parts that can actually contribute to the result are read (plus a read-ahead window of at most `max_threads` parts that keeps reading parallel), which significantly reduces peak memory consumption. [#106215](https://github.com/ClickHouse/ClickHouse/pull/106215) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Aggregation queries without aggregate functions now use `HashSet`-based methods instead of `HashMap` (for supported key types). Speedups up to 1.8x times were observed. [#108862](https://github.com/ClickHouse/ClickHouse/pull/108862) ([Nikita Taranov](https://github.com/nickitat)). * Avoid `JOIN` runtime-filter overhead when the probe side is estimated to be small. The threshold is controlled by the new `join_runtime_filter_min_probe_rows` setting (default `1000`). [#104860](https://github.com/ClickHouse/ClickHouse/pull/104860) ([Vladimir Cherkasov](https://github.com/vdimir)). * Queries with predicates like `nullIf(key, sentinel) = const` can now use primary-key, partition, and skip-index pruning. [#107308](https://github.com/ClickHouse/ClickHouse/pull/107308) ([Aditya Kumar](https://github.com/adityaksolves)). * Added the `jemalloc_merge_tree_arenas` server setting to control dedicated jemalloc arena(s) for `MergeTree` part and table metadata. On many-core servers, a per-CPU pool can reduce allocator lock contention between concurrent merges, and the dedicated arena now only holds long-lived metadata. [#109490](https://github.com/ClickHouse/ClickHouse/pull/109490) ([Raúl Marín](https://github.com/Algunenano)). * Sped up ZSTD decompression on `x86_64` for columns with small match offsets, such as fixed-width integer columns, by vectorizing short-offset overlapping copies with SSSE3 `pshufb`. For example, decompression of a `UInt64` column is up to ~1.2x faster on AMD Zen 5. [#110909](https://github.com/ClickHouse/ClickHouse/pull/110909) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Speed up `groupUniqArray` for numeric types by up to 3x by using CRC32 hashing, as in `uniqExact`, and by inlining per-row insertion. [#110917](https://github.com/ClickHouse/ClickHouse/pull/110917) ([Manuel](https://github.com/raimannma)). * Faster `estimateCompressionRatio` with the `NONE` codec. [#111107](https://github.com/ClickHouse/ClickHouse/pull/111107) ([Raufs Dunamalijevs](https://github.com/rienath)). * Lowered the default `max_snapshot_commit_thread_pool_size` to `16`, matching `backup_threads`, to reduce CFS throttling when committing snapshots for tables with many parts. [#111417](https://github.com/ClickHouse/ClickHouse/pull/111417) ([Han Fei](https://github.com/hanfei1991)). * Reduce CPU usage when committing snapshots for tables with huge numbers of files. [#112297](https://github.com/ClickHouse/ClickHouse/pull/112297) ([Han Fei](https://github.com/hanfei1991)). * Skip async-insert deduplication hash prewarming when deduplication is disabled. [#111549](https://github.com/ClickHouse/ClickHouse/pull/111549) ([Sema Checherinda](https://github.com/CheSema)). * A `JOIN` whose `ON` condition is always false, for example `ON 1 = 2`, `ON NULL`, or `a.t = 'A' AND a.t = 'B'`, no longer reads the non-contributing side. Controlled by the `query_plan_short_circuit_constant_false_join` setting (enabled by default). [#110234](https://github.com/ClickHouse/ClickHouse/pull/110234) ([Groene AI](https://github.com/groeneai)). * Speed up reading of `Iceberg` tables by prefetching the next manifest file while the current manifest is being parsed, overlapping I/O with CPU work. [#108543](https://github.com/ClickHouse/ClickHouse/pull/108543) ([Asya Shneerson](https://github.com/asya-ch)). * Queries with `FINAL` that read in primary-key order with a small `LIMIT` no longer use `split_intersecting_parts_ranges_into_layers_final`. [#110431](https://github.com/ClickHouse/ClickHouse/pull/110431) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Improve performance of `RIGHT` and `FULL JOIN` with the `parallel_hash` algorithm when the right side contains `NULL` join keys or rows filtered by the `ON` expression, by avoiding null-map rebuilding. [#111258](https://github.com/ClickHouse/ClickHouse/pull/111258) ([Hechem Selmi](https://github.com/m-selmi)). * Speed up `countSubstringsCaseInsensitiveUTF8` by fixing quadratic complexity that could make the function take minutes on a haystack of a few megabytes with many matches. [#112003](https://github.com/ClickHouse/ClickHouse/pull/112003) ([Groene AI](https://github.com/groeneai)). * The Prometheus query API (`/api/v1/query`, `/api/v1/query_range`) of the experimental `TimeSeries` engine now executes queries in parallel instead of on a single thread. [#112108](https://github.com/ClickHouse/ClickHouse/pull/112108) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Auto-created `timestamp` and `value` columns of the `TimeSeries` samples inner table now default to `CODEC(DoubleDelta, ZSTD(1))` and `CODEC(Gorilla, ZSTD(1))` respectively. [#112110](https://github.com/ClickHouse/ClickHouse/pull/112110) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Fix a performance regression in `system.query_log` ingestion. Building the `Settings` and `asynchronous_read_counters` map columns was about six times slower than before, which could make `SYSTEM FLUSH LOGS query_log` exceed its 180-second timeout on servers that log many queries with many changed settings. Closes [#112191](https://github.com/ClickHouse/ClickHouse/issues/112191). [#112210](https://github.com/ClickHouse/ClickHouse/pull/112210) ([Groene AI](https://github.com/groeneai)). * Improve performance of RIGHT and FULL hash JOIN with multiple `OR` disjuncts in the ON section, and of RIGHT/FULL JOIN with residual (inequality) conditions. [#111590](https://github.com/ClickHouse/ClickHouse/pull/111590) ([Hechem Selmi](https://github.com/m-selmi)). * Reduce memory usage of `UNLOCK SNAPSHOT` by not loading per-data-part metadata that the unlock path does not use, avoiding out-of-memory for snapshots with a very large number of parts. [#111849](https://github.com/ClickHouse/ClickHouse/pull/111849) ([Julia Kartseva](https://github.com/jkartseva)). * Restore memory pre-allocation for dynamic paths of a `JSON` column during asynchronous `INSERT` flushes, avoiding repeated reallocations while batched rows are appended. [#112222](https://github.com/ClickHouse/ClickHouse/pull/112222) ([Groene AI](https://github.com/groeneai)). * Re-enable the consecutive-keys optimization for `TTL ... GROUP BY` aggregation during merges. It had been accidentally disabled on that path, so results stayed correct but merges did more work than necessary. [#112314](https://github.com/ClickHouse/ClickHouse/pull/112314) ([Groene AI](https://github.com/groeneai)). * Speed up reading `Nested` columns from binary-encoded types such as `Native` and `RowBinary` when `input_format_binary_decode_types_in_binary_format` is enabled, and report malformed nested `Tuple` type names as a syntax error instead of spending exponential time parsing them. [#112560](https://github.com/ClickHouse/ClickHouse/pull/112560) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved performance of squashing and merging `Dynamic` and `Object` columns with many dynamic variants or JSON paths. [#110583](https://github.com/ClickHouse/ClickHouse/pull/110583) ([Christoph Viebig](https://github.com/cv4g)). * Speed up `min` and `max` on 128-bit and 256-bit types (`Int128`, `UInt128`, `Int256`, `UInt256`, `Decimal128`, `Decimal256`). Up to 1.8x faster for wide integers and up to 3.4x faster for wide decimals. [#111965](https://github.com/ClickHouse/ClickHouse/pull/111965) ([Manuel](https://github.com/raimannma)). * Enabled by default (and promoted to beta) two on-disk features: 1) `packed_skip_index_max_bytes = '1M'`: skip-index substreams up to 1 MiB are now written into a single `skp_idx.packed` archive per part, reducing object count and write requests on object storage. Compatibility: only newly written parts are affected. Versions before 26.6 do not use the packed indices for pruning (they are treated as not materialized), and 26.7 and newer read them. 26.6 cannot read these parts: after a rollback to 26.6, `count()`, queries with a `WHERE` clause, and merges fail with `UNKNOWN_FORMAT_VERSION` on tables that have such parts. Set `packed_skip_index_max_bytes = 0` to keep the standalone per-file layout. To keep a rollback to 26.6 possible, make sure the setting is 0 for every table before upgrading, for example with `packed_skip_index_max_bytes = 0` in the `merge_tree` section of the server configuration or a `compatibility` below 26.8 in the default profile. The value comes from the last of these that sets it: `compatibility`, the `merge_tree` section, the `replicated_merge_tree` section (for `Replicated` tables), and the table's own `SETTINGS`. 2) `compute_exact_num_defaults_for_sparse_columns` plus `optimize_trivial_count_with_sparsity_filter`: exact per-column `num_defaults` counters are persisted and used to answer `SELECT count() FROM t WHERE ` without a data scan when the predicate splits rows into defaults vs non-defaults. Compatibility: the `exact_num_defaults` flag in `serialization.json` is ignored by older versions, so parts remain fully readable after a downgrade; the count rewrite only applies to parts that carry exact counters, so results stay correct on tables with mixed parts. [#111816](https://github.com/ClickHouse/ClickHouse/pull/111816) ([Raúl Marín](https://github.com/Algunenano)). * Add optional caching for tokens missing from text indexes. [#112742](https://github.com/ClickHouse/ClickHouse/pull/112742) ([Rory Shanks](https://github.com/rorylshanks)). * Fixed quadratic reallocation when replicating an `Array` whose elements hold `JSON` values in shared data, for example a constant `Array(JSON)` passed to an aggregate function. Such queries copied a number of bytes that grew quadratically with the row count. [#112897](https://github.com/ClickHouse/ClickHouse/pull/112897) ([Groene AI](https://github.com/groeneai)). * Use the primary key index for `pointInPolygon` when the point argument is a whole key column of type `Point` (or another `Tuple` of two numeric elements), e.g. `pointInPolygon(coord, [...])` for a table ordered by `coord`. Previously only the `pointInPolygon((x, y), [...])` form with two scalar key columns was analyzed. Closes [#54805](https://github.com/ClickHouse/ClickHouse/issues/54805). [#112956](https://github.com/ClickHouse/ClickHouse/pull/112956) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Skip the data-reading step entirely for `TTLDrop` merges to reduce memory usage, especially for read and prefetch buffers. Closes [#105639](https://github.com/ClickHouse/ClickHouse/issues/105639). [#105859](https://github.com/ClickHouse/ClickHouse/pull/105859) ([Pavel Kruglov](https://github.com/Avogar)). * Reduce memory allocations when reading backup metadata by moving file entries into in-memory maps instead of copying them. [#111718](https://github.com/ClickHouse/ClickHouse/pull/111718) ([Julia Kartseva](https://github.com/jkartseva)). * Use the text index when its expression contains an empty-string comparison such as `arrayFilter(s -> s != '', ...)`, even when `optimize_empty_string_comparisons = 1` rewrites it to `notEmpty`. Closes [#111788](https://github.com/ClickHouse/ClickHouse/issues/111788). [#112115](https://github.com/ClickHouse/ClickHouse/pull/112115) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Restore query condition cache pruning after a `lightweight DELETE`: repeated selective queries over a table that had ever been touched by a `lightweight DELETE` no longer fall back to reading every mark. [#112947](https://github.com/ClickHouse/ClickHouse/pull/112947) ([Nikita Fomichev](https://github.com/fm4v)). * Higher-order functions no longer physically copy captured columns for every array element when `enable_lazy_columns_replication` is enabled, reducing memory usage and copying time for lambdas that capture large columns. [#111581](https://github.com/ClickHouse/ClickHouse/pull/111581) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Fixed a ~1.5x performance regression introduced in 26.5 in `GROUP BY` over `Dynamic` keys and `JSON` path subcolumns. [#112316](https://github.com/ClickHouse/ClickHouse/pull/112316) ([Utkal Singh](https://github.com/Utkal059)). * `text_index_posting_list_apply_mode` now defaults to `lazy`, so text index queries decode posting lists on demand at packed-block granularity with a cursor instead of eagerly materializing them into bitmaps for text indexes with `posting_list_codec` set. [#113521](https://github.com/ClickHouse/ClickHouse/pull/113521) ([Anton Popov](https://github.com/CurtizJ)). * Speed up PromQL queries over `TimeSeries` tables: `timeSeriesIdToGroup` now reuses the previous row's group when consecutive rows carry the same series id, skipping the per-row id serialization and hash lookup. [#113580](https://github.com/ClickHouse/ClickHouse/pull/113580) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * PromQL aggregation operators `topk`, `bottomk`, and `limitk` now use a streaming, parallel execution plan with bounded memory instead of collecting all series into a single row. Queries over high-cardinality metrics that previously failed with `MEMORY_LIMIT_EXCEEDED` now run in bounded memory. [#113656](https://github.com/ClickHouse/ClickHouse/pull/113656) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). [#114409](https://github.com/ClickHouse/ClickHouse/pull/114409) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up division of 256-bit integers (`Int256`, `UInt256`, and `Decimal256`), affecting `intDiv`, `modulo`, `Decimal256` arithmetic, and conversion of these types to text. [#112477](https://github.com/ClickHouse/ClickHouse/pull/112477) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Improve `MergeTree` read performance for `JSON` subcolumns by reusing resolved subcolumn metadata during required conversions. [#113008](https://github.com/ClickHouse/ClickHouse/pull/113008) ([Rory Shanks](https://github.com/rorylshanks)). * Speed up building text and bloom-filter-based skip indexes with the `sparseGrams` tokenizer. [#110541](https://github.com/ClickHouse/ClickHouse/pull/110541) ([UnamedRus](https://github.com/UnamedRus)). * Snapshot creation now uses a dedicated thread pool, so a heavy concurrent `BACKUP` no longer starves it. [#113509](https://github.com/ClickHouse/ClickHouse/pull/113509) ([Han Fei](https://github.com/hanfei1991)). * Evaluate PromQL subexpressions shared by multiple plan steps (the `topk`/`bottomk`/`limitk` operand, the left side of `or`) once instead of twice: the SQL generated from PromQL now materializes shared subqueries. This removes a duplicated scan of the samples table; `topk` over `rate` gets ~2x faster cold, and kubernetes-mixin rule queries improve by 10-29%. [#113772](https://github.com/ClickHouse/ClickHouse/pull/113772) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * You can now enable early short-circuit folding for builtin `and` and `or` during analysis with the `enable_function_early_short_circuit` setting. This can skip executing dead single-row `count()` scalar subqueries while preserving normal type inference and semantic validation. Resolves [#83017](https://github.com/ClickHouse/ClickHouse/issues/83017). [#83505](https://github.com/ClickHouse/ClickHouse/pull/83505) ([fhw12345](https://github.com/fhw12345)). * Combine I/O cost with selectivity in `PREWHERE` condition ordering. This fixes a performance regression of `PREWHERE` execution in some cases introduced after [#101275](https://github.com/ClickHouse/ClickHouse/pull/101275). Part of [#110462](https://github.com/ClickHouse/ClickHouse/issues/110462). [#110695](https://github.com/ClickHouse/ClickHouse/pull/110695) ([Pavel Kruglov](https://github.com/Avogar)). * Improved the performance of `levenshteinDistance` (and `editDistance`) for medium-length strings on ARM by avoiding per-row heap allocations in the blocked Myers code path. [#108185](https://github.com/ClickHouse/ClickHouse/pull/108185) ([Groene AI](https://github.com/groeneai)). * Add the `merge_use_batch_sorting_queue` `MergeTree` setting to optionally use the batch sorting queue for ordinary `MergeTree` merges, reducing CPU and wall-clock time for merge workloads where sorted merging is a significant cost. [#108468](https://github.com/ClickHouse/ClickHouse/pull/108468) ([Rory Shanks](https://github.com/rorylshanks)). * Added the `merge_tree_min_bytes_per_read_stream` setting to reduce excessive read streams and pipeline overhead for ordinary local unordered narrow-column `MergeTree` scans on high core-count servers. [#109035](https://github.com/ClickHouse/ClickHouse/pull/109035) ([Jiebin Sun](https://github.com/jiebinn)). * Speed up function `toStartOfInterval` (aliases `time_bucket`, `date_bin`) by up to 5x for `SECOND`, `MINUTE` and `HOUR` intervals. [#109729](https://github.com/ClickHouse/ClickHouse/pull/109729) ([Manuel](https://github.com/raimannma)). * The native protocol now sends `String` columns as a separate stream of cumulative byte offsets (the same layout `Array` uses for its offsets) followed by the concatenated data, instead of a per-value varint length prefix, once both peers are on protocol revision 54489 or newer. A client can then read `String` columns with exact buffer preallocation and a single bulk copy, about 4 times faster than the per-value layout. Old clients and servers are unaffected: the layout is negotiated through the protocol revision. The same layout is available in the `Native` and `Buffers` formats through the new settings `output_format_native_write_string_with_size_stream` / `input_format_native_read_string_with_size_stream` (off by default, so the formats stay portable). [#110320](https://github.com/ClickHouse/ClickHouse/pull/110320) ([vahid-sohrabloo](https://github.com/vahid-sohrabloo)). * Avoid eager JSON serialization of Iceberg manifest entries when iceberg_metadata_log_level is below the call site's threshold. [#110437](https://github.com/ClickHouse/ClickHouse/pull/110437) ([Andy Bradshaw](https://github.com/andybradshaw)). * Improve full sorting merge join performance up to 1.5-3x times on sparse scenarios. [#111200](https://github.com/ClickHouse/ClickHouse/pull/111200) ([Vladimir Cherkasov](https://github.com/vdimir)). * Improve efficiency of minmax and basic column statistics for floating point columns. [#111221](https://github.com/ClickHouse/ClickHouse/pull/111221) ([Christoph Viebig](https://github.com/cv4g)). * Enable `join_runtime_filter_from_fixed_hash_table` when the probe and build join key types differ (for example, a Nullable probe key with a non-Nullable build key). [#111244](https://github.com/ClickHouse/ClickHouse/pull/111244) ([Hechem Selmi](https://github.com/m-selmi)). * Returns `COUNT()` queries directly from the text index cardinality metadata. [#111494](https://github.com/ClickHouse/ClickHouse/pull/111494) ([Elmi Ahmadov](https://github.com/ahmadov)). * Sped up row-wise hashing of JSON/`Object` column values stored in shared data (used e.g. by `uniq`/`uniqExact` over a JSON column and by hashed `GROUP BY`/`DISTINCT` keys) by avoiding a per-value `ColumnDynamic` reconstruction. Hash values are unchanged. [#111501](https://github.com/ClickHouse/ClickHouse/pull/111501) ([Valery Petrov](https://github.com/valerypetrov)). * Functions `mapContainsKeyLike`, `mapContainsValueLike`, `mapExtractKeyLike`, and `mapExtractValueLike` with a non-constant pattern use less memory and time when enable_lazy_columns_replication is enabled. The pattern is no longer eagerly copied per map entry at capture time; the copy is deferred to the execution of the internal lambda and is freed sooner, so one fewer expanded copy of the pattern exists at peak. [#111749](https://github.com/ClickHouse/ClickHouse/pull/111749) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Speed up sorting and other comparison-based operations over `JSON` columns whose paths are stored in shared data. Previously each value comparison materialized a temporary `Dynamic` column and ran a full binary deserialization for both sides, making `ORDER BY` over such columns pathologically slow. [#111894](https://github.com/ClickHouse/ClickHouse/pull/111894) ([Groene AI](https://github.com/groeneai)). * `clickhouse-client` no longer pings the server before every query, which removes a network round trip per query. It also no longer re-establishes the connection - silently starting a new session, and so losing temporary tables, the current database and session settings - just because the server was too slow to answer a ping. [#111990](https://github.com/ClickHouse/ClickHouse/pull/111990) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use the transitive conditions optimization more wisely: the additional filter created by transitivity is now used only for index analysis where the whole filter cannot be applied. See the `optimize_and_compare_chain` setting. [#112076](https://github.com/ClickHouse/ClickHouse/pull/112076) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Added two settings, `shrink_over_allocated_columns_min_waste_ratio` and `shrink_over_allocated_columns_min_waste_bytes`, that make INSERTs shrink over-allocated columns (whose reserved memory exceeds used memory due to power-of-two growth of variable-length columns) to fit before materialization and part writing, reducing peak memory usage. Disabled by default.. [#112161](https://github.com/ClickHouse/ClickHouse/pull/112161) ([Pavel Kruglov](https://github.com/Avogar)). * Lazy column replication for `ARRAY JOIN` is no longer lost when a query plan is serialized. With `serialize_query_plan = 1`, the node executing a remote plan fragment used to ignore `enable_lazy_columns_replication` and always materialize every replicated copy, even though the setting defaults to enabled. [#112330](https://github.com/ClickHouse/ClickHouse/pull/112330) ([Groene AI](https://github.com/groeneai)). * Formatting `Decimal` values as text is much faster — up to 190 times for a `Decimal256` with a large scale, and about 1.6 times for a `Decimal64`. On processors with AVX-512 IFMA, converting 64-bit integers to text is about 1.4 times faster. Also fixed the rounding of `toDecimalString`, which dropped a carry out of the fractional part: `toDecimalString(toDecimal64('9.995', 3), 2)` returned `9.00` and now returns `10.00`. [#112457](https://github.com/ClickHouse/ClickHouse/pull/112457) ([Konstantin Bogdanov](https://github.com/thevar1able)). * A `WHERE` equality is now merged into the `JOIN` condition when its operands have different types but a common supertype, such as `Int32` and `Nullable(Int32)`. This allows a CROSS join to be converted to an INNER Join. [#112630](https://github.com/ClickHouse/ClickHouse/pull/112630) ([Hechem Selmi](https://github.com/m-selmi)). * Queries on Iceberg tables with many delete files now start faster. ClickHouse reads and decodes the delete manifest files concurrently instead of one at a time, so their storage reads overlap. The new setting `iceberg_delete_manifest_decode_concurrency` (default `4`) controls how many are decoded at the same time. [#112679](https://github.com/ClickHouse/ClickHouse/pull/112679) ([Asya Shneerson](https://github.com/asya-ch)). * Asynchronous logging now uses a bounded lock-free queue: threads writing log messages never block on a mutex or wake up the logging threads. [#112803](https://github.com/ClickHouse/ClickHouse/pull/112803) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `randomHadamardTransform` of a constant vector is now evaluated once instead of once per row. This speeds up vector search queries that rotate the query vector with `randomHadamardTransform` before comparing it against a `QBit` column. [#112921](https://github.com/ClickHouse/ClickHouse/pull/112921) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Restores set sharing between the part tasks of one mutation filtering on `IN (subquery)` over a primary key column. `enable_sharing_sets_for_mutations` is on by default, but the set built for primary key analysis stopped participating in the shared cache, so a table with N parts materialized the same set N times. [#112941](https://github.com/ClickHouse/ClickHouse/pull/112941) ([Groene AI](https://github.com/groeneai)). * Fixed cubic complexity of query planning for a JOIN with a `Merge` table over many tables with different structures. Such queries could previously spend minutes in query planning without responding to `max_execution_time` or `KILL QUERY`. [#113140](https://github.com/ClickHouse/ClickHouse/pull/113140) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The bucket-region cache of the `S3` client now also works for data lake catalogs; previously every request to them resolved the region again. [#113330](https://github.com/ClickHouse/ClickHouse/pull/113330) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed a performance regression where an aggregation without `GROUP BY` keys serialized its whole input plan subtree, including the full contents of constant-folded literals, on every execution to compute a hash-table-size cache key it can never use. A query such as `SELECT quantileMerge(arrayJoin(arrayMap(x -> state, range(5000000))))` spent about 63 ms per execution in query-plan optimization and now spends 0.15 ms. [#113333](https://github.com/ClickHouse/ClickHouse/pull/113333) ([Groene AI](https://github.com/groeneai)). * Add opt-in Parquet serialization of UInt128/UInt256/Int128/Int256 as DECIMAL to enable row group and page pruning. [#113347](https://github.com/ClickHouse/ClickHouse/pull/113347) ([Ivan Babrou](https://github.com/bobrik)). * Speed up query planning when `uniq_v2` column statistics are used. The estimated number of distinct values is now cached instead of being recomputed on every request, which query planning issues many times per query. (issue [#113038](https://github.com/ClickHouse/ClickHouse/issues/113038)). [#113357](https://github.com/ClickHouse/ClickHouse/pull/113357) ([Groene AI](https://github.com/groeneai)). * Do not prefetch unneeded substreams when reading a whole JSON path from advanced shared data. [#113380](https://github.com/ClickHouse/ClickHouse/pull/113380) ([Pavel Kruglov](https://github.com/Avogar)). * Filter pushdown now works for `Tuple` subcolumns in Parquet and ORC files. A predicate such as `WHERE tup.1 = 555555` over `file`, `s3` or `url` now prunes row groups and row index strides using the tuple element's own statistics instead of reading the whole file. [#113383](https://github.com/ClickHouse/ClickHouse/pull/113383) ([Groene AI](https://github.com/groeneai)). * Replaced the per-bucket hash map inside the `timeSeries*ToGrid` aggregate functions with a flat sorted array of samples: sample ingestion becomes an O(1) append for in-order inputs (the overwhelmingly common case) and the per-bucket copy-and-sort at finalization is gone. [#113681](https://github.com/ClickHouse/ClickHouse/pull/113681) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Improve PromQL query performance over the TimeSeries engine: the internal selector SQL now reads the samples-table primary-key columns directly (without no-op casts) and checks the timestamp range before the series-id set, which makes cold selector-heavy queries 30-44% faster. [#113768](https://github.com/ClickHouse/ClickHouse/pull/113768) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Use typed id maps in the time-series tags collector. [#113839](https://github.com/ClickHouse/ClickHouse/pull/113839) ([Vitaly Baranov](https://github.com/vitlibar)). * Skip the redundant sortedness scan when writing MergeTree parts whose sorting keys are all constant. [#113899](https://github.com/ClickHouse/ClickHouse/pull/113899) ([Perfloop Agent](https://github.com/perfloop-agent)). * PromQL selectors that match all series of one metric now filter the samples table of a `TimeSeries` table with a continuous primary-key range on `id` during index analysis instead of a large `id IN ` condition, when the id layout is a two-component tuple with the canonical id generator. Removes the dominant single-threaded index-analysis cost of selector-heavy PromQL queries: up to −45% cold latency on dashboard and rule query shapes, −11% cold geomean over the full suite on a 62-billion-sample table. [#114131](https://github.com/ClickHouse/ClickHouse/pull/114131) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Speed up vectorized TimeSeries tag transformations by replacing hash-based remapping with direct indexing when tag-group IDs occupy a compact range. [#114244](https://github.com/ClickHouse/ClickHouse/pull/114244) ([Minh Vu](https://github.com/fallintoplace)). * The Prometheus HTTP API endpoints `/api/v1/query` and `/api/v1/query_range` now evaluate a PromQL subquery shared by several plan steps once, as the `prometheusQuery` and `prometheusQueryRange` table functions already did. Previously a request made with `enable_analyzer = 0`, or by a user whose profile sets it, rescanned the shared subquery once per referencing step. [#114261](https://github.com/ClickHouse/ClickHouse/pull/114261) ([Groene AI](https://github.com/groeneai)). * Bounded the cost of estimating the selectivity of `col IN (...)` from column statistics, which could add hundreds of milliseconds to the planning of a single query. The estimator no longer runs the subquery behind `col IN (subquery)` to fill a set it only needs one selectivity number from — an unbuilt set is skipped instead. For a set larger than the new `statistics_max_set_size_for_exact_selectivity_estimation` setting (default 10000), the selectivity is now derived from the size of the set and its bounding range. [#114389](https://github.com/ClickHouse/ClickHouse/pull/114389) ([Nikita Taranov](https://github.com/nickitat)). * Parsing an HTTP request that carries many query parameters or headers with the same name (for example thousands of repeated `role=` parameters) no longer takes quadratic time. [#114410](https://github.com/ClickHouse/ClickHouse/pull/114410) ([James](https://github.com/sanjams2)). * Fixed a regression in 26.7: for a `MergeTree` table ordered by `toUnixTimestamp` (or another integer conversion) of a `DateTime64` column, a plain range filter on that column no longer used the primary key and read all granules of the matched parts. Additionally, a filter like `toInt64(ts) >= c` over a table ordered by the raw `DateTime64` column now uses the primary key. [#114413](https://github.com/ClickHouse/ClickHouse/pull/114413) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The heavy consistency check in `ColumnArray` is now performed only in debug builds. [#114469](https://github.com/ClickHouse/ClickHouse/pull/114469) ([Anton Popov](https://github.com/CurtizJ)). * Do not merge-sort a distributed gather whose sort description is all-constant. [#114626](https://github.com/ClickHouse/ClickHouse/pull/114626) ([Groene AI](https://github.com/groeneai)). * Speed up set building for `IN (subquery)` on partitioned `MergeTree` tables by keeping each partition's rows within a single stream and deduplicating each stream independently, so the single set-filling transform — previously hashing every row serially — only sees unique rows. This applies when the partition expression is a deterministic function of the subquery's output columns. The optimization is not applied when the largest partition holds more than twice the rows of the average partition; the new setting `force_creating_set_partitions_independently` (disabled by default) bypasses this check. Controlled by the new setting `allow_creating_set_partitions_independently` (enabled by default). [#114645](https://github.com/ClickHouse/ClickHouse/pull/114645) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Parallelize the single-level to two-level conversion of per-thread aggregation states before merging. This fixes a 2-3x regression of short `GROUP BY` queries with heavy aggregate states (e.g. `COUNT(DISTINCT ...)`, ClickBench Q10/Q11) on machines with very many cores, introduced in 26.7 development builds when the aggregation memory accounting was made accurate. [#114691](https://github.com/ClickHouse/ClickHouse/pull/114691) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up the `timeSeries*ToGrid` aggregate functions: per-sample bucket math now avoids Int128 division, and consecutive samples in the same bucket skip the hash-map lookup. [#114889](https://github.com/ClickHouse/ClickHouse/pull/114889) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Appending to a system log queue no longer deep-copies the whole queue when it grows. [#115030](https://github.com/ClickHouse/ClickHouse/pull/115030) ([Groene AI](https://github.com/groeneai)). * Process samples of the `timeSeries*ToGrid` aggregate functions in runs of consecutive samples falling into one grid bucket, speeding up the sample-ingestion stage of `timeSeriesRateToGrid` and related functions by 1.2-2x on sorted time series data. [#115041](https://github.com/ClickHouse/ClickHouse/pull/115041) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Use compact presence masks for PromQL set operators. [#115224](https://github.com/ClickHouse/ClickHouse/pull/115224) ([Minh Vu](https://github.com/fallintoplace)). * Faster evaluation of PromQL instant queries: the `timeSeries*ToGrid` aggregate functions use the vectorized sample-classification path for single-point grids, and `timeSeriesIdToGroup` fills its result column directly. [#115267](https://github.com/ClickHouse/ClickHouse/pull/115267) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * A `JOIN` whose `ON` section has several `OR`-ed equalities no longer materializes the right-hand key columns that the query does not select. [#115465](https://github.com/ClickHouse/ClickHouse/pull/115465) ([Nikita Taranov](https://github.com/nickitat)). * Naming a `DateTime` or `DateTime64` type no longer builds a UTC time zone lookup table that nothing reads, which makes constructing these types — and therefore starting up `clickhouse local` and `clickhouse client` — faster. [#115488](https://github.com/ClickHouse/ClickHouse/pull/115488) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Attaching the system tables no longer copies each table's metadata, including its full column description, twice just to set the table comment. [#115490](https://github.com/ClickHouse/ClickHouse/pull/115490) ([Konstantin Bogdanov](https://github.com/thevar1able)). * The client hostname is now resolved when it is first reported instead of on every `clickhouse client` and `clickhouse local` invocation. [#115491](https://github.com/ClickHouse/ClickHouse/pull/115491) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed quadratic time when the functions `h3kRing`, `h3ToChildren`, `h3PolygonToCells` and `h3PolygonToCellsWithContainment` build their result over many rows. A query over 196 000 rows spent 838 seconds copying memory in a CI stress run; it now takes about 5 seconds. [#115773](https://github.com/ClickHouse/ClickHouse/pull/115773) ([Groene AI](https://github.com/groeneai)). * Speed up duplicate-series checks for all-zero PromQL condition vectors. [#115822](https://github.com/ClickHouse/ClickHouse/pull/115822) ([Minh Vu](https://github.com/fallintoplace)). * Speeds up `GROUP BY` without aggregate functions over `String`, `FixedString` and `LowCardinality` keys: the query no longer rewrites an unused placeholder into the hash table for every row, only for the rows that add a new key. [#115834](https://github.com/ClickHouse/ClickHouse/pull/115834) ([Nikita Taranov](https://github.com/nickitat)). * Speed up `toUTCTimestamp`/`to_utc_timestamp` and `fromUTCTimestamp`/`from_utc_timestamp` on `DateTime64` arguments, restoring the throughput they had before the overflow fix in [#109738](https://github.com/ClickHouse/ClickHouse/issues/109738). Results are unchanged. Closes [#115802](https://github.com/ClickHouse/ClickHouse/issues/115802). [#115838](https://github.com/ClickHouse/ClickHouse/pull/115838) ([Groene AI](https://github.com/groeneai)). * Reduced memory usage and query-plan optimization time for keyed aggregations over large constant-folded literals. The aggregation hash-table-statistics cache key is now hashed as it is serialized instead of being accumulated in memory first, so a plan carrying a large literal no longer holds a copy of it during optimization. For an 80 MB literal, peak memory drops by about 250 MB and optimization time by about 2.5x. The computed key is unchanged. [#115847](https://github.com/ClickHouse/ClickHouse/pull/115847) ([Groene AI](https://github.com/groeneai)). #### Improvement * Outgoing OpenAI-format HTTP requests are now tagged with the SQL name of the calling AI function for upstream endpoint identification. [#110449](https://github.com/ClickHouse/ClickHouse/pull/110449) ([George Larionov](https://github.com/george-larionov)). * Reduce cancellation latency for positive forms of `LIMIT BY`: `KILL QUERY` and Ctrl+C now interrupt within a single chunk instead of waiting for the whole step to finish. [#106070](https://github.com/ClickHouse/ClickHouse/pull/106070) ([Roman Vasin](https://github.com/rvasin)). * When an `INSERT` fails because a materialized view's `TO` target table rejects the write while the insert pipeline is being built, the error message now names the materialized view and its target table instead of only reporting the bare storage error. [#107234](https://github.com/ClickHouse/ClickHouse/pull/107234) ([Groene AI](https://github.com/groeneai)). * With `input_format_protobuf_oneof_presence`, ClickHouse no longer requires the `oneof` presence enum to list exactly the tags declared in the Protobuf schema: extra tags without matching columns in the target table are now allowed, which makes schema changes easier. [#109174](https://github.com/ClickHouse/ClickHouse/pull/109174) ([Ilya Golshtein](https://github.com/ilejn)). * The transposed distance functions over `QBit` (`L2DistanceTransposed`, `cosineDistanceTransposed`, `dotProductTransposed`, and their quantized variants) now accept a reference vector longer than the number of searched dimensions, ignoring the extra trailing elements. This allows reusing a full-size query vector for a reduced-dimension Matryoshka search without slicing it first. [#109388](https://github.com/ClickHouse/ClickHouse/pull/109388) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `basic` statistics `NullCount` sub-statistic is renamed to `DefaultCount` and now counts rows equal to the column type's default value, not only `NULL` rows of `Nullable` columns. [#109977](https://github.com/ClickHouse/ClickHouse/pull/109977) ([Han Fei](https://github.com/hanfei1991)). * Support concurrent writes to `Iceberg` tables on a local disk: `LocalObjectStorage` now implements conditional writes, so the compare-and-swap that publishes a new snapshot through `version-hint.text` works there and concurrent writers cannot lose each other's updates. [#112556](https://github.com/ClickHouse/ClickHouse/pull/112556) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE ... MOVE/REPLACE PARTITION` between two `MergeTree` tables with incompatible granularity now throws `BAD_ARGUMENTS` instead of `LOGICAL_ERROR`. [#110535](https://github.com/ClickHouse/ClickHouse/pull/110535) ([Groene AI](https://github.com/groeneai)). * The `Values` format no longer records false parse errors in `system.errors` and `system.error_log` when streaming parsing successfully falls back to SQL expression parsing. [#111141](https://github.com/ClickHouse/ClickHouse/pull/111141) ([Pablo Marcos](https://github.com/pamarcos)). * Added the `DuplicationDataHashComputations` `ProfileEvent`, which counts column-wise data-hash computations performed while deduplicating `INSERT` blocks into `*MergeTree` tables. [#111173](https://github.com/ClickHouse/ClickHouse/pull/111173) ([Valery Petrov](https://github.com/valerypetrov)). * Highlight the `./clickhouse` and `sudo ./clickhouse install` commands in the universal installation script output. [#111514](https://github.com/ClickHouse/ClickHouse/pull/111514) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Cartesian joins and analyzer-planned constant-predicate joins no longer fail because of an incompatible `join_algorithm` setting when the analyzer is used. [#108289](https://github.com/ClickHouse/ClickHouse/pull/108289) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * `bitmaskToArray` and `bitmaskToList` now support `Int128`, `UInt128`, `Int256`, and `UInt256` arguments. `bitPositionsToArray` is also faster for big-integer arguments because it now scales with the number of set bits rather than the highest set-bit position. [#110743](https://github.com/ClickHouse/ClickHouse/pull/110743) ([Manuel](https://github.com/raimannma)). * Fix a crash in JIT-compiled expressions on native from-source macOS/aarch64 builds by using the correct LLVM host triple for the host OS. [#111591](https://github.com/ClickHouse/ClickHouse/pull/111591) ([Raúl Marín](https://github.com/Algunenano)). * Fix `JSON_EXISTS`, `JSON_VALUE`, and `JSON_QUERY` on `Dynamic` input when tuple path arguments contain `LowCardinality` elements, avoiding `LOGICAL_ERROR` exceptions. Closes [#110345](https://github.com/ClickHouse/ClickHouse/issues/110345). [#109944](https://github.com/ClickHouse/ClickHouse/pull/109944) ([Groene AI](https://github.com/groeneai)). * Corrupt `ORC` files whose type tree declares more columns than the stripe footer now fail schema inference with `CANNOT_EXTRACT_TABLE_STRUCTURE` instead of aborting in debug/sanitizer builds. [#110967](https://github.com/ClickHouse/ClickHouse/pull/110967) ([Groene AI](https://github.com/groeneai)). * Rotated system log tables, such as `system.metric_log`, can now be marked `table_readonly` even when their metadata exceeds `max_query_size`, so rotation no longer logs `QUERY_IS_TOO_LARGE` and continues normally. [#111703](https://github.com/ClickHouse/ClickHouse/pull/111703) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In Play, added a button to close all tabs except the current one. [#111835](https://github.com/ClickHouse/ClickHouse/pull/111835) ([Mikhail Artemenko](https://github.com/Michicosun)). Fixed keyboard focus after activating it with Enter or Space when the button hides itself. [#111899](https://github.com/ClickHouse/ClickHouse/pull/111899) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In Play, holding `Ctrl` (`Cmd` on Mac) while hovering over a table cell whose value is an image URL now shows an image preview. [#109347](https://github.com/ClickHouse/ClickHouse/pull/109347) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * On Linux, ClickHouse now adds a `system.warnings` entry at startup when `rseq` is unavailable, because per-CPU profile counters then fall back to a slower `sched_getcpu` path. [#109283](https://github.com/ClickHouse/ClickHouse/pull/109283) ([Azat Khuzhin](https://github.com/azat)). * `COUNT(*)` over the `file` table function no longer throws an exception when schema inference wraps columns in `Nullable`. Closes [#102044](https://github.com/ClickHouse/ClickHouse/issues/102044). [#102509](https://github.com/ClickHouse/ClickHouse/pull/102509) ([Rory Shanks](https://github.com/rorylshanks)). * Functions `L1Normalize`, `L2Normalize`, `LinfNormalize`, and `LpNormalize` (and their aliases) now work for `Array` arguments, not only `Tuple` arguments, consistently with `L2Norm` and the distance functions. [#110052](https://github.com/ClickHouse/ClickHouse/pull/110052) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the legacy Apache Arrow-based `ORC` reader. The `ORC` input format now always uses the native ClickHouse decoder, which was already the default. The `input_format_orc_use_fast_decoder` setting is obsolete and has no effect. [#110074](https://github.com/ClickHouse/ClickHouse/pull/110074) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Made commonly used `clickhouse-client` arguments configurable from XML (``, ``, ``, ``, ``, ``), and preserved client-config values when the corresponding defaulted CLI flags are omitted, for settings such as `history_max_entries`, `suggestion_limit`, `progress`, `progress-table`, `enable_progress_table_toggle`, `print-memory-to-stderr`, `chime-threshold-seconds`, and `profile-events-delay-ms`. [#110607](https://github.com/ClickHouse/ClickHouse/pull/110607) ([Larry Snizek](https://github.com/larry-cdn77)). * Added the `analyzer_compatibility_apply_final_to_all_joined_tables` setting to restore the old behavior where `FINAL` on the left-most table of a `JOIN` also applies to the other joined tables. The setting is registered in the settings changes history, so `compatibility` with versions before 26.6 restores the old semantics automatically. [#111589](https://github.com/ClickHouse/ClickHouse/pull/111589) ([Nikita Fomichev](https://github.com/fm4v)). * In Play, the `+` (new tab) button now renders at the correct height in Firefox. [#109524](https://github.com/ClickHouse/ClickHouse/pull/109524) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A nested correlated `EXISTS` subquery that references a column from a scope beyond its immediate outer query now fails with `NOT_IMPLEMENTED` instead of the internal `NOT_FOUND_COLUMN_IN_BLOCK` exception. [#110310](https://github.com/ClickHouse/ClickHouse/pull/110310) ([Groene AI](https://github.com/groeneai)). * Formatting an `EXECUTE AS` query after AST rewrites no longer throws a client-side exception, for example in the AST fuzzer. [#111725](https://github.com/ClickHouse/ClickHouse/pull/111725) ([Groene AI](https://github.com/groeneai)). * Improve `DESCRIBE` query support for parameterized views, and support parameterized views in scalar expressions in the new analyzer. Closes [#66307](https://github.com/ClickHouse/ClickHouse/issues/66307). Closes [#69598](https://github.com/ClickHouse/ClickHouse/issues/69598). [#68978](https://github.com/ClickHouse/ClickHouse/pull/68978) ([Dmitry Novik](https://github.com/novikd)). * Malformed `CREATE INDEX` and `CREATE HYPOTHETICAL INDEX` expressions that cannot survive a format-parse-format round trip now report an exception instead of hitting an `Inconsistent AST formatting` logical error in debug and sanitizer builds. [#109175](https://github.com/ClickHouse/ClickHouse/pull/109175) ([Groene AI](https://github.com/groeneai)). * Web UI: fix the user/password field staying red even when the credentials are correct and the server is reachable, and re-check credentials after editing the connection so a correct value turns green. [#109414](https://github.com/ClickHouse/ClickHouse/pull/109414) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `text_index_max_processed_tokens_before_flush` and `text_index_max_memory_usage_before_flush` settings to control when text index builders flush temporary segments. [#111573](https://github.com/ClickHouse/ClickHouse/pull/111573) ([Rory Shanks](https://github.com/rorylshanks)). * Added the `system_cache_extensions` setting to control which file extensions are stored in the system cache when `use_split_cache` is enabled. [#111575](https://github.com/ClickHouse/ClickHouse/pull/111575) ([Kirill](https://github.com/kirillgarbar)). * Error messages on musl-based builds now show the human-readable `strerror` text instead of an empty or numeric value such as `strerror: 0`. [#111620](https://github.com/ClickHouse/ClickHouse/pull/111620) ([Konstantin Bogdanov](https://github.com/thevar1able)). * The effective named collections storage type is now available as `named_collections_storage.type` in `system.server_settings` and via `getServerSetting('named_collections_storage_type')`. [#111806](https://github.com/ClickHouse/ClickHouse/pull/111806) ([Pablo Marcos](https://github.com/pamarcos)). * Relaxes an overly strict permutation size check in `ColumnReplicated::permute` so that it follows the general `IColumn::permute` contract, which allows a permutation shorter than the column when a limit is set. [#109897](https://github.com/ClickHouse/ClickHouse/pull/109897) ([Groene AI](https://github.com/groeneai)). * Improve planning of streaming queries by unifying the internal streaming read plan, which avoids double-optimizing direct text-index reads. [#111821](https://github.com/ClickHouse/ClickHouse/pull/111821) ([Mikhail Artemenko](https://github.com/Michicosun)). * Web UI: the gap between the `Run one` and `Run all` buttons now matches the left margin before the first button. [#111988](https://github.com/ClickHouse/ClickHouse/pull/111988) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Memory reservation scheduling (experimental) no longer evicts an allocation when a pending memory release would free the room needed by an over-limit reservation increase: the eviction decision now waits for in-flight decreases to be applied, avoiding unnecessary query kills under memory pressure. [#112299](https://github.com/ClickHouse/ClickHouse/pull/112299) ([Sergei Trifonov](https://github.com/serxa)). * Iceberg v2 requires `sequence_number` in manifest files, but other engines such as BigQuery can write manifest files without it. ClickHouse can now read such tables. [#112431](https://github.com/ClickHouse/ClickHouse/pull/112431) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Show warning when statistics for join reordering are missing. [#107666](https://github.com/ClickHouse/ClickHouse/pull/107666) ([Vladimir Cherkasov](https://github.com/vdimir)). * Added MySQL-compatibility columns to `INFORMATION_SCHEMA` views: `SCHEMATA.DEFAULT_COLLATION_NAME` and `DEFAULT_ENCRYPTION`; `TABLES.ENGINE` and related MySQL columns; and `COLUMNS.COLUMN_KEY`, `PRIVILEGES`, `GENERATION_EXPRESSION`, `SRS_ID`. This lets MySQL-aware clients run their catalog introspection queries without hitting `UNKNOWN_IDENTIFIER`. [#109351](https://github.com/ClickHouse/ClickHouse/pull/109351) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * An unqualified reference to an `INNER JOIN` key that is equated in the `ON` condition (e.g. `SELECT id FROM a INNER JOIN b ON a.id = b.id`) is no longer reported as an ambiguous identifier, since both sides are guaranteed to carry the same value. [#109366](https://github.com/ClickHouse/ClickHouse/pull/109366) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Return HTTP code `403 Forbidden` (instead of `500 Internal Server Error`) for `ACCESS_DENIED` exceptions over the HTTP interface. [#111043](https://github.com/ClickHouse/ClickHouse/pull/111043) ([Schum](https://github.com/Schum-io)). * Write the UNIQUE KEY index SST directly into part storage instead of staging it in a local temporary file, removing the dependency on a local temporary volume and avoiding an extra copy. [#111189](https://github.com/ClickHouse/ClickHouse/pull/111189) ([johnjing](https://github.com/JingYanchao)). * `UTMToGeo` now accepts the MGRS latitude band letter returned by `geoToUTM` as its fourth argument (in addition to the integer hemisphere flag), so a `geoToUTM` result round-trips through `UTMToGeo` directly. [#111521](https://github.com/ClickHouse/ClickHouse/pull/111521) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` ("Part level Min-Max index was constructed from unexpected columns set") that could abort a background merge in debug/sanitizer builds after `part_minmax_index_columns` was lowered (for example from `with_block_number_offset` back to `partition_key_only`). [#111511](https://github.com/ClickHouse/ClickHouse/pull/111511) ([Groene AI](https://github.com/groeneai)). * Fixes `ProfileEvents['JoinResultRowCount']` under-reporting the result size of a `JOIN` that spills to disk. Rows emitted from delayed buckets were counted only in `JoinDelayedJoinedTransformRowCount` and were missing from the total. Query results were always correct; only the profile event was wrong. [#112673](https://github.com/ClickHouse/ClickHouse/pull/112673) ([Groene AI](https://github.com/groeneai)). * `SYSTEM DISABLE FAILPOINT` now rejects a fail point name that does not exist, raising `BAD_ARGUMENTS` like `SYSTEM ENABLE FAILPOINT` already did. Previously a mistyped name reported success while leaving the intended fail point enabled, with no indication that nothing had been disabled. [#112680](https://github.com/ClickHouse/ClickHouse/pull/112680) ([Groene AI](https://github.com/groeneai)). * Added a new column `skipping_indices_types` to the `system.tables` table. It contains the distinct types of data-skipping indices defined for each table. [#106388](https://github.com/ClickHouse/ClickHouse/pull/106388) ([Anton Popov](https://github.com/CurtizJ)). * Added a sanity check that a distributed query always carries a known client version, throwing a logical error instead of silently forwarding a zero version to remote shards. Server-initiated queries (background flushes, streaming consumers, dictionary reloads, asynchronous insert flushes) now report the server's own version as the initiator version. [#109408](https://github.com/ClickHouse/ClickHouse/pull/109408) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduced the verbosity of the per-part primary-key binary-search diagnostics in `MergeTree` by lowering them from trace logging to test logging. [#110324](https://github.com/ClickHouse/ClickHouse/pull/110324) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The hive partitioning sample path for object storage tables (e.g. `S3`) is resolved on the first use of the table instead of `CREATE`/`ATTACH`, so an unreachable endpoint no longer blocks table creation and server startup. [#111842](https://github.com/ClickHouse/ClickHouse/pull/111842) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `ReadBufferFromPocoSocketBase::setReceiveTimeout` applying its argument as seconds rather than microseconds, and fixed builds configured with `-DENABLE_LIBFIU=OFF`, which did not compile. In builds without libfiu, `SYSTEM ENABLE FAILPOINT` and related commands now throw an exception instead of silently doing nothing. [#112767](https://github.com/ClickHouse/ClickHouse/pull/112767) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `clickhouse-local` under-reporting `rows_read` and the other progress counters in the `statistics` of the `JSON`/`XML` output and in the progress bar: a progress increment that arrived while the progress packet was being assembled was discarded. [#112958](https://github.com/ClickHouse/ClickHouse/pull/112958) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the error reported when an asynchronous `INSERT` into a `Distributed` table needs a queue directory whose name exceeds the 255-byte filesystem limit (reachable with `use_compact_format_in_distributed_parts_names = 0`). It was a `LOGICAL_ERROR` for a shard with `internal_replication` and an unattributed `Code: 1001. std::exception` otherwise; both now report `ARGUMENT_OUT_OF_BOUND` naming the table, the cluster and the limit. [#113083](https://github.com/ClickHouse/ClickHouse/pull/113083) ([Groene AI](https://github.com/groeneai)). * Reject out-of-range `execution_time` quota limits with `BAD_ARGUMENTS` instead of relying on undefined behavior when scaling them to internal nanoseconds. [#113178](https://github.com/ClickHouse/ClickHouse/pull/113178) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Automatic parallel replicas now support `JOIN` queries. [#106073](https://github.com/ClickHouse/ClickHouse/pull/106073) ([Nikita Taranov](https://github.com/nickitat)). * Support refresh-token authentication for the `OneLake` catalog. [#110413](https://github.com/ClickHouse/ClickHouse/pull/110413) ([alesapin](https://github.com/alesapin)). * The `write_snapshot_version` `Keeper` setting is now hot-reloadable. [#112150](https://github.com/ClickHouse/ClickHouse/pull/112150) ([alesapin](https://github.com/alesapin)). * Parallelize `SYSTEM DROP FILESYSTEM CACHE`. [#112532](https://github.com/ClickHouse/ClickHouse/pull/112532) ([Kseniia Sumarokova](https://github.com/kssenii)). * Unsupported nesting of a `SQL SECURITY DEFINER` or `SQL SECURITY NONE` view over `s3Cluster`, `urlCluster`, or `fileCluster` is now rejected with a normal query error instead of hitting a `LOGICAL_ERROR` in debug and sanitizer builds. [#113371](https://github.com/ClickHouse/ClickHouse/pull/113371) ([Groene AI](https://github.com/groeneai)). * Added the setting `analyzer_compatibility_multiple_joins_qualify_column_names` (default `false`). When enabled and the `FROM` clause of a query contains two or more `JOIN`s, result column names produced by the analyzer mimic the old analyzer's multiple-joins rewrite: columns expanded from `*` are named `.`, and an unaliased column reference in the `SELECT` list keeps its name exactly as written. This makes outer queries that reference such qualified names, like `SELECT ll.Date FROM (SELECT * FROM t AS ll JOIN t1 ON ... JOIN t2 ON ...)`, work as they did with the old analyzer. It also fixes the analyzer losing qualified result column names for columns expanded from `*` when `group_by_use_nulls` is combined with `ROLLUP`, `CUBE`, or `GROUPING SETS`, which produced duplicate result column names and broke outer references to those columns. [#110746](https://github.com/ClickHouse/ClickHouse/pull/110746) ([Dmitry Novik](https://github.com/novikd)). * Renamed `allow_experimental_delta_kernel_rs` to `allow_delta_kernel_rs`. The old name is kept as an alias. [#113476](https://github.com/ClickHouse/ClickHouse/pull/113476) ([Kseniia Sumarokova](https://github.com/kssenii)). * When the `remote`/`remoteSecure` table functions or the `Remote`/`RemoteSecure` table engines are given a missing named collection together with a `key = value` override, ClickHouse now reports the missing named collection instead of reparsing the call positionally and reporting an unrelated error. [#113510](https://github.com/ClickHouse/ClickHouse/pull/113510) ([Groene AI](https://github.com/groeneai)). * Shell startup with `clickhouse-bootstrap` completion no longer dumps environment variables when the completion is loaded from the user completion hook. [#113694](https://github.com/ClickHouse/ClickHouse/pull/113694) ([Azat Khuzhin](https://github.com/azat)). * Added the `sainte_lague` value for `shared_merge_tree_merge_coordinator_distribution_algorithm` and made it the default instead of `water_filling`. [#113442](https://github.com/ClickHouse/ClickHouse/pull/113442) ([Mikhail Artemenko](https://github.com/Michicosun)). * Added the `filesystem_cache_wait_for_concurrent_download_timeout_milliseconds` setting to bound how long a read waits for another query's filesystem-cache download of a single chunk (about 1 MiB), not a whole file segment. [#113322](https://github.com/ClickHouse/ClickHouse/pull/113322) ([Kseniia Sumarokova](https://github.com/kssenii)). * Added the Keeper setting `min_time_between_fsyncs_ms` (default `5` ms), which batches changelog appends that arrive shortly after the previous flush instead of starting another flush immediately. [#113749](https://github.com/ClickHouse/ClickHouse/pull/113749) ([Michael Kolupaev](https://github.com/al13n321)). * Kafka tables with Keeper-stored offsets (`kafka_keeper_path`) now throw `ABORTED` instead of `LOGICAL_ERROR` when the table becomes inactive after a Keeper session loss, for example during a direct read or materialized-view streaming. [#113913](https://github.com/ClickHouse/ClickHouse/pull/113913) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed inconsistent AST formatting of `viewIfPermitted`: the table function form no longer formats an operator such as `not` incorrectly in its `ELSE` branch, and the expression form `viewIfPermitted(...)` no longer gets a spurious `ELSE`. [#113652](https://github.com/ClickHouse/ClickHouse/pull/113652) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a spurious `Failed to drop temporary table after refresh. Table ... is left behind and requires manual cleanup.` error logged when a refreshable materialized view refresh is cancelled (for example by server shutdown) while `database_atomic_wait_for_drop_and_detach_synchronously` is enabled. The temporary table was in fact dropped, so no manual cleanup was needed. [#113957](https://github.com/ClickHouse/ClickHouse/pull/113957) ([Groene AI](https://github.com/groeneai)). * The server now throws `UNKNOWN_ELEMENT_IN_CONFIG` on unknown configuration options in the server config, helping catch typos and misconfigured options early. The check can be disabled with ``. [#100332](https://github.com/ClickHouse/ClickHouse/pull/100332) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, the browser history and the URL are no longer updated on every keypress; they are recorded when a query is successfully run or on a tab change. [#113596](https://github.com/ClickHouse/ClickHouse/pull/113596) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * ClickHouse now reports a stack overflow instead of dying silently: the fatal signal handlers run on the alternative signal stack, so `SIGSEGV`, `SIGABRT`, `SIGILL`, `SIGBUS`, `SIGSYS`, `SIGFPE`, and `SIGTRAP` produce a stack trace even when the faulting thread has exhausted its stack. [#113927](https://github.com/ClickHouse/ClickHouse/pull/113927) ([Groene AI](https://github.com/groeneai)). * `ALTER TABLE ... MODIFY COLUMN Tuple(...)` on a named `Tuple` is now metadata-only when only adding subfields, matching the speed of top-level `ADD COLUMN`. Gated behind `SET `allow_metadata_only_named_tuple_alter` = 1`. [#107305](https://github.com/ClickHouse/ClickHouse/pull/107305) ([Amos Bird](https://github.com/amosbird)). * Add `input_format_json_max_object_size` setting to limit JSON object size on parsing. Closes [#106704](https://github.com/ClickHouse/ClickHouse/issues/106704). [#107669](https://github.com/ClickHouse/ClickHouse/pull/107669) ([Pavel Kruglov](https://github.com/Avogar)). * Avoid logging `sasl.kerberos.kinit.cmd configuration parameter is ignored.` for `Kafka` engine configurations that do not use Kerberos keytab authentication. [#108235](https://github.com/ClickHouse/ClickHouse/pull/108235) ([Ivan Shelestov](https://github.com/Shelestov7)). * Add system.s3(azure)_queue_metadata. [#108522](https://github.com/ClickHouse/ClickHouse/pull/108522) ([Kseniia Sumarokova](https://github.com/kssenii)). * Log tolerated connection failures to remote MySQL/PostgreSQL databases as warnings instead of errors. [#109472](https://github.com/ClickHouse/ClickHouse/pull/109472) ([Shaohua Wang](https://github.com/tiandiwonder)). * When neither `port` nor `secure` is specified, `clickhouse-client` tries both the default port 9000 and the secure port 9440 concurrently and uses the one that answers first, so `clickhouse-client --host play.clickhouse.com --user play` connects over TLS without `--secure`, even though the plain port of that server is silently dropped rather than refused. If the port that answered turns out to be unusable — a secure port with an untrusted certificate, for example — the client falls back to the other one, because the protocol was not requested explicitly. [#110130](https://github.com/ClickHouse/ClickHouse/pull/110130) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Clicking the cloud logo in the Playground now opens the ClickHouse website in a new tab, so you don't lose progress while queries are running. [#110422](https://github.com/ClickHouse/ClickHouse/pull/110422) ([William Hatcher](https://github.com/williamhatcher)). * Add STREAM BOUNDED modifier, which read only the first snapshot of a streaming query, then finish instead of subscribing for updates. [#110653](https://github.com/ClickHouse/ClickHouse/pull/110653) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Add information of internal state of joins to `EXPLAIN ANALYZE`. [#110892](https://github.com/ClickHouse/ClickHouse/pull/110892) ([Kirill Kopnev](https://github.com/Fgrtue)). * When an `RWLockImpl::getLock()` re-entrancy assertion fires (`RWLock is already locked in exclusive mode` / `Cannot acquire exclusive lock while RWLock is already locked`), the error message now names the other `query_id`s currently owning the lock and how many locks the requesting `query_id` already holds, to make such lock-ordering errors diagnosable. [#110971](https://github.com/ClickHouse/ClickHouse/pull/110971) ([Groene AI](https://github.com/groeneai)). * ClickHouse Keeper now logs the "Client has not sent any data" event at `Information` level instead of `Warning`, matching the server's TCP handler. This avoids spurious warnings from TCP health-check probes (for example Kubernetes `tcpSocket` liveness probes) that open and immediately close a connection to the Keeper client port. [#111217](https://github.com/ClickHouse/ClickHouse/pull/111217) ([Zeynel](https://github.com/ZeynelKoca)). * `MaterializedPostgreSQL` now preserves unchanged PostgreSQL `TOAST` values during updates instead of replacing them with default values. [#111552](https://github.com/ClickHouse/ClickHouse/pull/111552) ([OrpheusAgent](https://github.com/OrpheusAgent)). * keeper: Increase default `write_snapshot_version` to 8 and make it hot-reloadable. [#111715](https://github.com/ClickHouse/ClickHouse/pull/111715) ([Michael Kolupaev](https://github.com/al13n321)). * Add STREAM UNORDERED modifier: skip the per-snapshot commit-order sort. [#111794](https://github.com/ClickHouse/ClickHouse/pull/111794) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Added a new MergeTree setting `text_index_version` that controls the on-disk format version of text indexes: `v0_initial`, `v1_with_codec`, or `v2_with_positions`. During a rolling upgrade or before a downgrade, set it to an older version so that newer servers keep writing text index parts in a format older servers can still read; the compatibility setting automatically adjusts it. [#111803](https://github.com/ClickHouse/ClickHouse/pull/111803) ([Anton Popov](https://github.com/CurtizJ)). * AI text functions (`aiGenerate`, `aiClassify`, `aiExtract`, `aiTranslate`) now reject truncated or otherwise incomplete provider responses (e.g. when the model hits the `max_tokens` limit) instead of silently returning partial output. Behavior follows the `ai_function_throw_on_error` setting. [#111830](https://github.com/ClickHouse/ClickHouse/pull/111830) ([George Larionov](https://github.com/george-larionov)). * Improve reduced-precision `L2DistanceTransposedQuantized`, `cosineDistanceTransposedQuantized`, and `dotProductTransposedQuantized` by reconstructing `p < 8` `QBit(Int8)` codes with Gaussian conditional-mean prefix centroids. Full precision (`p = 8`) remains bit-exact; reduced-precision approximate distances may change, and recall/latency gains are workload-dependent. [#111867](https://github.com/ClickHouse/ClickHouse/pull/111867) ([Sergey Kuznetsov](https://github.com/skuznetsov)). * Fixed the _query condition cache_ not being populated for granules eliminated by any but the last conjunct of a `WHERE` clause with multiple `AND` conditions. [#112083](https://github.com/ClickHouse/ClickHouse/pull/112083) ([Shankar Iyer](https://github.com/shankar-iyer)). * Added a new setting `input_format_json_max_string_column_growth_step` that caps the power-of-two growth of internal String buffers when building a JSON column, reducing peak memory when inserting large JSON documents. Disabled by default. [#112159](https://github.com/ClickHouse/ClickHouse/pull/112159) ([Pavel Kruglov](https://github.com/Avogar)). * Reduced peak memory when writing JSON columns with bucketed shared data serialization (`map_with_buckets` and advanced) by splitting the shared data into buckets one at a time on write instead of materializing all buckets simultaneously. This is mostly beneficial for Array(JSON) columns: for scalar JSON the shared data is split per granule (bounded by index_granularity_bytes), so the saving is small, whereas a single Array(JSON) granule can hold many nested rows and thus a large amount of shared data, where the one-bucket-at-a-time split significantly lowers peak memory. [#112172](https://github.com/ClickHouse/ClickHouse/pull/112172) ([Pavel Kruglov](https://github.com/Avogar)). * The documentation of a setting in `system.documentation` — shown by the built-in `/docs` page and by the `help` command — now includes the history of the changes of its default value: the version in which the setting was introduced and every later change of its default, with the previous value, the new value and the reason for the change. [#112177](https://github.com/ClickHouse/ClickHouse/pull/112177) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `EXPLAIN ANALYZE` now works for streaming queries. [#112445](https://github.com/ClickHouse/ClickHouse/pull/112445) ([Kirill Kopnev](https://github.com/Fgrtue)). * The modifiers of a column declaration - `COMMENT`, `CODEC`, `STATISTICS`, `TTL`, `COLLATE`, `PRIMARY KEY` and per-column `SETTINGS` - can now be written in any order in `CREATE TABLE`, and each of them at most once. Previously only one fixed order was accepted, and, for example, `x UInt64 CODEC(ZSTD) COMMENT 'text'` was a syntax error. In `ALTER TABLE ... ADD COLUMN` / `MODIFY COLUMN`, the supported modifiers - `COMMENT`, `CODEC`, `STATISTICS`, `TTL` and per-column `SETTINGS` - can also be written in any order; per-column `SETTINGS` in `ADD COLUMN` and a declared `STATISTICS` in `ADD COLUMN` / `MODIFY COLUMN` are now applied instead of being silently dropped, and `COLLATE` and `PRIMARY KEY` in these `ALTER` commands now throw an exception instead of being silently ignored. [#112788](https://github.com/ClickHouse/ClickHouse/pull/112788) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `pread_threadpool` read method needs the `preadv2` system call with the `RWF_NOWAIT` flag to read the data that is already in the page cache without handing the read off to a thread pool. It is now checked at start time whether that system call can be used, and if it cannot - the Linux kernel is older than 5.11, or a `seccomp` profile of a container runtime rejects the system call - the default value of `local_filesystem_read_method` is switched to `pread`, and the reason is reported in the server log. Previously, every read paid for a thread pool hand-off on such systems, and a `seccomp` profile that answers `EPERM` made queries fail with `CANNOT_READ_FROM_FILE_DESCRIPTOR`. [#112945](https://github.com/ClickHouse/ClickHouse/pull/112945) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Report malformed Parquet thrift metadata as INCORRECT_DATA. [#113311](https://github.com/ClickHouse/ClickHouse/pull/113311) ([Groene AI](https://github.com/groeneai)). * Fixed GCD codec handling of signed fixed-width values with negative numbers. Previously it computed the divisor from signed values directly, which could miss the common divisor and significantly reduce compression efficiency for signed integer and decimal columns. It now computes the divisor from magnitudes for signed types while preserving existing behavior for unsigned types and decompression. [#113798](https://github.com/ClickHouse/ClickHouse/pull/113798) ([Kirill Shcherbatov](https://github.com/kshcherbatov)). * Add settings `enable_alp_codec`, `enable_sz3_codec`, `enable_zxc_codec`, and `enable_quantized_codec` to enable each experimental compression codec individually. [#113824](https://github.com/ClickHouse/ClickHouse/pull/113824) ([Raufs Dunamalijevs](https://github.com/rienath)). * Allow `ALTER TABLE ... MODIFY COLUMN` of a column whose subcolumns are used in the primary or partition key, as long as the subcolumns used in the key keep an on-disk-compatible type. Previously any such ALTER was forbidden. [#113862](https://github.com/ClickHouse/ClickHouse/pull/113862) ([Pavel Kruglov](https://github.com/Avogar)). * Add `Filesystem cache` dashboard to `system.dashboards`. [#113884](https://github.com/ClickHouse/ClickHouse/pull/113884) ([Kseniia Sumarokova](https://github.com/kssenii)). * Put the per-buffer-refill cache logs behind `filesystem_cache_verbose_logging`. [#113885](https://github.com/ClickHouse/ClickHouse/pull/113885) ([Groene AI](https://github.com/groeneai)). * When a column's declared type and its data diverge during a `MergeTree` read (mixed type provenance, e.g. after an `ALTER TABLE ... MODIFY COLUMN` whose mutation has not finished), the server now reports a clear exception naming the column and both structures, instead of an unchecked cast: debug and sanitizer builds previously aborted with a bare `Bad cast from type A to B` naming no column, and release builds walked mismatched memory silently. [#114087](https://github.com/ClickHouse/ClickHouse/pull/114087) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A window function with `PARTITION BY` now works under `make_distributed_plan`. When the plan shape allows, the window runs in parallel across buckets that each receive complete partitions. [#114111](https://github.com/ClickHouse/ClickHouse/pull/114111) ([Vighnesh Pathrikar](https://github.com/VighneshPath)). * Web UI: the hourglass loading indicator in the databases panel is animated again. [#114187](https://github.com/ClickHouse/ClickHouse/pull/114187) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * 64-bit hash function is used now for external nullable fixed-width aggregation methods (avoids collision disaster for very high cardinality aggregation). [#114210](https://github.com/ClickHouse/ClickHouse/pull/114210) ([Nikita Taranov](https://github.com/nickitat)). * `Distributed` and `Buffer` tables that omit their column list now always infer the structure under the creating user's access rights. [#114211](https://github.com/ClickHouse/ClickHouse/pull/114211) ([Pedro Ferreira](https://github.com/PedroTadim)). * `x IN (subquery)` over a `LowCardinality` column now returns `LowCardinality(UInt8)` — the same type as `x IN (literal list)`. Previously the two forms returned different types, and distributed plans with such an `IN` needed a special case in the plan type check. [#114229](https://github.com/ClickHouse/ClickHouse/pull/114229) ([Alexander Gololobov](https://github.com/davenger)). * Enabled `ie_join` by default: the default value of `join_algorithm` is now `direct,parallel_hash,hash,ie_join`. A `JOIN` whose `ON` section has only inequality conditions (two comparisons `<`, `<=`, `>`, `>=` between expressions of the joined tables) is now executed with the sort-based IEJoin algorithm instead of a `CROSS JOIN` with a filter, and `LEFT`/`RIGHT`/`FULL`/`SEMI`/`ANTI` joins with such conditions are supported. Since `ie_join` is last in the list, it is used only when the other algorithms do not apply. [#114327](https://github.com/ClickHouse/ClickHouse/pull/114327) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed charts in the Web UI showing the data of a mirrored point in the hover balloon for a result ordered by `x` descending. `Date`, `Date32`, `DateTime` and `DateTime64` columns can now be plotted; previously only integer Unix timestamps worked. Dates on the axes and in the balloons are rendered in ISO 8601, and quantities with the `K`/`M`/`G`/... suffixes, the same way the advanced dashboard does. [#114349](https://github.com/ClickHouse/ClickHouse/pull/114349) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support parsing integers that exceed the 64-bit range in the `JSON` data type, `JSONExtract`, and `isValidJSON`. Such values are read into wide integer types such as `Int128`/`UInt256` instead of causing the whole document to be rejected. [#114379](https://github.com/ClickHouse/ClickHouse/pull/114379) ([Pavel Kruglov](https://github.com/Avogar)). * QueryRunner tables now start worker threads on demand and release them once idle, instead of occupying threads for the table's whole lifetime. [#114522](https://github.com/ClickHouse/ClickHouse/pull/114522) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Added observability for the stacks of fibers, which are used for asynchronous communication with remote replicas: profile events `FiberStackAllocs`, `FiberStackAllocBytes`, `FiberStackAllocNanoseconds`, `FiberStackFreeNanoseconds`, and metrics `FiberStacks`, `FiberStackBytes`. [#114541](https://github.com/ClickHouse/ClickHouse/pull/114541) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * PromQL: support `@ start()` and `@ end()` timestamp modifiers. [#114558](https://github.com/ClickHouse/ClickHouse/pull/114558) ([Minh Vu](https://github.com/fallintoplace)). * Propagate the OpenTelemetry trace context into the fibers used for asynchronous connection establishment and remote query execution (hedged requests, `async_socket_for_remote`, `async_query_sending_for_remote`). Spans of remote queries in a distributed query are now correctly parented under the initiator's `Connection::sendQuery` span instead of being attached directly to the client-supplied trace context, and each asynchronous task execution produces its own span. [#114813](https://github.com/ClickHouse/ClickHouse/pull/114813) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * With this change, while using the WHATIF indices, the projection-served baseline reports per-candidate `not_applicable` instead of failing the statement. [#114846](https://github.com/ClickHouse/ClickHouse/pull/114846) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `DROP DATABASE` now drops tables in reverse topological order of both loading and referential dependencies (previously only loading dependencies), so a crash in the middle of `DROP DATABASE` cannot leave a table whose dependencies were already dropped. [#114952](https://github.com/ClickHouse/ClickHouse/pull/114952) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI dashboards, the global error screen is replaced by a note above the dashboards, so a failing chart no longer hides the whole page. [#115019](https://github.com/ClickHouse/ClickHouse/pull/115019) ([Mikhail Artemenko](https://github.com/Michicosun)). * Spill the adaptive aggregator's learning-phase tables under memory pressure. [#115038](https://github.com/ClickHouse/ClickHouse/pull/115038) ([Groene AI](https://github.com/groeneai)). * `clickhouse-local` now answers CORS preflight requests with the same permissive headers as `clickhouse-server`, so the HTTP interface opened by `SYSTEM START LISTEN HTTP` can be used from a browser out of the box, including the Web UI opened from a `file://` URL. [#115045](https://github.com/ClickHouse/ClickHouse/pull/115045) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support PostgreSQL-style regular expression match operators: `~` (an alias for the `match` function), `~*` (case-insensitive match), `!~` and `!~*` (negations). The `\d`, `\dt`, `\dv` commands of `psql` now work when connected to ClickHouse over the PostgreSQL compatibility protocol, and a failed query no longer terminates the connection. [#115066](https://github.com/ClickHouse/ClickHouse/pull/115066) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-client` no longer prints the `Connecting to ...` message twice when it asks for a password interactively. [#115081](https://github.com/ClickHouse/ClickHouse/pull/115081) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `EXPLAIN WHATIF` now says why the empirical estimate was skipped, in a new `empirical_reason` line shown when `empirical_status` is `unsupported`. [#115140](https://github.com/ClickHouse/ClickHouse/pull/115140) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * When no join algorithm enabled by the `join_algorithm` setting can execute a `JOIN`, the error message now names the algorithms that were tried and the strictness and kind of the JOIN that failed, instead of reporting only that none of them worked. [#115226](https://github.com/ClickHouse/ClickHouse/pull/115226) ([Alexey Elkin](https://github.com/elkinal)). * Vector search queries can now use the vector similarity index when the reference vector is an integer array literal, e.g. `ORDER BY L2Distance(vec, [1, 2])`. Previously, such queries silently fell back to a brute-force scan. [#115255](https://github.com/ClickHouse/ClickHouse/pull/115255) ([Robert Schulze](https://github.com/rschu1ze)). * Azure client-side network failures (connection reset/refused, DNS and TLS errors) are now retried as transport errors instead of surfacing as a synthetic `500`, matching the S3 client. [#115269](https://github.com/ClickHouse/ClickHouse/pull/115269) ([Arsen Muk](https://github.com/arsenmuk)). * The documentation of all SQL statements (e.g. name, syntax, description, examples) can now be retrieved from system table `system.statements`. [#115341](https://github.com/ClickHouse/ClickHouse/pull/115341) ([Robert Schulze](https://github.com/rschu1ze)). * `BACKUP` no longer leaves a lock file behind in its destination when the attempt that created it fails before writing anything -- whether it fails while claiming the destination or while opening an archive. No later backup could match such a lock, so every retry to the same destination failed until the file was deleted by hand. Backups to `S3` and `Azure` destinations also create the lock exclusively, and a lock that belongs to another backup is reported as a concurrent backup rather than as whatever error the storage returned. [#115387](https://github.com/ClickHouse/ClickHouse/pull/115387) ([Julia Kartseva](https://github.com/jkartseva)). * Support bearer-token authentication in BuilderRWBufferFromHTTP (attempt 2). [#115400](https://github.com/ClickHouse/ClickHouse/pull/115400) ([Sergei Trifonov](https://github.com/serxa)). * Add some dimensional metrics for S3Queue. [#115422](https://github.com/ClickHouse/ClickHouse/pull/115422) ([Bharat Nallan](https://github.com/bharatnc)). * Added `keyword` as an alias for text index tokenizer `array` for improved compatibilty with OpenSearch, Elasticsearch, SolR, and Lucene. [#115507](https://github.com/ClickHouse/ClickHouse/pull/115507) ([Robert Schulze](https://github.com/rschu1ze)). * Allow to create own s3 tables. [#115652](https://github.com/ClickHouse/ClickHouse/pull/115652) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Do not log per-thread untracked memory on non-server wide MEMORY_LIMIT_EXCEEDED errors. [#115658](https://github.com/ClickHouse/ClickHouse/pull/115658) ([Azat Khuzhin](https://github.com/azat)). * The Prometheus remote-write protocol now supports asynchronous inserts: data from multiple concurrent remote-write requests is batched before forming parts, following the `async_insert` setting (enabled by default; add `async_insert=0` to the handler URL to keep the synchronous behavior). A request is acknowledged only after the data is flushed to all inner tables of the target `TimeSeries` table. [#115688](https://github.com/ClickHouse/ClickHouse/pull/115688) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Makes all `timeSeries*` aggregate functions handle duplicate timestamps by one rule (the greatest value wins, NaN loses to any other value), fixing the order-dependent results of `timeSeriesInstantRateToGrid`, `timeSeriesInstantDeltaToGrid`, `timeSeriesLastTwoSamples` and `timeSeriesResampleToGridWithStaleness` when some values at a shared timestamp are NaN. The rule is now documented and tested. [#115920](https://github.com/ClickHouse/ClickHouse/pull/115920) ([Vitaly Baranov](https://github.com/vitlibar)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fix column-name corruption and `BAD_ARGUMENTS` exceptions with `inject_random_order_for_select_without_order_by`: multi-column `SELECT` queries no longer fail, and output column names for `CREATE TABLE AS SELECT`, `CREATE VIEW AS SELECT`, and named formats such as `JSONEachRow` are preserved instead of being replaced with internal `__subquery_column_` aliases. Closes [#102812](https://github.com/ClickHouse/ClickHouse/issues/102812). Closes [#101107](https://github.com/ClickHouse/ClickHouse/issues/101107). [#105896](https://github.com/ClickHouse/ClickHouse/pull/105896) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` in the native `Parquet` V3 reader when a `PREWHERE` has conjuncts that share a common intermediate expression. [#107059](https://github.com/ClickHouse/ClickHouse/pull/107059) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` for queries that use the `_part_starting_offset` or `_part_offset` virtual columns in `WHERE` together with lazy materialization. [#108287](https://github.com/ClickHouse/ClickHouse/pull/108287) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix `Not found column or subcolumn c.null in block` when running `WHERE col IS NULL` or `IS NOT NULL` on a schema-evolved `Iceberg` column that was added after older data files were written, with `optimize_functions_to_subcolumns = 1` (the default). [#109515](https://github.com/ClickHouse/ClickHouse/pull/109515) ([Groene AI](https://github.com/groeneai)). * Fix false `ICEBERG_SPECIFICATION_VIOLATION` errors when reading an `Iceberg` table whose decimal or other parameterized primitive type is serialized with different whitespace across metadata files, such as `decimal(20,0)` in the table metadata and `decimal(20, 0)` in the manifest. [#109676](https://github.com/ClickHouse/ClickHouse/pull/109676) ([Groene AI](https://github.com/groeneai)). * Fix deferred `PREWHERE` with `optimize_move_to_prewhere = 1` when `PREWHERE` is applied after `FINAL`: `WHERE` conditions are no longer moved ahead of a deferred row policy, which could change query results. [#109703](https://github.com/ClickHouse/ClickHouse/pull/109703) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `BAD_GET` when comparing an `Array(String)` column with an array literal such as `arr = ['x']` and the column has a `text`, `tokenbf_v1`, or `ngrambf_v1` skip index. Such comparisons now fall back to a full scan instead of failing the query. [#110057](https://github.com/ClickHouse/ClickHouse/pull/110057) ([Groene AI](https://github.com/groeneai)). * Fix wrong results for `has`, `mapContainsKey`, and `mapContainsValue` with an empty needle when a text index is present. [#110246](https://github.com/ClickHouse/ClickHouse/pull/110246) ([Robert Schulze](https://github.com/rschu1ze)). * Fix `Host is empty in S3 URI` when a scalar subquery is used as the URL argument of the `s3` table function, or as an argument of any other table function, in `CREATE TABLE ... AS SELECT` queries. [#110254](https://github.com/ClickHouse/ClickHouse/pull/110254) ([Sema Checherinda](https://github.com/CheSema)). * Fix `ON CLUSTER` privilege checks for `SYSTEM STOP/START CLEANUP` and `SYSTEM STOP/START VIRTUAL PARTS UPDATE`: they no longer require `SYSTEM PULLING REPLICATION LOG` and now use `SYSTEM CLEANUP` or `SYSTEM VIRTUAL PARTS UPDATE`. [#110289](https://github.com/ClickHouse/ClickHouse/pull/110289) ([Groene AI](https://github.com/groeneai)). * Fix a read-only object-storage replica, with `read_only = true`, not discovering new parts via `refresh_parts_interval`, and `table_disk = true` failing on such a disk with `table_disk is not supported for non-ObjectStorage disks`. [#110460](https://github.com/ClickHouse/ClickHouse/pull/110460) ([Julia Kartseva](https://github.com/jkartseva)). * Fix writes through data-lake table functions such as `INSERT INTO FUNCTION icebergS3(...)` producing `Parquet` files whose string columns lost the `String` type annotation, making the written data unreadable for external readers such as Spark. [#110465](https://github.com/ClickHouse/ClickHouse/pull/110465) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix schema inference failing with `Map cannot have a key of type LowCardinality(Nullable(String))` when reading `ORC` files whose map keys are dictionary-encoded, including files written by Spark and files written by ClickHouse itself with `output_format_orc_dictionary_key_size_threshold` enabled. [#110492](https://github.com/ClickHouse/ClickHouse/pull/110492) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix wrong `IS NULL` / `IS NOT NULL` / `count()` results with `optimize_functions_to_subcolumns` on a column made `Nullable` by a metadata-only `ALTER MODIFY COLUMN T` to `Nullable(T)`. For parts written before the conversion, the `.null` subcolumn is now derived from the physically present parent column instead of the storage-type default. [#110584](https://github.com/ClickHouse/ClickHouse/pull/110584) ([Groene AI](https://github.com/groeneai)). * Fix wrong query results from the `optimize_and_compare_chain` optimization when transitive conditions mix types with different comparison semantics, such as `Enum` with `String`, `Decimal` with `Float64`, or `FixedString` values of different widths. [#110621](https://github.com/ClickHouse/ClickHouse/pull/110621) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix wrong results for some read-in-order queries when the optimizer widens the sort-key prefix. [#110725](https://github.com/ClickHouse/ClickHouse/pull/110725) ([Groene AI](https://github.com/groeneai)). * Fix `CREATE` statements with a `COMMENT` clause to raise a syntax error when the comment string literal is missing, instead of silently accepting a misparsed query. For example, `CREATE VIEW v COMMENT AS SELECT 1` no longer creates a view without a comment. [#111159](https://github.com/ClickHouse/ClickHouse/pull/111159) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix wrong results from `SummingMergeTree` and `CoalescingMergeTree` with `FINAL` when `optimize_move_to_prewhere_if_final = 1` and the `WHERE` clause is an implicit-boolean condition on a sorting-key column, such as `WHERE k`. [#111342](https://github.com/ClickHouse/ClickHouse/pull/111342) ([Groene AI](https://github.com/groeneai)). * Fix spurious `Directory '...' was created concurrently with remote path '...'` exceptions for metadata-only operations on object-storage-backed databases and tables, such as `DROP DATABASE`. [#111353](https://github.com/ClickHouse/ClickHouse/pull/111353) ([Mikhail Artemenko](https://github.com/Michicosun)). * Data-lake REST catalogs now report stale credentials with the proper exception instead of silently treating them as a missing table or returning `UNKNOWN_TABLE`. [#111379](https://github.com/ClickHouse/ClickHouse/pull/111379) ([alesapin](https://github.com/alesapin)). * Fix `SQL SECURITY DEFINER` and `SQL SECURITY NONE` not being honored for parameterized views when the analyzer is disabled (`enable_analyzer = 0`). Previously, selecting from such a view as a user without privileges on the underlying table failed with `ACCESS_DENIED`, even though the same view works with the analyzer enabled. [#111458](https://github.com/ClickHouse/ClickHouse/pull/111458) ([Groene AI](https://github.com/groeneai)). * Reject a `Nullable(Nothing)` column, or any other type not allowed in tables, at `CREATE TABLE` time when the schema is inferred from the storage, for example in a columnless `remote`, `Remote`, or `Distributed` table over a view that does `SELECT NULL`, instead of persisting metadata that fails to load on the next server restart and blocks loading of unrelated tables. [#111487](https://github.com/ClickHouse/ClickHouse/pull/111487) ([Groene AI](https://github.com/groeneai)). * Fix `std::bad_variant_access` and incorrect `Iceberg` schema evolution for nested `Array(Map(...))` columns whose inner `Struct` is altered. [#111489](https://github.com/ClickHouse/ClickHouse/pull/111489) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix `ALTER MODIFY COLUMN` failing when converting a `Nullable` column to a different type with a `DEFAULT`, for example from `Nullable(UInt8)` to `String`, `LowCardinality(String)`, or `Array(UInt8)`. [#102156](https://github.com/ClickHouse/ClickHouse/pull/102156) ([DQ](https://github.com/il9ue)). * Fix `BAD_ARGUMENTS` (`Dictionary not found`) from `dictGet` after `DETACH DICTIONARY ... PERMANENTLY` is rejected with `HAVE_DEPENDENT_OBJECTS`; the dictionary now remains usable when the detach is rejected. [#105259](https://github.com/ClickHouse/ClickHouse/pull/105259) ([Groene AI](https://github.com/groeneai)). * ClickHouse-written data files for `Iceberg` tables now embed `Iceberg` field IDs in `Avro` and `ORC`, `ORC` `String` columns are written as `string` instead of binary, and optional complex `ORC` fields (`Array`, `Map`, `Tuple`) now preserve the correct `iceberg.required` metadata. This fixes silent `NULL` results after column renames and restores compatibility with spec-compliant readers such as Spark and `iceberg-java`. [#109994](https://github.com/ClickHouse/ClickHouse/pull/109994) ([Groene AI](https://github.com/groeneai)). [#111775](https://github.com/ClickHouse/ClickHouse/pull/111775) ([Groene AI](https://github.com/groeneai)). * Fix reading files on `plain_rewritable` disks after an existing path is rewritten with content of a different size: stale in-memory metadata could otherwise trigger `UNEXPECTED_END_OF_FILE`. [#110304](https://github.com/ClickHouse/ClickHouse/pull/110304) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix drift of the per-user memory tracker caused by system log records such as `query_log` and `query_views_log`, which could otherwise trigger a false `max_memory_usage_for_user` limit for users with continuous workloads. [#110708](https://github.com/ClickHouse/ClickHouse/pull/110708) ([Raúl Marín](https://github.com/Algunenano)). * Fix a rare race where a comment-only or settings-only `ALTER` on a `ReplicatedMergeTree` table could silently drop a column added by a concurrent `ALTER ... ADD COLUMN`, leaving a replica with outdated table metadata. [#111029](https://github.com/ClickHouse/ClickHouse/pull/111029) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix incorrect primary-key pruning on tables with a descending sorting-key suffix, such as `ORDER BY (g, r DESC)`, which could drop matching rows. [#111059](https://github.com/ClickHouse/ClickHouse/pull/111059) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix a peer-certificate memory leak on every TLS handshake when certificate verification is enabled. [#111425](https://github.com/ClickHouse/ClickHouse/pull/111425) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix mutations with a query parameter as the partition in `ALTER TABLE ... UPDATE/DELETE ... IN PARTITION {param:Type}`: the serialized partition value can now be parsed back correctly, so the `ALTER` no longer writes mutation entries that break table loading. [#111518](https://github.com/ClickHouse/ClickHouse/pull/111518) ([Michael Kolupaev](https://github.com/al13n321)). * Fix a macOS/aarch64 crash when a stack trace is captured on a coroutine or fiber stack, for example by the memory profiler. [#111656](https://github.com/ClickHouse/ClickHouse/pull/111656) ([Raúl Marín](https://github.com/Algunenano)). * Fix server crash when reading `Protobuf` data with `input_format_allow_errors_num > 0` and a valid message precedes a bad but skippable message in the same block. [#107739](https://github.com/ClickHouse/ClickHouse/pull/107739) ([Andrey Tsarevskiy | Андрей Царевский ](https://github.com/atsarevskiy)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a query over a `Distributed` table uses `_shard_num` or `shardNum` that becomes `Nullable`, for example under `FULL JOIN` with `join_use_nulls = 1`, together with `ORDER BY` and the analyzer. [#109798](https://github.com/ClickHouse/ClickHouse/pull/109798) ([Groene AI](https://github.com/groeneai)). * Fix wrong results and `UNKNOWN_ELEMENT_OF_ENUM` exceptions in `LEFT`/`RIGHT`/`FULL`/`ASOF JOIN` with `join_algorithm = 'full_sorting_merge'`, where non-joined rows could be filled with `0` instead of the column type default. [#111197](https://github.com/ClickHouse/ClickHouse/pull/111197) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` exception when a higher-order function's lambda body is a constant-condition `if` or `multiIf` that mixes a `Bool` literal branch with a `UInt8` comparison branch, for example `arrayFilter(p -> multiIf(false, true, p = 'ALL'), ['ALL'])`. [#111245](https://github.com/ClickHouse/ClickHouse/pull/111245) ([Groene AI](https://github.com/groeneai)). * A `{_partition_id}` placeholder in the path of `S3`, `AzureBlobStorage`, `URL`, and similar file-like engines without an explicit `partition_strategy` now implies the `wildcard` strategy again, restoring compatibility for pre-26.6 DDL that started failing with `BAD_ARGUMENTS`. [#111279](https://github.com/ClickHouse/ClickHouse/pull/111279) ([Nikita Fomichev](https://github.com/fm4v)). * Fix `NUMBER_OF_COLUMNS_DOESNT_MATCH` for `GROUP BY` queries over a `Merge` table wrapping a `Distributed` table when the grouping key is a function of a column and an aggregate reads the same column. [#111334](https://github.com/ClickHouse/ClickHouse/pull/111334) ([Groene AI](https://github.com/groeneai)). * `GRANT role TO role` on the same role is now rejected with `BAD_ARGUMENTS` instead of silently creating a self-referential entry in `system.role_grants`. [#103315](https://github.com/ClickHouse/ClickHouse/pull/103315) ([zxuhan7](https://github.com/zxuhan)). * Fix writes to `Iceberg` tables when the metadata file has no `refs` section. [#106896](https://github.com/ClickHouse/ClickHouse/pull/106896) ([Aleksandr Musorin](https://github.com/AVMusorin)). * Fixed `system.session_log` becoming unreadable after an Arrow Flight login: the `interface` column is an `Enum8`, but the `ArrowFlight` value was missing from its enumeration, so a login over the Arrow Flight protocol wrote the raw value `10` and any `SELECT` reading such a row threw the `Unexpected value 10 in enum` exception. [#110758](https://github.com/ClickHouse/ClickHouse/pull/110758) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a signed integer overflow in `toStartOfInterval` with subsecond intervals: for `DateTime64` values within one interval of the lower bound of `Int64`, the rounded result silently wrapped to a positive garbage value; now the function throws a `DECIMAL_OVERFLOW` exception. [#111370](https://github.com/ClickHouse/ClickHouse/pull/111370) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `readWKTPoint` and `readWKT` returning uninitialized (scalar) or previous-row (vectorized) coordinates for a dimension-tagged empty point such as `POINT M EMPTY`. Such inputs are now rejected with `CANNOT_PARSE_TEXT`, consistent with `POINT EMPTY`. [#111517](https://github.com/ClickHouse/ClickHouse/pull/111517) ([Groene AI](https://github.com/groeneai)). * Fix `Unexpected number of columns in result sample block` exceptions for `RIGHT` and `FULL JOIN` when the left side carries a duplicated same-named column, for example `1 AS b, 1 AS b`, that survives through `UNION ALL`. [#111554](https://github.com/ClickHouse/ClickHouse/pull/111554) ([Groene AI](https://github.com/groeneai)). * Fixed `CANNOT_CONVERT_TYPE` and `LOGICAL_ERROR` exceptions in distributed queries using the `-Tuple` combinator with a `RESPECT NULLS` or `IGNORE NULLS` modifier under nested combinators, for example `anyRespectNullsStateTuple(...) IGNORE NULLS`. [#111570](https://github.com/ClickHouse/ClickHouse/pull/111570) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed SQL syntax highlighting in Play on iPhone, where iOS Safari's automatic text inflation could misalign the highlight overlay. [#111683](https://github.com/ClickHouse/ClickHouse/pull/111683) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `UNKNOWN_QUERY_PARAMETER` when a parameterized view is called with a subquery-valued argument whose type is `Array`, `Tuple`, or `LowCardinality`, for example `SELECT * FROM v(param = (SELECT groupArray(x) FROM t))`. [#111790](https://github.com/ClickHouse/ClickHouse/pull/111790) ([Groene AI](https://github.com/groeneai)). * Fix `TYPE_MISMATCH` (`Key type for complex key ... does not match`) when a `JOIN` onto a dictionary uses a `Nullable`, `LowCardinality`, or `LowCardinality(Nullable)` join key while the dictionary key is not wrapped. The direct-join dictionary lookup now normalizes the key to the dictionary's declared key type, as `dictGet` and `dictHas` already do, and a `NULL` key never matches. [#111857](https://github.com/ClickHouse/ClickHouse/pull/111857) ([Groene AI](https://github.com/groeneai)). * With `fsync_part_directory = 1`, renaming a part directory from its temporary to its final name is now itself made durable by fsyncing the parent directory, and for moves across parents the source parent too. Previously only the moved directory was fsynced, so a power loss right after a part was committed could leave the part missing on disk and lose rows despite the setting being enabled. [#111861](https://github.com/ClickHouse/ClickHouse/pull/111861) ([Groene AI](https://github.com/groeneai)). * Fixed hangs in `randChiSquared`, `randStudentT`, `randFisherF`, and `randBinomial` with extreme parameters. Such queries could not be interrupted by `max_execution_time` or `KILL QUERY`. These parameters are now rejected even when `max_rand_distribution_parameter` or `max_rand_distribution_trials` is set to `0`, and `max_rand_distribution_trials` can no longer be raised above its default of `10^9` for `randBinomial`. [#111909](https://github.com/ClickHouse/ClickHouse/pull/111909) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results when the analyzer rewrites comparison chains such as `x = c1 OR x = c2 OR ...`, `x != c1 AND x != c2 AND ...`, or `has(const_array, x)` to `IN` and `NOT IN`. It also fixes `Illegal type ... of argument of function in` exceptions for expressions with `Dynamic`, `Array(Dynamic)`, or `JSON` structure. [#111924](https://github.com/ClickHouse/ClickHouse/pull/111924) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and `LOGICAL_ERROR` when selecting a `Nullable(Tuple(...))` column together with one of its null-carrying element subcolumns (`Variant`, `Dynamic`, or `LowCardinality`) from a `MergeTree` table. [#111949](https://github.com/ClickHouse/ClickHouse/pull/111949) ([Groene AI](https://github.com/groeneai)). * Reject native-protocol handshakes that use `USER_INTERSERVER_MARKER` for an unknown cluster, or for a cluster configured without a `` element. Previously such handshakes could enter interserver mode before authentication and read information about local tables. [#99675](https://github.com/ClickHouse/ClickHouse/pull/99675) ([Dan Anderson](https://github.com/MillaFleurs)). * Fix `JOIN` against `EmbeddedRocksDB`, `Redis`, and `KeeperMap` tables that could silently return zero rows when `DirectKeyValueJoin` is used with `String` or `Nullable(String)` keys. [#105253](https://github.com/ClickHouse/ClickHouse/pull/105253) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix coordinated refreshable materialized views in a `Replicated` database getting stuck after re-attach on a Keeper without the `MULTI_READ` feature flag; the view now stops cleanly with status `Disabled` instead of logging `Unexpected exception in refresh scheduling` forever. [#108890](https://github.com/ClickHouse/ClickHouse/pull/108890) ([Groene AI](https://github.com/groeneai)). * Fix the ZooKeeper client sending an overflowed `int32` length prefix for requests larger than `INT32_MAX` bytes. Oversized requests are now rejected on the client side. The limit is configurable via `max_request_size` under `` and is otherwise learned from Keeper. Closes [#109165](https://github.com/ClickHouse/ClickHouse/issues/109165). [#109190](https://github.com/ClickHouse/ClickHouse/pull/109190) ([Arsen Muk](https://github.com/arsenmuk)). * Fix `S3Queue`/`AzureQueue` ordered mode with persistent processing nodes: bucket locks are now refreshed during streaming, so TTL cleanup controlled by `persistent_processing_node_ttl_seconds` does not remove locks of a live processor. If lock ownership is nevertheless lost, it is detected and reported, and streaming recovers with a fresh file iterator. [#110292](https://github.com/ClickHouse/ClickHouse/pull/110292) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix wrong data and possible out-of-bounds reads when reading `String` columns with `string_serialization_version = 'with_size_stream'` after skipping leading rows, and when reading sub-objects of `JSON` columns stored with `object_shared_data_serialization_version = 'map'`. [#110471](https://github.com/ClickHouse/ClickHouse/pull/110471) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CREATE TABLE` and `ALTER TABLE` with the analyzer accepting a `DEFAULT` or `MATERIALIZED` expression that references a virtual column such as `_table` or `_database`. These expressions are now rejected at definition time instead of failing later on insert with `NOT_FOUND_COLUMN_IN_BLOCK`. [#111776](https://github.com/ClickHouse/ClickHouse/pull/111776) ([Groene AI](https://github.com/groeneai)). * Fix a race in executable UDFs, the `executable` table engine, and executable dictionaries where an `fcntl` on an already-closed file descriptor could corrupt an unrelated descriptor of a concurrent query and make queries hang indefinitely. [#111779](https://github.com/ClickHouse/ClickHouse/pull/111779) ([Raúl Marín](https://github.com/Algunenano)). * Creating deprecated `minmax` statistics now emits a warning instead of throwing an exception. [#111785](https://github.com/ClickHouse/ClickHouse/pull/111785) ([Han Fei](https://github.com/hanfei1991)). * Fix `Iceberg` tables written by ClickHouse being unreadable by external readers such as PyIceberg and Spark. Manifest-list and manifest `Avro` schemas now include the required `Iceberg` `field-id` properties. [#111786](https://github.com/ClickHouse/ClickHouse/pull/111786) ([Groene AI](https://github.com/groeneai)). * Regenerate materialized-into-part `block_number` and `block_offset` columns during part attach/detach/move/clone operations, so they no longer become stale after part manipulation. [#111801](https://github.com/ClickHouse/ClickHouse/pull/111801) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix the `query_plan_split_filter` optimization leaving an internal `__split_filter` column in one branch of `INTERSECT` or `UNION`, which could cause a `Block structure mismatch in IntersectOrExceptStep stream: different number of columns` exception. [#111930](https://github.com/ClickHouse/ClickHouse/pull/111930) ([Groene AI](https://github.com/groeneai)). * Fix `AMBIGUOUS_COLUMN_NAME` for queries that repeat a `JOIN ON` condition in `WHERE` with `join_use_nulls = 1`; such queries now return their result instead of failing. [#112007](https://github.com/ClickHouse/ClickHouse/pull/112007) ([Groene AI](https://github.com/groeneai)). * Fix DLF authentication in the `Paimon` REST catalog: every request after the first no longer fails with HTTP 401, and table-existence checks now detect HTTP 404 correctly. [#112015](https://github.com/ClickHouse/ClickHouse/pull/112015) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fix `LOGICAL_ERROR` during concurrent `Kafka` table cleanup in Keeper; creation or removal no longer fails if the table's Keeper path disappears after session expiry. [#112078](https://github.com/ClickHouse/ClickHouse/pull/112078) ([Groene AI](https://github.com/groeneai)). * Fix merges and mutations when the part-level min-max index is enabled for `_block_number` and `_block_offset` columns. [#109281](https://github.com/ClickHouse/ClickHouse/pull/109281) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix failed `INSERT` into `DeltaLakeLocal` or `deltaLake` tables leaving an unfinalized data file after writing had already started. [#109328](https://github.com/ClickHouse/ClickHouse/pull/109328) ([Groene AI](https://github.com/groeneai)). * Fix credential leaks in `SHOW CREATE`, `system.query_log`, server logs, and `EXPLAIN` output for `s3`/`s3Cluster`, the `S3`-backed engines, `BACKUP ... TO S3`, and secret arguments of `encrypt`, `decrypt`, and HMAC functions built by expressions or SQL UDFs. [#109768](https://github.com/ClickHouse/ClickHouse/pull/109768) ([Raúl Marín](https://github.com/Algunenano)). * Fix reading large `Parquet` files from uncompressed `STORED` zip entries via `s3` and `file`. [#110495](https://github.com/ClickHouse/ClickHouse/pull/110495) ([Eva Wuuu](https://github.com/Gudauu)). * Fix `EXPLAIN ANALYZE` hanging on queries with a streaming `FROM ... STREAM` read hidden in a subquery, CTE, or view. Such queries are now rejected with `NOT_IMPLEMENTED`, like top-level streaming reads already are. [#110935](https://github.com/ClickHouse/ClickHouse/pull/110935) ([Groene AI](https://github.com/groeneai)). * Reject vectors whose squared magnitude overflows to infinity when using a `vector_similarity` index with `i8` quantization. Previously, such vectors produced silent garbage. [#111085](https://github.com/ClickHouse/ClickHouse/pull/111085) ([Groene AI](https://github.com/groeneai)). * Fsync the text index files of a merged or materialized part when `min_rows_to_fsync_after_merge`, `min_compressed_bytes_to_fsync_after_merge`, or `fsync_part_directory` are set. Previously, these files were the only ones in the part left unsynced, so a power loss right after a merge could leave a committed but broken part and cause data loss. [#111335](https://github.com/ClickHouse/ClickHouse/pull/111335) ([Groene AI](https://github.com/groeneai)). * Fix path validation against `user_files`. [#111442](https://github.com/ClickHouse/ClickHouse/pull/111442) ([Alexander Tokmakov](https://github.com/tavplubix)). * Fix `SYSTEM REFRESH` on a stopped `RabbitMQ` table sometimes failing to consume the queued backlog. The refresh operation now waits for the AMQP delivery loop to start before it spends its one out-of-order streaming cycle. [#111480](https://github.com/ClickHouse/ClickHouse/pull/111480) ([Groene AI](https://github.com/groeneai)). * Restrict local object access to the `user_files` path. [#111483](https://github.com/ClickHouse/ClickHouse/pull/111483) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix a segfault due to out-of-bounds memory access when deserializing a malformed aggregate function state containing a `String` value. Such states are now validated and rejected. [#111606](https://github.com/ClickHouse/ClickHouse/pull/111606) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix integer overflow while parsing `system.zookeeper_info`. [#111629](https://github.com/ClickHouse/ClickHouse/pull/111629) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a `FINAL` query filters on a sorting-key column that is not in the `SELECT` list and the filter is moved to `PREWHERE`, for example `SELECT s FROM t FINAL WHERE k GROUP BY s` with `optimize_move_to_prewhere_if_final = 1`. [#111721](https://github.com/ClickHouse/ClickHouse/pull/111721) ([Groene AI](https://github.com/groeneai)). * Fix partitioned `INSERT` into `DeltaLake` with `allow_experimental_delta_lake_writes = 1` corrupting partition values that contain path characters and rejecting `NULL` partition values. Partition values are now percent-encoded into a single path segment, and `NULL` or empty-string values are written as `__HIVE_DEFAULT_PARTITION__` with a null `partitionValues` entry. [#111793](https://github.com/ClickHouse/ClickHouse/pull/111793) ([Groene AI](https://github.com/groeneai)). * Fix an exception and silently skipped data when reading a dynamic subcolumn through a `Buffer` table. [#111948](https://github.com/ClickHouse/ClickHouse/pull/111948) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `clickhouse-client` losing the current database after `USE {db:Identifier}`. If the client had to reconnect later, it silently switched the session to the `default` database. [#111982](https://github.com/ClickHouse/ClickHouse/pull/111982) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` exception (`Type mismatch when building statistics for column`) during `MergeTree` mutations after a metadata-only `ALTER MODIFY COLUMN` changes a column type while the column carries statistics. Columns actually rewritten by the mutation now have their statistics recomputed, and untouched columns keep their existing statistics. [#112009](https://github.com/ClickHouse/ClickHouse/pull/112009) ([Groene AI](https://github.com/groeneai)). * Fix `Paimon` table reads that could fail while the mutable `snapshot/LATEST` hint was being replaced concurrently. Invalid or stale hints are now ignored, and the snapshot is loaded from the catalog instead. [#112010](https://github.com/ClickHouse/ClickHouse/pull/112010) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fix reading `GeoParquet`, `Arrow`, and `ArrowStream` files whose WKT-encoded geometry column uses a non-uppercase type keyword such as `point(1 2)` or an empty container such as `LINESTRING EMPTY` or `POLYGON()`. This also fixes reading back files that store the output of `wkt` for an empty geometry. [#112020](https://github.com/ClickHouse/ClickHouse/pull/112020) ([Groene AI](https://github.com/groeneai)). * Fix query-level settings overrides not being applied to Azure native copy. [#112037](https://github.com/ClickHouse/ClickHouse/pull/112037) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix `query_masking_rules` rewriting the type of an `EPHEMERAL` column, which could make `CREATE TABLE` store the wrong type or fail. [#112048](https://github.com/ClickHouse/ClickHouse/pull/112048) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Table function parameters no longer accept aggregate states, which previously triggered a logical error because such parameters are unsupported. Closes [#112085](https://github.com/ClickHouse/ClickHouse/issues/112085). [#112087](https://github.com/ClickHouse/ClickHouse/pull/112087) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix wrong results for a `JOIN` whose `ON` expression contains a constant conjunct, for example `ON l.id = r.id AND CAST(NULL AS Nullable(UInt8))`. [#112155](https://github.com/ClickHouse/ClickHouse/pull/112155) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix wrong results for `hasAllTokens` and other multi-token text-index searches on builds that do not use `simdcomp`, such as `aarch64`, when `text_index_posting_list_apply_mode = 'lazy'`. [#112178](https://github.com/ClickHouse/ClickHouse/pull/112178) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a parameterized view is called with an argument that is not a `parameter = value` assignment. Such calls are now rejected with `UNKNOWN_QUERY_PARAMETER` on both execution paths. [#112194](https://github.com/ClickHouse/ClickHouse/pull/112194) ([Groene AI](https://github.com/groeneai)). * Fix parameterized views whose `SELECT` body uses `INTERSECT`, `EXCEPT`, or a `UNION` chain that mixes `UNION DISTINCT` with `UNION ALL` being misclassified as ordinary views, which made their metadata unloadable. [#112211](https://github.com/ClickHouse/ClickHouse/pull/112211) ([Groene AI](https://github.com/groeneai)). * Fix schema inference for the `TSV` family inferring `String` for decimal values written with a leading zero, such as `0.0` or `0.5`, and consuming such a first row as a header. Also fix `TSKV` inferring a numeric type for an escape-encoded value such as `x=1\x2E5`. Closes [#112105](https://github.com/ClickHouse/ClickHouse/issues/112105). [#112226](https://github.com/ClickHouse/ClickHouse/pull/112226) ([Groene AI](https://github.com/groeneai)). * Fix `system.s3_queue_settings` reporting incorrect values for `bucketing_mode`, `partitioning_mode`, `partition_regex`, and `partition_component`. [#112300](https://github.com/ClickHouse/ClickHouse/pull/112300) ([Bharat Nallan](https://github.com/bharatnc)). * Fix wrong results, wrongly ordered `ORDER BY` output, and `BAD_TYPE_OF_FIELD` exceptions when the primary key is an `Array` or `Tuple` and the query applies `plus`, `minus`, `multiply`, `divide`, or `intDiv` to it. [#112339](https://github.com/ClickHouse/ClickHouse/pull/112339) ([Groene AI](https://github.com/groeneai)). * Fix remote execution of queries containing `PASTE JOIN`, for example through a `Distributed` table or a `*Cluster` table function. ClickHouse no longer formats the query as the invalid syntax `ALL PASTE JOIN`. [#112404](https://github.com/ClickHouse/ClickHouse/pull/112404) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`Invalid number of rows in Chunk`) in `JoiningTransform` for a `LEFT JOIN` with unique right keys, a mixed `ON` condition, and a small `max_joined_block_size_rows`. [#106928](https://github.com/ClickHouse/ClickHouse/pull/106928) ([Groene AI](https://github.com/groeneai)). * Fixed the `_tags` virtual column for the `S3Queue` table engine: it was declared but never populated, so `SELECT _tags` always returned an empty map. It now returns the object tags, like the `S3` engine and the `s3` table function. [#108676](https://github.com/ClickHouse/ClickHouse/pull/108676) ([Groene AI](https://github.com/groeneai)). * Fix data loss of a column with no default expression when a merge runs concurrently with `ALTER TABLE... RENAME COLUMN`, or when the column's only values come from a lightweight `UPDATE`. The column could be dropped from the merged part, so all of its values read back as NULL. [#109356](https://github.com/ClickHouse/ClickHouse/pull/109356) ([Groene AI](https://github.com/groeneai)). * Fix skip indices being dropped from a data part (or `CHECK TABLE` failing with `UNEXPECTED_FILE_IN_DATA_PART`) after a full-part-rewrite mutation, such as `ALTER TABLE... DROP COLUMN` of a `MATERIALIZED` column, on a `Wide` part. [#109616](https://github.com/ClickHouse/ClickHouse/pull/109616) ([Groene AI](https://github.com/groeneai)). * Fixed two bugs on the `DateTime64` path of `toUTCTimestamp`/`fromUTCTimestamp` (and their `to_utc_timestamp`/`from_utc_timestamp` aliases) and `toTime64`. First, a signed integer overflow near the `Int64` boundary (now computed in `Int128` and clamped to the representable range). Second, a wrong result for negative fractional values near a timezone-offset boundary: the seconds split truncated toward zero, so `timezoneOffset()` inspected the next second and could pick the wrong side of a DST/offset change. [#109738](https://github.com/ClickHouse/ClickHouse/pull/109738) ([Groene AI](https://github.com/groeneai)). * Fix a logical error ("Cannot determine row level filter; 0 columns deleted, 0 columns added") when reading from a `Merge` table whose child table has a row policy whose filter is a bare existing column (for example `CREATE ROW POLICY... USING flag`). [#109843](https://github.com/ClickHouse/ClickHouse/pull/109843) ([Groene AI](https://github.com/groeneai)). * Fixed two errors raised when a query with `GROUP BY GROUPING SETS` (or a scalar or mutation subquery containing one) runs with `force_aggregation_in_order = 1`: `Trying to get name of not a column: ExpressionList` under the old analyzer (`enable_analyzer = 0`), and `Memory bound merging of aggregated results is not supported for grouping sets.` for a distributed query with `enable_memory_bound_merging_of_aggregation_results = 1` under the analyzer. [#109862](https://github.com/ClickHouse/ClickHouse/pull/109862) ([Groene AI](https://github.com/groeneai)). * Fix parsing of numeric `1`/`0` as `Bool` values inside container types (e.g. `Array(Bool)`, `Tuple(Bool,...)`). Previously `[1,0]` failed with `CANNOT_READ_ARRAY_FROM_TEXT` while `[true,false]` worked. [#109976](https://github.com/ClickHouse/ClickHouse/pull/109976) ([Groene AI](https://github.com/groeneai)). * Fixed a spurious `ILLEGAL_COLUMN` error from `ALTER TABLE... ADD COLUMN IF NOT EXISTS` when the column already exists at apply time. This happened when the same column was added twice with `IF NOT EXISTS` in one statement, or when a concurrent `ALTER` added the column between the prepare and apply phases. [#110080](https://github.com/ClickHouse/ClickHouse/pull/110080) ([Groene AI](https://github.com/groeneai)). * Fixed a wrong result where `length` (and the QBit dimension) on a `FixedString` column returned `0` instead of the fixed size `N` under parallel replicas when the query had a selective filter. [#110427](https://github.com/ClickHouse/ClickHouse/pull/110427) ([Groene AI](https://github.com/groeneai)). * Fixed `flipCoordinates` losing the `Geometry` type: for a `Geometry` argument the result is now typed `Geometry` again instead of the underlying `Variant(...)`, so it can be passed directly to functions like `areaCartesian`. [#110694](https://github.com/ClickHouse/ClickHouse/pull/110694) ([Groene AI](https://github.com/groeneai)). * Fixed `optimize_injective_functions_in_group_by` changing query results for `GROUP BY GROUPING SETS` and `GROUP BY... WITH TOTALS`. The setting is documented as result-preserving, but eliminating an injective function of a grouping key produced a wrong value (or dropped a row) for rows where that key is absent from the aggregated set (a `GROUPING SETS` non-member set) or for the `WITH TOTALS` row. The wrong value could coincide with a genuine key value, making a super-aggregate row indistinguishable from a real group in the same result set. [#110721](https://github.com/ClickHouse/ClickHouse/pull/110721) ([Groene AI](https://github.com/groeneai)). * Numeric `1`/`0` in JSON `Bool` parsing now honors the `allow_special_bool_values` setting. Previously JSON input such as `{"v":1}` for `Variant(Bool, UInt32)` was read as `Bool` even with `allow_special_bool_values_inside_variant = 0`, because the JSON `Bool` parsers ignored the setting and `Bool` has a higher `Variant` deserialize priority than integer types. [#110835](https://github.com/ClickHouse/ClickHouse/pull/110835) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `block.rows() == getRows()` (an out-of-bounds read and broken insert deduplication in release builds) when an `INSERT` flows through a dependent materialized view whose target is an `Alias` and whose inner query changes the number of rows, with a deduplicating table reachable behind the alias hop. [#111103](https://github.com/ClickHouse/ClickHouse/pull/111103) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a logical error `Block structure mismatch` (in debug and sanitizer builds) and an `Illegal types of arguments` error for set operations over compatible aggregate-state columns (e.g. `quantileState` and `quantilesState(0.9)`) nested inside container columns such as `Tuple`, `Array`, `Map`, `Nullable`, or `Variant`. [#111191](https://github.com/ClickHouse/ClickHouse/pull/111191) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `DISTINCT` with an ordered `LIMIT`/`OFFSET` returning wrong rows when the DISTINCT early-stop limit hint could not bound the head of the result: a negative `LIMIT` (returned the head instead of the tail), a fractional `LIMIT`/`OFFSET` (the fraction is only resolved after a full read), and a bare `OFFSET` with no `LIMIT` (the tail after the offset was dropped). For example `SELECT DISTINCT intDiv(x, 100) FROM t ORDER BY intDiv(x, 100) LIMIT -1` over a multi-part table returned the smallest distinct value instead of the largest. [#111326](https://github.com/ClickHouse/ClickHouse/pull/111326) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when a query has a `set` skip index and a predicate of the shape `(x AND ) OR y`, where the NULL is produced by a function (e.g. `toInt64OrNull('x')`, `nullIf(1, 1)`, `CAST(NULL AS Nullable(Int64))`). Such a value was wrapped by `__bitWrapperFunc` and propagated `NULL` instead of the "unknown" mask, so the set index could prune granules that actually match and the query returned fewer rows than it should. [#111604](https://github.com/ClickHouse/ClickHouse/pull/111604) ([Groene AI](https://github.com/groeneai)). * Fix `DELETE`/`UPDATE` on an Iceberg table silently removing the wrong rows (or throwing `NOT_FOUND_COLUMN_IN_BLOCK`) when the table contains data files written both before and after `ALTER TABLE... ADD COLUMN` and `optimize_move_to_prewhere` is enabled. [#111693](https://github.com/ClickHouse/ClickHouse/pull/111693) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` ("Invalid number of rows in Chunk") — and, under hardened builds, a related out-of-bounds abort — for `ORDER BY ... WITH FILL ... INTERPOLATE` over a `Distributed` table with two or more shards, over `clusterAllReplicas`, or over custom-key parallel replicas, including on empty results. `WITH FILL` is now applied only on the node producing the final result, which also fixes duplication of the generated fill rows across shards/replicas. [#111919](https://github.com/ClickHouse/ClickHouse/pull/111919) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Keep `randBinomial` usable with the degenerate probabilities `0` and `1` for any number of trials. [#112021](https://github.com/ClickHouse/ClickHouse/pull/112021) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes wrong results caused by a vector-search query poisoning the query condition cache. A `SELECT... WHERE ORDER BY LIMIT n` query over a table with a `vector_similarity` index could record granules as not matching ``, so a later ordinary `SELECT... WHERE ` on the same table silently returned fewer rows. Vector-search reads no longer write to the query condition cache. [#112086](https://github.com/ClickHouse/ClickHouse/pull/112086) ([Groene AI](https://github.com/groeneai)). * Fixes a bug where a `CREATE VIEW` or `CREATE FUNCTION` whose definition calls `substr`, `mid` or `byteSlice` in a form the `substring` grammar cannot re-parse is accepted, but the metadata ClickHouse writes for it is not valid SQL. Reading that definition afterwards fails with `SYNTAX_ERROR`, and at server startup it aborts metadata loading, so the server cannot start at all. Such definitions now round-trip, and existing metadata files of the form `substring(x, a, b, c)` become readable again, so an affected server starts up without manual intervention. [#112195](https://github.com/ClickHouse/ClickHouse/pull/112195) ([Groene AI](https://github.com/groeneai)). * Fixes a `TimeSeries` table whose local metadata diverged from Keeper making replica recovery in a `Replicated` database hang forever, so the replica never served the database again and stayed broken across restarts. The inner tables of such a table are now dropped while the recovery metadata transaction is still available, instead of being deferred to a background task that could not execute the `DROP` at all. [#112218](https://github.com/ClickHouse/ClickHouse/pull/112218) ([Groene AI](https://github.com/groeneai)). * Fixed reading a subcolumn of a `Nullable(Tuple(...))` column whose element is a non-nullable `LowCardinality(T)` when the same query also reads the parent subcolumn. On a Compact part the parent returned wrong values, or the query failed with `Unexpected return type from assumeNotNull`, or the server was killed by a `string_view` hardening assertion, because the extracted subcolumn was deserialized into a `LowCardinality(Nullable(T))` buffer whose substreams were then shared with the parent's read. [#112232](https://github.com/ClickHouse/ClickHouse/pull/112232) ([Groene AI](https://github.com/groeneai)). * Fixed a permanent server hang on macOS caused by `pthread_rwlock` losing wakeups when the sampling query profiler is enabled (Apple bug FB24027930): `pthread_rwlock` is now replaced with a signal-robust implementation on Darwin. [#112267](https://github.com/ClickHouse/ClickHouse/pull/112267) ([Raúl Marín](https://github.com/Algunenano)). * Fix `max_execution_time` and `KILL QUERY` being ignored while a query evaluates `geohashesInBox`, `arrayFold`, `sleep`, `base58Decode`, `h3PolygonToCells`, or `h3PolygonToCellsWithContainment`. Such queries could keep running for minutes after cancellation. This now also covers evaluations inside expressions stored in table metadata, such as sorting keys, skip indexes, and `PARTITION BY` keys. [#112273](https://github.com/ClickHouse/ClickHouse/pull/112273) ([Groene AI](https://github.com/groeneai)). [#113456](https://github.com/ClickHouse/ClickHouse/pull/113456) ([Groene AI](https://github.com/groeneai)). * Fixes a logical error `Cannot add step Expression to QueryPlan because it has incompatible header with root step JoinLazyColumnsStep` for a query with `FINAL`, a `PREWHERE` filter, a small `LIMIT` and `make_distributed_plan = 1`, when the selected columns are listed in an order that differs from the table's column order. Lazy materialization replaced a plan node without preserving that node's output header, which made the query plan inconsistent. [#112303](https://github.com/ClickHouse/ClickHouse/pull/112303) ([Groene AI](https://github.com/groeneai)). * Fixes writing Iceberg tables whose schema contains a tuple nested inside a tuple. The published table metadata replaced the inner tuple's field names with positional names `1`, `2`,..., so with the default `Parquet` format the `INSERT` failed with `INCORRECT_DATA` and with `Avro` the column could not be read back. [#112480](https://github.com/ClickHouse/ClickHouse/pull/112480) ([Groene AI](https://github.com/groeneai)). * Fix `ATTEMPT_TO_READ_AFTER_EOF` error when merging parts with a text index if one of the merged parts was empty, for example after a mutation that deleted all rows of the part. [#112490](https://github.com/ClickHouse/ClickHouse/pull/112490) ([Anton Popov](https://github.com/CurtizJ)). * Fixes wrong results and a logical error when an argument that must be constant, such as the scale of `toDecimal32` or the timezone of `toDateTime`, comes from a scalar subquery and the old analyzer is used. Such an argument was analyzed as if its value were zero or empty, so the declared result type disagreed with the computed value, and `CREATE VIEW` and `CREATE MATERIALIZED VIEW` persisted a wrong column type. [#112492](https://github.com/ClickHouse/ClickHouse/pull/112492) ([Groene AI](https://github.com/groeneai)). * Fixes a `LOGICAL_ERROR` ("Part... contains column... that is absent in table...") when a `ReplicatedMergeTree` replica mutates a part it fetched from another replica before applying its own pending `ALTER... ADD COLUMN`. Such a mutation is now postponed until the metadata change is applied. [#112510](https://github.com/ClickHouse/ClickHouse/pull/112510) ([Groene AI](https://github.com/groeneai)). * Fixed incorrect query results when using GROUP BY with ORDER BY and `arrayJoin` in the projection. [#101775](https://github.com/ClickHouse/ClickHouse/pull/101775) ([Yash ](https://github.com/Onyx2406)). * Fix a server crash (out-of-bounds access) in `S3Queue`/`AzureQueue` with `enable_hash_ring_filtering = 1` when a batch contained a non-processable file and the Keeper request to set the batch as processing failed at the same time. [#108977](https://github.com/ClickHouse/ClickHouse/pull/108977) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `Stream .variant_discr... is not found` (server abort in debug and sanitizer builds) when `ALTER TABLE... MODIFY COLUMN` changes a column into a type with dynamic subcolumns (for example `Variant` into `Dynamic`) on a table with Wide parts, and the part is later read or merged. [#110204](https://github.com/ClickHouse/ClickHouse/pull/110204) ([Groene AI](https://github.com/groeneai)). * Fix a signed-integer overflow (UBSan) when converting an extreme `DateTime64` value (e.g. `INT64_MIN`) to `Time`/`Time64` in a timezone with a negative offset. [#110451](https://github.com/ClickHouse/ClickHouse/pull/110451) ([Groene AI](https://github.com/groeneai)). * Fixed a MergeTree table with a `_part_offset` or commit-order projection becoming permanently unattachable after `allow_part_offset_column_in_projections` or `allow_commit_order_projection` was disabled and the table was detached or the server restarted. These two settings are CREATE-time gates, so they are no longer re-checked on ATTACH. [#110570](https://github.com/ClickHouse/ClickHouse/pull/110570) ([Groene AI](https://github.com/groeneai)). * With `analyzer_compatibility_join_using_top_level_identifier = 1`, an identifier in `JOIN... USING` can now be resolved from an alias defined on a subexpression inside the SELECT list (for example, `SELECT uniqExact(lower(x) AS id) FROM t1 JOIN t2 USING (id)`), matching the old analyzer behavior. Previously only top-level projection aliases were considered, and such queries failed with an `UNKNOWN_IDENTIFIER` exception even with the setting enabled. The error hint suggesting the setting is now also produced when the matching alias is nested. [#110739](https://github.com/ClickHouse/ClickHouse/pull/110739) ([Dmitry Novik](https://github.com/novikd)). * Fixes an issue where unauthenticated requests could cause ClickHouse to resolve hostnames supplied in forwarded client-address headers. ClickHouse now accepts only numeric IP addresses from these headers. Invalid values are ignored for forwarded-address quota attribution without DNS resolution. When `auth_use_forwarded_address` is enabled, invalid forwarded addresses are rejected during authentication. Rejections are logged at debug level. [#111060](https://github.com/ClickHouse/ClickHouse/pull/111060) ([Dmitriy Borisenko](https://github.com/otselnik)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` when a row policy and `additional_table_filters` reference the same column that is not otherwise selected by the query. [#111099](https://github.com/ClickHouse/ClickHouse/pull/111099) ([Groene AI](https://github.com/groeneai)). * Fix `UNKNOWN_IDENTIFIER` error for columns qualified by CTE name (`cte_name.column`) in queries stored in views when the analyzer is enabled. [#111386](https://github.com/ClickHouse/ClickHouse/pull/111386) ([Dmitry Novik](https://github.com/novikd)). * Fix a `LOGICAL_ERROR` (`Got read request from replica N for unknown stream...`) that could abort the server when reading with parallel replicas and the query is fully answered by a projection optimization (exact-count, minmax-count, or a stored aggregate/normal projection selecting no ranges) on the initiator. [#111689](https://github.com/ClickHouse/ClickHouse/pull/111689) ([Groene AI](https://github.com/groeneai)). * Fixed a signed integer overflow in the three-argument overload of `toStartOfInterval`: for an `origin` argument near the lower bound of `Int64` the function returned a wrapped-around value instead of reporting an error. [#112045](https://github.com/ClickHouse/ClickHouse/pull/112045) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a `LOGICAL_ERROR` exception (`Block structure mismatch in joined block stream`) on a multi-table join whose two sub-joins both produce no output columns, for example `SELECT count() FROM t1, t2, t3, t4 WHERE (t1.b = t2.b) AND (t3.a = t4.a)` with `query_plan_optimize_join_order_limit` set to `0` or `1`. [#112205](https://github.com/ClickHouse/ClickHouse/pull/112205) ([Groene AI](https://github.com/groeneai)). * Fixed the `values` table function rendering a non-literal `Bool` constant (e.g. `CAST(true, 'Nullable(Bool)')`) inserted into a `String`/textual column as `1`/`0` instead of `true`/`false`. [#112308](https://github.com/ClickHouse/ClickHouse/pull/112308) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix a `LOGICAL_ERROR` (`partitions_count > 0`) when partitioning `Iceberg` data chunks emptied by position deletes, e.g. during compaction of a partitioned `Iceberg` table with fully deleted data files. [#112432](https://github.com/ClickHouse/ClickHouse/pull/112432) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixed schema inference proposing a numeric type for text values the value parser cannot read back. Inference accepted a dangling exponent (`1e+`), a missing mantissa (`.`) or a repeated sign (`++inf`), which `readFloatTextPrecise` then rejected with `CANNOT_PARSE_NUMBER` when reading the inferred schema; on a `Dynamic` column this was an error at INSERT time. Inference now validates the number with the parser the reader actually uses. This also removes a case where the inferred type depended on `max_read_buffer_size`, and lets valid leading-zero exponent forms such as `0e5` infer `Float64` in TSV as they already do in CSV. Follows [#112226](https://github.com/ClickHouse/ClickHouse/issues/112226) and closes the exponent half of [#112105](https://github.com/ClickHouse/ClickHouse/issues/112105). [#112453](https://github.com/ClickHouse/ClickHouse/pull/112453) ([Groene AI](https://github.com/groeneai)). * Report the correct position for errors in PromQL queries. [#112494](https://github.com/ClickHouse/ClickHouse/pull/112494) ([Minh Vu](https://github.com/fallintoplace)). * Fixes a `LOGICAL_ERROR` (`Cannot add transform Resize to Pipes because it has no outputs`) when a query aggregating over a cluster runs with `serialize_query_plan = 1` and plan-based parallel replicas, caused by the merge step resizing the pipeline to zero streams. [#112506](https://github.com/ClickHouse/ClickHouse/pull/112506) ([Groene AI](https://github.com/groeneai)). * Fixes an unsigned underflow in the table name length check that allowed creating a table which could never be dropped. When the escaped database name reached 214 bytes, the limit computed for the `metadata_dropped` filename wrapped around, so `CREATE TABLE` was accepted and the subsequent `DROP TABLE` or `DROP DATABASE` failed with `File name too long`. Such a name is now rejected with `ARGUMENT_OUT_OF_BOUND`, the same error already returned just below that boundary. [#112527](https://github.com/ClickHouse/ClickHouse/pull/112527) ([Groene AI](https://github.com/groeneai)). * Fix the `Context has expired` exception when `getClientHTTPHeader` is used inside a subquery. [#112533](https://github.com/ClickHouse/ClickHouse/pull/112533) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `s3_allow_server_credentials_in_user_queries` incorrectly rejecting a query-supplied `extra_credentials(role_arn = '...')`. STS assume-role with an explicit role is now always allowed: it does not expose the server's own credentials to the query, and it is the documented way to grant access to a private S3 bucket. [#112603](https://github.com/ClickHouse/ClickHouse/pull/112603) ([Raúl Marín](https://github.com/Algunenano)). * Fixes the read counters (`read_rows`, `read_bytes`, `ProfileEvents['SelectedRows']`, `ProfileEvents['SelectedBytes']`) and the `READ_ROWS` and `READ_BYTES` quotas being over-charged for queries whose `ORDER BY` spills to disk. Rows re-read from the external sort temporary files were counted again as source reads, so a spilling query over-reported what it read and could be rejected with `QUOTA_EXCEEDED` even though it stayed within its quota. [#112645](https://github.com/ClickHouse/ClickHouse/pull/112645) ([Groene AI](https://github.com/groeneai)). * Fixes an Iceberg table whose write format is `Avro` serializing a field declared `"required": false` whose type is a `list`, `map` or `struct` as required, with no `["null", T]` union, so that the data file's own schema disagreed with the table metadata and other Iceberg readers saw a required field. Such a field is now written as the `["null", T]` union the Iceberg spec uses for an optional one. [#112648](https://github.com/ClickHouse/ClickHouse/pull/112648) ([Groene AI](https://github.com/groeneai)). * Fixes `DeltaLake` tables returning too few rows when the `WHERE` predicate contains a sub-expression the delta-kernel predicate translator cannot handle, positioned under a `NOT`. Such a sub-expression is now reported to delta-kernel as an explicit unknown predicate instead of as the end of the child iterator, which used to silently truncate the enclosing conjunction. [#112652](https://github.com/ClickHouse/ClickHouse/pull/112652) ([Groene AI](https://github.com/groeneai)). * Fixes heap corruption, which could crash the server at shutdown, caused by the `const` method `AccessRights::getFilters` modifying an access-rights tree shared by concurrent queries. [#112692](https://github.com/ClickHouse/ClickHouse/pull/112692) ([Groene AI](https://github.com/groeneai)). * Fixes a logical error in query formatting when an `ARRAY JOIN` clause carries no expressions. Such a clause is now rejected while parsing with `SYNTAX_ERROR`, consistent with an empty `USING`, instead of being accepted and formatted into text that cannot be parsed back. [#112699](https://github.com/ClickHouse/ClickHouse/pull/112699) ([Groene AI](https://github.com/groeneai)). * Fix a query hang in local execution of distributed query plans (`distributed_plan_execute_locally = 1`): the in-memory exchange reader could block the only pipeline thread before the sources producing its data were started. [#111762](https://github.com/ClickHouse/ClickHouse/pull/111762) ([Alexander Gololobov](https://github.com/davenger)). * Fix `Iceberg` fields declared with `"required": false` whose type is `list`, `map`, or `struct` being written as `REQUIRED` in the `Parquet` footer. Such fields are now written as `OPTIONAL`, matching the published `Iceberg` schema and other readers. [#112669](https://github.com/ClickHouse/ClickHouse/pull/112669) ([Groene AI](https://github.com/groeneai)). * Fixed a heap-use-after-free in the pipeline executor that could crash the server when a query removes processors at runtime (for example `FINAL` with lazy reads or external sort), found by the AST fuzzer. [#111017](https://github.com/ClickHouse/ClickHouse/pull/111017) ([Groene AI](https://github.com/groeneai)). * Fix a use-after-free (segmentation fault) when a query with a shared storage snapshot (setting `enable_shared_storage_snapshot_in_query`) strips data parts from the snapshot — e.g. a streaming subquery reading the same `MergeTree` table that another part of the query reads concurrently. [#111192](https://github.com/ClickHouse/ClickHouse/pull/111192) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject unsupported uses of `AggregateFunction` columns in TTL expressions at `CREATE TABLE` time (e.g. `TTL toDateTime(state)`) instead of failing later during TTL execution with `ILLEGAL_TYPE_OF_ARGUMENT`. This also covers states carried inside `Variant` alternatives and `Dynamic` values. Valid state-aware consumers such as `finalizeAggregation` are still accepted. [#107366](https://github.com/ClickHouse/ClickHouse/pull/107366) ([Ria Khatoniar](https://github.com/Ria-K912)). * Fix a `LOGICAL_ERROR` (`CTE '...' does not have query tree, but was not planned yet`) when a `MATERIALIZED` CTE is referenced from more than one join-tree position (for example two joined subqueries, or a joined subquery plus a scalar subquery in `WHERE`) in a query over a `Distributed` table. The query now executes instead of raising the exception (it aborted the server in debug/sanitizer builds). [#108924](https://github.com/ClickHouse/ClickHouse/pull/108924) ([Groene AI](https://github.com/groeneai)). * Preserve original key order in bucketed Map serialization to fix comparison operations that depend on it. [#109178](https://github.com/ClickHouse/ClickHouse/pull/109178) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed `made_current_at` in `system.iceberg_history` becoming `1970-01-01` for a retained snapshot after `ALTER TABLE... EXECUTE expire_snapshots` on an Iceberg table. Also fixed `system.iceberg_history` returning no rows for a table whose metadata has no `snapshot-log`. [#111781](https://github.com/ClickHouse/ClickHouse/pull/111781) ([Groene AI](https://github.com/groeneai)). * Fixes a `FixedString` text try-parse that leaves partially appended bytes in the column when parsing fails, which can cause a `Sizes of nested column and null map of Nullable column are not equal` logical error during serialization of `Variant`/`Array`/`Nullable` columns, or silently shifted result bytes. [#111908](https://github.com/ClickHouse/ClickHouse/pull/111908) ([Groene AI](https://github.com/groeneai)). * Fixes `replaceOne`, `replaceAll`, `replaceRegexpOne` and `replaceRegexpAll` ignoring `max_execution_time` and `KILL QUERY`. A single call now checks for cancellation while it works, so a long-running replacement over a large value or a large number of rows stops instead of holding a server thread until it finishes. Such a call has no partial result to return, so under `timeout_overflow_mode = 'break'` it raises the deadline as an error: always when constant-folded, and in the pipeline whenever this checkpoint is reached before the executor's between-block check, which still stops the query silently. [#112483](https://github.com/ClickHouse/ClickHouse/pull/112483) ([Groene AI](https://github.com/groeneai)). * Fixes a logical error and a wrong dependency record when an ordinary `VIEW` body references a table or dictionary through a query parameter, as in `CREATE VIEW v AS SELECT x FROM {db:Identifier}.t`. Such a view no longer records a referential dependency on a same-named table of the current database, so unrelated tables can be dropped again. [#112704](https://github.com/ClickHouse/ClickHouse/pull/112704) ([Groene AI](https://github.com/groeneai)). * Fixes a wrongly typed read of a `Nested` element subcolumn whose column was dropped and re-added. Selecting such a subcolumn together with its own parent column returned the whole element in the subcolumn's slot while the block still declared the element type, which produced arbitrary values in release builds and a logical error in debug and sanitizer builds. [#112769](https://github.com/ClickHouse/ClickHouse/pull/112769) ([Groene AI](https://github.com/groeneai)). * Fixed `Cannot read all data of type FixedString` when reading the `product_quantization_codebook` subcolumn of a column with a `Quantized('product',...)` codec. The per-part codebook is written after the data of all granules, so it is not delimited by marks, and a read whose mark range ended before the part's final mark could read it short. [#112818](https://github.com/ClickHouse/ClickHouse/pull/112818) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a segfault and silent data corruption when an `INSERT` into the `File` engine or the `file` table function appends to a non-empty file in a format that does not support appending, such as `Avro`. Writing through a file descriptor or a partitioned path bypassed the existing check, so the format prefix was suppressed and a second header was written after the existing bytes, leaving the file unreadable. Such an `INSERT` is now rejected with `CANNOT_APPEND_TO_FILE`, as it already is for a plain path. [#112839](https://github.com/ClickHouse/ClickHouse/pull/112839) ([Groene AI](https://github.com/groeneai)). * Fix `connectionId` raising `Context has expired` when it is used inside a subquery by capturing the client's connection id when the function is built. Closes [#112533](https://github.com/ClickHouse/ClickHouse/issues/112533). [#112870](https://github.com/ClickHouse/ClickHouse/pull/112870) ([ClickGap AI Bot](https://github.com/clickgapai)). * Fixed parsing of integers that do not fit into the target type in `Poco::NumberParser`, which is used to read JSON numbers. Such a number was silently parsed as a wrong value (for example, `18446744073709551617` became `1`) instead of being rejected, and a `UInt64` number above the `Int64` maximum did not survive an AST JSON round trip. [#112904](https://github.com/ClickHouse/ClickHouse/pull/112904) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `IS DISTINCT FROM` for `Array` and `Map` values compared with `NULL`. Closes [#103042](https://github.com/ClickHouse/ClickHouse/issues/103042). [#103162](https://github.com/ClickHouse/ClickHouse/pull/103162) ([yanglongwei](https://github.com/ylw510)). * Prevent exhausted retries for leader-executed replicated distributed DDL from permanently blocking subsequent `ON CLUSTER` DDL queries. [#108505](https://github.com/ClickHouse/ClickHouse/pull/108505) ([Ahaan Limaye](https://github.com/ahaanlimaye)). * Fix a `LOGICAL_ERROR` ("Trying to extract chunk from ChunkBuffer before all inputs are finished") when a set operation (`INTERSECT` / `UNION ALL` / `EXCEPT`) combines branches that use correlated subqueries and `correlated_subqueries_default_join_kind = 'left'`. [#108554](https://github.com/ClickHouse/ClickHouse/pull/108554) ([Groene AI](https://github.com/groeneai)). * Conversion of out-of-range floating-point values (`BFloat16`, `Float32`, `Float64`) and of `UInt64` values above `Int64::max()` to `DateTime` and `Time` now reliably saturates to the range boundaries on all architectures (previously the result of an out-of-range conversion was architecture-dependent and could wrap around on x86-64), and non-finite values (`NaN`, `inf`) now throw an exception instead of producing an arbitrary result. The same fix is applied to the float-to-`Date` conversion path of `toDate` and to the conversions of `UInt64` and of wide integers (`(U)Int128`, `(U)Int256`) to `Date` and `Date32`, which could return a pre-epoch day instead of saturating. Conversion of a number to `Time` now saturates to the range of the type for every numeric source type (previously values from narrow unsigned or wide integer sources were stored unclamped and clamped only when printed), and the accurate cast to `Time` no longer rejects the negative values that `Time` supports, nor silently saturates the values that do not fit into it. The accurate cast of a number to `Date32` (`accurateCast`, `accurateCastOrNull`) no longer silently saturates an out-of-range or non-finite value, but throws or returns `NULL` as it already did for `Date` and `DateTime`, and the `OrDefault` flavours return the default value of `Date32` for such an input. The accurate cast of a non-integral floating-point number to `Date`, `Date32`, `DateTime` or `Time` is now rejected instead of being silently truncated. Also fixed the `toDateTime32` function documentation, which mistakenly showed `toDateTime64` examples and described the `DateTime64` value range. [#110459](https://github.com/ClickHouse/ClickHouse/pull/110459) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ALTER TABLE... DETACH PARTITION/PART` now fsyncs the clone it creates under `detached/` (the part subtree plus the directory chain up to the disk root) when the table's `fsync_part_directory` setting is enabled, for both Full (directory) and Packed part storage. Previously the clone was never fsynced while the empty covering part that removes the rows from the active set was, so a power loss right after the acknowledgement could destroy the detached data on btrfs (the un-synced clone directory entries roll back, leaving `detached/` empty and nothing to `ATTACH`). Gated on the existing `fsync_part_directory` setting; default behavior is unchanged. [#111426](https://github.com/ClickHouse/ClickHouse/pull/111426) ([Groene AI](https://github.com/groeneai)). * Fix `CREATE OR REPLACE` leaving the replaced object behind under an internal name when the caller lacks the drop privilege on it. [#111466](https://github.com/ClickHouse/ClickHouse/pull/111466) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed a crash when running a `STREAM` read (`enable_streaming_queries = 1`) over a table with more than one partition and a row policy defined on it. In debug and sanitizer builds it failed with the `Filter column... not found in DAG outputs` logical error; in release builds it could segfault. [#111572](https://github.com/ClickHouse/ClickHouse/pull/111572) ([Groene AI](https://github.com/groeneai)). * Fix `count()` on Iceberg tables whose snapshot summary is inaccurate or corrupted: the trivial count optimization now derives exact row counts from the manifest files, and falls back to a real scan when row-level deletes make metadata counts inexact. Also fix Iceberg `OPTIMIZE TABLE` writing incorrect manifest-list counts and lineage, and make `totalBytes` metadata robust against corrupted summaries. [#111617](https://github.com/ClickHouse/ClickHouse/pull/111617) ([Melvyn Peignon](https://github.com/melvynator)). * Fix a DEFAULT column missing in a part being filled with the type default instead of the DEFAULT expression when the PREWHERE condition was split into multiple read steps. Closes [#111757](https://github.com/ClickHouse/ClickHouse/issues/111757). [#111795](https://github.com/ClickHouse/ClickHouse/pull/111795) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `Expected UInt64 column for __grouping_set, got BLOB` and `ColumnBLOB should be converted to a regular column before usage` logical errors on a distributed query executed on a shard with `enable_parallel_blocks_marshalling` (enabled by default), for example with `GROUP BY GROUPING SETS` over a cluster that has a local replica, or when reading a `Merge` table that spans a local and a distributed table. `BlocksMarshallingStep` is no longer added to plans whose blocks are consumed in the same process. [#111997](https://github.com/ClickHouse/ClickHouse/pull/111997) ([Groene AI](https://github.com/groeneai)). * Operations now fail when corrupt MergeTree statistics are loaded instead of ignoring the error. [#112201](https://github.com/ClickHouse/ClickHouse/pull/112201) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fixes a logical error when PREWHERE is used on a Merge table over another Merge table or a materialized view whose column type differs. Also fixes a row policy being silently ignored on tables that read from remote servers (a Distributed table, or a table wrapping one), which could expose rows the policy was meant to hide; such queries are now rejected with an ILLEGAL_PREWHERE error. Define these policies on the underlying local tables on each remote server instead; note that such a policy is still not applied to reads shipped with `serialize_query_plan = 1`, which is a separate, pre-existing bug. [#112326](https://github.com/ClickHouse/ClickHouse/pull/112326) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixes `connection_pool_max_wait_ms` so that its documented and default value `0` means an infinite timeout. A query that found the connection pool full used to retry in a tight loop, consuming CPU and logging `No free connections in pool. Waiting 0 ms.` on every iteration. It now waits until a connection is returned to the pool. Closes [#112053](https://github.com/ClickHouse/ClickHouse/issues/112053). [#112380](https://github.com/ClickHouse/ClickHouse/pull/112380) ([Groene AI](https://github.com/groeneai)). * Fixes silent loss of acknowledged Iceberg writes on HDFS: an Iceberg commit publishes each new metadata file with a conditional write, but `HDFSObjectStorage` dropped the condition and performed a plain overwrite, so concurrent writers were all acknowledged while one writer's snapshot became unreachable. HDFS now refuses a conditional write instead, so storage-managed Iceberg writes on HDFS, `CREATE TABLE... ENGINE = IcebergHDFS(...)` with an explicit column list included, now fail. Reads are unaffected. [#112437](https://github.com/ClickHouse/ClickHouse/pull/112437) ([Groene AI](https://github.com/groeneai)). * Fix an out-of-bounds read (a crash in release builds) in insert deduplication when an `INSERT` flows through a materialized view whose inner query changes the number of rows and the deduplicating target table is partitioned. `DeduplicationInfo::filterToPartition` attributed each deduplication token to partitions by walking the token's source-row range over a partition selector sized to the (smaller) view-output block, reading past its bounds. It now keeps every token in every partition for any materialized-view target — where the source-row-to-partition mapping no longer exists after the row count changed — instead of the out-of-bounds walk (and instead of the `NOT_IMPLEMENTED` that a later guard raised for the same insert). [#112649](https://github.com/ClickHouse/ClickHouse/pull/112649) ([Sema Checherinda](https://github.com/CheSema)). * Fixes a logical error (which aborts in debug and sanitizer builds) when `finalizeAggregation` is applied to a column whose aggregate states come from different functions that share a state representation but finalize to different types — for example `quantileState` and `quantilesState(0.9)` brought together by a `UNION` or produced via `arrayReduce`. Such queries now raise a normal error instead of crashing. [#112662](https://github.com/ClickHouse/ClickHouse/pull/112662) ([Zain Ul Abidin](https://github.com/zainulabidin302)). * Fix `DELETE` on `KeeperMap` tables deleting only the first `max_block_size` matched rows while reporting the mutation as successfully completed. Also fix `DELETE` with `keeper_map_strict_mode = 1` not being atomic: it could fall back to unversioned removals and delete rows that were concurrently updated, and it applied the matched rows block by block so a conflict could leave the delete partially applied. [#112777](https://github.com/ClickHouse/ClickHouse/pull/112777) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ALTER` on `MergeTree` tables being applied while an incomplete `RENAME COLUMN` alter-mutation is still in progress. That could update the storage metadata incompatibly and block later mutations completely. [#112783](https://github.com/ClickHouse/ClickHouse/pull/112783) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fixed server startup and `ATTACH` failing with `WITH RECURSIVE is not supported with the old analyzer` for a view with a recursive CTE when `enable_analyzer = 0` is the server default. [#112784](https://github.com/ClickHouse/ClickHouse/pull/112784) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixes a logical error when a `Join` table engine is used on the right side of a JOIN whose `ON` section contains a condition referencing columns of both tables, for example `ON (t.key = j.key) AND (t.a < j.a)`. Such a query is now rejected with a clear exception instead of raising `required columns:... but not found any in left table` or `std::bad_variant_access`. [#112815](https://github.com/ClickHouse/ClickHouse/pull/112815) ([Groene AI](https://github.com/groeneai)). * Fixes wrong results for a `JOIN` whose `ON` clause carries a mixed condition, that is a cross-side non-equi residual such as `ON (t1.key = t2.key) AND (t1.a * 10 < t2.a)`. Only the hash family evaluates such a condition, but `full_sorting_merge`, `partial_merge` and the direct key-value join claimed they could run these joins and then silently ignored the residual, returning extra rows. They now decline, so a hash algorithm is used instead. [#112831](https://github.com/ClickHouse/ClickHouse/pull/112831) ([Groene AI](https://github.com/groeneai)). * Fix `ReadBufferFromEncryptedFile: Wrong file position` logical error when reading a `Compact` part from an `encrypted` disk with direct I/O, which made merges get stuck in `system.replication_queue`. [#112943](https://github.com/ClickHouse/ClickHouse/pull/112943) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a quadratic slowdown when parsing a PromQL query with a long tail of unrecognized characters (e.g. a `FixedString` value padded with NUL bytes): the parse of a megabyte-sized invalid input used to keep a thread busy for tens of minutes and was uncancellable, because it happens during query analysis. Now the parser stops at the first error. [#112944](https://github.com/ClickHouse/ClickHouse/pull/112944) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject PromQL durations whose units are repeated or not ordered from longest to shortest. [#112954](https://github.com/ClickHouse/ClickHouse/pull/112954) ([Minh Vu](https://github.com/fallintoplace)). * Fix a hang when writing `Parquet` with `output_format_parquet_parallel_encoding` enabled (the default) and `max_threads` greater than 1. If the encoder failed to schedule an additional thread, its live-thread counter underflowed and the write could stop making progress permanently instead of finishing or reporting an error. [#112959](https://github.com/ClickHouse/ClickHouse/pull/112959) ([Groene AI](https://github.com/groeneai)). [#113170](https://github.com/ClickHouse/ClickHouse/pull/113170) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `geohashesInBox` spending an unbounded amount of time on a box of zero area, such as `geohashesInBox(0., 0., 180., 0., 12)`. Such a query returned the correct single geohash but burned roughly 5e8 empty loop iterations per row and ignored both `max_execution_time` and `KILL QUERY`. [#112976](https://github.com/ClickHouse/ClickHouse/pull/112976) ([Groene AI](https://github.com/groeneai)). * Fixed undefined behavior (null pointer passed to `memcpy`) in `replaceRegexpOne`/`replaceRegexpAll` when the replacement string substitutes a capturing group that did not participate in the match, e.g. `replaceRegexpAll('abc', '(a)|(b)', '\\2')`. [#113012](https://github.com/ClickHouse/ClickHouse/pull/113012) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject a `SETTINGS` change marked as written without a value when it carries a value other than `true`, and never elide the value of such a change when formatting a query. Previously a crafted AST JSON payload could execute a `Bool` setting with `false` while `system.query_log` and `formatQueryFromJSON` showed the valueless form. [#113025](https://github.com/ClickHouse/ClickHouse/pull/113025) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed undefined behavior and a silent scheduling error when a background task is scheduled with a very large delay, for example by a refreshable materialized view with a huge `REFRESH AFTER` period. Delays are now bounded to the largest representable value instead of overflowing. [#113041](https://github.com/ClickHouse/ClickHouse/pull/113041) ([Groene AI](https://github.com/groeneai)). * Fixed an out-of-bounds write when reading a `Parquet` `DECIMAL` column whose physical type is wider than the type its declared precision maps to, for example `DECIMAL(9, 2)` stored as physical `INT64`. Such files are valid `Parquet` and other writers produce them, but the reader sized the destination column from the declared precision while the decoder wrote the physical width, corrupting memory. Reading such a file now also raises `DECIMAL_OVERFLOW` when a value does not fit the declared precision, instead of returning corrupted data, and reads losslessly with a type hint at least as wide as the physical type. [#113046](https://github.com/ClickHouse/ClickHouse/pull/113046) ([Groene AI](https://github.com/groeneai)). * Fixed `intDiv` on a non-constant column with the constant divisor -1 silently wrapping the division of the minimal signed number (e.g. `intDiv(-9223372036854775808, -1)`) instead of throwing the `ILLEGAL_DIVISION` exception like the other execution paths do. The silent wrap could also produce wrong `ORDER BY`/`DISTINCT` results, because the read-in-order optimization correctly treats `intDiv` by a negative constant as monotonic, and the wrapped value violated the resulting sort order. [#113048](https://github.com/ClickHouse/ClickHouse/pull/113048) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an incompatibility that text indexes created in ClickHouse 26.3 with `unicode_word` tokenizer cannot be loaded in later versions. [#113061](https://github.com/ClickHouse/ClickHouse/pull/113061) ([Robert Schulze](https://github.com/rschu1ze)). * Fixed a `Pipeline stuck` logical error in queries that scatter data by partition, such as a window function with `PARTITION BY` or a join with `join_algorithm = 'parallel_full_sorting_merge'`, when one shard's downstream finished while a block was only partially distributed. [#113190](https://github.com/ClickHouse/ClickHouse/pull/113190) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when a hash join has a single `LowCardinality` key of a type wider than 8 bytes (`UInt128`, `Int128`, `UInt256`, `Int256`, and their `Nullable` variants). Such a join silently returned rows whose key values are not equal. [#113230](https://github.com/ClickHouse/ClickHouse/pull/113230) ([Groene AI](https://github.com/groeneai)). * Fix `ALTER TABLE ... ADD PROJECTION` over a table-level `ALIAS` column leaving the table unusable, with subsequent `INSERT` failing with `UNKNOWN_IDENTIFIER` (`Missing columns`), when the session had `optimize_respect_aliases = 0`. [#109091](https://github.com/ClickHouse/ClickHouse/pull/109091) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` and `LOGICAL_ERROR` exceptions in a `direct` `JOIN` over a `MergeTree` right table, and related aborts in debug and sanitizer builds. [#109932](https://github.com/ClickHouse/ClickHouse/pull/109932) ([Groene AI](https://github.com/groeneai)). * Fix comparisons that previously threw `ILLEGAL_TYPE_OF_ARGUMENT` when the operands have no least common supertype. Arrays such as `Array(Int64)` vs `Array(UInt64)` and `Array(Int256)` vs `Array(UInt256)` can now be compared lexicographically with `=`, `!=`, `<`, `<=`, `>`, `>=`, `IS DISTINCT FROM`, and `IS NOT DISTINCT FROM`. `isDistinctFrom` and `isNotDistinctFrom` now also compare supported non-array numeric and decimal pairs by value instead of throwing, and equality comparison for arrays nested inside `Tuple(Nullable(...))` is also fixed. Closes [#33897](https://github.com/ClickHouse/ClickHouse/issues/33897). [#110245](https://github.com/ClickHouse/ClickHouse/pull/110245) ([Diego Gomes Tomé](https://github.com/diegomestre2)). * Fix a `LOGICAL_ERROR` (`Bad cast from ColumnString to ColumnLowCardinality`) during primary-key index analysis when a `LowCardinality` key column is wrapped in a nested `CAST` chain that re-introduces `LowCardinality`, e.g. `WHERE CAST(CAST(s, 'LowCardinality(String)'), 'String') < '5'`. In debug and sanitizer builds this aborted the server; in release it failed the query. [#111050](https://github.com/ClickHouse/ClickHouse/pull/111050) ([Groene AI](https://github.com/groeneai)). * Fix wrong results for a `FULL JOIN` on a `Join` engine table with `USING` when the left key is `Nullable` and the storage key is not: an unmatched left row with a `NULL` key returned `0` instead of `NULL`. [#111371](https://github.com/ClickHouse/ClickHouse/pull/111371) ([Groene AI](https://github.com/groeneai)). * Fix stale and deleted row versions being visible when a `MaterializedPostgreSQL` database is read through a `Merge` table. Such reads now go through the `MaterializedPostgreSQL` wrapper, with the same `FINAL` and `_sign = 1` filtering as direct reads. [#112382](https://github.com/ClickHouse/ClickHouse/pull/112382) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Bad cast from ColumnNullable to ColumnString` when a mutation that rewrites the whole part runs on a part whose column type is older than the one in the table metadata, for example after `ALTER TABLE ... MODIFY COLUMN` followed by `ALTER TABLE ... MATERIALIZE PROJECTION`. [#112501](https://github.com/ClickHouse/ClickHouse/pull/112501) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix row-based input formats ignoring `KILL QUERY` and `max_execution_time` for minutes while parsing a block. They now check for cancellation every 8192 rows, so a killed `INSERT` (including a background async-insert flush) stops promptly instead of running to the end of the block. [#112646](https://github.com/ClickHouse/ClickHouse/pull/112646) ([Groene AI](https://github.com/groeneai)). * Fix `RENAME TABLE` checking the maximum table name length against the source database instead of the destination one. Renaming a table into a database with a long name could produce a table that `DROP TABLE` could not remove, and renaming a table out of such a database was wrongly rejected. [#113019](https://github.com/ClickHouse/ClickHouse/pull/113019) ([Groene AI](https://github.com/groeneai)). * Fix `UNKNOWN_IDENTIFIER` when selecting from a `VIEW` whose declared column types differ from the types its inner query produces, for example `SELECT sum(length(arr)) FROM v` where `v` declares `arr Array(UInt8)` over a `String` column. With the default `optimize_functions_to_subcolumns = 1`, the optimizer rewrote `length(arr)` into a read of the `arr.size0` subcolumn and forwarded that name into the inner query, which has no such subcolumn. [#113057](https://github.com/ClickHouse/ClickHouse/pull/113057) ([Groene AI](https://github.com/groeneai)). * Fixed an exception (in release builds, an error that prevents the server from starting) when a mutation predicate uses the `merge` table function without an explicit database name. The database of the mutated table is now used, the same way as for table names. [#113185](https://github.com/ClickHouse/ClickHouse/pull/113185) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results and a `LOGICAL_ERROR` when reading a subcolumn of a dropped-and-re-added `Nested` member together with a present member of the same group, on a `MergeTree` table with `share_nested_offsets = 1`. The present members returned values from a later block whenever the read spanned more than one block. [#113225](https://github.com/ClickHouse/ClickHouse/pull/113225) ([Groene AI](https://github.com/groeneai)). * Fix a hang of `ALTER`, `OPTIMIZE`, `TRUNCATE`, and partition-manipulation statements with `alter_sync = 2` on `ReplicatedMergeTree` when the table starts shutting down while the statement waits for another replica, for example because of a concurrent `DROP DATABASE ... SYNC`. Such a statement now fails with `UNFINISHED` and also honors `max_execution_time` and `KILL QUERY`. [#113228](https://github.com/ClickHouse/ClickHouse/pull/113228) ([Groene AI](https://github.com/groeneai)). * Do not write ANSI escape sequences to `INTO OUTFILE` when `stdout` is a TTY. [#113252](https://github.com/ClickHouse/ClickHouse/pull/113252) ([Azat Khuzhin](https://github.com/azat)). * Fix a rare use-after-free in the async insert queue when different entries have identical deadlines. [#113363](https://github.com/ClickHouse/ClickHouse/pull/113363) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix silently empty tables when more than one `ENGINE = Backup` database is created from the same backup, or when such a database is recreated under a different name. The storage policy that backs a `Backup` database is now identified by a name that also covers the local database name, so each database gets the disk built for its own data path. [#113378](https://github.com/ClickHouse/ClickHouse/pull/113378) ([Groene AI](https://github.com/groeneai)). * Fix rows never being deleted when a `MergeTree` table has two or more `TTL ... DELETE WHERE` rules whose time expression is identical and which differ only in the `WHERE` condition. After a TTL delete merge, the part's `rows_where_ttl_info` was reset to zero, so no further TTL merge was ever scheduled for it and the expired rows were retained indefinitely with no error or log message. Closes [#113116](https://github.com/ClickHouse/ClickHouse/issues/113116). [#113384](https://github.com/ClickHouse/ClickHouse/pull/113384) ([Groene AI](https://github.com/groeneai)). * A query over a `Merge` table (or the `merge` table function) that matches many tables now reacts to `KILL QUERY` and `max_execution_time` while the per-table query plans are being built, instead of only after the planning of all children has finished. With `timeout_overflow_mode = 'break'`, it now also stops building query plans for the remaining matched tables and returns the partial result without an error, as that mode documents. [#113415](https://github.com/ClickHouse/ClickHouse/pull/113415) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#113642](https://github.com/ClickHouse/ClickHouse/pull/113642) ([Groene AI](https://github.com/groeneai)). * Fix `clickhouse-client` exiting with code `139` instead of `0` after cancelling a query with Ctrl+C, or after an `INSERT` that reads data from stdin. [#113431](https://github.com/ClickHouse/ClickHouse/pull/113431) ([Groene AI](https://github.com/groeneai)). * Allow the `preprocessor` and `postprocessor` expressions of a text index to reference `ALIAS` columns. [#110014](https://github.com/ClickHouse/ClickHouse/pull/110014) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixed several cases where parallel replicas returned duplicated rows: `distributed_aggregation_memory_efficient` was ignored by plan-based parallel replicas, so the buckets of the memory-efficient merge arrived out of order and were merged twice; and a shard was read from all of its replicas although the data was not split between them, when parallel replicas were not applicable to the query or when `parallel_replicas_mode` requested custom-key filtering without `parallel_replicas_custom_key`. [#112942](https://github.com/ClickHouse/ClickHouse/pull/112942) ([Nikita Taranov](https://github.com/nickitat)). * Fixed `CREATE TABLE ... AS SELECT` becoming unkillable when its populating `SELECT` failed and `database_atomic_wait_for_drop_and_detach_synchronously` was enabled: the internal cleanup `DROP` of the temporary table waited for the background drop queue and ignored both `KILL QUERY` and `max_execution_time`. [#113504](https://github.com/ClickHouse/ClickHouse/pull/113504) ([Groene AI](https://github.com/groeneai)). * Fixed `ALTER TABLE ... EXECUTE remove_orphan_files` deleting a live `metadata/version-hint.text` file of an `Iceberg` table. Its reachable-set entry was built by string concatenation assuming a trailing slash on a value that never has one, so the real hint file was never recognized as reachable and was removed once older than the `older_than` threshold. With `iceberg_use_version_hint = 1`, the table then failed to read at all. [#113548](https://github.com/ClickHouse/ClickHouse/pull/113548) ([Groene AI](https://github.com/groeneai)). * Fixed a parameterized view losing its database when a query referencing it without qualification was sent to a shard, so the shard resolved the name against its own default database. Depending on what that database contained, the query either failed with `UNKNOWN_FUNCTION` or silently returned rows from a different view. Affects `IN`, `JOIN`, and `remote` through a `Distributed` table. [#113550](https://github.com/ClickHouse/ClickHouse/pull/113550) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` when a query filters `system.databases` on the `name` column with a subquery inside `indexHint`, for example `SELECT count() FROM system.databases WHERE indexHint(name IN (SELECT 'foo'))`. [#113556](https://github.com/ClickHouse/ClickHouse/pull/113556) ([Groene AI](https://github.com/groeneai)). * Fixed an endless loop in background merges and `OPTIMIZE` when `merge_max_block_size_bytes` was set below the `byteSize()` of an empty column. An empty `LowCardinality` column still carries its dictionary, so the byte limit could be reached before a single row was merged; the merge then produced empty blocks forever, burning 100% of a CPU core and never releasing its background-pool slot. [#113585](https://github.com/ClickHouse/ClickHouse/pull/113585) ([Groene AI](https://github.com/groeneai)). * Fixed a query over a `Variant` column ignoring `max_execution_time` and `KILL QUERY`. Calling a function with several `Variant` arguments resolved the function once per alternative of every argument, and neither of the two loops doing that work checked for cancellation, so such a query could not be stopped until it finished. [#113612](https://github.com/ClickHouse/ClickHouse/pull/113612) ([Groene AI](https://github.com/groeneai)). * Fixed `Unknown compression method` for the `auto` and `none` compression hints when written in any letter case other than all-lowercase, for example `file('data.csv.gz', 'CSV', 'x String', 'AUTO')`. Codec names such as `GZIP` were already accepted case-insensitively; the two special hints now behave the same way. [#113631](https://github.com/ClickHouse/ClickHouse/pull/113631) ([Groene AI](https://github.com/groeneai)). * Fix constant folding under the new analyzer changing the type of a constant `Geometry` value. `Geometry` is a named `Variant` whose alternatives can share the same storage layout (`LineString`/`Ring`, `MultiLineString`/`Polygon`, and empties), so two folded constants that differed only in their alternative could collapse into one, e.g. `MULTIPOLYGON EMPTY` became `LINESTRING()`. [#110709](https://github.com/ClickHouse/ClickHouse/pull/110709) ([Groene AI](https://github.com/groeneai)). * Fix an exception (`Cannot calculate columns sizes when columns or checksums are not initialized`) during a merge of a single-column `MergeTree` table whose only column has a fully expiring column `TTL`, with `min_bytes_for_wide_part = 1`. [#111169](https://github.com/ClickHouse/ClickHouse/pull/111169) ([Groene AI](https://github.com/groeneai)). * Failing to start a background schedule pool (for example the `iceberg` pool under global thread-pool exhaustion) no longer aborts the server: the error is now a recoverable exception. Reading `system.background_schedule_pool` also no longer creates those pools as a side effect. [#111209](https://github.com/ClickHouse/ClickHouse/pull/111209) ([Groene AI](https://github.com/groeneai)). * Fix an out-of-bounds read in the native protocol on corrupted index data. [#112331](https://github.com/ClickHouse/ClickHouse/pull/112331) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a deadlock that permanently hangs a `Filesystem` or `HDFS` database when two queries resolve the same table name at the same time. Every later query against that database blocks forever and cannot be interrupted, so the server needs a restart to recover. [#112495](https://github.com/ClickHouse/ClickHouse/pull/112495) ([Groene AI](https://github.com/groeneai)). * Fix `max_execution_time` and `KILL QUERY` being silently ineffective for a distributed query that is establishing connections to replicas. Such a query now stops at the next connection attempt instead of working through every remaining replica and retry. [#112689](https://github.com/ClickHouse/ClickHouse/pull/112689) ([Groene AI](https://github.com/groeneai)). * Fix `DROP TABLE` and `SYSTEM STOP VIEW` hanging when a refreshable materialized view is blocked while planning its refresh query. [#113188](https://github.com/ClickHouse/ClickHouse/pull/113188) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `CREATE TABLE` with `ENGINE = Distributed(...)` or `ENGINE = Buffer(...)` and no column list revealing the structure of a referenced table the creating user is not allowed to see. For a local `CREATE`, the structure is now inferred under the user's own context, so `SHOW COLUMNS` on the referenced table is required. A `CREATE` replayed from the DDL queue (`ON CLUSTER`, or inside a `Replicated` database) is not covered. [#113220](https://github.com/ClickHouse/ClickHouse/pull/113220) ([Groene AI](https://github.com/groeneai)). [#113372](https://github.com/ClickHouse/ClickHouse/pull/113372) ([Groene AI](https://github.com/groeneai)). * Fixed reading the internal `_temporary_and_external_tables` database through the `merge` table function and the `Merge` table engine, which allowed one session to read the temporary tables of other sessions and other users. A database regexp now skips that database, and naming it explicitly is denied, the same way as direct access to it is. [#113224](https://github.com/ClickHouse/ClickHouse/pull/113224) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `countSubstrings`, `countSubstringsCaseInsensitive`, `countSubstringsCaseInsensitiveUTF8`, `countMatches`, and `countMatchesCaseInsensitive` ignoring `KILL QUERY` and `max_execution_time`: a call over many rows, many matches, or one large value could run for minutes after the query was cancelled. [#113369](https://github.com/ClickHouse/ClickHouse/pull/113369) ([Groene AI](https://github.com/groeneai)). [#113886](https://github.com/ClickHouse/ClickHouse/pull/113886) ([Groene AI](https://github.com/groeneai)). * Fixed a mutation of a Compact part producing a different result depending on whether the part's serialization info was still in memory or had been reloaded from `serialization.json`, when the table uses `serialization_info_version = 'basic'`. On `ReplicatedMergeTree`, this made two replicas holding a byte-identical source part write different mutated parts, and the mutation failed with `CHECKSUM_DOESNT_MATCH`. Closes [#113500](https://github.com/ClickHouse/ClickHouse/issues/113500). [#113588](https://github.com/ClickHouse/ClickHouse/pull/113588) ([Groene AI](https://github.com/groeneai)). * Fixed `made_current_at` in `system.iceberg_history` reporting the earliest time a snapshot became current instead of the latest one, when the same `snapshot-id` appears more than once in the `Iceberg` `snapshot-log` (as a rollback leaves it). [#113598](https://github.com/ClickHouse/ClickHouse/pull/113598) ([Groene AI](https://github.com/groeneai)). * Fix wrong values for virtual columns such as `_table` or `_database` selected from the right side of a `JOIN` against a key-value source such as `EmbeddedRocksDB`, `KeeperMap`, `Redis`, or a dictionary. The query could return a data-column value instead, or fail with a `LOGICAL_ERROR` exception. [#113698](https://github.com/ClickHouse/ClickHouse/pull/113698) ([Groene AI](https://github.com/groeneai)). * Fix `EXPLAIN ANALYZE` over a `Merge` table with several children, a bare-column `PREWHERE`, and a separate `WHERE` failing with `Required output position N is out of range for pass-through inputs`. Closes [#113621](https://github.com/ClickHouse/ClickHouse/issues/113621). [#113725](https://github.com/ClickHouse/ClickHouse/pull/113725) ([Groene AI](https://github.com/groeneai)). * Fixed `clickhouse-disks move` of a directory failing with `DIRECTORY_ALREADY_EXISTS` on disks with `metadata_type = plain_rewritable` (for example `s3_plain_rewritable`), which also left an empty destination directory behind. [#113727](https://github.com/ClickHouse/ClickHouse/pull/113727) ([Groene AI](https://github.com/groeneai)). * Fix `Merge` table reads failing with `No set is registered for key ...` when a child declares an `ALIAS` column containing `IN` and the children disagree about that column's default. [#113757](https://github.com/ClickHouse/ClickHouse/pull/113757) ([Groene AI](https://github.com/groeneai)). * Fixed `UNKNOWN_IDENTIFIER` with `enable_analyzer = 0` when a `WHERE` refers to a column produced by `untuple` inside a subquery, for example `SELECT t.keys FROM (SELECT untuple(arrayJoin(m)) AS t FROM ...) WHERE t.values > 0`. The `enable_optimize_predicate_expression` rewrite pushed such a predicate into the subquery as a `HAVING`, where the name does not yet exist. [#113760](https://github.com/ClickHouse/ClickHouse/pull/113760) ([Groene AI](https://github.com/groeneai)). * Fix `ALTER TABLE ... MODIFY COLUMN` failing with `Cannot specify codec for column type ALIAS` when one statement both turns an `ALIAS` column into a physical one (`DEFAULT` or `MATERIALIZED`) and specifies a `CODEC`. Closes [#113693](https://github.com/ClickHouse/ClickHouse/issues/113693). [#113853](https://github.com/ClickHouse/ClickHouse/pull/113853) ([Groene AI](https://github.com/groeneai)). * Fix `ORDER BY ALL` and `GROUP BY ALL` over a joined `Merge` table under the old analyzer (`allow_experimental_analyzer = 0`). The query no longer fails with an internal exception, and debug and sanitizer builds no longer abort. [#109139](https://github.com/ClickHouse/ClickHouse/pull/109139) ([Groene AI](https://github.com/groeneai)). * Fix a sporadic `Cannot parse string ... as UInt64 ... While executing ValuesBlockInputFormat` exception that could occur when inserting values whose expressions contain a numeric cast, such as `initializeAggregation('sumState', 0::UInt64)`. [#110637](https://github.com/ClickHouse/ClickHouse/pull/110637) ([Groene AI](https://github.com/groeneai)). * Fixed the periodic local disk check reporting failures for a healthy disk. When a server runs an embedded Keeper alongside a `local` disk, two check threads could probe the same file, transiently log `FILE_DOESNT_EXIST`, `CANNOT_READ_ALL_DATA`, or `CANNOT_UNLINK`, and temporarily count the disk in the `BrokenDisks` metric. [#110773](https://github.com/ClickHouse/ClickHouse/pull/110773) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results and a logical error `Cannot serialize FutureSetFromSubquery with no query plan` for queries over `Merge` tables and the `merge` table function when plan-based parallel replicas were enabled. [#112849](https://github.com/ClickHouse/ClickHouse/pull/112849) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A table in the `URL` database engine no longer discloses whether a local file exists to a user without the read source grant. Both `EXISTS TABLE`, which requires only the `SHOW TABLES` privilege, and table-name resolution now wait for the grant check before probing the filesystem. [#113029](https://github.com/ClickHouse/ClickHouse/pull/113029) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a TTL expression becoming unusable when constant folding pruned the columns it refers to, for example `TTL d + INTERVAL 1 DAY DELETE WHERE isNull(x)` over a non-`Nullable` `x`, which made every `INSERT` into the table fail with `Missing columns`. Also fixed the server failing to start when a table had a TTL expression over a `Variant` column created with `variant_throw_on_type_mismatch = 0`. [#113042](https://github.com/ClickHouse/ClickHouse/pull/113042) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed cancellation of `postgresql` table function and `PostgreSQL` table engine reads. Cancelling while the read was still starting up could race on the transaction pointer and be lost, leaving the query running until PostgreSQL finished it. [#113150](https://github.com/ClickHouse/ClickHouse/pull/113150) ([Groene AI](https://github.com/groeneai)). * `UNDROP TABLE` can now be interrupted by `KILL QUERY` while it waits for running queries to release the dropped table. [#113263](https://github.com/ClickHouse/ClickHouse/pull/113263) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a memory leak of the whole XML parser when a `Poco` XML handler throws mid-parse, for example on `Poco::XML::NamePool` overflow while parsing a document with many unique element names. [#113348](https://github.com/ClickHouse/ClickHouse/pull/113348) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed writes to the `HDFS` disk: they no longer fail with `Parent directory doesn't exist` when the generated object key contains nested directory prefixes, because the missing parent directories are now created. [#113396](https://github.com/ClickHouse/ClickHouse/pull/113396) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results and, in builds with assertions enabled, an aborted assertion for a `JOIN` onto a dictionary whose `ON` clause has a non-equi condition over both tables, such as `ON (t.key = d.key) AND (t.a * 10 < d.a)`, when `join_use_nulls` is enabled. [#113534](https://github.com/ClickHouse/ClickHouse/pull/113534) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a query hanging for up to `receive_timeout` seconds when it finishes very quickly on the initiator. [#113551](https://github.com/ClickHouse/ClickHouse/pull/113551) ([Nikita Taranov](https://github.com/nickitat)). * Fix `UPDATE` and lightweight `DELETE` reading the wrong tables when the database of the updated table differs from the session database. Unqualified table names and the `merge` table function are now resolved against the database of the updated table, so the statement no longer silently updates or deletes rows selected from a different table. [#113748](https://github.com/ClickHouse/ClickHouse/pull/113748) ([Groene AI](https://github.com/groeneai)). * Fix a spurious `TOO_MANY_PARTS` rejection of a parallel `INSERT`: when `max_insert_threads` parallelizes a plain `INSERT`, the query no longer counts parts committed by its own sibling sinks. Closes [#114015](https://github.com/ClickHouse/ClickHouse/issues/114015). [#114016](https://github.com/ClickHouse/ClickHouse/pull/114016) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NO_SUCH_COLUMN_IN_TABLE` when a query with parallel replicas selects an `ALIAS` column of a table shipped as a `GLOBAL JOIN` temporary table, `MULTIPLE_EXPRESSIONS_FOR_ALIAS` when both sides of a distributed `JOIN` declare an `ALIAS` column with the same name, and `UNKNOWN_IDENTIFIER` when a `Distributed` table declares an `ALIAS` column over another `ALIAS` column, including as a `JOIN USING` key. [#107700](https://github.com/ClickHouse/ClickHouse/pull/107700) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fixed reading a CSV `Tuple` whose leading element is `NULL`. With `input_format_csv_deserialize_separate_columns_into_tuple` and `input_format_null_as_default` enabled, a leading `\N` in a row such as `\N,1` was taken for the whole column being `NULL`, so the row was replaced by a default and the remaining element cells caused a parse error. The leading `NULL` is now the value of the first element, with `input_format_null_as_default` applying to that element. A row that gives a single `\N` field for a whole bare `Tuple` is now short by the remaining elements and is rejected; set `input_format_csv_deserialize_separate_columns_into_tuple = 0` to read such a field as the whole column again. [#109744](https://github.com/ClickHouse/ClickHouse/pull/109744) ([Groene AI](https://github.com/groeneai)). * Fix a rare `Logical error: 'No more packets are available.'` for a valid distributed query with a `LIMIT`. In debug and sanitizer builds it also aborted the server. [#110334](https://github.com/ClickHouse/ClickHouse/pull/110334) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (`!part.empty()`) when a `Distributed` table defined with an empty remote database name is used in a `JOIN` with `distributed_product_mode = 'local'`. [#110677](https://github.com/ClickHouse/ClickHouse/pull/110677) ([Groene AI](https://github.com/groeneai)). * Fix resolution of fully qualified column names such as `db.table.column` in the analyzer for tables that have the same name as their database. Closes [#82084](https://github.com/ClickHouse/ClickHouse/issues/82084). [#110976](https://github.com/ClickHouse/ClickHouse/pull/110976) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed hash joins losing their post-build optimizations (conversion to a fixed hash table, the shared runtime filter, right-table reranging) whenever an automatic spill-to-disk threshold was configured, which has been the default since 26.5. [#111972](https://github.com/ClickHouse/ClickHouse/pull/111972) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a bug where `ALTER TABLE ... MODIFY COLUMN` that changes a `JSON` type hint (with `allow_experimental_json_lazy_type_hints` enabled) on a column whose subcolumn is used in the primary/sorting key, partition key, or an explicit skip index could corrupt the on-disk key or index. Ordinary reads masked it via the cached structure, but after it was reloaded (for example on server restart) queries failed with `CANNOT_READ_ALL_DATA` or returned wrong results. Such `ALTER`s are now rejected whenever they change the on-disk type of a referenced `JSON` subcolumn. Type-hint changes on columns outside those structures, or changes that leave the on-disk type of the referenced subcolumns unchanged (for example adding an unrelated typed path), remain metadata-only. [#112302](https://github.com/ClickHouse/ClickHouse/pull/112302) ([Pavel Kruglov](https://github.com/Avogar)). * Fix wrong results when `query_plan_join_swap_table` swaps an `ANY JOIN` while `join_any_take_last_row = 1`: ClickHouse now keeps the original join side order so `ANY JOIN` still returns the last matching row. Closes [#111645](https://github.com/ClickHouse/ClickHouse/issues/111645). [#112458](https://github.com/ClickHouse/ClickHouse/pull/112458) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed `listObjects` on a prefixed-endpoint `AzureBlobStorage` disk returning wrong blob names on every page after the first, because pagination bypassed the client wrapper that strips the endpoint prefix. Pagination now re-enters the wrapper via the continuation token so every page's blob names are correct. [#112872](https://github.com/ClickHouse/ClickHouse/pull/112872) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed unbounded memory allocation when reading a `Parquet` file whose Thrift metadata declares a container or string length larger than the metadata itself. Such a length is now rejected instead of being allocated for. [#113212](https://github.com/ClickHouse/ClickHouse/pull/113212) ([Groene AI](https://github.com/groeneai)). * Fixed `Cannot insert element into Set` (`NOT_IMPLEMENTED`) when `read_in_order_use_virtual_row` is enabled and a `MergeTree` read-in-order query combines `WITH FILL ... INTERPOLATE` with an `IN` or `has` filter. [#113264](https://github.com/ClickHouse/ClickHouse/pull/113264) ([Groene AI](https://github.com/groeneai)). * Fix `optimize_read_in_order` being silently skipped for some `ORDER BY` queries, including queries that use a virtual row or `toString`/`CAST(..., 'Nullable(String)')` over a `String` sorting key. Also fix wrong results for filters or `ORDER BY` on `toString` of `Time`, `Time64`, or `DateTime` values in time zones with daylight saving time. [#113291](https://github.com/ClickHouse/ClickHouse/pull/113291) ([Vladimir Cherkasov](https://github.com/vdimir)). * Reject invalid `parallel_replicas_custom_key` expressions such as `tuple()` or `(x, y)` with `ILLEGAL_TYPE_OF_COLUMN_FOR_FILTER` instead of reading past the end of the custom-key type list. [#113359](https://github.com/ClickHouse/ClickHouse/pull/113359) ([Nikita Taranov](https://github.com/nickitat)). * Fix a server restart failure after the server is killed during `DROP TABLE` of a table that uses a named collection: a later `DROP NAMED COLLECTION` could succeed, leaving table metadata that references a missing named collection and making the next startup fail with `NAMED_COLLECTION_DOESNT_EXIST`. [#113524](https://github.com/ClickHouse/ClickHouse/pull/113524) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes two cases where an AWS Glue `DataLakeCatalog` database reported a table as absent instead of reporting an error. `SHOW TABLES` and `system.tables` no longer fail with `DATALAKE_DATABASE_ERROR` ("Exception calling GetTables ... not found") when a Glue database is absent while its tables are listed, for example because an unrelated database in the same AWS account was dropped concurrently; such a database now contributes no tables instead of aborting the listing. `EXISTS TABLE` no longer returns `0` for a table that exists when the underlying `GetTable` call fails for a reason other than the table being absent, such as an authentication, DNS, or 5xx failure. Every other Glue error still propagates in both paths. [#113591](https://github.com/ClickHouse/ClickHouse/pull/113591) ([Groene AI](https://github.com/groeneai)). * Fixed a Keeper durability bug where a crash while persisting the Raft state file (storing `term` and `voted_for`) could leave the node with no valid state and no backup. On restart the node could reset to `term = 0` and grant a vote it had already granted in that term, which could cause two leaders and loss of committed data. The previous state is now kept until the new state file is fully written and synced. [#113654](https://github.com/ClickHouse/ClickHouse/pull/113654) ([Alexey Bakharew](https://github.com/alexbakharew)). * Fix a data race when a `MergeTree` table is destroyed while its background statistics-refresh, parts-refresh, or outdated/unexpected-parts-loading tasks are still running. [#113722](https://github.com/ClickHouse/ClickHouse/pull/113722) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ORDER BY ALL` in a `WINDOW VIEW`, and `UNKNOWN_IDENTIFIER` errors for `INTERPOLATE`, `WINDOW`, and `LIMIT BY` in a query over a `Merge` or `Distributed` table with a join, with the old analyzer. [#113759](https://github.com/ClickHouse/ClickHouse/pull/113759) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a memory leak in the experimental `SZ3` compression codec: when compression of poorly compressible or corrupted data failed with an exception, temporary scratch buffers were left allocated. [#113916](https://github.com/ClickHouse/ClickHouse/pull/113916) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix quadratic complexity of parsing the address argument of the `remote`, `mysql`, and similar table functions: a very long address (for example a megabyte-sized zero-padded `FixedString`) made the query hang during analysis, where it could not be cancelled. [#113918](https://github.com/ClickHouse/ClickHouse/pull/113918) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `OPTIMIZE TABLE` on an `Iceberg` table failing with `Unsupported snapshot's operation type OVERWRITE` when the table's history contained a row delete that removed more than one row per position-delete file. Closes [#113745](https://github.com/ClickHouse/ClickHouse/issues/113745). [#113964](https://github.com/ClickHouse/ClickHouse/pull/113964) ([Groene AI](https://github.com/groeneai)). * Do not replicate `ALTER TABLE ATTACH PARTITION FROM` for replicated db. [#94835](https://github.com/ClickHouse/ClickHouse/pull/94835) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * Invalid Azure upload-size settings (for example `azure_min_upload_part_size = 0` or `azure_max_blocks_in_multipart_upload = 0`) are now rejected early with a clear error instead of triggering an internal exception (`second_size > 0`) deep in the write path. This applies to both query settings and Azure disk configuration. [#103394](https://github.com/ClickHouse/ClickHouse/pull/103394) ([Statxc](https://github.com/statxc)). * Fixed incorrect SQL literal escaping in `StorageSQLite` and `sqlite()` table function when pushing `WHERE` predicates to SQLite: single quotes and control characters (`\n`, `\r`, `\t`, `\`) were escaped with backslashes, which SQLite does not interpret, causing syntax errors or wrong query results. Also fixed the escaping of string literals pushed down to PostgreSQL: strings nested inside `IN` lists kept ClickHouse escaping, and control characters were sent as backslash sequences that PostgreSQL reads back as different bytes. [#104217](https://github.com/ClickHouse/ClickHouse/pull/104217) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` exception in `INNER JOIN ... ON arrayJoin(...) = ...` queries when `query_plan_convert_join_to_in` is enabled and the SELECT references the source array column. [#104809](https://github.com/ClickHouse/ClickHouse/pull/104809) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix IN and NOT IN expressions with non-constant right-hand side operands referencing columns from the current row, and align new analyzer tuple right-hand side handling with existing ClickHouse IN semantics. Under the old analyzer, a bare source column as the right-hand side (`x IN (arr)`) still resolves as a table name and stays out of scope. [#104993](https://github.com/ClickHouse/ClickHouse/pull/104993) ([Yue Ni](https://github.com/niyue)). * Fixed the error `Cannot add column ...: column with this name already exists` on `INSERT SELECT` from a table function such as `file`, `s3` or `input`, when the same source column is selected more than once. [#105982](https://github.com/ClickHouse/ClickHouse/pull/105982) ([etienne](https://github.com/crakjie)). * Reject `max_replication_lag_to_enqueue = 0` on `Replicated` databases. The value made the post-recovery unsynced check trivially true and aborted the server with `LOGICAL_ERROR` in debug and sanitizer builds. `0` is now rejected at parse time with `BAD_ARGUMENTS` from every source (`CREATE DATABASE ... SETTINGS`, `` server-config block, `ATTACH` replay, and upgrade-time replay of existing metadata). The smallest valid value is `1`. Closes [#98823](https://github.com/ClickHouse/ClickHouse/issues/98823). [#106006](https://github.com/ClickHouse/ClickHouse/pull/106006) ([Groene AI](https://github.com/groeneai)). * Fix `Logical error: 'Pipeline stuck'` in queries that use `enable_sharding_aggregator = 1` together with a `UNION ALL` and `max_streams_for_union_step` smaller than the pipeline width. `BufferedShardByHashTransform` now finishes empty-queue output ports as soon as its input is exhausted, and keeps pulling input when a demanded empty port has nothing drainable even if a sibling queue hit the back-pressure cap. [#106251](https://github.com/ClickHouse/ClickHouse/pull/106251) ([Groene AI](https://github.com/groeneai)). * Fixed a signed integer overflow when a wait-timeout setting, such as `interactive_delay`, was set to a huge value: the wait could time out immediately instead of waiting, and the server could abort under the Undefined Behavior Sanitizer. [#106961](https://github.com/ClickHouse/ClickHouse/pull/106961) ([Groene AI](https://github.com/groeneai)). * Exporting `Time`/`Time64` values outside a valid time-of-day (negative or `>= 24h`) to the `Arrow` format is now rejected with a clear error instead of writing invalid Arrow `time32`/`time64` data. [#107179](https://github.com/ClickHouse/ClickHouse/pull/107179) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix wrong results for `GROUP BY` on a `Distributed` table over duplicate `ALIAS` columns that expand to the same expression (e.g. two columns both defined as `toString(x)`) under two-level / memory-efficient distributed aggregation. The duplicate keys were collapsed into one on the shards but merged by the full key set on the initiator, so under two-level aggregation equal groups from different shards landed in different buckets and were not merged, returning a group once per shard with a split `count()`. [#108855](https://github.com/ClickHouse/ClickHouse/pull/108855) ([Groene AI](https://github.com/groeneai)). * Fixed a spurious `Inconsistent AST formatting` error for `ALTER TABLE ... MOVE PART ... TO SHARD ''`. The query formatter dropped the `SHARD` keyword, so the reformatted query could not be parsed back, raising a `LOGICAL_ERROR` (a handled exception in release builds, an abort in debug and sanitizer builds). [#109060](https://github.com/ClickHouse/ClickHouse/pull/109060) ([Groene AI](https://github.com/groeneai)). * Rejects non-monotonic `Array` offsets on every read path that decodes them as absolute values, instead of only in the `Native` format. A corrupt `set` skip index granule previously passed unchecked and was then read out of bounds. [#109212](https://github.com/ClickHouse/ClickHouse/pull/109212) ([Groene AI](https://github.com/groeneai)). * `JSONExtract` now honours `cast_string_to_date_time_mode` when converting string JSON values to `DateTime`/`DateTime64`, consistently with `CAST`. Closes [#109126](https://github.com/ClickHouse/ClickHouse/issues/109126). [#109252](https://github.com/ClickHouse/ClickHouse/pull/109252) ([Utkal Singh](https://github.com/Utkal059)). * Fix possible logical error "Unexpected substream ... for column ..." during bump of compatibility setting. [#109496](https://github.com/ClickHouse/ClickHouse/pull/109496) ([Pavel Kruglov](https://github.com/Avogar)). * set missing coordination component in system.detached_parts size calculation. [#109553](https://github.com/ClickHouse/ClickHouse/pull/109553) ([Den Kalantaevskii](https://github.com/Diskein)). * Fixed the block-wait timeout of streaming inserts (`input_format_max_block_wait_ms`) potentially never expiring while the query profiler is active: the file-descriptor poll restarted with the full timeout after every profiler signal, so a stalled input source could delay the partial-block flush indefinitely instead of flushing when the wait limit is reached. [#109602](https://github.com/ClickHouse/ClickHouse/pull/109602) ([Shaohua Wang](https://github.com/tiandiwonder)). * Constant folding during query planning no longer executes a function whose argument type differs from the type the function was resolved for during analysis. Such a mismatch used to raise a `LOGICAL_ERROR` while the output header was being computed, which is an assertion failure in debug and sanitizer builds. Folding is now skipped instead, and a genuinely invalid query still reports a recoverable `ILLEGAL_TYPE_OF_ARGUMENT` error from the normal execution path. [#109747](https://github.com/ClickHouse/ClickHouse/pull/109747) ([Groene AI](https://github.com/groeneai)). * Fixes a mutation that references the `_sample_factor` virtual column failing mid-execution with `Unexpected const virtual column: _sample_factor` and leaving the mutation stuck. A mutation now reads it as `1`, the same value a query without `SAMPLE` returns. [#109813](https://github.com/ClickHouse/ClickHouse/pull/109813) ([Groene AI](https://github.com/groeneai)). * Fix three issues with `use_constant_folding_in_index_analysis`: a logical error `Invalid partition key size` that could abort a `SELECT` using a normal projection when the table sets `part_minmax_index_columns = '`with_block_number_offset`'`; wrong results (silently dropped rows) for a filter on a modulo partition key such as `PARTITION BY id % 200`; and a crash (data race) when a `text` index with a `sparseGrams` tokenizer is queried with a `LIKE` predicate over many partitions with `max_threads > 1`. [#109896](https://github.com/ClickHouse/ClickHouse/pull/109896) ([Groene AI](https://github.com/groeneai)). * Fix a data race (and rare crash) when a parsing/analysis error is reported while reading a file/object storage table. A parse error raised during lazy `KeyCondition` initialization was shared as a single exception object across reader threads, and two threads could concurrently append `(in file/uri ...)` to its message, racing on the exception message string. [#109911](https://github.com/ClickHouse/ClickHouse/pull/109911) ([Groene AI](https://github.com/groeneai)). * Fix propagation of query settings in `accurateCastOrDefault`, and preserve source `NULL`s encoded by `Dynamic` and `Variant`. Closes [#109943](https://github.com/ClickHouse/ClickHouse/issues/109943). [#109946](https://github.com/ClickHouse/ClickHouse/pull/109946) ([Pavel Kruglov](https://github.com/Avogar)). [#114912](https://github.com/ClickHouse/ClickHouse/pull/114912) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed incorrect results from `subBitmap`, `bitmapSubsetInRange` and `bitmapSubsetLimit` for bitmaps still held in the small representation, including `UInt64` and `Int64` element values above `2^32`, which were truncated to 32 bits. Comparisons in bitmap functions over signed element types now consistently use the unsigned value of the element type, so in an `Int8` bitmap the element `-1` is compared as `255` instead of as the sign-extended `4294967295`; this also fixes `bitmapMin`, `bitmapMax`, `bitmapContains` and `bitmapTransform` on bitmaps that have grown past the small representation. Queries that passed sign-extended thresholds have to be adjusted: over an `Int8` bitmap, `bitmapSubsetInRange(bm, 4294967168, 4294967296)` becomes `bitmapSubsetInRange(bm, 128, 256)`. Also fixed `groupNumericIndexedVector` returning different results for `Int8` and `Int16` index columns than for wider index types, and `numericIndexedVectorGetValue` returning `0` for negative indexes. Corrected the bitmap function documentation, including the subset functions that were described as using 1-based indexing and the signed `bitmapBuild` / `bitmapToArray` support. [#110072](https://github.com/ClickHouse/ClickHouse/pull/110072) ([Rami Darwiche](https://github.com/RamiDarwiche)). * Fix `optimize_aggregation_in_order` ignoring query cancellation. `AggregatingInOrderTransform` now checks for cancellation while aggregating a chunk, so a query stopped by `KILL QUERY` or by `max_execution_time` (in the default `timeout_overflow_mode = 'throw'`) stops promptly instead of running the whole chunk to completion. [#110104](https://github.com/ClickHouse/ClickHouse/pull/110104) ([Groene AI](https://github.com/groeneai)). * Fixed a race between `KILL TRANSACTION` and `ALTER TABLE ... UPDATE/DELETE` executed inside a transaction: a rollback landing between the mutation's registration in the transaction and its registration in the table left an orphaned mutation entry, which raised a logical error (`Cannot find transaction ... that has started mutation ...`) in background jobs and blocked all subsequent mutations of the affected parts until server restart. [#110226](https://github.com/ClickHouse/ClickHouse/pull/110226) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix a `LOGICAL_ERROR` (`Left and right columns have same names`) in the join order optimizer that could be triggered by a comma join over a multi-table view when `query_plan_merge_expression_into_join` is enabled. [#110227](https://github.com/ClickHouse/ClickHouse/pull/110227) ([Groene AI](https://github.com/groeneai)). * Join-order cardinality estimation now composes column statistics over the parts surviving partition/PK pruning instead of all active parts. Previously a query pruned to a small partition was planned against table-wide statistics (observed 2500x row overestimation flipping the hash-join build side), and disabling statistics paradoxically produced a better plan. Also fixed: lazy FINAL was silently disabled for MergeTree relations under a JOIN because the join-order optimizer's memoized index-analysis result was mistaken for an applied projection. [#110283](https://github.com/ClickHouse/ClickHouse/pull/110283) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fixed wrong results from correlated scalar subqueries when the outer query has duplicate values for the correlated column and the subquery is decorrelated via the CROSS JOIN path. Previously aggregates such as `count()` and `sum()` were multiplied by the duplicate count. This affects the experimental `allow_experimental_correlated_subqueries` feature. [#110311](https://github.com/ClickHouse/ClickHouse/pull/110311) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Function node with name '...' is not resolved as ordinary function`) that could occur with `optimize_or_like_chain` enabled when an aggregate or window function appeared on the left-hand side of a `LIKE`/`ILIKE`/`match` inside an `OR` chain. [#110641](https://github.com/ClickHouse/ClickHouse/pull/110641) ([Groene AI](https://github.com/groeneai)). * Fix a `TTL ... DELETE WHERE ` never removing rows that started matching `` because of an `ALTER TABLE ... UPDATE` on a column referenced in the `WHERE`. The mutation now recalculates the row TTL, so the newly matching expired rows are deleted instead of being silently retained. [#110648](https://github.com/ClickHouse/ClickHouse/pull/110648) ([Groene AI](https://github.com/groeneai)). * Fix an accounting leak in memory tracking: when an allocation was rejected by a parent memory tracker (for example, the server-wide memory limit), the thread, query, and user level trackers kept the rejected amount. The error accumulated over time, inflating `MemoryTracking` and per-query/per-user memory usage, and could lead to spurious `MEMORY_LIMIT_EXCEEDED` errors. [#110784](https://github.com/ClickHouse/ClickHouse/pull/110784) ([Seva Potapov](https://github.com/seva-potapov)). * Fixed non-canonical WKB serialization of an empty polygon: `wkb(POLYGON EMPTY)` now emits `numRings = 0` instead of a spurious single zero-point ring, so a WKB round-trip of a standard empty polygon is identity-preserving. [#110796](https://github.com/ClickHouse/ClickHouse/pull/110796) ([Groene AI](https://github.com/groeneai)). * Fixes a case when using CTE column aliases with `INTERSECT` or `EXCEPT` queries, e.g. `WITH t (a, b) AS (SELECT 1, 2 INTERSECT SELECT 1, 2) SELECT a, b FROM t`, the query was failing with UNKNOWN_IDENTIFIER error. Closes [#104770](https://github.com/ClickHouse/ClickHouse/issues/104770). [#110885](https://github.com/ClickHouse/ClickHouse/pull/110885) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a `std::length_error` (`LOGICAL_ERROR`) when reading a MergeTree table with a pathological `max_streams_for_merge_tree_reading` value (near the maximum of `UInt64`) together with `allow_asynchronous_read_from_io_pool_for_merge_tree = 1`. The setting is now clamped to the same ceiling as `max_threads`. [#110936](https://github.com/ClickHouse/ClickHouse/pull/110936) ([Groene AI](https://github.com/groeneai)). * Fixes a crash (null pointer dereference during query analysis) when `getSubcolumn` is called on a `JSON` argument with an internal type-hint subcolumn name, that is one starting with a backtick type marker. [#110940](https://github.com/ClickHouse/ClickHouse/pull/110940) ([Groene AI](https://github.com/groeneai)). * Fixed `Nested type LineString cannot be inside Nullable type (ILLEGAL_TYPE_OF_ARGUMENT)` when reading a nullable MySQL spatial column (`LINESTRING`, `POLYGON`, `MULTILINESTRING`, `MULTIPOLYGON`, `MULTIPOINT`, or the generic `GEOMETRY`) through the `mysql` table function or the `MySQL` table/database engines. Such columns now fall back to `Nullable(String)` holding the value as MySQL returns it (a 4-byte SRID prefix followed by the WKB payload). `POINT` keeps mapping to `Nullable(Point)`. [#110943](https://github.com/ClickHouse/ClickHouse/pull/110943) ([Groene AI](https://github.com/groeneai)). * Fixes `Native` serialization of `SimpleAggregateFunction` wrappers containing versioned aggregate states for older peers. [#110997](https://github.com/ClickHouse/ClickHouse/pull/110997) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR: ReadBuffer is canceled. Can't read from it.` that could occur in a `grace_hash` join when reading spilled blocks failed mid-read while multiple threads shared the reader. [#111056](https://github.com/ClickHouse/ClickHouse/pull/111056) ([Groene AI](https://github.com/groeneai)). * Each SSH key now counts individually toward the `max_authentication_methods_per_user` limit, so the limit applies consistently regardless of syntax. Previously, multiple keys in a single `ssh_key` method counted as one. [#111181](https://github.com/ClickHouse/ClickHouse/pull/111181) ([Yecine Megdiche](https://github.com/TheMC47)). * Fix a `LOGICAL_ERROR` (`Reading from materialized CTE '...' before its materialization completed - DelayedPortsProcessor gate is missing in the query plan`) when a reused materialized CTE (`enable_materialized_cte`) is read on a branch whose `WITH TOTALS` or `extremes` output is discarded, including outer aggregation, `INSERT ... SELECT`, set operations and `EXPLAIN ANALYZE`. [#111194](https://github.com/ClickHouse/ClickHouse/pull/111194) ([Groene AI](https://github.com/groeneai)). * Fixed `map[key]` returning the value of a later duplicate key instead of the first one. When a `Map` column contained duplicate keys, the result of `map[key]` could depend on preceding rows in the same block and on the `optimize_functions_to_subcolumns` setting, so a `SELECT` and a `WHERE` over the same row could disagree. [#111246](https://github.com/ClickHouse/ClickHouse/pull/111246) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `!rhs_literal->getValue().isNull()` in the logical-expression optimizer when an `AND` chain compares a column against a NULL-valued constant of a non-`Nullable` type (for example a NULL-valued `Variant`) with `use_variant_default_implementation_for_comparisons = 0`. [#111292](https://github.com/ClickHouse/ClickHouse/pull/111292) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` / `THERE_IS_NO_COLUMN` errors (and, on some releases, silently wrong results) for a `RIGHT JOIN` when `parallel_replicas_min_number_of_rows_per_replica` is set and a left-table column is projected. [#111332](https://github.com/ClickHouse/ClickHouse/pull/111332) ([Groene AI](https://github.com/groeneai)). * `RESTORE` now fsyncs the restored part files when the destination table has `fsync_after_insert` enabled, so a restored part is as durable against power loss as an inserted one. Previously restored files were only written and closed (never fsynced), so a power loss right after `RESTORE` returned `RESTORED` could leave the parts torn and the table empty. [#111378](https://github.com/ClickHouse/ClickHouse/pull/111378) ([Groene AI](https://github.com/groeneai)). * Fix a rare Keeper data-loss window on local snapshot disks: the snapshot write path now fsyncs the snapshot directory after removing the `tmp_` marker, so a power loss can no longer drop the marker unlink while keeping the snapshot content. Previously such a crash made Keeper treat a good, fully-synced snapshot as incomplete and delete it on restart, losing acknowledged writes even with `force_sync=true`. Snapshot disks backed by object storage are not covered. [#111397](https://github.com/ClickHouse/ClickHouse/pull/111397) ([Groene AI](https://github.com/groeneai)). * Fix an out-of-bounds read (release) / logical error (debug) in the lazy materialization optimization, and a logical error in `Pipe::unitePipes` for `Merge` tables, when the direct-read-from-text-index optimization leaves an unused index virtual column in the read header of a `MergeTree` table with multiple `text` indexes. [#111467](https://github.com/ClickHouse/ClickHouse/pull/111467) ([Groene AI](https://github.com/groeneai)). * Fixed the `_time` virtual column of the `url`/`urlCluster` table functions always being `NULL` on musl-based builds, caused by non-portable parsing of the HTTP `Last-Modified` header. [#111619](https://github.com/ClickHouse/ClickHouse/pull/111619) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed a `LOGICAL_ERROR` (`Cannot create part with type Compact and storage type Full because table does not support polymorphic parts`) raised when a `MergeTree` table loads a pre-existing `Compact` part while its current settings disable polymorphic parts (non-adaptive granularity). This could happen, for example, when a read-only table shares an on-disk path with another table and reloads its parts via `SYSTEM RESTART DISK` or on server startup. In debug and sanitizer builds the exception aborts the server. [#111626](https://github.com/ClickHouse/ClickHouse/pull/111626) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE TABLE ... ENGINE = KeeperMap` failing with `Coordination error: No node, path .../metadata/drop_lock_version` when another table on the same Keeper path is dropped concurrently. The create now treats the vanished metadata subtree as a completed cleanup and retries, as it already does for the same situation one statement later. [#111966](https://github.com/ClickHouse/ClickHouse/pull/111966) ([Groene AI](https://github.com/groeneai)). * Fix a race between a `ReplicatedMergeTree` mutation and the removal of old temporary directories that could publish a mutated part with some of its files missing, making reads of that part and all its later mutations fail. [#111992](https://github.com/ClickHouse/ClickHouse/pull/111992) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `quantileDeterministic`, `quantilesDeterministic` and `medianDeterministic` returning different results for the same data on every run when the query uses parallel replicas, a distributed table, or external aggregation, or when the states are stored in a table (including inside a `Dynamic` value). States written before this version under the unversioned type spelling keep the previous behavior, since their layout cannot change. [#112052](https://github.com/ClickHouse/ClickHouse/pull/112052) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes `serialize_string_in_memory_with_zero_byte` being dropped from the serialized query plan. With `serialize_query_plan = 1`, a cluster running with the setting disabled could diverge on the in-memory string representation between the initiator and the node executing the deserialized plan, because the setting was read on deserialization but never written on serialization and therefore always resolved to its default on the executing node. [#112131](https://github.com/ClickHouse/ClickHouse/pull/112131) ([Groene AI](https://github.com/groeneai)). * Fixed a stack frame being displayed as `?` instead of its function name in fatal log messages and in the `trace_full` column of `system.crash_log`. It affected frames of ordinary functions that have code attributed to a libc++ `__functional` header, which is common in release builds. [#112152](https://github.com/ClickHouse/ClickHouse/pull/112152) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes silently missing rows when a query filters on `intDiv(constant, key)` or `divide(constant, key)`. Primary-key and partition analysis reports the wrong direction of monotonicity for a constant divided by a key column, so matching granules are pruned away and the query returns fewer rows with no error. Also fixes `intDiv` with an unsigned constant dividend whose high bit is set, stops claiming monotonicity when a `Decimal` operand makes the division compute in the decimal's native width, and stops key analysis from raising `Cannot compare DB::IPv4 with long` for a valid query that divides an `IPv4`/`IPv6` constant by a key column. [#112156](https://github.com/ClickHouse/ClickHouse/pull/112156) ([Groene AI](https://github.com/groeneai)). * Fixed `DROP TABLE ... SYNC` hanging forever when a query over `system.replicas` had been killed before its status requests were processed. [#112164](https://github.com/ClickHouse/ClickHouse/pull/112164) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed a server abort (`SIGABRT`) when a `BUFFERED_V1` WebAssembly UDF returns an empty result buffer; the query now fails with a catchable `WASM_ERROR` and the server stays up. [#112182](https://github.com/ClickHouse/ClickHouse/pull/112182) ([Nikolai Ovchinnikov](https://github.com/kolsha)). * Fixed a use-after-free when a `JOIN` with `join_algorithm = 'direct'` onto a dictionary was given a join key that is stored with sparse serialization. `getColumnVectorData` returned a reference to a temporary column, so the dictionary lookup read freed memory: release builds could return wrong results and debug or sanitizer builds aborted. [#112327](https://github.com/ClickHouse/ClickHouse/pull/112327) ([Groene AI](https://github.com/groeneai)). * Fixes cases where row policy was not used for MergeTree Projections during query execution. [#112329](https://github.com/ClickHouse/ClickHouse/pull/112329) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed data corruption when inserting into an Iceberg table reached through a `MATERIALIZED VIEW`. The data file was written against the target's `CREATE`-time column types instead of its live Iceberg schema, so a `DateTime` value was stored as a bare Avro `int` and read back as microseconds, and with `DateTime64(0)` or `DateTime64(9)` the file could not be read back at all. As a consequence, `INSERT INTO ` against a target whose Iceberg schema is wider than the view's declared columns now raises `TYPE_MISMATCH` instead of writing a corrupt row. [#112359](https://github.com/ClickHouse/ClickHouse/pull/112359) ([Groene AI](https://github.com/groeneai)). * Keeper no longer refuses a `Set` request with empty data when it is over `max_memory_usage_soft_limit`. Such a request cannot increase the amount of stored data, and refusing it prevented clients from re-establishing their session, which could keep tables read-only for the whole duration of a Keeper memory event. [#112386](https://github.com/ClickHouse/ClickHouse/pull/112386) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a Keeper follower dropping out of quorum for a long time, causing mass `Session expired` errors, after it successfully installed a large snapshot: the leader discarded the follower's acknowledgement whenever applying the snapshot took longer than the snapshot-install timeout, and then walked that follower's log index backwards one entry per round trip. Also added the Keeper setting `nuraft_snapshot_sync_ctx_timeout_ms` to control that timeout, which previously could only be changed through `raft_limits_response_limit` and therefore not without also slowing down dead-peer detection. [#112405](https://github.com/ClickHouse/ClickHouse/pull/112405) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix data races in profile events and memory tracker. [#112466](https://github.com/ClickHouse/ClickHouse/pull/112466) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Security related bugfix. [#112479](https://github.com/ClickHouse/ClickHouse/pull/112479) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fixes reading a stale secondary (skip) index after an `ALTER TABLE ... MODIFY COLUMN` type change whose mutation never ran, for example because `KILL MUTATION` removed it: the granules on disk were written with the old type but decoded with the new one, raising `LOGICAL_ERROR`, requesting multi-exabyte allocations, or silently returning a wrong result. Also fixes a wrong result from an index over an expression whose meaning changes while every stored byte stays identical, so no mutation is created at all: a `MODIFY COLUMN` altering only a `DateTime` timezone, or only a custom type name such as `UInt8` to `Bool`. Such an index is now skipped for the affected part. Closes [#112213](https://github.com/ClickHouse/ClickHouse/issues/112213). [#112484](https://github.com/ClickHouse/ClickHouse/pull/112484) ([Groene AI](https://github.com/groeneai)). * Fixed a crash when reading a Parquet file with inconsistent bloom filter metadata. Such files could also silently return fewer rows than they should. [#112498](https://github.com/ClickHouse/ClickHouse/pull/112498) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix AsynchronousBoundedReadBuffer's readBigAt data race. Closes [#109678](https://github.com/ClickHouse/ClickHouse/issues/109678). [#112573](https://github.com/ClickHouse/ClickHouse/pull/112573) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fixes a column alias list such as `(SELECT 1) AS t(x)` or `WITH t(x) AS (SELECT 1)` not being visible inside the `view` table function, where a reference to `t.x` failed with `UNKNOWN_IDENTIFIER` although the same construct works outside `view`. [#112690](https://github.com/ClickHouse/ClickHouse/pull/112690) ([Groene AI](https://github.com/groeneai)). * Fixed the logical error `SortingAggregatedTransform already got bucket with number N` that could occur in multi-layer distributed `GROUP BY` queries using two-level aggregation. [#112794](https://github.com/ClickHouse/ClickHouse/pull/112794) ([Nikita Taranov](https://github.com/nickitat)). * Fixes a `NATS` table with `nats_stream` set silently consuming nothing after the NATS server is restarted. The `JetStream` subscription is now re-established automatically instead of requiring `DETACH TABLE` and `ATTACH TABLE`. [#112828](https://github.com/ClickHouse/ClickHouse/pull/112828) ([Groene AI](https://github.com/groeneai)). * Fixed a transient Azure Blob Storage error on the read or merge path — an RBAC permission-propagation `403 Forbidden`, a credential authentication failure, or a connect timeout surfaced as `408` — being misclassified as non-retryable and reported as `POTENTIALLY_BROKEN_DATA_PART` (Code 740), raising a false critical alert for a healthy part. These transient failures are now treated as retryable, and a genuinely persistent one surfaces as a plain Azure error instead of a phantom broken data part. [#112871](https://github.com/ClickHouse/ClickHouse/pull/112871) ([Arsen Muk](https://github.com/arsenmuk)). * Fixed Azure batch object deletion recording no `system.blob_storage_log` Delete events when the batch request itself failed, leaving the whole batch unlogged. A Delete event is now recorded for each object on a batch-level failure. [#112873](https://github.com/ClickHouse/ClickHouse/pull/112873) ([Arsen Muk](https://github.com/arsenmuk)). * Fix the per-part minmax index over `_block_number` and `_block_offset` being lost after a table reload when the part was produced by a mutation that does not rewrite the whole part. [#112878](https://github.com/ClickHouse/ClickHouse/pull/112878) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a logical error and wrong results when a non equi `JOIN ON` condition that is evaluated during the join contains `arrayJoin`. A `hash`, `parallel_hash` or `grace_hash` join now rejects such a condition with `INVALID_JOIN_ON_EXPRESSION`. Where the expansion depends on one side only, move it into an `ARRAY JOIN` in a subquery before the join; a condition whose `arrayJoin` argument reads columns from both sides has to be restructured. [#112889](https://github.com/ClickHouse/ClickHouse/pull/112889) ([Groene AI](https://github.com/groeneai)). * Fix wrong results of `ORDER BY` when reading from a `Merge` table over `Distributed` tables: rows could be returned out of order, and rows belonging in the result could be missing when `LIMIT` was used. [#112931](https://github.com/ClickHouse/ClickHouse/pull/112931) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `Bool` constant on the right-hand side of `IN` being converted to a `String` left-hand side as `'1'`/`'0'` instead of the canonical `'true'`/`'false'` (now consistent with `CAST` and the `values` table function). [#113051](https://github.com/ClickHouse/ClickHouse/pull/113051) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fixes `views` column in `query_log` for inserts to an `Alias` table whose target triggers materialized views, which was always empty. [#113076](https://github.com/ClickHouse/ClickHouse/pull/113076) ([Enric Calabuig](https://github.com/eclbg)). * Fixed `LOGICAL_ERROR` when reading a table with a `set` skip index defined on an expression and the query repeats that expression with a `Nullable` operand, for example `WHERE intDiv(id, (SELECT 1000)) = 3` over `INDEX b_set intDiv(id, 1000) TYPE set(100)`. A scalar subquery is enough to trigger it, since it is typed `Nullable`. [#113244](https://github.com/ClickHouse/ClickHouse/pull/113244) ([George Larionov](https://github.com/george-larionov)). * Fix a logical error (exception) `Chunk info was not set for chunk in MergingAggregatedTransform` when the setting `inject_random_order_for_select_without_order_by` is enabled and an aggregation query reads from a `Merge` table containing a `Distributed` child: the random `ORDER BY rand()` wrapper is no longer injected into queries planned only up to an intermediate stage. [#113266](https://github.com/ClickHouse/ClickHouse/pull/113266) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a query plan optimization stall when a `WHERE` clause contains a large string constant with many dots and the table carries a `bloom_filter`, `tokenbf_v1`, `ngrambf_v1` or `text` skip index. Matching a filter column name against `JSONAllPaths(...)` index columns enumerated every dot split of the name, which made skip-index condition building quadratic in the constant's length. Closes [#113003](https://github.com/ClickHouse/ClickHouse/issues/113003). [#113289](https://github.com/ClickHouse/ClickHouse/pull/113289) ([Groene AI](https://github.com/groeneai)). * Fixed `ATTACH` of a `Kafka` table when `kafka_num_consumers` exceeds the limit derived from the number of CPU cores. [#113390](https://github.com/ClickHouse/ClickHouse/pull/113390) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed a permanent `DROP TABLE` hang and memory leak after a query using a materialized CTE failed. The table stayed stuck in the drop queue until the server was restarted. [#113397](https://github.com/ClickHouse/ClickHouse/pull/113397) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix a logical error `IDENTIFIER is not a table expression` when a remote-family table function with a `view(...)` argument is used on the left side of a `JOIN ... USING` whose key is a `SELECT` list alias, with `analyzer_compatibility_join_using_top_level_identifier = 1`. [#113400](https://github.com/ClickHouse/ClickHouse/pull/113400) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Paimon` tables partitioned by a `TIMESTAMP` column with a precision higher than milliseconds, which failed with `scale 6 is not supported, only support scale <= 3`. Closes [#112768](https://github.com/ClickHouse/ClickHouse/issues/112768). [#113401](https://github.com/ClickHouse/ClickHouse/pull/113401) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fixes a segfault and a `LOGICAL_ERROR` (`No host found for exchange stream final_result__0_0`) when a subquery sets `distributed_plan_execute_locally` in its own `SETTINGS` clause under `make_distributed_plan`. The distributed plan and the code executing it read the setting from two different contexts and could disagree, so the initiator built a self-contradictory pipeline. [#113422](https://github.com/ClickHouse/ClickHouse/pull/113422) ([Groene AI](https://github.com/groeneai)). * Fixed reading Paimon tables that contain a nullable `ARRAY` or `MAP` column. Such a table could not be read at all, because the schema mapper wrapped the composite type in `Nullable`, which ClickHouse forbids, so both `DESC` and `SELECT` failed with `Nested type Array(Nullable(Int32)) cannot be inside Nullable type`. A nullable composite column is now mapped to a non-`Nullable` composite type and a `NULL` value is read as an empty one. Closes [#113337](https://github.com/ClickHouse/ClickHouse/issues/113337). [#113450](https://github.com/ClickHouse/ClickHouse/pull/113450) ([Groene AI](https://github.com/groeneai)). * Fix part and granule pruning for a `JOIN ON` condition involving constant columns of the other side. [#113484](https://github.com/ClickHouse/ClickHouse/pull/113484) ([Vladimir Cherkasov](https://github.com/vdimir)). * Stopped writing a part whose `Map` layout contradicts its own metadata, for a `Map` value inside a `Dynamic` column. Tables differing only in `map_serialization_version` shared one cached serialization object, so a part could get the `basic` Map layout while its metadata declared `with_buckets`, and reading it later failed with `LOGICAL_ERROR: Stream ...buckets_info ... is not found`. Parts an earlier version already wrote this way stay unreadable and have to be dropped. [#113514](https://github.com/ClickHouse/ClickHouse/pull/113514) ([Groene AI](https://github.com/groeneai)). * Fixed the inferred column type depending on the order of previous queries for a source whose schema is cached. Settings that change an inferred type were missing from the schema inference cache key, so the first query's value decided the type every later query saw: `input_format_try_infer_exponent_floats`, `max_parser_depth`, `input_format_json_infer_array_of_dynamic_from_array_of_different_types`, and for `Parquet` also `input_format_parquet_local_time_as_utc`, `input_format_parquet_allow_geoparquet_parser`, `input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference` and `schema_inference_make_json_columns_nullable`. Also registers the missing cache key getter for `Form` and makes the `Template` getter use its own row format's escaping rule. [#113533](https://github.com/ClickHouse/ClickHouse/pull/113533) ([Groene AI](https://github.com/groeneai)). * Fix a theoretical data race in profile events when using the `trace_profile_events_list` setting to write stack traces of certain profile events to `system.trace_log`. [#113553](https://github.com/ClickHouse/ClickHouse/pull/113553) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed `RENAME DATABASE` accepting a target name long enough to make the database's tables impossible to drop. The table name length check now runs regardless of `check_table_dependencies`, and it also covers detached tables, which were never checked and so were affected even at default settings. Closes [#101747](https://github.com/ClickHouse/ClickHouse/issues/101747). [#113557](https://github.com/ClickHouse/ClickHouse/pull/113557) ([Groene AI](https://github.com/groeneai)). * Fixed the `LOGICAL_ERROR` `Reading from materialized CTE '...' before its materialization completed - DelayedPortsProcessor gate is missing in the query plan` raised when a `Merge` table with more than one child reads a materialized CTE that the outer query references. [#113558](https://github.com/ClickHouse/ClickHouse/pull/113558) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a row policy containing a scalar subquery being evaluated only once and then reused forever after the table had been read through a `Merge` table. The parsed policy condition is cached and shared by all queries, and reading it through `Merge` rewrote the cached expression in place, which also caused a data race between concurrent queries using the same policy. [#113563](https://github.com/ClickHouse/ClickHouse/pull/113563) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Unsupported JOIN keys of type keys256 in StorageJoin` when reading a `Join`-engine table whose key is composite (several columns) or a single 16/32-byte value such as `UUID`, `Int256` or `FixedString(4)`. Such a table could be created, written and joined against, but never read back with `SELECT`. Closes [#74362](https://github.com/ClickHouse/ClickHouse/issues/74362) and [#77394](https://github.com/ClickHouse/ClickHouse/issues/77394). [#113578](https://github.com/ClickHouse/ClickHouse/pull/113578) ([Groene AI](https://github.com/groeneai)). * In the Web UI, opening a link with `run=1` no longer makes other tabs' URLs auto-run their queries on the next page reload. [#113595](https://github.com/ClickHouse/ClickHouse/pull/113595) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error comparing arrays whose element type is Nothing. [#113609](https://github.com/ClickHouse/ClickHouse/pull/113609) ([Groene AI](https://github.com/groeneai)). * Backups now report the underlying lock-file access error instead of incorrectly reporting a concurrent backup when the destination storage cannot read the .lock file. [#113637](https://github.com/ClickHouse/ClickHouse/pull/113637) ([Kirill Shokhin](https://github.com/kirillshokhin)). * Fix a logical error `Sending a distributed query with unknown (zero) client version` when a structure-less `Distributed` table over a remote shard is attached or loaded from metadata written by an older server version. [#113675](https://github.com/ClickHouse/ClickHouse/pull/113675) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Treat a NUL byte as an ordinary literal in match() patterns. [#113690](https://github.com/ClickHouse/ClickHouse/pull/113690) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix `theilsU` over a window frame returning an arbitrary value instead of 0 when the first argument is constant within the frame. [#113691](https://github.com/ClickHouse/ClickHouse/pull/113691) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Logical error: Duplicate announcement received for replica number N` when automatic parallel replicas (`automatic_parallel_replicas_mode`) adopts a plan whose single-node variant was sharded by `query_plan_join_shard_by_pk_ranges`. [#113730](https://github.com/ClickHouse/ClickHouse/pull/113730) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CANNOT_CONVERT_TYPE` and an exception in `GroupingAggregatedTransform` when reading a `Merge` table with one child being a `Distributed` table under custom-key parallel replicas (`parallel_replicas_mode = '`custom_key_sampling`'` or `'custom_key_range'`). Closes [#113741](https://github.com/ClickHouse/ClickHouse/issues/113741). [#113742](https://github.com/ClickHouse/ClickHouse/pull/113742) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` when a `SELECT` used the `STREAM` modifier on a named table together with parallel replicas in the read-tasks mode. With a materialized CTE used as an `IN` set in `PREWHERE`, the server raised `Reading from materialized CTE ... DelayedPortsProcessor gate is missing in the query plan` (a server abort in debug and sanitizer builds). A table carrying `STREAM` is now rejected when parallel replicas are requested: at `enable_parallel_replicas = 2` the query fails with `SUPPORT_IS_DISABLED`, at `1` it runs without them, exactly as already happens for `FINAL`. [#113754](https://github.com/ClickHouse/ClickHouse/pull/113754) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash when joining with a constant `ON` expression against a right side that contains an `ARRAY JOIN`. Reading a lazily replicated right-side column used a freed `ColumnReplicated`. Closes [#113859](https://github.com/ClickHouse/ClickHouse/issues/113859). [#113901](https://github.com/ClickHouse/ClickHouse/pull/113901) ([Groene AI](https://github.com/groeneai)). * Fix undefined behavior and wrong results in `avg` over `Date`/`DateTime`/`DateTime64`/`Time`/`Time64`: the average is now computed exactly in integer space, fixing both the `Int64`-boundary overflow (UB, wrong result on x86) and `Float64` precision loss above 2^53 (visible at nanosecond scale). [#113912](https://github.com/ClickHouse/ClickHouse/pull/113912) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results in `JOIN` with `join_algorithm = 'partial_merge'` when the right side carries totals: the right side could be left unmerged, so `LEFT` joins substituted default values for real matches and `INNER`/`RIGHT` joins returned no rows at all. In the `RIGHT`/`FULL` non-joined path the same state dereferenced a null pointer, which crashed the server. [#113936](https://github.com/ClickHouse/ClickHouse/pull/113936) ([Groene AI](https://github.com/groeneai)). * Fixed a defect in the `MySQL` integrations where overriding a TLS credential of a named collection with an empty `ssl_ca_pem`/`ssl_cert_pem`/`ssl_key_pem` value was accepted when the collection stored the credential in the contents form, silently dropping the configured CA or client certificate instead of rejecting the override. [#113947](https://github.com/ClickHouse/ClickHouse/pull/113947) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a `std::out_of_range` exception (`Logical error: 'std::exception. Code: 1001, type: std::out_of_range'`, which also aborts the server in debug and sanitizer builds) in the Parquet v3 reader when a column needed only for filter evaluation was scheduled for decoding after its output slot had already been dropped. [#113956](https://github.com/ClickHouse/ClickHouse/pull/113956) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Unexpected return type`) when a filter over a view whose column type differs from the underlying storage (e.g. `information_schema.tables`, where `engine` is `Nullable(String)`, over `system.tables`, where it is `String`) is pushed down to the storage. [#113984](https://github.com/ClickHouse/ClickHouse/pull/113984) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix ClickHouse Keeper refusing to start with `CORRUPTED_DATA` after a crash during cross-segment Raft log truncation (`writeAt`), which could leave an acknowledged log entry stranded behind stale changelog segments. [#114003](https://github.com/ClickHouse/ClickHouse/pull/114003) ([Nishant Agarwal](https://github.com/nishant-uxs)). * A table or database whose name is substituted into a `ReplicatedMergeTree` ZooKeeper path through the `{database}` or `{table}` macro is now rejected if the name would not be a single safe path component. Previously a name containing `/` was spliced in unescaped, so the table registered its znodes inside another table's keeper subtree and permanently broke `ALTER` and `OPTIMIZE` there. A `.` or `..` component, a control character and macro syntax are rejected too. Existing tables keep loading and attaching. [#114006](https://github.com/ClickHouse/ClickHouse/pull/114006) ([Groene AI](https://github.com/groeneai)). * Fixes a memory leak with `optimize_aggregation_in_order = 1` when the table sorting key is a strict prefix of the `GROUP BY` key and an aggregate function whose state owns heap memory is used, such as `quantileDD`. Server memory grows with every such query until restart. [#114010](https://github.com/ClickHouse/ClickHouse/pull/114010) ([Groene AI](https://github.com/groeneai)). * Fixes a logical error when a distributed query plan contains a window function producing an aggregate function state, for example `theilsUState(a, b) OVER (ORDER BY a)` with `make_distributed_plan = 1`. Affects the `cramersV`, `cramersVBiasCorrected`, `theilsU` and `contingency` families. [#114020](https://github.com/ClickHouse/ClickHouse/pull/114020) ([Groene AI](https://github.com/groeneai)). * Fix wrong results for `SELECT DISTINCT` with a `LIMIT` that has to read its whole input: `LIMIT n WITH TIES` dropped tying rows, `exact_rows_before_limit = 1` reported a truncated `rows_before_limit_at_least`, and `GROUP BY ... WITH TOTALS` computed the totals row from a prefix of the stream. [#114025](https://github.com/ClickHouse/ClickHouse/pull/114025) ([Groene AI](https://github.com/groeneai)). * Fixed `isProbablePrime` on `UInt128`/`UInt256` ignoring `max_execution_time` and `KILL QUERY`. A block of wide values ran to completion inside a single function call with no cancellation point, so a query could run for minutes past its deadline. [#114032](https://github.com/ClickHouse/ClickHouse/pull/114032) ([Groene AI](https://github.com/groeneai)). * Fixes a missing `system.query_views_log` row for a materialized view that fails while its dependencies are being collected with `materialized_views_ignore_errors` enabled. In debug and sanitizer builds this also aborts the server with `std::out_of_range` instead of logging the view. [#114064](https://github.com/ClickHouse/ClickHouse/pull/114064) ([Groene AI](https://github.com/groeneai)). * Fixed `Code: 46. DB::Exception: Unknown function exists. (UNKNOWN_FUNCTION)` thrown when a `PREWHERE` clause contains an `IN (subquery)` predicate and `rewrite_in_to_join = 1` (or `make_distributed_plan = 1`, which force-enables it) is set. The same query spelled with `WHERE` worked correctly. [#114067](https://github.com/ClickHouse/ClickHouse/pull/114067) ([RohithPariki](https://github.com/RohithPariki)). * Fixed `ORDER BY ... LIMIT` returning fewer rows than requested, possibly none, when a row policy was the only filter of the query and the sort column had a `minmax` skip index. The top-K optimization narrowed the read before the row policy was applied. [#114073](https://github.com/ClickHouse/ClickHouse/pull/114073) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes `hasAny`, `hasAll`, `has`, `indexOf`, `mapContainsKey`, `mapContainsValue` and `mapContains` returning too few rows, or failing with `TOO_LARGE_STRING_SIZE`, when a `bloom_filter` index is queried with a `FixedString` constant. The index hashed the padded form of the constant while the function compares the unpadded one, so a matching granule was skipped. [#114089](https://github.com/ClickHouse/ClickHouse/pull/114089) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` "Not-ready Set is passed as the second argument for function `globalNullIn`" and the loss of the query's real error message, which could happen when statistics part pruning speculatively executed an `IN`/`GLOBAL IN` subquery during index analysis and that subquery failed. [#114121](https://github.com/ClickHouse/ClickHouse/pull/114121) ([Groene AI](https://github.com/groeneai)). * Respect the `allow_calculating_subcolumns_sizes_for_merge_tree_reading` setting in the PREWHERE optimization. [#114146](https://github.com/ClickHouse/ClickHouse/pull/114146) ([Pavel Kruglov](https://github.com/Avogar)). * Reject corrupted/malicious dictionary where `lcp` exceeds the previous token or `lcp + data_size` overflows, before the buffer write. [#114155](https://github.com/ClickHouse/ClickHouse/pull/114155) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix `min` and `max` on `DateTime64` columns returning wrong results once the aggregate is JIT compiled and the data contains timestamps before 1970. `is_signed` was not specialised for `DateTime64` and `Time64`, which derive from `Decimal64`, so the generated code compared tick counts as unsigned integers and a pre-1970 timestamp won `max` and lost `min`. [#114168](https://github.com/ClickHouse/ClickHouse/pull/114168) ([Groene AI](https://github.com/groeneai)). * Fixed an exception when reading a `MergeTree` table by primary-key range layers when a layer produces an empty pipe. [#114177](https://github.com/ClickHouse/ClickHouse/pull/114177) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the error code reported for some malformed `Replicated`-serialized columns received over the Native protocol: these are now rejected with `INCORRECT_DATA` instead of `LOGICAL_ERROR` (which aborted the server in debug and sanitizer builds). [#114186](https://github.com/ClickHouse/ClickHouse/pull/114186) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed `ATTACH PARTITION FROM`, `REPLACE PARTITION`, `MOVE PARTITION TO TABLE` and adding a `ReplicatedMergeTree` replica failing with `Tables have different ...`, `METADATA_MISMATCH` or `INCOMPATIBLE_COLUMNS` for tables whose definitions were written with redundant parentheses, such as `PARTITION BY (a)`, `ORDER BY (b)`, `INDEX ix (b * c) TYPE minmax`, `PROJECTION p (SELECT (b) ...)`, `CONSTRAINT c CHECK (a > 0)`, `TTL (d + INTERVAL 10 YEAR)` or `DEFAULT (a + 1)`. [#114188](https://github.com/ClickHouse/ClickHouse/pull/114188) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `ALP`, `FPC` and `GCD` omitting their type-derived parameter from the codec hash. In a Compact part the writer shares one compressed stream per distinct codec hash, so two columns of different value width under the same codec collided and shared one codec object carrying the wrong width. This rejected valid inserts with `CANNOT_COMPRESS` when the widths disagreed, and otherwise compressed one of the columns worse than it should have. [#114189](https://github.com/ClickHouse/ClickHouse/pull/114189) ([Groene AI](https://github.com/groeneai)). * Fix `std::out_of_range: map::at: key not found` when a `MergeTree` key expression contains a `mortonEncode` or `hilbertEncode` call that is not itself a key column, for example `ORDER BY mortonEncode(hilbertEncode(x, y), x, y)` or `PARTITION BY mortonEncode(x, y) % 4`. [#114233](https://github.com/ClickHouse/ClickHouse/pull/114233) ([Groene AI](https://github.com/groeneai)). * Clear plain LIMIT/OFFSET in the window-view backfill source query. [#114247](https://github.com/ClickHouse/ClickHouse/pull/114247) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` in `arrayAutocorrelation` when the argument is a non-empty array whose element type is `Nothing`, for example `SELECT arrayAutocorrelation([arrayMax([])])`. Such an argument is now rejected with `ILLEGAL_COLUMN`. The empty array literal `[]` keeps returning `[]`. [#114249](https://github.com/ClickHouse/ClickHouse/pull/114249) ([Groene AI](https://github.com/groeneai)). * Fix a Keeper server never joining the cluster when `nuraft_use_bg_thread_for_snapshot_io` is enabled: the log batch sent to a joining server was silently dropped, so the membership change never completed. [#114275](https://github.com/ClickHouse/ClickHouse/pull/114275) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix `XDG_CACHE_HOME` being read from `XDG_STATE_HOME`. [#114278](https://github.com/ClickHouse/ClickHouse/pull/114278) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Register Iceberg namespace in the catalog before writing table files (needed for SeaweedFS). [#114285](https://github.com/ClickHouse/ClickHouse/pull/114285) ([Azat Khuzhin](https://github.com/azat)). * Fix inconsistent results when a filter on a non-key column of the right table was pushed down below an ANY INNER JOIN. [#114313](https://github.com/ClickHouse/ClickHouse/pull/114313) ([Kirill Shokhin](https://github.com/kirillshokhin)). * Fix bash completion with bash-completion 2.12+ (_comp_get_words). [#114339](https://github.com/ClickHouse/ClickHouse/pull/114339) ([Azat Khuzhin](https://github.com/azat)). * Fixes a crash when reading a data lake table whose schema declares a column with an empty name. Malformed Iceberg metadata is rejected with `ICEBERG_SPECIFICATION_VIOLATION`; a schema supplied by a catalog, Delta Lake or Paimon is rejected with `AMBIGUOUS_COLUMN_NAME`. The check is unconditional, so an Iceberg table that merely retains an unused historical schema with an empty field name also becomes unreadable instead of aborting once that schema is read. [#114394](https://github.com/ClickHouse/ClickHouse/pull/114394) ([Groene AI](https://github.com/groeneai)). * Fixes a bug where a client connecting to ClickHouse Keeper during a Raft leader change could be held for the whole `session_timeout_ms` (30 seconds by default) before its connection was rejected, instead of being rejected as soon as the in-flight request was dropped. The connecting client can now reconnect to another replica sooner. [#114401](https://github.com/ClickHouse/ClickHouse/pull/114401) ([Groene AI](https://github.com/groeneai)). * `BACKUP`, `RESTORE` and `ENGINE = Backup` now authorize the backup location against the user's `SOURCES` grants, the same way `s3()`, `file()` and `azureBlobStorage()` do. Previously a user holding `BACKUP` but not `SOURCES` could write a backup to, and read one from, any location the server could reach. Writing a backup now requires the `WRITE` direction on the destination's source (`WRITE ON S3`, `WRITE ON AZURE`, `WRITE ON FILE`) and reading one requires `READ`; `Disk(...)` destinations stay restricted by `backups.allowed_disk` only. [#114405](https://github.com/ClickHouse/ClickHouse/pull/114405) ([Groene AI](https://github.com/groeneai)). * Fixes `CREATE OR REPLACE` leaking an internal `_tmp_replace_*` table, and `CREATE OR REPLACE VIEW` failing with `NOT_IMPLEMENTED`, when `ignore_drop_queries_probability` is set. The DROPs it issues internally are steps of one user statement, so DROP fault injection no longer applies to them. [#114420](https://github.com/ClickHouse/ClickHouse/pull/114420) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Bad cast from type DB::ASTLiteral to DB::ASTPartition`) and unparseable formatted SQL when an `ALTER TABLE ... MOVE` command deserialized from a JSON AST (`dialect = clickhouse_json`) paired the `PART` form with `TO TABLE`, or the `PARTITION` form with `TO SHARD`. Such payloads are now rejected with `BAD_ARGUMENTS`. [#114429](https://github.com/ClickHouse/ClickHouse/pull/114429) ([Groene AI](https://github.com/groeneai)). * Do not remove columns when it's referenced in both `PREWHERE` and `WHERE` filter conditions even if it's rewritten into a text index virtual column. [#114460](https://github.com/ClickHouse/ClickHouse/pull/114460) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed a `Not-ready Set is passed as the second argument for function 'in'` error when a mutation whose predicate contains `IN (subquery)` is cancelled, for example by `KILL MUTATION` or `DETACH DATABASE`, while the subquery's set is still being built. Related: [#107619](https://github.com/ClickHouse/ClickHouse/issues/107619). [#114463](https://github.com/ClickHouse/ClickHouse/pull/114463) ([Groene AI](https://github.com/groeneai)). * `ALTER TABLE ... DROP COLUMN` of a column that is used in the sorting, primary or partition key now fails with `ALTER_OF_COLUMN_IS_FORBIDDEN` and an explanation, the same as `ALTER TABLE ... CLEAR COLUMN` does. Previously it failed with a confusing `UNKNOWN_IDENTIFIER: Missing columns` error coming from the recalculation of the key expressions. The same applies to dropping or clearing a whole `Nested` group by its common prefix when a column of the group is used in a key; previously that could silently destroy the group's data or leave a mutation that never finishes. Dropping a column that is used only in a `TTL` expression now also reports which TTL expression it breaks. [#114468](https://github.com/ClickHouse/ClickHouse/pull/114468) ([Maksim Dergousov](https://github.com/m7kss1)). * Fixed reading a `Tuple` column whose first element is `NULL` in the `CustomSeparated`, `Regexp` and `Template` formats with the `CSV` escaping rule. A bare `Tuple` occupies one field per element there, so the leading `\N` field is that element and not the whole column. Previously it was consumed as the whole column, shifting the remaining element fields onto the following columns, so `CustomSeparated` could not read back its own output. Closes [#114402](https://github.com/ClickHouse/ClickHouse/issues/114402). [#114471](https://github.com/ClickHouse/ClickHouse/pull/114471) ([Groene AI](https://github.com/groeneai)). * Fixes `arrayExists(x -> x = needle, arr)` returning a wrong result, or failing with `TOO_LARGE_STRING_SIZE`, when the array element and the needle are different string types (for example a `String` needle against an `Array(FixedString(N))` element). The `optimize_rewrite_array_exists_to_has` optimization, enabled by default, rewrote such a call to `has`, which does not compare zero-padded the way `=` does. [#114496](https://github.com/ClickHouse/ClickHouse/pull/114496) ([Groene AI](https://github.com/groeneai)). * Reading an Iceberg table whose metadata describes a schema evolution that the Iceberg specification forbids no longer aborts the server. Four spec violations in `IcebergSchemaProcessor` were reported as `LOGICAL_ERROR`, which is treated as a failed assertion, or reached a fatal assertion unchecked; they now raise `ICEBERG_SPECIFICATION_VIOLATION`. [#114518](https://github.com/ClickHouse/ClickHouse/pull/114518) ([Groene AI](https://github.com/groeneai)). * Fixed the `Regexp` input format silently discarding the unparsed rest of a matched field, which stored a truncated or fabricated value instead of reporting an error: `v=-1` into `UInt64` read `0`, and `v=2020-01-01junk` into `Date` read `1975-07-14` under the `JSON` rule. Malformed fields are now rejected under the `Escaped`, `CSV` and `JSON` rules, matching the formats those rules are documented as behaving like. Under `Raw`, a matched field containing a tab is now read as a whole instead of being truncated at the tab. [#114520](https://github.com/ClickHouse/ClickHouse/pull/114520) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` when filtering an Iceberg column that `ALTER TABLE ... MODIFY COLUMN` made `Nullable`. Closes [#85029](https://github.com/ClickHouse/ClickHouse/issues/85029). [#114521](https://github.com/ClickHouse/ClickHouse/pull/114521) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE` and `ALTER TABLE` now reject lossy codec such as `SZ3` on a column used in the sorting key. [#114531](https://github.com/ClickHouse/ClickHouse/pull/114531) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` when reading a Delta Lake table whose `metaData.partitionColumns` names a column that `metaData.schemaString` does not declare. Such metadata is now rejected with `BAD_ARGUMENTS` on the delta-kernel reader and `INCORRECT_DATA` on the legacy reader, whenever the table or its schema is read from a snapshot, not only when the query carries a predicate. [#114538](https://github.com/ClickHouse/ClickHouse/pull/114538) ([Groene AI](https://github.com/groeneai)). * Reject a Variant whose ORC branches read back as one type. [#114540](https://github.com/ClickHouse/ClickHouse/pull/114540) ([Groene AI](https://github.com/groeneai)). * Fix text index evaluation of `LIKE/ILIKE` operator built on Map or JSON containers. [#114544](https://github.com/ClickHouse/ClickHouse/pull/114544) ([Elmi Ahmadov](https://github.com/ahmadov)). * **Support quoted label names in PromQL grouping modifiers.**. [#114545](https://github.com/ClickHouse/ClickHouse/pull/114545) ([Minh Vu](https://github.com/fallintoplace)). * Fixes incorrect, often sign-flipped, `Time64` literals and `IN`-list constants produced when rescaling a lower-scale `Decimal64` overflowed `Int64`. Such a conversion now reports `DECIMAL_OVERFLOW`, matching the `DateTime64` branch and explicit `CAST`. [#114546](https://github.com/ClickHouse/ClickHouse/pull/114546) ([Groene AI](https://github.com/groeneai)). * **Accept trailing commas in PromQL grouping label lists.**. [#114548](https://github.com/ClickHouse/ClickHouse/pull/114548) ([Minh Vu](https://github.com/fallintoplace)). * Support quoted metric and label names in PromQL selectors. [#114551](https://github.com/ClickHouse/ClickHouse/pull/114551) ([Minh Vu](https://github.com/fallintoplace)). * A query could be cancelled up to 1 ms before its `max_execution_time` had passed, failing with a self-contradictory `Timeout exceeded: elapsed 999.672 ms, maximum: 1000 ms`. [#114559](https://github.com/ClickHouse/ClickHouse/pull/114559) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed reading a table while a mutation is in progress: a `MATERIALIZED` column could return a stale value, and selecting one together with an updated column could fail with an error. [#114561](https://github.com/ClickHouse/ClickHouse/pull/114561) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixes `clickhouse-local` accepting a `CREATE TABLE` whose name is too long for the configured `--default_database`, which produced a table that could not be dropped. `DatabaseOverlay` now checks the table name length like the on-disk database it writes to. [#114567](https://github.com/ClickHouse/ClickHouse/pull/114567) ([Groene AI](https://github.com/groeneai)). * Fixed an RBAC bypass in the `Alias` table engine that allowed users without privileges on the target table to reveal its schema, row count, size, and existence. [#114596](https://github.com/ClickHouse/ClickHouse/pull/114596) ([Kai Zhu](https://github.com/nauu)). * Fixed hash-table statistics — and with them aggregation hash-table preallocation — being silently and permanently disabled for the whole server when a query using the lazy `FINAL` optimization (`query_plan_optimize_lazy_final`) was the first aggregation to run after server startup. [#114597](https://github.com/ClickHouse/ClickHouse/pull/114597) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a column that was renamed, dropped and added again in one `ALTER` returning the dropped column's values instead of its own default while the mutation was still pending. [#114601](https://github.com/ClickHouse/ClickHouse/pull/114601) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed an infinite, uncancellable loop in functions `hop` and `windowID` when the span of an interval argument in seconds is a multiple of 2^32 (for example, `toIntervalDay(2147483648)`): the wrapped subtraction dodged the time-overflow check, and with constant arguments the loop ran at analysis time, where the query could not even be killed. The same interval could also spin a background thread of a `WINDOW VIEW` forever; such a window view is now rejected at creation. [#114607](https://github.com/ClickHouse/ClickHouse/pull/114607) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a bug where an access entity carrying a `Map`-valued setting, such as a settings profile with `http_response_headers` or `additional_table_filters`, is stored in a form that ClickHouse cannot read back, leaving the entity permanently unloadable after a restart. [#114620](https://github.com/ClickHouse/ClickHouse/pull/114620) ([Groene AI](https://github.com/groeneai)). * Fixed a refreshable materialized view leaking its rotated-out target table when `ignore_drop_queries_probability` is enabled. The `DROP` a refresh issues to clean up the previous target is a step of the refresh, not a `DROP` the user asked for, so the fault injection no longer applies to it. [#114622](https://github.com/ClickHouse/ClickHouse/pull/114622) ([Groene AI](https://github.com/groeneai)). * Fixes `has`, `indexOf`, `countEqual`, `mapContainsKey`, `mapContainsValue` and `Map` subscript returning "not found" for a constant `LowCardinality` needle equal to the element type's default value, such as an empty `String` or a zero number. [#114624](https://github.com/ClickHouse/ClickHouse/pull/114624) ([Groene AI](https://github.com/groeneai)). * Fixed logical error `Multi-block postings must be compressed` in queries over tables with a text index in the lazy posting-list apply mode, when the query was canceled during the read. [#114636](https://github.com/ClickHouse/ClickHouse/pull/114636) ([Anton Popov](https://github.com/CurtizJ)). * Fixed formatting of a subquery argument of the `view` and `viewIfPermitted` table functions when the enclosing query has a trailing `SETTINGS` clause. Such a query was formatted as `view((SELECT ...))`, which cannot be parsed back, so the query failed the internal format-parse-format check and raised `Inconsistent AST formatting`. [#114658](https://github.com/ClickHouse/ClickHouse/pull/114658) ([Groene AI](https://github.com/groeneai)). * Fixed heap memory corruption when reading Parquet through an input format that owns its read buffer, for example a dictionary with `SOURCE(FILE(... format 'Parquet'))`. Background prefetch and decode tasks could still read and write through the buffer after the pipeline released it, which could abort the server. [#114668](https://github.com/ClickHouse/ClickHouse/pull/114668) ([Groene AI](https://github.com/groeneai)). * Fix reading a subcolumn (`.size0`, `.null`, a tuple element, `String.size`) through a `Merge` table when the underlying table declares the parent column as `ALIAS`. Such a read returned the type default, and when the parent was selected in the same query it could return the value of an unrelated column. [#114673](https://github.com/ClickHouse/ClickHouse/pull/114673) ([Groene AI](https://github.com/groeneai)). * Fix `ANY RIGHT JOIN` and `SEMI RIGHT JOIN` with several `OR`-ed conditions in the `ON` section returning some rows of the right table twice and losing the matches of others. A written `ANY LEFT JOIN` was affected too, because the planner may swap the tables and execute it as `ANY RIGHT JOIN`. [#114676](https://github.com/ClickHouse/ClickHouse/pull/114676) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed wrong results of the trivial `GROUP BY ... LIMIT` optimization (setting `optimize_trivial_group_by_limit_query`) for queries with `DISTINCT`, `QUALIFY`, window functions, or `arrayJoin` in the projection: these consume or filter the groups after the aggregation, so capping the aggregation at `LIMIT + OFFSET` keys could return too few rows or wrong values. The optimization no longer applies to such queries. [#114695](https://github.com/ClickHouse/ClickHouse/pull/114695) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed rows missing from the result of a `SELECT` filtered by `pointInPolygon` over a MergeTree primary key when the polygon argument is an invalid constant literal and `validate_polygons = 0`. Primary key analysis derived granule pruning from a polygon it never validated, and dropped granules holding matching rows. [#114710](https://github.com/ClickHouse/ClickHouse/pull/114710) ([Groene AI](https://github.com/groeneai)). * Fixes a server crash that happens when aggregate states of the `-OrNull` or `-OrDefault` combinators are merged with `group_by_overflow_mode = 'any'` and a non-zero `max_rows_to_group_by`, for example when a query reads through an aggregate projection. [#114740](https://github.com/ClickHouse/ClickHouse/pull/114740) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` ("Unexpected number of kept output positions after removing unused columns from `ReadFromMergeTree`") raised by a `SELECT ... FINAL` whose `PREWHERE` uses a column that `FINAL` needs for merging (`ver`, `is_deleted`, `sign`) and which does not select that column. [#114746](https://github.com/ClickHouse/ClickHouse/pull/114746) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Block structure mismatch`) when reading a `Merge` table with `make_distributed_plan` enabled over children with internal set operations. [#114753](https://github.com/ClickHouse/ClickHouse/pull/114753) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject an absurdly large `kafka_num_consumers` with a clear error instead of failing an allocation inside the `Kafka` table engine. Previously, with `kafka_disable_num_consumers_limit` enabled, such a value produced a `std::length_error` exception. [#114764](https://github.com/ClickHouse/ClickHouse/pull/114764) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR: Context has expired` exception when a stored expression containing `accurateCastOrDefault` or a `toOrDefault` function is evaluated after the query that created it has finished, for example a `MATERIALIZED` column default on `INSERT` or a mutation expression. [#114769](https://github.com/ClickHouse/ClickHouse/pull/114769) ([Groene AI](https://github.com/groeneai)). * Fixes a `LOGICAL_ERROR` (`Cannot find input column ... on its position in inputs of expression actions DAG`) when a query has a `JOIN` above a table whose text-indexed column is used by both a `PREWHERE` and a `WHERE` text-search predicate. [#114771](https://github.com/ClickHouse/ClickHouse/pull/114771) ([Groene AI](https://github.com/groeneai)). * Fix lost wakeup of restarting thread on startup from the attach thread. [#114802](https://github.com/ClickHouse/ClickHouse/pull/114802) ([Azat Khuzhin](https://github.com/azat)). * `OPTIMIZE TABLE` on an object-storage table whose engine does not implement compaction reported success while doing nothing. It now raises an error, like every other engine that does not support the statement. This affects `DeltaLake`, `Paimon`, `Hudi` and plain object-storage engines (`S3`, `GCS`, `COSN`, `OSS`, `AzureBlobStorage`, `HDFS`, and object-storage-backed `URL`); `Iceberg`, which does implement compaction, is unchanged. Closes [#114765](https://github.com/ClickHouse/ClickHouse/issues/114765). [#114805](https://github.com/ClickHouse/ClickHouse/pull/114805) ([Groene AI](https://github.com/groeneai)). * `timeSeriesRange` and `timeSeriesFromGrid` threw a false `DECIMAL_OVERFLOW` for timestamps before 1970 because the start/end timestamps were read as unsigned values. Also, for `DateTime`/`UInt32` timestamps the step was silently truncated to its lower 32 bits. Now all the calculations are done in `Int64`. [#114815](https://github.com/ClickHouse/ClickHouse/pull/114815) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixes a server abort (`Logical error: Part patch-... doesn't exist`) when a `DROP TABLE` runs concurrently with a lightweight `UPDATE` on a `ReplicatedMergeTree` table. The update now holds the table's shared lock until its patch part is committed, so the `DROP` waits instead of removing it. [#114826](https://github.com/ClickHouse/ClickHouse/pull/114826) ([Groene AI](https://github.com/groeneai)). * Fix few bugs in async inserts (possible crash in case of some queries failed with `MEMORY_LIMIT_EXCEEDED` and never worked `async_insert_queue_flush_on_shutdown`). [#114839](https://github.com/ClickHouse/ClickHouse/pull/114839) ([Azat Khuzhin](https://github.com/azat)). * Fixed `LOGICAL_ERROR: Unexpected return type from tuple. Expected Tuple(..., UInt8). Got Tuple(..., LowCardinality(UInt8))` when a `LowCardinality` column compared with `IN (subquery)` is read through `tuple(...)` across a subquery boundary and `enable_analyzer = 0`. With the old analyzer, `x IN (subquery)` over a `LowCardinality` column now returns `LowCardinality(UInt8)`, the same type as `x IN (literal list)` and the same as with `enable_analyzer = 1`. [#114856](https://github.com/ClickHouse/ClickHouse/pull/114856) ([Nikita Fomichev](https://github.com/fm4v)). * The AI-function quota settings (`ai_function_max_api_calls_per_query`, `ai_function_max_input_tokens_per_query`, `ai_function_max_output_tokens_per_query`) were enforced per block rather than per query, letting a query exceed the configured limit in proportion to the number of blocks and threads processed. They now bound the whole query per server: the API-call limit is an exact cap (atomic reservation), and the token limits are best-effort. A distributed query enforces the limits independently on each participating server / fragment. [#114885](https://github.com/ClickHouse/ClickHouse/pull/114885) ([George Larionov](https://github.com/george-larionov)). * Fixes a server abort that happens when the global thread pool is momentarily saturated while tables are still loading. `AsyncLoader` now keeps running if it cannot spawn an extra loader worker and some worker will still drain the queued jobs, instead of calling `std::terminate`. [#114897](https://github.com/ClickHouse/ClickHouse/pull/114897) ([Groene AI](https://github.com/groeneai)). * Fixed undefined behavior and incorrect boundary results in `quantile` and `quantiles` for `DateTime64` and wide `Decimal` values. [#114919](https://github.com/ClickHouse/ClickHouse/pull/114919) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix server abort on a corrupt ORC stripe footer. [#114924](https://github.com/ClickHouse/ClickHouse/pull/114924) ([Groene AI](https://github.com/groeneai)). * Fix possible crash due to ThreadGroup use-after-free (for async INSERTs/MVs/EXPLAIN ANALYZE). [#114940](https://github.com/ClickHouse/ClickHouse/pull/114940) ([Azat Khuzhin](https://github.com/azat)). * Fixed a hang when dropping a `TimeSeries` table whose name sorts lexicographically below its inner tables' names, for example `` DROP TABLE `-ts` `` or dropping a materialized view declared with `ENGINE = TimeSeries`. The drop self-deadlocked on the DDL guard and could not be cancelled with `KILL QUERY`. [#114953](https://github.com/ClickHouse/ClickHouse/pull/114953) ([Groene AI](https://github.com/groeneai)). * Fix `BAD_DATA_PART_NAME` ("Trying to get part name in new format for old format version") on a `ReplicatedMergeTree` created with the deprecated positional syntax under `allow_deprecated_syntax_for_merge_tree`. A mutation could be reported as done on a replica that had not applied it, and the table could stay readonly after a restart. [#114980](https://github.com/ClickHouse/ClickHouse/pull/114980) ([Groene AI](https://github.com/groeneai)). * Fix the logical error `Not-ready Set is passed as the second argument for function 'in'` for a query with an `IN` subquery reading from a remote cluster: the query now fails with the actual error from the subquery, such as a connection failure. [#114983](https://github.com/ClickHouse/ClickHouse/pull/114983) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error for IN with an identical non-constant tuple on both sides in the old analyzer. [#114988](https://github.com/ClickHouse/ClickHouse/pull/114988) ([Groene AI](https://github.com/groeneai)). * Fix reading a tuple element through a positional Tuple structure hint. [#115000](https://github.com/ClickHouse/ClickHouse/pull/115000) ([Groene AI](https://github.com/groeneai)). * Do not alias pre-aggregation states of a zero-size aggregate in arrayReduceInRanges. [#115008](https://github.com/ClickHouse/ClickHouse/pull/115008) ([Groene AI](https://github.com/groeneai)). * Fixed an `INSERT` into a `Log`, `TinyLog` or `StripeLog` table that fails while committing the recorded file sizes. Such an insert could keep its rows in the table even though it reported an error, or leave the data files of an array column inconsistent with each other. Reading a `Log` or `TinyLog` table whose array column holds no elements while its offsets claim some now raises an error naming the column instead of reading past the end of the elements. A partially written elements column was already rejected. [#115027](https://github.com/ClickHouse/ClickHouse/pull/115027) ([Groene AI](https://github.com/groeneai)). * Fix an adaptive aggregator abort on lazily replicated columns. [#115034](https://github.com/ClickHouse/ClickHouse/pull/115034) ([Groene AI](https://github.com/groeneai)). * Fixed an out-of-bounds read in `pointInPolygon` with a constant polygon whose bounding box is unbounded, for example when a coordinate reaches `+-DBL_MAX`. Such a polygon is now rejected with `BAD_ARGUMENTS`, and a polygon with an empty ring is treated as having no interior. Previously the query could return a wrong result, or abort the server in builds with hardening or sanitizers enabled. [#115035](https://github.com/ClickHouse/ClickHouse/pull/115035) ([Groene AI](https://github.com/groeneai)). * Fixed an `out_of_range` exception raised as an internal `Code: 1001` error when a `sequenceMatch`, `sequenceCount` or `sequenceMatchEvents` pattern contains the event number `0`, for example `sequenceMatch('(?0)')`. Such a pattern is now rejected with `BAD_ARGUMENTS`. A temporal condition holding a lone sign, such as `sequenceMatch('(?1)(?t>+)(?2)')`, was silently treated as `(?t>0)` and is now rejected with `SYNTAX_ERROR`. [#115056](https://github.com/ClickHouse/ClickHouse/pull/115056) ([Groene AI](https://github.com/groeneai)). * Fixed `Found patch part ... that intersects mutation with version ...` (`LOGICAL_ERROR`) on tables with lightweight updates. A merge of patch parts could produce a patch whose range of data versions spans the data version of an existing part; that patch can then be neither applied nor skipped, so every later operation on the partition failed and the replication queue stalled. Such merges are no longer assigned. [#115064](https://github.com/ClickHouse/ClickHouse/pull/115064) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a `LOGICAL_ERROR` (`Cache file segment is in detached state`) when reading through the filesystem cache with the experimental `use_reader_executor` setting. The regression was introduced by [#110029](https://github.com/ClickHouse/ClickHouse/issues/110029) and is not in any release, so no backport is owed. [#115070](https://github.com/ClickHouse/ClickHouse/pull/115070) ([Groene AI](https://github.com/groeneai)). * Arrow Flight server: accept `Basic` authentication credentials encoded in Base64 without padding, as sent by Go-based Flight clients such as the ADBC Flight SQL driver, and return a proper authentication error instead of a generic `Unexpected error in RPC handling` when the `authorization` header is malformed. [#115085](https://github.com/ClickHouse/ClickHouse/pull/115085) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix autocompletion (hints and TAB completion from `system.completions`) in the embedded client used by the SSH interface and the Web terminal: suggestions were loaded through a separate connection that authenticated as the `default` user with an empty password, which failed on servers where `default` has a password, and otherwise leaked names of entities the logged-in user has no access to. Suggestions are now loaded through the session's own connection. [#115086](https://github.com/ClickHouse/ClickHouse/pull/115086) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server that refuses to start after a table was created with `CREATE TABLE ... AS url('http://host/**/', ...)` while `allow_experimental_url_wildcard_from_index_pages` was enabled. Loading such a table's metadata re-evaluated the experimental check and failed with `SUPPORT_IS_DISABLED`, aborting startup. [#115095](https://github.com/ClickHouse/ClickHouse/pull/115095) ([Groene AI](https://github.com/groeneai)). * Rewrite bucketIndexForTimestamp in 64-bit arithmetic. [#115097](https://github.com/ClickHouse/ClickHouse/pull/115097) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed the formatting of `ALTER TABLE ... MATERIALIZE INDEX`, `MATERIALIZE STATISTICS` and `MATERIALIZE PROJECTION`, which dropped the `IF EXISTS` clause, so the formatted query was not the query that was parsed. Also rejected four `ALTER ... STATISTICS` AST JSON payloads that the SQL parser cannot produce and that formatted into a different command. [#115099](https://github.com/ClickHouse/ClickHouse/pull/115099) ([Groene AI](https://github.com/groeneai)). * Fixed an exception (`Block structure mismatch in Expression for FilterSortedStreamByRange`) for some `SELECT ... FINAL` queries with `PREWHERE` over a table whose sorting key is an expression. [#115113](https://github.com/ClickHouse/ClickHouse/pull/115113) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix clickhouse-local CORS CLI overrides. [#115129](https://github.com/ClickHouse/ClickHouse/pull/115129) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Disable distributed index analysis with projections (wrong results). [#115132](https://github.com/ClickHouse/ClickHouse/pull/115132) ([Azat Khuzhin](https://github.com/azat)). * Fixed `if` and `multiIf` returning a large positive value instead of a negative one when a `Time` branch is combined with `Time64`, `DateTime` or `DateTime64` and the expression is JIT-compiled. [#115146](https://github.com/ClickHouse/ClickHouse/pull/115146) ([Groene AI](https://github.com/groeneai)). * A web terminal embedded in another page, such as the SQL console or `play.html`, receives its credentials from that page and no longer opens a connection of its own before they arrive. Previously it connected twice - once with an empty password, then again with the real credentials - and printed `Connecting...` twice. [#115148](https://github.com/ClickHouse/ClickHouse/pull/115148) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed wrong results when a `WHERE` clause contained a `NULL` expression under a `NOT`, for example `SELECT count() FROM t WHERE NOT (id >= 20 AND NULL)`. Index analysis treated such a condition as provably true for the whole range, so the filter was skipped: `count()` returned rows the query rejects, and `ALTER TABLE ... DELETE` with that condition removed rows it should have kept. [#115152](https://github.com/ClickHouse/ClickHouse/pull/115152) ([Groene AI](https://github.com/groeneai)). * Fixed `EXPLAIN ANALYZE` losing the join statistics of a view declared `SQL SECURITY DEFINER` or `SQL SECURITY NONE` whose body contains a JOIN. In debug and sanitizer builds the query aborted with the logical error `JoinStep analyzed without the analyze mode`. [#115156](https://github.com/ClickHouse/ClickHouse/pull/115156) ([Groene AI](https://github.com/groeneai)). * Fixed an out-of-bounds read while deserializing corrupted posting lists of a text index. [#115222](https://github.com/ClickHouse/ClickHouse/pull/115222) ([Anton Popov](https://github.com/CurtizJ)). * Fixed wrong results for `SELECT count(arrayJoin(arr))` with the default `optimize_trivial_count_query = 1`. The stored row count was returned instead of the number of array elements, and on `file()` and `url()` the query returned `0`. Closes [#115123](https://github.com/ClickHouse/ClickHouse/issues/115123). [#115227](https://github.com/ClickHouse/ClickHouse/pull/115227) ([Groene AI](https://github.com/groeneai)). * Fixes `NOT_FOUND_COLUMN_IN_BLOCK` on every `INSERT` into a table that declares `STATISTICS(...)` on an `ALIAS` or `EPHEMERAL` column, which made such tables read-only. [#115231](https://github.com/ClickHouse/ClickHouse/pull/115231) ([Groene AI](https://github.com/groeneai)). * Fix the final statistics of a finished query (`N rows in result`, `Read ... rows`) being rendered in an invisible color in the Web UI. [#115245](https://github.com/ClickHouse/ClickHouse/pull/115245) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed silently incorrect results and undefined behavior when reading a malformed `AggregateFunction(uniq, ...)` state. The `skip_degree` field of the serialized state was used as a shift exponent without validation, so an out-of-range value produced an arbitrary cardinality instead of an error. Such states are now rejected with `INCORRECT_DATA`. [#115276](https://github.com/ClickHouse/ClickHouse/pull/115276) ([Groene AI](https://github.com/groeneai)). * Fixed a row policy bypass where the mergeTreeIndex table function exposed primary key and minmax index values of the rows hidden by a SELECT row policy on the source table; reading mergeTreeIndex for a table with a row policy is now denied. [#115304](https://github.com/ClickHouse/ClickHouse/pull/115304) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `CREATE QUOTA` and `ALTER QUOTA` silently discarding all but the last `FOR INTERVAL` block when the separating comma between intervals was omitted. Such a statement succeeded while installing none of the earlier limits. The intervals now accumulate again, as they did before 2020. Closes [#115282](https://github.com/ClickHouse/ClickHouse/issues/115282). [#115320](https://github.com/ClickHouse/ClickHouse/pull/115320) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results from `if`, `multiIf` and `CASE` when one branch is a `Date` or `Date32` and the result type is `DateTime` or `DateTime64`. With `compile_expressions = 1` (the default) the compiled expression copied the date's day count into the timestamp column without converting days to seconds, so `if(cond, toDate32('1900-01-01'), toDateTime64('1970-01-01', 3))` returned `1969-12-31 23:59:34.433`. Closes [#115272](https://github.com/ClickHouse/ClickHouse/issues/115272). [#115325](https://github.com/ClickHouse/ClickHouse/pull/115325) ([Groene AI](https://github.com/groeneai)). * Do not leave metadata referencing a dropped named collection. [#115326](https://github.com/ClickHouse/ClickHouse/pull/115326) ([Groene AI](https://github.com/groeneai)). * Fixed `timezoneOffset` (alias `timeZoneOffset`) returning an offset wrong by exactly one day for time zones whose UTC offset differs from their 1970-01-01 offset by a whole day, such as `Pacific/Kiritimati` and `Pacific/Apia` after they crossed the international date line, and for most zones before 1970. This also fixes `parseDateTime`, `parseDateTimeOrNull`, `parseDateTime64`, `parseDateTimeInJodaSyntax`, `EXTRACT(TIMEZONE_HOUR / TIMEZONE_MINUTE ...)`, `formatDateTime`'s `%z`, `toUTCTimestamp` and `fromUTCTimestamp`, which consume that offset. Closes [#115281](https://github.com/ClickHouse/ClickHouse/issues/115281). [#115332](https://github.com/ClickHouse/ClickHouse/pull/115332) ([Groene AI](https://github.com/groeneai)). * Fixed a bypass of settings constraints: `SET = DEFAULT` reset a setting without checking the constraints from the user's profile, so a `MIN`/`MAX`/`CONST` constraint could be escaped by resetting the setting instead of assigning to it, and a session could leave readonly mode with `SET readonly = DEFAULT`. [#115334](https://github.com/ClickHouse/ClickHouse/pull/115334) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Arrow Flight server: reject malformed Basic authentication credentials that omit the username/password separator. [#115347](https://github.com/ClickHouse/ClickHouse/pull/115347) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` (`Join is supported only for pipelines with one output port`) for a `full_sorting_merge` JOIN with `query_plan_join_shard_by_pk_ranges` enabled when one of the two sides is pruned to no parts at all - by the primary key, by column statistics, or because the table is empty. [#115352](https://github.com/ClickHouse/ClickHouse/pull/115352) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Disable the query condition cache for filters over _part_starting_offset. [#115358](https://github.com/ClickHouse/ClickHouse/pull/115358) ([Azat Khuzhin](https://github.com/azat)). * Fixed `randConstant` returning two different values in one query when its calls carry different aliases. `randConstant` is documented to hold a single value for the whole query, and `SELECT randConstant() = randConstant()` already returned `1`, but adding aliases as in `SELECT randConstant() AS a, randConstant() AS b` produced two unrelated values. [#115384](https://github.com/ClickHouse/ClickHouse/pull/115384) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `uniq`, `uniqExact`, `uniqHLL12` and `uniqTheta` returning a wrong result for an argument wrapped in an injective function that hides nullability, such as `uniqExact(tuple(x))` over a `Nullable` column. The `optimize_injective_functions_inside_uniq` optimization removed the wrapping function, after which NULL rows were skipped instead of counted. [#115466](https://github.com/ClickHouse/ClickHouse/pull/115466) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix `GroupingAggregatedTransform` producing duplicate chunks for the same bucket. [#115487](https://github.com/ClickHouse/ClickHouse/pull/115487) ([Nikita Taranov](https://github.com/nickitat)). * Fixed the error message of `geoToH3` naming the wrong argument position and the wrong type when a coordinate is not `Float64`. Under the default `geotoh3_argument_order = 'lat_lon'` the message pointed at the other argument and printed that argument's type. Closes [#101813](https://github.com/ClickHouse/ClickHouse/issues/101813). [#115503](https://github.com/ClickHouse/ClickHouse/pull/115503) ([Groene AI](https://github.com/groeneai)). * Reject a bare carriage return (`\r`), in addition to a line feed (`\n`), in HTTP header names and values that pass through the HTTP header filter. This closes an HTTP header injection (request/response splitting) gap for features that pass user-provided headers, such as the `url` table function and engine, HTTP dictionaries, and data lake catalogs. [#115510](https://github.com/ClickHouse/ClickHouse/pull/115510) ([Sergei Trifonov](https://github.com/serxa)). * Fixed caches configured by a maximum number of entries (`SLRU` policy) silently stopping to admit new entries once that many entries had been accessed at least twice. Affected caches include the Iceberg and Paimon metadata file caches, the Parquet metadata cache, the text index caches, the vector similarity index cache, the compiled expression cache and the ObjectStorageQueue file status cache. [#115517](https://github.com/ClickHouse/ClickHouse/pull/115517) ([Groene AI](https://github.com/groeneai)). * Fixed a hang of up to one hour in `BACKUP`/`RESTORE ... TO S3`, GCS disks and the `s3` table function when the named collection sets `http_client = 'gcp_oauth'` and the Google OAuth2 token endpoint accepts the connection but never answers. The bearer token request no longer inherits the caller's data transfer request timeout; its response wait is bounded at 10 seconds, matching the STS `AssumeRole` request cap. [#115531](https://github.com/ClickHouse/ClickHouse/pull/115531) ([Groene AI](https://github.com/groeneai)). * Fix `dictGet` and its variations in distributed queries: an unqualified dictionary name is now bound to the current database of the initiator, instead of being resolved against the current database of each shard, where it either was not found or resolved to a different dictionary of the same name. [#115541](https://github.com/ClickHouse/ClickHouse/pull/115541) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a race that could make a `STREAM BOUNDED` query return fewer rows than were committed before it started, usually none. [#115543](https://github.com/ClickHouse/ClickHouse/pull/115543) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed Keeper snapshots being written as empty files when `compress_snapshots_with_zstd_format` is disabled. The server logged `Created persistent snapshot` but the snapshot file was 0 bytes, and the node then refused to start with `CORRUPTED_DATA`. Closes [#115388](https://github.com/ClickHouse/ClickHouse/issues/115388). [#115553](https://github.com/ClickHouse/ClickHouse/pull/115553) ([Groene AI](https://github.com/groeneai)). * Fix `read_rows`, `read_bytes`, `written_rows` and `written_bytes` in `system.query_thread_log` reporting a thread lifetime running total instead of the values for the logged query. Rows of long-lived pooled threads, in particular the query's initiating `HTTPHandler` thread, were affected. [#115596](https://github.com/ClickHouse/ClickHouse/pull/115596) ([Groene AI](https://github.com/groeneai)). * Fixed an out-of-bounds read of the cached dictionary hashes when aggregating or joining on a `LowCardinality(String)` key whose dictionary grows while it is being aggregated, reachable from `TTL ... GROUP BY`. It could yield a wrong hash, and a reported heap-buffer-overflow in sanitizer builds. [#115604](https://github.com/ClickHouse/ClickHouse/pull/115604) ([Groene AI](https://github.com/groeneai)). * `CREATE TABLE` and `ALTER TABLE` now reject a lossy codec such as `SZ3` on a column used in the partition key. [#115626](https://github.com/ClickHouse/ClickHouse/pull/115626) ([Groene AI](https://github.com/groeneai)). * Fix invalid backslash escape characters in text, tokenbf_v1 and ngrambf_v1 indexes. [#115634](https://github.com/ClickHouse/ClickHouse/pull/115634) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed `removePartAndCoveredParts` when it is called for a part that is present in the active data part set. [#115640](https://github.com/ClickHouse/ClickHouse/pull/115640) ([Mikhail Artemenko](https://github.com/Michicosun)). * Allow KeeperMap readers to accept shared metadata when equivalent primary keys differ only by redundant outer parentheses. [#115642](https://github.com/ClickHouse/ClickHouse/pull/115642) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fixed a bug where restoring a database or table under a new name dropped the table aliases from the bodies of stored views, so reading a restored view failed with `UNKNOWN_IDENTIFIER`. Closes [#115479](https://github.com/ClickHouse/ClickHouse/issues/115479). [#115645](https://github.com/ClickHouse/ClickHouse/pull/115645) ([Groene AI](https://github.com/groeneai)). * Fixed `Code: 349. Cannot convert NULL value to non-Nullable type` when a `has` or `notHas` predicate carries a NULL array element and the primary key is a `String`, `Array(String)` or `Map(String, String)`, and fixed wrong results when that element is a `Dynamic` or `Variant` holding NULL. The NULL is now dropped from the set during key analysis, as it already was for other key types. [#115647](https://github.com/ClickHouse/ClickHouse/pull/115647) ([Groene AI](https://github.com/groeneai)). * Fixed a bug in the function `formatRowNoNewline` that could produce incorrect results or a logical error when a row is formatted to an empty result. [#115669](https://github.com/ClickHouse/ClickHouse/pull/115669) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject zero and negative `step` values in one-point Prometheus range queries. [#115679](https://github.com/ClickHouse/ClickHouse/pull/115679) ([Minh Vu](https://github.com/fallintoplace)). * Reject PromQL timestamps and durations that exceed the Int64 range. [#115682](https://github.com/ClickHouse/ClickHouse/pull/115682) ([Minh Vu](https://github.com/fallintoplace)). * Fix server crashes on deeply nested input in `readWKB`, `JSONMergePatch`, the address parser of the `url`/`remote`/`cluster`/`mysql`/`postgresql` table functions, the `CURSOR` clause parser and the CapnProto schema parser. Fix a crash in the parser of the `EXECUTE` statement when an argument is not a literal. [#115700](https://github.com/ClickHouse/ClickHouse/pull/115700) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Do not return uninitialized memory in the result of a binary string literal whose length is not a multiple of eight, and from the LZ4 decompressor when a compressed block has no body. [#115704](https://github.com/ClickHouse/ClickHouse/pull/115704) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Bound the size of the startup message of the PostgreSQL wire protocol, which is read before authentication. [#115708](https://github.com/ClickHouse/ClickHouse/pull/115708) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `MULTIPLE_EXPRESSIONS_FOR_ALIAS` error in queries where a column alias coincides with the alias of a table expression in the `FROM` section (an analyzer regression in version 25.2). [#115714](https://github.com/ClickHouse/ClickHouse/pull/115714) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `clickhouse-local` terminating the whole process when a connection handler of a listener started with `SYSTEM START LISTEN` threw, for example because a client sent a malformed packet or reset a keep-alive socket. Such a process now logs the error and keeps serving, like `clickhouse-server` does. [#115715](https://github.com/ClickHouse/ClickHouse/pull/115715) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR: No available columns` when a query reads no columns from a table expression that has no columns at all, for example `SELECT count() FROM loop(db, tbl)` where `tbl` is an `Alias` table whose target was dropped, or a parameterized view. Such a query now fails with `UNSUPPORTED_METHOD` instead of aborting the server in debug and sanitizer builds. [#115716](https://github.com/ClickHouse/ClickHouse/pull/115716) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` "Reading from materialized CTE 'X' before its materialization completed - DelayedPortsProcessor gate is missing in the query plan" when a `MATERIALIZED` CTE is read from inside an `IN` subquery over a `Distributed` table. Closes [#113184](https://github.com/ClickHouse/ClickHouse/issues/113184). [#115740](https://github.com/ClickHouse/ClickHouse/pull/115740) ([Groene AI](https://github.com/groeneai)). * Reject invalid names of projections and indexes. [#115824](https://github.com/ClickHouse/ClickHouse/pull/115824) ([Alexander Tokmakov](https://github.com/tavplubix)). * Reject PromQL vector matching modifiers when one operand is a scalar. [#115828](https://github.com/ClickHouse/ClickHouse/pull/115828) ([Minh Vu](https://github.com/fallintoplace)). * Fixed `cast_keep_nullable` not preserving nullability when the target type is `LowCardinality`. Casting a NULL-capable value to `LowCardinality(T)` threw `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN` instead of producing `LowCardinality(Nullable(T))`. Closes [#103485](https://github.com/ClickHouse/ClickHouse/issues/103485). [#115851](https://github.com/ClickHouse/ClickHouse/pull/115851) ([Groene AI](https://github.com/groeneai)). * Add Objects' paths to deduplication hashing, preventing incorrect deduplication. [#115866](https://github.com/ClickHouse/ClickHouse/pull/115866) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix `TYPE_MISMATCH` ("CAST AS Array can only be performed between same-dimensional array types") for a query with a constant of type `Array(Map(...))`, such as `has([map('k', 'v')], m)`, executed over a `Distributed` table or with parallel replicas. [#115883](https://github.com/ClickHouse/ClickHouse/pull/115883) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Unknown identifier` raised during query plan optimization under `make_distributed_plan = 1` when a window query's sort column passes through an `Expression` step that does not mention it. Closes [#115737](https://github.com/ClickHouse/ClickHouse/issues/115737). [#115890](https://github.com/ClickHouse/ClickHouse/pull/115890) ([Groene AI](https://github.com/groeneai)). * Fixed `h3kRing`, `h3HexRing`, `h3Line` and `h3ToChildren` ignoring `max_execution_time` and `KILL QUERY` while expanding a block of rows. [#115893](https://github.com/ClickHouse/ClickHouse/pull/115893) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading through a `Merge` table that matches an `Alias` table whose target is a parameterized view no longer raises `LOGICAL_ERROR: Table has no columns.` (which aborts the server in debug and sanitizer builds). Such a table now reports `STORAGE_REQUIRES_PARAMETER`, the same error the parameterized view itself reports. Any other matched table that cannot supply columns now reports `UNSUPPORTED_METHOD` instead of an internal error. [#115896](https://github.com/ClickHouse/ClickHouse/pull/115896) ([Groene AI](https://github.com/groeneai)). * Fixed the on-the-fly preview of a pending mutation whose expression references `_sample_factor`. With `apply_mutations_on_fly = 1` and a `SAMPLE` clause, the expression was evaluated with the reading query's sample factor instead of the value the background materialization persists, so previewed rows disagreed with the eventual materialized result, and a pending `ALTER TABLE t DELETE WHERE _sample_factor > 1` could make a `SAMPLE` read return no rows at all. The query's own `_sample_factor` column is unchanged. [#115906](https://github.com/ClickHouse/ClickHouse/pull/115906) ([Groene AI](https://github.com/groeneai)). * Fixed a case where `ATTACH TABLE` carrying an `ENGINE = URL(...)` definition did not check the `TABLE ENGINE` grant of the engine the URL scheme dispatches to. A user holding only `TABLE ENGINE ON URL` could attach and then read a `File`, `S3`, `AzureBlobStorage` or `HDFS` backed table, which `CREATE TABLE` correctly rejects. [#115914](https://github.com/ClickHouse/ClickHouse/pull/115914) ([Groene AI](https://github.com/groeneai)). * Do not let an unparsable KeeperMap metadata node stop the server from starting. [#115941](https://github.com/ClickHouse/ClickHouse/pull/115941) ([Groene AI](https://github.com/groeneai)). * Fix a crash of the `showCertificate` function on a server that obtains its TLS certificate with ACME (Let's Encrypt). The function now reports the certificate that is actually served, also after a certificate reload. [#115976](https://github.com/ClickHouse/ClickHouse/pull/115976) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a server abort in debug and sanitizer builds when inserting into a materialized view whose `TO` target is another materialized view. Such an insert already failed on released versions, with an uninformative `Code: 1001 STD_EXCEPTION`; it now reports `NOT_IMPLEMENTED` naming the offending view. [#115985](https://github.com/ClickHouse/ClickHouse/pull/115985) ([Groene AI](https://github.com/groeneai)). * Statistics files with format version `V3` are now read instead of rejected with `ILLEGAL_STATISTICS`. Such files were produced only by builds of `master` from a narrow window, but parts containing them were completely unreadable, and on a readonly disk `ALTER TABLE ... MATERIALIZE STATISTICS` cannot regenerate them. [#116030](https://github.com/ClickHouse/ClickHouse/pull/116030) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `LOGICAL_ERROR: Index with name auto_minmax_index_ already exists` when a single `ALTER TABLE` statement combined `RENAME COLUMN` with another command, such as `MODIFY SETTING`, on a table with implicit minmax indices (`add_minmax_index_for_numeric_columns` and friends). Renaming a plain column also no longer leaves a following `DROP COLUMN` failing with `UNKNOWN_IDENTIFIER`, nor a following `ADD COLUMN` without its implicit index. [#116063](https://github.com/ClickHouse/ClickHouse/pull/116063) ([Groene AI](https://github.com/groeneai)). * Fix stateless workers crash with object storage. [#116067](https://github.com/ClickHouse/ClickHouse/pull/116067) ([Konstantin Vedernikov](https://github.com/scanhex12)). #### Build/Testing/Packaging Improvement * Provide signed artifacts for macOS. [#115169](https://github.com/ClickHouse/ClickHouse/pull/115169) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix building from source with clang 23. [#112288](https://github.com/ClickHouse/ClickHouse/pull/112288) ([Azat Khuzhin](https://github.com/azat)). * Reduced the dependencies and the code size of the SQL parser, which can now be built on its own — see `utils/wasm-parser` for a standalone WebAssembly build of it. The tables of SQL keywords and access types are now built at compile time instead of on every server start. [#112067](https://github.com/ClickHouse/ClickHouse/pull/112067) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Building from source now fails during CMake generation instead of corrupting `StorageSystemLicenses.generated.cpp` when license generation fails. [#111222](https://github.com/ClickHouse/ClickHouse/pull/111222) ([Mikhail Artemenko](https://github.com/Michicosun)). * Updated the bundled Apache Thrift library to version 0.24.0. [#111278](https://github.com/ClickHouse/ClickHouse/pull/111278) ([Groene AI](https://github.com/groeneai)). * Add `libucontext` for musl builds, enabling the `boost::context` `ucontext` backend used by sanitizer builds. [#111437](https://github.com/ClickHouse/ClickHouse/pull/111437) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix source-level portability issues when building with musl. [#111631](https://github.com/ClickHouse/ClickHouse/pull/111631) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Editing `Core/ProtocolDefines.h` no longer triggers a near-full rebuild when building from source: the include was removed from `BlockInfo.h` and added only in the files that use its constants. [#111650](https://github.com/ClickHouse/ClickHouse/pull/111650) ([Alexander Gololobov](https://github.com/davenger)). * Fix a startup abort in macOS debug builds by disabling memory-tracker debug checks for allocations made by system libraries. [#112080](https://github.com/ClickHouse/ClickHouse/pull/112080) ([Arsen Muk](https://github.com/arsenmuk)). * Updated the bundled `libarchive` to version 3.8.9. [#112434](https://github.com/ClickHouse/ClickHouse/pull/112434) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Updated `StringZilla` to version 5.0.5. [#112319](https://github.com/ClickHouse/ClickHouse/pull/112319) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update `jwt-cpp` to v0.7.2. [#112595](https://github.com/ClickHouse/ClickHouse/pull/112595) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update the bundled SQLite amalgamation from 3.41.2 to 3.53.4. [#112599](https://github.com/ClickHouse/ClickHouse/pull/112599) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Updated `aws-c-http` to version 0.11.0. [#112413](https://github.com/ClickHouse/ClickHouse/pull/112413) ([Govind R Nair](https://github.com/Revertionist)). * Update `libcotp` to v4.2.1. [#112596](https://github.com/ClickHouse/ClickHouse/pull/112596) ([Konstantin Bogdanov](https://github.com/thevar1able)). * OpenSSL can now be built from its portable C sources on architectures that have no hand-written assembly in the build description, instead of failing at the final link with undefined symbols. The new `OPENSSL_NO_ASM` CMake option selects the same path explicitly. [#112863](https://github.com/ClickHouse/ClickHouse/pull/112863) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added an `ENABLE_ORC` CMake option (on by default), so a build can have Parquet without ORC and without the protobuf and `protoc` that ORC brings with it. [#112864](https://github.com/ClickHouse/ClickHouse/pull/112864) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `Common/Exception.cpp` no longer requires ClickHouse's patched libc++ to compile, so a build linking a different C++ standard library gets exceptions without a throw-site stack trace rather than not compiling. [#112865](https://github.com/ClickHouse/ClickHouse/pull/112865) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Stack unwinding on aarch64 musl builds now continues through signal frames and blocked syscalls instead of stopping at the libc frame, improving stack traces from the fatal error handler and the sampling query profiler. [#112592](https://github.com/ClickHouse/ClickHouse/pull/112592) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update the vendored `libexpat` to 2.8.2. [#112593](https://github.com/ClickHouse/ClickHouse/pull/112593) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update `simdjson` to v4.6.5. [#112594](https://github.com/ClickHouse/ClickHouse/pull/112594) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update Apache Arrow to 25.0.0 and Apache ORC to 2.3.1. [#112604](https://github.com/ClickHouse/ClickHouse/pull/112604) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Coverage builds (`WITH_COVERAGE`) now link the profile runtime built in-tree from the vendored LLVM sources instead of the archive bundled with the host clang toolchain. [#112607](https://github.com/ClickHouse/ClickHouse/pull/112607) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update Protocol Buffers to v35.1 and gRPC to v1.83.0. [#112614](https://github.com/ClickHouse/ClickHouse/pull/112614) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Added an experimental WebAssembly (`wasm64`, via Emscripten) target: `emcmake cmake` now configures the tree and the lower layers build. Also fixed `-DENABLE_LIBFIU=OFF`, which did not build on any platform. [#112911](https://github.com/ClickHouse/ClickHouse/pull/112911) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed every `-Wunique-object-duplication` site: header-defined mutable singletons that would get one copy per shared object instead of one per process. Mostly by moving the definition into a `.cpp`. [#112995](https://github.com/ClickHouse/ClickHouse/pull/112995) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Installing a `tgz` package over a dangling symlink now preserves the symlink chain and applies the packaged target's mode and ownership instead of a `umask`-derived mode. [#112766](https://github.com/ClickHouse/ClickHouse/pull/112766) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In `aarch64` musl builds, `memmove` now uses the ARM optimized-routines assembly implementation (shared with `memcpy`) instead of the generic C word-at-a-time loop. [#113502](https://github.com/ClickHouse/ClickHouse/pull/113502) ([Konstantin Bogdanov](https://github.com/thevar1able)). * In musl builds, `getaddrinfo` with `AI_ADDRCONFIG` now follows glibc semantics, so `localhost` no longer resolves to an unusable `::1` first in IPv4-only environments, and `sysconf` returns glibc's buffer-size hints for `getpwuid_r`/`getgrgid_r` instead of -1, unbreaking libhdfs3. [#113485](https://github.com/ClickHouse/ClickHouse/pull/113485) ([Konstantin Bogdanov](https://github.com/thevar1able)). * In musl builds, `sched_getcpu` is now an rseq read (~1ns) instead of a syscall, matching glibc and removing the system-time overhead of jemalloc's `percpu_arena` on allocation-heavy workloads; musl also registers rseq eagerly and exports the glibc `__rseq_*` ABI. [#113501](https://github.com/ClickHouse/ClickHouse/pull/113501) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix local (non-CI) builds compiling all vendored Rust code (the `PCO` codec, Delta Lake reads via `delta-kernel-rs`, `prqlc`, `chdig`, `wasmtime`, the client's fuzzy history search via `skim`) with `opt-level = 0`. The `SANITIZE` CMake variable was declared with `option`, so its default value was the BOOL `OFF`, which failed the `STREQUAL ""` "no sanitizer" checks, and every build configured without an explicit `-DSANITIZE=...` took the sanitizer workaround path that disables Rust optimization. [#114083](https://github.com/ClickHouse/ClickHouse/pull/114083) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Push docker image layers zstd-compressed. [#114342](https://github.com/ClickHouse/ClickHouse/pull/114342) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Build Debian packages for the release builds only. [#114739](https://github.com/ClickHouse/ClickHouse/pull/114739) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed building `unit_tests_dbms` in configurations without the full library set (`ENABLE_LIBRARIES=0` or `ENABLE_ROCKSDB=0`). [#115655](https://github.com/ClickHouse/ClickHouse/pull/115655) ([Alexey Milovidov](https://github.com/alexey-milovidov)). ### ClickHouse release 26.7, 2026-07-22. [Presentation](https://presentations.clickhouse.com/2026-release-26.7/), [Video](https://www.youtube.com/watch?v=mKBNLaFOVDA) #### Backward Incompatible Change * Mark rotated non-replicated `MergeTree` system log tables without TTL as `table_readonly` to avoid unnecessary background operations. The `table_readonly` `MergeTree` setting now also suppresses all background work on a plain `MergeTree` table — regular, TTL (`DELETE`/`MOVE`/recompression) and recompression merges, background mutations, and background part moves — and rejects the mutating partition commands (`ATTACH`/`MOVE`/`DROP`/`DROP DETACHED`/`FETCH`/`REPLACE PARTITION` and `MOVE PARTITION ... TO TABLE` into the table), in addition to the inserts, mutations, and `OPTIMIZE` it already rejected. As a result, a `table_readonly` table with a TTL no longer reclaims its expired data while the setting is enabled. **Downgrade warning:** the rotated system log tables (for example `system.text_log_0`) now store `table_readonly = true` in their metadata. The `table_readonly` setting exists since 26.3, so servers older than 26.3 fail to start after a downgrade with `UNKNOWN_SETTING` (`Unknown setting 'table_readonly': for storage MergeTree: Cannot attach table`). Before downgrading to a version older than 26.3, drop the rotated `system.*_log_N` tables (it is safe to drop them), or run `ALTER TABLE system.
_log_N RESET SETTING table_readonly` on each of them. [#95079](https://github.com/ClickHouse/ClickHouse/pull/95079) ([Mathuranath Metivier](https://github.com/matt-metivier)). * S3 access originating from user SQL no longer resolves the server's own cloud credentials (environment, IMDS/IRSA, instance profile, AWS config files, `role_arn`-based STS, GCP OAuth metadata) by default; such a request must use explicit credentials or `NOSIGN`. Named collections now default `use_environment_credentials` to `0`. The previous behaviour can be restored per request with `use_environment_credentials = 1` (or globally via the `` config) together with the new setting `s3_allow_server_credentials_in_user_queries` (disabled by default). Explicitly supplied and operator-provisioned config credentials are unaffected. On server startup or `RESTORE`, a persistent `S3`/`S3Queue` table, dynamic S3 disk, or `DataLakeCatalog` whose definition resolves such credentials is loaded anonymously (inaccessible until re-credentialed) instead of aborting startup, controlled by the new server setting `s3_load_table_anonymously_if_credentials_restricted` (enabled by default). [#106855](https://github.com/ClickHouse/ClickHouse/pull/106855) ([Raúl Marín](https://github.com/Algunenano)). * The `**/` glob (any number of directories) now also matches the same directory. Previously, `**/` in glob patterns was not handled as a special case, so `data/**/file.txt` would not match `data/file.txt` (zero directory levels). [#97676](https://github.com/ClickHouse/ClickHouse/pull/97676) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject the `zip`/`zipx` backup archive format for backups stored on object storage — direct `S3(...)`/`AzureBlobStorage(...)` destinations and `Disk(...)` destinations backed by S3 or Azure — for both `BACKUP` and `RESTORE`. Zip requires seeking to read its central directory, which is very slow over object storage. Use a tar-based format such as `tar.gz` instead. [#101770](https://github.com/ClickHouse/ClickHouse/pull/101770) ([Yash ](https://github.com/Onyx2406)). * Extended the supported range of `DateTime64` from `[1900-01-01, 2299-12-31]` to `[0000-01-01, 9999-12-31]`. Values outside the former range are now computed correctly (via cctz) instead of being clamped to the boundary. With precision 8 or 9 the range remains narrower because the ticks are stored as `Int64` (with nanosecond precision the maximum is still `2262-04-11`). Backward compatibility: if you used out-of-range date-times before, but relied on the very specific saturation rules inside the old range, keep in mind that now the results are changing to be more correct. [#107907](https://github.com/ClickHouse/ClickHouse/pull/107907) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `AggregatingMergeTree` now rejects, at table creation time, schemas where a column is neither part of the sorting key nor an aggregate-state measure (`AggregateFunction`/`SimpleAggregateFunction`). Such columns are silently collapsed to an arbitrary value during background merges, producing wrong results for queries that `GROUP BY` or filter on them. Set `allow_dimensions_outside_sorting_key = 1` to restore the previous behavior. Closes [#751](https://github.com/ClickHouse/ClickHouse/issues/751). [#108087](https://github.com/ClickHouse/ClickHouse/pull/108087) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the config-based workload scheduling configuration (the `resources` and `workload_classifiers` server config sections). Use `CREATE RESOURCE` and `CREATE WORKLOAD` queries instead. The obsolete config sections are ignored with a warning. [#108286](https://github.com/ClickHouse/ClickHouse/pull/108286) ([Sergei Trifonov](https://github.com/serxa)). * ClickHouse now always uses the unified insert deduplication hash for both synchronous and asynchronous inserts; the legacy per-insert deduplication behaviours are removed. The server setting `insert_deduplication_version` is kept as a migration guard: the server refuses to start if it is set to a legacy value (`old_separate_hashes` or `compatible_double_hashes`). To upgrade from a version that used a legacy value, first run a release that supports `compatible_double_hashes` (which writes both the legacy and unified hashes). For replicated tables run it for at least `replicated_deduplication_window_seconds` (one hour by default; the default windows retain the unified hashes of all inserts for that window, which is considered enough for an insert retry loop). For non-replicated tables with `non_replicated_deduplication_window` > 0 that window is count-based rather than time-based, so run `compatible_double_hashes` for at least that many inserts. Then remove the setting (or set it to `new_unified_hash`) before upgrading to this version. [#108361](https://github.com/ClickHouse/ClickHouse/pull/108361) ([Sema Checherinda](https://github.com/CheSema)). If you never set `insert_deduplication_version`, you can ignore this item. * Removed the long-deprecated functions `snowflakeToDateTime`, `snowflakeToDateTime64`, `dateTimeToSnowflake` and `dateTime64ToSnowflake`. They were deprecated back in v24.6 in favor of `snowflakeIDToDateTime`, `snowflakeIDToDateTime64`, `dateTimeToSnowflakeID` and `dateTime64ToSnowflakeID`, which should be used instead. The setting `allow_deprecated_snowflake_conversion_functions` (which used to re-enable them) is now obsolete and has no effect. [#108711](https://github.com/ClickHouse/ClickHouse/pull/108711) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The setting `use_legacy_to_time` is now `0` by default, so `toTime` converts values into the `Time` data type instead of converting a date with time to a fixed date. The legacy behavior is still available via the `toTimeWithFixedDate` function or by setting `use_legacy_to_time = 1`. [#108729](https://github.com/ClickHouse/ClickHouse/pull/108729) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Naive Bayes models (used by `naiveBayesClassifier`) are now configured as a dictionary with the `NAIVE_BAYES` layout, built at load time from a table of pre-aggregated per-class n-gram counts, instead of the previous server-side configuration (an XML config file referencing serialized `.bin` model files), which is no longer supported — existing models must be recreated as dictionaries. Three new functions are added alongside `naiveBayesClassifier`: `naiveBayesClassifierWithProb` returns the predicted class together with its probability, `naiveBayesClassifierWithAllProbs` returns every class with its probability, and `naiveBayesNgrams` tokenizes text into n-grams the same way the dictionary does, for building the training data from raw labelled text. Additionally, several performance optimizations have been applied: on a 9.7 MiB code-point trigram language model, it uses ~49× less memory (from 1.84 GiB to 38 MiB), loads ~11× faster, and classifies ~11× faster. [#108773](https://github.com/ClickHouse/ClickHouse/pull/108773) ([Nihal Z. Miaji](https://github.com/nihalzp)). * The `hasColumnInTable` function no longer accepts the optional `hostname`, `username`, and `password` arguments for checking a column on an arbitrary remote server; only the `hasColumnInTable(database, table, column)` form remains. The removed remote mode was a security concern: it let any user trigger outbound connections to arbitrary hosts and leaked credentials into query logs, and it was not gated by any privilege. [#110881](https://github.com/ClickHouse/ClickHouse/pull/110881) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### New Feature * Added `EXPLAIN ANALYZE` for examination of query performance: the query is executed, and the actual execution metrics are rendered in the familiar query plan format. [#106586](https://github.com/ClickHouse/ClickHouse/pull/106586) ([Kirill Kopnev](https://github.com/Fgrtue)). [#110668](https://github.com/ClickHouse/ClickHouse/pull/110668) ([Kirill Kopnev](https://github.com/Fgrtue)). * Added support for `WHERE` clauses in projection definitions. Projections with `WHERE` only materialize rows matching the predicate, and the optimizer can use them (cost-based) when the query's `WHERE` implies the projection's `WHERE`. [#102347](https://github.com/ClickHouse/ClickHouse/pull/102347) ([S Bala Vignesh](https://github.com/SBALAVIGNESH123)). * Added the `skip_unavailable_shards_mode` setting (also available as a `Distributed` engine setting) to control which exceptions from a remote shard are silently ignored when `skip_unavailable_shards` is enabled. [#79091](https://github.com/ClickHouse/ClickHouse/pull/79091) ([cjw](https://github.com/byte-sourcerer)). * Add `groupFormat` aggregate function that formats rows in each group using a specified output format and returns the result as a string. [#93201](https://github.com/ClickHouse/ClickHouse/pull/93201) ([Yang Hu](https://github.com/wandersofb)). * Added `AWS_MSK_IAM` as a supported value for `kafka_sasl_mechanism`, enabling ClickHouse to authenticate with Amazon MSK using IAM roles without managing SASL/SCRAM credentials. [#96100](https://github.com/ClickHouse/ClickHouse/pull/96100) ([kalavt](https://github.com/kalavt)). * Add aggregate function combinator `-Tuple`, which applies the underlying aggregate function to each element of a `Tuple` column independently and returns a `Tuple` of the results, preserving element names: `sumTuple(t)` for `t = (a, b)` returns `(sum(a), sum(b))`. Aggregate functions with several arguments take one tuple per argument, paired by position: `corrTuple((a1, a2), (b1, b2))` returns `(corr(a1, b1), corr(a2, b2))`. Unlike `-ForEach` over arrays, elements may have different types, and per-element result types and names are preserved. [#98190](https://github.com/ClickHouse/ClickHouse/pull/98190) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Text indexes now support a `postprocessor` argument — an arbitrary expression that transforms each token after tokenization (for example, `lower`). [#98939](https://github.com/ClickHouse/ClickHouse/pull/98939) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#108606](https://github.com/ClickHouse/ClickHouse/pull/108606) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Added the `system.stemmers` system table, which lists all languages that can be specified for the `stem` function. [#100611](https://github.com/ClickHouse/ClickHouse/pull/100611) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Add support for `WITH TIES` for negative `LIMIT`. [#100930](https://github.com/ClickHouse/ClickHouse/pull/100930) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Added support for the standard SQL `AT TIME ZONE` and `AT LOCAL` postfix operators as syntactic sugar for `toTimeZone`. The expression `expr AT TIME ZONE zone` is now equivalent to `toTimeZone(expr, zone)`, and `expr AT LOCAL` is equivalent to `toTimeZone(expr, timeZone())`. [#106092](https://github.com/ClickHouse/ClickHouse/pull/106092) ([lizepeng](https://github.com/lvzhipin03)). * The `URL` table engine and `url` table function now dispatch to the appropriate backend based on the URL scheme: `file://` is served by the `File` engine, `s3://`/`gs://`/`gcs://`/`oss://` by `S3`, `az://`/`azure://`/`abfss://`/`abfs://` by `AzureBlobStorage`, `hdfs://` by `HDFS`, and `http(s)://` by the `URL` engine as before. The `url_base` setting is applied before scheme dispatch. Only the S3 schemes resolved by the default `url_scheme_mappers` are dispatched; other S3-compatible vendor schemes (`cos`, `obs`, ...) are not, and require using the `s3` engine/function directly. [#106093](https://github.com/ClickHouse/ClickHouse/pull/106093) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `Remote` and `RemoteSecure` table engines, the persistent counterparts of the `remote` and `remoteSecure` table functions. `CREATE TABLE ... ENGINE = Remote('addresses', db, table, ...)` now works in addition to `CREATE TABLE ... AS remote(...)`. [#106189](https://github.com/ClickHouse/ClickHouse/pull/106189) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added engine-agnostic `SYSTEM STOP`, `SYSTEM START`, `SYSTEM PAUSE`, `SYSTEM CANCEL`, and `SYSTEM REFRESH` commands, and their `... ALL BACKGROUND` server-wide forms, to control the background activity of `Kafka`, `RabbitMQ`, `NATS`, `S3Queue`/`AzureQueue` tables and refreshable materialized views through one unified interface. For refreshable materialized views they alias the existing `SYSTEM ... VIEW` commands. As part of supporting these controls on NATS, JetStream tables now acknowledge messages only after a successful insert (at-least-once, previously messages were auto-acknowledged on delivery and could be lost if the insert failed). A new `nats_wait_for_flush_interval` setting (default `false`, preserving the previous low-latency behaviour) optionally keeps a consumption cycle open for the whole flush interval instead of flushing as soon as the queue drains. A new `nats_commit_on_select` setting makes a direct `SELECT` on a JetStream table consume (acknowledge) the messages it reads. [#107476](https://github.com/ClickHouse/ClickHouse/pull/107476) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)). * Added support for the `mysql`, `postgresql`, and `sqlite` table functions and table engines to accept a user's query (instead of a table name) and pass it to the external database as is, written either as a subquery `(SELECT ...)` or as `query('SELECT ...')`. The structure of the resulting table is inferred from the query result, and such a table is read-only. Closes [#46758](https://github.com/ClickHouse/ClickHouse/issues/46758). Based on the initial implementation by [Denis Vidiaev](https://github.com/the-devid) ([#79652](https://github.com/ClickHouse/ClickHouse/pull/79652)). [#107740](https://github.com/ClickHouse/ClickHouse/pull/107740) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added tabs to the ClickHouse Web UI (Play) for working with multiple queries at once. Tabs, their titles, parameters, active state, and small result snapshots are persisted across reloads (large and image results are not restored), integrate with browser history and the URL, and can be switched with the mouse wheel. [#107826](https://github.com/ClickHouse/ClickHouse/pull/107826) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Introduce the `QueryRunner` table engine. Records inserted into a `QueryRunner` table represent queries that the engine executes. The engine can be used for asynchronous query execution, batch execution of generated queries, directing queries to remote clusters, benchmarks, fuzzing, and testing with shadow traffic. [#107888](https://github.com/ClickHouse/ClickHouse/pull/107888) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add the `GeoJSON` output format for writing [GeoJSON](https://geojson.org/) `FeatureCollection` documents, producing one feature per row. The single geo-typed column (`Point`, `LineString`, `MultiLineString`, `Polygon`, `MultiPolygon`, `Ring`, or `Geometry`) becomes each feature's `geometry`; a column named `id` becomes the feature `id`; and any remaining columns become the feature `properties`, where a lone object-typed column named `properties` (a `JSON`, `Map`, or named `Tuple`) is written directly as the `properties` object. A `Ring` is written as a single-ring `Polygon`. The new `format_geojson_validate_geometry` setting (enabled by default) controls whether geometries that are not valid GeoJSON shapes — such as a `Polygon` with fewer than four points or an unclosed `Polygon` ring — are rejected when reading and writing. Additionally, the `GeoJSON` input format now infers the `id` column as `Nullable(String)` instead of `String`, so a feature with an absent or explicit `"id": null` member is read as `NULL` and kept distinct from an empty string `""`. Based on the initial implementation by [Mark Needham](https://github.com/mneedham) ([#98124](https://github.com/ClickHouse/ClickHouse/pull/98124)). [#108065](https://github.com/ClickHouse/ClickHouse/pull/108065) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Added the `dotProductTransposed` function (alias `scalarProductTransposed`) that computes the approximate inner product between a `QBit` column and a reference vector, complementing the existing `L2DistanceTransposed` and `cosineDistanceTransposed` functions. [#108100](https://github.com/ClickHouse/ClickHouse/pull/108100) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Added quantized transposed distance functions `cosineDistanceTransposedQuantized`, `L2DistanceTransposedQuantized` and `dotProductTransposedQuantized` that operate on a `QBit(Int8)` of `quantizeBFloat16ToInt8` Lloyd-Max codes, dequantizing the stored codes on the fly. A floating-point reference vector is the full-precision query, compared at `Float32` precision (a `Float64` query is narrowed); an `Array(Int8)` reference is itself dequantized, for a symmetric quantized-vs-quantized distance. [#109405](https://github.com/ClickHouse/ClickHouse/pull/109405) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added an optional stride parameter to the `QBit` data type (`QBit(T, dimension, stride)`) that stores groups of dimensions in separate streams, so a vector search can read only the first dimensions efficiently (e.g. for Matryoshka embeddings). The transposed distance functions accept an optional fourth `used_dims` argument to read a reduced number of dimensions. [#108103](https://github.com/ClickHouse/ClickHouse/pull/108103) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added support for the `Int8` element type in the `QBit` data type, enabling storage and transposed-distance vector search (`L2DistanceTransposed`, `cosineDistanceTransposed`) over quantized 8-bit integer vectors. [#108105](https://github.com/ClickHouse/ClickHouse/pull/108105) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * New function `randomHadamardTransform(vector[, seed[, output_dims]])`: a deterministic randomized Hadamard transform of a float vector — an orthogonal, norm-preserving rotation useful for preprocessing embeddings before quantization, and (when truncated) as a Johnson–Lindenstrauss / subsampled-randomized-Hadamard random projection. [#108227](https://github.com/ClickHouse/ClickHouse/pull/108227) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support configuring lazy loading of dictionaries on a per-dictionary basis with the `dictionary_lazy_load` setting in the dictionary definition, overriding the global `dictionaries_lazy_load` server setting, so that some dictionaries can be loaded lazily while others are loaded eagerly. [#108314](https://github.com/ClickHouse/ClickHouse/pull/108314) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Added a built-in documentation search page, available at the `/docs` path of the HTTP interface, that provides instant search over the `system.documentation` table and renders the reference documentation (with syntax highlighting, math, and cross-links). [#108345](https://github.com/ClickHouse/ClickHouse/pull/108345) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added function `xxHash64Spark`, which computes Spark-compatible `xxHash64` values for `String` and `NULL` inputs using seed `42` and returns `Int64`. [#108436](https://github.com/ClickHouse/ClickHouse/pull/108436) ([Lalit Yadav](https://github.com/lalitium)). * `ALTER USER`, `ALTER ROLE` and `ALTER SETTINGS PROFILE` now accept `SET name = value` as an alias for `MODIFY SETTING name = value`. It changes individual settings in place while keeping the rest, unlike the bare `SETTINGS` clause which replaces the whole settings list. This makes it less likely to accidentally wipe a user's other settings. [#108722](https://github.com/ClickHouse/ClickHouse/pull/108722) ([Groene AI](https://github.com/groeneai)). * Support `ALTER TABLE ... MODIFY CONSTRAINT [IF EXISTS] name CHECK expr` to change the expression of an existing constraint in place. [#108768](https://github.com/ClickHouse/ClickHouse/pull/108768) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: added a per-column color coding switch (the 🌈 icon in a column header) to toggle between bar, heatmap, categorical, and no visualization. The selected modes are remembered in the URL and browser history. [#108873](https://github.com/ClickHouse/ClickHouse/pull/108873) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `Keeper` now correctly handles `ZooKeeper`'s `CreateContainer` opcode (19), gated behind a new opt-in `create_container` feature flag (disabled by default, like `create_ttl`). Once enabled after a full ensemble upgrade, external `ZooKeeper` clients (e.g. Java `ZooKeeper` 3.9+) can create container nodes against `Keeper` instead of getting `ConnectionLoss`; container nodes report the expected `ephemeralOwner` sentinel value, and a background GC thread on the leader auto-deletes childless containers. A plain `Create` request carrying the `CONTAINER` create-mode flag without opcode 19 continues to be rejected (`ZBADARGUMENTS`): no known `ZooKeeper` client emits that combination. This is server-side protocol compatibility only: ClickHouse's own `ZooKeeper` client (`zkutil`) is unchanged and has no API to create container nodes itself. [#108908](https://github.com/ClickHouse/ClickHouse/pull/108908) ([unintended](https://github.com/unintended)). * Added functions `geoToUTM`, `UTMToGeo`, `geoToMGRS` and `MGRSToGeo` for converting between WGS84 geographic coordinates and the UTM and MGRS coordinate systems. [#108939](https://github.com/ClickHouse/ClickHouse/pull/108939) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `digits(n, offset[, length])` function, which returns the digits of `n` starting at the 1-based `offset` and spanning `length` digits, or to the end of the number if `length` is omitted. [#109012](https://github.com/ClickHouse/ClickHouse/pull/109012) ([Umang Agrawal](https://github.com/1000ms)). * Add the `sqr` arithmetic function for calculating the square of a number. [#109061](https://github.com/ClickHouse/ClickHouse/pull/109061) ([Lalit Yadav](https://github.com/lalitium)). * Added a new function `dictGetRoot` which returns the topmost ancestor (the root) of a key in a hierarchical dictionary. It is a convenient equivalent of `dictGetHierarchy(dict_name, key)[-1]`. [#109459](https://github.com/ClickHouse/ClickHouse/pull/109459) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `SYSTEM UNLOAD DICTIONARY` and `SYSTEM UNLOAD DICTIONARIES` commands to release dictionary memory without dropping the dictionary definition. Dictionaries will be reloaded lazily on the next access. [#109639](https://github.com/ClickHouse/ClickHouse/pull/109639) ([Matheus Nerone](https://github.com/Manerone)). * Added functions `geometryIntersectCartesian` and `geometryIntersectSpherical` that return whether two geometries intersect. Unlike `polygonsIntersectCartesian`/`polygonsIntersectSpherical`, they accept any geometry data type (`Point`, `LineString`, `MultiLineString`, `Ring`, `Polygon`, `MultiPolygon`), including the common `Geometry` type, and the two arguments may be of different types. [#110062](https://github.com/ClickHouse/ClickHouse/pull/110062) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Query parameters (`{name:Type}` substitutions) can now be used as setting values, both in the `SETTINGS` clause of a query (such as `SELECT` or `INSERT`) and in standalone `SET` queries, e.g., `SELECT ... SETTINGS max_threads = {threads:UInt64}` and `SET max_threads = {threads:UInt64}`. [#108760](https://github.com/ClickHouse/ClickHouse/pull/108760) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Experimental Feature * Wildcard expansion for the `url` table function and the `URL` table engine: wildcards in the URL path are expanded by listing HTTP index pages (HTML or plaintext directory listings) and extracting matching URLs, with limits on the index page size and on the number of directories read. Enabled by the `allow_experimental_url_wildcard_from_index_pages` setting. [#95181](https://github.com/ClickHouse/ClickHouse/pull/95181) ([Yue Ni](https://github.com/niyue)). * Text indexes can now store token positions (the `support_phrase_search` index argument) to enable exact phrase matching with direct reads for the `hasPhrase` function. Requires enabling the `MergeTree` setting `allow_experimental_text_index_phrase_search`. [#103172](https://github.com/ClickHouse/ClickHouse/pull/103172) ([Elmi Ahmadov](https://github.com/ahmadov)). The argument was initially introduced under the name `positions`. [#109900](https://github.com/ClickHouse/ClickHouse/pull/109900) ([Elmi Ahmadov](https://github.com/ahmadov)). * Support manifest file compaction for `Iceberg` tables via `OPTIMIZE TABLE ... MANIFEST` (guarded by the `allow_experimental_iceberg_compaction` setting). Closes [#95174](https://github.com/ClickHouse/ClickHouse/issues/95174). [#98178](https://github.com/ClickHouse/ClickHouse/pull/98178) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). Fix `OPTIMIZE TABLE ... MANIFEST` for `Iceberg` tables when manifest partition tuples do not match their partition spec. [#111368](https://github.com/ClickHouse/ClickHouse/pull/111368) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Add a Real Doubles (RD) variant and an optional `ALP(AUTO|STD|RD)` argument to the experimental `ALP` codec. Bare `ALP` now auto-selects between the existing `STD` scheme and `RD` instead of always using `STD`. [#99654](https://github.com/ClickHouse/ClickHouse/pull/99654) ([Nazarii Piontko](https://github.com/nazarii-piontko)). * Add an in-memory SLRU cache for deserialized `Paimon` metadata files (manifest lists and manifests). When enabled via the `use_paimon_metadata_files_cache` setting, repeated queries against the same `Paimon` table skip re-downloading and re-parsing metadata from object storage. [#104657](https://github.com/ClickHouse/ClickHouse/pull/104657) ([XiaoBinMu](https://github.com/Binnn-MX)). * Added "drivers" for executable user-defined functions. A driver declared in `` can be used in `CREATE FUNCTION name ARGUMENTS (...) RETURNS T ENGINE = DriverName(...) AS '...code...'` to compile or otherwise process a user code snippet at function-creation time and produce a runnable executable UDF. The resulting configuration is stored in ``, and the originating query is persisted as `ATTACH FUNCTION` so the function survives server restarts. A proof-of-concept `c_function_body` driver compiles and runs C function bodies inside sandboxed Docker containers in `executable_pool` mode. Based on the initial implementation by [Daniil Timižev](https://github.com/dahbka-lis) ([#77128](https://github.com/ClickHouse/ClickHouse/pull/77128)). [#105131](https://github.com/ClickHouse/ClickHouse/pull/105131) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Serve `SELECT count(*) FROM t WHERE col default(col)` from per-column sparsity statistics in `serialization.json` without any data scan when the predicate exactly partitions the column into defaults and non-defaults (under the new experimental setting `optimize_trivial_count_with_sparsity_filter`, off by default). [#105890](https://github.com/ClickHouse/ClickHouse/pull/105890) ([Raúl Marín](https://github.com/Algunenano)). * Prometheus Query API requests to `/api/v1/query` and `/api/v1/query_range` are now recorded in `system.query_log` with `read_rows` and `read_bytes` metrics. [#106611](https://github.com/ClickHouse/ClickHouse/pull/106611) ([James Cunningham](https://github.com/JTCunning)). * The experimental `ReaderExecutor` read path (`use_reader_executor`, off by default) can now hold a remote source connection open and reuse it across sequential reads, reducing the number of object storage requests on scans. [#107735](https://github.com/ClickHouse/ClickHouse/pull/107735) ([Sema Checherinda](https://github.com/CheSema)). It now also supports reading encrypted files, with a global encryption-header cache configured by the `encryption_header_cache_size` server setting and cleared by `SYSTEM DROP ENCRYPTION HEADERS CACHE`. [#109702](https://github.com/ClickHouse/ClickHouse/pull/109702) ([Sema Checherinda](https://github.com/CheSema)). * Support distributed query-plan reads for `SELECT ... FINAL` on `MergeTree`-family tables when `make_distributed_plan` is enabled. [#108148](https://github.com/ClickHouse/ClickHouse/pull/108148) ([Alexander Gololobov](https://github.com/davenger)). * Add experimental plan-based parallel replicas execution for `MergeTree` queries. [#108504](https://github.com/ClickHouse/ClickHouse/pull/108504) ([Igor Nikonov](https://github.com/devcrafter)). * Added the experimental `Quantize` vector codec family and an opt-in two-stage approximate vector-search rewrite over quantized companion streams. [#108565](https://github.com/ClickHouse/ClickHouse/pull/108565) ([Shankar Iyer](https://github.com/shankar-iyer)). * Revive the experimental `SZ3` error-bounded lossy compression codec for `Float32`/`Float64` (and arrays of them) columns, originally implemented by [Konstantin Vedernikov](https://github.com/scanhex12) ([#83088](https://github.com/ClickHouse/ClickHouse/pull/83088)). It requires `allow_experimental_codecs`. [#108788](https://github.com/ClickHouse/ClickHouse/pull/108788) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Fixed reading data compressed with `SZ3('ALGO_LORENZO_REG', ...)`, which previously failed with `CORRUPTED_DATA`, and avoided undefined behavior when compressing non-finite floating-point values (`NaN` and infinities). [#110762](https://github.com/ClickHouse/ClickHouse/pull/110762) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the experimental `ZXC` codec — an asymmetric LZ codec with slow compression and very fast decompression, with a compression ratio between `LZ4` and `ZSTD`. It requires `allow_experimental_codecs = 1`, and `system.compression_codecs` reports it as experimental. [#110620](https://github.com/ClickHouse/ClickHouse/pull/110620) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#111007](https://github.com/ClickHouse/ClickHouse/pull/111007) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * You can now authenticate to OneLake using a pre-obtained bearer token via the `onelake_bearer_token` setting, instead of a `onelake_client_id` and `onelake_client_secret`. This avoids sharing a long-lived client secret. The token is not refreshed, so the database must be recreated once it expires. [#109104](https://github.com/ClickHouse/ClickHouse/pull/109104) ([Asya Shneerson](https://github.com/asya-ch)). * AI functions no longer take the named collection as the first positional argument. Credentials come from the `ai_function_text_default_credentials` / `ai_function_embedding_default_credentials` settings or from a `credentials` key in an optional trailing `Map(String, String)` argument, which also carries tunables (`model`, `max_tokens`, `temperature`, `system_prompt`, `instructions`, `dimensions`). [#109232](https://github.com/ClickHouse/ClickHouse/pull/109232) ([George Larionov](https://github.com/george-larionov)). The `aiEmbed` function now takes `model` as a required positional argument (`aiEmbed(text, model[, params])`) instead of reading it from the named collection, which ensures reproducibility for embeddings. [#110619](https://github.com/ClickHouse/ClickHouse/pull/110619) ([George Larionov](https://github.com/george-larionov)). * Fix `data_uncompressed_bytes` for skip indices packed into `skp_idx.packed` (via `packed_skip_index_max_bytes`): the reported uncompressed size was the compressed size, which could also prevent `distributed_index_analysis` from activating. [#109272](https://github.com/ClickHouse/ClickHouse/pull/109272) ([Raúl Marín](https://github.com/Algunenano)). * Added experimental packed data part storage for `MergeTree` tables, which stores most of a part's files in a single `data.packed` archive instead of a file per stream (projections and a few service files, such as `txn_version.txt`, are still written separately). It is controlled by the `min_bytes_for_full_part_storage`, `min_rows_for_full_part_storage`, and `min_level_for_full_part_storage` settings and is disabled by default. Once a table writes packed parts, older server versions cannot read them, so enabling this format prevents downgrading the server. [#108118](https://github.com/ClickHouse/ClickHouse/pull/108118) ([Raúl Marín](https://github.com/Algunenano)). * Added an experimental table function `eval`, which evaluates a constant expression to a query string and executes the resulting single `SELECT` query. The feature is disabled by default and can be enabled with the setting `allow_experimental_eval_table_function`. Author: [Yue Ni](https://github.com/niyue). [#110132](https://github.com/ClickHouse/ClickHouse/pull/110132) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `zstd` compression support to the experimental Prometheus remote-write v1 HTTP handler. [#110907](https://github.com/ClickHouse/ClickHouse/pull/110907) ([James Cunningham](https://github.com/JTCunning)). * Writing to `DeltaLake` tables (the `allow_experimental_delta_lake_writes` setting) was promoted to Beta. [#107034](https://github.com/ClickHouse/ClickHouse/pull/107034) ([Kseniia Sumarokova](https://github.com/kssenii)). * Added `ProfileEvents` `DistributedPlanRemoteTasks`, `DistributedPlanLocalExecution`, and `DistributedPlanHostsUsed` to observe the execution of experimental distributed query plans (`make_distributed_plan`). [#107985](https://github.com/ClickHouse/ClickHouse/pull/107985) ([Shankar Iyer](https://github.com/shankar-iyer)). * Removed the experimental RocksDB-based storage in Keeper (`experimental_use_rocksdb`). A better new on-disk storage is coming soon. [#108000](https://github.com/ClickHouse/ClickHouse/pull/108000) ([Michael Kolupaev](https://github.com/al13n321)). * Distributed index analysis (an experimental feature controlled by the `distributed_index_analysis` setting) now sends big lists of data part names to remote servers as scalars instead of embedding them into the query text. This avoids possible failures due to the `max_ast_elements` / `max_query_size` limits and removes the AST parsing overhead. [#110419](https://github.com/ClickHouse/ClickHouse/pull/110419) ([Azat Khuzhin](https://github.com/azat)). * Optimize the experimental `timeSeries*ToGrid` aggregation functions: buckets are now aligned to both step and window, and the final aggregation is much faster when the window is much larger than the step. The serialization format of the aggregation state was changed in an incompatible way (these functions are experimental). [#106724](https://github.com/ClickHouse/ClickHouse/pull/106724) ([Vitaly Baranov](https://github.com/vitlibar)). * External target tables of a `TimeSeries` table are now registered as referential dependencies: similarly to materialized views, this disallows dropping an external target table before dropping the `TimeSeries` table that references it (if `check_referential_table_dependencies` is enabled). [#108388](https://github.com/ClickHouse/ClickHouse/pull/108388) ([Vitaly Baranov](https://github.com/vitlibar)). * Add the PromQL function `increase`. [#111023](https://github.com/ClickHouse/ClickHouse/pull/111023) ([Vitaly Baranov](https://github.com/vitlibar)). #### Performance Improvement * `JOIN`s can now use the primary key index or skip indexes on the left-hand side table to prune granules. Controlled by the setting `enable_join_runtime_filters_index_analysis`. [#109085](https://github.com/ClickHouse/ClickHouse/pull/109085) ([Shankar Iyer](https://github.com/shankar-iyer)). * Serialization and deserialization of vectors of trivially serializable types now use a single buffer I/O call instead of processing individual elements. [#89842](https://github.com/ClickHouse/ClickHouse/pull/89842) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Optimize inverse dictionary lookups: a constant equality predicate such as `WHERE dictGet(dict, attr, key_expr) = value` is now constant-folded directly into a key filter (`key_expr = const`, `key_expr IN [...]`, or `WHERE 0`) instead of being rewritten to an `IN (SELECT ... FROM dictionary(...))` subquery, and the constant path of `dictGetKeys` now executes in parallel. [#91164](https://github.com/ClickHouse/ClickHouse/pull/91164) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Add a new setting `merge_tree_generic_exclusion_search_max_steps` that limits the number of steps the generic exclusion search algorithm spends analyzing the primary key index of each data part. The budget is spent on the largest remaining key ranges first, so even a small budget prunes the bulk of the data; ranges that were not fully analyzed are read whole, so query results stay correct but more granules may be read. The limit is applied loosely: it may be exceeded by at most `merge_tree_coarse_index_granularity` steps, plus one step for each range the part is already divided into (for example, by the query condition cache). The default value 0 means unlimited steps. [#92779](https://github.com/ClickHouse/ClickHouse/pull/92779) ([Michael Jarrett](https://github.com/EmeraldShift)). * Queries with `ORDER BY` and `LIMIT` on views over `Distributed` tables now use the merge-sorted-streams optimization: the outer `ORDER BY`/`LIMIT` is pushed into a simple view's inner query, so each shard sorts its data locally and the coordinator merges pre-sorted streams instead of performing a full sort. [#94102](https://github.com/ClickHouse/ClickHouse/pull/94102) ([matanper](https://github.com/matanper)). * Enable `optimize_or_like_chain` by default. Closes [#87779](https://github.com/ClickHouse/ClickHouse/issues/87779). [#94517](https://github.com/ClickHouse/ClickHouse/pull/94517) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve the performance of fetching a single user record from `system.users` when the server has many users. [#96699](https://github.com/ClickHouse/ClickHouse/pull/96699) ([Alistair Evans](https://github.com/alistairjevans)). * Reading in order with parallel replicas now uses the same logic of splitting the table into `max_threads` parts as local reading, for better parallelism. [#101434](https://github.com/ClickHouse/ClickHouse/pull/101434) ([Nikita Taranov](https://github.com/nickitat)). * Use partition minmax index bounds to prune more granules during primary key analysis for `MergeTree` tables, when a primary key column is also an input column of the partition key. For example, in a `MergeTree` table with `ORDER BY (id, event_time)` and `PARTITION BY toYYYYMM(event_time)`, ClickHouse will use the partition minmax index on `event_time` during primary key index analysis to make more informed granule-pruning decisions. Controlled by the new setting `use_partition_minmax_for_primary_key_pruning` (enabled by default). [#103480](https://github.com/ClickHouse/ClickHouse/pull/103480) ([UnamedRus](https://github.com/UnamedRus)). * Enable the query condition cache for queries that use Top-K dynamic filtering (`ORDER BY ... LIMIT n`). The cache entry is partitioned by the Top-K plan parameters, so the same query reuses cached `WHERE` filter results, while a different `LIMIT`, sort column, or sort direction produces a fresh entry. [#104478](https://github.com/ClickHouse/ClickHouse/pull/104478) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `use_constant_folding_in_index_analysis` setting (disabled by default). When enabled, `MergeTree` primary-key, `MinMax`, and skip-index analysis fold partition-level constants into the filter predicate separately for each part, improving pruning for filters whose branches depend on partition values, e.g. `(a = 1 AND b >= 1) OR (a = 2 AND b > 10)` with `PARTITION BY a`. [#104582](https://github.com/ClickHouse/ClickHouse/pull/104582) ([Mikhail Artemenko](https://github.com/Michicosun)). * Push `LIMIT` into aggregation-in-order to enable early termination when the `GROUP BY` key matches the `ORDER BY` key and the table sorting key, significantly reducing the number of rows read. [#104859](https://github.com/ClickHouse/ClickHouse/pull/104859) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Updating profile events is now up to 30 times faster, using per-CPU atomics for server-wide and per-user counters. Controlled by the `user_profile_events_per_cpu` server setting (it uses around 640 KiB per user on 64 cores, so with a very large number of users it may make sense to keep it disabled). [#105056](https://github.com/ClickHouse/ClickHouse/pull/105056) ([Azat Khuzhin](https://github.com/azat)). * Enable `allow_aggregate_partitions_independently` by default. When a `GROUP BY` key suits the partition key, ClickHouse can aggregate each partition independently and skip the global merging step. Runtime heuristics automatically skip the optimization when the partition layout would make it unfavorable (too few partitions, too many partitions, or significantly skewed partition sizes). [#105128](https://github.com/ClickHouse/ClickHouse/pull/105128) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Vector search queries with rescoring (`vector_search_with_rescoring = 1`) now compute the exact distance only for the rows returned by the vector index, applying an exact row-position filter instead of brute-force rescoring neighbouring rows from the same `MergeTree` granule. [#105591](https://github.com/ClickHouse/ClickHouse/pull/105591) ([Sergey Kuznetsov](https://github.com/skuznetsov)). [#108846](https://github.com/ClickHouse/ClickHouse/pull/108846) ([Sergey Kuznetsov](https://github.com/skuznetsov)). * Improve the performance of the `partial_merge` join algorithm on `FixedString` keys by comparing runs of values at once instead of one comparison call per value. [#105737](https://github.com/ClickHouse/ClickHouse/pull/105737) ([Artem Zuikov](https://github.com/4ertus2)). * `SHOW TABLES` and `system.tables` on data lake catalogs (Iceberg REST, Glue, Unity, Hive, Paimon) now push namespace-bound predicates down to the catalog, avoiding a full catalog scan. Closes [#105022](https://github.com/ClickHouse/ClickHouse/issues/105022). [#106029](https://github.com/ClickHouse/ClickHouse/pull/106029) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Bound untracked memory per CPU to avoid overcommit and out-of-memory errors (the limits are configured with the `max_per_cpu_untracked_memory` and `per_cpu_untracked_memory_thread_buffer` server settings). [#106055](https://github.com/ClickHouse/ClickHouse/pull/106055) ([Azat Khuzhin](https://github.com/azat)). * Speed up operators that scan a sorted stream for runs of equal key values — `DISTINCT` in order, `LIMIT BY` in order, negative `LIMIT BY` in order, `full_sorting_merge` and `partial_merge` joins. [#106502](https://github.com/ClickHouse/ClickHouse/pull/106502) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Implemented a native reader and writer for the `Arrow` and `ArrowStream` formats that does not use the Apache Arrow library, avoiding extra data copies and conversions. It is now the default (settings `input_format_arrow_use_native_reader` and `output_format_arrow_use_native_writer`) and is faster for both reading and writing. [#106522](https://github.com/ClickHouse/ClickHouse/pull/106522) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Re-enable LTO for jemalloc after fixing a thread-cache corruption that appeared with LTO, which improves performance. [#106898](https://github.com/ClickHouse/ClickHouse/pull/106898) ([Azat Khuzhin](https://github.com/azat)). * Reduced memory usage and improved the performance of `JOIN`s. The right-hand side of a hash join now uses a compact 8-byte index-based row reference, so the hash-table entries of `ALL` joins are as small as those of `ANY` joins for every key type. Benchmarked on large `parallel_hash` join queries — `INNER` and `ANY` joins of 100–300 million row tables on `UInt64` and `String` keys, with both unique and duplicated keys: the median query became about 12% faster and used about 15% less peak memory, with no query becoming slower. `INNER` joins on `UInt64` keys gained the most (median about 21% faster with 38% less memory). [#107189](https://github.com/ClickHouse/ClickHouse/pull/107189) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * Single-column non-Nullable `LowCardinality(String)` join keys are now natively supported in the hash join. [#107264](https://github.com/ClickHouse/ClickHouse/pull/107264) ([Nikita Taranov](https://github.com/nickitat)). * Added the `dpsub` join-order enumeration algorithm (dynamic programming over subsets), providing optimal join plans with lower optimization overhead than `dpsize` and support for non-inner joins. [#107351](https://github.com/ClickHouse/ClickHouse/pull/107351) ([Fisnik Kastrati](https://github.com/fkastrati)). * Speed up filesystem cache loading on startup by encoding each cache file's size in its name (`_`), avoiding a `stat` per file. Loading of cache files written by older versions remains supported. [#107415](https://github.com/ClickHouse/ClickHouse/pull/107415) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve the performance of `arrayElement` on `Array(LowCardinality(String))` and of the map `LIKE` functions on `Map`s with `LowCardinality(String)` keys or values, by avoiding unnecessary string materialization. [#107450](https://github.com/ClickHouse/ClickHouse/pull/107450) ([Michael Jarrett](https://github.com/EmeraldShift)). * Avoid converting single-level aggregation to two-level aggregation when memory usage is small, by tracking the memory of the aggregation state itself instead of the memory of the whole query. [#107490](https://github.com/ClickHouse/ClickHouse/pull/107490) ([Hechem Selmi](https://github.com/m-selmi)). * Speed up analysis of aggregating queries: the cache key for the aggregation hash-table size statistics is now computed from the query plan instead of building the full query AST. [#107643](https://github.com/ClickHouse/ClickHouse/pull/107643) ([Dmitry Novik](https://github.com/novikd)). * Improve the performance of `INTERSECT ALL` and `EXCEPT ALL` (the default mode for `INTERSECT` and `EXCEPT`) by several times, by keying the multiset on the row value instead of hashing each row with SipHash. [#107649](https://github.com/ClickHouse/ClickHouse/pull/107649) ([Raúl Marín](https://github.com/Algunenano)). * Rebuild row policy filters without holding the cache lock, improving query scalability under concurrent `CREATE ROW POLICY` and `DROP ROW POLICY`. [#107917](https://github.com/ClickHouse/ClickHouse/pull/107917) ([Azat Khuzhin](https://github.com/azat)). * Speed up the `match`, `extract`, `extractAll`, `replaceRegexpOne` and `replaceRegexpAll` functions for simple regular expressions by compiling them to native code with LLVM. Controlled by the new setting `compile_regular_expressions` (enabled by default); patterns outside the supported subset transparently fall back to the RE2 engine. [#108004](https://github.com/ClickHouse/ClickHouse/pull/108004) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Sped up ZSTD decompression on AArch64 (ARM) for columns with small match offsets, such as fixed-width integer columns, by vectorizing short-offset overlapping copies with NEON. For example, decompression of a `UInt64` column is up to ~2.3x faster on AWS Graviton 4. [#108049](https://github.com/ClickHouse/ClickHouse/pull/108049) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use the `libdeflate` library for gzip/zlib/deflate compression and decompression, making it faster (compression ~1.15× with a better ratio, decompression ~1.4–1.5×) for `.gz`/HTTP/`url`/`s3` data and the Parquet GZIP codec. [#108074](https://github.com/ClickHouse/ClickHouse/pull/108074) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved performance of text-search queries combined with a primary key filter. [#108114](https://github.com/ClickHouse/ClickHouse/pull/108114) ([Anton Popov](https://github.com/CurtizJ)). * Use runtime statistics collected during the first run of a query to optimize subsequent runs: the `hash` join algorithm can automatically switch to `parallel_hash` based on the collected statistics. [#108125](https://github.com/ClickHouse/ClickHouse/pull/108125) ([Hechem Selmi](https://github.com/m-selmi)). The size of join runtime filters is optimized using the hash table size from runtime join statistics, controlled by the new setting `join_runtime_filter_size_from_hash_table_stats`. [#108313](https://github.com/ClickHouse/ClickHouse/pull/108313) ([Hechem Selmi](https://github.com/m-selmi)). * LZ4 decompression speed was improved for fixed-size, low-cardinality columns. [#108175](https://github.com/ClickHouse/ClickHouse/pull/108175) ([Nikita Taranov](https://github.com/nickitat)). * Speed up the default-whitespace `trimLeft`/`trimRight`/`trimBoth` functions (and their aliases `ltrim`/`rtrim`/`trim`): consecutive rows that need no trimming are copied in a single batch, and the per-row space scan runs only for rows that actually have a leading or trailing space. [#108177](https://github.com/ClickHouse/ClickHouse/pull/108177) ([Groene AI](https://github.com/groeneai)). * Improve the performance of `replaceAll` and `replaceRegexpAll` when the pattern is a single character and the replacement is a single character (for example `replaceRegexpAll(s, ' ', '_')`). Such replacements no longer change the string layout, so the column is now copied once and matching bytes are rewritten in place instead of running a per-match search loop. [#108178](https://github.com/ClickHouse/ClickHouse/pull/108178) ([Groene AI](https://github.com/groeneai)). * Improve performance of the `pointInPolygon` function with a constant polygon. The preprocessed-polygon cache is now keyed on the raw constant arguments, so a constant polygon is parsed only once (on a cache miss) instead of being re-parsed on every input block. [#108184](https://github.com/ClickHouse/ClickHouse/pull/108184) ([Groene AI](https://github.com/groeneai)). * Speed up parsing of the canonical `YYYY-MM-DD hh:mm:ss` date-time representation in best-effort mode (`date_time_input_format = 'best_effort'`, `cast_string_to_date_time_mode = 'best_effort'`), which is the default. This recovers a parsing performance regression introduced when those defaults were switched from `basic` to `best_effort`. [#108187](https://github.com/ClickHouse/ClickHouse/pull/108187) ([Groene AI](https://github.com/groeneai)). * Avoid maintaining `system.predicate_statistics_log` selectivity counters on the `MergeTree` read path when the feature is disabled (`predicate_statistics_sample_rate = 0`, the default). This removes a per-granule atomic update and an `O(rows)` filter popcount from every read, recovering a performance regression that was most visible on AArch64. [#108190](https://github.com/ClickHouse/ClickHouse/pull/108190) ([Groene AI](https://github.com/groeneai)). * Speed up parsing of floating-point numbers from text with `precise_float_parsing = 1`, making it as fast as or faster than the default parser on almost all inputs. Also bumps the bundled `fast_float` library to v8.2.10. [#108205](https://github.com/ClickHouse/ClickHouse/pull/108205) ([Raúl Marín](https://github.com/Algunenano)). * Use the `simdutf` library for base64 encoding and decoding functions, improving their performance. [#108333](https://github.com/ClickHouse/ClickHouse/pull/108333) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Optimize `DISTINCT` for expensive high-cardinality keys. [#108366](https://github.com/ClickHouse/ClickHouse/pull/108366) ([Nihal Z. Miaji](https://github.com/nihalzp)). * On non-Linux builds (macOS and FreeBSD), jemalloc no longer purges dirty pages eagerly (a `madvise` syscall per `free`); it now uses a background purging thread with a finite `dirty_decay_ms`, like on Linux. This significantly speeds up allocation-churning workloads such as recursive CTEs (~2x on a recursive-CTE benchmark on macOS) with negligible RSS impact. [#108430](https://github.com/ClickHouse/ClickHouse/pull/108430) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up query planning under the analyzer for expressions that reference the same `WITH` alias or repeat the same subexpression many times (for example deeply nested `if`/`multiIf` chains), by not rebuilding shared subexpressions when constructing the actions DAG. [#108523](https://github.com/ClickHouse/ClickHouse/pull/108523) ([Dmitry Novik](https://github.com/novikd)). * Runtime filters are now built on equality keys even when the `ON` clause also contains non-equality predicates, reducing the number of rows entering the hash join for joins that mix equality and non-equality conditions. `LEFT ANTI JOIN` is unaffected. [#108579](https://github.com/ClickHouse/ClickHouse/pull/108579) ([Antonio Álvarez Caballero](https://github.com/29antonioac)). * Enable `-fno-math-errno` globally across the codebase, which allows the compiler to optimize math functions better. [#108628](https://github.com/ClickHouse/ClickHouse/pull/108628) ([Nikita Taranov](https://github.com/nickitat)). * Lower the software-prefetch threshold for aggregation and join hash tables from `4 * L2 cache size` to `L2 cache size`, so prefetch is enabled once the hash table no longer fits in L2. This recovers a regression on `GROUP BY` and `JOIN` over medium-sized hash tables (~1-8 MiB) on platforms whose reported L2 is large (e.g. AArch64 with a 2 MiB L2). [#108655](https://github.com/ClickHouse/ClickHouse/pull/108655) ([Groene AI](https://github.com/groeneai)). * Improve window function performance. Window queries over wide tables that include `SELECT *` now use significantly less memory and run faster. Window aggregates, including `quantile*`, `uniq*`, and `groupArray`, are faster with `OVER ()` and `OVER (ORDER BY ... RANGE ...)`. The `cume_dist` function is also faster, and windows using `PARTITION BY` or `ORDER BY` now perform better on columns with many repeated values. [#108688](https://github.com/ClickHouse/ClickHouse/pull/108688) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Sped up query analysis for queries with many or very large `AND`-chains of comparisons: the `optimize_and_compare_chain` optimization is now bounded by a work budget (new setting `optimize_and_compare_chain_max_hash_work`) instead of hashing a large fraction of the query tree, and lambda resolution no longer recomputes lambda-body hashes for its recursion guard. [#108757](https://github.com/ClickHouse/ClickHouse/pull/108757) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a performance regression when reading `JSON`/`Dynamic` columns with multiple threads: a per-value lookup of the `input_format_binary_max_type_complexity` setting caused reference-count contention on the shared query context, serializing parallel reads; the setting value is now cached per thread. [#108797](https://github.com/ClickHouse/ClickHouse/pull/108797) ([Seva Potapov](https://github.com/seva-potapov)). * Fixed a performance regression (introduced in [#71781](https://github.com/ClickHouse/ClickHouse/issues/71781)) where reading many small files from object storage via the `s3` and other table functions stopped prefetching and fell back to synchronous reads, significantly slowing single-threaded and low-concurrency reads of lots of tiny files. [#108872](https://github.com/ClickHouse/ClickHouse/pull/108872) ([Nikita Fomichev](https://github.com/fm4v)). The initial small-object prefetch is now also issued when object storage reads go through the filesystem cache (`filesystem_cache_name`), so reads of many small files (e.g. `S3Queue` ingestion) with the cache enabled are no longer synchronous and latency-bound. [#109478](https://github.com/ClickHouse/ClickHouse/pull/109478) ([Nikita Fomichev](https://github.com/fm4v)). * Parse deeply nested array and tuple literals in linear instead of quadratic time. [#108892](https://github.com/ClickHouse/ClickHouse/pull/108892) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduce the overhead coming from lock contention in the `parallel_hash` join algorithm. [#108938](https://github.com/ClickHouse/ClickHouse/pull/108938) ([Hechem Selmi](https://github.com/m-selmi)). * Speed up `estimateCompressionRatio` for `T64`-encoded columns by computing the compressed size analytically instead of compressing. [#109054](https://github.com/ClickHouse/ClickHouse/pull/109054) ([Raufs Dunamalijevs](https://github.com/rienath)). * Remove a single-threaded bottleneck in the shuffle step of distributed query plans: each upstream stream now scatters its rows by destination bucket independently instead of funneling the whole data stream through a single transform. [#109206](https://github.com/ClickHouse/ClickHouse/pull/109206) ([Alexander Gololobov](https://github.com/davenger)). * Reduced Keeper server idle CPU usage. [#109255](https://github.com/ClickHouse/ClickHouse/pull/109255) ([Michael Kolupaev](https://github.com/al13n321)). * Speed up serialization and merges of `JSON` columns whose shared data contains many sparse paths: when flattening shared data into per-path columns, the per-row scan over all accumulated path columns is removed, and gaps are backfilled in bulk instead of inserting default values one cell at a time. [#109341](https://github.com/ClickHouse/ClickHouse/pull/109341) ([Groene AI](https://github.com/groeneai)). * Reduce CPU overhead of expression evaluation for queries with a large number of columns (e.g. vector search over a `QBit` column with a small stride, where each vector expands into hundreds of bit-plane sub-columns that are all fed into a single `*DistanceTransposed` call). [#109380](https://github.com/ClickHouse/ClickHouse/pull/109380) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve the performance of H3 geo functions (`h3ToGeoBoundary`, `h3ToGeo`, `h3CellAreaM2`/`h3CellAreaRads2`, `geoToH3`) by computing paired sine/cosine together and eliminating redundant trigonometric calls in the coordinate transforms. Results are unchanged (bit-for-bit identical). [#109399](https://github.com/ClickHouse/ClickHouse/pull/109399) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve performance of comparisons (`<`, `>`, `<=`, `>=`) of wide integer types (`Int128`, `UInt128`, `Int256`, `UInt256` and types based on them, such as `Decimal128`) by up to 7 times, and of converting signed integers to `Int256` (up to 7 times on mixed-sign data), by making the comparison and sign-extension code branchless. [#109474](https://github.com/ClickHouse/ClickHouse/pull/109474) ([Manuel](https://github.com/raimannma)). * Optimized `uniqCombined` (including the case of aggregation without keys) and `uniqHLL12`. [#109794](https://github.com/ClickHouse/ClickHouse/pull/109794) ([Anton Popov](https://github.com/CurtizJ)). [#109831](https://github.com/ClickHouse/ClickHouse/pull/109831) ([Anton Popov](https://github.com/CurtizJ)). * Functions `arraySort` and `arrayReverseSort` are several times faster over numeric arrays (including `Decimal` and `DateTime64`) when called without a lambda. [#109832](https://github.com/ClickHouse/ClickHouse/pull/109832) ([Manuel](https://github.com/raimannma)). * Speed up `addDays`, `addWeeks`, `subtractDays`, and `subtractWeeks` on `DateTime` and `DateTime64` values in fixed-offset time zones (such as UTC) by taking an arithmetic fast path. [#109836](https://github.com/ClickHouse/ClickHouse/pull/109836) ([Manuel](https://github.com/raimannma)). * Optimized analysis of the text index. [#109886](https://github.com/ClickHouse/ClickHouse/pull/109886) ([Anton Popov](https://github.com/CurtizJ)). * Parallelize processing of non-joined rows in the `parallel_hash` join algorithm in two more cases: when there is a residual filter, and in the case of small join keys (single-level hash map). [#110008](https://github.com/ClickHouse/ClickHouse/pull/110008) ([Hechem Selmi](https://github.com/m-selmi)). * Built-in web UI responses (the Play UI, dashboards, and static-file handlers) are now compressed according to the client's `Accept-Encoding` header, using zstd, gzip, deflate, brotli, lz4, xz, snappy, or bzip2. For example, the `/play` page shrinks from ~350 KB uncompressed to under 120 KB with zstd, reducing load times on slow connections. Already-compressed responses are left untouched, and `Vary: Accept-Encoding` is emitted for correct CDN and cache partitioning. [#110108](https://github.com/ClickHouse/ClickHouse/pull/110108) ([Sayantanu Dey](https://github.com/sayantanu-dey)). * Push a filter below a window function or below `LIMIT BY` when it is safe: a predicate written above them (an outer `WHERE` around the subquery, or a conjunct in `QUALIFY`) that references only the window `PARTITION BY` columns, or only the `LIMIT BY` key columns (for `LIMIT n BY` with `n >= 1` and no `OFFSET`), now reaches storage and enables primary key and partition pruning, skip indexes, and projections. For example, `SELECT ... row_number() OVER (PARTITION BY key ORDER BY ts) AS rn ... QUALIFY rn = 1 AND key = 'x'` no longer reads and windows the whole table. [#110114](https://github.com/ClickHouse/ClickHouse/pull/110114) ([Groene AI](https://github.com/groeneai)). [#110116](https://github.com/ClickHouse/ClickHouse/pull/110116) ([Groene AI](https://github.com/groeneai)). * With `join_use_nulls = 1`, a null-rejecting `WHERE` that references columns from both sides of an outer join (e.g. `WHERE l.k = 42 AND r.k = 42`) now converts the join to `INNER` (or `LEFT`/`RIGHT`) and prunes the primary key, the same as with `join_use_nulls = 0`. Previously the conversion was skipped in this case and both tables were read in full. [#110121](https://github.com/ClickHouse/ClickHouse/pull/110121) ([Groene AI](https://github.com/groeneai)). * Functions with a single non-const `Nullable` argument now share the argument's null map with the result instead of allocating and merging a new one. [#110151](https://github.com/ClickHouse/ClickHouse/pull/110151) ([Manuel](https://github.com/raimannma)). * Avoid a heap allocation per hyperrectangle check in primary key index analysis, speeding up mark filtering by 5-26% depending on the query shape. [#110153](https://github.com/ClickHouse/ClickHouse/pull/110153) ([Manuel](https://github.com/raimannma)). * Improve performance of functions `arrayMin` and `arrayMax` over numeric arrays by ~1.3-1.5x by using a vectorized reduction instead of a per-element comparison loop. [#110163](https://github.com/ClickHouse/ClickHouse/pull/110163) ([Manuel](https://github.com/raimannma)). * Vectorize decompression of the `Delta` codec. Decoding is now 1.5–5 times faster for 8/16/32-bit data types, making scans of `Delta`-compressed columns up to 20% faster. [#110189](https://github.com/ClickHouse/ClickHouse/pull/110189) ([Manuel](https://github.com/raimannma)). * Speed up string search functions (`like`, `position`, `match`, `countSubstrings`, `hasToken`, etc.) over `Enum` columns with a constant needle by searching only the distinct enum names and mapping the results back per row, instead of searching every row. [#110325](https://github.com/ClickHouse/ClickHouse/pull/110325) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up aggregate functions `groupBitOr`/`groupBitAnd`/`groupBitXor` and the variance family (`varPop`, `varSamp`, `stddevPop`, `stddevSamp`, `skewPop`, `skewSamp`, `kurtPop`, `kurtSamp`, `covarPop`, `covarSamp`, `corr`) with vectorized batch processing: up to 4x faster with the `-If` combinator on unpredictable conditions, and up to 3x faster for the variance family without it. [#110461](https://github.com/ClickHouse/ClickHouse/pull/110461) ([Manuel](https://github.com/raimannma)). * Speed up the analysis of the text index (the stage that selects which granules to read): use exclusion search instead of iterating over all granules, and avoid creating a temporary bitmap for each range check. [#110530](https://github.com/ClickHouse/ClickHouse/pull/110530) ([Anton Popov](https://github.com/CurtizJ)). * Account for the on-disk size of `Map` subcolumns in the automatic `PREWHERE` optimization. This fixes a performance regression in `PREWHERE` introduced in [#99200](https://github.com/ClickHouse/ClickHouse/pull/99200). Part of [#110462](https://github.com/ClickHouse/ClickHouse/issues/110462). [#110623](https://github.com/ClickHouse/ClickHouse/pull/110623) ([Pavel Kruglov](https://github.com/Avogar)). * Speed up parsing of `UUID` values from text (e.g. `JSONExtract` into `LowCardinality(UUID)`, `toUUID`, `CAST AS UUID`) by validating and converting hex digits in a single pass instead of two. [#110625](https://github.com/ClickHouse/ClickHouse/pull/110625) ([Groene AI](https://github.com/groeneai)). * Significantly improve performance of the `timeSeries*ToGrid` aggregate functions by using faster sorting and hash table implementations. [#110875](https://github.com/ClickHouse/ClickHouse/pull/110875) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Queries like `SELECT * FROM t WHERE id` now use index skipping on the `id` column. Closes [#89222](https://github.com/ClickHouse/ClickHouse/issues/89222). [#89603](https://github.com/ClickHouse/ClickHouse/pull/89603) ([Aditya Chopra](https://github.com/adityachopra29)). * Added the `query_plan_merge_expression_into_join` setting (enabled by default) to allow merging expression steps into `JOIN` steps during the join reordering optimization. This enables join reordering across subqueries that wrap joins (e.g., when a `JOIN` is inside a subquery with computed columns), leading to better optimization of complex join trees. [#98533](https://github.com/ClickHouse/ClickHouse/pull/98533) ([Vladimir Cherkasov](https://github.com/vdimir)). * Optimize `AND` chains with multiple comparison conditions on the same expression: detect contradictions (e.g., `a < 3 AND a > 5` → `false`) and prune redundant conditions (e.g., `a = 3 AND a < 5` → `a = 3`). [#99736](https://github.com/ClickHouse/ClickHouse/pull/99736) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Materialized CTEs referenced from multiple branches of a `UNION` query are now materialized once and shared across all branches. Previously, each branch received its own copy of the CTE, which was inlined and evaluated separately. [#102107](https://github.com/ClickHouse/ClickHouse/pull/102107) ([Dmitry Novik](https://github.com/novikd)). * Threads in `BackgroundSchedulePool` are now created lazily on demand instead of all at once at server startup. A new server setting `background_schedule_pool_initial_size` (default `16`) controls how many workers are pre-spawned; the pool grows up to `background_schedule_pool_size` when needed. This reduces the number of idle threads on lightly-loaded servers. Closes [#85265](https://github.com/ClickHouse/ClickHouse/issues/85265). [#105066](https://github.com/ClickHouse/ClickHouse/pull/105066) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Skip unnecessary mark file loads for the `JSON` type's advanced shared data serialization. [#107051](https://github.com/ClickHouse/ClickHouse/pull/107051) ([Pavel Kruglov](https://github.com/Avogar)). * The cache of preprocessed constant polygons for the function `pointInPolygon` is now shared between queries and threads and bounded by the new server setting `point_in_polygon_cache_size` (default 256 MiB, `0` disables the cache, changeable at runtime). Previously, the cache kept up to `max_threads` copies of each preprocessed polygon, grew without bound across distinct polygons, and was never released until server restart. It can be cleared with `SYSTEM DROP POINT IN POLYGON CACHE`, and the current usage is reported in `system.metrics` as `PointInPolygonCacheBytes`, `PointInPolygonCacheCells`, and `PointInPolygonCacheSizeLimit`. Closes [#106393](https://github.com/ClickHouse/ClickHouse/issues/106393). [#107247](https://github.com/ClickHouse/ClickHouse/pull/107247) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Keeper no longer holds its internal snapshots lock while serializing and writing a snapshot to disk or moving snapshots between disks, so snapshot transfers to other nodes and Raft processing are no longer stalled by local snapshot creation. [#107595](https://github.com/ClickHouse/ClickHouse/pull/107595) ([Antonio Andelic](https://github.com/antonio2368)). * Parallelize the filesystem cache background eviction (controlled by the `keep_free_space_ratio` settings). Added a new setting `keep_free_space_eviction_threads` to control the parallelism, and increased the default of `keep_free_space_remove_batch` from `100` to `250`. [#108147](https://github.com/ClickHouse/ClickHouse/pull/108147) ([Kseniia Sumarokova](https://github.com/kssenii)). * Limit the number of concurrently staged parts in the Azure Blob Storage read-then-write copy (used by backups when native copy is unavailable) to `max_inflight_parts_for_one_file`, preventing excessive memory usage when many large files are copied in parallel. [#108232](https://github.com/ClickHouse/ClickHouse/pull/108232) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Enabled all three text index caches globally — previously, they were only enabled within queries. Also, the posting lists cache size is now zero, which effectively disables it again, because posting lists are large and caching them is too costly. [#108274](https://github.com/ClickHouse/ClickHouse/pull/108274) ([Robert Schulze](https://github.com/rschu1ze)). * Add streaming in-memory marks compression during marks loading to reduce peak memory usage. Closes [#108285](https://github.com/ClickHouse/ClickHouse/issues/108285). [#108325](https://github.com/ClickHouse/ClickHouse/pull/108325) ([Pavel Kruglov](https://github.com/Avogar)). * The new filesystem cache setting `reserve_granularity` (default 4 MiB) reserves space ahead in coarser granules, reducing lock contention on the filesystem cache space reservation hot path. [#108369](https://github.com/ClickHouse/ClickHouse/pull/108369) ([Kseniia Sumarokova](https://github.com/kssenii)). * Reduce the amount of memory used by the filesystem cache metadata (per file segment and per key). [#108477](https://github.com/ClickHouse/ClickHouse/pull/108477) ([Kseniia Sumarokova](https://github.com/kssenii)). * Queries to `system.iceberg_history` with `WHERE` filters on the `database` and `table` columns now access only the matching databases, which can save time if you have many unrelated remote databases on the server. [#108492](https://github.com/ClickHouse/ClickHouse/pull/108492) ([Den Kalantaevskii](https://github.com/Diskein)). * Removed unnecessary locking in the filesystem cache to reduce lock contention. [#108932](https://github.com/ClickHouse/ClickHouse/pull/108932) ([Kseniia Sumarokova](https://github.com/kssenii)). * Reduced memory usage of `BACKUP` and `RESTORE` metadata handling. Opening a backup (for `RESTORE`, or as the base of an incremental `BACKUP`) now parses the `.backup` metadata as a stream instead of loading it into an in-memory XML document tree, which for large (especially incremental) backups avoids allocating a multi-gigabyte DOM tree. [#109107](https://github.com/ClickHouse/ClickHouse/pull/109107) ([Julia Kartseva](https://github.com/jkartseva)). Finalizing a `BACKUP` no longer copies the file infos of all files into a temporary vector, which, for backups of millions of files, transiently cost several gigabytes. [#109861](https://github.com/ClickHouse/ClickHouse/pull/109861) ([Julia Kartseva](https://github.com/jkartseva)). Writing backup entries also no longer copies the current host's file-info list. [#111162](https://github.com/ClickHouse/ClickHouse/pull/111162) ([Julia Kartseva](https://github.com/jkartseva)). * Reduce memory consumption in the aggregator. [#109224](https://github.com/ClickHouse/ClickHouse/pull/109224) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Support the query condition cache for local `Parquet` files read via the `File` table engine (previously supported only for object storage and data lakes). [#109247](https://github.com/ClickHouse/ClickHouse/pull/109247) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduced the memory usage of the text index header cache by 2-2.5 times. More headers of data parts fit into the cache (setting `text_index_header_cache_size`), reducing disk reads for text search queries. [#109332](https://github.com/ClickHouse/ClickHouse/pull/109332) ([Anton Popov](https://github.com/CurtizJ)). * The transposed vector distance functions `L2DistanceTransposed`, `cosineDistanceTransposed` and `dotProductTransposed` now apply the partial bit-plane read optimization to `Nullable(QBit)` columns, reading only the requested bit planes instead of the whole column. [#109358](https://github.com/ClickHouse/ClickHouse/pull/109358) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `IS NOT DISTINCT FROM` and `IS TRUE` now use primary-key and minmax indexes to prune granules, the same as `=`. Previously `k IS NOT DISTINCT FROM 42` and `(k = 42) IS TRUE` scanned all granules. [#110006](https://github.com/ClickHouse/ClickHouse/pull/110006) ([Groene AI](https://github.com/groeneai)). * Pooled connections are no longer pinged before each use. This removes a `Ping`-`Pong` round trip that was added to every reused connection, reducing the latency of distributed queries and of `clickhouse-benchmark`. A stale pooled connection is detected with a zero-timeout poll (a non-blocking check that adds no round trip) and recovered by reconnecting. [#110068](https://github.com/ClickHouse/ClickHouse/pull/110068) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed spurious reconnections of pooled HTTP keep-alive connections over TLS (HTTPS, S3). The stale-connection check used `poll` on the socket, which misreports a live secure connection carrying an unread TLS post-handshake record (a session ticket or `KeyUpdate`) as closed; it now uses a non-blocking `MSG_PEEK` on plain sockets and `SSL_peek`/`SSL_has_pending` on TLS sockets, which also correctly detects an orderly TLS shutdown and a TLS record that has only partially arrived. [#110402](https://github.com/ClickHouse/ClickHouse/pull/110402) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Queries accelerated by TopK dynamic filtering (`ORDER BY ... LIMIT k`) now make fuller use of the query condition cache: the cache is populated even when lazy materialization is applied to the query, and such queries can reuse entries previously written by an ordinary query that had the same `WHERE` predicate. [#110507](https://github.com/ClickHouse/ClickHouse/pull/110507) ([Shankar Iyer](https://github.com/shankar-iyer)). * A `LIKE`/`NOT LIKE` pattern without wildcards (`%`, `_`) now uses an exact primary key range, so it reads the same number of granules as the equivalent `=`/`!=` predicate instead of a wider prefix range. [#107077](https://github.com/ClickHouse/ClickHouse/pull/107077) ([Groene AI](https://github.com/groeneai)). * Fix redundant file segment lock contention (a performance regression introduced in 26.1) when reading data through the filesystem cache. [#109577](https://github.com/ClickHouse/ClickHouse/pull/109577) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fixed skip indexes defined on `Tuple` subcolumns not being used when the field is accessed via `tupleElement(t, 'name')`, `tupleElement(t, N)`, or `t.N` while the full tuple is also read in the same query (e.g. `SELECT *`). All of these forms now prune granules the same way as the named subcolumn access `t.name`. [#110056](https://github.com/ClickHouse/ClickHouse/pull/110056) ([Groene AI](https://github.com/groeneai)). * Fixed slow `ORDER BY ... LIMIT` queries on `Distributed` tables when `prefer_localhost_replica` selects a local replica: the preliminary limit was removed from the local-shard plan, preventing lazy materialization and dynamic top-K filtering. [#110136](https://github.com/ClickHouse/ClickHouse/pull/110136) ([Michael Jarrett](https://github.com/EmeraldShift)). * Fixed the text index not being used for the `ILIKE` operator when the index is defined over an expression rather than a bare column (for example `assumeNotNull(col)` with a `lower(...)` preprocessor): such queries were reading all granules instead of using the index. This closes [#110350](https://github.com/ClickHouse/ClickHouse/issues/110350). [#110595](https://github.com/ClickHouse/ClickHouse/pull/110595) ([Elmi Ahmadov](https://github.com/ahmadov)). * Reduce lock contention in the filesystem cache: queries reading from the cache no longer block on the priority queue lock when updating an entry's LRU priority. [#109065](https://github.com/ClickHouse/ClickHouse/pull/109065) ([Antonio Andelic](https://github.com/antonio2368)). * Text index queries that use a filter-only `postprocessor` expression (for example `if(..., '', token)`) now discard empty tokens up front instead of materializing them row by row, improving performance. [#109049](https://github.com/ClickHouse/ClickHouse/pull/109049) ([Elmi Ahmadov](https://github.com/ahmadov)). * Reduce lock contention in system log queues, lowering query tail latency on busy clusters. [#110539](https://github.com/ClickHouse/ClickHouse/pull/110539) ([Sean Haynes](https://github.com/seandhaynes)). * Improved the performance of substring search functions (`position`, `countSubstrings`, `multiSearch*`, `replaceAll`, `replaceOne`, and their case-insensitive and UTF-8 variants) with a non-constant needle by up to 1.5x by using SIMD-backed string searchers on the per-row path instead of a naive implementation. [#110580](https://github.com/ClickHouse/ClickHouse/pull/110580) ([Raúl Marín](https://github.com/Algunenano)). * Lazy materialization is now applied to `ReplacingMergeTree` queries with `FINAL`, a filter, and a small `LIMIT` even without `ORDER BY`. Columns not needed by the filter and the `FINAL` merge are fetched only for the rows that remain after the limit, which makes such queries much faster when they select wide columns with a selective filter. Controlled by `query_plan_optimize_lazy_materialization` and `query_plan_max_limit_for_lazy_materialization`. [#110722](https://github.com/ClickHouse/ClickHouse/pull/110722) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Improved async insert performance when deduplication is enabled and one flush is split across multiple partitions: ClickHouse now reuses the computed deduplication hashes across partitions instead of recomputing them for each one. [#111150](https://github.com/ClickHouse/ClickHouse/pull/111150) ([Valery Petrov](https://github.com/valerypetrov)). #### Improvement * Added a setting `input_format_csv_missing_nullable_as_empty_string` (disabled by default). When enabled, a missing value of a `Nullable(String)` column in CSV input is read as an empty `String` instead of `NULL`, regardless of `input_format_csv_empty_as_default`. [#58225](https://github.com/ClickHouse/ClickHouse/pull/58225) ([kevinyhzou](https://github.com/KevinyhZou)). [#107577](https://github.com/ClickHouse/ClickHouse/pull/107577) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Functions `toDateOrNull`, `toDateTimeOrNull` and `toDateTime64OrNull` now accept integer arguments of all native integer types (interpreted the same way as by `toDate`, `toDateTime` and `toDateTime64`, with an optional timezone argument), returning `NULL` for values out of range of the result type. For example, `toDateTimeOrNull(1583851242, 'Asia/Shanghai')` returns `2020-03-10 22:40:42` and `toDateTimeOrNull(4294967296)` returns `NULL`. [#79791](https://github.com/ClickHouse/ClickHouse/pull/79791) ([Jitendra](https://github.com/jitendra1411)). * Added an `is_wildcard` column to the `system.grants` table that indicates whether a grant uses wildcard prefix matching (e.g., `GRANT SELECT ON db*.*`). Previously, wildcard and exact grants on the same name were indistinguishable in `system.grants`. Closes [#92835](https://github.com/ClickHouse/ClickHouse/issues/92835). [#98577](https://github.com/ClickHouse/ClickHouse/pull/98577) ([DQ](https://github.com/il9ue)). * Allow Snappy compression in the HTTP interface (`Accept-Encoding: snappy`) and add the `snappy_mode` setting to choose between the Hadoop Snappy block format and the Snappy framing format for generic `file`/`url` snappy I/O. [#100752](https://github.com/ClickHouse/ClickHouse/pull/100752) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support compatibility with the old analyzer under the `analyzer_compatibility_allow_non_aggregate_in_having` setting. If enabled, non-aggregate conjunctions are moved from `HAVING` to `WHERE`. [#104232](https://github.com/ClickHouse/ClickHouse/pull/104232) ([Dmitry Novik](https://github.com/novikd)). * Added a new server setting `memory_worker_rss_speculative_reserve_ratio` (default `1.0`) which makes the global memory tracker speculatively reserve memory on top of the observed RSS when the RSS growth outpaces the tracker's bookkeeping between samples. With it, allocations get `MEMORY_LIMIT_EXCEEDED` earlier, and the kernel OOM-killer is less likely to fire first. Set the ratio to `0` to disable the speculation. [#104976](https://github.com/ClickHouse/ClickHouse/pull/104976) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added opt-in Prometheus metrics for filesystem cache eviction activity (`filesystem_cache_evictions_total`, `filesystem_cache_evicted_bytes_total`, `filesystem_cache_evicted_segment_hits`, `filesystem_cache_evicted_segment_size_bytes`, and their per-user variants labeled with `user_id`), exposed per cache via `system.dimensional_metrics`, `system.histogram_metrics`, and the Prometheus endpoint. They are controlled by the cache disk-config settings `expose_prometheus_eviction_metrics` and `expose_prometheus_eviction_metrics_per_user` (both off by default), which can be toggled at runtime via `SYSTEM RELOAD CONFIG`. [#105020](https://github.com/ClickHouse/ClickHouse/pull/105020) ([Sacheendra Talluri](https://github.com/sacheendra)). * Make `EXPLAIN [PLAN] actions=1, compact=1, pretty=1` the default. [#105036](https://github.com/ClickHouse/ClickHouse/pull/105036) ([Kirill Kopnev](https://github.com/Fgrtue)). * Bump `delta-kernel-rs` (the library behind the `DeltaLake` integration) to v0.23.0. [#105861](https://github.com/ClickHouse/ClickHouse/pull/105861) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Add `du` and `wc` commands to `clickhouse-disks`. They, respectively, print the total size in bytes for a given file or directory, and list the number of bytes, lines, and words in a file. [#106268](https://github.com/ClickHouse/ClickHouse/pull/106268) ([Asya Shneerson](https://github.com/asya-ch)). * Added a new asynchronous metric `UntrackedMemory` (visible in `system.asynchronous_metrics`) that reports memory already allocated by threads but not yet accounted in the global memory tracking counter: each thread accumulates small allocations locally and reports them in bulk. This helps explain discrepancies between `MemoryTracking` and the process's actual memory usage. `MEMORY_LIMIT_EXCEEDED` error messages now also include the amount of untracked memory, making it easier to understand why a query or the server hit its memory limit. [#106386](https://github.com/ClickHouse/ClickHouse/pull/106386) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Upgrade ClickStack (the observability UI) to version 2.28.0. [#106406](https://github.com/ClickHouse/ClickHouse/pull/106406) ([Aaron Knudtson](https://github.com/knudtty)). * Added support for `BFloat16` in `dotProduct` and improved its performance by batching the SIMD path. [#106569](https://github.com/ClickHouse/ClickHouse/pull/106569) ([Nikita Taranov](https://github.com/nickitat)). * Made the settings `max_named_collection_num_to_throw`, `max_table_num_to_throw`, `max_replicated_table_num_to_throw`, `max_view_num_to_throw`, `max_dictionary_num_to_throw`, and `max_database_num_to_throw` changeable without a server restart. [#106821](https://github.com/ClickHouse/ClickHouse/pull/106821) ([Maxim Orlovsky](https://github.com/UberDever)). * Support the `REDUCED_REDUNDANCY`, `STANDARD_IA`, `ONEZONE_IA`, `GLACIER_IR`, and `EXPRESS_ONEZONE` values (in addition to `STANDARD` and `INTELLIGENT_TIERING`) for the `s3_storage_class_name` setting. [#107251](https://github.com/ClickHouse/ClickHouse/pull/107251) ([Aditya Kumar](https://github.com/adityaksolves)). * Added the create-time `materialized_postgresql_use_extended_date_and_time_types` setting for the `MaterializedPostgreSQL` database engine. By default (enabled), PostgreSQL `date`/`timestamp` columns are inferred as `Date32`/`DateTime64`; setting it to `0` at `CREATE DATABASE` time infers the narrower `Date`/`DateTime` types. The setting is not applicable to the `MaterializedPostgreSQL` table engine. [#107428](https://github.com/ClickHouse/ClickHouse/pull/107428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `SOME` / `ALL` array quantifier (`expr OP SOME(array)` / `expr OP ALL(array)`) now also supports the keyword comparison predicates `IS DISTINCT FROM` and `IS NOT DISTINCT FROM`, and the string-search predicates `LIKE`, `ILIKE`, `NOT LIKE`, `NOT ILIKE`, and `REGEXP`, rewritten to `arrayExists` / `arrayAll`. [#107454](https://github.com/ClickHouse/ClickHouse/pull/107454) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a memory leak that occurred when opening a SQLite database failed (for example, when the `sqlite` table function or the `SQLite` database engine is given a path that cannot be opened). [#107807](https://github.com/ClickHouse/ClickHouse/pull/107807) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI (`play.html`), the run shortcut hint now shows `Cmd+Enter` on Mac and `Ctrl+Enter` on other platforms instead of always showing `Ctrl/Cmd+Enter`, and the `, +Shift to run all` part of the hint is shown only when there are multiple queries. [#107817](https://github.com/ClickHouse/ClickHouse/pull/107817) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#108957](https://github.com/ClickHouse/ClickHouse/pull/108957) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `Alias` table engine now supports reading with parallel replicas when the target table is of the `MergeTree` family. [#107830](https://github.com/ClickHouse/ClickHouse/pull/107830) ([Kai Zhu](https://github.com/nauu)). * Added the `uniq_v2` column statistics type — a lightweight alternative to the `uniq` statistics, based on the `uniqCombined64` sketch. [#107863](https://github.com/ClickHouse/ClickHouse/pull/107863) ([Han Fei](https://github.com/hanfei1991)). Deprecated the `minmax` column statistics type and changed the default value of `auto_statistics_types` to `basic, uniq_v2`. [#108680](https://github.com/ClickHouse/ClickHouse/pull/108680) ([Han Fei](https://github.com/hanfei1991)). * AI functions (`aiGenerate`, `aiEmbed`, `aiClassify`, `aiExtract`, `aiTranslate`) now retry transient network failures (connection resets, TLS connect failures, timeouts, unreachable addresses) when `ai_function_max_retries` is set, matching the retry behavior of the `url` table function. Previously, only provider-side HTTP error responses were retried. [#107927](https://github.com/ClickHouse/ClickHouse/pull/107927) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `INSERT` into a `MergeTree` table now honors query cancellation and `max_execution_time` while writing many parts, instead of potentially running long after being killed. [#107929](https://github.com/ClickHouse/ClickHouse/pull/107929) ([Michael Kolupaev](https://github.com/al13n321)). * In the Web UI (`play.html`), the Web Terminal icon can now be opened in a new browser tab with a middle-button click or Ctrl/Cmd/Shift+click. [#108006](https://github.com/ClickHouse/ClickHouse/pull/108006) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * AI SQL generation in the client no longer sends the `temperature` parameter unless it is explicitly set via `ai.temperature` in the configuration. This fixes AI SQL generation for models that reject the `temperature` parameter. [#108014](https://github.com/ClickHouse/ClickHouse/pull/108014) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `system.masking_policies` table and `SHOW MASKING POLICIES` introspection to open-source builds. Masking policies themselves remain a ClickHouse Cloud feature, so the table is empty in open-source builds, but introspection queries no longer fail with an error. [#108030](https://github.com/ClickHouse/ClickHouse/pull/108030) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The Keeper-based Kafka consumer now randomizes the order in which it acquires temporary partition locks across replicas, improving fairness of partition distribution and reducing lock contention. Previously, the intended shuffle had no effect. [#108033](https://github.com/ClickHouse/ClickHouse/pull/108033) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added autocompletion to the Web UI (`play.html`), based on `system.completions` and the WASM-based SQL lexer. [#108059](https://github.com/ClickHouse/ClickHouse/pull/108059) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-client` and `clickhouse-local` now show as-you-type autocompletion hints (inline "ghost" text) for the best matching suggestion when the cursor is at the end of the input. The most relevant suggestions (recently used and identifiers already present in the query) are ranked first. Navigate with Up/Down (or Ctrl-Up/Ctrl-Down); accept a single or selected hint with Tab or Right, and accept a selected hint with Enter; Tab also opens the classic completion list. Controlled by the new `--hints` option (on by default; requires `--highlight`). [#108070](https://github.com/ClickHouse/ClickHouse/pull/108070) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support the `length` function for the `QBit` data type — it returns the dimension of the vector as a constant. [#108071](https://github.com/ClickHouse/ClickHouse/pull/108071) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Support `CAST` from a `QBit` to an `Array`, reconstructing the original vector — the inverse of the existing `Array` to `QBit` conversion. [#108072](https://github.com/ClickHouse/ClickHouse/pull/108072) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Requests to REST data lake catalogs (such as OneLake) now include a ClickHouse `User-Agent` header. [#108117](https://github.com/ClickHouse/ClickHouse/pull/108117) ([Konstantin Vedernikov](https://github.com/scanhex12)). * `arrayFold` now respects query cancellation and `max_execution_time`. Previously, a fold over a very long array ran entirely inside a single function call and could not be interrupted, so `KILL QUERY` and time limits were ignored until the fold finished. [#108192](https://github.com/ClickHouse/ClickHouse/pull/108192) ([Groene AI](https://github.com/groeneai)). * `MVTEncodeGeom` now snaps geometry to the integer pixel grid before clipping and clips polygons with the `wagyu` library, so the clipped output is valid (self-intersecting rings are repaired) and edge-aligned, matching PostGIS `ST_AsMVTGeom`. [#108248](https://github.com/ClickHouse/ClickHouse/pull/108248) ([Saarthak Gupta](https://github.com/saarthak2002)). * Add per-phase query pre-execution `ProfileEvents`: `QueryParseMicroseconds`, `QueryAnalysisMicroseconds`, `QueryPlanBuildMicroseconds` and `QueryPipelineBuildMicroseconds`. They expose where time is spent before query execution (parsing, analysis, query plan building, pipeline building) and are available in `system.query_log` and `system.events`. [#108282](https://github.com/ClickHouse/ClickHouse/pull/108282) ([Jordi Villar](https://github.com/jrdi)). * `ALTER TABLE` operations that would produce table metadata exceeding `max_query_size` are now rejected upfront, preventing tables from becoming unloadable by components such as DDL distribution and replica recovery. [#108283](https://github.com/ClickHouse/ClickHouse/pull/108283) ([Andrew Kravchuk](https://github.com/lockie)). * Added `settings` and `engine_settings` columns to `system.backups` and `system.backup_log`. `settings` exposes the backup/restore-specific settings requested for an operation (e.g., `allow_s3_native_copy`, `deduplicate_files`, `structure_only`), and `engine_settings` exposes the settings effectively used by the backup engine's reader and writer (e.g., the S3 request settings such as `allow_native_copy`, which may differ from what was requested after merging the endpoint configuration). This makes it possible to see which settings a `BACKUP`/`RESTORE` operation actually ran with. [#108334](https://github.com/ClickHouse/ClickHouse/pull/108334) ([Julia Kartseva](https://github.com/jkartseva)). * Added a `source` column to `system.documentation` containing the path of the source file where each entity's documentation is defined. The table now also documents compression codecs, profile events, current metrics, asynchronous metrics, and the system tables themselves (with their columns), and the documentation of settings now includes their type and default value. [#108346](https://github.com/ClickHouse/ClickHouse/pull/108346) ([Alexey Milovidov](https://github.com/alexey-milovidov)). An empty setting default value is now rendered as *empty string* instead of empty backticks. [#108708](https://github.com/ClickHouse/ClickHouse/pull/108708) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support asynchronous reads from remote replicas (`async_socket_for_remote`) on macOS by implementing the epoll-based polling primitives on top of `kqueue`. This lets distributed queries read shards in parallel on macOS instead of serially. [#108403](https://github.com/ClickHouse/ClickHouse/pull/108403) ([Raúl Marín](https://github.com/Algunenano)). * Support `BFloat16` in binary math functions. [#108442](https://github.com/ClickHouse/ClickHouse/pull/108442) ([Zhang Yifan](https://github.com/zhangyifan27)). * `MergeTree` can now read a compressed stream whose blocks use different codecs. This is the read-side prerequisite for adaptive codec selection ([#105404](https://github.com/ClickHouse/ClickHouse/issues/105404)). [#108592](https://github.com/ClickHouse/ClickHouse/pull/108592) ([Raufs Dunamalijevs](https://github.com/rienath)). * The automatic value of `max_threads` and similar settings is now shown in `system.settings` as `auto(8)` instead of `'auto(8)'`; the surrounding single quotes were a long-standing artifact baked into the value. Cross-version compatibility is preserved: the legacy quoted form is still accepted when parsing settings received from older servers. [#108657](https://github.com/ClickHouse/ClickHouse/pull/108657) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Several usability fixes for the `/schema` Web UI: load the schema on Enter, theme-aware scrollbars, readable table names, consistent per-database grouping of independent tables, no text selection while dragging, and a `play`-style authentication form (Credential Management API, URL-supplied credentials with password stripping, and live credential validation). [#108724](https://github.com/ClickHouse/ClickHouse/pull/108724) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The text index lazy posting-list apply mode is no longer experimental and can be selected with `text_index_posting_list_apply_mode = 'lazy'` without `allow_experimental_text_index_lazy_apply`. The density-threshold setting was renamed from `text_index_density_threshold` to `text_index_lazy_intersection_density_threshold`. [#108814](https://github.com/ClickHouse/ClickHouse/pull/108814) ([Anton Popov](https://github.com/CurtizJ)). * Add a `delta_sharing` catalog type to `DataLakeCatalog` for Databricks Delta Sharing's Iceberg REST endpoint, whose namespaces are flat and which ignores the `parent` listing filter. Use it instead of `catalog_type = 'rest'` for such endpoints, where `SHOW TABLES` would otherwise hang. [#108865](https://github.com/ClickHouse/ClickHouse/pull/108865) ([Seva Potapov](https://github.com/seva-potapov)). * Added a new `clickhouse-client`/`clickhouse-local` option `--echo-query-separator` that prints a custom separator before the formatted echoed query, making it easier to tell the typed query apart from its reformatted echo. Disabled by default. [#108888](https://github.com/ClickHouse/ClickHouse/pull/108888) ([David Meng](https://github.com/davidmenggx)). * Improvements to the database panel of the Web UI: the number of tables is shown in parentheses next to a database name when it is expanded. [#108891](https://github.com/ClickHouse/ClickHouse/pull/108891) ([Alexey Milovidov](https://github.com/alexey-milovidov)). A refresh button next to an expanded database reloads its list of tables. [#108958](https://github.com/ClickHouse/ClickHouse/pull/108958) ([Alexey Milovidov](https://github.com/alexey-milovidov)). When the panel is reopened, the previously expanded databases are restored. [#108964](https://github.com/ClickHouse/ClickHouse/pull/108964) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow calculating the combined skip-index benefit in `EXPLAIN WHATIF`: it shows the data ratio for the intersection of all existing suitable hypothetical indices. [#108934](https://github.com/ClickHouse/ClickHouse/pull/108934) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed the horizontal scroll position of the result table being reset in the Web UI when clicking a link in a cell and returning to the page. [#108941](https://github.com/ClickHouse/ClickHouse/pull/108941) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `MySQL` database engine, table engine, and table function now map MySQL's spatial column types (`LINESTRING`, `POLYGON`, `MULTILINESTRING`, `MULTIPOLYGON`, and the generic `GEOMETRY`) to the corresponding ClickHouse geometric types instead of `String`. This is controlled by the new `geometry` flag of the `mysql_datatypes_support_level` setting, enabled by default. `POINT` is still always converted to `Point`. The generic `GEOMETRY` column maps to the umbrella `Geometry` type; reading a value whose subtype has no ClickHouse counterpart (`MULTIPOINT`, `GEOMETRYCOLLECTION`) throws an exception at read time. [#108944](https://github.com/ClickHouse/ClickHouse/pull/108944) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI (`play.html`): full keyboard navigation — arrow-key navigation in the database panel and result table, keyboard access to the toolbar buttons, theme switcher, and download menu, and visible focus outlines. [#108972](https://github.com/ClickHouse/ClickHouse/pull/108972) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * HTTP interface: when credentials are provided both via URL parameters (`user`/`password`) and an `Authorization` header, the URL parameters now take precedence instead of the request being rejected. This fixes downloading results from the Web UI (`play.html`) failing with `AUTHENTICATION_FAILED` (error 516) when the browser has remembered Basic credentials. [#108980](https://github.com/ClickHouse/ClickHouse/pull/108980) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Restore `show_data_lake_catalogs_in_system_tables` as the setting that controls only `DataLakeCatalog` visibility in `system.tables`, `system.columns`, and `system.completions`. Added `show_remote_databases_in_system_tables`, enabled by default, to let users hide `MySQL` and `PostgreSQL` databases from those system tables separately. [#109082](https://github.com/ClickHouse/ClickHouse/pull/109082) ([Pablo Marcos](https://github.com/pamarcos)). * Use the precise (closest-representable) float parsing algorithm by default and apply the `precise_float_parsing` setting to input formats (`CSV`, `TSV`, `JSON`, `VALUES`, ...) and numeric literals, not just `toFloat*`/`CAST`. Set `precise_float_parsing = 0` for the previous, faster in some cases but less accurate, behavior. Closes [#60146](https://github.com/ClickHouse/ClickHouse/issues/60146). Closes [#74647](https://github.com/ClickHouse/ClickHouse/issues/74647). Closes [#68914](https://github.com/ClickHouse/ClickHouse/issues/68914). [#109086](https://github.com/ClickHouse/ClickHouse/pull/109086) ([Raúl Marín](https://github.com/Algunenano)). * A query with a single CTE is now formatted with the same newline and indentation as a query with multiple CTEs. Previously `WITH a AS (...)` kept the CTE on the same line as `WITH`, while two or more CTEs put `WITH` on its own line with each CTE indented. [#109092](https://github.com/ClickHouse/ClickHouse/pull/109092) ([Groene AI](https://github.com/groeneai)). * Web UI: when the highlighted completion is the same as the already-typed word, or the same word in a different case (e.g. `HASH` offered for a typed `hash`), pressing Right/Tab/Enter no longer rewrites the word or gets swallowed by the autocompletion — the key moves the caret or inserts a newline as expected. [#109106](https://github.com/ClickHouse/ClickHouse/pull/109106) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#109384](https://github.com/ClickHouse/ClickHouse/pull/109384) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added rainbow parentheses, matched-bracket, matching-identifier, and digit-group highlighting to the Web UI (`play.html`), matching `clickhouse-client`. [#109108](https://github.com/ClickHouse/ClickHouse/pull/109108) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, `Tab` and `Shift+Tab` now indent and unindent the selected lines by 4 spaces. [#109110](https://github.com/ClickHouse/ClickHouse/pull/109110) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: highlight the position of a syntax error in the query editor with a red background. The highlight is cleared as soon as the user returns focus to the editor. [#109112](https://github.com/ClickHouse/ClickHouse/pull/109112) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-compressor --stat` and default-codec detection for old parts now report corrupted block headers as corruption instead of a misleading end-of-file error. [#109157](https://github.com/ClickHouse/ClickHouse/pull/109157) ([Raufs Dunamalijevs](https://github.com/rienath)). * ClickHouse Keeper now respects the `os_collect_psi_metrics` setting and skips PSI metrics collection when it is disabled. [#109179](https://github.com/ClickHouse/ClickHouse/pull/109179) ([Maxim Orlovsky](https://github.com/UberDever)). * In the Web UI, when query tabs are shown, the connection parameters (host, user, password) are hidden behind a key button in the top-right corner and shown in a drop-down on demand. [#109243](https://github.com/ClickHouse/ClickHouse/pull/109243) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: show a single-value result (one row, one column, e.g. `SHOW CREATE TABLE`) with full height instead of clamping it to three lines. [#109245](https://github.com/ClickHouse/ClickHouse/pull/109245) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, clicking a table in the databases panel with the middle mouse button, with Shift, or with the platform's new-tab modifier (Cmd on macOS, Ctrl elsewhere) now opens its query in a new Web UI tab. [#109246](https://github.com/ClickHouse/ClickHouse/pull/109246) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Key the query condition cache for remote (object storage) `Parquet` files by ETag in addition to the path, so that overwriting an object in place no longer serves stale cached results. [#109310](https://github.com/ClickHouse/ClickHouse/pull/109310) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, empty strings and NULLs are no longer colorized in the categorical coloring mode. [#109342](https://github.com/ClickHouse/ClickHouse/pull/109342) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: clicking a table's icon in the database panel now shows the list of its columns, with type icons, a size bar proportional to the compressed size, and a tooltip with the compressed/uncompressed size and compression ratio. [#109346](https://github.com/ClickHouse/ClickHouse/pull/109346) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: show the full tab title as a tooltip on hover when it is truncated. [#109354](https://github.com/ClickHouse/ClickHouse/pull/109354) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, middle-clicking a tab's title (or Ctrl/Shift/Cmd+clicking it) now duplicates the tab. [#109357](https://github.com/ClickHouse/ClickHouse/pull/109357) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: resize the final progress and query statistics areas uniformly so they do not wrap awkwardly when the browser window is narrow. [#109363](https://github.com/ClickHouse/ClickHouse/pull/109363) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: selecting a database in the database panel now uses it as the default database for queries run from the editor. [#109372](https://github.com/ClickHouse/ClickHouse/pull/109372) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support `reinterpret` of an `Array` of fixed-size elements as a `String`, the inverse of the existing `reinterpret` of a `String`/`FixedString` as an `Array`. [#109383](https://github.com/ClickHouse/ClickHouse/pull/109383) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI, do not display the per-column color-coding toggles when the result has no more than a single row. [#109385](https://github.com/ClickHouse/ClickHouse/pull/109385) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow `CAST` between `QBit` types that differ in the element type and/or the stride, as long as the dimension stays the same (for example `CAST(x AS QBit(Float64, N))` from a `QBit(Float32, N)`). Stride-only changes are lossless; element-type changes follow the corresponding `Array` conversion semantics (for example `Float32` to `BFloat16` may lose precision, and `accurateCast` / `accurateCastOrNull` reject rows that are not exactly representable). [#109387](https://github.com/ClickHouse/ClickHouse/pull/109387) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Functions `quantizeBFloat16ToInt8` and `dequantizeInt8ToBFloat16` now also accept `Array` and `QBit` arguments, applying the Lloyd-Max codec to the whole vector (returning `Array`/`QBit` of the corresponding element type), in addition to the existing scalar overloads. [#109398](https://github.com/ClickHouse/ClickHouse/pull/109398) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: the Documentation link now carries the current user name in the URL. [#109419](https://github.com/ClickHouse/ClickHouse/pull/109419) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support `arraySum`, `arrayAvg`, and `arrayProduct` for arrays of `BFloat16`. [#109420](https://github.com/ClickHouse/ClickHouse/pull/109420) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The MySQL-style format specifier `%f` in `parseDateTime` / `parseDateTime64` (and their `OrZero` / `OrNull` variants) now accepts between 1 and 6 fractional digits, interpreted as left-aligned microseconds like MySQL's `STR_TO_DATE`, instead of requiring exactly 6. Also fixed misaligned `PrettyCompact` tables in the built-in function documentation examples. [#109421](https://github.com/ClickHouse/ClickHouse/pull/109421) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: query tabs are now persistent — each tab keeps its rendered result (including images and charts) and its running query when you switch away and back, and long queries continue running in the background. Tab titles show a spinner, progress bar, and completion check-mark for their query. [#109425](https://github.com/ClickHouse/ClickHouse/pull/109425) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: correctly display an error when a query fails after some of its result has already been streamed, instead of showing a client-side `SyntaxError: Unexpected end of JSON input` (or, with `http_write_exception_in_output_format`, marking the failed query as successful). [#109430](https://github.com/ClickHouse/ClickHouse/pull/109430) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added pinned columns to the Web UI: a column can be pinned from its header so that it stays visible (stuck to the edge) while the results table is scrolled horizontally. [#109439](https://github.com/ClickHouse/ClickHouse/pull/109439) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the advanced dashboard (`/dashboard`), charts for specific metrics can now be added directly by the metric name as it appears in the source code or documentation, which makes it easier to add multiple metrics during debugging sessions. [#109449](https://github.com/ClickHouse/ClickHouse/pull/109449) ([Mikhail Artemenko](https://github.com/Michicosun)). * Web UI: fixed the connection settings drop-down closing when selecting text with the mouse and dragging past its border. [#109451](https://github.com/ClickHouse/ClickHouse/pull/109451) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Web UI: hovering over a result table column header now shows the full column name and type as a tooltip, so truncated headers can be read in full. [#109463](https://github.com/ClickHouse/ClickHouse/pull/109463) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Fixed column header titles being clipped with an ellipsis even when the column was wide enough to show the full title. [#110397](https://github.com/ClickHouse/ClickHouse/pull/110397) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support SSD cache dictionaries (`SSD_CACHE` layout) and the `FileLog` table engine on macOS builds. [#109493](https://github.com/ClickHouse/ClickHouse/pull/109493) ([Raúl Marín](https://github.com/Algunenano)). * Web UI: do not restore tabs with empty queries when the page is opened. [#109529](https://github.com/ClickHouse/ClickHouse/pull/109529) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix unreadable as-you-type autocompletion hints and bright colors in the interactive client on terminals whose `TERM` does not contain `256` (e.g. Ghostty, kitty, Alacritty, foot): bright colors are now emitted as aixterm bright color codes unconditionally instead of falling back to bold + dark color, which modern terminals render as a dark, hard-to-read color on dark backgrounds. [#109622](https://github.com/ClickHouse/ClickHouse/pull/109622) ([Alasdair Brown](https://github.com/sdairs)). * Enable jemalloc per-CPU arenas and allocation profiling on macOS builds. [#109684](https://github.com/ClickHouse/ClickHouse/pull/109684) ([Raúl Marín](https://github.com/Algunenano)). * Allow altering some authentication settings of a `OneLake` data lake catalog database with `ALTER DATABASE ... MODIFY SETTING`. [#110019](https://github.com/ClickHouse/ClickHouse/pull/110019) ([alesapin](https://github.com/alesapin)). * The `Hive` engine now reads `ORC` file metadata (min/max indexes, row counts) with ClickHouse's native `ORC` reader instead of the Apache Arrow `ORC` adapter. [#110086](https://github.com/ClickHouse/ClickHouse/pull/110086) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduced the binary size by ~10.5 MB by executing comparison and arithmetic operations on rarely used mixed type pairs (`Decimal` vs integer of a different width, and pairs involving `Int128`/`UInt128`/`Int256`/`UInt256`) via a conversion to a common type instead of a dedicated compiled kernel for every combination of types. Same-type pairs, commonly used pairs, and the memory-bound `plus`/`minus`/`multiply` keep their dedicated kernels; results, result types and exceptional cases are unchanged. [#110131](https://github.com/ClickHouse/ClickHouse/pull/110131) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A subquery on the right side of `IN` whose single column is an array one dimension deeper than the left argument is now interpreted as the set of the array's elements (like an array literal or an array-returning function), instead of failing with a confusing type-mismatch error. For example, `x IN (SELECT groupArray(x) FROM ...)` now works. [#110169](https://github.com/ClickHouse/ClickHouse/pull/110169) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading from a `SQLite` table engine or `sqlite` table function no longer busy-spins a full CPU core when the SQLite database is locked by another connection. The read now idles while waiting for the lock and stays cancellable. [#110248](https://github.com/ClickHouse/ClickHouse/pull/110248) ([Groene AI](https://github.com/groeneai)). * Round elapsed time, rate, and ratio values in log and exception messages to three digits, so numbers like `1.345844286 sec.` are no longer printed at full precision. [#110277](https://github.com/ClickHouse/ClickHouse/pull/110277) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI (Play), `BFloat16` is now recognized as a floating point number and its columns are right-aligned and colorized accordingly. [#110433](https://github.com/ClickHouse/ClickHouse/pull/110433) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support the `SETTINGS` clause for the `PostgreSQL` table engine and the `postgresql` table function (for example `SETTINGS postgresql_connection_pool_size = 50`), bringing feature parity with the `MySQL` engine. [#110614](https://github.com/ClickHouse/ClickHouse/pull/110614) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `SHOW CREATE TABLE` (and `SHOW CREATE VIEW` / `SHOW CREATE DICTIONARY`) now suggests a similarly-named table in the error message when the requested table does not exist, the same way `SELECT` queries do. [#110633](https://github.com/ClickHouse/ClickHouse/pull/110633) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enable the `merge_selector_enable_heuristic_to_lower_max_parts_to_merge_at_once` setting by default: the merge selector now automatically lowers the maximum number of parts to merge at once based on how full the partition is. See [#91163](https://github.com/ClickHouse/ClickHouse/pull/91163). [#110726](https://github.com/ClickHouse/ClickHouse/pull/110726) ([Mikhail Artemenko](https://github.com/Michicosun)). * Accept PostgreSQL cleanup commands `RESET`, `UNLISTEN`, and `DISCARD` as no-ops in the PostgreSQL wire protocol instead of failing them with a syntax error. This improves compatibility with drivers such as Skunk that send `RESET ALL` and `UNLISTEN *` during connection setup and cleanup. [#110780](https://github.com/ClickHouse/ClickHouse/pull/110780) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Change the default value of the `auto_statistics_types` setting from `basic, uniq` to `basic, uniq_v2`. [#110878](https://github.com/ClickHouse/ClickHouse/pull/110878) ([Han Fei](https://github.com/hanfei1991)). * `randomHadamardTransform` now computes an exact, length-preserving transform for any vector length whose largest odd factor is at most 64 (for example `3584 = 512 * 7`, common in embedding models), extending the previous `2^N`, `2^k * {12, 20}`, and `2^k * 9` families. A full transform of a length that cannot be represented exactly now raises an exception instead of silently zero-padding to a longer vector; pass `output_dims` to compute a truncated projection of an arbitrary length. [#111006](https://github.com/ClickHouse/ClickHouse/pull/111006) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` exception during backup of a `Replicated` database that is being dropped and recreated concurrently; such backups now fail cleanly with `CANNOT_GET_REPLICATED_DATABASE_SNAPSHOT`. [#100651](https://github.com/ClickHouse/ClickHouse/pull/100651) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Surface the real underlying error when a zip archive cannot be unpacked (for example, when the archive is read from S3 and the read buffer refuses a seek). Previously, the actual error was hidden behind the generic `Couldn't unpack zip archive: Code = -100` / `Couldn't open zip archive` message. [#105103](https://github.com/ClickHouse/ClickHouse/pull/105103) ([Groene AI](https://github.com/groeneai)). * Fix a confusing "Maybe you meant X?" hint after a server restart (or `DETACH DATABASE`/`ATTACH DATABASE`): dropping an already-dropped table could suggest the just-dropped name as the alternative. [#106238](https://github.com/ClickHouse/ClickHouse/pull/106238) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (`Logical error: 'removed'`, aborting the server in debug builds) in the background table-drop queue, triggered when the same explicit UUID is reused across several `CREATE OR REPLACE TABLE` queries, which enqueues more than one dropped table sharing that UUID. [#107031](https://github.com/ClickHouse/ClickHouse/pull/107031) ([Groene AI](https://github.com/groeneai)). * Fixed an `Inconsistent AST formatting` logical error that could abort the server in debug and sanitizer builds when an aliased lambda was used as the operand of an access operator (tuple element `.N` or array element `[]`) at a non-first position of an expression list, e.g. `SELECT 1, ((p0, p1) -> p0 AS a7).4[3] FROM t`. [#107092](https://github.com/ClickHouse/ClickHouse/pull/107092) ([Groene AI](https://github.com/groeneai)). * A cluster table function (`urlCluster`, `fileCluster`, `s3Cluster`, ...) nested inside another distributed query, such as `clusterAllReplicas(..., urlCluster(...))`, is now rejected with a `BAD_ARGUMENTS` error instead of failing with a logical error (`Distributed task iterator is not initialized`). [#107107](https://github.com/ClickHouse/ClickHouse/pull/107107) ([Groene AI](https://github.com/groeneai)). * `OPTIMIZE ... DRY RUN` interrupted by a query timeout (`max_execution_time` with `timeout_overflow_mode = 'break'`) now returns `TIMEOUT_EXCEEDED` instead of a logical error about `rows_sources` (which aborted the server in debug/sanitizer builds). [#107114](https://github.com/ClickHouse/ClickHouse/pull/107114) ([Groene AI](https://github.com/groeneai)). * Stop logging a benign `Net Exception: Socket is not connected` error during `ZooKeeper` session finalize on macOS. `clickhouse keeper-client` no longer prints this spurious error to stderr at exit. [#107438](https://github.com/ClickHouse/ClickHouse/pull/107438) ([Groene AI](https://github.com/groeneai)). * Fixed a signed integer overflow when a refreshable materialized view retries a failed refresh with `refresh_retries` set to a very large value (near `Int64` max). The overflow could abort the server in builds with the undefined behavior sanitizer. [#108005](https://github.com/ClickHouse/ClickHouse/pull/108005) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (server abort in debug/sanitizer builds) when a `JOIN` uses the null-safe comparison operator (`<=>` / `IS NOT DISTINCT FROM`) and one of the keys is a scalar subquery, e.g. `... JOIN t2 ON (SELECT x FROM t1) <=> t2.k`, with the old analyzer (`enable_analyzer = 0`). Such a query now returns a regular `NOT_FOUND_COLUMN_IN_BLOCK` error instead of a logical error. [#108123](https://github.com/ClickHouse/ClickHouse/pull/108123) ([Groene AI](https://github.com/groeneai)). * Vector search queries that select the `_distance` column now return a proper error instead of failing with a logical error. [#108423](https://github.com/ClickHouse/ClickHouse/pull/108423) ([Robert Schulze](https://github.com/rschu1ze)). * Fix a `std::future_error` (`The associated promise has been destructed prior to the associated state becoming ready`) that could surface, and abort the server in debug and sanitizer builds, when scheduling the final asynchronous S3/Azure multipart-upload completion task failed (for example under thread-pool exhaustion). The real scheduling error is now reported instead. [#108730](https://github.com/ClickHouse/ClickHouse/pull/108730) ([Groene AI](https://github.com/groeneai)). * Avoid excessive server log output and an oversized error message when compiling a very large regular expression (for example a `LIKE` or `match` pattern with hundreds of thousands of wildcards); such patterns now fail with a clear `CANNOT_COMPILE_REGEXP` error. [#108821](https://github.com/ClickHouse/ClickHouse/pull/108821) ([Raúl Marín](https://github.com/Algunenano)). * Fixed undefined behaviour when stringifying an out-of-range protocol packet type (e.g. in the `Unexpected packet from server` / `Received ... packet` error messages) for a desynced or fuzzed connection. [#108885](https://github.com/ClickHouse/ClickHouse/pull/108885) ([Groene AI](https://github.com/groeneai)). * Fixed a confusing internal error (`Method getResultType is not supported for TABLE query tree node`) when a table expression was used as the left argument of the `IN` operator; such queries now produce a clear error message. [#109412](https://github.com/ClickHouse/ClickHouse/pull/109412) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed reading an Iceberg table with a metadata file whose version number is all digits but exceeds the 32-bit integer range (for example `v99999999999999999999.metadata.json`). Such a name previously produced an opaque `std::out_of_range` (`STD_EXCEPTION`) instead of a clear `BAD_ARGUMENTS` error. [#109619](https://github.com/ClickHouse/ClickHouse/pull/109619) ([Groene AI](https://github.com/groeneai)). * Fixed a possible `Logical error: 'ReadBuffer is canceled. Can't read from it.'` when a row-based input format (e.g. `TSV`/`CSV`) failed to parse input and the underlying read had already been canceled (for example a malformed HTTP chunk or a truncated body). Building the verbose parse diagnostics no longer reads from a canceled buffer. [#109708](https://github.com/ClickHouse/ClickHouse/pull/109708) ([Groene AI](https://github.com/groeneai)). * Row policy filter expressions using `arrayJoin` (or its `unnest` alias) are now rejected with a clear error; previously such policies raised a `column->size() == num_rows` logical error at query time. [#109753](https://github.com/ClickHouse/ClickHouse/pull/109753) ([Raúl Marín](https://github.com/Algunenano)). [#109973](https://github.com/ClickHouse/ClickHouse/pull/109973) ([Raúl Marín](https://github.com/Algunenano)). * Reading a Paimon table whose schema contains an unsupported nested type (for example a `ROW` field) now reports a clear `BAD_ARGUMENTS` error naming the unsupported type, instead of a bare `DB::Exception. (OK)` with error code 0 and no message. [#109762](https://github.com/ClickHouse/ClickHouse/pull/109762) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`std::length_error`) when a query with the experimental setting `make_distributed_plan = 1` used a very large `distributed_plan_default_reader_bucket_count` or `distributed_plan_default_shuffle_join_bucket_count`. Such values are now rejected with `INVALID_SETTING_VALUE`. [#109770](https://github.com/ClickHouse/ClickHouse/pull/109770) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`query tree node does not have valid source node`) when a recursive CTE resolved an identifier from an outer scope as a correlated column (with `allow_experimental_correlated_subqueries = 1`). Such queries now return a clear `UNSUPPORTED_METHOD` error. [#109863](https://github.com/ClickHouse/ClickHouse/pull/109863) ([Groene AI](https://github.com/groeneai)). * Fix a signed integer overflow when `ORDER BY ... WITH FILL` skips a very large gap over an `Int64`/`UInt64` column (e.g. a step of 2 across a gap near 2^63). The overflow was undefined behavior under sanitizers and silently wrapped in release builds. [#109937](https://github.com/ClickHouse/ClickHouse/pull/109937) ([Groene AI](https://github.com/groeneai)). * Fix a `Bad cast from type DB::FunctionNode to DB::ConstantNode` logical error (server abort in debug/sanitizer builds) when running `SELECT ... ORDER BY ... WITH FILL` against a `Distributed` table with a low `optimize_const_name_size`. [#109938](https://github.com/ClickHouse/ClickHouse/pull/109938) ([Groene AI](https://github.com/groeneai)). * Fixed an `Invalid number of rows in Chunk` logical error when an `INTERPOLATE` target is an alias of a `WITH FILL` column with the old analyzer (`enable_analyzer = 0`). Such queries are now rejected with a clear `INVALID_WITH_FILL_EXPRESSION` error. [#110103](https://github.com/ClickHouse/ClickHouse/pull/110103) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (server abort in debug/sanitizer builds) when the `indexHint`/`ignore`/`isZeroOrNull` functions are given an argument whose type resolves to `Nothing`, e.g. inside expressions like `indexHint(assumeNotNull(materialize(NULL)))`. [#110192](https://github.com/ClickHouse/ClickHouse/pull/110192) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error `Too large size (...) passed to allocator` that could occur when an out-of-range value was set for a read buffer size setting such as `max_read_buffer_size` or `max_read_buffer_size_local_fs`. Such values are now clamped to 256 MiB by the settings sanity check, and additionally at the consumption site for `INSERT ... FROM INFILE`, which reads files inside `clickhouse-client` where the sanity check does not apply. [#110207](https://github.com/ClickHouse/ClickHouse/pull/110207) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `CHECK TABLE` now reports the actual corruption for a projection part that failed to load its metadata, instead of a misleading `Columns doesn't match ... Expected: 0 columns` error. [#110262](https://github.com/ClickHouse/ClickHouse/pull/110262) ([Raúl Marín](https://github.com/Algunenano)). * A type mismatch when a value read from a PostgreSQL source (e.g. the `postgresql` table function or the `PostgreSQL` table engine) cannot be parsed into the declared column type (for example a `text` column declared as `Int32`) is now reported as a proper query error; previously it aborted the server in debug and sanitizer builds. [#110264](https://github.com/ClickHouse/ClickHouse/pull/110264) ([Groene AI](https://github.com/groeneai)). * Support the query profiler, memory and trace profilers, and `system.trace_log` symbolization on macOS. [#109825](https://github.com/ClickHouse/ClickHouse/pull/109825) ([Raúl Marín](https://github.com/Algunenano)). * Update `chdig` to [v26.7.1](https://github.com/azat/chdig/releases/tag/v26.7.1): compressed stack traces for sharing, query patterns, separate-arena memory support, Perfetto improvements, and compatibility fixes. [#110938](https://github.com/ClickHouse/ClickHouse/pull/110938) ([Azat Khuzhin](https://github.com/azat)). * Fix incorrect monotonicity inference for `intDiv` on an unsigned key divided by a signed constant, which could disable primary-key index pruning for `IN`/`NOT IN` predicates that cross the signed boundary of the result type; debug builds also hit a `LOGICAL_ERROR` `Invalid binary search result in MergeTreeSetIndex`. [#107586](https://github.com/ClickHouse/ClickHouse/pull/107586) ([Groene AI](https://github.com/groeneai)). * Added the `allow_lossy_numeric_supertype` setting (disabled by default). When enabled, `if`, `multiIf`, `coalesce`, `ifNull`, `array`, and `map` over numeric arguments that have no lossless common type (for example `Decimal` and `Float64`, or `Int64` and `Float64`) resolve to `Float64` instead of `Variant`, so the result can be used with aggregate functions such as `sum`, `avg`, `min`, and `max`. Relevant aggregate-function error messages now mention the setting by name. Closes [#106707](https://github.com/ClickHouse/ClickHouse/issues/106707). [#107236](https://github.com/ClickHouse/ClickHouse/pull/107236) ([Groene AI](https://github.com/groeneai)). * Start the Prometheus endpoint (for metrics-only configurations) and asynchronous metrics collection before tables are loaded, so metrics are visible during the potentially long metadata loading phase. [#108402](https://github.com/ClickHouse/ClickHouse/pull/108402) ([Christoph Wurm](https://github.com/cwurm)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fix `TRUNCATE TABLE` and `DROP PARTITION` failing on tables with many deduplication blocks, where removing them in a single ZooKeeper request could exceed the default 1 MB `jute.maxbuffer` limit. [#105991](https://github.com/ClickHouse/ClickHouse/pull/105991) ([Clayton McClure](https://github.com/cmcclure-twilio)). This only affects you if you use Apache ZooKeeper (not recommended) instead of ClickHouse Keeper. * Fix the usage of qualified column names (`database.table.column`) in the `WHERE` clause of mutations, such as `DELETE FROM` and `ALTER TABLE ... UPDATE/DELETE`, over `MergeTree`-family tables; previously such queries failed with a missing-columns error. Closes [#71760](https://github.com/ClickHouse/ClickHouse/issues/71760). [#109491](https://github.com/ClickHouse/ClickHouse/pull/109491) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix the query result cache for PromQL queries. The `promql` dialect bypassed the non-deterministic-function check and could serve stale results anchored at `now()`; the Prometheus HTTP API (`/api/v1/query`, `/api/v1/query_range`) never stored cache entries at all. [#110887](https://github.com/ClickHouse/ClickHouse/pull/110887) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Fix the server failing to start when the `TZ` environment variable is empty. Closes [#68920](https://github.com/ClickHouse/ClickHouse/issues/68920). [#68921](https://github.com/ClickHouse/ClickHouse/pull/68921) ([Ardenwick](https://github.com/ardenwick)). * Fixed the server failing to start when a `Backup` database engine refers to a backup that has become unavailable (for example, when its files were deleted or the underlying storage is inaccessible). The database is now loaded without tables instead of preventing the whole server from starting. [#83188](https://github.com/ClickHouse/ClickHouse/pull/83188) ([Vitaly Orlov](https://github.com/orloffv)). * When using background inserts into tables with the `Distributed` engine, delays caused by repeated errors when sending data to remote shards now decrease once the errors are resolved. Previously, the delay would only increase and never reset. [#87378](https://github.com/ClickHouse/ClickHouse/pull/87378) ([Andrei Kochemirovskii](https://github.com/Blackmorse)). * Fix `broken_data_files` in `system.distribution_queue` and the `BrokenDistributedFilesToInsert` metric always reporting `0` for broken files discovered when scanning the `Distributed` async-insert queue at startup. [#92124](https://github.com/ClickHouse/ClickHouse/pull/92124) ([KG.Xu](https://github.com/squalfof)). * Fix recovery of logging after a disk-full error. Previously, when the log disk became full, ClickHouse would enter a failed state and continuously spam error messages to syslog (potentially writing 100+ GB/hour), never recovering even after disk space was freed. Now the logging system automatically recovers once disk space becomes available, without requiring a server restart. [#93127](https://github.com/ClickHouse/ClickHouse/pull/93127) ([jaehanbyun](https://github.com/jaehanbyun)). * Forward static S3 credentials to Unity `DataLakeCatalog` table reads when `vended_credentials` is disabled, preventing anonymous object-storage access failures. [#96910](https://github.com/ClickHouse/ClickHouse/pull/96910) ([kgeg401](https://github.com/kgeg401)). * Fix a crash in the schema parsing code of the `DeltaLake` table engine. [#97112](https://github.com/ClickHouse/ClickHouse/pull/97112) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix a security issue where an unauthenticated TCP client could probe table existence and replication status via the interserver port. [#99854](https://github.com/ClickHouse/ClickHouse/pull/99854) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix a syntax error when an alias follows a subquery in `DESCRIBE TABLE`. Fixes [#100031](https://github.com/ClickHouse/ClickHouse/issues/100031). [#100205](https://github.com/ClickHouse/ClickHouse/pull/100205) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed an exception in correlated subqueries when outer columns become `Nullable` under `group_by_use_nulls` with `ROLLUP`/`CUBE`. [#100365](https://github.com/ClickHouse/ClickHouse/pull/100365) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server abort (in sanitizer builds) and silent data loss during table startup when a rolled-back transactional `MergeTree` part intersects a committed part. [#100992](https://github.com/ClickHouse/ClickHouse/pull/100992) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fixed `indexOfAssumeSorted` returning incorrect results for `Array(LowCardinality(String))` columns in `MergeTree` tables. [#101771](https://github.com/ClickHouse/ClickHouse/pull/101771) ([Yash ](https://github.com/Onyx2406)). * Fix incorrect primary-key pruning when a table's sorting key wraps a `Date` column in `toDateTime` (for example, `ORDER BY toDateTime(date_column)`) and a query filters on the original column with a comparison like `WHERE date_column >= '...'`. `toDateTime(Date)` overflows for `Date` values beyond the `DateTime` range (after `2106-02-07`), so the stored key is non-monotonic; ClickHouse no longer uses primary key pruning for this key/predicate combination because doing so could drop granules that contain matching rows. Closes [#101744](https://github.com/ClickHouse/ClickHouse/issues/101744). [#101814](https://github.com/ClickHouse/ClickHouse/pull/101814) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed `intDivOrNull`, `moduloOrNull` and `positiveModuloOrNull` returning `0` instead of `NULL`, and `intDivOrNull` and `intDivOrZero` raising an exception instead of returning `NULL` / `0`, when the division leads to a floating-point exception (division by zero or `INT_MIN / -1`), including for mixed signed/unsigned arguments. [#101976](https://github.com/ClickHouse/ClickHouse/pull/101976) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `LOGICAL_ERROR` exceptions when reading `Iceberg` or `DeltaLake` tables in corner cases such as concurrent `Iceberg` metadata updates or reads through the `merge` table function over `DeltaLake` tables. [#102033](https://github.com/ClickHouse/ClickHouse/pull/102033) ([Groene AI](https://github.com/groeneai)). * Fix outbound HTTP requests (e.g. the `aiGenerate` function, the `url` table function, S3) failing with `No route to host` on hosts that advertise both IPv4 and IPv6 addresses when only one address family is routable. The HTTP connection pool now falls back to the next resolved address when the first one fails with a network error, instead of propagating the error on the very first request. [#103786](https://github.com/ClickHouse/ClickHouse/pull/103786) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a server abort during `RESTORE` of backups containing tables with cyclic dependencies. [#103824](https://github.com/ClickHouse/ClickHouse/pull/103824) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Schema inference for Arrow `time32`/`time64` columns (e.g. `SELECT * FROM file(..., 'Arrow')`) now infers `Time64` instead of `DateTime64`; anything downstream relying on the old inferred type will observe the new type after an upgrade. Also, `Time`/`Time64` values can now be exported to Arrow with the appropriate Apache Arrow time type selected by precision. Fixes [#104038](https://github.com/ClickHouse/ClickHouse/issues/104038). [#104316](https://github.com/ClickHouse/ClickHouse/pull/104316) ([/bin/cat](https://github.com/Maoyao233)). * Fixed a race between `ALTER TABLE ... RENAME COLUMN` and a concurrent `OPTIMIZE TABLE ... FINAL` (or any background merge) in `MergeTree` that could silently replace the renamed column's data with default values. [#104822](https://github.com/ClickHouse/ClickHouse/pull/104822) ([Groene AI](https://github.com/groeneai)). * Fixed a `Block structure mismatch` logical error during a concurrent `INSERT` while an `ALTER TABLE ... RENAME COLUMN` is in flight, in an `Atomic` database with `lazy_load_tables = 1` after `DETACH DATABASE`/`ATTACH DATABASE`. [#104852](https://github.com/ClickHouse/ClickHouse/pull/104852) ([Groene AI](https://github.com/groeneai)). * Fix `groupConcat` when the parametric and two-argument spellings are mixed, e.g. `groupConcat(',', 2)(x, '/')`: the row-limit parameter was silently dropped, so every row was returned instead of the requested number; the delimiter from the second argument now correctly overrides the parameter. [#104882](https://github.com/ClickHouse/ClickHouse/pull/104882) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix a `LOGICAL_ERROR` (`Expected one block from input stream`) thrown by `KILL QUERY`/`KILL MUTATION`/`KILL PART_MOVE_TO_SHARD`/`KILL TRANSACTION` when their `WHERE` clause contains a per-row subquery, or when `max_block_size` is small enough that the internal `SELECT` over the relevant `system.*` table emits more than one block. [#104927](https://github.com/ClickHouse/ClickHouse/pull/104927) ([Groene AI](https://github.com/groeneai)). * Fixed `CAST` from smaller to larger interval units returning wrong results (for example, `CAST(toIntervalSecond(60) AS IntervalMinute)` returned `0` instead of `1`). Closes [#104986](https://github.com/ClickHouse/ClickHouse/issues/104986). [#105058](https://github.com/ClickHouse/ClickHouse/pull/105058) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix a logical error (`Assertion 'row < chunk.getNumRows()' failed`) in `LIMIT ... WITH TIES` queries running with the read-in-order pipeline (`optimize_read_in_order = 1` and `read_in_order_use_virtual_row_per_block = 1`). [#105102](https://github.com/ClickHouse/ClickHouse/pull/105102) ([Groene AI](https://github.com/groeneai)). * Fix a leak in refreshable materialized views (`MATERIALIZED VIEW ... REFRESH ...`): the temporary inner table that a refresh rotates out via `EXCHANGE TABLES` could not be dropped if it was larger than `max_table_size_to_drop`, so every subsequent refresh created yet another temporary inner table, growing the view's data directory until the disk was exhausted. The refresh task's internal drop now bypasses `max_table_size_to_drop` and `max_partition_size_to_drop`; those safety nets still apply to user-issued `DROP TABLE` of the view itself. Closes [#104900](https://github.com/ClickHouse/ClickHouse/issues/104900). [#105106](https://github.com/ClickHouse/ClickHouse/pull/105106) ([Groene AI](https://github.com/groeneai)). * Fix an unexpected `DATA_TYPE_CANNOT_BE_USED_IN_KEY` error on `ALTER` queries that do not change the sorting key (changing settings, comments, codecs, adding a non-key column, etc.) for `MergeTree` tables that have a `SimpleAggregateFunction` (or another type allowed only with `allow_suspicious_primary_key = 1`) in the sorting key. [#105111](https://github.com/ClickHouse/ClickHouse/pull/105111) ([Groene AI](https://github.com/groeneai)). * Fixed a use-after-free when querying `system.clusters` replica-state columns (such as `is_active`, `unsynced_after_recovery`, `recovery_time`) while a `Replicated` database is being dropped or detached. Also, benign Keeper exceptions swallowed in this code path (the affected database is treated as transiently unavailable and skipped) are now logged at the `Information` level instead of `Error`, so they no longer reach clients at the default `send_logs_level = 'warning'`. [#105149](https://github.com/ClickHouse/ClickHouse/pull/105149) ([Groene AI](https://github.com/groeneai)). * Fixed an issue where running multiple statements in `clickhouse-local --ignore-error` would echo every subsequent statement inside the error message for each lexical error or unmatched-parenthesis error. [#105480](https://github.com/ClickHouse/ClickHouse/pull/105480) ([Groene AI](https://github.com/groeneai)). * Fix a rare server crash when refreshable materialized views are dropped or replaced concurrently with their refresh scheduling. [#105588](https://github.com/ClickHouse/ClickHouse/pull/105588) ([Groene AI](https://github.com/groeneai)). * Reject `ALTER TABLE ... DELETE` and `ALTER TABLE ... UPDATE` on `Iceberg` tables whose data file format is not `Parquet` with a clear `NOT_IMPLEMENTED` error instead of crashing the server or silently corrupting the table. [#105893](https://github.com/ClickHouse/ClickHouse/pull/105893) ([Groene AI](https://github.com/groeneai)). * Fix wrong results from a view defined with `EXCEPT` or `INTERSECT` whose operand is a `UNION` chain, after `DETACH`/`ATTACH` or a server restart. The formatter was missing parentheses around `UNION` children of `INTERSECT`/`EXCEPT`, so the stored SQL was reparsed with reversed precedence. [#105935](https://github.com/ClickHouse/ClickHouse/pull/105935) ([Groene AI](https://github.com/groeneai)). * Fix `JSON` data misdetected as `TSKV` during format auto-detection. Closes [#100797](https://github.com/ClickHouse/ClickHouse/issues/100797). [#106009](https://github.com/ClickHouse/ClickHouse/pull/106009) ([Pavel Kruglov](https://github.com/Avogar)). * Forbid creating a `minmax` skip index directly on `JSON` columns — it could later fail inserts with a `NO_COMMON_TYPE` exception when mixed-type arrays were inserted. Create the index on typed subcolumns instead (e.g. `INDEX idx json.field TYPE minmax`). [#106094](https://github.com/ClickHouse/ClickHouse/pull/106094) ([linhaojie](https://github.com/linhaojie)). * Fixed a logical error (`Parsed partition value ... doesn't match partition value for an existing part with the same partition ID`) thrown by `OPTIMIZE TABLE ... PARTITION ...` and other queries that resolve a partition value, on tables with a `Time`-typed partition key. [#106202](https://github.com/ClickHouse/ClickHouse/pull/106202) ([Groene AI](https://github.com/groeneai)). * Fix a data part being incorrectly marked as having a broken projection after a lightweight delete leaves no projection part behind — either when the projection is rebuilt with zero output rows or when it is dropped (`lightweight_mutation_projection_mode = 'rebuild'`/`'drop'`). The broken state previously disabled projection optimization for queries on that part. [#106273](https://github.com/ClickHouse/ClickHouse/pull/106273) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `removeDirectory` on `plain` object storage disks (such as `s3_plain`) removing all files inside a non-empty directory as if the removal were recursive; removing a non-empty directory now fails with `CANNOT_RMDIR`, and recursive removal handles the contents explicitly. [#106281](https://github.com/ClickHouse/ClickHouse/pull/106281) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Fixed a logical error (`Part ... doesn't exist`) that could abort the server during recovery of a quorum `INSERT` from a Keeper hardware error when the quorum was concurrently marked as failed. [#106424](https://github.com/ClickHouse/ClickHouse/pull/106424) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` exception about duplicate column names when applying row policies. [#106438](https://github.com/ClickHouse/ClickHouse/pull/106438) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix `CREATE OR REPLACE TABLE` and `REPLACE TABLE` leaving a stranded `_tmp_replace_*` table on disk and replacing the original table with an empty one when the existing target exceeds `max_table_size_to_drop`. The size check now runs before the swap, so a violation aborts cleanly with the user-visible table name in the error. [#106782](https://github.com/ClickHouse/ClickHouse/pull/106782) ([Groene AI](https://github.com/groeneai)). * Fix `JSON_QUERY`/`JSON_VALUE`/`JSON_EXISTS` returning `Dynamic` instead of `String`/`UInt8` on `Dynamic` arguments. Closes [#106461](https://github.com/ClickHouse/ClickHouse/issues/106461). [#106877](https://github.com/ClickHouse/ClickHouse/pull/106877) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed wrong results when querying a `ReplacingMergeTree` table with `FINAL` and a filter on a text index while `query_plan_optimize_lazy_final` was enabled. The lazy `FINAL` optimization built reading steps that did not reproduce the direct read from the text index, so the filter dropped all matching rows. [#106894](https://github.com/ClickHouse/ClickHouse/pull/106894) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Apply `query_masking_rules` to messages appended to exceptions, so URL-encoded credentials in `(in file/uri ...)` suffixes of errors from the `jdbc`/`odbc` table functions are not leaked when masking rules are configured. [#106916](https://github.com/ClickHouse/ClickHouse/pull/106916) ([Gaurav Dubey](https://github.com/gaurav0107)). * Fixed parts being marked as broken and detached on any part reload (server restart, `DETACH`/`ATTACH`) for tables with a `LowCardinality(Nullable(...))` column in the partition key. Since 26.5, the per-part minmax index file was not written when such a column's minimum and maximum were `NULL`, while the part consistency check still required the file. Parts written by affected versions lack the minmax index file and still need to be re-attached manually. Closes [#106837](https://github.com/ClickHouse/ClickHouse/issues/106837). [#106945](https://github.com/ClickHouse/ClickHouse/pull/106945) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix a `LOGICAL_ERROR` (`Unexpected expression in JOIN ON section. Expected boolean (UInt8), got 'Nothing'`) when a non-equi `JOIN ... ON` predicate references a `Nothing`-typed column, such as one produced by `ARRAY JOIN []`. [#106981](https://github.com/ClickHouse/ClickHouse/pull/106981) ([Groene AI](https://github.com/groeneai)). * Fix a rare `LOGICAL_ERROR` (`Attempt to release query context that does not exist`) and the accompanying server crash when reading `MergeTree` tables through a filesystem cache disk created with `enable_filesystem_query_cache_limit = 1`. [#107028](https://github.com/ClickHouse/ClickHouse/pull/107028) ([Groene AI](https://github.com/groeneai)). * Fix a server crash when moving an empty part to a `plain_rewritable` disk (for example, with `ALTER TABLE ... MOVE PARTITION ... TO DISK` for an empty part kept by `remove_empty_parts = 0`). [#107040](https://github.com/ClickHouse/ClickHouse/pull/107040) ([Groene AI](https://github.com/groeneai)). * Fix incorrect row order in `ORDER BY` queries over `UNION ALL` with `optimize_read_in_order` and `read_in_order_use_virtual_row` enabled. Closes [#106879](https://github.com/ClickHouse/ClickHouse/issues/106879). [#107053](https://github.com/ClickHouse/ClickHouse/pull/107053) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a `LOGICAL_ERROR` (`Unsupported argument types`) in the `geohashesInBox` function when its coordinate arguments mixed constant and non-constant values, or when a coordinate was a `BFloat16`. Mixed const/non-const `Float32` arguments now work, and `BFloat16` arguments are rejected with a clear error. [#107063](https://github.com/ClickHouse/ClickHouse/pull/107063) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results (duplicate rows) for `SELECT DISTINCT` over a `GROUP BY` with `WITH CUBE`, `WITH ROLLUP`, or `GROUPING SETS` on the same keys. The `query_plan_remove_redundant_distinct` optimization no longer removes the `DISTINCT` in these cases, because the grouping modifiers emit extra rows with the key columns defaulted, which can collide with real group values. [#107072](https://github.com/ClickHouse/ClickHouse/pull/107072) ([Groene AI](https://github.com/groeneai)). * Fix a wrong result in an `OUTER JOIN` when the `WHERE` filter is a conjunction that contains a constant term (for example `p AND 1`, or `WHERE p QUALIFY NULL` where the `QUALIFY NULL` is merged into `WHERE`). The constant conjunct could be pushed into the non-preserved side of the join and removed from the post-join filter, so the join produced non-matched rows with the side columns defaulted, and those rows incorrectly passed the filter. [#107084](https://github.com/ClickHouse/ClickHouse/pull/107084) ([Groene AI](https://github.com/groeneai)). * Fixed `executable_pool` user-defined functions configured with `` not picking up changes to the underlying script. Previously, only `SYSTEM RELOAD FUNCTIONS` would re-read an edited script; periodic `` reloads kept executing the old version. [#107087](https://github.com/ClickHouse/ClickHouse/pull/107087) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` exception (`Cannot find input column ... on its position in inputs of expression actions DAG`) for a correlated scalar subquery over a source projecting the same column identifier twice (e.g. `SELECT number, *`) when `correlated_subqueries_default_join_kind = 'left'`. [#107112](https://github.com/ClickHouse/ClickHouse/pull/107112) ([Groene AI](https://github.com/groeneai)). * Fixed a data race on the server-global trace collector between a worker thread starting its profiler and server shutdown. [#107307](https://github.com/ClickHouse/ClickHouse/pull/107307) ([Groene AI](https://github.com/groeneai)). * Incremental backups no longer store `S3` credentials in the `` locator of the `.backup` metadata file. Backups created with `use_same_s3_credentials_for_base_backup = 1`, or with explicit base backup credentials matching this backup locator, store a non-secret marker and are restored without extra restore-time settings; for backups created with different explicit base backup credentials or extra base authentication arguments, pass them to `RESTORE` with the `base_backup` setting. Backups created by older versions with embedded credentials remain restorable. [#107357](https://github.com/ClickHouse/ClickHouse/pull/107357) ([Pablo Marcos](https://github.com/pamarcos)). * Fixed the `CSVWithNames` and `CSVWithNamesAndTypes` header having fewer columns than the data when `output_format_csv_serialize_tuple_into_separate_columns` is enabled (the default). The header (and the types row) now flattens `Tuple` columns into their leaf fields with dotted names (e.g. `t.a`, `t.b`), so the header column count matches the data. A new setting `output_format_csv_header_serialize_tuple_into_separate_columns` (default `1`) controls this and can be set to `0` to restore the previous single-name header. [#107371](https://github.com/ClickHouse/ClickHouse/pull/107371) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Iceberg` v3 tables whose `Parquet` data files contain reserved row-lineage columns (such as `_row_id`); the native `Parquet` reader no longer raises `ICEBERG_SPECIFICATION_VIOLATION` for reserved field IDs that are not part of the table schema. [#107377](https://github.com/ClickHouse/ClickHouse/pull/107377) ([Greg Maher](https://github.com/gregakinman)). * Fixed a rare server abort during shutdown (`std::future_error`: `The associated promise has been destructed prior to the associated state becoming ready`) caused by a scheduled task being dropped from a thread pool queue before it ran. [#107383](https://github.com/ClickHouse/ClickHouse/pull/107383) ([Groene AI](https://github.com/groeneai)). * Fix the `MaterializedPostgreSQL` table and database engines so that a single table or database can be replicated from a non-default PostgreSQL schema (`materialized_postgresql_schema`), including the case where tables with the same name exist in several schemas of the same database (previously they would share a publication and replication slot and cross-talk). [#107425](https://github.com/ClickHouse/ClickHouse/pull/107425) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `MaterializedPostgreSQL` stopping replication of an entire database (with `LOGICAL_ERROR: Columns number mismatch`) when a single replicated table's structure changed while the server was down. Now only the affected table is skipped and can be recovered with `DETACH`/`ATTACH`. [#107427](https://github.com/ClickHouse/ClickHouse/pull/107427) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `BACKUP` of a `MaterializedPostgreSQL` database no longer hangs forever with `Table ... were created or changed its definition during scanning`, and now actually backs up the table data (delegated to the underlying `ReplacingMergeTree`), which can be restored as a standalone `ReplacingMergeTree`. [#107433](https://github.com/ClickHouse/ClickHouse/pull/107433) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a regression where setting `compatibility = '26.6'` (which implicitly enables the `hive` partition strategy) silently accepted `{_partition_id}` in S3/object storage table paths instead of raising `BAD_ARGUMENTS`. [#107437](https://github.com/ClickHouse/ClickHouse/pull/107437) ([Lefteris](https://github.com/LefterisXefteris)). * Fixed a memory leak in the bundled `mongo-c-driver` that could occur when reading from MongoDB (for example, via a MongoDB dictionary or the `mongodb` table function) if a retryable read error was followed by a failed retry server selection. [#107448](https://github.com/ClickHouse/ClickHouse/pull/107448) ([Groene AI](https://github.com/groeneai)). * Fix an exception when inserting into an `Iceberg` table whose metadata was created by an external engine and omits the optional `snapshots`, `metadata-log`, or `snapshot-log` arrays. Such inserts now succeed instead of failing. [#107473](https://github.com/ClickHouse/ClickHouse/pull/107473) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `DELETE FROM` (lightweight delete) requiring the `ALTER UPDATE` privilege in addition to `ALTER DELETE`. A user granted only `ALTER DELETE` can now run `DELETE FROM`, as documented. [#107491](https://github.com/ClickHouse/ClickHouse/pull/107491) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` when reading Hive-partitioned files with `use_hive_partitioning = 1` and a `WHERE`/`PREWHERE` clause that filters a real (non-virtual) column while a Hive partition column is also selected. [#107505](https://github.com/ClickHouse/ClickHouse/pull/107505) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`Different order of columns in UNION subquery`) when the old analyzer (`enable_analyzer = 0`) reads a subset of columns from a subquery whose `UNION` children are `INTERSECT`/`EXCEPT` set operations. [#107511](https://github.com/ClickHouse/ClickHouse/pull/107511) ([Groene AI](https://github.com/groeneai)). * Fix `NULL` values being silently converted to empty strings when inserting `Arrow`/`ORC` data into a `LowCardinality(Nullable(...))` column. This was a regression introduced in 26.5. [#107532](https://github.com/ClickHouse/ClickHouse/pull/107532) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix a logical error (`Stream ... variant_discr ... is not found`) that could occur when merging or reading a `MergeTree` part produced by a mutation of a table with a `Dynamic` column. [#107562](https://github.com/ClickHouse/ClickHouse/pull/107562) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a logical error (`WhichDataType(const_type).isArray()`) during primary-key analysis when `pointInPolygon` is called with a constant polygon argument of a wrapper type such as `Variant` or `Dynamic` (for example, `pointInPolygon((x, y), if(c, [(0, 0), ...], NULL))`). [#107589](https://github.com/ClickHouse/ClickHouse/pull/107589) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (`Last stored last_written_position in meta file ... is bigger than current last_written_pos`) in the `FileLog` engine that could happen when a watched file was deleted and recreated reusing the same inode. [#107617](https://github.com/ClickHouse/ClickHouse/pull/107617) ([Groene AI](https://github.com/groeneai)). * Fix a rare server abort (`std::future_error`: `The associated promise has been destructed prior to the associated state becoming ready`) during ZooKeeper client request processing, when an asynchronous Keeper request was dropped while being sent, for example under memory pressure. [#107647](https://github.com/ClickHouse/ClickHouse/pull/107647) ([Groene AI](https://github.com/groeneai)). * Fixed `RESTORE` for `ReplicatedMergeTree` tables so duplicate-content parts from a backup are preserved instead of being silently deduplicated. [#107652](https://github.com/ClickHouse/ClickHouse/pull/107652) ([Pablo Marcos](https://github.com/pamarcos)). * Fix runtime join filter producing wrong results for `JSON` columns. Closes [#107646](https://github.com/ClickHouse/ClickHouse/issues/107646). [#107663](https://github.com/ClickHouse/ClickHouse/pull/107663) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed incorrect results from distributed queries selecting `ALIAS` columns that share a common subexpression: columns could be misaligned and values returned under the wrong column. [#107675](https://github.com/ClickHouse/ClickHouse/pull/107675) ([Vladimir Cherkasov](https://github.com/vdimir)). * Quotas keyed by `normalized_query_hash` now account all resources (`read_rows`, `read_bytes`, `result_rows`, `result_bytes`, `execution_time`, `written_bytes`, `errors`) per query pattern, like the query-count counters, instead of accounting them against a single shared per-user bucket. [#107681](https://github.com/ClickHouse/ClickHouse/pull/107681) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `Block structure mismatch in UnionStep stream` logical error (server abort on debug/sanitizer builds, `Code: 49` on release builds) that occurred when sibling branches of a `UNION`/`INTERSECT`/`EXCEPT` differed only in their `WHERE` predicate and one branch's predicate was constant-folded to a `Const` column. [#107719](https://github.com/ClickHouse/ClickHouse/pull/107719) ([Groene AI](https://github.com/groeneai)). * Fix logical errors (`Bad cast`) caused by inconsistent stripping of `LowCardinality` nested inside `Variant` and `Dynamic` columns, for example in `concat`, `format`, and primary key analysis. Closes [#107598](https://github.com/ClickHouse/ClickHouse/issues/107598). [#107773](https://github.com/ClickHouse/ClickHouse/pull/107773) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed a severe work-distribution skew with parallel replicas when the cluster contains inactive replicas (for example, stale entries left after autoscaling). Such replicas are no longer counted by the reading coordinator, so work is balanced across the online replicas instead of piling onto a single one. [#107805](https://github.com/ClickHouse/ClickHouse/pull/107805) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `Not-ready Set is passed as the second argument` exception that could occur when building an `IN` subquery set during primary key analysis failed silently (for example, a subquery timeout with `overflow_mode = 'break'`), leaving the set permanently unbuilt for the query pipeline. [#107924](https://github.com/ClickHouse/ClickHouse/pull/107924) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a server crash in predicate pushdown (`enable_optimize_predicate_expression`, legacy analyzer) when a `UNION` subquery appears inside a `JOIN ... ON` condition. [#107930](https://github.com/ClickHouse/ClickHouse/pull/107930) ([Groene AI](https://github.com/groeneai)). * Fix spurious `CHECKSUM_DOESNT_MATCH` and `Parquet` read errors (and, for some formats, silently wrong results) when an S3 or S3-compatible (e.g. GCS) object is overwritten in place while being read. Such reads now fail with a clear, retryable error instead of returning data stitched from two object versions; controlled by the new setting `s3_validate_etag_on_read` (on by default). [#107934](https://github.com/ClickHouse/ClickHouse/pull/107934) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `SELECT` failing with `NOT_FOUND_COLUMN_IN_BLOCK` on a table that has a `set` data-skipping index when a row policy filters it using an always-true condition combined with a check on a non-indexed column. [#107971](https://github.com/ClickHouse/ClickHouse/pull/107971) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed executable user-defined function command parameter parsing so placeholders with empty or invalid names are not accepted as parameters. [#107983](https://github.com/ClickHouse/ClickHouse/pull/107983) ([Goutam Adwant](https://github.com/goutamadwant)). * Fix performance regression for `Map` subcolumns with `PREWHERE`. Closes [#107912](https://github.com/ClickHouse/ClickHouse/issues/107912). Caused by [#99200](https://github.com/ClickHouse/ClickHouse/pull/99200). [#107988](https://github.com/ClickHouse/ClickHouse/pull/107988) ([Pavel Kruglov](https://github.com/Avogar)). * Fix an out-of-bounds write when reading a `Variant` column from a malformed `Native` block whose discriminators reference a variant index that does not exist. [#107991](https://github.com/ClickHouse/ClickHouse/pull/107991) ([uwezkhan](https://github.com/uwezkhan)). * Fix wrong query results from incorrect primary key and partition pruning when `toStartOfDay`, `toStartOfISOYear`, `toDaysSinceYearZero`, `toRelativeSecondNum`, `toRelativeMinuteNum`, `toRelativeHourNum`, `toRelativeWeekNum`, `toRelativeDayNum`, `toMonthNumSinceEpoch`, or `toYearNumSinceEpoch` is applied to a `Date32` key column containing dates outside the range the function can represent (in particular dates before `1970-01-01`). These functions report themselves as monotonic to the primary index, but previously wrapped around for such arguments, so the index could prune granules that actually contained matching rows; they now saturate at the bounds of their result type and stay monotonic over the whole `Date32` range. [#108018](https://github.com/ClickHouse/ClickHouse/pull/108018) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `accurateCastOrNull` of a `Tuple` whose element is `Dynamic`/`Variant`: a genuine source `NULL` was treated as a conversion failure. For a `Nullable` target element a source `NULL` now stays an element `NULL` (matching a plain `Tuple(Nullable(...))` source); for a non-`Nullable` target element a source `NULL` now nulls the whole tuple instead of producing the element default; parse/overflow failures still null the whole tuple. [#108023](https://github.com/ClickHouse/ClickHouse/pull/108023) ([Groene AI](https://github.com/groeneai)). Also fix a logical error when casting a `Dynamic` or `Variant` column nested inside a `Tuple` to a non-`Nullable` element type with `accurateCastOrNull` or `accurateCastOrDefault`. [#108061](https://github.com/ClickHouse/ClickHouse/pull/108061) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `numericIndexedVectorPointwiseMultiply` returning an empty result when multiplying by an all-ones vector that has a different BSI (bit-sliced index) configuration. [#108027](https://github.com/ClickHouse/ClickHouse/pull/108027) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `toTime64` and `CAST(... AS Time64)` not clamping out-of-range values to the `Time64` range in `saturate` and `ignore` overflow modes, which could produce values that display identically but compare as different. [#108028](https://github.com/ClickHouse/ClickHouse/pull/108028) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `getClientHTTPHeader` is now correctly treated as non-deterministic, so its result is no longer incorrectly reused by the query result cache. [#108029](https://github.com/ClickHouse/ClickHouse/pull/108029) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-client --port 9440` again automatically enables a secure (TLS) connection; this was broken by a refactoring that stopped passing the port to the secure-connection check. [#108032](https://github.com/ClickHouse/ClickHouse/pull/108032) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A transient error while refreshing data parts of a read-only table no longer permanently stops the background refresh task. [#108034](https://github.com/ClickHouse/ClickHouse/pull/108034) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed memory/CPU/mutation overload warnings in `system.warnings` lagging one asynchronous-metrics cycle behind (and being absent on the first cycle). [#108035](https://github.com/ClickHouse/ClickHouse/pull/108035) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `mongodb` table function now accepts `oid_columns` passed as a named argument (e.g. `oid_columns='_id'`), instead of rejecting it with `BAD_ARGUMENTS`. [#108039](https://github.com/ClickHouse/ClickHouse/pull/108039) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `Bad cast from type DB::ColumnNullable to DB::ColumnVector<...>` exception (logical error) when a qualified asterisk (`t.*`) selects a `JOIN USING` key and that join is nested below a `PASTE`/`CROSS`/comma join or an outer `ON` join, with `join_use_nulls = 0`. [#108043](https://github.com/ClickHouse/ClickHouse/pull/108043) ([Groene AI](https://github.com/groeneai)). * Fixed incorrect handling of zero-width assertions (`\b`, `^`, `$`) in the regexp "match all" functions `extractAll`, `extractAllGroupsVertical`, `extractAllGroupsHorizontal`, `countMatches` and `splitByRegexp`. For example, `extractAll('new york is the greatest', '\b(\w)')` now correctly returns the first letter of each word instead of every letter. [#108047](https://github.com/ClickHouse/ClickHouse/pull/108047) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `CREATE TABLE ... AS SELECT` on `Atomic` databases leaving an empty table behind when the query fails — for example when the user has no access to a table referenced from a subquery. Previously a retry reported that the table already exists instead of the original error. The table is now created via a temporary table and becomes visible only after it has been fully populated. [#108048](https://github.com/ClickHouse/ClickHouse/pull/108048) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow querying a `range_hashed` or `complex_key_range_hashed` dictionary that uses a `DateTime64`, `Decimal` or floating-point range with a matching argument to `dictGet`/`dictHas`. Previously such queries failed with `must be convertible to Int64`, and open-ended intervals of `Decimal`/`DateTime64` ranges incorrectly returned the default value. [#108052](https://github.com/ClickHouse/ClickHouse/pull/108052) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an exception (`CANNOT_PARSE_TEXT`) when loading a dictionary with a composite key whose key columns are not the first columns in the dictionary definition, with a local `ClickHouse` source using an explicit query. Source columns are now matched to the dictionary structure by name instead of by position. [#108053](https://github.com/ClickHouse/ClickHouse/pull/108053) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the `values` table function failing with `ARGUMENT_OUT_OF_BOUND` when a decimal literal that is not exactly representable in a narrow floating-point column (such as `0.1` for a `Float32` column) is used, e.g. `SELECT * FROM values('x Float32', 0.1)`. Such values are now accepted and converted to the nearest representable value, consistent with `CAST` and `INSERT ... VALUES`. [#108055](https://github.com/ClickHouse/ClickHouse/pull/108055) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` (`No available columns`) when executing `SELECT count()` (or other trivial queries) on a table where the user is granted `SELECT` access only on an `ALIAS` column. [#108056](https://github.com/ClickHouse/ClickHouse/pull/108056) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `mapFilter`, `mapSort` (and its variants) and `mapConcat` dropping `LowCardinality` from the key and value types of a `Map`. Previously `mapFilter` over a `Map(LowCardinality(String), String)` column returned `Map(String, String)`, which could corrupt the metadata of a table created via `CREATE TABLE ... AS SELECT` and make `CHECK TABLE` fail. [#108057](https://github.com/ClickHouse/ClickHouse/pull/108057) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results when using `hasToken` on text indexes with a tokenizer other than `splitByNonAlpha`. [#108066](https://github.com/ClickHouse/ClickHouse/pull/108066) ([Robert Schulze](https://github.com/rschu1ze)). * Fixed a freeze of `SYSTEM RELOAD DICTIONARIES` (and `RELOAD DICTIONARY`) when many MySQL dictionaries defined in XML shared a single connection pool via `share_connection`. Such pools now honor `connection_pool_size` and `connection_wait_timeout` (default 5 seconds) from the configuration, matching dictionaries defined through named collections. [#108083](https://github.com/ClickHouse/ClickHouse/pull/108083) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow `EXISTS ` for a user that has only the `SHOW DICTIONARIES` privilege on the dictionary (previously it required `SHOW TABLES`). [#108084](https://github.com/ClickHouse/ClickHouse/pull/108084) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `CREATE TABLE dst AS src SETTINGS ...` (and similar variants with `ORDER BY`/`PARTITION BY` but without an explicit `ENGINE`) silently dropping the source table's engine, storage clauses (such as table `TTL` and keys) and settings. The engine and the non-overridden storage clauses are now inherited from the source table, and the specified settings are merged on top of the source table's settings. [#108085](https://github.com/ClickHouse/ClickHouse/pull/108085) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `INSERT` into `MergeTree` tables failing with `filesystem error: in rename: Permission denied` on filesystems backed by Windows (WSL, CIFS/SMB, Docker Desktop bind mounts), caused by the part writer leaving file descriptors open across the part-directory rename. [#108089](https://github.com/ClickHouse/ClickHouse/pull/108089) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash (null pointer dereference) when querying a `Hive` table without a `WHERE` clause, e.g. `SELECT * FROM hive_table`. [#108094](https://github.com/ClickHouse/ClickHouse/pull/108094) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` ("Unexpected return type from if") when reading a column under `apply_mutations_on_fly = 1` after an `ALTER UPDATE col = ... WHERE ` with a non-constant or false condition followed by `ALTER MODIFY COLUMN col `. [#108128](https://github.com/ClickHouse/ClickHouse/pull/108128) ([Groene AI](https://github.com/groeneai)). * Fix `toStartOfInterval` and `dateTrunc` returning a rounded value instead of the start of the containing interval when the input has finer precision than the interval unit, e.g. `toStartOfInterval(toDateTime64('2023-10-09 10:11:12.000999', 6), INTERVAL 1 millisecond)` returned `10:11:12.001` instead of `10:11:12.000`. The overload with an explicit origin was affected too. Closes [#103535](https://github.com/ClickHouse/ClickHouse/issues/103535). [#108186](https://github.com/ClickHouse/ClickHouse/pull/108186) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Hide sensitive information in the `view_query` column of `system.query_views_log`. [#108214](https://github.com/ClickHouse/ClickHouse/pull/108214) ([Valerii Mordovskii](https://github.com/Fidelaggio)). * Fixed a wedge on a `MergeTree` table attached with a legacy `hypothesis` skip index (a removed index type kept only for `ATTACH` compatibility). Previously a lightweight `DELETE`, `OPTIMIZE`, a plain `INSERT`, or a filtered `SELECT` on such a table failed with `ILLEGAL_INDEX` ("Index of type 'hypothesis' is no longer supported"), and the `DELETE` mutation retried forever. The dead index is now treated as inert: it is carried forward untouched during merge/mutation, skipped on insert and query planning, and can still be dropped with `ALTER TABLE ... DROP INDEX`. [#108217](https://github.com/ClickHouse/ClickHouse/pull/108217) ([Groene AI](https://github.com/groeneai)). * Fix the setting `type_json_allow_duplicated_key_with_literal_and_nested_object` not working with typed paths in `JSON`. [#108218](https://github.com/ClickHouse/ClickHouse/pull/108218) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a `LOGICAL_ERROR` (`Different list of shards in child plans`) when running a query with `make_distributed_plan = 1` over a `MergeTree` table that has a normal/aggregate projection. [#108256](https://github.com/ClickHouse/ClickHouse/pull/108256) ([Groene AI](https://github.com/groeneai)). * Fixed `replaceRegexpOne` and `replaceRegexpAll` so `.` matches newline characters by default, consistently with other regular expression functions. [#108265](https://github.com/ClickHouse/ClickHouse/pull/108265) ([linjiayu1025-collab](https://github.com/linjiayu1025-collab)). * Fix a logical error when casting an `Array(Dynamic)` or `Array(Variant)` to `QBit` with `accurateCastOrNull`, e.g. `accurateCastOrNull(CAST(range(114), 'Array(Dynamic)'), 'QBit(Float32, 114)')`. [#108288](https://github.com/ClickHouse/ClickHouse/pull/108288) ([Groene AI](https://github.com/groeneai)). * `index_granularity_bytes` is now honored for `AggregateFunction` state columns. Previously the granule byte cap was ignored for such columns (for example `uniqExact` states in `AggregatingMergeTree` tables and aggregating projections), producing granules far larger than the configured limit and increasing read amplification and query-time memory. [#108297](https://github.com/ClickHouse/ClickHouse/pull/108297) ([Groene AI](https://github.com/groeneai)). * Fix lightweight `UPDATE`/`DELETE` conditions being evaluated twice, which could lead to incorrect behavior for non-deterministic conditions and extra work for deterministic ones. Closes [#86032](https://github.com/ClickHouse/ClickHouse/issues/86032). [#108323](https://github.com/ClickHouse/ClickHouse/pull/108323) ([ofeliacode](https://github.com/ofeliacode)). * Fix an exception in a predefined HTTP handler when a header whose `headers_regexp` regular expression allows an empty value is absent from the request. [#108324](https://github.com/ClickHouse/ClickHouse/pull/108324) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix `arrayPartialSort` with a non-constant limit argument. [#108327](https://github.com/ClickHouse/ClickHouse/pull/108327) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed a cancelled or `KILL`ed `INSERT` continuing to run for a long time while building a skip index (for example an unbounded `set(0)` index on a high-cardinality column). The index build now stops promptly when the query is cancelled. [#108351](https://github.com/ClickHouse/ClickHouse/pull/108351) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `SELECT` from the `primes` table function not responding to cancellation: with a large `step` (or `limit`) the query could keep running for a long time after `KILL QUERY` or a timeout. It now stops promptly. [#108353](https://github.com/ClickHouse/ClickHouse/pull/108353) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed parsing of data-type names whose name contains the substring `INT` (for example a function-like name such as `quantileInterpolatedWeighted` used in a data-type position). Such names were mistaken for MySQL integer types and had their first `(...)` argument group silently consumed as a display-width modifier, which could break the query formatting round-trip. [#108354](https://github.com/ClickHouse/ClickHouse/pull/108354) ([Groene AI](https://github.com/groeneai)). * Fixed an inconsistency where a malformed `ARRAY JOIN` followed by a comma and a parenthesized table list was misparsed as a cross join instead of being rejected. [#108365](https://github.com/ClickHouse/ClickHouse/pull/108365) ([Raúl Marín](https://github.com/Algunenano)). * Hide secret arguments of functions such as `encrypt`, `decrypt`, and `HMAC` in `EXPLAIN actions`, `EXPLAIN header`, and `EXPLAIN PIPELINE` output when `format_display_secrets_in_show_and_select` is disabled (the default). [#108386](https://github.com/ClickHouse/ClickHouse/pull/108386) ([Raúl Marín](https://github.com/Algunenano)). * Fix `CREATE OR REPLACE` of a refreshable materialized view with a `TO` target: it was incorrectly rejected because the target table is already owned by the view being replaced. [#108392](https://github.com/ClickHouse/ClickHouse/pull/108392) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed a server crash (`Received signal 4`, illegal instruction) that could be triggered by a malformed or desynchronized native TCP protocol stream. The client-supplied initial address is now validated to be a numeric `host:port` before it is parsed, instead of letting a non-numeric port reach the trapped `getservbyname` libc function. [#108410](https://github.com/ClickHouse/ClickHouse/pull/108410) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash during shutdown when shutting down a database throws an exception (for example when a table's flush hits a ZooKeeper timeout). The rest of the shutdown sequence is no longer skipped, so system logs are flushed and their threads are joined before the server exits. [#108417](https://github.com/ClickHouse/ClickHouse/pull/108417) ([Groene AI](https://github.com/groeneai)). * Fixed regular expression functions (`match`, `extract`, `extractAll`, `replaceRegexpOne`, `replaceRegexpAll`, etc.) silently returning wrong results when the pattern contained a NUL (`\0`) byte. The NUL is now treated as an ordinary literal byte, consistent with RE2. [#108427](https://github.com/ClickHouse/ClickHouse/pull/108427) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `clickhouse-local` returning an empty reply to HTTP `OPTIONS` requests when no `http_options_response` is configured, which made the web UI (`/play`) show the connection as broken even though queries worked. [#108428](https://github.com/ClickHouse/ClickHouse/pull/108428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` ("Invalid action query tree node ...") exception when a distributed query both deduplicated structurally-identical duplicate-`ALIAS` columns and referenced a table function with a `key = value` named-collection argument (for example `oss(s3_conn, filename = '...')`). [#108435](https://github.com/ClickHouse/ClickHouse/pull/108435) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash (`LOGICAL_ERROR: 'Unexpected exception in refresh scheduling'`) that could happen on startup when a coordinated refreshable materialized view in a `Replicated` database was attached on a Keeper that does not support the `MULTI_READ` feature flag (for example after a Keeper downgrade). The view is now stopped gracefully instead of aborting the server. [#108441](https://github.com/ClickHouse/ClickHouse/pull/108441) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error (e.g. `Column _part not found`) when selecting virtual columns from a `MergeTree` table under parallel replicas, including via `SELECT *` with `asterisk_include_virtual_columns = 1`. [#108451](https://github.com/ClickHouse/ClickHouse/pull/108451) ([Groene AI](https://github.com/groeneai)). * `SYSTEM RESET DDL WORKER` now requires the new `SYSTEM RESET DDL WORKER` privilege. Previously any authenticated user (including `readonly` ones) could run it and repeatedly reset the DDL worker state, blocking `ON CLUSTER` DDL. [#108460](https://github.com/ClickHouse/ClickHouse/pull/108460) ([Groene AI](https://github.com/groeneai)). * Restrict the model path of `catboostEvaluate` to the `user_files` directory, like the `file` table function and the dictionary sources. Previously the function accepted an arbitrary filesystem path with no containment check, which allowed probing the existence of and triggering reads of files outside `user_files`. Models must now be located inside `user_files`. [#108463](https://github.com/ClickHouse/ClickHouse/pull/108463) ([Groene AI](https://github.com/groeneai)). * The `hasColumnInTable` function now requires the `SHOW COLUMNS` privilege on the target table, the same grant required by `DESCRIBE` and `SHOW CREATE TABLE`. Previously any user could call `hasColumnInTable` to probe column names and test table and database existence without any access check. [#108464](https://github.com/ClickHouse/ClickHouse/pull/108464) ([Groene AI](https://github.com/groeneai)). * Fixed excessive memory allocation when deserializing crafted aggregate function states for `mannWhitneyUTest`, `rankCorr`, `largestTriangleThreeBuckets`, `quantileGK`, `sequenceMatch`/`sequenceCount` and `groupArrayIntersect`. A malformed state could declare a huge element count and make the server try to allocate tens of gigabytes from a few bytes of input; such states are now rejected with `TOO_LARGE_ARRAY_SIZE`. [#108465](https://github.com/ClickHouse/ClickHouse/pull/108465) ([Groene AI](https://github.com/groeneai)). * Validate the redirect target of an S3 `301 Moved Permanently` response against `remote_url_allow_hosts` (`RemoteHostFilter`). Previously the AWS SDK 301 handling followed the attacker-supplied endpoint from the response (`Location` header or `` element) without the host check, so a malicious or compromised S3-compatible server could redirect ClickHouse to an arbitrary host (SSRF). The 307 redirect path already performed this check; both paths are now consistent. [#108466](https://github.com/ClickHouse/ClickHouse/pull/108466) ([Groene AI](https://github.com/groeneai)). * Fixed a credential leak where the legacy `storage_aws_access_key_id`, `storage_aws_secret_access_key`, `storage_catalog_credential`, and `storage_auth_header` settings of the `DataLakeCatalog` database engine were shown in plaintext in `system.databases.engine_full` and `SHOW CREATE DATABASE`. They are now redacted as `[HIDDEN]`. [#108470](https://github.com/ClickHouse/ClickHouse/pull/108470) ([Groene AI](https://github.com/groeneai)). * Fixed `ssl_certificate` user identification so a single `*` wildcard matches exactly one name component (RFC 6125 6.4.3). Previously a wildcard in a `CN` or `DNS:` SAN subject (for example `*.corp.example.com`) also matched multi-label names such as `evil.deep.corp.example.com`, letting a holder of a certificate for a deeper subdomain authenticate as the wildcard user. `URI:` SAN matching is unchanged. [#108472](https://github.com/ClickHouse/ClickHouse/pull/108472) ([Groene AI](https://github.com/groeneai)). * Fixed SQL injection in the MySQL wire protocol: the `SHOW TABLE STATUS LIKE` argument and the `SET ` value sent over the MySQL interface (port 9004) are now parsed and re-quoted instead of being concatenated verbatim into the rewritten query. Also fixed `KILL QUERY ` over the MySQL interface silently ignoring multi-digit connection ids. [#108474](https://github.com/ClickHouse/ClickHouse/pull/108474) ([Groene AI](https://github.com/groeneai)). * Sensitive values passed as HTTP query-string parameters (for example the `param_*` query-parameter binding such as `param_secret_key`/`param_aws_secret_access_key`, the `password` parameter, or S3-style `signature` parameters) are no longer written verbatim to `system.text_log` or OpenTelemetry spans. The `Request URI` log line now redacts the values of parameters whose names look sensitive, replacing them with `[HIDDEN]`. [#108475](https://github.com/ClickHouse/ClickHouse/pull/108475) ([Groene AI](https://github.com/groeneai)). * Enforce the `http_forbid_headers` configuration during schema inference for the `url` table function and `URL` storage engine. Previously a forbidden header was still sent over the network while inferring the schema (for example with the `Regexp` format or `DESCRIBE`), and its response body could be reflected back in a parse error, leaking secrets. The header filter is now checked before any network access on the inference path, matching the existing behavior of `urlCluster`. [#108476](https://github.com/ClickHouse/ClickHouse/pull/108476) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when using `SELECT ... SAMPLE ...` together with the query condition cache (setting `use_query_condition_cache`, enabled by default). [#108488](https://github.com/ClickHouse/ClickHouse/pull/108488) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash (stack overflow) caused by deeply nested expressions such as `[[[ ... ]]]` or `array(array( ... ))` when `max_parser_depth` is set to a large value. [#108493](https://github.com/ClickHouse/ClickHouse/pull/108493) ([Raúl Marín](https://github.com/Algunenano)). * Fixed the `RabbitMQ` table engine hanging indefinitely on `DROP TABLE` or server shutdown when the broker closes the AMQP connection due to missed heartbeats without sending a TCP RST. All blocking event loop calls now have a 30-second timeout so shutdown always completes. [#108497](https://github.com/ClickHouse/ClickHouse/pull/108497) ([Aly Ayman](https://github.com/alyelmorsy-dataengineer)). * Secondary queries executed as part of internal queries will now be logged as internal queries. [#108506](https://github.com/ClickHouse/ClickHouse/pull/108506) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * The `ArrowFlight` table engine and the `arrowFlight` table function now honor the `` allow-list. Previously, the connection host and port were not validated, so a query could reach hosts not present in the allow-list. [#108507](https://github.com/ClickHouse/ClickHouse/pull/108507) ([Groene AI](https://github.com/groeneai)). * The MySQL wire protocol commands `COM_FIELD_LIST` (`mysql_list_fields`) and `COM_INIT_DB` (`USE database`) now enforce the same access control as their SQL equivalents (`SHOW COLUMNS`/`DESCRIBE` and `USE`). Previously, they could disclose column names of tables the user only had partial column grants on, and switch the current database without the `SHOW DATABASES` privilege. [#108508](https://github.com/ClickHouse/ClickHouse/pull/108508) ([Groene AI](https://github.com/groeneai)). * Match `http_forbid_headers` case-insensitively. HTTP header names are case-insensitive, so forbidding `Authorization` now also blocks `authorization`, `AUTHORIZATION`, and other case variants. Configured `header_regexp` patterns are now matched case-insensitively without needing an explicit `(?i)` flag. [#108509](https://github.com/ClickHouse/ClickHouse/pull/108509) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash (segmentation fault) that could occur when a distributed query referenced a not-yet-materialized `MATERIALIZED` CTE as an external table and the remote source was created lazily. [#108547](https://github.com/ClickHouse/ClickHouse/pull/108547) ([Groene AI](https://github.com/groeneai)). * Fixed wrong results when the query condition cache (`use_query_condition_cache = 1`) reused a skip-index-derived mark exclusion for a query that ran a different set of skip indexes, for example with `use_skip_indexes = 0`, `ignore_data_skipping_indices`, or a different `use_skip_indexes_for_disjunctions` mode. Skip-index-derived cache entries are now keyed by the effective set of skip indexes that ran. [#108548](https://github.com/ClickHouse/ClickHouse/pull/108548) ([Groene AI](https://github.com/groeneai)). * Fixed several bugs in `changeYear`, `changeMonth`, `changeDay`, `changeHour`, `changeMinute` and `changeSecond` with `DateTime64` arguments: the result column was created with the hardcoded default scale 3 instead of the argument's scale, which led to a `LOGICAL_ERROR` (`writeSlice expects same column types`) when the result was passed to `arrayPushBack`/`arrayPushFront`/`arrayConcat`; nanosecond-precision (scale 9) inputs threw `DECIMAL_OVERFLOW`; and pre-epoch sub-second inputs returned the wrong calendar second. Also fixed `timeSlots` over `DateTime64` ignoring the scale of the optional `Size` argument in its declared return type and returning wrong timestamps for the largest scale in `Size`. [#108551](https://github.com/ClickHouse/ClickHouse/pull/108551) ([Groene AI](https://github.com/groeneai)). [#108681](https://github.com/ClickHouse/ClickHouse/pull/108681) ([Takumi Hara](https://github.com/takumihara)). [#108994](https://github.com/ClickHouse/ClickHouse/pull/108994) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error when a compound predicate is aliased in `GROUP BY` and referenced again, with `enable_identifier_resolve_cache` enabled (the default). [#108553](https://github.com/ClickHouse/ClickHouse/pull/108553) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Equal values are not contiguous within the range assumed to be sorted`) when running `DISTINCT` over a `STREAM` read (`SELECT DISTINCT ... FROM table STREAM`). Read-in-order optimizations are no longer applied to `STREAM` reads, which produce rows in commit order rather than sorting-key order. [#108568](https://github.com/ClickHouse/ClickHouse/pull/108568) ([Groene AI](https://github.com/groeneai)). * Fixed projections returning a column's type default (e.g. `0`) instead of its declared `DEFAULT` value (e.g. `-1`) in two cases: when reading a column that was added with `ALTER TABLE ... ADD COLUMN ... DEFAULT` after the projection was created (reads from the base table were already correct), and after the column's `TTL` expired on a wide part. [#108569](https://github.com/ClickHouse/ClickHouse/pull/108569) ([Shaohua Wang](https://github.com/tiandiwonder)). [#109013](https://github.com/ClickHouse/ClickHouse/pull/109013) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `ReadBuffer is canceled. Can't read from it.` error that could occur when an upload to S3 (for example a backup) is retried after a transient read error. [#108573](https://github.com/ClickHouse/ClickHouse/pull/108573) ([Groene AI](https://github.com/groeneai)). * Fix `ATTACH PARTITION ... FROM` rejecting tables whose primary keys are equivalent but declared differently (one explicit `PRIMARY KEY`, the other implicit from `ORDER BY`). [#108590](https://github.com/ClickHouse/ClickHouse/pull/108590) ([Jordi Orihuela](https://github.com/jordiori)). * Fixed direct read from the text index failing when multiple text indexes are partially materialized. Fixes [#108530](https://github.com/ClickHouse/ClickHouse/issues/108530). [#108607](https://github.com/ClickHouse/ClickHouse/pull/108607) ([Anton Popov](https://github.com/CurtizJ)). * Fix a metadata disclosure where `DESCRIBE loop('db', 'table')` and `DESCRIBE loop()` bypassed the `SHOW COLUMNS` / source access check, letting an unprivileged user read a table's column schema. [#108624](https://github.com/ClickHouse/ClickHouse/pull/108624) ([Groene AI](https://github.com/groeneai)). * Fixed the `_etag` virtual column for the `S3Queue` and `AzureQueue` table engines: it was declared but never populated, so `SELECT _etag` always returned an empty string. It now returns the object ETag, like the `S3` engine and the `s3` table function. [#108625](https://github.com/ClickHouse/ClickHouse/pull/108625) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Arrow`, `ArrowStream` and Arrow-based `ORC` files whose timestamp column carries a fixed numeric UTC offset (e.g. `+05:30`, `-08:00`, `00:00`) or the non-IANA marker `fixed` as its timezone, which previously failed with `Cannot load time zone ...` (`BAD_ARGUMENTS`). [#108633](https://github.com/ClickHouse/ClickHouse/pull/108633) ([Groene AI](https://github.com/groeneai)). * Fixed an exception (`LOGICAL_ERROR: Join is supported only for pipelines with one output port` or `NOT_IMPLEMENTED: MergeJoinAlgorithm is not implemented for strictness Semi`) when `join_algorithm = 'full_sorting_merge'` was used for a `SEMI`/`ANTI` join, including the joins produced by decorrelating an `EXISTS` correlated subquery. `full_sorting_merge` now declines strictness/kind combinations it cannot execute, so the planner falls back to another enabled algorithm or reports that the join cannot be executed. [#108658](https://github.com/ClickHouse/ClickHouse/pull/108658) ([Groene AI](https://github.com/groeneai)). * Fixed a heap buffer overflow when building a text index with `positions = 1` over many distinct short tokens. [#108659](https://github.com/ClickHouse/ClickHouse/pull/108659) ([Groene AI](https://github.com/groeneai)). * A query reading from or writing to S3 that is cancelled (e.g. by `KILL QUERY`) while an S3 request is in flight is now reported as cancelled instead of with a misleading network/S3 error. In particular, a cancelled backup or restore to S3 now reports `BACKUP_CANCELLED`/`RESTORE_CANCELLED` in `system.backups`. [#108673](https://github.com/ClickHouse/ClickHouse/pull/108673) ([Julia Kartseva](https://github.com/jkartseva)). * Fixed a startup failure where a `DataLakeCatalog` database created by an older version (25.12 or earlier) with a malformed `auth_header` could not be attached after upgrading to 26.2 or later, preventing the server from starting. The `auth_header` is now validated only on `CREATE`, and on `ATTACH` the catalog is built lazily on first use instead of during startup, so a single misconfigured or unreachable catalog database no longer blocks server startup. [#108674](https://github.com/ClickHouse/ClickHouse/pull/108674) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (`Sort order of blocks violated`) during a merge, and incorrect reads, when a non-nullable `LowCardinality` element of a `Nullable(Tuple(...))` column is used as a subcolumn (for example as a sort key). [#108679](https://github.com/ClickHouse/ClickHouse/pull/108679) ([Groene AI](https://github.com/groeneai)). * Fix a possible crash (use-after-free) when `SYSTEM FLUSH DISTRIBUTED` runs concurrently with `DROP TABLE` of the same `Distributed` table. [#108684](https://github.com/ClickHouse/ClickHouse/pull/108684) ([Groene AI](https://github.com/groeneai)). * Fix a crash in Base64 decoding of the Azure account key, reachable via `azureBlobStorage` (and related Azure table functions and storages) when the key is not valid Base64: a byte `>= 0x80` caused an out-of-bounds read, and any invalid byte caused undefined behaviour. Invalid keys are now rejected cleanly. [#108716](https://github.com/ClickHouse/ClickHouse/pull/108716) ([Groene AI](https://github.com/groeneai)). * Fix `NUMBER_OF_COLUMNS_DOESNT_MATCH` error when a subquery on a `Distributed` table reads two or more `ALIAS` columns that expand to the same expression (for example `a1 String ALIAS toString(x), a2 String ALIAS toString(x)`) and the subquery feeds an outer query, e.g. `SELECT count() FROM (SELECT a1, a2 FROM dist GROUP BY a1, a2)`. [#108725](https://github.com/ClickHouse/ClickHouse/pull/108725) ([Groene AI](https://github.com/groeneai)). * Fixed `CREATE OR REPLACE MATERIALIZED VIEW ... POPULATE` leaving the new view unsubscribed from its source table, which silently dropped every row inserted after the replace. [#108728](https://github.com/ClickHouse/ClickHouse/pull/108728) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an exception (`Cannot find sharding key column`, and in debug and sanitizer builds a server abort) when a `Distributed` table's sharding key is an expression the analyzer const-folds, for example `if(1, toInt32(id), toInt32(id) + 1)`, with `optimize_skip_unused_shards = 1`. [#108737](https://github.com/ClickHouse/ClickHouse/pull/108737) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (`Bad cast from type DB::ColumnNullable to DB::ColumnString`) and possible wrong results when using `group_by_use_nulls` with a `LowCardinality` constant grouping key in `GROUPING SETS`/`ROLLUP`/`CUBE`. [#108771](https://github.com/ClickHouse/ClickHouse/pull/108771) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed partition and primary key pruning being silently disabled when a `LowCardinality(FixedString)` (or `LowCardinality(Nullable(FixedString))`) key column is wrapped in a function in the key, for example `PARTITION BY sipHash64(k) % N` with `WHERE k = 'literal'`. Such queries scanned all partitions instead of pruning them. [#108777](https://github.com/ClickHouse/ClickHouse/pull/108777) ([Groene AI](https://github.com/groeneai)). * Fixed a logical error (a server abort in debug and sanitizer builds) when an `INSERT ... SELECT` through an `Alias` table, or into a `TimeSeries` table, is cancelled without an exception, for example with `timeout_overflow_mode = 'break'`. [#108783](https://github.com/ClickHouse/ClickHouse/pull/108783) ([Groene AI](https://github.com/groeneai)). [#108796](https://github.com/ClickHouse/ClickHouse/pull/108796) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix a `Bad cast from type DB::ColumnSparse to DB::ColumnString` logical error (a server abort in debug and sanitizer builds) when `groupConcat` is applied to a `Tuple` whose `String` element is stored sparse. [#108790](https://github.com/ClickHouse/ClickHouse/pull/108790) ([Groene AI](https://github.com/groeneai)). * Fix `TYPE_MISMATCH` in `mapFilter`, `mapSort`, `mapReverseSort`, `mapPartialSort` and `mapConcat` when a `Map` value contains a nested `Map` with `LowCardinality`, e.g. `mapFilter((k, v) -> 1, map('a'::LowCardinality(String), map('x'::LowCardinality(String), 'y')))`. [#108798](https://github.com/ClickHouse/ClickHouse/pull/108798) ([Groene AI](https://github.com/groeneai)). * Fixed `insert_quorum = 'auto'` not rejecting inserts up front when fewer than a majority of replicas were alive. Such inserts now fail immediately with `TOO_FEW_LIVE_REPLICAS` instead of writing a local part and later timing out with `UNKNOWN_STATUS_OF_INSERT`. [#108800](https://github.com/ClickHouse/ClickHouse/pull/108800) ([Gagan Dhakrey](https://github.com/gagandhakrey)). * Fixed a bug where the `WITH TOTALS` row of a `JOIN` could contain default values (such as `0`) instead of the constant values coming from a constant subquery on one side of the join. The wrong result appeared only for some join orderings or with the `query_plan_join_swap_table` setting. [#108807](https://github.com/ClickHouse/ClickHouse/pull/108807) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a segmentation fault (null-pointer dereference) that could occur when server shutdown ran concurrently with the shutdown of an `S3Queue`/`AzureQueue` table. [#108810](https://github.com/ClickHouse/ClickHouse/pull/108810) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed a startup abort (`Cannot allocate ThreadStack`, `EINVAL`) on glibc builds running on CPUs with a large signal-stack size (for example Intel Sapphire Rapids / Granite Rapids with an AMX-aware kernel), where the signal alt-stack size was not rounded up to a multiple of the page size. [#108813](https://github.com/ClickHouse/ClickHouse/pull/108813) ([Groene AI](https://github.com/groeneai)). * Fix distributed queries occasionally failing with `UNEXPECTED_PACKET_FROM_SERVER` (`expected TablesStatusResponse, got ProfileInfo`) when a connection that a previous cancelled query left out of sync was reused from the pool. [#108854](https://github.com/ClickHouse/ClickHouse/pull/108854) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a logical error (an exception in release builds, a server abort in debug and sanitizer builds) that could happen for a function call with more than one `LowCardinality` argument over a distributed table, for example `concatAssumeInjective((SELECT toLowCardinality('p')), s)` used as a `GROUP BY` key with the `remote` table function. The messages were `Default functions implementation for LowCardinality is supported only with a single LowCardinality argument` or `Expected the argument ... to have N rows, but it has M`. [#108871](https://github.com/ClickHouse/ClickHouse/pull/108871) ([Groene AI](https://github.com/groeneai)). * Fix `reinterpret(x, 'Decimal128(scale)')` (and `Decimal32`/`Decimal64`/`Decimal256`/`DateTime64` targets) producing a result column whose internal scale was the source scale instead of the requested target scale when the source and target had the same physical type. The values were correct, but the column object was structurally inconsistent with its declared type. [#108878](https://github.com/ClickHouse/ClickHouse/pull/108878) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`New empty part is about to materialize but the directory already exist`) that could abort the server in debug and sanitizer builds when a `DROP`/`DETACH`/`MOVE`/`REPLACE PARTITION` on a `MergeTree` table ran after a previous such operation was interrupted (for example a rolled-back transaction or a crash) and left a stale `tmp_empty_` directory behind. The stale directory is now reclaimed instead of failing. [#108879](https://github.com/ClickHouse/ClickHouse/pull/108879) ([Groene AI](https://github.com/groeneai)). * Fixed ClickHouse Keeper returning incorrect `dataLength = 0` in the `Stat` of `Create2` responses, and the ClickHouse ZooKeeper client not deserializing the `Stat` of `Create2` responses. [#108909](https://github.com/ClickHouse/ClickHouse/pull/108909) ([unintended](https://github.com/unintended)). * Fixed `clickhouse-local` rejecting an empty option value written adjacent to `=` (for example `--format_csv_null_representation=''`); it now behaves the same as `--format_csv_null_representation ""` and `SET format_csv_null_representation = ''`. [#108910](https://github.com/ClickHouse/ClickHouse/pull/108910) ([Vismay](https://github.com/vismaytiwari)). * Fixed a `LOGICAL_ERROR` (`Inconsistent AST formatting`) that aborted the server in debug and sanitizer builds when formatting a `Tuple` data type that mixes named and unnamed elements (for example `Tuple(a UInt8, UInt16)`). [#108915](https://github.com/ClickHouse/ClickHouse/pull/108915) ([Groene AI](https://github.com/groeneai)). * Fix a segmentation fault when merging `uniqExact` aggregate states with `GROUPING SETS`, `ROLLUP` or `CUBE` and `max_threads > 1`. [#108928](https://github.com/ClickHouse/ClickHouse/pull/108928) ([Raúl Marín](https://github.com/Algunenano)). * Fixed possible wrong results or out-of-bounds reads when an `INSERT` is rolled back after a mid-batch error (for example in `Buffer` tables, asynchronous inserts, or the `Kafka`/`RabbitMQ`/`FileLog` engines) while lazy column replication (`enable_lazy_columns_replication`) is in effect. [#108935](https://github.com/ClickHouse/ClickHouse/pull/108935) ([Groene AI](https://github.com/groeneai)). * `join_any_take_last_row` is now respected by all supported hash-based join paths, including joins that use automatic spilling to disk. [#108936](https://github.com/ClickHouse/ClickHouse/pull/108936) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed a bug in the analyzer where `FINAL` on one table of a `JOIN` (e.g. `FROM t1 FINAL JOIN t2`) was incorrectly applied to the other joined tables as well, which could make such queries slower. [#108979](https://github.com/ClickHouse/ClickHouse/pull/108979) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix a `LOGICAL_ERROR` ("Cannot find `__grouping_set` column in header of MergingAggregatedTransform with grouping sets", or "Chunk info was not set for chunk in MergingAggregatedTransform") that could occur, with the analyzer disabled, for a `UNION ALL`/`INTERSECT`/`EXCEPT` where one branch uses `GROUP BY GROUPING SETS` with parallel replicas and another branch uses `FINAL`. [#109003](https://github.com/ClickHouse/ClickHouse/pull/109003) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` in the automatic parallel replicas planner that could occur when `automatic_parallel_replicas_mode` is enabled together with `parallel_replicas_min_number_of_rows_per_replica` greater than 0. [#109011](https://github.com/ClickHouse/ClickHouse/pull/109011) ([Groene AI](https://github.com/groeneai)). * Fixed an excessive memory allocation during schema inference of the `MsgPack` format: a corrupted input whose array/map/string/binary header declares a huge element count no longer drives a single multi-gigabyte allocation and is rejected as malformed input. This fixes an out-of-memory condition in release builds (and an `allocation-size-too-big` abort under sanitizers). [#109019](https://github.com/ClickHouse/ClickHouse/pull/109019) ([Groene AI](https://github.com/groeneai)). * Fixed `use_client_time_zone` being ignored for `DateTime`/`DateTime64` string literals interpreted on the server (asynchronous `INSERT`, `SELECT` literals). The client now propagates its local time zone as `session_timezone` when `use_client_time_zone` is enabled, so server-side parsing matches the synchronous `INSERT` path. [#109051](https://github.com/ClickHouse/ClickHouse/pull/109051) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error when `query_plan_optimize_lazy_final` is enabled together with a `WHERE` filter partially pushed to `PREWHERE` (via `optimize_move_to_prewhere_if_final`). [#109073](https://github.com/ClickHouse/ClickHouse/pull/109073) ([Groene AI](https://github.com/groeneai)). * Fixed an error when using a qualified asterisk like `b.*` in a query where `b` is a table (or subquery/CTE) alias that shares its name with a non-compound column of another table. Such queries no longer fail and correctly select all columns of `b`. [#109078](https://github.com/ClickHouse/ClickHouse/pull/109078) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix a `Block structure mismatch in JoinStep` logical error (a server abort in debug and sanitizer builds) when a correlated subquery is decorrelated into a join and the source relation carries a column name more than once, e.g. `WITH t AS (SELECT number, * FROM numbers(3)) SELECT *, (SELECT t.number WHERE t.number >= 0) FROM t` with `correlated_subqueries_default_join_kind = 'right'`. [#109114](https://github.com/ClickHouse/ClickHouse/pull/109114) ([Groene AI](https://github.com/groeneai)). * Fixed a hang where a query reading through the filesystem cache could keep waiting on an in-progress download after being cancelled with `KILL QUERY`, and where dropping or `SYSTEM STOP VIEW`-ing a refreshable materialized view whose refresh was stuck in such a wait would block. [#109116](https://github.com/ClickHouse/ClickHouse/pull/109116) ([Murphy](https://github.com/murphy-4o)). * Fixed the `parallel_view_processing` setting being ignored when inserting into a table with materialized views — since 25.10, views always ran in parallel regardless of the setting. [#109170](https://github.com/ClickHouse/ClickHouse/pull/109170) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed a parser inconsistency where an `INSERT` column list accepted a qualified column matcher written as `t.* LIKE ''` / `t.* ILIKE ''` but rejected its canonical `t.COLUMNS('')` form, breaking the query format round-trip (and aborting the server in debug and sanitizer builds). [#109176](https://github.com/ClickHouse/ClickHouse/pull/109176) ([Groene AI](https://github.com/groeneai)). * Fixed non-monotonic `ProfileEvents` increments: the `NetworkReceiveBytes` and `AIOWriteBytes` counters could be incremented by a negative amount (wrapping around to a huge value) on a socket timeout or an AIO failure, and `MemoryOvercommitWaitTimeMicroseconds` was measured with a non-monotonic clock. [#109180](https://github.com/ClickHouse/ClickHouse/pull/109180) ([Azat Khuzhin](https://github.com/azat)). * Fix schema inference for the `Arrow`, `ArrowStream`, `Avro` formats and the legacy `ORC` and `Parquet` readers returning `Nullable(Tuple)` for nullable struct columns while the `Nullable(Tuple)` type is not allowed (`allow_experimental_nullable_tuple_type` is disabled). `DESCRIBE` returned a type that `CREATE TABLE` rejects, so creating a table or inserting data using the inferred schema failed with the error `Nullable Tuple type is not allowed`. [#109185](https://github.com/ClickHouse/ClickHouse/pull/109185) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed a wrong result of correlated `EXISTS` and scalar subqueries when the correlation appears only in the subquery projection together with a non-correlated `WHERE` clause (the subquery could evaluate to false / `NULL` for every row). Fixes [#105760](https://github.com/ClickHouse/ClickHouse/issues/105760). [#109186](https://github.com/ClickHouse/ClickHouse/pull/109186) ([Dmitry Novik](https://github.com/novikd)). * Fixed `CREATE USER ... HOST LIKE 'a', 'b'` (and the equivalent `ALTER USER`) silently keeping only the first pattern. All specified `HOST LIKE` patterns are now stored in the user's host allow-list. [#109187](https://github.com/ClickHouse/ClickHouse/pull/109187) ([Groene AI](https://github.com/groeneai)). * Fixed a text index defined on `mapValues(map)` or `mapKeys(map)` being silently not used when a table was queried through a `Distributed` engine table with the analyzer. The index was used for the local table and via the `cluster`/`remote` table functions, but a query through a `Distributed` engine table skipped it (and failed with `INDEX_NOT_USED` under `force_data_skipping_indices`). [#109188](https://github.com/ClickHouse/ClickHouse/pull/109188) ([Groene AI](https://github.com/groeneai)). * Fixed a false `LOGICAL_ERROR` (`There was an error on : Cannot obtain error message (probably it's a bug)`) that could abort the server in debug and sanitizer builds when running a DDL query on a `Replicated` database and its per-host `finished` status node in Keeper had already been cleaned up. [#109192](https://github.com/ClickHouse/ClickHouse/pull/109192) ([Groene AI](https://github.com/groeneai)). * Fix `SAMPLE` ratios with an exponent whose magnitude overflows `Int32` (e.g. `SAMPLE 1e-3000000000`) being silently treated as `SAMPLE 1`. [#109197](https://github.com/ClickHouse/ClickHouse/pull/109197) ([Raúl Marín](https://github.com/Algunenano)). * Fix the access check for `SYSTEM PREWARM PRIMARY INDEX CACHE ... ON CLUSTER`, which incorrectly required the `SYSTEM PREWARM MARK CACHE` privilege instead of `SYSTEM PREWARM PRIMARY INDEX CACHE`. [#109198](https://github.com/ClickHouse/ClickHouse/pull/109198) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `ALTER` queries failing for `MergeTree` tables created with a custom `disk` setting (`SETTINGS disk = disk(...)`). Fixes [#63019](https://github.com/ClickHouse/ClickHouse/issues/63019). [#109199](https://github.com/ClickHouse/ClickHouse/pull/109199) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fixed a server crash when states of the aggregate functions `cramersV`, `cramersVBiasCorrected`, `contingency` or `theilsU` from a window context (`OVER ()`) and from a plain aggregation are combined through chained set operations (`EXCEPT`, `INTERSECT`) or nested `UNION ALL` and then read, for example via the `-Merge` combinator. [#109200](https://github.com/ClickHouse/ClickHouse/pull/109200) ([Groene AI](https://github.com/groeneai)). * Fix a wrong result for `ANY JOIN` with a constant `ON` condition (e.g. `t1 ANY INNER JOIN t2 ON 1`): the query returned the full Cartesian product instead of the `ANY` join result. [#109207](https://github.com/ClickHouse/ClickHouse/pull/109207) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a server crash on asynchronous insert with deduplication when `optimize_on_insert` makes the inserted block empty (for example, rows summing to zero in `SummingMergeTree`). [#109229](https://github.com/ClickHouse/ClickHouse/pull/109229) ([Den Kalantaevskii](https://github.com/Diskein)). * Fix `SYSTEM DROP REPLICA` and `SYSTEM DROP DATABASE REPLICA ... FROM ZKPATH` to reject empty or root-only ZooKeeper paths at parse time with `BAD_ARGUMENTS`, route auxiliary Keeper paths to the named Keeper, and canonicalize Keeper paths so the local self-protection checks cannot mis-target or bypass the intended replica. [#109230](https://github.com/ClickHouse/ClickHouse/pull/109230) ([Groene AI](https://github.com/groeneai)). * Fix S3 settings priority so a URL-scoped `` endpoint block takes precedence over the top-level `` defaults. [#109251](https://github.com/ClickHouse/ClickHouse/pull/109251) ([Bharat Nallan](https://github.com/bharatnc)). * Fixed a bug where column `DEFAULT` values were not applied for `INSERT INTO TABLE FUNCTION` (for example `remote` or `file`) with inline `VALUES` data when the server parses the inline data itself (`send_table_structure_on_insert_with_inline_data = 0`). An explicit `NULL` inserted into a non-`Nullable` column with a `DEFAULT` became `0` instead of the declared default. It now behaves like a plain table `INSERT` and the HTTP protocol. [#109258](https://github.com/ClickHouse/ClickHouse/pull/109258) ([Groene AI](https://github.com/groeneai)). * Fix `CREATE TABLE ... AS SELECT` from `s3Cluster` (and other cluster table functions such as `fileCluster`/`urlCluster`) failing with `NOT_FOUND_COLUMN_IN_BLOCK` inside a `Replicated` database. [#109266](https://github.com/ClickHouse/ClickHouse/pull/109266) ([Groene AI](https://github.com/groeneai)). * Fixed `SHOW TABLES` and `SELECT ... FROM system.tables` returning different results for data lake catalog databases: tables that ClickHouse cannot read are now consistently hidden from both. [#109273](https://github.com/ClickHouse/ClickHouse/pull/109273) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fixed a bug where merging a `MergeTree` table that has projections while the `enable_block_number_column` or `enable_block_offset_column` setting is enabled produced projection parts containing a spurious `_block_number`/`_block_offset` column. The merged projection part then no longer matched the projection definition and the insert-produced projection parts, so `CHECK TABLE` and `OPTIMIZE ... DRY RUN` reported it as corrupted (`CORRUPTED_DATA`). [#109284](https://github.com/ClickHouse/ClickHouse/pull/109284) ([Groene AI](https://github.com/groeneai)). * Allow reading an Avro `array` of two-field `{key, value}` records into a ClickHouse `Map` column. This is the encoding Iceberg and Spark produce for a `MAP` with a non-string key (Avro native maps only support string keys). Previously such a file could be read as `Array(Tuple(key, value))` but not as `Map(K, V)`, which failed with `Type Map(...) is not compatible with Avro array`. [#109289](https://github.com/ClickHouse/ClickHouse/pull/109289) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash on `CREATE HYPOTHETICAL INDEX ... TYPE set` (and `ngrambf_v1`/`tokenbf_v1`) when the required index argument is omitted. Such statements are now rejected with a clear error. [#109294](https://github.com/ClickHouse/ClickHouse/pull/109294) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` (`Current component is empty`) that could be raised by mutation operations such as `KILL MUTATION` on a `MergeTree` table that stores its metadata in Keeper, when the `enforce_keeper_component_tracking` server setting is enabled. [#109297](https://github.com/ClickHouse/ClickHouse/pull/109297) ([Groene AI](https://github.com/groeneai)). * Fixed reading `ORC` timestamps beyond ~year 2262 into a `DateTime64` column with a scale coarser than 9. The native `ORC` reader used to convert every timestamp through a fixed `DateTime64(9)` intermediate, which overflows `Int64` at nanosecond scale and rejected such values with `VALUE_IS_OUT_OF_RANGE_OF_DATA_TYPE`, even when the requested `DateTime64` scale (for example `DateTime64(6)`, as produced by Iceberg) can represent the value. Timestamps are now read directly at the requested scale, and `date_time_overflow_behavior` is honored when even the target scale cannot hold the value. [#109302](https://github.com/ClickHouse/ClickHouse/pull/109302) ([Groene AI](https://github.com/groeneai)). * `clickhouse-local` acting as a server (after `SYSTEM START LISTEN HTTP`) now serves HTTP connections using the `TabSeparated` default output format, matching `clickhouse-server`, instead of the interactive `PrettyCompact` default. This fixes connecting to `clickhouse-local` over HTTP with drivers such as `clickhouse-connect`. The interactive terminal session still renders results as `PrettyCompact`. [#109362](https://github.com/ClickHouse/ClickHouse/pull/109362) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a transient `Code: 499 ... InvalidPart` (`S3_ERROR`) failure on S3 multipart uploads. MinIO can briefly report a just-uploaded part as missing on `CompleteMultipartUpload`; this error is now retried like the already-handled `NoSuchKey`, bounded by `s3_max_unexpected_write_error_retries`. [#109364](https://github.com/ClickHouse/ClickHouse/pull/109364) ([Groene AI](https://github.com/groeneai)). * Fix `UNKNOWN_IDENTIFIER` error on `ALTER TABLE ... DROP COLUMN` when another column has a `DEFAULT` or `MATERIALIZED` expression that defines and references an inline alias. [#109374](https://github.com/ClickHouse/ClickHouse/pull/109374) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a data race (and resulting rare crash) in the parallel merge of `uniqExact` aggregate states with two-level sets, which could occur with `GROUPING SETS`, `ROLLUP` and `CUBE`. [#109389](https://github.com/ClickHouse/ClickHouse/pull/109389) ([Groene AI](https://github.com/groeneai)). * Fixed an error (`NOT_FOUND_COLUMN_IN_BLOCK`, or `std::bad_function_call` in older versions) when running `CREATE TABLE ... ON CLUSTER ... AS SELECT` reading a `Distributed` table on a cluster with two or more shards. [#109407](https://github.com/ClickHouse/ClickHouse/pull/109407) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed merges for tables with `TTL ... GROUP BY` (rollup): parts that were already fully aggregated could be rescheduled for merging again and again indefinitely, and tables with `MATERIALIZED` columns or enabled persistent virtual columns were handled incorrectly — such columns are now aggregated with `any`. Closes [#105647](https://github.com/ClickHouse/ClickHouse/issues/105647). [#109410](https://github.com/ClickHouse/ClickHouse/pull/109410) ([Mikhail Artemenko](https://github.com/Michicosun)). [#109532](https://github.com/ClickHouse/ClickHouse/pull/109532) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fixed `IN` with a bare array column on the right argument silently returning a wrong (always-false) result or throwing an exception, so that `x IN arr` behaves like `has(arr, x)`. [#109416](https://github.com/ClickHouse/ClickHouse/pull/109416) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `UNKNOWN_IDENTIFIER` errors on `ALTER TABLE` operations (`RENAME`/`ADD`/`MODIFY COLUMN`, and `MATERIALIZE INDEX` on a table with freshly inserted parts) for `MergeTree` tables with an implicit min-max index over the persistent virtual columns `_block_number`/`_block_offset` (enabled by `add_minmax_index_for_block_number_column`/`add_minmax_index_for_block_offset_column`). [#109428](https://github.com/ClickHouse/ClickHouse/pull/109428) ([Groene AI](https://github.com/groeneai)). [#110236](https://github.com/ClickHouse/ClickHouse/pull/110236) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` in `arrayFold` over a non-const `Array(LowCardinality(T))` argument (for example `arrayFold((acc, x) -> acc + x, materialize([1, 2, 3]::Array(LowCardinality(Int64))), toInt64(0))`), which failed with `Arguments of 'plus' have incorrect data types` (a server abort in debug and sanitizer builds). [#109462](https://github.com/ClickHouse/ClickHouse/pull/109462) ([Groene AI](https://github.com/groeneai)). * Fixed `CHECK TABLE` on a `ReplicatedMergeTree` table reporting a healthy part as broken when a transient, retryable ZooKeeper error (e.g. connection loss) occurred during the check; such errors are now surfaced as a retryable query error instead. [#109465](https://github.com/ClickHouse/ClickHouse/pull/109465) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix segfaults in `groupArrayLastMerge` and the `largestTriangleThreeBuckets` aggregate function: deserialization of aggregate function states is now validated, so a broken state does not lead to an out-of-bounds memory access. [#109485](https://github.com/ClickHouse/ClickHouse/pull/109485) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). [#109492](https://github.com/ClickHouse/ClickHouse/pull/109492) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed the global `max_rows_in_join` / `max_bytes_in_join` limit not being enforced for the `parallel_hash` join algorithm, where a query with `join_overflow_mode = 'throw'` could silently succeed instead of raising `SET_SIZE_LIMIT_EXCEEDED`. [#109488](https://github.com/ClickHouse/ClickHouse/pull/109488) ([Hechem Selmi](https://github.com/m-selmi)). * Fixed reading Paimon tables partitioned by a `BIGINT` column whose value does not fit into `Int32`. Such partition values were truncated (for example `9223372036854775807` became `-1`), which produced a wrong partition path and failed with a filesystem error. [#109510](https://github.com/ClickHouse/ClickHouse/pull/109510) ([Groene AI](https://github.com/groeneai)). * Fix a crash when reading Iceberg tables with equality delete files. If a column is nullable in the equality delete file but non-nullable in the table schema (or vice versa), the values read from the delete file were inserted into a column of a different type through an unchecked cast (a column type confusion), corrupting the column and crashing the server. [#109551](https://github.com/ClickHouse/ClickHouse/pull/109551) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed the MySQL `connect_timeout`, read and write timeouts not taking effect: a connection attempt to an unresponsive MySQL server could hang for more than two minutes even with `connect_timeout = 1`, because the sampling query profiler's periodic signals kept resetting the MySQL client's internal poll deadline. [#109592](https://github.com/ClickHouse/ClickHouse/pull/109592) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a `LOGICAL_ERROR` (`creation_csn is not set while removal_csn is set to 1`) that could be thrown when a non-transactional `TRUNCATE` ran concurrently with an uncommitted transaction that had inserted into the same table. Such a `TRUNCATE` no longer removes parts created by not-yet-committed transactions. [#109598](https://github.com/ClickHouse/ClickHouse/pull/109598) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix `TYPE_MISMATCH` errors when reading `Parquet` struct columns whose physical nullability differs from the requested ClickHouse type: a non-nullable `Parquet` group can now be read as `Nullable(Tuple(...))`, and the native `Parquet` reader can read a physically nullable struct (a `Parquet` `OPTIONAL` group) as `Nullable(Tuple(...))`. [#109615](https://github.com/ClickHouse/ClickHouse/pull/109615) ([Groene AI](https://github.com/groeneai)). [#109898](https://github.com/ClickHouse/ClickHouse/pull/109898) ([Groene AI](https://github.com/groeneai)). * Fixed silent data loss when an `INSERT ... VALUES` in a multi-query stream is followed by a trailing SQL comment (for example `VALUES (1) -- comment`) under the server-side inline insert parsing path (`send_table_structure_on_insert_with_inline_data = 0`). The trailing comment was scanned as row data past the terminating `;`, causing the following queries in the stream to be silently skipped. [#109643](https://github.com/ClickHouse/ClickHouse/pull/109643) ([Groene AI](https://github.com/groeneai)). * Fix a segfault when a query arrives in a narrow window during server shutdown after an error at startup. [#109675](https://github.com/ClickHouse/ClickHouse/pull/109675) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed stale skip indices (`text`, `bloom_filter`, etc.) and projections after materializing lightweight updates with `ALTER TABLE ... APPLY PATCHES`. Previously the index and projection files that depend on a patch-updated column were left unchanged, so queries using them could return wrong results (e.g. `hasToken` missing an updated row, or a projection returning stale aggregates once the spent patch part was removed). [#109709](https://github.com/ClickHouse/ClickHouse/pull/109709) ([Groene AI](https://github.com/groeneai)). * Fix `system.tables` silently skipping databases for users with per-database grants when the query reads only the `name`/`database` columns. Introduced in 26.2. [#109723](https://github.com/ClickHouse/ClickHouse/pull/109723) ([Samay Sharma](https://github.com/samay-sharma)). * Fix a `LOGICAL_ERROR` when writing a `LowCardinality(Time)` column to the `Arrow` format with `output_format_arrow_low_cardinality_as_dictionary = 1`. [#109730](https://github.com/ClickHouse/ClickHouse/pull/109730) ([Groene AI](https://github.com/groeneai)). * Fix `ObjectStorageQueue`/`AzureQueue` (and the `azureBlobStorage` table function) silently skipping objects when the object storage returns an empty listing page together with a continuation token. Azure Blob Storage does this (e.g. when a listing crosses an internal partition boundary), and the async list iterator was treating the empty page as the end of the listing and dropping the token. It now follows the continuation token. [#109761](https://github.com/ClickHouse/ClickHouse/pull/109761) ([Gal Ben Moshe](https://github.com/GalBenMoshe)). * Fixed partition pruning returning no rows for Iceberg tables partitioned by a timestamp (`DateTime64`) column. [#109764](https://github.com/ClickHouse/ClickHouse/pull/109764) ([Den Kalantaevskii](https://github.com/Diskein)). * Fix an `ILLEGAL_COLUMN` exception in the `conv` function for `FixedString` arguments; they are now correctly cast to `String`. Closes [#109670](https://github.com/ClickHouse/ClickHouse/issues/109670). [#109771](https://github.com/ClickHouse/ClickHouse/pull/109771) ([hp](https://github.com/hp77-creator)). * The `getClientHTTPHeader` function now treats header names as case-insensitive, according to [RFC 9110](https://www.rfc-editor.org/info/rfc9110/#section-5.1); in particular, the `authorization` header is now filtered out regardless of case. Closes [#103957](https://github.com/ClickHouse/ClickHouse/issues/103957). [#109791](https://github.com/ClickHouse/ClickHouse/pull/109791) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix `max_execution_time` (with `timeout_overflow_mode = 'throw'`) sometimes never cancelling a query: when the internal timeout watcher was already waiting for a query with a later deadline, a query with an earlier deadline registered afterwards could be missed entirely, letting it run long past its time limit. [#109792](https://github.com/ClickHouse/ClickHouse/pull/109792) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed the file name of gzip-compressed Iceberg metadata files. ClickHouse wrote them as `v{N}.gzip.metadata.json` (the HTTP `Content-Encoding` token), while the Iceberg spec expects the `gz` extension `v{N}.gz.metadata.json`. As a result Spark and other Hadoop-catalog readers could not find the metadata written by ClickHouse. ClickHouse now writes `v{N}.gz.metadata.json` and still reads the legacy `gzip` name for backward compatibility. [#109812](https://github.com/ClickHouse/ClickHouse/pull/109812) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_IMPLEMENTED` error ("Method getDataAt is not supported for Nullable(String)") that could be thrown by a join with `enable_join_runtime_filters = 1` (on by default since 26.2) when the build-side join key was `LowCardinality(Nullable(...))` with `NULL` values and the runtime filter fell back to its bloom filter. [#109824](https://github.com/ClickHouse/ClickHouse/pull/109824) ([Groene AI](https://github.com/groeneai)). * External database engines (e.g. `PostgreSQL`) no longer push down range comparisons (`>=`, `>`, `<=`, `<`) on `UUID` columns. ClickHouse and external databases order UUIDs differently, so pushing these down silently dropped rows from the result; such predicates are now evaluated by ClickHouse instead. [#109833](https://github.com/ClickHouse/ClickHouse/pull/109833) ([Vismay](https://github.com/vismaytiwari)). * Fixed a server crash when a lambda expression was passed where a higher-order function expects a concrete value (for example the accumulator of `arrayFold`, as in `arrayFold(lambda, arr, another_lambda)`). Such queries are now rejected with `ILLEGAL_TYPE_OF_ARGUMENT` instead of crashing when `enable_analyzer = 0`. [#109840](https://github.com/ClickHouse/ClickHouse/pull/109840) ([Groene AI](https://github.com/groeneai)). * Fixed the `icebergLocal` table function and `IcebergLocal` engine declaring `azure` instead of `local` as their object storage type, which made them fail the disk type check (`Disk type doesn't match`) when used with a local disk via `SETTINGS disk = '...'`. [#109872](https://github.com/ClickHouse/ClickHouse/pull/109872) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix reading Iceberg tables partitioned by the same source column more than once (for example `PARTITIONED BY (hours(ts), ts)`). Such tables previously failed with `Cannot add column ...: column with this name already exists (ILLEGAL_COLUMN)`. [#109895](https://github.com/ClickHouse/ClickHouse/pull/109895) ([Groene AI](https://github.com/groeneai)). * Fix wrong query results caused by the primary key index incorrectly pruning granules for tables with a reversed (`DESC`) key column when parts have no final mark (non-adaptive granularity, `index_granularity_bytes = 0`). [#109901](https://github.com/ClickHouse/ClickHouse/pull/109901) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed `generateRandomStructure` occasionally producing an invalid structure string with two data types concatenated (for example `Decimal32(7)IPv4`) when type nesting exceeded the internal depth limit, which made the result unparseable. [#109928](https://github.com/ClickHouse/ClickHouse/pull/109928) ([Groene AI](https://github.com/groeneai)). * Fixed `NOT_IMPLEMENTED` error (`Method getDataAt is not supported for Nullable(String) in case if value is NULL`) when a `text` index is built on `Array(LowCardinality(Nullable(String)))` (including `Nested` fields stored that way) and an indexed array contains a `NULL` element. `NULL` array elements are now skipped during index construction, matching `Array(Nullable(String))`. [#110055](https://github.com/ClickHouse/ClickHouse/pull/110055) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when a `minmax` skip index is built on a `LowCardinality(Nullable(...))` column: `WHERE`/`PREWHERE`/`HAVING ... IS NULL` predicates pushed to storage previously pruned every granule and returned 0 rows even though the column contained `NULL` values. [#110061](https://github.com/ClickHouse/ClickHouse/pull/110061) ([Groene AI](https://github.com/groeneai)). * Fix `distinct_overflow_mode = 'break'`: `DISTINCT` now returns the partial result accumulated up to the limit and stops reading the source, as documented. Previously the chunk that crossed `max_rows_in_distinct` / `max_bytes_in_distinct` was discarded (truncated or empty results) and the query kept reading and inserting into the hash set to the end of the input, which could end in `MEMORY_LIMIT_EXCEEDED`. [#110075](https://github.com/ClickHouse/ClickHouse/pull/110075) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fixed a logical error `ChunkInfoRowNumbers does not exist` on `OPTIMIZE TABLE` of an Iceberg table containing a data file in a non-Parquet format (e.g. `ORC`) newer than all position delete files. [#110107](https://github.com/ClickHouse/ClickHouse/pull/110107) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `max_bytes_in_distinct` and `max_bytes_in_set`: string keys are stored in an arena that was not counted by the limit checks, so `DISTINCT` / `IN` over string keys could hold memory exceeding the byte limit by the whole key payload (unbounded in the key length). The limits now account for the arena, matching their documentation; queries with string keys close to a byte limit may now trip it earlier (correctly). [#110120](https://github.com/ClickHouse/ClickHouse/pull/110120) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)). * Fix a `LOGICAL_ERROR` (`Bad cast from type ColumnLowCardinality to ColumnString`) when `identity` (or a scalar subquery result) wraps a value containing a nested `LowCardinality` and the query uses `WITH TOTALS`/`WITH ROLLUP`. [#110138](https://github.com/ClickHouse/ClickHouse/pull/110138) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Npy` files with zero-sized inner dimensions: the number of materialized rows and the optimized and non-optimized `count` results now agree. [#110146](https://github.com/ClickHouse/ClickHouse/pull/110146) ([Yanjun Qiu](https://github.com/qiuyanjun888)). * Fixed a quadratic-time blowup (and unresponsiveness to `max_execution_time`) in aggregation in order when grouping by multiple keys whose sort order is only a prefix of the grouping key. [#110159](https://github.com/ClickHouse/ClickHouse/pull/110159) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error (`Unexpected number of columns in result sample block`) when a filter containing a correlated subquery (for example `exists((SELECT ...))`) is optimized with `convert_query_to_cnf` or `optimize_and_compare_chain`. [#110187](https://github.com/ClickHouse/ClickHouse/pull/110187) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an exception (`UNION mode UNION_DEFAULT must be normalized`) when a `DELETE` or `ALTER UPDATE` mutation used a set operation (`UNION`/`UNION ALL`/`UNION DISTINCT`/`EXCEPT`/`INTERSECT`) inside a subquery in its `WHERE` condition or in an `UPDATE` assignment. [#110196](https://github.com/ClickHouse/ClickHouse/pull/110196) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `ReadBuffer is canceled. Can't read from it.` logical error (server abort in debug/sanitizer builds) that could occur while reading a zip archive (e.g. during `RESTORE` from a zip backup) after a prior read from the same archive failed mid-stream. [#110197](https://github.com/ClickHouse/ClickHouse/pull/110197) ([Groene AI](https://github.com/groeneai)). * Fixed slow server shutdown and unresponsive `KILL MUTATION` when a mutation with `x IN (subquery)` was building the subquery set during primary-key analysis; the set build is now cancelled promptly. [#110198](https://github.com/ClickHouse/ClickHouse/pull/110198) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Keeper now rejects the coordination settings `max_requests_batch_size` and `max_requests_append_size` being set to `0` instead of getting stuck in an infinite append-entries loop in a multi-node setup. [#110200](https://github.com/ClickHouse/ClickHouse/pull/110200) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `NOT_FOUND_COLUMN_IN_BLOCK` error when using `GROUP BY ALL` over a tuple expression together with `ORDER BY`. [#110206](https://github.com/ClickHouse/ClickHouse/pull/110206) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `NOT_FOUND_COLUMN_IN_BLOCK` error on `INSERT` into a table that has a `CHECK` constraint referencing a subcolumn (such as `x.null` of a `Nullable` column or `arr.size0` of an `Array`). [#110208](https://github.com/ClickHouse/ClickHouse/pull/110208) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix reading Iceberg tables whose default sort order references a column that needs quoting (e.g. `@timestamp`). Such tables were unreadable because the synthesized storage `ORDER BY` was built from the raw column name and failed to parse with `SYNTAX_ERROR`. [#110233](https://github.com/ClickHouse/ClickHouse/pull/110233) ([Groene AI](https://github.com/groeneai)). * Fixed `system.tables` for a `DataLakeCatalog` (Iceberg/Glue) database aborting the whole query, or silently dropping tables, when a single table's metadata is unresolvable. Such a table now stays listed by name with default/NULL values for the columns that need the opened storage object (`engine`, `total_rows`, etc.), regardless of `database_datalake_require_metadata_access`. Direct access to the broken table still reports the error. [#110242](https://github.com/ClickHouse/ClickHouse/pull/110242) ([Groene AI](https://github.com/groeneai)). * Fixed a possible crash (heap-buffer-overflow) when a `quantileTDigest`-family aggregate-function state column was used as a `GROUP BY` key and serialized concurrently by several threads. [#110263](https://github.com/ClickHouse/ClickHouse/pull/110263) ([Groene AI](https://github.com/groeneai)). * Fixed `INSERT` into a Microsoft Fabric / OneLake `DataLakeCatalog` table failing with `IncorrectEndpointError` (HTTP 400) by routing ADLS Gen2 (DFS) writes to the `.dfs` endpoint host instead of the `.blob` host. Note: with `remote_url_allow_hosts`, both the `.blob` (read) and `.dfs` (write) Fabric hosts must be allow-listed for `INSERT`. [#110290](https://github.com/ClickHouse/ClickHouse/pull/110290) ([Mohammad Lareb Zafar](https://github.com/zlareb1)). * Fixed a server abort when parsing certain PRQL queries (`SET dialect = 'prql'`). A panic inside the `prqlc` compiler was aborting the process instead of being reported as a query error. [#110316](https://github.com/ClickHouse/ClickHouse/pull/110316) ([Groene AI](https://github.com/groeneai)). * Fix silently wrong query results when a projection part written before the projection's column set changed (for example, after upgrading across versions that materialize an `ALIAS` column's source differently, or after `ALTER TABLE ... MODIFY COLUMN` re-points an `ALIAS` column used by a projection) was read through the projection or merged: the missing column was filled with default values instead of the real data. Such parts are now read from the base table, and merges rebuild the projection from the base data. [#110328](https://github.com/ClickHouse/ClickHouse/pull/110328) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed `ORDER BY ... WITH FILL` not respecting `max_execution_time` and being slow to cancel when generating a large fill range (especially with `INTERPOLATE`). [#110332](https://github.com/ClickHouse/ClickHouse/pull/110332) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error (a server abort in debug/sanitizer builds) when a multi-command `ALTER` such as `UPDATE ..., DELETE ...` was issued on an Iceberg table. Such mutations are now rejected with a clear `NOT_IMPLEMENTED` error. [#110347](https://github.com/ClickHouse/ClickHouse/pull/110347) ([Groene AI](https://github.com/groeneai)). * Fixed a parser bug where `COMMENT` was incorrectly consumed as an implicit alias when a `SELECT` query ended directly after a bare table identifier (e.g. `... FROM t COMMENT 'x'`), causing a misleading syntax error instead of the comment being applied to the view/table. [#110372](https://github.com/ClickHouse/ClickHouse/pull/110372) ([Aditya Kumar](https://github.com/adityaksolves)). * Fixed a server crash (native stack overflow) when a deeply nested `Array`/`Tuple`/`Map`/`Object` literal is copied or destroyed, for example a query with a very deeply nested literal at a raised `max_parser_depth`. [#110393](https://github.com/ClickHouse/ClickHouse/pull/110393) ([Raúl Marín](https://github.com/Algunenano)). * Fixed two cases where vector search could return fewer rows than expected: `vector_search_index_fetch_multiplier` values below 1.0 could truncate the computed fetch count to zero and produce an empty result (such values are now rejected), and the vector search optimization is now skipped for `LIMIT ... WITH TIES` queries, because the optimization bounds the search to exactly N candidates and dropped rows tied with the N-th row. [#110452](https://github.com/ClickHouse/ClickHouse/pull/110452) ([Tamish Mhatre](https://github.com/tamish560)). [#110453](https://github.com/ClickHouse/ClickHouse/pull/110453) ([Tamish Mhatre](https://github.com/tamish560)). * Fix the logical error `Expected CommonSubplanReferenceStep to reference CommonSubplanStep`, the error `Subplan cannot be used to build pipeline`, and the logical error `Trying to extract chunk from ChunkBuffer before all inputs are finished` for `IN (subquery)` where the subquery contains a correlated subquery and the set is built during index analysis. [#110491](https://github.com/ClickHouse/ClickHouse/pull/110491) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix substitution of query parameters inside the definitions of a named `WINDOW` clause. Previously, parameters there were silently kept unsubstituted, and an unset `Identifier` parameter could lead to the exception `Logical error: '!part.empty()'` during `EXPLAIN SYNTAX`. [#110506](https://github.com/ClickHouse/ClickHouse/pull/110506) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed server startup failure for `MergeTree` tables on object storage disks when a leftover `txn_version.txt.tmp` file has broken disk-level metadata. [#110519](https://github.com/ClickHouse/ClickHouse/pull/110519) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error in the filesystem cache (`Expected file ... not to exist`) that could occur when a background cache download failed with a non-filesystem error (for example while running `OPTIMIZE` on an Iceberg table), leaving an empty orphan cache file behind. [#110549](https://github.com/ClickHouse/ClickHouse/pull/110549) ([Groene AI](https://github.com/groeneai)). * Fixed reading a corrupted `Native`-format stream whose `Dynamic` type count or `JSON`/`Object` path count is close to `SIZE_MAX`: such malformed input is now rejected with a clear `INCORRECT_DATA` error instead of an uncaught `std::length_error`. [#110590](https://github.com/ClickHouse/ClickHouse/pull/110590) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server crash in the `MaterializedPostgreSQL` database engine that could happen when a table was detached (or its structure changed) while it was still queued for synchronization. [#110596](https://github.com/ClickHouse/ClickHouse/pull/110596) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `SET param_` in the client (and `--param_` flags) misbehaving when the parameter name matches a builtin setting name such as `limit`, `offset`, `max_threads`, or `log_comment`. Depending on the colliding setting, the query failed with `CANNOT_PARSE_QUOTED_STRING`, the value was silently normalized (e.g. `max_threads=0` became `auto(N)`), or a setting alias name was lost. Such a parameter also broke every subsequent `SET param_*` in the same session. [#110597](https://github.com/ClickHouse/ClickHouse/pull/110597) ([Raúl Marín](https://github.com/Algunenano)). * Fix reading subcolumns of a column that has a `DEFAULT` expression and is not materialized in a part (e.g. after `ALTER TABLE ADD COLUMN` or during `ALTER TABLE MATERIALIZE COLUMN`): the subcolumn was filled with type default values instead of the evaluated `DEFAULT` expression, and the transposed vector distance functions (`cosineDistanceTransposed` and others) on such `QBit` columns failed with the `SIZES_OF_ARRAYS_DONT_MATCH` error. This closes [#110634](https://github.com/ClickHouse/ClickHouse/issues/110634). [#110636](https://github.com/ClickHouse/ClickHouse/pull/110636) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` `Invalid partition key size: 0` that could fail an `INSERT` into a partitioned `MergeTree` table with `non_replicated_deduplication_window` enabled, when an async insert batch was only partially deduplicated (some `insert_deduplication_token` values were duplicates and some were new). [#110651](https://github.com/ClickHouse/ClickHouse/pull/110651) ([Groene AI](https://github.com/groeneai)). * A background asynchronous insert flush now stops promptly when killed with `KILL QUERY`; previously a flush of a large buffered payload kept parsing to the end and ignored the cancellation. [#110652](https://github.com/ClickHouse/ClickHouse/pull/110652) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed wrong results from `GROUP BY` with `optimize_aggregation_in_order` and `DISTINCT` with `optimize_distinct_in_order` placed over a `partial_merge` JOIN. The read-in-order optimization was propagated through the partial-merge join, which re-sorts its left input by the join key and therefore does not preserve the left stream's original order, so rows were grouped incorrectly. [#110671](https://github.com/ClickHouse/ClickHouse/pull/110671) ([Groene AI](https://github.com/groeneai)). * Fixed a signed integer overflow in `toStartOfInterval` with an extreme `MONTH`, `WEEK`, or `DAY` interval count on `Date`/`Date32`/`DateTime64` values. [#110688](https://github.com/ClickHouse/ClickHouse/pull/110688) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error for queries with a subquery in `FROM` when parallel replicas run in `custom_key_sampling` / `custom_key_range` mode. Such queries (e.g. `SELECT * FROM (SELECT id, k, v FROM t WHERE id < 20) ORDER BY k`) previously failed with `Not found column __table2.id in block ...`. [#110690](https://github.com/ClickHouse/ClickHouse/pull/110690) ([Groene AI](https://github.com/groeneai)). * Fixed `readWKT` rejecting WKT strings with leading whitespace (e.g. `readWKT(' POINT(1 2)')`), which are accepted by the typed `readWKTPoint`/`readWKTPolygon`/... readers and by the WKT grammar. [#110706](https://github.com/ClickHouse/ClickHouse/pull/110706) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Column ... already added for reading`, a server abort in debug/sanitizer builds) when the same text-index predicate is used in both `PREWHERE` and `WHERE` of a query over a `Merge` table on top of a `Distributed` table. [#110710](https://github.com/ClickHouse/ClickHouse/pull/110710) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error for the transposed distance functions over `QBit` columns (such as `cosineDistanceTransposed`) called with a scalar-subquery or constant reference vector when parallel replicas are enabled. This closes [#110719](https://github.com/ClickHouse/ClickHouse/issues/110719). [#110729](https://github.com/ClickHouse/ClickHouse/pull/110729) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a rare `LOGICAL_ERROR` (`Unexpected number of parts to remove from parts_queue`; a server abort in debug/sanitizer builds) on `ReplicatedMergeTree` when two partition operations (for example two `MOVE PARTITION`/`REPLACE PARTITION` queries, or one racing a background `DROP_RANGE`) cancelled the background part check over overlapping ranges at the same time. [#110738](https://github.com/ClickHouse/ClickHouse/pull/110738) ([Groene AI](https://github.com/groeneai)). * Fixed a `Hash is not set for serialization` logical error that could occur when a column with a `Quantized(...)` codec was wrapped inside another serialization, for example in a `Merge` table over sources whose same-named column has different types (merged into a `Variant`). [#110776](https://github.com/ClickHouse/ClickHouse/pull/110776) ([Groene AI](https://github.com/groeneai)). * Fix a server hang when multiplying a `median`/`quantile` aggregate function state by a huge integer constant (e.g. `medianState(x) * 18446744073709551615`). The query became unresponsive to cancellation and could run indefinitely. [#110779](https://github.com/ClickHouse/ClickHouse/pull/110779) ([Groene AI](https://github.com/groeneai)). * Fix a server crash (integer division by zero) when a quota with a zero-length interval (for example `CREATE QUOTA q FOR INTERVAL 0 SECOND MAX queries = 1000`) was consumed. A non-positive quota interval duration is now rejected at quota creation. [#110846](https://github.com/ClickHouse/ClickHouse/pull/110846) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixed `throwIf(notLike(col, pattern))` (and other `throwIf` over a function) throwing unconditionally when `col` is `LowCardinality`. The default `LowCardinality` implementation ran `throwIf` on the whole dictionary, which always holds the reserved default value even when no row references it, so a non-zero value in that unused slot made `throwIf` throw for data that does not satisfy the condition. [#110864](https://github.com/ClickHouse/ClickHouse/pull/110864) ([Groene AI](https://github.com/groeneai)). * Propagate a query's active roles (`SET ROLE` or the `role` parameter) to remote nodes on parallel-replica and `Distributed` reads over a cluster secured with an interserver secret; previously the remote nodes fell back to the user's default roles, evaluating row policies inconsistently with the initiator. [#110867](https://github.com/ClickHouse/ClickHouse/pull/110867) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed Iceberg tables producing duplicate field IDs after `ALTER TABLE ... ADD COLUMN` when the initial schema contains nested fields (`Tuple`/`Array`/`Map`). `last-column-id` now records the maximum assigned field ID including nested children, so a subsequently added column no longer reuses a nested field ID and reads no longer fail with `ICEBERG_SPECIFICATION_VIOLATION` (`Duplicate field id`). [#110884](https://github.com/ClickHouse/ClickHouse/pull/110884) ([Groene AI](https://github.com/groeneai)). * Fixed a crash in FIPS builds when using an Ed25519 SSH key (`CREATE USER ... IDENTIFIED WITH ssh_key ... TYPE 'ssh-ed25519'`, or such a key in `users.xml`). Ed25519 is not FIPS-approved; such keys are now rejected with a clear `LIBSSH_ERROR` error. [#110891](https://github.com/ClickHouse/ClickHouse/pull/110891) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix the reduced-precision results of the transposed distance functions over `QBit` columns (`cosineDistanceTransposed`, `L2DistanceTransposed`, `dotProductTransposed`). Values truncated to `precision` bit planes were reconstructed with the dropped bits zero-filled, which biased every reconstructed value towards zero and was degenerate at low precision: for `QBit(BFloat16)` at precision 1 only the sign bit survives, so every value reconstructed to `±0.0` and the distance was the same constant for every row, carrying no ranking information. Truncated values are now reconstructed to a bounded midpoint of the remaining precision cell (raw `Int8` codes to their cell centre; exact zeros and infinities are preserved), so a smaller `precision` trades accuracy for speed without wildly biased results. Full-precision results are unchanged. This closes [#110898](https://github.com/ClickHouse/ClickHouse/issues/110898). [#110911](https://github.com/ClickHouse/ClickHouse/pull/110911) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a server crash (use-after-free) during graceful shutdown after Iceberg tables were queried from sessions that also used temporary tables. [#110914](https://github.com/ClickHouse/ClickHouse/pull/110914) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed a `LOGICAL_ERROR: Cannot sum Bools` (a server abort in debug/sanitizer builds) when merging `sumMap`/`sumMapWithOverflow`/`sumMapFiltered` aggregate states with `Bool` values that were serialized in the old (version 0) state format. The same fix also corrects two silent wrong-result cases with version-0 `Bool` states: map keys of type `Bool` were not deduplicated across states, and zero-value compaction dropped the wrong entries. [#110922](https://github.com/ClickHouse/ClickHouse/pull/110922) ([Groene AI](https://github.com/groeneai)). * Fixed a crash when an aggregate function with the `-Tuple` combinator was used with `RESPECT NULLS` and produced an intermediate state (`-State`, distributed aggregation, `WITH ROLLUP`/`WITH CUBE`). The combined function was named after the wrong state variant, so its serialized `AggregateFunction` type re-resolved to an incompatible in-memory layout on a round-trip. [#110930](https://github.com/ClickHouse/ClickHouse/pull/110930) ([Groene AI](https://github.com/groeneai)). * Fix `PREWHERE` (and row-level policies) on an Iceberg column that was renamed by schema evolution silently returning `0` rows for old data files. Such queries now return the correct rows, matching `WHERE`. [#110941](https://github.com/ClickHouse/ClickHouse/pull/110941) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`LazyMaterializingTransform: Number of rows in lazy chunk N does not match number of offsets M`) that could happen for vector search queries with `vector_search_with_rescoring = 1` combined with lazy materialization when several distances tie. [#111003](https://github.com/ClickHouse/ClickHouse/pull/111003) ([Groene AI](https://github.com/groeneai)). * Fixed reading `Arrow`/`ArrowStream` data with empty nested `Array`/`Map` columns produced by Apache Arrow Java < 19.0.0 (bundled with Apache Spark), which were previously rejected with an `INCORRECT_DATA` error about the offsets buffer being too small. [#111101](https://github.com/ClickHouse/ClickHouse/pull/111101) ([Raúl Marín](https://github.com/Algunenano)). * Fix stale reads on `plain_rewritable` disks with the page cache enabled. [#111105](https://github.com/ClickHouse/ClickHouse/pull/111105) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix `CANNOT_CONVERT_TYPE` error for constants of `Variant` type (including `Geometry`) in distributed queries and queries with parallel replicas. [#111136](https://github.com/ClickHouse/ClickHouse/pull/111136) ([Nikita Fomichev](https://github.com/fm4v)). * Fix a bug where reading large files from HDFS could fail or read incorrectly when a single read request exceeded the `hdfsPread` 32-bit size limit (`INT_MAX`). [#98470](https://github.com/ClickHouse/ClickHouse/pull/98470) ([Arup Chauhan](https://github.com/arup-chauhan)). * Fix incorrect use of the text index for equality comparisons with an empty needle. [#108340](https://github.com/ClickHouse/ClickHouse/pull/108340) ([Robert Schulze](https://github.com/rschu1ze)). * Fixed a hang in `SYSTEM SYNC MERGES` with the `Manual` merge selector when a scheduled merge could not be placed into a full background pool. [#108770](https://github.com/ClickHouse/ClickHouse/pull/108770) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ATTACH TABLE ... AS REPLICATED` discarding committed data and resurrecting pre-mutation rows when a stale `txn_version.txt.tmp` file was left on a data part. [#108772](https://github.com/ClickHouse/ClickHouse/pull/108772) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a possible race condition in `ALTER TABLE ... MOVE PARTITION TO TABLE` with non-replicated `MergeTree` tables that could produce intersecting data parts, because the commit of moved parts and block number allocation were not under the same lock. [#108922](https://github.com/ClickHouse/ClickHouse/pull/108922) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix reading `Iceberg` tables written by Databricks UniForm, which writes a degenerate placeholder schema (with an empty field list) into manifest files. [#108945](https://github.com/ClickHouse/ClickHouse/pull/108945) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix a use-after-free when reading a non-partitioned `Hive` table with multiple threads. [#109164](https://github.com/ClickHouse/ClickHouse/pull/109164) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the console log level not being reverted after server startup when `logger.startup_console_log_level` is set: the level was captured from the wrong configuration key, so the console stayed at the elevated startup level for the lifetime of the server. Closes [#103472](https://github.com/ClickHouse/ClickHouse/issues/103472). [#109858](https://github.com/ClickHouse/ClickHouse/pull/109858) ([Garrett Thomas](https://github.com/garrettthomaskth)). * Fix snapshot-isolation anomalies in `plain_rewritable` disk metadata transactions. Fixes [#92055](https://github.com/ClickHouse/ClickHouse/issues/92055). [#110948](https://github.com/ClickHouse/ClickHouse/pull/110948) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix an abort (`front() called on an empty vector`) in `clickhouse-client --login` when the host was not passed as an explicit `--host` argument, for example when it came from `--connection`, the configuration file, or the `CLICKHOUSE_HOST` environment variable. Closes [#103603](https://github.com/ClickHouse/ClickHouse/issues/103603). [#110279](https://github.com/ClickHouse/ClickHouse/pull/110279) ([Christoph Wurm](https://github.com/cwurm)). * Fix a logical error (`!part.empty()`) when a lambda argument is a backtick-quoted identifier whose name contains a dot (for example `__table1.`). Such a query previously produced a handled exception in release builds and aborted the server in debug and sanitizer builds while formatting an error message. [#108735](https://github.com/ClickHouse/ClickHouse/pull/108735) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`No set is registered for key`) in `ALTER TABLE ... DROP COLUMN`, `ALTER TABLE ... DELETE`/`UPDATE` mutations, and lightweight `DELETE FROM`, on tables that have an `ALIAS` column whose expression uses an `IN` operator (including through another `ALIAS` column). [#111039](https://github.com/ClickHouse/ClickHouse/pull/111039) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixed silent data loss with async inserts and deduplication (`async_insert=1`, `async_insert_deduplicate=1`). When several async-insert entries with distinct `insert_deduplication_token` values were coalesced into one flush that wrote to disjoint partitions, each token was registered in the deduplication log of every partition the flush touched, not only the partition its own rows landed in. A later insert reusing one of those tokens in a partition it never wrote to was then silently deduplicated away. Tokens are now registered only against the partition their rows actually landed in. Closes [#111031](https://github.com/ClickHouse/ClickHouse/issues/111031). [#111049](https://github.com/ClickHouse/ClickHouse/pull/111049) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` exception (`ScatterExchangeStep should have one source shard, got 8`) and duplicated rows when querying a `Merge` table over `Distributed` tables or views with the experimental setting `make_distributed_plan = 1`. Closes [#107946](https://github.com/ClickHouse/ClickHouse/issues/107946). [#108401](https://github.com/ClickHouse/ClickHouse/pull/108401) ([Groene AI](https://github.com/groeneai)). * Serialize a `Nullable(Tuple(...))` column as a single `CSV` field instead of flattening it into separate columns. Previously a non-null value was written as several `CSV` fields while `NULL` was a single `\N`, giving a row-dependent column count that broke `CSVWithNames`/`CSVWithNamesAndTypes` headers and round-trips. [#108959](https://github.com/ClickHouse/ClickHouse/pull/108959) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`KeyCondition uses PREWHERE output`) in the `Parquet` v3 reader when a format-level key condition references a column that is not read from the `Parquet` file. [#109267](https://github.com/ClickHouse/ClickHouse/pull/109267) ([Groene AI](https://github.com/groeneai)). * Fix wrong results for table-function reads with `FINAL` or `SAMPLE` under `serialize_query_plan = 1`. Also fix the aggregation hash-table statistics cache key for table functions so different table functions no longer share the same preallocation entry. [#109847](https://github.com/ClickHouse/ClickHouse/pull/109847) ([Azat Khuzhin](https://github.com/azat)). * Fixed cleanup of a `BACKUP ... TO Memory(...)` that fails before finalization: the failed backup was left registered, so its name could not be reused. [#109947](https://github.com/ClickHouse/ClickHouse/pull/109947) ([Julia Kartseva](https://github.com/jkartseva)). * Fix extreme interval arithmetic for `WITH FILL STEP` and the `add*` / `subtract*` date/time interval functions: out-of-range deltas no longer rely on signed overflow, and `WITH FILL` with a huge `YEAR` step no longer stalls by running the calendar backward. [#110158](https://github.com/ClickHouse/ClickHouse/pull/110158) ([Groene AI](https://github.com/groeneai)). * Fix `411 Length Required` errors from Azure services: the Poco-based Azure HTTP transport now sets `Content-Length` from the request body for SDK clients that do not set it themselves, such as Azure Key Vault. [#110299](https://github.com/ClickHouse/ClickHouse/pull/110299) ([Konstantin Bogdanov](https://github.com/thevar1able)). * The lazy `FINAL` optimization (`query_plan_optimize_lazy_final`, disabled by default) is no longer applied when the query reads in sorting-key order, because its replacement plan does not preserve that order and could return incorrectly ordered results. It is also no longer applied when a `LIMIT` smaller than the number of rows to read applies directly to the reading step, where the mandatory set-building pass defeats early termination. [#110576](https://github.com/ClickHouse/ClickHouse/pull/110576) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed `readWKT` and `readWKTPoint` returning uninitialized (scalar) or stale prior-row (vectorized) coordinates for `POINT EMPTY`. `POINT EMPTY` is now rejected with `CANNOT_PARSE_TEXT`, since ClickHouse `Point` is a fixed `Tuple(Float64, Float64)` with no empty representation. [#110692](https://github.com/ClickHouse/ClickHouse/pull/110692) ([Groene AI](https://github.com/groeneai)). * Fix wrong timezone handling when materializing right-side columns in hash joins. Closes [#111033](https://github.com/ClickHouse/ClickHouse/issues/111033). [#111074](https://github.com/ClickHouse/ClickHouse/pull/111074) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed wrong `count()` results and dropped rows when a `String` (or narrower `FixedString`) key or minmax skip index is filtered by a comparison with a wider `FixedString` constant, e.g. `toFixedString('abc', 257) = string_col`. Primary-key and minmax pruning built the range from the NUL-padded constant and wrongly skipped matching granules. [#111106](https://github.com/ClickHouse/ClickHouse/pull/111106) ([Groene AI](https://github.com/groeneai)). * Fixed asynchronous `Native`-format inserts so one buffered entry that becomes incompatible after `ALTER ... MODIFY COLUMN` no longer causes the whole batch to fail. Closes [#111064](https://github.com/ClickHouse/ClickHouse/issues/111064). [#111108](https://github.com/ClickHouse/ClickHouse/pull/111108) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix `CREATE OR REPLACE` of a dictionary with an object of another kind: it failed with `CANNOT_DETACH_DICTIONARY_AS_TABLE` after the replace was already committed, leaving an orphan `_tmp_replace_*` table. [#111142](https://github.com/ClickHouse/ClickHouse/pull/111142) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `INCOMPATIBLE_TYPE_OF_JOIN` for an `ANY` join on a `Join` engine table when a `WHERE` filter on a right-side column allowed the query planner to rewrite the join to `SEMI` or `ANTI`. A `Join` engine table has a fixed declared strictness that cannot be changed, so the conversion is now declined for such tables. [#111362](https://github.com/ClickHouse/ClickHouse/pull/111362) ([Groene AI](https://github.com/groeneai)). * Fix the `format` table function returning no columns when the input data parses to zero rows (for example an empty JSON document or a GeoJSON `FeatureCollection` with no features). It now returns an empty result with the correct columns instead of throwing `NOT_FOUND_COLUMN_IN_BLOCK`. Closes [#111390](https://github.com/ClickHouse/ClickHouse/issues/111390). [#111428](https://github.com/ClickHouse/ClickHouse/pull/111428) ([Groene AI](https://github.com/groeneai)). #### Build/Testing/Packaging Improvement * Sanitizer builds now capture the sanitizer report into the global buffer `sanitizer_report`, so that the core dump analyzer can read it from the core dump. [#109333](https://github.com/ClickHouse/ClickHouse/pull/109333) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Build `musl` from source instead of using vendored binaries. [#97452](https://github.com/ClickHouse/ClickHouse/pull/97452) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use SIMD `memcmp`/`memcpy`/`memmove`/`memset`/`bcmp`/`memmem` implementations from LLVM-libc. [#107566](https://github.com/ClickHouse/ClickHouse/pull/107566) ([Konstantin Bogdanov](https://github.com/thevar1able)). Fix silent `memmove` corruption in `-O0` debug builds (`-DDEBUG_O_LEVEL=0`) on machines with AVX-512. [#110904](https://github.com/ClickHouse/ClickHouse/pull/110904) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Remove usages of the `curl` library across the codebase and bundled third-party libraries. [#108296](https://github.com/ClickHouse/ClickHouse/pull/108296) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Updated `mariadb-connector-c` to 3.4.9. [#108328](https://github.com/ClickHouse/ClickHouse/pull/108328) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update `libssh` to 0.12.0. [#108329](https://github.com/ClickHouse/ClickHouse/pull/108329) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix a hang on transaction `COMMIT` on macOS. [#108375](https://github.com/ClickHouse/ClickHouse/pull/108375) ([Raúl Marín](https://github.com/Algunenano)). * Added a `COMPRESS_DEBUG_SECTIONS` CMake option that compresses DWARF debug sections in object files (`-gz`), auto-enabled where the compiler supports it, to shrink Debug build directories; final binaries are left uncompressed, so runtime stack-trace symbolization is unaffected. [#109306](https://github.com/ClickHouse/ClickHouse/pull/109306) ([Murphy](https://github.com/murphy-4o)). The bundled LLVM toolchain is now built with zlib, so its clang supports `-gz=zlib`, and the missing `clang++-21` symlink was added. [#109596](https://github.com/ClickHouse/ClickHouse/pull/109596) ([Murphy](https://github.com/murphy-4o)). * Debug builds now compile the vendored Rust crates with reduced debug info (`debug = 1`, cargo's `limited` level) and no incremental cache (`REDUCE_RUST_DEBUG_INFO`, on by default), shrinking Rust build artifacts (~12 → ~7 GB from scratch) and preventing unbounded cargo cache growth across rebuilds. Set `-DREDUCE_RUST_DEBUG_INFO=OFF` to restore full DWARF and incremental compilation. [#109471](https://github.com/ClickHouse/ClickHouse/pull/109471) ([Murphy](https://github.com/murphy-4o)). * Support the streaming distributed query exchange and `STREAM` reads on macOS. [#109769](https://github.com/ClickHouse/ClickHouse/pull/109769) ([Raúl Marín](https://github.com/Algunenano)). * Bump `libarchive` from 3.8.7 to 3.8.8. [#109874](https://github.com/ClickHouse/ClickHouse/pull/109874) ([Robert Schulze](https://github.com/rschu1ze)). * Bump `c-ares` from 1.34.6 to 1.34.8. [#109905](https://github.com/ClickHouse/ClickHouse/pull/109905) ([Robert Schulze](https://github.com/rschu1ze)). * Update `krb5` to fix CVE-2026-40355 and CVE-2026-40356. [#109910](https://github.com/ClickHouse/ClickHouse/pull/109910) ([Robert Schulze](https://github.com/rschu1ze)). * Removed the `libpng` third-party dependency. `FORMAT PNG` now uses a small built-in PNG encoder (backed by the bundled zlib) instead. [#110051](https://github.com/ClickHouse/ClickHouse/pull/110051) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Track raw C allocations (not going through `operator new`) in the memory tracker on macOS by wrapping the jemalloc malloc zone, so memory limits are enforced for them. [#110370](https://github.com/ClickHouse/ClickHouse/pull/110370) ([Raúl Marín](https://github.com/Algunenano)). * The `-distroless` Docker images for `clickhouse-server` and `clickhouse-keeper` are now built from the shell-free production stage; previously they were built from the debug stage and included `/busybox/sh`. Closes [#105677](https://github.com/ClickHouse/ClickHouse/issues/105677). [#105678](https://github.com/ClickHouse/ClickHouse/pull/105678) ([ashishch432](https://github.com/ashishch432)). * Update `abseil-cpp` to LTS `20260526.0`. [#108991](https://github.com/ClickHouse/ClickHouse/pull/108991) ([Konstantin Bogdanov](https://github.com/thevar1able)). ### ClickHouse release 26.6, 2026-06-25. [Presentation](https://presentations.clickhouse.com/2026-release-26.6/), [Video](https://www.youtube.com/watch?v=-NmqMH9y4EY) #### Backward Incompatible Change * The change removes `allow_experimental_query_deduplication` feature. The feature was experimental for a long time, has no tests and not going to be supported (initial tests were unstable and has been removed in [#49579](https://github.com/ClickHouse/ClickHouse/pull/49579)). [#99398](https://github.com/ClickHouse/ClickHouse/pull/99398) ([Igor Nikonov](https://github.com/devcrafter)). * The XRay-based `SYSTEM INSTRUMENT` feature now uses `arguments` instead of `parameters`: the `system.instrumentation.parameters` column was renamed to `arguments`, and the `SYSTEM INSTRUMENT ADD ... HANDLER PARAMETERS ...` syntax was renamed to `SYSTEM INSTRUMENT ADD ... HANDLER ARGUMENTS ...`. Queries or automation that use the old column or syntax must be updated to use the new names; the old names are no longer supported. [#103854](https://github.com/ClickHouse/ClickHouse/pull/103854) ([Pablo Marcos](https://github.com/pamarcos)). * The setting `show_data_lake_catalogs_in_system_tables` has been renamed to `show_remote_databases_in_system_tables` and broadened: when its value is 0 (the default), `MySQL` and `PostgreSQL` databases are also hidden from `system.tables`, `system.columns`, and `system.completions`, in addition to data lake catalogs. The old setting name is kept as an alias. [#104416](https://github.com/ClickHouse/ClickHouse/pull/104416) ([Pablo Marcos](https://github.com/pamarcos)). * The default x86 build now targets x86-64-v3 (AVX2) instead of x86-64-v2 (SSE4.2). This requires a CPU with AVX2 support (Intel Haswell or later, AMD Excavator or later). If your CPU does not support AVX2, use the `amd64compat` build which targets plain x86-64. [#105019](https://github.com/ClickHouse/ClickHouse/pull/105019) ([Raúl Marín](https://github.com/Algunenano)). * Reject nested `Dynamic`/`Variant` in `min`/`max` aggregates and `minmax` indexes. Previously only top-level `Dynamic`/`Variant` were checked. Closes [#104747](https://github.com/ClickHouse/ClickHouse/issues/104747). [#105468](https://github.com/ClickHouse/ClickHouse/pull/105468) ([Pavel Kruglov](https://github.com/Avogar)). * `icebergHash` and `icebergBucket` now reject `Int128`, `UInt128`, `Int256`, `UInt256`, and `Decimal256` arguments with an explicit error. Previously they silently truncated wider values and produced colliding hashes. The Iceberg spec only defines hashing for 32/64-bit integers and decimals with precision up to 38; cast to `Int64` or `Decimal128` to keep the previous behaviour for values that fit. [#105866](https://github.com/ClickHouse/ClickHouse/pull/105866) ([Raúl Marín](https://github.com/Algunenano)). * Added granular control over query echoing in the CLI. The `--echo` option now accepts an optional boolean value and works in both interactive and batch mode. New options `--echo-formatted` and `--echo-query-id` control whether echoed queries are formatted and whether the `query_id` is printed, respectively. The `--hilite`/`--highlight` option now also controls highlighting of echoed queries. As a side effect, `--echo` is now a boolean-value option, so a positional query directly after a bare `--echo` is treated as its value; use `--echo --query "..."` or `--echo=false` instead. [#106191](https://github.com/ClickHouse/ClickHouse/pull/106191) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the experimental KQL (Kusto) functions `array_sort_asc` and `array_sort_desc`, and their SQL backends `kql_array_sort_asc` and `kql_array_sort_desc`. These functions were experimental, implemented with low quality, and the source of correctness and parser bugs. Queries using these names now return `UNKNOWN_FUNCTION`. [#108101](https://github.com/ClickHouse/ClickHouse/pull/108101) ([Groene AI](https://github.com/groeneai)). * `ALTER TABLE ... REPLACE PARTITION ... FROM ...` no longer silently drops the destination partition's data when the source table has no parts in the requested partition. Previously such a request removed the destination partition and wrote nothing in its place — a data-loss footgun. It is now rejected with `BAD_ARGUMENTS` by default. To restore the previous silent-clear behavior, set the new setting `allow_replace_partition_from_empty_source = 1`, or set `compatibility` to `26.5` or lower; to explicitly drop destination data, use `ALTER TABLE ... DROP PARTITION`. [#104939](https://github.com/ClickHouse/ClickHouse/pull/104939) ([Groene AI](https://github.com/groeneai)). * The default of the server setting `insert_deduplication_version` changes from `compatible_double_hashes` to `new_unified_hash`. Insert deduplication now works on the whole inserted block (per insert) rather than per part/partition: a retry of the same insert is still deduplicated, but two different inserts that produce an identical part are no longer cross-deduplicated, and reordered inserts of the same rows are no longer deduplicated. Set `insert_deduplication_version = compatible_double_hashes` to restore the previous behavior. Instances upgrading directly from a release that defaulted to `old_separate_hashes` should first run with `compatible_double_hashes` until the longest relevant deduplication window has elapsed before relying on `new_unified_hash`. [#107886](https://github.com/ClickHouse/ClickHouse/pull/107886) ([Sema Checherinda](https://github.com/CheSema)). #### New Feature * Added support for continuous queries over `MergeTree` tables, implemented via a series of snapshot reads. This is a first step toward general support for streaming queries. [#105114](https://github.com/ClickHouse/ClickHouse/pull/105114) ([Mikhail Artemenko](https://github.com/Michicosun)). * Added hypothetical (what-if) skip indexes. Use `CREATE HYPOTHETICAL INDEX ... ON t (expr) TYPE ...` to define a session-scoped virtual skip index, then `EXPLAIN WHATIF SELECT ...` to estimate its skip ratio and cost without materializing it. Defined indexes are visible in the new `system.hypothetical_indexes` table. [#104608](https://github.com/ClickHouse/ClickHouse/pull/104608) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Add an embedded `/schema` web UI to `clickhouse-server` that visualizes the dependency graph between tables, materialized views, refreshable materialized views, dictionaries, distributed tables and views. [#105276](https://github.com/ClickHouse/ClickHouse/pull/105276) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Added the `PNG` output format support, allowing query output results to be directly rendered as PNG images. [#74691](https://github.com/ClickHouse/ClickHouse/pull/74691) ([Maksim Dergousov](https://github.com/m7kss1)). * Added detection of the AI coding agent (Claude Code, Cursor, Codex, Gemini CLI, Goose, etc.) that invoked `clickhouse-client` or `clickhouse-local`, based on environment variables. The detected agent is reported in the new `client_agent` column of `system.query_log`, `system.query_thread_log` and `system.processes`. [#106619](https://github.com/ClickHouse/ClickHouse/pull/106619) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new system table `system.documentation` that collects the embedded reference documentation of the uniform components of the system (functions, table engines, data types, settings, formats, and others) into a single table, with the documentation rendered as Markdown. [#107463](https://github.com/ClickHouse/ClickHouse/pull/107463) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add the `GeoJSON` input format for reading [GeoJSON](https://geojson.org/) `FeatureCollection` documents, producing one row per feature with `id` (`String`), `geometry` (`Geometry`), and `properties` (`Nullable(JSON)`) columns. `Point`, `LineString`, `MultiLineString`, `Polygon`, and `MultiPolygon` geometries are read into ClickHouse's native `Geometry` type. `GeometryCollection` and `MultiPoint`, which the `Geometry` type cannot represent, raise an exception by default, or can be stored as `NULL` via the `input_format_geojson_unsupported_geometry_handling` setting. Contributes to [#91533](https://github.com/ClickHouse/ClickHouse/issues/91533). [#98124](https://github.com/ClickHouse/ClickHouse/pull/98124) ([Mark Needham](https://github.com/mneedham)). * Added functions for serving Mapbox Vector Tiles directly from SQL: `MVTEncodeGeom` projects a geometry into the pixel space of a slippy-map tile and clips it, `MVTEncode` aggregates the projected geometries of a group into the binary bytes of a single-layer tile, and `MVTBoundingBox` / `MVTBoundingBoxMercator` return the bounding box of a tile for restricting rows to it. Point, line and polygon geometry is supported. Also available under the PostGIS aliases `ST_AsMVTGeom` and `ST_AsMVT`. [#106107](https://github.com/ClickHouse/ClickHouse/pull/106107) ([Saarthak Gupta](https://github.com/saarthak2002)). * `clickhouse-local` now supports `SYSTEM START LISTEN` and `SYSTEM STOP LISTEN` queries, allowing it to be transformed into a server accepting TCP and HTTP connections. [#101143](https://github.com/ClickHouse/ClickHouse/pull/101143) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added an interactive `help` command to `clickhouse-client` and `clickhouse-local`. Typing `help ` (also `/help`, `man`, `/man`) shows the documentation of a function, table engine, data type, format, setting, or other component, rendered from Markdown in the terminal with syntax-highlighted SQL. When the word is ambiguous all matches are shown, and when it is not found, similar names are suggested. [#108042](https://github.com/ClickHouse/ClickHouse/pull/108042) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support writing into Azure Data Lake Storage Gen2. [#105406](https://github.com/ClickHouse/ClickHouse/pull/105406) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Adding `RowBinaryWithNamesAndTypesAndDefaults` format to get better support for schema evolution. [#105736](https://github.com/ClickHouse/ClickHouse/pull/105736) ([Mark Zitnik](https://github.com/mzitnik)). * Implement `ADD ENUM VALUES` in `ALTER TABLE` queries to simplify appending new values to an existing `Enum` type without the need to specify all current `Enum` values again. [#93830](https://github.com/ClickHouse/ClickHouse/pull/93830) ([Ilya Golshtein](https://github.com/ilejn)). * Refreshable materialized view now supports `REFRESH DEPENDS ON` to trigger refreshes on another RMV's refreshes instead of time-based schedule. (`REFRESH EVERY ... DEPENDS ON` already existed, but couldn't be used reliably for this use case.) See `CREATE MATERIALIZED VIEW` documentation. [#104440](https://github.com/ClickHouse/ClickHouse/pull/104440) ([Michael Kolupaev](https://github.com/al13n321)). * Add support for selecting columns by name pattern with `* LIKE ''` and `* ILIKE ''`, including qualified forms such as `table.* LIKE ''` and `table.* ILIKE ''`. `LIKE` matches column names case-sensitively, `ILIKE` case-insensitively. [#104569](https://github.com/ClickHouse/ClickHouse/pull/104569) ([Yue Ni](https://github.com/niyue)). * Added `ESCAPE` syntax that defines a custom escape character in a `LIKE` pattern, allowing you to treat `%` and `_` as literal characters instead of wildcards. [#99774](https://github.com/ClickHouse/ClickHouse/pull/99774) ([Ilya Yatsishin](https://github.com/qoega)). * Add `materialize_projections_on_insert` and `materialize_projections_on_merge` `MergeTree` table settings. When `materialize_projections_on_insert = 0`, `INSERT`s skip building projection parts, which improves insert throughput for tables with many projections. When `materialize_projections_on_merge = 1`, a merge rebuilds a projection that is missing from all of its source parts, so projections can be built during merges instead of on insert. Merges still combine only parts that share the same set of projections. [#100993](https://github.com/ClickHouse/ClickHouse/pull/100993) ([Christoph Wurm](https://github.com/cwurm)). * Add a new immutable `MergeTree` setting `allow_tuple_element_aggregation`, disabled by default. When enabled, `SummingMergeTree`, `AggregatingMergeTree` and `CoalescingMergeTree` recursively flatten `Tuple` columns and aggregate each leaf element independently during merges, exactly as if it were a top-level column — `SummingMergeTree` sums it, `AggregatingMergeTree` merges its aggregate-function state, and `CoalescingMergeTree` keeps its last non-NULL value. The setting must be specified at table creation time and is silently ignored by engines that do not support it. [#98039](https://github.com/ClickHouse/ClickHouse/pull/98039) ([johnjing](https://github.com/JingYanchao)). * New functions `quantizeBFloat16ToInt8` and `dequantizeInt8ToBFloat16`: a scalar codec that compresses embedding components to 8-bit using a 256-level Gaussian Lloyd-Max quantizer, from which Int4/Int2/binary codes can be extracted by bit-truncation. [#108102](https://github.com/ClickHouse/ClickHouse/pull/108102) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add functions `arrayTopK` and `arrayBottomK`: - `arrayTopK(k, array)` returns the K largest elements in descending order - `arrayBottomK(k, array)` returns the K smallest elements in ascending order. [#104563](https://github.com/ClickHouse/ClickHouse/pull/104563) ([Vitaly Baranov](https://github.com/vitlibar)). * Added a new system table `system.iceberg_files` exposing per-file metadata for `Iceberg` tables, with one row per data or delete file in each table's current snapshot. This closes [#98777](https://github.com/ClickHouse/ClickHouse/issues/98777). [#104415](https://github.com/ClickHouse/ClickHouse/pull/104415) ([Asya Shneerson](https://github.com/asya-ch)). * Added `system.constraints` table that provides information about all `CHECK` and `ASSUME` constraints across all tables, including constraint name, type, and expression. [#105337](https://github.com/ClickHouse/ClickHouse/pull/105337) ([Pedro Ferreira](https://github.com/PedroTadim)). * Added a new `system.dictionary_layouts` table that lists the available dictionary layouts together with embedded documentation (`description`, `syntax`, `examples`, `introduced_in`, `related`). [#106182](https://github.com/ClickHouse/ClickHouse/pull/106182) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new `system.dictionary_sources` table that lists the available dictionary sources together with embedded documentation (`description`, `syntax`, `examples`, `introduced_in`, `related`). [#106184](https://github.com/ClickHouse/ClickHouse/pull/106184) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new `system.data_skipping_index_types` table that lists the available data skipping index types together with embedded documentation (`description`, `syntax`, `examples`, `introduced_in`, `related`). [#106186](https://github.com/ClickHouse/ClickHouse/pull/106186) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new `system.disk_types` table that lists the available disk types together with embedded documentation (`description`, `syntax`, `examples`, `introduced_in`, `related`). [#106187](https://github.com/ClickHouse/ClickHouse/pull/106187) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implemented `SYSTEM RESTART DISK `: it now reloads a disk's in-memory metadata and re-scans the data parts of readonly-replica tables located on it. This lets a readonly replica of a table on shared `plain_rewritable` storage observe data written by another server on demand, without waiting for `refresh_parts_interval` or restarting the server. [#106645](https://github.com/ClickHouse/ClickHouse/pull/106645) ([Jordi Villar](https://github.com/jrdi)). * PostgreSQL-style `expr OP SOME(array)` / `expr OP ALL(array)` (non-subquery right-hand side) is now supported and rewritten to `has` / `NOT has` for `=`/`<>`, or to `arrayExists` / `arrayAll` lambdas for other comparison operators. `ANY` is not accepted for the array form because `any` is also an aggregate function; use `SOME` instead. The subquery form of `ANY`/`SOME`/`ALL` continues to be lowered to `IN` / `NOT IN`. [#105129](https://github.com/ClickHouse/ClickHouse/pull/105129) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added two new `load_balancing` strategies, `hostname_longest_common_prefix` and `hostname_longest_common_suffix`, which prefer the replica whose hostname shares the longest common prefix (respectively, suffix) with the initiator's hostname. They are useful when the data center is encoded as a prefix or suffix of hostnames whose numeric segments have variable length, where the existing `nearest_hostname` and `hostname_levenshtein_distance` strategies pick the wrong replica. [#107360](https://github.com/ClickHouse/ClickHouse/pull/107360) ([Denny [DBA at Innervate]](https://github.com/den-crane)). * Added TLS client certificate information (subjects, serial number, issuer, and validity period) to the `system.session_log` table to improve observability of certificate-based authentication. [#107679](https://github.com/ClickHouse/ClickHouse/pull/107679) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add an optional `external_id` credential for `S3` role-based access. [#106941](https://github.com/ClickHouse/ClickHouse/pull/106941) ([Elian Gidoni](https://github.com/eliangidoni)). * Added a new `S3Queue` setting `after_processing_move_preserve_path`. When enabled together with `after_processing='move'` and `after_processing_move_prefix`, processed objects are moved while preserving their full source path under the destination prefix instead of being flattened to just the file name. [#105354](https://github.com/ClickHouse/ClickHouse/pull/105354) ([Asya Shneerson](https://github.com/asya-ch)). * Add `message_queue_disable_insertion` server setting to disable insertion from message queue engines (`Kafka`, `RabbitMQ`, `NATS`) into attached materialized views. Useful for read-only replica scenarios. [#104911](https://github.com/ClickHouse/ClickHouse/pull/104911) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Added `LOCALTIME` and `LOCALTIMESTAMP` (SQL-standard / PostgreSQL syntax). `LOCALTIMESTAMP` is an alias for `now()` (returns `DateTime`); `LOCALTIME` returns the current time of day as a `Time` value. [#106139](https://github.com/ClickHouse/ClickHouse/pull/106139) ([Thomas Cabral](https://github.com/thomas-cabral)). * Added `date_part('unit', expr)` as syntactic sugar for `EXTRACT(unit FROM expr)`. Standard interval kinds and the PostgreSQL extras (`epoch`, `dow`, `doy`, `isodow`, `isoyear`, `century`, `decade`, `millennium`) are all supported. [#105127](https://github.com/ClickHouse/ClickHouse/pull/105127) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * PostgreSQL-compatible `EXTRACT(TIMEZONE_HOUR FROM dt)` and `EXTRACT(TIMEZONE_MINUTE FROM dt)` for the hour and minute parts of a timezone offset, and `EXTRACT( FROM INTERVAL n )` / `date_part('', INTERVAL n )` for extracting the value out of an interval. [#106227](https://github.com/ClickHouse/ClickHouse/pull/106227) ([Vinayak Joshi](https://github.com/vinayakj592)). * Added `min_by` and `max_by` aggregate aliases for `argMin` and `argMax`. [#105712](https://github.com/ClickHouse/ClickHouse/pull/105712) ([Joey Yu](https://github.com/itsjoeoui)). These aliases are stupid, real engineers use the original ClickHouse names. * Added `REGEXP_SUBSTR` as a case-insensitive alias of `regexpExtract` for Oracle/MySQL/Snowflake compatibility. [#105122](https://github.com/ClickHouse/ClickHouse/pull/105122) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `SESSION_USER` as a case-insensitive alias of `currentUser()` for PostgreSQL / SQL-standard compatibility. [#106081](https://github.com/ClickHouse/ClickHouse/pull/106081) ([Takumi Hara](https://github.com/takumihara)). * Support a trailing `NULL` / `NOT NULL` modifier in `ALTER TABLE ... ADD/MODIFY COLUMN`, mirroring `CREATE TABLE`. [#106150](https://github.com/ClickHouse/ClickHouse/pull/106150) ([Takumi Hara](https://github.com/takumihara)). * You can now use `h3PolygonToCellsWithContainment` function from h3 to use their new algorithm which supports center-based, fully-contained, and overlapping containment modes. [#104455](https://github.com/ClickHouse/ClickHouse/pull/104455) ([Youssef Kadry](https://github.com/yousefQadry)). * Adds `IPV4_PREFIX_BITS and IPV6_PREFIX_BITS` flags when Keying done by `IP_ADDRESS` or `FORWARDED_IP_ADDRESS`. [#89270](https://github.com/ClickHouse/ClickHouse/pull/89270) ([Aditya Chopra](https://github.com/adityachopra29)). * `formatReadableTimeDelta` now accepts an `INTERVAL` expression of type other than `Month` and `Year` as input. [#64315](https://github.com/ClickHouse/ClickHouse/pull/64315) ([Francisco J. Jurado Moreno](https://github.com/Beetelbrox)). * User can now specify an optional precision argument to the `formatReadableSize`, `formatReadableDecimalSize` and `formatReadableQuantity` functions, controlling the number of digits after the decimal point. Default is 2, preserving the prior behavior. [#104648](https://github.com/ClickHouse/ClickHouse/pull/104648) ([Antonio Filipovic](https://github.com/antoniofilipovic)). * Add `output_format_float_precision` setting to control the number of decimal digits in floating-point text output. [#99721](https://github.com/ClickHouse/ClickHouse/pull/99721) ([phulv94](https://github.com/phulv94)). * Added a setting `output_format_always_write_decimal_point_in_float_and_decimal` to always print a decimal point for floating-point and `Decimal` numbers in text formats, even when the value is a whole number. For example, output `1.` instead of `1`. Disabled by default. [#62614](https://github.com/ClickHouse/ClickHouse/pull/62614) ([Ilya Yatsishin](https://github.com/qoega)). * Add support for new HTTP handler configuration elements `` and `` (match all paths under a base path, useful e.g. for Prometheus handlers), as well as ``, ``, and ``. The old `regex:...` form is now obsolete. [#107492](https://github.com/ClickHouse/ClickHouse/pull/107492) ([Vitaly Baranov](https://github.com/vitlibar)). * Support `basic` statistics, a compact per-column statistic that stores numeric min/max, average string length, and NULL counts where applicable. [#106048](https://github.com/ClickHouse/ClickHouse/pull/106048) ([Han Fei](https://github.com/hanfei1991)). * Support TTL nodes in ClickHouse Keeper, opt-in via the `create_ttl` feature flag. [#100397](https://github.com/ClickHouse/ClickHouse/pull/100397) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Introduce a memory reservation feature for workloads. See the [workload scheduling](https://clickhouse.com/docs/operations/workload-scheduling) documentation. [#82414](https://github.com/ClickHouse/ClickHouse/pull/82414) ([Sergei Trifonov](https://github.com/serxa)). #### Experimental Feature * Multi-stage distributed query execution: the planner splits the query plan into stages connected by scatter / broadcast / gather / shuffle exchanges and dispatches the plan fragments to worker nodes. The data between stages is streamed via TCP or passed via temporary files in shared object storage, the path supports distributed shuffle and broadcast hash joins, shuffle aggregation, and distributed sort. The feature is experimental and is disabled by default. [#106020](https://github.com/ClickHouse/ClickHouse/pull/106020) ([Alexander Gololobov](https://github.com/davenger)). The experimental distributed query plan engine (`make_distributed_plan`) can now use a different task-dispatch and streaming-exchange port per worker, configured per replica in `` with `stateless_worker_port` and `streaming_exchange_port`. When unset, the previous server-level ports (`stateless_worker_client.port` and `distributed_query.streaming_exchange_port`) are used. This makes it possible to run several workers on one host. [#107885](https://github.com/ClickHouse/ClickHouse/pull/107885) ([Alexander Gololobov](https://github.com/davenger)). * Added the experimental `dphyp` join reordering algorithm for inner joins as an option for the `query_plan_optimize_join_order_algorithm` setting, and the `query_plan_optimize_join_order_max_searched_plans` setting, which bounds the join-order search and falls back to the next algorithm in the chain when the bound is exceeded; set it to `0` to keep the previous unbounded search behavior. [#98798](https://github.com/ClickHouse/ClickHouse/pull/98798) ([Alexander Gololobov](https://github.com/davenger)). * Add `aiEmbed` function for using LLM APIs to generate embeddings from within ClickHouse. [#102922](https://github.com/ClickHouse/ClickHouse/pull/102922) ([George Larionov](https://github.com/george-larionov)). * Added an experimental Linux OOM canary: a sacrificial child process that the Linux OOM killer targets before the main server process. When enabled with `oom_canary_enable`, it sheds memory pressure by purging allocator caches, cancelling queries, cancelling merges, and recording an event in `system.crash_log`. The OOM response requires cgroup v2 `memory.events` OOM-kill evidence. [#101942](https://github.com/ClickHouse/ClickHouse/pull/101942) ([Peng](https://github.com/fastio)). * The web terminal interface at `/webterminal` is now a production feature and is enabled by default. It is controlled by the new `enable_webterminal` server setting; the former `allow_experimental_webterminal` setting is deprecated but still honored for backward compatibility. To disable the endpoint, set `enable_webterminal` to `false`. [#106255](https://github.com/ClickHouse/ClickHouse/pull/106255) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * PromQL HTTP endpoints now support specifying the database and table via URL query parameters (`database=`, `table=`, or `table=db.table`). The previous implicit `default.prometheus` table fallback is removed: if neither the handler config nor the URL specifies a table, the handler now returns `The time series table name is not set`. To keep the old behavior, configure `
default.prometheus
` or pass `table=...` in the URL. [#107553](https://github.com/ClickHouse/ClickHouse/pull/107553) ([Vitaly Baranov](https://github.com/vitlibar)). * Added support for the PromQL `histogram_quantile` function in `prometheusQuery` and `prometheusQueryRange` table functions. This enables computing quantiles over classic Prometheus histogram buckets identified by the `le` label. [#103477](https://github.com/ClickHouse/ClickHouse/pull/103477) ([Joe Smith](https://github.com/Yasumoto)). * Added lazy posting list apply mode for the text index. When enabled via `SET allow_experimental_text_index_lazy_apply = 1` and `SET text_index_posting_list_apply_mode = 'lazy'`, posting lists are decoded on demand at packed-block granularity using a cursor-based approach instead of being fully materialized into Roaring Bitmaps, reducing memory usage and CPU time for selective text index queries. [#100035](https://github.com/ClickHouse/ClickHouse/pull/100035) ([Peng](https://github.com/fastio)). * Added support for the [AssemblyScript](https://www.assemblyscript.org) ABI for WebAssembly UDFs. You can try the AssemblyScript ABI without installing the toolchain locally by using the browser-based [playground](https://pastila.nl/?0009878e/b9768c4e2144d0c2f3dd512e7a1aaf69.html), which compiles AssemblyScript code and loads the resulting WebAssembly module into local ClickHouse. [#104606](https://github.com/ClickHouse/ClickHouse/pull/104606) ([Vladimir Cherkasov](https://github.com/vdimir)). * Allow for attaching Prometheus handlers on the main HTTP port with an optional prefix. [#104975](https://github.com/ClickHouse/ClickHouse/pull/104975) ([James Cunningham](https://github.com/JTCunning)). * Added an experimental `MergeTree` setting `packed_skip_index_max_bytes` that bundles small skip-index substreams into a single `skp_idx.packed` archive per part, reducing inode pressure when many skip indices are defined on a table. The decision is per substream at write time: substreams whose serialized size stays under the threshold go into the archive, anything larger keeps the standalone `skp_idx_.idx2` / `.mrk2` layout. A single part can mix layouts. Full-text indices are not supported and are always per-file. Default is 0 (packing disabled). [#105321](https://github.com/ClickHouse/ClickHouse/pull/105321) ([Raúl Marín](https://github.com/Algunenano)). * Support `Buffers` serialization for WebAssembly UDFs using `ABI BUFFERED_V1`, and add `webassembly_udf_enable_fuel` as a persisted WASM UDF function setting. [#105574](https://github.com/ClickHouse/ClickHouse/pull/105574) ([Antonio Andelic](https://github.com/antonio2368)). * Add an experimental `use_reader_executor` setting (default off) that routes reads through a new pipeline-based `ReaderExecutor` instead of the legacy chain of read buffers. [#106570](https://github.com/ClickHouse/ClickHouse/pull/106570) ([Sema Checherinda](https://github.com/CheSema)). Added observability metrics for the experimental `use_reader_executor` read path, including a modeled read-cost KPI (`ReaderExecutorModeledCostMsPerRequestedMiB`). [#106968](https://github.com/ClickHouse/ClickHouse/pull/106968) ([Sema Checherinda](https://github.com/CheSema)). Introduced the `ChainedBuffers` buffer behind the experimental `use_reader_executor` read path, with a `ReaderExecutorChainedBufferBytes` metric. [#107210](https://github.com/ClickHouse/ClickHouse/pull/107210) ([Sema Checherinda](https://github.com/CheSema)). * Descending sort order in `MergeTree` sorting keys (e.g. `ORDER BY (time DESC, key)`) is now always supported and no longer requires the experimental setting `allow_experimental_reverse_key`, which became obsolete. [#106440](https://github.com/ClickHouse/ClickHouse/pull/106440) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * `Nullable(Tuple(...))` is now Beta. Disabled by default, set `enable_nullable_tuple_type = 1` to use it. [#107754](https://github.com/ClickHouse/ClickHouse/pull/107754) ([Nihal Z. Miaji](https://github.com/nihalzp)). #### Performance Improvement * Various changes to Keeper that make it around 2x faster overall (better batching, pipelining messages to leader, pipelining log appends). [#101757](https://github.com/ClickHouse/ClickHouse/pull/101757) ([Michael Kolupaev](https://github.com/al13n321)). * Reduce per-query overhead for simple `SELECT` queries (parsing, analysis and planning). For example, `SELECT count() FROM hits` from a single connection is roughly 50% faster. [#104513](https://github.com/ClickHouse/ClickHouse/pull/104513) ([Raúl Marín](https://github.com/Algunenano)). * Optimize primary key index analysis for long and high-cardinality primary keys. For a long primary key, the run time of index analysis now mainly depends on the complexity of the query's filter (the key columns it actually uses), not on the length of the primary key — so extending the sorting key has negligible extra overhead on index analysis for queries that filter on only a few of its columns. For a high-cardinality primary key, where ClickHouse keeps only a selective prefix of the key columns in memory and does not load the trailing ones, index analysis now works on just that in-memory prefix instead of the whole key. The optimization is enabled by default and can be turned off with the new setting `use_lightweight_primary_key_index_analysis`. Closes [#65103](https://github.com/ClickHouse/ClickHouse/issues/65103). [#91836](https://github.com/ClickHouse/ClickHouse/pull/91836) ([Nihal Z. Miaji](https://github.com/nihalzp)). * The default x86 build now targets x86-64-v3 (AVX2) instead of x86-64-v2 (SSE4.2). This requires a CPU with AVX2 support (Intel Haswell or later, AMD Excavator or later). If your CPU does not support AVX2, use the `amd64compat` build which targets plain x86-64. [#105019](https://github.com/ClickHouse/ClickHouse/pull/105019) ([Raúl Marín](https://github.com/Algunenano)). * Use `ipnsort` and `driftsort` (stable sorting) — C++ ports of the Rust standard library's sort implementations — for the general-purpose comparison sorts. Speeds up `ORDER BY` on non-numeric columns and stable sorts, and removes a worst case on reverse-sorted input. [#106650](https://github.com/ClickHouse/ClickHouse/pull/106650) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * New `GROUP BY` optimization for high cardinality evenly distributed keys that scatters rows across threads by hashing the grouping key, so each thread aggregates a disjoint subset of keys without a merge phase. Set `enable_sharding_aggregator = 1` to enable it. [#104233](https://github.com/ClickHouse/ClickHouse/pull/104233) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Enable hash table prefetching for string key `GROUP BY` queries, improving high-cardinality string aggregation performance by ~8%. [#101007](https://github.com/ClickHouse/ClickHouse/pull/101007) ([Le Zhang](https://github.com/damahua)). * Reduced peak memory usage when merging partial two-level aggregation results with large aggregate states (e.g. `groupArray`), by freeing each bucket's source states incrementally during the merge instead of keeping them all alive until the merge completes. [#102330](https://github.com/ClickHouse/ClickHouse/pull/102330) ([Yuri Fedoseev](https://github.com/yurifedoseev)). * Memory representation of `Enum` `DataType` was optimized. Memory footprint of tables that have `Enum` type in it can improve up to 10x times for `Enum` component. Operation time to retrieve name by value(number) is similar or faster. Search by name is slightly slower, but this access direction is less common. [#95668](https://github.com/ClickHouse/ClickHouse/pull/95668) ([Ilya Yatsishin](https://github.com/qoega)). * Add identifier resolution caching to prevent duplicate identifier resolution during query analysis. [#88043](https://github.com/ClickHouse/ClickHouse/pull/88043) ([Max Justus Spransy](https://github.com/maxjustus)). * Improve performance of query analysis for queries over tables with many columns: avoid computing column node hashes (which include the whole source table expression) when not needed. Analyzing nested `SELECT *` subqueries over a table with ~1200 columns is now about 50 times faster. [#106957](https://github.com/ClickHouse/ClickHouse/pull/106957) ([Dmitry Novik](https://github.com/novikd)). * Sped up query analysis for queries with many or deeply nested function calls by removing a redundant query-tree hash from the function resolution cache. [#107516](https://github.com/ClickHouse/ClickHouse/pull/107516) ([Dmitry Novik](https://github.com/novikd)). * Fixed long login and query-startup stalls with replicated access storage when many access entities (row policies, roles, quotas, settings profiles) change at once. Each per-entity cache now recomputes once per notification batch instead of once per changed entity, removing quadratic work that could hold the access lock for minutes. [#107672](https://github.com/ClickHouse/ClickHouse/pull/107672) ([Azat Khuzhin](https://github.com/azat)). * Share the hash join's `FixedHashMap` as the join runtime filter on the probe side. When the build-side hash table is (or can be converted to) a `FixedHashMap`, it is published as the runtime filter and replaces the `Set`/`BloomFilter` that `BuildRuntimeFilterStep` would otherwise install. Controlled by the new setting `join_runtime_filter_from_fixed_hash_table` (default `true`). [#105640](https://github.com/ClickHouse/ClickHouse/pull/105640) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Lazy application of selector and replication indexes in case of `JOIN` followed by a selective `LIMIT` or `TopN` or another `JOIN`. To control the number of payload columns for enabling lazy selector indexes use the setting `query_plan_min_columns_for_join_lazy_indexing` (0 means the optimization is disabled). To control the `LIMIT` for which the optimization is applied use the setting `query_plan_max_limit_for_join_lazy_indexing`. [#106566](https://github.com/ClickHouse/ClickHouse/pull/106566) ([Hechem Selmi](https://github.com/m-selmi)). * DP (dynamic programming) JOIN reordering is now allowed with parallel replicas. [#105889](https://github.com/ClickHouse/ClickHouse/pull/105889) ([Nikita Taranov](https://github.com/nickitat)). * Improved query plan when JOIN uses runtime filters (default-on `enable_join_runtime_filters`): the join-reorder cost model now sees through `WindowTransform` (and other row-preserving plan steps) on the right subtree and uses the underlying row count and per-column NDV instead of falling back to no statistics. [#107229](https://github.com/ClickHouse/ClickHouse/pull/107229) ([UnamedRus](https://github.com/UnamedRus)). * De-virtualize the emission of non-joined rows in RIGHT and FULL join by batching them. [#105679](https://github.com/ClickHouse/ClickHouse/pull/105679) ([Hechem Selmi](https://github.com/m-selmi)). * Support packed `keys32` and `keys64` methods in `HashJoin`. [#107202](https://github.com/ClickHouse/ClickHouse/pull/107202) ([Nikita Taranov](https://github.com/nickitat)). * Support packed `keys32` and `keys64` methods in `Set`. [#107564](https://github.com/ClickHouse/ClickHouse/pull/107564) ([Nikita Taranov](https://github.com/nickitat)). * More optimal handling of nullable columns in aggregation. [#106015](https://github.com/ClickHouse/ClickHouse/pull/106015) ([Nikita Taranov](https://github.com/nickitat)). * Improve performance of `LIMIT BY` queries. [#103349](https://github.com/ClickHouse/ClickHouse/pull/103349) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up `ORDER BY ... LIMIT BY` queries by running `LIMIT BY` inside each parallel sorted stream during `Sort` when `LIMIT BY`'s columns are a prefix of the `ORDER BY`. This reduces the number of rows flowing through the final sort merge and any downstream pipeline steps. This optimization is controlled by the new setting `query_plan_push_limit_by_into_sort` (enabled by default). [#104000](https://github.com/ClickHouse/ClickHouse/pull/104000) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up `SELECT ... LIMIT N BY ` queries when `` are a prefix of the table's sorting key, or become one after `WHERE col = const` fixes leading columns. With this enabled the `MergeTree` reads data in primary-key order and `LIMIT BY` first filters in streaming mode with O(1) memory per sorted stream which filters out most of the data, then finally running normal `LIMIT BY` on reduced data to get the final result. Controlled by the new setting `optimize_limit_by_in_order` (enabled by default). [#105135](https://github.com/ClickHouse/ClickHouse/pull/105135) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up `LIMIT BY` queries on partitioned `MergeTree` tables by running `LIMIT BY` inside each partition's stream in parallel, instead of merging all streams into one before applying the limit. This applies when the partition expression is a deterministic function of the `LIMIT BY` columns, so no `LIMIT BY` group can span two partitions. Controlled by the new setting `allow_limit_by_partitions_independently` (enabled by default). [#105126](https://github.com/ClickHouse/ClickHouse/pull/105126) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up `LIMIT BY` queries by removing redundant key expressions: a key that is a deterministic function of the other keys is dropped (e.g. `LIMIT 5 BY x, f(x)` becomes `LIMIT 5 BY x`), and an injective function of a key is replaced by its argument (e.g. `LIMIT 5 BY toString(x)` becomes `LIMIT 5 BY x`). This evaluates fewer and cheaper expressions per row. Controlled by the new settings `optimize_limit_by_function_keys` and `optimize_injective_functions_in_limit_by`, both enabled by default. [#106818](https://github.com/ClickHouse/ClickHouse/pull/106818) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Speed up sharded aggregation, `grace_hash` join bucketing, and parallel window partitioning by replacing per-chunk column hashing with a kernel that uses hardware CRC32C. [#106538](https://github.com/ClickHouse/ClickHouse/pull/106538) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). * Optimize `L1Distance`, `L2Distance`, `L2SquaredDistance`, `LinfDistance`, and `cosineDistance` for array arguments with multi-target auto-vectorization on x86-64-v4/v3. For non-const array arguments, all of these except `LpDistance`, which uses `std::pow` with a runtime exponent, can now use compiler-vectorized kernels. The existing AVX-512 and Sapphire Rapids `BFloat16` const-argument fast paths for `L2Distance` and `cosineDistance` are preserved. [#101310](https://github.com/ClickHouse/ClickHouse/pull/101310) ([Peng](https://github.com/fastio)). * Improved performance of the `{Norm}Distance` and `dotProduct` functions. [#106007](https://github.com/ClickHouse/ClickHouse/pull/106007) ([Nikita Taranov](https://github.com/nickitat)). * Improved performance of the `arrayNorm` function. [#106211](https://github.com/ClickHouse/ClickHouse/pull/106211) ([Nikita Taranov](https://github.com/nickitat)). * The performance of `dotProduct` improved in some cases. [#106210](https://github.com/ClickHouse/ClickHouse/pull/106210) ([Nikita Taranov](https://github.com/nickitat)). * Improved performance of the AVX-512 const-left paths for `L2Distance` and `cosineDistance` on `Array(Float32)`, `Array(Float64)`, and `Array(BFloat16)` when the array length is exactly divisible by the vectorized processing width. [#104625](https://github.com/ClickHouse/ClickHouse/pull/104625) ([Sergey Kuznetsov](https://github.com/skuznetsov)). * Enabled AVX-512 codegen for amd and better codegen for `arrayDistance`. [#106505](https://github.com/ClickHouse/ClickHouse/pull/106505) ([Nikita Taranov](https://github.com/nickitat)). * Improved performance of the `L2DistanceTransposed` and `cosineDistanceTransposed` functions for the `QBit` data type. [#106701](https://github.com/ClickHouse/ClickHouse/pull/106701) ([Raufs Dunamalijevs](https://github.com/rienath)). * Improve performance of the `encrypt`, `decrypt`, `tryDecrypt`, `aes_encrypt_mysql`, and `aes_decrypt_mysql` functions by up to an order of magnitude, recovering the performance lost in the BoringSSL to OpenSSL 3.x migration (24.4). [#107339](https://github.com/ClickHouse/ClickHouse/pull/107339) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Improve performance of the `encrypt`, `decrypt`, and `halfMD5` functions by avoiding implicit per-row OpenSSL provider lookups in OpenSSL 3.x. [#99105](https://github.com/ClickHouse/ClickHouse/pull/99105) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Add SIMD implementation of `SHA1` that hashes multiple inputs in parallel using AVX-512, improving throughput. [#105459](https://github.com/ClickHouse/ClickHouse/pull/105459) ([Joanna Hulboj](https://github.com/jh0x)). * Added an AArch64 ASIMD/NEON backend for the multi-buffer `MD5` implementation. [#105563](https://github.com/ClickHouse/ClickHouse/pull/105563) ([Venkata Vineel ](https://github.com/vyalamar)). * Squash source blocks before calculating projection during `MATERIALIZE PROJECTION` to reduce the number of temporary projection parts and merge overhead. ~3.4x speedup on a 50M-row table. [#100047](https://github.com/ClickHouse/ClickHouse/pull/100047) ([Amos Bird](https://github.com/amosbird)). * Reduced `INSERT` latency for `MergeTree` tables with skip indices on `ORDER BY` columns by avoiding redundant column permutations during part writing (~17% on traces-shaped workloads). [#101101](https://github.com/ClickHouse/ClickHouse/pull/101101) ([Amos Bird](https://github.com/amosbird)). * Enabled Arrow's SIMD code paths (NEON on ARM, SSE4.2/AVX2/AVX512 on x86) and used them to speed up decoding of `BYTE_STREAM_SPLIT`-encoded Parquet `FLOAT`/`DOUBLE` columns. [#106376](https://github.com/ClickHouse/ClickHouse/pull/106376) ([Raúl Marín](https://github.com/Algunenano)). * Improved insertion performance for `LowCardinality` columns with `bloom_filter` indexes. [#106410](https://github.com/ClickHouse/ClickHouse/pull/106410) ([Michael Jarrett](https://github.com/EmeraldShift)). * Parallelize blob copies in `DiskObjectStorageTransaction::copyFile` using a per-disk thread pool. [#105089](https://github.com/ClickHouse/ClickHouse/pull/105089) ([Asya Shneerson](https://github.com/asya-ch)). * Avoid reading file contents when using the `One` input format with file-like table functions such as `file` and `s3`. [#105157](https://github.com/ClickHouse/ClickHouse/pull/105157) ([Yue Ni](https://github.com/niyue)). * Speed up `clickhouse` startup on macOS by about 3x by not exporting dynamic symbols, which the dynamic linker would otherwise process on every launch. [#107768](https://github.com/ClickHouse/ClickHouse/pull/107768) ([Raúl Marín](https://github.com/Algunenano)). * Speed up filesystem cache loading on startup by avoiding a redundant directory open per cache key. [#107414](https://github.com/ClickHouse/ClickHouse/pull/107414) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up filesystem cache loading on startup by letting directory-listing threads help load cache metadata once listing is finished, instead of leaving half the threads idle. [#107416](https://github.com/ClickHouse/ClickHouse/pull/107416) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduced `clickhouse` process startup overhead by updating the bundled WasmEdge library, which no longer generates a random hash secret in a global initializer at every startup. [#107869](https://github.com/ClickHouse/ClickHouse/pull/107869) ([Raúl Marín](https://github.com/Algunenano)). * The `MergeTree` primary key and skip indexes can now prune granules for filters where `ifNull` or `coalesce` wraps a condition, such as `ifNull(key = 0, 0)` or `coalesce(key = 0, 0)`. Such predicates — often emitted by query generators to turn a possibly-`NULL` comparison into a definite boolean — were previously opaque to index analysis and could not skip granules. This extends the existing `allow_key_condition_coalesce_rewrite` setting (enabled by default). Closes [#106264](https://github.com/ClickHouse/ClickHouse/issues/106264). [#106272](https://github.com/ClickHouse/ClickHouse/pull/106272) ([Andy Zhao](https://github.com/andyzzhao)). * Skip index evaluation on `DateTime64` types should consume lesser CPU in 26.6. [#107707](https://github.com/ClickHouse/ClickHouse/pull/107707) ([Shankar Iyer](https://github.com/shankar-iyer)). * Improved performance of text index analysis in multi-token searches by optimizing handling of rare tokens. [#98226](https://github.com/ClickHouse/ClickHouse/pull/98226) ([Anton Popov](https://github.com/CurtizJ)). * Use a cache for the posting-list compressed segments of the text index. [#106299](https://github.com/ClickHouse/ClickHouse/pull/106299) ([Anton Popov](https://github.com/CurtizJ)). * Improved the performance of generic exclusion search for queries that select large consecutive ranges of the table. [#93813](https://github.com/ClickHouse/ClickHouse/pull/93813) ([Michael Jarrett](https://github.com/EmeraldShift)). * `S3` clients with the same endpoint and bucket share a cache, avoiding duplicate region discovery. Closes [#92482](https://github.com/ClickHouse/ClickHouse/issues/92482). [#96802](https://github.com/ClickHouse/ClickHouse/pull/96802) ([Andrey Zvonov](https://github.com/zvonand)). * If a query contains a `has(, )`, the analyzer will transform the node to use a faster `in` implementation if it is feasible to do so. [#97341](https://github.com/ClickHouse/ClickHouse/pull/97341) ([Shankar Iyer](https://github.com/shankar-iyer)). * When a new job is scheduled in `ThreadPool`, wake the most recently idle worker thread (LIFO) instead of an arbitrary one. This reduces memory fragmentation from per-thread allocator caches. The order in which jobs are processed is not changed. [#100177](https://github.com/ClickHouse/ClickHouse/pull/100177) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved performance of approximate runtime filters and bloom filter indexes. [#100201](https://github.com/ClickHouse/ClickHouse/pull/100201) ([Christoph Viebig](https://github.com/cv4g)). * Allow `ASOF JOIN` to use the `parallel_hash` join algorithm, parallelizing the build across distinct equality-key values. Previously `ASOF` was unconditionally opted out of `parallel_hash`. [#105375](https://github.com/ClickHouse/ClickHouse/pull/105375) ([Greg Maher](https://github.com/gregakinman)). * Avoid reading unbounded inputs for `INTERSECT` and `EXCEPT` queries when an empty input makes the result known to be empty. [#105393](https://github.com/ClickHouse/ClickHouse/pull/105393) ([Yue Ni](https://github.com/niyue)). * Add `constinit` for various `MemoryTracker` blockers. [#105490](https://github.com/ClickHouse/ClickHouse/pull/105490) ([Azat Khuzhin](https://github.com/azat)). * Improve performance of `bitmapContains` for non-`UInt64` `groupBitmap` states by avoiding repeated `rb_max` calls during range checks. [#105960](https://github.com/ClickHouse/ClickHouse/pull/105960) ([Yue Ni](https://github.com/niyue)). * Faster discontinuous queries for `LowCardinality` columns backed by a single dictionary. [#103662](https://github.com/ClickHouse/ClickHouse/pull/103662) ([Ivan Babrou](https://github.com/bobrik)). * Added a fast path for the `HashJoin` probe when the block has a single row. [#106008](https://github.com/ClickHouse/ClickHouse/pull/106008) ([Nikita Taranov](https://github.com/nickitat)). * Predicates from queries like `SELECT ... FROM view(SELECT ... FROM remote(...)) WHERE ... ` are now pushed into the query sent to the remote shard. The optimization `allow_push_predicate_ast_for_distributed_subqueries` lacked a case for table functions. [#105986](https://github.com/ClickHouse/ClickHouse/pull/105986) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Turn on `enable_join_transitive_predicates` by default. [#103724](https://github.com/ClickHouse/ClickHouse/pull/103724) ([Alexander Gololobov](https://github.com/davenger)). * Added support for functions `multiSearchAny`, `multiSearchAnyUTF8`, and `multiMatchAny` in text indexes. Also improved text index analysis for the function `match`: now patterns with alternative groups can skip more granules. [#106279](https://github.com/ClickHouse/ClickHouse/pull/106279) ([Anton Popov](https://github.com/CurtizJ)). * Allow filters introduced after the initial PREWHERE selection (predicate pushdown, runtime filters, or explicit PREWHERE plus a WHERE set by the planner) to be merged into the existing PREWHERE on a second optimizer pass instead of staying as a separate Filter step above the `MergeTree` read. [#105445](https://github.com/ClickHouse/ClickHouse/pull/105445) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * The `QueryConditionCache` now records individually filtered-out granules even within read batches that partially pass PREWHERE, reducing the number of marks re-read by subsequent queries with the same condition. [#105335](https://github.com/ClickHouse/ClickHouse/pull/105335) ([Han Fei](https://github.com/hanfei1991)). * Added Keeper setting `nuraft_use_bg_thread_for_snapshot_io`, enabled by default, to make `NuRaft` read snapshot objects from a background thread instead of Raft worker threads. [#106285](https://github.com/ClickHouse/ClickHouse/pull/106285) ([Antonio Andelic](https://github.com/antonio2368)). * Reduce peak memory usage when applying received snapshots in ClickHouse Keeper with `KeeperMemoryStorage`. [#105851](https://github.com/ClickHouse/ClickHouse/pull/105851) ([Antonio Andelic](https://github.com/antonio2368)). * Route LLVM/JIT allocations to a dedicated jemalloc arena. Reduces RSS fragmentation caused by interleaving long-lived JIT bookkeeping with short-lived query allocations. Adds new asynchronous metrics `jemalloc.jit_arena.active_bytes`, `jemalloc.jit_arena.dirty_bytes`, `CompiledExpressionCacheBytesMax`, and `CompiledExpressionCacheCountMax` for observability. `SYSTEM DROP COMPILED EXPRESSION CACHE` now also drops the underlying CHJIT instances and purges the JIT arena. [#104113](https://github.com/ClickHouse/ClickHouse/pull/104113) ([Raúl Marín](https://github.com/Algunenano)). * Fix excessive memory reservation when reading sparse dictionary-encoded `Nullable(String)` columns from `Parquet` with the native V3 reader. [#102805](https://github.com/ClickHouse/ClickHouse/pull/102805) ([Francisco](https://github.com/Milias)). * Fixed `CREATE TABLE ... CLONE AS (and REPLACE / ATTACH PARTITION ... FROM)` on encrypted disks copying data instead of hardlinking it. [#106731](https://github.com/ClickHouse/ClickHouse/pull/106731) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Improve cardinality estimation in the query plan optimizer: a column produced by a deterministic single-argument function (e.g. `toYear(date)`) now inherits its argument's number of distinct values as an upper bound instead of being left without statistics, leading to more accurate join reordering. [#107757](https://github.com/ClickHouse/ClickHouse/pull/107757) ([Alexander Gololobov](https://github.com/davenger)). * Reduced CPU overhead of asynchronous logging under high log rates by notifying the log consumer only on the empty-to-non-empty queue transition instead of on every message. [#107352](https://github.com/ClickHouse/ClickHouse/pull/107352) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Remove duplicate calculations during query execution in more scenarios. [#106113](https://github.com/ClickHouse/ClickHouse/pull/106113) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Parallelize the processing of a recursive CTE's result: a `GROUP BY` or other operation over a large `WITH RECURSIVE` result is no longer limited to a single thread. [#107694](https://github.com/ClickHouse/ClickHouse/pull/107694) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved the performance of case-insensitive substring search (`positionCaseInsensitiveUTF8`, `ILIKE`, `multiSearchAnyCaseInsensitiveUTF8`, and related functions) by adding a NEON SIMD path on ARM and a wider AVX2 kernel on x86. [#107882](https://github.com/ClickHouse/ClickHouse/pull/107882) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a ~16% throughput regression of the `FPC` floating-point codec on ARM that was introduced when its predictor tables started using `VectorWithMemoryTracking`. [#108182](https://github.com/ClickHouse/ClickHouse/pull/108182) ([Groene AI](https://github.com/groeneai)). #### Improvement * The background `MemoryWorker` now periodically updates the server's hard memory limit based on the current memory usage and the amount of memory the kernel reports as available, so ClickHouse leaves room for other processes running on the same host. The formula is `(resident memory + system MemAvailable) * max_server_memory_usage_to_ram_ratio`. The same `max_server_memory_usage_to_ram_ratio` server setting controls both the startup cap and the dynamic adjustment; set it to `0` to disable both. To keep only the static startup/reload cap (the behavior of previous versions), set the new server setting `memory_worker_dynamic_hard_limit` to `0`. [#104964](https://github.com/ClickHouse/ClickHouse/pull/104964) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support C# PostgreSQL client in `PostgreSQL` protocol. This closes [#18611](https://github.com/ClickHouse/ClickHouse/issues/18611). [#80785](https://github.com/ClickHouse/ClickHouse/pull/80785) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Mutations now use the analyzer. [#98884](https://github.com/ClickHouse/ClickHouse/pull/98884) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Added granular control over query echoing in the CLI. The `--echo` option now accepts an optional boolean value and works in both interactive and batch mode. New options `--echo-formatted` and `--echo-query-id` control whether echoed queries are formatted and whether the `query_id` is printed, respectively. The `--hilite`/`--highlight` option now also controls highlighting of echoed queries. As a side effect, `--echo` is now a boolean-value option, so a positional query directly after a bare `--echo` is treated as its value; use `--echo --query "..."` or `--echo=false` instead. [#106191](https://github.com/ClickHouse/ClickHouse/pull/106191) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support schema evolution together with concurrent consistency checks for data lake catalogs. [#106102](https://github.com/ClickHouse/ClickHouse/pull/106102) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Allow cache disk for datalake table engines. [#102017](https://github.com/ClickHouse/ClickHouse/pull/102017) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Fix `DataLakeCatalog` databases with `catalog_type = 'onelake'` reading table data by using the OneLake Blob endpoint (`.blob.fabric.microsoft.com`) by default. Set `onelake_use_blob_endpoint = false` to keep the previous DFS endpoint (`.dfs.fabric.microsoft.com`) behavior. [#106843](https://github.com/ClickHouse/ClickHouse/pull/106843) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Update default of `S3(Azure)Queue` setting `persistent_processing_node_ttl_seconds` from `1 hour` to `6 hours` (which stands for cleanup time due to service unclean restart etc). The previous default was fine for processing nodes ttl, but it is also used for bucket lock ttl, which can be hold for longer time, therefore 1 hour was not enough. [#106838](https://github.com/ClickHouse/ClickHouse/pull/106838) ([Kseniia Sumarokova](https://github.com/kssenii)). * Decreased the default size of the `Iceberg` metadata cache from 1 GB to 128 MB. [#106492](https://github.com/ClickHouse/ClickHouse/pull/106492) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Propagate the `S3Queue` streaming task `query_id` to dependent table inserts. [#106494](https://github.com/ClickHouse/ClickHouse/pull/106494) ([Christoph Wurm](https://github.com/cwurm)). * Populate `used_storages` in `system.query_log` with the storage engine name when querying tables via `DataLakeCatalog`. [#100706](https://github.com/ClickHouse/ClickHouse/pull/100706) ([Melvyn Peignon](https://github.com/melvynator)). * You can check which 'operation' and what 'summary' Iceberg's snapshots have. Useful for testing and debugging. [#106246](https://github.com/ClickHouse/ClickHouse/pull/106246) ([Den Kalantaevskii](https://github.com/Diskein)). * Complete the final `S3` multipart upload request asynchronously via the task tracker, so it can overlap with the last part upload instead of running serially in finalize. [#105487](https://github.com/ClickHouse/ClickHouse/pull/105487) ([Asya Shneerson](https://github.com/asya-ch)). * Added four new `MergeTree` table settings to control default parameters of text indexes: `text_index_dictionary_block_size`, `text_index_dictionary_block_frontcoding_compression`, `text_index_posting_list_block_size`, and `text_index_posting_list_codec`. These settings allow tuning text index behavior at the table level without specifying parameters in every index definition. Explicit per-index arguments still take precedence. [#100626](https://github.com/ClickHouse/ClickHouse/pull/100626) ([Anton Popov](https://github.com/CurtizJ)). * Allow projections to override additional `MergeTree` settings (compression, part format, serialization) beyond just `index_granularity`, with an allow-list of allowed settings and validation. [#101170](https://github.com/ClickHouse/ClickHouse/pull/101170) ([Amos Bird](https://github.com/amosbird)). * Add `current_projection`, `current_projection_progress`, `projections_completed`, and `projections_remaining` columns to `system.merges` to expose projection merge progress. [#102611](https://github.com/ClickHouse/ClickHouse/pull/102611) ([Amos Bird](https://github.com/amosbird)). * Fixed `PREWHERE` with `IN` subquery on primary key columns not using primary key index for granule pruning, causing full table scans instead of reading only relevant granules. [#102570](https://github.com/ClickHouse/ClickHouse/pull/102570) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Function `h3PolygonToCells` now enforces the maximum array size across all polygons of a `MultiPolygon`, validates the underlying H3 library return codes, and rejects `MultiLineString` arguments instead of silently returning an empty result. [#106399](https://github.com/ClickHouse/ClickHouse/pull/106399) ([Raúl Marín](https://github.com/Algunenano)). * Add a new keeper-only `system.keeper_snapshots` table with information about local ClickHouse Keeper snapshots. [#105571](https://github.com/ClickHouse/ClickHouse/pull/105571) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add a new keeper-only `system.keeper_changelogs` table with information about local ClickHouse Keeper changelog (Raft log) files. [#105617](https://github.com/ClickHouse/ClickHouse/pull/105617) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add a keeper-only `system.keeper_cluster` table. Contains one row per Raft cluster member as seen by the current Keeper. [#105646](https://github.com/ClickHouse/ClickHouse/pull/105646) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add more Keeper profile (server-side + client-side) events for watches. Closes [#97703](https://github.com/ClickHouse/ClickHouse/issues/97703). [#105336](https://github.com/ClickHouse/ClickHouse/pull/105336) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added Keeper `coordination_settings` for `NuRaft` uncommitted log entry admission limiting and append-entries backward-probe throttling. [#106108](https://github.com/ClickHouse/ClickHouse/pull/106108) ([Antonio Andelic](https://github.com/antonio2368)). * Add `enable_compression` setting for the `mysql` table function, the `MySQL` table engine, the `MySQL` database engine, dictionary `SOURCE(MYSQL)`, and named collections. When enabled, ClickHouse negotiates MySQL protocol-level compression for all data transferred over the connection. [#103229](https://github.com/ClickHouse/ClickHouse/pull/103229) ([Bernard Lim](https://github.com/bernardlim)). * Reduced cancellation latency for queries running over the PostgreSQL wire protocol: `KILL QUERY` now interrupts output serialization within a single chunk instead of waiting for the entire chunk to be sent to the client. [#106535](https://github.com/ClickHouse/ClickHouse/pull/106535) ([Roman Vasin](https://github.com/rvasin)). * Reduced cancellation latency for queries running over the MySQL wire protocol: `KILL QUERY` now interrupts output serialization within a single chunk instead of waiting for the entire chunk to be sent to the client. [#107228](https://github.com/ClickHouse/ClickHouse/pull/107228) ([Roman Vasin](https://github.com/rvasin)). * Fixed reading of the `auth_response` length in the MySQL handshake, where a length byte `>= 128` was interpreted as a multi-gigabyte value because it was read as a signed `char`. [#107384](https://github.com/ClickHouse/ClickHouse/pull/107384) ([uwezkhan](https://github.com/uwezkhan)). * `MaterializedPostgreSQL` now maps PostgreSQL `numeric(p, 0)` columns with precision greater than 76 (e.g. `numeric(78, 0)` used for 256-bit integers) to ClickHouse `Int256` instead of failing with "Precision too big". Values that do not fit into `Int256` are rejected with a clear error. [#107431](https://github.com/ClickHouse/ClickHouse/pull/107431) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Send an exception to the client when TCP connection setup fails. [#107317](https://github.com/ClickHouse/ClickHouse/pull/107317) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Users can now insert `Avro` `Fixed` fields for 8-bit/16-bit/32-bit/64-bit integer variants. [#98139](https://github.com/ClickHouse/ClickHouse/pull/98139) ([Patrick Pichler](https://github.com/patrickpichler)). * The `AvroConfluent` format now retries the Confluent Schema Registry HTTP client on transient failures (transport timeouts, connection refused, DNS errors, HTTP `5xx`/`408`/`429`) with exponential backoff, instead of aborting the INSERT on the first network glitch. New settings `format_avro_schema_registry_max_retries` (default `5`) and `format_avro_schema_registry_retry_initial_backoff_ms` (default `100`) control the policy. Schema-validation errors (HTTP `409`, malformed Avro JSON) remain fatal. [#106661](https://github.com/ClickHouse/ClickHouse/pull/106661) ([Groene AI](https://github.com/groeneai)). * Record privileges in `system.query_log.used_privileges` for all granted access checks, including those that go through the non-throwing `isGranted` path (`checkAccessWithFilter`). Previously only privileges checked via throwing entry points (`checkAccess` / `checkGrantOption`) were recorded, which made `READ ON FILE` / `READ ON S3` / `READ ON AZURE` / `READ ON URL` invisible in the audit log for `DESCRIBE`, `CREATE TABLE AS`, and similar queries that use `file` / `s3` / `azure` / `url` table functions. Access enforcement is unchanged. [#104693](https://github.com/ClickHouse/ClickHouse/pull/104693) ([Alexey Bakharew](https://github.com/alexbakharew)). * Added asynchronous metrics `TotalUncompressedBytesOfMergeTreeTables` and `TotalUncompressedBytesOfMergeTreeTablesSystem`, reporting the total uncompressed size of data stored in MergeTree-family tables. [#106364](https://github.com/ClickHouse/ClickHouse/pull/106364) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Adds the `GlobalMemoryLimitExceeded` profile event so operators can monitor when the server-wide memory limit is hit. [#106466](https://github.com/ClickHouse/ClickHouse/pull/106466) ([Sacheendra Talluri](https://github.com/sacheendra)). * Populate the `ExecutableUserDefinedFunction*` `ProfileEvents` counters for `executable_pool` UDFs. Per-call invocation count, wall and pool-wait time, child-process CPU and peak memory, and stdin/stdout bytes are now visible in `system.query_log.ProfileEvents`. [#105010](https://github.com/ClickHouse/ClickHouse/pull/105010) ([Xu Jia](https://github.com/XuJia0210)) [#105618](https://github.com/ClickHouse/ClickHouse/pull/105618) ([Ilya Andreev](https://github.com/andreev-io)). * Added asynchronous metrics `ExecutableUserDefinedFunctionMemoryResidentBytes` and `ExecutableUserDefinedFunctionProcesses`, reporting the resident memory (`VmRSS`) and number of live processes of `executable` and `executable_pool` user-defined functions, including descendant processes and idle pool workers. [#107300](https://github.com/ClickHouse/ClickHouse/pull/107300) ([Hanzi Jiang](https://github.com/HanziJiang)). * Added three Linux-only async metrics — `MemoryThreadStacksCount`, `MemoryThreadStacksVirtual`, and `MemoryThreadStacksResident` — that report pthread stack memory separately from the rest of anonymous memory. Enabled via `asynchronous_metrics_enable_heavy_metrics`. Requires Linux 5.17+ (`prctl(PR_SET_VMA_ANON_NAME)`) and readable `/proc/self/smaps`; otherwise the metrics are absent and the limitation is reported in `system.warnings`. [#106230](https://github.com/ClickHouse/ClickHouse/pull/106230) ([Raúl Marín](https://github.com/Algunenano)). * Expose the `MemoryThreadStacks*` asynchronous metrics (resident/virtual size and count of thread stacks) on macOS. [#107752](https://github.com/ClickHouse/ClickHouse/pull/107752) ([Raúl Marín](https://github.com/Algunenano)). * Table engines now carry embedded documentation, introspectable via the new `description`, `syntax`, `examples`, `introduced_in`, and `related` columns of the `system.table_engines` table. [#106177](https://github.com/ClickHouse/ClickHouse/pull/106177) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Database engines now carry embedded documentation, introspectable via the new `description`, `syntax`, `examples`, `introduced_in`, and `related` columns of the `system.database_engines` table. [#106178](https://github.com/ClickHouse/ClickHouse/pull/106178) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Data types now carry embedded documentation, introspectable via the new `description`, `syntax`, `examples`, `introduced_in`, and `related` columns of the `system.data_type_families` table. [#106180](https://github.com/ClickHouse/ClickHouse/pull/106180) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Input/output formats now carry embedded documentation, introspectable via the new `description`, `examples`, `introduced_in`, and `related` columns of the `system.formats` table. [#106181](https://github.com/ClickHouse/ClickHouse/pull/106181) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Aggregate function combinators now carry embedded documentation, introspectable via the new `description`, `syntax`, `examples`, `introduced_in`, and `related` columns of the `system.aggregate_function_combinators` table. [#106185](https://github.com/ClickHouse/ClickHouse/pull/106185) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added the `--chime` command-line option to `clickhouse-client`, `clickhouse-local`, and the embedded client. When a query finishes (on success or on error) after running for at least N seconds, the client writes the ASCII `BEL` control character (`\x07`) to stderr. Terminals decide whether to make a sound or a visual flash based on the user's preferences. Enabled by default with a 5-second threshold; pass `--chime N` for a custom threshold or `--chime 0` to disable. Closes [#92718](https://github.com/ClickHouse/ClickHouse/issues/92718). [#104545](https://github.com/ClickHouse/ClickHouse/pull/104545) ([Groene AI](https://github.com/groeneai)). * In `clickhouse-client`, when the terminal does not support bracketed paste, embedded newlines from a pasted multi-line query are now folded into the same edit buffer instead of switching to the continuation prompt for every line. The pasted query stays under a single prompt and the arrow keys navigate across its lines. This is a best-effort improvement that relies on the typical TTY-buffered paste behavior. [#104299](https://github.com/ClickHouse/ClickHouse/pull/104299) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Now also works with syntax highlighting disabled (`--highlight 0`). [#106665](https://github.com/ClickHouse/ClickHouse/pull/106665) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add MySQL-style runtime pager control for `clickhouse-client`/`clickhouse-local`. [#105706](https://github.com/ClickHouse/ClickHouse/pull/105706) ([Azat Khuzhin](https://github.com/azat)). * `clickhouse-client` now respects non-UTF-8 terminals: when the terminal character encoding is not UTF-8 (e.g. with `LANG=C`), the `Pretty` formats fall back to ASCII grid borders instead of printing Unicode box-drawing characters that would corrupt the terminal. [#106213](https://github.com/ClickHouse/ClickHouse/pull/106213) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Sanitize server-supplied display strings in `clickhouse-client`'s `Hello` packet handling (server name, time zone, display name, password-rule patterns and messages) so a hostile server cannot inject bytes that the client renders verbatim, and cap their size to defend against unbounded allocation. [#105243](https://github.com/ClickHouse/ClickHouse/pull/105243) ([Raúl Marín](https://github.com/Algunenano)). * The Web UI now displays the result as an image when a query uses an image output format such as `FORMAT PNG`, instead of showing the raw bytes as a table. [#107638](https://github.com/ClickHouse/ClickHouse/pull/107638) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add setting `output_format_pretty_use_nbsp_for_padding` to render table-layout padding in table-style `Pretty` formats as `U+00A0` NO-BREAK SPACE when `output_format_pretty_grid_charset` is `UTF-8`. This helps copied `Pretty` output keep table alignment in tools that collapse regular spaces. The setting is disabled by default, and `ASCII` charset output keeps regular spaces. [#103559](https://github.com/ClickHouse/ClickHouse/pull/103559) ([Ashrith Bandla](https://github.com/ashrithb)). * The universal install script now also installs `clickhousectl` into `~/.local/bin` on Linux and macOS, with a `chctl` symlink. Set `CLICKHOUSE_ONLY=1` to skip. [#105399](https://github.com/ClickHouse/ClickHouse/pull/105399) ([Alasdair Brown](https://github.com/sdairs)). * Fixed a spurious `Syntax error: Unterminated quoted string` that could occur intermittently when installing ClickHouse via `curl https://clickhouse.com/ | sh` while a `clickhouse` binary already existed in the current directory. [#106417](https://github.com/ClickHouse/ClickHouse/pull/106417) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the left panel height on the `play.html` page when the web terminal is open, so the terminal toggle button at the bottom stays visible. [#105087](https://github.com/ClickHouse/ClickHouse/pull/105087) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Avoid selecting the only query in the Web UI after running it. [#105711](https://github.com/ClickHouse/ClickHouse/pull/105711) ([Joey Yu](https://github.com/itsjoeoui)). * The Web SQL UI now expands the query editor when a query is at most 30% taller than the current editor height. [#105713](https://github.com/ClickHouse/ClickHouse/pull/105713) ([Joey Yu](https://github.com/itsjoeoui)). * In the Web UI, lay out the bottom menu icons (Web Terminal and GitHub) horizontally when the left panel is expanded, giving the list of databases and tables more vertical space. [#106427](https://github.com/ClickHouse/ClickHouse/pull/106427) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In the Web UI (`play.html`), pressing `Escape` now deactivates the currently selected result-table cell. [#107777](https://github.com/ClickHouse/ClickHouse/pull/107777) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improvements to the database panel in the Web SQL UI (Play): a tooltip on the database icon, a button to hide the panel, full-width clickable database/table names, and a fix for the panel width shifting when the scrollbar appears. [#108013](https://github.com/ClickHouse/ClickHouse/pull/108013) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add a server setting (`min_allocation_size_to_throw_on_memory_limit`) to allow any allocation to throw `MEMORY_LIMIT_EXCEEDED`, this should help with preventing OOMs. [#105265](https://github.com/ClickHouse/ClickHouse/pull/105265) ([Azat Khuzhin](https://github.com/azat)). * Allow `NONE` authentication for SSH (prevents asking password for `no_password` users). [#105476](https://github.com/ClickHouse/ClickHouse/pull/105476) ([Azat Khuzhin](https://github.com/azat)). * Reset the supplementary group list in `clickhouse su` before dropping privileges, matching the behavior of `/bin/su` and `sudo`. Previously the dropped process inherited the invoker's supplementary groups. [#105247](https://github.com/ClickHouse/ClickHouse/pull/105247) ([Raúl Marín](https://github.com/Algunenano)). * Add `BFloat16` support for numeric predicate (`isFinite`, `isNaN`) functions. [#105391](https://github.com/ClickHouse/ClickHouse/pull/105391) ([Mohamed Hussain S](https://github.com/mohhddhassan)). * Reduce memory usage of polygon dictionaries (`polygon_index_cell`, `polygon_index_each`) and make `system.dictionaries.bytes_allocated` correctly reflect the lookup index they build. [#105431](https://github.com/ClickHouse/ClickHouse/pull/105431) ([Raúl Marín](https://github.com/Algunenano)). * Fixes Kafka table engine consumers that kept using a short poll interval after partition assignment, so they return to the configured `kafka_poll_timeout_ms` and avoid excessive empty polls, smaller inserted parts, and extra merge overhead after rebalances. [#100431](https://github.com/ClickHouse/ClickHouse/pull/100431) ([sugaf1204](https://github.com/sugaf1204)). * Identify columns by position (instead of by name) when removing unused columns in the query plan. This enables unused-column removal when duplicated column names are present. [#100586](https://github.com/ClickHouse/ClickHouse/pull/100586) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * ClickHouse throws a proper exception for attempts to use `table_readonly` for replicated engines, with better exception handling for DDL queries (and `DatabaseReplicated`) related to this setting. [#100950](https://github.com/ClickHouse/ClickHouse/pull/100950) ([alesapin](https://github.com/alesapin)). `OPTIMIZE TABLE ... ON CLUSTER` and other DDL no longer hang when the target table has `table_readonly = 1`. The setting now throws a new dedicated error code `TABLE_IS_PERMANENTLY_READ_ONLY`, which `DDLWorker` treats as non-retriable (distinct from the transient `TABLE_IS_READ_ONLY` that arises during temporary `ReplicatedMergeTree` ZooKeeper disconnects). The `table_readonly` setting is also now explicitly rejected for `ReplicatedMergeTree`, both at creation and via `ALTER MODIFY SETTING`. Follow-up to [#97652](https://github.com/ClickHouse/ClickHouse/issues/97652). [#105109](https://github.com/ClickHouse/ClickHouse/pull/105109) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed refreshable materialized view refreshes that could fail with `UNKNOWN_TABLE` after the target table had been replaced (for example via `EXCHANGE TABLES` or drop-and-recreate): the refresh now resolves the target table by name instead of relying on a stale UUID. [#102724](https://github.com/ClickHouse/ClickHouse/pull/102724) ([Seva Potapov](https://github.com/seva-potapov)). * Added a setting `wait_for_part_commit_in_dependent_materialized_views` (default `false`). When a materialized view's `SELECT` reads back from its own source table (for example via an `INNER JOIN` to the source), the cascade could miss the row currently being inserted, because the inserted part is committed only after the dependent views run. Enabling this setting commits the part before the dependent views are triggered, so they observe the in-flight row, at the cost of less insert parallelism. [#105943](https://github.com/ClickHouse/ClickHouse/pull/105943) ([Elmi Ahmadov](https://github.com/ahmadov)). * Do not swallow fatal errors (e.g. `MEMORY_LIMIT_EXCEEDED`) in `tryDeserialize`. [#106808](https://github.com/ClickHouse/ClickHouse/pull/106808) ([Azat Khuzhin](https://github.com/azat)). * Check `CREATE TEMPORARY TABLE` grant lazily in `InverseDictionaryLookupPass`. [#107098](https://github.com/ClickHouse/ClickHouse/pull/107098) ([Azat Khuzhin](https://github.com/azat)). * Fix an `ILLEGAL_TYPE_OF_ARGUMENT` error for distributed queries with `serialize_query_plan = 1` that contain a lambda with a constant argument (e.g. `arrayMap(t -> t.2, ...)`). Constant columns of `ActionsDAG` `INPUT` nodes are now preserved during query plan serialization. [#107124](https://github.com/ClickHouse/ClickHouse/pull/107124) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Deserialization of the states of the `contingency`, `cramersV`, `cramersVBiasCorrected`, and `theilsU` aggregate functions now validates that the stored counts form a consistent contingency table and throws a `CORRUPTED_DATA` exception otherwise. Closes [#106899](https://github.com/ClickHouse/ClickHouse/issues/106899). [#107185](https://github.com/ClickHouse/ClickHouse/pull/107185) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Preserve original error codes for exceptions wrapped in `arrow::Status`. [#107267](https://github.com/ClickHouse/ClickHouse/pull/107267) ([Azat Khuzhin](https://github.com/azat)). * Support `keyed_by_normalized_query_hash` for quotas defined in the static server configuration (`users.xml`), matching the existing `CREATE QUOTA ... KEYED BY normalized_query_hash` DDL syntax. [#107654](https://github.com/ClickHouse/ClickHouse/pull/107654) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The compatibility setting no longer applies obsolete settings, so it does not mark them as changed or produce obsolete-setting warnings. [#107737](https://github.com/ClickHouse/ClickHouse/pull/107737) ([Maxim Orlovsky](https://github.com/UberDever)). * Async inserts no longer log the full list of `query_id`s at `trace`/`debug` level, which since 26.2 could blow up `text_log` on services with heavy async insert traffic. Detailed lines are now at `test` level. [#107852](https://github.com/ClickHouse/ClickHouse/pull/107852) ([Sema Checherinda](https://github.com/CheSema)). * Filesystem cache improvement: stop invalidated priority entries from pinning `KeyMetadata`. Follow up to [#106387](https://github.com/ClickHouse/ClickHouse/pull/106387). [#107903](https://github.com/ClickHouse/ClickHouse/pull/107903) ([Kseniia Sumarokova](https://github.com/kssenii)). * `FileCache` metrics improvement. [#106116](https://github.com/ClickHouse/ClickHouse/pull/106116) ([Kseniia Sumarokova](https://github.com/kssenii)). * Do not print the "Stack trace (when copying this message, always include the lines below):" preamble in exception messages when the stack trace is actually empty. [#106524](https://github.com/ClickHouse/ClickHouse/pull/106524) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a potential overflow in `roundUpToMultiple` when computing filesystem cache boundary alignment for very large offsets. [#92579](https://github.com/ClickHouse/ClickHouse/pull/92579) ([Bharat Nallan](https://github.com/bharatnc)). * `EXPLAIN SYNTAX` now formats operators as function calls consistently in explain output (for example `plus(1, 2)` instead of `1 + 2`). [#94681](https://github.com/ClickHouse/ClickHouse/pull/94681) ([Mohamed Abdelhalim ](https://github.com/1abdelhalim)). * Improved table name hints in error messages: no longer suggests the exact same name, and includes the database name in the suggestion (e.g., "Maybe you meant `other_db.my_table`?"). [#95116](https://github.com/ClickHouse/ClickHouse/pull/95116) ([Mathuranath Metivier](https://github.com/matt-metivier)). * Improved error message for unresolved identifiers in queries without a `FROM` clause to suggest adding one. [#101769](https://github.com/ClickHouse/ClickHouse/pull/101769) ([Yash ](https://github.com/Onyx2406)). * Add `EXPLAIN PIPELINE` option to compact repeated processor chains. [#104662](https://github.com/ClickHouse/ClickHouse/pull/104662) ([Yue Ni](https://github.com/niyue)). * Fix a heap-buffer-overflow read in syntax-error message construction (`UTF8::computeWidthImpl` via `parseQuery.cpp`) that occurred when the parser had backtracked past the first semicolon / end-of-stream token; release builds were splicing bytes from neighboring heap memory into the displayed error, ASan builds aborted. [#105086](https://github.com/ClickHouse/ClickHouse/pull/105086) ([Groene AI](https://github.com/groeneai)). * Make hive partition strategy a default under compatibility setting `file_like_engine_default_partition_strategy`. [#86746](https://github.com/ClickHouse/ClickHouse/pull/86746) ([Kseniia Sumarokova](https://github.com/kssenii)). * Use a larger (8 MiB) thread stack on macOS to match the Linux default, fixing rare server crashes from stack overflow during JIT compilation. [#107033](https://github.com/ClickHouse/ClickHouse/pull/107033) ([Raúl Marín](https://github.com/Algunenano)). * The temporary storage size limit of `clickhouse-local` is no longer hard-coded to 1 GiB. The default is raised to 1 TiB and can be configured with the `max_temporary_data_on_disk_size` server setting. [#106689](https://github.com/ClickHouse/ClickHouse/pull/106689) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `read-checksums` command to `clickhouse-disks`. It reads the `checksums.txt` file of a `MergeTreeDataPart` and prints it as a human-readable, tab-separated table. [#106901](https://github.com/ClickHouse/ClickHouse/pull/106901) ([Asya Shneerson](https://github.com/asya-ch)). * Added a `read-bitmap` command to `clickhouse-disks`. [#107834](https://github.com/ClickHouse/ClickHouse/pull/107834) ([murphy-4o](https://github.com/murphy-4o)). * Chdig: enable TLS for secure connections (previous leads to either "unknown setting `skip_verify`" or "connection reset by peer"). [#105864](https://github.com/ClickHouse/ClickHouse/pull/105864) ([Azat Khuzhin](https://github.com/azat)). * [`chdig` v26.5.1](https://github.com/azat/chdig/releases/tag/v26.5.1). [#105918](https://github.com/ClickHouse/ClickHouse/pull/105918) ([Azat Khuzhin](https://github.com/azat)). * [`chdig` v26.6.1](https://github.com/azat/chdig/releases/v26.6.1) - `{asynchronous_,}metric_log`, query patterns, heatmaps, logs/traces stored on disk (previously it can use >40GiB for 1 hour of traces). [#107678](https://github.com/ClickHouse/ClickHouse/pull/107678) ([Azat Khuzhin](https://github.com/azat)). * The server config (`programs/server/config.xml`) now sets `keep_alive_timeout` to 30 (formerly 10), aligning with the actual documented and implemented default. [#107779](https://github.com/ClickHouse/ClickHouse/pull/107779) ([Dan Checkoway](https://github.com/dcheckoway)). * The default for the server setting `disk_connections_rcvbuf` changed from `0` (kernel TCP autotuning) to `204800` (200 KB), capping the per-socket TCP receive buffer for object-storage (`S3`/`Azure`/`GCS`) disk connections. [#107859](https://github.com/ClickHouse/ClickHouse/pull/107859) ([Sema Checherinda](https://github.com/CheSema)). * The `PREWHERE` optimizer now groups conjuncts that reference the same column set before estimating selectivity, so that conditions like `a > 2500 AND a < 2502` are evaluated together as a combined range rather than as two independent predicates, producing a more accurate selectivity estimate. [#106337](https://github.com/ClickHouse/ClickHouse/pull/106337) ([Han Fei](https://github.com/hanfei1991)). * Add silk fiber aware secure and plain socket implementations. [#107680](https://github.com/ClickHouse/ClickHouse/pull/107680) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed a garbled error message (swapped arguments) and a typo in the `nested` function. [#108031](https://github.com/ClickHouse/ClickHouse/pull/108031) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a `sed` command to `clickhouse-disks` that applies a sed expression to a given path, in-place. [#107131](https://github.com/ClickHouse/ClickHouse/pull/107131) ([Asya Shneerson](https://github.com/asya-ch)). * Added the create-time `materialized_postgresql_use_extended_date_and_time_types` setting for the `MaterializedPostgreSQL` database engine. By default (enabled), PostgreSQL `date`/`timestamp` columns are inferred as `Date32`/`DateTime64`; setting it to `0` at `CREATE DATABASE` time infers the narrower `Date`/`DateTime` types. [#107428](https://github.com/ClickHouse/ClickHouse/pull/107428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Bug Fix (user-visible misbehavior in an official stable release) * Make possible to drop detached parts with the `tryN` suffix. [#58957](https://github.com/ClickHouse/ClickHouse/pull/58957) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix a `MULTIPLE_EXPRESSIONS_FOR_ALIAS` exception for queries with duplicate projection aliases (for example `SELECT *, day + 365 AS day`) inside nested subqueries when running with parallel replicas. [#80310](https://github.com/ClickHouse/ClickHouse/pull/80310) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a bug with `splitMultipartQuery` throwing an "Empty query" error for queries that end with comment after semicolon. [#85491](https://github.com/ClickHouse/ClickHouse/pull/85491) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix an exception with `ARRAY JOIN`s: `Function writeSlice expects same column types for GenericArraySlice and GenericArraySink` which happens when `LowCardinality` numeric types are used (issue [#57243](https://github.com/ClickHouse/ClickHouse/issues/57243)). [#91784](https://github.com/ClickHouse/ClickHouse/pull/91784) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` exception when using `LIMIT BY` with constant columns alongside `DISTINCT` and `ORDER BY` with the new analyzer. [#93195](https://github.com/ClickHouse/ClickHouse/pull/93195) ([Ashrith Bandla](https://github.com/ashrithb)). * Fix `NOT NULL` columns being silently created as `Nullable` when `data_type_default_nullable = 1` and the table is created in a `Replicated` database or via `ON CLUSTER`. [#97572](https://github.com/ClickHouse/ClickHouse/pull/97572) ([xiaohuanlin](https://github.com/xiaohuanlin)). * Fixed HiveCatalog connection stability by adding automatic retry mechanism and reconnection logic for handling TTransportException errors when communicating with Hive Metastore. [#98471](https://github.com/ClickHouse/ClickHouse/pull/98471) ([Dmitriy Borisenko](https://github.com/otselnik)). * Fixed `nested` function (used internally by `ARRAY JOIN`) stripping `LowCardinality` from column types, causing `Array(LowCardinality(String))` to become `Array(String)` in output. Closes [#95582](https://github.com/ClickHouse/ClickHouse/issues/95582). [#98974](https://github.com/ClickHouse/ClickHouse/pull/98974) ([Yash ](https://github.com/Onyx2406)). * Fix "Distributed task iterator is not initialized" exception when using `url`, `s3`, or similar table functions in queries with parallel replicas enabled. [#100146](https://github.com/ClickHouse/ClickHouse/pull/100146) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception when reading Iceberg tables whose format version was upgraded by an external tool (e.g. Spark). [#100407](https://github.com/ClickHouse/ClickHouse/pull/100407) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Functions `like`, `ilike`, `notLike`, `notILike`, and `match` now support constant haystack with non-constant needle (e.g. `'foo' LIKE pattern_column`), which previously threw `ILLEGAL_COLUMN`. [#100479](https://github.com/ClickHouse/ClickHouse/pull/100479) ([Yash ](https://github.com/Onyx2406)). * Fixed wrong row count returned by a `MaterializedView` query with `query_plan_enable_optimizations = 0` when the view maps an integer column to a `Bool` column. [#100692](https://github.com/ClickHouse/ClickHouse/pull/100692) ([Maksim Moisiuk](https://github.com/Maximus5)). * Fix an exception (`LOGICAL_ERROR: 'PREWHERE passed to format that doesn't support it'`) when reading Iceberg tables containing ORC data files with PREWHERE optimization enabled. [#101206](https://github.com/ClickHouse/ClickHouse/pull/101206) ([Groene AI](https://github.com/groeneai)). * Fix race in RestCatalog. [#101216](https://github.com/ClickHouse/ClickHouse/pull/101216) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` error when selecting from a VIEW over a table with a normal projection. [#101218](https://github.com/ClickHouse/ClickHouse/pull/101218) ([Amos Bird](https://github.com/amosbird)). * Fix stale metadata in FileLog when a file is deleted and recreated. [#101408](https://github.com/ClickHouse/ClickHouse/pull/101408) ([Azat Khuzhin](https://github.com/azat)). * Fix join reordering silently dropping unmatched rows of a `RIGHT`/`LEFT JOIN` when it is comma-joined (cross) with another table, e.g. `t1 RIGHT JOIN t2 ON t1.c = t2.c, t3`. The query previously returned the inner-join result instead of the outer-join one. [#101684](https://github.com/ClickHouse/ClickHouse/pull/101684) ([Groene AI](https://github.com/groeneai)). * Fixed `FORMAT` clause being consumed by `INSERT` instead of applying to `EXPLAIN` output in `EXPLAIN INSERT INTO ... SELECT ... FORMAT ...`. [#101772](https://github.com/ClickHouse/ClickHouse/pull/101772) ([Yash ](https://github.com/Onyx2406)). * Fixed incorrect compression codec selection for `MergeTree` parts when the table-level `default_compression_codec` setting was explicitly configured. Parts written on insert, during merges, and for projections used the server-wide default codec instead of the table-level setting (the empty part produced by a fully-deleting mutation is now covered as well). [#101784](https://github.com/ClickHouse/ClickHouse/pull/101784) ([Yash ](https://github.com/Onyx2406)). * Reject negative `Float64` values (e.g. `-100.5`) in workload settings like `max_bytes_per_second`, `max_cpus`, etc. Previously only negative integers were validated, allowing negative floats to silently create broken scheduler nodes. Closes [#101825](https://github.com/ClickHouse/ClickHouse/issues/101825). [#101842](https://github.com/ClickHouse/ClickHouse/pull/101842) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` ("Port is not connected", code 49) that could occur when executing queries involving a VIEW with aggregation inside a JOIN. [#102574](https://github.com/ClickHouse/ClickHouse/pull/102574) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix inconsistent part metadata after mutations of columns with non-default serializations. [#102817](https://github.com/ClickHouse/ClickHouse/pull/102817) ([Eduard Karacharov](https://github.com/korowa)). * Fix updating metadata before executing mutation resolves [#96806](https://github.com/ClickHouse/ClickHouse/issues/96806). [#102882](https://github.com/ClickHouse/ClickHouse/pull/102882) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix SELECT queries being significantly slower when concurrent INSERTs are running. Previously an INSERT pipeline reserved CPU slots up to `max_threads` at query start even when most slots were never used, starving concurrent SELECTs. CPU slot allocation is now demand-driven: the pipeline only requests slots as it actually pushes parallelizable work. Applies both to concurrency control and to the preemptive CPU scheduler for workloads. New server setting `concurrent_threads_lazy_allocation` (default `true`) acts as a rollback lever. [#102928](https://github.com/ClickHouse/ClickHouse/pull/102928) ([Seva Potapov](https://github.com/seva-potapov)). * Fix `NULL` propagation when reading subcolumns extracted from `Nullable(Tuple(...))` columns. For example, for `tup Nullable(Tuple(s Nullable(String)))`, `SELECT tup.s` now correctly returns `NULL` in rows where the outer tuple is `NULL` instead of garbage values. This covers all element types that can represent `NULL`: `Nullable`, `Dynamic`, `Variant` and `LowCardinality(Nullable(...))`. Closes [#105356](https://github.com/ClickHouse/ClickHouse/issues/105356). [#102942](https://github.com/ClickHouse/ClickHouse/pull/102942) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Throw `INCORRECT_DATA` instead of `LOGICAL_ERROR` when client-supplied data in the Native format is truncated or malformed. [#102975](https://github.com/ClickHouse/ClickHouse/pull/102975) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Added cancellation check to ObjectStorage read operation. Closes [#98165](https://github.com/ClickHouse/ClickHouse/issues/98165). [#103016](https://github.com/ClickHouse/ClickHouse/pull/103016) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Respect `input_format_binary_max_type_complexity` in decoding aggregate function parameters. Closes [#102903](https://github.com/ClickHouse/ClickHouse/issues/102903). [#103026](https://github.com/ClickHouse/ClickHouse/pull/103026) ([Pavel Kruglov](https://github.com/Avogar)). * Fix nullable inference for geo columns in Arrow-based Parquet reader. Closes [#101845](https://github.com/ClickHouse/ClickHouse/issues/101845). [#103032](https://github.com/ClickHouse/ClickHouse/pull/103032) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed `recursiveRemoveLowCardinality` erasing custom geometry type names (e.g. `LineString` vs `Ring`, `MultiLineString` vs `Polygon`), which caused misinterpretation of the geometry type. Closes [#103207](https://github.com/ClickHouse/ClickHouse/issues/103207). [#103041](https://github.com/ClickHouse/ClickHouse/pull/103041) ([Joanna Hulboj](https://github.com/jh0x)). * Fixed `input_format_max_block_size_bytes` being silently ignored during `INSERT` parsing when `max_insert_block_size_bytes` is `0` (the default). The setting now correctly limits the size of blocks produced by row input formats. [#103068](https://github.com/ClickHouse/ClickHouse/pull/103068) ([Kirill Kopnev](https://github.com/Fgrtue)). * Fixed ClickHouse occasionally producing invalid GSSAPI tokens due to incorrect stripping of trailing null bytes. [#103114](https://github.com/ClickHouse/ClickHouse/pull/103114) ([Michael Jarrett](https://github.com/EmeraldShift)). * Fix analyzer-time constant folding for short-circuit functions (`if`, `multiIf`, `and`, `or`, etc.) so that statically unreachable branches no longer raise exceptions at analysis time. For example, `WITH 0 AS n SELECT multiIf(n = 0, 0, intDiv(100, n))` now correctly returns `0` instead of failing with a division-by-zero error. [#103157](https://github.com/ClickHouse/ClickHouse/pull/103157) ([Peng](https://github.com/fastio)). * `EXPLAIN SYNTAX` expands parameterized views. [#103263](https://github.com/ClickHouse/ClickHouse/pull/103263) ([Jordi Villar](https://github.com/jrdi)). * Fix data corruption when writing Parquet (and other trailer-bearing formats such as ORC and Arrow) to HDFS via `INSERT INTO FUNCTION hdfs(...)`. Since 26.1, `WriteBufferFromHDFS` did not flush its working buffer on `finalize()`, so the last up to `DBMS_DEFAULT_BUFFER_SIZE` bytes of every file were silently lost, including the Parquet `PAR1` footer. Reading such files returned `Not a Parquet file (wrong magic bytes at the end of file)`. [#103268](https://github.com/ClickHouse/ClickHouse/pull/103268) ([Groene AI](https://github.com/groeneai)). * Fix wrong results and a possible logical error for correlated subqueries when a join size limit (`max_rows_in_join` / `max_bytes_in_join`) is set together with `join_overflow_mode = 'break'`. The join created internally to evaluate a correlated subquery now ignores those user limits, so it can no longer stop early and drop rows. [#103322](https://github.com/ClickHouse/ClickHouse/pull/103322) ([Groene AI](https://github.com/groeneai)). * Fix a bug where `ALTER TABLE ... MODIFY SETTING` on an `EmbeddedRocksDB` table could persist an invalid setting value to the table metadata file even when the server rejected the query. On the next server restart the table would fail to attach with `CANNOT_PARSE_BOOL` (or a similar parsing error), and in databases where load failures are fatal the server would refuse to start. Invalid setting values are now rejected before any metadata is written. [#103417](https://github.com/ClickHouse/ClickHouse/pull/103417) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort when creating a table with an object-storage engine (`AzureBlobStorage`, `DeltaLakeAzure`, `S3`, `HDFS`) using an unsupported number of arguments. The server now returns a clean `NUMBER_OF_ARGUMENTS_DOESNT_MATCH` error instead of aborting in debug/sanitizer builds. [#103544](https://github.com/ClickHouse/ClickHouse/pull/103544) ([Groene AI](https://github.com/groeneai)). * Fix a server crash in `JSON` and `Dynamic` data type parameter parsing when the abstract syntax tree (AST) was structurally malformed (for example produced by the AST fuzzer): the `equals(name, value)` parameter expression could be left with fewer than two children, which `DataTypeObject` and `DataTypeDynamic` then dereferenced without bounds checking. [#103545](https://github.com/ClickHouse/ClickHouse/pull/103545) ([Groene AI](https://github.com/groeneai)). * Fix logical errors in analyzer with column names collision in lambda used in prewhere. Closes [#103584](https://github.com/ClickHouse/ClickHouse/issues/103584). [#103627](https://github.com/ClickHouse/ClickHouse/pull/103627) ([Pavel Kruglov](https://github.com/Avogar)). * Fix sporadic `Logical error: 'Database not found'` from `DataLakeConfiguration::getCatalog` when an `Iceberg` engine table is loaded inside a regular database during async metadata loading. [#103775](https://github.com/ClickHouse/ClickHouse/pull/103775) ([Groene AI](https://github.com/groeneai)). * Fix `MULTIPLE_EXPRESSIONS_FOR_ALIAS` errors thrown by remote replicas when running queries that reference projection aliases inside `PREWHERE` / `WHERE` / `HAVING` / `QUALIFY` (e.g. `SELECT x AS a, y AS b, (a AND b) AS c FROM t PREWHERE c`) or `SELECT *` over self-joins with overlapping column names, with parallel replicas and `parallel_replicas_local_plan = 0`. Closes [#74324](https://github.com/ClickHouse/ClickHouse/issues/74324). [#103806](https://github.com/ClickHouse/ClickHouse/pull/103806) ([Groene AI](https://github.com/groeneai)). * Fix server abort when a query uses nested `coalesce`/`ifNull` comparisons (e.g. `WHERE coalesce(a, b, coalesce(c, d), e) = const`) on a `MergeTree` table with multiple `minmax` skip indexes and `use_skip_indexes_for_disjunctions = 1`. The skip-index rewrite of `(coalesce(...), const)` is now applied recursively to inner coalesce arguments, so the per-index `KeyCondition` RPN matches the template's RPN as the disjunction-tracking code already assumes. [#103929](https://github.com/ClickHouse/ClickHouse/pull/103929) ([Groene AI](https://github.com/groeneai)). * Fix `max_rows_to_transfer` and `max_bytes_to_transfer` being silently ignored for `GLOBAL IN` and `GLOBAL JOIN` queries under the new analyzer. The settings now raise `SET_SIZE_LIMIT_EXCEEDED` (or break, depending on `transfer_overflow_mode`) when the materialised external table exceeds the configured limit, matching the behavior of the old analyzer. Closes [#103333](https://github.com/ClickHouse/ClickHouse/issues/103333). [#104119](https://github.com/ClickHouse/ClickHouse/pull/104119) ([Groene AI](https://github.com/groeneai)). * Fix excessive catalog/S3 metadata reads when an INSERT or DDL statement references a non-existent table in a DataLake catalog database with `show_data_lake_catalogs_in_system_tables` enabled. The typo-hint suggestion path loaded full per-table Iceberg metadata for the whole catalog, which could exhaust memory on large catalogs. [#104124](https://github.com/ClickHouse/ClickHouse/pull/104124) ([DQ](https://github.com/il9ue)). * Fix a crash (`LOGICAL_ERROR` "Columns are assumed to be of identical types, but they are different in Nullable" in debug builds, a `ColumnString::compareAt` null-pointer SIGSEGV in release builds) and a wrong result that could occur in queries using `direct` `join_algorithm` with a `MergeTree` right-side table when the lookup plan reordered columns during optimization. The right-side columns could land in wrong-named slots, so a filter or comparison on the join key saw a column of the wrong type. Surfaced by AST fuzzer as STID 2139-5111 and reported in [#107272](https://github.com/ClickHouse/ClickHouse/issues/107272). [#104174](https://github.com/ClickHouse/ClickHouse/pull/104174) ([Groene AI](https://github.com/groeneai)). * Reject out-of-range integer partition ID for `Date` instead of silent overflow or `LOGICAL_ERROR` on `ALTER`. [#104250](https://github.com/ClickHouse/ClickHouse/pull/104250) ([Azat Khuzhin](https://github.com/azat)). * Disable `additional_table_filters` with parallel replicas and the analyzer without query plan serialization. Previously it could lead to incorrect results. [#104296](https://github.com/ClickHouse/ClickHouse/pull/104296) ([Azat Khuzhin](https://github.com/azat)). * Fix `dictGetOrNull` silently overwriting other columns in the `SELECT` projection with `NULL` when called with a `Nullable` key column whose values are missing in the dictionary. The function was mutating an input-aliased null map in place; it now deep-clones the result column before mutation. Closes [#73633](https://github.com/ClickHouse/ClickHouse/issues/73633). [#104327](https://github.com/ClickHouse/ClickHouse/pull/104327) ([Groene AI](https://github.com/groeneai)). * Fix a broken patch part after `ALTER TABLE ... DROP PARTITION ID 'patch-...'` followed by `DETACH`/`ATTACH TABLE`. Previously the empty covering part was written without `partition.dat` and `source_parts.dat`, leading to a `broken-on-start_patch-...` entry in `system.detached_parts` after the next attach or server restart. Closes [#93132](https://github.com/ClickHouse/ClickHouse/issues/93132). Closes [#102103](https://github.com/ClickHouse/ClickHouse/issues/102103). [#104353](https://github.com/ClickHouse/ClickHouse/pull/104353) ([Groene AI](https://github.com/groeneai)). * Fixed streaming `INSERT` with `input_format_max_block_wait_ms` for the `HTTP` interface and for `INSERT SELECT FROM input`, so partial input blocks are flushed before the request finishes. [#104534](https://github.com/ClickHouse/ClickHouse/pull/104534) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a bug where queries combining `arrayJoin` with `ORDER BY ... LIMIT` after a `JOIN` could silently return zero rows. The query plan optimization that lifts function evaluation above the `SortingStep` no longer applies when the lifted expression contains `arrayJoin`, since `arrayJoin` can change the number of rows. Closes [#82279](https://github.com/ClickHouse/ClickHouse/issues/82279). [#104558](https://github.com/ClickHouse/ClickHouse/pull/104558) ([Groene AI](https://github.com/groeneai)). * Fix a rare server abort during `MergeTree` family table destruction. The abort could occur when `shutdown` raised an exception before the background-job scheduler was fully stopped: a background task firing in that narrow window would dispatch a virtual call through a partially-destroyed storage object and hit `__cxa_pure_virtual`. [#104561](https://github.com/ClickHouse/ClickHouse/pull/104561) ([Groene AI](https://github.com/groeneai)). * Fix inconsistent result in `JSON` combined subcolumn with a type hint containing null value. [#104584](https://github.com/ClickHouse/ClickHouse/pull/104584) ([Pavel Kruglov](https://github.com/Avogar)). * Fix wrong results or missed projection when an aggregate projection contains multiple `sumIf` aggregates with different `IN (...)` conditions. [#104765](https://github.com/ClickHouse/ClickHouse/pull/104765) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix wrong row count from `arrayJoin()` used inside `JOIN ON`: the result was multiplied by the array length an extra time. [#104785](https://github.com/ClickHouse/ClickHouse/pull/104785) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix `deltaSumTimestamp` returning wrong results for signed integer types crossing zero. Resolves [#104750](https://github.com/ClickHouse/ClickHouse/issues/104750). [#104830](https://github.com/ClickHouse/ClickHouse/pull/104830) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix block structure mismatch in UnionStep after filter push-down with NULL constants. Closes [#104821](https://github.com/ClickHouse/ClickHouse/issues/104821). [#104853](https://github.com/ClickHouse/ClickHouse/pull/104853) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `Logical error: 'Metadata is not initialized'` raised by `DELETE FROM` on a freshly-attached `Iceberg`, `DeltaLake`, or `Hudi` table whose metadata file is corrupted or unloadable. A regular user-facing exception is reported instead, and the server keeps running. Closes [#104891](https://github.com/ClickHouse/ClickHouse/issues/104891). [#104917](https://github.com/ClickHouse/ClickHouse/pull/104917) ([Groene AI](https://github.com/groeneai)). * Fixed the `max_memory_usage_soft_limit` was not updated on-the-fly when the configuration changed. The soft memory limit was calculated once at startup and cached, so subsequent config reloads had no effect. After this fix, the limit is recalculated and applied immediately on every config reload, no Keeper restart required. [#104940](https://github.com/ClickHouse/ClickHouse/pull/104940) ([Kai Zhu](https://github.com/nauu)). * Fix JIT symbol resolution on macOS so that sort, expression, and aggregate JIT compilation no longer fail with `CANNOT_COMPILE_CODE: Could not find symbol _`. [#104946](https://github.com/ClickHouse/ClickHouse/pull/104946) ([Alexei Fedotov](https://github.com/leshikus)). * Fix an exception (`Received signal 6 (abort)` in vector hardening, observable as `Tuple::back()` on an empty vector) when executing a query of the form `SELECT ... FROM WHERE/HAVING/GROUP BY ... (sharding_key_column IN tuple())` with `optimize_skip_unused_shards_rewrite_in` enabled under the new analyzer. [#104966](https://github.com/ClickHouse/ClickHouse/pull/104966) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `Logical error: 'Inconsistent AST formatting'` server abort (STID 1941-1bfa) on queries like `CREATE TABLE t (c0 Int CODEC(not((not(materialize(1), materialize(2)))), ZSTD)) ENGINE = Memory` in debug / sanitiser builds. The formatter no longer emits redundant outer parens around a multi-argument `tuple(...)` function call that appears inside `CODEC` / `STATISTICS` / `BACKUP_NAME` argument lists (where the operator form `(a, b)` is disabled), keeping the format-parse-format round-trip stable. [#104991](https://github.com/ClickHouse/ClickHouse/pull/104991) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` thrown by `ALTER TABLE ... MATERIALIZE INDEX` on parts that were created in 25.8 and contain a skip index over a column that was added with a separate `ALTER TABLE ... ADD COLUMN`. The mutation now correctly reads every column required by every pre-existing skip index and projection on the part during force-recalculation. Closes [#104872](https://github.com/ClickHouse/ClickHouse/issues/104872). [#105039](https://github.com/ClickHouse/ClickHouse/pull/105039) ([Groene AI](https://github.com/groeneai)). * Fixed two `LOGICAL_ERROR: Reading from materialized CTE 'X' before it has been materialized` shapes thrown by queries with `enable_materialized_cte`: (1) a reused materialized CTE filtered by `IN (subquery)` over another materialized CTE, and (2) a materialized CTE referenced both directly and inside a `WHERE ... IN (...)` filter that hits a MergeTree primary key through a nested IN-subquery. `EXPLAIN` on the same queries was affected too because the bugs fired during plan optimization. Closes [#101940](https://github.com/ClickHouse/ClickHouse/issues/101940), closes [#102320](https://github.com/ClickHouse/ClickHouse/issues/102320). [#105041](https://github.com/ClickHouse/ClickHouse/pull/105041) ([Dmitry Novik](https://github.com/novikd)). * Fix `countmin` statistics `PREWHERE` selectivity estimation for `Float32` columns. Two bugs caused `countmin` statistics to silently produce wrong estimates for `Float32` columns: `ConditionSelectivityEstimator` skipped statistics entirely when the column type (`Float32`) was narrower than the literal type (`Float64`), falling back to default selectivity and ignoring the sketch; and `StatisticsCountMinSketch::estimateEqual` hashed the first N bytes of a `Field` object, but `Field` stores `Float32` as `Float64` (`NearestFieldType` = `Float64`), so the byte pattern used at query time differed from the actual `Float32` bytes stored during build. [#105047](https://github.com/ClickHouse/ClickHouse/pull/105047) ([Han Fei](https://github.com/hanfei1991)). * `ATTACH TABLE name ;` queries that supply storage clauses (`ORDER BY`, `PARTITION BY`, `PRIMARY KEY`, `SAMPLE BY`, `TTL`, `UNIQUE KEY`, or engine `SETTINGS`) without an `ENGINE` now throw `BAD_ARGUMENTS` instead of silently re-attaching the table with its stored definition and discarding the user-supplied clauses. Query-level session `SETTINGS` (such as `log_comment`) are still applied. Use `ATTACH TABLE t;` to re-attach with stored metadata, or `ALTER TABLE t MODIFY SETTING ...` after `ATTACH` to change settings. [#105068](https://github.com/ClickHouse/ClickHouse/pull/105068) ([Groene AI](https://github.com/groeneai)). * Fixes a use-after-free in `runningAccumulate` when called on a column whose aggregate function returns its own state (e.g. `uniqStateOrDefaultState`, `sumStateOrDefaultState`, `uniqStateForEachState`). [#105085](https://github.com/ClickHouse/ClickHouse/pull/105085) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed a bug in the Web UI (`play.html`): running a query while the cursor was past the only trailing `;` reported "empty query" instead of running the query. [#105107](https://github.com/ClickHouse/ClickHouse/pull/105107) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR: 'No user in current context, it's a bug'` when a `CREATE TABLE ... AS SELECT` inside a `Replicated` database reads from `system.current_roles` or `system.enabled_roles`. The query now succeeds and returns an empty result set in this internal DDL worker context. [#105150](https://github.com/ClickHouse/ClickHouse/pull/105150) ([Groene AI](https://github.com/groeneai)). * Fixed `ILLEGAL_TYPE_OF_ARGUMENT` when merging `quantileExactWeightedInterpolated`, `quantileDD`, or `quantilePrometheusHistogram` aggregate states with their plural `quantilesXxxMerge` counterparts (and vice versa). The singular and plural variants of these three quantile families share the same internal aggregate state but were not listed in the internal name-mapping table, so cross-function state merge — and the function-fusion optimization for these families — were rejected. [#105189](https://github.com/ClickHouse/ClickHouse/pull/105189) ([Groene AI](https://github.com/groeneai)). * Fix incorrect results of `toStartOfWeek`, `toLastDayOfWeek`, `toMonday`, `toStartOfMonth`, `toLastDayOfMonth`, `toStartOfQuarter` and `toStartOfYear` for `Date32` and `DateTime64` arguments whose result falls outside the `Date` range: instead of overflowing into arbitrary dates, results before `1970-01-01` are now clamped to `1970-01-01` and results after `2149-06-06` are clamped to `2149-06-06`. This also fixes wrong query results (incorrectly pruned parts and granules) when such functions were used in `WHERE` over a `Date32` or `DateTime64` key containing out-of-range values. [#105244](https://github.com/ClickHouse/ClickHouse/pull/105244) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix a logical error in `arrayRemove` when the first argument is an array of `Variant` whose alternatives are all incompatible with the type of the second argument and `variant_throw_on_type_mismatch` is disabled. The function now treats the comparison as "never equal" and returns the array unchanged instead of triggering a server-side `assertTypeEquality` failure. [#105248](https://github.com/ClickHouse/ClickHouse/pull/105248) ([Groene AI](https://github.com/groeneai)). * Query can be executed successfully even if `FileCache` fails to create some directory on disk for caching. [#105250](https://github.com/ClickHouse/ClickHouse/pull/105250) ([Den Kalantaevskii](https://github.com/Diskein)). * Fix server abort with `Assertion 'px != 0' failed` in `ExtremesTransform` when a query with `extremes = 1` hits an exception (for example `MEMORY_LIMIT_EXCEEDED`) while the transform is building its extremes columns from the first chunk. The client now receives the original exception instead of crashing the server. [#105264](https://github.com/ClickHouse/ClickHouse/pull/105264) ([Groene AI](https://github.com/groeneai)). * `LEFT ANTI JOIN` now correctly returns the right-side join key column with default values for unmatched rows, instead of duplicating the left key value. Previously, `SELECT a.x AS l, b.x AS r FROM a LEFT ANTI JOIN b ON a.x = b.x` could return `r = a.x` for the unmatched rows when it should return the default (`0` for `Int32`, `''` for `String`, `NULL` for `Nullable`). Only the join key column was affected; non-key right columns were already defaulted correctly. The same bug appeared in `RIGHT ANTI JOIN` whenever the optimizer swapped sides. Closes [#99959](https://github.com/ClickHouse/ClickHouse/issues/99959). [#105278](https://github.com/ClickHouse/ClickHouse/pull/105278) ([Groene AI](https://github.com/groeneai)). * Fix data loss in `clickhouse-local` when started repeatedly with a non-default `--default_database` (for example `clickhouse-local --path X -- --default_database=mydb`): the lookup for the persisted database UUID was hardcoded to `metadata/default`, so each restart minted a fresh UUID and the previous run's tables became invisible. Closes [#101831](https://github.com/ClickHouse/ClickHouse/issues/101831). [#105284](https://github.com/ClickHouse/ClickHouse/pull/105284) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `Mutation of `Memory` table produced incomplete output` raised when running `ALTER TABLE APPLY PATCHES` or `ALTER TABLE APPLY DELETED MASK`. `Memory` tables do not own patch parts or deletion masks, so both commands are now correctly treated as no-ops. [#105286](https://github.com/ClickHouse/ClickHouse/pull/105286) ([Groene AI](https://github.com/groeneai)). * Fix long-running queries with Unity Catalog by passing a `table_id` parameter in the query request. [#105303](https://github.com/ClickHouse/ClickHouse/pull/105303) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix `CANNOT_CONVERT_TYPE` for `Merge` over `Merge` over `Distributed` with `distributed_group_by_no_merge=1`. [#105330](https://github.com/ClickHouse/ClickHouse/pull/105330) ([Azat Khuzhin](https://github.com/azat)). * Fix `clickhouse local` and `clickhouse client` silently ignoring `--query`/`-q` when a positional file argument was also given. Both forms now produce `BAD_ARGUMENTS` consistently. [#105334](https://github.com/ClickHouse/ClickHouse/pull/105334) ([Raúl Marín](https://github.com/Algunenano)). * Fix `REPLACE TEMPORARY TABLE` silently creating a new table when the target didn't exist. It now throws `UNKNOWN_TABLE`, matching `REPLACE TABLE` semantics. Use `CREATE OR REPLACE TEMPORARY TABLE` to keep the create-or-replace behavior. [#105373](https://github.com/ClickHouse/ClickHouse/pull/105373) ([Groene AI](https://github.com/groeneai)). * Fixed long-running reads (e.g. `INSERT ... SELECT`) from a `DataLakeCatalog` database with `catalog_type = 'glue'` failing with an `ExpiredToken` error once the STS session credentials captured at table-metadata load time crossed their expiry. The AWS STS credentials are now refreshed mid-query, so such reads continue past the token's expiry. [#105381](https://github.com/ClickHouse/ClickHouse/pull/105381) ([Pratima Patel](https://github.com/pratimapatel2008)). * Fix logical error in negative `LIMIT BY` in some cases when used with `ARRAY JOIN`. [#105403](https://github.com/ClickHouse/ClickHouse/pull/105403) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix inflated `read_bytes` (and the derived bytes/s shown in `system.query_log`, progress bar, etc.) when reading Parquet files. The previous implementation reported the row group's total compressed size on every chunk, so reading K of N columns overcounted by `N / K`. It is now summed only across the selected columns. Also fixes file-level progress tracking for Iceberg tables, which previously never reported the data file size. [#105413](https://github.com/ClickHouse/ClickHouse/pull/105413) ([Groene AI](https://github.com/groeneai)). * Fixed a possible server segfault in cluster table functions (`s3Cluster`, `urlCluster`, `fileCluster`, ...) when the planner produces a `SELECT` with the `recursive_with` flag set but no `WITH` expression. Closes [#105370](https://github.com/ClickHouse/ClickHouse/issues/105370). [#105433](https://github.com/ClickHouse/ClickHouse/pull/105433) ([Groene AI](https://github.com/groeneai)). * Fix wrong result of `countDistinct` / `uniqExact` on `Nullable` columns when `count_distinct_optimization = 1` (the NULL group was incorrectly counted). [#105439](https://github.com/ClickHouse/ClickHouse/pull/105439) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a heap-buffer-overflow when reading Arrow or ArrowStream files with corrupted intermediate offsets in a binary or string column, and a null-pointer dereference when reading geo-tagged Arrow columns. [#105449](https://github.com/ClickHouse/ClickHouse/pull/105449) ([Raúl Marín](https://github.com/Algunenano)). * Validate `Dynamic`/`Variant` types in window `PARTITION BY`, it was not checked before when `allow_suspicious_types_in_group_by` is disabled. Closes [#105028](https://github.com/ClickHouse/ClickHouse/issues/105028). [#105450](https://github.com/ClickHouse/ClickHouse/pull/105450) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `input_format_json_empty_as_default` not working in JSONStrings* formats. Closes [#104913](https://github.com/ClickHouse/ClickHouse/issues/104913). [#105453](https://github.com/ClickHouse/ClickHouse/pull/105453) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `NULL` insert into `Variant` column via `VALUES` format during expression parsing. Closes [#104909](https://github.com/ClickHouse/ClickHouse/issues/104909). [#105455](https://github.com/ClickHouse/ClickHouse/pull/105455) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `dictGetOrDefault` throwing `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN` when the default argument is a `Nullable` expression and `short_circuit_function_evaluation` is enabled with a mix of found and not-found dictionary keys. [#105461](https://github.com/ClickHouse/ClickHouse/pull/105461) ([Raúl Marín](https://github.com/Algunenano)). * Fix silent value truncation in `Dynamic` columns when the `Quoted` (Values) text reader falls back to `String` for incomplete inferred types like `Map(Array(Nothing), ...)`. Previously the fallback wrapped the raw field with naive `'...'` concatenation, so an inner single quote (always present for such fallbacks) prematurely terminated the quoted string and the rest of the value was lost. [#105463](https://github.com/ClickHouse/ClickHouse/pull/105463) ([Groene AI](https://github.com/groeneai)). * Fix SIGSEGV when reading `Dynamic` subcolumns from compressed `Memory` table after `ALTER`. Closes [#104901](https://github.com/ClickHouse/ClickHouse/issues/104901). [#105464](https://github.com/ClickHouse/ClickHouse/pull/105464) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `toFloat64`/`toUInt32`/`toString`/etc. on `Dynamic` ignoring `cast_keep_nullable`. Closes [#104789](https://github.com/ClickHouse/ClickHouse/issues/104789). [#105467](https://github.com/ClickHouse/ClickHouse/pull/105467) ([Pavel Kruglov](https://github.com/Avogar)). * Include `skip_first_lines` in schema cache key for `WithNames` formats. Closes [#104527](https://github.com/ClickHouse/ClickHouse/issues/104527). [#105469](https://github.com/ClickHouse/ClickHouse/pull/105469) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a server segfault in `uniqStateOrNull` / `uniqStateOrDefault` / `uniqOrNullState` (and similar combinator chains over `uniq`) when used with `GROUP BY ... WITH ROLLUP`, `WITH CUBE`, or `WITH TOTALS` and a Nullable argument. [#105470](https://github.com/ClickHouse/ClickHouse/pull/105470) ([Groene AI](https://github.com/groeneai)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` exception when combining `ORDER BY ... WITH FILL INTERPOLATE` and `LIMIT N BY` with the analyzer enabled. Closes [#103474](https://github.com/ClickHouse/ClickHouse/issues/103474). [#105481](https://github.com/ClickHouse/ClickHouse/pull/105481) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix `toStartOfMillisecond` and `toStartOfMicrosecond` returning a result off by nearly a second for negative (pre-epoch) `DateTime64` values, and fix `UndefinedBehaviorSanitizer` signed-integer-overflow in `toStartOfSecond`, `toStartOfMillisecond`, and `toStartOfMicrosecond` for `DateTime64` inputs near `INT64_MIN`. [#105482](https://github.com/ClickHouse/ClickHouse/pull/105482) ([Groene AI](https://github.com/groeneai)). * Add a new compatibility setting `analyzer_compatibility_prefer_alias_over_subcolumn` (disabled by default). When enabled, the new analyzer prefers the alias-prefix interpretation over `Tuple`-subcolumn / dotted-column matches for multi-part identifiers, restoring the previous interpreter's behavior. This avoids `AMBIGUOUS_IDENTIFIER` (and related) errors when a query joins a table whose name matches an inner table of a CTE/subquery that uses `SELECT *` over a join, where asterisk-renamed columns (e.g. `b.id`) would otherwise leak the inner table identifiers into the outer scope. [#105491](https://github.com/ClickHouse/ClickHouse/pull/105491) ([Vladimir Cherkasov](https://github.com/vdimir)). * A vector search query executed with the _index only_ optimization now returns the correct value for `L2Distance()` if it is in the `SELECT` list. The function was incorrectly returning the _L2-squared_ distance value. Note that the vector search query was returning the correctly ranked results, only applications **explicitly retrieving and interpreting** the `L2Distance()` value would have seen the squared value. [#105495](https://github.com/ClickHouse/ClickHouse/pull/105495) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fixed a rare issue in new Kafka table engine where messages could be incorrectly handled during topic partition reassignment, potentially leading to skipped messages after offset rollback. [#105500](https://github.com/ClickHouse/ClickHouse/pull/105500) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix an exception (`Trying to execute PLACEHOLDER action` logical error) when a `MATERIALIZED` CTE whose body is a correlated subquery is used as the right-hand side of `IN`. Such a CTE is now rejected at analysis time, consistent with how the same pattern is already rejected when the CTE is referenced directly in `FROM`. [#105518](https://github.com/ClickHouse/ClickHouse/pull/105518) ([Groene AI](https://github.com/groeneai)). * Fix stdout/stdin compression not applied when format is explicitly specified. Closes [#104441](https://github.com/ClickHouse/ClickHouse/issues/104441). [#105528](https://github.com/ClickHouse/ClickHouse/pull/105528) ([Pavel Kruglov](https://github.com/Avogar)). * Fix SEGFAULT in `singleValueOrNull` deserialization for `JSON` type. Closes [#103630](https://github.com/ClickHouse/ClickHouse/issues/103630). [#105535](https://github.com/ClickHouse/ClickHouse/pull/105535) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `clickhouse-format --backslash` silently dropping `INSERT` `VALUES` data. Closes [#103533](https://github.com/ClickHouse/ClickHouse/issues/103533). [#105536](https://github.com/ClickHouse/ClickHouse/pull/105536) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `variant_throw_on_type_mismatch/dynamic_throw_on_type_mismatch=false` not catching exceptions during function execution. Closes [#103484](https://github.com/ClickHouse/ClickHouse/issues/103484). [#105543](https://github.com/ClickHouse/ClickHouse/pull/105543) ([Pavel Kruglov](https://github.com/Avogar)). * Fix ignoring `input_format_try_infer_datetimes` during insertion into shared data in `JSON`. Closes [#103221](https://github.com/ClickHouse/ClickHouse/issues/103221). [#105544](https://github.com/ClickHouse/ClickHouse/pull/105544) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `histogram` producing wrong results for small unsorted inputs. Closes [#103109](https://github.com/ClickHouse/ClickHouse/issues/103109). [#105548](https://github.com/ClickHouse/ClickHouse/pull/105548) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `DateTime` wrapping around for out-of-range values in `JSONExtract` and text deserializations. Closes [#103094](https://github.com/ClickHouse/ClickHouse/issues/103094). [#105551](https://github.com/ClickHouse/ClickHouse/pull/105551) ([Pavel Kruglov](https://github.com/Avogar)). * Fix usage of insertion table in table functions when `optimize_trivial_insert_select` is enabled. Closes [#103083](https://github.com/ClickHouse/ClickHouse/issues/103083). [#105555](https://github.com/ClickHouse/ClickHouse/pull/105555) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `CASE` expression returning the `ELSE` branch instead of the matching `THEN` when both the expression and a `WHEN` value were `NULL`. [#105556](https://github.com/ClickHouse/ClickHouse/pull/105556) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a crash in `replxx` caused by hitting the `FD_SETSIZE` limitation of `select` (now replaced with `poll`). Previously, with SSH or the web terminal enabled, this could crash the server. [#105559](https://github.com/ClickHouse/ClickHouse/pull/105559) ([Azat Khuzhin](https://github.com/azat)). * Materialize subcolumns in `executePartitionByExpression` before executing expression. Closes [#103057](https://github.com/ClickHouse/ClickHouse/issues/103057). [#105573](https://github.com/ClickHouse/ClickHouse/pull/105573) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` exception when `TTL` expression references a subcolumn. [#105578](https://github.com/ClickHouse/ClickHouse/pull/105578) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a server crash (SIGSEGV) reachable by any user with `CREATE TABLE` rights when sending `CREATE TABLE ... TO INNER UUID '...'` without an `ENGINE` clause over HTTP or the native protocol. The same bug also crashed the client. The parser now reports a proper `BAD_ARGUMENTS` error instead of dereferencing a null pointer. [#105579](https://github.com/ClickHouse/ClickHouse/pull/105579) ([Groene AI](https://github.com/groeneai)). * Fix `estimateCompressionRatio` window function losing accumulated data between rows. Closes [#101738](https://github.com/ClickHouse/ClickHouse/issues/101738). [#105581](https://github.com/ClickHouse/ClickHouse/pull/105581) ([Pavel Kruglov](https://github.com/Avogar)). * Fix crash when inserting tuples of different sizes in the same VALUES clause into a `String` column. Closes [#101727](https://github.com/ClickHouse/ClickHouse/issues/101727). [#105582](https://github.com/ClickHouse/ClickHouse/pull/105582) ([Pavel Kruglov](https://github.com/Avogar)). * Hive partition value extraction now honors the `cast_string_to_date_time_mode` setting and accepts ISO 8601 timestamps with timezone suffixes (e.g. `+0000`, `+00:00`, `Z`) in partition keys by default. [#105584](https://github.com/ClickHouse/ClickHouse/pull/105584) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes `NOT_IMPLEMENTED` error on `toString` from `DateTime` with Timezone containing `NULL` value. Closes [#103712](https://github.com/ClickHouse/ClickHouse/issues/103712). [#105587](https://github.com/ClickHouse/ClickHouse/pull/105587) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix OOB read in flattened `Dynamic` column `Native` deserialization. [#105666](https://github.com/ClickHouse/ClickHouse/pull/105666) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a `LOGICAL_ERROR` (`Unexpected return type from if`) raised during query planning for `if` expressions whose result type is `Variant` and whose second-or-third branch is a constant-condition `if` over a `UInt64` literal that fits into `Int64`. Closes [#105649](https://github.com/ClickHouse/ClickHouse/issues/105649). [#105680](https://github.com/ClickHouse/ClickHouse/pull/105680) ([Groene AI](https://github.com/groeneai)). * Fix wrong results when reading an Iceberg table with `iceberg_use_version_hint = 1` after another writer (such as the `icebergLocal`/`icebergS3` table function) without the setting advances the table. `version-hint.text` is now kept in sync by every writer once the file exists, so subsequent readers using the hint see the latest snapshot. [#105682](https://github.com/ClickHouse/ClickHouse/pull/105682) ([Groene AI](https://github.com/groeneai)). * Fixed a silent under-count in `SELECT` queries when `use_query_condition_cache = 1` (default). A query of the shape `PREWHERE pk_prefix = X WHERE non_pk IN (...)` against a column with a bloom-filter skip index poisoned the `QueryConditionCache` for the `pk_prefix = X` predicate, so a subsequent benign `SELECT count() ... WHERE pk_prefix = X` returned an incorrect, under-counted result. Affected all 26.x releases. [#104781](https://github.com/ClickHouse/ClickHouse/issues/104781). [#105686](https://github.com/ClickHouse/ClickHouse/pull/105686) ([Groene AI](https://github.com/groeneai)). * Fixed `singleValueOrNullMerge` returning a concrete value instead of `NULL` when merging a state that had already observed multiple distinct values. [#105734](https://github.com/ClickHouse/ClickHouse/pull/105734) ([Minh Vu](https://github.com/fallintoplace)). * Fix `Template` input format error recovery after malformed rows. [#105735](https://github.com/ClickHouse/ClickHouse/pull/105735) ([Yue Ni](https://github.com/niyue)). * Fix a crash in the `mongodb` table function, MongoDB storage and MongoDB dictionary source when the collection name is empty or contains NUL bytes. [#105776](https://github.com/ClickHouse/ClickHouse/pull/105776) ([Raúl Marín](https://github.com/Algunenano)). * Fix `Keeper` snapshot cleanup after failed writes so partial snapshots are cleaned up safely and failed writes can be retried without advancing `latest_snapshot_meta`. [#105779](https://github.com/ClickHouse/ClickHouse/pull/105779) ([Antonio Andelic](https://github.com/antonio2368)). * Fix `ALTER TABLE ... CLEAR COLUMN` being rejected for explicit `SummingMergeTree` and `CoalescingMergeTree` `columns_to_sum` columns. [#105785](https://github.com/ClickHouse/ClickHouse/pull/105785) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed an assertion in `DatabaseCatalog::tryGetDatabase` (and resulting `UNKNOWN_DATABASE` error in release builds) when creating a parameterized view whose database name is supplied via a query parameter, for example `CREATE VIEW {db:Identifier}.v AS SELECT {p:UInt64}`. Parameters in the DDL parts of the `CREATE` statement are now substituted at create time, while parameters in the SELECT body remain available for substitution at view-call time. [#105799](https://github.com/ClickHouse/ClickHouse/pull/105799) ([Groene AI](https://github.com/groeneai)). * Fix `Bad get: has Decimal32, requested Decimal128` from `sumMap` and `sumMapWithOverflow` over a `Nested(... Nullable(Decimal(P, S)))` column when the aggregate state is serialised (e.g. parallel replicas, `sumMapState` via a binary-state formatter, external aggregation). [#105816](https://github.com/ClickHouse/ClickHouse/pull/105816) ([Groene AI](https://github.com/groeneai)). * Fix `Expected ColumnLowCardinality, got String` / `Bad cast from type DB::ColumnString to DB::ColumnLowCardinality` errors when `apply_mutations_on_fly = 1` is used on a table with pending on-fly `UPDATE`/`DELETE` mutations queued before an `ALTER MODIFY COLUMN ... LowCardinality(...)` mutation. [#105847](https://github.com/ClickHouse/ClickHouse/pull/105847) ([Raúl Marín](https://github.com/Algunenano)). * Iceberg writes now preserve NULL values in `Nullable(T)` partition columns. Previously, a NULL written by ClickHouse showed up as the default value of the inner type (`0` for `int`) when read back by Spark or other Iceberg readers. Closes [#105852](https://github.com/ClickHouse/ClickHouse/issues/105852). [#105862](https://github.com/ClickHouse/ClickHouse/pull/105862) ([Groene AI](https://github.com/groeneai)). * Fixed `Parquet` and `ORC` filter pushdown for `IN (subquery)` predicates, allowing row-group/page/bloom-filter pruning to work for `file`, `url`, `s3`, and object-storage reads. [#105863](https://github.com/ClickHouse/ClickHouse/pull/105863) ([Arsen Muk](https://github.com/arsenmuk)). * Fix read-in-order for `Merge` tables (with a new analyzer). [#105867](https://github.com/ClickHouse/ClickHouse/pull/105867) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed Iceberg v2 merge-on-read position deletes returning wrong rows when a single delete file references multiple data files and several such delete files apply to the same data file. The streaming reader (`use_roaring_bitmap_iceberg_positional_deletes = 0`) now filters delete-file rows by `file_path` in C++ instead of relying on Parquet row-group pruning, restoring the ascending-positions invariant. [#105888](https://github.com/ClickHouse/ClickHouse/pull/105888) ([Groene AI](https://github.com/groeneai)). * Fix `Cannot find column` error for distributed queries with `IN Array(...)` filter for the new analyzer. [#105894](https://github.com/ClickHouse/ClickHouse/pull/105894) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix a SIGSEGV in `ALTER TABLE ... MODIFY COLUMN ... Nullable(...)` on `MergeTree` tables when a column with `STATISTICS` is concurrently dropped by another `ALTER`. Closes [#105912](https://github.com/ClickHouse/ClickHouse/issues/105912). [#105917](https://github.com/ClickHouse/ClickHouse/pull/105917) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash that could occur when a query reading from PostgreSQL — via the postgresql table function, the PostgreSQL table engine, or a dictionary with a PostgreSQL source — was cancelled (for example with KILL QUERY) and cancelling the remote PostgreSQL query failed. [#105949](https://github.com/ClickHouse/ClickHouse/pull/105949) ([Rory Shanks](https://github.com/rorylshanks)). * Fix `SYSTEM INSTRUMENT ADD` formatting so handler arguments are separated by a single space, and reject invalid `SLEEP` instrumentation argument lists with more than two values or a range where the minimum is greater than the maximum. [#105984](https://github.com/ClickHouse/ClickHouse/pull/105984) ([Pablo Marcos](https://github.com/pamarcos)). * Fix possibly wrong results for queries that combine an outer join with a subsequent inner join referencing the outer join's null-supplying side. Join reordering could pick a plan that pulled inner-join conditions into the outer join's ON clause. [#105992](https://github.com/ClickHouse/ClickHouse/pull/105992) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a possible crash due to a too-large string literal sent within the query. [#105996](https://github.com/ClickHouse/ClickHouse/pull/105996) ([Nikita Taranov](https://github.com/nickitat)). * Fix `INVALID_WITH_FILL_EXPRESSION` exception when using `INTERPOLATE ()` (empty) with a sorting prefix in `ORDER BY` and `use_with_fill_by_sorting_prefix` enabled. Sorting prefix columns are now correctly excluded from the implicit interpolation set, matching the behavior of explicitly named `INTERPOLATE (col)`. [#106001](https://github.com/ClickHouse/ClickHouse/pull/106001) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix `ALTER TABLE` partition operations silently failing for `Bool` partition keys. Closes [#101722](https://github.com/ClickHouse/ClickHouse/issues/101722). [#106004](https://github.com/ClickHouse/ClickHouse/pull/106004) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `JSONExtractRaw` and `JSONHas` for typed `JSON` paths with default values. Closes [#101721](https://github.com/ClickHouse/ClickHouse/issues/101721). [#106005](https://github.com/ClickHouse/ClickHouse/pull/106005) ([Pavel Kruglov](https://github.com/Avogar)). * Fix incorrect '/'-prefix addition on empty basepaths in data lake configurations. Closes [#105989](https://github.com/ClickHouse/ClickHouse/issues/105989). [#106013](https://github.com/ClickHouse/ClickHouse/pull/106013) ([thewisenerd](https://github.com/thewisenerd)). * Fix `IcebergLocal` table engine becoming read-only after a `DETACH` + `ATTACH` cycle or a server restart, which made every subsequent `INSERT` fail with `Local object storage Local is readonly. (READONLY)`. [#106016](https://github.com/ClickHouse/ClickHouse/pull/106016) ([Groene AI](https://github.com/groeneai)). * Fix `Keeper` failures during follower catch-up when the new request dispatcher response queue could fill before the response thread started. [#106049](https://github.com/ClickHouse/ClickHouse/pull/106049) ([Antonio Andelic](https://github.com/antonio2368)). * Better compatibility with the old analyzer. If the table has columns like `x.a Array, x.b Array, x String`, prefer arrays for `ARRAY JOIN x`. [#106069](https://github.com/ClickHouse/ClickHouse/pull/106069) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed the filesystem cache being silently disabled for Azure Blob Storage (e.g. Delta Lake tables over Azure) because object metadata did not include the blob ETag. [#106091](https://github.com/ClickHouse/ClickHouse/pull/106091) ([thewisenerd](https://github.com/thewisenerd)). * Fix `system.dictionaries` returning 0 rows with partial `SHOW DICTIONARIES` revoke. [#106105](https://github.com/ClickHouse/ClickHouse/pull/106105) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed a server crash when querying DeltaLake tables with `allow_experimental_delta_kernel_rs` enabled and a credential or option that contained invalid bytes (the Rust FFI panicked across the `extern "C"` boundary). [#106109](https://github.com/ClickHouse/ClickHouse/pull/106109) ([Raúl Marín](https://github.com/Algunenano)). * Fix incorrect results for queries against tables whose `ORDER BY` contains a monotonically decreasing function such as `(c0 / -42)` or `intDiv(c0, -42)`. Predicates on the underlying column (for example, `c0 < 0`) could wrongly prune granules that contained matching rows, producing missing results. Closes [#106084](https://github.com/ClickHouse/ClickHouse/issues/106084). Closes [#106124](https://github.com/ClickHouse/ClickHouse/issues/106124). Closes [#106080](https://github.com/ClickHouse/ClickHouse/issues/106080). [#106136](https://github.com/ClickHouse/ClickHouse/pull/106136) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix support for using `WASM` SQL UDFs in `MATERIALIZED VIEW` definitions. [#106161](https://github.com/ClickHouse/ClickHouse/pull/106161) ([Yue Ni](https://github.com/niyue)). * Iceberg partition pruning now correctly handles `WHERE partition_col = (SELECT ... FROM ...)` filters where the analyzer wraps the scalar subquery result in an internal `_CAST(Const, 'TargetType')` with matching source and target types. Previously such filters disabled partition pruning and triggered a full table scan. [#106204](https://github.com/ClickHouse/ClickHouse/pull/106204) ([Groene AI](https://github.com/groeneai)). * Fix clickhouse local's `--query_id` parameter to allow it to specify custom query id. [#106205](https://github.com/ClickHouse/ClickHouse/pull/106205) ([Yue Ni](https://github.com/niyue)). * Fixed the S3 storage class (`s3_storage_class` / `s3_storage_class_name`) being ignored for objects written via multipart upload on `S3` disks and object storage, which caused large objects to be created with the default `STANDARD` class. The option name is now accepted both as `s3_storage_class` and `s3_storage_class_name` for disks, object storage and backups. [#106214](https://github.com/ClickHouse/ClickHouse/pull/106214) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed Keeper sometimes getting stuck on startup when setting `nuraft_max_log_gap_in_stream` is set to non-default value (default is 0, i.e. disable pipelining of `append_entries` requests). [#106220](https://github.com/ClickHouse/ClickHouse/pull/106220) ([Michael Kolupaev](https://github.com/al13n321)). * Validate corrupted `DDSketch` aggregate-function state (bin keys) on insertion instead of accepting malformed data. [#106236](https://github.com/ClickHouse/ClickHouse/pull/106236) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed coordinator mode mismatch when subqueries override in-order settings. [#106243](https://github.com/ClickHouse/ClickHouse/pull/106243) ([Nikita Taranov](https://github.com/nickitat)). * Fix `optimize_skip_unused_shards` did not apply (and `force_optimize_skip_unused_shards` falsely failing) when a `Distributed` table is queried through a `Merge` table or the `merge` table function, with the predicate applied above it. [#106250](https://github.com/ClickHouse/ClickHouse/pull/106250) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix `INTERPOLATE ()` throwing `INVALID_WITH_FILL_EXPRESSION` when `ORDER BY` columns are aliased in the `SELECT` list with the old analyzer. Closes [#106248](https://github.com/ClickHouse/ClickHouse/issues/106248). [#106252](https://github.com/ClickHouse/ClickHouse/pull/106252) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fixed an abort when deserialising a malformed `AggregateFunction(uniqTheta, ...)` state from `RowBinary` input or a query parameter. The bad input is now rejected with `CORRUPTED_DATA` instead of escaping as a `std::exception` and aborting the process via `abortOnFailedAssertion`. [#106260](https://github.com/ClickHouse/ClickHouse/pull/106260) ([Groene AI](https://github.com/groeneai)). * Reject out-of-range `IntervalKind` bytes during `RowBinaryWithNamesAndTypes` type decoding (`input_format_binary_decode_types_in_binary_format = 1`) with a clear `INCORRECT_DATA` error instead of constructing a `DataTypeInterval` with an invalid kind that could subsequently trip undefined behavior in hash paths. [#106261](https://github.com/ClickHouse/ClickHouse/pull/106261) ([Groene AI](https://github.com/groeneai)). * Fix `SLRU` downgrade failure rollback issue in `26.1+`. Fix `System`/`Data` cache split priority with `keep_free_space_ratio` feature. [#106286](https://github.com/ClickHouse/ClickHouse/pull/106286) ([Kseniia Sumarokova](https://github.com/kssenii)). * Read the `null` subcolumn as a `JSON` path in `Nullable(JSON)` instead of the null-map. Closes [#106085](https://github.com/ClickHouse/ClickHouse/issues/106085). [#106295](https://github.com/ClickHouse/ClickHouse/pull/106295) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `RestCatalog::empty` returning the wrong result when an Iceberg REST catalog contains tables. [#106301](https://github.com/ClickHouse/ClickHouse/pull/106301) ([Lefteris](https://github.com/LefterisXefteris)). * Fixed an exception for `INNER JOIN` queries with an empty left `MergeTree` table when `enable_parallel_replicas`, `query_plan_use_new_logical_join_step`, and `query_plan_optimize_join_order_algorithm = 'greedy'` are enabled. [#106338](https://github.com/ClickHouse/ClickHouse/pull/106338) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed an incorrect conversion of subnormal `Float16` values to `Float32` (e.g. when reading them from Numpy `.npy` files), caused by an off-by-one mantissa shift. [#106343](https://github.com/ClickHouse/ClickHouse/pull/106343) ([Joanna Hulboj](https://github.com/jh0x)). * Fix a crash and a possible `NOT_FOUND_COLUMN_IN_BLOCK` error when constraint-based optimization (`optimize_using_constraints`) is used with correlated subqueries. [#106349](https://github.com/ClickHouse/ClickHouse/pull/106349) ([Raúl Marín](https://github.com/Algunenano)). * Fixed an out-of-bounds read in `sipHash64Keyed`, `sipHash128Keyed` and `sipHash128ReferenceKeyed` when hashing a column whose arrays are all empty and the key is not constant. [#106355](https://github.com/ClickHouse/ClickHouse/pull/106355) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `SYSTEM RELOAD CONFIG` discarding per-endpoint Azure Blob Storage settings (such as `use_native_copy`), which caused a disk's configured settings to be ignored for `BACKUP`/`RESTORE` until the server was restarted. [#106357](https://github.com/ClickHouse/ClickHouse/pull/106357) ([Julia Kartseva](https://github.com/jkartseva)). * Fix `regexpExtract(haystack, pattern)` so that patterns without a capturing group return the whole match instead of throwing `INDEX_OF_POSITIONAL_ARGUMENT_IS_OUT_OF_RANGE`. [#106374](https://github.com/ClickHouse/ClickHouse/pull/106374) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` exception during cache predownload when a remote S3 object is overwritten with shorter content between listing and reading. [#106375](https://github.com/ClickHouse/ClickHouse/pull/106375) ([Nikita Fomichev](https://github.com/fm4v)). * Fix memory usage growth in filesystem cache. [#106387](https://github.com/ClickHouse/ClickHouse/pull/106387) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix multiple heap out-of-bounds reads in the Arrow IPC format reader (`ArrowColumnToCHColumn`). A malformed Arrow file could declare more rows than its buffers contain, declare list/struct/map child lengths inconsistent with their parent, supply non-monotonic list offsets, or truncate a child validity bitmap, causing reads past the end of heap allocations. This is reachable by any user with `SELECT` privilege via `file()`, `format()`, table functions, or ArrowFlight inputs. All data, offsets, view-struct, and validity-bitmap buffers are now validated before any raw pointer access, and list/struct/map shapes and offsets are checked for consistency. [#106395](https://github.com/ClickHouse/ClickHouse/pull/106395) ([Raúl Marín](https://github.com/Algunenano)). * Fixed backups failing with `FILE_DOESNT_EXIST` when a refreshable materialized view's REPLACE target is collected on a Replicated or Shared database whose materialized view isn't yet instantiated on the backup-initiating replica. [#106411](https://github.com/ClickHouse/ClickHouse/pull/106411) ([Julia Kartseva](https://github.com/jkartseva)). * Fix a logical error `Column identifier ... is already registered` when a mutation (`DELETE`/`UPDATE`) predicate contains an `IN`/`EXISTS` subquery that reads from a default table expression nested in another subquery. [#106414](https://github.com/ClickHouse/ClickHouse/pull/106414) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error (`Left and right columns have same names`) in the join order optimizer that could occur for joins executed with parallel replicas when two relations in the join graph share column names. [#106418](https://github.com/ClickHouse/ClickHouse/pull/106418) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a logical error when building a polygon dictionary from source data containing `NaN` or infinite point coordinates. Such coordinates are now rejected with a clear error. [#106423](https://github.com/ClickHouse/ClickHouse/pull/106423) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a bug where the `used_privileges` and `missing_privileges` columns of `system.query_log` could contain privilege strings leaked from unrelated earlier queries of a different user, database, or session. [#106425](https://github.com/ClickHouse/ClickHouse/pull/106425) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Functions `base58Encode`, `base58Decode` and `tryBase58Decode` now respect `max_execution_time` and query cancellation on large inputs, and reject inputs larger than 10 KB instead of running for a very long time. The limit is configurable via the new setting `function_base58_max_input_size` (`0` disables it). [#106428](https://github.com/ClickHouse/ClickHouse/pull/106428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix sporadic incorrect results for `ORDER BY ... DESC` queries when reading wide parts in reverse order with `read_in_order_use_virtual_row_per_block = 1` and a small `max_block_size`. [#106429](https://github.com/ClickHouse/ClickHouse/pull/106429) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a `NOT_FOUND_COLUMN_IN_BLOCK` exception when querying an Iceberg or S3 table with a compound `WHERE` containing `IS NOT NULL` on a column that is not in the `SELECT` list, using the Parquet V3 native reader. [#106443](https://github.com/ClickHouse/ClickHouse/pull/106443) ([Shaohua Wang](https://github.com/tiandiwonder)). * Allows user to specify headers for `HTTPDictionary` using named collections. [#106459](https://github.com/ClickHouse/ClickHouse/pull/106459) ([Jan Rada](https://github.com/ZelvaMan)). * Fix a case where a worker thread could stay attached to a stale thread group after `CurrentThread::attachToGroup` failed part way through, causing later tasks on the same thread to fail with `Thread is already attached to a group`. [#106462](https://github.com/ClickHouse/ClickHouse/pull/106462) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fixed an exception when a vector search query uses a vector index and uses another skip index like `minmax` and `use_skip_indexes_on_data_read = 1`. [#106473](https://github.com/ClickHouse/ClickHouse/pull/106473) ([Shankar Iyer](https://github.com/shankar-iyer)). * Malformed `Avro` enum values are now validated and rejected with an error instead of causing an out-of-bounds read and abort when deserializing corrupted `Avro` data. [#106476](https://github.com/ClickHouse/ClickHouse/pull/106476) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix inflated progress reporting when reading from Iceberg tables with `_file` or `_path` filters. Previously, `total_bytes_to_read` progress included all files from the manifest regardless of filtering. [#106491](https://github.com/ClickHouse/ClickHouse/pull/106491) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixed a `Bad cast exception` for Redis dictionaries that use `STORAGE_TYPE 'simple'` with a `cache`/`direct` layout and a single string (complex) key; such dictionaries now work, and composite keys over `simple` storage report a clear error. [#106501](https://github.com/ClickHouse/ClickHouse/pull/106501) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix wrong-results bug where `WHERE c0 = const` returned no rows for tables with `ORDER BY f(c0)` when `f(const)` evaluates to NaN, e.g. `ORDER BY sqrt(c0)` with a negative constant. The primary-key analysis incorrectly pruned every granule and poisoned the query condition cache for subsequent queries. [#106507](https://github.com/ClickHouse/ClickHouse/pull/106507) ([Groene AI](https://github.com/groeneai)). * Fix `LIMIT WITH TIES` and fractional `LIMIT WITH TIES` not respecting the collation from `ORDER BY ... COLLATE` when determining ties. Rows that are equal according to the collation (for example `'1'` and `'01'` under numeric collation) were compared byte-wise, so some tied rows were wrongly dropped from the result. [#106539](https://github.com/ClickHouse/ClickHouse/pull/106539) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `DISTINCT` in order and `LIMIT BY` in order optimizations (including negative `LIMIT BY`) returning wrong results when the input is sorted with a collation (`ORDER BY ... COLLATE`). Rows that are equal according to the collation (for example `'a'` and `'A'` under a case-insensitive collation) are ordered by collation key and are not adjacent by value, so the in order optimization is now skipped when a collator is used. [#106564](https://github.com/ClickHouse/ClickHouse/pull/106564) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed wrong results for `SELECT c, count() FROM t WHERE c GROUP BY c` against tables with the implicit `_minmax_count_projection`, an explicit aggregate projection, or a normal projection: every group used to collapse into one row with a constant key and the total row count. [#106590](https://github.com/ClickHouse/ClickHouse/pull/106590) ([Groene AI](https://github.com/groeneai)). * Fixes a bug where users could not use scalar subqueries in the first argument of `IN` where the second argument is a non-constant tuple. [#106610](https://github.com/ClickHouse/ClickHouse/pull/106610) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix exception `Mutation of `Memory` table produced incomplete output` raised when running `ALTER TABLE ` commands that have no per-row data effect on a `Memory` engine, namely `APPLY PATCHES`, `APPLY DELETED MASK`, `MATERIALIZE STATISTICS`, `MATERIALIZE INDEX`, `MATERIALIZE PROJECTION`, and `REWRITE PARTS`. `Memory` tables do not own those structures, so such commands are now treated as no-ops. [#106621](https://github.com/ClickHouse/ClickHouse/pull/106621) ([Groene AI](https://github.com/groeneai)). * Fix `session_timezone` being ignored when serializing `LowCardinality(DateTime)` columns to text formats (CSV, TSV, JSONEachRow, etc.). Previously, after the first write of a `LowCardinality(DateTime)` column on a server, every subsequent query that wrote such a column rendered the wall-clock string in whichever timezone was first seen, regardless of `session_timezone`. [#106634](https://github.com/ClickHouse/ClickHouse/pull/106634) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` "Trying to get name of not a column: `ExpressionList`" raised by queries that pass an asterisk inside `multiIf` to a table function argument, e.g. `numbers(multiIf(*, ...), 2)`. The query now rejects the unresolvable matcher with `UNSUPPORTED_METHOD`. [#106647](https://github.com/ClickHouse/ClickHouse/pull/106647) ([Groene AI](https://github.com/groeneai)). * Fix server failing to start with `Too many marks in file ...skp_idx_idx.cmrk4, marks expected 0 (bytes size 0)` when a `MergeTree` table has a skip-index part with zero granules and a non-empty marks file on disk. [#106675](https://github.com/ClickHouse/ClickHouse/pull/106675) ([Groene AI](https://github.com/groeneai)). * Reject pathological `file` glob patterns that would cause unbounded recursion in directory listing. A maximum recursion depth of 1000 is now enforced; queries that exceed it raise `TOO_DEEP_RECURSION` instead of crashing the server with a stack overflow. [#106676](https://github.com/ClickHouse/ClickHouse/pull/106676) ([Groene AI](https://github.com/groeneai)). * Fix server abort `Bad cast from type DB::ColumnNothing to DB::ColumnVector` in `FunctionIf::executeForConstAndNullableCondition` when the `if`/`multiIf` condition constant-folds to `Const(Nullable(Nothing))` (for example, an out-of-range subscript on an empty array, like `arraySort(x -> x, [])[toNullable(1)]`). [#106678](https://github.com/ClickHouse/ClickHouse/pull/106678) ([Groene AI](https://github.com/groeneai)). * Fix exception `'Trying to read from input() twice.'` raised when the table function `input` is wrapped in a non-MATERIALIZED CTE that is referenced from more than one place in the query. The query is now rejected with a clean `INVALID_USAGE_OF_INPUT` error at planning time. `input` is a one-shot client stream and can only be consumed by a single source in the query plan. [#106682](https://github.com/ClickHouse/ClickHouse/pull/106682) ([Groene AI](https://github.com/groeneai)). * Make `FORMAT` apply to the `EXPLAIN` output of `EXPLAIN ... INSERT ... SELECT ... FORMAT ...` also when `SETTINGS` precede the `FORMAT` or the output format is `Values`. Follow-up to [#101772](https://github.com/ClickHouse/ClickHouse/issues/101772). [#106686](https://github.com/ClickHouse/ClickHouse/pull/106686) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a rare spurious `RESOURCE_ACCESS_DENIED` error ("Scheduler queue with resource request is about to be destructed") for a query that was actually granted access to a workload resource, caused by a reused thread-local request object retaining a previous request's failure. [#106690](https://github.com/ClickHouse/ClickHouse/pull/106690) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a race between destroying `INATSConsumer` objects and calling `INATSConsumer::onMsg` on them via callbacks in the NATS library. [#106692](https://github.com/ClickHouse/ClickHouse/pull/106692) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixed `match`, `extract`, `extractAll` and `countMatches` returning wrong results for regular expressions containing hex or octal escapes. [#106709](https://github.com/ClickHouse/ClickHouse/pull/106709) ([ofeliacode](https://github.com/ofeliacode)). * Keeper's internal Raft TLS now honors the `openSSL.client.verificationMode` setting. Previously peer certificate verification was always enabled for inter-Keeper Raft communication regardless of this setting, so `none` was silently ignored. Now `none` explicitly disables Raft peer-certificate verification, while an absent setting keeps the previous secure-by-default behavior. Configurations that explicitly set `none` will stop verifying Raft peer certificates after upgrade, matching the configured intent. [#106726](https://github.com/ClickHouse/ClickHouse/pull/106726) ([Antonio Andelic](https://github.com/antonio2368)). * Fix logical error in startup scripts in `SYSTEM RELOAD CONFIG`. Additionally, load startup scripts on `SYSTEM RELOAD CONFIG` (currently private-only functionality). [#106727](https://github.com/ClickHouse/ClickHouse/pull/106727) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Revert a change that made `sumMap` / the `-Map` combinator reject custom-named numeric value types (such as `SimpleAggregateFunction(sum, T)` and `Bool`) with `ILLEGAL_TYPE_OF_ARGUMENT: Values for -Map cannot be summed`, breaking previously-working aggregations. [#106729](https://github.com/ClickHouse/ClickHouse/pull/106729) ([Nikita Fomichev](https://github.com/fm4v)). * Fixed multiple memory-safety and resource-exhaustion issues in format readers reachable from untrusted input: a heap out-of-bounds read in the native Parquet reader's `DataPageV2` definition/repetition level-length handling, a stack overflow on Parquet files with deeply nested schemas, and allocations that ignored `max_memory_usage` when parsing GeoParquet WKB/WKT geometry and Avro strings/bytes. [#106739](https://github.com/ClickHouse/ClickHouse/pull/106739) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a heap buffer overflow (server crash) in `decodeHTMLComponent` when decoding strings containing the expanding HTML entities `≫⃒` or `≪⃒`, reachable by any user with a single `SELECT`. [#106741](https://github.com/ClickHouse/ClickHouse/pull/106741) ([Raúl Marín](https://github.com/Algunenano)). * Reject `CREATE TABLE`, `ALTER TABLE ADD INDEX`, and `CREATE INDEX` with `GRANULARITY 0` for skipping indexes with a clear `BAD_ARGUMENTS` error. Previously this triggered an `Inconsistent AST formatting` exception in debug builds. [#106783](https://github.com/ClickHouse/ClickHouse/pull/106783) ([Groene AI](https://github.com/groeneai)). * Fixed Azure BACKUP/RESTORE ignoring endpoint settings for legacy-form `azure_blob_storage` disks. [#106784](https://github.com/ClickHouse/ClickHouse/pull/106784) ([Julia Kartseva](https://github.com/jkartseva)). * Fixed a `Bad cast` exception while pruning parts by `MinMax` statistics when a key column is `LowCardinality` and the predicate constant is `LowCardinality(Nullable(...))`. [#106793](https://github.com/ClickHouse/ClickHouse/pull/106793) ([Groene AI](https://github.com/groeneai)). * Fix inconsistent columns (that leads to `LOGICAL_ERROR` later) on exception (i.e. `MEMORY_LIMIT_EXCEEDED`) during parsing. [#106802](https://github.com/ClickHouse/ClickHouse/pull/106802) ([Azat Khuzhin](https://github.com/azat)). * Fix after configuring `keeper_server.http_control.secure_port`, the server returned an HTTP response when making requests to HTTPS clients. [#106822](https://github.com/ClickHouse/ClickHouse/pull/106822) ([linjiayu1025-collab](https://github.com/linjiayu1025-collab)). * Fix logical error in `parseDateTime` with non-ASCII input bytes. [#106856](https://github.com/ClickHouse/ClickHouse/pull/106856) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `SHOW CREATE ROW POLICY` emitting `restrictive`/`permissive` in lowercase instead of uppercase, inconsistent with other keywords. [#106865](https://github.com/ClickHouse/ClickHouse/pull/106865) ([Valery Petrov](https://github.com/valerypetrov)). * Fixes the case when parallel replicas wasn't applied for view with `UNION` due to empty table in the `UNION`. [#106900](https://github.com/ClickHouse/ClickHouse/pull/106900) ([Igor Nikonov](https://github.com/devcrafter)). * Fix server crash (SIGSEGV) when reading truncated `Protobuf` data with `input_format_allow_errors_num > 0`. [#106905](https://github.com/ClickHouse/ClickHouse/pull/106905) ([Andrey Tsarevskiy | Андрей Царевский ](https://github.com/atsarevskiy)). * Fix `THERE_IS_NO_COLUMN` exception for distributed queries involving `optimize_rewrite_aggregate_function_with_if` optimization when the aggregate function argument requires a cast to `Nullable` type. [#106908](https://github.com/ClickHouse/ClickHouse/pull/106908) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix an exception (`LOGICAL_ERROR`) in the join runtime filter when the join key contains a `Variant` or `Dynamic` type nested inside a `Tuple`, `Array`, or `Map` and the right side of the join has a single distinct value. [#106931](https://github.com/ClickHouse/ClickHouse/pull/106931) ([Groene AI](https://github.com/groeneai)). * Fixed a signed integer overflow (undefined behavior) in `arrayLevenshteinDistanceWeighted` and `arraySimilarity` when the weight arrays contain large integer values. The weighted distance is now accumulated in a wide integer for integral weights, so large integer weights no longer overflow and stay exact. [#106934](https://github.com/ClickHouse/ClickHouse/pull/106934) ([Groene AI](https://github.com/groeneai)). * Fix a server crash (null pointer dereference) when running `TRUNCATE` or `DROP` on an `EmbeddedRocksDB` table whose RocksDB handle was released, for example a `read_only` table whose data directory was emptied by a prior `TRUNCATE`. [#106940](https://github.com/ClickHouse/ClickHouse/pull/106940) ([Groene AI](https://github.com/groeneai)). * Fix an exception (`std::length_error` reported as a `LOGICAL_ERROR`) when reading from a `*Cluster` table function such as `urlCluster` with a very large `max_streams_for_files_processing_in_cluster_functions` setting. The number of streams is now bounded to a sane value. [#106946](https://github.com/ClickHouse/ClickHouse/pull/106946) ([Groene AI](https://github.com/groeneai)). * Fix a server crash (null pointer dereference of `DB::IConnections`) in `RemoteQueryExecutor` when a distributed query is cancelled right before it is sent to a shard. [#106950](https://github.com/ClickHouse/ClickHouse/pull/106950) ([Groene AI](https://github.com/groeneai)). * Fixed `elapsed_us` always being zero, and `read_rows`/`read_bytes` being undercounted, in `system.processors_profile_log` and `system.query_log` for asynchronous insert flush (`AsyncInsertFlush`) queries. [#106982](https://github.com/ClickHouse/ClickHouse/pull/106982) ([Christoph Wurm](https://github.com/cwurm)). * Fix a crash (`Source column is not Map` / `SIGSEGV`) when merging sorted blocks that contain a `Variant` column with a `Map` variant whose local storage order differs from its global order. [#107011](https://github.com/ClickHouse/ClickHouse/pull/107011) ([Groene AI](https://github.com/groeneai)). * Fix a logical error `conflicted_part_name.has_value()` raised during a synchronous insert into a `ReplicatedMergeTree` table when the inserted block was fully deduplicated and the conflicting part's deduplication node had already been removed (for example by a concurrent `DROP PARTITION`). [#107026](https://github.com/ClickHouse/ClickHouse/pull/107026) ([Groene AI](https://github.com/groeneai)). * Fixed an exception (logical error `this->visited_views == right->visited_views`) on `INSERT` when two materialized views on the same source table write to the same target table and a dependent view reads that target, with `materialized_views_squash_parallel_inserts` enabled. [#107027](https://github.com/ClickHouse/ClickHouse/pull/107027) ([Groene AI](https://github.com/groeneai)). * Fixed a `Block structure mismatch in UnionStep stream` logical error (server abort on debug/sanitizer builds, `Code: 49` on release builds) that occurred when one branch of a `UNION`/`INTERSECT`/`EXCEPT` read a `Sparse`-serialized column while the sibling branch read the same column as a full one (for example when pushing to a materialized view). [#107041](https://github.com/ClickHouse/ClickHouse/pull/107041) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` exception when inserting into a DeltaLake table with columns that do not match its write schema (for example a `Nested` column that flattens to subcolumns, or a table function with an explicit column subset). Such inserts now fail with a user-facing `INCOMPATIBLE_COLUMNS` error instead. Closes [#87402](https://github.com/ClickHouse/ClickHouse/issues/87402). [#107058](https://github.com/ClickHouse/ClickHouse/pull/107058) ([Groene AI](https://github.com/groeneai)). * Fix `TYPE_MISMATCH` error ("Cannot convert string ... to type ...") for `ORDER BY ... LIMIT n` queries when lazy materialization placed another column before the sort column. The top-K threshold is now read from the correct sort column. [#107060](https://github.com/ClickHouse/ClickHouse/pull/107060) ([Groene AI](https://github.com/groeneai)). * Fixed a syntax error when a `FORMAT`, `SETTINGS`, or `INTO OUTFILE` clause follows `SHOW ROW POLICIES` or `SHOW MASKING POLICIES` (e.g. `SHOW ROW POLICIES FORMAT TabSeparated`). [#107061](https://github.com/ClickHouse/ClickHouse/pull/107061) ([Groene AI](https://github.com/groeneai)). * Fix a compound `ALTER TABLE ... RENAME COLUMN a TO b, RENAME COLUMN c TO a` that reuses a freed column name (a "swap") between columns of different types. The materialized part recorded the wrong column type, so a later `SELECT` failed with `Conversion between numeric types and IPv6 is not supported` (or aborted on part load in debug builds). [#107064](https://github.com/ClickHouse/ClickHouse/pull/107064) ([Groene AI](https://github.com/groeneai)). * Fixed non-deterministic results of the `roundDown` function when the boundaries array contains `NaN`. The same input value could return a finite boundary or `NaN` depending on the surrounding rows in a batch. `NaN` boundaries are now ignored, so the result depends only on the finite boundaries. [#107065](https://github.com/ClickHouse/ClickHouse/pull/107065) ([Groene AI](https://github.com/groeneai)). * Fixed `quantileTDigest` and `quantileTDigestWeighted` throwing `DECIMAL_OVERFLOW` for `Date` and `DateTime` arguments when the interpolated quantile is fractional but in range (for example `quantileTDigestWeighted(date, weight)` on values that all fit in the type). The fractional result is now truncated to the result type, matching `quantilesTDigestWeighted`; genuine out-of-range values still raise an error. Closes: [#106722](https://github.com/ClickHouse/ClickHouse/issues/106722). [#107066](https://github.com/ClickHouse/ClickHouse/pull/107066) ([Groene AI](https://github.com/groeneai)). * Fix `trimLeft`, `trimRight`, and `trimBoth` (and aliases `ltrim`, `rtrim`, `trim`) throwing `TOO_LARGE_STRING_SIZE` when the custom trim character set is longer than 16 characters. Trim sets of any length are now supported again. [#107071](https://github.com/ClickHouse/ClickHouse/pull/107071) ([Nikita Fomichev](https://github.com/fm4v)). * Fixed silent truncation of out-of-range integer values in `Enum8`/`Enum16` type definitions. `Enum8('a' = 200)` now throws `ARGUMENT_OUT_OF_BOUND` instead of silently creating `Enum8('a' = -56)`. [#107081](https://github.com/ClickHouse/ClickHouse/pull/107081) ([Groene AI](https://github.com/groeneai)). * Fix wrong row order (and a `LOGICAL_ERROR` "Rows are not sorted with permutation" in debug builds) for multi-column `ORDER BY ... LIMIT` queries that sort by `Nullable` columns when several rows tie on the leading columns. [#107094](https://github.com/ClickHouse/ClickHouse/pull/107094) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`Expected the argument N to have X rows, but it has Y`) when executing a function over a `Dynamic` column built by a JOIN over `Dynamic` (for example the non-joined rows of a `RIGHT`/`FULL JOIN`, or a correlated `EXISTS` subquery decorrelated into a join). [#107095](https://github.com/ClickHouse/ClickHouse/pull/107095) ([Groene AI](https://github.com/groeneai)). * Fix spurious ZooKeeper session recreation on config reload. [#107096](https://github.com/ClickHouse/ClickHouse/pull/107096) ([Azat Khuzhin](https://github.com/azat)). * Do not hold mutex across ZooKeeper reads in access entities refresh. [#107097](https://github.com/ClickHouse/ClickHouse/pull/107097) ([Azat Khuzhin](https://github.com/azat)). * Fix a `Logical error: Too large size passed to allocator` exception on `INSERT` into a `MergeTree` table when `adaptive_write_buffer_initial_size` is set to an extremely large value. The adaptive write buffer initial size is now clamped to the buffer maximum. [#107104](https://github.com/ClickHouse/ClickHouse/pull/107104) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL ERROR` (`Bad cast from type DB::ColumnString to DB::ColumnLowCardinality`) when a `Variant` constant containing a `LowCardinality` member is compared to a key column whose key expression is a non-monotonic deterministic function (for example a `minmax` skip index over `sipHash64(col)`). [#107111](https://github.com/ClickHouse/ClickHouse/pull/107111) ([Groene AI](https://github.com/groeneai)). * Fix a `Bad cast from type DB::IColumn const* to DB::ColumnNullable const*` logical error when a qualified asterisk (`t.*`) over a `JOIN ... USING` key is passed to an aggregate function and the other side of an outer JOIN has a `Nullable` key (with `join_use_nulls = 0`). [#107129](https://github.com/ClickHouse/ClickHouse/pull/107129) ([Groene AI](https://github.com/groeneai)). * Fix `ALTER TABLE ... ON CLUSTER` batches that mix `MODIFY SETTING`/`RESET SETTING` with a comment change (for example `MODIFY COMMENT 'x', MODIFY SETTING old_parts_lifetime = 123`) being applied only on the leader replica, leaving the other replicas diverged. Also fix a positional or per-column-`SETTINGS` `MODIFY COLUMN ... COMMENT` being misclassified as a local comment-only metadata change, which left the column reorder out of the replicated metadata and could cause `INCOMPATIBLE_COLUMNS` on replica restart. [#107142](https://github.com/ClickHouse/ClickHouse/pull/107142) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` ("Table expression ... data must be initialized") raised when a qualified asterisk matcher (for example `x.*`) referenced a recursive CTE by name inside its own recursive term. Such matchers now expand the recursive table's columns, the same way a qualified column (`x.a`) or an unqualified matcher (`*`) already does in that position. [#107144](https://github.com/ClickHouse/ClickHouse/pull/107144) ([Groene AI](https://github.com/groeneai)). * Fixed wrong query results caused by the query condition cache when on-fly mutations (`apply_mutations_on_fly`) or patch parts filtered rows before `PREWHERE`. A query reading with `apply_mutations_on_fly = 1` could poison the cache so that a later query with `apply_mutations_on_fly = 0` and the same predicate skipped marks it should have read and returned too few rows. The same fix also covers row-level security policies, which are prepended as a filter ahead of `PREWHERE`: a query run under a restrictive row policy could poison the cache for a later query that uses the same predicate without that policy. [#107145](https://github.com/ClickHouse/ClickHouse/pull/107145) ([Groene AI](https://github.com/groeneai)). * Fixed `BACKUP` to `AzureBlobStorage`: copying a data file inside a backup wrote the destination object outside the backup directory. Backup object existence checks on `S3` destinations now use exact `HeadObject` requests instead of prefix listing, preventing false matches of similarly-prefixed keys. [#107153](https://github.com/ClickHouse/ClickHouse/pull/107153) ([Pablo Marcos](https://github.com/pamarcos)). * Fixed a signed integer overflow in `quantileExactExclusive`, `quantilesExactExclusive`, `quantileExactInclusive` and `quantilesExactInclusive` that could produce a wrong result for `Int64` inputs spanning a large range. [#107154](https://github.com/ClickHouse/ClickHouse/pull/107154) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` ("Unexpected exception in refresh scheduling") that could put the server into a crash-loop on restart when a refreshable materialized view has a `REFRESH ... DEPENDS ON ` dependency whose unqualified name matches a temporary table or a CTE name. [#107156](https://github.com/ClickHouse/ClickHouse/pull/107156) ([Groene AI](https://github.com/groeneai)). * Make sure we don't run startup scripts on config reload, as that is semantically wrong and unexpected by users - and has been known to be error-prone. [#107187](https://github.com/ClickHouse/ClickHouse/pull/107187) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Cap the max value for `queue_size` for pre-reserve. [#107205](https://github.com/ClickHouse/ClickHouse/pull/107205) ([Elian Gidoni](https://github.com/eliangidoni)). * Fix the `Argument ... of GROUPING function is not a part of GROUP BY clause` error for queries that use the `grouping` function with the `group_by_use_nulls` setting enabled. [#107206](https://github.com/ClickHouse/ClickHouse/pull/107206) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix incorrect result order for `ORDER BY` over `UNION ALL` with `optimize_read_in_order` enabled when the union pipeline was narrowed due to `max_streams_for_union_step` settings; narrowing is now skipped when the plan relies on sorted UNION output streams. Closes [#106880](https://github.com/ClickHouse/ClickHouse/issues/106880). [#107208](https://github.com/ClickHouse/ClickHouse/pull/107208) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a possible null pointer dereference while resolving proxy configuration during late server shutdown. [#107231](https://github.com/ClickHouse/ClickHouse/pull/107231) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix lightweight `UPDATE` queries with legacy parallel replicas enabled for non-replicated `MergeTree` tables. [#107246](https://github.com/ClickHouse/ClickHouse/pull/107246) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort (`std::out_of_range` logical error) when inserting into an `Iceberg` table whose write block column names do not match the field ids of the latest schema (for example after a concurrent writer renames a column within the `iceberg_metadata_staleness_ms` window). The insert now fails with a clean query error instead of crashing the server. [#107279](https://github.com/ClickHouse/ClickHouse/pull/107279) ([Groene AI](https://github.com/groeneai)). * Fix the stuck shutdown bug on server and local due to static thread pools keeping idle threads indefinitely if `max_*_thread_pool_free_size > 0`. [#107291](https://github.com/ClickHouse/ClickHouse/pull/107291) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix the `s3` table function silently ignoring a lowercase positional `partition_strategy` (e.g. `hive`). [#107297](https://github.com/ClickHouse/ClickHouse/pull/107297) ([Julia Kartseva](https://github.com/jkartseva)). * Fix `low_cardinality_allow_in_native_format=0` handling in parallel blocks marshalling. [#107319](https://github.com/ClickHouse/ClickHouse/pull/107319) ([Azat Khuzhin](https://github.com/azat)). * Fix wrong (often empty) results from `ORDER BY LIMIT n` when the `use_skip_indexes_for_top_k` optimization is active and a part with a `minmax` skip index on the sort column has had rows removed by a lightweight `DELETE`. The optimization no longer ranks the stale minmax of lightweight-deleted parts ahead of the parts that hold the live top rows. [#107320](https://github.com/ClickHouse/ClickHouse/pull/107320) ([Groene AI](https://github.com/groeneai)). * Fixed a bug in ClickHouse Keeper where the snapshot metadata reported via `last_snapshot` (and `zk_latest_snapshot_size` in `mntr`) could move backwards after a stale or duplicated snapshot install, which could also let a same-index local snapshot overwrite a registered snapshot file in place while it was still being streamed to a peer or uploaded to S3. [#107321](https://github.com/ClickHouse/ClickHouse/pull/107321) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed possible server stack overflow (crash) when reading a deeply nested schema or value in the `MsgPack`, `BSON`, `ORC`, `Parquet`, `JSON`, `DeltaLake`, `Iceberg` and `Paimon` formats. Such deeply nested input is now rejected with an exception. [#107341](https://github.com/ClickHouse/ClickHouse/pull/107341) ([Raúl Marín](https://github.com/Algunenano)). * Fix a possible logical error in `SYSTEM SYNC DATABASE REPLICA ... STRICT` and make the `STRICT` modifier actually take effect for database replicas. [#107344](https://github.com/ClickHouse/ClickHouse/pull/107344) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixes a server abort in debug/sanitizer builds when reading a `DeltaLake` table whose ClickHouse schema names a column that is absent from the Delta column mapping, for example after a column was renamed or dropped in the Delta log. The query now fails with a catchable `INCORRECT_DATA` error instead. [#107347](https://github.com/ClickHouse/ClickHouse/pull/107347) ([Groene AI](https://github.com/groeneai)). * Fix `ORDER BY ... WITH FILL` producing extra rows when an `ORDER BY` column before the fill column uses a `COLLATE` collation. The rows are now grouped by the sorting prefix using that collation, matching the sort order. [#107365](https://github.com/ClickHouse/ClickHouse/pull/107365) ([Groene AI](https://github.com/groeneai)). * Fixed a crash (`LOGICAL_ERROR` in debug builds) and a silent wrong-results bug (in release builds) when reading an Iceberg table whose metadata re-binds an existing `schema-id` to a different schema across metadata versions. Such metadata is now rejected with `ICEBERG_SPECIFICATION_VIOLATION`. [#107370](https://github.com/ClickHouse/ClickHouse/pull/107370) ([Groene AI](https://github.com/groeneai)). * Fix a `LOGICAL_ERROR` (`Variant N (T) has size X, but expected Y`) when a function such as `toString` or `concat` is applied to a `Variant` or `Dynamic` column that holds a single non-empty variant together with NULLs, and the function returns its input column unchanged. [#107374](https://github.com/ClickHouse/ClickHouse/pull/107374) ([Groene AI](https://github.com/groeneai)). * Fix `Logical error: 'Duplicate announcement received for replica number N'` that could occur with parallel replicas when a scalar subquery contained nested subqueries reading the same table. [#107381](https://github.com/ClickHouse/ClickHouse/pull/107381) ([Groene AI](https://github.com/groeneai)). * Fix `getServerSetting` to return the live effective value for runtime-changeable server settings (such as `max_server_memory_usage`, `mark_cache_size`, `max_concurrent_queries`, thread pool sizes, etc.), matching what `system.server_settings` reports. [#107388](https://github.com/ClickHouse/ClickHouse/pull/107388) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `arrayResize` with a `Decimal` size argument: the size is now interpreted by its real value (e.g. `arrayResize([1, 2, 3], 1.5::Decimal(2, 1))` returns one element) instead of the raw unscaled representation. [#107389](https://github.com/ClickHouse/ClickHouse/pull/107389) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `LOGICAL_ERROR` (`block.rows() == getRows()`) raised on an async `INSERT` into an `Alias` table when `use_strict_insert_block_limits` was enabled. [#107400](https://github.com/ClickHouse/ClickHouse/pull/107400) ([Groene AI](https://github.com/groeneai)). * Fix a logical error (`Unexpected return type from equals. Expected Nullable(UInt8). Got UInt8`) when the disjunction (partial predicate) push-down optimization pushes a condition over a `USING` key whose type is widened by the JOIN. Also fix a server crash (segmentation fault) in the analyzer when resolving identifiers in `JOIN ... USING` queries that contain constant-foldable `if`/`multiIf` branches referencing unknown identifiers. [#107407](https://github.com/ClickHouse/ClickHouse/pull/107407) ([Groene AI](https://github.com/groeneai)). * Fixed a heap buffer overflow (server crash) in `windowFunnel` when finalizing a crafted aggregate-function state with an out-of-range event type, reachable by any user with a single `SELECT`. [#107412](https://github.com/ClickHouse/ClickHouse/pull/107412) ([uwezkhan](https://github.com/uwezkhan)). * Fixed undefined behavior when a non-finite floating-point value (such as `nan` or `inf`) is passed as the timestamp/duration argument of the `prometheusQuery` / `prometheusQueryRange` table functions. Such an argument now raises `BAD_ARGUMENTS` instead of producing a garbage timestamp. [#107417](https://github.com/ClickHouse/ClickHouse/pull/107417) ([Groene AI](https://github.com/groeneai)). * Fix `MaterializedPostgreSQL` silently stopping replication of changes when the PostgreSQL database or table name contains upper-case letters (the `pgoutput` consumer requested an unquoted, lower-cased publication name that did not match the case-preserving publication). [#107423](https://github.com/ClickHouse/ClickHouse/pull/107423) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an exception (`Logical error: Not-ready Set is passed as the second argument for function 'in'`) when a key expression (`ORDER BY`, `PRIMARY KEY`, `PARTITION BY`, or a skip `INDEX`) contained an `IN` operator with a table on the right-hand side, e.g. `ORDER BY (x IN some_table)`. Such key expressions are now rejected at table creation time. [#107424](https://github.com/ClickHouse/ClickHouse/pull/107424) ([Groene AI](https://github.com/groeneai)). * Fix incorrect results from the `optimize_rewrite_aggregate_function_with_if` optimization for aggregate functions that preserve `NULL` payload values (the `*_respect_nulls` family: `anyRespectNulls`, `first_value_respect_nulls`, `anyLast_respect_nulls`, `last_value_respect_nulls`). The optimization no longer rewrites `f(if(cond, x, NULL))` into the `-If` form for such functions. [#107430](https://github.com/ClickHouse/ClickHouse/pull/107430) ([Groene AI](https://github.com/groeneai)). * Fix a spurious `filesystem error: in last_write_time: No such file or directory` exception when listing a local-disk object storage directory (e.g. an Iceberg table on a `local` disk) while files are being concurrently replaced. A concurrently removed entry is now omitted from the listing instead of aborting it. [#107432](https://github.com/ClickHouse/ClickHouse/pull/107432) ([Groene AI](https://github.com/groeneai)). * Fix `THERE_IS_NO_COLUMN` error when `optimize_if_transform_strings_to_enum = 1` and the optimized `if`/`transform`-over-string-literals expression is a `GROUP BY` or `ORDER BY` key over a Distributed table or parallel replicas. [#107455](https://github.com/ClickHouse/ClickHouse/pull/107455) ([Groene AI](https://github.com/groeneai)). * Fixed a rare server crash in `DISTINCT` processing that could occur when an allocation failed (for example, when hitting a memory limit) while the set of distinct keys was being initialized. [#107467](https://github.com/ClickHouse/ClickHouse/pull/107467) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR: Unexpected return type from materialize` (and similar type-mismatch errors) when `apply_mutations_on_fly = 1` is used on a table with a pending on-fly `UPDATE` whose target column is also read as a function input by an earlier on-fly `UPDATE`, before an `ALTER MODIFY COLUMN ... LowCardinality(...)` mutation. [#107475](https://github.com/ClickHouse/ClickHouse/pull/107475) ([Groene AI](https://github.com/groeneai)). * Fixed stale AWS STS credentials when reading `DeltaLake` tables over `S3`: the engine now keeps the freshly-credentialed `S3` client when refreshing its snapshot, so long-running reads no longer fail after the STS token's TTL expires. The STS assume-role credentials provider now also honors credential refresh for any `S3` access using STS. [#107480](https://github.com/ClickHouse/ClickHouse/pull/107480) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fixed `SELECT ... FINAL` and `OPTIMIZE TABLE ... FINAL` returning duplicate rows after `CREATE TABLE ... CLONE AS`, `ATTACH PARTITION ... FROM` or `MOVE PARTITION ... TO TABLE` adopted parts from a plain `MergeTree` into a `ReplacingMergeTree`, `SummingMergeTree` or `AggregatingMergeTree`. The adopted part's merge level is now reset to 0 when the source and destination engines differ, so the destination deduplicates it on the next merge. [#107481](https://github.com/ClickHouse/ClickHouse/pull/107481) ([Groene AI](https://github.com/groeneai)). * Fix an integer underflow in the PostgreSQL wire protocol parser where a message with a length field smaller than 4 caused a `size - 4` wraparound and an oversized `resize`/`ignore`. [#107485](https://github.com/ClickHouse/ClickHouse/pull/107485) ([uwezkhan](https://github.com/uwezkhan)). * Query cancellation is now better tracked while waiting for the quorum in ReplicatedMergeTree. [#107513](https://github.com/ClickHouse/ClickHouse/pull/107513) ([Nikita Taranov](https://github.com/nickitat)). * Fixed a `Not-ready Set is passed as the second argument for function 'in'` (`LOGICAL_ERROR`) when querying a table with a `PARTITION BY` key and an `IN`/`NOT IN` subquery wrapped inside a larger expression, for example `WHERE (c0 IN (SELECT ...)) != 0`. [#107515](https://github.com/ClickHouse/ClickHouse/pull/107515) ([Groene AI](https://github.com/groeneai)). * Fixed `currentUser()`, `user()`, `SESSION_USER` and `authenticatedUser()` evaluating to an empty string on the asynchronous insert flush path (with `async_insert = 1`). This affected `DEFAULT`/`MATERIALIZED` column expressions and materialized views that reference these functions, which silently stored an empty string instead of the inserting user. [#107541](https://github.com/ClickHouse/ClickHouse/pull/107541) ([Groene AI](https://github.com/groeneai)). * With `enable_analyzer = 1` (the default), querying a table by its bare name when it only exists in another database now suggests the right table, e.g. `SELECT * FROM functions` reports `Maybe you meant system.functions?`. Previously the new analyzer gave a hint-less `Unknown table expression identifier` error, while the old analyzer already produced the helpful suggestion. [#107550](https://github.com/ClickHouse/ClickHouse/pull/107550) ([Groene AI](https://github.com/groeneai)). * Account memory used by the rapidjson library (in `prettyPrintJSON`, `JSONMergePatch` and the rapidjson JSON parser) against the memory tracker, so pathological inputs are rejected with `MEMORY_LIMIT_EXCEEDED` instead of allocating without bound. [#107555](https://github.com/ClickHouse/ClickHouse/pull/107555) ([Raúl Marín](https://github.com/Algunenano)). * Fix a `LOGICAL_ERROR` (`updateFormatPrewhereInfo called more than once`) raised when querying a `file()`, `url()`, or object-storage source with both an explicit `PREWHERE` and a `WHERE` while `optimize_prewhere_after_pushdown` was enabled. [#107568](https://github.com/ClickHouse/ClickHouse/pull/107568) ([Groene AI](https://github.com/groeneai)). * Fix a crash (null pointer dereference) that could happen when a distributed query plan was executed locally (`make_distributed_plan` + `distributed_plan_execute_locally`) with `log_formatted_queries = 1`. [#107570](https://github.com/ClickHouse/ClickHouse/pull/107570) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort (`std::terminate`, signal 6) during teardown of a distributed-plan query (`make_distributed_plan = 1`) when a worker status-check failed to re-schedule the next check (for example `CANNOT_SCHEDULE_TASK` on shutdown or `MEMORY_LIMIT_EXCEEDED`). The query now fails cleanly and the server keeps running. [#107575](https://github.com/ClickHouse/ClickHouse/pull/107575) ([Groene AI](https://github.com/groeneai)). * Added missing bounds checks in Elf and DWARF parsing. [#107579](https://github.com/ClickHouse/ClickHouse/pull/107579) ([Michael Kolupaev](https://github.com/al13n321)). * Added missing bounds checks in ORC reader. [#107580](https://github.com/ClickHouse/ClickHouse/pull/107580) ([Michael Kolupaev](https://github.com/al13n321)). * Fixed an exception (`Digest does not match` logical error) that could happen on `RENAME TABLE`, `RENAME DATABASE`, or `CREATE OR REPLACE TABLE` involving a `TimeSeries` table inside a `Replicated` database. Renaming a `TimeSeries` table is now supported. [#107583](https://github.com/ClickHouse/ClickHouse/pull/107583) ([Groene AI](https://github.com/groeneai)). * Fix an unauthenticated memory-exhaustion denial of service on the MySQL protocol port. [#107599](https://github.com/ClickHouse/ClickHouse/pull/107599) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix `DROP TABLE` of a `TimeSeries` table in a `Replicated` database, which previously leaked the inner tables and left the background drop task retrying forever (`DROP TABLE ... SYNC` would hang). [#107604](https://github.com/ClickHouse/ClickHouse/pull/107604) ([Groene AI](https://github.com/groeneai)). * Fixed two path-traversal issues in the replicated part fetch protocol that could let a malicious replica write files outside the part directory. [#107606](https://github.com/ClickHouse/ClickHouse/pull/107606) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed 'Account must be specified error' when reading a Delta Lake table over Azure. [#107620](https://github.com/ClickHouse/ClickHouse/pull/107620) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fixed `ALTER TABLE ... REPLACE PARTITION` on a plain `MergeTree` table resurrecting the replaced-out parts after a server restart, which made the table return both the replacement rows and the stale replaced rows. [#107623](https://github.com/ClickHouse/ClickHouse/pull/107623) ([Groene AI](https://github.com/groeneai)). * Fix a server crash when reading a materialized view whose target is a `Distributed` table while the query runs with `enable_analyzer = 0`. [#107653](https://github.com/ClickHouse/ClickHouse/pull/107653) ([Groene AI](https://github.com/groeneai)). * When several quotas are assigned to the same user or role, all of them are now enforced together (a query is rejected if any of them is exceeded), instead of only one quota being enforced and chosen non-deterministically. `SHOW QUOTA` and `system.quota_usage` now show all quotas enforced for the current user. [#107664](https://github.com/ClickHouse/ClickHouse/pull/107664) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `s3` and other object storage table functions throwing `LOGICAL_ERROR` instead of `BAD_ARGUMENTS` when a key-value argument is duplicated, e.g. `s3('http://...', format = 'CSV', format = 'TSV')`. [#107670](https://github.com/ClickHouse/ClickHouse/pull/107670) ([Groene AI](https://github.com/groeneai)). * Fixed the MySQL interface being unusable with `MySQL Connector/J` 8.2.0 and newer (including 9.x). The `info` field of the `OK` packet is now length-encoded, matching the MySQL server, so the JDBC driver can connect. [#107693](https://github.com/ClickHouse/ClickHouse/pull/107693) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` ("Input nodes size mismatch in dag") when a query with `make_distributed_plan = 1` joins on a function-wrapped key whose two sides have no common type (for example `ON intDiv(-1, t1.key) = t2.key` with a `UInt64` right key). [#107701](https://github.com/ClickHouse/ClickHouse/pull/107701) ([Groene AI](https://github.com/groeneai)). * Reading Arrow/ArrowStream data with empty `String`/`Binary` columns produced by Apache Arrow Java < 19.0.0 (including Apache Spark) no longer throws `INCORRECT_DATA`. [#107764](https://github.com/ClickHouse/ClickHouse/pull/107764) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a `BAD_GET` exception ("Bad get: has String, requested UInt64") from `countmin` column statistics estimation when a query compares a column against a literal of a different type that is not pre-coerced, such as `numeric_col IN (SELECT '5')` or `string_col IN (SELECT 5)`. [#107793](https://github.com/ClickHouse/ClickHouse/pull/107793) ([Groene AI](https://github.com/groeneai)). * Fixed the `odbc` and `jdbc` table functions hanging for minutes and ignoring query cancellation (`KILL QUERY`, `max_execution_time`) when the bridge becomes unresponsive while inferring the remote table structure. [#107809](https://github.com/ClickHouse/ClickHouse/pull/107809) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an exception (`Logical error: 'index >= result.start'`) when formatting a malformed query that mixes the positional and named secret-argument forms of the `s3`/`gcs` table functions, e.g. `s3('url', 'a', 'b', secret_access_key = 'c')`. [#107818](https://github.com/ClickHouse/ClickHouse/pull/107818) ([Groene AI](https://github.com/groeneai)). * Fix set skip index not pruning granules over `LowCardinality` columns. [#107868](https://github.com/ClickHouse/ClickHouse/pull/107868) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fixed insert deduplication computing wrong hashes for `String` and `Array` columns with the server setting `insert_deduplication_version = new_unified_hash`: identical inserts could fail to deduplicate because the deduplication hash depended on the row's position within the inserted block. [#107915](https://github.com/ClickHouse/ClickHouse/pull/107915) ([Sema Checherinda](https://github.com/CheSema)). * Fix incorrect results of `ORDER BY` over `Nullable` columns with multiple sort keys on macOS (Apple Silicon), caused by a missing sign-extension in the JIT-compiled sort comparator. [#107973](https://github.com/ClickHouse/ClickHouse/pull/107973) ([Raúl Marín](https://github.com/Algunenano)). * Fix performance regression when reading `Dynamic` columns with multiple threads. Caused by [#100730](https://github.com/ClickHouse/ClickHouse/pull/100730). Closes [#107942](https://github.com/ClickHouse/ClickHouse/issues/107942). [#107997](https://github.com/ClickHouse/ClickHouse/pull/107997) ([Pavel Kruglov](https://github.com/Avogar)). * The deprecated data lake setting `storage_catalog_url` is now correctly rejected by the catalog guard (previously only `storage_catalog_type` and `storage_aws_access_key_id` were checked), and the error message lists all deprecated settings. [#108040](https://github.com/ClickHouse/ClickHouse/pull/108040) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a `Bad cast from type DB::ColumnSparse to DB::ColumnVector` logical error when a `LIKE` query reads from a `text` index via the direct-read fallback path over a column stored sparse. [#108068](https://github.com/ClickHouse/ClickHouse/pull/108068) ([Groene AI](https://github.com/groeneai)). * Always run access cache batch-finished handlers, even on empty batch. [#108124](https://github.com/ClickHouse/ClickHouse/pull/108124) ([Azat Khuzhin](https://github.com/azat)). * Fixed a `LOGICAL_ERROR` (`Column identifier is already registered`) for some queries with `UNION ALL`. [#100770](https://github.com/ClickHouse/ClickHouse/pull/100770) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed a use-after-free of the workload storage mutex during server shutdown. [#101447](https://github.com/ClickHouse/ClickHouse/pull/101447) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fixed a deadlock in `ALTER DATABASE MODIFY COMMENT` with the `Shared` catalog. [#103683](https://github.com/ClickHouse/ClickHouse/pull/103683) ([Nikolay Degterinsky](https://github.com/evillique)). * An empty Unicode-quoted identifier now raises a `SYNTAX_ERROR` instead of `CANNOT_PARSE_QUOTED_STRING`. [#104173](https://github.com/ClickHouse/ClickHouse/pull/104173) ([leonard9893](https://github.com/leonard9893)). * Preserve the original parameter types of `timeSeries*` aggregate functions so that `AggregateFunction` types round-trip correctly across table reattach and parallel replicas. [#104812](https://github.com/ClickHouse/ClickHouse/pull/104812) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed a `LOGICAL_ERROR` for a materialized CTE under `serialize_query_plan` with parallel replicas. [#104896](https://github.com/ClickHouse/ClickHouse/pull/104896) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed signed integer overflow in the `timeSeries*ToGrid` functions when the staleness (window) parameter is near `INT64_MAX`. [#105319](https://github.com/ClickHouse/ClickHouse/pull/105319) ([Groene AI](https://github.com/groeneai)). * A `compatibility` setting value no longer reverts `apply_row_policy_after_final` to `false`, so row policies are always applied correctly with `FINAL`. [#105637](https://github.com/ClickHouse/ClickHouse/pull/105637) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed `clickhouse chdig client` by honoring `SETPGROUP`/`RESETIDS`/`SETSIGDEF` in the `posix_spawn` stub. [#105858](https://github.com/ClickHouse/ClickHouse/pull/105858) ([Azat Khuzhin](https://github.com/azat)). * Fixed an exception during merge-tree sanity checks on projections when the merge/mutate executor was not initialized. [#105919](https://github.com/ClickHouse/ClickHouse/pull/105919) ([Mikhail Artemenko](https://github.com/Michicosun)). * During index analysis on projections, the `minmax` index is now loaded from the projection itself rather than from the parent part, producing correct results. [#105940](https://github.com/ClickHouse/ClickHouse/pull/105940) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fixed an exception in `addMonotonicChain` for `materialize(monotonic_chain(...))`. [#106050](https://github.com/ClickHouse/ClickHouse/pull/106050) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed `RESTORE` failing on backups that include `MASKING POLICY` dependents. [#106086](https://github.com/ClickHouse/ClickHouse/pull/106086) ([Julia Kartseva](https://github.com/jkartseva)). * Fixed possible memory corruption when expanding SQL user-defined functions with `prefer_column_name_to_alias` enabled. [#106121](https://github.com/ClickHouse/ClickHouse/pull/106121) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * A dictionary using the bridge connection is now reloaded after a server restart, re-establishing the connection. Closes [#106151](https://github.com/ClickHouse/ClickHouse/issues/106151). [#106152](https://github.com/ClickHouse/ClickHouse/pull/106152) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fixed the filesystem cache for `LocalObjectStorage`. [#106239](https://github.com/ClickHouse/ClickHouse/pull/106239) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fixed the lazy remote source for table functions with `use_delayed_remote_source`. [#106470](https://github.com/ClickHouse/ClickHouse/pull/106470) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixed `timeSeriesLastToGrid` for timestamps before the grid start and out-of-window timestamps. [#106504](https://github.com/ClickHouse/ClickHouse/pull/106504) ([Vitaly Baranov](https://github.com/vitlibar)). [#106577](https://github.com/ClickHouse/ClickHouse/pull/106577) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed a `LOGICAL_ERROR` (`Inconsistent AST formatting`) on a function name containing query parameters. [#106635](https://github.com/ClickHouse/ClickHouse/pull/106635) ([Vitaly Baranov](https://github.com/vitlibar)). * Fixed a logical error when a table is dropped before distributed plan task deserialization. [#106944](https://github.com/ClickHouse/ClickHouse/pull/106944) ([Alexander Gololobov](https://github.com/davenger)). * Use exact comparison for `Nullable` `GROUP BY` keys. [#107050](https://github.com/ClickHouse/ClickHouse/pull/107050) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed a `LOGICAL_ERROR` (`Trying to get name of not a column`) raised when a table-function argument was not a column expression (for example an unresolved `*` matcher from `multiIf`/`CASE`). [#107411](https://github.com/ClickHouse/ClickHouse/pull/107411) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed unbounded growth of the time-zone cache (`DateLUT`) when many distinct invalid time-zone names are processed. [#107464](https://github.com/ClickHouse/ClickHouse/pull/107464) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Write the `part_log` entry before reporting a plain `MergeTree` mutation as done. [#107741](https://github.com/ClickHouse/ClickHouse/pull/107741) ([Azat Khuzhin](https://github.com/azat)). * `arrayFold` now sanity-checks its array argument against malformed input. [#107932](https://github.com/ClickHouse/ClickHouse/pull/107932) ([Michael Kolupaev](https://github.com/al13n321)). * Fixed a rare data race and possible use-after-free on the cached skip-index archive reader of a `MergeTree` part, which could happen when a query read skip indices while the part was being moved or renamed. [#107995](https://github.com/ClickHouse/ClickHouse/pull/107995) ([Raúl Marín](https://github.com/Algunenano)). * Fix performance regression for `Map` subcolumns with `PREWHERE`. [#107988](https://github.com/ClickHouse/ClickHouse/pull/107988) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `parseDateTimeBestEffort` with a timezone throwing `CANNOT_PARSE_DATETIME` on NULL rows of `toString(Nullable(DateTime64))`. [#108310](https://github.com/ClickHouse/ClickHouse/pull/108310) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed the `ArrowFlight` table function and engine rejecting a named collection that omits the optional `dataset` key with a `No such key 'dataset'` error. [#108041](https://github.com/ClickHouse/ClickHouse/pull/108041) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an `Inconsistent AST formatting` logical error for a no-argument window function inside a `CODEC` or engine declaration (e.g. `CODEC(cume_dist() OVER (...))`); such a function now keeps its parentheses so the query survives a format/parse round-trip. [#107806](https://github.com/ClickHouse/ClickHouse/pull/107806) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `hasToken` with a separator-containing needle silently returning results via a text index instead of raising `BAD_ARGUMENTS`. [#108189](https://github.com/ClickHouse/ClickHouse/pull/108189) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * The setting `use_skip_indexes_on_data_read` can now be reverted to its pre-26.1 default (`false`) via the `compatibility` setting, providing an escape hatch for a performance regression where the on-data-read path defeats `minmax`/`set`/`bloom_filter` skip-index mark-range pruning. [#108330](https://github.com/ClickHouse/ClickHouse/pull/108330) ([egor romanov](https://github.com/egor-click)). * Fixed a possible crash (null pointer dereference) when the `database`/`db` override of a named collection passed to `remote`/`remoteSecure` is not a constant database name, e.g. `remote(nc, database = (SELECT 1))`. The query now fails with a clear error instead of crashing. [#108271](https://github.com/ClickHouse/ClickHouse/pull/108271) ([Groene AI](https://github.com/groeneai)). * Fixed refreshable materialized view getting stuck if the ZooKeeper connection is lost at the wrong moment. [#108234](https://github.com/ClickHouse/ClickHouse/pull/108234) ([Michael Kolupaev](https://github.com/al13n321)). * Fix `Bad cast from type DB::ColumnVector<...> to DB::ColumnTuple` when reading an `Array(Tuple(...))` column whose value is filled with defaults, e.g. after `ALTER TABLE ... ADD COLUMN` or after an unfinished `ALTER TABLE ... CLEAR COLUMN` mutation applied on the fly. [#107232](https://github.com/ClickHouse/ClickHouse/pull/107232) ([Groene AI](https://github.com/groeneai)). * Fix a `Bad cast from type DB::ColumnDynamic to DB::ColumnNullable` logical error when an explicit reference to a `JOIN ... USING` key whose common supertype is `Dynamic` is passed to an aggregate function, for example `count(t.key) IGNORE NULLS`. [#107289](https://github.com/ClickHouse/ClickHouse/pull/107289) ([Groene AI](https://github.com/groeneai)). * Fix silent data loss on plain (non-replicated) `MergeTree` when `REPLACE PARTITION`, `MOVE PARTITION`, `DETACH PARTITION`, or `DETACH PART` is run on a partition that still has unapplied lightweight `UPDATE` patches. These operations now reject the command, as `ReplicatedMergeTree` already does. [#107386](https://github.com/ClickHouse/ClickHouse/pull/107386) ([Groene AI](https://github.com/groeneai)). * Fix a crash (SIGSEGV in release builds, type-mismatch assertion in debug builds) in `has` over a `Map` with a `Dynamic` key when the lookup argument is `LowCardinality`, e.g. `has(map('a'::Dynamic, ...), toLowCardinality('b'))`. [#107956](https://github.com/ClickHouse/ClickHouse/pull/107956) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` ("Block structure mismatch ... between `ConvertingTransform` and `RemovingReplicatedColumnsTransform`") when inserting into a materialized view whose `TO` target table declares a column with a wider `Enum` than the view's `SELECT` produces. The valid `Enum` widening is now applied. [#107648](https://github.com/ClickHouse/ClickHouse/pull/107648) ([Groene AI](https://github.com/groeneai)). * Fix `NUMBER_OF_COLUMNS_DOESNT_MATCH` error when querying a `Distributed` table (or using parallel replicas) that has several `ALIAS` columns expanding to the same expression and referencing them together with `ORDER BY`/`GROUP BY`/`HAVING`. [#107913](https://github.com/ClickHouse/ClickHouse/pull/107913) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fixed undefined behavior (null pointer passed to `memcpy`) in the `detectCharset` and `detectLanguageUnknown` functions when the input string is larger than 32768 bytes and no character set can be detected. [#108250](https://github.com/ClickHouse/ClickHouse/pull/108250) ([Groene AI](https://github.com/groeneai)). * Fix `signed integer overflow` (undefined behavior) in `dateDiff` with `hour` and `minute` units on extreme `DateTime64` values close to the `Int64` range limits. [#108229](https://github.com/ClickHouse/ClickHouse/pull/108229) ([Groene AI](https://github.com/groeneai)). * Fix `Too many marks` for a text index on an empty merged part. [#106867](https://github.com/ClickHouse/ClickHouse/pull/106867) ([Azat Khuzhin](https://github.com/azat)). * Fix a `LOGICAL_ERROR` ("Unexpected return type from if") when reading a column under `apply_mutations_on_fly = 1` after an `ALTER UPDATE col = ... WHERE ` with a non-constant or false condition followed by `ALTER MODIFY COLUMN col `. [#108128](https://github.com/ClickHouse/ClickHouse/pull/108128) ([Groene AI](https://github.com/groeneai)). * Fix a server abort (`Logical error` in `IColumn::insertFrom`) when casting an `Array(Dynamic)` or `Array(Variant)` to `QBit` with `accurateCastOrNull`, e.g. `accurateCastOrNull(CAST(range(114), 'Array(Dynamic)'), 'QBit(Float32, 114)')`. [#108288](https://github.com/ClickHouse/ClickHouse/pull/108288) ([Groene AI](https://github.com/groeneai)). * Fixed a syntax error when an alias follows a subquery in `DESCRIBE TABLE (...) AS ...`. [#100205](https://github.com/ClickHouse/ClickHouse/pull/100205) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * `getClientHTTPHeader` is now correctly treated as non-deterministic, so its result is no longer incorrectly reused by the query result cache. [#108029](https://github.com/ClickHouse/ClickHouse/pull/108029) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Build/Testing/Packaging Improvement * Build `delta-kernel-rs` with the `default-engine-native-tls` feature so that its own HTTP client (`reqwest`) can reuse the OpenSSL that ClickHouse already links in. This is only a partial `native-tls` enablement: `object_store` still forces `reqwest/rustls-tls-native-roots`, so `reqwest` ends up with both the `native-tls` and `rustls` backends. `delta-kernel-rs` remains disabled under MSan for now because `ring` is still compiled in via `object_store` (both transitively through `rustls` and directly for AWS request HMAC signing), and its hand-written assembly does not generate the symbols MSan requires (tracked upstream at [arrow-rs-object-store#585](https://github.com/apache/arrow-rs-object-store/pull/585)). [#96856](https://github.com/ClickHouse/ClickHouse/pull/96856) ([Austin Bonander](https://github.com/abonander)). * Add PGO (Profile-Guided Optimization) and BOLT (Binary Optimization and Layout Tool) post-link optimization for the `clickhouse` binary. Profiles are collected from CI workloads and applied as a best-effort step during release builds — both stages fall back to non-optimized output if a profile is stale or incompatible. As of today, there are no benefits from PGO. [#100938](https://github.com/ClickHouse/ClickHouse/pull/100938) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `--storage` mode to `clickhouse keeper-bench` that benchmarks `KeeperStorage` in-process (no network, no raft, no state machine), using the same `setup`/`generator` config sections as the network mode. [#103081](https://github.com/ClickHouse/ClickHouse/pull/103081) ([Michael Kolupaev](https://github.com/al13n321)). * Fix build with `ENABLE_AWS_S3=OFF`. [#105390](https://github.com/ClickHouse/ClickHouse/pull/105390) ([Yue Ni](https://github.com/niyue)). * Update `mongo-cxx-driver` to r4.3.0. [#105522](https://github.com/ClickHouse/ClickHouse/pull/105522) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix build against RapidJSON. [#105674](https://github.com/ClickHouse/ClickHouse/pull/105674) ([Ilya Golshtein](https://github.com/ilejn)). * Speed up the build by removing transitive includes from widely-used base/ headers and stripping unused code from the vendored `Poco/Logger.h`. [#105702](https://github.com/ClickHouse/ClickHouse/pull/105702) ([Raúl Marín](https://github.com/Algunenano)). * Updated `libxml2` from 2.15.1 to 2.15.3. [#105985](https://github.com/ClickHouse/ClickHouse/pull/105985) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `expat` 2.8.1 inside Poco. [#105988](https://github.com/ClickHouse/ClickHouse/pull/105988) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Bump `thrift` to `v0.23.0`. [#105993](https://github.com/ClickHouse/ClickHouse/pull/105993) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `postgres` tag `REL_18_4`. [#105994](https://github.com/ClickHouse/ClickHouse/pull/105994) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Upgrade `krb5` to 1.22.2. [#106076](https://github.com/ClickHouse/ClickHouse/pull/106076) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update bundled `curl` to 8.20.0. [#106077](https://github.com/ClickHouse/ClickHouse/pull/106077) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Bumped `mariadb-connector-c` to 3.1.29. All the ClickHouse-relevant patches preserved. [#106287](https://github.com/ClickHouse/ClickHouse/pull/106287) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Bump distroless base image digests for libssl3t64 CVE fixes. [#106360](https://github.com/ClickHouse/ClickHouse/pull/106360) ([Rahul Nair](https://github.com/motsc)). * Use `wasmtime` v45.0.1. [#106836](https://github.com/ClickHouse/ClickHouse/pull/106836) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Bump `openssl` to 3.5.7. [#106844](https://github.com/ClickHouse/ClickHouse/pull/106844) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Distroless Docker build now also updates the `:X.Y-distroless` and `:X.Y.Z-distroless` floating tags, not just the full-version tag. [#106932](https://github.com/ClickHouse/ClickHouse/pull/106932) ([Rahul Nair](https://github.com/motsc)). * Disable unused `curl` features, such as NTLM auth, cookies, alt-svc, HSTS, DNS-over-HTTPS, netrc, MIME, AWS SigV4. [#107226](https://github.com/ClickHouse/ClickHouse/pull/107226) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Updated `NuRaft` with a fix for a snapshot-install livelock that occurred when snapshot IO ran on a background thread. The `nuraft_use_bg_thread_for_snapshot_io` Keeper setting remains disabled by default; CI now randomizes it to cover both modes. [#107338](https://github.com/ClickHouse/ClickHouse/pull/107338) ([Antonio Andelic](https://github.com/antonio2368)). * Fix building `abseil` when the ClickHouse checkout path contains a substring matching a header extension. [#107349](https://github.com/ClickHouse/ClickHouse/pull/107349) ([zhiqiang](https://github.com/zhiqiang-hhhh)). * Determine the L2 cache size from `CPUID` on x86_64 instead of the glibc-specific `sysconf(_SC_LEVEL2_CACHE_SIZE)`, which is not available under musl libc. [#107469](https://github.com/ClickHouse/ClickHouse/pull/107469) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Increase the stack size for musl builds to 8 MiB to support deeply-nested queries. [#107470](https://github.com/ClickHouse/ClickHouse/pull/107470) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Derive the set of C library symbols localized in Rust static libraries from the actual reference libraries instead of a hand-maintained allowlist, so all compiler-rt builtins and platform libm functions are covered. [#107571](https://github.com/ClickHouse/ClickHouse/pull/107571) ([Raúl Marín](https://github.com/Algunenano)). * Switch the distroless server and keeper images to `gcr.io/distroless/base-nossl-debian13` so they only ship the libraries ClickHouse actually links against and stop pulling in ones we statically link ourselves. [#107837](https://github.com/ClickHouse/ClickHouse/pull/107837) ([Rahul Nair](https://github.com/motsc)). * Re-enable 50 stateless tests on macOS (`arm_darwin`) that were stale-skipped, and use `fsync` instead of `F_FULLFSYNC` for directory syncing on macOS. [#107887](https://github.com/ClickHouse/ClickHouse/pull/107887) ([Raúl Marín](https://github.com/Algunenano)). ### ClickHouse release 26.5, 2026-05-21. [Presentation](https://presentations.clickhouse.com/2026-release-26.5/), [Video](https://www.youtube.com/watch?v=P1IDAvsi7p8) #### Backward Incompatible Change * Changed defaults of `date_time_input_format` and `cast_string_to_date_time_mode` from `basic` to `best_effort`. Queries that previously failed to parse non-basic datetime strings (e.g. `2024 April 4`, `Apr 15, 2020 10:30:00`) may now succeed by default. To keep the old strict parsing behavior, set these settings to `basic` (or use `compatibility`). [#89334](https://github.com/ClickHouse/ClickHouse/pull/89334) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Tuple element name `null` is now forbidden because it conflicts with the subcolumn name used for Nullable null maps, causing ambiguous subcolumn resolution. [#98377](https://github.com/ClickHouse/ClickHouse/pull/98377) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add a setting `dynamic_disk_allow_from_env`, `dynamic_disk_allow_from_zk`, `dynamic_disk_allow_include` to disallow usage of `from_env`, `from_zk`, `include` in dynamic disks. Backward Incompatible Change because prohibits by default the behaviour which used to be allowed by default. [#99138](https://github.com/ClickHouse/ClickHouse/pull/99138) ([Kseniia Sumarokova](https://github.com/kssenii)). * You can no longer use the obsolete Arrow-based Parquet reader and writer. The native implementation will be used instead. [#100949](https://github.com/ClickHouse/ClickHouse/pull/100949) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `SHOW CREATE TABLE t` now prefers the temporary table when both a permanent and a temporary table named `t` exist and no database is specified, matching the existing behavior of `DESCRIBE TABLE`. Additionally, `DESCRIBE TEMPORARY TABLE` syntax is now supported. [#100966](https://github.com/ClickHouse/ClickHouse/pull/100966) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduced default `http_max_fields` from 1,000,000 to 1,000 and `http_max_field_name_size` from 128 KB to 4 KB to limit pre-authentication memory usage by HTTP connections. Added `http_max_request_header_size` and `http_headers_read_timeout` settings. Users who rely on the previous higher limits can restore them via settings. [#103285](https://github.com/ClickHouse/ClickHouse/pull/103285) ([Sema Checherinda](https://github.com/CheSema)). * Added a `histograms` nested column to `system.metric_log` that snapshots every registered histogram metric per row, with a new `system_metric_log_show_zero_values_in_histograms` setting to control zero-value emission. Deprecates the `system.histogram_metric_log` table. [#103770](https://github.com/ClickHouse/ClickHouse/pull/103770) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * `CAST` to `DateTime` or `DateTime64` without an explicit time zone now preserves the time zone of its source argument (when the source is a `DateTime`/`DateTime64` with an explicit time zone), matching the behavior of the `toDateTime`/`toDateTime64` functions. Closes [#55072](https://github.com/ClickHouse/ClickHouse/issues/55072). [#104433](https://github.com/ClickHouse/ClickHouse/pull/104433) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Removed the `kql` table function. Use `SET dialect = 'kusto'` to run queries in the KQL dialect. [#105101](https://github.com/ClickHouse/ClickHouse/pull/105101) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The window functions `RANK` and `DENSE_RANK` now reject arguments and throw `NUMBER_OF_ARGUMENTS_DOESNT_MATCH`, in line with the SQL standard. Previously, queries such as `RANK(x) OVER (ORDER BY id)` were silently accepted with the argument ignored. To restore the previous lenient behavior, set `allow_rank_dense_rank_arguments = 1`. Closes [#49526](https://github.com/ClickHouse/ClickHouse/issues/49526). [#104324](https://github.com/ClickHouse/ClickHouse/pull/104324) ([Groene AI](https://github.com/groeneai)). #### New Feature * Added a new setting `max_bytes_ratio_before_external_join`, mirroring `max_bytes_ratio_before_external_group_by` and `max_bytes_ratio_before_external_sort`. It expresses the spill-to-disk threshold for hash joins as a fraction of available memory; combined with the absolute `max_bytes_before_external_join`, the smaller resulting threshold applies. [#103862](https://github.com/ClickHouse/ClickHouse/pull/103862) ([Alexey Milovidov](https://github.com/alexey-milovidov)). The `new max_bytes_ratio_before_external_join` setting is now enabled by default at 0.5, mirroring `max_bytes_ratio_before_external_group_by` and `max_bytes_ratio_before_external_sort`. Hash joins automatically spill to grace hash join once the right-side data exceeds half of the available system memory (when memory limits are configured). [#104285](https://github.com/ClickHouse/ClickHouse/pull/104285) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add table function `filesystem`. It allows to represent directory structure as a table, to query files' metadata and contents with SQL. Originally [#42039](https://github.com/ClickHouse/ClickHouse/pull/42039) by @perst20. See [#42039](https://github.com/ClickHouse/ClickHouse/issues/42039). See [#50208](https://github.com/ClickHouse/ClickHouse/issues/50208). [#53610](https://github.com/ClickHouse/ClickHouse/pull/53610) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow passing bare function names to higher-order functions like `arrayMap`, `arrayFilter`, etc. For example, `arrayMap(negate, [1, 2, 3])` is now equivalent to `arrayMap(x -> negate(x), [1, 2, 3])`. [#101033](https://github.com/ClickHouse/ClickHouse/pull/101033) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add setting `send_table_structure_on_insert_with_inline_data` and `--inline-insert-data` client option to allow the server to parse inline INSERT data itself over the native protocol, avoiding the round-trip to receive table structure and improving performance for many small inserts. [#101034](https://github.com/ClickHouse/ClickHouse/pull/101034) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `tokenizeQuery` and `highlightQuery` functions for SQL query tokenization and syntax highlighting. `tokenizeQuery` returns lexer tokens with byte offsets and token types; `highlightQuery` returns parser-based syntax highlighting ranges with highlight categories (keyword, identifier, function, number, string, etc.). [#101054](https://github.com/ClickHouse/ClickHouse/pull/101054) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `url_base` setting to resolve relative URLs in the `url` table function and `URL` table engine, following RFC 3986 semantics. [#101113](https://github.com/ClickHouse/ClickHouse/pull/101113) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support negative values in the `LIMIT BY` clause to select rows from the *end* of each group instead of the beginning. For example, `LIMIT -2 BY id` returns the last two rows per `id`. Negative offsets (`LIMIT -1 OFFSET -1 BY id`) and mixed signs (`LIMIT -2 OFFSET 1 BY id`) are supported as well. [#103222](https://github.com/ClickHouse/ClickHouse/pull/103222) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Support `json_value` function to output with `tuple` and `array` to improve performance of multiple json query. [#78362](https://github.com/ClickHouse/ClickHouse/pull/78362) ([kevinyhzou](https://github.com/KevinyhZou)). Support multi-path `Tuple`/`Array` JSONPath argument in `JSON_VALUE`, `JSON_EXISTS`, and `JSON_QUERY`, by @KevinyhZou. [#101102](https://github.com/ClickHouse/ClickHouse/pull/101102) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A new `kafka_autodetect_client_rack` parameter is introduced. If set, discover the Availability Zone via cloud facilities and propagate it as the `client.rack` parameter of `librdkafka` to avoid cross-zone communications. [#81323](https://github.com/ClickHouse/ClickHouse/pull/81323) ([Ilya Golshtein](https://github.com/ilejn)). * Add `Read` event type to `system.blob_storage_log` for tracking object storage read operations, controlled by new setting `enable_blob_storage_log_for_read_operations`. [#96867](https://github.com/ClickHouse/ClickHouse/pull/96867) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Users can now see ZooKeeper watches issued by `clickhouse-server` using the new `system.zookeeper_watches` table. [#99277](https://github.com/ClickHouse/ClickHouse/pull/99277) ([Den Kalantaevskii](https://github.com/Diskein)). * Added `Shards` profile event that counts the number of shards involved in distributed queries, summed across all tables. [#99470](https://github.com/ClickHouse/ClickHouse/pull/99470) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `WASM` UDFs can be declared `DETERMINISTIC` and become a subject of constant folding. [#100005](https://github.com/ClickHouse/ClickHouse/pull/100005) ([Vasily Chekalkin](https://github.com/bacek)). * Add `parallel_replicas_prefer_local_replica` setting: when disabled, parallel replicas are selected purely by the load balancing algorithm, allowing even `max_parallel_replicas = 1` queries to be directed to another host. [#100139](https://github.com/ClickHouse/ClickHouse/pull/100139) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `{disk,storage,http}_connections_rcvbuf` and `{disk,storage,http}_connections_sndbuf` server settings to control TCP socket buffer sizes on outgoing HTTP connections, allowing operators to override kernel autotuning and cap per-connection memory usage. [#100478](https://github.com/ClickHouse/ClickHouse/pull/100478) ([Sema Checherinda](https://github.com/CheSema)). * Support `CREATE OR REPLACE MATERIALIZED VIEW` with the same atomic swap semantics as `CREATE OR REPLACE TABLE`. Works with inner tables, `TO` tables, `POPULATE`, `REFRESH`, and `ON CLUSTER`. [#100539](https://github.com/ClickHouse/ClickHouse/pull/100539) ([DQ](https://github.com/il9ue)). * Add `s3_read_request_duration_microseconds` and `s3_read_request_bytes` histogram metrics to observe S3 GET request connection lifetime and bytes consumed, visible in `system.histogram_metrics` and the Prometheus endpoint. [#102058](https://github.com/ClickHouse/ClickHouse/pull/102058) ([Sema Checherinda](https://github.com/CheSema)). * Add `Paimon`, `PaimonS3`, `PaimonAzure`, `PaimonHDFS`, and `PaimonLocal` table engines with incremental read support backed by Keeper snapshot progress tracking. Incremental mode returns only new rows since the last committed snapshot. Targeted snapshot delta reads are available via `paimon_target_snapshot_id`, and per-query snapshot caps via `max_consume_snapshots`. Background metadata refresh is configurable with `paimon_metadata_refresh_interval_sec`. Gated by `allow_experimental_paimon_storage_engine`. [#102343](https://github.com/ClickHouse/ClickHouse/pull/102343) ([XiaoBinMu](https://github.com/Binnn-MX)). * Added a new Kafka table setting `kafka_map_virtual_columns_on_write`. When enabled, columns named `_key`, `_timestamp`, `_headers.name` and `_headers.value` in the Kafka table schema are produced as the corresponding Kafka message key, timestamp and headers on `INSERT`, and are excluded from the message payload. [#103243](https://github.com/ClickHouse/ClickHouse/pull/103243) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `SYSTEM PAUSE VIEW [db.]name` and `SYSTEM PAUSE VIEWS` queries for refreshable materialized views. Unlike `SYSTEM STOP VIEW`, `SYSTEM PAUSE VIEW` does not interrupt the currently running refresh — the in-flight refresh is allowed to complete and only subsequent refreshes are prevented. Undone by `SYSTEM START VIEW` or `SYSTEM START VIEWS`, which now uniformly clear both the stopped and paused states. [#103252](https://github.com/ClickHouse/ClickHouse/pull/103252) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Added function `regexpPosition` (with PostgreSQL-compatible aliases `regexpInstr` and `regexp_instr`) that returns the byte position of the N-th regex match in a string. Supports start offset, return-after-match mode, regex flags, and capture-group selection. [#104172](https://github.com/ClickHouse/ClickHouse/pull/104172) ([Abhinav Agarwal](https://github.com/aagarwal-gtr)). * New functions `isPrime` and `isProbablePrime` for primality checks. `isPrime` returns an exact result for unsigned integers up to `UInt64`. `isProbablePrime` also supports `UInt128` and `UInt256`; for those wider types, `0` means definitely composite and `1` means probably prime. The optional second argument of `isProbablePrime`, `rounds`, controls the confidence (capped at `256`); the default of `25` rounds bounds the false-positive rate for a random composite below `10^-15`, and `isProbablePrime` is cancellable. [#104234](https://github.com/ClickHouse/ClickHouse/pull/104234) ([Nihal Z. Miaji](https://github.com/nihalzp)). [#104639](https://github.com/ClickHouse/ClickHouse/pull/104639) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#104806](https://github.com/ClickHouse/ClickHouse/pull/104806) ([Nihal Z. Miaji](https://github.com/nihalzp)). * `clear` and `/clear` now clear the terminal in the clickhouse command-line tools instead of running as a mistaken query. [#104318](https://github.com/ClickHouse/ClickHouse/pull/104318) ([Tyler Hannan](https://github.com/tylerhannan)). * Allow the `file` table function to accept an `Array(String)` of paths in `SELECT` queries. [#104442](https://github.com/ClickHouse/ClickHouse/pull/104442) ([Yue](https://github.com/niyue)). * Add `deterministic` and `higher_order` columns to `system.functions` table. [#104479](https://github.com/ClickHouse/ClickHouse/pull/104479) ([Pedro Ferreira](https://github.com/PedroTadim)). * Add an optional encoding variant parameter to `bech32Encode` (`bech32` / `bech32m`) and a raw decode mode to `bech32Decode` for non-SegWit address encoding (e.g. Cosmos SDK, Injective, Osmosis). [#98986](https://github.com/ClickHouse/ClickHouse/pull/98986) ([Yash ](https://github.com/Onyx2406)). * You can now write data in `AvroConfluent` format, which was previously input-only. This enables producing Confluent Schema Registry-framed Avro messages directly from ClickHouse, for example when writing to Kafka. The schema is automatically registered with the registry. Use the new `output_format_avro_confluent_subject` setting to specify the subject name. [#101935](https://github.com/ClickHouse/ClickHouse/pull/101935) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Added the `prettyPrintJSON` function to format a JSON string into a human-readable form. Useful for dashboards or reports where prettifying previously required an extra step on the client side. Closes [#62523](https://github.com/ClickHouse/ClickHouse/issues/62523). [#102594](https://github.com/ClickHouse/ClickHouse/pull/102594) ([Dmitry Prokofyev](https://github.com/prokofyevDmitry)). * Added `STRING_AGG` as a case-insensitive alias of `groupConcat` for PostgreSQL / SQL-standard compatibility. [#105125](https://github.com/ClickHouse/ClickHouse/pull/105125) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Individual subquery results can now be cached independently using `SETTINGS use_query_cache = true` on specific subqueries, without caching the entire outer query. A new setting `query_cache_for_subqueries = true` enables bulk propagation of `use_query_cache` into all subqueries. Note: `use_query_cache` on the outer query no longer auto-propagates to subqueries. [#99804](https://github.com/ClickHouse/ClickHouse/pull/99804) ([Vincent Voyer](https://github.com/vvo)). * Added lexer-based syntax highlighting to the query editor in the Web UI (`play.html`), modelled after `clickhouse-client` colors. [#105105](https://github.com/ClickHouse/ClickHouse/pull/105105) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Experimental Feature * Added an experimental web terminal interface at `/webterminal` that provides an interactive `clickhouse-client` session in the browser over WebSocket. Disabled by default; enable with the `allow_experimental_webterminal` server setting. See it [here](https://play.clickhouse.com/webterminal?user=play). [#100277](https://github.com/ClickHouse/ClickHouse/pull/100277) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#105191](https://github.com/ClickHouse/ClickHouse/pull/105191) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#105059](https://github.com/ClickHouse/ClickHouse/pull/105059) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `Kafka2` engine (experimental, with Keeper-based offset storage) now supports direct `SELECT` queries and the `kafka_commit_on_select` setting. [#100276](https://github.com/ClickHouse/ClickHouse/pull/100276) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `WASM` UDFs can now coerce more numeric types: smaller integers to wider integers (e.g. `Int8` to `UInt64`), and any integer to floating point (e.g. `Int32` to `Float32`). [#100435](https://github.com/ClickHouse/ClickHouse/pull/100435) ([Vasily Chekalkin](https://github.com/bacek)). * Support `LIKE` function in `DELETE FROM system.webassembly_modules` query. [#104397](https://github.com/ClickHouse/ClickHouse/pull/104397) ([Vladimir Cherkasov](https://github.com/vdimir)). * Support geo types for `Iceberg`. [#103113](https://github.com/ClickHouse/ClickHouse/pull/103113) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Add prepared statements functionality to ArrowFlight SQL server. [#103047](https://github.com/ClickHouse/ClickHouse/pull/103047) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Improved KQL parser (for the Kusto language support) robustness by preserving parser depth and backtracks counters across KQL parser stages, so parser limits are tracked consistently for complex KQL queries. [#103528](https://github.com/ClickHouse/ClickHouse/pull/103528) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). #### Performance Improvement * Reuse the Parquet footer metadata cache when reading local Parquet files via the `file` table function or `File` table engine. Previously the cache was only consulted for object-storage backends. [#104260](https://github.com/ClickHouse/ClickHouse/pull/104260) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Push `ORDER BY ... LIMIT n` through `LEFT`/`RIGHT` joins when the sort key only references columns from the side preserved by the join, restricting how many rows the preserved-side input must produce before joining. Controlled by the new setting `query_plan_top_k_through_join` (default enabled). [#104268](https://github.com/ClickHouse/ClickHouse/pull/104268) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enable `use_top_k_dynamic_filtering` and `use_skip_indexes_for_top_k` settings by default to improve performance of `ORDER BY ... LIMIT N` queries. [#99537](https://github.com/ClickHouse/ClickHouse/pull/99537) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Restrict `use_top_k_dynamic_filtering` to fixed-length sort columns by default, avoiding regressions on `ORDER BY LIMIT N` queries where the per-row threshold comparison cost exceeds the I/O savings. The previous behavior is available via the new `use_top_k_dynamic_filtering_for_variable_length_types` setting. [#104216](https://github.com/ClickHouse/ClickHouse/pull/104216) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use the oversize-arena feature of `jemalloc` to reduce page faults. [#103958](https://github.com/ClickHouse/ClickHouse/pull/103958) ([Nikita Taranov](https://github.com/nickitat)). * Limit simultaneously active streams in `UNION ALL` to reduce peak memory usage. [#100176](https://github.com/ClickHouse/ClickHouse/pull/100176) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Slightly optimize the userspace page cache. [#100300](https://github.com/ClickHouse/ClickHouse/pull/100300) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Now the userspace page cache is always better than the OS page cache. * Cold reads of object storage through the userspace page cache (`use_page_cache_for_object_storage = 1`) are now significantly faster, because consecutive cache misses are coalesced into a single HTTP request instead of one request per `page_cache_block_size` block. [#104230](https://github.com/ClickHouse/ClickHouse/pull/104230) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up the index analysis involving type casting and function application. Closes [#55653](https://github.com/ClickHouse/ClickHouse/issues/55653). [#100366](https://github.com/ClickHouse/ClickHouse/pull/100366) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Speed up huge queries on `Merge` tables over a huge number of underlying tables. Closes [#32465](https://github.com/ClickHouse/ClickHouse/issues/32465). [#100369](https://github.com/ClickHouse/ClickHouse/pull/100369) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove vtable from setting field types, reducing `Settings` copy size by ~28x and improving cache locality via typed-array layout for all settings types. [#102269](https://github.com/ClickHouse/ClickHouse/pull/102269) ([Raúl Marín](https://github.com/Algunenano)). * Add `max_threads_min_free_memory_per_thread` and `max_insert_threads_min_free_memory_per_thread` settings to automatically reduce query parallelism when the server is low on free memory. [#100383](https://github.com/ClickHouse/ClickHouse/pull/100383) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reduce memory usage on low-memory systems (< 4 GiB). [#100389](https://github.com/ClickHouse/ClickHouse/pull/100389) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `OptimizeTrivialGroupByLimitPass`. For trivial `SELECT ... FROM t GROUP BY k LIMIT n` queries (no `HAVING`, `ORDER BY`, or window functions), the analyzer now sets `max_rows_to_group_by = n + offset` with `group_by_overflow_mode = 'any'`, so aggregation stops once `n` distinct keys have been produced instead of grouping the entire input. Controlled by the new setting `optimize_trivial_group_by_limit_query` (enabled by default). [#104473](https://github.com/ClickHouse/ClickHouse/pull/104473) ([Amos Bird](https://github.com/amosbird)) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Granule-level implicit min-max index for the `_part_offset` and `_block_number` virtual columns, including for projections. Enables fast pruning of granules based on virtual column predicates. [#103952](https://github.com/ClickHouse/ClickHouse/pull/103952) ([Mikhail Artemenko](https://github.com/Michicosun)). [#104746](https://github.com/ClickHouse/ClickHouse/pull/104746) ([Mikhail Artemenko](https://github.com/Michicosun)). [#105137](https://github.com/ClickHouse/ClickHouse/pull/105137) ([Mikhail Artemenko](https://github.com/Michicosun)). * For the `prealloc_serialized` family of aggregation methods, precompute per-row hashes during the batch-serialization pass and use them to (a) skip rehashing in `emplaceKey`/`findKey` and (b) software-prefetch the next row's bucket. Speeds up multi-key string/serialized aggregation by hiding hash-table cache-miss latency. [#104475](https://github.com/ClickHouse/ClickHouse/pull/104475) ([Amos Bird](https://github.com/amosbird)) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Query condition cache for `Iceberg` tables. [#102115](https://github.com/ClickHouse/ClickHouse/pull/102115) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Optimize `cramersV`, `cramersVBiasCorrected`, `theilsU`, and `contingency` when used with window functions. Closes [#83521](https://github.com/ClickHouse/ClickHouse/issues/83521). [#93384](https://github.com/ClickHouse/ClickHouse/pull/93384) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Added a query optimization that rewrites `tupleElement(dictGet('dict', ('a', 'b', 'c'), key), N)` into `dictGet('dict', 'a', key)`, avoiding fetching unnecessary dictionary attributes. Controlled by the `optimize_dictget_tuple_element` setting (enabled by default). [#100186](https://github.com/ClickHouse/ClickHouse/pull/100186) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enable buffering for sorting steps on the initiator for distributed queries with sorting. [#100661](https://github.com/ClickHouse/ClickHouse/pull/100661) ([Nikita Taranov](https://github.com/nickitat)). * Performance improvement for `partial_merge` JOIN. [#100945](https://github.com/ClickHouse/ClickHouse/pull/100945) ([Artem Zuikov](https://github.com/4ertus2)). * Slightly reduced memory over-allocation in `partial_merge` JOIN. [#100963](https://github.com/ClickHouse/ClickHouse/pull/100963) ([Artem Zuikov](https://github.com/4ertus2)). * Optimize allocations/deallocations by caching sampling settings instead of traversing the entire memory-tracker hierarchy. [#101267](https://github.com/ClickHouse/ClickHouse/pull/101267) ([Azat Khuzhin](https://github.com/azat)). * Reduce memory allocation overhead during S3 multipart uploads by preallocating internal tracking containers. [#101799](https://github.com/ClickHouse/ClickHouse/pull/101799) ([Gagan Dhakrey](https://github.com/gagandhakrey)). * Add software prefetch in hash join probe phase to reduce memory access latency for large hash tables, controlled by setting `enable_software_prefetch_in_join`. [#102444](https://github.com/ClickHouse/ClickHouse/pull/102444) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Optimize `MemoryTracker` layout, improving its performance by ~25%. [#103464](https://github.com/ClickHouse/ClickHouse/pull/103464) ([Azat Khuzhin](https://github.com/azat)). * Rewrite `coalesce(a, b, ...) const` and `ifNull(a, b) const` predicates before index analysis so per-column primary key and skip indexes on each argument can prune granules. Controlled by the new setting `allow_key_condition_coalesce_rewrite` (on by default). [#103468](https://github.com/ClickHouse/ClickHouse/pull/103468) ([Manuel](https://github.com/raimannma)). * Significantly improved query performance when reading Iceberg catalogs with large JSON metadata files by optimizing escaped slash processing. [#103998](https://github.com/ClickHouse/ClickHouse/pull/103998) ([Mohaidoss](https://github.com/mohaidoss)). * Avoid index uncompressed cache overhead when the cache is disabled (server setting `index_uncompressed_cache_size = 0`, which is the default). [#104063](https://github.com/ClickHouse/ClickHouse/pull/104063) ([Michael Kolupaev](https://github.com/al13n321)). * Fix `ORDER BY ... LIMIT` queries with small limit reading excessive granules when combined with a non-selective `WHERE` filter. Previously, any filter above an in-order read disabled the per-mark-range task split, so the pipeline could not cancel between granules and read the whole part for each stream. [#104112](https://github.com/ClickHouse/ClickHouse/pull/104112) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed missing hash table sizes cache reuse after reordering JOIN sides. [#104131](https://github.com/ClickHouse/ClickHouse/pull/104131) ([Nikita Taranov](https://github.com/nickitat)). * When `fsync_after_insert` is enabled, fsync of part files is now performed in parallel using the IO thread pool, speeding up insert finalization on wide tables. The full ClickBench `hits` load is about 22% faster. [#104137](https://github.com/ClickHouse/ClickHouse/pull/104137) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Vectorise `find_first_symbols`, `find_first_not_symbols`, `find_last_symbols_or_null`, `find_last_not_symbols_or_null`, and `splitInto` on AArch64 using NEON. Previously these helpers had a SIMD path only on x86 (SSE2 / SSE4.2) and fell through to a scalar loop on ARM. Speeds up TSV parsing by ~2x, URL functions and `splitByChar` by ~1.3x on the ClickBench `hits` dataset; very dense JSON parsing (sub-16-byte field cadence) regresses slightly in line with the existing SSE2 trade-off. [#104228](https://github.com/ClickHouse/ClickHouse/pull/104228) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reading data through a trivial `ALIAS` column (e.g. `some_alias['key']` where `some_alias ALIAS m`) now goes through the same per-subcolumn read path as referencing the underlying column directly, restoring large I/O savings for `Map`, `Array`, `Tuple` and `Nullable` aliased columns. [#104245](https://github.com/ClickHouse/ClickHouse/pull/104245) ([Raúl Marín](https://github.com/Algunenano)). * Reduced lock contention on the asynchronous remote-FS read path by making the per-query `AsyncReadCounters` lockless. [#104374](https://github.com/ClickHouse/ClickHouse/pull/104374) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Slightly better query plan for projections in-order scan. [#103723](https://github.com/ClickHouse/ClickHouse/pull/103723) ([Mikhail Artemenko](https://github.com/Michicosun)). * Extend `optimizeUseNormalProjections` to also handle a specific case where nothing was filtered but the projection's sorting key can remove the sorting step from the query plan. [#104680](https://github.com/ClickHouse/ClickHouse/pull/104680) ([Mikhail Artemenko](https://github.com/Michicosun)). * Allow `json.path[]` syntax sugar and explicit type hints to work correctly on typed `JSON` paths. [#99179](https://github.com/ClickHouse/ClickHouse/pull/99179) ([Pavel Kruglov](https://github.com/Avogar)). * Optimize `json.path[N].nested.path` (which expands to `tupleElement(tupleElement(json.path[N], 'nested'), 'path')`) into `json.path[].nested.path[N]`, which reads much less data. [#99802](https://github.com/ClickHouse/ClickHouse/pull/99802) ([Pavel Kruglov](https://github.com/Avogar)). * Optimize `SHOW TABLES` access checks in `system.parts` by hoisting database-level grant checks out of the per-table loop. [#100860](https://github.com/ClickHouse/ClickHouse/pull/100860) ([Shaohua Wang](https://github.com/tiandiwonder)). * Remove a redundant per-table `SHOW TABLES` access check in the `SELECT name` / `SELECT database, name` fast path of `system.tables`. [#100881](https://github.com/ClickHouse/ClickHouse/pull/100881) ([Shaohua Wang](https://github.com/tiandiwonder)). * Enable JIT compilation on macOS. [#100947](https://github.com/ClickHouse/ClickHouse/pull/100947) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Skip deferring the row policy after `FINAL` when it depends only on sorting-key columns and is deterministic, and skip the corresponding `PREWHERE` deferral that the row policy was forcing in that case. [#102884](https://github.com/ClickHouse/ClickHouse/pull/102884) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Added support for `hasAny` and `hasAll` as filter predicates in text indexes. [#103266](https://github.com/ClickHouse/ClickHouse/pull/103266) ([Anton Popov](https://github.com/CurtizJ)). * Route long-lived MergeTree heap state (per-part and per-table metadata) to a dedicated jemalloc arena, exposed via `jemalloc.mergetree_arena.*` async metrics. Reduces default-arena fragmentation in steady-state and keeps per-part survivors from pinning otherwise-decayable pages. [#104136](https://github.com/ClickHouse/ClickHouse/pull/104136) ([Raúl Marín](https://github.com/Algunenano)). * Improve ZooKeeper client timeout handling under heavy load when many requests are pipelined on a single session. The ZooKeeper client now uses a progress-based timeout: as long as any data is received from the server within `session_timeout_ms`, the wait is extended. A hard cap of `3 * session_timeout_ms` per request still bounds caller latency. Closes [#100466](https://github.com/ClickHouse/ClickHouse/issues/100466). [#104351](https://github.com/ClickHouse/ClickHouse/pull/104351) ([Antonio Andelic](https://github.com/antonio2368)). * Move per-event trace flags into a separate array and allocate them lazily, reducing the overhead of `ProfileEvents` tracing. [#105030](https://github.com/ClickHouse/ClickHouse/pull/105030) ([Azat Khuzhin](https://github.com/azat)). * Add `compareTrackAt` for `ColumnDecimal`, improving comparison performance for `Decimal` columns. [#105110](https://github.com/ClickHouse/ClickHouse/pull/105110) ([Artem Zuikov](https://github.com/4ertus2)). * Added a SIMD implementation of `MD5` (`AVX2` / `AVX512`) that hashes multiple inputs in parallel, improving throughput. [#105161](https://github.com/ClickHouse/ClickHouse/pull/105161) ([Joanna Hulboj](https://github.com/jh0x)). #### Improvement * Improve the help output for all ClickHouse applications: `--help` now consistently returns exit code `0`, prints to `stdout`, and a top-level `clickhouse --help` lists all subcommands. Adds a `--no-sudo` option for `clickhouse start` / `restart` (useful in Docker), and a `clickhouse help` subcommand. Continuation of [#58244](https://github.com/ClickHouse/ClickHouse/issues/58244) from @qoega. [#98148](https://github.com/ClickHouse/ClickHouse/pull/98148) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The default of `input_format_column_name_matching_mode` is changed from `match_case` to `auto`. Input formats that match input column names against the table schema (`JSONEachRow`, `CSVWithNames`, `JSONColumns`, `BSONEachRow`, `RowBinaryWithNames`, etc.) now first try a case-sensitive match and fall back to case-insensitive matching when the case-sensitive match misses. The previous strict behavior is preserved under `compatibility`. [#104320](https://github.com/ClickHouse/ClickHouse/pull/104320) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `STDDEV` as a case-insensitive alias of `stddevSamp` for PostgreSQL/SQL-standard compatibility. [#105120](https://github.com/ClickHouse/ClickHouse/pull/105120) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `array_to_string` as a case-insensitive alias of `arrayStringConcat` for PostgreSQL compatibility. [#105121](https://github.com/ClickHouse/ClickHouse/pull/105121) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `unnest` as a case-insensitive alias of `arrayJoin` for PostgreSQL compatibility (function-call form). [#105124](https://github.com/ClickHouse/ClickHouse/pull/105124) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The progress bar in `clickhouse-client` now shows temporary data on disk usage (e.g. for external sort, aggregation, or JOIN) next to RAM, including a per-host breakdown for distributed queries. [#105190](https://github.com/ClickHouse/ClickHouse/pull/105190) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `system.predicate_statistics_log`, a new sampled log of predicate filter selectivity and MergeTree index-granule pruning per query. Disabled by default; enable via the `predicate_statistics_sample_rate` server setting. Needed for automatic index and projection recommendations. [#98727](https://github.com/ClickHouse/ClickHouse/pull/98727) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Allow skipping local shard with missing table when `skip_unavailable_shards` is enabled. [#100141](https://github.com/ClickHouse/ClickHouse/pull/100141) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add startup warnings (visible in `system.warnings`) when a Linux mdraid array is being resynchronized or is in a degraded state, as both can affect disk I/O performance or indicate disk failures. [#100941](https://github.com/ClickHouse/ClickHouse/pull/100941) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * WASM UDFs now appear in `system.functions` with correct `origin` (`WasmUserDefined`), `syntax`, `arguments`, and `returned_value` columns populated from their ClickHouse type metadata. Previously they were either missing or listed with the wrong origin as duplicates. [#101053](https://github.com/ClickHouse/ClickHouse/pull/101053) ([Vasily Chekalkin](https://github.com/bacek)). * The `generate_series` table function now supports negative step values for generating descending sequences, e.g. `SELECT * FROM generate_series(99, 0, -1)`. [#101056](https://github.com/ClickHouse/ClickHouse/pull/101056) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `Context has expired` exceptions raised by some functions (`dotProduct`, `formatRow`, `structureToCapnProtoSchema`/`structureToProtobufSchema`, user-defined functions) when used in deferred execution paths such as `DEFAULT`/`MATERIALIZED` expressions and table engine settings, by ensuring those functions hold a strong context where required. [#101109](https://github.com/ClickHouse/ClickHouse/pull/101109) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix spurious `Cache limits violated` logical errors on priorities created with zero limits (such as the per-query filesystem cache priority used when `enable_filesystem_query_cache_limit` is on). [#101428](https://github.com/ClickHouse/ClickHouse/pull/101428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Auto-detect the region for `s3express` endpoints. [#101520](https://github.com/ClickHouse/ClickHouse/pull/101520) ([Pradeep Chhetri](https://github.com/chhetripradeep)). * Support field ids for data files in `Iceberg` writes. Closes [#102322](https://github.com/ClickHouse/ClickHouse/issues/102322). [#102362](https://github.com/ClickHouse/ClickHouse/pull/102362) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The left panel in the web UI (`play.html`) now scrolls independently, stays visible when the page is scrolled, and clicking a table scrolls to the query area. [#102498](https://github.com/ClickHouse/ClickHouse/pull/102498) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow opening the Play UI in a new tab via Ctrl/Cmd/Shift+click or middle-click on the ClickHouse logo in the left panel. [#102501](https://github.com/ClickHouse/ClickHouse/pull/102501) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * New metrics to track memory used by projection primary keys and projection index granularity across all tables. [#102587](https://github.com/ClickHouse/ClickHouse/pull/102587) ([Narasimha Pakeer](https://github.com/npakeer)). * Apply `max_network_bandwidth_for_user` and `max_network_bandwidth_for_all_users` to remote filesystem reads/writes. [#103080](https://github.com/ClickHouse/ClickHouse/pull/103080) ([Azat Khuzhin](https://github.com/azat)). * Improve the resilience of backups when Refreshable Materialized Views are constantly refreshed by making decisions based on the snapshotted tables' state rather than a global one. [#103384](https://github.com/ClickHouse/ClickHouse/pull/103384) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * `SELECT * FROM system.databases` now always lists data lake catalog databases regardless of the `show_data_lake_catalogs_in_system_tables` setting. Previously they were hidden by default, which was inconsistent with `SHOW DATABASES` that always showed them. [#103444](https://github.com/ClickHouse/ClickHouse/pull/103444) ([Alsu Giliazova](https://github.com/alsugiliazova)). * Align the default of the server setting `concurrent_threads_soft_limit_ratio_to_cores` in code with the shipped `config.xml` value (`2`), so the default `max_min_fair` concurrent threads scheduler caps query processing threads at 2x cores out of the box even when the shipped `config.xml` is not used. [#103446](https://github.com/ClickHouse/ClickHouse/pull/103446) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved sanitizer robustness in parser and string-function edge cases by handling empty input in `getURLScheme`, avoiding null-pointer arithmetic in `Lexer::nextToken` `max_query_size` checks, and short-circuiting UTF-8 subsequence evaluation on empty haystacks before decoding. [#103489](https://github.com/ClickHouse/ClickHouse/pull/103489) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Improved correctness of monotonicity inference for `divide(0, x)` and `intDiv(0, x)`: previously the functions were unconditionally reported as monotonic, but `0 / x` is non-monotonic on any range that includes 0 because `0 / 0` is undefined (`NaN`/`Inf` for `divide`, division-by-zero exception for `intDiv`). The false claim caused `KeyCondition::applyMonotonicFunctionsChainToRange` to produce ranges with `left > right`, which tripped a `LOGICAL_ERROR` `Invalid binary search result in MergeTreeSetIndex` in debug builds when an `IN` / `NOT IN` expression on the primary key wrapped the key in `divide(0, key)` or `intDiv(0, key)`. Release builds were unaffected — the assertion is gated by `#ifndef NDEBUG`. [#103621](https://github.com/ClickHouse/ClickHouse/pull/103621) ([Groene AI](https://github.com/groeneai)). * Replace the `HTTPConnectionPool*TCP{Rcv,Snd}BufBytes_{p50,p75,p90,p95}` async metrics with a bucket-based histogram `http_pool_tcp_buf_bytes` (labels `group`, `direction`) in `system.histogram_metrics`. Per-group total async metrics are preserved. [#103704](https://github.com/ClickHouse/ClickHouse/pull/103704) ([Sema Checherinda](https://github.com/CheSema)). * The server settings `{disk,storage,http}_connections_{rcvbuf,sndbuf}` are now reported as changeable without restart in `system.server_settings`. Their values can be updated via `SYSTEM RELOAD CONFIG`; the runtime application path was already in place since [#100478](https://github.com/ClickHouse/ClickHouse/issues/100478). [#103772](https://github.com/ClickHouse/ClickHouse/pull/103772) ([Sema Checherinda](https://github.com/CheSema)). * Add support for resolved symbols (`Array(String)`) in `flameGraph`. [#103816](https://github.com/ClickHouse/ClickHouse/pull/103816) ([Azat Khuzhin](https://github.com/azat)). * CLI client can now specify `false` in its config in environments where terminal colors clash with the parentheses (similarly, `clickhouse format` now has `--no_rainbow_parentheses`). [#103851](https://github.com/ClickHouse/ClickHouse/pull/103851) ([Larry Snizek](https://github.com/larry-cdn77)). * Fix `MultiVolnitsky` UTF-8 case-insensitive search by rolling back partial `putNGram` inserts on failure and switching failed needles to fallback searchers, avoiding inconsistent state. [#103864](https://github.com/ClickHouse/ClickHouse/pull/103864) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Implement `sched_getcpu` via `rseq` TLS in glibc-compatibility musl. [#104016](https://github.com/ClickHouse/ClickHouse/pull/104016) ([Azat Khuzhin](https://github.com/azat)). * Replicated refreshable materialized view in APPEND mode will no longer refresh twice if connection to zookeeper was briefly lost during refresh. [#104051](https://github.com/ClickHouse/ClickHouse/pull/104051) ([Michael Kolupaev](https://github.com/al13n321)). * Allow Distributed tables without an explicit column list to validate sharding keys against the inferred remote table structure. [#104111](https://github.com/ClickHouse/ClickHouse/pull/104111) ([Yue](https://github.com/niyue)). * Fix REST catalog with `Azure` `abfss` path. [#104120](https://github.com/ClickHouse/ClickHouse/pull/104120) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Honor `role_arn` and `role_session_name` auth settings in the Keeper S3 snapshot client, allowing snapshot uploads to use STS `AssumeRole`-based authentication. [#104140](https://github.com/ClickHouse/ClickHouse/pull/104140) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a memory leak when running queries with `MATERIALIZED` CTEs. [#104153](https://github.com/ClickHouse/ClickHouse/pull/104153) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `lowerUTF8` / `upperUTF8` exception when processing large non-ASCII data sets (e.g. during text-index builds) where the accumulated output buffer would exceed 2 GiB; reject excessively long single rows with a clear error. [#104229](https://github.com/ClickHouse/ClickHouse/pull/104229) ([Shaohua Wang](https://github.com/tiandiwonder)). * Harden MergeTree column-statistics loading against transient I/O failures to prevent permanently disabling statistics-based optimizations for affected parts. [#104372](https://github.com/ClickHouse/ClickHouse/pull/104372) ([zoomxi](https://github.com/zoomxi)). * `INSERT INTO ... SELECT FROM input(...)` no longer requires the `CREATE TEMPORARY TABLE` grant. [#104470](https://github.com/ClickHouse/ClickHouse/pull/104470) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added a new MergeTree setting `concurrent_part_removal_threshold_for_remote_disk` (default `16`) that is used instead of `concurrent_part_removal_threshold` when at least one part being removed is stored on a remote disk. The previous threshold of `100` could make `DROP TABLE` and other part-removal operations on object storage backends stall for tens of seconds because removals were performed serially even though each one is a separate network round-trip. The new setting causes the parallel removal path to be entered much sooner on remote storage, while leaving local-disk behavior unchanged. [#104676](https://github.com/ClickHouse/ClickHouse/pull/104676) ([Groene AI](https://github.com/groeneai)). * New setting `defer_partition_pruning_after_final` (default `1`) makes the 26.3-introduced behavior of skipping partition pruning under `FINAL` opt-out. Set it to `0` to restore pre-26.3 partition pruning — substantially faster on event-log workloads where same-PK rows cannot span partitions. `compatibility = '26.2'` flips it to `0` automatically. [#104705](https://github.com/ClickHouse/ClickHouse/pull/104705) ([Nikita Fomichev](https://github.com/fm4v)). * Fix `intExp2` returning wrong results for out-of-range inputs (shifts `>= 64` under JIT, and integers with magnitude exceeding `INT_MAX` on both code paths). [#105054](https://github.com/ClickHouse/ClickHouse/pull/105054) ([Raúl Marín](https://github.com/Algunenano)). * Quote shell arguments built from CLI options and parsed input in `clickhouse install` and `clickhouse git-import` so paths, user/group names, and commit hashes containing whitespace or shell metacharacters are handled correctly. [#105232](https://github.com/ClickHouse/ClickHouse/pull/105232) ([Raúl Marín](https://github.com/Algunenano)). * The `Alias` table engine is now non-experimental and available without the `allow_experimental_alias_table_engine` setting. [#103488](https://github.com/ClickHouse/ClickHouse/pull/103488) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added `--queries-format` option to `clickhouse-benchmark` to choose between the default tab-escaped one-query-per-line input (`tsv`) and parsing standard input as a script of multiple queries separated by `;` (`script`). [#99972](https://github.com/ClickHouse/ClickHouse/pull/99972) ([Aleksandr Musorin](https://github.com/AVMusorin)). * The element-wise tuple operators `tuplePlus`, `tupleMinus`, `tupleMultiply`, `tupleDivide`, `tupleModulo`, `tupleIntDiv`, and `tupleIntDivOrZero` are now variadic and accept two or more tuples of the same size, applied element-wise as a left-fold. They previously accepted exactly two arguments only. [#104659](https://github.com/ClickHouse/ClickHouse/pull/104659) ([Aruj Bansal](https://github.com/arujbansal)). * Added a new built-in web UI at `/processors-profile` that visualizes the pipeline of any past `SELECT` query as a heatmap, sourced from `system.query_log` and `system.processors_profile_log`. Each processor is colored by its `elapsed_us` and shows per-processor stats (rows, bytes, wait times) on hover. [#104614](https://github.com/ClickHouse/ClickHouse/pull/104614) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Setting `query_plan_use_logical_join_step` (and its alias `query_plan_use_new_logical_join_step`) is now obsolete and has no effect; the logical join step is always used. [#104017](https://github.com/ClickHouse/ClickHouse/pull/104017) ([Vladimir Cherkasov](https://github.com/vdimir)). * The `user` input on `play.html` and the other built-in web pages (`dashboard.html`, `jemalloc.html`, `merges.html`, `binary.html`, `webterminal.html`) is no longer pre-filled with the literal text `default`. It now shows a `user` placeholder hint and is empty by default. When left empty, the page does not send a `user=` URL parameter (and the WebTerminal auth JSON omits the `user` field), so the server applies its usual fallback to the `default` user — and HTTP credentials supplied via other channels (`X-ClickHouse-User`, HTTP Basic, per-handler `` configuration) are no longer overridden. [#105254](https://github.com/ClickHouse/ClickHouse/pull/105254) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `ORC` reader: decouple offset-based read (`readBigAt`) from `use_prefetch`, so EC-encoded data on `HDFS` can be read correctly even when `use_prefetch = false` (e.g. by Gluten). [#103348](https://github.com/ClickHouse/ClickHouse/pull/103348) ([zhanglistar](https://github.com/zhanglistar)). * Files downloaded from `play.html` with the `CSVWithNames` format now use the `.csv` file extension. [#103737](https://github.com/ClickHouse/ClickHouse/pull/103737) ([JackFielding](https://github.com/JackFielding)). * More input validation for the BigLake catalog. [#105117](https://github.com/ClickHouse/ClickHouse/pull/105117) ([Konstantin Vedernikov](https://github.com/scanhex12)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fix inconsistent formatting by remembering if an expression was parenthesized. [#92340](https://github.com/ClickHouse/ClickHouse/pull/92340) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix accurate comparison between Decimal and Float types. [#94293](https://github.com/ClickHouse/ClickHouse/pull/94293) ([zoomxi](https://github.com/zoomxi)). * Allow positional arguments in distributed queries. [#94359](https://github.com/ClickHouse/ClickHouse/pull/94359) ([simonmichal](https://github.com/simonmichal)). * Fix assertion failure in `DatabaseCatalog::updateDependencies` when dropping and recreating a materialized view with the same name; also fix view dependencies being silently lost during `RENAME TABLE` or `EXCHANGE TABLES` when referential dependencies were empty. [#98779](https://github.com/ClickHouse/ClickHouse/pull/98779) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a server abort (assertion failure in `DatabaseCatalog::getTableImpl`) that occurred when creating a table via `ON CLUSTER` with a UUID identical to an existing database's UUID. [#98861](https://github.com/ClickHouse/ClickHouse/pull/98861) ([xiaohuanlin](https://github.com/xiaohuanlin)). * Fixed usage of skip indexes on data read (with enabled setting `use_skip_indexes_on_data_read`) and existing patch parts created by lightweight updates. [#99543](https://github.com/ClickHouse/ClickHouse/pull/99543) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Not-ready Set" exception when a filter with `IN (subquery)` is moved to PREWHERE by the query optimizer. [#100375](https://github.com/ClickHouse/ClickHouse/pull/100375) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Cannot find column" error when combining `ADD COLUMN` with `RENAME COLUMN` in a single `ALTER TABLE` statement. [#100387](https://github.com/ClickHouse/ClickHouse/pull/100387) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix views with mixed `UNION` and `INTERSECT`/`EXCEPT` operators returning wrong results after `DETACH`/`ATTACH` or server restart. [#100390](https://github.com/ClickHouse/ClickHouse/pull/100390) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Trying to execute PLACEHOLDER action" exception that could occur during stress testing with the AST fuzzer when a correlated subquery appeared in an IN clause. [#100398](https://github.com/ClickHouse/ClickHouse/pull/100398) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception when inserting into an Iceberg table with a `Date` column partitioned by `year`, `month`, or `day` transforms. [#100404](https://github.com/ClickHouse/ClickHouse/pull/100404) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix skip index being used with incompatible data after `ALTER TABLE MODIFY COLUMN` changes the column type, which could cause server crashes in sanitizer builds or incorrect query results. [#100526](https://github.com/ClickHouse/ClickHouse/pull/100526) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix credential leak in `query_log` for `paimonCluster`, `paimonS3Cluster`, `paimonAzureCluster`, and `deltaLakeS3` table functions. [#100529](https://github.com/ClickHouse/ClickHouse/pull/100529) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fix `DROP TABLE` on Kafka engine tables potentially hanging indefinitely due to a deadlock in `rd_kafka_consumer_close`. [#100604](https://github.com/ClickHouse/ClickHouse/pull/100604) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Replace the hardcoded `source_table_engines` list with runtime lookup via `StorageFactory` and `DatabaseFactory`. Add `source_access_type` to `DatabaseFactory::EngineFeatures` so that `CREATE DATABASE` with source engines (PostgreSQL, MySQL, S3, etc.) requires the same source grants as `CREATE TABLE`. Fixes `GRANT TABLE ENGINE ON *` failing with `table_engines_require_grant=false`. Closes [#71544](https://github.com/ClickHouse/ClickHouse/issues/71544). [#100746](https://github.com/ClickHouse/ClickHouse/pull/100746) ([pufit](https://github.com/pufit)). * Fix server crash when reading from a table whose `ALIAS` column contained a correlated subquery nested inside a function call (e.g. `ALIAS toString(intDivOrZero(x, (SELECT ...)))`). `CREATE TABLE` and `ALTER TABLE` now reject any subquery at any depth in `DEFAULT` / `ALIAS` / `MATERIALIZED` column expressions with `THERE_IS_NO_DEFAULT_VALUE`. Previously, the shallow validation accepted nested subqueries, so the correlated case crashed at read time and the nested non-correlated case appeared to work but could lead to unexpected behavior; both are now rejected at DDL time. [#100753](https://github.com/ClickHouse/ClickHouse/pull/100753) ([Groene AI](https://github.com/groeneai)). * Fixed a server crash (`Logical error: Bad cast from type DB::FunctionNode to DB::ColumnNode`) that could occur when using queries with correlated subqueries on tables that have `CONSTRAINT ... ASSUME` definitions, when `optimize_substitute_columns` and `convert_query_to_cnf` settings are enabled. [#100756](https://github.com/ClickHouse/ClickHouse/pull/100756) ([Groene AI](https://github.com/groeneai)). * Fix an exception that could occur when inserting a row into a JSON column where the row introduced a new dynamic field alongside a typed field with an incompatible value, and the column was subsequently used as a GROUP BY key. [#100758](https://github.com/ClickHouse/ClickHouse/pull/100758) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix workload IO scheduling being silently bypassed for S3/object-storage writes that go through the `DiskObjectStorageTransaction` path (used by `s3_with_keeper` disks and explicitly when `use_fake_transaction=false`). Previously, INSERT writes into MergeTree tables on such disks ignored `CREATE RESOURCE` / `CREATE WORKLOAD` throttling because the resource link was never injected into `WriteSettings` for the transactional write path. [#100777](https://github.com/ClickHouse/ClickHouse/pull/100777) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fix shared schema-cache collision between `Protobuf` and `ProtobufList`: reading a message first as `FORMAT Protobuf` and then as `FORMAT ProtobufList` could fail because `ProtobufList` forced the cached schema into its envelope form. `ProtobufList` now falls back to the cached message type when no envelope schema is available. [#100849](https://github.com/ClickHouse/ClickHouse/pull/100849) ([Callum Cooper](https://github.com/callumc34)). * Fix heap-buffer-overflow in Rust CXX bridge due to `std::exception` ABI mismatch. [#100931](https://github.com/ClickHouse/ClickHouse/pull/100931) ([Azat Khuzhin](https://github.com/azat)). * `FunctionVariantAdaptor` now throws `ILLEGAL_TYPE_OF_ARGUMENT` when all `Variant` alternatives are incompatible with a function, instead of silently returning `Nullable(Nothing)`. Previously, a `WHERE function(variant_col)` predicate where no alternative type was compatible would return 0 rows with no error, while the equivalent `SELECT` context already threw correctly. [#100939](https://github.com/ClickHouse/ClickHouse/pull/100939) ([Vasily Chekalkin](https://github.com/bacek)). * `TRUNCATE ALL TABLES` no longer fails when the database contains views. [#100943](https://github.com/ClickHouse/ClickHouse/pull/100943) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `optimize_rewrite_array_exists_to_has` optimization to correctly handle type-incompatible cases (e.g. Date vs String) and re-enable it by default. [#100944](https://github.com/ClickHouse/ClickHouse/pull/100944) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix S3 settings priority so that `storage_configuration` disk settings override global `` section, and user/profile/query-level settings override both. [#100975](https://github.com/ClickHouse/ClickHouse/pull/100975) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix sparse serialization losing the sign of negative zero (-0.0) for BFloat16, Float32, and Float64 columns. [#100983](https://github.com/ClickHouse/ClickHouse/pull/100983) ([Takumi Hara](https://github.com/takumihara)). * Fixed two bugs in the WKT geometry parser used when reading WKT-encoded GeoParquet and Arrow files: `MULTILINESTRING` geometries were incorrectly parsed as `Polygon` (causing an exception for typed columns and silent data corruption for mixed `Geometry` columns), and a malformed WKT string with no type keyword triggered undefined behaviour instead of a clean error. [#100997](https://github.com/ClickHouse/ClickHouse/pull/100997) ([Vasily Chekalkin](https://github.com/bacek)). * Fix `grouping`/`GROUPING SETS` queries on Distributed tables with a single shard that failed with "Method executeImpl is not supported for 'grouping' function". [#101030](https://github.com/ClickHouse/ClickHouse/pull/101030) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` "Column identifier is already registered" in the planner that could occur when the same table expression was processed multiple times. [#101048](https://github.com/ClickHouse/ClickHouse/pull/101048) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR exception "Column identifier is already registered" when `additional_result_filter` setting is used with UNION or EXCEPT queries. [#101051](https://github.com/ClickHouse/ClickHouse/pull/101051) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed server crash (Logical error: Bad cast from ColumnVector to ColumnNullable) when using `* APPLY` with aggregate functions and `group_by_use_nulls=1` with GROUPING SETS, ROLLUP, or CUBE. [#101062](https://github.com/ClickHouse/ClickHouse/pull/101062) ([Groene AI](https://github.com/groeneai)). * Fix infinite loop when `input_format_csv_skip_first_lines` or `input_format_tsv_skip_first_lines` exceeds the number of lines in the file. [#101111](https://github.com/ClickHouse/ClickHouse/pull/101111) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix out-of-bounds access in lazy materialization optimization that could cause an exception in debug builds. [#101144](https://github.com/ClickHouse/ClickHouse/pull/101144) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix type inference in recursive CTEs: column types are now iteratively widened via `getLeastSupertype` across the non-recursive and recursive sides of the `UNION ALL` until convergence, preventing integer overflow in expressions like `x + 1`. [#101155](https://github.com/ClickHouse/ClickHouse/pull/101155) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Clamp MergeTree compress block size settings (max_compress_block_size, min_compress_block_size, marks_compress_block_size, primary_key_compress_block_size) to 256 MiB to prevent server crash when extreme values are set via CREATE TABLE SETTINGS. [#101159](https://github.com/ClickHouse/ClickHouse/pull/101159) ([Groene AI](https://github.com/groeneai)). * Fix a LOGICAL_ERROR crash in `QueryAnalyzer::resolve()` when a ROW POLICY uses a scalar subquery in its `USING` clause (e.g. `USING (SELECT 1)`). Closes [#100695](https://github.com/ClickHouse/ClickHouse/issues/100695). [#101263](https://github.com/ClickHouse/ClickHouse/pull/101263) ([Groene AI](https://github.com/groeneai)). * Fix undefined behavior in `MergeTreeDataPartWriterCompact::cancel` when a stream allocation fails. [#101292](https://github.com/ClickHouse/ClickHouse/pull/101292) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes a case where timezone was not included during the assignment in the ALTER statement. Closes [#101328](https://github.com/ClickHouse/ClickHouse/issues/101328). Related https://github.com/ClickHouse/ClickHouse/pull/100647. [#101403](https://github.com/ClickHouse/ClickHouse/pull/101403) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix server crash (LOGICAL_ERROR) for INSERT SELECT queries with ORDER BY ALL when the SELECT pipeline produces multiple streams. [#101443](https://github.com/ClickHouse/ClickHouse/pull/101443) ([Groene AI](https://github.com/groeneai)). * Fix incorrect query results caused by join reordering pushing an INNER JOIN filter (referencing only the preserved side of an outer join) into the outer join’s ON clause. [#101504](https://github.com/ClickHouse/ClickHouse/pull/101504) ([Vladimir Cherkasov](https://github.com/vdimir)). * Skip applying lazy materialization for plans with arrayJoin, which could lead to limit not being respected. Close [#101608](https://github.com/ClickHouse/ClickHouse/issues/101608). [#101644](https://github.com/ClickHouse/ClickHouse/pull/101644) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix incorrect JOIN query results when `mergeFilterIntoJoinCondition` optimization silently dropped a WHERE equality condition with mismatched types. [#101652](https://github.com/ClickHouse/ClickHouse/pull/101652) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Fix the `error` test hint not working for syntax/parsing errors — previously `-- { error SYNTAX_ERROR }` on a malformed query would fail with "Expected server error" instead of matching the client-side parse error. [#101675](https://github.com/ClickHouse/ClickHouse/pull/101675) ([Groene AI](https://github.com/groeneai)). * Fix `INTO OUTFILE ... TRUNCATE` not actually using atomic rename — the write went directly to the original file instead of a temp file, so on query failure the original content was destroyed. Now the data is written to a temp file and renamed only on success. [#101884](https://github.com/ClickHouse/ClickHouse/pull/101884) ([Pablo Marcos](https://github.com/pamarcos)). * Fixed an exception `Unsupported DeltaLake type: varchar(n)` when reading Delta Lake tables whose schema contains varchar(n) or char(n) column types. These types are now mapped to String, consistent with how the Delta Lake protocol stores them as plain byte arrays in Parquet. [#101973](https://github.com/ClickHouse/ClickHouse/pull/101973) ([Flavio Malavazi](https://github.com/flaviomalavazi)). * Fix silent data loss when reading tar archives from S3 with `schema_inference_mode=union` and heterogeneous Parquet schemas — the Parquet metadata cache incorrectly reused the first file's metadata for all subsequent files in the archive. [#101990](https://github.com/ClickHouse/ClickHouse/pull/101990) ([Ahaan Limaye](https://github.com/ahaanlimaye)). * Fix server crash (LOGICAL_ERROR) when executing ALTER TABLE UPDATE/DELETE on Iceberg tables when no prior SELECT or INSERT was done on the table in the same server lifetime. [#102113](https://github.com/ClickHouse/ClickHouse/pull/102113) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix possible crash when an ALTER query executed on tables from in-memory database (e.g. temporary tables). [#102360](https://github.com/ClickHouse/ClickHouse/pull/102360) ([Den Kalantaevskii](https://github.com/Diskein)). * Fixed `formatQuery` producing duplicate grants like `GRANT FILE ON *.*, FILE ON *.* TO x` instead of `GRANT FILE ON *.* TO x` when formatting backward-compatible READ/WRITE grants on the same source. [#102411](https://github.com/ClickHouse/ClickHouse/pull/102411) ([Groene AI](https://github.com/groeneai)). * Fix lost parallelism for aggregation after read-in-order queries when `max_streams_to_max_threads_ratio` is greater than 1. [#102467](https://github.com/ClickHouse/ClickHouse/pull/102467) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a crash (`LOGICAL_ERROR: Unknown virtual column`) when selecting subcolumns (like `.null` for Nullable, `.size0` for Array, tuple elements, or map keys/values) from a Buffer table engine. [#102470](https://github.com/ClickHouse/ClickHouse/pull/102470) ([Groene AI](https://github.com/groeneai)). * Fix exception "Cannot fold actions for projection" in join reorder optimization when a LEFT/RIGHT JOIN with `join_use_nulls` is combined with other joins involving more than two tables. [#102516](https://github.com/ClickHouse/ClickHouse/pull/102516) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed silent overflow in scalar pointwise operations on numericIndexedVector (e.g. `numericIndexedVectorPointwiseAdd`) when the scalar is out of range. Such inputs now raise `INCORRECT_DATA` instead of returning corrupted values. [#102546](https://github.com/ClickHouse/ClickHouse/pull/102546) ([FriendLey](https://github.com/FriendLey)). * Fix inconsistent AST formatting for INSERT with SAMPLE and query-level OFFSET. Closes [#102523](https://github.com/ClickHouse/ClickHouse/issues/102523). [#102547](https://github.com/ClickHouse/ClickHouse/pull/102547) ([zoomxi](https://github.com/zoomxi)). * Fix `LOGICAL_ERROR` exception in `groupConcat` when deserializing a malformed aggregate function state. [#102558](https://github.com/ClickHouse/ClickHouse/pull/102558) ([Christoph Wurm](https://github.com/cwurm)). * Now we do not accept garbage at the end of Time64 CSV values with `input_format_csv_use_default_on_bad_values=0`. Closes [#102490](https://github.com/ClickHouse/ClickHouse/issues/102490). [#102596](https://github.com/ClickHouse/ClickHouse/pull/102596) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix UNKNOWN_ELEMENT_OF_ENUM exception when inserting default value to JSON column with Enum typed paths. Closes [#102359](https://github.com/ClickHouse/ClickHouse/issues/102359). [#102687](https://github.com/ClickHouse/ClickHouse/pull/102687) ([Pavel Kruglov](https://github.com/Avogar)). * Fix columns_substreams.txt corruption during column rename in some cases. Closes [#102259](https://github.com/ClickHouse/ClickHouse/issues/102259). [#102689](https://github.com/ClickHouse/ClickHouse/pull/102689) ([Pavel Kruglov](https://github.com/Avogar)). * Fix inserting into Time data type during JSON parsing. Closes [#102016](https://github.com/ClickHouse/ClickHouse/issues/102016). [#102690](https://github.com/ClickHouse/ClickHouse/pull/102690) ([Pavel Kruglov](https://github.com/Avogar)). * Fix hex encoding of content sample in cached schema filenames. Closes [#101904](https://github.com/ClickHouse/ClickHouse/issues/101904). [#102703](https://github.com/ClickHouse/ClickHouse/pull/102703) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed sort order violation (crash in debug, silent data corruption in release) during `SummingMergeTree` merge when the `ORDER BY` key is a hash expression over a `Float32` column containing signaling NaN values. [#102791](https://github.com/ClickHouse/ClickHouse/pull/102791) ([Groene AI](https://github.com/groeneai)). * Make `DETACH DATABASE ... SYNC` and other operations that call `waitDetachedTableNotInUse` cancellable via `KILL QUERY` and responsive to server shutdown, preventing indefinite hangs when a concurrent query holds a table reference. [#102804](https://github.com/ClickHouse/ClickHouse/pull/102804) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed possible incorrect result of `ANY RIGHT JOIN`. [#102893](https://github.com/ClickHouse/ClickHouse/pull/102893) ([Nikita Taranov](https://github.com/nickitat)). * Fix S3 requests failing with `ios_base::clear: unspecified iostream_category error` instead of being retried, caused by Poco `BufferedStreamBuf::flushBuffer` not handling short writes from the socket layer. [#102894](https://github.com/ClickHouse/ClickHouse/pull/102894) ([Sema Checherinda](https://github.com/CheSema)). * Fix `input('auto')` table function failing via HTTP interface in INSERT SELECT queries. [#102902](https://github.com/ClickHouse/ClickHouse/pull/102902) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Disable trivial LIMIT optimization with row policies/additional_table_filters (to allow parallel index analysis). [#102921](https://github.com/ClickHouse/ClickHouse/pull/102921) ([Azat Khuzhin](https://github.com/azat)). * Hide secret key in `HMAC` SQL function. Fix [#102927](https://github.com/ClickHouse/ClickHouse/issues/102927). [#102997](https://github.com/ClickHouse/ClickHouse/pull/102997) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix `MULTIPLE_EXPRESSIONS_FOR_ALIAS` exception raised on distributed queries that reference the same `quantile` call multiple times (e.g. in `SELECT`, `HAVING`, and `ORDER BY`) with `optimize_syntax_fuse_functions` enabled. [#103014](https://github.com/ClickHouse/ClickHouse/pull/103014) ([tanner-bruce](https://github.com/tanner-bruce)). * Fixes a case found by CI where a not-ready set was passed when the filter depends on the left-side column. Closes [#102966](https://github.com/ClickHouse/ClickHouse/issues/102966). [#103029](https://github.com/ClickHouse/ClickHouse/pull/103029) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix castOrNull to JSON in some cases. Closes [#101818](https://github.com/ClickHouse/ClickHouse/issues/101818). [#103036](https://github.com/ClickHouse/ClickHouse/pull/103036) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `SELECT DISTINCT` silently returning incomplete results when an aggregate projection matched the query and some parts of the table had no projection data (e.g. the projection was added on a table that already held data, and `MATERIALIZE PROJECTION` was not run). Closes [#102951](https://github.com/ClickHouse/ClickHouse/issues/102951). [#103052](https://github.com/ClickHouse/ClickHouse/pull/103052) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `TOO_FEW_ARGUMENTS_FOR_FUNCTION` exception when a `WHERE` predicate such as `AND(OR(A, A), A)` collapses to a single top-level argument during common expression extraction in the analyzer. [#103072](https://github.com/ClickHouse/ClickHouse/pull/103072) ([Peng](https://github.com/fastio)). * Fix a logical error `Function writeSlice expects same column types for GenericArraySlice and GenericArraySink` raised when evaluating `if`/`ifNull` over tuples, maps, or arrays containing a `QBit` element. `ColumnQBit::structureEquals` incorrectly compared the inner tuple of one `QBit` column against the outer wrapper of the other, so structurally identical `QBit` columns were reported as different. [#103084](https://github.com/ClickHouse/ClickHouse/pull/103084) ([Groene AI](https://github.com/groeneai)). * Make `DETACH DATABASE` with `database_atomic_wait_for_drop_and_detach_synchronously` respect `KILL QUERY` instead of hanging indefinitely when a table reference is held. [#103095](https://github.com/ClickHouse/ClickHouse/pull/103095) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix segfault when deleting WASM module using non-identifier predicate (e.g. `WHERE 1=1`). [#103101](https://github.com/ClickHouse/ClickHouse/pull/103101) ([Joe Redfern](https://github.com/JosephRedfern)). * Fix backward compatibility break where old clients fail with `UNEXPECTED_PACKET_FROM_SERVER` when inserting into a newer server via `remote()` or distributed tables, caused by unconditional `sendProgress` at the end of `processInsertQuery`. [#103148](https://github.com/ClickHouse/ClickHouse/pull/103148) ([Sema Checherinda](https://github.com/CheSema)). * IN with non-constant tuple second argument must not CAST tuple elements down to the LHS type, this can overflow. Closes [#103055](https://github.com/ClickHouse/ClickHouse/issues/103055). [#103169](https://github.com/ClickHouse/ClickHouse/pull/103169) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix use-of-uninitialized-value in protocol function. [#103187](https://github.com/ClickHouse/ClickHouse/pull/103187) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `LOGICAL_ERROR` (`Bad cast … to ColumnLowCardinality`) during MergeTree index analysis when a `WHERE` clause compares a `LowCardinality` key column with a constant cast to a type that contains nested `LowCardinality` (for example `Variant(LowCardinality(Date), String)`). [#103211](https://github.com/ClickHouse/ClickHouse/pull/103211) ([Groene AI](https://github.com/groeneai)). * Fix `Logical error: 'index < bucket_count'` in the `timeSeries*ToGrid` aggregate function family (e.g. `timeSeriesResampleToGridWithStaleness`, `timeSeriesChangesToGrid`, `timeSeriesResetsToGrid`, `timeSeriesRateToGrid`) when called with extreme timestamp parameters that would overflow signed 64-bit arithmetic in the bucket count computation. Also cap the total number of grid buckets at 16 million to prevent accidental large-memory allocation from adversarial inputs. [#103223](https://github.com/ClickHouse/ClickHouse/pull/103223) ([Groene AI](https://github.com/groeneai)). * Fix `Logical error: 'Port is already connected'` exception during pipeline expansion in the lazy `FINAL` path for `ReplacingMergeTree`. The bug affected queries with `query_plan_optimize_lazy_final = 1` when `ReadFromMergeTree::initializePipeline` inserted internal transforms (e.g. a `Resize`) that wired the sub-pipeline's processors together, causing `LazyUnorderedReadFromMergeTreeSource::expandPipeline` to try to connect output ports that were already connected. [#103230](https://github.com/ClickHouse/ClickHouse/pull/103230) ([Groene AI](https://github.com/groeneai)). * Fix wrong results from `LIMIT BY` and `DISTINCT` when the input is a `UNION ALL` of sorted subqueries. The query plan optimizer incorrectly treated such a union as globally sorted, causing extra rows to be returned. [#103231](https://github.com/ClickHouse/ClickHouse/pull/103231) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix possible crash during statistics calculation in Map type with lazy replication. Closes [#102390](https://github.com/ClickHouse/ClickHouse/issues/102390). [#103273](https://github.com/ClickHouse/ClickHouse/pull/103273) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a data race for ClickHouse, MySQL, PostgreSQL and XDBC dictionary sources where `clone() const` reads from and `isModified() const` writes to the same `invalidate_query_response` string. [#103277](https://github.com/ClickHouse/ClickHouse/pull/103277) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix incorrect monotonicity detection for `Date32`. Closes [#101265](https://github.com/ClickHouse/ClickHouse/issues/101265). [#103283](https://github.com/ClickHouse/ClickHouse/pull/103283) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Cap pre-auth TCP Hello packet strings to 64 KB and add `handshake_timeout_milliseconds` server setting to limit total handshake time, preventing unauthenticated clients from consuming excessive memory or holding threads indefinitely. [#103284](https://github.com/ClickHouse/ClickHouse/pull/103284) ([Sema Checherinda](https://github.com/CheSema)). * Fix Parquet ColumnIndex stats min_value > max_value for String columns. [#103334](https://github.com/ClickHouse/ClickHouse/pull/103334) ([Saurabh Kumar Ojha](https://github.com/saurabhojha)). * Fix incorrect handling of NULL rows for `Nullable(Tuple(...))` inputs in `flattenTuple` and `tupleToNameValuePairs`. `flattenTuple` now preserves the outer null map so NULL rows stay NULL. `tupleToNameValuePairs` now changes the result's value type to `Nullable(T)` when possible when the input is `Nullable(Tuple(...))`, so NULL rows produce `[('a', NULL), ('b', NULL)]` instead of `[('a', 0), ('b', 0)]`. When the element type cannot be wrapped in `Nullable` (e.g. `Array`), default values are used instead of NULL. Closes [#103312](https://github.com/ClickHouse/ClickHouse/issues/103312). [#103383](https://github.com/ClickHouse/ClickHouse/pull/103383) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Check for malformed flattened Dynamic data in Native format. [#103392](https://github.com/ClickHouse/ClickHouse/pull/103392) ([Pavel Kruglov](https://github.com/Avogar)). * Fix stale RPC causing replication stall after snapshot sync. [#103406](https://github.com/ClickHouse/ClickHouse/pull/103406) ([Seva Potapov](https://github.com/seva-potapov)). * Fix a `LOGICAL_ERROR` "Cannot pop N rows from X" that aborted the server (in debug / sanitizer builds) when reading a malformed BSON document containing a value whose BSON type is incompatible with the target column type inside a `Nullable(T)` or `Array(Nullable(T))` column. The malformed row now produces a clean `ILLEGAL_COLUMN` exception as intended. [#103418](https://github.com/ClickHouse/ClickHouse/pull/103418) ([Groene AI](https://github.com/groeneai)). * Fix a server abort in `arrayFill` and `arrayReverseFill` when applied to an `Array(String)` column whose first row is empty together with a constant-false lambda. The aggregate / sub-array loop underflowed a `size_t` index to `SIZE_MAX`, which then caused an out-of-bounds read inside `ColumnString::doInsertManyFrom`. Same bug family as [#12263](https://github.com/ClickHouse/ClickHouse/issues/12263). [#103424](https://github.com/ClickHouse/ClickHouse/pull/103424) ([Groene AI](https://github.com/groeneai)). * Fixes ACCESS_DENIED / UNKNOWN_TABLE in downstream APPEND RMVs with SQL SECURITY DEFINER when an upstream REPLACE RMV's EXCHANGE flips the target storage identity mid-resolution. [#103427](https://github.com/ClickHouse/ClickHouse/pull/103427) ([Alexander Gololobov](https://github.com/davenger)). * Populate `_time` column from `url` table function. [#103437](https://github.com/ClickHouse/ClickHouse/pull/103437) ([Nikita Taranov](https://github.com/nickitat)). * Fix potential out-of-bounds read in the `ALP` codec decompression path when processing malformed input with an invalid bit-width. [#103457](https://github.com/ClickHouse/ClickHouse/pull/103457) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix a bug where `SYSTEM SYNC FILESYSTEM CACHE '' ON CLUSTER ...` could lose the cache name while formatting the query, causing remote nodes to sync all filesystem caches instead of only the requested cache. [#103469](https://github.com/ClickHouse/ClickHouse/pull/103469) ([Asish Kumar](https://github.com/officialasishkumar)). * Fix processing of URL-encoded path in `deltaLakeAzure`. Closes [#103509](https://github.com/ClickHouse/ClickHouse/issues/103509). [#103525](https://github.com/ClickHouse/ClickHouse/pull/103525) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix heap-use-after-free in multiplying an `AggregateFunction` state by an integer (e.g. `quantilesExactState(...) * N`). The exponentiation-by-squaring loop merged the state with itself, which is undefined when the aggregate function's `merge` reallocates its internal storage. Closes STID 0988-40af. [#103536](https://github.com/ClickHouse/ClickHouse/pull/103536) ([Groene AI](https://github.com/groeneai)). * Fixed a rare `LOGICAL_ERROR` "Part X intersects previous part Y" raised during `ReplicatedMergeTree` table startup when two empty unexpected parts on disk had overlapping but non-containing block ranges. The exception aborted the table-attach thread and prevented the table from coming up. [#103537](https://github.com/ClickHouse/ClickHouse/pull/103537) ([Groene AI](https://github.com/groeneai)). * Fix `Logical error: Incorrect mark rows for part ...` (debug-only assertion) that was triggered by mutations on `MergeTree` tables with non-adaptive index granularity (`index_granularity_bytes = 0`) whose last data mark was incomplete (most reliably reproduced via `DETACH`/`ATTACH` followed by a lightweight `DELETE`). [#103538](https://github.com/ClickHouse/ClickHouse/pull/103538) ([Groene AI](https://github.com/groeneai)). * Fix `clickhouse-local` returning 0 rows silently when reading from `/proc` and `/sys` pseudo-files via the `file()` table function (e.g. `SELECT * FROM file('/proc/cpuinfo', 'RawBLOB')`). [#103548](https://github.com/ClickHouse/ClickHouse/pull/103548) ([Ashrith Bandla](https://github.com/ashrithb)). * Fixed handling of the `max_string_length` argument for the `GenerateRandom` table engine. [#103550](https://github.com/ClickHouse/ClickHouse/pull/103550) ([Alex Kuleshov](https://github.com/0xAX)). * Fix masking nested credentials in logs. [#103552](https://github.com/ClickHouse/ClickHouse/pull/103552) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix SVE detection using SVE instructions when unavailable. [#103568](https://github.com/ClickHouse/ClickHouse/pull/103568) ([Raúl Marín](https://github.com/Algunenano)). * Fix `GenerateRandom` storage args parsing. [#103574](https://github.com/ClickHouse/ClickHouse/pull/103574) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix infinite loop on `WITH FILL` for data starting with ±inf. [#103580](https://github.com/ClickHouse/ClickHouse/pull/103580) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix `JSONExtract` into a `Variant` type silently truncating fractional JSON numbers. Previously, `JSONExtract('{"x": 3.14}', 'x', 'Variant(Int64, Float64)')` returned `Int64=3` and `JSONExtract('{"x": 3.14}', 'x', 'Variant(String, Int64)')` returned `Int64=3`, dropping the fractional part. The fractional value is now preserved losslessly: a `Float64`/`Decimal` member claims it when present, otherwise a `String` member captures the original JSON. `Variant` types containing only integer members (e.g. `Variant(Int64, Int32)`) and direct integer extraction (`JSONExtract(json, 'Int64')`, `JSONExtractInt`, etc.) are unchanged. [#103620](https://github.com/ClickHouse/ClickHouse/pull/103620) ([Groene AI](https://github.com/groeneai)). * Fix `SYSTEM INSTRUMENT REMOVE` without arguments producing `std::bad_optional_access` (error code 1001) instead of `SYNTAX_ERROR` (error code 62). [#103622](https://github.com/ClickHouse/ClickHouse/pull/103622) ([Pablo Marcos](https://github.com/pamarcos)). * Fix logical error / undefined behavior in `windowID` and `tumble` window view when called with a timezone string as the 3rd argument. [#103641](https://github.com/ClickHouse/ClickHouse/pull/103641) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * S3 client logging now treats HTTP 400 responses that include a non-empty x-amz-bucket-region header (wrong SigV4 signing region) as an informational wrong-region case with a clearer log line instead of the generic error-style path. The STS web identity credentials provider is added to the S3 credentials chain only when web identity is configured, which reduces spurious warnings for deployments that do not use it. Closes [#99140](https://github.com/ClickHouse/ClickHouse/issues/99140). [#103673](https://github.com/ClickHouse/ClickHouse/pull/103673) ([MeltonSmith](https://github.com/MeltonSmith)). * Fix `SYSTEM SYNC REPLICA . IF EXISTS` to silently succeed when the database does not exist, matching the existing behaviour for a missing table and the precedent set by `DROP TABLE IF EXISTS`. Previously the query threw `UNKNOWN_DATABASE` despite `IF EXISTS`. Closes [#103629](https://github.com/ClickHouse/ClickHouse/issues/103629). [#103689](https://github.com/ClickHouse/ClickHouse/pull/103689) ([Groene AI](https://github.com/groeneai)). * Fix logical error `Arguments of 'plus' have incorrect data types` thrown by `FunctionBinaryArithmetic::executeImpl2` when a `MergeTree` table had `Array(LowCardinality(...))` in its sort key and a `WHERE` clause used `plus` / `minus` between that column and an `Array` constant of a different element type. `KeyCondition::getMonotonicityForRange` now strips `LowCardinality` recursively before invoking the inner numeric dispatch. [#103701](https://github.com/ClickHouse/ClickHouse/pull/103701) ([Groene AI](https://github.com/groeneai)). * Fix dynamic cache resize race in 26.1+. [#103702](https://github.com/ClickHouse/ClickHouse/pull/103702) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fixed an issue where text and bloom_filter skip indices on `ALIAS` columns whose expression contained a lambda with captured constants (e.g. `arrayMap((k, v) -> concat(k, '=', v), mapKeys(m), mapValues(m)))` were silently ignored. [#103708](https://github.com/ClickHouse/ClickHouse/pull/103708) ([Anton Popov](https://github.com/CurtizJ)). * Fix `position` and `positionCaseInsensitive` with a `start_pos` close to `UINT64_MAX` getting stuck or causing a segmentation fault due to overflowing pointer arithmetic. [#103766](https://github.com/ClickHouse/ClickHouse/pull/103766) ([Raúl Marín](https://github.com/Algunenano)). * Fixes undefined behavior while parsing dateTime with a double fractional part. [#103773](https://github.com/ClickHouse/ClickHouse/pull/103773) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a data-correctness regression in 26.x where JIT-compiled `if` and `multiIf` with a `Decimal` result type and a non-`Decimal` (integer or float) literal in one branch silently returned a value `10^scale` too small. The slow (non-JIT) path was unaffected. Workaround on affected versions: `SET compile_expressions = 0`. [#103809](https://github.com/ClickHouse/ClickHouse/pull/103809) ([Groene AI](https://github.com/groeneai)). * Fixes forwarding the tokenizer from the text index to supported functions. [#103826](https://github.com/ClickHouse/ClickHouse/pull/103826) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix wrong results from `numbers`, `generate_series`, and similar range-honoring sources when the `WHERE` clause uses `IN` or `NOT IN` over a tuple whose elements deduplicate to a single key column (for example, `WHERE tuple(number, number) NOT IN (tuple(1, 2))`). Closes [#103660](https://github.com/ClickHouse/ClickHouse/issues/103660). [#103835](https://github.com/ClickHouse/ClickHouse/pull/103835) ([Groene AI](https://github.com/groeneai)). * Keep the auto-spilling hash join's actual memory peak under `max_bytes_before_external_join`. Previously, statistics-driven preallocation, in-place hash table doubling, and unbounded `GraceHashJoin` in-memory buckets could each push the query past the configured cap and trip `MEMORY_LIMIT_EXCEEDED`. [#103838](https://github.com/ClickHouse/ClickHouse/pull/103838) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a hang in `clickhouse-local` shutdown when system logs (such as `text_log` or `filesystem_cache_log`) are configured. `SystemLogs::flushImpl` was calling `BaseDaemon::instance().flushTextLogs()`, which throws `std::bad_cast` outside `clickhouse-server` and left the saving threads running until `pthread_cond_destroy` blocked the destruction of the system log queues. [#103874](https://github.com/ClickHouse/ClickHouse/pull/103874) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject `MATERIALIZED` CTE queries whose body resolves to different inferred column types in different references with a clear `TYPE_MISMATCH` error instead of crashing the server with a `Bad cast` `LOGICAL_ERROR`. This previously happened when the CTE body referenced identifiers from outer scope (for example an alias from the calling subquery's projection) that were inlined as different constants per reference. [#103879](https://github.com/ClickHouse/ClickHouse/pull/103879) ([Groene AI](https://github.com/groeneai)). * Fix a use-of-uninitialized-value in the JOIN-conversion query plan optimizers (`tryConvertAnyOuterJoinToInnerJoin`, `tryConvertAnyJoinToSemiOrAntiJoin`) when the filter on top of an `ANY` `OUTER` `JOIN` contains a non-deterministic function such as `rand`, `now`, or `rowNumberInAllBlocks`. The optimizer no longer attempts to constant-fold such filters and leaves the JOIN unchanged, which also prevents incorrect conversions to `INNER`/`SEMI`/`ANTI JOIN` that could silently drop rows. [#103880](https://github.com/ClickHouse/ClickHouse/pull/103880) ([Groene AI](https://github.com/groeneai)). * Fix a use-after-free in `StorageKafka2` that could crash the server when a Keeper session is replaced while the consumer is holding ephemeral topic-partition locks. [#103890](https://github.com/ClickHouse/ClickHouse/pull/103890) ([Groene AI](https://github.com/groeneai)). * Fixed a `LOGICAL_ERROR` "Primary key type mismatch" thrown when joining an `EmbeddedRocksDB` table via `JOIN ... USING (key)` against a subquery whose key column has a different type (e.g. `Nullable(UInt64)`, `Int64`, `Decimal`) than the storage's primary key. The planner now declines `DirectKeyValueJoin` on type mismatch and falls back to `HashJoin`, which handles the type conversion. [#103928](https://github.com/ClickHouse/ClickHouse/pull/103928) ([Groene AI](https://github.com/groeneai)). * `SET max_threads = DEFAULT` (and the same for `max_final_threads` and `max_parsing_threads`) no longer loses the auto state. Previously, after resetting one of these settings, `system.settings` would report the resolved core count (e.g. `32`) instead of `'auto(32)'`, and the server would behave as if the value had been explicitly pinned. [#103991](https://github.com/ClickHouse/ClickHouse/pull/103991) ([Groene AI](https://github.com/groeneai)). * Fix `SIZES_OF_ARRAYS_DONT_MATCH` when reading `AggregateFunction(topK(N), String)` columns whose persisted `alpha_size` was grown by a pre-25.12 deserialize+serialize cycle. [#104002](https://github.com/ClickHouse/ClickHouse/pull/104002) ([Raúl Marín](https://github.com/Algunenano)). * Fixed SQL injection vulnerability in PostgreSQL, Cassandra and XDBC dictionary sources: string keys containing single quotes were escaped with `\'` (backslash), which these backends treat as a literal backslash rather than an escape sequence, allowing arbitrary SQL to be injected into dictionary lookup queries. `ExternalQueryBuilder` now emits SQL-standard `''` escaping for those backends; ClickHouse and MySQL dictionary sources continue to use backslash escaping. [#104009](https://github.com/ClickHouse/ClickHouse/pull/104009) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixed possible type mismatch in aggregation when a column type was altered to/from LowCardinality, and min-max projection wasn't rebuilt. [#104013](https://github.com/ClickHouse/ClickHouse/pull/104013) ([Nikita Taranov](https://github.com/nickitat)). * Fix infinite loop of query optimizations caused by not propagating prevention of unused column removal when merging expressions. [#104083](https://github.com/ClickHouse/ClickHouse/pull/104083) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix `LOGICAL_ERROR` in `StreamingStorageRegistry::renameTable` when batch-renaming streaming tables (S3Queue, Kafka, RabbitMQ) by using UUID-based identity tracking instead of name-based. [#104101](https://github.com/ClickHouse/ClickHouse/pull/104101) ([Nikita Taranov](https://github.com/nickitat)). * Fixed wrong results when a CTE constructed with `UNION ALL` contained a `SELECT DISTINCT` branch and the outer query projected only a subset of the CTE's columns. The new analyzer's `RemoveUnusedProjectionColumnsPass` was incorrectly removing the un-referenced column from the inner `DISTINCT` projection, causing rows that should have remained distinct (same value in the projected column but different value in the dropped column) to collapse into one. [#104114](https://github.com/ClickHouse/ClickHouse/pull/104114) ([Groene AI](https://github.com/groeneai)). * Fix several edge case correctness issues in setting `optimize_inverse_dictionary_lookup` where the optimization could silently drop rows or suppress exceptions. Closes [#103270](https://github.com/ClickHouse/ClickHouse/issues/103270). Close [#103085](https://github.com/ClickHouse/ClickHouse/issues/103085). [#104133](https://github.com/ClickHouse/ClickHouse/pull/104133) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix write out of bounds while deserializing `quantileTiming` state. [#104141](https://github.com/ClickHouse/ClickHouse/pull/104141) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix undefined behavior when an out-of-range `Float64` value is converted to a wide integer type (`UInt64`, `Int64`, `Int128`, `UInt128`, `Int256`, `UInt256`). Previously, values equal to `Float64(numeric_limits::max())` (which rounds up to a value above the actual maximum) bypassed the bounds check and produced UB in the subsequent cast. This affected aggregate-function parameter parsing (`topK`, `histogram`, `uniqUpTo`, `groupArrayInsertAt`, etc.) and integer-typed settings via `SET = `. Closes [#103817](https://github.com/ClickHouse/ClickHouse/issues/103817). [#104154](https://github.com/ClickHouse/ClickHouse/pull/104154) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` exception "Unexpected return type from comparison. Expected `UInt8`. Got `Const(Nullable(UInt8))`" when comparing a nested `Tuple(Tuple(Nullable(...)))` (or deeper nesting) with a `String` literal. The comparison's return type is now correctly inferred as `Nullable(UInt8)`, matching the runtime behaviour. [#104171](https://github.com/ClickHouse/ClickHouse/pull/104171) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort (`UndefinedBehaviorSanitizer: reference binding to null pointer`, segfault in release builds) on `SELECT ... FROM FINAL PREWHERE is_deleted = AND ` when `query_plan_optimize_lazy_final = 1`. The lazy-FINAL non-intersecting reading step lost the `is_deleted` column from its output header because prewhere consumed it as an input without re-exposing it as an output. The downstream `addIsDeletedFilter` step then dereferenced a null `ActionsDAG::Node` pointer. [#104177](https://github.com/ClickHouse/ClickHouse/pull/104177) ([Groene AI](https://github.com/groeneai)). * When calling system.failpoints table, it would use a failpoint, thus possibly disabling it. [#104237](https://github.com/ClickHouse/ClickHouse/pull/104237) ([Pedro Ferreira](https://github.com/PedroTadim)). * Fix `MemorySanitizer: use-of-uninitialized-value` in `detectLanguage*` functions when the input contains a UTF-8 character ending exactly at the buffer boundary. [#104257](https://github.com/ClickHouse/ClickHouse/pull/104257) ([Raúl Marín](https://github.com/Algunenano)). * Fix a logical error (`Bad cast from type DB::CachedObjectStorage to DB::S3ObjectStorage`) that aborted argument parsing of data-lake table functions and engines (`icebergS3`, `deltaLakeS3`, etc.) when called with `SETTINGS disk = '...'` against a disk whose underlying object storage is wrapped by a decorator such as a filesystem cache. Closes [#89300](https://github.com/ClickHouse/ClickHouse/issues/89300). [#104258](https://github.com/ClickHouse/ClickHouse/pull/104258) ([Groene AI](https://github.com/groeneai)). * Fix parsing of parenthesized subscript expressions on columns named `values`, for example `(values['a'])`, so they are no longer interpreted as SQL-standard `VALUES` table expressions. [#104312](https://github.com/ClickHouse/ClickHouse/pull/104312) ([Desel72](https://github.com/Desel72)). * Fix a server abort/`LOGICAL_ERROR` in the filesystem cache background eviction thread (`SLRUFileCachePriority::collectEvictionInfo`) that could fire when `keep_free_space_size_ratio` or `keep_free_space_elements_ratio` was high enough to trigger eviction while all cache entries had already been promoted to the SLRU protected queue (probationary empty). [#104313](https://github.com/ClickHouse/ClickHouse/pull/104313) ([Groene AI](https://github.com/groeneai)). * Fix exceptions (`NOT_FOUND_COLUMN_IN_BLOCK`, `LOGICAL_ERROR`, `AMBIGUOUS_COLUMN_NAME`) when using projections with UNION ALL views, window functions, or alias-column name collisions. Regression from [#88798](https://github.com/ClickHouse/ClickHouse/issues/88798). [#104317](https://github.com/ClickHouse/ClickHouse/pull/104317) ([Amos Bird](https://github.com/amosbird)). * Fixes a possible underflow in parsing postgres arrays. [#104322](https://github.com/ClickHouse/ClickHouse/pull/104322) ([Grant Holly](https://github.com/grantholly-clickhouse)). * Functions that return a non-`Nullable` type (such as `Array`, `Tuple`, or `Map`) now accept `Nullable` arguments. Affected functions include `extractAll`, `extractAllGroups`, `extractAllGroupsHorizontal`, `extractAllGroupsVertical`, `extractGroups`, `splitByChar`, `splitByString`, `splitByRegexp`, `splitByWhitespace`, `splitByNonAlpha`, and `alphaTokens`. `NULL` input rows produce the default value of the result type (e.g. an empty array) instead of raising "Nested type is not allowed inside Nullable type". [#104326](https://github.com/ClickHouse/ClickHouse/pull/104326) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix iceberg stats with partitioned table. This closes [#104321](https://github.com/ClickHouse/ClickHouse/issues/104321). [#104329](https://github.com/ClickHouse/ClickHouse/pull/104329) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix a fatal logical error at server startup on macOS (and similar jemalloc builds) where `Jemalloc::verifySetup` incorrectly reported a `jemalloc_enable_background_threads` mismatch because optional `background_thread` / `max_background_threads` mallctls are absent; verification is skipped when those mallctls are unavailable, and `getValue` no longer leaves the output uninitialized on failure. Closes [#102183](https://github.com/ClickHouse/ClickHouse/issues/102183). [#104330](https://github.com/ClickHouse/ClickHouse/pull/104330) ([SAYON DEEP](https://github.com/sayondeep)). * Fix a correctness regression where `SELECT` queries with `max_rows_to_read_leaf` and `read_overflow_mode_leaf = 'throw'` could incorrectly throw `TOO_MANY_ROWS` even when a skip index would have reduced the read below the leaf cap. The symmetric non-leaf settings (`max_rows_to_read` / `read_overflow_mode`) were already handled. [#104331](https://github.com/ClickHouse/ClickHouse/pull/104331) ([Groene AI](https://github.com/groeneai)). * `toUUID`, `toUUIDOrNull`, `toUUIDOrZero`, `toUUIDOrDefault`, `CAST` to `UUID` and `Nullable(UUID)`, `accurateCastOrNull` to `UUID`, and the input formats that parse a UUID from text (`Avro`, `MsgPack`, `JSONExtract`, ...) now reject strings that have the right length but contain non-hexadecimal characters. Previously such inputs were silently turned into a fabricated UUID by walking off the end of the hex digit lookup table; now `toUUID` throws `CANNOT_PARSE_UUID`, and the `Or*` variants return `NULL` / the zero UUID / the supplied default. [#104370](https://github.com/ClickHouse/ClickHouse/pull/104370) ([Groene AI](https://github.com/groeneai)). * Fix `Bad cast` `LOGICAL_ERROR` in `caseWithExpression` (and SQL `CASE expr WHEN ... THEN ... ELSE ...`) when the THEN-only supertype and the (THENs + ELSE) supertype land in different `ColumnDecimal` storages -- for example THEN values `(UInt16, Int8)` with an ELSE of `Decimal(9, 2)`. Closes [#104335](https://github.com/ClickHouse/ClickHouse/issues/104335). [#104378](https://github.com/ClickHouse/ClickHouse/pull/104378) ([Groene AI](https://github.com/groeneai)). * Internal failures reported via `logExceptionBeforeStart` (asynchronous insert flushes, materialized-view refreshes, parse errors raised during internal queries) now correctly increment the `FailedInternalQuery`, `FailedInternalSelectQuery`, and `FailedInternalInsertQuery` ProfileEvents alongside the user-visible `FailedQuery`, `FailedSelectQuery`, and `FailedInsertQuery` counters. Previously these internal counters stayed at zero for failures occurring before query execution started, undercounting a major class of internal failures. [#104399](https://github.com/ClickHouse/ClickHouse/pull/104399) ([Groene AI](https://github.com/groeneai)). * Fix `Code: 36. BAD_ARGUMENTS Expected literal, got {name:Type}` thrown when a query parameter is used inside the `base_backup` setting of a `BACKUP` or `RESTORE` statement (e.g. `BACKUP ... SETTINGS base_backup = S3({backup_name:String}, ...)`). The regression was introduced in `26.1.5` by PR [#99205](https://github.com/ClickHouse/ClickHouse/issues/99205). Closes [#103324](https://github.com/ClickHouse/ClickHouse/issues/103324). [#104413](https://github.com/ClickHouse/ClickHouse/pull/104413) ([Groene AI](https://github.com/groeneai)). * Fix a server abort in `IcebergLocal`/`IcebergS3` writes when `ALTER TABLE ... DROP COLUMN` is followed by an `INSERT` with `iceberg_metadata_staleness_ms` greater than zero. `ALTER` now invalidates the local Iceberg metadata files cache so subsequent reads and writes see the new schema. [#104419](https://github.com/ClickHouse/ClickHouse/pull/104419) ([Groene AI](https://github.com/groeneai)). * Fix `DROP ROLE`, `DROP USER`, `DROP SETTINGS PROFILE`, `DROP ROW POLICY`, `DROP QUOTA` and `DROP MASKING POLICY` to remove references to the dropped entity from any other access entity that referenced it (e.g. a user's `DEFAULT ROLE` list, a settings profile's `TO` list, a row policy's grantees) and persist the cleanup to disk. Previously the in-memory state appeared correct because `SHOW CREATE` filters unknown UUIDs, but the on-disk `.sql` files retained dangling `ID('')` entries and the references were resurrected on the next server restart, surfacing as `ACCESS_ENTITY_NOT_FOUND` errors during distributed query execution. [#104427](https://github.com/ClickHouse/ClickHouse/pull/104427) ([Groene AI](https://github.com/groeneai)). * Fixed a server abort triggered by `azureBlobStorage`, `AzureBlobStorage`-engine, and DeltaLake-on-Azure with a connection string whose `BlobEndpoint` URL has an empty, non-numeric, or out-of-range port (e.g. `BlobEndpoint=http://host:abc/`). The server now returns a clean `BAD_ARGUMENTS` error instead of aborting in debug/sanitizer builds. [#104460](https://github.com/ClickHouse/ClickHouse/pull/104460) ([Groene AI](https://github.com/groeneai)). * Allow an unquoted identifier as the user name in the `remote` and `remoteSecure` table functions, mirroring how the database and table arguments accept unquoted identifiers. Previously such a query failed with a misleading authentication error referring to the `default` user. [#104465](https://github.com/ClickHouse/ClickHouse/pull/104465) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `Logical error: Incorrect ASTSelectWithUnionQuery (modes: M, selects: N)` triggered when a SQL user-defined function body contains a parenthesized inner `UNION ALL` (e.g. `CREATE FUNCTION f AS x -> (SELECT 1 UNION ALL (SELECT 1 UNION ALL SELECT 1))`). [#104477](https://github.com/ClickHouse/ClickHouse/pull/104477) ([Groene AI](https://github.com/groeneai)). * Fix `uniqThetaIntersect` returning the cardinality of the first argument instead of `0` when the second argument is an empty `uniqTheta` state -- for example the result of `uniqThetaMergeStateIf(s, predicate)` when the predicate excludes every row. [#104529](https://github.com/ClickHouse/ClickHouse/pull/104529) ([Groene AI](https://github.com/groeneai)). * Fix `SHOW TABLES` and `system.tables` silently truncating the listing for `DataLakeCatalog` databases backed by Iceberg REST catalogs (`iceberg-rest`, `onelake`, `biglake`). When the catalog server paginates the list-tables or list-namespaces response (e.g. Microsoft Fabric / OneLake beyond ~50 tables per namespace), tables on later pages were silently invisible to `SHOW TABLES` and `system.tables`, even though they were queryable via direct `SELECT`. `RestCatalog` now follows the `next-page-token` continuation token defined by the Iceberg REST OpenAPI spec, matching the existing behavior of `PaimonRestCatalog` and `UnityCatalog`. [#104531](https://github.com/ClickHouse/ClickHouse/pull/104531) ([Groene AI](https://github.com/groeneai)). * The `input` table function now infers its structure from the surrounding `INSERT` query's `FORMAT` clause when that format has a fixed schema (`LineAsString`, `RawBLOB`, `JSONAsString`, etc.), so users no longer have to repeat the structure as `input('line String')` for these formats. [#104532](https://github.com/ClickHouse/ClickHouse/issues/104532). [#104533](https://github.com/ClickHouse/ClickHouse/pull/104533) ([Groene AI](https://github.com/groeneai)). * Updated datatype of fields in Iceberg history from `Int32` to `Int64`. Closes [#94176](https://github.com/ClickHouse/ClickHouse/issues/94176). [#104579](https://github.com/ClickHouse/ClickHouse/pull/104579) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fixed `Inconsistent AST formatting` `LOGICAL_ERROR` when parsing a function call where a lambda follows a comma, e.g. ``SELECT substring(x, `x` -> `x`)``. The parser used to silently merge the preceding arguments into the lambda's left-hand side, producing a single-argument call that could not be re-parsed back to the same AST. It now preserves the function's original arity. [#104626](https://github.com/ClickHouse/ClickHouse/pull/104626) ([Groene AI](https://github.com/groeneai)). * `CHECK TABLE t` now prefers a `TEMPORARY` table over a permanent one with the same name when no database qualifier is given, matching the precedence already used by `SHOW CREATE TABLE`, `DESCRIBE TABLE`, `OPTIMIZE TABLE`, and `ALTER TABLE`. Previously `CHECK TABLE t` skipped temporary tables entirely, so it failed with `UNKNOWN_TABLE` on a temporary `Log` or `File` table even though those engines support `CHECK`. Follow-up to [#100966](https://github.com/ClickHouse/ClickHouse/pull/100966). [#104637](https://github.com/ClickHouse/ClickHouse/pull/104637) ([Groene AI](https://github.com/groeneai)). * Fix Keeper termination and restart loop when `get /keeper/availability_zone` is sent with `quorum_reads=true` to a Keeper without `` configured. [#104663](https://github.com/ClickHouse/ClickHouse/pull/104663) ([myeongjun](https://github.com/myeongjjun)). * Fix a TOCTOU data race in `FutureSetFromTuple::buildOrderedSetInplace` that results in a logical error. [#104673](https://github.com/ClickHouse/ClickHouse/pull/104673) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix several functions returning different results for the same input depending on whether arguments arrived as columns or constants: `bitRotateLeft` / `bitRotateRight` (boundary shift counts), `length(FixedString)` / `concatWithSeparator` with `LowCardinality(Nullable)` inputs, `roundDown` on NaN, and `rightUTF8` on invalid UTF-8. [#104710](https://github.com/ClickHouse/ClickHouse/pull/104710) ([Raúl Marín](https://github.com/Algunenano)). * Fixed `anyHeavy` returning a non-heavy value when the most frequent value was the column default and the data lived across multiple `MergeTree` parts (sparse-column read path). [#104712](https://github.com/ClickHouse/ClickHouse/pull/104712) ([Raúl Marín](https://github.com/Algunenano)). * Fix several defects in setting-constraint handling: `MergeTreeSettings` constraints declared on a setting's canonical name could be bypassed by writing to an alias of that setting; the `disallowed_values` constraint check threw an exception on clamp paths (secondary queries, `ON CLUSTER` workers, `SQL SECURITY DEFINER` views) instead of silently dropping the change. [#104737](https://github.com/ClickHouse/ClickHouse/pull/104737) ([Raúl Marín](https://github.com/Algunenano)). * Fix a `LOGICAL_ERROR` exception (`Metadata is not initialized`) raised by `OPTIMIZE TABLE`, `ALTER TABLE ... DELETE`, `ALTER TABLE ... ADD COLUMN` and other ALTER variants on a lazily-attached `Iceberg` / `IcebergLocal` / `DeltaLake` / `Hudi` table whose metadata had not been loaded yet (typical after a server restart, or after a previous metadata write failed and left a corrupted metadata file on disk). The operation now either proceeds normally if the metadata loads successfully, or surfaces the underlying load failure as a regular user-facing exception instead of aborting the server in debug / sanitizer builds. [#104738](https://github.com/ClickHouse/ClickHouse/pull/104738) ([Groene AI](https://github.com/groeneai)). * Fix segfault due to a use-after-free bug in `AvroConfluentRowInputFormat`. [#104751](https://github.com/ClickHouse/ClickHouse/pull/104751) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Scalar variants of `numericIndexedVectorPointwiseMultiply`, `numericIndexedVectorPointwiseDivide`, `numericIndexedVectorPointwiseEqual`, and `numericIndexedVectorPointwiseNotEqual` now raise `INCORRECT_DATA` when called with a `UInt64` scalar above `Int64::max`. [#104784](https://github.com/ClickHouse/ClickHouse/pull/104784) ([FriendLey](https://github.com/FriendLey)). * Fix a bug where manually overriding a setting via its alias name (e.g. `SET enable_analyzer = 1` instead of `SET allow_experimental_analyzer = 1`) after applying a `compatibility` setting could cause that override to be reverted by a subsequent change of the `compatibility` setting. [#104829](https://github.com/ClickHouse/ClickHouse/pull/104829) ([Raúl Marín](https://github.com/Algunenano)). * Fix AWS logger being disabled after https://github.com/ClickHouse/ClickHouse/commit/0e8ad4355c9d. [#104837](https://github.com/ClickHouse/ClickHouse/pull/104837) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix three more functions returning different results for the same input depending on whether arguments arrived as columns or constants: `transform` (and `caseWithExpression` through it) with a constant `Date`/`Date32`/`Enum`/`FixedString` default, comparison operators between a `String` and a constant `FixedString`, and `if`/`ifNull`/`nullIf` with a `FixedString` branch under a constant condition. [#104858](https://github.com/ClickHouse/ClickHouse/pull/104858) ([Raúl Marín](https://github.com/Algunenano)). * Fix `Bad cast from type DB::ColumnConst to DB::ColumnNullable` server abort during partition pruning for `MergeTree` tables when the partition expression contains a function chain that collapses to a single constant value (such as `floor(NULL, toRelativeYearNum(...))`). [#104861](https://github.com/ClickHouse/ClickHouse/pull/104861) ([Groene AI](https://github.com/groeneai)). * Fix a `ThreadSanitizer` data race in the `ContextData` copy constructor: `table_function_results` was copied from the source object without acquiring `table_function_results_mutex`, so a concurrent `Context::executeTableFunction` writer could race against the unsynchronized read in the copy constructor. [#104879](https://github.com/ClickHouse/ClickHouse/pull/104879) ([Groene AI](https://github.com/groeneai)). * Fix data part consistency checks for types with dynamic structure and detect corrupted columns_substreams.txt. Resubmit of https://github.com/ClickHouse/ClickHouse/pull/103858 with additional changes. [#104888](https://github.com/ClickHouse/ClickHouse/pull/104888) ([Pavel Kruglov](https://github.com/Avogar)). * Fix race between DROP and UNDROP in DatabaseCatalog. [#104915](https://github.com/ClickHouse/ClickHouse/pull/104915) ([Azat Khuzhin](https://github.com/azat)). * Fix filesystem cache dynamic resize with partially downloaded segments, including restore accounting after failed eviction. [#104921](https://github.com/ClickHouse/ClickHouse/pull/104921) ([Antonio Andelic](https://github.com/antonio2368)). * `DETACH TABLE` on a temporary table (without the `TEMPORARY` keyword) now correctly raises `SYNTAX_ERROR`, matching the behavior of `DETACH TEMPORARY TABLE`. Previously it silently set an internal `is_detached` flag and returned without error. To remove a temporary table, use `DROP TEMPORARY TABLE` or `DROP TABLE` (the latter resolves the temporary table via `Context::ResolveExternal`). Closes [#103475](https://github.com/ClickHouse/ClickHouse/issues/103475). [#104943](https://github.com/ClickHouse/ClickHouse/pull/104943) ([Groene AI](https://github.com/groeneai)). * Make the `filesystem` table function honor `max_memory_usage` / `max_server_memory_usage` when loading file content. Previously, large or numerous parallel content reads could push past the limit without raising `MEMORY_LIMIT_EXCEEDED` and end up OOM-killed instead. [#104956](https://github.com/ClickHouse/ClickHouse/pull/104956) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a `LOGICAL_ERROR` thrown when filtering `system.detached_tables` by `uuid` (e.g. `SELECT count() FROM system.detached_tables WHERE uuid = '...'`). The query now returns the expected result instead of aborting the server. [#104979](https://github.com/ClickHouse/ClickHouse/pull/104979) ([Groene AI](https://github.com/groeneai)). * `flattenTuple` no longer raises a `LOGICAL_ERROR` when called on a tuple whose nested structure contains only empty `Tuple()` leaves (e.g. `Tuple(c0 Array(Tuple()))` or `Tuple(c0 Tuple())`). Such inputs now produce a user-facing `ILLEGAL_TYPE_OF_ARGUMENT` exception explaining that the flatten result would be an empty tuple. [#104989](https://github.com/ClickHouse/ClickHouse/pull/104989) ([Groene AI](https://github.com/groeneai)). * Fix a server abort and a silent-wrong-result bug when querying `loop(remote(...))` (or any `loop()` wrapping a storage that can defer aggregation) with `GROUP BY`. The outer planner used to add `MergingAggregatedStep` based on the inner storage's reported processing stage, but `LoopSource` always materialises its inner select with `QueryProcessingStage::Complete` and emits plain column chunks, so `MergingAggregatedTransform` tripped a `LOGICAL_ERROR` (`Chunk info was not set for chunk in MergingAggregatedTransform`) under `enable_parallel_replicas = 1` and silently dropped the outer aggregation otherwise. [#105001](https://github.com/ClickHouse/ClickHouse/pull/105001) ([Groene AI](https://github.com/groeneai)). * `clickhouse-benchmark --reconnect` (bare, no value) was inadvertently broken in 25.4 by a change that turned `--reconnect` into an integer option, requiring a value. The bare form now works again and is equivalent to `--reconnect=1` (reconnect on every query). [#105006](https://github.com/ClickHouse/ClickHouse/pull/105006) ([Groene AI](https://github.com/groeneai)). * Fixed malformed JSON output for column names ending with incomplete UTF-8 sequences. [#105012](https://github.com/ClickHouse/ClickHouse/pull/105012) ([Pablo Marcos](https://github.com/pamarcos)). * Fix silent data loss after `EXCHANGE TABLES` or `CREATE OR REPLACE TABLE` of a materialized view's source table. The `MV`'s source-view dependency edge is now kept on the original name so it continues to fire on inserts. Regression introduced by [#98779](https://github.com/ClickHouse/ClickHouse/issues/98779); restores the pre-regression behavior. [#105029](https://github.com/ClickHouse/ClickHouse/pull/105029) ([Sema Checherinda](https://github.com/CheSema)). * Fix `CANNOT_COMPILE_CODE Could not find symbol __fixunsdfti` when JIT-compiling expressions that convert a `Float` to `UInt128`, such as `toUInt128()`. The unsigned 128-bit float-to-int compiler-rt builtins were missing from the JIT symbol resolver. Closes [#105031](https://github.com/ClickHouse/ClickHouse/issues/105031). [#105048](https://github.com/ClickHouse/ClickHouse/pull/105048) ([Raúl Marín](https://github.com/Algunenano)). * Fix `clickhouse-local` not printing log messages from a failed query when `send_logs_level` is set. [#105067](https://github.com/ClickHouse/ClickHouse/pull/105067) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results when the same parameterized view is referenced more than once in the same query with different argument values. Previously, the analyzer collapsed the calls into one, silently dropping all but the first filter. [#105170](https://github.com/ClickHouse/ClickHouse/pull/105170) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `TOTALS` row being rendered twice at the bottom of the result table in the `play.html` web UI. [#103803](https://github.com/ClickHouse/ClickHouse/pull/103803) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Reject non-finite vectors (`NaN`, `±Inf`) during vector search (either as searched vectors or as reference vectors); they previously caused undefined behavior in `usearch`. [#104079](https://github.com/ClickHouse/ClickHouse/pull/104079) ([Groene AI](https://github.com/groeneai)). * PromQL: fix the aggregation operator for empty vectors. [#104425](https://github.com/ClickHouse/ClickHouse/pull/104425) ([Vitaly Baranov](https://github.com/vitlibar)). * PromQL: fix error handling in the Prometheus query API. [#104741](https://github.com/ClickHouse/ClickHouse/pull/104741) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix a race in `MergeTreeTransaction::afterCommit` where, after a connection loss between writing the commit `CSN` to ZooKeeper and finalizing the transaction, the `COMMIT` response could reach the client before the new `creation_csn` / `removal_csn` became visible in `system.parts`. [#104708](https://github.com/ClickHouse/ClickHouse/pull/104708) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix exponential memory growth in the KQL parser when converting nested array indexing (`arr[arr[arr[...]]]`). [#105142](https://github.com/ClickHouse/ClickHouse/pull/105142) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results for `RIGHT ANY JOIN` when the right table has multiple rows per key and the output block is split due to size limits. Closes [#99431](https://github.com/ClickHouse/ClickHouse/issues/99431). [#102064](https://github.com/ClickHouse/ClickHouse/pull/102064) ([Vladimir Cherkasov](https://github.com/vdimir)). * Check for stack overflow in `Avro` reader during nested-type deserialization. [#102417](https://github.com/ClickHouse/ClickHouse/pull/102417) ([Pavel Kruglov](https://github.com/Avogar)). * Fix duplicate rows in `system.completions` for `MergeTree` settings — each setting name appeared twice because both `getMergeTreeSettings` and `getReplicatedMergeTreeSettings` were dumped despite having identical setting names. Closes [#102013](https://github.com/ClickHouse/ClickHouse/issues/102013). [#102015](https://github.com/ClickHouse/ClickHouse/pull/102015) ([Groene AI](https://github.com/groeneai)). * Fix `StorageObjectStorageQueue` (`S3Queue`, `AzureQueue`) blocking shutdown until partially-processed files were fully read from object storage. The source now aborts the read immediately on shutdown; deduplication ensures rows already streamed into the destination table before shutdown are not duplicated when the file is retried on next start. [#103126](https://github.com/ClickHouse/ClickHouse/pull/103126) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix multi-block inserts to the `Alias` table engine with `insert_deduplication_token` so that all blocks are kept. [#103246](https://github.com/ClickHouse/ClickHouse/pull/103246) ([Enric Calabuig](https://github.com/eclbg)). * Fix `JSONHas` and `JSONExtractBool` on native `JSON` columns returning the extracted value (cast to `UInt8`) instead of `0`/`1`. [#103313](https://github.com/ClickHouse/ClickHouse/pull/103313) ([zxuhan7](https://github.com/zxuhan)). * Protect the `CustomSeparated` input format against malformed or adversarial data that omits `format_custom_row_after_delimiter`. Header detection, schema inference, and variable-column rows previously accumulated fields unboundedly and could allocate many gigabytes of memory before failing. Reads now fail with `INCORRECT_DATA` once a single row contains more than 1,000,000 fields. [#103404](https://github.com/ClickHouse/ClickHouse/pull/103404) ([Groene AI](https://github.com/groeneai)). * Fix `RESTORE` of replicated `MergeTree` tables so that restored part attachment uses `backup_restore_keeper_max_retries` instead of the regular insert Keeper retry budget. [#104610](https://github.com/ClickHouse/ClickHouse/pull/104610) ([Pablo Marcos](https://github.com/pamarcos)). * Fix wrong results from `WHERE p AND ` on `MergeTree` tables. Previously such queries returned zero rows because part pruning derived a `NULL` default from the `LowCardinality(Nullable(...))` type and synthesized a `notEquals(x, NULL)` guard, pruning every part. [#104767](https://github.com/ClickHouse/ClickHouse/pull/104767) ([Groene AI](https://github.com/groeneai)). * Fix incorrect aggregation result when grouping by a non-injective function of a column that is also the partition key (e.g. `PARTITION BY a` with `GROUP BY intDiv(a, 2)` or `a % 2`). Values from different partitions that mapped to the same group were not merged, producing duplicate group rows under `allow_aggregate_partitions_independently = 1`. [#104869](https://github.com/ClickHouse/ClickHouse/pull/104869) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix a race in `Keeper` snapshot transfer where a snapshot could be removed or moved while it was being sent to a recovering follower. [#104941](https://github.com/ClickHouse/ClickHouse/pull/104941) ([Antonio Andelic](https://github.com/antonio2368)). * Fix a silent wrong-result bug in `hilbertEncode` and `mortonEncode` where a non-constant `Tuple` first argument (range mask) used row 0's values to drive the bit-shift for every row. The functions now compute the per-row mask values, so the result no longer depends on the input being constant or on the block size. [#104992](https://github.com/ClickHouse/ClickHouse/pull/104992) ([Groene AI](https://github.com/groeneai)). * Fix `clickhouse-local` silently ignoring user-level configuration (`profiles`, `users`, `quotas`, access-control settings) when the configuration file is auto-discovered from `./clickhouse-local.xml`, `~/.clickhouse-local/config.xml`, or `/etc/clickhouse-local/config.xml`. Previously these settings were applied only when `--config-file` was passed or `./config.xml` existed in the current directory; now all discovery paths behave consistently. [#105008](https://github.com/ClickHouse/ClickHouse/pull/105008) ([Groene AI](https://github.com/groeneai)). * Fix `BFloat16` columns silently returning zero matches when compared against a string literal (e.g. `WHERE bf16_col = '49.9'`). [#105042](https://github.com/ClickHouse/ClickHouse/pull/105042) ([Raúl Marín](https://github.com/Algunenano)). * Fix `CLEAR COLUMN` and TTL handling during merges for `SummingMergeTree`, `AggregatingMergeTree`, and `CoalescingMergeTree` when merge-required columns are absent or expired. Closes [#101953](https://github.com/ClickHouse/ClickHouse/issues/101953). [#105203](https://github.com/ClickHouse/ClickHouse/pull/105203) ([Antonio Andelic](https://github.com/antonio2368)). * Fix a race between `DNSCacheUpdater::run` and `Context::reloadClusterConfig` by adding an explicit null-guard for `shared->clusters_config` before dereferencing it. [#105220](https://github.com/ClickHouse/ClickHouse/pull/105220) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix `best_effort` date-time parsing for `toDateTime64` with `ms` and `ns` precision. [#105233](https://github.com/ClickHouse/ClickHouse/pull/105233) ([Kaviraj Kanagaraj](https://github.com/kavirajk)). * Fix `OSIOWaitMicroseconds` reporting thread-lifetime I/O wait instead of per-query I/O wait. [#105246](https://github.com/ClickHouse/ClickHouse/pull/105246) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * `Hudi` table engine now raises `INCORRECT_DATA` (a regular query-level exception) instead of `LOGICAL_ERROR` when a parquet file in the table directory does not match the Hudi `[FileId]_[FileWriteToken]_[Timestamp].[extension]` naming convention. Previously such file names caused an exception in debug builds. [#105266](https://github.com/ClickHouse/ClickHouse/pull/105266) ([Groene AI](https://github.com/groeneai)). * Fix deadlock in `ParallelFormattingOutputFormat` on schedule failure: when `scheduleFormatterThreadForUnitWithNumber` throws (e.g. `CANNOT_SCHEDULE_TASK`), the unit was left in `READY_TO_FORMAT` with no formatter thread to process it, leading to a hang. The schedule call is now wrapped in `try`/`catch` and any failure is routed through `onBackgroundException` so the collector exits cleanly. [#105275](https://github.com/ClickHouse/ClickHouse/pull/105275) ([Azat Khuzhin](https://github.com/azat)). * Fix silent data loss in `Distributed` async inserts when recovering from an abnormal shutdown: if the last `.bin` file in a saved batch was intact but a middle one was corrupted, `DistributedAsyncInsertBatch::recoverBatch` would only validate the last file's header and then `sendBatch` would mark the entire batch — including the intact files — as broken, losing their rows. Each file's header is now validated individually so only the actually broken file is moved to `broken/` and the surviving rows reach the remote shard. [#105281](https://github.com/ClickHouse/ClickHouse/pull/105281) ([Groene AI](https://github.com/groeneai)). * Fix a regression in `clickhouse extract-from-config --try`: when the config used `from_env` attributes and had no `include_from` element (or had one pointing to a missing file), all `from_env` substitutions were silently dropped and returned empty strings. This broke the Docker entrypoint port discovery in 26.2.5. Closes [#101704](https://github.com/ClickHouse/ClickHouse/issues/101704). [#105283](https://github.com/ClickHouse/ClickHouse/pull/105283) ([Groene AI](https://github.com/groeneai)). * Fix `INSERT INTO` a `SQLite` table failing with a SQLite syntax error when the inserted value is an `Enum`, `JSON`, or `AggregateFunction` whose text representation contains a single quote. The `output_format_values_escape_quote_with_quote` setting is now honored by the corresponding serializations (previously only `String` and `FixedString` honored it). [#105285](https://github.com/ClickHouse/ClickHouse/pull/105285) ([Groene AI](https://github.com/groeneai)). * Fix `Code: 44. ILLEGAL_COLUMN: Cannot add column ...: column with this name already exists` thrown by distributed queries with `allow_push_predicate_ast_for_distributed_subqueries = 1` (the default) when a `GLOBAL IN` tuple is matched against a subquery whose projection contains duplicate column names, e.g. `(x, y) GLOBAL IN (SELECT number, number FROM numbers(5))`. [#105290](https://github.com/ClickHouse/ClickHouse/pull/105290) ([Groene AI](https://github.com/groeneai)). * Fix `reinterpret` to `Array(LowCardinality(...))` returning a confusing `NOT_IMPLEMENTED` error at runtime; it now returns `ILLEGAL_TYPE_OF_ARGUMENT` during type checking. [#105301](https://github.com/ClickHouse/ClickHouse/pull/105301) ([Raúl Marín](https://github.com/Algunenano)). * `minMap`/`maxMap` (array form) and `minMappedArrays`/`maxMappedArrays` now treat `NaN` consistently with `ORDER BY`: `NaN` is treated as last (returned only when all values are `NaN`). Previously, results depended on the position of `NaN` in the data due to IEEE 754 unordered comparison semantics, and disagreed with the `Map`-argument form of `minMap`/`maxMap` that was fixed in [#100448](https://github.com/ClickHouse/ClickHouse/pull/100448). [#105331](https://github.com/ClickHouse/ClickHouse/pull/105331) ([Raúl Marín](https://github.com/Algunenano)). * Fix a memory leak in the per-table `ColumnsDescription` cache for `MergeTree`-family tables with `Nested` columns. Entries were never evicted across `ALTER ADD COLUMN`/`DROP COLUMN` cycles when `share_nested_offsets = 1` (the default). [#105376](https://github.com/ClickHouse/ClickHouse/pull/105376) ([Groene AI](https://github.com/groeneai)). * Fix `BAD_ARGUMENTS` exception "It's a bug! Only integer types are supported by `__bitWrapperFunc`" thrown when a `SELECT` with a `TYPE set(N)` skip index has a `WHERE` atom whose result type is `Float`, `BFloat16`, or any other non-integer type (for example `WHERE c0 + 0.1` or `WHERE log(c0)`). The skip index now falls back to the regular filter path in that case. [#105384](https://github.com/ClickHouse/ClickHouse/pull/105384) ([Groene AI](https://github.com/groeneai)). #### Build/Testing/Packaging Improvement * Stop using the system `compiler-rt` libraries and headers. Closes [#91475](https://github.com/ClickHouse/ClickHouse/issues/91475). [#102857](https://github.com/ClickHouse/ClickHouse/pull/102857) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Refresh the distroless Docker base image to fix OpenSSL CVEs in `libssl3t64`. [#103583](https://github.com/ClickHouse/ClickHouse/pull/103583) ([Rahul Nair](https://github.com/motsc)). * Reliably receive the GPG key during releases by chaining multiple Ubuntu keyservers, instead of timing out on `keyserver.ubuntu.com`. [#103834](https://github.com/ClickHouse/ClickHouse/pull/103834) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Remove and forbid build-time `CMake` checks (`check_*`, `try_compile`, `try_run`). The compiler and toolchain are fixed, so feature detection at configure time is unnecessary and is now blocked project-wide; any version-specific behaviour must be gated on `CMAKE_CXX_COMPILER_VERSION` explicitly. [#103980](https://github.com/ClickHouse/ClickHouse/pull/103980) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Bump `libarchive` from 3.8.6 to 3.8.7. [#104047](https://github.com/ClickHouse/ClickHouse/pull/104047) ([Robert Schulze](https://github.com/rschu1ze)). * Update `mongo-c-driver` to 2.3.0. [#104300](https://github.com/ClickHouse/ClickHouse/pull/104300) ([Raúl Marín](https://github.com/Algunenano)). * Update LLVM dependencies for 22.x. [#104381](https://github.com/ClickHouse/ClickHouse/pull/104381) ([Joshua Carp](https://github.com/jmcarp)). * Make the embedded-client and PTY descriptor classes build on macOS and FreeBSD by using portable POSIX `posix_openpt` / `grantpt` / `unlockpt` and removing Linux-only `#if` guards. [#104436](https://github.com/ClickHouse/ClickHouse/pull/104436) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Upgrade `librdkafka` to version 2.14.1. [#105222](https://github.com/ClickHouse/ClickHouse/pull/105222) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). ### ClickHouse release 26.4, 2026-04-30. [Presentation](https://presentations.clickhouse.com/2026-release-26.4/), [Video](https://www.youtube.com/watch?v=9lSVy7k2EoI) #### Backward Incompatible Change * The `IN` operator now uses exact value semantics for `Bool` type: only `0` and `1` values in the set match `Bool` values. Previously, numeric values greater than `255` in the `IN` set were incorrectly clamped to true when compared against `Bool`, so `SELECT CAST(1, 'Bool') IN (256)` returned 1. Now it correctly returns `0`. Closes [#92980](https://github.com/ClickHouse/ClickHouse/issues/92980). [#93115](https://github.com/ClickHouse/ClickHouse/pull/93115) ([Ashrith Bandla](https://github.com/ashrithb)). * The H3 library has been updated to v4, which improves the precision of length, area, and other metric calculations. This change is backward incompatible because the new results differ from previous ones. [#100348](https://github.com/ClickHouse/ClickHouse/pull/100348) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Disallows using `SELECT` as a bareword identifier in a `WITH` expression list element. [#101059](https://github.com/ClickHouse/ClickHouse/pull/101059) ([Aruj Bansal](https://github.com/arujbansal)). * This patch changes how the merge table will handle virtuals. If the underlying table contains `_table` or `_database`, these columns will be read from storage; otherwise, they will be filled after the read step using the expression step. [#101742](https://github.com/ClickHouse/ClickHouse/pull/101742) ([Mikhail Artemenko](https://github.com/Michicosun)). * The `IN` operator now rejects lossy `Decimal` conversions inside composite types (`Tuple`, `Array`, `Map`) as well, making its behavior consistent with top-level scalar comparisons. Previously, precision checks were enforced only for top-level scalar values: for example, `CAST('33.3', 'Decimal64(1)') IN (33.33)` correctly returned `0`, but `CAST(['33.3'], 'Array(Decimal64(1))') IN ([33.33])` incorrectly returned `1` because the lossy conversion happened inside an `Array`. Now both cases correctly return `0`. [#101812](https://github.com/ClickHouse/ClickHouse/pull/101812) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Reduced default `http_max_fields` from 1,000,000 to 1,000 and `http_max_field_name_size` from 128 KB to 4 KB to limit pre-authentication memory usage by HTTP connections. Added `http_max_request_header_size` and `http_headers_read_timeout` settings. Users who rely on the previous higher limits can restore them via settings. [#103285](https://github.com/ClickHouse/ClickHouse/pull/103285) ([Sema Checherinda](https://github.com/CheSema)). #### New Feature * Add automatic spilling to hash and parallel hash joins by converting them to grace hash join when memory limit is reached. This behavior is controlled by `max_bytes_before_external_join`. [#97813](https://github.com/ClickHouse/ClickHouse/pull/97813) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Add Arrow Flight SQL support. [#91170](https://github.com/ClickHouse/ClickHouse/pull/91170) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Add incremental read support for `Paimon` table engines with Keeper-backed snapshot progress tracking, including targeted snapshot delta reads via `paimon_target_snapshot_id`, and extend test coverage for type mapping, partition pruning, and incremental read scenarios. [#93655](https://github.com/ClickHouse/ClickHouse/pull/93655) ([XiaoBinMu](https://github.com/Binnn-MX)). * The `stem` function is now non-experimental (previously, setting `allow_experimental_nlp_functions` had to be enabled). [#102399](https://github.com/ClickHouse/ClickHouse/pull/102399) ([Jimmy Aguilar Mena](https://github.com/Ergus)). You can now stem all the words/tokens in columns `String`, `FixedString`, `Array([Fixed]String)`, `Nullable`, `LowCardinality` and `Const` easily with the `stem` function. [#99137](https://github.com/ClickHouse/ClickHouse/pull/99137) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Implement new behavior of `max_insert_block_size_rows`, `max_insert_block_size_bytes`, `min_insert_block_size_rows`, `min_insert_block_size_bytes` in squashing under compatibility setting `use_strict_insert_block_limits`. [#94207](https://github.com/ClickHouse/ClickHouse/pull/94207) ([Kirill Kopnev](https://github.com/Fgrtue)). * Add function `arrayAutocorrelation(arr [, max_lag])` that computes the normalized autocorrelation of a numeric array for each lag. Supports integer, float, and decimal array types. [#94776](https://github.com/ClickHouse/ClickHouse/pull/94776) ([Wenyu Chen](https://github.com/wenyuchen96)). * A SQL function `obfuscateQuery`. Closes [#98010](https://github.com/ClickHouse/ClickHouse/issues/98010). [#98305](https://github.com/ClickHouse/ClickHouse/pull/98305) ([Xuewei Wang](https://github.com/Sallery-X)). * Add support for Map and JSON/Object types as dictionary attributes. Now dictionaries can store and retrieve complex types including Map(String, String), Map(String, Array(String)), JSON, and Nullable(JSON) types in both FLAT and HASHED layouts. [#98627](https://github.com/ClickHouse/ClickHouse/pull/98627) ([yanglongwei](https://github.com/ylw510)). * Added two new MergeTree settings — `replicated_fetches_min_part_level` and `replicated_fetches_min_part_level_timeout_seconds` — that allow replicas to skip fetching freshly-inserted (unmerged) parts from peers, reducing replication overhead during heavy ingestion. [#98625](https://github.com/ClickHouse/ClickHouse/pull/98625) ([tanner-bruce](https://github.com/tanner-bruce)). * Add MergeTree skip index support for JSON columns using JSONAllPaths with bloom_filter, tokenbf_v1, ngrambf_v1, and text (inverted) index types, enabling granule skipping based on the set of JSON paths present in each granule. [#98886](https://github.com/ClickHouse/ClickHouse/pull/98886) ([Pavel Kruglov](https://github.com/Avogar)). * The `printf` function now supports non-constant format strings, allowing different format patterns per row based on column values. [#98991](https://github.com/ClickHouse/ClickHouse/pull/98991) ([Yash ](https://github.com/Onyx2406)). * Add a new projection index, `commit_order`, that reorganizes data in insertion order. [#99004](https://github.com/ClickHouse/ClickHouse/pull/99004) ([Mikhail Artemenko](https://github.com/Michicosun)). * Add `highlight` function that wraps occurrences of search terms in a text string with HTML tags (default ``/``). Supports ASCII case-insensitive matching, automatic merging of overlapping matches, and custom open/close tags. [#99131](https://github.com/ClickHouse/ClickHouse/pull/99131) ([Peng](https://github.com/fastio)). * Implement quotas by normalized query hash to protect public ClickHouse services from abuse. 1. Support `NORMALIZED_QUERY_HASH` as a quota key type - separate quota buckets per unique normalized query, so `CREATE QUOTA q KEYED BY normalized_query_hash` tracks each distinct query independently. 2. Support `QUERIES_PER_NORMALIZED_HASH` as a quota resource type -- limits max executions of any single normalized query within an interval, so `MAX queries_per_normalized_hash = 100` prevents any one query pattern from running more than 100 times. [#99586](https://github.com/ClickHouse/ClickHouse/pull/99586) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Users can now write join queries using the `NATURAL JOIN` syntax, which automatically matches on all columns sharing the same name and deduplicates those columns in the result. [#99840](https://github.com/ClickHouse/ClickHouse/pull/99840) ([Peter Nguyen](https://github.com/petern48)). * Support `SET TIME ZONE 'tz'` as alias for `SET session_timezone`. [#99883](https://github.com/ClickHouse/ClickHouse/pull/99883) ([phulv94](https://github.com/phulv94)). * Added support for parameterized queries in the Web UI (`play.html`): query parameters like `{name:Type}` are detected and input fields are shown for filling in their values. [#100041](https://github.com/ClickHouse/ClickHouse/pull/100041) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support SQL standard `VALUES` clause as a table expression in `FROM`, e.g. `SELECT * FROM (VALUES (1, 'a'), (2, 'b')) AS t(id, val)`. [#100143](https://github.com/ClickHouse/ClickHouse/pull/100143) ([Desel72](https://github.com/Desel72)). * Add PostgreSQL-compatible units to the `EXTRACT` operator: `EPOCH`, `DOW`, `DOY`, `ISODOW`, `ISOYEAR`, `WEEK`, `CENTURY`, `DECADE`, `MILLENNIUM`. Also fix `EXTRACT(WEEK FROM date)` which previously threw an error. [#100274](https://github.com/ClickHouse/ClickHouse/pull/100274) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added support for SQL-standard compound interval literals with `TO` range qualifiers, e.g. `INTERVAL '1:30' HOUR TO MINUTE`. Internally decomposed into sums of intervals. [#100453](https://github.com/ClickHouse/ClickHouse/pull/100453) ([Desel72](https://github.com/Desel72)). * Add asynchronous metrics for kernel TCP receive and transmit buffer memory (`sk_rmem_alloc`, `sk_wmem_alloc`) of HTTP connection pool sockets, reported as p50/p75/p90/p95 percentiles and totals per connection group. [#100575](https://github.com/ClickHouse/ClickHouse/pull/100575) ([Sema Checherinda](https://github.com/CheSema)). * Added a jemalloc profiling web UI for ClickHouse Keeper, available at `/jemalloc` on the HTTP control port. [#100606](https://github.com/ClickHouse/ClickHouse/pull/100606) ([murphy-4o](https://github.com/murphy-4o)). * Implement command `SYSTEM FLUSH OBJECT STORAGE QUEUE db.table PATH 'x'` for ordered and unordered modes. [#100709](https://github.com/ClickHouse/ClickHouse/pull/100709) ([Bharat Nallan](https://github.com/bharatnc)). * Added function `JSONAllValues` that returns all values from a `JSON` column as `Array(String)`, with values serialized in text representation and ordered by their path names. Added support of text index for `JSONAllValues` expression on `JSON` columns. When a text index is created on `JSONAllValues(json_column)`, it is automatically used to filter queries on `JSON` subcolumns (e.g., `json_column.key1 = 'value'`). [#100730](https://github.com/ClickHouse/ClickHouse/pull/100730) ([Anton Popov](https://github.com/CurtizJ)). * Adds a new setting `input_format_column_name_matching_mode` which allows different case sensitivities for input formats. [#99346](https://github.com/ClickHouse/ClickHouse/pull/99346) ([manerone](https://github.com/Manerone)). * Add `watch` command to `clickhouse-keeper-client` with watch support in `get`, `exists`, and `ls` commands. [#100834](https://github.com/ClickHouse/ClickHouse/pull/100834) ([Den Kalantaevskii](https://github.com/Diskein)). * Added `getChildrenRecursive` (ListRecursive) request to ClickHouse Keeper and `lsr` command to `clickhouse-keeper-client`. This closes [#99916](https://github.com/ClickHouse/ClickHouse/issues/99916). [#100998](https://github.com/ClickHouse/ClickHouse/pull/100998) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Add new function `arrayTranspose` taking a two-dimensional array (matrix) and transposing it: `SELECT arrayTranspose([[1, 2, 3], [4, 5, 6]])`. [#101214](https://github.com/ClickHouse/ClickHouse/pull/101214) ([Vitaly Baranov](https://github.com/vitlibar)). * `auto_statistics_types mergetree` setting defaults to `'minmax, uniq'` — minmax and uniq statistics are created automatically for all suitable columns in new tables - `materialize_statistics_on_insert` defaults to false — statistics are now built during merges rather than at insert time, reducing insert overhead. use `SET materialize_statistics_on_insert = 1` to restore the old behavior. [#101275](https://github.com/ClickHouse/ClickHouse/pull/101275) ([Han Fei](https://github.com/hanfei1991)). * Add `prefer_dependency_replica` refresh setting for materialized view dependency chains to reduce missing data from cross-replica replication lag. [#101591](https://github.com/ClickHouse/ClickHouse/pull/101591) ([Seva Potapov](https://github.com/seva-potapov)). * Adds a `hasPhrase` (alias `matchPhrase`) function for phrase search (continuous sequences of tokens). Search is brute-force, i.e. not supported by the text index yet. [#101997](https://github.com/ClickHouse/ClickHouse/pull/101997) ([Elmi Ahmadov](https://github.com/ahmadov)). * Add `s3_read_request_duration_microseconds` and `s3_read_request_bytes` histogram metrics to observe S3 GET request connection lifetime and bytes consumed, visible in `system.histogram_metrics` and the Prometheus endpoint. [#102058](https://github.com/ClickHouse/ClickHouse/pull/102058) ([Sema Checherinda](https://github.com/CheSema)). * `Date` and `Date32` values can now be added to `Time` and `Time64` values using the `+` operator, producing a `DateTime` or `DateTime64` result. For example, `SELECT toDate('2024-01-15') + toTime('14:30:25')` returns `2024-01-15 14:30:25`. The result is computed in the session timezone, and out-of-range results are handled according to the `date_time_overflow_behavior` setting. Closes [#95914](https://github.com/ClickHouse/ClickHouse/issues/95914). [#102421](https://github.com/ClickHouse/ClickHouse/pull/102421) ([Nihal Z. Miaji](https://github.com/nihalzp)). * The text index is now GA and stays enabled regardless of the `compatibility` setting, preventing unexpected disabling during backup restores or when running in compatibility mode. [#101518](https://github.com/ClickHouse/ClickHouse/pull/101518) ([Nikita Fomichev](https://github.com/fm4v)). #### Experimental Feature * Add `ALTER TABLE ... EXECUTE remove_orphan_files` for Iceberg tables to identify and remove unreferenced files from object storage. [#99127](https://github.com/ClickHouse/ClickHouse/pull/99127) ([murphy-4o](https://github.com/murphy-4o)). * Add `query_plan_optimize_join_order_randomize` setting that randomizes statistics used for join reordering, useful for testing. [#100643](https://github.com/ClickHouse/ClickHouse/pull/100643) ([Vladimir Cherkasov](https://github.com/vdimir)). * Add AI function support to ClickHouse, allowing users to call OpenAI and Anthropic endpoints using SQL. `aiGenerate` is included as the first such function. [#100831](https://github.com/ClickHouse/ClickHouse/pull/100831) ([George Larionov](https://github.com/george-larionov)). * Add AI functions: `aiClassify`, `aiExtract`, and `aiTranslate` for utilizing LLM APIs in ClickHouse. [#100832](https://github.com/ClickHouse/ClickHouse/pull/100832) ([George Larionov](https://github.com/george-larionov)). * Added `system.histogram_metric_log`, a new system table that periodically snapshots all histogram metrics (e.g. S3/Azure latencies, keeper request processing stages durations). Also, the `value` column of `system.histogram_metrics` changes to `Float64` as it's more flexible and compatible with the Prometheus data model. [#103046](https://github.com/ClickHouse/ClickHouse/pull/103046) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). The table structure is likely to be changed in future releases. #### Performance Improvement * ClickHouse is now able to prune entire data parts in SELECT queries based on min/max statistics. [#94140](https://github.com/ClickHouse/ClickHouse/pull/94140) ([zoomxi](https://github.com/zoomxi)). * Reduce lock contention during readonly operations on ReplicatedMergeTree tables with finished mutations. [#95771](https://github.com/ClickHouse/ClickHouse/pull/95771) ([Eduard Karacharov](https://github.com/korowa)). * Respect `optimize_read_in_order` when reading projections. Closes [#89453](https://github.com/ClickHouse/ClickHouse/issues/89453). [#95885](https://github.com/ClickHouse/ClickHouse/pull/95885) ([Andrey Zvonov](https://github.com/zvonand)). * Small set of improvements in Hash Join and Concurrent Hash Join. [#96663](https://github.com/ClickHouse/ClickHouse/pull/96663) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Optimize `DISTINCT` transform by disabling `LowCardinality` columns optimization when input data is almost distinct. [#97113](https://github.com/ClickHouse/ClickHouse/pull/97113) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Performance optimization for `LIKE` queries from [#97723](https://github.com/ClickHouse/ClickHouse/issues/97723). Now these queries can use text indices. [#98149](https://github.com/ClickHouse/ClickHouse/pull/98149) ([Elmi Ahmadov](https://github.com/ahmadov)). * Vectorized math functions (`exp`, `log`, `sigmoid`, `tanh`) are now accelerated on AArch64 (using NEON/SVE) and on FreeBSD/Darwin, where they previously used a slower scalar fallback. [#98230](https://github.com/ClickHouse/ClickHouse/pull/98230) ([Raúl Marín](https://github.com/Algunenano)). * Queries filtering on `MergeTree` primary key columns with regexp alternations over literal strings, such as `^(abc-1|abc-2)`, can now use primary key pruning when the alternatives share a common prefix. [#98988](https://github.com/ClickHouse/ClickHouse/pull/98988) ([Yash ](https://github.com/Onyx2406)). * Generalize `ORDER BY ... LIMIT` top-k dynamic filtering to support `Nullable`, `String`, and `COLLATE` types. [#99033](https://github.com/ClickHouse/ClickHouse/pull/99033) ([murphy-4o](https://github.com/murphy-4o)). * Speedup hash join on `Int32` and `Int64` keys with small range by using a direct-index hash table. [#99275](https://github.com/ClickHouse/ClickHouse/pull/99275) ([Hechem Selmi](https://github.com/m-selmi)). * Faster discontinuous queries for `LowCardinality` columns with a single dictionary. [#99285](https://github.com/ClickHouse/ClickHouse/pull/99285) ([Ivan Babrou](https://github.com/bobrik)). * Speed up `var*Stable` and `stddev*Stable` functions for `Float64` columns by devirtualizing the inner loop. Note: this enables compiler optimizations (FMA/registers) that alter floating-point results at the ULP level. [#99460](https://github.com/ClickHouse/ClickHouse/pull/99460) ([Riyane El Qoqui](https://github.com/riyaneel)). * Use optimised Firedancer base58 encode for inputs of 32/64 bytes (automatic for `base58Encode`). Allow using optimised base58 decode if decoded result is 32/64 bytes (explicit with `base58Decode('...', 32)` or alike). [#99461](https://github.com/ClickHouse/ClickHouse/pull/99461) ([Joanna Hulboj](https://github.com/jh0x)). * Enable linker section-based optimizations (`-ffunction-sections`, `-fdata-sections`, `--icf=all`) to reduce binary size and improve instruction cache utilization. [#99474](https://github.com/ClickHouse/ClickHouse/pull/99474) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix negative scaling for short queries with aggregation on machines with many cores. When a query reads few marks, the pipeline no longer expands to `max_threads` after aggregation, avoiding overhead from mostly-empty streams. [#99493](https://github.com/ClickHouse/ClickHouse/pull/99493) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve the performance of queries with parallel replicas by correctly selecting the reading task size. [#99801](https://github.com/ClickHouse/ClickHouse/pull/99801) ([Nikita Taranov](https://github.com/nickitat)). * Allow prefetching when reading a remote file through the userspace page cache. [#99919](https://github.com/ClickHouse/ClickHouse/pull/99919) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Avoid unnecessary computation of String `.size` subcolumn during subcolumns enumeration. [#99941](https://github.com/ClickHouse/ClickHouse/pull/99941) ([Pavel Kruglov](https://github.com/Avogar)). * Make clickhouse-client's progress bar less jittery when working from a hotel with clusters with a very large number of replicas. [#100145](https://github.com/ClickHouse/ClickHouse/pull/100145) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Start `MemoryWorker` in `clickhouse-local` when page cache is enabled, so that the userspace page cache can be actually used. [#100306](https://github.com/ClickHouse/ClickHouse/pull/100306) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Optimize queries by pushing the `LIMIT` clause down into the `UNION ALL`. [#100364](https://github.com/ClickHouse/ClickHouse/pull/100364) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add JIT compilation support for `String` and `FixedString` column comparisons in `ORDER BY`, improving merge-phase sort performance by 6–17% for string-heavy sort keys. Co-authored with @lgbo-ustc. [#100577](https://github.com/ClickHouse/ClickHouse/pull/100577) ([Raúl Marín](https://github.com/Algunenano)). * When `read_in_order_use_virtual_row` is enabled together with the new `read_in_order_use_virtual_row_per_block` setting, virtual row boundary information is now emitted after each block read from `MergeTree`, allowing the merge to reprioritize sources mid-stream for parts whose data is fully filtered out by `WHERE`/`PREWHERE`/`JOIN`. Close [#99945](https://github.com/ClickHouse/ClickHouse/issues/99945). [#100603](https://github.com/ClickHouse/ClickHouse/pull/100603) ([Vladimir Cherkasov](https://github.com/vdimir)). * Faster Float-to-String conversion for large integer values by extending the `itoa` fast path with dragonbox-compatible rounding. [#100649](https://github.com/ClickHouse/ClickHouse/pull/100649) ([Raúl Marín](https://github.com/Algunenano)). * Replace `dragonbox` with `zmij` for 1.5x-3x faster Float-to-String conversion. [#100650](https://github.com/ClickHouse/ClickHouse/pull/100650) ([Raúl Marín](https://github.com/Algunenano)). * Faster `Int128`/`UInt128` to string conversion by replacing software division with Barrett reduction and unrolling the conversion loop. [#100671](https://github.com/ClickHouse/ClickHouse/pull/100671) ([Raúl Marín](https://github.com/Algunenano)). * Avoid spawning redundant threads in `uniqExact` parallel merge. [#100686](https://github.com/ClickHouse/ClickHouse/pull/100686) ([Jiebin Sun](https://github.com/jiebinn)). * Add batch parallel merge for `uniqExact`. [#100687](https://github.com/ClickHouse/ClickHouse/pull/100687) ([Jiebin Sun](https://github.com/jiebinn)). * Better parallelization of queries with simple views (with underlying `MergeTree` table) executed with parallel replicas. [#100815](https://github.com/ClickHouse/ClickHouse/pull/100815) ([Igor Nikonov](https://github.com/devcrafter)). * Implements support of parallel replicas over simple views (including eligible `UNION ALL` views over `MergeTree` tables) when `parallel_replicas_allow_view_over_mergetree=1`. This allows to parallelize view's outer query instead of inner one which increases query parallelization across nodes. [#100958](https://github.com/ClickHouse/ClickHouse/pull/100958) ([Igor Nikonov](https://github.com/devcrafter)). * Optimise reading in order of the primary key for `full_sorting_merge` when filters with `IN` are present in the query plan. [#101261](https://github.com/ClickHouse/ClickHouse/pull/101261) ([Nikita Taranov](https://github.com/nickitat)). * Optimization allocations/deallocations by caching sampling settings instead of traverse all memory tracker hierarchy. [#101267](https://github.com/ClickHouse/ClickHouse/pull/101267) ([Azat Khuzhin](https://github.com/azat)). * Fix significant INSERT performance regression when `deduplicate_insert = 'enable'` (default since 26.2) by deferring data hash computation from squashing to the sink and using batch column hashing via `updateHashWithValueRange`, reducing overhead from ~2.5s to ~0.5s for 5M rows with 22 columns. [#101494](https://github.com/ClickHouse/ClickHouse/pull/101494) ([Sema Checherinda](https://github.com/CheSema)). * Reduce profiled lock overhead by using `try_lock` to avoid timing uncontended acquisitions and removing hold-time measurement. [#101502](https://github.com/ClickHouse/ClickHouse/pull/101502) ([Antonio Andelic](https://github.com/antonio2368)). * Replace hand-written AVX-512 intrinsics in `arrayDotProduct` with platform-independent auto-vectorizable loops, adding AVX2 and ARM NEON support. [#101571](https://github.com/ClickHouse/ClickHouse/pull/101571) ([Peng](https://github.com/fastio)). * Improve performance in `INSERT VALUES` for `Map`, `Array`, and `Tuple` columns when values are passed as escaped strings (e.g. `'{\'key\':1}'`), avoiding unnecessary fallback to the SQL expression parser. [#102119](https://github.com/ClickHouse/ClickHouse/pull/102119) ([Joanna Hulboj](https://github.com/jh0x)). * Fixed excessive `RabbitMQ` table engine CPU usage. [#102711](https://github.com/ClickHouse/ClickHouse/pull/102711) ([Jaap Elst](https://github.com/jaapieaapie1)). * The JOIN order optimizer now infers transitive equi-join predicates from existing join conditions. For example, given `A.x = B.x AND B.x = C.x`, the equivalence `A.x = C.x` is recognized, allowing the optimizer to consider direct joins between transitively-connected tables. This can improve plan quality for star and snowflake schemas where dimension tables connect through a shared fact table. The feature is controlled by the new `enable_join_transitive_predicates` setting (off by default). [#98479](https://github.com/ClickHouse/ClickHouse/pull/98479) ([Alexander Gololobov](https://github.com/davenger)). * Optimize `TRUNCATE DATABASE TABLES LIKE` by pre-cancelling merges in parallel. [#98597](https://github.com/ClickHouse/ClickHouse/pull/98597) ([Shaohua Wang](https://github.com/tiandiwonder)). * Add monotonicity support for multiply, enabling primary key pruning for `key * constant` expressions. [#98983](https://github.com/ClickHouse/ClickHouse/pull/98983) ([Amos Bird](https://github.com/amosbird)). * Cache dictionaries no longer take an exclusive lock in `hasKeys`; this reduces lock contention by using a shared lock for cache reads. [#100796](https://github.com/ClickHouse/ClickHouse/pull/100796) ([liuguangliang](https://github.com/sourcelliu)). * Inline VIEW subquery in the query tree to allow more optimisations to be applied to the VIEW. [#100830](https://github.com/ClickHouse/ClickHouse/pull/100830) ([Dmitry Novik](https://github.com/novikd)). * Optimize cache loading on server startup. [#101500](https://github.com/ClickHouse/ClickHouse/pull/101500) ([Kseniia Sumarokova](https://github.com/kssenii)). * Implement lazy column materialization for ReplacingMergeTree with FINAL in case the predicate is selective enough. [#101647](https://github.com/ClickHouse/ClickHouse/pull/101647) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Re-enable the `optimize_rewrite_array_exists_to_has` optimization (off by default since 23.10). It rewrites `arrayExists(x -> x = elem, arr)` into the much faster `has(arr, elem)` and now correctly skips the rewrite when the array element type and `elem` are not compatible for `has` (e.g. `Date` vs `String`), so previously breaking queries continue to work. Closes [#71431](https://github.com/ClickHouse/ClickHouse/issues/71431). [#100944](https://github.com/ClickHouse/ClickHouse/pull/100944) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Improvement * Improved EXPLAIN PLAN pretty=1 output: print top-level query output columns, show join relation labels/symbols with estimated result rows and locality, and include per-step output columns for join/source steps. The changes cover Information Deficit part from [#98117](https://github.com/ClickHouse/ClickHouse/issues/98117). [#99462](https://github.com/ClickHouse/ClickHouse/pull/99462) ([Kirill Kopnev](https://github.com/Fgrtue)). * Add MergeTree table setting `share_nested_offsets` (default `true`). When set to `false`, Array columns with dotted names (e.g. `n.a`, `n.b`) are treated as independent columns instead of sharing offset files and validating equal array sizes as part of legacy `Nested` semantics. [#98416](https://github.com/ClickHouse/ClickHouse/pull/98416) ([Amos Bird](https://github.com/amosbird)). * Users can now specify multiple authentication methods in users.xml/yaml configuration (in SQL it was always possible). [#91998](https://github.com/ClickHouse/ClickHouse/pull/91998) ([Flip-Liquid](https://github.com/Flip-Liquid)). * Auto reload Raft inter-node connections which are using TLS. [#93455](https://github.com/ClickHouse/ClickHouse/pull/93455) ([Evgeny](https://github.com/evkuzin)). * Extend `cast_keep_nullable` to work with Dynamic/JSON types. When set, casting NULL from types that can be Nullable will return NULL, otherwise NULL will throw `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN` error. [#96504](https://github.com/ClickHouse/ClickHouse/pull/96504) ([Seva Potapov](https://github.com/seva-potapov)). * Reduced the memory footprint from the internal data structures (`ISerialization` objects) by introducing an object pool. [#96563](https://github.com/ClickHouse/ClickHouse/pull/96563) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Add support of `password` and `identity` fields to keeper-client XML config. [#96800](https://github.com/ClickHouse/ClickHouse/pull/96800) ([Grigorii Sokolik](https://github.com/GSokol)). * Improve `Iceberg` writes for the unity catalog. [#98162](https://github.com/ClickHouse/ClickHouse/pull/98162) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Add setting `finalize_projection_parts_synchronously` to allow synchronous finalization of projection parts during INSERT, reducing peak memory usage for tables with many projections while preserving existing async behavior by default. [#98228](https://github.com/ClickHouse/ClickHouse/pull/98228) ([Amos Bird](https://github.com/amosbird)). * Add `projections_duration_ms` column to `system.part_log` that records per-projection merge/rebuild duration in milliseconds. [#98292](https://github.com/ClickHouse/ClickHouse/pull/98292) ([Amos Bird](https://github.com/amosbird)). * Improve canceling queries using ExpressionTransform and NumbersRangedSource by KILL QUERY and cancel query (Ctrl+C) in clickhouse-client. [#98908](https://github.com/ClickHouse/ClickHouse/pull/98908) ([Roman Vasin](https://github.com/rvasin)). * Replace the hardcoded `source_table_engines` list with a runtime lookup via `StorageFactory::getAllStorages()`. This adds access checks for some missing table engines and closes [#71544](https://github.com/ClickHouse/ClickHouse/issues/71544). [#98984](https://github.com/ClickHouse/ClickHouse/pull/98984) ([pufit](https://github.com/pufit)). * Add a setting to control type mismatch behavior for Variant and Dynamic (throw or return null). [#99085](https://github.com/ClickHouse/ClickHouse/pull/99085) ([Bharat Nallan](https://github.com/bharatnc)). * Improve `Iceberg` and Spark compatibility: fix inconsistent path handling caused by mixed usage of storage paths and metadata paths; enforce that `Iceberg` tables write down a table location that is either a URL or an absolute path; add a fallback for counting file sizes in `Azure` because some ClickHouse readers don't support byte counting after traversal; handle `version-hint.txt` in a manner compatible with Spark; introduce type-level abstractions that make it harder to mix up path types in the future; add tests for `Azure` and `Local` that verify cross-engine interoperability without intermediate uploading/downloading; fix usage of position deletes, which previously relied on path inference heuristics where that approach is inappropriate. [#99163](https://github.com/ClickHouse/ClickHouse/pull/99163) ([Daniil Ivanik](https://github.com/divanik)). [#100420](https://github.com/ClickHouse/ClickHouse/pull/100420) ([Daniil Ivanik](https://github.com/divanik)). * Fix possible race condition in `IPartitionStrategy::cached_result` introduced in https://github.com/ClickHouse/ClickHouse/pull/92844. [#99400](https://github.com/ClickHouse/ClickHouse/pull/99400) ([Arthur Passos](https://github.com/arthurpassos)). * Users can now write ClickHouse Interval datatypes in the Arrow Format. [#99519](https://github.com/ClickHouse/ClickHouse/pull/99519) ([Peter Nguyen](https://github.com/petern48)). * Adds native support for importing and exporting `UUID` data types in `Arrow` and `Parquet` formats. Users can now directly query and transfer UUID data between ClickHouse and other data tools without requiring manual string conversions or workarounds. Automated logical inference for top-level UUIDs, and support for explicit schema hint for nested UUIDs. [#99521](https://github.com/ClickHouse/ClickHouse/pull/99521) ([Ivan](https://github.com/ivanmantova)). * Support `7z` archives on object storage. Closes [#70968](https://github.com/ClickHouse/ClickHouse/issues/70968). [#99600](https://github.com/ClickHouse/ClickHouse/pull/99600) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `ObjectStorageListedObjects`, `ObjectStorageGlobFilteredObjects`, `ObjectStoragePredicateFilteredObjects`, and `ObjectStorageReadObjects` ProfileEvents for introspection of object storage (`S3`, `Azure`, etc.) file listing and reading pipeline. [#99778](https://github.com/ClickHouse/ClickHouse/pull/99778) ([Sema Checherinda](https://github.com/CheSema)). * Fix `merge` table function failing with `UNKNOWN_IDENTIFIER` error when querying columns not present in all underlying distributed/remote tables. [#99833](https://github.com/ClickHouse/ClickHouse/pull/99833) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Now we include commit time in total mutation execution time metric for ReplicatedMergeTree. It was lost after [#96376](https://github.com/ClickHouse/ClickHouse/issues/96376). [#99936](https://github.com/ClickHouse/ClickHouse/pull/99936) ([alesapin](https://github.com/alesapin)). * Add a write-ahead log for blob objects pending removal in `MetadataStorageFromDisk`, improving durability and consistency between metadata and remote object storage when objects are deleted. [#100019](https://github.com/ClickHouse/ClickHouse/pull/100019) ([Maksim Kita](https://github.com/kitaisreal)). * Disable AI SQL generation (`??` command) in the embedded client (SSH and WebSocket protocols) to prevent access to the server's environment variables. [#100290](https://github.com/ClickHouse/ClickHouse/pull/100290) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Change the interface for Iceberg inserts with the catalog. Deprecate settings: `storage_catalog_type`, `storage_aws_access_key_id`, etc. [#100334](https://github.com/ClickHouse/ClickHouse/pull/100334) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Render tabs as 4 spaces when pasting into clickhouse-client. Closes [#100405](https://github.com/ClickHouse/ClickHouse/issues/100405). [#100416](https://github.com/ClickHouse/ClickHouse/pull/100416) ([Raúl Marín](https://github.com/Algunenano)). * Avoid scanning the whole remote data lake catalog for “Maybe you meant …” table hints when `show_data_lake_catalogs_in_system_tables` is disabled. [#100452](https://github.com/ClickHouse/ClickHouse/pull/100452) ([Alsu Giliazova](https://github.com/alsugiliazova)). * Apply `distributed_index_analysis_min_indexes_bytes_to_activate` after partition pruning. [#100477](https://github.com/ClickHouse/ClickHouse/pull/100477) ([Azat Khuzhin](https://github.com/azat)). * Fix assertion failure in Parquet bloom filter push down when using empty IN/NOT IN clauses. [#100543](https://github.com/ClickHouse/ClickHouse/pull/100543) ([zoomxi](https://github.com/zoomxi)). * MinMax column statistics now store the minimum and maximum values as Field (typed) instead of Float64. The serialized format includes the column type name alongside the values. The statistics file version is bumped to V2; files written by older versions require re-materialization (ALTER TABLE … MATERIALIZE STATISTICS ALL). fix [#53140](https://github.com/ClickHouse/clickhouse-private/issues/53140). [#100605](https://github.com/ClickHouse/ClickHouse/pull/100605) ([Han Fei](https://github.com/hanfei1991)). * Update `cppkafka` to include fix for Consumer close deadlock. [#100612](https://github.com/ClickHouse/ClickHouse/pull/100612) ([Azat Khuzhin](https://github.com/azat)). * Object information used for parsing data files in Iceberg now contains the number of file rows and file size in bytes parsed from manifest file. [#100645](https://github.com/ClickHouse/ClickHouse/pull/100645) ([Daniil Ivanik](https://github.com/divanik)). * Add `use_separate_cache_arena` configuration parameter to be able to control separation of the cache memory arena. [#100664](https://github.com/ClickHouse/ClickHouse/pull/100664) ([Seva Potapov](https://github.com/seva-potapov)). * Adds native support for importing Apache Arrow's `StringView` and `BinaryView` data types into ClickHouse `String` columns, improving compatibility for Arrow-based ingestion. [#100762](https://github.com/ClickHouse/ClickHouse/pull/100762) ([Ivan](https://github.com/ivanmantova)). * A few Keeper server settings are now hot-reloaded if config file is changed at runtime: max_requests_batch_size, max_requests_batch_bytes_size, max_request_size, quorum_reads. [#100773](https://github.com/ClickHouse/ClickHouse/pull/100773) ([Michael Kolupaev](https://github.com/al13n321)). * Increment profile events `MemoryAllocatedWithoutCheck/MemoryAllocatedWithoutCheckBytes` in release build. [#100899](https://github.com/ClickHouse/ClickHouse/pull/100899) ([Pavel Kruglov](https://github.com/Avogar)). * Cgroupv2 memory tracking now excludes `slab_reclaimable` from kernel memory, giving a more accurate measure of non-reclaimable memory usage. [#100901](https://github.com/ClickHouse/ClickHouse/pull/100901) ([Antonio Andelic](https://github.com/antonio2368)). * `use_partition_pruning = 0` now also disables `MinMax` index pruning and count optimization on partition key columns, in addition to disabling pruning based on partition keys. [#100904](https://github.com/ClickHouse/ClickHouse/pull/100904) ([Nihal Z. Miaji](https://github.com/nihalzp)). * `pretty=1` in `EXPLAIN [PLAN]` now prints expressions in a human readable format. [#100927](https://github.com/ClickHouse/ClickHouse/pull/100927) ([Kirill Kopnev](https://github.com/Fgrtue)). * `accurateCastOrNull` and `accurateCastOrDefault` now support `Tuple` target types, including nested `Tuples` with `Nullable` elements. Previously these functions rejected `Tuple` targets because `Tuple` could not be inside `Nullable`. Closes [#100820](https://github.com/ClickHouse/ClickHouse/issues/100820). [#100942](https://github.com/ClickHouse/ClickHouse/pull/100942) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix chart duplication in Play UI when switching between light and dark themes. [#101058](https://github.com/ClickHouse/ClickHouse/pull/101058) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Update chdig to [v26.3.1](https://github.com/azat/chdig/releases/tag/v26.3.1) (perfetto UI, sparklines to summary for CPU/Memory/Merges/Queries), system.warnings, regexp search in logs). [#101092](https://github.com/ClickHouse/ClickHouse/pull/101092) ([Azat Khuzhin](https://github.com/azat)). Update chdig to [v26.4.3](https://github.com/azat/chdig/releases/tag/v26.4.3) (perfetto improvements, fixes for sharing via pastila.nl, flamegraph diffs, change settings in realtime). [#103145](https://github.com/ClickHouse/ClickHouse/pull/103145) ([Azat Khuzhin](https://github.com/azat)). * You can now have a trailing comma in the `WITH` clause before a `SELECT` query. [#101093](https://github.com/ClickHouse/ClickHouse/pull/101093) ([Aruj Bansal](https://github.com/arujbansal)). * Add `compress_per_column_in_compact_parts` MergeTree setting to control how compressed blocks are organized within Compact parts. When `true` (default, preserving current behavior), each column starts a new compressed block, allowing selective decompression. When `false`, all columns within a granule are packed into the same compressed block, improving compression ratio and read performance for workloads that always read all columns. [#101114](https://github.com/ClickHouse/ClickHouse/pull/101114) ([Amos Bird](https://github.com/amosbird)). * Show table info balloon in Play UI only when hovering over the table name, not the entire row. [#101118](https://github.com/ClickHouse/ClickHouse/pull/101118) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add engine-specific icons and improve table list UX in the Play UI sidebar. [#101134](https://github.com/ClickHouse/ClickHouse/pull/101134) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support `Nullable(Tuple)` for `Arrow`, `ArrowStream`, `ORC`, legacy `Parquet` formats. [#101272](https://github.com/ClickHouse/ClickHouse/pull/101272) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Display TOTALS row as a table footer in the web UI (play.html). [#101286](https://github.com/ClickHouse/ClickHouse/pull/101286) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support multi-query mode in the web UI (`play.html`): run multiple queries at once with parallel execution of `SELECT`-like queries and per-query result display. [#101290](https://github.com/ClickHouse/ClickHouse/pull/101290) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix column resize in play.html web UI after the result table was refactored into a web component. [#101295](https://github.com/ClickHouse/ClickHouse/pull/101295) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add ability limit amount of jemalloc profile flushes on MEMORY_LIMIT_EXCEEDED per time interval. [#101396](https://github.com/ClickHouse/ClickHouse/pull/101396) ([Azat Khuzhin](https://github.com/azat)). * Added keeper settings `nuraft_streaming_mode` (by default `false`), `nuraft_max_log_gap_in_stream`, `nuraft_max_bytes_in_flight_in_stream`. Closes [#90743](https://github.com/ClickHouse/ClickHouse/issues/90743). [#101427](https://github.com/ClickHouse/ClickHouse/pull/101427) ([Kseniia Sumarokova](https://github.com/kssenii)). * Added `CGroupMemoryUsedWithoutPageCache` async metric that reports cgroup memory usage excluding both the kernel OS page cache and the ClickHouse userspace page cache, mirroring `MemoryResidentWithoutPageCache`. Also clarified the `CGroupMemoryUsed` metric description. [#101513](https://github.com/ClickHouse/ClickHouse/pull/101513) ([Francesco Ciocchetti](https://github.com/primeroz)). * Add parser-level syntactic sugar for the SQL standard `OVERLAY` function syntax. The `overlay` function already exists; this adds support for the keyword-based form using `PLACING`, `FROM`, and `FOR` as separators. [#101681](https://github.com/ClickHouse/ClickHouse/pull/101681) ([Desel72](https://github.com/Desel72)). * Added column alias `INDEX_LENGTH` to system table `information_schema.tables`, analogous to existing uppercase aliases in this table. [#101705](https://github.com/ClickHouse/ClickHouse/pull/101705) ([Robert Schulze](https://github.com/rschu1ze)). * System table `information_schema.tables` now ignores inactive table parts. This makes the shown table size values more realistic. [#101706](https://github.com/ClickHouse/ClickHouse/pull/101706) ([Robert Schulze](https://github.com/rschu1ze)). * The `ngrams` function now rejects invalid ngram lengths. Example: `SELECT ngrams('abc', 0)` now returns an error. [#101922](https://github.com/ClickHouse/ClickHouse/pull/101922) ([Robert Schulze](https://github.com/rschu1ze)). * A follow-up for [#91820](https://github.com/ClickHouse/ClickHouse/issues/91820) and [#90837](https://github.com/ClickHouse/ClickHouse/issues/90837): filter unsupported algorithms from the error message; run FIPS-specific tests in FIPS builds. [#102067](https://github.com/ClickHouse/ClickHouse/pull/102067) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Limit cell height to three lines in the Web UI (`play.html`), with expandable cells on click. [#102154](https://github.com/ClickHouse/ClickHouse/pull/102154) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added new option that allows to force (virtual/path) style for S3 endpoints. Resolves [#82019](https://github.com/ClickHouse/ClickHouse/issues/82019); [#76007](https://github.com/ClickHouse/ClickHouse/issues/76007) Continue of https://github.com/ClickHouse/ClickHouse/pull/83168. [#102378](https://github.com/ClickHouse/ClickHouse/pull/102378) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The `restore_replace_external_engines_to_null` setting now also skips restoring databases with external engines (e.g. `DataLakeCatalog`, `MySQL`, `PostgreSQL`, `S3`) instead of failing or initiating external connections. [#102400](https://github.com/ClickHouse/ClickHouse/pull/102400) ([Nikita Fomichev](https://github.com/fm4v)). * Add text index analysis support for the `hasPhrase` function via the `HINT` mode. [#102438](https://github.com/ClickHouse/ClickHouse/pull/102438) ([Elmi Ahmadov](https://github.com/ahmadov)). * Treat STATISTICS as read-only in ColumnDependency to fix LOGICAL_ERROR during MATERIALIZE STATISTICS ALL. [#102627](https://github.com/ClickHouse/ClickHouse/pull/102627) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Create and populate `system.asynchronous_metric_log` in keeper-as-server mode. [#102664](https://github.com/ClickHouse/ClickHouse/pull/102664) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add `default_system_log_flush_policy.skip_alias_columns` config option to allow omitting ALIAS columns from system log tables, fixing S3-backed system logs that reject ALIAS columns. [#102669](https://github.com/ClickHouse/ClickHouse/pull/102669) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Don't enable auto statistics for system tables. They rarely have chance to use them. [#102862](https://github.com/ClickHouse/ClickHouse/pull/102862) ([Han Fei](https://github.com/hanfei1991)). * Support `array` tokenizer for the LIKE optimization. [#102880](https://github.com/ClickHouse/ClickHouse/pull/102880) ([Elmi Ahmadov](https://github.com/ahmadov)). * Send MemoryAllocatedWithoutCheck even in release builds. [#103064](https://github.com/ClickHouse/ClickHouse/pull/103064) ([Azat Khuzhin](https://github.com/azat)). * Expose per-thread untracked_memory in system.stack_trace. [#103065](https://github.com/ClickHouse/ClickHouse/pull/103065) ([Azat Khuzhin](https://github.com/azat)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fix `Block structure mismatch in stream` error caused by unnecessary columns returned from Lazy materialization. Fixes [#95191](https://github.com/ClickHouse/ClickHouse/issues/95191). [#96682](https://github.com/ClickHouse/ClickHouse/pull/96682) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix a logical error with data masking policy query with `ON CLUSTER`. [#97594](https://github.com/ClickHouse/ClickHouse/pull/97594) ([Bharat Nallan](https://github.com/bharatnc)). * Fix a bug when using Unity catalog on top of GCS. [#98456](https://github.com/ClickHouse/ClickHouse/pull/98456) ([Melvyn Peignon](https://github.com/melvynator)). * `DataLakeCatalog` now respects the server's `http_forbid_headers` configuration when validating the `auth_header` setting. [#98827](https://github.com/ClickHouse/ClickHouse/pull/98827) ([Michael Anastasakis](https://github.com/michael-anastasakis)). * Fix N+1 `HeadObject` calls for S3 brace-expansion globs. [#99219](https://github.com/ClickHouse/ClickHouse/pull/99219) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Validate setting changes in create queries when the engine itself also supports settings. [#99279](https://github.com/ClickHouse/ClickHouse/pull/99279) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed `ALTER TABLE UPDATE/DELETE` failing with `Missing columns` error when a table has a MATERIALIZED column whose expression depends on an EPHEMERAL column. [#99281](https://github.com/ClickHouse/ClickHouse/pull/99281) ([Yash ](https://github.com/Onyx2406)). * Credentials in JDBC, ODBC, and NATS connection strings are now masked in query logs and `SHOW CREATE` output, preventing accidental exposure of sensitive information. For URI-style connection strings (e.g. `{scheme}://{user}:{password}@{host}`), only the password portion is masked while the rest remains visible for easier debugging. The `nats_token` setting is now also masked. [#99344](https://github.com/ClickHouse/ClickHouse/pull/99344) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix parseDateTimeBestEffort incorrectly parsing words starting with month prefixes in DD-month-YYYY format. Closes [#99345](https://github.com/ClickHouse/ClickHouse/issues/99345#event-23517658314). [#99350](https://github.com/ClickHouse/ClickHouse/pull/99350) ([Pavel Kruglov](https://github.com/Avogar)). * Fix ignoring of TABLE_UUID_MISMATCH for non analyzer. [#99380](https://github.com/ClickHouse/ClickHouse/pull/99380) ([Azat Khuzhin](https://github.com/azat)). * Fix a bug where explicit settings sent alongside `compatibility` in the same request could be silently ignored when their value matched the server default. [#99402](https://github.com/ClickHouse/ClickHouse/pull/99402) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fixes cases where numbers with leading zeros in hive partitioning path were causing errors. Fixes [#98801](https://github.com/ClickHouse/ClickHouse/issues/98801). [#99458](https://github.com/ClickHouse/ClickHouse/pull/99458) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix heap-use-after-free when a table is dropped concurrently with a running read query (19 occurrences in CI over the last 90 days). [#99483](https://github.com/ClickHouse/ClickHouse/pull/99483) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed a bug in Keeper where a read request could get stuck (causing session to time out) if a different unrelated session on the same server was closed at just the wrong moment. [#99484](https://github.com/ClickHouse/ClickHouse/pull/99484) ([Michael Kolupaev](https://github.com/al13n321)). * Validate column structure before applying patches. [#99531](https://github.com/ClickHouse/ClickHouse/pull/99531) ([Seva Potapov](https://github.com/seva-potapov)). * Fix vertical merge `rows_sources` assertion failure when `SYSTEM STOP/START MERGES` toggles rapidly during merge of table with `Dynamic` columns. [#99532](https://github.com/ClickHouse/ClickHouse/pull/99532) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect partition pruning for `toWeek()` that caused queries with `WHERE toWeek(date, mode) = N` to return empty results for weeks 49-52 on tables partitioned by `toYYYYMM(date)`. [#99542](https://github.com/ClickHouse/ClickHouse/pull/99542) ([Takumi Hara](https://github.com/takumihara)). * Fix exception in functions operating on ColumnReplicated with unreferenced rows produced by JOIN. [#99564](https://github.com/ClickHouse/ClickHouse/pull/99564) ([Hechem Selmi](https://github.com/m-selmi)). * Fix `CLEAR COLUMN` not rebuilding projections and not reevaluation materialized columns that depend on the cleared column, which could cause exceptions or data corruption during subsequent merges. [#99565](https://github.com/ClickHouse/ClickHouse/pull/99565) ([Desel72](https://github.com/Desel72)). * Fixed an exception (`Bad get: has Tuple, actual type String`) in `ConditionSelectivityEstimator` when a query uses IN with a single scalar query parameter (e.g. `WHERE col IN ({p:String})`) on a table that has column statistics and `use_statistics` is enabled. [#99614](https://github.com/ClickHouse/ClickHouse/pull/99614) ([Ilya Yatsishin](https://github.com/qoega)). * Part with unknown projections should not be marked as lost forever. [#99623](https://github.com/ClickHouse/ClickHouse/pull/99623) ([Sema Checherinda](https://github.com/CheSema)). * Fix a rare logical error exception during vertical merge when `SYSTEM STOP MERGES` and `SYSTEM START MERGES` are executed concurrently. [#99628](https://github.com/ClickHouse/ClickHouse/pull/99628) ([Desel72](https://github.com/Desel72)). * Fix dangling reference in injectRequiredColumns causing crash during merge. [#99679](https://github.com/ClickHouse/ClickHouse/pull/99679) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix undefined behavior in Avro format reader when reading numeric values that overflow the target column type. Now queries fail on overflows instead of silently producing incorrect values. [#99697](https://github.com/ClickHouse/ClickHouse/pull/99697) ([asyablue22](https://github.com/asyablue22)). * Fix parsing of shell-style quotes in arguments for the `executable` table function. [#99794](https://github.com/ClickHouse/ClickHouse/pull/99794) ([Nikita Semenov](https://github.com/leftmain)). * Fix false-positive abort in `NativeReader` when deserializing a Native format stream with a row-count mismatch: changed from `LOGICAL_ERROR` to `INCORRECT_DATA` so the error is handled as a data error rather than triggering `abort()` in sanitizer/debug builds. [#99822](https://github.com/ClickHouse/ClickHouse/pull/99822) ([Rahul Nair](https://github.com/motsc)). * Fix process abort in `Tuple` column deserialization when the serialization kind in the binary stream is `DETACHED`. [#99823](https://github.com/ClickHouse/ClickHouse/pull/99823) ([Rahul Nair](https://github.com/motsc)). * Fix false `LOGICAL_ERROR` exception during filesystem cache dynamic resize due to a race condition in SLRU sub-queue promotion. [#99850](https://github.com/ClickHouse/ClickHouse/pull/99850) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix async insert queries reporting zero `written_rows`, `read_rows`, and `result_rows` in `query_log` and client output. [#99879](https://github.com/ClickHouse/ClickHouse/pull/99879) ([Sema Checherinda](https://github.com/CheSema)). * Fix exception "Bad cast from type X to Y" in `KILL QUERY` when the internal query against system tables returns columns wrapped in `ColumnConst`. [#99881](https://github.com/ClickHouse/ClickHouse/pull/99881) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error with correlated subquery within untuple argument. [#99917](https://github.com/ClickHouse/ClickHouse/pull/99917) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixes an exception when calling `right`, `rightUTF8`, or other substring functions with a length of INT64_MIN (-9223372036854775808), which previously caused undefined behavior due to integer overflow. The functions now correctly report an ARGUMENT_OUT_OF_BOUND error. [#99934](https://github.com/ClickHouse/ClickHouse/pull/99934) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Now ClickHouse should properly handle spark-style tables (where we have full absolute path for each file or relative path to common table path). Fixes [#92348](https://github.com/ClickHouse/ClickHouse/issues/92348). [#99935](https://github.com/ClickHouse/ClickHouse/pull/99935) ([alesapin](https://github.com/alesapin)). * Fix "Inconsistent AST formatting" exception for `ALTER TABLE ... MODIFY QUERY` with nested subqueries containing `SETTINGS` when the `ALTER` itself also has `SETTINGS`. [#99938](https://github.com/ClickHouse/ClickHouse/pull/99938) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Revert [#97114](https://github.com/ClickHouse/ClickHouse/issues/97114) "Move join step row estimation before check for 1 child" due to suspected performance regression. [#99957](https://github.com/ClickHouse/ClickHouse/pull/99957) ([Alexander Gololobov](https://github.com/davenger)). * Fixes a bug where ClickHouse could skip files if the `Content-Length` header was missing in their HEAD request response (for example, because of decompressive transcoding in GCS). [#99971](https://github.com/ClickHouse/ClickHouse/pull/99971) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix assertion failure (exception in debug builds, incorrect results in release builds) when multiplying `NumericIndexedVector` aggregate states by an even integer constant, caused by self-XOR on aliased Roaring bitmaps in `pointwiseAddInplace`. [#99976](https://github.com/ClickHouse/ClickHouse/pull/99976) ([Desel72](https://github.com/Desel72)). * Prevent `Unexpected return type` exception in legacy filter pushdown through chained `JOIN USING` when key types change after join conversions. [#99999](https://github.com/ClickHouse/ClickHouse/pull/99999) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception "Unexpected node type for table expression ... Actual IDENTIFIER" when a scalar subquery is used inside an unresolved table function argument, e.g. `SELECT * FROM remote('localhost', view(SELECT 2 AS x), concat(x, (SELECT 1)))`. [#100014](https://github.com/ClickHouse/ClickHouse/pull/100014) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `INSERT` with `VALUES` failing when the data was followed by a trailing SQL comment (`--` or `/* */`) on the next line. The comment is now skipped instead of being parsed as another row. [#100016](https://github.com/ClickHouse/ClickHouse/pull/100016) ([Pratima Patel](https://github.com/pratimapatel2008)). * Fix exception in `arrayRemove` when comparing tuples with NULL components. [#100017](https://github.com/ClickHouse/ClickHouse/pull/100017) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix cross-user data leak in `system.asynchronous_inserts`: any user with `SELECT` on the table could see pending async insert entries belonging to other users. Entries are now filtered by the current user, unless the user has the `SHOW_USERS` privilege. [#100024](https://github.com/ClickHouse/ClickHouse/pull/100024) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fixes case where Time64 to UInt64 cast could clamp values to 24 hours. [#100025](https://github.com/ClickHouse/ClickHouse/pull/100025) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix local server crash when CREATE DICTIONARY has a definition with list value containing non-existing function. [#100036](https://github.com/ClickHouse/ClickHouse/pull/100036) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix `CSV`, `MsgPack` format not being able to parse `Nullable(Tuple)` properly. Closes [#99753](https://github.com/ClickHouse/ClickHouse/issues/99753). [#100038](https://github.com/ClickHouse/ClickHouse/pull/100038) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `CREATE VIEW` failing with `UNKNOWN_IDENTIFIER` when using a WITH function-expression alias (e.g. `tuple(...)`) as the right-hand side of `IN`. [#100042](https://github.com/ClickHouse/ClickHouse/pull/100042) ([Peng](https://github.com/fastio)). * Fix timeseries aggregate functions (e.g. `timeSeriesResampleToGridWithStaleness`) failing with `ILLEGAL_TYPE_OF_ARGUMENT` when used with `initializeAggregation` or `AggregatingMergeTree` under parallel replicas. [#100053](https://github.com/ClickHouse/ClickHouse/pull/100053) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Make correctly processing negative values inside NumericIndexedVectorDataBSI. [#100086](https://github.com/ClickHouse/ClickHouse/pull/100086) ([Daniil Ivanik](https://github.com/divanik)). * Fix `accurateCastOrDefault` and `to*OrDefault` functions not preserving Const column type for constant inputs. [#100132](https://github.com/ClickHouse/ClickHouse/pull/100132) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Omitted query parameters with `LowCardinality(Nullable(T))` type now correctly default to NULL, same as `Nullable(T)`. [#100144](https://github.com/ClickHouse/ClickHouse/pull/100144) ([Denys Melnyk](https://github.com/germiBest)). * Fix use-of-uninitialized-value in StringSearcher.h. [#100225](https://github.com/ClickHouse/ClickHouse/pull/100225) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Allow cancellation of scalar subqueries and other analysis-time pipelines via Ctrl+C. Previously, pressing Ctrl+C during a long-running scalar subquery had no effect until the subquery completed. Also fix the progress bar and JSON statistics to correctly report rows read during scalar subquery execution, both in `clickhouse-client` and `clickhouse-local`. Co-authored with @YjyJeff. [#100230](https://github.com/ClickHouse/ClickHouse/pull/100230) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a `LOGICAL_ERROR` exception in queries involving `Dynamic` columns with cross joins and runtime filters, caused by `ColumnVariant::filter` sharing variant column pointers instead of cloning them in the `hasOnlyNulls` optimization path. Closes https://github.com/ClickHouse/ClickHouse/pull/100147. [#100234](https://github.com/ClickHouse/ClickHouse/pull/100234) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed array of variant bug which may reinterpret data type upon calling `arrayFirst`/`arrayLast` function. For example, previously `Array(Variant(Date, Bool))` is converted to `Bool` when the actual underlying variant type is `Date`. [#100255](https://github.com/ClickHouse/ClickHouse/pull/100255) ([timothygk](https://github.com/timothygk)). * A few minor changes to functions: h3 functions now validate boundaries better; readWKB checks the size limits (a new setting, `max_wkb_geometry_elements`); random generator functions limit the maximum iterations in their computations. A follow-up for [#93543](https://github.com/ClickHouse/ClickHouse/issues/93543). [#100270](https://github.com/ClickHouse/ClickHouse/pull/100270) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an issue where cutURLParameter could incorrectly skip parameters when they appeared as substrings of other parameters. [#100280](https://github.com/ClickHouse/ClickHouse/pull/100280) ([Nikita Semenov](https://github.com/leftmain)). * Fix exception when Iceberg metadata file path setting contains a null byte. [#100283](https://github.com/ClickHouse/ClickHouse/pull/100283) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed the quadratic number of run queries when `distributed_index_analysis` is used with predicates containing `IN` subqueries. [#100287](https://github.com/ClickHouse/ClickHouse/pull/100287) ([Anton Popov](https://github.com/CurtizJ)). * Fix "Block structure mismatch" exception when using `GROUP BY ... WITH TOTALS HAVING` combined with `UNION DISTINCT` and nullable expressions. [#100293](https://github.com/ClickHouse/ClickHouse/pull/100293) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR exception in `estimateCompressionRatio` when `block_size_bytes` parameter is extremely large. [#100298](https://github.com/ClickHouse/ClickHouse/pull/100298) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Inconsistent AST formatting" exception in debug builds when using `GROUP BY CUBE(...) WITH ROLLUP` or similar combinations. [#100376](https://github.com/ClickHouse/ClickHouse/pull/100376) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when creating a view with column aliases and `SELECT *` or `EXCEPT`/`INTERSECT` queries. [#100386](https://github.com/ClickHouse/ClickHouse/pull/100386) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `DROP TABLE` hanging indefinitely on Kafka engine tables when consumers are stuck in a rebalance after a heartbeat error. [#100388](https://github.com/ClickHouse/ClickHouse/pull/100388) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ReadBuffer is canceled. Can't read from it.` exception in backup/restore operations using zip archives. [#100400](https://github.com/ClickHouse/ClickHouse/pull/100400) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `TOO_MANY_ROWS` exception for `SELECT count()` queries with `max_rows_to_read` / `force_primary_key` when data is split across multiple parts with non-aligned granule boundaries. [#100408](https://github.com/ClickHouse/ClickHouse/pull/100408) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `system.completions` to correctly filter databases, tables, and columns by access rights in all grant combinations: per-table, per-db, and per-column revoke. [#100432](https://github.com/ClickHouse/ClickHouse/pull/100432) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix SEGFAULT in NuRaft due to race condition. [#100444](https://github.com/ClickHouse/ClickHouse/pull/100444) ([Pablo Marcos](https://github.com/pamarcos)). * `min`/`max`/`argMin`/`argMax` now treat NaN consistently with `ORDER BY`: NaN is always skipped (returned only when all values are NaN). Previously, results depended on NaN position in the data due to IEEE 754 unordered comparison semantics. [#100448](https://github.com/ClickHouse/ClickHouse/pull/100448) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a copy-paste bug where `delta_lake_snapshot_end_version` set without `delta_lake_snapshot_start_version` was silently ignored instead of producing a `BAD_ARGUMENTS` error. [#100454](https://github.com/ClickHouse/ClickHouse/pull/100454) ([Mohammad Lareb Zafar](https://github.com/zlareb1)). * `StorageRabbitMQ::shutdown` is not idempotent (it unconditionally accesses weak pointers and then destroys the corresponding shared pointers), but is now called twice: once in `StreamingStorageRegistry` and then in `DatabaseCatalog`. This fix makes the method idempotent and adds defensive null checks. [#100455](https://github.com/ClickHouse/ClickHouse/pull/100455) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix `LOGICAL_ERROR` exception when using `accurateCastOrNull` with `QBit` target type. [#100470](https://github.com/ClickHouse/ClickHouse/pull/100470) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix LOGICAL_ERROR exception "Stream ... not found" when inserting into a table with nested `Array(JSON)` columns in wide parts with `optimize_on_insert=0`. [#100475](https://github.com/ClickHouse/ClickHouse/pull/100475) ([Pavel Kruglov](https://github.com/Avogar)). * Validate file entry paths in backup metadata to reject path traversal, absolute paths, and empty names during `RESTORE`. [#100483](https://github.com/ClickHouse/ClickHouse/pull/100483) ([Pablo Marcos](https://github.com/pamarcos)). * Fix `LIMIT m OFFSET n WITH TIES` syntax not working. This syntax is equivalent to `LIMIT n, m WITH TIES` which already worked. [#100491](https://github.com/ClickHouse/ClickHouse/pull/100491) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix exception "No set is registered for key" when using `IN` with `Nullable(Tuple)` columns that have named fields and `LowCardinality` elements. [#100523](https://github.com/ClickHouse/ClickHouse/pull/100523) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix heap-buffer-overflow in usearch `sorted_buffer_gt::insert()` that could crash or silently corrupt memory during vector similarity search. [#100537](https://github.com/ClickHouse/ClickHouse/pull/100537) ([Dustin Healy](https://github.com/dustinhealy)). * Fix `EXECUTE AS` ignoring `FORMAT` and `INTO OUTFILE` clauses specified in the query. [#100538](https://github.com/ClickHouse/ClickHouse/pull/100538) ([pufit](https://github.com/pufit)). * Fix inconsistent AST formatting for SAMPLE with query-level OFFSET. Closes [#100576](https://github.com/ClickHouse/ClickHouse/issues/100576). [#100579](https://github.com/ClickHouse/ClickHouse/pull/100579) ([Pavel Kruglov](https://github.com/Avogar)). * Fix polaris catalog with azure. Since 25.12 this catalog with azure started to add bucket at the beginning of the path. For example `abfss://polaris-polaris@.windows.net/polaris-polaris/` instead of `abfss://polaris-polaris@.windows.net/`. This PR cuts bucket in the path. [#100583](https://github.com/ClickHouse/ClickHouse/pull/100583) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix type mismatch exception in transform when default column is const on some blocks. Closes [#100574](https://github.com/ClickHouse/ClickHouse/issues/100574). [#100616](https://github.com/ClickHouse/ClickHouse/pull/100616) ([Pavel Kruglov](https://github.com/Avogar)). * Fixes NOT_FOUND_COLUMN_IN_BLOCK cases where the projection SELECT part has columns that do not exist in the original SELECT part of the query. Closes [#100194](https://github.com/ClickHouse/ClickHouse/issues/100194). [#100623](https://github.com/ClickHouse/ClickHouse/pull/100623) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Validate Npy format shape dimensions against file size and overflow limits to prevent denial of service from crafted `.npy` files with unreasonably large dimensions. Also reject empty shapes and cap per-row memory to 2 GiB. [#100625](https://github.com/ClickHouse/ClickHouse/pull/100625) ([Raúl Marín](https://github.com/Algunenano)). * Fix `session_timezone` being ignored when parsing `DateTime` values during async inserts (TCP) and all inserts over HTTP. [#100647](https://github.com/ClickHouse/ClickHouse/pull/100647) ([Sema Checherinda](https://github.com/CheSema)). * Allow passing sharding key to `cluster()` and `clusterAllReplicas()` table functions when using a table function as the source (e.g. `cluster('name', view(...), sharding_key)`). [#100665](https://github.com/ClickHouse/ClickHouse/pull/100665) ([Sergey Veletskiy](https://github.com/velom)). * Fix server crash (assertion failure) when using parametric aggregate functions with Array combinator and NULL arguments, such as `quantileIfArrayArray(0.5)([[NULL]], [[1]])`. [#100679](https://github.com/ClickHouse/ClickHouse/pull/100679) ([nerve-bot](https://github.com/nerve-bot)). * Fix exception when computing common supertype for empty and non-empty tuples with `use_variant_as_common_type` enabled. [#100699](https://github.com/ClickHouse/ClickHouse/pull/100699) ([Antonio Andelic](https://github.com/antonio2368)). * Server no longer fails to start when an Azure blob storage disk is configured but the endpoint is temporarily unreachable (e.g. DNS failure). [#100701](https://github.com/ClickHouse/ClickHouse/pull/100701) ([Raúl Marín](https://github.com/Algunenano)). * Fix undefined behavior in `positiveModulo` when the unsigned divisor does not fit in the signed result type. [#100705](https://github.com/ClickHouse/ClickHouse/pull/100705) ([Raúl Marín](https://github.com/Algunenano)). * Fix server crash (logical error "Unexpected return type from __topKFilter") when `use_top_k_dynamic_filtering` is enabled and the `ORDER BY` column has `Dynamic` or `Variant` type. [#100742](https://github.com/ClickHouse/ClickHouse/pull/100742) ([Groene AI](https://github.com/groeneai)). * Fix server crash when using `has()` function with PREWHERE/WHERE on a Tuple key containing LowCardinality elements. [#100760](https://github.com/ClickHouse/ClickHouse/pull/100760) ([Groene AI](https://github.com/groeneai)). * Fix `file_offset_of_buffer_end <= getFileSize()` assertion failure (exception in debug builds) when reading from `Log` or `StripeLog` tables on S3 object storage with concurrent writes. [#100763](https://github.com/ClickHouse/ClickHouse/pull/100763) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an exception in the statistics selectivity estimator when a WHERE clause contains a function expression (e.g. toDecimal64(col, 3)) on a table with statistics enabled. The estimator now skips such predicates instead of attempting an invalid type cast. [#100764](https://github.com/ClickHouse/ClickHouse/pull/100764) ([Han Fei](https://github.com/hanfei1991)). * Fixes a rare case where join with reordering can produce a wrong result. [#100790](https://github.com/ClickHouse/ClickHouse/pull/100790) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix incorrect `AggregateFunction` argument types in optimized trivial count, which caused `NUMBER_OF_ARGUMENTS_DOESNT_MATCH` exception when querying expressions like `count(v0 + v1)` on distributed tables. [#100794](https://github.com/ClickHouse/ClickHouse/pull/100794) ([YjyJeff](https://github.com/YjyJeff)). * Some catalogs can show some secrets in the `SETTINGS` section of `select * from system.databases` query result. This PR prevents such behavior. [#100800](https://github.com/ClickHouse/ClickHouse/pull/100800) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix undefined behavior (signed integer overflow) in `toStartOfInterval` when using Week, Quarter, or Year intervals with an origin argument and extreme interval values. [#100817](https://github.com/ClickHouse/ClickHouse/pull/100817) ([Raúl Marín](https://github.com/Algunenano)). * Fix `If`, `Distinct`, `DistinctIf`, `IfState ` aggregate function combinators with `Tuple` return type and one or more `Nullable` argument not being able to read older serialized states after introduction of `Nullable(Tuple)`. Closes [#98917](https://github.com/ClickHouse/ClickHouse/issues/98917). [#100826](https://github.com/ClickHouse/ClickHouse/pull/100826) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix segfault in s3Cluster and distributed queries due to connection pool use-after-free. [#100837](https://github.com/ClickHouse/ClickHouse/pull/100837) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix null pointer dereference segfault when loading dictionaries during server shutdown. `Context::getUserDefinedSQLObjectsStorage` (dereferences `user_defined_sql_objects_storage`) is called by dictionary threads concurrently with the main thread calling `Context::shutdown` (sets `user_defined_sql_objects_storage` to null). We need to make sure we disable future updates in the dictionaries loader, kill the currently running dictionary queries and join the dictionary loading threads - all before running `Context::shutdown`. Similar to what we do with normal queries. [#100839](https://github.com/ClickHouse/ClickHouse/pull/100839) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix buffer overflow in `ULIDStringToDateTime` when input contains non-ASCII bytes. [#100843](https://github.com/ClickHouse/ClickHouse/pull/100843) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix crash (`LOGICAL_ERROR`) when querying a `Merge` table (or `merge()` table function) that wraps multiple tables including a `Distributed` table, with `distributed_group_by_no_merge=1` enabled. [#100859](https://github.com/ClickHouse/ClickHouse/pull/100859) ([Groene AI](https://github.com/groeneai)). * Cast_keep_nullable when enabled will not throw when casting dynamic null to variant. [#100864](https://github.com/ClickHouse/ClickHouse/pull/100864) ([Seva Potapov](https://github.com/seva-potapov)). * Fix `clickhouse-keeper-client` `get`, `exists`, and `ls` commands printing duplicate `watch_id` error messages to stdout instead of stderr. [#100893](https://github.com/ClickHouse/ClickHouse/pull/100893) ([Mohammad Lareb Zafar](https://github.com/zlareb1)). * Fix exception in `intDiv`/`intDivOrZero` on arrays of nullable tuples, e.g. `SELECT intDiv([divide((1, 2), ... AND NULL)], 2)`. [#100895](https://github.com/ClickHouse/ClickHouse/pull/100895) ([Raúl Marín](https://github.com/Algunenano)). * Evaluate engine arguments for `StorageAlias` before storing the definition, so that expressions like `currentDatabase()` are resolved to literals before being saved to the database. [#100902](https://github.com/ClickHouse/ClickHouse/pull/100902) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix `processAndOptimizeTextIndexFunctions` when `query_plan_merge_expressions = 0`, where `ExpressionStep` is directly above `ReadFromMergeTree`. Fixes [#100879](https://github.com/ClickHouse/ClickHouse/issues/100879). [#100909](https://github.com/ClickHouse/ClickHouse/pull/100909) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Update replxx to include fix for out-of-bounds access in do_complete_line. [#100925](https://github.com/ClickHouse/ClickHouse/pull/100925) ([Azat Khuzhin](https://github.com/azat)). * Fix wrong results when JOIN with shard-by-PK optimization uses query condition cache and some parts are filtered out by cached conditions. [#100926](https://github.com/ClickHouse/ClickHouse/pull/100926) ([Groene AI](https://github.com/groeneai)). * Fix `divide` and `intDiv` returning `ILLEGAL_DIVISION` when used in filter expressions during index analysis in some cases. [#100928](https://github.com/ClickHouse/ClickHouse/pull/100928) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed "Target table doesn't exist" errors for materialized views with inner tables during async startup, caused by incorrect startup dependency ordering. [#100946](https://github.com/ClickHouse/ClickHouse/pull/100946) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix undefined behavior (signed integer overflow) in parseDateTimeBestEffort when parsing datetime strings with more than 18 fractional-second digits. [#100948](https://github.com/ClickHouse/ClickHouse/pull/100948) ([Vasily Chekalkin](https://github.com/bacek)). * Fixed a crash when using a text search index with an `IN` clause containing a tuple subquery, e.g. `WHERE (id, str) IN (SELECT (id, str) FROM ...)`, or when the number of columns in the subquery does not match the tuple on the left side of `IN`. [#100959](https://github.com/ClickHouse/ClickHouse/pull/100959) ([Anton Popov](https://github.com/CurtizJ)). * Fixed crash when building a polygon dictionary from a `MergeTree` table that uses sparse columns serialization. [#100964](https://github.com/ClickHouse/ClickHouse/pull/100964) ([Anton Popov](https://github.com/CurtizJ)). * Fix logical error "Invalid action query tree node" when using `INTERSECT ALL` / `UNION ALL` with constant-folded expressions. [#100977](https://github.com/ClickHouse/ClickHouse/pull/100977) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `sumCountOrDefault` aggregate function with one or more `Nullable` argument not being able to read older serialized states after introduction of `Nullable(Tuple)`. Closes [#100882](https://github.com/ClickHouse/ClickHouse/issues/100882). [#101021](https://github.com/ClickHouse/ClickHouse/pull/101021) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix crash (`Logical error: isConst/isSparse/isReplicated assertTypeEquality`) in merge algorithms when lazy column replication (`enable_lazy_columns_replication`) produces `ColumnReplicated` columns that flow into merge-sort pipelines with late-arriving inputs. [#101036](https://github.com/ClickHouse/ClickHouse/pull/101036) ([Groene AI](https://github.com/groeneai)). * Fix incorrect `UNKNOWN_IDENTIFIER` error when the same alias is used for multiple expressions in `SELECT`; the correct `MULTIPLE_EXPRESSIONS_FOR_ALIAS` error is now reported. [#101040](https://github.com/ClickHouse/ClickHouse/pull/101040) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix ALIAS columns with `DateTime`/`DateTime64` types not applying timezone conversion when the declared timezone differs from the expression timezone. [#101043](https://github.com/ClickHouse/ClickHouse/pull/101043) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix row policies not being recorded in `query_log` for views, subqueries, and `INSERT ... SELECT`. Even though row policies were applied during query planning, they were not propagated from sub-planners to the parent planner for logging. Row policies (for logging only) are now kept in `QueryAccessInfo` so that both planners and sub-planners can populate them. [#101044](https://github.com/ClickHouse/ClickHouse/pull/101044) ([Narasimha Pakeer](https://github.com/npakeer)). * Fix exception in `DirectJoinMergeTreeEntity` when pipeline blocks contain `ColumnConst` columns that are merged with regular columns. [#101046](https://github.com/ClickHouse/ClickHouse/pull/101046) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix spurious space in CTE column alias formatting (`WITH t (a, b)` → `WITH t(a, b)`). [#101049](https://github.com/ClickHouse/ClickHouse/pull/101049) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `remote`/`cluster` table functions failing with nested table functions like `merge` when the analyzer is enabled. [#101055](https://github.com/ClickHouse/ClickHouse/pull/101055) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `OFFSET` being applied twice in distributed queries when `prefer_localhost_replica=1`, producing fewer rows than expected. [#101071](https://github.com/ClickHouse/ClickHouse/pull/101071) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix crash when using the `Regexp` format with an invalid regular expression in `format_regexp` setting. [#101074](https://github.com/ClickHouse/ClickHouse/pull/101074) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix "Illegal type Decimal64 of start parameter" error for timeseries aggregate functions when using `serialize_query_plan=1` with parallel replicas. [#101083](https://github.com/ClickHouse/ClickHouse/pull/101083) ([Groene AI](https://github.com/groeneai)). * Fix exception in `optimizeLazyMaterialization` when a projection with PREWHERE is used with `ORDER BY ... LIMIT`. [#101115](https://github.com/ClickHouse/ClickHouse/pull/101115) ([Anton Popov](https://github.com/CurtizJ)). * Fix server crash (SIGABRT) when using aggregate functions with the internal-only `Null` combinator (e.g. `sumNull`, `avgNull`) and `aggregate_functions_null_for_empty = 1` setting enabled. [#101147](https://github.com/ClickHouse/ClickHouse/pull/101147) ([Groene AI](https://github.com/groeneai)). * Fix a use-after-free in the filesystem cache write path that could cause reads from freed memory when logging completed file segments (detected by MemorySanitizer in BuzzHouse). [#101161](https://github.com/ClickHouse/ClickHouse/pull/101161) ([Groene AI](https://github.com/groeneai)). * Fix server crash with "Trying to attach external table to a ready set without explicit elements" when distributed index analysis encounters a GLOBAL IN predicate whose set was built without explicit elements. [#101178](https://github.com/ClickHouse/ClickHouse/pull/101178) ([Groene AI](https://github.com/groeneai)). * Fix `MAX`/`MIN` aggregate functions on `Decimal` columns returning incorrect results when JIT compilation is enabled (after the compilation threshold is reached). [#101203](https://github.com/ClickHouse/ClickHouse/pull/101203) ([Raúl Marín](https://github.com/Algunenano)). * Fix `minmax_count_projection` and trivial `COUNT(*)` optimizations being permanently disabled after a lightweight delete, even after all parts with a mask of lightweight delete were merged away. [#101212](https://github.com/ClickHouse/ClickHouse/pull/101212) ([Anton Popov](https://github.com/CurtizJ)). * Fix a case which can lead to `Having zero bytes, ...` logical error from cache arising from a remote object being overwritten in between `list` and `read` which previously resulted in a stale object metadata. [#101219](https://github.com/ClickHouse/ClickHouse/pull/101219) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix server crash (LOGICAL_ERROR: Bad cast from ColumnVector to ColumnLowCardinality) when querying a MergeTree table with `ORDER BY CAST(lc_column, 'Type')` where `lc_column` has a LowCardinality type. [#101220](https://github.com/ClickHouse/ClickHouse/pull/101220) ([Groene AI](https://github.com/groeneai)). * Fix cleanup of stale processing nodes in `S3Queue`. [#101230](https://github.com/ClickHouse/ClickHouse/pull/101230) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix UB in mergeTreeAnalyzeIndexes() in case of invalid optimizations argument. [#101253](https://github.com/ClickHouse/ClickHouse/pull/101253) ([Azat Khuzhin](https://github.com/azat)). * Fix `Logical error: 'partitions_count > 0'` exception when performing consecutive `ALTER TABLE UPDATE` on a partitioned Iceberg table. [#101278](https://github.com/ClickHouse/ClickHouse/pull/101278) ([Desel72](https://github.com/Desel72)). * Fix wrong query results when a large integer constant (e.g. 256, 2147483648) is used as a boolean predicate in a WHERE clause with AND on MergeTree tables. For example, `SELECT count() FROM t WHERE (2147483648 > b) AND 2147483648` would incorrectly return 0 instead of matching all rows. [#101287](https://github.com/ClickHouse/ClickHouse/pull/101287) ([Groene AI](https://github.com/groeneai)). * Fix insert-select from delta lake cluster with replicated merge tree. [#101299](https://github.com/ClickHouse/ClickHouse/pull/101299) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed crash with "Logical error: Reading from materialized CTE before materialization" when a scalar subquery references a chain of dependent materialized CTEs. [#101305](https://github.com/ClickHouse/ClickHouse/pull/101305) ([Groene AI](https://github.com/groeneai)). * Fix data race on `storage_id` in `IStorage::getDependentViewsByColumn`. [#101385](https://github.com/ClickHouse/ClickHouse/pull/101385) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix BACKUP FROM SNAPSHOT AST formatting and cloning. [#101405](https://github.com/ClickHouse/ClickHouse/pull/101405) ([Pablo Marcos](https://github.com/pamarcos)). * Fix LOGICAL_ERROR crash "Current component is empty" when querying `system.part_moves_between_shards` with `enforce_keeper_component_tracking` enabled. [#101462](https://github.com/ClickHouse/ClickHouse/pull/101462) ([Groene AI](https://github.com/groeneai)). * Fix segmentation fault in DataTypeDynamic::create() when the fuzzer generates a malformed Dynamic type AST. [#101464](https://github.com/ClickHouse/ClickHouse/pull/101464) ([Groene AI](https://github.com/groeneai)). * Throw an error when `delta_lake_snapshot_version` or CDF version settings are used without DeltaKernel enabled, instead of silently returning wrong data. [#101489](https://github.com/ClickHouse/ClickHouse/pull/101489) ([Desel72](https://github.com/Desel72)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` exception when using ARRAY JOIN with JOIN USING and `analyzer_compatibility_join_using_top_level_identifier` setting enabled. Close [#101240](https://github.com/ClickHouse/ClickHouse/issues/101240). [#101507](https://github.com/ClickHouse/ClickHouse/pull/101507) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix Iceberg INSERT retry loop failing when the table was created with `iceberg_metadata_file_path` and the target metadata version already exists. [#101548](https://github.com/ClickHouse/ClickHouse/pull/101548) ([Groene AI](https://github.com/groeneai)). * Strip Nullable from result column in arrayIntersect and related functions to avoid serialization/deserialization mismatch. [#101569](https://github.com/ClickHouse/ClickHouse/pull/101569) ([George Larionov](https://github.com/george-larionov)). * Fix server crash (LOGICAL_ERROR) when SELECT-ing from a materialized view backed by an IcebergLocal table engine. [#101577](https://github.com/ClickHouse/ClickHouse/pull/101577) ([Groene AI](https://github.com/groeneai)). * Fix incorrect error message when calling `intExp10` with NaN argument — it said `intExp2` instead of `intExp10`. [#101582](https://github.com/ClickHouse/ClickHouse/pull/101582) ([Krishna Chaitanya](https://github.com/Krishnachaitanyakc)). * Fix `allow_statistics=0` not blocking `ALTER TABLE ADD STATISTICS` and `ALTER TABLE DROP STATISTICS` after refactoring in [#100288](https://github.com/ClickHouse/ClickHouse/issues/100288). [#101585](https://github.com/ClickHouse/ClickHouse/pull/101585) ([Krishna Chaitanya](https://github.com/Krishnachaitanyakc)). * Fix `KeeperMap` `CREATE TABLE` failing with "Cannot create metadata for table" when leftover ZooKeeper nodes from a pre-25.1 partial drop are missing the `drop_lock_version` node. [#101623](https://github.com/ClickHouse/ClickHouse/pull/101623) ([Antonio Andelic](https://github.com/antonio2368)). * Fix possible logical error during reading Map subcolumns. Closes [#100769](https://github.com/ClickHouse/ClickHouse/issues/100769). Closes [#101336](https://github.com/ClickHouse/ClickHouse/issues/101336). [#101641](https://github.com/ClickHouse/ClickHouse/pull/101641) ([Pavel Kruglov](https://github.com/Avogar)). * Fix exact subcolumn match priority over prefix match in `getSubcolumnData` to avoid possible crash. Closes [#101271](https://github.com/ClickHouse/ClickHouse/issues/101271). [#101645](https://github.com/ClickHouse/ClickHouse/pull/101645) ([Pavel Kruglov](https://github.com/Avogar)). * Fix crash (LOGICAL_ERROR: "ColumnUnique can't contain null values") when comparing a `LowCardinality` column with a `Variant` NULL constant while `use_variant_default_implementation_for_comparisons` is disabled. [#101690](https://github.com/ClickHouse/ClickHouse/pull/101690) ([Groene AI](https://github.com/groeneai)). * What: Added empty-stream guard to Bzip2ReadBuffer so it returns EOF instead of throwing UNEXPECTED_END_OF_FILE when the inner stream is empty. [#101691](https://github.com/ClickHouse/ClickHouse/pull/101691) ([ClickGap AI Bot](https://github.com/clickgapai)). * What: Fixed the inverted description text for the `alterable` column in `system.s3_queue_settings` and `system.azure_queue_settings` — swapped `0` and `1` meanings to match the actual code behavior. [#101703](https://github.com/ClickHouse/ClickHouse/pull/101703) ([ClickGap AI Bot](https://github.com/clickgapai)). * Fix positiveModulo(tuple, number) incorrectly dispatching to division instead of modulo. [#101709](https://github.com/ClickHouse/ClickHouse/pull/101709) ([ClickGap AI Bot](https://github.com/clickgapai)). * Fixes a crash when `thread_pool_size` is configured on a cache-wrapped disk. Previously, `FileCacheSettings::loadFromConfig()` rejected `thread_pool_size` as an unknown setting, preventing the server from starting. The setting is a valid `IDisk` parameter that controls the number of threads used for disk-to-disk copy operations during background part moves. [#101712](https://github.com/ClickHouse/ClickHouse/pull/101712) ([Francisco](https://github.com/Milias)). * Fix `RANGE_HASHED` dictionary creation silently accepting a non-existent `MAX` range attribute and using the wrong type configuration when min and max range attributes had different types. The bug was a copy-paste error in `buildRangeConfiguration` that looked up `min_attr_name` instead of `max_attr_name` for the max attribute. [#101732](https://github.com/ClickHouse/ClickHouse/pull/101732) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix use-after-free crash in CPU lease scheduler when the wait timer outlives the worker thread whose `ProfileEvents::Counters` it references. [#101761](https://github.com/ClickHouse/ClickHouse/pull/101761) ([Antonio Andelic](https://github.com/antonio2368)). * Fixes bug in arrayLevenshteinDistanceWeighted and arraySimilarity functions. Closes [#101725](https://github.com/ClickHouse/ClickHouse/issues/101725). [#101767](https://github.com/ClickHouse/ClickHouse/pull/101767) ([Mikhail f. Shiryaev](https://github.com/Felixoid)). * Fix Prometheus Query API ignoring POST form bodies. [#101794](https://github.com/ClickHouse/ClickHouse/pull/101794) ([James Cunningham](https://github.com/JTCunning)). * Fix exception escaping from S3 `Client::~Client` destructor causing server termination. [#101798](https://github.com/ClickHouse/ClickHouse/pull/101798) ([Gagan Dhakrey](https://github.com/gagandhakrey)). * Fix use-after-scope in parallel deserialization of `Object` type dynamic paths, which could cause crashes when reading tables with many dynamic paths. [#101823](https://github.com/ClickHouse/ClickHouse/pull/101823) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed incorrect output in function `formatDateTime` with formatter`%W` with certain (non-default) formatting settings. [#101847](https://github.com/ClickHouse/ClickHouse/pull/101847) ([Robert Schulze](https://github.com/rschu1ze)). * Fix `shouldPatchFunction` false negative in `SYSTEM INSTRUMENT ADD` when the search string first appears inside a template argument of the demangled symbol name. [#101885](https://github.com/ClickHouse/ClickHouse/pull/101885) ([Pablo Marcos](https://github.com/pamarcos)). * Fix UDF registry loss when ZooKeeper session expires during periodic refresh — all user-defined functions could become unavailable until a full refresh succeeds. [#101891](https://github.com/ClickHouse/ClickHouse/pull/101891) ([Nikita Fomichev](https://github.com/fm4v)). * Fixed `system.codecs` description for `AES_256_GCM_SIV` to report `AES-256` instead of `AES-128`. [#101917](https://github.com/ClickHouse/ClickHouse/pull/101917) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix using wrong extremes in min-max index created on JSON column leading to wrong query result. Closes [#101700](https://github.com/ClickHouse/ClickHouse/issues/101700). [#101918](https://github.com/ClickHouse/ClickHouse/pull/101918) ([Pavel Kruglov](https://github.com/Avogar)). * The `splitByString` tokenizer now rejects empty separator strings. [#101928](https://github.com/ClickHouse/ClickHouse/pull/101928) ([Robert Schulze](https://github.com/rschu1ze)). * Fix `materialize_skip_indexes_on_merge=false` not suppressing text (full-text) indexes during merge. Previously, only non-text skip indexes (minmax, set, bloom_filter) were suppressed; text indexes continued to be built, wasting CPU and I/O. [#101932](https://github.com/ClickHouse/ClickHouse/pull/101932) ([Groene AI](https://github.com/groeneai)). * The `sparseGrams` tokenizer generated longer tokens than the provided max length (this was due to a hard-coded `+2` in the implementation). [#101934](https://github.com/ClickHouse/ClickHouse/pull/101934) ([Elmi Ahmadov](https://github.com/ahmadov)). * Fix SIGSEGV in `MergeTreeDataPartWriterWide::cancel` when a stream constructor throws during `addStreams`, leaving a null entry in `column_streams`. [#101936](https://github.com/ClickHouse/ClickHouse/pull/101936) ([Antonio Andelic](https://github.com/antonio2368)). * Fixes an exception when querying Merge or Distributed tables with a full-text index and combined filter conditions that mix `has*Tokens` with LIKE, while `query_plan_direct_read_from_text_index` is enabled. [#101939](https://github.com/ClickHouse/ClickHouse/pull/101939) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix undefined behaviour when parsing native protocol query packets with invalid `QueryProcessingStage` values. [#101972](https://github.com/ClickHouse/ClickHouse/pull/101972) ([Raúl Marín](https://github.com/Algunenano)). * Close TCP connection when an exception occurs during initial query parsing to prevent reading garbage from a desynchronized stream. [#101989](https://github.com/ClickHouse/ClickHouse/pull/101989) ([Raúl Marín](https://github.com/Algunenano)). * Fix SLRU race bug in filesystem cache 26.1+, which can lead to space reservation logical error. In debug build it can also lead to failed assert: `'Previous state is Evicting, but expected state to be Active while setting Evicting flag for 2c1e3484ecdc6b78a8978fa5b17c5097:0:339 (state: Evicting)'.`. [#101991](https://github.com/ClickHouse/ClickHouse/pull/101991) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix exception when casting a string with trailing data to empty `Tuple()` type. [#102011](https://github.com/ClickHouse/ClickHouse/pull/102011) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes incorrect row ordering in queries that use ORDER BY with the grace_hash join algorithm. Affected queries could return results in the wrong order, producing silently incorrect output. [#102036](https://github.com/ClickHouse/ClickHouse/pull/102036) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixes a LOGICAL ERROR (Unexpected size of index type) that could occur in RIGHT JOIN and FULL JOIN queries when the `max_bytes_in_join` setting was configured. [#102042](https://github.com/ClickHouse/ClickHouse/pull/102042) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fixes cases where Time with negative values was returning wrong result on comparison with DateTime. Closes [#101670](https://github.com/ClickHouse/ClickHouse/issues/101670). [#102056](https://github.com/ClickHouse/ClickHouse/pull/102056) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix crash in UDF refresh caused by `ZooKeeperRetriesControl` retrying on a stale (expired) ZooKeeper session without renewing it. [#102059](https://github.com/ClickHouse/ClickHouse/pull/102059) ([Nikita Fomichev](https://github.com/fm4v)). * Fix missing spaces when formatting unlock snapshot. close https://github.com/clickhouse/clickhouse/issues/101723. [#102063](https://github.com/ClickHouse/ClickHouse/pull/102063) ([Han Fei](https://github.com/hanfei1991)). * Fix crash (SIGSEGV) when querying a view with a WHERE clause and the inner query produces columns with different types than the view metadata (e.g. Nullable from LEFT JOIN with `join_use_nulls`). [#102085](https://github.com/ClickHouse/ClickHouse/pull/102085) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix `VectorSimilarityIndexCache` entries never being evicted after part removal due to mismatched cache keys. [#102152](https://github.com/ClickHouse/ClickHouse/pull/102152) ([Seva Potapov](https://github.com/seva-potapov)). * NACK broken messages when using rabbitmq storage. [#102157](https://github.com/ClickHouse/ClickHouse/pull/102157) ([Seva Potapov](https://github.com/seva-potapov)). * Fix logical error when parsing incorrect empty tuple string. [#102289](https://github.com/ClickHouse/ClickHouse/pull/102289) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix incorrect aggregation results (duplicate rows) when using `optimize_aggregation_in_order=1` with GROUP BY columns ordered differently from the table's sorting key. [#102299](https://github.com/ClickHouse/ClickHouse/pull/102299) ([Groene AI](https://github.com/groeneai)). * Fix crash in `IcebergLocal` `ALTER TABLE ... UPDATE` when using Avro format, caused by `LowCardinality`/`Nullable` wrapper types not being unwrapped before serialization. [#102337](https://github.com/ClickHouse/ClickHouse/pull/102337) ([Desel72](https://github.com/Desel72)). * Fixed a segmentation fault in mutations on materialized columns without an expression. Closes [#102185](https://github.com/ClickHouse/ClickHouse/issues/102185). [#102342](https://github.com/ClickHouse/ClickHouse/pull/102342) ([zoomxi](https://github.com/zoomxi)). * Coalescing merge tree fix for array type. This closes [#89509](https://github.com/ClickHouse/ClickHouse/issues/89509). [#102384](https://github.com/ClickHouse/ClickHouse/pull/102384) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix segfault (or LOGICAL_ERROR in debug builds) when reading Parquet files with bloom filter push down enabled and WHERE clause equality/inequality conditions. The crash occurred due to an out-of-bounds memory access in the Parquet prefetcher's bloom filter data retrieval, and could also cause non-deterministic wrong query results. [#102385](https://github.com/ClickHouse/ClickHouse/pull/102385) ([Groene AI](https://github.com/groeneai)). * Fix `LOGICAL_ERROR` abort during SLRU filesystem cache dynamic resize caused by shared eviction statistics across sub-queues and incorrect recovery path for failed candidates. [#102396](https://github.com/ClickHouse/ClickHouse/pull/102396) ([Antonio Andelic](https://github.com/antonio2368)). * Fix failure to initialize on a fresh replica Alias tables without a target table in the Database Replicated. Closes [#101320](https://github.com/ClickHouse/ClickHouse/issues/101320). [#102397](https://github.com/ClickHouse/ClickHouse/pull/102397) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix out-of-bounds read in string search functions (`countSubstrings`, `position`, etc.) when searching for a needle consisting entirely of null bytes. [#102401](https://github.com/ClickHouse/ClickHouse/pull/102401) ([Raúl Marín](https://github.com/Algunenano)). * Full text index settings (`enable_full_text_index`, `allow_experimental_full_text_index`, `use_skip_indexes_on_data_read`) are no longer disabled when the `compatibility` setting points to a version older than 26.1. Previously this could prevent `SharedDatabaseCatalog` from creating tables with text indexes. [#102422](https://github.com/ClickHouse/ClickHouse/pull/102422) ([Nikita Fomichev](https://github.com/fm4v)). * Fix out of bounds read in `printf` with trailing `%`. [#102472](https://github.com/ClickHouse/ClickHouse/pull/102472) ([Raúl Marín](https://github.com/Algunenano)). * Fixes a chassert exception `ReadBuffer is canceled` in debug builds in AsynchronousMetrics, caused by rewind not resetting the buffer cancellation flag. [#102524](https://github.com/ClickHouse/ClickHouse/pull/102524) ([Yuri Fedoseev](https://github.com/yurifedoseev)). * Fixed `hasToken` / `hasTokenOrNull` with separator-only needles (e.g. `'()'`, `'!!!'`) on columns with a text index: previously the index silently skipped all granules instead of throwing `BAD_ARGUMENTS` (for `hasToken`) or returning `NULL` (for `hasTokenOrNull`). [#102544](https://github.com/ClickHouse/ClickHouse/pull/102544) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix OOMs on huge multi requests in the keeper. For OpenTelemetry tracing, we unconditionally allocate >1 KiB for OpenTelemetry spans in `ZooKeeperRequest` objects - meaning, for really huge multi requests, we try to allocate >10 GiB extra memory. To fix this, we now keep shared data in static memory and use `std::unique_ptr` over `std::optional` in `ZooKeeperOpentelemetrySpans`. [#102586](https://github.com/ClickHouse/ClickHouse/pull/102586) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix `NamedCollection` CurrentMetric being inflated by `CREATE NAMED COLLECTION IF NOT EXISTS` on existing collections, and not being initialized for collections loaded from config or SQL storage at startup. Closes [#102507](https://github.com/ClickHouse/ClickHouse/issues/102507). [#102598](https://github.com/ClickHouse/ClickHouse/pull/102598) ([Pablo Marcos](https://github.com/pamarcos)). * Fix exception in `getStructureOfRemoteTable` when local shard returns empty columns due to concurrent DDL. [#102604](https://github.com/ClickHouse/ClickHouse/pull/102604) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception when multiple concurrent `CREATE TABLE IF NOT EXISTS` queries target the same `S3Queue` table on a Shared database. [#102610](https://github.com/ClickHouse/ClickHouse/pull/102610) ([Nikita Taranov](https://github.com/nickitat)). * Fix LOGICAL_ERROR crash "Unexpected number of rows in column subchunk" in native Parquet V3 reader when reading nullable columns with a WHERE filter. [#102628](https://github.com/ClickHouse/ClickHouse/pull/102628) ([Groene AI](https://github.com/groeneai)). * Fix `AzureWriteMicroseconds` profile event description saying "read" instead of "write". [#102639](https://github.com/ClickHouse/ClickHouse/pull/102639) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Bug with row policy that gives 'Not found column in block' exception in some special cases. [#102648](https://github.com/ClickHouse/ClickHouse/pull/102648) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed a server exception in ClusterDiscovery when a static cluster (defined in config) temporarily had no live nodes. [#102661](https://github.com/ClickHouse/ClickHouse/pull/102661) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix wrong date data type inference in case of overflow after timezone adjustment. Closes [#102601](https://github.com/ClickHouse/ClickHouse/issues/102601). [#102674](https://github.com/ClickHouse/ClickHouse/pull/102674) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `SYSTEM WAIT VIEW` hanging forever when the refreshable materialized view is dropped while the wait is in progress. [#102681](https://github.com/ClickHouse/ClickHouse/pull/102681) ([Nikolay Degterinsky](https://github.com/evillique)). * Fix CASE with Dynamic expression returning ELSE for all rows. Closes [#102511](https://github.com/ClickHouse/ClickHouse/issues/102511). [#102684](https://github.com/ClickHouse/ClickHouse/pull/102684) ([Pavel Kruglov](https://github.com/Avogar)). * Fix flattened Dynamic type serialization with binary encoded data types. Closes [#101911](https://github.com/ClickHouse/ClickHouse/issues/101911). [#102692](https://github.com/ClickHouse/ClickHouse/pull/102692) ([Pavel Kruglov](https://github.com/Avogar)). * Fix format_schema_source='query' silently ignoring multi-row results. Closes [#101905](https://github.com/ClickHouse/ClickHouse/issues/101905). [#102698](https://github.com/ClickHouse/ClickHouse/pull/102698) ([Pavel Kruglov](https://github.com/Avogar)). * Report actual exit code via SSH client instead of mapping all errors to `1`. Closes [#101741](https://github.com/ClickHouse/ClickHouse/issues/101741). [#102700](https://github.com/ClickHouse/ClickHouse/pull/102700) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix losing HTTP headers for dynamic/predefined query handlers. Closes [#101846](https://github.com/ClickHouse/ClickHouse/issues/101846). [#102706](https://github.com/ClickHouse/ClickHouse/pull/102706) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Apply Poisson sampling correction to collapsed jemalloc heap profiles to match jeprof output. Previously the collapsed format underestimated actual allocation sizes by not accounting for the sampling probability. [#102759](https://github.com/ClickHouse/ClickHouse/pull/102759) ([Antonio Andelic](https://github.com/antonio2368)). * Fix the crash in the function `hasPhrase` with a `NULL` argument. [#102802](https://github.com/ClickHouse/ClickHouse/pull/102802) ([Nikita Taranov](https://github.com/nickitat)). * Fix server crash (SIGSEGV) when reading Avro files with recursive schemas containing cyclic symbolic type references. Now such schemas are detected and rejected with a clear error message instead of crashing. [#102853](https://github.com/ClickHouse/ClickHouse/pull/102853) ([Groene AI](https://github.com/groeneai)). * Fix server crash (LOGICAL_ERROR assertion) when a function on a `Variant` column hits a memory limit or other non-type-conversion exception during result casting in `FunctionVariantAdaptor`. The exception is now propagated correctly instead of being misclassified as an internal error. [#102855](https://github.com/ClickHouse/ClickHouse/pull/102855) ([Groene AI](https://github.com/groeneai)). * Fix server crash in debug/sanitizer builds when `std::length_error` is thrown during schema inference (e.g., from extreme `input_format_msgpack_number_of_columns` values or malformed input data). [#102859](https://github.com/ClickHouse/ClickHouse/pull/102859) ([Groene AI](https://github.com/groeneai)). * Make null representation in serialization of replicated and sparse columns respect settings (e.g. format_tsv_null_representation). [#102888](https://github.com/ClickHouse/ClickHouse/pull/102888) ([Hechem Selmi](https://github.com/m-selmi)). * Backported in [#103499](https://github.com/ClickHouse/ClickHouse/issues/103499): Fix S3 requests failing with `ios_base::clear: unspecified iostream_category error` instead of being retried, caused by Poco `BufferedStreamBuf::flushBuffer` not handling short writes from the socket layer. [#102894](https://github.com/ClickHouse/ClickHouse/pull/102894) ([Sema Checherinda](https://github.com/CheSema)). * Fix `minmax_count_projection` and trivial `COUNT(*)` optimizations being permanently disabled after a lightweight delete, even after all parts with a mask of lightweight delete were merged away. [#102900](https://github.com/ClickHouse/ClickHouse/pull/102900) ([Anton Popov](https://github.com/CurtizJ)). * Fix random crashes in jemalloc due to LTO. [#102913](https://github.com/ClickHouse/ClickHouse/pull/102913) ([Azat Khuzhin](https://github.com/azat)). * Optimize row policy OR-chains to IN in the new analyzer. [#102915](https://github.com/ClickHouse/ClickHouse/pull/102915) ([Azat Khuzhin](https://github.com/azat)). * Fix jemalloc metadata corruption caused by page cache freeing with wrong alignment that may lead to crashes. [#102918](https://github.com/ClickHouse/ClickHouse/pull/102918) ([Azat Khuzhin](https://github.com/azat)). * Plain INSERTs without materialized views no longer request excessive ConcurrencyControl slots and threads (`max_threads` instead of `max_insert_threads`), preventing CC slot starvation and thread count blowup on clusters with high INSERT throughput. [#102961](https://github.com/ClickHouse/ClickHouse/pull/102961) ([Sema Checherinda](https://github.com/CheSema)). * Reintroduce ArrowMemoryPool to allow throwing MEMORY_LIMIT_EXCEEDED to avoid kernel OOM. [#102999](https://github.com/ClickHouse/ClickHouse/pull/102999) ([Azat Khuzhin](https://github.com/azat)). * Fix `cast_string_to_date_time_mode` being ignored for `CAST` to `Nullable(DateTime)`. Closes [#101840](https://github.com/ClickHouse/ClickHouse/issues/101840). [#103035](https://github.com/ClickHouse/ClickHouse/pull/103035) ([Pavel Kruglov](https://github.com/Avogar)). * Added support for `ALIAS` columns in text index direct read optimization. [#103037](https://github.com/ClickHouse/ClickHouse/pull/103037) ([Anton Popov](https://github.com/CurtizJ)). * Backported in [#103454](https://github.com/ClickHouse/ClickHouse/issues/103454): Fix `SELECT DISTINCT` silently returning incomplete results when an aggregate projection matched the query and some parts of the table had no projection data (e.g. the projection was added on a table that already held data, and `MATERIALIZE PROJECTION` was not run). Closes [#102951](https://github.com/ClickHouse/ClickHouse/issues/102951). [#103052](https://github.com/ClickHouse/ClickHouse/pull/103052) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix wrong results returned by `WHERE x AND toNullable(N)` on `MergeTree` tables when `N` is an integer wider than `UInt8` (e.g. `256`, `65535`, `2147483648`, or any negative integer). The filter incorrectly discarded all rows because `splitFilterNodeForAllowedInputs` used a `NULL` zero when converting the `Nullable` remainder of a reduced `AND` to boolean, turning the comparison into `NULL` under three-valued logic. [#103077](https://github.com/ClickHouse/ClickHouse/pull/103077) ([Groene AI](https://github.com/groeneai)). * Fixed incorrect argument type reported in error messages of string search functions (e.g. `locate`, `position`) when arguments are passed in swapped order (`locate(needle, haystack)` with `function_locate_has_mysql_compatible_argument_order = 1`). [#103102](https://github.com/ClickHouse/ClickHouse/pull/103102) ([Alex Kuleshov](https://github.com/0xAX)). * Fix waitForPause hanging indefinitely when disableFailPoint is called with no thread paused at the failpoint. [#103119](https://github.com/ClickHouse/ClickHouse/pull/103119) ([Shaohua Wang](https://github.com/tiandiwonder)). * Cap pre-auth TCP Hello packet strings to 64 KB and add `handshake_timeout_milliseconds` server setting to limit total handshake time, preventing unauthenticated clients from consuming excessive memory or holding threads indefinitely. [#103284](https://github.com/ClickHouse/ClickHouse/pull/103284) ([Sema Checherinda](https://github.com/CheSema)). * Fix Parquet ColumnIndex stats min_value > max_value for String columns. [#103334](https://github.com/ClickHouse/ClickHouse/pull/103334) ([Saurabh Kumar Ojha](https://github.com/saurabhojha)). * Check for malformed flattened Dynamic data in Native format. [#103392](https://github.com/ClickHouse/ClickHouse/pull/103392) ([Pavel Kruglov](https://github.com/Avogar)). * Populate `_time` column from `url` table function. [#103437](https://github.com/ClickHouse/ClickHouse/pull/103437) ([Nikita Taranov](https://github.com/nickitat)). * Fix SVE detection using SVE instructions when unavailable. [#103568](https://github.com/ClickHouse/ClickHouse/pull/103568) ([Raúl Marín](https://github.com/Algunenano)). #### Build/Testing/Packaging Improvement * Libstemmer (Snowball) dependency now uses newer version v3.0.1. [#99256](https://github.com/ClickHouse/ClickHouse/pull/99256) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Randomize settings in clickhouse-test: `use_skip_indexes_for_top_k`, `use_top_k_dynamic_filtering`, `query_plan_max_limit_for_top_k_optimization`. [#91782](https://github.com/ClickHouse/ClickHouse/pull/91782) ([Nikita Fomichev](https://github.com/fm4v)). * Implement a stress test for functions, checking the soundness of their various properties. [#93543](https://github.com/ClickHouse/ClickHouse/pull/93543) ([Michael Kolupaev](https://github.com/al13n321)). * Provide own CMake configuration for `llvm-project` instead of importing upstream one. [#97453](https://github.com/ClickHouse/ClickHouse/pull/97453) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Randomize more `optimize_*` settings in the test infrastructure to improve coverage of query optimization passes. [#97547](https://github.com/ClickHouse/ClickHouse/pull/97547) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use Rust toolchain nightly-2026-03-22. [#98602](https://github.com/ClickHouse/ClickHouse/pull/98602) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `wasmtime` v42.0.1. [#98603](https://github.com/ClickHouse/ClickHouse/pull/98603) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `llvm-project` 22.1.1. [#98882](https://github.com/ClickHouse/ClickHouse/pull/98882) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Replace `SANITIZE_COVERAGE` (custom sanitizer callbacks, symbol-level granularity) with LLVM source-based coverage (`WITH_COVERAGE`, `-fprofile-instr-generate -fcoverage-mapping`) for the nightly per-test coverage pipeline. The server now reads its own coverage mapping from ELF sections at startup and collects `(file, line_start, line_end)` tuples per test via a new `SYSTEM SET COVERAGE TEST 'name'` command. Test selection in targeted CI checks uses line-range queries against a new `checks_coverage_lines` CIDB table and ranks candidate tests by how many changed diff lines they cover. [#99513](https://github.com/ClickHouse/ClickHouse/pull/99513) ([Nikita Fomichev](https://github.com/fm4v)). * Fix llvm-libc link error with -O0 build. [#100023](https://github.com/ClickHouse/ClickHouse/pull/100023) ([Zheguang Zhao](https://github.com/zheguang)). * Fixed two error messages on container startup in case the configuration did not contain the `logger.log` and `logger.errorlog` settings (e.g. when all log messages are supposed to go to STDOUT/STDERR). [#100239](https://github.com/ClickHouse/ClickHouse/pull/100239) ([Simon](https://github.com/simonhammes)). * Prefer `ld64.lld` over Apple's `ld` (cctools-port ld64) for macOS builds. This should significantly reduce Darwin link times, as cctools-port's ld64 is very slow with `-ffunction-sections` and `-dead_strip`. Falls back to `ld` if `ld64.lld` is not available. [#100275](https://github.com/ClickHouse/ClickHouse/pull/100275) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Tests to ensure that directories do not register any secrets (REST + Glue). [#100307](https://github.com/ClickHouse/ClickHouse/pull/100307) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Repeat recently modified tests with different randomized settings. [#100385](https://github.com/ClickHouse/ClickHouse/pull/100385) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Track `#embed` file dependencies in CMake and enable ccache depend mode for correct rebuilds. [#100411](https://github.com/ClickHouse/ClickHouse/pull/100411) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add correctness tests for TPC-H queries. [#100580](https://github.com/ClickHouse/ClickHouse/pull/100580) ([Raufs Dunamalijevs](https://github.com/rienath)). * Use `aws-sdk-cpp` 1.11.771. [#100582](https://github.com/ClickHouse/ClickHouse/pull/100582) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Allow performance test XMLs to reference external SQL query files and settings via file attribute. [#100747](https://github.com/ClickHouse/ClickHouse/pull/100747) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix compilation with `-march=x86-64-v4` by adding missing `TargetSpecific.h` include in `LowerUpperImpl.h`. [#100932](https://github.com/ClickHouse/ClickHouse/pull/100932) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix build on gentoo by adding `--no-default-config` to cxxflags. [#100973](https://github.com/ClickHouse/ClickHouse/pull/100973) ([Isak Ellmer](https://github.com/spinojara)). * Added `utils/auto-bisect/` — a shell-based bisection framework that downloads pre-built CI binaries and runs a user-supplied test script to find the first commit that introduced a regression, without requiring a local build. [#100989](https://github.com/ClickHouse/ClickHouse/pull/100989) ([Nikita Fomichev](https://github.com/fm4v)). * Disable ThinLTO by default in CMake so that local developer builds no longer implicitly enable it. CI release builds are unaffected since they explicitly pass `-DENABLE_THINLTO=1`. [#101041](https://github.com/ClickHouse/ClickHouse/pull/101041) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add TPC-DS SF1 benchmark to performance tests. [#101209](https://github.com/ClickHouse/ClickHouse/pull/101209) ([Raufs Dunamalijevs](https://github.com/rienath)). * Added stateless tests for MergeTree lightweight deletes covering: `has_lightweight_delete` flag lifecycle, `COUNT(*)` correctness with `optimize_trivial_count_query`, `MATERIALIZED`/`DEFAULT` column integrity, `ReplicatedMergeTree` flag recovery, `read_in_order` with deleted rows, multiple delete/merge cycles, `_row_exists` column hiding, predicate variety, and `ALTER DELETE` RBAC enforcement. [#101792](https://github.com/ClickHouse/ClickHouse/pull/101792) ([Nikita Fomichev](https://github.com/fm4v)). * Distroless Docker images are now published with ubuntu and alpine variants for tagged releases. [#101941](https://github.com/ClickHouse/ClickHouse/pull/101941) ([Rahul Nair](https://github.com/motsc)). * Stack traces now show clean bare relative paths (e.g. `src/Common/Exception.cpp`) instead of paths polluted with the build directory (e.g. `./ci/tmp/fast_build/./src/Common/Exception.cpp`). [#102000](https://github.com/ClickHouse/ClickHouse/pull/102000) ([Raúl Marín](https://github.com/Algunenano)). * Add a CI style check that rejects files larger than 5 MB committed to the repository, with a whitelist for existing legitimate test data. Remove unused 14 MB `zookeeper_log.parquet`. [#102080](https://github.com/ClickHouse/ClickHouse/pull/102080) ([Raúl Marín](https://github.com/Algunenano)). * Remove ~400 unused `#include` directives from headers to reduce compilation times. [#102585](https://github.com/ClickHouse/ClickHouse/pull/102585) ([Raúl Marín](https://github.com/Algunenano)). * Use `wasmtime` v43.0.1. [#102603](https://github.com/ClickHouse/ClickHouse/pull/102603) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `openssl` 3.5.6. [#102606](https://github.com/ClickHouse/ClickHouse/pull/102606) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `xz` 5.8.3. [#102607](https://github.com/ClickHouse/ClickHouse/pull/102607) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Upgrade distroless Docker image base from Debian 12 (glibc 2.36, OpenSSL 3.0) to Debian 13 (glibc 2.41, OpenSSL 3.5), reducing CVE surface to zero reachable vulnerabilities. [#101678](https://github.com/ClickHouse/ClickHouse/pull/101678) ([Rahul Nair](https://github.com/motsc)). ### ClickHouse release 26.3 LTS, 2026-03-26. [Presentation](https://presentations.clickhouse.com/), [Video](https://www.youtube.com/watch?v=_bY0ucNB1lQ) #### Backward Incompatible Change * Downgrading after upgrading may cause data loss. Propagate data types serialization versions to nested data types. For example, String serialization version `with_size_stream` previously was applied only on top-level String columns and Tuple elements. Now it's applied to any String type inside any nested type like `Array`/`Map`/`Variant`/`JSON`/etc. This behaviour is controlled by MergeTree setting `propagate_types_serialization_versions_to_nested_types` that is now enabled by default. After this change, newly created data parts cannot be read by older versions, but old parts can be read on new version with no problems. **Upgrade is safe, but downgrade is not — if you need to roll back after upgrading to 26.3, data written by 26.3 in columns with nested types will be unreadable!** See [#101429](https://github.com/ClickHouse/ClickHouse/issues/101429) for details. [#94859](https://github.com/ClickHouse/ClickHouse/pull/94859) ([Pavel Kruglov](https://github.com/Avogar)). * Remove the `hypothesis` skip index type. It was an obscure, experimental feature with limited practical use. Creating tables with `INDEX ... TYPE hypothesis` will now produce an error. [#96874](https://github.com/ClickHouse/ClickHouse/pull/96874) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove the experimental `detectProgrammingLanguage` function. [#99567](https://github.com/ClickHouse/ClickHouse/pull/99567) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `NOT` operator precedence to match the SQL standard: `NOT` now binds looser than `IS NULL`, `BETWEEN`, `LIKE`, and arithmetic operators. For example, `NOT (x) IS NULL` is now parsed as `NOT (x IS NULL)` instead of `(NOT x) IS NULL`. This may change the result of queries that relied on the previous (non-standard) behavior. [#97680](https://github.com/ClickHouse/ClickHouse/pull/97680) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Corrects the metadata of normal projections so that projections with multi-column sorting keys are properly recognized. Builds on top of [#90429](https://github.com/ClickHouse/ClickHouse/issues/90429). [#91352](https://github.com/ClickHouse/ClickHouse/pull/91352) ([Amos Bird](https://github.com/amosbird)). * Fixed skip index files not respecting replace_long_file_name_to_hash setting, causing "File name too long" errors and broken index reads for indices with long names. Skip index filenames are now hashed when they exceed max_file_name_length, similar to column files. This is backward compatible (new servers read old parts), but downgrading (or old servers during a rolling upgrade) may cause long-named indices to be ignored. [#97128](https://github.com/ClickHouse/ClickHouse/pull/97128) ([Raúl Marín](https://github.com/Algunenano)). * Turn on async insert by default. ClickHouse will be batching all small inserts by default now. This setting is set under compatibility. If you set `compatibility=` then default value would be the previous one, `false`. You can turn off/on async inserts at several levels: in the config in users profiles, for the session, for the query, or for the MergeTree table. [#97590](https://github.com/ClickHouse/ClickHouse/pull/97590) ([Sema Checherinda](https://github.com/CheSema)). * Change the default value of `mysql_datatypes_support_level` from empty to `decimal,datetime64,date2Date32`, enabling proper mapping of MySQL `DATE` to `Date32`, `DECIMAL`/`NUMERIC` to `Decimal`, and `DATETIME`/`TIMESTAMP` with precision to `DateTime64` by default. Previously, MySQL `DATE` columns were mapped to `Date` which cannot represent dates before 1970-01-01, causing data corruption. [#97716](https://github.com/ClickHouse/ClickHouse/pull/97716) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Accept array of part names instead of regexp for `mergeTreeAnalyzeIndexes{,UUID}` since regexp is slow (_Experimental feature_). [#98474](https://github.com/ClickHouse/ClickHouse/pull/98474) ([Azat Khuzhin](https://github.com/azat)). * Change default `stderr_reaction` from `throw` to `log_last` for executable UDFs. UDFs that write warnings to stderr no longer fail when exit code is 0. Exit code exceptions now include stderr content. [#99232](https://github.com/ClickHouse/ClickHouse/pull/99232) ([Xu Jia](https://github.com/XuJia0210)). #### New Feature * Added bucketed serialization for Map columns in MergeTree (`map_serialization_version = 'with_buckets'`). Keys are split into hash-based buckets so that reading a single key (`m['key']`) only reads one bucket instead of the entire column, providing 2-49x speedup for single-key lookups depending on map size. The number of buckets and the bucketing strategy can be controlled by new MergeTree settings: `map_serialization_version`, `max_buckets_in_map`, `map_buckets_strategy`, `map_buckets_coefficient`, and `map_buckets_min_avg_size`. [#99200](https://github.com/ClickHouse/ClickHouse/pull/99200) ([Pavel Kruglov](https://github.com/Avogar)). * Support materialized CTE. Allow evaluating CTEs only once during query execution and store their results in temporary tables. Closes [#53449](https://github.com/ClickHouse/ClickHouse/issues/53449). [#94849](https://github.com/ClickHouse/ClickHouse/pull/94849) ([Dmitry Novik](https://github.com/novikd)). * Allow certain SQL-standard functions without parentheses for compatibility, such as `NOW`. Closes [#52102](https://github.com/ClickHouse/ClickHouse/issues/52102). [#95949](https://github.com/ClickHouse/ClickHouse/pull/95949) ([Aly Kafoury](https://github.com/AlyHKafoury)). * You can now use the natural sort key function as `naturalSortKey(s)`. [#90322](https://github.com/ClickHouse/ClickHouse/pull/90322) ([Nazarii Piontko](https://github.com/nazarii-piontko)). * You can now use native JSON/Object input for JSONExtract functions. Closes [#88370](https://github.com/ClickHouse/ClickHouse/issues/88370). [#96711](https://github.com/ClickHouse/ClickHouse/pull/96711) ([Fisnik Kastrati](https://github.com/fkastrati)). * If a query parameter has `Nullable` type and is not specified, we will assume that its value is `NULL`. [#93869](https://github.com/ClickHouse/ClickHouse/pull/93869) ([Vikash Kumar](https://github.com/vikashkumar2020)). * Support auxiliary ZooKeeper for `Replicated` database. [#95590](https://github.com/ClickHouse/ClickHouse/pull/95590) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Support `has` function for JSON type to check path existence, similar to Map. [#96927](https://github.com/ClickHouse/ClickHouse/pull/96927) ([DQ](https://github.com/il9ue)). * Added the `mergeTreeTextIndex(database, table, index)` table function, which allows reading data directly from a text index. This function can be used for introspection or for performing aggregations on top of text index data. [#97003](https://github.com/ClickHouse/ClickHouse/pull/97003) ([Anton Popov](https://github.com/CurtizJ)). * Add `table_readonly` MergeTree setting to mark tables as read-only, preventing inserts and modifications. [#97652](https://github.com/ClickHouse/ClickHouse/pull/97652) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add new setting `use_partition_pruning` and alias `use_partition_key`. Set it to `false` to disable partition pruning based on the partition key. [#97888](https://github.com/ClickHouse/ClickHouse/pull/97888) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Implements `ALTER TABLE ... EXECUTE expire_snapshots('')` for Iceberg tables. [#97904](https://github.com/ClickHouse/ClickHouse/pull/97904) ([murphy-4o](https://github.com/murphy-4o)). [#99130](https://github.com/ClickHouse/ClickHouse/pull/99130) * Allow each `type=http` entry in `` to specify a custom `` key pointing to a separate `` config section, enabling different HTTP routing rules per port. [#98414](https://github.com/ClickHouse/ClickHouse/pull/98414) ([Amos Bird](https://github.com/amosbird)). * Add `pretty=1` option to `EXPLAIN` for tree-style indented output and `compact=1` to collapse `Expression` steps, making query plans more readable. [#98500](https://github.com/ClickHouse/ClickHouse/pull/98500) ([Kirill Kopnev](https://github.com/Fgrtue)). * Add `restore_access_entities_with_current_grants` server setting. When enabled, restored users/roles from backups have their grants limited to what the restoring user is allowed to grant (same semantics as `GRANT CURRENT GRANTS`), instead of failing with `ACCESS_DENIED`. [#98795](https://github.com/ClickHouse/ClickHouse/pull/98795) ([pufit](https://github.com/pufit)). * Add `caseFoldUTF8` and `removeDiacriticsUTF8` functions for Unicode case folding and diacritical mark removal. [#98973](https://github.com/ClickHouse/ClickHouse/pull/98973) ([George Larionov](https://github.com/george-larionov)). * Add `normalizeUTF8NFKCCasefold` string function for NFKC_Casefold Unicode normalization, which combines NFKC normalization with case folding. [#99276](https://github.com/ClickHouse/ClickHouse/pull/99276) ([George Larionov](https://github.com/george-larionov)). * Add `unicode_word` tokenizer for full-text indexes and the `tokens` function. It splits text using Unicode word boundary rules: ASCII words are formed with connector characters (underscore, colon, dot, single quote), while non-ASCII Unicode characters become single-character tokens. Note: 26.3 accepts this tokenizer only under the name `unicode_word`; it was renamed to `asciiCJK` in 26.4. [#99357](https://github.com/ClickHouse/ClickHouse/pull/99357) ([Amos Bird](https://github.com/amosbird)). * Added `max_skip_unavailable_shards_num` and `max_skip_unavailable_shards_ratio` settings to limit how many shards can be silently skipped when `skip_unavailable_shards` is enabled. If the number or ratio of unavailable shards exceeds the configured threshold, an exception is thrown instead of returning silently incomplete results. [#99369](https://github.com/ClickHouse/ClickHouse/pull/99369) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Users can now use `SOME` keyword for subquery expressions. It behaves identically to `ANY`. [#99842](https://github.com/ClickHouse/ClickHouse/pull/99842) ([Artem Kytkin](https://github.com/Vinceent)). * Add `output_format_trim_fixed_string` setting to strip trailing null bytes from `FixedString` values in text output formats. [#97558](https://github.com/ClickHouse/ClickHouse/pull/97558) ([NeedmeFordev](https://github.com/spider-yamet)). * Support parenthesized table join expressions in FROM clause, e.g. `SELECT * FROM (t1 CROSS JOIN t2)`. [#97650](https://github.com/ClickHouse/ClickHouse/pull/97650) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implement function `toDaysInMonth`: it returns the number of days in the month of the specified date. [#99227](https://github.com/ClickHouse/ClickHouse/pull/99227) ([Vitaly Baranov](https://github.com/vitlibar)). #### Experimental Feature * Add experimental support for WebAssembly-based user-defined functions (UDFs), allowing custom function logic to be implemented in WebAssembly and executed within ClickHouse. Special thanks to [Alexey Smirnov](https://github.com/lioshik) for contributing the Wasmtime backend support. [#88747](https://github.com/ClickHouse/ClickHouse/pull/88747) ([Vladimir Cherkasov](https://github.com/vdimir)). Incremental improvements for WASM UDF support. [#99373](https://github.com/ClickHouse/ClickHouse/pull/99373) ([Vasily Chekalkin](https://github.com/bacek)). * Add support for external SQL dialects using `polyglot` library. [#99496](https://github.com/ClickHouse/ClickHouse/pull/99496) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `ALP` floating-point compression codec (without ALP_rd fallback for non-compressible doubles). [#91362](https://github.com/ClickHouse/ClickHouse/pull/91362) ([Nazarii Piontko](https://github.com/nazarii-piontko)). * Add experimental lazy type hints for `JSON` columns. When enabled via `allow_experimental_json_lazy_type_hints`, `ALTER TABLE ... MODIFY COLUMN json JSON(path TypeName)` that only adds or modifies type hints completes instantly as a metadata-only operation, without rewriting historical data. Type hints are applied at query time for old parts and materialized during INSERTs and background merges. [#97412](https://github.com/ClickHouse/ClickHouse/pull/97412) ([tanner-bruce](https://github.com/tanner-bruce)). * Enable parallel reads from YTsaurus table engine. [#97343](https://github.com/ClickHouse/ClickHouse/pull/97343) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). #### Performance Improvement * Improve the performance of data lakes. In previous versions, reading from object storage didn't resize the pipeline to the number of processing threads. This gives orders of magnitude improvements (~40x) on multi-core machines. [#99548](https://github.com/ClickHouse/ClickHouse/pull/99548) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Now, the relationship between `enable_parallel_replicas` and `automatic_parallel_replicas_mode` is as follows. A query can use parallel replicas only if `enable_parallel_replicas > 0`. Additionally, if `automatic_parallel_replicas_mode=1`, the decision whether to use parallel replicas is made during planning based on the previously collected statistics. If `automatic_parallel_replicas_mode=0`, parallel replicas will be used for all supported queries regardless of any statistics. One notable exemption is distributed insert-select with parallel replicas: in this case, queries will always be executed as if `automatic_parallel_replicas_mode=0`. [#97517](https://github.com/ClickHouse/ClickHouse/pull/97517) ([Nikita Taranov](https://github.com/nickitat)). * Allow partition pruning when the predicate contains any comparison operator (`=`, `<`, `>`, `!=`) and the partition key is wrapped in a deterministic function chain (e.g. `PARTITION BY x` and predicates like `cityHash64(x) % 5 > 2`, `toYYYYMM(x) < 2026`, `toYYYYMM(x) = 2026`, or `toYYYYMM(x) != 2026` will all use the partition key for pruning). Closes [#28800](https://github.com/ClickHouse/ClickHouse/issues/28800). [#98432](https://github.com/ClickHouse/ClickHouse/pull/98432) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Allow read-in-order optimization and primary-key pruning when the `CAST` target type is `Nullable` and the conversion is monotonic; for example, with `PRIMARY KEY x`, ClickHouse can use read-in-order optimization for `ORDER BY x::Nullable(UInt64)` and can apply primary-key pruning for predicates such as `WHERE x::Nullable(UInt64) > 500000`. [#98482](https://github.com/ClickHouse/ClickHouse/pull/98482) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Allow index pruning and filter pushdown when an integral column is compared with a float literal; for example, predicates like `WHERE x < 10.5` can now use the primary key for pruning, and filters such as `prime < 1e9` or `number < 1e5` are now pushed down for the `primes()` and `numbers()` table functions instead of causing unbounded execution. Closes [#85167](https://github.com/ClickHouse/ClickHouse/issues/85167). [#98516](https://github.com/ClickHouse/ClickHouse/pull/98516) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Added a new SLRU cache for Parquet metadata to improve read performance by removing the need to re-download files just to read metadata. [#98140](https://github.com/ClickHouse/ClickHouse/pull/98140) ([Grant Holly](https://github.com/grantholly-clickhouse)). * Support swapping sides of ANTI, SEMI and FULL joins based on optimizer statistics. [#97498](https://github.com/ClickHouse/ClickHouse/pull/97498) ([Hechem Selmi](https://github.com/m-selmi)). * Optimize granules skipping for `pointInPolygon` for large polygons and fix `pointInPolygon` index analysis throwing during primary key pruning. [#91633](https://github.com/ClickHouse/ClickHouse/pull/91633) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Improve performance of `levenshteinDistance` function. [#94543](https://github.com/ClickHouse/ClickHouse/pull/94543) ([Joanna Hulboj](https://github.com/jh0x)). * Optimize batch decimal type conversions by avoiding per-element function calls. [#95923](https://github.com/ClickHouse/ClickHouse/pull/95923) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Iceberg tables now support asynchronous metadata prefetching via `iceberg_metadata_async_prefetch_period_ms` table setting, which periodically pre-populates the metadata cache. Additionally, the `iceberg_metadata_staleness_ms` query setting allows SELECT queries to use cached metadata if it is fresher than the specified staleness, eliminating calls to the Iceberg catalog during request processing. [#96191](https://github.com/ClickHouse/ClickHouse/pull/96191) ([Arsen Muk](https://github.com/arsenmuk)). * `S3Queue` ordered mode uses S3 ListObjectsV2 StartAfter to avoid re-listing full prefix history, reducing ListObjects calls. [#96370](https://github.com/ClickHouse/ClickHouse/pull/96370) ([Venkata Vineel ](https://github.com/vyalamar)). * Lower memory usage for inserts deduplication. In general the original block is needed for deduplication, but for sync insert we could omit it and save the memory for good. [#96661](https://github.com/ClickHouse/ClickHouse/pull/96661) ([Sema Checherinda](https://github.com/CheSema)). * Use an arch-specific value for cache line size instead of a hardcoded value of 64. [#97357](https://github.com/ClickHouse/ClickHouse/pull/97357) ([Nikita Taranov](https://github.com/nickitat)). * Slightly optimized reading from the text index dictionary, improving the overall performance of text index analysis. [#97519](https://github.com/ClickHouse/ClickHouse/pull/97519) ([Anton Popov](https://github.com/CurtizJ)). * Speed up `LZ4` decompression of 16 byte blocks in ARM. [#97774](https://github.com/ClickHouse/ClickHouse/pull/97774) ([Raúl Marín](https://github.com/Algunenano)). * Refactor tokenization to a new high-performance interface, replacing the old iterator-style API to support SIMD and stateful tokenizers. Part of [#90268](https://github.com/ClickHouse/ClickHouse/issues/90268). [#97871](https://github.com/ClickHouse/ClickHouse/pull/97871) ([Amos Bird](https://github.com/amosbird)). * Improved performance of text index analysis for queries with combined conditions involving both indexed and non-indexed columns. Previously, early exit optimization during index analysis was incorrectly disabled in such cases. [#98096](https://github.com/ClickHouse/ClickHouse/pull/98096) ([Anton Popov](https://github.com/CurtizJ)). * Improve the performance of queries with constant expressions that generate very long arrays or maps. [#98287](https://github.com/ClickHouse/ClickHouse/pull/98287) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix key condition analysis for `DateTime64` primary keys compared with integer constants, which previously resulted in no granule pruning. [#98410](https://github.com/ClickHouse/ClickHouse/pull/98410) ([Amos Bird](https://github.com/amosbird)). * The setting `optimize_syntax_fuse_functions` is enabled by default. [#98424](https://github.com/ClickHouse/ClickHouse/pull/98424) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Optimize `avgWeighted` aggregate function by using local accumulators instead of per-row store-forwarding through aggregate state, improving performance by up to 27% for Nullable inputs. [#98793](https://github.com/ClickHouse/ClickHouse/pull/98793) ([Antonio Andelic](https://github.com/antonio2368)). * Improves performance and reduces memory usage for parallel window functions in certain scenarios, and for `arrayFold` workloads with large arrays. This can also reduce page-fault pressure and improve stability under tight memory limits for affected queries. [#98892](https://github.com/ClickHouse/ClickHouse/pull/98892) ([filimonov](https://github.com/filimonov)). * Improve performance of sorted merges. [#99013](https://github.com/ClickHouse/ClickHouse/pull/99013) ([Artem Zuikov](https://github.com/4ertus2)). * Optimize `INTERSECT ALL` and `EXCEPT ALL`. [#99097](https://github.com/ClickHouse/ClickHouse/pull/99097) ([Raufs Dunamalijevs](https://github.com/rienath)). * Support the `read_in_order_use_virtual_row` optimization for reverse-order reads. [#99198](https://github.com/ClickHouse/ClickHouse/pull/99198) ([Vladimir Cherkasov](https://github.com/vdimir)). * Reduce cache contention in `RIGHT` and `FULL` JOINs by checking if JoinUsedFlags is set before writing. [#99274](https://github.com/ClickHouse/ClickHouse/pull/99274) ([Hechem Selmi](https://github.com/m-selmi)). * Optimize `PrefetchingHelper::calcPrefetchLookAhead` by replacing floating-point math with pure integer arithmetic, improving instruction cache layout and reducing cycle overhead during aggregation loops. [#99327](https://github.com/ClickHouse/ClickHouse/pull/99327) ([Riyane El Qoqui](https://github.com/riyaneel)). * Reduce memory consumption of Keeper by replacing `absl::flat_hash_set` with `CompactChildrenSet` for storing node children. The new container stores 0–1 children inline without heap allocation, which covers the majority of Keeper nodes. This reduces `KeeperMemNode` size from 144 to 128 bytes. [#99860](https://github.com/ClickHouse/ClickHouse/pull/99860) ([Antonio Andelic](https://github.com/antonio2368)). * Aggregate projections are now correctly supported in views. Fixes [#32753](https://github.com/ClickHouse/ClickHouse/issues/32753). [#88798](https://github.com/ClickHouse/ClickHouse/pull/88798) ([Amos Bird](https://github.com/amosbird)). * Support OUTER to INNER join conversion optimization with `join_use_nulls`. Closes [#90978](https://github.com/ClickHouse/ClickHouse/issues/90978). [#95968](https://github.com/ClickHouse/ClickHouse/pull/95968) ([Vladimir Cherkasov](https://github.com/vdimir)). * Improve subcolumns reading by correct sizes calculation before reading. It reduces memory usage and speeds up the subcolumns reading. [#96251](https://github.com/ClickHouse/ClickHouse/pull/96251) ([Pavel Kruglov](https://github.com/Avogar)). * Make mark, uncompressed and page caches to use separate jemalloc arena to avoid memory fragmentation when short lived allocations, i.e., for queries and requests are mixed with longer lived allocations for caches. [#96812](https://github.com/ClickHouse/ClickHouse/pull/96812) ([Seva Potapov](https://github.com/seva-potapov)). [#98812](https://github.com/ClickHouse/ClickHouse/pull/98812). [#99021](https://github.com/ClickHouse/ClickHouse/pull/99021) * Tables with `DELETE TTL` rules can now use the vertical merge algorithm. [#97332](https://github.com/ClickHouse/ClickHouse/pull/97332) ([murphy-4o](https://github.com/murphy-4o)). * Apply data skipping indexes during distributed index analysis. [#97767](https://github.com/ClickHouse/ClickHouse/pull/97767) ([Azat Khuzhin](https://github.com/azat)). * Marks of secondary indexes are now prewarmed when the `prewarm_mark_cache` setting is enabled (loaded into the index mark cache during data part fetches and table startup). [#97772](https://github.com/ClickHouse/ClickHouse/pull/97772) ([Anton Popov](https://github.com/CurtizJ)). * Reduced locking during access control. [#97894](https://github.com/ClickHouse/ClickHouse/pull/97894) ([Nikita Taranov](https://github.com/nickitat)). * When apply_row_policy_after_final or apply_prewhere_after_final is enabled, compound AND conditions in row policies and PREWHERE are now decomposed to extract sorting-key atoms for primary key index analysis. Previously, if a deferred filter contained a mix of sorting-key and non-sorting-key predicates (e.g. x > 1 AND y != 'foo'), the entire expression was excluded from index analysis. Now sorting-key atoms (like x > 1) are extracted and used for granule pruning, even from nested AND expressions. [#98513](https://github.com/ClickHouse/ClickHouse/pull/98513) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Reduce lock contention in MergeTreeBackgroundExecutor by making task resources release without acquiring the lock. Closes [#93620](https://github.com/ClickHouse/ClickHouse/issues/93620). [#98604](https://github.com/ClickHouse/ClickHouse/pull/98604) ([Dmitry Novik](https://github.com/novikd)). * Fix excessive memory usage (~514 MiB) during format auto-detection when reading non-Arrow data (e.g. JSON from `url` or `file` without explicit format), caused by the ArrowStream reader misinterpreting the first bytes as a huge metadata length. [#98893](https://github.com/ClickHouse/ClickHouse/pull/98893) ([Konstantin Bogdanov](https://github.com/thevar1able)). #### Improvement * Makes it possible to parse GeoParquet files that contain different Geo type in the same column. [#97851](https://github.com/ClickHouse/ClickHouse/pull/97851) ([Mark Needham](https://github.com/mneedham)). * Introduce `tokensForLikePattern` SQL function that tokenizes LIKE patterns while respecting wildcard semantics: `%` and `_` are treated as wildcards, escaped wildcards (`\%`, `\_`) are treated as literals, and tokens adjacent to unescaped wildcards are discarded. [#97872](https://github.com/ClickHouse/ClickHouse/pull/97872) ([Amos Bird](https://github.com/amosbird)). * Add a `{_schema_hash}` placeholder for the S3 table engine that inserts a hash of the table's column definitions into the S3 path. [#98265](https://github.com/ClickHouse/ClickHouse/pull/98265) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * `SymbolIndex`, `addressToSymbol`, `system.symbols`, and `buildId` now work on macOS by parsing Mach-O symbol tables. [#99014](https://github.com/ClickHouse/ClickHouse/pull/99014) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `system.stack_trace` table now works on macOS, allowing introspection of all server thread stack traces. [#98982](https://github.com/ClickHouse/ClickHouse/pull/98982) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add per-server LDAP config option `` (default `false`) to control whether the LDAP client follows referrals. Disabling referral chasing avoids timeouts and hangs when searching from an Active Directory domain-root base DN. Referral-related log messages are moved from `warn` to `trace`. [#96765](https://github.com/ClickHouse/ClickHouse/pull/96765) ([paf91](https://github.com/paf91)). * We now track any data skipping indices that were used during query execution in the query_log table in a new column named `skip_indices`. Fixes [#78676](https://github.com/ClickHouse/ClickHouse/issues/78676). Original author @pheepa. [#87862](https://github.com/ClickHouse/ClickHouse/pull/87862) ([Grant Holly](https://github.com/grantholly-clickhouse)). * ACCESS_DENIED hints no longer reveal column names unless the user can show all required columns; database/table names remain visible in the hint. [#91067](https://github.com/ClickHouse/ClickHouse/pull/91067) ([filimonov](https://github.com/filimonov)). * Add a dedicated cleanup thread for MergeTree to prevent cleanup delays under heavy merge load. This resolves [#86181](https://github.com/ClickHouse/ClickHouse/issues/86181). [#91574](https://github.com/ClickHouse/ClickHouse/pull/91574) ([Amos Bird](https://github.com/amosbird)). * Reload cluster config if IPs of local server’s hostname changed, instead of IPs of any host. Fixes [#81215](https://github.com/ClickHouse/ClickHouse/issues/81215), [#70156](https://github.com/ClickHouse/ClickHouse/issues/70156) and [#65268](https://github.com/ClickHouse/ClickHouse/issues/65268). [#93726](https://github.com/ClickHouse/ClickHouse/pull/93726) ([Zhigao Hong](https://github.com/zghong)). * Allow optimize_aggregators_of_group_by_keys to correctly optimize aggregate functions in GROUPING SETS queries. [#93935](https://github.com/ClickHouse/ClickHouse/pull/93935) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Keeper-bench: report errors in metrics and generate json metrics file for `--input-request-log` mode. [#95748](https://github.com/ClickHouse/ClickHouse/pull/95748) ([Mohammad Lareb Zafar](https://github.com/zlareb1)). * Add new clause ROLE to CREATE USER. [#97074](https://github.com/ClickHouse/ClickHouse/pull/97074) ([Vitaly Baranov](https://github.com/vitlibar)). * You can now set internal_replication settings for a cluster created by the Replicated database. [#97228](https://github.com/ClickHouse/ClickHouse/pull/97228) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)). * New setting `allow_nullable_tuple_in_extracted_subcolumns` controls whether extracted `Tuple(...)` subcolumns from `Tuple`, `Variant`, `Dynamic` and `JSON` are returned as `Nullable(Tuple(...))` (`NULL` for missing rows) or as `Tuple(...)` (default tuple values for missing rows). Disabled by default, this setting can only be changed by restarting the server. [#97299](https://github.com/ClickHouse/ClickHouse/pull/97299) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Add information about deferred filters as a separate item to EXPLAIN query output (when using Row Policies/PREWHERE with FINAL). Related: [#91065](https://github.com/ClickHouse/ClickHouse/pull/91065). [#97374](https://github.com/ClickHouse/ClickHouse/pull/97374) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Enable `type_json_allow_duplicated_key_with_literal_and_nested_object` by default. It allows to avoid errors about duplicated keys during parsing of JSONs like `{"a" : 42, "a" : {"b" : 42}}` that can be formatted by ClickHouse from original JSON data `{"a" : 42, "a.b" : 42}`. [#97423](https://github.com/ClickHouse/ClickHouse/pull/97423) ([Pavel Kruglov](https://github.com/Avogar)). * Keeper improvement: `find_super_nodes` is a very useful command for debugging unexpected growth in the node count in Keeper. Unfortunately, if there are multiple super nodes, it's almost impossible to find more than one, because the command gets stuck forever traversing the children of the first encountered super node. This PR forbids traversing the children of super nodes. [#97819](https://github.com/ClickHouse/ClickHouse/pull/97819) ([pufit](https://github.com/pufit)). * Initial completion support for `clickhouse-keeper-client`. [#97828](https://github.com/ClickHouse/ClickHouse/pull/97828) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Flush async logging buffers in case of crash. [#97836](https://github.com/ClickHouse/ClickHouse/pull/97836) ([Azat Khuzhin](https://github.com/azat)). * Enable the impersonate feature by default (see [EXECUTE AS target_user](https://clickhouse.com/docs/sql-reference/statements/execute_as)). [#97870](https://github.com/ClickHouse/ClickHouse/pull/97870) ([Vitaly Baranov](https://github.com/vitlibar)). * Improve canceling queries with SQLite table engine by KILL QUERY and cancel query (Ctrl+C) in clickhouse-client. [#97944](https://github.com/ClickHouse/ClickHouse/pull/97944) ([Roman Vasin](https://github.com/rvasin)). * Add server setting `jemalloc_profiler_sampling_rate` to control jemalloc's `lg_prof_sample` and expose it as `jemalloc.prof.lg_sample` asynchronous metric. [#97945](https://github.com/ClickHouse/ClickHouse/pull/97945) ([Antonio Andelic](https://github.com/antonio2368)). * Support weights in concurrent bounded queue implementation. [#97962](https://github.com/ClickHouse/ClickHouse/pull/97962) ([Daniil Ivanik](https://github.com/divanik)). * Add sslmode to the allowed keys for PostgreSQL dictionary sources. Previously, sslmode was not in the dictionary_allowed_keys allowlist in PostgreSQLDictionarySource.cpp, making it impossible to configure SSL mode for PostgreSQL dictionary connections. This prevented dictionaries from connecting to PostgreSQL servers that require SSL (e.g., AWS RDS, which enforces SSL by default), as the connection would fail TLS negotiation and the server would reject the unencrypted fallback. [#98014](https://github.com/ClickHouse/ClickHouse/pull/98014) ([mcalfin](https://github.com/mcalfin)). * Show a clear "no such file" error when passing a non-existent file path to `clickhouse` or `clickhouse-local`, instead of a confusing generic message. [#98048](https://github.com/ClickHouse/ClickHouse/pull/98048) ([Raúl Marín](https://github.com/Algunenano)). * Text indexes can now be built on top of `Nullable([Fixed]String)` and `Array(Nullable([Fixed]String))` columns. [#98118](https://github.com/ClickHouse/ClickHouse/pull/98118) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Avoid dropping named collections that are dependencies of dictionary sources. [#98127](https://github.com/ClickHouse/ClickHouse/pull/98127) ([Pablo Marcos](https://github.com/pamarcos)). * Enable `grace_hash` join algorithm for queries with totals. [#98144](https://github.com/ClickHouse/ClickHouse/pull/98144) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Cancel background merges early in DROP DATABASE for ordinary shared merge tree. [#98161](https://github.com/ClickHouse/ClickHouse/pull/98161) ([Shaohua Wang](https://github.com/tiandiwonder)). * Improve canceling queries with MongoDB and MySQL by KILL QUERY and cancel query (Ctrl+C) in clickhouse-client. [#98187](https://github.com/ClickHouse/ClickHouse/pull/98187) ([Roman Vasin](https://github.com/rvasin)). * Remove NetlinkMetricsProvider and use procfs exclusively for per-thread taskstats metric collection. Netlink-based collection is problematic in containerized environments and has worse tail latency under contention. [#98229](https://github.com/ClickHouse/ClickHouse/pull/98229) ([Amos Bird](https://github.com/amosbird)). * Refactor Iceberg manifest file handling to fix issues with manifest file caching. [#98231](https://github.com/ClickHouse/ClickHouse/pull/98231) ([Daniil Ivanik](https://github.com/divanik)). * Now we take into account cases where the sorting key of the table can be an expression like `toDate(time)`, and can make the decisions about not-deferring expressions like this if they are part of filters. [#98237](https://github.com/ClickHouse/ClickHouse/pull/98237) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Add a new `MaxAllocatedEphemeralLockSequentialNumber` metric for the maximum sequential number allocated for ephemeral lock znodes in ZooKeeper. [#98243](https://github.com/ClickHouse/ClickHouse/pull/98243) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Update clickstack to version 2.20.0. [#98252](https://github.com/ClickHouse/ClickHouse/pull/98252) ([Aaron Knudtson](https://github.com/knudtty)). * Added a new profile event `KeeperRequestTotalWithSubrequests` that counts each subrequest within a multi-request individually, providing better visibility into the actual Keeper workload. The existing `KeeperRequestTotal` event continues to count each multi-request as a single request. [#98348](https://github.com/ClickHouse/ClickHouse/pull/98348) ([Antonio Andelic](https://github.com/antonio2368)). * `SYSTEM RELOAD DICTIONARIES` now reloads dictionaries in topological order so that dictionaries sourcing from other dictionaries see fresh data after reload. [#98356](https://github.com/ClickHouse/ClickHouse/pull/98356) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Restart the statistics cache after changing the MergeTree setting. [#98520](https://github.com/ClickHouse/ClickHouse/pull/98520) ([Han Fei](https://github.com/hanfei1991)). * Only "alive" replicas (that are connectable) participate in distributed index analysis. [#98521](https://github.com/ClickHouse/ClickHouse/pull/98521) ([Azat Khuzhin](https://github.com/azat)). * Add a setting `access_control_improvements.disallow_config_defined_profiles_for_sql_defined_users` (disabled/allowed by default) that disallows using config-defined settings profiles (except for the `default` profile) for SQL-defined users. [#98662](https://github.com/ClickHouse/ClickHouse/pull/98662) ([Alexander Tokmakov](https://github.com/tavplubix)). * Cap the number of nodes used in the automatic parallel replicas heuristic to the actual number of nodes in the cluster (instead of only the `max_parallel_replicas` setting). [#98668](https://github.com/ClickHouse/ClickHouse/pull/98668) ([Nikita Taranov](https://github.com/nickitat)). * Implement hedged requests and asynchronous reading for distributed index analysis. [#98724](https://github.com/ClickHouse/ClickHouse/pull/98724) ([Azat Khuzhin](https://github.com/azat)). * Deserialization of binary `AggregateFunction` states now requires consuming the full input. If extra redundant trailing bytes are present, ClickHouse throws an exception instead of accepting malformed state data. [#98786](https://github.com/ClickHouse/ClickHouse/pull/98786) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Make TRUNCATE DATABASE respond to query cancellation. [#98828](https://github.com/ClickHouse/ClickHouse/pull/98828) ([Shaohua Wang](https://github.com/tiandiwonder)). * Improve `keeper-bench` with request pipelining, warmup period, per-operation stats, reproducible seeds, and better error handling. [#98906](https://github.com/ClickHouse/ClickHouse/pull/98906) ([Antonio Andelic](https://github.com/antonio2368)). * Support SAMPLE clause in distributed index analysis. [#98931](https://github.com/ClickHouse/ClickHouse/pull/98931) ([Azat Khuzhin](https://github.com/azat)). * Show chart title in dashboard even when query returns empty result or encounters an error. [#98975](https://github.com/ClickHouse/ClickHouse/pull/98975) ([Yash ](https://github.com/Onyx2406)). * Analyzer error messages no longer dump all columns of a table (which could produce 150KB+ exceptions). Column lists are now capped at 10 entries. [#99002](https://github.com/ClickHouse/ClickHouse/pull/99002) ([Yash ](https://github.com/Onyx2406)). * Properly return column stats from sub-queries with joins so that parent query can use them for join reordering. [#99096](https://github.com/ClickHouse/ClickHouse/pull/99096) ([Alexander Gololobov](https://github.com/davenger)). * Mark ZooKeeper session as expired immediately when finalization starts, instead of waiting for the send thread to exit. This allows other threads to establish a new session without delay. [#99102](https://github.com/ClickHouse/ClickHouse/pull/99102) ([Raúl Marín](https://github.com/Algunenano)). * Use more math functions from LLVM-libc: `exp`, `exp2`, `expm1`, `fabs`, `fabsl`, `floor`, `fmodl`, `log`, `log2`, `logf`, `pow`, `scalbn`, `scalbnl`, `copysignl`, `nan`, `nanf`, `nanl`, and the `explogxf` shared constants. [#99118](https://github.com/ClickHouse/ClickHouse/pull/99118) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Reduce memory usage and fix potential duplicate output in `system.jemalloc_profile_text` collapsed format. [#99121](https://github.com/ClickHouse/ClickHouse/pull/99121) ([Antonio Andelic](https://github.com/antonio2368)). * Add `is_subrequest` column to `system.aggregated_zookeeper_log` to separate standalone requests from sub-requests inside Multi/MultiRead requests. Previously, sub-requests were aggregated into the same buckets as standalone requests, and since each sub-operation was logged with the total multi-request duration, the average latency became misleading. Sub-requests now have zero latency. [#99169](https://github.com/ClickHouse/ClickHouse/pull/99169) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Allow `ALTER TABLE MODIFY COLUMN x TTL ...` command without specifying column type. [#99208](https://github.com/ClickHouse/ClickHouse/pull/99208) ([Nikolay Degterinsky](https://github.com/evillique)). * Skip stale Keeper requests for sessions that have already disconnected, avoiding unnecessary Raft round-trips. The number of tracked finished sessions is capped by the `max_finished_sessions_cache_size` coordination setting. [#99246](https://github.com/ClickHouse/ClickHouse/pull/99246) ([Antonio Andelic](https://github.com/antonio2368)). * Support text index built on `mapValues(map)` with `IN` operator. [#99286](https://github.com/ClickHouse/ClickHouse/pull/99286) ([Anton Popov](https://github.com/CurtizJ)). * Shell-like completion support in clickhouse keeper-client (handle completions of arguments in quotes, i.e. `'foo ba'`, handle escaped arguments, i.e. `foo\ ba`, make `ls` print quoted nodes, if they have whitespaces). [#99312](https://github.com/ClickHouse/ClickHouse/pull/99312) ([Azat Khuzhin](https://github.com/azat)). * Prevent Keeper `mntr` command from getting stuck because of lock contention. [#99472](https://github.com/ClickHouse/ClickHouse/pull/99472) ([Antonio Andelic](https://github.com/antonio2368)). * Reduce lock contention in Keeper dispatcher by invoking callbacks and dispatching read requests outside mutex scope, and add profiled lock guards for observability. [#99751](https://github.com/ClickHouse/ClickHouse/pull/99751) ([Antonio Andelic](https://github.com/antonio2368)). * Tolerate missing padding at the end of the last block of parquet files. [#99857](https://github.com/ClickHouse/ClickHouse/pull/99857) ([Seva Potapov](https://github.com/seva-potapov)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fixes how an Alias table target is saved as a DDL dependency when not fully qualified: it's now saved with the Alias table database instead of the session database. [#95175](https://github.com/ClickHouse/ClickHouse/pull/95175) ([Enric Calabuig](https://github.com/eclbg)). * Fix wrong result or exception during reading subcolumns of ALIAS columns. [#95408](https://github.com/ClickHouse/ClickHouse/pull/95408) ([Pavel Kruglov](https://github.com/Avogar)). * Fix missing column when non-standard identifier alias in JOIN with old analyzer. Fixes [#25594](https://github.com/ClickHouse/ClickHouse/issues/25594), [#47288](https://github.com/ClickHouse/ClickHouse/issues/47288) and [#53263](https://github.com/ClickHouse/ClickHouse/issues/53263). [#95679](https://github.com/ClickHouse/ClickHouse/pull/95679) ([Zhigao Hong](https://github.com/zghong)). * Fixes a crash in Kusto dialect functions `bin()`, `bin_at()`, `extract()`, and `indexof()` when empty arguments are provided. [#95736](https://github.com/ClickHouse/ClickHouse/pull/95736) ([NeedmeFordev](https://github.com/spider-yamet)). * Forbids mounting local_object_storage (which is used by datalakes above local filesystem and maybe by LocalDisk) anywhere apart from user_files_path in clickhouse-client. [#96201](https://github.com/ClickHouse/ClickHouse/pull/96201) ([Daniil Ivanik](https://github.com/divanik)). * In `DeltaLake` table engine fix logical race on snapshot version change, remove redundant heavy snapshot reloads. [#96226](https://github.com/ClickHouse/ClickHouse/pull/96226) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix logical error on attaching a part in MergeTree if there were several chained renames between detaching and attaching. [#96351](https://github.com/ClickHouse/ClickHouse/pull/96351) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a bug where explicit settings sent alongside `compatibility` in the same request could be silently ignored when their value matched the server default. [#97078](https://github.com/ClickHouse/ClickHouse/pull/97078) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix the client reporting `NETWORK_ERROR` instead of the actual parsing error (with the correct row number) when an INSERT with parallel parsing encounters invalid data. [#97339](https://github.com/ClickHouse/ClickHouse/pull/97339) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `sumCount` aggregate function not being able to read older serialized states after introduction of `Nullable(Tuple)`. Closes [#97370](https://github.com/ClickHouse/ClickHouse/issues/97370). [#97502](https://github.com/ClickHouse/ClickHouse/pull/97502) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix exception in tuple comparison involving `Nothing` type elements (e.g., comparing with `NULL` tuple elements) when used with `GROUPING SETS` and `ORDER BY`. [#97509](https://github.com/ClickHouse/ClickHouse/pull/97509) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix non-deterministic `uncompressed_hash` computation for Compact MergeTree parts when multiple compression codecs are used, which could cause incorrect deduplication behavior. [#97522](https://github.com/ClickHouse/ClickHouse/pull/97522) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error about missing stream during INSERT SELECT with JSON and buckets in shared data. Closes [#97331](https://github.com/ClickHouse/ClickHouse/issues/97331). [#97523](https://github.com/ClickHouse/ClickHouse/pull/97523) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `MEMORY_LIMIT_EXCEEDED` exceptions being incorrectly reported as `CORRUPTED_DATA` during SummingMergeTree and CoalescingMergeTree merges. [#97537](https://github.com/ClickHouse/ClickHouse/pull/97537) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix "Context has expired" exception for correlated subqueries containing table functions like `url()`. [#97544](https://github.com/ClickHouse/ClickHouse/pull/97544) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exceptions and incorrect behavior in `optimize_syntax_fuse_functions` with aggregate projections, Date types, and column name preservation. [#97545](https://github.com/ClickHouse/ClickHouse/pull/97545) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove incorrect `replaceRegexpOne` to `extract` query rewrite that produced wrong results when the regexp didn't match; also fix exception when `replaceRegexpOne` is used with `GROUP BY ... WITH CUBE` and `group_by_use_nulls=1`. [#97546](https://github.com/ClickHouse/ClickHouse/pull/97546) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `DROP DATABASE` with `database_atomic_wait_for_drop_and_detach_synchronously` hanging indefinitely when the query is killed. [#97586](https://github.com/ClickHouse/ClickHouse/pull/97586) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `KILL QUERY` not being able to terminate queries stuck in `WITH FILL` generation, dictionary loading via `dictGet`, or `ALTER DELETE` with `mutations_sync=1` on `ReplicatedMergeTree`. [#97589](https://github.com/ClickHouse/ClickHouse/pull/97589) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `loop` table function was calling `inner_storage->read()` directly, bypassing the interpreter layer where row policies, column-level grants, and other security checks are applied. This allowed a user restricted by row policies to read all rows via `loop(table)` even when a direct SELECT returned zero rows. [#97682](https://github.com/ClickHouse/ClickHouse/pull/97682) ([pufit](https://github.com/pufit)). * Fixes incorrect partition pruning in cases of using pre-epoch DateTime64 with `toDate()` function. [#97746](https://github.com/ClickHouse/ClickHouse/pull/97746) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * After this patch, `hasPartitionId` will return false if another partition with a higher partition ID exists in the data part set. [#97748](https://github.com/ClickHouse/ClickHouse/pull/97748) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix possible crashes during reading of empty granules in advanced shared data in JSON. Closes [#97563](https://github.com/ClickHouse/ClickHouse/issues/97563). [#97778](https://github.com/ClickHouse/ClickHouse/pull/97778) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `Cannot schedule a file` `LOGICAL_ERROR` on `INSERT` into `Distributed` due to race between `DROP` and `INSERT`. [#97822](https://github.com/ClickHouse/ClickHouse/pull/97822) ([Azat Khuzhin](https://github.com/azat)). * Fixed ClickHouse server crash/assert in call to `mapContainsKey/mapContainsKeyLike` with `tokenbf_v1` skip index. [#97826](https://github.com/ClickHouse/ClickHouse/pull/97826) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fix LOGICAL_ERROR exceptions caused by `LowCardinality` inside compound types (`Variant`, `Dynamic`, `Tuple`) in `concatWithSeparator`, `format`, `IN` subqueries, `GLOBAL IN`, and joins with runtime filters. [#97831](https://github.com/ClickHouse/ClickHouse/pull/97831) ([Raúl Marín](https://github.com/Algunenano)). * Fix `LOGICAL_ERROR` exception `Chunk info was not set for chunk in MergingAggregatedTransform` when using `ARRAY JOIN` with `merge()` table function over multiple Distributed tables combined with `GROUP BY`. [#97838](https://github.com/ClickHouse/ClickHouse/pull/97838) ([Raúl Marín](https://github.com/Algunenano)). * Fix server crash (`std::terminate`) caused by an uncaught exception in the HTTP connection pool destructor when the connection group hard limit is reached under high concurrency. The exception `HTTP_CONNECTION_LIMIT_REACHED` could escape `~PooledConnection` when recycling a connection back to the pool, leading to `SIGABRT`. [#97850](https://github.com/ClickHouse/ClickHouse/pull/97850) ([Antonio Andelic](https://github.com/antonio2368)). * Fix incorrect result when `grace_hash` algorithm is used with non-equi joins and left block cannot be processed completely because of the size constraints on the join result. [#97866](https://github.com/ClickHouse/ClickHouse/pull/97866) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix performance inefficiency in DeltaLake metadata scanning introduced in [#96686](https://github.com/ClickHouse/ClickHouse/pull/96686). [#97880](https://github.com/ClickHouse/ClickHouse/pull/97880) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix data race in ZooKeeper client between sendThread and receiveThread. [#97887](https://github.com/ClickHouse/ClickHouse/pull/97887) ([Pablo Marcos](https://github.com/pamarcos)). * Fixes a bug where it was not possible to use CTE with distributed insert selects. Continuation of https://github.com/ClickHouse/ClickHouse/pull/87789. Closes [#95837](https://github.com/ClickHouse/ClickHouse/issues/95837). [#97889](https://github.com/ClickHouse/ClickHouse/pull/97889) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix exception from `CachedOnDiskReadBufferFromFile::readBigAt`. Closes [#97325](https://github.com/ClickHouse/ClickHouse/issues/97325). [#97890](https://github.com/ClickHouse/ClickHouse/pull/97890) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix `LOGICAL_ERROR` exception in `Alias` engine with materialized columns due to column mismatch. Closes [#97907](https://github.com/ClickHouse/ClickHouse/issues/97907). [#97921](https://github.com/ClickHouse/ClickHouse/pull/97921) ([Kai Zhu](https://github.com/nauu)). * Fix Keeper data loss after restart when using Azure Blob Storage with `s3_plain` metadata for log storage. [#97987](https://github.com/ClickHouse/ClickHouse/pull/97987) ([Antonio Andelic](https://github.com/antonio2368)). * Fix JIT miscompilation of `sign` function for integer types wider than `Int8` — values outside the -128..127 range could produce incorrect sign. [#98012](https://github.com/ClickHouse/ClickHouse/pull/98012) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `DUPLICATE_COLUMN` exception and silent NULLs when reading Delta Lake tables that use column mapping "name" mode with struct fields whose names contain dots (e.g. `` STRUCT<`a.foo`: STRING, `b.foo`: STRING> ``). [#98013](https://github.com/ClickHouse/ClickHouse/pull/98013) ([Caio Ishizaka Costa](https://github.com/ch-caioishizaka)). * Fix mutation after lightweight update and secondary indices. [#98044](https://github.com/ClickHouse/ClickHouse/pull/98044) ([Raúl Marín](https://github.com/Algunenano)). * Fix incorrect result of FINAL queries when mixing primary key and non primary key skip indexes. [#98097](https://github.com/ClickHouse/ClickHouse/pull/98097) ([Raúl Marín](https://github.com/Algunenano)). * Enforce READ ON FILE checks for scalar file() and DESCRIBE TABLE file(). [#98115](https://github.com/ClickHouse/ClickHouse/pull/98115) ([Nikolay Degterinsky](https://github.com/evillique)). * Fixes a crash where querying files with a glob pattern (e.g., `file('dir/**', 'LineAsString')`) would throw an unhandled filesystem exception (`STD_EXCEPTION`) if the directory contained a dangling symlink. Dangling symlinks are now silently skipped, and the query returns results from all valid files. [#98143](https://github.com/ClickHouse/ClickHouse/pull/98143) ([Mark Andreev](https://github.com/mrk-andreev)). * Fix segfault in query plan optimization when converting outer join to inner join with `arrayJoin` in filter expression. [#98147](https://github.com/ClickHouse/ClickHouse/pull/98147) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ProtobufList` format not working with Kafka engine due to read state not being reset between messages. [#98151](https://github.com/ClickHouse/ClickHouse/pull/98151) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error with analyzer_compatibility_join_using_top_level_identifier and ARRAY JOIN, closes [#98164](https://github.com/ClickHouse/ClickHouse/issues/98164). [#98179](https://github.com/ClickHouse/ClickHouse/pull/98179) ([Vladimir Cherkasov](https://github.com/vdimir)). * Set `Watch` component for watch responses in `aggregated_zookeeper_log` instead of leaving it empty. [#98202](https://github.com/ClickHouse/ClickHouse/pull/98202) ([Antonio Andelic](https://github.com/antonio2368)). * If the partition key columns are not covered by the sorting key, then partition pruning could incorrectly skip partitions containing rows that should "win" during FINAL deduplication. [#98242](https://github.com/ClickHouse/ClickHouse/pull/98242) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix logical error "Bad cast from type DB::ColumnConst to DB::ColumnArray" in `kql_array_sort_asc`/`kql_array_sort_desc` when called with constant array arguments. [#98251](https://github.com/ClickHouse/ClickHouse/pull/98251) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix out-of-bounds access in `ColumnConst::getExtremes` that could cause a crash when `extremes = 1` is enabled. [#98263](https://github.com/ClickHouse/ClickHouse/pull/98263) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix potential deadlock when two concurrent `MOVE PARTITION` operations work with the same pair of tables in opposite directions. [#98264](https://github.com/ClickHouse/ClickHouse/pull/98264) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * HTTP server now returns an error message in the response body for 400 Bad Request responses caused by malformed headers, instead of an empty body. [#98268](https://github.com/ClickHouse/ClickHouse/pull/98268) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong results with distributed index analysis (experimental feature) and query condition cache. [#98269](https://github.com/ClickHouse/ClickHouse/pull/98269) ([Azat Khuzhin](https://github.com/azat)). * Fixed LOGICAL_ERROR exception "Invalid binary search result in `MergeTreeSetIndex`" triggered by `toDate` conversion on key columns with data crossing the 65535 boundary. [#98276](https://github.com/ClickHouse/ClickHouse/pull/98276) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception when a RIGHT JOIN wrapped in a CROSS JOIN is swapped by the `query_plan_join_swap_table` optimization in the legacy join step code path. [#98279](https://github.com/ClickHouse/ClickHouse/pull/98279) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Validate corrupted data during `DDSketch` deserialization to prevent segfaults, exceptions, infinite loops, and OOM when reading corrupted `quantilesDD` aggregate function states. [#98284](https://github.com/ClickHouse/ClickHouse/pull/98284) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR "Trying to execute PLACEHOLDER action" when correlated columns from outer queries are referenced inside lambda functions such as `arrayMap`. [#98285](https://github.com/ClickHouse/ClickHouse/pull/98285) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception in `caseWithExpression` when `CASE` expression involves `materialize(NULL)` or other `Nullable(Nothing)` arguments. [#98290](https://github.com/ClickHouse/ClickHouse/pull/98290) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix bad cast exception when filtering `_table` virtual column in `merge` table function. [#98291](https://github.com/ClickHouse/ClickHouse/pull/98291) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix sporadic deduplication failure where re-inserts were incorrectly deduplicated due to inconsistent cleanup ordering between `blocks/` and `deduplication_hashes/` ZooKeeper directories. [#98293](https://github.com/ClickHouse/ClickHouse/pull/98293) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when `ORDER BY ... WITH FILL` is used together with `LIMIT BY`. [#98361](https://github.com/ClickHouse/ClickHouse/pull/98361) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix silent data corruption when inserting a Parquet/Arrow `Date` column into an `Enum` column — now properly rejects the incompatible type conversion instead of storing invalid enum values. [#98364](https://github.com/ClickHouse/ClickHouse/pull/98364) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when reading an Arrow file with `Array` column into a table with `Nested` column. [#98365](https://github.com/ClickHouse/ClickHouse/pull/98365) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `MATERIALIZE INDEX` and `MATERIALIZE PROJECTION` mutations getting stuck when the index or projection is dropped before the mutation finishes. [#98369](https://github.com/ClickHouse/ClickHouse/pull/98369) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when reading from `Nullable(Tuple(...))` where a Tuple element name collides with the Nullable `null` subcolumn. [#98372](https://github.com/ClickHouse/ClickHouse/pull/98372) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "Column ... query tree node does not have valid source node" when joining a `Merge` table (wrapping a `Distributed` table) with another table. [#98376](https://github.com/ClickHouse/ClickHouse/pull/98376) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect Parquet `Bool` to `FixedString` conversion in native V3 reader that produced raw bytes instead of string representation. [#98378](https://github.com/ClickHouse/ClickHouse/pull/98378) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `tryGetColumnDescription` to filter subcolumns by parent column kind, consistent with other column lookup methods. [#98391](https://github.com/ClickHouse/ClickHouse/pull/98391) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Accept base64 credentials without padding in HTTP Basic Auth. Some HTTP clients omit trailing `=` padding in the `Authorization: Basic` header, which previously caused authentication failures. [#98392](https://github.com/ClickHouse/ClickHouse/pull/98392) ([Amos Bird](https://github.com/amosbird)). * Fix incorrect partition pruning results after merging parts with `Nullable` partition key columns, caused by wrong min-max index bounds. [#98405](https://github.com/ClickHouse/ClickHouse/pull/98405) ([Amos Bird](https://github.com/amosbird)). * Fix rare exception in the pipeline executor, that could manifest as a `Received signal 6` (only in debug builds), when pipeline expansion races with query cancellation. [#98428](https://github.com/ClickHouse/ClickHouse/pull/98428) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "Column identifier is already registered" when `count_distinct_optimization` is used with a `QUALIFY` clause. [#98433](https://github.com/ClickHouse/ClickHouse/pull/98433) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "cannot be inside Nullable type" when using `IN`/`NOT IN` with `LowCardinality` column arguments (e.g. `a NOT IN (b)` where `a` is `LowCardinality(String)`). [#98443](https://github.com/ClickHouse/ClickHouse/pull/98443) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Pipeline stuck" exception in `full_sorting_merge` joins caused by a deadlock in `PingPongProcessor` when the `FilterBySetOnTheFly` optimization created a circular dependency with `MergeJoinTransform`. [#98454](https://github.com/ClickHouse/ClickHouse/pull/98454) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception "Projection cannot increase the number of rows in a block" when merging parts with TTL that deletes all rows and an aggregate projection with a constant `GROUP BY` key. [#98458](https://github.com/ClickHouse/ClickHouse/pull/98458) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception when `CROSS JOIN` is used together with `INNER JOIN USING`. [#98459](https://github.com/ClickHouse/ClickHouse/pull/98459) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix null pointer dereference in `dictGetOrDefault` when the key argument is `Nullable`. [#98460](https://github.com/ClickHouse/ClickHouse/pull/98460) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception in `DISTINCT` queries when using aggregate projections and `materialize` causes `LowCardinality` type differences between the query and the projection. [#98462](https://github.com/ClickHouse/ClickHouse/pull/98462) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR exception when `arrayJoin` is used in a filter expression with OUTER JOIN and `join_use_nulls` enabled. [#98464](https://github.com/ClickHouse/ClickHouse/pull/98464) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception "Replica decided to read in WithOrder mode, not in ReverseOrder" when using parallel replicas with `optimize_aggregation_in_order`. [#98467](https://github.com/ClickHouse/ClickHouse/pull/98467) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed ClickHouse Keeper disconnecting Java ZooKeeper clients after `addWatch` request. The Java client expects a 4-byte `ErrorResponse` body in the `addWatch` response, but Keeper was sending an empty body, causing `EOFException` and session disconnect. This broke Apache Curator's `CuratorCache` and any Java application using persistent watches. Fixes [#98079](https://github.com/ClickHouse/ClickHouse/issues/98079). [#98499](https://github.com/ClickHouse/ClickHouse/pull/98499) ([Antonio Andelic](https://github.com/antonio2368)). * Fix `zk_followers` and `zk_synced_followers` Keeper metrics not decreasing when a follower goes down. Add new `zk_learners` and `zk_synced_non_voting_followers` metrics to the `mntr` four-letter-word command. Fixes [#54173](https://github.com/ClickHouse/ClickHouse/issues/54173). [#98504](https://github.com/ClickHouse/ClickHouse/pull/98504) ([Antonio Andelic](https://github.com/antonio2368)). * Fix a LOGICAL_ERROR exception in `renameAndCommitEmptyParts` that could occur when `TRUNCATE TABLE` runs concurrently with `OPTIMIZE TABLE` using MergeTree transactions. [#98508](https://github.com/ClickHouse/ClickHouse/pull/98508) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed Keeper's secure raft port ignoring `cipherList` and `dhParamsFile` from `openSSL` configuration, always using defaults instead of user-specified values. Close [#51188](https://github.com/ClickHouse/ClickHouse/issues/51188). [#98509](https://github.com/ClickHouse/ClickHouse/pull/98509) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed misleading Keeper log messages like "Receiving request for session X took 9963 ms" where the reported time was actually spent waiting idle in `poll()` between heartbeats, not performing the operation itself. Fixes [#79026](https://github.com/ClickHouse/ClickHouse/issues/79026). [#98510](https://github.com/ClickHouse/ClickHouse/pull/98510) ([Antonio Andelic](https://github.com/antonio2368)). * Fix unexpected result with read_in_order_use_virtual_row and monotonic functions, close [#97837](https://github.com/ClickHouse/ClickHouse/issues/97837). [#98514](https://github.com/ClickHouse/ClickHouse/pull/98514) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix `LOGICAL_ERROR: Not-ready Set is passed as the second argument for function 'in'` when using `PREWHERE` with `IN` subquery on MergeTree tables. [#98522](https://github.com/ClickHouse/ClickHouse/pull/98522) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix Keeper TCP connections preventing graceful server shutdown by not responding to shutdown signal. [#98525](https://github.com/ClickHouse/ClickHouse/pull/98525) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "Sorting column wasn't found in the ActionsDAG's outputs" when `query_plan_convert_join_to_in` is enabled with `query_plan_merge_expressions = 0`. [#98526](https://github.com/ClickHouse/ClickHouse/pull/98526) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix MongoDB dictionary source failing with named collections. Closes [#97840](https://github.com/ClickHouse/ClickHouse/issues/97840). [#98528](https://github.com/ClickHouse/ClickHouse/pull/98528) ([Pablo Marcos](https://github.com/pamarcos)). * Fixed LOGICAL_ERROR when Identifier is empty after parameter substitution. [#98530](https://github.com/ClickHouse/ClickHouse/pull/98530) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)). * Fix pipeline deadlock when using `sort_overflow_mode = 'break'` together with window functions. [#98543](https://github.com/ClickHouse/ClickHouse/pull/98543) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix column rollback in Buffer engine during handling an exception during appending a new block. Old logic could lead to corrupted in-memory state of columns. [#98551](https://github.com/ClickHouse/ClickHouse/pull/98551) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed exception `Bad cast from type ColumnConst to ColumnDynamic` in null-safe comparison (`<=>` / `IS NOT DISTINCT FROM`) with const `Dynamic` or `Variant` columns and `NULL`. Also fixed `IS DISTINCT FROM` with `Dynamic`/`Variant` vs `NULL` always incorrectly returning 0. [#98553](https://github.com/ClickHouse/ClickHouse/pull/98553) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed usage of the text index with other skip indexes. Previously, logical errors such as "Trying to get non-existing mark" could be thrown when a query filter utilized a text index and other regular skip indexes simultaneously. [#98555](https://github.com/ClickHouse/ClickHouse/pull/98555) ([Anton Popov](https://github.com/CurtizJ)). * Fix logical error "TABLE_FUNCTION is not allowed in expression context" when a table function with an alias appears multiple times in the same query scope (e.g. in both `PREWHERE` and `QUALIFY` clauses). [#98557](https://github.com/ClickHouse/ClickHouse/pull/98557) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix distributed index analysis with expressions (not just columns) in PK (leads to zero filtering of redundant granules on remote replicas). [#98561](https://github.com/ClickHouse/ClickHouse/pull/98561) ([Azat Khuzhin](https://github.com/azat)). * Disallow dropping column when its subcolumns are used in other columns default/alias expressions and use analyzer for default expressions on alter drop column. [#98569](https://github.com/ClickHouse/ClickHouse/pull/98569) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Fix S3 requests being incorrectly retried on non-retryable errors (including `HTTP_CONNECTION_LIMIT_REACHED`) in the HTTP client. [#98598](https://github.com/ClickHouse/ClickHouse/pull/98598) ([Sema Checherinda](https://github.com/CheSema)). * Fixes a decimal overflow when partition pruning with DateTime64. [#98628](https://github.com/ClickHouse/ClickHouse/pull/98628) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix two bugs in JIT expression compilation: a copy-paste error in `nativeCast` type checking that made integer-to-integer and float-to-float cast branches unreachable, and incorrect `nullptr` TargetMachine passed to LLVM `PassBuilder` preventing target-specific optimization passes from being registered. [#98660](https://github.com/ClickHouse/ClickHouse/pull/98660) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix RBAC bypass that allowed users to DESCRIBE any table via `remote()`, `remoteSecure()`, `cluster()`, or `clusterAllReplicas()` pointed at localhost, without requiring `SHOW_COLUMNS` privilege. [#98669](https://github.com/ClickHouse/ClickHouse/pull/98669) ([pufit](https://github.com/pufit)). * Fix `BAD_GET` exception and incorrect query results when a non-boolean expression (e.g. `sin(col)`) is used in both WHERE and SELECT with a JOIN, due to filter push-down optimization corrupting shared DAG nodes. [#98681](https://github.com/ClickHouse/ClickHouse/pull/98681) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR "Replica decided to read in Default mode, not in WithOrder" when using `read_in_order_through_join` with parallel replicas. [#98685](https://github.com/ClickHouse/ClickHouse/pull/98685) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "Bad cast from type `DB::TableFunctionNode` to `DB::QueryNode`" when using `input` table function as an argument of `remote`. [#98694](https://github.com/ClickHouse/ClickHouse/pull/98694) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix outdated data parts resurrection caused by incorrectly cleaning up empty covering parts. [#98698](https://github.com/ClickHouse/ClickHouse/pull/98698) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix exception in `LogicalExpressionOptimizerPass` when a boolean function in an `equals` comparison returns a `Variant` type. [#98712](https://github.com/ClickHouse/ClickHouse/pull/98712) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `parseDateTimeBestEffort` incorrectly parsing words starting with month/weekday prefixes. Closes [#97965](https://github.com/ClickHouse/ClickHouse/issues/97965). [#98742](https://github.com/ClickHouse/ClickHouse/pull/98742) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `UNKNOWN_IDENTIFIER` exception when querying `merge()` table function or `Merge` engine over tables with JSON columns that have different parameters (e.g. different `SKIP` fields) and ALIAS columns referencing JSON sub-paths, with the new analyzer enabled. Closes [#97812](https://github.com/ClickHouse/ClickHouse/issues/97812). [#98753](https://github.com/ClickHouse/ClickHouse/pull/98753) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `optimize_skip_unused_shards` optimization with the analyzer in case `Distributed` storage is used in a `View`. [#98754](https://github.com/ClickHouse/ClickHouse/pull/98754) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix tuple subcolumn access by name (e.g. `SELECT x.a` for `Tuple(a UUID, b Int32)`) for external tables passed via `--external` in `clickhouse-client`. Closes [#96925](https://github.com/ClickHouse/ClickHouse/issues/96925). [#98755](https://github.com/ClickHouse/ClickHouse/pull/98755) ([Pavel Kruglov](https://github.com/Avogar)). * Fix `reverseUTF8` exception on invalid (truncated) UTF-8 input. [#98770](https://github.com/ClickHouse/ClickHouse/pull/98770) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix detecting set skip index usefulness with OR with false (i.e. or(x, 0)) predicate. [#98776](https://github.com/ClickHouse/ClickHouse/pull/98776) ([Azat Khuzhin](https://github.com/azat)). * Fix a `LOGICAL_ERROR` exception (Block structure mismatch in `removeUnusedColumns`) that could occur with `FINAL` + `PREWHERE` + constant `WHERE` expression + column-independent aggregates like `count()`. [#98778](https://github.com/ClickHouse/ClickHouse/pull/98778) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Make `system.trace_log` entries for ClickHouse dictionaries' auto-reloads have non-empty query IDs. [#98784](https://github.com/ClickHouse/ClickHouse/pull/98784) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixes a crash where we could de-reference a null pointer in system tables created between the time when we snapshot the tables in the `IDatabaseTablesIterator::table()` call and the tables changing in another thread during later iteration. [#98792](https://github.com/ClickHouse/ClickHouse/pull/98792) ([Grant Holly](https://github.com/grantholly-clickhouse)). * Fix `SYSTEM START REPLICATED VIEW` not waking up the refresh task. [#98797](https://github.com/ClickHouse/ClickHouse/pull/98797) ([Pablo Marcos](https://github.com/pamarcos)). * Fix exception "Inconsistent table names" when using `view()` table function containing JOINs inside another JOIN (only with old analyzer). [#98809](https://github.com/ClickHouse/ClickHouse/pull/98809) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix adjusting RLIMIT_SIGPENDING (via pending_signals). [#98829](https://github.com/ClickHouse/ClickHouse/pull/98829) ([Azat Khuzhin](https://github.com/azat)). * Fix exception when composing `loop` with cluster table functions. [#98860](https://github.com/ClickHouse/ClickHouse/pull/98860) ([Konstantin Bogdanov](https://github.com/thevar1able)). * LEFT ANTI JOIN with multiple join key columns returned wrong results when `enable_join_runtime_filters=1` (which is default). [#98871](https://github.com/ClickHouse/ClickHouse/pull/98871) ([Alexander Gololobov](https://github.com/davenger)). * Fix `WITH FILL STALENESS` producing extra filled rows when data is read in multiple chunks (e.g., with small `index_granularity`). [#98895](https://github.com/ClickHouse/ClickHouse/pull/98895) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "RPNBuilderFunctionTreeNode has A arguments, attempted to get argument at index B" LOGICAL_ERROR. [#98900](https://github.com/ClickHouse/ClickHouse/pull/98900) ([Azat Khuzhin](https://github.com/azat)). * Fix memory tracking drift caused by failed allocations not being rolled back, `nallocx(0)` undefined behavior, and off-by-one in global peak tracking. Extend tracking to cover `io_uring` ring buffers. [#98915](https://github.com/ClickHouse/ClickHouse/pull/98915) ([Antonio Andelic](https://github.com/antonio2368)). * Forbid attaching to local data lake tables outside user paths, not only creating them. [#98936](https://github.com/ClickHouse/ClickHouse/pull/98936) ([Daniil Ivanik](https://github.com/divanik)). * Fix a race condition that could cause a "ReadBuffer is canceled" exception in queries using `urlCluster` or similar cluster table functions. [#98955](https://github.com/ClickHouse/ClickHouse/pull/98955) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception in financial functions (`financialNetPresentValue`, `financialInternalRateOfReturn`, etc.) when `BFloat16` type arguments are passed. [#98958](https://github.com/ClickHouse/ClickHouse/pull/98958) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix skip indexes (and primary key conditions) not being applied for ALIAS columns when query plan expression merging is disabled (query_plan_merge_expressions = 0 or query_plan_enable_optimizations = 0). [#98960](https://github.com/ClickHouse/ClickHouse/pull/98960) ([Peng](https://github.com/fastio)). * Increment `InsertQuery` ProfileEvent for async inserts. Closes [#98626](https://github.com/ClickHouse/ClickHouse/issues/98626). [#98962](https://github.com/ClickHouse/ClickHouse/pull/98962) ([Narasimha Pakeer](https://github.com/npakeer)). * Fix exception "Inconsistent KeyCondition behavior" in debug builds when primary key contains NaN float values, by making `accurateLess` and `accurateEquals` handle NaN consistently with ClickHouse sort order. Closes [#98075](https://github.com/ClickHouse/ClickHouse/issues/98075). [#98964](https://github.com/ClickHouse/ClickHouse/pull/98964) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * SummingMergeTree no longer sums Bool (and other domain type) columns. Bool values are kept as-is instead of being arithmetically summed. [#98976](https://github.com/ClickHouse/ClickHouse/pull/98976) ([Yash ](https://github.com/Onyx2406)). * Fixes an exception Scalar doesn't exist that occurred when querying a remote shard with `optimize_const_name_size` set and `enable_scalar_subquery_optimization` = 0. Large constants replaced with `__getScalar` references in the remote query were not being sent to the shard, causing the query to fail. [#98979](https://github.com/ClickHouse/ClickHouse/pull/98979) ([andriibeee](https://github.com/andriibeee)). * Fix `NOT_FOUND_COLUMN_IN_BLOCK` for some queries with `GROUP BY` and expressions that include inverse dictionary lookup, `Date/DateTime` conversion comparisons, and tuple comparisons. Closes [#98888](https://github.com/ClickHouse/ClickHouse/issues/98888). [#98980](https://github.com/ClickHouse/ClickHouse/pull/98980) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed undefined behavior (null pointer dereference) when altering a version/sign/is_deleted column to `EPHEMERAL` or `ALIAS` in MergeTree engines. Such alterations are now properly rejected. [#98985](https://github.com/ClickHouse/ClickHouse/pull/98985) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed an issue where `system.grants` omitted the regular expression parameters for `URL` and `S3` grants in the `access_object` column. [#98987](https://github.com/ClickHouse/ClickHouse/pull/98987) ([DQ](https://github.com/il9ue)). * Fixed Iceberg BigLake reads: ADC credentials are now forwarded to the GCS S3 client (fixing 403 errors), OAuth2 credentials are URL-encoded before sending (fixing auth failures for tokens with special characters), and namespace traversal no longer aborts on BigLake HTTP 400 responses. [#98998](https://github.com/ClickHouse/ClickHouse/pull/98998) ([Nikita Fomichev](https://github.com/fm4v)). * Fixed `clickhouse-client` failing to switch timezone when `TZ` environment variable uses POSIX file path syntax (e.g. `TZ=:/etc/localtime`). [#99000](https://github.com/ClickHouse/ClickHouse/pull/99000) ([Yash ](https://github.com/Onyx2406)). * Fix incorrect or less pruning when `startsWith`, `LIKE`, `NOT LIKE` used with `FixedString` column. Additionally, `FixedString` to `String` cast function can now prune granules when wrapped around key column. Closes [#98940](https://github.com/ClickHouse/ClickHouse/issues/98940). [#99001](https://github.com/ClickHouse/ClickHouse/pull/99001) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed `windowFunnel` with `strict_deduplication` returning incorrect level when a duplicate event was encountered. [#99003](https://github.com/ClickHouse/ClickHouse/pull/99003) ([Yash ](https://github.com/Onyx2406)). * Fixes a bug where EXISTS would ignore LIMIT and OFFSET clauses in subqueries, causing incorrect results when the subquery returned no rows due to an offset or a zero limit. Closes [#88722](https://github.com/ClickHouse/ClickHouse/issues/88722). [#99005](https://github.com/ClickHouse/ClickHouse/pull/99005) ([andriibeee](https://github.com/andriibeee)). * Fix "Block structure mismatch" exception when filter push-down optimization encounters an AND expression that short-circuits to a constant with `GROUPING SETS`. [#99010](https://github.com/ClickHouse/ClickHouse/pull/99010) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when reading patch parts (lightweight updates) without `_part_offset` column in the query plan. [#99023](https://github.com/ClickHouse/ClickHouse/pull/99023) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * A query like `SELECT * FROM table WHERE pk_id = ''` where `pk_id` is the primary key and of `String` type will now correctly use the primary key index for filtering granules. [#99027](https://github.com/ClickHouse/ClickHouse/pull/99027) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fix `DEPENDENCIES_NOT_FOUND` exception in Kafka engine when materialized view is detached while the background thread is streaming data. [#99028](https://github.com/ClickHouse/ClickHouse/pull/99028) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception when creating a table with an `EPHEMERAL` column that has the same name as a virtual column (e.g., `_part_offset`). [#99031](https://github.com/ClickHouse/ClickHouse/pull/99031) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix misleading "inflate failed: buffer error" when reading non-existent compressed files via `url()` table function with glob patterns. Now returns empty result as expected when `http_skip_not_found_url_for_globs` is enabled. [#99034](https://github.com/ClickHouse/ClickHouse/pull/99034) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix server crash (std::terminate) when executing `ALTER TABLE ... DROP PART` on a patch part after a schema change (e.g. ADD COLUMN). The crash was caused by missing system columns (`_part`) in the empty coverage part metadata, leading to an uncaught exception inside a NOEXCEPT_SCOPE. [#99036](https://github.com/ClickHouse/ClickHouse/pull/99036) ([Peng](https://github.com/fastio)). * ClickHouse server process could crash if there was a memory limit exceeded exception thrown during a _cached disk read_. That is now fixed. [#99042](https://github.com/ClickHouse/ClickHouse/pull/99042) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fix `LOGICAL_ERROR` when querying a table that has both a ROW POLICY and an ALIAS column using `dictGet`. The issue was caused by premature access to the table expression during ALIAS column resolution in the new analyzer. [#99065](https://github.com/ClickHouse/ClickHouse/pull/99065) ([Peng](https://github.com/fastio)). * Fix out of bounds error when user tries to query only virtual columns from Iceberg table with Avro format for data. It's super rare scenario so not marking it as critical. Fixes [#88238](https://github.com/ClickHouse/ClickHouse/issues/88238). [#99080](https://github.com/ClickHouse/ClickHouse/pull/99080) ([alesapin](https://github.com/alesapin)). * Fix segfault in recursive CTE with `remote()` + `view()`. [#99081](https://github.com/ClickHouse/ClickHouse/pull/99081) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Skip unnecessary extra index analysis when read-in-order optimization is applied. [#99084](https://github.com/ClickHouse/ClickHouse/pull/99084) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed a crash triggered by a memory limit exception thrown during patch part application. [#99086](https://github.com/ClickHouse/ClickHouse/pull/99086) ([Anton Popov](https://github.com/CurtizJ)). * Fix debug assertion in `DDLWorker` caused by stale `first_failed_task_name` after a ZooKeeper entry is deleted during reinitialization recovery. [#99099](https://github.com/ClickHouse/ClickHouse/pull/99099) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed rebuild of text indexes on merges with TTL. [#99107](https://github.com/ClickHouse/ClickHouse/pull/99107) ([Anton Popov](https://github.com/CurtizJ)). * Fix crash in `ALTER TABLE ... REMOVE SETTINGS` query for Iceberg table engine. Fixes [#86330](https://github.com/ClickHouse/ClickHouse/issues/86330). [#99108](https://github.com/ClickHouse/ClickHouse/pull/99108) ([alesapin](https://github.com/alesapin)). * Fixes a bug in `query_plan_convert_any_join_to_semi_or_anti_join` optimization, returning an incorrect result for unmatched rows. Related: https://github.com/ClickHouse/ClickHouse/pull/95995. [#99112](https://github.com/ClickHouse/ClickHouse/pull/99112) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix LOGICAL_ERROR exception in `ASTColumnsExceptTransformer::transform`. [#99119](https://github.com/ClickHouse/ClickHouse/pull/99119) ([Pablo Marcos](https://github.com/pamarcos)). * Fix RBAC bypass that allowed users to obtain table structure via `DESCRIBE TABLE` or `CREATE TABLE AS` on table functions (`mysql()`, `postgresql()`, `sqlite()`, `arrowFlight()`, `jdbc()`, `odbc()`, etc.) without the required source access privileges. For functions that infer schema from remote servers, this also allowed triggering outbound connections (SSRF) without authorization. [#99122](https://github.com/ClickHouse/ClickHouse/pull/99122) ([pufit](https://github.com/pufit)). * Fix Keeper crash (segfault in NuRaft) during dynamic reconfiguration and leadership transfer. [#99133](https://github.com/ClickHouse/ClickHouse/pull/99133) ([JIaQi Tang](https://github.com/JiaQiTang98)). * Fix crash on usage of Buffer table with SAMPLE when destination does not support it. [#99141](https://github.com/ClickHouse/ClickHouse/pull/99141) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix LOGICAL_ERROR due to patch parts column order mismatch. [#99164](https://github.com/ClickHouse/ClickHouse/pull/99164) ([Pablo Marcos](https://github.com/pamarcos)). * Fix very rare crash when Iceberg table contains files of mixed format (ORC and Parquet). Fixes [#88126](https://github.com/ClickHouse/ClickHouse/issues/88126). [#99168](https://github.com/ClickHouse/ClickHouse/pull/99168) ([alesapin](https://github.com/alesapin)). * Fix max_execution_time not being applied for backup/restore. [#99205](https://github.com/ClickHouse/ClickHouse/pull/99205) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix `insert_deduplication_token` being silently ignored for `INSERT SELECT` queries without `ORDER BY ALL`. Previously, deduplication was disabled entirely for unsorted `INSERT SELECT`, even when an explicit user token was provided. Now, providing `insert_deduplication_token` is sufficient to enable deduplication regardless of `ORDER BY ALL`. [#99206](https://github.com/ClickHouse/ClickHouse/pull/99206) ([Desel72](https://github.com/Desel72)). * Fix excessive access checks during `InverseDictionaryLookupPass` optimization by checking `CREATE_TEMPORARY_TABLE` grant once before the pass instead of for every visited node. [#99210](https://github.com/ClickHouse/ClickHouse/pull/99210) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix `clickhouse format --obfuscate` producing invalid SQL by obfuscating skip index types, compression codec names, database engine names, and dictionary layout/source definitions. [#99260](https://github.com/ClickHouse/ClickHouse/pull/99260) ([Raúl Marín](https://github.com/Algunenano)). * Fixed a bug where, in some cases, comparing between Time[64] and DateTime[64] types was confusing; now, in cases like this, Time[64] values are promoted to DateTime[64] by adding the `1970-01-01` as a date part. [#99267](https://github.com/ClickHouse/ClickHouse/pull/99267) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Clamp settings constraints in DDL worker for distributed DDL queries. [#99317](https://github.com/ClickHouse/ClickHouse/pull/99317) ([Pablo Marcos](https://github.com/pamarcos)). * Fix minor issues with TOTP authentication: the `--one-time-password` CLI option with empty password, validation of `` and `` configuration values. [#99322](https://github.com/ClickHouse/ClickHouse/pull/99322) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix logical error `unordered_map::at: key not found` in Avro output format when serializing `Enum8`/`Enum16` columns with values not present in the enum definition. [#99332](https://github.com/ClickHouse/ClickHouse/pull/99332) ([Desel72](https://github.com/Desel72)). * Fix CHECK TABLE with sparse serialization inside Tuple with Dynamic. Closes [#96588](https://github.com/ClickHouse/ClickHouse/issues/96588). [#99351](https://github.com/ClickHouse/ClickHouse/pull/99351) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed too strict validation of text index preprocessor. [#99359](https://github.com/ClickHouse/ClickHouse/pull/99359) ([Anton Popov](https://github.com/CurtizJ)). * Fix compatibility when upgrading replicated tables with implicit minmax indices from 25.10 to newer versions. [#99392](https://github.com/ClickHouse/ClickHouse/pull/99392) ([Raúl Marín](https://github.com/Algunenano)). * Removed support for negated functions (`notEquals`, `notLike`, `notIn`) in text index analysis. These functions could never skip any granules, so analyzing the index for them only added overhead without any benefit. [#99393](https://github.com/ClickHouse/ClickHouse/pull/99393) ([Anton Popov](https://github.com/CurtizJ)). * Fix `optimize_skip_unused_shards` with a new analyzer for the case when a `Distributed` table was used inside an IN subquery. [#99436](https://github.com/ClickHouse/ClickHouse/pull/99436) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix heap-use-after-free in `INTERSECT`/`EXCEPT` when the query produces duplicate column names. [#99471](https://github.com/ClickHouse/ClickHouse/pull/99471) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error in `ALTER TABLE ... DROP PART` when a typed query parameter is used for the part name. [#99489](https://github.com/ClickHouse/ClickHouse/pull/99489) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed `NOT_FOUND_COLUMN_IN_BLOCK` exception when a text index predicate (e.g. `hasAllTokens`) is referenced in both `SELECT` and `WHERE` clauses via an alias. [#99504](https://github.com/ClickHouse/ClickHouse/pull/99504) ([Anton Popov](https://github.com/CurtizJ)). * Fix incorrect results when using `hasAllTokens` with OR across columns that have separate text indexes. [#99505](https://github.com/ClickHouse/ClickHouse/pull/99505) ([Anton Popov](https://github.com/CurtizJ)). * Initialize page cache in `clickhouse-local` so that `page_cache_max_size` setting takes effect. [#99510](https://github.com/ClickHouse/ClickHouse/pull/99510) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed rare incorrect marking of a data part as broken and detaching it after `DETACH/ATTACH TABLE` query. [#99529](https://github.com/ClickHouse/ClickHouse/pull/99529) ([Anton Popov](https://github.com/CurtizJ)). * Fix `std::length_error` exception when querying empty system tables with Pretty format via HTTP interface. [#99541](https://github.com/ClickHouse/ClickHouse/pull/99541) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` when using `ALTER TABLE ADD COLUMN` to create an `EPHEMERAL` column with the same name as a virtual column (e.g. `_part_offset`). [#99549](https://github.com/ClickHouse/ClickHouse/pull/99549) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `VectorSimilarityIndexCache` entries never being evicted after part removal due to mismatched cache keys. [#99575](https://github.com/ClickHouse/ClickHouse/pull/99575) ([Seva Potapov](https://github.com/seva-potapov)). * Forbid reading Google credentials from a local file. This setting is insecure because it allows reading other credentials if the file path is known. [#99584](https://github.com/ClickHouse/ClickHouse/pull/99584) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix performance degradation in the analyzer. Prune unused columns from ARRAY JOIN. [#99587](https://github.com/ClickHouse/ClickHouse/pull/99587) ([Dmitry Novik](https://github.com/novikd)). * Fixed reading of text index in table with existing lightweight deletes and row policies. [#99661](https://github.com/ClickHouse/ClickHouse/pull/99661) ([Anton Popov](https://github.com/CurtizJ)). * Fix nullptr dereference in Parquet reader when filter-in-decoder path encounters filtered-out pages. Closes [#99676](https://github.com/ClickHouse/ClickHouse/issues/99676). [#99677](https://github.com/ClickHouse/ClickHouse/pull/99677) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect seek in AsynchronousReadBufferFromFileDescriptor with O_DIRECT. Closes [#99358](https://github.com/ClickHouse/ClickHouse/issues/99358). [#99678](https://github.com/ClickHouse/ClickHouse/pull/99678) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed heap-buffer-overflow in `CompressionCodecT64` and process abort in `CompressionCodecMultiple` when decompressing malformed compressed data. Both issues were found by new libFuzzer targets. The codecs now throw an exception instead of crashing. [#99680](https://github.com/ClickHouse/ClickHouse/pull/99680) ([Rahul](https://github.com/motsc)). * Delay processing until server has finished loading all the tables. [#99700](https://github.com/ClickHouse/ClickHouse/pull/99700) ([Seva Potapov](https://github.com/seva-potapov)). * Fix MySQL dictionary source bypassing `RemoteHostFilter` for inline DDL params. [#99720](https://github.com/ClickHouse/ClickHouse/pull/99720) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix logical error when iterating over data lake tables in `system.tables`. [#99739](https://github.com/ClickHouse/ClickHouse/pull/99739) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed analysis of predicates with the `IN` function by text index with the preprocessor. Fixed the collision of searched tokens in the text index, which could lead to incorrect results. [#99755](https://github.com/ClickHouse/ClickHouse/pull/99755) ([Anton Popov](https://github.com/CurtizJ)). * Fix infinite loop when reading `Npy` format files with negative shape dimensions. [#99812](https://github.com/ClickHouse/ClickHouse/pull/99812) ([Desel72](https://github.com/Desel72)). * Fix global-buffer-overflow in `CRC32` function on `FixedString` arguments when evaluated with zero rows during query plan header computation. [#99835](https://github.com/ClickHouse/ClickHouse/pull/99835) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix crash (null pointer dereference) when executing `ALTER TABLE ... MODIFY COLUMN ... COMMENT` on Iceberg tables. [#99838](https://github.com/ClickHouse/ClickHouse/pull/99838) ([Desel72](https://github.com/Desel72)). * Fix `aggregate_functions_null_for_empty` setting to work with aggregate functions returning non-Nullable types such as `Array` or `Map` (e.g., `groupArray`, `sumMap`). [#99839](https://github.com/ClickHouse/ClickHouse/pull/99839) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix LOGICAL_ERROR exception in `midpoint` function when called with mixed signed/unsigned integer types. [#99867](https://github.com/ClickHouse/ClickHouse/pull/99867) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Block structure mismatch" exception in queries with HAVING clause where the filter expression contains both an aggregate wrapped in a NULL-producing function and `materialize(0)`. [#99915](https://github.com/ClickHouse/ClickHouse/pull/99915) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix assertion failure in `sipHash128Keyed` (and similar keyed hash functions) when the data argument is a Map with array keys or other nested array types. [#99921](https://github.com/ClickHouse/ClickHouse/pull/99921) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception "Not-ready Set" in `IN` function during query plan optimization with `convertAnyJoinToSemiOrAntiJoin`. [#99939](https://github.com/ClickHouse/ClickHouse/pull/99939) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Build/Testing/Packaging Improvement * Reduce compilation time by removing heavy header includes and moving expensive template instantiations out of headers. [#97893](https://github.com/ClickHouse/ClickHouse/pull/97893) ([Raúl Marín](https://github.com/Algunenano)). * Reduce compile time of arithmetic functions and related headers by shrinking template dispatch matrices and removing heavy includes. [#98204](https://github.com/ClickHouse/ClickHouse/pull/98204) ([Raúl Marín](https://github.com/Algunenano)). * Use `mongo-c-driver` 2.2.2. [#98304](https://github.com/ClickHouse/ClickHouse/pull/98304) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `postgres` REL_18_3. [#98306](https://github.com/ClickHouse/ClickHouse/pull/98306) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Enable jemalloc allocator for UBSan builds to avoid RSS accumulation from glibc malloc's poor memory reclamation behavior. [#98444](https://github.com/ClickHouse/ClickHouse/pull/98444) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use Rust v0 symbol mangling and strip internal symbols from PRQL library to reduce symbol name bloat from parser combinator libraries. [#98446](https://github.com/ClickHouse/ClickHouse/pull/98446) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add TPC-H benchmark suite and TPC-DS README to `tests/benchmarks`. [#98495](https://github.com/ClickHouse/ClickHouse/pull/98495) ([Raufs Dunamalijevs](https://github.com/rienath)). * Add correctness tests for all 99 TPC-DS queries. [#99204](https://github.com/ClickHouse/ClickHouse/pull/99204) ([Raufs Dunamalijevs](https://github.com/rienath)). * Add integration test reproducing DDL CREATE TABLE + ALTER with offline replica bug ([#44070](https://github.com/ClickHouse/ClickHouse/issues/44070)), marked as expected failure. [#99259](https://github.com/ClickHouse/ClickHouse/pull/99259) ([Raufs Dunamalijevs](https://github.com/rienath)). * Integrate jemalloc with `je_` prefix and remove usage of linker's `--wrap`. [#99342](https://github.com/ClickHouse/ClickHouse/pull/99342) ([Azat Khuzhin](https://github.com/azat)). ### ClickHouse release 26.2, 2026-02-26. [Presentation](https://presentations.clickhouse.com/2026-release-26.2/), [Video](https://www.youtube.com/watch?v=7qHba08vNfo) #### Backward Incompatible Change * Deduplication is turned ON for all inserts by default. It was OFF before for async inserts and for MV's, but it was ON for sync inserts. The goal is to have the same defaults for both ways of inserts. If you have deduplication explicitly disabled on your cluster, you have to explicitly set `deduplicate_insert='backward_compatible_choice'` to keep the old behavior. The same with `deduplicate_blocks_in_dependent_materialized_views`. [#95970](https://github.com/ClickHouse/ClickHouse/pull/95970) ([Sema Checherinda](https://github.com/CheSema)). * Improved storage format of statistics. All statistics are now stored in a single file. [#93414](https://github.com/ClickHouse/ClickHouse/pull/93414) ([Anton Popov](https://github.com/CurtizJ)). If you didn't explicitly enable table statistics, you can ignore this item. * Limit S3(Azure)Queue in-memory metadata. System tables are renamed from `azure_queue` to `azure_queue_metadata_cache` and `system.s3queue` to `s3queue_metadata_cache`. [#95809](https://github.com/ClickHouse/ClickHouse/pull/95809) ([Kseniia Sumarokova](https://github.com/kssenii)). * Previously, applying a function to a `Variant` column silently returned NULLs when a variant sub-type was incompatible with the function; now it throws an exception, which may break queries that relied on the silent NULL behavior. [#95811](https://github.com/ClickHouse/ClickHouse/pull/95811) ([Bharat Nallan](https://github.com/bharatnc)). * `DATE` columns from PostgreSQL are now inferred as `Date32` in ClickHouse (in previous versions they were inferred as `Date`, which led to overflow of the values outside of a narrow range). Allow inserting `Date32` values back to PostgreSQL. Closes [#73084](https://github.com/ClickHouse/ClickHouse/issues/73084). [#95999](https://github.com/ClickHouse/ClickHouse/pull/95999) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The semantics of the `do_not_merge_across_partitions_select_final` setting were made more obvious. Previously, the feature could be automatically enabled when the setting was not explicitly set in the configs. It caused confusion repeatedly and, unfortunately, led to some issues in production. Now, the rules are simpler: `do_not_merge_across_partitions_select_final=1` enables the functionality unconditionally. If `do_not_merge_across_partitions_select_final=0`, then automatic is used only if the new setting `enable_automatic_decision_for_merging_across_partitions_for_final=1` and not used otherwise. To preserve the old behaviour as much as possible, the defaults were set to `do_not_merge_across_partitions_select_final=0` and `enable_automatic_decision_for_merging_across_partitions_for_final=1`. [#96110](https://github.com/ClickHouse/ClickHouse/pull/96110) ([Nikita Taranov](https://github.com/nickitat)). * When creating an S3 table with explicitly specified columns, ClickHouse now validates that those column names actually exist in the remote file's schema. Queries that previously worked with mismatched column names will now fail at table creation time. This closes [#96089](https://github.com/ClickHouse/ClickHouse/issues/96089). [#96194](https://github.com/ClickHouse/ClickHouse/pull/96194) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Forbid using subqueries in ORDER BY and other table key expressions. [#96847](https://github.com/ClickHouse/ClickHouse/pull/96847) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enable `apply_row_policy_after_final` by default. Initially, when `optimize_move_to_prewhere_if_final=0`, both ROW POLICY and PREWHERE respect FINAL and were applied after FINAL. This was broken by [#87303](https://github.com/ClickHouse/ClickHouse/issues/87303), which ignored the `optimize_move_to_prewhere_if_final` for the ROW POLICY filter. To fix this, this PR enables the setting `apply_row_policy_after_final` introduced in [#91065](https://github.com/ClickHouse/ClickHouse/issues/91065). With `apply_row_policy_after_final` enabled, ROW POLICY would continue to respect FINAL by default, as previously. This PR is an incompatible change because it changes the behaviour for `optimize_move_to_prewhere_if_final=1`. Now, to get the ROW POLICY applied before FINAL, `apply_row_policy_after_final` should be used instead of `optimize_move_to_prewhere_if_final`. [#97279](https://github.com/ClickHouse/ClickHouse/pull/97279) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * The `Date` type is now serialized as Arrow's native `date32` type in Arrow/ArrowStream formats, instead of `uint16`. Tools like PyArrow will now correctly see the column as a date type. The old behavior can be restored with the `output_format_arrow_date_as_uint16` setting. Reading old Arrow files that used `uint16` for `Date` columns is still supported. [#96860](https://github.com/ClickHouse/ClickHouse/pull/96860) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### New Feature * Users can now use ClickStack (an observability UI) directly from ClickHouse, useful for debugging and local development. [#96597](https://github.com/ClickHouse/ClickHouse/pull/96597) ([Aaron Knudtson](https://github.com/knudtty)). * Support time-based one-time password (TOTP) as an authentication method. [#71273](https://github.com/ClickHouse/ClickHouse/pull/71273) ([Vladimir Cherkasov](https://github.com/vdimir)). * Add `lazy_load_tables` database setting. When enabled, tables are not loaded during database startup — a lightweight `StorageTableProxy` is created instead and the real table engine is materialized on first access. [#96283](https://github.com/ClickHouse/ClickHouse/pull/96283) ([xiaohuanlin](https://github.com/xiaohuanlin)). * Added `input_format_max_block_wait_ms` setting to emit data blocks by timeout and allowed processing of remaining data when an HTTP connection is closed unexpectedly. [#94509](https://github.com/ClickHouse/ClickHouse/pull/94509) ([Mostafa Mohamed Salah](https://github.com/Sasao4o)). * Google BigLake catalog integration. This closes [#95339](https://github.com/ClickHouse/ClickHouse/issues/95339). [#97104](https://github.com/ClickHouse/ClickHouse/pull/97104) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Added system table `system.tokenizers` which shows all available tokenizers. [#96753](https://github.com/ClickHouse/ClickHouse/pull/96753) ([Robert Schulze](https://github.com/rschu1ze)). * Add new system table `system.user_defined_functions` to monitor UDF loading status and configuration. [#90340](https://github.com/ClickHouse/ClickHouse/pull/90340) ([Xu Jia](https://github.com/XuJia0210)). * Add `system.jemalloc_stats` table exposing jemalloc memory allocator statistics (via `malloc_stats_print`) for diagnosing memory usage on servers built with jemalloc. Also add a `/jemalloc.html` HTTP endpoint on the ClickHouse HTTP interface for interactive visualization of these statistics. [#97077](https://github.com/ClickHouse/ClickHouse/pull/97077) ([Antonio Andelic](https://github.com/antonio2368)). * Added `system.jemalloc_profile_text` table for reading and analyzing jemalloc heap profiles. The output format is controlled by the `jemalloc_profile_text_output_format` setting (raw, symbolized, or collapsed; default collapsed). Inline frame resolution is controlled by `jemalloc_profile_text_symbolize_with_inline` (when enabled, inline frames are included at the cost of slower symbolization; when disabled, they are skipped for faster output). For the collapsed format, `jemalloc_profile_text_collapsed_use_count` controls whether stacks are weighted by live allocation count (true) or live bytes (false, default). This enables easier memory profiling and flame graph visualization of jemalloc heap profiles. Fixes [#93248](https://github.com/ClickHouse/ClickHouse/issues/93248). [#97218](https://github.com/ClickHouse/ClickHouse/pull/97218) ([Antonio Andelic](https://github.com/antonio2368)). * Add the `default_dictionary_database` setting, which lets ClickHouse resolve external dictionaries referenced without a database qualifier in a specified default database. This simplifies migration from XML-defined global dictionaries to SQL-defined per-database dictionaries—allowing existing dictionary queries (e.g. dictGet('name', …)) to continue working without modification. [#91412](https://github.com/ClickHouse/ClickHouse/pull/91412) ([Dmitrii Plotnikov](https://github.com/dimbo4ka)). * Support auxiliary zookeeper for `DatabaseReplicated`. [#91683](https://github.com/ClickHouse/ClickHouse/pull/91683) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Implement new table function `primes` and new system table `system.primes` that contains prime numbers in ascending order. Closes [#90839](https://github.com/ClickHouse/ClickHouse/issues/90839). [#92776](https://github.com/ClickHouse/ClickHouse/pull/92776) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Async inserts support parallel quorum. The inserted data is replicated to the quorum. If duplicates are found, query waits until previously inserted data is replicated as well. [#93356](https://github.com/ClickHouse/ClickHouse/pull/93356) ([Sema Checherinda](https://github.com/CheSema)). * Added functions `colorOKLABToSRGB`, `colorSRGBToOKLAB` to convert value from sRGB to OKLAB and vice versa. [#93361](https://github.com/ClickHouse/ClickHouse/pull/93361) ([Pranav Tiwari](https://github.com/pranavt84)). * A new `deduplicate_insert` setting which overrides `insert_deduplicate` and `async_insert_deduplicate`. [#94413](https://github.com/ClickHouse/ClickHouse/pull/94413) ([Sema Checherinda](https://github.com/CheSema)). * Server setting `insert_deduplication_version` makes it possible to migrate on unified deduplication hash. [#95409](https://github.com/ClickHouse/ClickHouse/pull/95409) ([Sema Checherinda](https://github.com/CheSema)). * Add `xxh3_128` hashing function. [#96055](https://github.com/ClickHouse/ClickHouse/pull/96055) ([Raúl Marín](https://github.com/Algunenano)). * Added `OPTIMIZE DRY RUN PARTS ` query to simulate merges without committing the result part. It may be useful for testing purposes: verifying merge correctness in the new version, deterministically reproducing merge-related bugs, and reliably benchmarking merge performance. [#96122](https://github.com/ClickHouse/ClickHouse/pull/96122) ([Anton Popov](https://github.com/CurtizJ)). * Add a new check enabled by default via setting `check_named_collection_dependencies` to avoid dropping named collections used by tables. [#96181](https://github.com/ClickHouse/ClickHouse/pull/96181) ([Pablo Marcos](https://github.com/pamarcos)). * Added `system.fail_points` to inspect existing failpoints in the server and whether they are enabled or not. This is going to help automate testing. [#96762](https://github.com/ClickHouse/ClickHouse/pull/96762) ([Pedro Ferreira](https://github.com/PedroTadim)). * Add role-based access to Glue catalog. Use settings `aws_role_arn` and, optionally, `aws_role_session_name`. [#90825](https://github.com/ClickHouse/ClickHouse/pull/90825) ([Antonio Andelic](https://github.com/antonio2368)). * Added a setting `add_minmax_index_for_temporal_columns` that, when enabled, automatically creates minmax indexes for all `Date`, `Date32`, `Time`, `Time64`, `DateTime`, and `DateTime64` columns. [#93355](https://github.com/ClickHouse/ClickHouse/pull/93355) ([Michael Jarrett](https://github.com/EmeraldShift)). * Support for extended table aliases for JOINs (queries like `SELECT * FROM (SELECT 1) AS t(a) JOIN (SELECT 1) AS u(b) ON a = b`). Closes [#95131](https://github.com/ClickHouse/ClickHouse/issues/95131). [#95331](https://github.com/ClickHouse/ClickHouse/pull/95331) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Added support for `ALTER TABLE RENAME COLUMN` for Iceberg tables. Previously only `ADD COLUMN, DROP COLUMN, and MODIFY COLUMN` were supported. [#97455](https://github.com/ClickHouse/ClickHouse/pull/97455) ([murphy-4o](https://github.com/murphy-4o)). #### Experimental Feature * The text index is now GA. [#96794](https://github.com/ClickHouse/ClickHouse/pull/96794) ([Robert Schulze](https://github.com/rschu1ze)). * The `QBit` data type for quantized bit-packed vector storage (used for approximate nearest-neighbor search) is now generally available and no longer requires enabling an experimental setting. [#95358](https://github.com/ClickHouse/ClickHouse/pull/95358) ([Raufs Dunamalijevs](https://github.com/rienath)). * Vector search in ClickHouse can now use replicas in the cluster to _distribute_ the load and search of vector index parts. This enables ClickHouse to support large vector indexes that exceed the memory capacity of a single VM. [#95876](https://github.com/ClickHouse/ClickHouse/pull/95876) ([Shankar Iyer](https://github.com/shankar-iyer)). * Add server-side AST fuzzer controlled by `ast_fuzzer_runs` and `ast_fuzzer_any_query` settings. When enabled, the server runs randomized mutations of each query after its normal execution, discarding the results. [#97568](https://github.com/ClickHouse/ClickHouse/pull/97568) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `iif` function to the experimental KQL dialect. [#94790](https://github.com/ClickHouse/ClickHouse/pull/94790) ([happyso](https://github.com/sunyeongchoi)). * Schema inference now respects `allow_experimental_nullable_tuple_type`. When enabled, it allows inferred tuple types to be `Nullable(Tuple(...))`, so missing nested objects can become `NULL` instead of a tuple of `NULL` elements. [#95525](https://github.com/ClickHouse/ClickHouse/pull/95525) ([Nihal Z. Miaji](https://github.com/nihalzp)). * The `use_statistics_cache` setting is now enabled by default, so column statistics are cached in memory to speed up query optimization without needing to reload them from each part. [#95950](https://github.com/ClickHouse/ClickHouse/pull/95950) ([Han Fei](https://github.com/hanfei1991)). #### Performance Improvement * Allow any deterministic expression in Primary Key to be used for data skipping (e.g. `ORDER BY cityHash64(user_id)`/ `ORDER BY length(user_id)`). For deterministic expressions, ClickHouse can apply the expression to query constants and use the result in the primary key index for predicates like `=`, `IN`, and `has`. If the expression is also injective (e.g. `ORDER BY hex(p)` or `ORDER BY reverse(tuple(reverse(p), hex(p)))`), we can effectively use the index for the negated forms: `!=`, `NOT IN`, and `NOT has`. Closes [#10685](https://github.com/ClickHouse/ClickHouse/issues/10685). Closes [#82161](https://github.com/ClickHouse/ClickHouse/issues/82161). [#92952](https://github.com/ClickHouse/ClickHouse/pull/92952) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Improved storage format of statistics. All statistics are now stored in a single file. [#93414](https://github.com/ClickHouse/ClickHouse/pull/93414) ([Anton Popov](https://github.com/CurtizJ)). * Allow parallelized read for remote table engines/functions in the filesystem cache. [#71781](https://github.com/ClickHouse/ClickHouse/pull/71781) ([Kseniia Sumarokova](https://github.com/kssenii)). * Allow using userspace page cache with local files and object storage table functions. [#77874](https://github.com/ClickHouse/ClickHouse/pull/77874) ([Michael Kolupaev](https://github.com/al13n321)). * Avoid unnecessary memcpy in userspace page cache. [#77884](https://github.com/ClickHouse/ClickHouse/pull/77884) ([Michael Kolupaev](https://github.com/al13n321)). * The default for `concurrent_threads_scheduler` is now `max_min_fair` instead of `fair_round_robin`. This improves fairness under high load by prioritizing queries with fewer allocated slots, so short-running queries aren't penalized by long-running ones. [#95300](https://github.com/ClickHouse/ClickHouse/pull/95300) ([Sergei Trifonov](https://github.com/serxa)). * If a `FINAL` query used primary key condition for filtering followed by skip indexes for other conditions, the `PrimaryKeyExpand` processing step will now only check the initial shortlisted primary key ranges for intersection. [#94903](https://github.com/ClickHouse/ClickHouse/pull/94903) ([Shankar Iyer](https://github.com/shankar-iyer)). * When using parallel replicas with table functions like `s3(...)`, queries with a single subquery wrapping the table function are now automatically parallelized across replicas, whereas previously only direct table function references were parallelized. Closes [#92264](https://github.com/ClickHouse/ClickHouse/issues/92264). [#96332](https://github.com/ClickHouse/ClickHouse/pull/96332) ([phulv94](https://github.com/phulv94)). * Enable splitting data and system files in cache into separate segments. [#87834](https://github.com/ClickHouse/ClickHouse/pull/87834) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * Speed up some hash join operations by implementing dynamic dispatch for `ColumnVector::replicate`. [#79573](https://github.com/ClickHouse/ClickHouse/pull/79573) ([Raúl Marín](https://github.com/Algunenano)). * Performance improvement for parallel hash join in cases of complex predicates. Previously, we were processing non-joined rows in one thread, which is suboptimal the idea of the optimization is to parallelize the processing of the non-joined rows across multiple threads. Can be toggled by the `parallel_non_joined_rows_processing` setting. Enabled by default. [#92068](https://github.com/ClickHouse/ClickHouse/pull/92068) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Slightly optimize parsing of JSON type. [#93614](https://github.com/ClickHouse/ClickHouse/pull/93614) ([Pavel Kruglov](https://github.com/Avogar)). * Improve memory footprint of AST. Optimization makes sense as fields are not used when highlighting is not used and there is no VALUES parsing. [#93974](https://github.com/ClickHouse/ClickHouse/pull/93974) ([Ilya Yatsishin](https://github.com/qoega)). * Optimize memory consumption of named Tuple AST objects. Place column names as strings in tuple object instead of having them in generic AST literal nodes. [#94704](https://github.com/ClickHouse/ClickHouse/pull/94704) ([Ilya Yatsishin](https://github.com/qoega)). * Devirtualization is improved with additional linker options. [#94737](https://github.com/ClickHouse/ClickHouse/pull/94737) ([Nikita Taranov](https://github.com/nickitat)). * Improve clone replica performance for ReplicatedMergeTree tables with many parts by batching ZooKeeper requests. [#94847](https://github.com/ClickHouse/ClickHouse/pull/94847) ([c-end](https://github.com/c-end)). * When read step already has PREWHERE filters a new filter couldn't be added. This change postpones PREWHERE optimization until after JOIN runtime filter optimization so that runtime filters can be also pushed to PREWHERE. [#95838](https://github.com/ClickHouse/ClickHouse/pull/95838) ([Alexander Gololobov](https://github.com/davenger)). * Speedup `T64` codec compression by using dynamic dispatch on x86. [#95881](https://github.com/ClickHouse/ClickHouse/pull/95881) ([Raúl Marín](https://github.com/Algunenano)). * Speed up `uniq` over numeric types by batching inserts when possible (not null, not -If, no GROUP BY, no IPv6 or String). [#95904](https://github.com/ClickHouse/ClickHouse/pull/95904) ([Raúl Marín](https://github.com/Algunenano)). * Low-level optimizations for Keeper: `ZooKeeper::observeOperations` has been found to account for >20% of the ZooKeeper receive thread CPU consumption. This change addresses that by: 1. For `AggregatedZooKeeperLog::stats`, use `CityHash64` instead of `SipHash` which is >10x faster. 2. For `Coordination::ErrorCounter`, use `std::array, N>` instead of `std::unordered_map` and `std::mutex`. [#95962](https://github.com/ClickHouse/ClickHouse/pull/95962) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Remove 64-byte alignment for ProfileEvents::Counter to save memory. [#96097](https://github.com/ClickHouse/ClickHouse/pull/96097) ([Azat Khuzhin](https://github.com/azat)). * Memory optimization: trim size of `CachedOnDiskReadBufferFromFile` structure 50x. [#96098](https://github.com/ClickHouse/ClickHouse/pull/96098) ([Azat Khuzhin](https://github.com/azat)). * Don't copy old data on hash table resizing if it's empty. [#96180](https://github.com/ClickHouse/ClickHouse/pull/96180) ([Raúl Marín](https://github.com/Algunenano)). * Support JOIN runtime filters for `RIGHT OUTER` JOINs. [#96183](https://github.com/ClickHouse/ClickHouse/pull/96183) ([Hechem Selmi](https://github.com/m-selmi)). * The optimization `enable_join_runtime_filters` is now the default. [#89314](https://github.com/ClickHouse/ClickHouse/pull/89314) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Previously, text index direct read optimization was applied only when all parts had a materialized text index. This PR adds partial support: if some parts have a materialized text index, those parts will use it, while parts without a materialized text index will fall back to executing the original filter expression. [#96411](https://github.com/ClickHouse/ClickHouse/pull/96411) ([Anton Popov](https://github.com/CurtizJ)). * Added `minmax` secondary indexes on time columns and `bloom_filter` indexes on `query_id`/`initial_query_id` columns to system log tables for faster filtering. [#96712](https://github.com/ClickHouse/ClickHouse/pull/96712) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Lazy materialization optimization is now applied to all branches of a `UNION ALL` query, not just the first one. Queries that combine multiple sorted and limited reads from different `MergeTree` tables via `UNION ALL` will now benefit from deferred column reading on every branch, reducing I/O. [#96832](https://github.com/ClickHouse/ClickHouse/pull/96832) ([Federico Ginosa](https://github.com/menxit)). * Optimize minmax skip index computation during INSERT by removing an unnecessary data copy and enabling vectorized min/max calculation for numeric columns. [#97392](https://github.com/ClickHouse/ClickHouse/pull/97392) ([Raúl Marín](https://github.com/Algunenano)). * Storage `DeltaLake` now takes `count()` result from delta lake metadata and shows correct table stats in system.tables (total bytes/rows). [#96190](https://github.com/ClickHouse/ClickHouse/pull/96190) ([Kseniia Sumarokova](https://github.com/kssenii)). * The unused columns are removed also from the reading step in case of reading from a MergeTree. It is especially useful when a filter is pushed down into `PREWHERE`. [#89982](https://github.com/ClickHouse/ClickHouse/pull/89982) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Improved processing `SHOW TABLES` query by fetching only names of tables and improved getLightweightTablesIterator to return structure containing only table names. resolves [#93835](https://github.com/ClickHouse/ClickHouse/issues/93835). [#94467](https://github.com/ClickHouse/ClickHouse/pull/94467) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Improve `assumeNotNull`, `coalesce`, `ifNull` to enable primary key and skip index pruning for range predicates when key columns are wrapped in these functions. Closes [#94689](https://github.com/ClickHouse/ClickHouse/issues/94689). [#94754](https://github.com/ClickHouse/ClickHouse/pull/94754) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Add with_data & with_stat extension to getChildren Keeper request. This allows fetching not only the list of children, but also their `stat` and/or `data` in a single operation. [#94826](https://github.com/ClickHouse/ClickHouse/pull/94826) ([Nikolay Degterinsky](https://github.com/evillique)). * The index analysis is done only once (in most cases) regardless of whether we end up executing a local plan or a plan with parallel replicas. [#94854](https://github.com/ClickHouse/ClickHouse/pull/94854) ([Nikita Taranov](https://github.com/nickitat)). * Allow to enable distributed index analysis based on amount of parts (`distributed_index_analysis_min_parts_to_activate`) and indexes size (`distributed_index_analysis_min_indexes_size_to_activate`). [#95216](https://github.com/ClickHouse/ClickHouse/pull/95216) ([Azat Khuzhin](https://github.com/azat)). * Enable PREWHERE optimization for Iceberg tables. [#95476](https://github.com/ClickHouse/ClickHouse/pull/95476) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Reduce the memory footprint of some AST classes. [#95514](https://github.com/ClickHouse/ClickHouse/pull/95514) ([Raúl Marín](https://github.com/Algunenano)). * Limit the number of pipeline streams generated with `split_intersecting_parts_ranges_into_layers` enabled. Helps to avoid excessive memory consumption. [#96478](https://github.com/ClickHouse/ClickHouse/pull/96478) ([Nikita Taranov](https://github.com/nickitat)). * Implement equivalent sets optimization for multiple joins. Queries with multiple consecutive `INNER JOIN` operations now benefit from improved filter pushdown optimization. When tables are joined on equivalent columns (e.g., `t1 JOIN t2 ON t1.id = t2.id JOIN t3 ON t2.id = t3.id WHERE t1.id > 10`), filters applied to any table in the chain are automatically pushed down to all tables. Close [#96550](https://github.com/ClickHouse/ClickHouse/issues/96550). [#96596](https://github.com/ClickHouse/ClickHouse/pull/96596) ([Vladimir Cherkasov](https://github.com/vdimir)). * Optimize delta lake metadata scan. Uses changes from delta-kernel PR https://github.com/delta-io/delta-kernel-rs/pull/1827. [#96686](https://github.com/ClickHouse/ClickHouse/pull/96686) ([Kseniia Sumarokova](https://github.com/kssenii)). * In Replicated database, don't update the cached cluster for every dummy query. [#96897](https://github.com/ClickHouse/ClickHouse/pull/96897) ([Tuan Pham Anh](https://github.com/tuanpach)). * Use the primary key index when filtering with `startsWithUTF8` if the prefix contains only ASCII characters. [#97055](https://github.com/ClickHouse/ClickHouse/pull/97055) ([vkcku](https://github.com/vkcku)). #### Improvement * Add OpenTelemetry tracing for Keeper requests. [#91332](https://github.com/ClickHouse/ClickHouse/pull/91332) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * New configuration options: `logger.startup_console_level` & `logger.shutdown_console_level` to allow for overriding the console log level during the startup & shutdown of ClickHouse respectively. [#95919](https://github.com/ClickHouse/ClickHouse/pull/95919) ([Garrett Thomas](https://github.com/garrettthomaskth)). * Respect command-line overrides when reloading configuration. Closes [#80294](https://github.com/ClickHouse/ClickHouse/issues/80294). [#80295](https://github.com/ClickHouse/ClickHouse/pull/80295) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Allow key-value overrides for named collection parameters in `mongodb` table function. [#89616](https://github.com/ClickHouse/ClickHouse/pull/89616) ([vanchaklar](https://github.com/vanchaklar)). * The read-in-order optimization for Iceberg tables now works with complex sorting functions like `icebergBucket` and `icebergTruncate`, not just simple column references. [#90256](https://github.com/ClickHouse/ClickHouse/pull/90256) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Add a new column named parts_postpone_reasons in system.mutations to improve diagnostics, which shows the parts postpone reasons. [#92206](https://github.com/ClickHouse/ClickHouse/pull/92206) ([Shaohua Wang](https://github.com/tiandiwonder)). * Track changes in the number of rows to read (due to inserts/deletes or query condition cache usage) in `DataflowStatisticsCache`. [#93636](https://github.com/ClickHouse/ClickHouse/pull/93636) ([Nikita Taranov](https://github.com/nickitat)). * Support SYSTEM RESET DDL WORKER [ON CLUSTER] query. It requests to reset the state of DDLWorker in its main thread. It is helpful to refresh the replica active when host IDs are updated. [#93780](https://github.com/ClickHouse/ClickHouse/pull/93780) ([Tuan Pham Anh](https://github.com/tuanpach)). * Support `mutation_ids` in `system.part_log` for event type of `MUTATE_PART` and `MUTATE_PART_START`. [#93811](https://github.com/ClickHouse/ClickHouse/pull/93811) ([Shaohua Wang](https://github.com/tiandiwonder)). * Background operations (Mutate, Merge) can now be configured independently via 'background' profile. Previously such operations shared settings with regular queries via 'default' profile. [#93905](https://github.com/ClickHouse/ClickHouse/pull/93905) ([Arsen Muk](https://github.com/arsenmuk)). * Add more information to `system.crash_log`. [#94112](https://github.com/ClickHouse/ClickHouse/pull/94112) [#95857](https://github.com/ClickHouse/ClickHouse/pull/95857) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Added new `QueryNonInternal` metric to track the number of executing non-internal queries. This metric is exposed as `ClickHouseMetrics_QueryNonInternal` and helps operators monitor query concurrency against the `max_concurrent_queries` limit, which only applies to non-internal queries. [#94284](https://github.com/ClickHouse/ClickHouse/pull/94284) ([Ashwath Singh](https://github.com/ashwath)). * Support input bytes statistics collection for columns from compact parts in `RuntimeDataflowStatisticsCacheUpdater`. [#94626](https://github.com/ClickHouse/ClickHouse/pull/94626) ([Nikita Taranov](https://github.com/nickitat)). * Add a check for Keeper misconfiguration leading to cluster assembly failures. Closes [#60932](https://github.com/ClickHouse/ClickHouse/issues/60932). [#94682](https://github.com/ClickHouse/ClickHouse/pull/94682) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Improve JSON prefixes deserialization during part loading. [#94848](https://github.com/ClickHouse/ClickHouse/pull/94848) ([Pavel Kruglov](https://github.com/Avogar)). * Refactor the write using full INSERT pipeline, which triggers materialized views on the target table. [#94890](https://github.com/ClickHouse/ClickHouse/pull/94890) ([Kai Zhu](https://github.com/nauu)). * Use vector similarity search plan optimizations only if the index exists for the search column. [#94998](https://github.com/ClickHouse/ClickHouse/pull/94998) ([Eduard Karacharov](https://github.com/korowa)). * Check for the total memory limit before the user authentication and throw `(total) memory limit exceeded` if the total limit is more than allowed. [#95003](https://github.com/ClickHouse/ClickHouse/pull/95003) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Added the `throw_on_unmatched_row_policies` configuration option which, when enabled, throws an exception if a user queries a table that has row policies but none of them apply to that user — preventing the ambiguous behavior of returning all rows due to access control misconfiguration. [#95014](https://github.com/ClickHouse/ClickHouse/pull/95014) ([Vitaly Baranov](https://github.com/vitlibar)). * Dynamic update s3 access tokens in long queries with unity catalog. This closes [#93981](https://github.com/ClickHouse/ClickHouse/issues/93981). [#95069](https://github.com/ClickHouse/ClickHouse/pull/95069) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Disable jemalloc's dirty page decay if ClickHouse is under sustained memory pressure for `memory_worker_decay_adjustment_period_ms` milliseconds. Enable jemalloc's dirty page decay back if ClickHouse is working under normal conditions for same amount of time. [#95145](https://github.com/ClickHouse/ClickHouse/pull/95145) ([Antonio Andelic](https://github.com/antonio2368)). * S3Queue auxiliary Zookeeper support using `keeper_path` setting from s3Queue. [#95203](https://github.com/ClickHouse/ClickHouse/pull/95203) ([Diego Nieto](https://github.com/lesandie)). * Respect `max_parts_to_merge_at_once` in TTL drop part merges. [#95315](https://github.com/ClickHouse/ClickHouse/pull/95315) ([Kseniia Sumarokova](https://github.com/kssenii)). * Add `connection_address` and `connection_port` to query_log to reflect physical connection (`address` and `port` are replaced when connected through proxy and auth_use_forwarded_address=1). [#95471](https://github.com/ClickHouse/ClickHouse/pull/95471) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix incorrect memory accounting for the query conditions cache. The key problem was that it didn't take into account the cache key that composed of several strings (like part_name, the table id and the whole SQL condition). [#95478](https://github.com/ClickHouse/ClickHouse/pull/95478) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Server started with the embedded configuration will allow to manipulate users and grants, saving them to the `access` directory, as the regular configuration does. This improves testing. Also enabled all access_control_improvements in the embedded config and in clickhouse-local. [#95481](https://github.com/ClickHouse/ClickHouse/pull/95481) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improved S3 authentication error messages to include a hint to check credentials when access is denied. [#95648](https://github.com/ClickHouse/ClickHouse/pull/95648) ([Gerald Latkovic](https://github.com/batkovic75)). * Enable statistics cache and set the update period of cache to 300s. [#95841](https://github.com/ClickHouse/ClickHouse/pull/95841) ([Han Fei](https://github.com/hanfei1991)). * Add component name to `system.aggregated_zookeeper_log`. [#95882](https://github.com/ClickHouse/ClickHouse/pull/95882) ([Antonio Andelic](https://github.com/antonio2368)). * Skip object storage reads when querying `DeltaLake` tables from `system.tables`. [#95899](https://github.com/ClickHouse/ClickHouse/pull/95899) ([Antonio Andelic](https://github.com/antonio2368)). * Enable `enable_max_bytes_limit_for_min_age_to_force_merge` by default if the `compatibility` setting is `26.2` or higher. [#95917](https://github.com/ClickHouse/ClickHouse/pull/95917) ([Christoph Wurm](https://github.com/cwurm)). * Delta Lake is now available on macOS. Closes #95979. [#95985](https://github.com/ClickHouse/ClickHouse/pull/95985) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In previous versions, when combining conflicting ALTER expressions with UPDATE and RENAME COLUMN, a logical error was thrown instead of a proper exception. Closes [#70678](https://github.com/ClickHouse/ClickHouse/issues/70678). [#96022](https://github.com/ClickHouse/ClickHouse/pull/96022) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve the help output for all ClickHouse applications and add a `--no-sudo` option with a few fixes. This is a continuation of [#58244](https://github.com/ClickHouse/ClickHouse/issues/58244) from [Ilya Yatsishin](https://github.com/qoega). [#96025](https://github.com/ClickHouse/ClickHouse/pull/96025) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add `distanceCosine` alias for `cosineDistance` because all other distance functions already have alias of this form. [#96065](https://github.com/ClickHouse/ClickHouse/pull/96065) ([Raufs Dunamalijevs](https://github.com/rienath)). * Add support for the `with_data` Keeper extension to improve table fetching in Database Replicated. [#96090](https://github.com/ClickHouse/ClickHouse/pull/96090) ([Nikolay Degterinsky](https://github.com/evillique)). * Update chdig to [v26.2.1](https://github.com/azat/chdig/releases/tag/v26.2.1) (new features and MacOS support). [#96113](https://github.com/ClickHouse/ClickHouse/pull/96113) ([Azat Khuzhin](https://github.com/azat)). * Improve filter pushdown for `numbers` and `primes`. ClickHouse can now derive conservative value bounds from `WHERE` conditions when exact bounds cannot be derived, and restrict sequence generation accordingly (for example, for `WHERE number % 5 < 2 AND number > 100 AND number < 300`, ClickHouse will only generate numbers between 100 and 300, then apply the predicate), avoiding unbounded scans. Closes [#84853](https://github.com/ClickHouse/ClickHouse/issues/84853). Closes [#93913](https://github.com/ClickHouse/ClickHouse/issues/93913). [#96115](https://github.com/ClickHouse/ClickHouse/pull/96115) ([Nihal Z. Miaji](https://github.com/nihalzp)). * The formatter previously wrapped SELECT in parentheses when a `COMMENT` clause was present to disambiguate parsing. Instead, output `COMMENT` before `AS SELECT`, which eliminates the ambiguity without parentheses. [#96293](https://github.com/ClickHouse/ClickHouse/pull/96293) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The `allow_impersonate_user` config setting is now located inside the `access_control_improvements` section rather than being a standalone server setting. [#96451](https://github.com/ClickHouse/ClickHouse/pull/96451) ([Vitaly Baranov](https://github.com/vitlibar)). * Make `core_dump.size_limit` configuration setting hot-reloadable, to avoid having to restart servers for configuration changes to take place. [#96524](https://github.com/ClickHouse/ClickHouse/pull/96524) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Improves CPU and real-time profiler interoperability with socket timeouts. [#96601](https://github.com/ClickHouse/ClickHouse/pull/96601) ([Sergei Trifonov](https://github.com/serxa)). * Prevent the resurrection of dropped data if ADD COLUMN is run quickly after the DROP COLUMN mutation. [#96713](https://github.com/ClickHouse/ClickHouse/pull/96713) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Change `function_id` type in `system.instrumentation` from LowCardinality(Int32) to Int32. [#96726](https://github.com/ClickHouse/ClickHouse/pull/96726) (Copilot). * Synchronous waiting for mutations will respect query cancellation and time limits. [#96756](https://github.com/ClickHouse/ClickHouse/pull/96756) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added system command `SYSTEM RELOAD DELTA KERNEL TRACING ` to be able to change delta-kernel logging, which can be useful for debugging. [#96763](https://github.com/ClickHouse/ClickHouse/pull/96763) ([Kseniia Sumarokova](https://github.com/kssenii)). * Filtering by IP address family, i.e. `dns_allow_resolve_names_to_ipv4/ipv6` settings, are applied even if DNS cache is disabled. [#96810](https://github.com/ClickHouse/ClickHouse/pull/96810) ([c-end](https://github.com/c-end)). * Better jemalloc introspection. [#96840](https://github.com/ClickHouse/ClickHouse/pull/96840) ([Azat Khuzhin](https://github.com/azat)). * Fix `/play` Web UI throwing `QUERY_CACHE_USED_WITH_SYSTEM_TABLE` when querying system tables. [#96869](https://github.com/ClickHouse/ClickHouse/pull/96869) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve Web UI: change favicon to indicate running query state; display errors from auxiliary queries (loading databases and tables) instead of silently ignoring them. Closes [#85055](https://github.com/ClickHouse/ClickHouse/issues/85055). [#96883](https://github.com/ClickHouse/ClickHouse/pull/96883) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Make the left panel in `/play` UI clickable to toggle the database list. [#96884](https://github.com/ClickHouse/ClickHouse/pull/96884) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `DROP DATABASE` now drops tables in reverse dependency order, improving crash-safety when the database contains tables with loading dependencies (e.g. `Distributed` tables using `joinGet`). [#97057](https://github.com/ClickHouse/ClickHouse/pull/97057) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Bump yaml-cpp to prevent skipping invalid YAML. [#97333](https://github.com/ClickHouse/ClickHouse/pull/97333) ([Azat Khuzhin](https://github.com/azat)). * Show a loading indicator in the `play.html` sidebar while tables are being fetched. [#97531](https://github.com/ClickHouse/ClickHouse/pull/97531) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add a copy-to-clipboard button for raw query results in the built-in web UI (play.html). [#97532](https://github.com/ClickHouse/ClickHouse/pull/97532) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix query obfuscator (`clickhouse-format --obfuscate`) to produce parseable SQL in more cases. [#97584](https://github.com/ClickHouse/ClickHouse/pull/97584) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Bug Fix (user-visible misbehavior in an official stable release) * After metadata-only ALTERs, such as extending the elements of Enums, the optimization of aggregation with projection may end up producing an exception. [#84143](https://github.com/ClickHouse/ClickHouse/pull/84143) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Materialized views now use the database where they were created as execution context, meaning that: - it is possible to omit explicit database qualification on names referred in view's select query - if no explicit database qualification is given, the same database where the materialized view was created is assumed. [#88193](https://github.com/ClickHouse/ClickHouse/pull/88193) ([Dmitry Kovalev](https://github.com/dk-github)). * Fix query parameter substitution in CREATE USER authentication methods when using ON CLUSTER. Query parameters in authentication methods (e.g., password) were not being replaced, causing UNKNOWN_QUERY_PARAMETER errors on remote nodes. [#92777](https://github.com/ClickHouse/ClickHouse/pull/92777) ([xiaohuanlin](https://github.com/xiaohuanlin)). * Fixed inconsistencies in text index analysis for `has`, `mapContainsKey`, and `mapContainsValue` functions. Previously, queries using these functions could return different results depending on whether the expression was evaluated with or without a text index. [#93578](https://github.com/ClickHouse/ClickHouse/pull/93578) ([Anton Popov](https://github.com/CurtizJ)). * Fix crash when attaching a table to a `MaterializedPostgreSQL` database if `dropReplicationSlot` throws during stack unwinding. [#96871](https://github.com/ClickHouse/ClickHouse/pull/96871) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Backups could crash the server if you do many concurrent backups clashing over the same files. [#93659](https://github.com/ClickHouse/ClickHouse/pull/93659) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixes queries with parallel replicas and JOIN with non-MT table. Closes [#92056](https://github.com/ClickHouse/ClickHouse/issues/92056). [#93902](https://github.com/ClickHouse/ClickHouse/pull/93902) ([Igor Nikonov](https://github.com/devcrafter)). * Fixes an issue when Iceberg columns with dot in names returned NULL as values. [#94335](https://github.com/ClickHouse/ClickHouse/pull/94335) ([Mikhail Koviazin](https://github.com/mkmkme)). * Fixed handling of UTF8 strings in `stringJaccardIndexUTF8` and improve performance. [#94613](https://github.com/ClickHouse/ClickHouse/pull/94613) ([Joanna Hulboj](https://github.com/jh0x)). * Fix possible overflows in `WITH FILL STALENESS` (that leads to UB or/and endless loops). Fix possible endless loop due to big jumps. Add old analyzer support (mostly for stress tests). [#94663](https://github.com/ClickHouse/ClickHouse/pull/94663) ([Azat Khuzhin](https://github.com/azat)). * Fix possible hung distributed queries when hostnames resolve to multiple addresses and a remote replica freezes. [#94726](https://github.com/ClickHouse/ClickHouse/pull/94726) ([c-end](https://github.com/c-end)). * Fix invalid result on joining multiple table expressions, when leftmost table expression is a `-Cluster` table function. Resolves [#89996](https://github.com/ClickHouse/ClickHouse/issues/89996). [#94748](https://github.com/ClickHouse/ClickHouse/pull/94748) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Fix incorrect primary key and skip index pruning for predicates involving `toWeek`, `toYearWeek`, `toStartOfWeek`, `toLastDayOfWeek`, and `toDayOfWeek`, and fix exceptions in some of these functions for valid queries with `LowCardinality(String)`. [#94816](https://github.com/ClickHouse/ClickHouse/pull/94816) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Remove unnecessary skip permissions check in `ATTACH` queries for a view with SQL Security. This prevents potential privilege escalation when a user attaches a view with a definer without validating required access. [#94865](https://github.com/ClickHouse/ClickHouse/pull/94865) ([pufit](https://github.com/pufit)). * Fixes a crash during `ReplicatedMergeTree` startup caused by concurrent removal of `delete_tmp_*` directories. [#94892](https://github.com/ClickHouse/ClickHouse/pull/94892) ([myeongjun](https://github.com/myeongjjun)). * Fix `INSERT` into Iceberg tables with materialized views losing deduplication information, which caused an exception. [#94938](https://github.com/ClickHouse/ClickHouse/pull/94938) ([Daniil Ivanik](https://github.com/divanik)). * Fix a bug where `SYSTEM DROP QUERY CACHE TAG 'TAGNAME' ON CLUSTER ` would drop the full cache on the cluster. [#94978](https://github.com/ClickHouse/ClickHouse/pull/94978) ([Rory Crispin](https://github.com/RoryCrispin)). * Preserve constant index granularity (use_const_adaptive_granularity) after Vertical merges (v2 with a fix for Nested, and in general). [#95013](https://github.com/ClickHouse/ClickHouse/pull/95013) ([Azat Khuzhin](https://github.com/azat)). * Fixes race in filesystem cache in version 26.1 after [ClickHouse/ClickHouse[#82764](https://github.com/ClickHouse/ClickHouse/issues/82764)](https://github.com/ClickHouse/ClickHouse/pull/82764). [#95042](https://github.com/ClickHouse/ClickHouse/pull/95042) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix postgresql() table function canceling by KILL QUERY and cancel query (Ctrl+C) in clickhouse-client. [#95136](https://github.com/ClickHouse/ClickHouse/pull/95136) ([Roman Vasin](https://github.com/rvasin)). * Fixed type inference for qualified columns from source tables when multiple joins are used with `USING` clause. Previously, subsequent joins incorrectly updated types of underlying source columns to a common supertype even when the column was not involved in that join (e.g., in `SELECT t2.a FROM t1 LEFT JOIN t2 USING (a) LEFT JOIN t3 USING (a)`, the `t2.a` column is only used by the first join, so its type should be the supertype of `t1.a` and `t2.a`, excluding `t3.a`). This could lead to logical errors or crashes when functions expected different column types than what actually appeared in the execution plan. [#95157](https://github.com/ClickHouse/ClickHouse/pull/95157) ([Vladimir Cherkasov](https://github.com/vdimir)). * Make column transform only once during getting content of manifest .avro list and files. [#95164](https://github.com/ClickHouse/ClickHouse/pull/95164) ([Daniil Ivanik](https://github.com/divanik)). * Fix incorrect calculation of JSON column sizes that could lead to excessive memory usage or wrong column statistics. [#95207](https://github.com/ClickHouse/ClickHouse/pull/95207) ([Azat Khuzhin](https://github.com/azat)). * Fixed inaccurate memory accounting when applying large patch parts after lightweight updates. Previously, applying large patches could cause excessive memory usage and result in the server process being killed by the OOM killer. [#95231](https://github.com/ClickHouse/ClickHouse/pull/95231) ([Anton Popov](https://github.com/CurtizJ)). * Fix undefined behavior that could cause incorrect results or an exception when a distributed query with `max_parallel_replicas` fell back to a local replica during index analysis. [#95263](https://github.com/ClickHouse/ClickHouse/pull/95263) ([Azat Khuzhin](https://github.com/azat)). * Fix aggregation of sparse columns for `sum` and timeseries when `group_by_overflow_mode` is set to `any`. [#95301](https://github.com/ClickHouse/ClickHouse/pull/95301) ([Mikhail Koviazin](https://github.com/mkmkme)). * Fix a reliability issue in `plain_rewritable` disk policy where a network error mid-way through unlinking a metadata file could leave the storage in an inconsistent state. [#95302](https://github.com/ClickHouse/ClickHouse/pull/95302) ([Mikhail Artemenko](https://github.com/Michicosun)). * Replace Date with Date32 for iceberg. [#95322](https://github.com/ClickHouse/ClickHouse/pull/95322) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The password argument of the `redis` table function now will be masked in the logs and system tables (e.g.: `query_log`). [#95325](https://github.com/ClickHouse/ClickHouse/pull/95325) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix a bug where tables could be dropped or altered while a distributed query was still executing against them, potentially causing exceptions or incorrect results. [#95356](https://github.com/ClickHouse/ClickHouse/pull/95356) ([Azat Khuzhin](https://github.com/azat)). * Fix a logical error in some cases when negative `LIMIT/OFFSET` is used in distributed queries. [#95357](https://github.com/ClickHouse/ClickHouse/pull/95357) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix a bug where clickhouse-client would ask for password twice when connecting using ssh. [#95372](https://github.com/ClickHouse/ClickHouse/pull/95372) ([Isak Ellmer](https://github.com/spinojara)). * Fix a data race in storage S3(Azure)Queue. [#95385](https://github.com/ClickHouse/ClickHouse/pull/95385) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix the prewhere filter error caused by lambda expressions in prewhere. [#95395](https://github.com/ClickHouse/ClickHouse/pull/95395) ([Xiaozhe Yu](https://github.com/wudidapaopao)). * Fix `optimize_syntax_fuse_functions` to not rewrite `sum/count/avg` into `sumCount()` when the aggregate argument is `Nullable`. Closes [#95390](https://github.com/ClickHouse/ClickHouse/issues/95390). [#95441](https://github.com/ClickHouse/ClickHouse/pull/95441) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Avoid possible crash for distributed queries in case of cancellation. [#95466](https://github.com/ClickHouse/ClickHouse/pull/95466) ([Aleksandr Musorin](https://github.com/AVMusorin)). * Fix deduplication for streaming from S3(Azure)Queue engine. [#95467](https://github.com/ClickHouse/ClickHouse/pull/95467) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix updating row policies assigned to the initial user in distributed queries. [#95469](https://github.com/ClickHouse/ClickHouse/pull/95469) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix check for encrypted disks over plain_rewritable (Fixes possible `It is not possible to register multiple plain-rewritable disks with the same object storage prefix`). [#95470](https://github.com/ClickHouse/ClickHouse/pull/95470) ([Azat Khuzhin](https://github.com/azat)). * The `mergeTreeProjection` table function was missing an access check, allowing users without SELECT permission on a table (but with permissions for table functions) to read data from its projections. This fix adds the same access check that `mergeTreeIndex` and `mergeTreeAnalyzeIndexes` already have. [#95480](https://github.com/ClickHouse/ClickHouse/pull/95480) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix possible logical error during reading of size subcolumn from dynamic subcolumns of Dynamic/JSON types. [#95573](https://github.com/ClickHouse/ClickHouse/pull/95573) ([Pavel Kruglov](https://github.com/Avogar)). * Fix regression in (experimental) zero‑copy replication introduced by [#94262](https://github.com/ClickHouse/ClickHouse/issues/94262) where shared parts could be deleted before other replicas finished fetching them. [#95597](https://github.com/ClickHouse/ClickHouse/pull/95597) ([filimonov](https://github.com/filimonov)). * Fix crash during `tupleElement` applied to arrays of JSON. Closes [#95581](https://github.com/ClickHouse/ClickHouse/issues/95581). [#95647](https://github.com/ClickHouse/ClickHouse/pull/95647) ([Pavel Kruglov](https://github.com/Avogar)). * Fix logical error exception when using a matcher (`*`) inside a lambda function within a VALUES clause in a JOIN with USING. Close [#93675](https://github.com/ClickHouse/ClickHouse/issues/93675). [#95661](https://github.com/ClickHouse/ClickHouse/pull/95661) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed `There was an error: Cannot obtain error message` logical error when waiting for a distributed DDL and dropping the Replicated database concurrently. Fixes [#95539](https://github.com/ClickHouse/ClickHouse/issues/95539). [#95664](https://github.com/ClickHouse/ClickHouse/pull/95664) ([Alexander Tokmakov](https://github.com/tavplubix)). * Fix `IN` function returning incorrect results with `NULL` values when `transform_null_in` is enabled. Closes [#65776](https://github.com/ClickHouse/ClickHouse/issues/65776). [#95674](https://github.com/ClickHouse/ClickHouse/pull/95674) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Correctly handle LowCardinality Nullable types in `CAST` when the setting `cast_keep_nullable` is enabled. Closes [#95670](https://github.com/ClickHouse/ClickHouse/issues/95670). [#95747](https://github.com/ClickHouse/ClickHouse/pull/95747) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix squashing partitioned delta lake data. [#95773](https://github.com/ClickHouse/ClickHouse/pull/95773) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix race condition for Nullable join column in runtime filters. [#95775](https://github.com/ClickHouse/ClickHouse/pull/95775) ([Hechem Selmi](https://github.com/m-selmi)). * Fix possible logical error in query with matcher (`*`, `table.*`) and `analyzer_compatibility_join_using_top_level_identifier` when `USING` column has different types in tables and select list. Close [#90477](https://github.com/ClickHouse/ClickHouse/issues/90477). [#95808](https://github.com/ClickHouse/ClickHouse/pull/95808) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix memory safety bugs in parallel thread pool operations (backups, aggregation, distributed queries) that could cause exceptions when an error occurred during task scheduling. [#95818](https://github.com/ClickHouse/ClickHouse/pull/95818) ([Raúl Marín](https://github.com/Algunenano)). * Fixes a crash on the DROP WORKLOAD while running concurrently with queries using the workload being dropped. [#95856](https://github.com/ClickHouse/ClickHouse/pull/95856) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix slow performance when querying system tables with a user that has limited grants on many databases. Closes [#89371](https://github.com/ClickHouse/ClickHouse/issues/89371). [#95874](https://github.com/ClickHouse/ClickHouse/pull/95874) ([pufit](https://github.com/pufit)). * Fix executing tupleElement on JSON with nested paths, previously it could lead to wrong query result. [#95907](https://github.com/ClickHouse/ClickHouse/pull/95907) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed a `NOT_SUPPORTED` error that could occur when using the `direct` join algorithm with an empty MergeTree table. [#95935](https://github.com/ClickHouse/ClickHouse/pull/95935) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix the client not suggesting and auto-completing alias names for settings, closes [#92190](https://github.com/ClickHouse/ClickHouse/issues/92190). [#95945](https://github.com/ClickHouse/ClickHouse/pull/95945) ([phulv94](https://github.com/phulv94)). * Fix event_date in system.asynchronous_metric_log. [#95947](https://github.com/ClickHouse/ClickHouse/pull/95947) ([Raúl Marín](https://github.com/Algunenano)). * Fix skipping paths in JSON data type. Previously with `JSON(SKIP path)` all JSON keys with prefix `path` were skipped, even keys like `"pathpath"`, so it could lead to data loss for these paths during insert. Now it's fixed and only key `"path"` is skipped. [#95948](https://github.com/ClickHouse/ClickHouse/pull/95948) ([Pavel Kruglov](https://github.com/Avogar)). * Part with unknown projections should not be marked as lost forever. [#95952](https://github.com/ClickHouse/ClickHouse/pull/95952) ([Mikhail Artemenko](https://github.com/Michicosun)). * Fix empty string becoming `NULL` in `Join` table with `Nullable(String)` key. Closes [#71414](https://github.com/ClickHouse/ClickHouse/issues/71414). [#96002](https://github.com/ClickHouse/ClickHouse/pull/96002) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Now the PostgreSQL engine can correctly read `BOOLEAN[]`. Closes [#72754](https://github.com/ClickHouse/ClickHouse/issues/72754). [#96006](https://github.com/ClickHouse/ClickHouse/pull/96006) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `ProtobufList` format for the case of reading from an empty file. Closes [#70059](https://github.com/ClickHouse/ClickHouse/issues/70059). [#96007](https://github.com/ClickHouse/ClickHouse/pull/96007) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ProtobufList` format producing ghost record for empty tables. Closes [#72596](https://github.com/ClickHouse/ClickHouse/issues/72596). [#96010](https://github.com/ClickHouse/ClickHouse/pull/96010) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `if` function type mismatch between `UInt64` and `Int32` in an unusual case of distributed queries and PREWHERE, with type inference. Closes [#70017](https://github.com/ClickHouse/ClickHouse/issues/70017). [#96012](https://github.com/ClickHouse/ClickHouse/pull/96012) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `JIT` compiled queries involving `Bool` types. [#96013](https://github.com/ClickHouse/ClickHouse/pull/96013) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error when reading UUID column from SQLite TEXT column. Closes [#71263](https://github.com/ClickHouse/ClickHouse/issues/71263). [#96016](https://github.com/ClickHouse/ClickHouse/pull/96016) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix SQLite engine type conversion for `DateTime`, `Date`, `UUID`, and other types. Closes [#73481](https://github.com/ClickHouse/ClickHouse/issues/73481). [#96017](https://github.com/ClickHouse/ClickHouse/pull/96017) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `FixedString` values were escaped incorrectly in queries to external databases, SQLite and PostgreSQL. Closes [#73519](https://github.com/ClickHouse/ClickHouse/issues/73519). Co-authored with @jh0x. [#96019](https://github.com/ClickHouse/ClickHouse/pull/96019) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix assertion failure in WindowTransform with a large PRECEDING offset. Closes [#75852](https://github.com/ClickHouse/ClickHouse/issues/75852). [#96026](https://github.com/ClickHouse/ClickHouse/pull/96026) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a bug with possible data corruption when concurrent async inserts are using the same parameter names but contain different values. [#96035](https://github.com/ClickHouse/ClickHouse/pull/96035) ([Seva Potapov](https://github.com/seva-potapov)). * Fix period for global profilers (controlled by `global_profiler_real_time_period_ns` and `global_profiler_cpu_time_period_ns`). Instead of set value, a truncated value was used, causing profiler to wakeup more than intended. [#96048](https://github.com/ClickHouse/ClickHouse/pull/96048) ([Antonio Andelic](https://github.com/antonio2368)). * Earlier if reference data file inside iceberg manifest file for position delete was present in an entry but was null, we didn't get correct bounds for a corresponding data files. This PR fixes this bug. [#96061](https://github.com/ClickHouse/ClickHouse/pull/96061) ([Daniil Ivanik](https://github.com/divanik)). * Fix revoking default roles. [#96103](https://github.com/ClickHouse/ClickHouse/pull/96103) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix use-after-free in the index analysis in a rare combination of disabled `use_primary_key` and a very large number of disjunctions of conditions that use the index. [#96112](https://github.com/ClickHouse/ClickHouse/pull/96112) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a regression with the `Gorilla` codec when an explicitly specified size does not correspond to the data type size, and the buffer size is too small. In previous versions, it threw an exception on decompression. Closes [#78253](https://github.com/ClickHouse/ClickHouse/issues/78253). [#96118](https://github.com/ClickHouse/ClickHouse/pull/96118) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Avoid a deadlock in dictionaries loaded when one dictionary references a Merge table that references it recursively. Closes [#78360](https://github.com/ClickHouse/ClickHouse/issues/78360). [#96120](https://github.com/ClickHouse/ClickHouse/pull/96120) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix use-of-uninitialized-value in `formatDateTime` with non-fixed-width formatters, such as MySQL and JODA-style. [#96133](https://github.com/ClickHouse/ClickHouse/pull/96133) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The combination of settings `use_const_adaptive_granularity` and `index_granularity_bytes` (which means "non-adaptive granularity") led to a miscalculation of the number of rows to read and an exception. [#96143](https://github.com/ClickHouse/ClickHouse/pull/96143) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Running an invalid ALTER UPDATE mutation on object storage file-like tables, such as S3 and Azure, could lead to a nullptr dereference. Closes [#92994](https://github.com/ClickHouse/ClickHouse/issues/92994). [#96162](https://github.com/ClickHouse/ClickHouse/pull/96162) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `AccessRights::contains` returning incorrect results with partial revokes. [#96170](https://github.com/ClickHouse/ClickHouse/pull/96170) ([pufit](https://github.com/pufit)). * Fix query condition cache hash collision for CTE folded constants, which could lead to a wrong query result. Closes [#96060](https://github.com/ClickHouse/ClickHouse/issues/96060). [#96172](https://github.com/ClickHouse/ClickHouse/pull/96172) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix possible deadlock in ProcessList. It can happen because of possible lock inversion if memory overcommit tracker triggers when we are adding task to cancellation checker. [#96182](https://github.com/ClickHouse/ClickHouse/pull/96182) ([Antonio Andelic](https://github.com/antonio2368)). * Fixed a bug where queries involving outer joins (LEFT, RIGHT, or FULL) combined with multiple INNER JOINs could return incorrect results due to illegal join reordering. When an outer join's ON condition referenced columns from multiple previously joined tables, the optimizer failed to account for all table dependencies and could reorder the joins incorrectly, producing missing rows. Close [#95972](https://github.com/ClickHouse/ClickHouse/issues/95972). [#96193](https://github.com/ClickHouse/ClickHouse/pull/96193) ([Vladimir Cherkasov](https://github.com/vdimir)). * When a table has no statistics defined, ClickHouse shouldn't try to load them. This avoids some overhead (100+ms) for checking if the statistics files exist. (issue [#96068](https://github.com/ClickHouse/ClickHouse/issues/96068)). [#96233](https://github.com/ClickHouse/ClickHouse/pull/96233) ([Han Fei](https://github.com/hanfei1991)). * Fix `optimize_syntax_fuse_functions` to not rewrite `sum/count/avg` into `sumCount()` when the aggregate argument is `LowCardinality(Nullable)`. Closes [#95390](https://github.com/ClickHouse/ClickHouse/issues/95390). [#96239](https://github.com/ClickHouse/ClickHouse/pull/96239) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix incorrect partition pruning for `not IN` and `not has` function in some cases. [#96241](https://github.com/ClickHouse/ClickHouse/pull/96241) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `stack-use-after-scope` in the vector similarity index. [#96259](https://github.com/ClickHouse/ClickHouse/pull/96259) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix test runner not recognizing error hint comments when a query is preceded by a SQL comment. [#96336](https://github.com/ClickHouse/ClickHouse/pull/96336) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Fix logical error in KeyCondition when a table has a nullable primary key, and the query uses the `coalesce` function, which has its first argument constant. [#96340](https://github.com/ClickHouse/ClickHouse/pull/96340) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * The interaction of `GROUPING SETS`, `group_by_use_nulls`, and the `Tuple` data type with `LowCardinality` inside it, could produce an unexpected block structure in the query pipeline, which led to a logical error. This appeared after the introduction of `Nullable` `Tuple`s. [#96358](https://github.com/ClickHouse/ClickHouse/pull/96358) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * It was possible to create a table with an empty expression `()` as an index, which led to an invalid memory access. [#96363](https://github.com/ClickHouse/ClickHouse/pull/96363) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed crash in old analyzer if JOIN and duplicated aliases. [#96405](https://github.com/ClickHouse/ClickHouse/pull/96405) ([Ilya Golshtein](https://github.com/ilejn)). * Fix `Nested columns sizes are inconsistent with local_discriminators` error due to a wrong in-place filtering optimization for Variant columns. [#96410](https://github.com/ClickHouse/ClickHouse/pull/96410) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `CREATE TABLE ... CLONE AS ...` ignoring full qualifier of source table. [#96415](https://github.com/ClickHouse/ClickHouse/pull/96415) ([Hasyimi Bahrudin](https://github.com/hasyimibhar)). * Fix `mysql` table function canceling by KILL QUERY and cancel query (Ctrl+C) in clickhouse-client. [#96437](https://github.com/ClickHouse/ClickHouse/pull/96437) ([Roman Vasin](https://github.com/rvasin)). * Fixes livelock in the cancellation checker thread for queries with high `max_execution_time` values. [#96450](https://github.com/ClickHouse/ClickHouse/pull/96450) ([Sergei Trifonov](https://github.com/serxa)). * Fix a logical error in some cases when fractional `LIMIT/OFFSET` is used in distributed queries. [#96475](https://github.com/ClickHouse/ClickHouse/pull/96475) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix null pointer dereference in certain expressions with lambda functions. [#96479](https://github.com/ClickHouse/ClickHouse/pull/96479) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results when `LowCardinality` columns are converted to `Nullable`. [#96483](https://github.com/ClickHouse/ClickHouse/pull/96483) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix a crash when creating an Iceberg table with an `ORDER BY` clause referencing a non-existent column or using a positional argument. Closes [#93280](https://github.com/ClickHouse/ClickHouse/issues/93280). [#96484](https://github.com/ClickHouse/ClickHouse/pull/96484) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix runtime filter exception for Tuple columns with Nullable subfields. [#96509](https://github.com/ClickHouse/ClickHouse/pull/96509) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception in Parquet V3 native reader when `PREWHERE` filter column contains non-boolean UInt8 values. [#96594](https://github.com/ClickHouse/ClickHouse/pull/96594) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix implicit index regeneration in replicated tables during metadata changes. [#96600](https://github.com/ClickHouse/ClickHouse/pull/96600) ([Raúl Marín](https://github.com/Algunenano)). * Fixes a datarace on DROP WORKLOAD. [#96614](https://github.com/ClickHouse/ClickHouse/pull/96614) ([Sergei Trifonov](https://github.com/serxa)). * Fix a bug in Iceberg table writes where partitioned inserts could produce incorrect data distribution across partition files. [#96620](https://github.com/ClickHouse/ClickHouse/pull/96620) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fixed `heap-use-after-free` in `CREATE TABLE` with constraints. [#96669](https://github.com/ClickHouse/ClickHouse/pull/96669) ([Nikita Taranov](https://github.com/nickitat)). * Validate witness version in bech32 to avoid buffer overflow. [#96671](https://github.com/ClickHouse/ClickHouse/pull/96671) ([Raúl Marín](https://github.com/Algunenano)). * Fix `system.tables` returning errors when a Data Lake REST catalog is created with an invalid `auth_header` setting. [#96680](https://github.com/ClickHouse/ClickHouse/pull/96680) ([Han Fei](https://github.com/hanfei1991)). * Fix `min(timestamp)` returning epoch (`1970-01-01`) via `_minmax_count_projection` after TTL merge when all rows in a block are filtered out. [#96703](https://github.com/ClickHouse/ClickHouse/pull/96703) ([Raquel Barbadillo](https://github.com/rbarbadillo)). * Improve validation of the `iceberg_metadata_file_path` setting to prevent path traversal and ensure the specified metadata file is within the table directory. [#96754](https://github.com/ClickHouse/ClickHouse/pull/96754) ([Daniil Ivanik](https://github.com/divanik)). * Fix crash in `ifNull` with `Variant` argument used in `GROUP BY`. [#96790](https://github.com/ClickHouse/ClickHouse/pull/96790) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fixed cache key collisions between tables with `table_disk=1` setting. [#96818](https://github.com/ClickHouse/ClickHouse/pull/96818) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix MemoryWorker's purging thread being stuck because of a race condition. [#96819](https://github.com/ClickHouse/ClickHouse/pull/96819) ([Antonio Andelic](https://github.com/antonio2368)). * Don't log data with credentials in iceberg catalogs. [#96831](https://github.com/ClickHouse/ClickHouse/pull/96831) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix exit status of clickhouse-client after server error. [#96841](https://github.com/ClickHouse/ClickHouse/pull/96841) ([Vitaly Baranov](https://github.com/vitlibar)). * Queries with CROSS JOINs and enabled parallel replicas could return incorrect result. Fixes [#74337](https://github.com/ClickHouse/ClickHouse/issues/74337). [#96848](https://github.com/ClickHouse/ClickHouse/pull/96848) ([Igor Nikonov](https://github.com/devcrafter)). * Fixed `ALTER TABLE DROP COLUMN` queries failing after a lightweight update was previously performed on the same column. [#96861](https://github.com/ClickHouse/ClickHouse/pull/96861) ([Anton Popov](https://github.com/CurtizJ)). * Fix stack overflow (crash) when creating archive-based backups (`.zip`, `.tzst`) to a `plain_rewritable` object storage disk. [#96872](https://github.com/ClickHouse/ClickHouse/pull/96872) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix server crash when backup fails due to full disk or other I/O errors on the destination filesystem. [#96873](https://github.com/ClickHouse/ClickHouse/pull/96873) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `EXCEPT ALL` and `INTERSECT ALL` ignoring row multiplicities and behaving like their `DISTINCT` counterparts. [#96876](https://github.com/ClickHouse/ClickHouse/pull/96876) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `std::terminate` exception in `indexOfAssumeSorted` when called with incompatible types (e.g., `IPv4` array with integer search value). [#96877](https://github.com/ClickHouse/ClickHouse/pull/96877) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception `Bad cast from type DB::ColumnNullable to DB::ColumnString` when using window functions with `group_by_use_nulls = 1` and CUBE/ROLLUP/GROUPING SETS. [#96878](https://github.com/ClickHouse/ClickHouse/pull/96878) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect results when JIT-compiled expressions convert `DateTime` to `DateTime64` (e.g., in `CASE`/`if`/`multiIf` with mixed DateTime types). The value was reinterpreted instead of properly scaled, producing wrong timestamps after expression compilation kicked in. [#96879](https://github.com/ClickHouse/ClickHouse/pull/96879) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception in `CoalescingMergeTree` when a skip index expression produces a constant column (e.g., `bloom_filter` on `ifNotFinite(1, c0)` for an integer column). [#96880](https://github.com/ClickHouse/ClickHouse/pull/96880) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix wrong port number in error message when accidentally connecting with HTTP to the TLS-enabled native protocol port. [#96881](https://github.com/ClickHouse/ClickHouse/pull/96881) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix per-subquery `SETTINGS` not being applied to table functions like `file` in CTEs and subqueries. [#96882](https://github.com/ClickHouse/ClickHouse/pull/96882) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix memory leak of BIO objects when reading X509 certificates. [#96885](https://github.com/ClickHouse/ClickHouse/pull/96885) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception in query analyzer when a lambda expression is passed where a concrete value is expected (e.g., as the accumulator argument of `arrayFold`). [#96892](https://github.com/ClickHouse/ClickHouse/pull/96892) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `ColumnNullable is not compatible with original` exception when casting complex nested types (Array of Nullable Tuple containing Map with Nullable Enum values). [#96924](https://github.com/ClickHouse/ClickHouse/pull/96924) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a race condition in sharded `HASHED` dictionary parallel loading that could occasionally cause some rows to not be loaded. [#96953](https://github.com/ClickHouse/ClickHouse/pull/96953) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a race condition between `REPLACE PARTITION` and background mutations that could result in both old and new data being visible after the replace. [#96955](https://github.com/ClickHouse/ClickHouse/pull/96955) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `arrayJoin` function producing duplicate rows when used with INNER JOIN and WHERE clause, caused by the partial predicate push-down optimization incorrectly pushing filters containing `arrayJoin` below a JOIN. [#96989](https://github.com/ClickHouse/ClickHouse/pull/96989) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix crash (SEGFAULT) in `clearCaches` caused by `BlockIO::operator=` not moving `query_metadata_cache`, leading to premature destruction of cached storage snapshots and use-after-free of `MergeTreeData` storage. [#96995](https://github.com/ClickHouse/ClickHouse/pull/96995) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix assertion failure in `IfTransformStringsToEnumPass` when the `if` or `transform` function returns `Nullable(String)` (e.g. with `GROUP BY ... WITH CUBE` and `group_by_use_nulls = true`). [#97002](https://github.com/ClickHouse/ClickHouse/pull/97002) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect data written during `INSERT ... SELECT` with `UNION ALL` and `JOIN`, where constant string columns could receive wrong values after block squashing. [#97019](https://github.com/ClickHouse/ClickHouse/pull/97019) ([Hasyimi Bahrudin](https://github.com/hasyimibhar)). * Fix `assert_cast` exception (or silent data corruption in release builds) when building column statistics after `ALTER TABLE MODIFY COLUMN` changes the column type. [#97027](https://github.com/ClickHouse/ClickHouse/pull/97027) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix reads of uninitialized memory in Azure Blob Storage, SSH protocol, and Arrow Flight interfaces. [#97053](https://github.com/ClickHouse/ClickHouse/pull/97053) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix cases where indexes were affecting the result for queries with row policy/PREWHERE and FINAL. [#97076](https://github.com/ClickHouse/ClickHouse/pull/97076) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix remaining race condition between `REPLACE PARTITION` and background mutations in `MergeTree` tables that could cause old data to reappear. [#97105](https://github.com/ClickHouse/ClickHouse/pull/97105) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix implicit indices with alias columns and do full validation before creating them. [#97115](https://github.com/ClickHouse/ClickHouse/pull/97115) ([Raúl Marín](https://github.com/Algunenano)). * Fix logical error in `FunctionVariantAdaptor` with functions requiring const arguments like `arrayROCAUC`. [#97116](https://github.com/ClickHouse/ClickHouse/pull/97116) ([Bharat Nallan](https://github.com/bharatnc)). * Fix stuck mutations when `PartCheckThread` re-enqueues a `GET_PART` for an already-mutated part, leaving phantom entries in `parts_to_do`. [#97162](https://github.com/ClickHouse/ClickHouse/pull/97162) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix query plan row count estimation for subqueries with `ORDER BY ... LIMIT`, which could cause the optimizer to choose a suboptimal join order. [#97193](https://github.com/ClickHouse/ClickHouse/pull/97193) ([Alexander Gololobov](https://github.com/davenger)). * Fix `LOGICAL_ERROR` exception in `FunctionVariantAdaptor` when a function operating on Variant columns returns `Nothing` type, which can happen with empty arrays in `UNION ALL` queries. [#97213](https://github.com/ClickHouse/ClickHouse/pull/97213) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a data race during S3 multipart copy operations (e.g., during `BACKUP`/`RESTORE` to S3) that could cause exceptions under concurrent access. [#97227](https://github.com/ClickHouse/ClickHouse/pull/97227) ([Azat Khuzhin](https://github.com/azat)). * Fix `LOGICAL_ERROR` exception when `arrayJoin` in `WHERE` clause references columns from both sides of a `JOIN`. [#97239](https://github.com/ClickHouse/ClickHouse/pull/97239) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception when reading `.size` subcolumn of a sparse `Nullable(String)` in a Tuple with PREWHERE. [#97264](https://github.com/ClickHouse/ClickHouse/pull/97264) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix exception "Number of rows in lazy chunk does not match number of offsets" in `LazyMaterializingTransform` when reading from tables with non-adaptive index granularity (`index_granularity_bytes = 0`) using `ORDER BY ... LIMIT`. [#97270](https://github.com/ClickHouse/ClickHouse/pull/97270) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `SYSTEM RESTART REPLICA` losing table from database when table re-creation fails with a non-ZooKeeper exception (e.g. memory limit), causing metadata digest mismatches in `DatabaseReplicated`. [#97276](https://github.com/ClickHouse/ClickHouse/pull/97276) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Field `readonly` in `system.merge_tree_settings` now properly reflects that certain merge tree settings (e.g. `index_granularity`) are unconditionally readonly. [#97277](https://github.com/ClickHouse/ClickHouse/pull/97277) ([Robert Schulze](https://github.com/rschu1ze)). * Fix a crash during `count()` optimization on `MergeTree` tables when the storage snapshot was created without data. [#97281](https://github.com/ClickHouse/ClickHouse/pull/97281) ([Pablo Marcos](https://github.com/pamarcos)). * Fix a possible crash when resolving function names from debug information for stack traces. [#97294](https://github.com/ClickHouse/ClickHouse/pull/97294) ([Azat Khuzhin](https://github.com/azat)). * Fix logical error with analyzer_compatibility_join_using_top_level_identifier and ALIAS columns. Close [#96228](https://github.com/ClickHouse/ClickHouse/issues/96228). [#97297](https://github.com/ClickHouse/ClickHouse/pull/97297) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix `LOGICAL_ERROR` exception in `applyOrder` when using text-indexed columns with `QUALIFY` clause. [#97313](https://github.com/ClickHouse/ClickHouse/pull/97313) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * System table `system.functions` now shows for internal functions `categories = 'Internal'` instead of `categories = ''`. [#97315](https://github.com/ClickHouse/ClickHouse/pull/97315) ([Robert Schulze](https://github.com/rschu1ze)). * Query with RIGHT JOIN chain and enabled parallel replicas can produce incorrect result. Fixes [#74341](https://github.com/ClickHouse/ClickHouse/issues/74341). [#97316](https://github.com/ClickHouse/ClickHouse/pull/97316) ([Igor Nikonov](https://github.com/devcrafter)). * Fix spurious `TABLE_UUID_MISMATCH` errors that could occur with refreshable materialized views and other scenarios where tables are renamed. [#97323](https://github.com/ClickHouse/ClickHouse/pull/97323) ([Azat Khuzhin](https://github.com/azat)). * Fix segfault in `StorageKeeperMap` backup due to use-after-free of dangling storage pointer in lazy backup batch. [#97336](https://github.com/ClickHouse/ClickHouse/pull/97336) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix the `exists` function with a scalar subquery inside an `ALTER UPDATE/DELETE` when `mutations_execute_subqueries_on_initiator` is enabled. The scalar subquery was incorrectly evaluated, which could lead to an error or a corrupt mutation command that made the table unloadable on the next server restart. [#97347](https://github.com/ClickHouse/ClickHouse/pull/97347) ([Kirill Kopnev](https://github.com/Fgrtue)). * Fix logical exception `Unexpected return type from equals. Expected Nullable(UInt8). Got Const(LowCardinality(Nullable(UInt8)))` when comparing NULL with a Variant column containing LowCardinality types. [#97379](https://github.com/ClickHouse/ClickHouse/pull/97379) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a possible race condition when `EXCHANGE TABLES` is executed in parallel with the sharded query cache enabled. [#97411](https://github.com/ClickHouse/ClickHouse/pull/97411) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix `LOGICAL_ERROR` exception in Array-to-`QBit` conversion when `nullable_source` from an outer `Tuple` wrapper replaces the converted array column with a mismatched column type. Closes [#97389](https://github.com/ClickHouse/ClickHouse/issues/97389). [#97413](https://github.com/ClickHouse/ClickHouse/pull/97413) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix AST formatting roundtrip inconsistency for aliased tuple literals inside parentheses, e.g., `(('a', 'b') AS x)` was incorrectly reformatted as `tuple(('a', 'b') AS x)`. [#97418](https://github.com/ClickHouse/ClickHouse/pull/97418) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an exception during asynchronous inserts with deduplication when a parsing failure produced an empty block with zero rows. [#97460](https://github.com/ClickHouse/ClickHouse/pull/97460) ([Sema Checherinda](https://github.com/CheSema)). * Fix exception "Number of rows in lazy chunk does not match number of offsets" in `LazyMaterializingTransform` when reading from tables with non-adaptive index granularity (`index_granularity_bytes = 0`) using `ORDER BY ... LIMIT`. [#97482](https://github.com/ClickHouse/ClickHouse/pull/97482) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix insert iceberg settings. Add alias for the `allow_experimental_insert_into_iceberg` setting. [#97483](https://github.com/ClickHouse/ClickHouse/pull/97483) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix `ACCESS_DENIED` for users without `CREATE TEMPORARY TABLE` permission when `optimize_inverse_dictionary_lookup` optimization rewrites `dictGet(...)` predicates. ClickHouse now skips the rewrite and executes the original expression. Closes [#97269](https://github.com/ClickHouse/ClickHouse/issues/97269). [#97484](https://github.com/ClickHouse/ClickHouse/pull/97484) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix assertion failure (exception in debug/sanitizer builds) in `Set` and `MergeTreeIndexSet` when processing columns with inner sparse subcolumns (e.g., `Tuple` columns from MergeTree parts with different sparse serialization profiles). [#97493](https://github.com/ClickHouse/ClickHouse/pull/97493) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a possible use after free in StorageKafka2. [#97520](https://github.com/ClickHouse/ClickHouse/pull/97520) ([Bharat Nallan](https://github.com/bharatnc)). * Fix `INTO OUTFILE` with `TRUNCATE` and `into_outfile_create_parent_directories` setting when the output path contains directories. [#97549](https://github.com/ClickHouse/ClickHouse/pull/97549) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `BAD_ARGUMENTS` error when querying tables with lambda expressions inside ALIAS columns through the `merge()` table function with the analyzer enabled. [#97551](https://github.com/ClickHouse/ClickHouse/pull/97551) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `system.zookeeper_info` exception when Keeper zxid is 0. [#97553](https://github.com/ClickHouse/ClickHouse/pull/97553) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a possible logical error in `ip_trie` dictionary when key type is not String. [#97555](https://github.com/ClickHouse/ClickHouse/pull/97555) ([Bharat Nallan](https://github.com/bharatnc)). * Fix REST catalog OAuth authentication not working for the base `RestCatalog` (it only worked for derived catalogs like `OneLakeCatalog`). This broke the default REST catalog after BigLake catalog was introduced. [#97561](https://github.com/ClickHouse/ClickHouse/pull/97561) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Geometry functions (`perimeterSpherical`, `areaSpherical`, etc.) now accept individual geometry subtypes (`Polygon`, `Ring`, `Point`, etc.) in addition to the `Geometry` variant type. [#97571](https://github.com/ClickHouse/ClickHouse/pull/97571) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `LOGICAL_ERROR` exception when using `isNull`/`isNotNull` on subcolumns of `Nullable(Tuple(... Nullable(T) ...))` types. Closes [#97224](https://github.com/ClickHouse/ClickHouse/issues/97224). [#97582](https://github.com/ClickHouse/ClickHouse/pull/97582) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix null pointer dereference when applying patch parts during lightweight updates. [#97583](https://github.com/ClickHouse/ClickHouse/pull/97583) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `BaseSettings::readBinary` passes the index from `accessor.find` to `field_infos[]` without checking for the not-found sentinel value (i.e., `-1`), which may cause a `std::vector` out-of-bounds access. The issue was caught thanks to libcxx hardening. This probably happened during query plan deserialization when a newer server sends a setting unknown to an older server. The string-based read method already handles this correctly; `readBinary` was missing the same check. [#97585](https://github.com/ClickHouse/ClickHouse/pull/97585) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix incorrect query results for `UNION ALL` queries where one branch had a constant-false predicate — the branch would incorrectly read data instead of returning nothing. [#97620](https://github.com/ClickHouse/ClickHouse/pull/97620) ([Bharat Nallan](https://github.com/bharatnc)). * Fix `IN (col)` with a single column reference failing with `UNSUPPORTED_METHOD` error. [#97646](https://github.com/ClickHouse/ClickHouse/pull/97646) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix logical error exception during `GROUP BY ... WITH ROLLUP/CUBE` when keys include `LowCardinality(Nullable(...))` inside `Nullable(Tuple(...))`. [#97647](https://github.com/ClickHouse/ClickHouse/pull/97647) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix AST formatting inconsistency for `NOT (1, 1, 1)` that could cause `LOGICAL_ERROR` in debug builds. [#97653](https://github.com/ClickHouse/ClickHouse/pull/97653) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix `keeper-converter` exception when encountering empty ZooKeeper transaction log files. [#97673](https://github.com/ClickHouse/ClickHouse/pull/97673) ([Alexey Milovidov](https://github.com/alexey-milovidov)). #### Build/Testing/Packaging Improvement * ClickHouse can be built with clang-23 (master). [#95578](https://github.com/ClickHouse/ClickHouse/pull/95578) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix force `is_local` to false when `bind_host` is configured and replace with integration test. Follow-up for [#74741](https://github.com/ClickHouse/ClickHouse/pull/74741). [#93109](https://github.com/ClickHouse/ClickHouse/pull/93109) [#96018](https://github.com/ClickHouse/ClickHouse/pull/96018) ([Zhigao Hong](https://github.com/zghong)). * Stress tests: fix stress and upgrade tests in CI. ignore `no-{build}` tags. add compatibility randomization. [#94693](https://github.com/ClickHouse/ClickHouse/pull/94693) ([Nikita Fomichev](https://github.com/fm4v)). * Publish parser_memory_profiler binary from build. Tool can be used to analyze AST memory consumption. [#95826](https://github.com/ClickHouse/ClickHouse/pull/95826) ([Ilya Yatsishin](https://github.com/qoega)). * Add `--symbolize` flag for `parser_memory_profiler` tool that produces `.heap.sym` files with resolved symbols in results. [#96477](https://github.com/ClickHouse/ClickHouse/pull/96477) ([Ilya Yatsishin](https://github.com/qoega)). * Pin third-party Docker images in integration tests to specific versions. [#96500](https://github.com/ClickHouse/ClickHouse/pull/96500) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Restore the possibility to link `OpenSSL` dynamically. This is not recommended and not used by any production builds, but the option still exists for enthusiasts on the Internet. [#96506](https://github.com/ClickHouse/ClickHouse/pull/96506) ([Govind R Nair](https://github.com/Revertionist)). * Reduce `magic_enum` range from [-100, 1000] to default [-128, 127] by using a per-type specialization for `Coordination::OpNum`, improving build time. [#96632](https://github.com/ClickHouse/ClickHouse/pull/96632) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove unnecessary C++ templates from Function classes to reduce build times. [#96646](https://github.com/ClickHouse/ClickHouse/pull/96646) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Move `StorageSystemLicenses` generation to configure time to improve build parallelism. [#96697](https://github.com/ClickHouse/ClickHouse/pull/96697) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Parallelize license scanning. [#96727](https://github.com/ClickHouse/ClickHouse/pull/96727) ([Raúl Marín](https://github.com/Algunenano)). * Add stateless functional test for SSH protocol support. [#96996](https://github.com/ClickHouse/ClickHouse/pull/96996) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add Kafka 3.9.0 to the stateless functional test infrastructure, enabling direct testing of Kafka and Kafka2 table engines using ClickHouse Keeper as ZooKeeper. Six new stateless tests cover basic produce/consume, virtual columns, INSERT, multiple formats, broken message handling, and Keeper-based offset storage. [#96997](https://github.com/ClickHouse/ClickHouse/pull/96997) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Add a CI workflow to build PGO+BOLT optimized clang toolchain. [#96991](https://github.com/ClickHouse/ClickHouse/pull/96991) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Use the PGO-optimized LLVM/Clang build in CI, which should give 20..30% build speed improvement. [#97031](https://github.com/ClickHouse/ClickHouse/pull/97031) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Replace math functions from glibc with llvm-libc implementations. [#90151](https://github.com/ClickHouse/ClickHouse/pull/90151) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Update Boost from 1.83 to 1.90, fixing a `devector` assertion failure in debug builds. [#97037](https://github.com/ClickHouse/ClickHouse/pull/97037) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Update `postgres` to REL_18_1. [#95189](https://github.com/ClickHouse/ClickHouse/pull/95189) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `libexpat` 2.7.3. [#95218](https://github.com/ClickHouse/ClickHouse/pull/95218) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `OpenSSL` 3.5.5. [#95345](https://github.com/ClickHouse/ClickHouse/pull/95345) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `simdjson` v4.2.4. [#97129](https://github.com/ClickHouse/ClickHouse/pull/97129) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `libarchive` 3.8.5. [#97131](https://github.com/ClickHouse/ClickHouse/pull/97131) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `fast_float` v8.2.3. [#97133](https://github.com/ClickHouse/ClickHouse/pull/97133) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Use `abseil-cpp` 20260107.1, `s2geometry` to v0.13.1. [#97134](https://github.com/ClickHouse/ClickHouse/pull/97134) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Bump `libxml2` to 2.15.1. [#95574](https://github.com/ClickHouse/ClickHouse/pull/95574) ([Robert Schulze](https://github.com/rschu1ze)). * Upgraded 7 Tier-3 integration test Docker images from EOL or removed base images to current supported versions. [#97314](https://github.com/ClickHouse/ClickHouse/pull/97314) ([Rahul](https://github.com/motsc)). * Add TPC-DS benchmark queries. [#97349](https://github.com/ClickHouse/ClickHouse/pull/97349) ([Raufs Dunamalijevs](https://github.com/rienath)). * Replace individual x86 instruction-set cmake options (`ENABLE_SSSE3`, `ENABLE_AVX2`, `NO_SSE3_OR_HIGHER`, `ARCH_NATIVE`, etc.) with a single numeric `X86_ARCH_LEVEL` option (`1`/`2`/`3`/`4`), matching the standard x86-64 microarchitecture levels already used by the runtime dispatch system. [#97354](https://github.com/ClickHouse/ClickHouse/pull/97354) ([Raúl Marín](https://github.com/Algunenano)). * Avoid instantiating `division_by_nullable=true` template variants for non-division operations in `FunctionBinaryArithmetic`, reducing compilation time and binary size. [#97496](https://github.com/ClickHouse/ClickHouse/pull/97496) ([Raúl Marín](https://github.com/Algunenano)). * Reduce the include footprint of `Exception.h` by removing it from high-fan-out headers like `typeid_cast.h`, `assert_cast.h`, `Context_fwd.h`, `IDataType.h`, and various Column headers. [#97497](https://github.com/ClickHouse/ClickHouse/pull/97497) ([Raúl Marín](https://github.com/Algunenano)). * Always use bundled `compiler-rt` headers (sanitizer and XRay interfaces) instead of the host compiler's headers, and build `compiler-rt` libraries from source by default. [#97499](https://github.com/ClickHouse/ClickHouse/pull/97499) ([Raúl Marín](https://github.com/Algunenano)). * Avoid including boost/multiprecision headers in `wide_integer_impl.h` on platforms with adequate `long double`, improving build time. [#96633](https://github.com/ClickHouse/ClickHouse/pull/96633) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implement LLVM Code Coverage job and enable it initially for master branch. [#90952](https://github.com/ClickHouse/ClickHouse/pull/90952) ([Alexey Bakharew](https://github.com/alexbakharew)). * Enable fast libcxx hardening for release builds. This is mostly needed for out-of-bounds checks. Given performance tests results, no noticeable performance impact is expected. [#94757](https://github.com/ClickHouse/ClickHouse/pull/94757) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). ### ClickHouse release 26.1, 2026-01-29. [Presentation](https://presentations.clickhouse.com/2026-release-26.1/), [Video](https://www.youtube.com/watch?v=fWuYt4M0xE4) #### Backward Incompatible Change * Fix inconsistent formatting caused by an incorrect substitution of aliases in the formatter. This closes [#82833](https://github.com/ClickHouse/ClickHouse/issues/82833). This closes [#82832](https://github.com/ClickHouse/ClickHouse/issues/82832). This closes [#68296](https://github.com/ClickHouse/ClickHouse/issues/68296). This change is potentially backward incompatible: when the analyzer is disabled, certain CREATE VIEW queries with IN referencing an alias cannot be processed. To prevent the incompatibility, enable the analyzer (it is enabled by default since 24.3). [#82838](https://github.com/ClickHouse/ClickHouse/pull/82838) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Codecs `DEFLATE_QPL` and `ZSTD_QAT` were removed. Users are advised to convert existing data compressed with `DEFLATE_QPL` or `ZSTD_QAT` to another codec before upgrade. Note that in order to use the codecs, settings `enable_deflate_qpl_codec` and `enable_zstd_qat_codec` had to be enabled. [#92150](https://github.com/ClickHouse/ClickHouse/pull/92150) ([Robert Schulze](https://github.com/rschu1ze)). * Improve UDF debugging by enabling stderr capture in `system.query_log.exception`. Previously, UDF stderr was only logged to files and not exposed in query logs, making debugging impossible. Now stderr triggers exceptions by default and is fully accumulated (up to 1MB) before throwing, so complete Python tracebacks and error messages appear in `system.query_log.exception` for effective troubleshooting. [#92209](https://github.com/ClickHouse/ClickHouse/pull/92209) ([Xu Jia](https://github.com/XuJia0210)). * Empty column list in `JOIN USING ()` clause is now considered a syntax error. Previously it was supposed to be `INVALID_JOIN_ON_EXPRESSION` during query execution. In some cases such as joining with `Join` storage it led to `LOGICAL_ERROR`, close [#82502](https://github.com/ClickHouse/ClickHouse/issues/82502). [#92371](https://github.com/ClickHouse/ClickHouse/pull/92371) ([Vladimir Cherkasov](https://github.com/vdimir)). * Use partial match for SKIP REGEXP in JSON type by default. Closes [#79250](https://github.com/ClickHouse/ClickHouse/issues/79250). [#92847](https://github.com/ClickHouse/ClickHouse/pull/92847) ([Pavel Kruglov](https://github.com/Avogar)). * Revert "Allow INSERT into simple ALIAS columns" (Reverts ClickHouse/ClickHouse[#84154](https://github.com/ClickHouse/ClickHouse/issues/84154)). It does not work with custom formats, and is not guarded with a setting. [#92849](https://github.com/ClickHouse/ClickHouse/pull/92849) ([Azat Khuzhin](https://github.com/azat)). * Setting to throw an error if a data lake catalog doesn't have access to object storage. [#93606](https://github.com/ClickHouse/ClickHouse/pull/93606) ([Konstantin Vedernikov](https://github.com/scanhex12)). * The `Lazy` database engine is removed and no longer available. Closes [#91231](https://github.com/ClickHouse/ClickHouse/issues/91231). [#93627](https://github.com/ClickHouse/ClickHouse/pull/93627) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove the `transposed_with_wide_view` mode of the `metric_log` - it is unusable due to a bug. It is no longer possible to define `system.metric_log` with this mode. This partially reverts [#78412](https://github.com/ClickHouse/ClickHouse/issues/78412). [#93867](https://github.com/ClickHouse/ClickHouse/pull/93867) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * CPU scheduling for workloads is now preemptive by default. See `cpu_slot_preemption` server setting. [#94060](https://github.com/ClickHouse/ClickHouse/pull/94060) ([Sergei Trifonov](https://github.com/serxa)). * Escape index filenames to prevent broken parts. With this change ClickHouse will fail to load indices with non-ascii characters in their name created by previous versions. To handle it you can use the merge tree setting `escape_index_filenames`. [#94079](https://github.com/ClickHouse/ClickHouse/pull/94079) ([Raúl Marín](https://github.com/Algunenano)). * Format settings `exact_rows_before_limit`, `rows_before_aggregation`, `cross_to_inner_join_rewrite`, `regexp_dict_allow_hyperscan`, `regexp_dict_flag_case_insensitive`, `regexp_dict_flag_dotall` and `dictionary_use_async_executor` were changed to be regular (non-format) settings now. This is a purely internal change without user-visible side effects except in the (unlikely) case that you specified any of these settings in Iceberg or DeltaLake or Kafka or S3 or S3Queue or Azure or Hive or RabbitMQ or Set or FileLog or NATS table engine definitions. In these cases, these settings were previously ignored, now such definitions throw an error. [#94106](https://github.com/ClickHouse/ClickHouse/pull/94106) ([Robert Schulze](https://github.com/rschu1ze)). * The `joinGet/joinGetOrNull` functions now enforce `SELECT` privileges on the underlying Join table. After this change, executing `joinGet('db.table', 'column', key)` requires the user to have `SELECT` privilege on both the key columns defined in the Join table and the attribute column being retrieved. Queries lacking these privileges will fail with `ACCESS_DENIED`. To migrate, grant the necessary permissions using `GRANT SELECT ON db.join_table TO user` for full table access, or `GRANT SELECT(key_col, attr_col) ON db.join_table TO user` for column-level access. This change affects all users and applications relying on `joinGet`/`joinGetOrNull` where explicit `SELECT` grants were not previously configured. [#94307](https://github.com/ClickHouse/ClickHouse/pull/94307) ([Vladimir Cherkasov](https://github.com/vdimir)). * Check `SHOW COLUMNS` for `CREATE TABLE ... AS ...` queries. Previously, it checked `SHOW TABLES`, which is an incorrect grant for this type of permission check. [#94556](https://github.com/ClickHouse/ClickHouse/pull/94556) ([pufit](https://github.com/pufit)). * Make the `Hash` output format independent of block sizes. [#94503](https://github.com/ClickHouse/ClickHouse/pull/94503) ([Alexey Milovidov](https://github.com/alexey-milovidov)). Note that this changes the output hash values compared to previous versions. #### New Feature * HTTP API and embedded Web UI for ClickHouse Keeper. [#78181](https://github.com/ClickHouse/ClickHouse/pull/78181) ([pufit](https://github.com/pufit) and [speeedmaster](https://github.com/speeedmaster)). * Async insert deduplication now works with dependent materialized views. When collision by block_id occurs, the original block is filtered to remove rows associated with the block_id, and the remaining rows are transformed with all relevant materialized views select queries, this rebuilds original block without conflicting rows. [#89140](https://github.com/ClickHouse/ClickHouse/pull/89140) ([Sema Checherinda](https://github.com/CheSema)). It is allowed to use deduplication with async inserts when materialized views are involved. [#93957](https://github.com/ClickHouse/ClickHouse/pull/93957) ([Sema Checherinda](https://github.com/CheSema)). * Introduced a new syntax and framework to simplify and extend projection index feature. This follows up https://github.com/ClickHouse/ClickHouse/pull/81021. [#91844](https://github.com/ClickHouse/ClickHouse/pull/91844) ([Amos Bird](https://github.com/amosbird)). * Add text index support for `Array` columns. [#89895](https://github.com/ClickHouse/ClickHouse/pull/89895) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Enable `use_variant_as_common_type` by default, which lets you use incompatible types inside an `Array`, in `UNION` queries, and in branches of `if`/`multiIf`/`case`. [#90677](https://github.com/ClickHouse/ClickHouse/pull/90677) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * New system table `zookeeper_info`. Implements [#88014](https://github.com/ClickHouse/ClickHouse/issues/88014). [#90809](https://github.com/ClickHouse/ClickHouse/pull/90809) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Support the `Variant` type in all functions. [#90900](https://github.com/ClickHouse/ClickHouse/pull/90900) ([Bharat Nallan](https://github.com/bharatnc)). * Adds a `ClickHouse_Info` metric to the Prometheus `/metrics` endpoint containing mainly version information so it's possible to build charts tracking detailed version information over time. [#91125](https://github.com/ClickHouse/ClickHouse/pull/91125) ([Christoph Wurm](https://github.com/cwurm)). * Introduce a new four letter `rcfg` command for keeper which allows to change cluster configuration. This command provides broader possibilities for configuration changes than standard `reconfigure` request. Command takes `json` string as an argument. The whole set of bytes sent to TCP interface should look like this: `rcfg{json_string_length_big_endian}{json_string}`. Some examples of command may look like this: `{"preconditions": {"leaders": [1, 2], "members": [1, 2, 3, 4, 5]}, "actions": [{"transfer_leadership": [3]}, {"remove_members": [1, 2]}, {"set_priority": [{"id": 4, "priority": 100}, {"id": 5, "priority": 100}]}, {"transfer_leadership": [4, 5]}, {"set_priority": [{"id": 3, "priority": 0}]}]}`. [#91354](https://github.com/ClickHouse/ClickHouse/pull/91354) ([alesapin](https://github.com/alesapin)). * Add function `reverseBySeparator` which reverses the order of substrings in a string separated by a specified separator. Close [#91463](https://github.com/ClickHouse/ClickHouse/issues/91463). [#91780](https://github.com/ClickHouse/ClickHouse/pull/91780) ([Xuewei Wang](https://github.com/Sallery-X)). * Adds new setting `max_insert_block_size_bytes` which control the formation of inserted blocks in finer detail. [#92833](https://github.com/ClickHouse/ClickHouse/pull/92833) ([Kirill Kopnev](https://github.com/Fgrtue)). * It is possible to execute DDL queries with `ON CLUSTER` clause for a Replicated database if the `ignore_on_cluster_for_replicated_database` setting is enabled. In this case, the cluster name will be ignored. [#92872](https://github.com/ClickHouse/ClickHouse/pull/92872) ([Kirill](https://github.com/kirillgarbar)). * Implement `mergeTreeAnalyzeIndexes` function. [#92954](https://github.com/ClickHouse/ClickHouse/pull/92954) ([Azat Khuzhin](https://github.com/azat)). * Add new setting `use_primary_key`. Set it to `false` to disable granule pruning based on the primary key. [#93319](https://github.com/ClickHouse/ClickHouse/pull/93319) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Add `icebergLocalCluster` table function. [#93323](https://github.com/ClickHouse/ClickHouse/pull/93323) ([Anton Ivashkin](https://github.com/ianton-ru)). * Added `cosineDistanceTransposed` function that approximates the [cosine distance](https://en.wikipedia.org/wiki/Cosine_similarity#Cosine_distance) between two points. [#93621](https://github.com/ClickHouse/ClickHouse/pull/93621) ([Raufs Dunamalijevs](https://github.com/rienath)). * Add `files` column to system.parts table that shows the number of files in each data part. [#94337](https://github.com/ClickHouse/ClickHouse/pull/94337) ([Match](https://github.com/gayanMatch)). * Adds a max-min fair scheduler for concurrency control. Provides better fairness under high oversubscription, where many queries compete for limited CPU slots. Short-running queries are not penalized by long-running queries that have accumulated more slots over time. Enabled by the `concurrent_threads_scheduler` server setting `max_min_fair` value. [#94732](https://github.com/ClickHouse/ClickHouse/pull/94732) ([Sergei Trifonov](https://github.com/serxa)). * Added the ability for ClickHouse client to override TLS SNI when connecting to the server. [#89761](https://github.com/ClickHouse/ClickHouse/pull/89761) ([Matt Klein](https://github.com/mattklein123)). * Support temporary tables in `joinGet` function calls. [#92973](https://github.com/ClickHouse/ClickHouse/pull/92973) ([Eduard Karacharov](https://github.com/korowa)). * Support deletion vectors in `DeltaLake` table engine. [#93852](https://github.com/ClickHouse/ClickHouse/pull/93852) ([Kseniia Sumarokova](https://github.com/kssenii)). * Support deletion vectors for `deltaLakeCluster`. [#94365](https://github.com/ClickHouse/ClickHouse/pull/94365) ([Kseniia Sumarokova](https://github.com/kssenii)). * Google cloud storage support for data lakes. [#93866](https://github.com/ClickHouse/ClickHouse/pull/93866) ([Konstantin Vedernikov](https://github.com/scanhex12)). #### Experimental Feature * Move `QBit` from Experimental to Beta. [#93816](https://github.com/ClickHouse/ClickHouse/pull/93816) ([Raufs Dunamalijevs](https://github.com/rienath)). * Add support for `Nullable(Tuple)`. Set `allow_experimental_nullable_tuple_type = 1` to enable it. [#89643](https://github.com/ClickHouse/ClickHouse/pull/89643) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Support Paimon REST catalog, continuing from https://github.com/ClickHouse/ClickHouse/pull/84423. [#92011](https://github.com/ClickHouse/ClickHouse/pull/92011) ([JIaQi Tang](https://github.com/JiaQiTang98)). #### Performance Improvement * Setting `use_skip_indexes_on_data_read` is now enabled by default. This setting allows filtering in a streaming fashion, at the same time as reading, improving query performance and startup time. [#93407](https://github.com/ClickHouse/ClickHouse/pull/93407) ([Shankar Iyer](https://github.com/shankar-iyer)). * Improve performance of `DISTINCT` on `LowCardinality` columns. Closes [#5917](https://github.com/ClickHouse/ClickHouse/issues/5917). [#91639](https://github.com/ClickHouse/ClickHouse/pull/91639) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Optimize `distinctJSONPaths` aggregate function so it reads only JSON paths from data parts and not the whole JSON column. [#92196](https://github.com/ClickHouse/ClickHouse/pull/92196) ([Pavel Kruglov](https://github.com/Avogar)). * More filters pushed down JOINs. [#85556](https://github.com/ClickHouse/ClickHouse/pull/85556) ([Nikita Taranov](https://github.com/nickitat)). * Support more cases for push down from join ON condition when the filter uses inputs only from one side. Support `ANY`, `SEMI`, `ANTI` joins. [#92584](https://github.com/ClickHouse/ClickHouse/pull/92584) ([Dmitry Novik](https://github.com/novikd)). * Allow using equivalent sets to push down filters for `SEMI JOIN`. Closes [#85239](https://github.com/ClickHouse/ClickHouse/issues/85239). [#92837](https://github.com/ClickHouse/ClickHouse/pull/92837) ([Dmitry Novik](https://github.com/novikd)). * Skip reading left side of hash join when right side is empty. Previously we were reading left side until first non-empty block, which might do a lot of work in case when there is heavy filtering or aggregation. [#94062](https://github.com/ClickHouse/ClickHouse/pull/94062) ([Alexander Gololobov](https://github.com/davenger)). * Using the "fastrange" (Daniel Lemire) method for partitioning data inside the query pipeline. This could improve parallel sorting and JOINs. [#93080](https://github.com/ClickHouse/ClickHouse/pull/93080) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Improve performance of window functions when PARTITION BY matches or is a prefix of the sorting key. [#87299](https://github.com/ClickHouse/ClickHouse/pull/87299) ([Nikita Taranov](https://github.com/nickitat)). * Outer filter is pushed down into views which allows applying PREWHERE on local and remote nodes. Resolves [#88189](https://github.com/ClickHouse/ClickHouse/issues/88189). [#88316](https://github.com/ClickHouse/ClickHouse/pull/88316) ([Igor Nikonov](https://github.com/devcrafter)). * Implement JIT compilations for more functions. Closes [#73509](https://github.com/ClickHouse/ClickHouse/issues/73509). [#88770](https://github.com/ClickHouse/ClickHouse/pull/88770) ([Alexey Milovidov](https://github.com/alexey-milovidov) with [Taiyang Li](https://github.com/taiyang-li)). * If a skip index used in a `FINAL` query is on a column that is part of the primary key, the additional step to check for primary key intersection in other parts is unnecessary and now not performed. Resolves [#85897](https://github.com/ClickHouse/ClickHouse/issues/85897). [#93899](https://github.com/ClickHouse/ClickHouse/pull/93899) ([Shankar Iyer](https://github.com/shankar-iyer)). * Optimize performance and memory usage for fractional `LIMIT` and `OFFSET`. [#91167](https://github.com/ClickHouse/ClickHouse/pull/91167) ([Ahmed Gouda](https://github.com/0xgouda)). * Fix using of faster random read logic for Parquet Reader V3 prefetcher. Closes [#90890](https://github.com/ClickHouse/ClickHouse/issues/90890). [#91435](https://github.com/ClickHouse/ClickHouse/pull/91435) ([Arsen Muk](https://github.com/arsenmuk)). * Improve performance of `icebergCluster`. Closes [#91462](https://github.com/ClickHouse/ClickHouse/issues/91462). [#91537](https://github.com/ClickHouse/ClickHouse/pull/91537) ([Yang Jiang](https://github.com/Ted-Jiang)). * Don't filter by virtual columns on constant filters. [#91588](https://github.com/ClickHouse/ClickHouse/pull/91588) ([c-end](https://github.com/c-end)). * Reduce INSERT/merges memory usage with wide parts for very wide tables by enabling adaptive write buffers. Add support of adaptive write buffers for encrypted disks. [#92250](https://github.com/ClickHouse/ClickHouse/pull/92250) ([Azat Khuzhin](https://github.com/azat)). * Improved performance of full text search with text index and `sparseGrams` tokenizer by reducing the number of searched tokens in the index. [#93078](https://github.com/ClickHouse/ClickHouse/pull/93078) ([Anton Popov](https://github.com/CurtizJ)). * Function `isValidASCII` was optimized for positive outcomes, i.e. all-ASCII input values. [#93347](https://github.com/ClickHouse/ClickHouse/pull/93347) ([Robert Schulze](https://github.com/rschu1ze)). * The read-in-order optimization now recognizes when ORDER BY columns are constant due to WHERE conditions, enabling efficient reverse-order reads. This benefits multi-tenant queries like `WHERE tenant='42' ORDER BY tenant, event_time DESC` which can now use InReverseOrder instead of requiring a full sort.". [#94103](https://github.com/ClickHouse/ClickHouse/pull/94103) ([matanper](https://github.com/matanper)). * Introduce Enum AST specialized class to store value parameters in (string, integer) pairs instead of ASTLiteral children to optimize memory consumption. [#94178](https://github.com/ClickHouse/ClickHouse/pull/94178) ([Ilya Yatsishin](https://github.com/qoega)). * Distributed index analysis on multiple replicas. Beneficial for shared storage and huge amount of data in cluster. This is applicable for SharedMergeTree (ClickHouse Cloud) and could be applicable for other types of MergeTree tables on a shared storage. [#86786](https://github.com/ClickHouse/ClickHouse/pull/86786) ([Azat Khuzhin](https://github.com/azat)). * Reduce overhead of join runtime filters by disabling them in the following cases: - too many bits are set in the bloom filter - too few rows are filtered out at runtime. [#91578](https://github.com/ClickHouse/ClickHouse/pull/91578) ([Alexander Gololobov](https://github.com/davenger)). * Use an in-memory buffer for correlated subqueries input to avoid evaluating it multiple times. Part of [#79890](https://github.com/ClickHouse/ClickHouse/issues/79890). [#91205](https://github.com/ClickHouse/ClickHouse/pull/91205) ([Dmitry Novik](https://github.com/novikd)). * Allow all replicas to steal orphaned ranges in parallel replicas reading. This improves load balancing and reduces long-tail latency. [#91374](https://github.com/ClickHouse/ClickHouse/pull/91374) ([zoomxi](https://github.com/zoomxi)). * External aggregation/sorting/join now respects query setting `temporary_files_codec` in all contexts. Fixed missing profile events for grace hash join. [#92388](https://github.com/ClickHouse/ClickHouse/pull/92388) ([Vladimir Cherkasov](https://github.com/vdimir)). * Make query memory usage detection for spilling to disk during aggregation/sorting more robust. [#92500](https://github.com/ClickHouse/ClickHouse/pull/92500) ([Azat Khuzhin](https://github.com/azat)). * Estimate total rows count and NDV (number of distinct values) statistics of aggregation key columns. [#92812](https://github.com/ClickHouse/ClickHouse/pull/92812) ([Alexander Gololobov](https://github.com/davenger)). * Optimize postings list compression with simdcomp. [#92871](https://github.com/ClickHouse/ClickHouse/pull/92871) ([Peng Jian](https://github.com/fastio)). * Refactor S3Queue Ordered mode processing with buckets. This should also improve performance, reducing the number of keeper requests. [#92889](https://github.com/ClickHouse/ClickHouse/pull/92889) ([Kseniia Sumarokova](https://github.com/kssenii)). * Functions `mapContainsKeyLike` and `mapContainsValueLike` can now leverage a text index on `mapKeys()` or `mapValues()`, respectively. [#93049](https://github.com/ClickHouse/ClickHouse/pull/93049) ([Michael Jarrett](https://github.com/EmeraldShift)). * Reduce memory usage on non-Linux systems (enable immediate purging of jemalloc dirty pages). [#93360](https://github.com/ClickHouse/ClickHouse/pull/93360) ([Eduard Karacharov](https://github.com/korowa)). * Force purging of jemalloc arenas in case the ratio of dirty pages size to `max_server_memory_usage` exceeds `memory_worker_purge_dirty_pages_threshold_ratio`. [#93500](https://github.com/ClickHouse/ClickHouse/pull/93500) ([Eduard Karacharov](https://github.com/korowa)). * Reduce memory usage for AST. [#93601](https://github.com/ClickHouse/ClickHouse/pull/93601) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * In some cases we've seen ClickHouse doesn't respect a memory limit when reading from a table. This behaviour is fixed. [#93715](https://github.com/ClickHouse/ClickHouse/pull/93715) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Enable `CHECK_STAT` and `TRY_REMOVE` Keeper extension by default. [#93886](https://github.com/ClickHouse/ClickHouse/pull/93886) ([Mikhail Artemenko](https://github.com/Michicosun)). * Parse lower and upper bounds of file names corresponding to position deletes from Iceberg manifest file entries for better selection of corresponding data files. [#93980](https://github.com/ClickHouse/ClickHouse/pull/93980) ([Daniil Ivanik](https://github.com/divanik)). * Add two more settings to control maximum number of dynamic subcolumns in JSON column. First is MergeTree setting `merge_max_dynamic_subcolumns_in_compact_part` (similar to already added `merge_max_dynamic_subcolumns_in_wide_part`) that limits number of dynamic subcolumns created during merge into a Compact part. Second is query level setting `max_dynamic_subcolumns_in_json_type_parsing` that limits number of dynamic subcolumns created during parsing of JSON data, it will allow to specify the limit on insert. [#94184](https://github.com/ClickHouse/ClickHouse/pull/94184) ([Pavel Kruglov](https://github.com/Avogar)). * Slightly optimize squashing of JSON columns for some cases. [#94247](https://github.com/ClickHouse/ClickHouse/pull/94247) ([Pavel Kruglov](https://github.com/Avogar)). * Lower the thread pool queue sizes based on the production experience. Add an explicit memory consumption check before reading any data from the MergeTree. [#94692](https://github.com/ClickHouse/ClickHouse/pull/94692) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Make sure the scheduler would prefer MemoryWorker thread under the CPU starvation, because it protects ClickHouse process from an existential threat. [#94864](https://github.com/ClickHouse/ClickHouse/pull/94864) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). * Run purging of jemalloc dirty pages in a different thread from main thread of MemoryWorker. If purging is slow, it could delay updates of RSS usage which could lead to out of memory kills of the process. Introduce new config `memory_worker_purge_total_memory_threshold_ratio` to start purging dirty pages based on ratio of total memory usage. [#94902](https://github.com/ClickHouse/ClickHouse/pull/94902) ([Antonio Andelic](https://github.com/antonio2368)). #### Improvement * `system.blob_storage_log` is now available for Azure Blob Storage. [#93105](https://github.com/ClickHouse/ClickHouse/pull/93105) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Implement `blob_storage_log` for Local and HDFS. Fix an error when `S3Queue` used something other than the disk name for logging in `blob_storage_log`. Add `error_code` column to `blob_storage_log`. Split the test configuration file to simplify local testing. [#93106](https://github.com/ClickHouse/ClickHouse/pull/93106) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * `clickhouse-client` and `clickhouse-local` will highlight digit groups (thousands, millions, etc.) inside numeric literals while typing. This closes [#93100](https://github.com/ClickHouse/ClickHouse/issues/93100). [#93108](https://github.com/ClickHouse/ClickHouse/pull/93108) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Adds support in `clickhouse-client` for command-line arguments with a space surrounding the equals sign. Closes [#93077](https://github.com/ClickHouse/ClickHouse/issues/93077). [#93174](https://github.com/ClickHouse/ClickHouse/pull/93174) ([Cole Smith](https://github.com/colesmith54)). * With `true`, the CLI client can now fall back to Ctrl-R for regular search like before, while Ctrl-T does fuzzy search. [#87785](https://github.com/ClickHouse/ClickHouse/pull/87785) ([Larry Snizek](https://github.com/larry-cdn77)). * The statement to clear caches `SYSTEM DROP [...] CACHE` gave the false impression that the statement disables the cache. ClickHouse now supports statement `SYSTEM CLEAR [...] CACHE` which is more obvious. The old syntax remains available. [#93727](https://github.com/ClickHouse/ClickHouse/pull/93727) ([Pranav Tiwari](https://github.com/pranavt84)). * Support multiple columns as primary key in `EmbeddedRocksDB`. Closes [#32819](https://github.com/ClickHouse/ClickHouse/issues/32819). [#33917](https://github.com/ClickHouse/ClickHouse/pull/33917) ([usurai](https://github.com/usurai)). * It is now possible to use non-constant IN for scalars (queries like `val1 NOT IN if(cond, val2, val3)`). [#93495](https://github.com/ClickHouse/ClickHouse/pull/93495) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Prevent `x-amz-server-side-encryption` headers from being propagated to `HeadObject`, `UploadPart` & `CompleteMultipartUpload` S3 requests as they're not supported. [#64577](https://github.com/ClickHouse/ClickHouse/pull/64577) ([Francisco J. Jurado Moreno](https://github.com/Beetelbrox)). * Tracking hive partitioning for ordered mode in S3Queue. Resolves [#71161](https://github.com/ClickHouse/ClickHouse/issues/71161). [#81040](https://github.com/ClickHouse/ClickHouse/pull/81040) ([Anton Ivashkin](https://github.com/ianton-ru)). * Optimize space reservation in filesystem cache. `FileCache::collectCandidatesForEviction` will be executed without unique lock. [#82764](https://github.com/ClickHouse/ClickHouse/pull/82764) ([Kseniia Sumarokova](https://github.com/kssenii)). * Support composite rotation strategy (size + time) for server log. [#87620](https://github.com/ClickHouse/ClickHouse/pull/87620) ([Jianmei Zhang](https://github.com/zhangjmruc)). * CLI client can now specify `false` instead of the command line `--no-warnings`. [#87783](https://github.com/ClickHouse/ClickHouse/pull/87783) ([Larry Snizek](https://github.com/larry-cdn77)). * Add support for the `avg` aggregate function with Date, DateTime and Time values as arguments. Closes [#82267](https://github.com/ClickHouse/ClickHouse/issues/82267). [#87845](https://github.com/ClickHouse/ClickHouse/pull/87845) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * The optimization `use_join_disjunctions_push_down` is enabled by default. [#89313](https://github.com/ClickHouse/ClickHouse/pull/89313) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support more table engines and data source kinds in the correlated subqueries. Closes [#80775](https://github.com/ClickHouse/ClickHouse/issues/80775). [#90175](https://github.com/ClickHouse/ClickHouse/pull/90175) ([Dmitry Novik](https://github.com/novikd)). * If the schema of parameterized view is specified explicitly, it is shown. Close [#88875](https://github.com/ClickHouse/ClickHouse/issues/88875), [#81385](https://github.com/ClickHouse/ClickHouse/issues/81385). [#90220](https://github.com/ClickHouse/ClickHouse/pull/90220) ([Grigorii Sokolik](https://github.com/GSokol)). * Correctly handle the gap in Keeper log entries if logs are before the last committed index. [#90403](https://github.com/ClickHouse/ClickHouse/pull/90403) ([Antonio Andelic](https://github.com/antonio2368)). * Improve `min_free_disk_bytes_to_perform_insert` setting to work correctly with JBOD volumes. [#90878](https://github.com/ClickHouse/ClickHouse/pull/90878) ([Aleksandr Musorin](https://github.com/AVMusorin)). * Make it possible to specify `storage_class_name` setting in named collections for `S3` table engine and `s3` table function. [#91926](https://github.com/ClickHouse/ClickHouse/pull/91926) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Support inserting auxiliary zookeeper by `system.zookeeper`. [#92092](https://github.com/ClickHouse/ClickHouse/pull/92092) ([RinChanNOW](https://github.com/RinChanNOWWW)). * Add new metrics for the keeper: `KeeperChangelogWrittenBytes`, `KeeperChangelogFileSyncMicroseconds`, `KeeperSnapshotWrittenBytes` and `KeeperSnapshotFileSyncMicroseconds` profile events as well as `KeeperBatchSizeElements` and `KeeperBatchSizeBytes` histogram metrics. [#92149](https://github.com/ClickHouse/ClickHouse/pull/92149) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Add a new setting, `trace_profile_events_list`, which limits tracing with `trace_profile_event` to the specified list of event names. This allows more precise data collection on large workloads. [#92298](https://github.com/ClickHouse/ClickHouse/pull/92298) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Support SYSTEM NOTIFY FAILPOINT for pausable failpoints. - Support SYSTEM WAIT FAILPOINT fp PAUSE/RESUME. [#92368](https://github.com/ClickHouse/ClickHouse/pull/92368) ([Shaohua Wang](https://github.com/tiandiwonder)). * Add `creation` (implicit/explicit) column to `system.data_skipping_indices`. [#92378](https://github.com/ClickHouse/ClickHouse/pull/92378) ([Raúl Marín](https://github.com/Algunenano)). * Allow passing the description of columns for YTsaurus dyn tables to the dictionary source. [#92391](https://github.com/ClickHouse/ClickHouse/pull/92391) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * In [#63985](https://github.com/ClickHouse/ClickHouse/pull/63985), we made it possible to specify all the parameters needed for TLS configuration on a per-port basis (see [composable protocols](https://clickhouse.com/docs/operations/settings/composable-protocols)), so we don't have to rely on global TLS config. However, the implementation still implicitly requires a global `openSSL.server` config section to exist, which conflicts with setups where different TLS configurations are needed for different ports. For example, in keeper-in-server deployments, we need separate TLS configs for inter-keeper communication and clickhouse client connections. [#92457](https://github.com/ClickHouse/ClickHouse/pull/92457) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Introduce a new setting `input_format_binary_max_type_complexity` that limits the total number of type nodes that can be decoded in binary format to prevent malicious payloads. [#92519](https://github.com/ClickHouse/ClickHouse/pull/92519) ([Raufs Dunamalijevs](https://github.com/rienath)). * Reflect running tasks in `system.background_schedule_pool{,_log}`. Add documentation. [#92587](https://github.com/ClickHouse/ClickHouse/pull/92587) ([Azat Khuzhin](https://github.com/azat)). * Execute current query in Ctrl+R search in client if no history match found. [#92749](https://github.com/ClickHouse/ClickHouse/pull/92749) ([Azat Khuzhin](https://github.com/azat)). * Support `EXPLAIN indices = 1` as an alias for `EXPLAIN indexes = 1`. Closes [#92483](https://github.com/ClickHouse/ClickHouse/issues/92483). [#92774](https://github.com/ClickHouse/ClickHouse/pull/92774) ([Pranav Tiwari](https://github.com/pranavt84)). * Parquet reader now allows reading Tuple or Map columns as JSON: `select x from file(f.parquet, auto, 'x JSON')` works even if the type of column `x` in `f.parquet` is tuple or map. [#92864](https://github.com/ClickHouse/ClickHouse/pull/92864) ([Michael Kolupaev](https://github.com/al13n321)). * Support empty tuples in parquet reader. [#92868](https://github.com/ClickHouse/ClickHouse/pull/92868) ([Michael Kolupaev](https://github.com/al13n321)). * Fallback to read-write copy for Azure Blob Storage when native copy fails with BadRequest (e.g. invalid block list). Previously this was only done for Unauthorized error which was seen while copying blob to different storage accounts. But we also sometimes see "The specified block list is invalid" error. So now updated the condition to fallback to read & write for all native copy fails. [#92888](https://github.com/ClickHouse/ClickHouse/pull/92888) ([Smita Kulkarni](https://github.com/SmitaRKulkarni)). * Fix EC2 metadata endpoint throttling when running many concurrent S3 queries with EC2 instance profile credentials. Previously, each query created its own `AWSInstanceProfileCredentialsProvider`, causing concurrent requests to the EC2 metadata service which could result in timeouts and `HTTP response code: 403` errors. Now the credentials provider is cached and shared across all queries. [#92891](https://github.com/ClickHouse/ClickHouse/pull/92891) ([Sav](https://github.com/sberss)). * Rework `insert_select_deduplicate` setting to add an ability to keep backward compatibility. [#92951](https://github.com/ClickHouse/ClickHouse/pull/92951) ([Sema Checherinda](https://github.com/CheSema)). * Log background tasks that are slower than average (`background_schedule_pool_log.duration_threshold_milliseconds=30`) to avoid excessive tasks logging. [#92965](https://github.com/ClickHouse/ClickHouse/pull/92965) ([Azat Khuzhin](https://github.com/azat)). * In previous versions, some of C++ function names were displayed incorrectly ("mangled") in the `system.trace_log` and `system.symbols`, and the `demangle` function didn't process them well. Closes [#93074](https://github.com/ClickHouse/ClickHouse/issues/93074). [#93075](https://github.com/ClickHouse/ClickHouse/pull/93075) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Introduced the `backup_data_from_refreshable_materialized_view_targets` backup setting to skip back up of refreshable materialized views. RMVs with APPEND refresh strategy which are always backed up. [#93076](https://github.com/ClickHouse/ClickHouse/pull/93076) ([Julia Kartseva](https://github.com/jkartseva)). [#93658](https://github.com/ClickHouse/ClickHouse/pull/93658) ([Julia Kartseva](https://github.com/jkartseva)) * Use minimal debug info instead of no debug info for heavy translation units, such as functions. [#93079](https://github.com/ClickHouse/ClickHouse/pull/93079) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Added MinIO compatibility support to AWS S3 C++ SDK by implementing error code mapping for MinIO-specific errors. This change allows ClickHouse to properly handle and retry MinIO server errors when using MinIO deployments instead of AWS S3, improving reliability for users running object storage on self-hosted MinIO clusters. [#93082](https://github.com/ClickHouse/ClickHouse/pull/93082) ([XiaoBinMu](https://github.com/Binnn-MX)). * Write symbolized jemalloc profiles (eliminating the need for a binary during heap profile generation). [#93099](https://github.com/ClickHouse/ClickHouse/pull/93099) ([Azat Khuzhin](https://github.com/azat)). * Resurrect `clickhouse git-import` tool - it was broken on large and invalid commits. See https://presentations.clickhouse.com/2020-matemarketing/. [#93202](https://github.com/ClickHouse/ClickHouse/pull/93202) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Don't show passwords from URL storage in query log. [#93245](https://github.com/ClickHouse/ClickHouse/pull/93245) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Support `Geometry` type for `flipCoordinates`. [#93303](https://github.com/ClickHouse/ClickHouse/pull/93303) ([Bharat Nallan](https://github.com/bharatnc)). * Improve the UX of SYSTEM INSTRUMENT ADD/REMOVE: use String literals for function names, patch all functions that match and allow using function_name in `REMOVE`. [#93345](https://github.com/ClickHouse/ClickHouse/pull/93345) ([Pablo Marcos](https://github.com/pamarcos)). * Add a new setting `materialize_statistics_on_merge` which enables/disables materializing statistics during merge. The default value is `1`. [#93379](https://github.com/ClickHouse/ClickHouse/pull/93379) ([Han Fei](https://github.com/hanfei1991)). * ClickHouse can now parse `SELECT` without parentheses around `DESCRIBE SELECT` queries. Closes [#58382](https://github.com/ClickHouse/ClickHouse/issues/58382). [#93429](https://github.com/ClickHouse/ClickHouse/pull/93429) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Add randomization of cache correctness checks under probability. [#93439](https://github.com/ClickHouse/ClickHouse/pull/93439) ([Kseniia Sumarokova](https://github.com/kssenii)). * Add setting `type_json_allow_duplicated_key_with_literal_and_nested_object` to allow duplicated paths in JSON where one is a literal and another is a nested object, e.g. `{"a" : 42, "a" : {"b" : 42}}`. Some data could be created before this restriction on duplicated paths was added in https://github.com/ClickHouse/ClickHouse/pull/79317 and further manipulation with this data can lead to errors now. With this setting, such old data cane still be used with no errors. [#93604](https://github.com/ClickHouse/ClickHouse/pull/93604) ([Pavel Kruglov](https://github.com/Avogar)). * Don't print values of simple types on separate lines in Pretty JSON. [#93836](https://github.com/ClickHouse/ClickHouse/pull/93836) ([Pavel Kruglov](https://github.com/Avogar)). * When there are many `alter table ... modify setting ...` statements, it's possible not to acquire lock for 5 seconds. Better to return `timeout` than `logical error`. [#93856](https://github.com/ClickHouse/ClickHouse/pull/93856) ([Han Fei](https://github.com/hanfei1991)). * Prevent excessive output on a syntax error. Before this change, it output the whole SQL script, which could contain a lot of queries. [#93876](https://github.com/ClickHouse/ClickHouse/pull/93876) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Do proper byte size calculation of the `check` request with stats in Keeper. [#93907](https://github.com/ClickHouse/ClickHouse/pull/93907) ([Mikhail Artemenko](https://github.com/Michicosun)). * Added `use_hash_table_stats_for_join_reordering` setting to control whether runtime hash table size statistics are used for join reordering. This setting is enabled by default, preserving the existing behavior of `collect_hash_table_stats_during_joins`. [#93912](https://github.com/ClickHouse/ClickHouse/pull/93912) ([Vladimir Cherkasov](https://github.com/vdimir)). * Users can now partially view nested global server settings in the `system.server_settings` table (e.g. `logger.level`). This only covers settings with a fixed structure (no lists, enumerations, repetitions etc.). [#94001](https://github.com/ClickHouse/ClickHouse/pull/94001) ([Hechem Selmi](https://github.com/m-selmi)). * `QBit` can now be compared for equality. [#94078](https://github.com/ClickHouse/ClickHouse/pull/94078) ([Raufs Dunamalijevs](https://github.com/rienath)). * When Keeper detects broken snapshot or inconsistent changelogs, throw exception instead of manually aborting or cleaning up files automatically. This should lead to a safer behaviour of Keeper relying on manual intervention. [#94168](https://github.com/ClickHouse/ClickHouse/pull/94168) ([Antonio Andelic](https://github.com/antonio2368)). * Fix leaving possible leftovers in case of `CREATE TABLE` fails. [#94174](https://github.com/ClickHouse/ClickHouse/pull/94174) ([Azat Khuzhin](https://github.com/azat)). * Fix uninitialized memory access (a bug in OpenSSL) when password protected TLS key is used. [#94182](https://github.com/ClickHouse/ClickHouse/pull/94182) ([Konstantin Bogdanov](https://github.com/thevar1able)). * Bump chdig to [v26.1.1](https://github.com/azat/chdig/releases/tag/v26.1.1). [#94290](https://github.com/ClickHouse/ClickHouse/pull/94290) ([Azat Khuzhin](https://github.com/azat)). * Support more generic partitioning for S3Queue ordered mode. [#94321](https://github.com/ClickHouse/ClickHouse/pull/94321) ([Bharat Nallan](https://github.com/bharatnc)). * Added alias `use_statistics` for setting `allow_statistics_optimize`. This is more consistent with existing settings `use_primary_key` and `use_skip_indexes`. [#94366](https://github.com/ClickHouse/ClickHouse/pull/94366) ([Robert Schulze](https://github.com/rschu1ze)). * Enabled setting `input_format_numbers_enum_on_conversion_error` for conversion from Numbers to Enums to check whether the element exists. [#94384](https://github.com/ClickHouse/ClickHouse/pull/94384) ([Elmi Ahmadov](https://github.com/ahmadov)). * In S3(Azure)Queue ordered mode clean up failed nodes by tracking limits (before that was done only in Unordered mode for both failed and processed, so now this will also be done for Ordered but only for failed nodes). [#94412](https://github.com/ClickHouse/ClickHouse/pull/94412) ([Kseniia Sumarokova](https://github.com/kssenii)). * Enable access management for `default` user in clickhouse-local. The default user in `clickhouse-local` was missing the access_management privilege, which caused operations like `DROP ROW POLICY IF EXISTS` to fail with `ACCESS_DENIED` error, even though the user should be unrestricted. [#94501](https://github.com/ClickHouse/ClickHouse/pull/94501) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Enable named collection for YTsaurus dictionaries and tables. [#94582](https://github.com/ClickHouse/ClickHouse/pull/94582) ([MikhailBurdukov](https://github.com/MikhailBurdukov)). * Add support for SQL-defined named collections in BACKUP/RESTORE for S3 and Azure Blob Storage. Closes [#94604](https://github.com/ClickHouse/ClickHouse/issues/94604). [#94605](https://github.com/ClickHouse/ClickHouse/pull/94605) ([Pablo Marcos](https://github.com/pamarcos)). * Support bucketing based on partition key for S3Queue in ordered mode. [#94698](https://github.com/ClickHouse/ClickHouse/pull/94698) ([Bharat Nallan](https://github.com/bharatnc)). * Add an asynchronous metric with the longest running merge elapsed time. [#94825](https://github.com/ClickHouse/ClickHouse/pull/94825) ([Raúl Marín](https://github.com/Algunenano)). * Add belonging file check before apply position delete using IcebergBitmapPositionDeleteTransform. [#94897](https://github.com/ClickHouse/ClickHouse/pull/94897) ([Yang Jiang](https://github.com/Ted-Jiang)). * Now `view_duration_ms` shows the time when group was active, not the sum of the threads duration in it. [#94966](https://github.com/ClickHouse/ClickHouse/pull/94966) ([Sema Checherinda](https://github.com/CheSema)). * Remove limit of the max number of search tokens in `hasAnyTokens` and `hasAllTokens` functions which was limited to 64. Example: `SELECT count() FROM table WHERE hasAllTokens(text, ['token_1', 'token_2', [...], 'token_65']]);` The query would result in a `BAD_ARGUMENTS` error because there are 65 search tokens. With this PR, the limit has been removed completely and the same query would run without an error. [#95152](https://github.com/ClickHouse/ClickHouse/pull/95152) ([Elmi Ahmadov](https://github.com/ahmadov)). * Add a setting `input_format_numbers_enum_on_conversion_error` for conversion from Numbers to Enums to check whether the element exists. Closes: [#56144](https://github.com/ClickHouse/ClickHouse/issues/56144). [#56240](https://github.com/ClickHouse/ClickHouse/pull/56240) ([Nikolay Degterinsky](https://github.com/evillique)). * Share format parser resources between data file and position delete file reading in Iceberg tables to reduce memory allocations. [#94701](https://github.com/ClickHouse/ClickHouse/pull/94701) ([Yang Jiang](https://github.com/Ted-Jiang)). #### Bug Fix (user-visible misbehavior in an official stable release) * Fixes a bug where predefined query handlers would have trailing whitespace interpreted as data during inserts. [#83604](https://github.com/ClickHouse/ClickHouse/pull/83604) ([Fabian Ponce](https://github.com/FabianPonce)). * Fix INCOMPATIBLE_TYPE_OF_JOIN error for Join storage and outer to inner join optimization applied. Resolves [#80794](https://github.com/ClickHouse/ClickHouse/issues/80794). [#84292](https://github.com/ClickHouse/ClickHouse/pull/84292) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix exception "Invalid number of rows in Chunk" when using hash join with `allow_experimental_join_right_table_sorting` enabled. [#86440](https://github.com/ClickHouse/ClickHouse/pull/86440) ([yanglongwei](https://github.com/ylw510)). * Always replace file names to hash in MergeTree if filesystem is case insensitive. Previously on systems with case insensitive filesystem (like MacOS) it could lead to data corruption when several column/subcolumn names differs only in the case. [#86559](https://github.com/ClickHouse/ClickHouse/pull/86559) ([Pavel Kruglov](https://github.com/Avogar)). * Add a full permissions check on the create stage for the underlying query inside a materialized view. [#89180](https://github.com/ClickHouse/ClickHouse/pull/89180) ([pufit](https://github.com/pufit)). * Fixed crash in `icebergHash` function on constant argument. [#90335](https://github.com/ClickHouse/ClickHouse/pull/90335) ([Michael Kolupaev](https://github.com/al13n321)). * Fix logical error when mutation without transaction mutates parts in an active transaction, which is rolled back finally. [#90469](https://github.com/ClickHouse/ClickHouse/pull/90469) ([Shaohua Wang](https://github.com/tiandiwonder)). * Update `system.warnings` correctly after an ordinary database was converted to an atomic database. [#90473](https://github.com/ClickHouse/ClickHouse/pull/90473) ([sdk2](https://github.com/sdk2)). * Fixes an assertion when reading from Parquet file, and part of a **prewhere** expression is used elsewhere in the query. [#90635](https://github.com/ClickHouse/ClickHouse/pull/90635) ([Max Kainov](https://github.com/maxknv)). * Fix crash in a single-node cluster when reading from Iceberg in split-by-buckets mode. This closes [#90913](https://github.com/ClickHouse/ClickHouse/issues/90913#issue-3668583963). [#91553](https://github.com/ClickHouse/ClickHouse/pull/91553) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix possible logical error in Log engine during subcolumns reading. Closes [#91710](https://github.com/ClickHouse/ClickHouse/issues/91710). [#91711](https://github.com/ClickHouse/ClickHouse/pull/91711) ([Pavel Kruglov](https://github.com/Avogar)). * Fix Logical error: 'Storage does not support transaction' during ATTACH AS REPLICATED. [#91772](https://github.com/ClickHouse/ClickHouse/pull/91772) ([Shaohua Wang](https://github.com/tiandiwonder)). * Fix for runtime filters working incorrectly when LEFT ANTI JOIN has extra post-condition. [#91824](https://github.com/ClickHouse/ClickHouse/pull/91824) ([Alexander Gololobov](https://github.com/davenger)). * Fixes an error where we have a null-safe comparison involving the Nothing type. Closes [#91834](https://github.com/ClickHouse/ClickHouse/issues/91834). Closes [#84870](https://github.com/ClickHouse/ClickHouse/issues/84870). Closes [#91821](https://github.com/ClickHouse/ClickHouse/issues/91821). [#91884](https://github.com/ClickHouse/ClickHouse/pull/91884) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix DELTA_BYTE_ARRAY decoding bugs in native Parquet reader affecting highly repetitive string data. [#91929](https://github.com/ClickHouse/ClickHouse/pull/91929) ([Daniel Muino](https://github.com/dmuino)). * Cache schema only for the file it was inferred from in globs instead of all files during schema inference. Closes [#91745](https://github.com/ClickHouse/ClickHouse/issues/91745). [#92006](https://github.com/ClickHouse/ClickHouse/pull/92006) ([Pavel Kruglov](https://github.com/Avogar)). * Fix the `Couldn't pack tar archive: Failed to write all bytes` error caused by an incorrect archive entry size header. Fixes [#89075](https://github.com/ClickHouse/ClickHouse/issues/89075). [#92122](https://github.com/ClickHouse/ClickHouse/pull/92122) ([Julia Kartseva](https://github.com/jkartseva)). * Release request stream in insert select to prevent closing http connection. [#92175](https://github.com/ClickHouse/ClickHouse/pull/92175) ([Sema Checherinda](https://github.com/CheSema)). * Fix logical error for queries with multiple JOINs with `USING` clause and `join_use_nulls`. [#92251](https://github.com/ClickHouse/ClickHouse/pull/92251) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix logical error while join reordering with join_use_nulls, close https://github.com/clickhouse/clickhouse/issues/90795. [#92289](https://github.com/ClickHouse/ClickHouse/pull/92289) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix inconsistent AST formatting of arrayElement with negate literal. Closes [#92288](https://github.com/ClickHouse/ClickHouse/issues/92288) Closes [#92212](https://github.com/ClickHouse/ClickHouse/issues/92212) Closes [#91832](https://github.com/ClickHouse/ClickHouse/issues/91832) Closes [#91789](https://github.com/ClickHouse/ClickHouse/issues/91789) Closes [#91735](https://github.com/ClickHouse/ClickHouse/issues/91735) Closes [#88495](https://github.com/ClickHouse/ClickHouse/issues/88495) Closes [#92386](https://github.com/ClickHouse/ClickHouse/issues/92386). [#92293](https://github.com/ClickHouse/ClickHouse/pull/92293) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a possible crash with `join_on_disk_max_files_to_merge` setting. [#92335](https://github.com/ClickHouse/ClickHouse/pull/92335) ([Bharat Nallan](https://github.com/bharatnc)). * Related issue #https://github.com/ClickHouse/support-escalation/issues/6365. [#92339](https://github.com/ClickHouse/ClickHouse/pull/92339) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix missing access check in `SYSTEM SYNC FILE CACHE`. Closes [#92101](https://github.com/ClickHouse/ClickHouse/issues/92101). [#92372](https://github.com/ClickHouse/ClickHouse/pull/92372) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix `count_distinct_optimization` pass over window functions and over multiple arguments. [#92376](https://github.com/ClickHouse/ClickHouse/pull/92376) ([Raúl Marín](https://github.com/Algunenano)). * Fix "Cannot write to finalized buffer" error when using certain aggregate functions with window functions. Closes [#91415](https://github.com/ClickHouse/ClickHouse/issues/91415). [#92395](https://github.com/ClickHouse/ClickHouse/pull/92395) ([Jimmy Aguilar Mena](https://github.com/Ergus)). * Fix logical error with `CREATE TABLE ... AS urlCluster()` and database engine `Replicated`. Closes [#92216](https://github.com/ClickHouse/ClickHouse/issues/92216). [#92418](https://github.com/ClickHouse/ClickHouse/pull/92418) ([Kseniia Sumarokova](https://github.com/kssenii)). * Inherit source part serialization info settings during mutation in MergeTree. It fixes possible incorrect result of the query over mutated part after changes in data types serialization. [#92419](https://github.com/ClickHouse/ClickHouse/pull/92419) ([Pavel Kruglov](https://github.com/Avogar)). * Fix possible conflict in column and subcolumn with the same name leading in using wrong serialization and query failures. Closes [#90219](https://github.com/ClickHouse/ClickHouse/issues/90219). Closes [#85161](https://github.com/ClickHouse/ClickHouse/issues/85161). [#92453](https://github.com/ClickHouse/ClickHouse/pull/92453) ([Pavel Kruglov](https://github.com/Avogar)). * Fix a `LOGICAL_ERROR`s that caused by not wanted modification of query plan when converting outer join to inner join. Also relax the requirements of optimization to be able to apply it in cases when injective functions are applied to the aggregating keys during joins. [#92503](https://github.com/ClickHouse/ClickHouse/pull/92503) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fix possible error `SIZES_OF_COLUMNS_DOESNT_MATCH` during sorting of emty tuple column. Closes [#92422](https://github.com/ClickHouse/ClickHouse/issues/92422). [#92520](https://github.com/ClickHouse/ClickHouse/pull/92520) ([Pavel Kruglov](https://github.com/Avogar)). * Check for incompatible typed paths in JSON type. Closes [#91577](https://github.com/ClickHouse/ClickHouse/issues/91577). [#92539](https://github.com/ClickHouse/ClickHouse/pull/92539) ([Pavel Kruglov](https://github.com/Avogar)). * Fix deadlock for SHOW CREATE DATABASE for Backup database. [#92541](https://github.com/ClickHouse/ClickHouse/pull/92541) ([Azat Khuzhin](https://github.com/azat)). * Use proper error code when validating hypothesis index. [#92559](https://github.com/ClickHouse/ClickHouse/pull/92559) ([Raúl Marín](https://github.com/Algunenano)). * Fix dynamic subcolumns resolution in column aliases in analyzer. Previously dynamic subcolumn in column alias was wrapped in `getSubcolumn` and in some cases could be not resolved at all. Closes [#91434](https://github.com/ClickHouse/ClickHouse/issues/91434). [#92583](https://github.com/ClickHouse/ClickHouse/pull/92583) ([Pavel Kruglov](https://github.com/Avogar)). * Prevent crash in tokens() with null second argument. [#92586](https://github.com/ClickHouse/ClickHouse/pull/92586) ([Raúl Marín](https://github.com/Algunenano)). * Fix potential crash caused by in place mutation of underlying const PREWHERE columns. This could've happened at column shrinking (`IColumn::shrinkToFit`) or filtering (`IColumn::filter`), which could've triggered concurrently from several threads. [#92588](https://github.com/ClickHouse/ClickHouse/pull/92588) ([Arsen Muk](https://github.com/arsenmuk)). * Creating and materializing text indexes on tables containing large parts (over 4,294,967,295 rows) is temporarily disabled. This limitation prevents incorrect query results, as the current index implementation does not yet support such large parts. [#92644](https://github.com/ClickHouse/ClickHouse/pull/92644) ([Anton Popov](https://github.com/CurtizJ)). * Fixes a logical error `Too large size (A) passed to allocator` while executing JOINs. Closes [#92043](https://github.com/ClickHouse/ClickHouse/issues/92043). [#92667](https://github.com/ClickHouse/ClickHouse/pull/92667) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Remove a bug that `ngrambf_v1` indexes with ngram length (1st parameter) > 8 would throw an exception. [#92672](https://github.com/ClickHouse/ClickHouse/pull/92672) ([Robert Schulze](https://github.com/rschu1ze)). * Fix uncaught exception during background named collections reload when zookeeper storage is used. Closes https://github.com/ClickHouse/clickhouse-private/issues/44180. [#92717](https://github.com/ClickHouse/ClickHouse/pull/92717) ([Kseniia Sumarokova](https://github.com/kssenii)). * Reworks incorrect logic in access grant checks for wildcard grants. The previous attempt https://github.com/ClickHouse/ClickHouse/pull/90928 addressed a critical vulnerability but ended up being too restrictive, resulting in some wildcard `GRANT` statements failing due to unrelated revokes. [#92725](https://github.com/ClickHouse/ClickHouse/pull/92725) ([pufit](https://github.com/pufit)). * Fix bug in data skipping logic when `not match(...)` is used in `WHERE` causing incorrect results. Closes [#92492](https://github.com/ClickHouse/ClickHouse/issues/92492). [#92726](https://github.com/ClickHouse/ClickHouse/pull/92726) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Do not attempt to delete temporary directories at startup if a MergeTree table is created over a read-only disk. [#92748](https://github.com/ClickHouse/ClickHouse/pull/92748) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix "Cannot add action to empty ExpressionActionsChain" for ALTER TABLE REWRITE PARTS (v2). [#92754](https://github.com/ClickHouse/ClickHouse/pull/92754) ([Azat Khuzhin](https://github.com/azat)). * Avoid crash due to reading from a disconnected `Connection`. [#92807](https://github.com/ClickHouse/ClickHouse/pull/92807) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix logical error ` Failed to set file processing within 100 retries` in storgae `S3Queue` in `Ordered` mode. It is now replaced with a warning. This error could happen before 25.10 version if keeper session expired, however it will still be a warning in 25.10+ versions, as it is still theoretically possible to get this error in case of high processing concurrency in `Ordered` mode. [#92814](https://github.com/ClickHouse/ClickHouse/pull/92814) ([Kseniia Sumarokova](https://github.com/kssenii)). * Previously, some queries that used PK sharding with a false condition were failing. Now they're not. Needed for https://github.com/ClickHouse/ClickHouse/pull/89313. [#92815](https://github.com/ClickHouse/ClickHouse/pull/92815) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fixed calculation of uncompressed sizes of text indexes in the `system.parts` table. [#92832](https://github.com/ClickHouse/ClickHouse/pull/92832) ([Anton Popov](https://github.com/CurtizJ)). * Fixed usage of primary index in lightweight updates that have an `IN` clause with subqueries in the predicate of the `WHERE` clause. [#92838](https://github.com/ClickHouse/ClickHouse/pull/92838) ([Anton Popov](https://github.com/CurtizJ)). * Fix creating type hint for path 'skip' in JSON. Closes [#92731](https://github.com/ClickHouse/ClickHouse/issues/92731). [#92842](https://github.com/ClickHouse/ClickHouse/pull/92842) ([Pavel Kruglov](https://github.com/Avogar)). * In the S3 table engine, we should avoid caching the partition key if there are non-deterministic functions. [#92844](https://github.com/ClickHouse/ClickHouse/pull/92844) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix possible error `FILE_DOESNT_EXIST` after mutation of a sparse column with `ratio_of_defaults_for_sparse_serialization=0.0`. Closes [#92633](https://github.com/ClickHouse/ClickHouse/issues/92633). [#92860](https://github.com/ClickHouse/ClickHouse/pull/92860) ([Pavel Kruglov](https://github.com/Avogar)). * Fix parquet schema inference in the old parquet reader (not used by default) when a JSON column comes after a Tupe column. Fix the old parquet reader (not used by default) failing on empty tuples. [#92867](https://github.com/ClickHouse/ClickHouse/pull/92867) ([Michael Kolupaev](https://github.com/al13n321)). * Fix logical error with multiple joins on constant condition and `join_use_nulls`, close [#92640](https://github.com/ClickHouse/ClickHouse/issues/92640). [#92892](https://github.com/ClickHouse/ClickHouse/pull/92892) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix possible error `NOT_FOUND_COLUMN_IN_BLOCK` during insert into a table with subcolumn in partition expression. Closes [#93210](https://github.com/ClickHouse/ClickHouse/issues/93210). Closes [#83406](https://github.com/ClickHouse/ClickHouse/issues/83406). [#92905](https://github.com/ClickHouse/ClickHouse/pull/92905) ([Pavel Kruglov](https://github.com/Avogar)). * Fix error `NO_SUCH_COLUMN_IN_TABLE` in Merge engine over tables with aliases. Closes [#88665](https://github.com/ClickHouse/ClickHouse/issues/88665). [#92910](https://github.com/ClickHouse/ClickHouse/pull/92910) ([Pavel Kruglov](https://github.com/Avogar)). * Fix NULL != NULL case for full_sorting_join on LowCardinality(Nullable(T)) column. [#92924](https://github.com/ClickHouse/ClickHouse/pull/92924) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fixed several crashes during merges of text indexes in `MergeTree` tables. [#92925](https://github.com/ClickHouse/ClickHouse/pull/92925) ([Anton Popov](https://github.com/CurtizJ)). * Restore LowCardinality wrappers on SET expression results if needed during TTL aggregation to prevent exceptions during table optimization. [#92971](https://github.com/ClickHouse/ClickHouse/pull/92971) ([Seva Potapov](https://github.com/seva-potapov)). * Fix logical error during index analysis when empty array is used in `has` function. Closes [#92906](https://github.com/ClickHouse/ClickHouse/issues/92906). [#92995](https://github.com/ClickHouse/ClickHouse/pull/92995) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix possible hung on terminating background schedule pool (may lead to server hungs on shutdown). [#93008](https://github.com/ClickHouse/ClickHouse/pull/93008) ([Azat Khuzhin](https://github.com/azat)). * Fix possible error FILE_DOESNT_EXIST after sparse column mutation when setting `ratio_of_defaults_for_sparse_serialization` was changed to `1.0` via alter. [#93016](https://github.com/ClickHouse/ClickHouse/pull/93016) ([Pavel Kruglov](https://github.com/Avogar)). * Fix bug in data skipping logic when `not materialize(...)` or `not CAST(...)` is used in WHERE causing incorrect results. Closes [#88536](https://github.com/ClickHouse/ClickHouse/issues/88536). [#93017](https://github.com/ClickHouse/ClickHouse/pull/93017) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix possible usage of outdated parts due to TOCTOU race for shared parts. [#93022](https://github.com/ClickHouse/ClickHouse/pull/93022) ([Azat Khuzhin](https://github.com/azat)). * Fix crash when deserialising malformed `groupConcat` aggregate state with out-of-bounds offsets. [#93028](https://github.com/ClickHouse/ClickHouse/pull/93028) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fix leaving connection in a broken state after preliminary cancellation distributed queries. [#93029](https://github.com/ClickHouse/ClickHouse/pull/93029) ([Azat Khuzhin](https://github.com/azat)). * Fix join results when the right-side join key is a sparse column. This closes [#92920](https://github.com/ClickHouse/ClickHouse/issues/92920). I can only reproduce the bug with `set compatibility='23.3'`. Not sure if it should be backported. [#93038](https://github.com/ClickHouse/ClickHouse/pull/93038) ([Amos Bird](https://github.com/amosbird)). * Fix possible `Cannot finalize buffer after cancellation` in `estimateCompressionRatio()`. Fixes: [#87380](https://github.com/ClickHouse/ClickHouse/issues/87380). [#93068](https://github.com/ClickHouse/ClickHouse/pull/93068) ([Azat Khuzhin](https://github.com/azat)). * Fixed merges of text indexes built on top of the complex expressions (such as `concat(col1, col2)`). [#93073](https://github.com/ClickHouse/ClickHouse/pull/93073) ([Anton Popov](https://github.com/CurtizJ)). * Fix applying projection when filter contains subcolumns. Closes [#92882](https://github.com/ClickHouse/ClickHouse/issues/92882). [#93141](https://github.com/ClickHouse/ClickHouse/pull/93141) ([Pavel Kruglov](https://github.com/Avogar)). * Fix logical error in some cases triggered when join runtime filters are added to query plan. It was caused by incorrectly returning duplicated const columns from one of join sides. [#93144](https://github.com/ClickHouse/ClickHouse/pull/93144) ([Alexander Gololobov](https://github.com/davenger)). * Special function `__applyFilter` used by join runtime filters was returning ILLEGAL_TYPE_OF_ARGUMENT in some valid cases. [#93187](https://github.com/ClickHouse/ClickHouse/pull/93187) ([Alexander Gololobov](https://github.com/davenger)). * Prevent different interpolated columns from collapse into the same column in a block when interpolated columns are effectively aliases of the same column. [#93197](https://github.com/ClickHouse/ClickHouse/pull/93197) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)). * Do not add runtime filter when joining with already filled right table. [#93211](https://github.com/ClickHouse/ClickHouse/pull/93211) ([Alexander Gololobov](https://github.com/davenger)). * Fix keeper persistent watches cleanup after dead session. This closes [#92480](https://github.com/ClickHouse/ClickHouse/issues/92480). [#93213](https://github.com/ClickHouse/ClickHouse/pull/93213) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix order by tuple in iceberg. This closes [#92977](https://github.com/ClickHouse/ClickHouse/issues/92977). [#93225](https://github.com/ClickHouse/ClickHouse/pull/93225) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix bug with S3Queue setting `s3queue_migrate_old_metadata_to_buckets`. Closes [#93392](https://github.com/ClickHouse/ClickHouse/issues/93392), [#93196](https://github.com/ClickHouse/ClickHouse/issues/93196), [#81739](https://github.com/ClickHouse/ClickHouse/issues/81739). [#93232](https://github.com/ClickHouse/ClickHouse/pull/93232) ([Kseniia Sumarokova](https://github.com/kssenii)). * Remove unused columns when the projection is rebuilt during the merge. It reduces memory usage and creates fewer temporary parts. [#93233](https://github.com/ClickHouse/ClickHouse/pull/93233) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fix unused columns removal from subqueries in the presence of a scalar correlated subquery. Before the fix column could have been removed if it was used only in the correlated subquery, and the query would fail with `NOT_FOUND_COLUMN_IN_BLOCK` error. [#93273](https://github.com/ClickHouse/ClickHouse/pull/93273) ([Dmitry Novik](https://github.com/novikd)). * Fix possible missing subcolumn in MV during alter of source table. Closes [#93231](https://github.com/ClickHouse/ClickHouse/issues/93231). [#93276](https://github.com/ClickHouse/ClickHouse/pull/93276) ([Pavel Kruglov](https://github.com/Avogar)). * Fix the `Merge` table engine query planning with the analyzer that could throw ILLEGAL_COLUMN for `hostName()` when merging local and remote/Distributed tables. Closes [#92059](https://github.com/ClickHouse/ClickHouse/issues/92059). [#93286](https://github.com/ClickHouse/ClickHouse/pull/93286) ([Jinlin](https://github.com/withlin)). * Fixes a case where NOT IN with non-constant array arguments was returning the wrong value + Support for non-constant Array functions. Closes [#14980](https://github.com/ClickHouse/ClickHouse/issues/14980). [#93314](https://github.com/ClickHouse/ClickHouse/pull/93314) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). * Fix `Not found column` for `use_top_k_dynamic_filtering` optimization. Fixes [#93186](https://github.com/ClickHouse/ClickHouse/issues/93186). [#93316](https://github.com/ClickHouse/ClickHouse/pull/93316) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Fixed rebuilding of text indexes created on top of subcolumns. [#93326](https://github.com/ClickHouse/ClickHouse/pull/93326) ([Anton Popov](https://github.com/CurtizJ)). * Fixed handling of empty array as a second argument in `hasAllTokens` and `hasAnyTokens` functions. [#93328](https://github.com/ClickHouse/ClickHouse/pull/93328) ([Anton Popov](https://github.com/CurtizJ)). * Fix logical error when runtime filters are used in a query with totals for right side table. [#93330](https://github.com/ClickHouse/ClickHouse/pull/93330) ([Alexander Gololobov](https://github.com/davenger)). * The server no longer crashes if function `tokens` is called with non-const tokenizer parameters (the 2th, 3rd, 4th parameter), e.g., `SELECT tokens(NULL, 1, materialize(1))`. [#93383](https://github.com/ClickHouse/ClickHouse/pull/93383) ([Robert Schulze](https://github.com/rschu1ze)). * Fixed integer overflow vulnerability in `groupConcat` state deserialisation that could cause memory safety issues with crafted aggregate states. [#93426](https://github.com/ClickHouse/ClickHouse/pull/93426) ([Raufs Dunamalijevs](https://github.com/rienath)). * Fixed text index analysis on array columns when the index contains no tokens (all arrays are empty or all tokens are skipped by the tokenizer). [#93457](https://github.com/ClickHouse/ClickHouse/pull/93457) ([Anton Popov](https://github.com/CurtizJ)). * Avoids oauth login in ClickHouse Client when username/password are within the connection string. [#93459](https://github.com/ClickHouse/ClickHouse/pull/93459) ([Krishna Mannem](https://github.com/kcmannem)). * Fix Azure ADLS Gen2 vended credentials support in DataLakeCatalog - parse `adls.sas-token.*` keys from Iceberg REST catalogs and fix ABFSS URL parsing. [#93477](https://github.com/ClickHouse/ClickHouse/pull/93477) ([Karun Anantharaman](https://github.com/karunmotorq)). * Fix GLOBAL IN support with analyzer (previously set was created on the remote node again). [#93507](https://github.com/ClickHouse/ClickHouse/pull/93507) ([Azat Khuzhin](https://github.com/azat)). * Fix extracting subcolumn during deserialization directly into Sparse columns. [#93512](https://github.com/ClickHouse/ClickHouse/pull/93512) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed direct reading from text index with duplicate search queries. [#93516](https://github.com/ClickHouse/ClickHouse/pull/93516) ([Anton Popov](https://github.com/CurtizJ)). * Fix for NOT_FOUND_COLUMN_IN_BLOCK error when runtime filter is enabled and joined tables have the same column returned multiple times (e.g. SELECT a, a, a FROM t). [#93526](https://github.com/ClickHouse/ClickHouse/pull/93526) ([Alexander Gololobov](https://github.com/davenger)). * Fix a bug where clickhouse-client would ask for password twice when connecting using ssh. [#93547](https://github.com/ClickHouse/ClickHouse/pull/93547) ([Isak Ellmer](https://github.com/spinojara)). * Make sure that zookeeper is finalized on shutdown (fix possible hung on shutdown in very unlikely cases). [#93602](https://github.com/ClickHouse/ClickHouse/pull/93602) ([Azat Khuzhin](https://github.com/azat)). * Fix LOGICAL_ERROR when restoring ReplicatedMergeTree with deduplication race. [#93612](https://github.com/ClickHouse/ClickHouse/pull/93612) ([Pablo Marcos](https://github.com/pamarcos)). * Fix using Sparse column for TTL update during direct deserialization into Sparse columns in some input formats. It fixes possible logical error `Unexpected type of result TTL column`. [#93619](https://github.com/ClickHouse/ClickHouse/pull/93619) ([Pavel Kruglov](https://github.com/Avogar)). * Fixed h3 index functions sometimes crashing or getting stuck when called on invalid inputs. [#93657](https://github.com/ClickHouse/ClickHouse/pull/93657) ([Michael Kolupaev](https://github.com/al13n321)). * The usage of `ngram_bf` index on a non-UTF-8 data led to an uninitialized memory read, with values that could reside in the resulting index structure. Closes [#92576](https://github.com/ClickHouse/ClickHouse/issues/92576). [#93663](https://github.com/ClickHouse/ClickHouse/pull/93663) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Validate that the decompressed buffer size is as expected. [#93690](https://github.com/ClickHouse/ClickHouse/pull/93690) ([Raúl Marín](https://github.com/Algunenano)). * Prevent users to get the list of columns from a table without checking `SHOW COLUMNS` permission using the `merge` table engine. [#93695](https://github.com/ClickHouse/ClickHouse/pull/93695) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)). * Fixed materialization of skip indexes created on top of subcolumns. [#93708](https://github.com/ClickHouse/ClickHouse/pull/93708) ([Anton Popov](https://github.com/CurtizJ)). * We store storages' shared pointers in `QueryPipeline::resources::storage_holders` to make sure that the `IStorage` objects are not destroyed while `PipelineExecutor` is alive. [#93746](https://github.com/ClickHouse/ClickHouse/pull/93746) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix attaching Replicated DBs when the interserver host changed after restarting. [#93779](https://github.com/ClickHouse/ClickHouse/pull/93779) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fix assert `!read_until_position` in `ReadBufferFromS3` which happened when cache is enabled. [#93809](https://github.com/ClickHouse/ClickHouse/pull/93809) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix logical error in a rare case when empty tuple is used with `Map` column. Closes [#93784](https://github.com/ClickHouse/ClickHouse/issues/93784). [#93814](https://github.com/ClickHouse/ClickHouse/pull/93814) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fixed `_part_offset` corruption when projections are rebuilt during merges, and optimized projection processing by avoiding unnecessary reads of the `_part_offset` column and skipping unneeded columns in projection calculations. This continues the optimizations introduced in [#93233](https://github.com/ClickHouse/ClickHouse/issues/93233). [#93827](https://github.com/ClickHouse/ClickHouse/pull/93827) ([Amos Bird](https://github.com/amosbird)). * Remove 'Bad version' handling. [#93843](https://github.com/ClickHouse/ClickHouse/pull/93843) ([Anton Ivashkin](https://github.com/ianton-ru)). * Fix `optimize_inverse_dictionary_lookup` not working with distributed query when key is signed integral type. Closes [#93259](https://github.com/ClickHouse/ClickHouse/issues/93259). [#93848](https://github.com/ClickHouse/ClickHouse/pull/93848) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix `lag`/`lead` not working with distributed `remote()` query. Closes [#90014](https://github.com/ClickHouse/ClickHouse/issues/90014). [#93858](https://github.com/ClickHouse/ClickHouse/pull/93858) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix system instrument dispatch bug. [#93937](https://github.com/ClickHouse/ClickHouse/pull/93937) ([Pablo Marcos](https://github.com/pamarcos)). * In https://github.com/ClickHouse/ClickHouse/pull/89173, we added an extra field to the structure that `TraceSender` sends through an internal pipe. However, the buffer size was not updated ([here](https://github.com/ClickHouse/ClickHouse/pull/89173/changes#diff-36ecfac5cde34c92c031652d8a77f0d12782cd5d43e68d6ef159e6d46a54224fL44)), therefore we are writing more data to buffer than `buffer_size` which results in multiple flushes. And because `TraceSender::send` is called from different threads, different threads' flushes may interleave which breaks the invariant that the receiving end (`TraceCollector`) relies on. [#93966](https://github.com/ClickHouse/ClickHouse/pull/93966) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix type conversion to super type during the join operation of the storage `Join` with `USING` clause. Fixes [#91672](https://github.com/ClickHouse/ClickHouse/issues/91672). Fixes [#78572](https://github.com/ClickHouse/ClickHouse/issues/78572). [#94000](https://github.com/ClickHouse/ClickHouse/pull/94000) ([Dmitry Novik](https://github.com/novikd)). * Fix for FilterStep not properly added when join runtime filter is applied over Merge table. [#94021](https://github.com/ClickHouse/ClickHouse/pull/94021) ([Alexander Gololobov](https://github.com/davenger)). * A `SELECT` query containing a predicate on multiple columns with bloom filter skip indexes and both `OR` and `NOT` conditions are present could return inconsistent results. That is fixed now. [#94026](https://github.com/ClickHouse/ClickHouse/pull/94026) ([Shankar Iyer](https://github.com/shankar-iyer)). * Fix CLEAR column with dependent indices. [#94057](https://github.com/ClickHouse/ClickHouse/pull/94057) ([Raúl Marín](https://github.com/Algunenano)). * Fix use-of-uninitialized-value in `ReadWriteBufferFromHTTP`. [#94058](https://github.com/ClickHouse/ClickHouse/pull/94058) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix bad check for typed paths in JSON. The check was introduced in https://github.com/ClickHouse/ClickHouse/pull/92842 and can lead to an error during existing tables startup. [#94070](https://github.com/ClickHouse/ClickHouse/pull/94070) ([Pavel Kruglov](https://github.com/Avogar)). * Fix the crash during filter analysis in the presence of OUTER JOIN. Fixes [#90979](https://github.com/ClickHouse/ClickHouse/issues/90979). [#94080](https://github.com/ClickHouse/ClickHouse/pull/94080) ([Dmitry Novik](https://github.com/novikd)). * Fix accuracy of `uniqTheta` when using UInt8 aggregation keys in parallel (`max_threads` > 1 - default). [#94095](https://github.com/ClickHouse/ClickHouse/pull/94095) ([Azat Khuzhin](https://github.com/azat)). * Fix crash caused by exception thrown from a `socket.setBlocking(true)` call inside `SCOPE_EXIT`. [#94100](https://github.com/ClickHouse/ClickHouse/pull/94100) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix data loss when `DROP PARTITION` removes parts created by later log entries in ReplicatedMergeTree. [#94123](https://github.com/ClickHouse/ClickHouse/pull/94123) ([Tuan Pham Anh](https://github.com/tuanpach)). * Fixed parquet reader v3 incorrectly handling arrays that cross page boundaries. This happens e.g. for files written by Arrow without enabling page statistics or page index. Affects only columns of Array data type. Likely symptom is that one array every ~1 MB of data gets truncated. Before this fix, use this setting as workaround: `input_format_parquet_use_native_reader_v3 = 0`. [#94125](https://github.com/ClickHouse/ClickHouse/pull/94125) ([Michael Kolupaev](https://github.com/al13n321)). * Fix too many watches in ReplicatedMergeTree while waiting for log entry. [#94133](https://github.com/ClickHouse/ClickHouse/pull/94133) ([Azat Khuzhin](https://github.com/azat)). * Functions `arrayShuffle`, `arrayPartialShuffle` and `arrayRandomSample` to materialize const columns - so that different rows get different results. [#94134](https://github.com/ClickHouse/ClickHouse/pull/94134) ([Joanna Hulboj](https://github.com/jh0x)). * Fix data race in evaluating table functions in materialized views. [#94171](https://github.com/ClickHouse/ClickHouse/pull/94171) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix nullptr dereference in `PostgreSQL` database engines (when the query is incorrect). Closes [#92887](https://github.com/ClickHouse/ClickHouse/issues/92887). [#94180](https://github.com/ClickHouse/ClickHouse/pull/94180) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix memory leak in refreshable materialized views using `SELECT` queries with multiple subqueries. [#94200](https://github.com/ClickHouse/ClickHouse/pull/94200) ([Antonio Andelic](https://github.com/antonio2368)). * Fix data race in `DataPartStorageOnDiskBase::remove` vs `system.parts`. Closes [#49076](https://github.com/ClickHouse/ClickHouse/issues/49076). [#94262](https://github.com/ClickHouse/ClickHouse/pull/94262) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Remove the wrong `noexcept` specifier at HashTable copy assignment that may lead to crash (`std::terminate`) on memory exceptions. [#94275](https://github.com/ClickHouse/ClickHouse/pull/94275) ([Nikita Taranov](https://github.com/nickitat)). * Previously, creating a projection with duplicate columns in GROUP BY (e.g., `GROUP BY c0, c0`) and inserting data caused a `std::length_error` if `optimize_row_order` is enabled. Closes [#94065](https://github.com/ClickHouse/ClickHouse/issues/94065). [#94277](https://github.com/ClickHouse/ClickHouse/pull/94277) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix obscure bug in ZooKeeper client on connect which leads to hungs and crashes. [#94320](https://github.com/ClickHouse/ClickHouse/pull/94320) ([Azat Khuzhin](https://github.com/azat)). * Fix function to subcolumns optimization not applied to subcolumns. [#94323](https://github.com/ClickHouse/ClickHouse/pull/94323) ([Pavel Kruglov](https://github.com/Avogar)). * Fix possibly incorrect result in nested RIGHT JOINs when `enable_lazy_columns_replication` is enabled. The bug caused all rows in replicated columns to incorrectly return the same value instead of their distinct values. Close [#93891](https://github.com/ClickHouse/ClickHouse/issues/93891). [#94339](https://github.com/ClickHouse/ClickHouse/pull/94339) ([Vladimir Cherkasov](https://github.com/vdimir)). * Fix filter pushdown for SEMI JOIN using equivalence sets. Do not push the filter down if argument types have changed. Fixes [#93264](https://github.com/ClickHouse/ClickHouse/issues/93264). [#94340](https://github.com/ClickHouse/ClickHouse/pull/94340) ([Dmitry Novik](https://github.com/novikd)). * Fix usage of DeltaLake CDF with database DataLake database engine (delta lake catalogs integration). Closes [#94122](https://github.com/ClickHouse/ClickHouse/issues/94122). [#94342](https://github.com/ClickHouse/ClickHouse/pull/94342) ([Kseniia Sumarokova](https://github.com/kssenii)). * Fix incorrect value of current metric `FilesystemCacheSizeLimit` in case `SLRU` cache policy was used. [#94363](https://github.com/ClickHouse/ClickHouse/pull/94363) ([Kseniia Sumarokova](https://github.com/kssenii)). * Creating a Backup database engine with less than two arguments now returns a more descriptive error message (`Wrong number of arguments` instead of `std::out_of_range: InlinedVector::at(size_type) const failed bounds check.`). [#94374](https://github.com/ClickHouse/ClickHouse/pull/94374) ([Robert Schulze](https://github.com/rschu1ze)). * Ignores impossible revokes of global grants on the database level for grants with grant option. [#94386](https://github.com/ClickHouse/ClickHouse/pull/94386) ([pufit](https://github.com/pufit)). * Fix reading sparse offsets from compact parts. Closes [#94385](https://github.com/ClickHouse/ClickHouse/issues/94385). [#94399](https://github.com/ClickHouse/ClickHouse/pull/94399) ([Pavel Kruglov](https://github.com/Avogar)). * Don't prevent ALTER of columns using implicit indexes, even if `alter_column_secondary_index_mode`'s `throw` mode is used. [#94425](https://github.com/ClickHouse/ClickHouse/pull/94425) ([Raúl Marín](https://github.com/Algunenano)). * Fix crash in `TCPHandler` when multiple `receivePacketsExpectQuery` calls read `Protocol::Client::IgnoredPartUUIDs`. [#94434](https://github.com/ClickHouse/ClickHouse/pull/94434) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fix masking sensitive data in `system.functions`. [#94436](https://github.com/ClickHouse/ClickHouse/pull/94436) ([Vitaly Baranov](https://github.com/vitlibar)). * Fix nullptr dereference with disabled `send_profile_events`. This feature was introduced recently for the ClickHouse Python driver. Closes [#92488](https://github.com/ClickHouse/ClickHouse/issues/92488). [#94466](https://github.com/ClickHouse/ClickHouse/pull/94466) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix text index .mrk incompatibility during merges. [#94494](https://github.com/ClickHouse/ClickHouse/pull/94494) ([Peng Jian](https://github.com/fastio)). * When `read_in_order_use_virtual_row` is enabled, the code was accessing index columns based on the full primary key size without checking if the index was truncated, leading to use-after-free / uninitialized memory. Closes [#85596](https://github.com/ClickHouse/ClickHouse/issues/85596). [#94500](https://github.com/ClickHouse/ClickHouse/pull/94500) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix an error due to a type mismatch when sending external tables for subqueries with GLOBAL IN if the types are Nullable. Closes [#94097](https://github.com/ClickHouse/ClickHouse/issues/94097). [#94511](https://github.com/ClickHouse/ClickHouse/pull/94511) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * In previous versions, queries with multiple index conditions over the same expression may erroneously throw an exception `Not found column`. Closes [#60660](https://github.com/ClickHouse/ClickHouse/issues/60660). [#94515](https://github.com/ClickHouse/ClickHouse/pull/94515) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix incorrect handling of Nullable join column in runtime filters. [#94555](https://github.com/ClickHouse/ClickHouse/pull/94555) ([Alexander Gololobov](https://github.com/davenger)). * Creating a workload in another workload that is currently in use no longer causes a crash. [#94599](https://github.com/ClickHouse/ClickHouse/pull/94599) ([Sergei Trifonov](https://github.com/serxa)). * Fix a crash during ANY LEFT JOIN optimization when `isNotNull` is evaluated on a missing column. [#94600](https://github.com/ClickHouse/ClickHouse/pull/94600) ([Molly](https://github.com/ggmolly)). * Fix default expression evaluation when referencing other columns with computed defaults. [#94615](https://github.com/ClickHouse/ClickHouse/pull/94615) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix permission issues in BACKUP/RESTORE operations. [#94617](https://github.com/ClickHouse/ClickHouse/pull/94617) ([Pablo Marcos](https://github.com/pamarcos)). * Fix crash due to incorrect type cast when the data type is `Nullable(DateTime64)`. [#94627](https://github.com/ClickHouse/ClickHouse/pull/94627) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)). * Fixes a bug where certain distributed queries with `ORDER BY` could return `ALIAS` columns with swapped values (i.e., column `a` showing column `b`’s data and vice versa). [#94644](https://github.com/ClickHouse/ClickHouse/pull/94644) ([filimonov](https://github.com/filimonov)). * Fix storing results of keeper-bench to file. [#94654](https://github.com/ClickHouse/ClickHouse/pull/94654) ([Antonio Andelic](https://github.com/antonio2368)). * Fix incorrect estimations with MinMax-type statistics when the column contains negative floating-point values. [#94665](https://github.com/ClickHouse/ClickHouse/pull/94665) ([zoomxi](https://github.com/zoomxi)). * Fix reading Parquet files when a map's key is a struct. [#94670](https://github.com/ClickHouse/ClickHouse/pull/94670) ([Konstantin Vedernikov](https://github.com/scanhex12)). * Fix possibly incorrect RIGHT join result when using complex ON conditions. Close [#92913](https://github.com/ClickHouse/ClickHouse/issues/92913). [#94680](https://github.com/ClickHouse/ClickHouse/pull/94680) ([Vladimir Cherkasov](https://github.com/vdimir)). * Preserve constant index granularity (use_const_adaptive_granularity) after Vertical merges. [#94725](https://github.com/ClickHouse/ClickHouse/pull/94725) ([Azat Khuzhin](https://github.com/azat)). * Fix mutation bug with scalar subqueries and table dependencies. If a table had dependencies (index or projections) over a column, scalar subqueries might be evaluated and cached without data and lead to incorrect changes. [#94731](https://github.com/ClickHouse/ClickHouse/pull/94731) ([Raúl Marín](https://github.com/Algunenano)). * Fix AsynchronousMetrics cpu_pressure fallback on error. [#94827](https://github.com/ClickHouse/ClickHouse/pull/94827) ([Raúl Marín](https://github.com/Algunenano)). * The `getURLHostRFC` function was missing bounds checks before dereferencing pointers. When an empty string was passed to `domainRFC`, it would read uninitialized memory, triggering MSan errors. [#94851](https://github.com/ClickHouse/ClickHouse/pull/94851) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix readonlyness of encrypted disks. [#94852](https://github.com/ClickHouse/ClickHouse/pull/94852) ([Azat Khuzhin](https://github.com/azat)). * Fix logical error in fractional `LIMIT/OFFSET` when using the old analyzer with Distributed tables. Closes [#94712](https://github.com/ClickHouse/ClickHouse/issues/94712). [#94999](https://github.com/ClickHouse/ClickHouse/pull/94999) ([Ahmed Gouda](https://github.com/0xgouda)). * Fix crash under some conditions when join runtime filters are enabled by default. [#95000](https://github.com/ClickHouse/ClickHouse/pull/95000) ([Alexander Gololobov](https://github.com/davenger)). * Improve masking passwords in url used in table engine `URL()` and table function `url()`. [#95006](https://github.com/ClickHouse/ClickHouse/pull/95006) ([Vitaly Baranov](https://github.com/vitlibar)). * Function `toStartOfInterval` now works in the same way as `toStartOfX`, where `X` is `Day, Week, Month, Quarter, Year` when the `enable_extended_results_for_datetime_functions` is on. [#95011](https://github.com/ClickHouse/ClickHouse/pull/95011) ([Kirill Kopnev](https://github.com/Fgrtue)). * Fix constant string comparisons not respecting the settings `cast_string_to_date_time_mode`, `bool_true_representation`, `bool_false_representation`, and `input_format_null_as_default`. Closes [#91681](https://github.com/ClickHouse/ClickHouse/issues/91681). [#95040](https://github.com/ClickHouse/ClickHouse/pull/95040) ([Nihal Z. Miaji](https://github.com/nihalzp)). * Fix data race in the filesystem cache. [#95064](https://github.com/ClickHouse/ClickHouse/pull/95064) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix a rare race condition in the Parquet reader. [#95068](https://github.com/ClickHouse/ClickHouse/pull/95068) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Fix crash in the top K optimization when `LIMIT` is zero. Closes [#93893](https://github.com/ClickHouse/ClickHouse/issues/93893). [#95072](https://github.com/ClickHouse/ClickHouse/pull/95072) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Converting from DateTime/integers to Time64 extracts the time-of-day component using `toTime`, which is not monotonic. The `ToDateTimeMonotonicity` template incorrectly claimed this conversion was monotonic, causing "Invalid binary search result in MergeTreeSetIndex" exception in debug builds. [#95125](https://github.com/ClickHouse/ClickHouse/pull/95125) ([Alexey Milovidov](https://github.com/alexey-milovidov)). * Recreated list of manifest file entries only if necessary (previously it was done on each iteration). [#95162](https://github.com/ClickHouse/ClickHouse/pull/95162) ([Daniil Ivanik](https://github.com/divanik)). #### Build/Testing/Packaging Improvement * Add a set of tools for profiling memory allocations in the ClickHouse SQL parser using jemalloc's heap profiling capabilities. [#94072](https://github.com/ClickHouse/ClickHouse/pull/94072) ([Ilya Yatsishin](https://github.com/qoega)). * Added a tool that simplifies debugging of memory allocations in parser. It uses jemalloc `stats.allocated` metric before and after we parse query to AST representation to show what is allocated. Also it supports memory profiling mode that dumps profile before and after to build reports where allocations occurred. [#93523](https://github.com/ClickHouse/ClickHouse/pull/93523) ([Ilya Yatsishin](https://github.com/qoega)). * Remove transitive libc++ includes. [#92523](https://github.com/ClickHouse/ClickHouse/pull/92523) ([Raúl Marín](https://github.com/Algunenano)). * Make some sequential tests parallel: https://github.com/ClickHouse/ClickHouse/pull/93030/changes#diff-c3a73510dae653c9bbfa24300b32f5d6ec663fd4e72cc4a3d5daa6e4342915df. [#93030](https://github.com/ClickHouse/ClickHouse/pull/93030) ([Nikita Fomichev](https://github.com/fm4v)). * Cleanup some build flags. [#93679](https://github.com/ClickHouse/ClickHouse/pull/93679) ([Raúl Marín](https://github.com/Algunenano)). * Bump c-ares from v1.34.5 to v1.34.6. This addresses c-ares' `CVE-2025-62408`, which is not relevant for ClickHouse. [#94129](https://github.com/ClickHouse/ClickHouse/pull/94129) ([Govind R Nair](https://github.com/Revertionist)). * Use `curl` 8.18.0. [#94742](https://github.com/ClickHouse/ClickHouse/pull/94742) ([Konstantin Bogdanov](https://github.com/thevar1able)).