# Changelog

This file documents the historical progress of the Micromegas project. For current focus, please see the main [README.md](./README.md).

## Unreleased

* **Web App:**
  * Fix `micromegas-screens`/`micromegas-query` CLI tools mis-decoding non-ASCII text (em dashes, accents, CJK) as mojibake when run under a non-UTF-8 platform locale: pin every text-mode file/stdin read and write to UTF-8 across `screens.py`, `query.py`, and the OIDC token cache, switch pulled screen files and plan/apply diffs to literal UTF-8 instead of `\uXXXX` escapes, and protect `apply`/`pull` from silently deleting or overwriting a screen whose local file can't be decoded or parsed (#1399)
  * Fix folder and screen rows in the Screens sidebar not behaving like real links: ctrl/cmd-click, middle-click, and right-click "open in new tab" now work as expected, since the rows render as actual anchors instead of `div`s intercepting every click (#1394)
  * Recognize UCUM/OTLP unit codes emitted by CloudWatch Metric Streams and other OTLP producers (`By`, `By/s`, `kBy`, `MiBy`, `MBit/s`, `{Count}`, `1`, `Cel`, ...), so a `1234567890 By/s` measure now renders as `1.1 GB/s` instead of a raw number with the code appended; `{...}` annotations are stripped by rule rather than by table entry, every size/bit prefix gains a working `/s` rate form, dimensionless units (`none`, `count`, `{Count}`, `1`) render as bare numbers, and equivalent spellings (`bytes`/`B`/`By`) now share a single Y axis instead of producing one per spelling (#1389)
  * Fix `micromegas-screens` CLI silently dropping a screen's folder assignment on import/pull, and showing a spurious diff that would wipe it on apply: thread `folder_path` through `WebClient.create_screen`/`update_screen` and the CLI's file read/write/diff paths end-to-end (#1362)
  * Add folder organization for saved screens: a `folders` API (list/create/rename/move/delete) backed by a materialized-path `folder_path` column on `screens`, a sidebar folder tree with breadcrumbs and drag-and-drop move, a folder picker in the Save/Save As dialog, and search that also matches folder names; folder rename/move/delete and screen create/move are serialized with Postgres advisory locks to close TOCTOU races between concurrent operations on the same path (#1159)
  * Fix categorical bar charts clipping the first/last bars and off-centering their x-axis labels: pad the categorical x-scale by half a slot via a new `buildXScale` helper so every bar sits fully inside the plot area
  * Add currency-aware value formatting: a metric `unit` recognized as an ISO 4217 currency code (e.g. `USD`, `CAD`, `EUR`) now renders as proper money (`"$1,234.56"`) in tooltips, the stats panel, and Y-axis ticks, instead of falling through to a bare number with the raw unit string appended (#1326)
  * Add an optional per-cell query time range override to every query-backed notebook cell (table, chart, log, property timeline, swimlane, transposed, flame graph, map, expression variables, image, and Perfetto export), so a cell can pin a fixed range or derive it from a variable, an upstream cell result, or a row/drag selection instead of always inheriting the screen's global range; a bad override now surfaces as a cell error uniformly, including on Perfetto export which previously fell back silently (#1314)
  * Add a per-cell "Wrap text" toggle to the notebook Log cell so long or multi-line `msg` values (e.g. stack traces) render wrapped instead of single-line-truncated, defaulting on and persisted in `options.wrapText`; the last column now bounds to the row's available width instead of a hardcoded 700px cap
  * Fix flamechart on-canvas labels bleeding into sibling spans and add ellipsis truncation for long span names (#1305); bound the hover tooltip's width/height, wrap embedded newlines, and position it from its measured size instead of hardcoded offsets (#1306)
* **Analytics:**
  * Fix the maintenance daemon's materialization pass aborting on the first view that fails, starving every view ordered after it for the rest of that pass (and, since the daemon's lookback windows are short, potentially forever): isolate per-view failures in `materialize_all_views` so every view still gets its own materialization attempt, aggregate the failures into a single reported error, and add a `materialize_view_failure` metric tagged by view for observability (#1393)
  * Add order-preserving k-way merge for `SqlBatchView`s with a non-temporal sort key: a new `ScanOrdering::PerFile` scan mode feeds each certified-sorted partition to DataFusion as its own file group so a `SortPreservingMergeExec` collapses them in one pass instead of buffering a full sort, gated by a per-partition `sort_order` degrade check; `SqlBatchView::with_merge_sort_order` lets a view declare its merge sort columns (adopted by `log_stats`), and `regenerate_partitions` can now upgrade partitions materialized before the declaration was added. **Breaking API change**: `View::get_scan_output_ordering` now returns `ScanOrdering` instead of `Vec<ScanSortColumn>`; `PartitionedTableProvider::with_ordering` is renamed to `with_scan_ordering` and now takes a single `scan_ordering: ScanOrdering` argument instead of an ordering `Vec<ScanSortColumn>` plus a separate `OrderingBounds`; `QueryMerger::with_merge_scan_ordering` now takes `ScanOrdering` instead of `Vec<ScanSortColumn>`; `make_partitioned_execution_plan` replaces its `output_ordering: &[ScanSortColumn]`/`ordering_bounds: OrderingBounds` parameters with a single `scan_ordering: &ScanOrdering`; and `fetch_sql_partition_spec` and `SqlPartitionSpec::new` both gain a required `sort_order: Option<Vec<String>>` parameter (#1392)
  * Set `write_partition`'s Parquet `max_row_group_size` to 128 Ki rows for finer row-group pruning (#1392)
  * Materialize OTLP `Summary` metrics (e.g. CloudWatch Metric Streams' `opentelemetry1.0` output) into `measures` as `count`/`sum`/`min`/`max` rows under suffixed metric names, instead of silently dropping them; bump `measures`' schema version so previously-ingested Summary blocks become eligible for re-materialization — note that the bump also retires every existing `measures` partition from query results (partitions are matched on an exact schema hash), so historical metrics stay invisible until `regenerate_partitions('measures', ...)` is run for the desired range; only the maintenance daemon's short trailing windows re-materialize on their own (#1359)
  * Fix a dictionary key overflow panic in the span/async-event/net-span/metrics/log-entries/images table builders and their OTLP companion processors: widen the `Int16`-keyed dictionary columns to `Int32` and replace panicking `append_value`/`append_values` calls with fallible equivalents, so a query batch with more than 32,767 distinct values in one of these columns no longer crashes the background query task (#1341)
  * Make `blocks_view` partition merges order-preserving so merged partitions stay internally sorted by `insert_time`, stream the Postgres-backed partition write path in bounded chunks instead of `fetch_all`-ing a whole insert range into memory, and add a `regenerate_partitions` admin table function to force-regenerate existing merged partitions from source; record a per-partition `sort_order` in `lakehouse_partitions` metadata, and enforce partition insert-time disjointness with a `btree_gist`-backed Postgres exclusion constraint (#1336)
  * Fix Perfetto trace export OOMing on wide-time-range, many-thread processes: declare `thread_spans`' existing scan ordering to DataFusion so `EnforceSorting` drops the per-thread `ORDER BY begin` sort instead of materializing a concurrent `ExternalSorter` per thread against the shared memory pool; `ORDER BY` stays in the query and is still honored, just free (#1297)
  * Hoist the per-segment `blocks_view` partition fetch out of the segment loop in JIT partition generation (`generate_process_jit_partitions`/`generate_stream_jit_partitions`): fetch the partition list once for the whole insert-time range and filter it in memory per segment instead of re-querying Postgres for every 1-hour segment, cutting a multi-day query's redundant round-trips from dozens-to-hundreds down to one, with no behavior change
* **Build:**
  * Bump the pinned Rust toolchain to 1.97.1, and regenerate the datafusion-wasm bindings whose internal closure-glue symbol names changed under the new compiler
  * Bump `datafusion` from 54.0 to 54.1.0 in both `rust/Cargo.toml` and `rust/datafusion-wasm/Cargo.toml` (bug-fix release; no Arrow version change), and regenerate the checked-in `datafusion-wasm` bindings
  * Bump the root workspace's `resolutions."react-router"` from `^7.18.0` to resolve Dependabot alert 395 (GHSA-qwww-vcr4-c8h2), pinning it via a `yarn patch` descriptor to exactly `8.3.0` (not a `^8.3.0` range — future 8.x patch releases won't be picked up automatically and will need the patch manually regenerated against the new version); the patch neutralizes the one line responsible for react-router 8.x's Jest breakage (`import.meta.hot` in framework-mode `loadRouteModule`, dead code never reached by this app's `react-router-dom-v5-compat`-only usage), and extend `grafana/jest.config.js`'s `transform` to also handle `.mjs` files so the new transitive `cookie-es` dependency parses; also bump `grafana/.nvmrc` and `.github/workflows/grafana-plugin.yml`'s `node-version` from Node 20 to the floating major `22`, and `grafana/package.json`'s `engines.node` from `>=20` to `>=22`, to satisfy `react-router@8.3.0`'s `engines.node >=22.22.0` floor, mirroring `analytics-web-app`'s #1351 precedent; also fix `build/grafana_ci.py`'s `run_cmd` to `nvm use` the explicit resolved Node version instead of a bare `nvm use`, since the bare form silently resolved the nearest `.nvmrc` by walking up from the invoking directory — a bug that would otherwise have silently run most of the Grafana CI suite under Node 20 (the root `.nvmrc`) after this branch's Node 22 bump
  * Bump `react-router-dom` to `react-router@^8.3.0` in `analytics-web-app` (the package was renamed starting with v8), and bump `.nvmrc` from Node 20 to the floating major `22` to satisfy `react-router@8.3.0`'s `engines.node >=22.22.0` floor, to resolve Dependabot alert 388 (GHSA-qwww-vcr4-c8h2); also bump the three Docker frontend-builder stages (`docker/analytics-web.Dockerfile`, `docker/monolith.Dockerfile`, `docker/all-in-one.Dockerfile`) from `node:20-alpine` to `node:22-alpine`, and `docker/github-runner.Dockerfile`'s CI runner image from nodesource `setup_20.x` to `setup_22.x` plus a new nvm install pre-provisioning both Node 20 and 22 (the runner is shared with the Grafana plugin build, which now also pins Node 22)
  * Bump `google.golang.org/grpc` to `1.82.1` in the grafana plugin's Go module to resolve Dependabot alert 381 (GHSA-hrxh-6v49-42gf)
  * Bump `dompurify` to `^3.4.12` (GHSA-c2j3-45gr-mqc4) and `fast-uri` to `^3.1.4` (GHSA-4c8g-83qw-93j6, GHSA-v2hh-gcrm-f6hx) via yarn `resolutions`/`overrides` across the root, `doc/intro-micromegas`, and `doc/high-frequency-observability` workspaces to resolve Dependabot alerts 383, 380, 379, 384, 382
  * Upgrade `react-router-dom` to `^7.18.0` in `analytics-web-app`, and force the transitive `react-router` resolution to `^7.18.0` in the root workspace (pulled in only via `@grafana/ui`'s legacy v5 routing compat shim), to resolve Dependabot alerts 386, 385, 378, 377, 376 (GHSA-wrjc-x8rr-h8h6, GHSA-337j-9hxr-rhxg, GHSA-jjmj-jmhj-qwj2) — no fix exists in the v6 line for these advisories
  * Bump `postcss` to `^8.5.18` via yarn `resolutions`/`overrides` across the root, `welcome`, `analytics-web-app`, `doc/notebooks`, `doc/unified-observability-for-games`, and `doc/high-frequency-observability` workspaces, fixing a path-traversal vulnerability (GHSA-r28c-9q8g-f849) found via `npm audit`
  * Bump the pinned Rust toolchain to 1.97.0; fix new `clippy::useless_borrows_in_formatting` lints the version tightened, and regenerate the datafusion-wasm bindings whose internal closure-glue symbol names changed under the new compiler
  * Pin the transitive `websocket-driver` dev dependency to `^0.7.5` via yarn `resolutions` to resolve Dependabot alerts #339/#340
  * Bump `js-yaml` to `^4.3.0`, `protobufjs` to `^7.6.5`, and `brace-expansion` to `^5.0.7`/`^2.1.2` via yarn `resolutions` across the root, `welcome/`, and `analytics-web-app/` workspaces to resolve Dependabot alerts #341-#347
  * Bump `brace-expansion` to `^5.0.8` via yarn `resolutions` across the root, `welcome/`, and `analytics-web-app/` workspaces (the older 1.x-4.x lines never received a patched release, only 5.x did) to resolve Dependabot alerts 396-398 (GHSA-mh99-v99m-4gvg)
  * Bump `tar` to `^7.5.19` (resolving to 7.5.20) across all yarn workspaces (root, `welcome/`, `analytics-web-app/`, and the `doc/*` sites) to resolve 24 Dependabot alerts (#348-371)
  * Bump `immutable` to `^5.1.8` (resolving to 5.1.9) in the root yarn workspace to resolve 2 Dependabot alerts (#374, #375)
  * Delete the unused `uri-handler` crate, and route `analytics-web-srv` and `object-cache-srv` through the public `micromegas` facade (`micromegas::auth`/`micromegas::object_cache`/`micromegas::tracing`) instead of depending on internal crates directly (#1256)
  * Migrate `analytics-web-app`'s test runner from Jest to Vitest, so tests are parsed by the same engine that builds the app and ESM-only dependencies stop requiring per-package carve-outs; drop the now-unreachable Babel/Jest packages and `handlebars` resolutions pin (#1345)
* **Grafana Plugin:**
  * Upgrade the Grafana plugin SDK from 11.6.7 to 12.4.6: replace the frozen `@grafana/experimental` with its successor `@grafana/plugin-ui`, bump `@grafana/tsconfig`/`@grafana/plugin-e2e`/`@testing-library/react` to satisfy peer dependencies, bump the local/e2e test Grafana instance to 12.4.6, and add e2e coverage for the `SQLEditor` swap (#848)
* **Caching:**
  * Fix `CacheClientStore::full_stream_with_fallback` surfacing the cache's own error (or, latently, silently truncating) on a `GET /obj` body that fails or ends short partway through: it now resumes the remainder from the direct store at the byte offset already delivered, so the only errors a client can ever surface are the direct store's own (#1360)
  * Add a client-side circuit breaker gating `CacheClientStore`'s reads and prefetches: after 5 consecutive unresponsive requests, reads and prefetches skip the cache entirely for a fixed 3s cooldown, with one probe request admitted per cooldown to detect recovery; replace the client's two flat 2s/15s timeouts with a `CacheClientConfig` (50ms connect, 500ms per-request abandon budget, 3s stall budget, 15s unchanged total deadline), all but the fixed budgets overridable via new `MICROMEGAS_OBJECT_CACHE_CLIENT_*` env vars (#1360)
  * Fix a benign but observable write race in `FoyerBackend::get`: replace foyer's `HybridCache::get` with a two-step read (RAM lookup, then direct disk load) whose disk→RAM promotion is gated on a caller-supplied expected length, plus a per-key single-flight around the disk load to keep read-coalescing parity; this makes foyer's inflight decoy-close-flag path unreachable and adds `object_cache_ram_tier_hit`/`object_cache_disk_tier_hit`, `range_cache_load_coalesced`, and `range_cache_promotion_len_mismatch` metrics (#1318)
  * Add `object_cache_promotion_count`/`object_cache_promotion_bytes` (`+ prefix`) metrics measuring disk→RAM promotion volume, emitted alongside `object_cache_disk_tier_hit` at the single validated-promotion crossing point in `FoyerBackend::promote_if_valid`; together with `object_cache_ram_tier_eviction_*` they make RAM-tier churn (promotion in vs. eviction out) directly observable (#1321)
  * Add an `object_cache_ram_tier_entries` saturation gauge alongside `object_cache_ram_tier_usage_bytes`, giving RAM-tier occupancy in entry count as well as bytes (#1322)
  * Fix `object_cache_get_bytes_served` recording a structural zero on the live `GET /obj/{key}` path: a `Content-Length`-framed HTTP body is never polled to a terminal `None`, so the byte-count callback fired from the drained stream never ran; `count_bytes_served` now fires as soon as the known expected payload is fully produced, before the final chunk is yielded, and both GET and `/ranges` call sites pass their known total (#1279)
  * Instrument object-cache eviction: emit RAM-tier eviction count/age metrics (tagged by `prefix` and `reason`) via a foyer `EventListener`, and a disk read-age estimate on disk-tier read hits, so it's observable what gets evicted and how long it lived before leaving a tier (#1281)
  * Fix a demand-fill leak where a cached block stored as a slice of its coalesced origin-GET buffer pinned the whole parent allocation, letting RAM-tier RSS run up to `max_coalesced_get_bytes / block_size`x its accounted weight; copy on demand admission in both `FoyerBackend` and the L1 `BoundedMemoryBackend` to detach the cached block, and export accounted RAM-tier usage as a new `object_cache_ram_tier_usage_bytes` saturation gauge (#1276)
  * Fix the same class of leak on the prefetch path: `FoyerBackend::put`'s prefetch arm stored the incoming slice verbatim, keeping its coalesced origin-GET parent buffer alive for as long as the entry lived in foyer's disk-write pipeline; copy before insert, mirroring the demand arm (#1317)
  * Add an `object_cache_head_requests` (`status`, `prefix`) counter to the `HEAD /obj/{key}` path in `object-cache-srv` by splitting `head_handler` into a thin counting wrapper + inner handler, mirroring the existing GET/`ranges` wrappers, so HEAD traffic volume and its status-code distribution are directly visible instead of only inferable as a residual of other call sites' size/HEAD-tier metrics (#1280)
  * Guard the foyer disk cache against stale on-disk formats: stamp a `DISK_FORMAT_VERSION` marker file in the disk dir and wipe it on a version mismatch or missing marker, so a value-layout change can never misdecode an old entry; add a `MAX_PLAUSIBLE_OBJECT_SIZE` sanity-check to the `size()` fast path (on both the cached-read and origin-HEAD paths) that rejects an implausible size instead of driving a catastrophic allocation, with `object_cache_disk_format_wiped` and `range_cache_size_implausible` metrics (#1287)
  * Fix `object-cache-srv`'s graceful shutdown dropping in-flight origin fetches instead of draining them: the grace period now also aborts the prefetch worker, waits for outstanding detached fetch tasks (coalesced block-run GETs and `size()` HEADs) to finish, and closes the foyer disk cache so a drained fetch's bytes survive a restart, all best-effort within whatever of the grace period axum's own HTTP drain didn't use; also fixes `FulfillGuard`'s shutdown-drop log incorrectly claiming a panic (#1291)
* **Docs:**
  * Fix `schema-reference.md`/`functions-reference.md`/`how_to_query` docs to match the `Int16`->`Int32` dictionary widening above, and correct stale `processes`/`streams` string-column types (Utf8, not `Int16` dictionaries) and a missing `images` table `Int32` list entry (#1341)
  * Rewrite the "Getting Started" page as a Docker-based quickstart (clone-free `curl` + in-repo compose options, self-contained monolith compose file with inline DB init, web app, optional Python sample query, stop/cleanup, troubleshooting); relocate the developer/build setup to the Build Guide and repoint entry-point links; note the Compose v2.23.1+ prerequisite where the shared compose file is invoked (#1273)
  * Split the root `CLAUDE.md` into per-subproject files (`rust/`, `python/`, `analytics-web-app/`, `grafana/`, `mkdocs/`) so subproject-specific guidance only loads when that subproject is touched, trim the root file to cross-cutting rules, and delete the now-fully-absorbed `AI_GUIDELINES.md`
* **Config:**
  * Consolidate ad-hoc `MICROMEGAS_*` env-var reads into typed config structs (`DataLakeConfig`, `WebServerConfig`, `CommonServerArgs`, `GatewayConfig`) and shared helpers (`parse_object_store_url`), replacing ~15 scattered `std::env::var` call sites; flatten `CommonServerArgs` into the service binaries and add an env-backed `--flightsql-url` flag to `http-gateway` to fail fast instead of reading the env per request (#1248)
* **Observability:**
  * Wrap DB query futures and the remaining JIT partition generation / block-write futures (partition cache fetches, DataFusion collects, block streaming, spawned task joins, logger writes) across analytics, ingestion, public, and analytics-web-srv with `instrument_named!`, so each round-trip shows up as its own span nested under the caller, separating DB/IO wait time from CPU time in traces
  * Add process RSS/virtual-size gauges (`process_resident_bytes`/`process_virtual_bytes`, allocator-agnostic via `sysinfo`) and jemalloc runtime stats (`jemalloc_allocated_bytes`/`jemalloc_resident_bytes`/`jemalloc_mapped_bytes`/`jemalloc_retained_bytes`, via a new `jemalloc-metrics` feature) to the shared `system_monitor` sampler every service runs, sampled every ~5s, so a production memory climb can be diagnosed as this process vs. another, a logical leak vs. allocator retention, or heap vs. non-heap growth (#1319)
  * Add a `pg_stat_*` self-observability collector to the maintenance daemon: a `PgStatsTask` samples PostgreSQL's standard `pg_stat_*` views (plus index/table sizes) once a minute and emits the readings as per-relation-tagged `pg_*` micromegas metrics, turning static questions like "which indexes are dead weight" into queryable runtime signals; includes an admin-docs section with sample unused-index and cache-hit-ratio queries (#1292)
  * Emit one structured JSON audit record per FlightSQL query under the `flightsql_query_audit` log target, unifying request attribution (client, user, email, service account, SQL, time range) with per-query cost (stage durations, output rows, bytes scanned) in one queryable row so expensive queries can be attributed to their source; retains the physical plan to read DataFusion metrics post-drain, makes the lakehouse parquet reader record `bytes_scanned` (previously structurally zero), and audits cancelled and setup-failed queries too (#1288)
* **Ingestion:**
  * Add a generic header-described `POST /ingestion/webhook` endpoint: any header-capable webhook producer (GitLab, GitHub, generic SaaS) can report directly to micromegas with no external transformer service, using `X-Micromegas-*` headers to synthesize an OTLP `Resource` and the verbatim request body as a single log record's `msg`, reusing the existing OTLP logs identity/block/write path end-to-end (#1296)
  * Add a `POST /ingestion/otlp/v1/metrics/firehose` endpoint speaking the Amazon Kinesis Data Firehose HTTP Endpoint Delivery protocol, so a CloudWatch Metric Stream (OpenTelemetry 1.0.0 output) can push metrics into micromegas as Metric Stream → Firehose → micromegas with no Lambda, Kinesis Data Stream, or collector in between: a thin envelope adapter unwraps the Firehose JSON/gzip batch, base64-decodes each record, and feeds it into the existing OTLP metrics decode/split/write path; it authenticates the `X-Amz-Firehose-Access-Key` header against the same API-key keyring and returns the Firehose ack response shape so failed batches retry and spill to S3 rather than dropping data (#1299)
  * Fix the still-unreleased Firehose metrics route (#1299) decoding each record as a single unframed protobuf message: CloudWatch Metric Streams actually pack one-or-more length-delimited `ExportMetricsServiceRequest` messages per record, so a batch of more than trivial size failed with errors like `invalid wire type value: 7`; decode and write each message as it's found instead, so a later malformed message can't discard already-decoded, already-written messages earlier in the same record (#1381)
  * Fix the still-unreleased Firehose metrics route (#1299) collapsing every CloudWatch Metric Stream onto one degenerate `process_id`: partition each matching resource into one synthetic `ResourceMetrics` per CloudWatch namespace (`AWS/RDS`, `AWS/ECS`, …, falling back to `AWS/Unknown` when a metric carries no `Namespace` datapoint attribute), with `exe` set to the namespace and `service.instance.id` set to the exporter ARN so different accounts/regions never collide; only newly ingested deliveries get per-namespace `process_id`/`exe` values under this scheme — the rewrite happens at ingestion time, so already-ingested CloudWatch Metric Stream blocks keep their old degenerate identity and re-materialization does not backfill it (#1387)
  * Add a `POST /ingestion/otlp/v1/logs/firehose/cloudwatch` route decoding the CloudWatch Logs subscription-filter delivery format, so RDS/ECS/Lambda logs can reach micromegas as CloudWatch Logs → subscription filter → Firehose → micromegas with no intermediate consumer: each per-record payload (gzip-compressed JSON, one `logGroup`/`logStream`/`owner` plus a batch of `logEvents`) is synthesized into an `ExportLogsServiceRequest` and fed into the existing OTLP logs decode/split/write path unchanged, reusing the Firehose envelope/auth machinery from #1299; per-record and aggregate-per-batch gunzip size caps guard against decompression bombs, and events with a negative timestamp are rejected rather than silently accepted (#1300)
* **Native / Blender:**
  * Add a dev-only **Keep Alive** add-on preference to the Blender extension: on disable the live telemetry session is parked on a `sys` attribute instead of shut down, and the next enable in the same process reuses it (same `session_id`), so the add-on no longer goes inactive on every re-enable — the native layer can only `mm_init` once per process, which tooling that re-enables the add-on on each launch (the VS Code Blender-Dev extension) hits every debug session; a parked session is reused even if the preference is later turned off, since no second one can be created, and the `atexit` flush hook is re-pointed at the current module instance on every `register()` so it stays attached to the namespace holding the live session across a reload that purges `sys.modules`
  * Capture redo-panel (F9) parameter edits in the Blender extension: Blender re-executes the same operator in place — same `wmOperator`, new parameters — and fires `undo_post` rather than `redo_post`, so the operator-history pointer diff saw an already-known entry and the edit was logged as a plain "undo" with the original parameters. The newest entry's formatted message is now baselined whenever the newest entry changes and diffed on `undo_post`/`redo_post`; a mismatch is logged to a new `blender.action_redo` target (and counted in `blender.action_captured`) in place of the "undo"/"redo" lifecycle log, since the underlying fact is a parameter change. Macro operators are skipped — their sub-operator values are unreachable from a stored history entry, so there is nothing to diff — and the baseline is deliberately not refreshed while the newest pointer is unchanged, so a poll landing between the edit and `undo_post` cannot mask it (#1365)

## v0.27.0 - 2026-07-12

* **Analytics:**
  * Move `ThreadSpansView::jit_update` off Postgres onto DataFusion views: replace the unprunable `blocks` full-table scan (TSC frequency estimation) and the `streams`/`processes` PK reads with `find_stream_from_view`/`find_process_with_latest_timing` against the DataFusion `streams`/`processes` views, and delete the now-dead `make_time_converter_from_db`/`find_stream` Postgres helpers (#1244)
* **Tests:**
  * Delete the orphaned, never-compiled `rust/http-gateway/src/config.rs` (duplicate of `HeaderForwardingConfig` in `public::servers::http_gateway`) and its dead tests; move three `tracing` crate inline `#[cfg(test)]` modules (`time.rs`, `logs/events.rs`, `string_id.rs`) into the crate's `tests/` folder per convention; mark `ingestion`'s `readiness.rs` live-dependency test `#[ignore]` instead of silently no-op-passing without a DB; replace a fixed 50ms sleep in `large_message_tests.rs` with a bounded TCP readiness poll (#1219)
  * Tighten timing/sleep-based tests: replace a meaningless fixed sleep in `cron_loop_drains` with a task-started signal, swap the random sleeps in `async_span_tests.rs` for fixed 1ms ones (dropping the `rand` dev-dependency), and correct the `thread_park_test` / `TracingRuntimeExt` docs to describe flush-on-thread-stop rather than a nonexistent `on_thread_park` callback (#1252)
* **Tracing:**
  * Harden the Rust telemetry sink's HTTP transport: priority queues with graded byte-budget dropping, lazy stream metadata (no `insert_stream` for idle/short-lived streams), per-priority retry tuning, and in-flight request gating with concurrent sends, porting the Unreal sink's resilience under backpressure and network flakiness to Rust (#1217)
* **Caching:**
  * Add a standalone range-aware S3 read cache (`micromegas-object-cache-srv`) backed by local SSD (Foyer RAM+disk): new `micromegas-object-cache` crate (cache engine + client `ObjectStore` layer) and `object-cache-srv` HTTP binary, wired into `BlobStorage`; clients fall back to the direct store on miss/error. Adds an `object-cache.Dockerfile`, MinIO-backed local test scripts, and admin docs (#1122)
  * Store `FoyerBackend` cache values as `Bytes` instead of `Vec<u8>` to avoid full-block copies on every RAM-tier read hit and fill (#1195)
  * Accept a list of allowed key prefixes in `object-cache-srv` (comma-separated `MICROMEGAS_OBJECT_CACHE_PREFIX` or repeated `--prefix`) so it can serve both `blobs/` and `views/`; the server now fails closed with no prefixes configured, requiring an explicit `--allow-all-prefixes` dev opt-out (#1204)
  * Install a byte weighter on `FoyerBackend`'s RAM tier so `ram_mb` bounds resident bytes instead of entry count, fixing an OOM risk under sustained load (#1207)
  * Rework the `object-cache` read path: single-flight fetch coalescing, a priority budget that lets demand reads jump ahead of prefetch, and a memory bound on in-flight prefetch; fixes a panic-leaking-a-held-permit bug, a batch-promotion race, and a permit undercount found in review (#1203)
  * Add `POST /prefetch` to `object-cache-srv` plus a `CacheClientStore::prefetch` client method, activating the prefetch-priority fill path from #1203: a bounded, load-shedding queue drives fills through the existing scheduler and returns `202 Accepted` immediately; prefetch fills are now admitted to the SSD tier only (never RAM), and a hit-path block-length guard heals cache entries poisoned by an undersized caller-supplied object size (#1198)
  * Stream `POST /prefetch` ingestion as NDJSON (one `PrefetchItem` per line) instead of buffering a whole JSON body, removing the request-body size ceiling: items are parsed incrementally and enqueued as they arrive, with a per-line cap the only remaining bound; deletes the `PrefetchRequest` wrapper type (#1218)
  * Stream `/ranges` and single-range GET responses from `object-cache-srv` over a bounded per-request window instead of assembling the whole response in memory: removes the 512 MiB `MAX_TOTAL_REQUESTED_BYTES` cap (and its `413`), reimplements `get_range`/`get_ranges` as collectors over a new `stream_ranges`, charges memory permits proportionally (capped at the window), adds a startup floor on `--memory-budget-mb` so a small budget fails fast instead of hanging, and adds a mid-stream direct-store fallback to the full-GET client path (#1222)
  * Warm the object cache at write time: once a freshly-materialized Parquet partition is durable in the origin store and committed to `lakehouse_partitions`, `write_partition_from_rows` fire-and-forget POSTs its key to `/prefetch` so the follow-up query's first read is a cache hit instead of a cold origin GET; adds a `PrefixPrefetch` adapter so warm keys match `PrefixStore`-derived read keys, surfaces the cache client's prefetch face and lake root on `DataLakeConnection` as a general `warm_object` primitive, and adds the `object_warm_requested` metric (#1201)
  * Add object-cache performance telemetry to locate bottlenecks and tune: per-stage latency spans + duration metrics (origin GET, backend read, fetch-permit wait), `prefix`/`class` dimensions on the hit-rate counters, and a saturation-monitor sampler for fetch-budget/mem-budget/prefetch-queue occupancy plus host NIC/SSD throughput; also counts all GET/`/ranges` outcomes (not just success) with a `status` dimension and fixes a double-counted `range_cache_size_backend_hit` on ranged GETs via new `*_with_size` read variants (#1206)
  * Make the foyer disk-cache write path tunable and observable: upgrade foyer 0.14 → 0.22, add `MICROMEGAS_OBJECT_CACHE_FLUSHERS` (default 2) and `MICROMEGAS_OBJECT_CACHE_WRITE_BUFFER_MB` (default 128) to size the flushers and write-buffer pool (submit-queue threshold pinned to 2× the buffer) so prefetch bursts stop overflowing foyer's submit queue, and emit `object_cache_foyer_disk_*` throughput gauges sourced from foyer's own `Statistics`, replacing the sysinfo-based `object_cache_ssd_*` gauges that read 0 in the deployed container (#1228)
  * Remove the postgres `partition_metadata` table (schema v6, `upgrade_v5_to_v6`): partition Parquet metadata is now read solely from the Parquet footer via the object-cache-backed reader, with the write-path `INSERT INTO partition_metadata` and the cleanup-path batch delete removed — fixing TOAST overhead on partition-retirement deletes and write-path overhead on every partition insert (#1121)
  * Add an in-process L1 read cache installed on the object stores DataFusion reads through (parquet partitions and static tables), caching hot row-group bytes for files of all sizes via the `object-cache` `RangeCache` core over a shared byte-bounded RAM backend (`MICROMEGAS_OBJECT_CACHE_L1_MB`, default 200, 0 disables); removes the old whole-file `FileCache`/`CachingReader` (and its `MICROMEGAS_FILE_CACHE_MB`/`MICROMEGAS_FILE_CACHE_MAX_FILE_MB` knobs) that only cached files ≤10 MB, and excludes raw-blob reads since those go through `BlobStorage` on a separate store reference (#1205)
  * Refactor `object-cache`'s 1290-line `range_cache.rs` into an `error`/`scheduler`/`fetch` submodule split and decompose the ~294-line `fetch_blocks` into cohesive helpers (`probe_blocks`, `register_missing`, `spawn_run_fetch`, `join_demand`/`join_prefetch`); pure behavior-preserving refactor with the public API and metrics unchanged (#1250)
  * Fix a lost-wakeup deadlock in `range_cache::InFlight::fulfill`: `watch::Sender::send` drops the result when the channel has zero receivers, and joiners subscribe lazily inside `join()`, so a fetch completing before any joiner subscribed would hang every later joiner forever; switch to `send_replace`, which stores the value unconditionally (#1259)
* **OTLP:**
  * Populate the `processes.username` column from `process.owner` (falling back through `process.user.name`, `process.real_user.name`, then `user.name`) and fold the resolved owner into the `process_id` derivation so processes that differ only by owning user get distinct ids; re-derivation stays under the existing `NS_OTEL_PROCESS_V1` namespace
* **Auth:**
  * Consolidate OIDC login-flow client construction into the `micromegas-auth` crate (new `oidc_client` module with a `DiscoveredProvider` that owns provider discovery + client building) so it lives in one place instead of being reimplemented in `analytics-web-srv`, and split the 909-line `analytics-web-srv/src/auth.rs` into focused `config`/`state`/`cookies`/`claims`/`handlers` submodules; behavior-preserving, public API unchanged (#1249)
* **Security:**
  * Gate the five mutating lakehouse SQL functions (`retire_partitions`, `materialize_partitions`, `regenerate_partitions`, `retire_partition_by_file`, `retire_partition_by_metadata`) on the authenticated caller's admin status: thread `is_admin` across the tower `AuthService` gRPC boundary via a new `x-auth-is-admin` header (mirroring the existing `x-auth-subject`/`x-allow-delegation` pattern), and only register these functions on a `SessionContext` when the caller is an admin. **Breaking behavior change**: any authenticated FlightSQL caller — including static API keys, which can never be admin — previously could call these functions; now only OIDC callers matched against `MICROMEGAS_ADMINS`/`MICROMEGAS_ANALYTICS_ADMINS` can, and everyone else gets "function not found". `--disable-auth` (local dev/monolith) continues to treat every caller as admin. **Breaking API change**: `register_lakehouse_functions`, `register_functions`, `make_session_context`, and `query` in the published `micromegas` crate (`rust/analytics/src/lakehouse/query.rs`) all gain a required `is_admin: bool` parameter (#1377)
  * Fix Dependabot alerts: bump `golang.org/x/net` to 0.55.0 in the Grafana plugin backend and `joserfc` to 1.7.2 in the Python client (#338, #337)
  * Upgrade `opentelemetry-proto` to 0.32 to resolve GHSA-w9wp-h8wv-79jx; treat the new profiling-only string-interning fields (`*_strindex`) as absent on non-profiling OTLP signals, per the OTLP spec (#336)
  * Harden the transit block parsing path (`parser.rs`, `dyn_string.rs`, `serialize.rs`, `parsing.rs`) against malformed/truncated payloads: replace unchecked arithmetic, slicing, and raw-pointer reads with checked variants that return `Err` instead of panicking or triggering UB; add a choke-point error log on parse failure and extensive corrupt-input regression tests (#1192)
* **Build:**
  * Add pre-merge supply-chain gates to Rust CI: `cargo audit` (RustSec advisory scan) and `cargo deny check licenses bans sources` (license/bans/sources policy), run against both the main workspace and the excluded `datafusion-wasm` tree; bump `crossbeam-epoch` and `quinn-proto` to clear advisories and document the remaining `rsa`/`quick-xml` ignores (#1246)
  * Add `dev_worker.py --size` to report the runner container and cache volume sizes, and make `--cleanup` also delete the cache volume
  * Fix `build_blender_plugin.py` artifact lookup when `CARGO_TARGET_DIR` is set; add `x86_64-pc-windows-gnu` to the pinned toolchain so the Windows cross-target installs automatically on fresh checkouts
  * Add Docker Hub publish pipeline for release: `build_docker_images.py` gains an arm64 buildx `--push` path and an `--all-arches` flag that builds both architectures in one run, with `--push` independently controlling publishing; sync the release runbook with a both-arch Docker Images phase (#1165)
  * Make `build.py` the single canonical generator for committed datafusion-wasm bindings; prune known wasm-pack leftovers from the output dir after each build so the copy loop is self-healing; document that `wasm-pack build` must not be run into the output dir (#1169)
  * Fix `capi-release`: route both build legs through the dev-worker runner (mold + mingw-w64 in the image), cross-compile the Windows artifacts on the Linux runner, and add the cross target to the pinned `1.96.0` toolchain the build actually uses (#1175)
* **Dependencies:**
  * Update DataFusion to 54.0 and rebuild the datafusion-wasm bindings
  * Migrate the internal proc-macro crates (`micromegas-proc-macros`, `micromegas-tracing-proc-macros`, `micromegas-derive-transit`) from `syn` 1.0 to 2.0 and bump the workspace pin, dropping the duplicate `syn 1.0.109` from the build; macro behavior unchanged (#1253)
* **Native / Blender:**
  * Add `micromegas-capi` C ABI crate (`cdylib` + `staticlib`) exposing init/shutdown, log, and int/float metric FFI over the existing Rust telemetry producer stack; ships a hand-authored C header and 8 smoke tests (#1160)
  * Add Blender Python extension (`blender/micromegas_blender/`) in Blender 4.2+ Extension format with `blender_manifest.toml`; captures user actions (modal recorder, `bpy.msgbus`, `bpy.app.handlers`), performance metrics, and process fingerprint via ctypes binding to `libmicromegas_capi` (#1160)
  * Add crash harvester: on startup scans for prior `*.crash.txt` files, claims each atomically via rename to prevent duplicate uploads, and ships as a CRITICAL log keyed to the prior session fingerprint (#1160)
  * Add `blender-extension` CI workflow: builds the cdylib for Linux and Windows, packages the extension zip, and uploads artifacts; add `capi-release` workflow: builds and publishes versioned `micromegas-capi` binaries (#1160)
  * Add docs for native C ABI integration (`mkdocs/docs/native/`) and the Blender add-on (`mkdocs/docs/blender/`) (#1160)
  * Make Blender per-process memory cross-platform and periodically sampled, and fix the misleading `blender.eval_ms` metric (renamed, no longer spans the file-load boundary) (#1168)
  * Expand the Blender process fingerprint and add Python exception capture via `sys.excepthook` plus semantic user-action capture, closing root-cause-analysis telemetry gaps (#1168)
  * Log add-on version and git commit hash on registration for easier triaging of field reports
  * Fix operator-history overflow gaps: drain the ring on every discrete input event via the recorder modal callback (per-keystroke cadence), lower the timer backstop from 1 s to 0.1 s, and set ring capacity to the correct Blender hard-cap of 32; adds `blender.action_gap` and `blender.action_captured` metrics (#1181)
  * Replace positional `bl_idname` string-diffing of the operator-history ring with per-entry object identity (`op.as_pointer()`) tracking, making new-entry detection exact and eliminating the false-gap WARN and duplicate action-log storm on repeating operator histories; gap detection reduces to a true full-ring turnover (#1181)
* **Docs:**
  * Add a caching architecture page describing the tiered L1/L2 object read path, and show the tiered read cache in the overview architecture diagram (#1238)
  * Add per-service admin guides for the ingestion, FlightSQL, and maintenance-daemon services
  * Refocus the object-cache admin guide on configuration, correcting the cache-warming and cache-client framing
  * Fill README gaps (web app, maintenance daemon, C ABI/Blender, monolith, cache) and fix code-verified inaccuracies across the docs (schema/config references, removed UDFs, binary names)

## June 2026 - v0.26.0

* **Security:**
  * Fix 24 Dependabot alerts: bump undici to ≥6.27.0 (resolves to 8.5.0) across all yarn workspaces (analytics-web-app, welcome, doc/intro-micromegas, doc/notebooks, doc/unified-observability-for-games, root) (#335, #334, #333, #332, #331, #330, #329, #328, #327, #326, #325, #324, #323, #322, #321, #320, #319, #318, #317, #316, #315, #314, #313, #312)
  * Fix 20 Dependabot alerts: bump protobufjs (≥7.6.3), tar (7.5.16), js-yaml (4.2.0), @babel/core (≥7.29.6), ws (8.21.0) across all npm workspaces; bump vite (≥7.3.5) in doc workspaces; bump cryptography (49.0.0) in Python; bump chi (5.2.4) in Grafana backend (#311, #310, #309, #308, #307, #306, #305, #304, #303, #302, #301, #300, #299, #298, #297, #296, #295, #294, #293, #292)
* **Tracing:**
  * Extend `#[micromegas_main]` with optional arguments (`ctrlc_handling`, `local_sink_enabled`, `local_sink_max_level`, `install_log_capture`, `system_metrics`, `telemetry_url`, `api_key`) so callers can configure the `TelemetryGuardBuilder` inline; invalid arguments now emit span-anchored `compile_error!` diagnostics instead of panicking
  * Add image streams: instrumented applications can send screenshots or other images as telemetry via `send_image()`; images are queryable via the `images` SQL table with a `data Binary` column holding raw bytes
* **Unreal:**
  * Add image stream support to the Unreal telemetry sink: wire `ImageStream`/`ImageBlock` into `Dispatch`, `EventSink`, and `HttpEventSink`; add `telemetry.screenshot` console command that captures the game viewport (or editor viewport via `WITH_EDITOR`) as a PNG and sends it as a telemetry image; gate capture with `telemetry.images.enable` CVar
  * Replace manual HTTP retry with `FHttpRetrySystem` (exponential backoff, per-priority retry budgets); add four priority queues (Metadata/Logs/Metrics/Traces) with configurable soft/hard byte-cap drop logic; gate concurrent uploads via `telemetry.max_in_flight_requests` CVar (#56, #43)
  * Add idle-aware spike sampling: suppress spike recording after `telemetry.sampling.interaction_timeout` seconds of no user input; add periodic heartbeat captures every `telemetry.sampling.heartbeat_interval` seconds of active time
  * Emit `TimeSinceLastInput` metric from `FSlateApplication` each frame, with a `BootTime` fallback so the value reads from boot instead of full uptime when no input has occurred yet
  * Replace `volatile` members in `HttpEventSink` with `std::atomic`; remove `QueueSize` and `RequestShutdown` volatile fields (#43)
  * Defer stream-metadata HTTP requests until the first block is enqueued so streams that never produce data generate no spurious uploads; fix `CurrentWorld` weak-ptr to survive same-name map reloads (PIE restart, same-level travel)
  * Emit per-tick camera and player metrics on non-dedicated-server builds: player position (X/Y/Z cm), camera position (X/Y/Z cm), camera orientation (Pitch/Yaw/Roll deg, FOV deg)
* **Deployment:**
  * Retire `telemetry-admin`'s four one-off subcommands (`materialize-partitions`, `retire-partitions`, `delete-old-data`, `delete-expired-temp` — superseded by the `materialize_partitions()`/`retire_partitions()` SQL functions and the daemon's own automatic hourly retention task) and rename the crate/binary to `telemetry-maintenance-srv`, dropping the now-single-mode `crond` subcommand so the binary just runs the daemon; the Docker image is renamed `micromegas-admin` → `micromegas-maintenance` (**breaking change for deployments** pulling the published image). Adds a configurable retention horizon via `--retention-days` / `MICROMEGAS_RETENTION_DAYS` (default 90, threaded through both the standalone daemon and the monolith's maintenance role) to replace the manual `delete-old-data` horizon. Note: the public crate's `daemon()` signature changed with a new `retention_days` parameter (#1266)
  * Add `linux/arm64` cross-compilation support to all production Dockerfiles and build script; builder stages pin to `$BUILDPLATFORM` and install the `aarch64-linux-gnu` cross toolchain so ARM64 images build natively without QEMU; `build_docker_images.py` gains an `--arm64` flag that drives `docker buildx build --platform linux/arm64 --load`; inline the wasm-builder stage to avoid buildx image-store isolation issues
  * Add `micromegas-monolith` single-process binary that runs ingestion, FlightSQL, maintenance, and web in one Tokio runtime sharing a single data-lake connection; includes Docker image, docker-compose stack, `--monolith` start-script mode, per-role auth, and role selection via `--roles` / `MICROMEGAS_MONOLITH_ROLES` (#1139)
* **Performance:**
  * Switch all production service binaries to jemalloc (`tikv-jemallocator`) as the global allocator; reduces allocation latency and memory fragmentation under multi-threaded workloads (#1129)
* **Services:**
  * Add deep `/ready` readiness probe to `telemetry-ingestion-srv`, `flight-sql-srv` (via sidecar HTTP listener on `--health-listen-addr`), and `analytics-web-srv`; each probe verifies its hard dependencies (PostgreSQL pool ping, blob storage list) with a 1s success cache and returns 503 when any dependency is unhealthy so ALBs can drain individual bad tasks during Aurora failover or object-store outages (#1038)
  * Add SIGTERM-driven graceful shutdown to `telemetry-ingestion-srv`, `flight-sql-srv`, `analytics-web-srv`, and `telemetry-admin crond`; in-flight requests, queries, and cron tasks drain within a configurable grace period (default 25s, `--shutdown-grace-period-seconds` or `MICROMEGAS_SHUTDOWN_GRACE_PERIOD_SECONDS`) instead of being killed on ECS task replacement. Note: the `daemon()` and `run_tasks_forever` signatures in the public crate changed (#1037)
  * Add optional `color` column to swimlane cells; per-segment colors support packed RGBA u32, `#rrggbb`/`#rrggbbaa` strings, and 4-byte binary values; falls back to the default theme color when absent (#1127)
  * Add optional `label` column to swimlane cells; labels render as truncated text inside each bar and appear in a hover tooltip alongside the lane name and time range
  * Route `formatDuration` through `formatTimeValue` so flamegraph tooltips show minutes, hours, and days for long spans instead of capping at seconds
  * Add per-row colors, user-selectable series colors, and reference line threshold indicators to XYChart; reference lines support named labels, units, dashed/solid style, and per-line color (#1043)
  * Accept orthographic GLB cameras in the map renderer; GLBs with an embedded `OrthographicCamera` no longer trigger the contract-error banner (#1145)
  * Add a per-Map-cell **Camera** setting with `perspective` (default) and `orthographic` modes; orthographic fits the camera to the projected map silhouette and maps Q/E to zoom (#1065)
  * Add a hover tooltip preview for map markers that renders the cell's `detailTemplate` as a small floating panel following the cursor, with a per-cell show/hide option (#1080)
  * Add `format_value(value, unit)` template function for adaptive unit formatting in Markdown templates (Map detail panel, Markdown cells, and table column overrides); surface unresolved-arg and unresolved-macro warnings via a banner (Map/Markdown) or column-header icon (tables) (#1086)
  * Rework map cell keyboard controls onto a single camera-relative orthonormal basis (A/D strafe, W/S up/down, Q/E forward/back) so key pairs no longer collapse onto the same direction at high camera tilt; radial zoom stays on Ctrl+wheel
  * Fix table column-override memo keying on fresh-per-render objects, causing `evaluateTemplate` to re-run every render; key on a content hash of only template-referenced inputs instead (#1092)
  * Route Map detail-panel `$column` macros through the evaluator's raw row + column-types channel so bare references carry their Arrow `DataType`; timestamps format as RFC3339 and `format_value()` receives full-precision raw values instead of pre-stringified ones (#1091)
  * Size non-fixed log columns (including `msg`) to their longest formatted value on the current page, capped and truncated with full text on hover; `msg` is no longer a special case
  * Add resizable columns to the log cell via draggable inline dividers; drag to pin a column width, right-click a divider to reset to auto or reset all, "Reset widths" button in bottom bar; pinned widths persist in cell options (#1130)
  * Add one-click copy icon to log rows; appears on hover, copies tab-separated row text to clipboard, briefly shows a green checkmark on success (#1131)
  * Add an `image` notebook cell that queries the `images` view and displays results as a navigable carousel; the `format` column is used as the image MIME type and decode failures surface a meaningful error message
  * Fix the data source selector silently rewriting persisted notebook config during render; out-of-scope `$var` references and deleted sources now display as-is (marked unavailable) instead of being switched to the default
  * Fix notebooks losing their `$var` data source when a cell is edited inside a group; group-sibling datasource variables are now valid selector options
* **Analytics:**
  * Accept runtime scalar expressions (CTEs, subqueries, CROSS JOIN columns) as `make_histogram` bounds; literal bounds continue to validate eagerly; NULL histogram rows propagate correctly through all consumers (#1135)
  * Replace SELECT+DELETE pairs in `delete_expired_blocks_batch` with atomic `DELETE … RETURNING` queries; eliminates double-counting and phantom deletes under concurrent writers (#1116)
  * Batch `retire_expired_partitions` to bound memory and transaction size on large backlogs; uses `DELETE … RETURNING` with a row-tuple subquery, matching the established batch pattern (#1111)
  * Prevent phantom empty partitions and detached writer tasks when span builds fail (crossing spans, net-span tree errors, merge stream errors); poison the write channel so `insert_partition` is skipped and the original error propagates to the query; surface full anyhow error chain at the DataFusion boundary
  * Batch `delete_expired_temporary_files` to avoid unbounded SQL/S3 operations on large `temporary_files` tables; add per-file `debug!`-level audit logging (#1108, #1109)
* **OTLP Ingestion:**
  * Backfill `observed_time_unix_nano` at ingestion time for OTLP log records where both timestamp fields are zero; block ID is derived from the pre-mutation bytes to preserve retry idempotency (#1123)
  * Accept `Content-Type: application/json` on all three OTLP/HTTP routes in addition to `application/x-protobuf`; response encoding mirrors the request; enables AWS EventBridge API Destinations to POST directly without a Lambda translation layer (#1115)
* **Build:**
  * Fix `cargo doc --workspace --all-features` hiding the `micromegas-perfetto` library modules by re-keying proto regeneration on the `MICROMEGAS_REGEN_PROTOS` env var instead of the `protogen` feature (#1079)
* **Refactoring:**
  * Remove unused map cell back-compat for the legacy `/maps/` `mapUrl` prefix and `markerColor`/`markerSize` scalar fallbacks; the feature is new so no saved config carries these shapes (#1077)
  * Split four high-complexity web-app files (PerformanceAnalysisPage, FlameGraphCell, MapViewer, XYChart) into pure-logic modules, THREE.js scene helpers, and thin React shells, with unit tests for the extracted logic; behavior unchanged (#1089)
  * Unify template and SQL macro value lookup behind a shared `resolveMacro` helper so both engines route every shape (`time`, `cellRow`, `selected`, `rowCol`, `varCol`, `var`) through one lookup; no behavior change (#1088)
* **Security:**
  * Bump react-router to 6.30.4 and @remix-run/router to 1.23.3 to fix open redirect CVE
  * Bump qs to 6.15.2 and js-cookie to 3.0.7 to fix Dependabot alerts (#256, #257)
  * Upgrade `dompurify` to 3.4.11 (prototype pollution CVE) and `@opentelemetry/core` to 2.8.0
* **Dependencies:**
  * Update DataFusion to 53.1 and rebuild the datafusion-wasm bindings (#1090)

## May 2026 - v0.25.0

* **HTTP Gateway:**
  * Add `GET /gateway/health` liveness endpoint for load balancer probes (#994)
* **OTLP Ingestion:**
  * Add native OTLP/HTTP ingestion for logs, metrics, and traces at `/ingestion/otlp/v1/{logs,metrics,traces}`; resource → `process_id` synthesis via stable UUIDv5; per-block format dispatch on a new `streams.format` column (schema v4)
  * Add `otel_logs_block_processor` (→ `log_entries`) and `otel_metrics_block_processor` (Sum/Gauge → `measures`)
  * Add `otel_spans` JIT view (per-process) materializing OTel spans with `trace_id`/`span_id` as `FixedSizeBinary`
* **Analytics:**
  * Add `net_spans` JIT view materializing Connection/Object/Property/RPC bandwidth spans with cumulative bit offsets
  * Add `rgba(r, g, b, a)` and `lerp_color(c1, c2, t)` scalar UDFs for building packed RGBA `u32` colors from SQL (#1062)
  * Add `color_scale(name, t, alpha)` scalar UDF for sampling built-in perceptually-uniform color scales (viridis, magma, plasma, inferno, cividis, turbo); returns packed RGBA `u32` and replaces the blue→red `lerp_color` pattern with one accessible call (#1069)
  * Add `bin_center(coord, cell_size)` scalar UDF for snapping coordinates to the centers of zero-centered bins; composes into 2D heatmap grids via `GROUP BY bin_center(x, cs), bin_center(y, cs)` (#1068)
  * Add `lerp(a, b, t)` and `unlerp(a, b, x)` scalar math UDFs for 1D remapping; `lerp(c, d, unlerp(a, b, x))` is the canonical `[a,b] → [c,d]` remap (#1083)
* **Web App:**
  * Extend flame graph cell to render bit-axis spans for `net_spans` and add bit-unit support to XYChart
  * Apply adaptive scaling to `bits/s` and `bytes/s` chart axes
  * Fix multi-series line chart rendering only points when series have sparsely-aligned X values
  * Resolve macros in chart series labels and chart cell titles
  * Defer notebook markdown cell render until sequential execution reaches it, preventing stale macro output on first paint (#1023)
  * Add map notebook cell rendering spatial events on a GLB model (#1033)
  * Convert map cell to native UE coordinates (Z-up) with GLB-embedded perspective camera, `MM_ambient_light`, and Neutral tone mapping; replace "Fit to data" with GLB-camera-seeded Reset (#1036)
  * Polish map interaction: cursor-anchored wheel zoom, right-mouse-drag orbit re-anchor, fix marker overlay visibility/picking, surface GLB contract errors in-cell (#1036)
  * Remove map cell `groundSnap` and `heightOffset` options; markers render at their native `(x, y, z)` coordinates
  * Simplify map navigation: WASD flies on hover (no right-click hold), Z resets view; remove right-scroll speed control, Q/E vertical movement, Shift boost, middle-mouse pan, and the speed/event-count overlays
  * Add `--remote-backend` flag to `start_analytics_web.py` for hybrid local-frontend + remote-backend setup (#1033)
  * Silence jsdom `navigation not implemented` console.error in AuthGuard test (#1047)
  * Upgrade to React 19.2, `@react-three/fiber` 9, `@react-three/drei` 10, `@testing-library/react` 16, and `lucide-react` 1.x; drop now-unused `react-reconciler` resolution (#1034)
  * Gate notebook auto-execution on the WASM engine being ready, eliminating an abort race that surfaced as an unhandled DOMException in Firefox (#1034)
  * Pin map `LoadingIndicator` to viewport center so drei's `<Html>` doesn't write a `NaN` transform before the camera is initialized (#1034)
  * Silence upstream `THREE.Clock`, `MouseEvent.mozPressure`, and `MouseEvent.mozInputSource` deprecation warnings emitted by R3F 9 / three 0.183 in dev (#1034)
  * Move map GLB assets and catalog off the static `ServeDir` to an object-store-backed `/api/maps/{catalog,blob}` (cookie-auth-gated, streaming `Body::from_stream`, pass-through `Content-Encoding`); `MICROMEGAS_MAPS_OBJECT_STORE_URI` configures the prefix, `start_analytics_web.py` derives a sane local-dev default
  * Add admin-only Maps management UI at `/admin/maps` with upload (server-side gzip, `.gz`-suffix storage), delete, drag-and-drop, and replace-confirm flow; per-route 256 MiB upload cap configurable via `MICROMEGAS_MAPS_MAX_UPLOAD_BYTES`; scope GLB load failures to the map cell with a retry affordance (#1050)
  * Replace map cell's hard-coded event detail panel with an authorable Markdown template rendered through macro substitution; relax the query contract so `time` is optional and every column is addressable as `$column`; publish map-cell selection to `cellSelections` for cross-cell `$mapcell.selected.col` references (#1053)
  * Refactor map cell to keep SQL results in Arrow Table format through the render path: replace eager `MapEvent[]` materialization with an `Overlay` struct (positions `Float32Array` + table) and on-demand row materialization; split InstancedMesh layout from selection/hover diff so selection changes touch O(1) instances; reject non-finite x/y/z at build time to avoid `InstancedMesh` bounding-sphere poisoning (#1035)
  * Generalize map cell to primitive overlays with shape dispatch (sphere/box), per-instance RGBA via an `instanceColorRGBA` attribute, and column-or-scalar bindings per visual channel (size, scaleX/Y/Z, color); color column accepts integer (packed RGBA u32), `#rrggbb[aa]` string, or 4-byte binary (DataFusion's `0xrrggbbaa` literal); fix SyntaxEditor cursor/overlay alignment in SQL mode (#1055)
  * Enlarge SQL query editors across the app: screen-level editors to 384px and per-cell editors to 240px; trim the flame chart cell description for new-cell dialog consistency
  * Refine map navigation: WASD pans horizontally (no elevation), add Q/E keyboard zoom, suppress the browser context menu when a right-drag releases off-canvas, and auto-size the event detail panel with the title rendered via the Markdown template
  * Remove admin-oriented `MICROMEGAS_MAPS_OBJECT_STORE_URI` / `.glb` drop hint from the map cell editor (admins use the Maps management UI)
  * Resolve `$var` macros in map cell primitive scalar bindings (size, scaleX/Y/Z, color); editor swaps the legacy number/color inputs for a text field with local-draft commit (Enter/blur commits, Escape cancels)
  * Require Ctrl/Cmd modifier for map cell wheel zoom so plain wheel scrolls the surrounding notebook page
  * Fix map cell event detail popup close button overlapping the first line of the markdown template (#1067)
  * Eliminate map cell first-paint camera snap by gating marker/camera mount until the GLB payload has propagated and promoting payload-extract/camera-seed effects to `useLayoutEffect`; restore cursor on controller/marker unmount; rename `UnrealCameraController` to `MapCameraController` (#1075)
* **Python:**
  * Switch `bulk_ingest` to accept `pyarrow.Table` directly for native pass-through of struct/list/binary columns
  * Resolve CLI connection settings from `~/.micromegas/config.json` with env-var override (#1033)
* **Unreal Engine:**
  * Add net trace support with connection/object/property/RPC scopes, runtime verbosity gating, and empty-scope elision
  * Enable crash reporting on Linux and skip telemetry flush during malloc-crash to avoid deadlock
* **Docs:**
  * Add Unreal net trace instrumentation guide and engine recipe
  * Document flame graph notebook cell type
  * Document `net_spans` view in schema reference and network-tracing guide
  * Add "An Introduction to Micromegas" presentation
  * Document map notebook cell type and hybrid local-frontend setup (#1033)
  * Document CLI configuration file and authentication settings (#1033)
  * Document `rgba` and `lerp_color` color functions in the SQL functions reference (#1062)
  * Document `color_scale` perceptual colormap function in the SQL functions reference (#1069)
  * Document `bin_center` binning function in the SQL functions reference (#1068)
  * Document `lerp` and `unlerp` math functions in the SQL functions reference (#1083)
  * Rework notebook cell-types reference: alphabetize sections, expand Map cell with channel mapping and color encodings, fact-check defaults/levels/bindings against the implementation, and move admin/maps content to the Admin → Web App page
* **Security:**
  * Bump rustls-webpki, rand, and uuid to fix Dependabot alerts (#210-213)
  * Bump postcss and uuid to fix Dependabot alerts (#214-221)
  * Bump urllib3, fast-uri, @babel/plugin-transform-modules-systemjs, and apache/thrift to fix Dependabot alerts (#222, #225-229)
  * Bump mermaid to 11.15.0 in `doc/intro-micromegas` and `doc/high-frequency-observability` to fix Dependabot alerts (#230-241)
  * Bump protobufjs, @protobufjs/utf8, brace-expansion, and authlib to fix Dependabot alerts (#178, #242-250)
* **Repo:**
  * Migrate Yarn 1 (Classic) to Yarn 4 (Berry) via corepack across all six yarn projects; corepack-only delivery, `nodeLinker: node-modules`; CI/Docker/build scripts updated; clean lock + zero install warnings (#1008)
* **CI:**
  * Switch dev-worker runners to ephemeral mode (one job per container, unique name per run) with build caches persisted in a named Docker volume (cargo, yarn, go, playwright); bake Go 1.25 into the runner image and skip `setup-go` on dev-worker; drop the now-redundant `--clear-cache` / `--rotate-cache` / `--rotate-at` flags

## April 2026 - v0.24.0

* **Analytics:**
  * Add `parse_block` table UDF for generic block inspection with transit-to-JSONB conversion (#1001)
* **CLI:**
  * Add unified diff output to `micromegas-screens plan` and `apply` for updated screens (#998)
* **Bug Fixes:**
  * Fix byteLength crash on 0-row Arrow tables in notebook status text (#1000)
  * Fix notebook variable URL desync on rapid updates and datasource reverting to default on change (#1003)
  * Fix flamechart WASD zoom continuing after key release in Chrome (#1013)
* **Web App:**
  * Show flamechart span duration in nanoseconds when below 1 microsecond
* **Docs:**
  * Add `parse_block` to functions reference (#1002)
  * Blog post: From Observability to Candor (#996)
* **Repo:**
  * Normalize line endings to LF for existing files (#997)
* **Dependencies:**
  * Update DataFusion to 52.5 (#1009)
  * Update pyarrow to ^23.0.0 (#1006)
* **Security:**
  * Update `rand` to 0.9 to fix unsoundness in `rand::rng()` (Dependabot #201, #202)
  * Bump protobufjs, protocol-buffers-schema, authlib, and rustls-webpki to fix Dependabot alerts (#205-209)
  * Bump dompurify to 3.4.0 to fix Dependabot alerts (#203, #204)
  * Bump pytest, cryptography, and opentelemetry-go SDK to fix Dependabot alerts (#198, #199, #200)
  * Bump vite, lodash, and lodash-es to fix Dependabot alerts
  * Bump serialize-javascript, handlebars, cryptography, and Pygments to fix Dependabot alerts
  * Bump picomatch, brace-expansion, yaml, and requests to fix Dependabot alerts

## March 2026 - v0.23.0

* **CI:**
  * Add container-based self-hosted runner infrastructure for faster CI builds on developer workstations
  * Add check-runner workflow to dynamically route builds between dev-worker and GitHub-hosted runners
  * Add nightly cache rotation with `--rotate-at` flag for built-in scheduling
* **Docs:**
  * Document JSONPath filter predicate syntax (SQL/JSON path) in query guide (#979)
* **Analytics:**
  * Add CSV table provider and `StaticTablesConfigurator` for auto-discovery of CSV/JSON tables via `MICROMEGAS_STATIC_TABLES_URL` (#946)
  * Downgrade extensionless file warning to debug in `StaticTablesConfigurator` (#954)
  * Scope merge session context to insert time range to reduce memory during compaction (#963)
  * Add `FlightSqlServer` builder to eliminate boilerplate when assembling a FlightSQL server (#955)
  * Add `Send + Sync` bounds to `MergerMaker` type alias for async view factories (#972)
  * Add `LakehouseContext::from_env()` convenience constructor to deduplicate initialization (#969)
  * Allow `jsonb_each` to accept arbitrary expression arguments like `jsonb_parse(...)` (#978)
  * Add `jsonb_array_elements` UDTF to unnest JSONB arrays into rows (#977)
  * Add `jsonb_array_length` scalar UDF for counting JSONB array elements (#976)
  * Allow `expand_histogram` to accept expression arguments and Dictionary-wrapped scalars (#983)
* **Ingestion:**
  * Add `WebIngestionService::from_env()` convenience constructor to deduplicate initialization (#973)
* **Object Storage:**
  * Use `parse_url_opts` to honor environment variable credentials for S3/GCS/Azure (#948)
  * Fix env var credential parsing by lowercasing keys for `object_store` case sensitivity bug (#951)
* **CLI:**
  * Add `--file` option to `micromegas-query` for reading SQL from a file or stdin (#941)
* **Web App:**
  * Add `managed_by` column to screens table for source-control tracking
  * Show warning banner when editing a source-controlled screen
  * Add Bearer token authentication to analytics-web-srv (alongside cookie auth)
  * Add `initialFrom`/`initialTo` options to flamegraph cell for pre-zoomed initial view
  * Resolve `$cell.selected.column` macros in table column overrides (#975)
  * Format timestamps in cell selection display panel
  * Show actual cell names in override editor help text
  * Halt notebook execution when a cell is blocked on a missing selection
* **CLI:**
  * Add `micromegas-screens` tool for managing screens as code with Terraform-inspired workflow (init, import, pull, plan, apply, list)
  * Add HTTP client (`WebClient`) for analytics-web-srv REST API
* **Claude Code Plugin:**
  * Add shareable micromegas plugin with micromegas-query skill for querying observability data via SQL
  * Extract pr, design, design-review, and branch-review skills to standalone dev-skills plugin (#960)
* **Repo:**
  * Add `.gitattributes` for LF line endings, ignore `.claude/settings.local.json` (#962)
* **Dependencies:**
  * Update lz4_flex to 0.12.1 to fix memory information leak vulnerability
  * Fix 6 dependabot security alerts: flatted 3.4.2, rustls-webpki 0.103.10, grpc-go 1.79.3
  * Fix rustls-webpki alert in datafusion-wasm Cargo.lock
  * Update DataFusion to 52.4.0 (#964)

## March 2026 - v0.22.0

* **Tracing:**
  * Fix async span depth across yield points and spawn boundaries with `SpanContextFuture` (#917)
  * Fix async span depth inconsistency by capturing depth at future creation time instead of poll time (#927)
* **Telemetry:**
  * Add default system properties (exe, username, hostname, CPU, memory, OS) to process metadata (#380)
* **Database Migration (REQUIRED before upgrade):**
  * Run the following SQL on your database before deploying this version:
    ```sql
    CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS processes_process_id_unique ON processes(process_id);
    CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS streams_stream_id_unique ON streams(stream_id);
    CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS blocks_block_id_unique ON blocks(block_id);
    ```
  * If you have existing duplicate rows, clean them up first using `delete_duplicate_blocks()`, `delete_duplicate_streams()`, `delete_duplicate_processes()` or manual deduplication
  * Validate concurrent indexes before completing v2→v3 migration to prevent invalid indexes from being silently accepted (#911)
* **Dependencies:**
  * Fix CVE-2026-32141: upgrade flatted to 3.4.1 to fix unbounded recursion DoS
  * Update tower-http from 0.5 to 0.6
  * Update quinn-proto to 0.11.14 to fix unauthenticated DoS vulnerability
  * Update DataFusion to 52.3 (#907)
  * Update DataFusion to 52.2 and remove LimitPushdown workaround (#882)
* **Notebook Enhancements:**
  * Add flame graph cell type with Three.js WebGL rendering for Perfetto-style CPU trace visualization (#917)
  * Rename $begin/$end macros to $from/$to and fix expansion in column overrides (#914)
  * Add Insert cell above/below options to cell context menu
  * Add Download CSV menu option for notebook cells (#900)
  * Add row selection to table cells with `$cell.selected.column` macros for interactive drill-down (#915)
  * Add cell result row references in macros — `$cell[N].column` syntax for chaining queries (#898)
  * Add Alt+PageUp/PageDown keyboard navigation for notebook cells (#909)
  * Close notebook cell editor on ESC key (#919)
  * Fix available variables showing orphaned URL params and unscoped variables
  * Fix timestamp values rendering as raw integers in macro expansion (#908, #910)
* **Analytics Web App:**
  * Fix config diff modal not showing non-cell changes (e.g. refresh interval) for notebooks
  * Fix time range picker calendar dismissing on every interaction (#930)
  * Fix page-level and swimlane cell scrollbar issues with flex layout
  * Fix swimlane TimeAxis tick alignment to prevent horizontal overflow
  * Add Ctrl+S keyboard shortcut to save screens
  * Fix crash on first notebook cell execution when loading state has no data field
  * Add auto-refresh feature with configurable intervals and execution-aware spinner (#892)
  * Add SQL editor improvements: horizontal scrolling, format button, and Ctrl+Enter to run
  * Replace sql-formatter-plus with sql-formatter (zero-dependency, actively maintained)
  * Fix concurrent token refresh race in authenticatedFetch
  * Remove local_query screen type (#871)
* **Security:**
  * Fix minimatch ReDoS vulnerabilities across JS packages (Dependabot alerts #116–#123)
  * Bump serialize-javascript to 7.0.3 and OpenTelemetry SDK to 1.40.0 (Dependabot security alerts)
  * Bump authlib to 1.6.9, dompurify to 3.2.7+, and immutable to 5.1.5 (Dependabot security alerts)
  * Bump dompurify to 3.3.2, jest to 30, and @types/jest to 30 (Dependabot security alerts)
  * Migrate window.location.href assignments to navigateTo() wrapper for jsdom 26 compatibility
* **Analytics:**
  * Add `jsonb_path_query_first` and `jsonb_path_query` UDFs for JSONPath traversal of JSONB columns (#920)
  * Extend `jsonb_each` to support array inputs (#920)
  * Bump jsonb dependency from 0.5.3 to 0.5.5 (#920)
  * Add `process_spans(process_id, types)` table function for cross-thread and async span analysis (#917)
  * Add `hash` column to `async_events` view for scope identification (#917)
  * Extract `get_process_thread_list` as shared utility for process stream discovery
* **Query CLI:**
  * Require `--begin` flag and default `--end` to now to prevent accidental full-range queries
  * Add `--all` flag to explicitly query the entire time range without time filtering
* **Documentation:**
  * Migrate all documentation URLs from madesroches.github.io to micromegas.info
  * Remove dead troubleshooting link from Grafana plugin README
* **Build:**
  * Add wasm-opt optimization step to WASM Docker builder for smaller binaries
  * Fix wasm-opt corrupting externref table by upgrading binaryen and enabling reference-types
  * Add SHA256 checksum verification for binaryen download in WASM builder

## February 2026 - v0.21.0
* **Security:**
  * Bump rollup to 4.59.0 to fix CVE (arbitrary file write) across all JS packages
* **Build:**
  * Gate server-only dependencies behind `server` feature flag on `micromegas-telemetry` and `micromegas` crates (#855)
* **DataFusion Extensions:**
  * Add `jsonb_each` table function to expand JSONB objects into key-value rows (#860)
* **Notebook Enhancements:**
  * Switch cell selection from single-click to double-click and add Edit cell to context menus
  * Move SERIES_COLORS to shared chart-constants module
  * Fix health check URL in start_analytics_web.py
  * Add column override support to transposed table cells (#868)
  * Add process info notebook with analysis links and clamped time range
  * Fix unsaved edits detection by snapshotting baseline JSON on edit start
  * Add copy and edit buttons to notebook view source panel
  * Add row hiding to transposed table cell via right-click context menu
  * Sort cell types alphabetically in Add Cell dialog
  * Stabilize hook references to prevent unnecessary re-renders in notebook components
* **Documentation:**
  * Add screenshots to notebook documentation pages
  * Fix rustdoc warnings in analytics crate
  * Add Interactive Notebooks for Observability presentation with Reveal.js + Micromegas brand theme
  * Add notebook documentation: cell types reference, variables, execution model, and web app overview
  * Reorganize mkdocs nav into Analytics Web App, Integrations, and Operations sections
  * Add notebook datasource option to dropdown variable cells (#861)
  * Add transposed table cell type for key-value layout of SQL results
  * Replace single-letter cell type icons with Lucide components
  * Stop markdown link clicks from opening cell editor in notebook tables
* **Analytics Web App:**
  * Add LZ4 compression to Arrow IPC streams for dramatically smaller network transfers
  * Raise FlightSQL client max decoding message size to 100MB
* **Python CLI:**
  * Add `micromegas-query` and `micromegas-logout` as installed CLI entry points via `pip install micromegas`
  * Move CLI module into package directory for proper entry point resolution
  * Remove legacy single-purpose CLI scripts superseded by general-purpose `micromegas-query`
* **Analytics Web App:**
  * Adapt chart Y-axis scale to visible series when hiding in multi-series charts (#836)
  * Hide Y-axis when all series for a unit are hidden
* **WASM UDF Extensions:**
  * Extract JSONB and histogram UDFs into shared `micromegas-datafusion-extensions` crate
  * Register extension UDFs in WASM query engine for browser-side SQL parity with server
  * Add WASM integration tests for JSONB and histogram UDFs
* **Tracing:**
  * Fix empty-string backward compatibility in opt_uuid_from_string for 0.20.0 clients (#850)
* **Security:**
  * Upgrade ajv from 6.12.6 to 6.14.0 to fix CVE-2025-69873 (dependabot #109)
  * Fix minimatch ReDoS vulnerability via resolution override to v10.x (dependabot alerts #104, #105)
  * Migrate Grafana eslint config to native flat config, remove @eslint/eslintrc dependency
* **Notebook Cross-Cell Queries:**
  * Add notebook-local query support via WASM DataFusion engine (#815)
  * Cells with `dataSource: 'notebook'` execute SQL in-browser against other cells' results
  * Remote cell results automatically registered in WASM context for cross-cell references
  * Add `execute_and_register` and `deregister_table` methods to WASM engine
  * Add "Notebook (local)" option to data source dropdown in cell editor
  * Live download progress (rows/bytes) and execution time in cell title bars
  * Extract shared `serialize_to_ipc` helper in datafusion-wasm crate
* **WASM Tracing (#817):**
  * Implement WASM support for micromegas-tracing and telemetry-sink
  * Add tracing to WASM query engine and fix wasm dispatch init
  * Add Send + Sync bounds to EventSink trait
  * Set default-features = false for micromegas-tracing in workspace
  * Remove unused feature flags from tracing and telemetry-sink
  * Enable SHOW TABLES in WASM query engine
* **Horizontal Group Cell:**
  * Implement horizontal group (hg) cell type for side-by-side cell layout in notebooks (#821)
  * Pass variable value/onValueChange to hg children, render combobox inline
  * Add DataSourceField to hg child editors
  * Show running status and spinner on hg cells
  * Add Run button for HG child cells in editor panel
  * Fix drag-into-group oscillation, drag-out, and reorder preview
  * Fix HG editor rename, child fetch progress, and child variable auto-run
  * Fix HG child content click to open child editor directly
  * Clean up hg child state on removal and migrate on rename
  * Add tests for HorizontalGroupCell (36 tests)
* **Multi-Query Chart Cells (#749):**
  * Implement multi-query chart cells with per-query data sources
  * Refactor CellState.data from Table | null to Table[] for multi-result support
  * Fix multi-series chart Y-axis unit auto-scaling
  * Fix chart cells defaulting to notebook instead of global data source
  * Fix single-query chart not using configured unit and label
  * Fix tooltip XSS and deduplicate SERIES_COLORS
  * Portal chart tooltips to document.body to prevent overflow clipping
  * Stop chart header clicks from selecting cell, fix clipped tooltips
* **Compact Notebook UI:**
  * Implement compact borderless notebook UI with minimal visual chrome
  * Add fade-on-idle behavior for cell metadata with three-state fade machine
  * Add always-visible loading spinner to cell status area
  * Unify status text placement between groups and cells
  * Restyle pagination bar to centered minimal design
  * Restyle notebook tables to minimal lines with prominent header
  * Restyle notebook log cells to match compact table design
  * Fix pagination overlap, hardcoded dark colors, and hidden HG status
  * Fix selection indicator layout shift with always-present border
  * Fix fade-on-idle reveal for fast cells and add hover delay
* **Notebook Enhancements:**
  * Add notebook pagination and per-cell auto-run (#823)
  * Add reference table cell type for inline CSV data (#824, #827)
  * Add duplicate cell action to notebook (#834)
  * Resolve relative time ranges at cell execution time in all renderers
  * Fix refresh not updating relative time spans
  * Fix data source race condition overwriting config on load
  * Stop column header sort click from opening cell editor (#829)
  * Deprecate non-notebook screen types for creation
* **Log Cell Improvements:**
  * Make log cell display resilient to unexpected or missing columns (#826)
  * Extract renderLogColumn to shared log-utils module
  * Preserve SQL column order in log renderers instead of reordering known columns
  * Hide data source selector for reference table cells
* **Code Quality:**
  * Extract 6 hooks from NotebookRenderer (1154 → 736 lines)
  * Extract buildCellRendererProps to unify cell rendering prop assembly
  * Consolidate AddCellModal and AddChildModal into shared component
  * Deduplicate datasourceVariables computation in NotebookRenderer
* **Security:**
  * Fix CVE-2025-69873: upgrade ajv 8.17.1 to 8.18.0
  * Bump qs to 6.14.2 to fix CVE-2026-2391
* **CI/Build:**
  * Run native and WASM CI checks in parallel
  * Fix wasm-builder to copy full Rust workspace for path dependencies
  * Add build-skip workflow for required check satisfaction
  * Install clang in wasm-builder Docker image for native dependency compilation
* **Documentation:**
  * Fix homepage and documentation URLs in Cargo.toml
  * Update README roadmap with v0.20.0 and current notebook focus
* **Scripts:**
  * Add --release flag to start_services.py and run binaries directly

## February 2026 - v0.20.0
* **Client-Side WASM Query Execution:**
  * Add `local_query` screen type running DataFusion SQL in the browser via WebAssembly (#806, #807, #808, #810)
  * Progressive row count and byte size display during source query fetch
  * Auto-run checkbox for local query execution on text changes
  * Rename datafusion-wasm to micromegas-datafusion-wasm with CI integration
  * Shared WASM builder Dockerfile stage for Docker builds
* **Configurable Data Sources:**
  * Add configurable data sources for analytics web app (#793)
  * Per-screen and per-cell data source selection (#794)
  * Datasource variable type for notebook data source selection (#800)
  * Data source selector on Processes, ProcessMetrics, and ProcessLog pages
  * Protected default data source from deletion and flag removal
* **Notebook Enhancements:**
  * Add Perfetto export cell type for notebooks (#771)
  * Add expression variable type for adaptive time_bin_duration (#782)
  * Add swimlane notebook cell type for visualizing concurrent events (#769)
  * Add drag-to-zoom time range selection to notebook charts (#768)
  * Add property timeline notebook cell type (#766, #762)
  * Re-execute notebook cells when time range changes (#768)
  * Add query guide links to SQL editor cells (#751)
  * Move variable cell input to title bar to reduce vertical space (#779)
  * Move save buttons to title bar, add config diff modal (#780)
  * Extract useExposeSaveRef hook, remove duplicate SaveFooter from renderers (#780)
  * Add zoom in/out buttons to time range control (#804)
* **Query & Data Features:**
  * Add multi-column query variables with $variable.column syntax (#753)
  * Add table URL support with column overrides (#750)
  * Add unit formatting system for charts (#755)
  * Allow hiding columns via right-click context menu (#790)
* **Client-Side Perfetto Trace Generation:**
  * Replace generate_trace endpoint with client-side trace fetching (#784)
  * Add gzip compression to analytics-web-srv endpoints (#784)
  * Add abort signal support for trace downloads
* **Performance Optimizations:**
  * Add parquet file content cache to reduce object storage reads (#757, #758)
  * Parallelize JIT for Perfetto trace thread span generation (#759, #772)
  * Implement pipelined query planning for Perfetto trace generation (#759)
* **Unreal Engine:**
  * Support 32-bit and 64-bit metrics (#786)
* **Dependencies:**
  * Update DataFusion to 52.1 and Arrow/Parquet to 57.2 (#756), Arrow to 57.3
* **Security:**
  * Update bytes crate to 1.11.1 to fix CVE-2026-25541 (#767)
  * Upgrade jsonwebtoken to 10.3 to fix type confusion vulnerability (#760)
  * Fix dependabot security alerts: protobuf and time (#787)
  * Bump cryptography from 46.0.3 to 46.0.5 (#801)
* **Analytics Web App:**
  * Add welcome landing page for madesroches.github.io/micromegas (#785)
  * Hide admin icon in sidebar for non-admin users (#802)
  * Add Process Details link to PivotButton navigation (#777)
  * Remove process list from available screen types (#791)
  * Fix Perfetto trace generation missing data source parameter (#805)
* **Documentation:**
  * Document delete_duplicate SQL functions and reorganize admin docs (#752)
  * Link documentation site in crate READMEs and PyPI metadata (#798)
  * Add GoatCounter analytics to all public pages (#796)
* **Code Quality:**
  * Remove old perf_report task folder
  * Remove column name transformation in process list tables (#744)
  * Refactor analytics-web-srv main.rs into focused functions
  * Delete orphaned queries.rs

## January 2026 - v0.19.0
* **User-Defined Screens:**
  * Add user-defined screens feature (#707)
  * Add table screen type with generic SQL viewer (#726)
  * Add notebook screen type with multi-cell layout (#728)
  * Refactor notebook cells to follow Open-Closed Principle (#729)
  * Notebook OCP refactoring and URL variable synchronization (#730)
  * Add syntax highlighting to notebook cell editors (#731)
  * Delta-based URL handling for notebook variables and time range (#734)
  * Add copy/paste support for time ranges (#735)
  * Decouple URL param ownership from ScreenPage to renderers (#736)
  * Add admin section with export/import screens (#737)
* **Data Integrity:**
  * Add delete_duplicate_streams and delete_duplicate_processes UDFs (#721)
* **Analytics & Query Features:**
  * Add expand_histogram table function and bar chart toggle (#720)
  * Unify chart and property timeline queries (#732)
  * Enable dictionary encoding preservation for web app (#727)
* **Analytics Web App:**
  * MVC view state refactor and XYChart generalization (#718)
  * Migrate remaining pages to useScreenConfig and remove useTimeRange (#719)
  * Add dynamic page titles (#712)
  * Consolidate API endpoints under /api prefix (#711)
  * Disable source maps in production builds (#710)
  * Fix blank page on hard refresh for deep URLs (#713)
* **Infrastructure:**
  * Add micromegas_app database creation to service startup (#705)
* **Security:**
  * Fix lodash prototype pollution vulnerability (CVE-2025-13465) (#725)
  * Fix Dependabot alert #91: upgrade diff to 8.0.3 (#708)
  * Fix dependabot alerts for grafana plugin dependencies (#704)
  * Fix 4 dependabot security alerts (#703)
* **Documentation:**
  * Add plans for unified metrics query and dictionary preservation (#724)
  * Add notebook screen design and generalized metrics chart plan (#716)
  * Update changelog and readme with unreleased changes (#722)
  * Update unified observability presentation slides (#706)
  * Add unified observability presentation link (#702)

## January 2026 - v0.18.0
* **Reliability & Data Integrity:**
  * Add periodic duplicate block cleanup to maintenance daemon (#700)
  * Prevent duplicate insertion for blocks, streams, and processes (#691)
  * Add delete_duplicate_blocks UDF (#689)
  * Fix queue_size going negative on timeout in http_event_sink (#699)
* **Ingestion & Client:**
  * Add proper HTTP error codes and client retry logic (#696)
* **Analytics & Query Features:**
  * Implement Arrow IPC streaming for query API (#685)
  * Enable SHOW TABLES and information_schema support (#687)
  * Add global LRU metadata cache for partition metadata (#674)
  * Add jsonb_object_keys UDF (#673)
  * Add property timeline feature for metrics visualization (#684)
* **Tracing & Instrumentation:**
  * Improve #[span_fn] rustdoc documentation (#676)
  * Fix async span parenting and add spawn_with_context helper (#675)
  * Add thread block parsing trace and tooling config (#686)
* **Analytics Web App:**
  * Migrate from Next.js to Vite for dynamic base path support (#667)
  * Pivot split button for process view navigation (#682)
  * Metrics chart scaling and time units improvements (#681)
  * Auto-refresh auth token on 401 API responses (#680)
  * Improve process info navigation and cleanup trace screen (#669)
  * Fix custom queries being reset when filters change (#670)
* **Python CLI:**
  * HTTPS URI support and executable scripts (#683)
* **Unreal Engine:**
  * Add more metrics and process info to telemetry plugin (#672)
* **Security:**
  * Fix urllib3 decompression bomb vulnerability (CVE-2026-21441) (#695)
  * Fix security vulnerabilities in qs and rsa dependencies (#693)
  * Fix esbuild security vulnerability (GHSA-67mh-4wv8-2f99) (#671)

## December 2025 - v0.17.0
 * **Analytics Web App Major Rework:**
   * Complete UI redesign with dark theme and Micromegas branding (#621, #622, #623)
   * Add Grafana-style time range picker with relative and absolute time support (#631)
   * Add performance analysis screen with thread coverage timeline (#642, #643)
   * Add Perfetto trace integration with split button for browser/download (#660, #661)
   * Add process metrics screen with time-series charting (#639)
   * Add process properties display panel (#634)
   * Add multi-word search to process list and log screens (#632, #633)
   * Allow custom limit values in process log view (#627, #628)
   * Improve time column formatting in process logs (#624)
   * Pass time range through process navigation links (#636)
   * Add schema documentation links to SQL panels (#635)
   * UX improvements and polish (#645, #647)
 * **Deployment & Configuration:**
   * Add per-service Docker images and modernize build scripts (#637, #649)
   * Add BASE_PATH and MICROMEGAS_PORT env vars for reverse proxy deployments (#650, #651, #654, #656, #658, #659)
 * **Unreal Engine:**
   * Add scalability and VSync context to telemetry (#625)
   * Document API key authentication (#629)
 * **Security & Bug Fixes:**
   * Fix CVE-2025-66478: Update Next.js to 15.5.7 (#626)
   * Fix urllib3 security vulnerabilities and OIDC token validation bug (#641)
   * Fix UTF-8 user attribution headers with percent-encoding (#638)
   * Handle empty MICROMEGAS_TELEMETRY_URL environment variable (#644)
 * **Documentation:**
   * Fix documentation dark mode readability (#648)
 * **Code Quality:**
   * Fix rustdoc bare URL warnings in auth crate (#630)

## November 2025 - v0.16.0
 * Released [version 0.16.0](https://crates.io/crates/micromegas)
 * **New: HTTP Gateway:**
   * Add HTTP Gateway with Authentication and Security Features (#597)
 * **Analytics Web App:**
   * Add OIDC authentication to analytics web app (#596)
 * **Authentication:**
   * Fix ID token expiration and add multi-provider OIDC support (#608)
   * Fix OIDC authentication and token refresh issues (#590)
 * **Analytics & Query Features:**
   * Optimize JSONB UDFs for dictionary-encoded column support (#593)
   * Fix timestamp binding in retire_partition_by_metadata UDF (#606)
   * Handle empty incompatible partitions and fix thrift buffer sizing (#602)
 * **Grafana Plugin:**
   * Fix Grafana plugin packaging and document release process (#601)
   * Fix secureJsonData undefined error and rename plugin to Micromegas FlightSQL (#603)
 * **Security & Dependencies:**
   * Fix js-yaml prototype pollution vulnerability (CVE-2025-64718) (#592)
   * Upgrade DataFusion from version 50.2.0 to 51.0.0 (#598)
   * Fix LIMIT pushdown in all TableProvider implementations (#600)
 * **Documentation:**
   * Document auth_provider parameter and deprecate headers in Python API (#595)
 * **Build & CI:**
   * Enable Claude to submit PR reviews and issue comments (#605)
   * Claude PR Assistant workflow (#604)

## November 2025 - v0.15.0
 * Released [version 0.15.0](https://crates.io/crates/micromegas)
 * **New: Authentication Framework (micromegas-auth crate):**
   * Add authentication framework with OIDC and API key support (#546)
   * Implement OIDC authentication for Rust services and Python client (#548)
   * Add OIDC authentication support to CLI tools (#549)
   * Add OAuth 2.0 client credentials support for service accounts (#552)
   * Add HTTP authentication to ingestion service (#551)
   * Unified JWKS architecture for service accounts (#547)
   * Refactor OIDC connection to library module (#588)
 * **Grafana Plugin (v0.15.0 - First release from main repo):**
   * Integrate Grafana FlightSQL datasource plugin into main repository (#554)
   * Implement OAuth 2.0 authentication for Grafana plugin (#564)
   * Add variable query editor and datasource migration tools (#585)
   * Rename Grafana plugin to follow official naming guidelines (#583)
   * Implement CI/CD pipeline for Grafana plugin (#558)
   * Update Grafana plugin SDK to 11.6.7 and fix security vulnerabilities (#555)
   * Fix 28 Dependabot security vulnerabilities (#556)
 * **Authentication & Security:**
   * Rework AuthProvider to use request validation (#571)
   * Refactor MultiAuthProvider for extensibility (#569)
   * Add client IP logging to server observability (#566)
   * Comprehensive authentication documentation in admin guide (#550)
 * **Unreal Engine:**
   * Modernize Unreal telemetry sink module (#584)
 * **Server Enhancements:**
   * Add gRPC health check endpoint (#570)
 * **Build & CI:**
   * Fix CI linker crashes and improve build reliability (#572)
   * Fix documentation build by installing mold linker (#573)
 * **Documentation:**
   * Add build tools installation before build steps (#582)
   * Update build prerequisites (#581)
   * Add documentation links to all Rust crate READMEs (#578)
   * Update high-frequency observability presentation (#574)
   * Clean up presentation files and update docs to use yarn (#568)
   * Clean up task documentation and improve authentication docs (#567)
   * Consolidate and streamline Grafana and monorepo documentation (#559)
   * Update documentation links to use hosted docs and fix markdown formatting (#563)
   * Remove Grafana section from README (#560)
 * **Planning:**
   * Add plan for query variable time filter feature (#580)
   * Grafana plugin repository merge planning and Phase 1.1 completion (#553)

## October 2025 - v0.14.0
 * Released [version 0.14.0](https://crates.io/crates/micromegas)
 * **Performance & Storage Optimizations:**
   * Complete properties to dictionary-encoded JSONB migration (#521)
   * Properties writing optimization with ProcessMetadata and BinaryColumnAccessor (#522, #524)
 * **Analytics & Query Features:**
   * Add Dictionary<Int32, Binary> support to jsonb_format_json UDF (#536)
   * Add SessionConfigurator for custom table registration (#531)
   * Add file existence validation to json_table_provider (#532)
   * Enable property_get UDF to access JSONB columns (#520)
   * Add support for empty lakehouse partitions (#537)
 * **Bug Fixes & Reliability:**
   * Fix NULL value handling in SQL-Arrow bridge with integration tests (#541)
   * Fix null decoding error in list_partitions table function (#540)
   * Fix null decoding error for file_path in retire_partitions (#539)
 * **Documentation & Presentations:**
   * Add High-Frequency Observability presentation (OSACON 2025) (#527, #528, #529, #533)
   * Update presentation template to new Vite-based build (#525)
 * **Security & Dependencies:**
   * Update Vite to 7.1.11 to fix security vulnerabilities (#526, #542)
   * Update DataFusion and Arrow Flight dependencies (#519)
   * cargo update (#530)
 * **Code Quality:**
   * Fix rustdoc HTML tag warnings in analytics crate (#534)
 * **Future Work:**
   * Analytics Server Authentication Plan (#543)

## September 2025 - v0.13.0
 * Released [version 0.13.0](https://crates.io/crates/micromegas)
 * **Performance & Storage Optimizations:**
   * Dictionary encoding for properties columns with comprehensive UDF support (#506, #507, #508, #510, #511)
   * Properties to JSONB UDF for efficient storage and querying (#515)
   * Arrow string column accessor with full dictionary encoding support (#511)
   * Production performance analysis of dictionary encoding effectiveness (#508)
   * Fixed parquet metadata race conditions with separation strategy (#502, #504)
   * Optimized lakehouse partition queries by removing unnecessary file_metadata fetches (#499)
   * Scalability improvements for high-volume environments (#497, #498)
 * **Schema Evolution & Admin Features:**
   * Incompatible partition retirement feature for schema evolution (#512)
   * Enhanced error logging in CompletionTrackedStream (#503)
   * Improved PostgreSQL container management in development environment (#500)
 * **Monitoring & Analytics:**
   * Added `log_stats` SQL aggregation view for log analysis by severity and service (#495, #505)
   * Enhanced documentation with log_stats view in schema reference
 * **Code Quality & Development:**
   * Organized project documentation and completed task archival (#513, #514, #517)
   * Dictionary encoding analysis archived due to Parquet limitations (#516)

## September 2025
 * Released [version 0.12.0](https://crates.io/crates/micromegas)
 * **Major Features:**
   * Comprehensive async span tracing with `micromegas_main` proc macro (#451)
   * Named async span event tracking with improved API ergonomics (#475)
   * Async span depth tracking for performance analysis (#474)
   * Async trait tracing support in `span_fn` macro (#469)
   * Perfetto async spans support with trace generation (#485)
   * HTTP gateway for easier interoperability (#433, #435, #436)
   * JSONB support for flexible data structures (#409)
 * **Infrastructure & Performance:**
   * Consolidate Perfetto trace generation to use SQL-powered implementation (#489)
   * Query latency tracking and async span instrumentation optimization (#468)
   * Replace custom interning logic with `internment` crate (#430)
   * Optimize view_instance metadata requests (#450)
   * Convert all unit tests to in-memory recording (#472)
 * **Documentation & Developer Experience:**
   * Complete Python API documentation with comprehensive docstrings (#491)
   * Complete SQL functions documentation with all missing UDFs/UDAFs/UDTFs (#470)
   * Visual architecture diagrams in documentation (#462)
   * Unreal instrumentation documentation (#492)
   * Automated documentation publishing workflow (#444)
 * **Security & Dependencies:**
   * Fix CVE-2025-58160: Update tracing-subscriber to 0.3.20 (#490)
   * Update DataFusion, tokio and other dependencies (#429, #476)
   * Rust edition 2024 upgrade with unsafe operations fixes (#408)
 * **Web UI & Export:**
   * Export Perfetto traces from web UI (#482)
   * Analytics web app build fixes and documentation updates (#483)
 * **Cloud & Deployment:**
   * Docker deployment scripts (#422)
   * Amazon Linux setup script (#423)
   * Cloud environment configuration support (#426)
   * Configurable PostgreSQL port via MICROMEGAS_DB_PORT (#425)

## July 2025
 * Released [version 0.11.0](https://crates.io/crates/micromegas)
 * Working on http gateway for easier interoperability
 * Add export mechanism to view materialization to send data out as it is ingested

## June 2025
 * Released [version 0.10.0](https://crates.io/crates/micromegas)
 * Process properties in measures and log_entries
 * Better histogram support
 * Processes and streams views now contain all processes/streams updated in the requested time range - based on SqlBatchView.

## May 2025
 * Released [version 0.8.0](https://crates.io/crates/micromegas) and [version 0.9.0](https://crates.io/crates/micromegas)
 * Frame budget reporting
 * Histogram support with quantile estimation
 * Run seconds & minutes tasks in parallel in daemon
 * GetPayload user defined function
 * Add bulk ingestion API for replication

## April 2025
 * Released [version 0.7.0](https://crates.io/crates/micromegas)
 * Perfetto trace server
 * DataFusion memory budget
 * Memory optimizations
 * Fixed interning of property sets
 * More flexible trace macros

## March 2025
 * Released [version 0.5.0](https://crates.io/crates/micromegas)
 * Better perfetto support
 * New rust FlightSQL client
 * Unreal crash reporting

## February 2025
 * Released [version 0.4.0](https://crates.io/crates/micromegas)
 * Incremental data reduction using sql-defined views
 * System monitor thread
 * Added support for ARM (& macos)
 * Deleted analytics-srv and the custom http python client to connect to it
 
## January 2025
 * Released [version 0.3.0](https://crates.io/crates/micromegas)
 * New FlightSQL python API
   * Ready to replace analytics-srv with flight-sql-srv

## December 2024
 * [Grafana plugin](https://github.com/madesroches/micromegas/tree/main/grafana)
 * Released [version 0.2.3](https://crates.io/crates/micromegas)
 * Properties on measures & log entries available in SQL queries

## November 2024
Released [version 0.2.1](https://crates.io/crates/micromegas)

 * FlightSQL support
 * Measures and log entries can now be tagged with properties
   * Not yet available in SQL queries

## October 2024
Released [version 0.2.0](https://crates.io/crates/micromegas)

 * Unified the query interface
   * Using `view_instance` table function to materialize just-in-time process-specific views from within SQL
 * Updated python doc to reflect the new API: https://pypi.org/project/micromegas/

## September 2024
Released [version 0.1.9](https://crates.io/crates/micromegas)

 * Updating global views every second
 * Caching metadata (processes, streams & blocks) in the lakehouse & allow sql queries on them

## August 2024
Released [version 0.1.7](https://crates.io/crates/micromegas)

 * New global materialized views for logs & metrics of all processes
 * New daemon service to keep the views updated as data is ingested
 * New analytics API based on SQL powered by Apache DataFusion

## July 2024
Released [version 0.1.5](https://crates.io/crates/micromegas)

Unreal
 * Better reliability, retrying failed http requests
 * Spike detection

Maintenance
 * Delete old blocks, streams & processes using cron task

## June 2024
Released [version 0.1.4](https://crates.io/crates/micromegas)

Good enough for dogfooding :)

Unreal
 * Metrics publisher
 * FName scopes

Analytics
 * Metric queries
 * Convert cpu traces in perfetto format

## May 2024
Released [version 0.1.3](https://crates.io/crates/micromegas)

Better unreal engine instrumentation
  * new protocol
  * http request callbacks no longer binded to the main thread
  * custom authentication of requests

Analytics
  * query process metadata
  * query spans of a thread

## April 2024
Telemetry ingestion from rust & unreal are working :) 

Released [version 0.1.1](https://crates.io/crates/micromegas)

Not actually useful yet, I need to bring back the analytics service to a working state.

## January 2024
Starting anew. I'm extracting the tracing/telemetry/analytics code from https://github.com/legion-labs/legion to jumpstart the new project. If you are interested in collaborating, please reach out.