--- name: create-explorer description: Author or modify an Our World in Data explorer (multi-dimensional dashboard with dropdown selectors, published from ETL via `viz://explorer//latest/`). Trigger when the user wants to build a new explorer, add/remove views or dimensions on an existing one, change the explorer's chart text or selection defaults, or finish an explorer migration once the snapshot/garden/grapher chain is already in place. metadata: internal: true owner: lucasrodes --- # Creating an Explorer Explorers are OWID's multi-dimensional dashboards (e.g. `ourworldindata.org/explorers/food-prices`). They're authored as YAML in this repo and published by ETL at `viz://explorer//latest/`. This skill is the explorer-flavored sibling of `/create-chart`. They use the same engine (`paths.create_chart` for charts and multidims, `paths.create_explorer` for explorers) and the same YAML schema for `dimensions` / `views` / `definitions.common_views`. The differences are: | | Chart / multidim | Explorer | |---|---|---| | Channel | `viz://chart/...` | `viz://explorer/...` | | Step file location | `etl/steps/viz/chart//latest/` | `etl/steps/viz/explorer//latest/` | | PathFinder method | `paths.create_chart(...)` | `paths.create_explorer(...)` | | Top-level config block | `title:`, `default_selection:`, `default_dimensions:` | `config:` block carrying legacy explorer settings (`explorerTitle`, `explorerSubtitle`, `selection`, `subNavId`, `entityType`, …) | | Slug convention | underscores in file paths and `short_name` | underscores in file path, **hyphens** in URL slug and `short_name` argument | | Verification | preview URL on staging | preview URL on staging + diff against `owid-grapher/explorers/.explorer.tsv` | | Save call | `c.save()` | `c.save(tolerate_extra_indicators=True)` (upstream grapher datasets usually have more indicators than the explorer references) | If you're modifying an existing explorer (adjusting chart text, swapping a catalogPath, adding a dimension choice, reordering views), most of the deeper sections below don't apply — find the existing `.config.yml`, edit, run `etlr`, done. The full structure is documented below for new explorers and substantial reshapes. ## When to use this skill - After the `/migrate-explorer-to-etl` skill has produced (or already located) the upstream snapshot/meadow/garden/grapher chain, and now needs the explorer step. - For a brand-new explorer where the data is already in ETL (skip directly to step 1). - When porting an existing explorer's view layout (e.g. full-YAML → table-driven, or moving FAUST text from per-view YAML up into indicator metadata). ## Step 1 — Files & directories Every explorer is exactly two files plus a DAG entry: ``` etl/steps/viz/explorer//latest/ ├── .py # Python uses snake_case └── .config.yml ``` ```bash mkdir -p etl/steps/viz/explorer//latest ``` **Hyphens vs underscores** (recurring source of confusion): - The Python file path uses **underscores**: `food_footprints.py`, `crop_yields.py`. - The explorer slug used in the URL and the `short_name=` argument keeps **hyphens**: `food-footprints`, `crop-yields`. - `paths.create_explorer(short_name="")` — pass the hyphenated slug. ## Step 2 — Pick a construction style The two ends of the spectrum, plus everything in between: | | Full-YAML | Programmatic / table-driven | |---|---|---| | Where views live | Hand-listed in `.config.yml` under `views:` | Auto-expanded by `paths.create_explorer(tb=tb, ...)` from columns whose `m.dimensions` is set | | Where chart text (FAUST) lives | Per-view `view.config.{title, subtitle, note}` in YAML | `presentation.{title_public, grapher_config}` on each indicator's garden metadata; common defaults via `definitions.common.presentation.grapher_config` | | Where chart-level config lives (`hasMapTab`, `tab`, `yAxis`, `chartTypes`) | Per-view `view.config` | Indicator's `presentation.grapher_config` — single source of truth, same as for any standalone chart on that indicator | | Map color scale | `view.indicators.y[i].display.{colorScaleScheme, colorScaleNumericBins}` (semicolon-string form, explorer-flavored override) | `presentation.grapher_config.map.colorScale.{baseColorScheme, binningStrategy, customNumericValues}` (canonical grapher form, inherited at chart render time) | | Python step content | Trivial: `paths.create_explorer(config=config, short_name=...).save(...)` | Loops columns to set `m.dimensions`, optionally post-processes (`sort_choices`, `group_views`, per-view `display` tweaks) | It's a spectrum, not a switch. Mix freely: use table-driven for the bulk of views, hand-list a handful of bespoke ones; or stay full-YAML but still push title/subtitle for the single-indicator views into garden metadata to remove duplication. **Strong fit for table-driven:** - **Single-indicator views dominate.** Each view is a thin wrapper around one indicator → that indicator's metadata is the right home for chart text. Avoids duplication between explorer YAML and the equivalent standalone chart, and keeps both in sync forever. - **Many views (>20)** following the cartesian product of a few dimensions. Hand-listing them is repetitive; auto-expansion plus YAML dimensions is significantly less code. - **Upstream is a dimensional table** (one row per country/year × dim_a × dim_b × …) with one indicator. `create_explorer(tb=tb, indicator_names=..., dimensions=...)` matches this shape directly — model: `migration/latest/migration_flows.py`. - **Same indicators back standalone grapher charts.** Pushing FAUST upstream means explorer view and standalone chart inherit the same text — no drift over time. **Stick with full-YAML when:** - **Few views (<10)**, all bespoke (different chart types / data sources / hand-tuned text). - **Multi-indicator views dominate.** FAUST cannot live on any single indicator when a view shows multiple indicators — you have to write it explicitly per view in the explorer YAML (or build views via `c.group_views(...)`). - **Single-shot migration with no plan to maintain.** The duplication of full-YAML doesn't matter if no one will edit it again. ## Step 3 — The Python step ### Full-YAML variant ```python """.""" from etl.helpers import PathFinder paths = PathFinder(__file__) def run() -> None: config = paths.load_config() c = paths.create_explorer( config=config, short_name="", # explorer slug ) c.save(tolerate_extra_indicators=True) ``` `tolerate_extra_indicators=True` is the common case: the upstream grapher dataset usually carries more indicators than the explorer references, and without this flag `c.save()` errors on the unused ones. ### Table-driven variant ```python """.""" from etl.helpers import PathFinder paths = PathFinder(__file__) # Map column → dimension tuple. "na" is the conventional empty slot for conditional dimensions # (e.g. cost_metric is meaningful only when type=cost; affordability views set cost_metric="na"). COLUMN_DIMENSIONS: dict[str, dict[str, str]] = { "": {"dim1": "value_a1", "dim2": "value_a2"}, "": {"dim1": "value_b1", "dim2": "value_b2"}, # ... } def run() -> None: config = paths.load_config() ds = paths.load_dataset("") tb = ds.read("", load_data=False) # metadata only — faster, we don't need values for column, dims in COLUMN_DIMENSIONS.items(): tb[column].m.dimensions = dims tb[column].m.original_short_name = "" c = paths.create_explorer( config=config, tb=tb, indicator_names=[""], dimensions={ "dim1": ["value_a1", "value_b1", ...], # explicit choice order "dim2": ["value_a2", "value_b2", ...], }, # common_view_config={...}, # only if not in indicator metadata short_name="", ) # Optional post-processing — see "Post-processing" below. # c.sort_choices({"dim1": lambda x: sorted(x)}) # c.group_views([...]) c.save(tolerate_extra_indicators=True) ``` Key APIs (see `etl/viz/chart/core/expand.py` and `etl/viz/chart/core/create.py`): - `tb[col].m.dimensions: dict[str, str]` — required per column. Each entry says "this column represents the (dim1=value, dim2=value) cell." Columns without `m.dimensions` are ignored by the expander. - `tb[col].m.original_short_name: str` — the unifying indicator name. With `indicator_names=[that_name]` and a single name, the expander treats all N columns as one logical indicator with N dimension combinations and drops the auto-added "indicator" pseudo-dimension. - `dimensions=` accepts: - `None` → all dimensions found, arbitrary order. - `list[str]` → restricts and orders dimensions, all values shown. - `dict[str, list[str] | "*"]` → restricts and orders both dimensions and choices. Use `"*"` for "all values, arbitrary order." - `common_view_config=` is applied uniformly to every auto-expanded view. Use it for fields that are truly shared and don't live at indicator level. **Prefer indicator-level `presentation.grapher_config`** for anything that should also flow to standalone charts. ## Step 4 — The config YAML > **Always block style.** Mappings and lists in explorer config YAML must use block style — one key per line, list items on their own line under `-`. Never use flow style (`{ key: value, ... }` or `[a, b, c]`) even for tiny per-view `dimensions:` blocks. PR review on a 45-view file is unreadable when half the views collapse to a single flow line. The only exception is markdown links inside a quoted-scalar `subtitle:`/`note:` (those `[text](url)` brackets are content, not YAML structure). ```yaml config: # Explorer settings rows — keys map verbatim from the legacy TSV settings section. explorerTitle: ... explorerSubtitle: ... isPublished: true hasMapTab: false hideAlertBanner: true hideAnnotationFieldsInTitle: true entityType: country # or "food", "region", etc. thumbnail: https://assets.ourworldindata.org/uploads/... wpBlockId: "12345" subNavId: explorers subNavCurrentId: selection: - - pickerColumnSlugs: [] # an empty list is OK; non-empty must be block-style yAxisMin: 0 # ... definitions: # Shared config applied to all views. Use this list (with optional `dimensions:` # filter per entry) — NOT YAML anchors and `<<:` merge keys. The framework merges # entries at expansion time; per-view `config:` blocks override anything here. common_views: - config: type: DiscreteBar hasMapTab: false # Dimension-filtered overrides apply only to matching views: # - dimensions: # metric: share # config: # note: "Share values sum to 100%" dimensions: # one entry per dropdown / radio / checkbox the user toggles - slug: # e.g. "metric" name: # e.g. "Metric" presentation: type: dropdown # or radio / checkbox choices: - slug: name: "" - slug: name: "..." views: # one entry per (dim1=x, dim2=y, …) tuple - dimensions: : # ... indicators: y: - catalogPath:
# # short form — see "catalogPath — short forms accepted" below display: # per-view, per-indicator overrides colorScaleNumericBins: 0;1;2 colorScaleScheme: PuBu config: # Only per-view overrides here. Common stuff lives in definitions.common_views. # No `<<:` merge keys, no `&anchor`s. title: ... subtitle: ... type: # LineChart, DiscreteBar, "LineChart DiscreteBar", StackedArea, … hasMapTab: false minTime: 1990 yAxisMin: 0 ``` For table-driven explorers, `views:` should still be present but is typically `views: []` — the explorer JSON schema requires the key, and `create_explorer(tb=tb, ...)` populates the views at runtime. #### `catalogPath` — short forms accepted The `Indicator.is_a_valid_path` check (`etl/viz/chart/model/view.py:62`) accepts three forms; pick the shortest one that still unambiguously resolves: | Form | Example | When to use | |---|---|---| | `table#indicator` | `global_carbon_budget#emissions_total` | Default. Resolved against the explorer's DAG dependencies via `tables_by_name` — fine as long as no two dependencies expose a table with the same name. | | `dataset/table#indicator` | `global_carbon_budget/global_carbon_budget#emissions_total` | When two upstream datasets happen to expose tables with the same `short_name`. | | `grapher////
#` | `grapher/gcp/2025-11-13/global_carbon_budget/global_carbon_budget#emissions_total` | Only when you need to pin a specific dataset version *separate from the one in the DAG* — almost never the right form to write by hand. | Short forms are expanded at `c.save()` time by `Indicator.expand_path(tables_by_name)`. If the table name doesn't exist in any dependency it raises `Table name '' not found in dependency tables`; if multiple dependencies expose the same table name, it raises and asks you to disambiguate with the medium form. **Default to `table#indicator`** when authoring YAML. The full path is verbose, drifts when upstream versions bump, and is only needed for genuinely ambiguous cases. ### Top-level `config:` settings — the most common keys | Key | Type | Notes | |---|---|---| | `explorerTitle` | string | Page title above the explorer. | | `explorerSubtitle` | string | One-liner under the title. | | `isPublished` | bool | `true` to publish; `false` keeps it draft. | | `hasMapTab` | bool | Whether any view shows the map tab by default. | | `entityType` | string | `country` (default), or `food`, `region`, `species`, etc. — controls picker labels. | | `selection` | list[str] | Default selected entities. | | `pickerColumnSlugs` | list[str] | Picker columns shown alongside the entity name. | | `subNavId` | string | Almost always `explorers`. | | `subNavCurrentId` | string | The slug — appears as the active nav item. | | `wpBlockId` | string | WordPress block ID for embedding (legacy). Stringify even when numeric. | | `thumbnail` | string | URL of preview image. | | `hideAlertBanner` | bool | Suppress the OWID-wide banner. | | `hideAnnotationFieldsInTitle` | bool | Drop time/entity from auto-titles. | | `yAxisMin` | number/string | Default Y-axis floor. | | `yScaleToggle` | bool | Allow user to toggle linear/log. | | `originUrl` | string | Path back to the topic page (e.g. `/environmental-impacts-of-food`). | ### Dimension presentation types - `dropdown`: shown as `