# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Development Commands ```bash # Docker (recommended) docker-compose up --build # Start all services docker-compose up db # Start only PostgreSQL/PostGIS # Local development (venv in ./env — Python 3.12; the image is python:3.12-slim, Django 5.2 LTS) source env/bin/activate # Activate virtual environment pipenv install # Install dependencies (Pipfile — there is no requirements.txt) # Faster equivalent for a fresh venv: uv venv --python 3.12 env && pipenv requirements --dev > /tmp/r.txt # && uv pip install --python env/bin/python -r /tmp/r.txt # Branches from before change python312-django52-upgrade need a 3.9 venv (kept as ../Mapsurvey/env39). # Deprecation gate (run once per Django bump): python -W error::django.utils.deprecation.RemovedInDjango60Warning # -W error::DeprecationWarning:survey -W error::DeprecationWarning:mapsurvey manage.py test survey python manage.py migrate # Apply database migrations python manage.py runserver # Start development server (port 8000) python manage.py createsuperuser # Create admin user python manage.py collectstatic # Collect static files # Testing (requires running PostGIS on port 5434) ./run_tests.sh survey # Run all survey app tests ./run_tests.sh survey -v2 # Verbose output ./run_tests.sh survey.tests.test_public_stories # One capability's tests ./run_tests.sh survey.tests.test_smoke.SmokeTest # One test class ``` ## Parallel worktrees (port isolation) `run_dev.sh`, `run_tests.sh`, and `run_e2e.sh` derive every host port from a single `PORT_OFFSET` so multiple worktrees can run dev + tests at the same time without colliding. To set up a worktree: `cp .env.ports.example .env.ports` and pick a unique offset (keep the registry in that file current). Ports = base + offset (PostGIS `5434`, Redis `6379`, web `8000`). `COMPOSE_PROJECT_NAME` isolates the docker stack per worktree. Offset `0` reproduces the original ports. `.env.ports` is gitignored. ## Testing Tests use Django's built-in test framework with PostGIS. Django automatically creates a separate `test_mapsurvey` database. **Prerequisites**: none — `run_tests.sh` starts the PostGIS and Redis containers itself and waits for both. Redis is not optional: `settings.py` falls back to `redis://localhost:6379/1` when `REDIS_URL` is unset, so on a machine running several projects the suite would otherwise connect to whichever project owns that port and write into its database. `CACHES` sets `IGNORE_EXCEPTIONS`, so that goes wrong silently rather than failing loudly. **Full suite**: `./run_tests.sh survey billing api mcp newsletter`. One area: a label (`survey.tests.test_public_stories`, `survey.tests.test_public_stories.SeedStoryTest`, `api.tests.test_v1_data`, `billing`). **Run defaults (change `test-suite-speed`, #272)**: `run_tests.sh` always passes `--noinput` and runs `--parallel auto` (one worker per core) unless you pass `--parallel` yourself; `TEST_PARALLEL=1` for `pdb` or a readable traceback order. `KEEPDB=1` adds `--keepdb` — opt-in, because Django does not re-migrate kept parallel clones, so after a new migration the workers would test the old schema. `TEST_RUNNER` (`mapsurvey/test_runner.py`) applies to every `manage.py test`: MD5 password hasher, a temporary `MEDIA_ROOT` (tests never write into `mediafiles/`), and a Redis database per parallel worker (`1 + worker id`, capped at 14 workers) — django-redis `cache.clear()` is `FLUSHDB`, so a shared database would let one worker wipe another's rate-limit counters. `tblib` (dev package) is required: without it a failure inside a worker aborts the whole run with a `TypeError` and a cascade of unrelated `IntegrityError`s. `pypdf` (dev package) reads the printable-form PDFs back; `reportlab` writes them (both since change printable-form — a venv from before needs `uv pip install reportlab pypdf`). Views leave `translation.activate()` behind, so the runner resets the language to `LANGUAGE_CODE` before every test (wrapping `SimpleTestCase._pre_setup`). A test that only passes serially is relying on another test's state; fix the test, never pin the suite to one process. **Test location — one module per capability (change `split-survey-tests`, #273)**: the survey app's tests are the package `survey/tests/`, with `test_.py` for each folder under `openspec/specs/` (`-` → `_`) and shared fixtures in `survey/tests/helpers.py`. A change adds its tests to the module of the capability its spec delta touches; a NEW capability adds its own module, so parallel branches touch different files. `SpecTestLineageTest` (`test_test_suite.py`) fails when a module names no spec and is not in `UNMAPPED_MODULES` (with a reason), or a spec has no module and is not in `SPECS_WITHOUT_SURVEY_TESTS` — both lists may only shrink, and the stale-entry test enforces it. `api/tests/`, `billing/tests/` and `mcp/tests/` follow the same one-module-per-capability rule. **Writing tests**: Use `django.test.TestCase` and GIVEN/WHEN/THEN pattern for docstrings. ## Load testing `loadtest/lecture-burst.js` (k6) reproduces a lecture-hall burst — N students opening the same map survey at once. It does **not** reproduce locally (a dev machine is far faster than the 0.5 CPU Render Starter instance previews run on; production is on Standard, 1 CPU / 2 GB, since the 2026-09-15 memory-limit incident), so run it against a Render PR preview, never production. Previews are opt-in (`previews.generation: manual` in `render.yaml`): put `[render preview]` in the PR title to get one; it is deleted after a day without new commits. Seed the preview's empty database first with `python manage.py seed_loadtest_survey`. See `loadtest/README.md`. ## Architecture Overview This is a Django-based geospatial survey platform using PostGIS for storing geographic data (points, lines, polygons). ### Project Structure - `mapsurvey/` - Django project settings and root URL configuration - `survey/` - Main application with all business logic ### Core Data Model Hierarchy ``` Organization └── SurveyHeader (survey definition) ├── SurveySection (logical groupings with map position) │ ├── Question (supports 12+ input types including GIS) │ └── OptionGroup → OptionChoice (reusable choice sets) └── SurveySession (user's survey attempt) └── Answer (stores responses with GIS geometry fields) ``` ### Key Patterns **Dynamic Form Generation**: `SurveySectionAnswerForm` in `survey/forms.py` dynamically builds form fields based on question `input_type`. Each type maps to specific Django fields and custom Leaflet widgets for GIS input. **Question Types**: `text`, `text_line`, `number`, `choice`, `multichoice`, `range`, `rating`, `datetime`, `point`, `line`, `polygon`, `image`, `html`, `budget` **Creator rich text (`Question.subtext`, `SurveySection.subheading`, `SurveyHeader.thanks_html`)**: everything a creator writes in their own words is authored in Quill and rendered to respondents as markup. All of it is unbounded `TextField` and all of it passes `survey/html_sanitize.py` on the way in — `QuestionForm.clean()`, `SurveySectionForm.clean_subheading()`, the translation savers in `editor_views.py`, `serialization._import_rich_text()` (ZIP import, which is also how AI generation writes), and `editor_question_preview_live` so the preview equals what a save keeps. **A new way to write these fields must go through the same helper**; the allow-list is what stands between a creator and stored XSS on every respondent. Use `coerce_creator_html`, not `sanitize_creator_html`, anywhere a value might not be from an editor. These fields hold plain text from before the editors existed (old rows, old ZIPs, AI drafts), and `nh3` would read "takes <5 minutes" as an unknown tag and delete it; `coerce_` escapes what carries no creator markup and sanitizes what does. Migration `0056` did the same one-off pass over the rows already in the database. The **Formatted Text block (`html`)** is the extreme case: it collects nothing and its `subtext` IS the whole body, rendered `|safe` in `html_text.html`. `Question.name` for `html` and `image` is an editor-only label that never reaches the respondent (see the `question-subtext` spec). **Embedding (`survey/embedding.py`, change embed-surveys, #284)**: a published survey can be framed by another site. `check_survey_access` calls `mark_embeddable`, `SurveyEmbedMiddleware` turns the mark into `Content-Security-Policy: frame-ancestors …` (from `SurveyHeader.embed_allowed_hosts` on the canonical row; empty = `*`) and drops `X-Frame-Options` on that response only — the editor, landing and every 404 keep `SAMEORIGIN`. Cross-site cookie attributes are OPT-IN and off in production: `CROSS_SITE_COOKIES=1` makes the session and CSRF cookies `SameSite=None; Secure` and `PartitionedCookieMiddleware` appends `Partitioned`. They were on from 2026-10-13 to 2026-10-22 and cost nine days of answers (#388): the Facebook/Instagram in-app browsers on iOS keep no `Partitioned` cookie, so every section POST failed CSRF, and Chromium creators ended up with two `csrftoken` cookies (Django reads the last, `document.cookie` readers the first). With the switch off the same middleware sweeps the stale partitioned pair from browsers that still send it. A framed survey keeps its session through the cookie-less carrier below instead — never turn the switch back on for embeds. The respondent page adds `MS-Embedded: 1` to its HTMX calls when framed, which is how the final redirect stays on the in-frame thanks page (`?next=`) instead of navigating the frame to `redirect_url`. The Share page hands out the snippet (`snippet_html`), `/oembed/` describes it for platforms that do oEmbed discovery, and `/embed.js` (served by a view so the path never changes) grows the host's iframe from the height messages the page posts. Safari and every iOS browser keep no cookies inside a third-party frame, so `EmbedSessionMiddleware` (the project's SessionMiddleware) lets a framed page carry its session key itself — `X-MS-Session` header, `ms_sid` form field with a same-host Origin, one-time `ms_h` handoff on redirects — only under `/surveys/`, only without a session cookie, never for a signed-in session, with CSRF still checked (the secret rides in the session). A new respondent view needs nothing: the mark happens in access control and the carrier in middleware; a new full-page respondent FORM needs `{% embed_session_field %}` next to its `{% csrf_token %}`. **Printable paper form (`survey/print/`, change `printable-form`, #489; epic #350 item 2)**: the Share page's *Print a paper form* (every status, every plan, `editor` role) downloads `/editor/surveys//print/form/?lang=&paper=` — `pdf.build_form` walks the RESPONDENT structure (`compute_visibility(surface='respondent')`, `group_runs`; operator-only items left out, rules as an "Answer only if …" line) and renders one `QUESTION_SHAPES[input_type]` flowable per question in reportlab (pure Python; the bundled DejaVu fonts in `survey/print/fonts/` cover everything but CJK). A map section prints a frame from `static_map.render_frame`: the Mapbox **Static Images** API for the basemap (same token, a separate metered SKU, cached a day per frame, fail-soft to a gridded empty frame) with the survey's reference layers drawn on it by Pillow, then a letter legend (A ●, B ━, C ▭, D ░ — never icons: they do not survive a photocopy) and a sub-question table per mark. Every page carries four 6 mm alignment squares and a **form code** — two vector QR (20 mm, level H, both bottom corners) with payload `MSFORM:
:::` (copy 0 = unnumbered; the old `ms:form:…` still parses, `print/form_code.py`) and the line `MS- · # · Page n of N` — that names a `PrintedForm` row — version, language, paper, `structure_hash`, and per page the map frame's rectangle in mm plus its geographic bounds — which is what item 4 (photo marks → coordinates) will read; a reprint of an unchanged survey reuses the row. Fixed form strings come from the RESPONDENT catalogs (75; `test_every_printed_form_string_is_translated_in_every_catalog`), the dialog's from the creator ones. Event `form_printed` is server-side. The flyer generator #220 is the next consumer of the same page template and static map. `PrintedForm.layout[page]["items"]` (since change `paper-photo-entry`) lists what starts on each page — `ItemMarker` flowables record it at draw time — so a photographed page is read against exactly those questions; the walk both share is `survey/print/structure.py::plan_items`. **Numbered copies** (`?mode=copies&copies=N`, 1…200, the dialog's default; `mode=single` = one copy to photocopy): `PrintRun` hands out copy numbers per CANONICAL survey under `select_for_update` on its header — they only grow, across forms, languages and papers, so a copy number names one questionnaire forever. The form is laid out once; `_CopiesCanvas` stores each body page as a Form XObject and stamps every copy's codes over a `doForm` (20 copies ≈ one copy in bytes). Never print copies by re-running the layout or by merging PDF pages: both were tried and were slow or huge. `LAYOUT_VERSION` in `pdf.py` enters the structure hash — bump it when the page chrome changes. `?keep=1` (the dialog's default) wraps a map section's heading, frame, legend and marks tables in one `KeepTogether` and is part of the hash too; file questions are not printed at all (`structure.plan_items` drops `FILE_INPUT_TYPES`). **Paper forms from photos (`survey/paper/`, `survey/paper_views.py`; change `paper-photo-entry`, #495; epic #350 item 3)**: *Responses → Enter a response ▾ → From photos of paper forms* opens **Paper responses** (`/editor/surveys//responses/paper/`, the operator-entry gate). The operator's unit is the **paper response** (`PaperSheet`, labelled "Copy 37" / "Response 4" by `.label` — no surface says sheet or batch); `PaperBatch` is a hidden per-survey inbox (`paper.inbox_for`) that only holds counters. Rows: `PaperUpload` (one file as received, plus where it was dropped: `sheet`, `expected_page`, `is_pdf`), `PaperPage` (one JPEG ≤ 2200 px on the PRIVATE tier with its code), `PaperSheet`, `SurveySession.paper_sheet`. **The code decides, never the order of arrival** (`survey/paper/sheets.py::place_page`): a numbered copy's page goes to that copy's response (unique per inbox) whatever card it was dropped on; an unnumbered page goes where the operator put it (*New response*) or to its PDF's response (one PDF = one form); a loose unnumbered photo, a page of a stored response or of another form is set aside with a reason; a second photo of the same page wins and the first is `replaced` (restorable), and once the new photo is read `review.supersede_machine_answers` drops the stored rows that still equal the earlier reading (the operator's edits stay). The first build grouped by upload order and split one form into two sessions — do not bring order-based grouping back. **No model ever reads a code**: zxing-cpp decodes it locally, at full resolution in ingest (PDF at 200 dpi) and once more on the stored JPEG. **The web process never decodes a byte**: the upload view hands the file to storage and enqueues `ingest_paper_upload`; `survey/paper/ingest.py` and `recognise_paper_page` (place, then `check_quota(org, 'paper_page')`, then ONE vision call per page bounded by the page's `items`, `extract.py` + `prompts.py`) run in Celery — on the dev stack the worker does not reload, restart it after touching this code. Gemini counts thinking tokens against `max_tokens` (extraction uses 16000; a 200-token call came back empty). `LLMProvider.complete_structured(images=[(mime, bytes)])` is the only way an image reaches a provider. Every call is an `AIGenerationEvent(kind='paper_page')` and an immediate `$ai_generation` (trace `paper-response-`, `survey/paper/accounting.py`) — PostHog prices it; nothing stores dollars. The check is page by page of the PRINTED form (`paper_response_page`): photo left, the questions printed on that page right (`_page_plan` over `layout[n]["items"]`, `questions_filter`, `save_section_answers(only_questions=)` so a section split over two pages keeps both halves), a stepper, prefill through `_build_section_context(prefill=…)` (stored rows always win), doubtful (`confidence < PAPER_DOUBTFUL_CONFIDENCE` or `unreadable`) and blank fields marked, a missing page attached in place; forms without an item map fall back to the section flow of operator entry. **Marks on the printed map (item 4, `survey/paper/marks.py`)**: on a page with a map frame the same call lists the frame section's geo questions with their legend letters (`print/structure.py::mark_letters`, shared with the printer) and returns `marks` (label, question, vertices in 0–1000 of the photo, the table row's answers) plus `map_frame`; registration is LOCAL — `find_frame_fiducials` (the 5-mm squares printed in the margins beside the frame: corners since `LAYOUT_VERSION = 3`, mid-sides too since 4; `fiducials`/`fiducial_dx_mm`/`fiducial_mids` in the frame record, targets from `fiducial_targets`) first, then `find_alignment_squares` (page corners, pure Pillow at full resolution, thresholds strict → loose, solid through — a QR finder centre is not a square — and a portrait quadrilateral), then the model's `map_frame_corners` (a trapezoid), then its box; one weighted least-squares homography over every landmark (`fit_registration`: fiducials ×50 when ≥ 3, else ×4; page squares ×1; the frame's border sampled all around by `find_frame_edges` ×10 and its corners from the fitted lines ×30) PLUS a thin-plate spline over the residuals, because paper on a lap is not a plane — `apply()` takes the `{h, tps}` registration. A page also bows INSIDE the frame where no border can see it, so the printed map is the last landmark: `survey/paper/texture.py` matches patches of the frame image kept at print time (`texture.save_reference` → `paper_refs/-.png` on the private tier, `ref_key` in the frame record; forms printed before it re-render, the grid stand-in never) against the photo rectified through the current registration (FFT normalised cross-correlation at a 6×4 grid of anchors, ×20, up to three match → refit rounds in `_refine_with_interior`); edge samples are re-anchored along their side on every refit (`_fit_with_edges`), since across the side they pin the paper and along it they only repeat the plane they were found with. For a `point` question the vertex is then moved onto the ink (`texture.snap_to_ink`: rectified photo minus reference, the nearest blob 0.8–4.5 mm within 7 mm with discness ≥ 0.5; letters fail it) — the model answers with the label beside the dot often enough that its vertex is a hint, not the answer. **The hybrid (`survey/paper/ink.py`)**: with a reference image the GEOMETRY of every mark comes from the ink — coloured against the paper around it (a blue pen: channel shares less their 8-mm mean) or darker than the paper's 8-mm and 3-mm means and than the darkest print within 3 px (a black pen), never on near-black print — not HSV saturation, not absolute chroma, not darkness alone (each failed on one of the two copies; `design.md`) — its components read as hatched cluster → spray, filled disc → point, closed loop → area, long stroke → line (pieces sewn across skips, an outline whose ends meet closed), glyph-sized rest → writing — and the model only NAMES them: `_match_marks` pairs each model mark with the ink of its kind along its vertices, ink the model missed becomes a numbered mark at confidence 0.5, a model mark the ink lacks keeps its vertices at 0.6, and ink under the model's `label_box` is writing. Never make the model's vertices the geometry again where ink is available. `mm_to_lnglat` goes through the frame record's Mercator bounds (linear in world pixels, never in degrees). The prompt asks for the symbol's ink, never the label beside it. The result is GeoJSON features in the `existing_geo_answers` shape, which `_build_section_context` restores onto the map from `prefill` (never into `initial`), and `_page_plan` gives a section's geo questions to the page with its frame and to NO other page — an empty POST of the heading's page would wipe the placed marks. That page (`paper_map_mode`) is NOT the respondent panel: form layout half and half, `_paper_marks.html` + `js/paper_marks.js` draw the printed frame as their own Leaflet map and the marks TABLE the paper has (rows A1…An, the sub-questions as columns), and a hidden input per geo question posts `feature|feature` in the dialect `save_section_answers` parses (`views.geo_answer_features` is the one builder of that feature shape). On paper, `pdf._sub_cell` prints a choice column's boxes in EVERY row. A table that runs onto the next page (`extract.table_items_for_page`: the frame's page up to the page of the question's own `question:` marker, which follows the table on paper) is asked for there as rows without geometry (`marks.table_rows`, a `rows_only` field); `merge_pages` hangs them on the map page's field as `rows_extra` and `_paper_map_context` joins them to the marks by label — a label with no mark is an `unplaced` row the check shows filled, with a place-it link. Images go when a response is stored (`delete_sheet_images`) or the response is deleted (`sheets.delete_sheet`: an open response goes whole, its copy number free again — there is no "skipped"), and by `sweep_paper_batches --delete` (in `reclaim.sh`, deletes open responses idle past `PAPER_BATCH_TTL_DAYS` the same way; stored rows stay). Export column `paper_copy`. Events: `paper_response_created`, `paper_page_recognised`, `paper_sheet_entered` (`fields_changed` vs the prefill = the reader's quality signal) — ids and counts, never content. **Reference overlay layers (`SurveyMapLayer`, kill switch `MAP_REFERENCE_LAYERS`)**: creator-uploaded GeoJSON rendered read-only beneath answer geometry on four surfaces — the respondent map, the editor preview, the Responses **Map pane**, its **Overview thumbnail** and the per-response **map modal** (the 200-px drawer thumbnail stays bare). One styling source: `partials/ref_layer_factory.html` (`window.RefLayerFactory.build`) is included before any consumer; `partials/reference_layers.html` (respondent) and `editor/partials/analytics_geo_map.html` (Responses) only fetch and place what it builds. Metadata comes from `survey/layers.py::build_map_layers_metadata`, geometry only from the gated endpoint `survey_layer_geojson` — never inlined. That endpoint admits any collaborator with the `viewer` role or above in every survey status (a closed survey is where responses get read); outsiders still go through `check_survey_access`, which is deliberately untouched. On Responses, layers are non-interactive whatever `show_popups` says, ignore per-section `hidden_layers` (the map aggregates all sections), and make the Map pane render even with zero geo answers. On the Map pane they are `reference` slots of the `LayerManager`, listed in a titled group beneath the answer layers in the Layers panel (own pane each, so stacking = panel order, never fetch order); order, visibility and opacity persist in `localStorage['rv2RefLayers:']` — browser-only, never the model. Cap: `MAX_LAYERS_PER_SURVEY = 10`. **Layer memory rules (change `layer-memory-diet`, after the 2026-09-15 memory-limit incident)**: `SurveyMapLayer.geojson` is up to 10 MB of text per row, and a gunicorn worker keeps the RSS of its largest request for life. The text is stored gzip-compressed in `geojson_gz` (a third of the bytes in the row, in the worker and on the wire); `geojson` is a property that decompresses on read and compresses on write, so readers and writers (including `create(geojson=…)`) keep the text API — but `.defer()`, `update_fields` and `.values()` name `geojson_gz`. The gated endpoint hands the stored bytes to a client that sends `Accept-Encoding: gzip` (every browser and `fetch()`) with `Content-Encoding: gzip` and decompresses only for one that does not. So `layers_for()`, `question_layers_for()` and every layer `get_object_or_404` in editor/object views DEFER `GEOMETRY_TEXT_FIELDS`; the only readers of the text are the gated endpoint (`_gated_layer(with_geojson=True)`), `download_data` and the ZIP export (`layers_for(survey).defer(None)`). `question.layer` cannot defer through the FK, and a full instance's `save()` writes the text back — on respondent and Responses paths use `layers.layer_lite(question)`. Property names for the editor's pickers are the stored `SurveyMapLayer.property_names`, written by `rebuild_layer`; never parse the GeoJSON to list them. Workers recycle after `GUNICORN_MAX_REQUESTS` (+ jitter) requests, and the worker class is `mapsurvey.gunicorn_workers.DrainingThreadWorker`, NOT stock `gthread`: gthread drops the connections it accepted in its last loop iteration when it recycles or gets a deploy's SIGTERM (one 502 per recycle on 2026-09-16); the draining worker stops accepting first and serves what it holds. `scripts/gunicorn_recycle_check.py` is the reproduction — run it against both classes after touching the gunicorn command line or bumping gunicorn. The `LayerMemoryDietTest` query-capture tests fail if a page starts selecting the column again. **Layer objects (`LayerObject`, `LayerObjectAsset`; change `overlay-features`)**: a layer is a container of objects — key, title, category, rich-text description, link, one-part geometry, raw imported properties, ordered attachments (image/audio/document/video files on the PUBLIC media tier under random `layer_assets/` keys, or YouTube/Vimeo embeds). `SurveyMapLayer.geojson` is a CACHE derived from them (`survey/layers.py::rebuild_layer`, reserved `_key/_title/_category/ _has_content/_cover` properties) — never edit it as a source; `geojson_legacy` holds the FD-1 text until one release after migration `0068` and then goes. Layers are owned by the CANONICAL survey and borrowed by draft copies and archived versions through `layers_for()`/`layer_owner()`; nothing copies them, so an object edit on a published survey is live for respondents (the object editor says so in a banner). The object editor is a full page, `/editor/surveys//layers//edit/` (`survey/layer_object_views.py`, `js/layer_editor.js`), with three ways in — draw, import GeoJSON, import CSV — plus content CSV and photo batches matched by key, then title. Per-object cards for respondents come from `survey_layer_object` under the same gate as the layer endpoint. **Objects on the map (`layer_objects`) and `thumbs`**: `layer_objects` is a question type bound to one layer (`Question.layer`, PROTECT — the settings card refuses to delete a bound layer and names the question); `min_objects` replaces `required`. **Sub-questions are the one mechanism** for "ask about an object on the map", shared by geo questions and `layer_objects` (`PARENT_TYPES`), with two entry points into the same modal: the *Sub-questions* section inside the question modal and the "+ Add Sub-question" button under the question. A geo question with no sub-questions is a normal state — hint, never block. Respondent side = variant A: the panel list (`partials/layer_objects_block.html`) is navigation; a row or a feature opens the SAME Leaflet popup respondent-placed features use, with the object card + the sub-question form + ✓, and nothing opens while a draw/crosshair mode is active. Answers about objects are rows on the sub-questions with `Answer.layer_object` set (partial unique per session/sub-question/object) and NEVER `parent_answer_id`; they post as `obj____` fields. `thumbs` (👍/👎) is a choice type with the fixed `THUMBS_CHOICES` (`1=up`, `0=down`), so every choice consumer works unchanged. Aggregates on every read surface come from one place, `survey/object_stats.py`. Variant B (object card in the panel, geo popups moved there too) is deferred as a future alternative view — do not reintroduce it ad hoc. **Marker icons (`Question.icon_class`)**: two value forms — a Font Awesome 5 class (`fas fa-bus`, legacy `fa fa-bus` and pasted Pro classes pass through untouched) and a map-set name (`maki:bus`, `temaki:bench`, drawn as `` from the sprites in `survey/assets/img/`). **Never build icon markup by hand**: on the server use `{% marker_icon value color=… %}` (`survey/marker_icons.py`), in the browser `MarkerIcon.element(value, color, cssClass)` (`survey/assets/js/marker_icon.js`), which is what `L.Icon.FontAwesome`, the crosshair overlay, the draw button, star ratings and the editor picker all do. Unknown `set:name` → default pin. The picker (`survey/assets/js/icon_picker.js`) reads `survey/assets/data/icon_catalog.json`, generated by `scripts/build_icon_catalog.py` from Font Awesome Free 5.15.4 metadata plus Maki and Temaki checkouts (not vendored — see the script docstring); the per-language search terms live in `scripts/icon_terms//*.json` and are merged at build time, so a "creators type X and find nothing" report (PostHog `icon_search_miss`) is fixed by adding a row there and rebuilding. Keep the Font Awesome CDN link in the three base templates on the version the catalog was built from. **Display styles and the icon grid (change `choice-icon-grid`, #341)**: which `display_style` values a type accepts is ONE table, `question_types.DISPLAY_STYLES_BY_TYPE` / `allowed_display_styles()` — respondent rendering (`resolve_display_style`), `QuestionForm.clean()`, both editor previews and ZIP import read it, and the modal's JS `DISPLAY_STYLE_TYPES` mirrors its keys; a new style is one row there, never another hard-coded tuple. `icon_grid` (choice and multichoice) renders `choice_icon_grid.html`: native radios/checkboxes wrapped in tile labels, so value, required check and write-in are the list's. Per-option `icon`, `group` and `color` are optional keys on the `choices` entries next to `other`, trimmed/capped by `question_types.normalise_choice_extras` (`color` only as `#rrggbb`) on every save path and on import, and ignored by every other consumer. Tile icons go through `{% marker_icon %}` and only when `marker_icons.is_drawable()` — no default pin on a tile. Icon colour = the option's `color`, else `Question.color` unless it is the model default `#000000` (= unset, CSS grey). The icon picker is ONE button (`js/icon_picker.js`, the input stays hidden as the value carrier) opening a fixed-position panel with Remove icon / No icon / "Use "; never put a text field or browse button back next to it. Single choice folds to the chosen row in `js/choice_icon_grid.js`; pop-ups that restore values with `.prop()` must call `window.syncIconGrids(root)` (done in `initPopupFields`). **Spray area (`spraycan`, change `spraycan-question-type`)**: the fourth geo type. The respondent paints a cloud of dots with a brush (`js/spray_layer.js`, `L.SprayLayer`, one canvas per cloud, one pre-rendered airbrush sprite stamped per dot — never a marker per dot, never a path per dot; redraws only on `moveend`/`zoomend`); the answer is ONE `Answer` row per respondent with a `MultiPoint` in `Answer.multipoint` (no cap by default; `validation_settings.max_dots` is the creator's optional one, `SPRAY_HARD_CEILING` only bounds scripted POSTs), normalised on save by `survey/spray.py` (1 m grid, duplicates dropped — the count means painted area × dwell, not mouse events), brush size in `Question.spray_brush`; `spray.describe()` is the "N dots · area" every surface shows. The column is NOT named after the type, so the old convention `getattr(answer, input_type)` / `f"{input_type}__isnull"` / `a.point or a.line or a.polygon` is gone: use `question_types.geo_column()` and `Answer.geometry`, and take geo-type membership from `question_types.GEO_TYPES` (every module's `GEO_INPUT_TYPES` is an alias of it). Two deliberate exceptions: `MULTI_FEATURE_TYPES` (min/max features — a cloud is one feature) and `SHARED_MAP_SOURCE_TYPES` (`layers.GEO_INPUT_TYPES` — a cloud never feeds a shared-map layer). Responses (Map pane and Overview thumbnail) draws a spray question as one `L.SprayAgreementLayer` — the public grid's look: share of respondents per geographic cell, each counted once, NOT `leaflet.heat` (its zoom scaling flattens it); the public page gets a server-side grid of distinct-respondent counts per cell with cells below K omitted (`survey/public_results_grid.py`), never the dots. **Budget questions (`survey/budget.py`, `survey/budget_stats.py`, change `budget-question`, #447)**: ONE question type and ONE group setting — never a geo type (the first build had one, `budget_items`, and the owner rejected it: "keep the geo questions as they are, just give each object a price"). `budget` (a `CHOICE_TYPES` member, NOT geo) has a `mode` — `split` (amounts per item that add up to the budget) or `projects` (pick items until the budget is spent) — and stores ONE `Answer` per respondent in `Answer.allocation = {"total", "items": {code: amount}}` (Projects: the cost at answer time); `selected_choices` is never set, so no choice consumer picks a budget up. Settings live in `validation_settings['budget']`; items are `choices` with the extras `description`, `cost`, `description_i18n` (and the icon grid's `icon`/`color` — the bar is stacked per item in the item's colour, a group member's `Question.color`) capped in `normalise_choice_extras`; `source == 'layer'` takes the items from `Question.layer`'s objects (cost from `cost_property`) and joins `layers.section_layer_ids`. **A budget on the map is a GROUP**: `QuestionGroup.budget = {total, unit, symbol}` (group dialog, empty total = ordinary group) and a PRICE on each member geo question, `validation_settings['price'] = {amount, per}` with `per` ∈ `feature | length | area` narrowed by type (`budget.PERS_BY_TYPE`; the Price block shows in the question dialog only inside a budget group). Creators find it in the "New question" picker: the **Budget group** card in Map questions is a picker ACTION (`question_types.PICKER_ACTIONS`, not an input type — the parity test is about `PICKER_TYPES`) that posts `budget=1` to `editor_group_create` and opens the group dialog, which reads "Budget group" with its own tips. The geo questions keep everything else. One validator for every record (`normalise_budget_settings(…, input_type | 'group')`, `normalise_price`), one cost (`feature_cost` = amount × `measure`: 1 / metres / m² in Web Mercator corrected by cos(lat) like `spray.cloud_area_m2`; the browser mirrors it with `distanceTo` / `L.GeometryUtil.geodesicArea`). The section POST has no error path for budgets either: `budget.parse_submission` stores nothing for an impossible allocation, `views._enforce_group_budgets` admits a group's marks in stored order (`budget.GroupSpend`) and deletes the rest, and the client (`js/budget_question.js`: `refreshGroups` from `refreshGeoQuestionUI`, `admit` from `draw:created`, `firstInvalid` from Next) says why — no new placement path. Aggregates: `budget_stats.aggregate` (question) and `group_aggregate` (group; Responses inserts a `budget_group` block before the group's first member); export writes a `cost` property per priced mark and a `: spent` column. Budget groups are FREE on every plan (owner 2026-11-10); Pro feature `budget_map` is asked only where a budget question becomes layer-sourced (`services.questions. _budget_pro_gate`) → `ProRequired`; respondents never meet it. A group's total/unit, a question's price, and a budget question's mode/total/unit/step/source/cost property/item costs are structural on a live survey (`versioning._budget_structural_changes`, `services.groups.update`). **Vendor facts on comparison pages (`survey/vendors.py`, change `market-guide-landscape-hub`, #259)**: every claim the marketing site makes about another vendor is a `Fact(text, source, verified)` in ONE registry — the public `https://` page it was read from and the ISO date — grouped into `CATEGORIES` and `CRITERIA`. The market guide `/participatory-mapping-tools/` (the hub above the `/alternatives/` pages; `/alternatives/` 301s to it) renders vendors × criteria from it, and the three `/alternatives/` pages render criteria × (Mapsurvey, vendor) through `partials/_vendor_vs_table.html`, so one correction reaches every page. **Never hand-write a competitor cell in a template**; a fact the page does not state is `Fact("Not published", , )`. No price figure for any vendor but Mapsurvey (owner rule 2026-10-06): the model, the trial and where a rate is listed; `test_no_competitor_price_figure_on_any_landing` scans every `Fact.text` and the rendered hub. Import-time `_validate()` and `VendorRegistryTest` refuse a fact without a source or with a bad date. The five category pages (#251/#252) still carry hand-written tables; moving them onto the registry is the follow-up after #253. Mapsurvey's own hosting fact is the United States (Render, Oregon) — the "EU-hosted / Frankfurt" wording that survived on the Maptionnaire page until this change must not come back. **API platform — the authorization layer (`api/`, change `auth-core`, #379; epic #378)**: every programmatic client (personal token, OAuth grant, the REST API #380 and the MCP server #306 on top) goes through ONE `api.auth.authenticate(request, audience)`. Bearer header only — a session cookie never authenticates an API path, which is why those views are CSRF-exempt and why `request.user` is never read there; use `request.principal`. Tokens are `oauth2_provider` `AccessToken` rows (hashed at rest, looked up by checksum) with our `TokenProfile` beside them (`kind`, `label`, `audiences` `api`/`mcp`, optional workspace binding, `revoked_at`); personal tokens are `msk_…`, minted only by `api.tokens.mint_personal_token()` on `/editor/account/connections/` (`/editor/account/mcp/` redirects there). Rights = scope ∩ organisation role ∩ survey role through `api.rights` (`survey_for` answers 404, never 403; `section_for`/`question_for` always filter by the survey — integer ids are IDORs otherwise). **Pro (`api` feature) is checked at the resource, per request, on the organisation that owns the object** (`api.gates.require_pro_api(principal, survey.organization)`), never at minting: a lapsed subscription stops access without revoking anything. Limits are per token (`api.limits`: read/write minute windows, a daily cap, named caps via `limits.hit(principal, "survey_create")`) plus a per-IP budget of failed bearers checked before the lookup; `429` + `Retry-After`, fail-open on a cache outage. Errors are the envelope in `api/errors.py` (`invalid_token`, `scope_missing`, `pro_required`, `not_found`, `rate_limited`, `server_error`). A new endpoint is `@api_endpoint(audience, scopes=…)` + `rights` + `gates` and calls the service layer — never a model directly (that is how an agent would skip `coerce_creator_html`). Token values never reach logs: `_posthog_scrub_tags` keeps only path and method under `API_PATH_PREFIXES`; audit and events carry ids. `OAUTH2_PROVIDER` pins RFC 9700 and `api.tests.test_settings` runs the toolkit's deploy check. Contract for resource servers: `api/README.md`. **REST API v1 (`api/v1/`, change `rest-api-v1`, #380)**: django-ninja at `/api/v1/`, OpenAPI at `/api/v1/openapi.json` (public). Routes declare `auth=Bearer()` (scopes checked before the body is parsed), get the survey through `deps.survey(request, uuid, role=…)` (404 invisible, 403 `forbidden_role` under the role, `pro_required` on the owning workspace) and write ONLY through `survey/services/` with `strict=True`; JSON is turned into the editor forms' dialect by `forms_bridge.py`, which decides nothing. Input schemas are `extra='forbid'` (an unknown field is a `422 validation` naming it). A structural change to a live survey is `409 draft_required` — the API never creates unpublished changes unasked. Export is `export.export_response`, the same function `download_data` returns; layer GeoJSON is `views.cached_geojson_response`. Ninja matches `{param}` as a string, so a literal path segment (`/surveys/import`, `/sections/order`) is registered before the parameterised route. The help centre's `reference/api.md` is GENERATED by `scripts/build_api_reference.py`; `ApiReferenceFreshnessTest` fails when a route or schema changed without regenerating it. **MCP server (`mcp/`, change `mcp-server`, #306; epic item 4)**: the second resource server, served by THIS process at `settings.MCP_RESOURCE_URL` = `/mcp` (no trailing slash; `/mcp/` routes too because `APPEND_SLASH` would 301 a POST). Streamable HTTP in **stateless mode**, hand-written in `mcp/views.py` + `mcp/protocol.py` (no SDK: the official one is ASGI and streams): one POST → one JSON response, `GET`/`DELETE` → 405, notifications → 202, NEVER an SSE stream (a worker held by a stream is the 2026-09-18 outage). Versions `2025-03-26..2025-11-25`; a 2026-07-28 client's `server/discover` gets `-32601` and falls back to `initialize`. Auth = `authenticate(request, 'mcp')`, 401 names `/.well-known/oauth-protected-resource/mcp` whose `resource` is the setting byte for byte. `Mcp-Session-Id` is SIGNED (token id + nonce, `mcp/sessions.py`), nothing stored; another token's id → 404; a present `Origin` not the issuer's (or `MCP_ALLOWED_ORIGINS`) → 403. Tools live in `mcp/tools/` as `@tool(...)` over `args_model(, survey=…, …_id=…)` — the REST schemas plus ids, `$ref`s inlined — and call `api.v1.deps` / `forms_bridge` / `out` / the services with `Actor(surface='mcp')`, never a model; `tools/list` omits tools the token's scopes lack and a call to one is `scope_missing`. Protocol errors (unknown tool, schema failure) are JSON-RPC `-32602`; every other refusal is an `isError` result carrying the REST envelope (two MCP-only codes: `confirmation_required`, `has_responses`). `delete_survey`, `publish_survey`, `publish_unpublished_changes` need `confirm: true`; `delete_survey` refuses a survey with answered external sessions. Every result is `{"note": UNTRUSTED_NOTE, "data": …}`; descriptions are literals. Limits: reads/writes share REST's windows by `read_only`, plus `limits.check_group('mcp.session')` (300 / 10 min). Event `mcp_tool_called` (tool, outcome, token id; never arguments). The help page `reference/mcp-tools.md` is GENERATED by `scripts/build_mcp_reference.py` (`test_reference` fails when stale). Vocabulary: "unpublished changes", never "draft", in tool names and descriptions. **The authorization server (change `oauth-server`, #313; epic item 2)**: django-oauth-toolkit's views mounted by hand in `api/urls.py` — RFC 8414 document at the root, `/oauth/authorize|token| revoke|register/`, nothing else (no introspection, device flow, OIDC, or protected-resource document; that one is each resource server's). Issuer = `settings.OAUTH_ISSUER` (the preview's own host under `IS_PULL_REQUEST`, else `SITE_URL` — `SITE_URL` is production on previews too). Clients register themselves (open DCR behind a per-IP budget, CIMD through the toolkit's SSRF-hardened fetcher) or are first-party rows in the admin. **Redirect URIs are ONE rule**, `api/oauth/redirects.py`: exact match, `https` only, except loopback `http://127.0.0.1|[::1]|localhost` with any port (Claude Code); the toolkit cannot express that, so `ALLOWED_REDIRECT_URI_SCHEMES` lists `http`, `oauth2_provider.E003` is silenced, and the helper runs at DCR, CIMD, `Application` `pre_save`, authorize and token time — never widen or bypass it. Every OAuth access token (`msa_…`; refresh `msr_…`) gets a `TokenProfile(kind="oauth")` in `api/oauth/validator.py` with audiences from `resource` (`api/oauth/audiences.py`, `API_RESOURCE_URLS`; none = both, unknown = `invalid_target`), so rights, the Pro gate and limits apply unchanged; `code_checksum` is how a code redeemed twice revokes its whole family. Consent (`api/authorize.html`) shows the redirect host ABOVE the attacker-chosen name, publish/delete as switches off by default, issued scope ⊆ requested (signed hidden field), and is remembered while an unrevoked refresh token covers the request. `/oauth/*` responses get DENY / `frame-ancestors 'none'` / no-store from `api.middleware`. Connected apps and *Disconnect* live on `/editor/account/connections/` (`api/oauth/grants.py`). Ops after a deploy: `scripts/oauth_reachability_check.py` and a bot-shaped curl of `/api/v1/me` — the Cloudflare in front of production is Render's edge, not a zone we manage, so there is no rule of ours to set. **Sites (`/p//`, epic #348; called "Landing pages" until 2026-11-09)**: a workspace's public site for an engagement — a landscape/planning firm's project site or a council's — built from a CLOSED set of blocks (`survey/sites.py::BLOCK_KINDS`, one cleaner per kind in `clean_block`, one public template `sites/blocks/.html`, one canvas include). Models `Site`, `SiteSurvey` (surveys linked many-to-many), `SiteBlock`, `SiteFile`, `EventRegistration`; Pro feature `sites` (trial: one site), a published site survives a lapse. The builder (`site_views.py`, `templates/sites/builder.html`, `js/site_builder.js`) is the page as visitors see it: the panel form (`sites/_block_form.html`) is the ONE serialisation of a block, canvas edits travel in its hidden fields, every change autosaves with `autosave=1` (JSON fragments; an EMPTY field waits, a wrong value is a 422). Owner rules: a block's primary actions happen in the block, refinements in Settings; reuse the app's dialogs and toolbars (no `confirm()`). **Every write goes through `survey/services/sites.py`** (change `sites-service-layer`, #617) — `SiteServiceLayerTest` fails if a view writes a row itself. Never name a template context variable `block` in a template extending a base (it is a `BlockNode`); the builder uses `b`/`edit_block`. CSS prefixes `lp-` (public) / `lb-` (builder) are historical internal names; the marketing `css/landing.css` and the SEO landings are a different thing. Plan of the next waves: ops `requirements/design/sites-v1.1-architecture-and-plan.md`. **Service layer (`survey/services/`, change `rest-api-v1`, #380)**: the ONE write path for surveys, sections, questions (choices, validation settings, visibility rules, translations, order) and the lifecycle (transition, unpublished changes create/publish/discard, restore version). The editor views and the REST API both call it; neither writes those models directly. A service takes an `Actor(user, ip, surface)` and plain values, never a `request` (`audit.record(actor, …)` is the request-free audit writer; `audit(request, …)` delegates to it). Input is validated by binding the SAME editor form (`SurveyHeaderForm`, `SurveySectionForm`, `QuestionForm`) to a dict — that is what keeps `coerce_creator_html`, `clean_redirect_url` and every `clean()` rule identical across surfaces — and `partial=True` gives PATCH semantics on top of forms written for full posts. Everything is validated before anything is written, inside one transaction (a layer created from `layer=answers:` goes with a refused save). Refusals are `services.errors` types (`ValidationFailed` with `fields` and, for questions, a `stage`; `StructureLocked` with `reasons` and the draft; `ContentLocked`, `AnswersAtRisk`, `TranslationGaps`, …) that each surface maps to its own response — the editor to exactly what it always returned. Questions speak the editor's dialect (`QuestionForm` fields + `choices_json`, `vs_*`, `sm_*`). Strict visibility checking (`visibility.validate_rule`) is opt-in (`strict=True`, the API); the editor keeps fail-open. Groups, duplicate, paste and move still live in their views. A test that patches a function the editor used to call directly patches it where the service imports it (`survey.services.locks.screen_survey`). **Billing — Pro on PayPro Global (`billing/`, change `paywall-payproglobal`, #268)**: PayPro is the Merchant of Record; we own the checkout link out (`billing/checkout.py`, the ONLY place a PayPro URL is built, with `x-org_id/x-user_id/x-plan/x-nonce` and `ipn-domain`), the webhook in (`POST /billing/ipn/` → `billing/ipn.py`: verify source IP + SHA256 `SIGNATURE` + MD5 `HASH` over the RAW strings, store once in `BillingEvent`, apply the state machine to `Subscription`) and the entitlement (`billing.entitlements.for_organization(org)` — Pro while `now < current_period_end` in active/past_due/suspended/terminated, never after refund/chargeback/finish). **One grant path**: only the webhook view writes `Subscription.status`/`current_period_end`; the return page polls, the billing-page API actions (`billing/ppg.py`, shown only when `PPG_API_SECRET_KEY` is set) ask PayPro and wait for the webhook. Features are the registry in `billing/features.py` (`unbranded`, `pro_help`); gate with `require_feature` / `{% has_feature org "key" as x %}`, never a column. `survey/ai/quota.py::check_quota` reads the same entitlement. `PPG_PRODUCTS` empty = nothing sold (rollback). Test-mode webhooks are applied only where `PPG_ACCEPT_TEST_IPN` is true (previews). The badge partial and `public_results.html` honour `show_branding` only under `unbranded`; the settings form disables the switch otherwise. Setup checklist, env vars and the PayPro email: `billing/README.md`. Reference digest of PayPro's docs lives outside the repo (local skill `payproglobal`). Tests: `./run_tests.sh billing` — one module per capability, `billing/tests/test_.py` ↔ the spec of the same name (the rule proposed for `survey` in #273); payload and subscription builders in `billing/tests/helpers.py`. **Pro trial (change `pro-trial-allowances`, #471)**: the SECOND entitlement source, next to `Subscription` — `EntitlementGrant` (`kind='trial'`; #322's academic grant is the next kind) with one `GrantAllowance(feature, limit, used)` per feature. Not days of Pro but counted uses for `TRIAL_DAYS` (90) after the OWNER's click (`POST /org//billing/trial/`, `billing/trial.py::claim`): `api` 500, `unbranded` 1, `pro_help` 1, `operator_entry` `None` = uncapped (counted, never refused; the paper cap arrives with #495) (`TRIAL_ALLOWANCES` in `billing/features.py`; empty = not offered, the rollback). One trial per OWNER (partial unique on `claimed_by`) and per workspace; never on Pro; AI generation is not in it (free on every plan). **It starts itself (change `pro-trial-auto`, #508)**: the first Pro use anyone allowed to make it makes in a workspace that never had a trial creates the grant for the earliest-joined owner without a trial (`started_by` = the actor, `trigger` = the feature key; a click is `trigger='click'`) and counts that use — never at registration or workspace creation. Every control that OFFERS a use asks `billing.trial.may(org, key)` (= `Entitlements.may` or `startable`), every point of use calls `billing.trial.use(org, key, actor)` (start + count), never `consume` directly; the view then calls `trial.flash_started(request, org)` for the one-time notice. The owner gets a start mail (on commit) and one "ends in 7 days" mail (`notify_trials_ending`, daily from `reclaim.sh`, `ending_notified_at`). **Every Pro control carries `{% pro_badge org "key" %}`** (`billing/_pro_badge.html`, `css/pro_badge.css`, violet `--pro` tokens — teal is for actions): plan / trial / startable ("free to try") / locked; never hand-write a Pro label. Navigation items (the sidebar) use `{% pro_nav_mark org "key" %}` instead (change `onboarding-friction`, #638): nothing unless the workspace is `locked`, then a grey lock without the word — a "PRO" in the sidebar read as "not for you" to the Free creators the trial is for. Tests that need a Free workspace REFUSED call `billing.tests.helpers.trial_used_elsewhere(org)`. `pro_help` is "Advice on your survey and results" (#438), listed in every menu below a free "Suggest an idea or report a bug" (`billing/_help_entries.html`). The webhook still owns `Subscription`; a grant is a different row with a different writer. **Three questions on `Entitlements`**: `has(key)` = the PLAN includes it (plan cards, `is_pro`, lapse task); `may(key)` = a NEW use is allowed (plan, or a live trial's remainder) — wrapped by `trial.may`, which every gate uses (`require_feature`, `require_pro_api`, `pro_organizations`, `SurveyHeaderForm.unbranded`); `covers(key)` = in force (plan, or a live trial includes it, spent or not) — what `{% has_feature %}` answers, because a survey unbranded under the trial stays unbranded after its one switch-off is spent. A use the plan does not cover is counted ONLY by `trial.consume()` (through `trial.use`) — one `UPDATE … WHERE ("limit" IS NULL OR used < "limit") RETURNING`, race-safe — at the point of use: `require_pro_api` (once per request per workspace; `mcp.protocol.call_tool` resets the memo per tool call), `services.surveys.update_settings` (a badge switch-off, inside the service transaction), `org_help` (a stored request), `operator_entry_views.entry_header` (one sheet; the sheet's own pages admit on `covers`). Expiry is a comparison, no cron. The viewer's state (`available | live | exhausted | expired | used_elsewhere`) is `trial.state_for(org, user)`, rendered by `billing/_trial_offer.html`; `pro_required` carries `trial: none|exhausted|expired`. Staff end a grant by setting `revoked_at` in the admin; the counters are read-only there. **Support chat (Chatwoot, change `chatwoot-support-chat`, #291)**: one partial, `partials/_support_chat.html`, included by the three base templates; config and the signed-in creator's identity come from `survey.context_processors.support_chat`. The identifier is `str(user.pk)`, signed with `identifier_hash = HMAC-SHA256(CHATWOOT_HMAC_TOKEN, pk)` on the server — the token never reaches the browser; empty `CHATWOOT_WEBSITE_TOKEN` = no widget, no third-party request (tests, local dev, previews without the env group). The creator bases include it with `editor_bubble=True` (change `onboarding-friction`, #638): the bubble shows on a desktop and never below 768 px, carries our own "×" that hides it in this browser (`localStorage['msSupportBubble']`), and "Show chat button" in the "?" menu (`.support-bubble-restore`, `showSupportBubble`) brings it back. The editor widget never speaks first, bubble or not: no unread pop-up; the "?" menu in the header (`editor/partials/_help_menu.html`) and "Help & support" in the ⋯/account menus stay the entry points, and an agent reply lights the `.support-unread` dot on them. `openSupportChat(event, context)` opens the chat and sends `context` to the agent as contact custom attributes (workspace, plan, page, survey are sent on every page) — the hook for help at the point of failure (#295). Chatwoot's own cookies outlive our session, so a page rendered for a different visitor clears them before the SDK loads; do not use `$chatwoot.reset()` for that (it reloads the frame without a second `chatwoot:ready`). **Help centre (`help/`, change `help-site`, #298)**: help.mapsurvey.org is a Starlight (Astro) site built from `help/` by its own Render static service (`mapsurvey-help`, `buildFilter: help/**`, automatic PR previews because static sites are free; previews mark every page `noindex`). Sections follow Diátaxis — `start-here/` (tutorial), `how-to/`, `reference/`, `troubleshooting/`, `learn/` (citizen engagement as a practice, written from `docs/research/survey-design-rules.md`), `use-cases/`, `faq.md` — and `help/CONTRIBUTING.md` holds the page shapes and the FAQ rule (a question is added only when someone asked it; source in a comment). A feature's guide ships in the feature's PR. **Product → help links go through `survey/help_links.py`** (`HELP_URL.` from the `help_site` context processor; `HELP_SITE_URL` empty = no links) and `HelpSiteLinksTest` fails when a registry path has no page under `help/src/content/docs/`, so a renamed page needs a `redirects` entry in `help/astro.config.mjs` and a registry update in the same commit. The "?" menu's "Help centre" item and the "Learn more" links (reference layers, embed, export, Pro) are the only entry points; the help site's "Still stuck?" link is `mapsurvey.org/?support=1`, which opens the chat for a signed-in creator (`open_on_load` in the support_chat context). `npm run build` in `help/` fails on a broken internal link (`starlight-links-validator`); the MCP pages are `draft: true` until mcp.mapsurvey.org is live. **Translation table (change `translation-table`, #454; epic #393)**: the **Translations** tab (`survey/translation_views.py`, `editor/survey_translations.html`, `js/translation_table.js`) is where translations are written from the UI; the Translations blocks of the section, question and group modals, the per-language columns of the choices table and the non-primary tabs of the thanks panel are READ-ONLY mirrors that link to the row (`?to=#row-`), and the editor views pass `translations=UNSET` to the services (the API and MCP still write through them). One module, `survey/translation_table.py`: an **occurrence** is one translatable field addressed by CODES, never ids — `section:[#n]:title|subheading|next_label`, `question::name|subtext`, `question::choice:`, `group::title|description`, `survey:thanks` — so nothing has to be copied on the draft copy, publish or restore; `occurrences()` walks the survey in respondent order, `rows_for()` groups identical normalised source text (`t:` keys; a group whose occurrences carry different translations renders split), and **`apply()` is the one writer** (the page, the CSV import and the future AI job) — it routes every value to the entity's service path (`save_translations`, `set_choice_names`, `set_thanks`), so caps, `coerce_creator_html`, the content lock and the re-screen are the modals'. **Outdated** = `TranslationSource` (keyed by the canonical survey + occurrence key + language) holds the hash of the primary text a translation was written against; every translation write records it for the fields whose value changed (`record_all` from the table), a base-text save never touches it, and a translation with no row reads as translated. `translation_gaps.survey_translation_gaps` stays the one gap engine — it now counts the Next label, the thank-you page and stale rows (`" (outdated)"`) — so badges, the publish warning, the API 409 and the table cannot disagree. CSV: `export_csv`/`parse_csv`/`values_from_csv` — the customer's Page/Element/source/translation layout plus `key` and `status`, matched by key then by source text, formula cells apostrophe-guarded, rich text as plain paragraphs. Languages are added from the tab through `services.surveys.update_settings` (owner only); the pair is `?from=&to=` and remembered in `localStorage['msTranslations:']`. Build takes `?question=&lang=` / `?group=` to open that modal at its Translations block. **Section layout controls (change `section-layout-controls`, #473)**: three independent per-section settings — the stage `SurveySection.layout` (`map | image | form`; `image` shows `stage_image`, else the survey's `cover_image`, else a tinted area), `panel_side` (`left | right`) and `panel_width` (`default` = 420 px, `third`, `half`), plus where the content sits inside the panel (change `panel-content-align`, #476): `content_width` (`full` | `column` ≤ 640 px, box-centred) and `content_valign` (`top` | `middle` — auto margins, so a long section scrolls from its title, never `justify-content: center`), both opt-in and ignored by `form`; and `keep_panel_open` (change `keep-panel-while-drawing`, #477, map sections, desktop): placement paths never call `toggleInfo(false)` themselves but `hidePanelForPlacement()`, and the crosshair, its pin position, the draw bar and a moved point centre on `visibleMapArea()` — a new placement path must use both (`test_placement_paths_go_through_one_helper` counts them). Map-less stages are ONE constant, `models.MAPLESS_LAYOUTS` (`section.has_map`): the section form, the question form's type filter and `layers.py` read it, so `image` excludes geo questions exactly as `form` does — never test `layout == 'form'` again. Rendering is body classes (`SurveySection.layout_classes()` on a direct load, `initSection()` step 0 on an HTMX swap from `#section-data[data-panel-side|data-panel-width|data-stage-image]`) plus one CSS variable `--panel-w`; the defaults set no class, so an untouched survey renders as before. Side and width rules live ONLY inside `@media (min-width: 769px)` in `main.css` — on a phone the panel stays the 88 % overlay and an `image` section is a picture band over one scrolling column. The picture is shared within a survey family and copied across surveys (`copy_section`, like `Question.image`), rides the ZIP as `images/sections/
_`, is read-only in the API (`stage_image` URL) and deleted on purge. **Hierarchical Questions/Answers**: Both Question and Answer models support self-referential parent relationships via `parent_question_id` and `parent_answer_id` for conditional sub-questions. **Conditional visibility (`CONDITIONAL_VISIBILITY` kill switch, default ON)**: a `visibility_rule` JSONField on `Question` and `SurveySection` (`{"question_code", "choice_codes"}`, any-of match on an earlier `choice`/`multichoice` answer) drives who sees what. One engine — `survey/visibility.py` — feeds the respondent form, POST discard/purge, section navigation, progress, and the editor badges/lint, so runtime and editor can never disagree. Hidden ⇒ never required, submitted answers discarded, abandoned branches purged server-side. Broken rules fail OPEN (shown to everyone) and are badged in the editor. Rules ride ZIP export/import (`_apply_visibility_rules` drops unresolvable ones with a report line) and duplication (`cloning.py` remaps intra-section controllers; cross-survey paste drops the rule). Do NOT reuse `parent_question_id` for visibility — that relation means "geo popup sub-question". Env var off ⇒ rules stored but inert, editor hides the Visibility block. **Question groups (`QuestionGroup`, change `question-groups`, #340)**: a container, not a question type — a title, a rich-text description (`coerce_creator_html` on every write), a display style (`plain | card | collapsible`) and one `visibility_rule` for a run of questions in a section or in a geo / Objects pop-up (`parent_question`). Answers stay rows on the questions; export, Responses and public results do not know groups exist. **One continuous order**: a group's `order_number` is a slot in the same sequence as its sibling questions and its members (`Question.group`, `SET_NULL`) follow it contiguously, so `section.questions()`/`subQuestions()` and the visibility engine's "controller is earlier" test are unchanged; anything that computes a new `order_number` must count groups (`survey/groups.py`: `max_order`, `next_order`, `shift_down`), and anything that DRAWS questions goes through `group_runs` (respondent: `SurveySectionAnswerForm.field_runs` → `partials/question_fields.html` → `partials/question_group.html`; editor: the `editor_runs` tag → `group_list_item.html`). Visibility treats a group as host kind `'group'` judged at its own order; a member is visible only if its group is (`VisibilityMap.group_visible`, `question_own` for the respondent page); client rule maps key groups `group:`, always before their members. Editor endpoints live in `survey/editor_group_views.py`; the one reorder endpoint for nested lists is `editor_items_reorder` (ownership-checked, renumbers 0..N, sets membership); a group change re-renders the section panel rather than patching rows. Deleting asks: only the group (members stay in place) or with its questions (the usual 409 answer-loss acknowledgement). Groups ride the ZIP (`groups`, `sub_question_groups`, `group`), every copy (`cloning.clone_groups`, `clone_top_level_questions`), comments (`group:` anchor) and content screening. AI generation writes no groups. **Pop-up rules (change `subquestion-visibility-rules`, #338)**: a sub-question of a geo / `layer_objects` question may carry the same `visibility_rule`, but its controller is an EARLIER `choice`/`multichoice` sub-question of the SAME pop-up (`parent_question_id`), never a section question. Those rules never enter `compute_visibility` (a pop-up is answered once per feature): `visibility.popup_visibility(parent, answers_by_code)` judges them per feature/object, and the section POST calls it for every geo feature (from its properties) and every object (`_save_object_answers`), storing nothing for a hidden follow-up. The respondent pop-up keeps every sub-question in its HTML; `conditional_visibility.js` toggles them per pop-up `` from `#section-data[data-popup-visibility-rules]` (bound in `initPopupFields`, i.e. after a feature's values are restored), and the geo shell prunes hidden cards' properties on Apply/close. Editor: `_visibility_block_context(..., 'subquestion', host, parent)` renders the block with one group ("This pop-up"); `visibility_info` resolves the kind from `parent_question_id_id`, so the recursive `question_list_item.html` badges sub-question rows unchanged. Import resolves a sub-question rule among the imported siblings; `clone_question` remaps it to the cloned sibling on every path, including cross-survey paste (the controller travels with the parent). **Shared map (`question`-sourced reference layers)**: a `SurveyMapLayer` with `source='question'` holds no creator objects — its `LayerObject`s are MATERIALISED from respondents' answers to the geo question named by `source_question_code` (a code, not a FK: layers belong to the canonical survey, question rows are copied per version). Materialisation runs at the end of the section POST (`layers.sync_question_layers_for_session`) and keys objects `s-`, never by answer id, because the POST deletes and re-inserts a session's answers on every submit and a re-keyed object would lose the reactions other respondents left on it. Respondents get `build_question_layer_geojson` per request (`no-store`, own marks omitted, only `status='visible'` from clean sessions); creator surfaces read the cached `layer.geojson` (every status, clean sessions) rebuilt on materialisation, moderation and session-status changes. Reactions are ordinary answers to the sub-questions of an "Objects on the map" question bound to that layer; `show_tallies`, `show_comments`, `approve_first` on the layer decide what other respondents see, and `LayerObject.status` / `Answer.hidden` are the creator's per-item moderation. The object editor is read-only for such layers; deleting the source geo question is refused. **Coverage surface (change `responses-coverage-surface`, issue #242, epic #245)**: on the Responses Map pane a LineString/Polygon answer layer's menu offers *Create Coverage*, a `coverage` slot of the `LayerManager` that draws, per world-anchored metre cell, the share of respondents whose shape covers it (polygons: inside, holes excluded; lines: within `corridorMeters`, default 20 m). It is `L.SprayAgreementLayer` (`js/spray_layer.js`) fed `{id, geometry}` members instead of `{id, dots}` clouds — ONE binning seam (`_bin` → `_rasterise`, an offscreen canvas at cell resolution clipped to the viewport, even-odd fill, stroke width = corridor in cells, alpha read-back) for spray, polygons and lines, so the three never disagree; do not add a second density renderer. Each respondent counts once per cell; the row reads "≤ N of M respondents". Derived in the browser only — exact counts for export and public results are #243, the agreement contour #244. **Reference layer style**: `SurveyMapLayer.style` (JSON) holds the base look beyond `color` (opacity, weight, fill_opacity, radius, icon) and at most one rule `by` one object property (categories or graduated classes, plus `other`). `layers.normalize_style` is the ONE validator — the update endpoint and the ZIP cleaner both run it, so a style that saves is a style that draws; never read `layer.style` raw. `ref_layer_factory.html`'s `styleFor()` is the ONE renderer (rule class → simplestyle in the file → base) for the respondent map and the Responses map alike; `layers.match_class` mirrors its matching for the server-side legend (`legend_for`, delivered as metadata). The object editor draws with its own code and does not honour rules yet. **In-app changelog (change `in-app-changelog`, issue #227)**: "What's new" entries are files in `survey/changelog/-.html` — a `---` header (`title`, `kind: new|fixed`, optional `link` URL name, optional `image` static path) followed by an HTML body, English only, shipped in the PR of the change they describe (format in `survey/changelog/README.md`; the loader is `survey/changelog.py`, read once per process). Seen-state is ONE watermark per creator, `CreatorPreferences.changelog_seen` = the newest id seen; ids start with the date, so "unseen" is a string comparison. The card (`editor/partials/_whats_new_card.html`, included from `editor_base.html` AND `base.html`, never from `base_survey_template.html`; the `whats_new` context processor also returns nothing under `/surveys/` and `/r/`) shows only the latest unseen entry, once; "Got it" / × / the page / "Don't show these" all move the watermark. `changelog_cards` off hides the card only — the page `/editor/whats-new/`, the gift icon and the account-menu item stay. New accounts start seen (`seed_changelog_watermark` in `signals.py`). `ChangelogEntriesTest.test_shipped_entries_load` parses every file, so a malformed entry fails CI, not every editor page. The first entry (`2026-10-02-whats-new.html`) announces the cards themselves; #226 (empty sessions hidden by default) writes the next. **Hero tier (change `changelog-hero`, #423)**: an entry with `tier: hero` (+ `art`/`image`, `link`+`link_query`, `cta_label`, optional `help`, `chips` with `{survey}`, `show_if`/`note_if` from the predicate registry `survey/changelog_audience.py`) renders as a full-width banner on the dashboard (`editor/partials/_whats_new_hero.html`, included from `editor.html` ONLY) instead of the corner card. A hero is *pending* while unseen, ≤30 days old and in audience; while pending no corner card renders anywhere (`whats_new.card` is None), a hero never becomes a corner card, and closing it marks everything seen. Both partials remember the closed id in `localStorage['msWhatsNewClosed']`, so a close whose POST failed stays closed — the second "close, close, close" click was the owner's reason for the rule. Budget: two heroes ≥14 days apart (`test_hero_budget`). Events carry `tier`; `changelog_card_clicked` (`target=cta|help`) is the interest signal per feature. **Comment threads (`CommentThread`/`Comment`/`CommentAttachment`, spec `survey-comment-threads`)**: workspace members discuss a survey where it lives — the survey as a whole, a question, a section, a respondent session or a public-results block — in ONE slide-in drawer (`editor/partials/_comments_drawer.html`, included once from `editor_base.html`; `js/comments_drawer.js` + `css/comments.css`). Row badges (`section_list_item`, `question_list_item`, `pr_block_list_item`) and the toolbar/modal buttons call `CommentsDrawer.open({anchor: ":"})`; the panel is an HTMX partial from `survey/comment_views.py`. **Anchoring rule**: `thread.survey` is always `canonical_of()`, question and section anchors are `code`s (never FKs — `publish_draft()` replaces the rows), session and block anchors are FKs; a `CheckConstraint` enforces exactly one anchor. All of that lives in `survey/comments.py` (`resolve_anchor`, `anchor_fields`, `open_counts`, `participants_of`, writes); views only check roles. Every role opens/replies/resolves; delete = author or owner. Bodies are PLAIN TEXT escaped on render — `coerce_creator_html` does not apply. Attachments sit on the private media tier under random keys and are served only by `attachment_download`. Badge counts come from one grouped query per page (`thread_counts` in every row-rendering context; the `thread_count` filter) and are refreshed client-side on `HX-Trigger: threadCountsChanged`, so a render site that forgets the context under-counts until the next action, never errors. Notification mail goes to the thread's participants (creator, authors, everyone mentioned), never the actor, one Celery task per recipient (`survey/tasks.py`) through `survey/mail.py::send_templated_mail` and `SITE_URL` (defaults to `NEWSLETTER_SITE_URL`); the same helper is where invitation/activation mail should move. Deep link: `?section|block|session=#thread-`. "New since you last looked" is `CommentSeen` (one row per member+survey, bumped when the drawer renders): `open_counts(survey, user)` also returns `new_keys`/`new_total`, which drive the red dot on badges, the toolbar count and the "N new" pill — so every render site passes `request.user`. Authors edit their own comments in place (`edited_at`, no re-notification). Response anchors are labelled with the Responses ordinal (`comments.session_seq`), never the session id. **Editing a live survey (`LIVE_SURVEY_EDITING` kill switch, default ON; change `edit-live-survey`)**: a published/closed survey takes SAFE edits in place (wording, help text, translations, added options, map/display settings) with no draft and no new version; STRUCTURAL edits (add/delete/reorder/paste, type, layer binding, visibility rules, section code, removing an ANSWERED option) go to the draft copy, which every creator surface calls **"Unpublished changes"** (Publish changes / Discard changes; "version" only on Responses and exports). One rule set in `versioning.py` (`is_live`, `structural_question_changes`, `structural_section_changes`, `removed_answered_choices`, shared with `check_draft_compatibility`). Views: `_edit_locks(survey)` → `(is_read_only, structure_locked)`; content endpoints call `_check_content_edit_allowed`, structural ones keep `_check_structural_edit_allowed` (403). Templates gate structural controls on `structure_locked` with `data-gate=""` (opens the gate modal in `survey_detail.html`, which posts `then` + `section_code` to `editor_create_draft`, `editor` role and up — owners alone publish/discard), content controls on `is_read_only`. A new editor control must pick one of the two — never `survey.status`. The draft copy and the editor's duplicate/paste take every plain `SurveySection`/`Question`/translation field from the model's field list (`cloning.plain_field_values`, `copy_section`); a field that must NOT travel is an explicit exclusion there, never an omission — #329 was a hand-written list in `versioning.py` that silently dropped `visibility_rule`, `layout`, `next_label` and `hidden_layers` on every publish. `DraftCopyKeepsFieldsTest` compares every concrete field between live and draft, and `report_draft_field_loss` (read-only) lists surveys whose publish already lost them. While a draft exists, Build redirects to it (`?live=1` shows the live survey read-only) and live content edits are refused, because publishing the draft replaces the live structure. Both confirmations (the gate and "Publish changes") carry "Don't ask again": ONE browser-side preference, `localStorage['msEditorConfirmations']` (`window.EditorConfirm` in `editor_base.html`), restored from the account menu; the compatibility dialog is never skipped. Switch off ⇒ the old read-only page and "draft" vocabulary. **After the last page (`SurveyHeader.redirect_url`, change `fix-redirect-url-validation`, #271)**: the value is `"#"` (built-in thanks page) or an absolute http(s) URL, nothing else. Never read it raw: respondent redirects go through `survey/redirects.py::completion_url(survey)`, imports through `clean_redirect_url`. It used to be free text, and hashtags, thank-you messages and `#https://…` (typed after the pre-filled `#`) kept respondents pressing Finish on the last page. Settings shows it as the "After the last page" choice; the form refuses anything that is not a full http(s) address. **Ways to start (change `first-survey-onboarding`)**: a workspace with no surveys and, by default, "+ New Survey" lead to the START PAGE `/editor/surveys/start/` (`editor_survey_start`) — two cards, *Survey wizard* / *Start from scratch* — not to the create page. The third card (*Use your AI assistant*) was removed in change `onboarding-friction` (#638): nothing on the first screen that does not make a survey; the agent page is reached from the sidebar's *MCP* item, the "?" menu and the help centre. Before the first survey (`app_nav.onboarding`: no canonical, non-trashed survey in the active workspace; owner/admin/editor) workspace-level pages open the sidebar as the rail unless the creator chose Expand (`msSidebar = 'wide'`). **One click on a card does its action** (change `start-cards-one-click`, #335): the blank card IS the submit `