--- title: Visual Regression Testing category: Contributing/Testing order: 4 --- # Visual regression testing InstUI runs visual regression tests on every pull request. They guard against unintended pixel-level changes to rendered components — a padding bump, an accidental color shift, a layout regression that a unit test wouldn't catch. ## How it works On every PR: 1. **Build & capture.** GitHub Actions builds InstUI, builds the `regression-test` Next.js app as a static site, and serves it. Cypress visits every page under `regression-test/src/app/*` once per theme (via the `?theme=` query param), waits for each to render, and calls `cy.screenshot()` directly in the spec, producing one PNG per page/theme combination. 2. **Diff.** The `ui-scripts visual-diff` command compares each screenshot against the matching baseline fetched from the `visual-baselines` branch of the repo. It uses [pixelmatch](https://github.com/mapbox/pixelmatch) with antialiasing detection enabled, so typical font-rendering noise doesn't register as a diff. 3. **Publish.** The diff script writes an HTML report (with baseline / actual / diff for every row, plus a lightbox viewer) and the action publishes it to the `gh-pages` branch under `visual-regression/pr-/`. The report is reachable at `https://instructure.design/visual-regression/pr-/index.html`. 4. **Comment.** A sticky PR comment summarizes the result, links to the report, and inlines each changed row's diff image inside a collapsed `
` block. 5. **Fail.** If any screenshot is marked `Changed` or `Removed`, the `Capture, diff, and publish report` job fails. `New` screenshots (no matching baseline) are tolerated so a contributor who adds a new component page doesn't need to pre-seed a baseline. 6. **Cleanup.** When the PR is closed or merged, a separate workflow removes `visual-regression/pr-/` from `gh-pages`. Baselines are refreshed automatically: every push to `master` triggers the `Update visual baselines` workflow, which re-runs Cypress and force-pushes the fresh screenshots to the `visual-baselines` branch. There is no manual review or approval step — merging is the approval. ## Local development ### Running the full loop against local changes From the repo root: ```sh --- type: code --- pnpm install && pnpm run bootstrap cd regression-test npm install npm run build # produces out/ npm start & # serves the static export on http://localhost:3000 npm run cypress # captures screenshots ``` The resulting PNGs land in `regression-test/cypress/screenshots/spec.cy.ts/`. To compare them against a local baseline set: ```sh --- type: code --- # Seed baselines by copying a known-good capture run mkdir -p .baselines cp cypress/screenshots/spec.cy.ts/*.png .baselines/ # Later, after a visual change, capture again and diff cp cypress/screenshots/spec.cy.ts/*.png .actual/ npm run visual-diff open .report/index.html ``` `npm run visual-diff` invokes the same `ui-scripts visual-diff` command CI uses. Pass extra flags to override the defaults: | Flag | Default | Description | | -------------------------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--actual-dir ` | `cypress/screenshots` | Newly captured screenshots. | | `--baseline-dir ` | `.baselines` | Baselines to compare against. | | `--output-dir ` | `visual-report` | Where the HTML report + diff PNGs are written. | | `--threshold <0..1>` | `0.1` | Per-pixel YIQ color threshold. Lower is stricter. | | `--fail-on-missing-baseline` / `--no-fail-on-missing-baseline` | on | Whether to exit 1 when actual screenshots have no baseline. | | `--pr-number ` | — | Rendered in the report header as a link to the PR. | | `--pr-url ` | — | Target of the PR link. | | `--meta ` | — | JSON mapping screenshot slug → visited URL path. Enables per-row source-file links in the report. Produced automatically by `spec.cy.ts` via `cy.task('recordMeta', ...)`. | | `--source-base-url ` | — | GitHub blob URL of `regression-test/src/app` on the branch being reviewed. Combined with `--meta` to build links like `treebrowser/page.tsx`. | | `--facets ` | — | Comma-separated facet suffixes (e.g. `canvas,light,dark`) rendered as one-click filter chips that match screenshots named `-`. | | `--app-path ` | — | Path, relative to the report root, where a static export of the app is published (CI uses `app`). Enables the lightbox **HTML** view, which iframes the live rendered page. | | `--a11y ` | — | JSON of axe violations keyed by screenshot slug. Enables everything in [Accessibility findings](#accessibility-findings) below. Produced automatically by `spec.cy.ts` via `cy.task('recordA11y', ...)`. | ### Interpreting the report The HTML report is a single self-contained page showing one section per screenshot. Each row has a status badge (`ok`, `changed`, `new`, `removed`), a three-up grid of baseline / actual / diff images, and — when metadata is available — a link to the source `page.tsx`. Top-bar controls: - **Filter buttons** (`All`, `Changed`, `New`, `Removed`, `Unchanged`, `⚠ A11y`) — the active one is highlighted. - **Search input** — live, debounced substring filter on the screenshot name. - **Theme chips** — narrow to one theme's screenshots. - **Lightbox** — click any thumbnail to open a fullscreen viewer. Inside: - `Baseline`, `Actual`, `Diff` buttons switch between the three images. - `Slider` mode overlays baseline and actual with a drag handle so you can scrub the boundary back and forth. - `⚠ N a11y` mode shows the screenshot at full size with every violating element boxed, and the findings docked alongside. Shown only for screenshots with findings. - `HTML` mode iframes the live rendered page (a static export of the app published next to the report) so you can inspect the real DOM, computed styles, and text alongside the pixels. Shown only when the page still exists and the app was published (`--app-path`). - `1:1` / `Fit` toggles between native pixel size (scrollable) and fit-to-window. - `‹` / `›` step through visible rows (respects the current filter). ### Accessibility findings axe runs against every page and theme, and the report shows what it found **on the picture**, so a designer can act on it without reading a selector. While the page is still on screen, `spec.cy.ts` records each violating element's bounding box, a human-readable name for it, and — for contrast failures — the foreground/background colors and ratios axe computed (see `cypress/support/a11y-capture.ts`). The report then renders: - **A run-level overview** at the top of the page: one line per rule with its impact, how many elements it affects, and how many screenshots it appears on. The same rule tripping in `canvas`, `light`, and `dark` collapses to a single line. Click a rule to narrow the list to just the screenshots it affects. - **A `⚠ N a11y` badge** in each affected screenshot's header, colored by the worst impact on it. That badge is the only a11y marking in the list — rows stay a clean three-up image comparison. **Click it** to open the lightbox's **A11y** view. - **The A11y view**: the screenshot at full size with every violating element boxed and numbered, colored by impact, and the findings listed beside it. Clicking a card highlights its box and vice versa. Findings are ordered worst-impact-first, so `#1` is always the most serious thing on that screenshot. - **One card per offending element**, leading with a plain-language headline ("Text contrast is too low") rather than the rule id, naming the element (`button “Archive this item”`), and suggesting a fix. Contrast failures show both colors as swatches with the measured and required ratios. The raw axe output — rule id, selector, and `failureSummary` — sits behind **Technical details**. The boxes and cards are rendered into an inert `