name: Post-Merge Verifier description: Smoke-tests the live app after a PR is merged. triggers: - context: projects: projectIds: - 019d8bf4-1ded-7317-be2f-555e8fb55ff9 pullRequest: webhookId: 019d8d97-eb6e-70c4-a97d-25952abbc5a7 events: - PULL_REQUEST_EVENT_MERGED action: limits: maxParallel: 1 maxTotal: 50 steps: - agent: prompt: | You are the Post-Merge Verifier. A PR was just merged. Verify the live app still works. The PR number is provided in the trigger context as the pull request that was merged. ## Step 1 — Determine if verification is needed Read the merged PR metadata: gh pr view --json title,labels Skip verification entirely if the PR title starts with `chore:`, `docs:`, or `refactor:`. These PRs don't affect the live app. For `feat:` and `fix:` PRs, proceed. ## Step 2 — Wait for deploy Wait 90 seconds for Vercel to deploy the merge to production. ## Step 3 — Run E2E test suite First, run the project's E2E test suite against the live site: BASE_URL=https://memo.software-factory.dev pnpm test:e2e If E2E tests fail, include the failures in the report (Step 4). If E2E tests pass, still run the ad-hoc smoke tests below for routes not yet covered by the E2E suite. ## Step 3b — Run ad-hoc smoke tests Write a Playwright script to /tmp/smoke-test.mjs. The script must only test routes that exist. Before testing a route, do a HEAD request first — if it returns 404, skip that check (do not count it as a failure). Test user credentials are available as env vars: TEST_USER_EMAIL, TEST_USER_PASSWORD Use these for any authenticated flows (e.g., login, dashboard access). ```js import { chromium } from 'playwright'; const BASE = 'https://memo.software-factory.dev'; const failures = []; const skipped = []; const browser = await chromium.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] }); const page = await browser.newPage(); const consoleErrors = []; page.on('console', m => { if (m.type() === 'error') consoleErrors.push(m.text()); }); // Helper: check if a route exists before testing it async function routeExists(path) { try { const res = await fetch(BASE + path, { method: 'HEAD', redirect: 'manual' }); return res.status !== 404; } catch { return false; } } // 1. Landing page (always exists) const res = await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }); if (!res || res.status() >= 400) failures.push('Landing page returned ' + (res?.status() ?? 'no response')); const title = await page.title(); if (!title) failures.push('Landing page has no title'); // 2. Sign-in page if (await routeExists('/sign-in')) { await page.goto(BASE + '/sign-in', { waitUntil: 'networkidle', timeout: 15000 }); const hasEmailInput = await page.locator('input[type=email]').count(); if (!hasEmailInput) failures.push('Sign-in page missing email input'); } else { skipped.push('/sign-in (not yet built)'); } // 3. Health endpoint const healthRes = await page.goto(BASE + '/api/health', { waitUntil: 'networkidle', timeout: 10000 }); const healthBody = await page.textContent('body'); if (!healthRes || healthRes.status() >= 400) failures.push('Health endpoint returned ' + (healthRes?.status() ?? 'no response')); if (healthBody && healthBody.includes('"status":"down"')) failures.push('Health endpoint reports down'); // 4. Dashboard (skip if not yet built — requires auth) if (await routeExists('/dashboard')) { const dashRes = await page.goto(BASE + '/dashboard', { waitUntil: 'networkidle', timeout: 15000 }); // Unauthenticated should redirect to /login, not 500 if (dashRes && dashRes.status() >= 500) failures.push('Dashboard returned ' + dashRes.status()); } else { skipped.push('/dashboard (not yet built)'); } // 5. Console errors (unauthenticated) if (consoleErrors.length) failures.push('Console errors (unauth): ' + consoleErrors.slice(0, 5).join('; ')); // --- Authenticated flow --- const EMAIL = process.env.TEST_USER_EMAIL; const PASSWORD = process.env.TEST_USER_PASSWORD; if (EMAIL && PASSWORD) { // 6. Sign in via the login form const authPage = await browser.newPage(); const authErrors = []; authPage.on('console', m => { if (m.type() === 'error') authErrors.push(m.text()); }); try { await authPage.goto(BASE + '/sign-in', { waitUntil: 'networkidle', timeout: 15000 }); await authPage.fill('input[type="email"]', EMAIL); await authPage.fill('input[type="password"]', PASSWORD); await authPage.click('button[type="submit"]'); // Wait for redirect away from sign-in (lands on /{workspaceSlug}) await authPage.waitForURL(url => !url.pathname.includes('/sign-in'), { timeout: 15000 }); const postLoginUrl = authPage.url(); const postLoginRes = authPage.context().pages().length; // sanity if (postLoginUrl.includes('/sign-in')) { failures.push('Login did not redirect — still on /sign-in'); } else { // 7. Workspace page loaded successfully const wsStatus = await authPage.evaluate(() => document.readyState); if (wsStatus !== 'complete') failures.push('Workspace page did not fully load'); // 8. Navigate to a page (editor) via sidebar button // The sidebar uses button elements for page navigation, not links. // Page buttons contain a relative timestamp (e.g. "Test Page\n21m ago"). const pageButtons = authPage.locator('button').filter({ hasText: /ago/ }); const pageBtnCount = await pageButtons.count(); if (pageBtnCount > 0) { await pageButtons.first().click(); // Wait for URL to change from /{workspace} to /{workspace}/{pageId} await authPage.waitForURL( url => url.pathname.split('/').filter(Boolean).length >= 2, { timeout: 10000 } ).catch(() => {}); const editorUrl = authPage.url(); if (editorUrl.includes('/sign-in')) { failures.push('Editor page redirected to sign-in (auth lost)'); } else if (new URL(editorUrl).pathname.split('/').filter(Boolean).length < 2) { skipped.push('Page button click did not navigate to editor route'); } else { // Check the editor container rendered const editorEl = await authPage.locator('[data-lexical-editor], [role="textbox"], [contenteditable="true"]').count(); if (editorEl === 0) { skipped.push('Editor element not found on page (may not be implemented yet)'); } } } else { skipped.push('No page buttons found in workspace to test editor route'); } } // 9. Console errors (authenticated) if (authErrors.length) failures.push('Console errors (auth): ' + authErrors.slice(0, 5).join('; ')); } catch (authErr) { failures.push('Authenticated flow error: ' + authErr.message); } await authPage.close(); } else { skipped.push('Authenticated routes (TEST_USER_EMAIL / TEST_USER_PASSWORD not set)'); } await browser.close(); if (skipped.length) console.log('Skipped: ' + skipped.join(', ')); if (failures.length) { console.error(JSON.stringify(failures)); process.exit(1); } console.log('OK'); ``` Run: `node /tmp/smoke-test.mjs` Delete: `rm -f /tmp/smoke-test.mjs` ## Step 3c — Interaction smoke test for feature PRs If the merged PR title starts with `feat:`, perform a lightweight interaction check on the affected feature area. This catches bugs where E2E tests exist but don't cover a specific interaction path. 1. Read the PR diff to identify which pages/components were added or modified. 2. Navigate to the affected page on the live site (authenticated if needed). 3. Perform basic interactions relevant to the feature: - Click buttons, open dropdowns, fill inputs - Verify dropdowns/popovers open and close correctly - Verify form submissions don't produce console errors - Verify no layout overflow or clipping issues (elements not cut off) 4. Take a screenshot of the page after interactions. 5. Check for console errors during the interaction flow. 6. If any interaction fails or produces errors, include it in the failure report. This step is a safety net — it does not replace E2E tests. If this step catches a bug that E2E tests missed, the bug issue should note that E2E coverage is incomplete for the affected interaction. ## Step 4 — Report results Find the merged PR number from the trigger context. If all checks pass: Comment on the merged PR: > ✅ Post-merge verification passed. [list which routes were tested and which were skipped] If any check fails: 1. Create a GitHub Issue: - Title: `bug: UI regression after PR # — ` - Labels: `bug`, `priority:1`, `status:backlog` - Body must include ALL of these sections: ## Description ## Acceptance Criteria - [ ] All previously passing smoke test checks pass again - [ ] - [ ] `pnpm lint && pnpm typecheck && pnpm test` pass ## Technical Notes - List all failures with details - PR that introduced the regression: # - Affected routes/components 2. Comment on the merged PR: > ❌ Post-merge verification failed. See #. ## Expanding checks As features ship, add new route checks to the Playwright script. Always use the `routeExists()` guard so the script doesn't fail on routes that haven't been built yet. Test user credentials for authenticated flows: TEST_USER_EMAIL, TEST_USER_PASSWORD (available as env vars) ## Do NOT - Retry failed checks — report the failure and stop. - Fix issues yourself — create the bug issue and let the Feature Builder or a human handle it. - Run verification for chore/docs/refactor PRs.