---
name: testing-strategy
description: "Writes and structures tests: TDD red-green-refactor, AAA pattern, test naming, pyramid placement, what to mock. Use when adding tests, fixing flaky or failing tests, deciding unit vs integration vs e2e, measuring coverage, or before implementing any new behavior."
license: MIT
metadata:
  author: lammanhhoang
  version: "1.0.0"
---

# Testing Strategy

## When to use

- Before implementing ANY new behavior (feature, bug fix, edge case handling).
- When adding tests to existing code, reviewing test quality, or restructuring a suite.
- When a test is flaky, slow, or broke after a refactor.
- When deciding where a test belongs: unit, integration, or e2e.
- When asked about coverage targets or mocking strategy.

When NOT to use: exploratory spikes you will throw away (delete the spike, then TDD the real thing), generated code you don't own, one-off migration scripts that run once under supervision.

## Iron Law

**NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST.**

A test written after the code, that passes on its first run, proves nothing. You never saw it fail, so you don't know it CAN fail. It may assert nothing, test the wrong thing, or pass for the wrong reason. The red step is the only evidence the test is wired to the behavior.

Already wrote the code? Recovery procedure — no exceptions:

```bash
git stash                # 1. Hide the implementation
# 2. Write the test against the code that no longer exists
<test command>           # 3. Run it. Watch it FAIL for the RIGHT reason
git stash pop            # 4. Restore the implementation
<test command>           # 5. Watch it pass
```

If step 3 passes with the implementation stashed, the test is broken. Fix the test before restoring anything.

If the implementation is already committed, check out the pre-implementation commit on a temp branch (or git revert it), verify the test fails there, then restore.

## Workflow: red-green-refactor

1. **Red.** Write ONE test for the next smallest behavior. Run it. Confirm it fails with the failure you expected (assertion error on the behavior — not an import error, not a typo). Paste the failure output.
2. **Green.** Write the MINIMUM production code to pass. Hardcoding is allowed; the next test forces generalization. Run the test. Paste the pass output.
3. **Refactor.** Clean up production code AND test code with the suite green. Run the full suite after every change. No new behavior in this step.
4. Repeat from 1 until the requirement is covered, including error paths.
5. Before claiming done: run the FULL suite and paste actual output (see Verification below).

## Rationalization table

| Excuse | Counter |
|---|---|
| "It's too simple to test" | Simple code breaks too; the test takes 2 minutes and pins the contract forever. |
| "I'll add tests after" | After-tests pass immediately and prove nothing; also "after" statistically never comes. |
| "Just this once" | Once is the unit of habit. The Iron Law has no exception clause. |
| "The deadline" | Debugging untested code is slower than TDD; you're borrowing time at 100% interest. |
| "It's a prototype" | Prototypes ship. If it might survive the week, it gets tests first. |
| "Manual testing confirmed it works" | Manual testing confirmed it worked once, on your machine, for the inputs you remembered to try. |
| "The test framework isn't set up" | Setting it up IS the first task. references/PATTERNS.md has the exact commands. |
| "This code is untestable" | Untestable = badly designed. The test is telling you to fix the coupling, not to skip the test. |

If you catch yourself producing a sentence like these: Stop. That's rationalization. Write the failing test.

## Red flags — stop immediately if you notice

- [ ] You are editing a production file and no new test failed in the last few minutes.
- [ ] A new test passed on its very first run.
- [ ] You are about to say "should work", "looks correct", or "I'm confident" instead of pasting test output.
- [ ] A test has an `if`, `for`, `try/catch`, or computes its own expected value.
- [ ] A test imports a private/underscore function or reaches into internal state.
- [ ] You are adding a retry, sleep, or `--rerun-failed` to make a flaky test pass.
- [ ] You are mocking a function that lives in the same module you are testing.
- [ ] Two tests share a mutable fixture and one mutates it.
- [ ] You raised a timeout to fix an intermittent failure.

## Test structure rules

**AAA with blank-line separation.** Arrange, Act, Assert — three blocks, one blank line between them. The Act is ONE line. Aim for one assertion focus per test (multiple assertions on the same result object are fine; asserting two unrelated behaviors is two tests).

**Naming: three parts.** Unit under test / scenario / expected outcome. A failure message must tell you what broke without opening the file.

```js
// BAD
it('works', () => { ... })
it('test order 2', () => { ... })

// GOOD
describe('createOrder', () => {
  describe('when stock is zero', () => {
    it('rejects with OutOfStock', async () => {
      const store = makeStore({ stock: 0 });          // Arrange

      const result = createOrder(store, 'sku-1', 1);   // Act (one line)

      await expect(result).rejects.toThrow(OutOfStock); // Assert
    });
  });
});
```

**Flat, short, zero logic.** No `if`, no loops, no `try/catch`-as-assertion in tests. A test with logic needs its own test. Parametrize instead of looping — `test.each` (vitest/jest), `@pytest.mark.parametrize`, table-driven tests (Go). Recipes: references/PATTERNS.md (load when writing tests in a specific framework).

**Black-box only.** Test public behavior through the public API. Never assert on private functions, internal call order, or intermediate state. Litmus test: if a pure refactor (no behavior change) breaks tests, the tests were coupled to internals — that is a design smell in the tests. Delete or rewrite them against the public contract; do not "update the mocks to match".

## Test doubles

Default: **stubs and spies. Avoid mocks** (pre-programmed interaction expectations that verify HOW instead of WHAT).

| Situation | Double | Why |
|---|---|---|
| Query the SUT reads from (repo returns data) | Stub | Canned answer; assert on SUT output |
| Command the SUT sends out (email, event publish) | Spy | Record the call; assert it happened with the right args |
| Network, HTTP APIs | Stub at the boundary (msw / `httptest` / `responses`) | Never let tests hit the wire |
| Clock / `Date.now` / `time.Now` | Fake timer or injected clock | Real time = flaky |
| Randomness / UUIDs | Seeded or injected generator | Determinism |
| Filesystem | Temp dir (`tmp_path`, `t.TempDir()`, `mkdtemp`) | Isolation without mocking `fs` |
| Your own module's internal helper | **NOTHING — do not double it** | Mocking internals welds tests to implementation |
| The module under test itself (partial mock) | **NEVER** | You'd be testing the mock |

Mock ONLY true externals: network, clock, randomness, filesystem, environment. Everything in-process and owned by you runs real.

**Per-test data ownership.** Every test constructs its own data via factory functions with overrides (`makeUser({ role: 'admin' })`). Never mutate a shared fixture; never depend on data another test created; never depend on test order. If tests only pass in order, they are broken now — the shuffle just hasn't happened yet (`go test -shuffle=on`; with pytest, install `pytest-randomly` — it randomizes by default; never pass `-p no:randomly` to hide order bugs).

## Test level placement

Component/API-level integration tests are the minimum bar and the best ROI: test a service through its API with real in-process collaborators and stubbed externals. Start there, not at either extreme — this distribution is deliberately trophy-shaped (integration-heavy) rather than the classic pyramid.

| Level | Scope | Doubles | Quantity |
|---|---|---|---|
| Unit | One module's logic, pure decisions | None or trivial stubs | Many, milliseconds each |
| Integration / component | Service through public API, real DB or containerized deps, stubbed 3rd parties | External boundaries only | The backbone of the suite |
| E2E | Full deployed stack, browser/client | Nothing | FEW — critical user journeys only (login, checkout-class flows), single digits to low tens |

**Push tests down.** If a failure appears at e2e and no lower-level test failed, that is a coverage gap AND a duplication signal: write the missing lower-level test that reproduces the failure, then delete the duplicated e2e coverage (keep the e2e only if it guards a genuine cross-system journey). Same rule one level down: integration failure with no unit failure on a logic bug → write the unit test.

Never test the same behavior at two levels on purpose. Duplicate coverage doubles maintenance and halves signal.

## Flaky tests protocol

A flaky test is worse than no test: it trains everyone to ignore red.

1. **Same day:** quarantine it — `it.skip` / `@pytest.mark.skip(reason="FLAKY #123")` / `t.Skip("FLAKY #123")` with a tracking issue. A quarantined test is a P1 bug, not an archive.
2. **Within the week:** fix the root cause. It is almost always one of: missing `await` / unresolved promise / race; shared state between tests (module-level variable, DB row, port); real time (timeouts, `Date.now`, midnight/timezone); test-order dependence; real network.
3. **Never** fix flakiness with retries, `--rerun-failed`, sleeps, or raised timeouts. Retry-looping a race condition ships the race to production.
4. Reproduce deterministically before fixing: `pytest --count=50 -x` (pytest-repeat), `go test -race -count=100 -run TestName`, `vitest --sequence.shuffle`.

## Verification before completion

Run the FULL suite — not just the new tests — and paste the ACTUAL output: pass/fail counts, the command used, exit status. "It should pass" is not a test result. "I ran the relevant tests" is not the full suite. Reasoning about code is not executing code.

```
$ npx vitest run
 Test Files  14 passed (14)
      Tests  212 passed (212)
```

No pasted output → not done. If the suite is too slow to run fully, that is a suite-health problem to raise, not a license to skip.

## Coverage

Coverage is a floor, not a goal. It measures execution, not verification: 100% coverage with assertion-free tests is worse than 70% honest coverage, because it manufactures false confidence. Rules:

- Set a floor (e.g., fail CI under 80% on changed lines) to catch untested new code; never chase 100%.
- Never write a test whose purpose is "make the coverage number go up". Write it for a behavior; coverage follows.
- To audit whether tests actually verify anything, use mutation testing (Stryker for JS/TS, mutmut for Python) on critical modules — surviving mutants are lies in your coverage report.

## What NOT to do

- Do not write production code before a failing test. (Iron Law. Stash procedure above.)
- Do not keep a test you never saw fail.
- Do not assert on internals, call order of private helpers, or mock return plumbing.
- Do not put logic in tests. Parametrize.
- Do not share mutable fixtures across tests.
- Do not mock the module under test or anything in-process that you own.
- Do not retry, sleep, or raise timeouts to silence flakiness.
- Do not add e2e coverage for behavior a lower level can prove.
- Do not report success without pasted full-suite output.
- Do not treat a coverage percentage as evidence of correctness.

## Files

- references/PATTERNS.md — per-framework recipes (vitest/jest, pytest, go test): setup, parametrized cases, async, fake timers, spies. Load when writing tests in one of these frameworks.
- assets/test-checklist.md — copy into PR descriptions or run as a pre-completion gate.