# Testing Guide

This document provides comprehensive information about testing in the Actual-sync project.

---

## 📊 Overview

The project uses **Jest** as the testing framework with comprehensive unit and integration tests covering:
- Configuration loading and validation
- Retry logic with exponential backoff
- Sync service integration
- Startup validation
- Health check endpoints
- Notification services
- Prometheus metrics
- Telegram bot commands
- Sync history tracking

**Current test count and coverage**: see the live **Tests** and **Coverage** badges at the top of the [README](../README.md) (regenerated each release from the Jest run). Enforced thresholds: 70% lines/functions/statements, 61% branches.

---

## 🚀 Quick Start

### Running Tests

```bash
# Run all tests
npm test

# Run tests in watch mode (re-runs on file changes)
npm run test:watch

# Run tests with coverage report
npm run test:coverage

# Prove the tests would actually catch the bugs they were written for
npm run test:mutation
```

### Mutation testing

`npm run test:mutation` reintroduces the original defect behind each shipped fix, one at a time, and asserts the suite **fails**. A mutation that survives is a fix with no test guarding it — a green suite that would not notice the bug coming back.

```bash
npm run test:mutation                    # every mutation, full suite each (~5 min)
npm run test:mutation -- --fast          # only the hinted test file per mutation (~1 min)
npm run test:mutation -- --ticket '#177' # one ticket's mutations
npm run test:mutation -- --list          # what is covered, without running anything
npm run test:mutation -- --recover       # restore after a hard kill
```

`mutationCatalog.test.js` is excluded from the runs. It asserts every anchor still exists, and a mutant replaces its anchor — so leaving it in made every mutation "caught" by construction, with the runner scoring its own self-check and unable to ever report a survivor. That bug shipped in the first version of this tool and is the reason the exclusion is pinned by a test.

It exists because three consecutive review rounds on a single change each found a fix that **no test protected**: source-text assertions that passed against a fully reintroduced bug, a parity regex satisfied by a leftover `require`, and a heuristic whose branch could be deleted with the suite still green. Reading a test and judging it plausible does not answer *"would this catch the bug?"* — only reintroducing the bug does.

The catalog covers behaviour at every level, not just service code — a mutation reintroducing the README's old "notifies you of failures" claim is caught by the doc-drift guard, and one relaxing the email `required: [from, to]` schema rule is caught by the config tests. If a guard cannot be broken by a mutation, it is not guarding anything.

**When you fix a bug, add a mutation for it.** The catalog lives in `scripts/mutations.js`; each entry names the file, the exact `anchor` text to replace, and the `mutant` that reintroduces the defect. `src/__tests__/mutationCatalog.test.js` runs in the normal suite and fails if any anchor no longer matches its file — without it, a refactor would silently turn mutations into no-ops and the catalog would rot into false confidence.

It is deliberately **not** wired into CI: it runs the whole suite once per mutation, which is far too slow per-PR. Run it when you change notification dispatch, config validation, or anything else the catalog covers, and before a release.

**It refuses to run without a green baseline.** The suite is run unmutated first, and the run aborts unless that baseline is genuinely clean: no failed tests, no suites that failed to *load*, and jest's own `success` flag set. Otherwise every mutation would score "caught" for a reason that has nothing to do with the mutation — a colleague's WIP test or a flake would produce a perfect score.

**It baselines what it is about to score.** In `--fast` mode each mutation is scored against its hinted test file alone, so each distinct hint is baselined alone — not the full suite. A full-suite baseline would not rule out the case that matters here: a file that passes as part of the suite but fails *standalone*, through leaked global state or an ordering dependency on a sibling file. That would fail under every mutant and score all of them `caught`, having proven nothing. Each distinct pattern is baselined once, so this is cheaper than the single full-suite baseline it replaced, not more expensive.

The load-error case is separate on purpose. A suite that throws while being imported contributes **zero failed tests** to jest's report while `numTotalTests` stays non-zero, so a "did tests run, did any fail" reading called it green. With one unresolvable import the runner printed `green (841 tests)` while `npm test` exited 1, and a mutant then scored a false `SURVIVED` for a defect that *was* guarded — the exact phantom coverage gap this tool exists to rule out.

**Scoring reads jest's JSON report, not its exit code.** jest exits 1 for "no tests found" and for a config error as well as for a real failure, so an exit-code reading counted those as caught having run nothing. A mutation is caught only when tests actually ran *and* at least one failed. A run with a load error, or one jest calls unsuccessful while reporting no failures, is `UNSCORED` rather than a verdict — the run fails, but it does not claim a coverage gap it has not demonstrated.

**The baseline is a single sample.** It rules out a suite that is already broken; it cannot rule out a genuinely flaky test, which would fail under some mutant and be recorded as `caught` for the wrong reason. If a mutation's result surprises you, re-run that ticket with `--ticket` before believing it.

**Safety.** Before a file is touched, both its original content and the mutant are journalled to disk atomically, so a hard kill is recoverable with `--recover`. `--recover` refuses to overwrite content that is neither the mutant nor the original — that is how it would otherwise destroy a repair you made by hand — and copies your version aside first. A pid lockfile blocks concurrent runs, and is only ever removed by the process that took it.

`--recover` also **refuses while a live run holds the lock**. Recovering mid-flight restores the original underneath the suite currently testing the mutant, so that suite passes against unmutated code and the mutation is scored `SURVIVED` — the tool reporting a coverage gap it just manufactured. `--recover` is dispatched before the catalog is loaded, so a broken `scripts/mutations.js` cannot block recovery of a file an earlier run left mutated.

Signal handlers are deliberately **not** installed. `main()` is synchronous, so a handler could never fire while jest is running; registering one only removed the default terminate behaviour, leaving a runner that `kill` could not stop. The journal is the recovery mechanism — if a run is killed, the next one refuses to start and tells you to run `--recover`.

One honest limitation: the uncommitted-changes check is a snapshot taken at startup, so a file created *during* a run is not covered by it. A file *modified* during a run is detected — the runner leaves your version alone and stops, rather than reverting it. That includes the case where the file was put **back to its original content** while the mutant was in place, which earlier read as the benign "the mutant write failed" path and said nothing.

**The runner mutates itself too.** `scripts/mutationTest.js` is in the catalog, which works because jest re-reads the file from disk while the running process keeps its own copy in the require cache. Those mutations are guarded by `src/__tests__/mutationRunner.test.js` and deliberately **not** by `mutationCatalog.test.js` — the runner excludes the catalog guard from the scored suite, so a guard placed there would score every one of them as `SURVIVED` regardless of the truth.

**It is not available inside the container.** `scripts/mutationTest.js` and `scripts/mutations.js` are excluded via `.dockerignore` (#180), so `npm run test:mutation` only works in a checkout. A tool whose job is rewriting source files in place has no business in a production image. `scripts/validateConfig.js` is *not* excluded — `npm run validate-config` in the container is the documented pre-upgrade check (#177) — so `scripts/` cannot be ignored wholesale, and a test in `docDriftGuards.test.js` pins both halves of that.

Exit codes: `0` all caught, `1` at least one survived (or a stale anchor / unscored run), `2` the runner refused to start or failed internally, `3` a file was left mutated and needs `--recover`. `1` is a claim about coverage, so an internal crash must never exit `1`.

### Documented config examples

`src/__tests__/configExamplesGuard.test.js` validates every fenced `json` config block in `README.md`, `docs/NOTIFICATIONS.md`, `docs/CONFIG.md`, `docs/MIGRATION.md` and `docs/DOCKER_DEPLOYMENT.md` against the real schema. A config snippet in the docs is advice people paste, so a wrong one is worse than no advice — this caught three invalid documented examples on its first run, each of which would have caused a startup failure.

If a documented JSON block is genuinely **not** configuration — an API response, a webhook payload — opt it out with a marker on the line before the fence:

```markdown
<!-- config-guard: skip -->
```

The marker applies only to the block immediately following it.

### Test Output

```
Test Suites: all passing
Tests:       all passing      # live count + coverage are on the README badges
Snapshots:   0 total
Time:        ~8 s
```

---

## 📁 Test Structure

```
src/
├── __tests__/
│   ├── helpers/
│   │   └── testHelpers.js         # Shared test utilities
│   ├── configLoader.test.js       # ConfigLoader unit tests
│   ├── retryLogic.test.js         # Retry logic unit tests
│   ├── syncService.test.js        # Sync service integration tests
│   └── startupValidation.test.js  # Startup validation tests
└── lib/
    └── configLoader.js            # Module under test
```

---

## 🧪 Test Suites

### 1. ConfigLoader Tests (`configLoader.test.js`)

Tests configuration loading, validation, and error handling:

- **Constructor**: Default and custom path handling
- **load()**: File existence, JSON parsing, schema validation
- **validateConfig()**: Schema validation with AJV
- **validateLogic()**: Business rules (duplicates, required fields, security warnings)
- **getConfig()**: Configuration retrieval
- **getServer()**: Server lookup by name
- **getServers()**: All servers retrieval

**Example Test**:
```javascript
test('should throw error for duplicate server names', () => {
    const config = {
        servers: [
            { name: 'Test', url: 'https://test1.com', ... },
            { name: 'Test', url: 'https://test2.com', ... }
        ]
    };
    const loader = new ConfigLoader(configPath);
    
    expect(() => loader.validateLogic(config))
        .toThrow('Duplicate server names');
});
```

### 2. Retry Logic Tests (`retryLogic.test.js`)

Tests exponential backoff and retry behavior:

- **Success scenarios**: First attempt, eventual success
- **Rate limit handling**: Exponential backoff, max retries
- **Network error handling**: ECONNRESET, ENOTFOUND, network-failure
- **Non-retryable errors**: Immediate failure
- **Exponential backoff**: Delay calculation validation
- **Edge cases**: Zero retries, null/undefined returns

**Example Test**:
```javascript
test('should retry with exponential backoff on rate limit', async () => {
    let attempts = 0;
    const rateLimitFn = jest.fn().mockImplementation(() => {
        attempts++;
        if (attempts < 3) {
            const error = new Error('Rate limit exceeded');
            error.code = 'NORDIGEN_ERROR';
            error.category = 'RATE_LIMIT_EXCEEDED';
            throw error;
        }
        return Promise.resolve('success');
    });
    
    const result = await runWithRetries(rateLimitFn);
    expect(result).toBe('success');
    expect(rateLimitFn).toHaveBeenCalledTimes(3);
});
```

### 3. Sync Service Integration Tests (`syncService.test.js`)

Tests end-to-end sync workflow with mocked Actual API:

- **Full workflow**: Init → Download → Sync → Bank Sync → Shutdown
- **Connection failures**: Authentication errors, network issues
- **Budget handling**: Download failures, sync failures
- **Account handling**: Empty lists, individual account failures
- **Cleanup**: Shutdown called even on errors

**Example Test**:
```javascript
test('should complete full sync workflow successfully', async () => {
    const server = {
        name: 'Test Server',
        url: 'https://test.example.com',
        password: 'test-password',
        syncId: 'test-sync-id',
        dataDir: '/tmp/test-data'
    };

    await syncBank(server);

    expect(actual.init).toHaveBeenCalled();
    expect(actual.downloadBudget).toHaveBeenCalled();
    expect(actual.sync).toHaveBeenCalled();
    expect(actual.runBankSync).toHaveBeenCalledTimes(2);
    expect(actual.shutdown).toHaveBeenCalled();
});
```

### 4. Startup Validation Tests (`startupValidation.test.js`)

Tests startup environment validation:

- **Node.js version**: Version detection and validation
- **Configuration directory**: Existence checks
- **Configuration file**: Existence, readability, JSON parsing
- **Dependencies**: node_modules and critical packages
- **Schema file**: Optional schema detection
- **Integration**: Complete valid setup workflow

**Example Test**:
```javascript
test('should detect invalid JSON in config.json', () => {
    const configFile = path.join(configDir, 'config.json');
    fs.writeFileSync(configFile, '{invalid json}');
    
    expect(() => {
        const content = fs.readFileSync(configFile, 'utf8');
        JSON.parse(content);
    }).toThrow();
});
```

---

## 🌐 Browser E2E Tests (#263)

The unit suite above mocks the Actual API and the filesystem; it never renders
the dashboard's HTML/JS or drives a real browser. `tests/e2e/` covers that gap
with Puppeteer against real backing services (a real `HealthCheckService` and
a real SQLite-backed `SyncHistoryService`), so it is a separate suite with its
own Jest config rather than another `describe()` block in the unit tests.

### Running It

```bash
npm run test:e2e
```

This must go through the npm script, not a bare `npx jest --config jest.e2e.config.js`:
Puppeteer ships ESM-only and is loaded with a dynamic `import()`, which Jest's
CommonJS runtime refuses unless Node itself starts with
`--experimental-vm-modules`; the npm script sets that flag.

### How a spec is structured

- **Fixture states** (`tests/e2e/fixtures/states/*.js`) seed a scenario -
  healthy, degraded, a longer history with a known failure count, and so on.
  Each state is just a module with a name string and some seed data, so a new
  scenario is a new small file, not a change to the harness itself.
- **The fixture server** (`tests/e2e/fixtures/index.js`) starts a real
  `HealthCheckService` on an OS-assigned port (`port: 0`) with a real
  `SyncHistoryService` on a temp SQLite file, seeded from the chosen state.
  Only `syncBank`, `getServers`, `getCronSchedules` and the notification
  service are stubbed - everything else is the real dashboard code and real
  routes.
- **The browser fixture** (`tests/e2e/fixtures/browser.js`) launches one
  shared headless Chrome for the whole run and exposes `withFixture` /
  `withDashboard` helpers. Every page it opens has a fixed viewport, UTC
  timezone, reduced motion, and a frozen clock (`@sinonjs/fake-timers`
  installed before any dashboard script runs, sharing the exact same "now"
  the fixture server uses) - so a screenshot or an assertion never depends on
  wall-clock time.
- **Network lockdown**: every request is intercepted. Only same-origin
  requests to the fixture server are allowed through (the dashboard serves
  Chart.js itself from its npm dependency, #244, so nothing is faked);
  everything else, including the browser's own `/favicon.ico` request on every
  navigation, is answered locally or blocked, so a spec can never reach the
  real network or silently consume its own rate-limit budget on a request it
  didn't intend to make.
- **On failure**, the harness saves a screenshot and the page's console log
  under `tests/e2e/artifacts/` (gitignored) instead of failing silently.

### What is covered

Login, the general 60-requests-per-minute rate limiter (the separate login
failure throttle from #246 has no E2E spec), manual sync triggering and error dismissal,
and the Overview, History, Analytics, Settings and Accounts tabs - each
against a fixture state built for that scenario. See the spec files under
`tests/e2e/*.e2e.js` for the exact assertions.

### The screenshot generator uses the same harness

`scripts/generateDashboardScreenshots.js` (`npm run screenshots`) renders the
documentation screenshots in `docs/screenshots/` from `tests/e2e/screenshots.json`
using this same fixture harness - there is no separate mocked code path to keep
in sync, and no running service to start first. See `docs/screenshots/README.md`
for the manifest format.

---

## 🛠️ Test Helpers

Located in `src/__tests__/helpers/testHelpers.js`:

### Available Utilities

```javascript
// Create temporary test directory
const tempDir = createTempDir();

// Clean up after tests
cleanupTempDir(tempDir);

// Create mock configuration object
const config = createMockConfig({ servers: [...] });

// Create mock Actual API instance
const api = createMockActualAPI();

// Create test configuration file
const configPath = createTestConfigFile(dir, config);

// Create test schema file
const schemaPath = createTestSchemaFile(dir);

// Suppress console output during tests
const { restore } = suppressConsole();

// Wait for specified time
await wait(1000); // 1 second
```

### Example Usage

```javascript
describe('My Test Suite', () => {
    let tempDir;

    beforeEach(() => {
        tempDir = createTempDir();
    });

    afterEach(() => {
        cleanupTempDir(tempDir);
    });

    test('my test', () => {
        const config = createMockConfig();
        const configPath = createTestConfigFile(tempDir, config);
        // ... test code
    });
});
```

---

## 📈 Coverage Reports

### Viewing Coverage

After running `npm run test:coverage`, open the HTML report:

```bash
# Generate coverage report
npm run test:coverage

# Open in browser (macOS)
open coverage/lcov-report/index.html

# Open in browser (Linux)
xdg-open coverage/lcov-report/index.html

# Open in browser (Windows)
start coverage/lcov-report/index.html
```

### Coverage Thresholds

Configured in `package.json`:

```json
"coverageThreshold": {
  "global": {
    "branches": 70,
    "functions": 70,
    "lines": 70,
    "statements": 70
  }
}
```

**Current coverage**: run `npm run test:coverage` for the authoritative per-file
report, or see the live **Coverage** badge on the [README](../README.md). The
enforced minimums are the thresholds shown above (70% lines/functions/statements,
61% branches).

### Excluded from Coverage

The following files are excluded as they are integration/orchestration code:
- `index.js` - Entry point
- `src/syncService.js` - Main service orchestration
- `src/__tests__/**` - Test files

---

## ✍️ Writing Tests

### Test File Naming

- Unit tests: `<module>.test.js`
- Integration tests: `<feature>.test.js`
- Place in `src/__tests__/` directory

### Test Structure

```javascript
const { testHelper } = require('./helpers/testHelpers');

describe('Feature Name', () => {
    beforeEach(() => {
        // Setup before each test
    });

    afterEach(() => {
        // Cleanup after each test
    });

    describe('Sub-feature', () => {
        test('should do something specific', () => {
            // Arrange
            const input = setupInput();
            
            // Act
            const result = performAction(input);
            
            // Assert
            expect(result).toBe(expected);
        });
    });
});
```

### Best Practices

1. **Descriptive names**: Use clear, descriptive test names
   ```javascript
   // Good
   test('should throw error for duplicate server names', () => {});
   
   // Bad
   test('duplicates', () => {});
   ```

2. **Arrange-Act-Assert**: Structure tests clearly
   ```javascript
   test('example', () => {
       // Arrange - Set up test data
       const input = { value: 42 };
       
       // Act - Execute the code
       const result = processInput(input);
       
       // Assert - Verify the outcome
       expect(result).toBe(84);
   });
   ```

3. **Isolation**: Each test should be independent
   ```javascript
   // Use beforeEach/afterEach for setup/cleanup
   beforeEach(() => {
       tempDir = createTempDir();
   });
   
   afterEach(() => {
       cleanupTempDir(tempDir);
   });
   ```

4. **Mock external dependencies**: Use Jest mocks
   ```javascript
   jest.mock('@actual-app/api', () => ({
       init: jest.fn().mockResolvedValue(undefined),
       sync: jest.fn().mockResolvedValue(undefined)
   }));
   ```

5. **Test edge cases**: Cover error paths
   ```javascript
   test('should handle empty input', () => {});
   test('should handle null input', () => {});
   test('should throw on invalid input', () => {});
   ```

---

## 🐛 Debugging Tests

### Run Single Test File

```bash
npm test -- configLoader.test.js
```

### Run Tests Matching Pattern

```bash
npm test -- --testNamePattern="retry"
```

### Run with Verbose Output

```bash
npm test -- --verbose
```

### Debug with VS Code

Add to `.vscode/launch.json`:

```json
{
  "type": "node",
  "request": "launch",
  "name": "Jest Debug",
  "program": "${workspaceFolder}/node_modules/.bin/jest",
  "args": ["--runInBand", "--no-cache"],
  "console": "integratedTerminal",
  "internalConsoleOptions": "neverOpen"
}
```

---

## 🔄 Continuous Integration

### GitHub Actions Example

```yaml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm ci
      - run: npm test
      - run: npm run test:coverage
      - name: Upload coverage
        uses: codecov/codecov-action@v3
```

---

## 📚 Additional Resources

- [Jest Documentation](https://jestjs.io/docs/getting-started)
- [Jest Matchers](https://jestjs.io/docs/expect)
- [Jest Mock Functions](https://jestjs.io/docs/mock-functions)
- [Testing Best Practices](https://github.com/goldbergyoni/javascript-testing-best-practices)

---

## 🤝 Contributing

When adding new features:

1. Write tests first (TDD) or alongside implementation
2. Ensure tests pass: `npm test`
3. If the change touches the dashboard's rendered UI, also run the browser
   E2E suite: `npm run test:e2e`
4. Check coverage: `npm run test:coverage`
5. Maintain >70% coverage threshold
6. Update this documentation if adding new test patterns

---

**Last Updated**: December 5, 2025