# no81_sync

Syncs No.81 forum content into local Markdown knowledge base files.

## What it exports

- Roles: all topic links found in `topic=1378.0`
- Rulebooks: all topics under `board=15.0`
- Completed records: all topics under `board=14.0;prefix=3`

## Output folders

- `kb/roles/`
- `kb/rulebooks/`
- `kb/records-completed/`
- `kb/index/`

## Setup

1. Create a local config file:

```bash
cp tools/no81_sync/config.example.env tools/no81_sync/.env
```

2. Fill in your SMF account in `tools/no81_sync/.env`.

   If your site blocks scripted `login2` posts (e.g. returns `Session 验证失败` on 403),
   you can use cookie auth instead:

   - log in once in browser
   - copy the Cookie header value from that logged-in request (recommended: include both auth cookie and PHPSESSID)
   - set `SMF_COOKIE=...` in `.env`
   - if you only copied the auth cookie value itself, also set `SMF_COOKIE_NAME=...` (e.g. `SMFCookie630`)

   When `SMF_COOKIE` is set, the script will use that session directly and skip username/password login.

3. Install dependencies:

```bash
python -m pip install -r tools/no81_sync/requirements.txt
```

## Run

Export all categories:

```bash
python tools/no81_sync/sync.py
```

Local maintenance/backfill (no re-fetch; updates existing markdown metadata and rebuilds indexes):

```bash
python tools/no81_sync/sync.py --mode maintenance-local
```

Export one category only:

```bash
python tools/no81_sync/sync.py --category roles
python tools/no81_sync/sync.py --category rulebooks
python tools/no81_sync/sync.py --category records
```

Dry run (no file writes):

```bash
python tools/no81_sync/sync.py --dry-run
```

## New index outputs

- `kb/index/topics.csv`
- `kb/index/topics.jsonl`
- `kb/index/authors.jsonl`
- `kb/index/authors_roles_sample.jsonl`
- `kb/index/warnings.txt`
- `kb/index/warnings.jsonl`

## Reporting posts (topic reply)

The sync can post start/finish status replies to a forum topic (default `topic=35`) using the same SMF credentials.

Config keys:

- `ENABLE_REPORT_POST=true|false`
- `REPORT_ON_START=true|false`
- `REPORT_ON_FINISH=true|false`
- `REPORT_TOPIC_ID=35`

Finish report includes counters: added / updated / unchanged / warnings / failed.

## Differential behavior

By default `SKIP_EXISTING_TOPICS=true` to avoid re-fetching already-synced topics in large libraries.

Set `SKIP_EXISTING_TOPICS=false` if you want to force re-check existing topics.

If you already completed a full sync before new metadata/index features were added, run:

```bash
python tools/no81_sync/sync.py --mode maintenance-local
```

This backfills existing files and rebuilds `topics/authors/warnings` indexes in batch.

## Notes

- The script uses SMF login via `action=login` -> `action=login2`.
- Some deployments may reject non-browser `login2` POST with session verification errors; use `SMF_COOKIE` fallback in that case.
- Export is text-first. Images/media are not downloaded.
- A local state file is used for incremental writes:
  - `tools/no81_sync/state/state.json`