# Sync — keeping your memory in sync across machines m3-memory's sync system keeps your memory database in sync between your local machine and a central PostgreSQL warehouse. This lets you switch machines (desktop ↔ laptop, work ↔ home) without losing context. This page covers the **default sync** — a SQLite local store (`agent_memory.db`) to a PostgreSQL warehouse. Bench result databases are **not** synced and are not auto-detected; if you run benchmarks (most users don't) and want their DBs included, add them to `M3_SYNC_DBS` and supply your own warehouse schema — see [What does NOT sync by default](#what-does-not-sync-by-default) below. > **If your local store is PostgreSQL**, this page's row-by-row bridge does not > apply to you — it opens the local side as SQLite. Use > [SYNC_PG_TO_PG.md](SYNC_PG_TO_PG.md) instead, which is the only supported path > for a PG primary. m3 refuses rather than falling back, because falling back > would sync nothing while reporting success. ## What gets synced - **`memory/agent_memory.db`** — your production memory: notes, decisions, facts, conversations, embeddings, relationships. Bidirectional row-level delta sync to PostgreSQL via `bin/pg_sync.py`. - **`memory/agent_chatlog.db`** — raw chat archive (Claude Code, Codex logs) awaiting promotion. Synced if the file exists; skipped silently if not. That's it. The repo does not ship sync support for bench result DBs or other custom databases. If you self-host a more complex layout (e.g., separate DBs for benchmarking), use `M3_SYNC_DBS` to add them — see the *Advanced* section at the bottom — but the repo will not auto-detect them and there is no shipped warehouse migration for non-default DBs. --- ## Setup You need: 1. A reachable PostgreSQL server (your warehouse). 2. The connection string stored in your OS keyring or env var. Set two env vars (typical values shown): ```bash export POSTGRES_SERVER=192.0.2.10 # your PostgreSQL host IP; or SYNC_TARGET_IP — same thing export M3_CDW_PG_URL='postgresql://user:pass@host:5432/agent_memory' # warehouse URL (PG_URL still works but is deprecated) ``` Or store `M3_CDW_PG_URL` in your OS keyring (macOS Keychain, Windows Credential Manager, Linux Secret Service) — the codebase uses `auth_utils.get_api_key` to look it up safely. Apply the warehouse schema (one-time per warehouse). Postgres-side migrations live in `memory/migrations/postgres/`: ```bash psql -h $POSTGRES_SERVER -U $PGUSER -d agent_memory \ -f memory/migrations/postgres/pg_warehouse_chatlog_v1.sql ``` Note: `memory/migrations/*.sql` (without the `postgres/` subdir) are SQLite migrations applied automatically on first connect. Don't put Postgres SQL there — `migrate_memory` will warn about malformed files. --- ## Running sync Manually: ```bash python bin/sync_all.py ``` What it does: 1. TCP-probes `$POSTGRES_SERVER` (3-second timeout). 2. If reachable, runs `bin/pg_sync.py` for `agent_memory.db`. 3. Logs to `logs/sync_all.log`. Dry-run (just check connectivity, don't write): ```bash python bin/sync_all.py --dry-run ``` --- ## Scheduling For unattended hourly sync: - **Linux/macOS**: cron — see `bin/pg_sync.sh` for a wrapper that handles env. - **Windows**: Scheduled Task. Action: `python.exe bin/sync_all.py`. Working dir: the repo root. Run as your user, schedule hourly. The scheduler tolerates outages — if the warehouse is unreachable, sync logs a warning and exits cleanly. Next run picks up where it left off. --- ## How conflict resolution works `pg_sync.py` uses **last-write-wins** based on `updated_at`. When the same row exists in both SQLite and Postgres with different `updated_at` values, the newer one wins. This means: - Edit a note on machine A, sync → warehouse has A's version. - Edit the same note on machine B before A's sync reaches B, then sync → whichever has the later `updated_at` wins. - Soft-deletes (rows with `is_deleted=1`) propagate cleanly. The deleted state replicates; the row stays in both DBs marked deleted. > **PostgreSQL primary?** If your primary store is PostgreSQL (not the default > SQLite) *and* you sync to a PostgreSQL warehouse, m3 can use a faster native > PostgreSQL-to-PostgreSQL path (`postgres_fdw`, set-based upserts) instead of > this row-by-row bridge. Same conflict rules; extra one-time setup. See > [SYNC_PG_TO_PG.md](SYNC_PG_TO_PG.md). --- ## What does NOT sync by default - **Bench result DBs** — out of scope for the repo. If you run benchmarks and want their results synced across machines, that's self-host territory: add the DBs to `M3_SYNC_DBS` and provide your own warehouse schema migration. - **`memory/local_*` rows** — anything tagged `scope='local'` is per-machine by design. - **`/tmp` scratch and `.scratch/`** — these are workspace, not memory. --- ## Troubleshooting **"PostgreSQL data warehouse unreachable"** → TCP probe failed. Check: - Is `POSTGRES_SERVER` set? - Can you `nc -zv $POSTGRES_SERVER 5432` from this host? - Is your warehouse running? **"Another sync is already in progress"** → A previous sync hung. Look in `logs/sync_all.log` for orphaned PIDs. The lock file is at `memory/.pg_sync.lock`; remove it manually if stale. **"Schema mismatch / missing column"** → You haven't applied the latest warehouse migration. See setup. **Hourly task stops running on Windows** → Task may auto-disable after repeated failures. Check `schtasks /Query /FO LIST /V | grep -i m3-memory` for `Status: Disabled`. Re-enable with `schtasks /Change /TN "" /ENABLE`. --- ## Multi-machine quick reference Setting up a second machine to sync against the same warehouse: 1. Clone the repo on machine B. 2. Set `POSTGRES_SERVER` and `M3_CDW_PG_URL` env vars (same warehouse as A; the `PG_URL` name still works but is deprecated). 3. First sync pulls everything from the warehouse — let it finish. 4. From then on, edits on either machine appear on the other after sync. Three-way sync (A ↔ warehouse ↔ B) works the same — the warehouse is the hub; peers don't talk to each other directly. --- ## Agent machine with no local m3 (MCP-only) — and why the chat log goes missing A common setup: **machine A** runs m3, **machine B** runs the coding agent (Claude Code, OpenCode) and reaches A's MCP server over the network. The memory tools work, but nothing is captured to the chat log. That is expected. **Chat capture is not part of the MCP server** — it is a hook that the agent fires in its own process: ``` agent on machine B ├─▶ hooks/chatlog/opencode_session_end.py │ └─▶ chatlog_ingest.py ← runs on machine B └─▶ MCP ─────────────────────────────▶ memory_search / memory_write on machine A ``` MCP carries the *tools* across the wire; it does not carry turn capture. m3 ships hooks for Claude Code, OpenCode, Gemini CLI and Aider, and each one runs locally. **So m3 has to be installed on machine B too.** Install it thin, then sync via the warehouse as described above. ### Don't load a second embedder on the agent machine The usual reason a small agent box falls over after installing m3 is the local GGUF embedder: `m3-embed-server` loads `bge-m3-Q4_K_M.gguf` into RAM at startup. On a 4-core / 16 GB machine, alongside the agent, that is enough to exhaust memory. Point machine B at machine A's embed server, and keep B from starting one of its own: ```bash # machine B M3_EMBED_URL=http://machine-a:8082 # where to send embed requests M3_EMBED_INPROC=0 # do not load a local GGUF in-process ``` `M3_EMBED_URL` alone is not sufficient. Tier-1 **auto-detects** a bge-m3 GGUF in the canonical model directories when `M3_EMBED_GGUF` is unset (`M3_EMBED_GGUF_AUTODETECT` defaults to `1`), so a machine that happens to have one — via LM Studio, say — will still load it in-process. In-process embedding is gated by `M3_EMBED_INPROC` / `.embed_config.json`: the design is *safe-by-default — route to the shared server unless inproc is clearly intended*, so being explicit here keeps B on that path. Then don't register the heavyweight services on B — skip `AgentOS_EmbedServer` (it is the sole embedder for the fleet, and belongs on A), and skip the cognitive loop if B doesn't need it. B needs the chatlog hooks, the MCP client, and its local SQLite store; the warehouse sync handles the rest. > ⚠️ The embed server binds `127.0.0.1` by default and has **no authentication**. > To serve another machine, A must bind a reachable interface — keep that on a > trusted LAN or a tunnel (Tailscale, WireGuard), never the open internet. ### Never share a SQLite file over a network mount Do not put `agent_memory.db` on an SMB/NFS share and point both machines at it. SQLite's locking relies on filesystem semantics network mounts don't reliably provide. It appears to work, which is what makes it dangerous — the failure surfaces later as a corrupt store or silently lost turns. Each machine keeps its own local SQLite; the warehouse is the meeting point. ### Pin both roots, or the chat log splits The **MCP server** reads its roots from its own registration's `env` block; the **chatlog hook** inherits the agent's *process* environment. Pin only one and the server reads the new root while hooks keep writing to the old one. On machine B, set `M3_ENGINE_ROOT` and `M3_CONFIG_ROOT` in **both** the `claude mcp add --env …` registration **and** inline on each hook `command`. ### Verifying capture on the agent machine ```bash m3 chatlog doctor # exits nonzero on capture warnings m3 chatlog status --json # last_write_at must advance after a session ``` If `last_write_at` doesn't move after a session on B, the hooks aren't firing — chase that independently of whether the MCP tools work. --- ## Advanced: M3_SYNC_DBS If you want to override the default DB list (e.g., to sync a custom named DB, or to skip auto-detection of bench DBs even when present): ```bash # Sync only agent_memory.db (no auto-detect) M3_SYNC_DBS=memory/agent_memory.db python bin/sync_all.py # Sync a custom set M3_SYNC_DBS=memory/agent_memory.db:custom/extra.db python bin/sync_all.py ``` Paths can be colon- or comma-separated, absolute or relative to the repo root.