# ARCHITECTURE.md How open-forge actually works as a system — the actors, the data flow, where state lives, what keeps quality up. Different audience from [CLAUDE.md](CLAUDE.md) (which is the *policy* — strict-doc rules, sanitization principles, processing workflow): ARCHITECTURE is *system shape*. > **Pointers**: For project intent (why / who / success / non-goals) see [BRD.md](BRD.md). For policy details (what's in scope, how to verify against upstream, sanitization rules) see [CLAUDE.md](CLAUDE.md). For end-user usage see [README.md](README.md). For the skill content the end-user's agent reads see [SKILL.md](plugins/open-forge/skills/open-forge/SKILL.md). ## System overview ``` (input pipe) end users ─┐ ┌─► marketplace (skill- │ ┌────────────────┐ │ (Claude Code assisted ├──► │ GitHub issues │ ←──── batch ─────── │ + 6 other feedback) │ │ ┌────────────┐ │ newsletter │ platforms) │ │ │ recipe-fb │ │ crawls │ maintainer ├──► │ │ nominate │ │ │ (manual) │ │ │ method │ │ │ │ │ └────────────┘ │ │ │ └────────┬───────┘ │ │ │ │ │ ▼ │ │ ┌────────────────┐ ┌────────────┐ │ │ │ AI session │───►│ PR on │────►│ merge → main │ │ (the bot │ │ main │ │ │ │ on schedule │ │ │ │ │ │ OR ad-hoc) │ └─────┬──────┘ │ │ └────────────────┘ │ │ │ ▲ ▼ │ │ │ ┌─────────┐ │ │ │ │ CI │ ──────┘ │ │ │ guards │ │ │ └─────────┘ │ │ │ (strict-doc-policy verification: │ re-fetch upstream → diff → patch) │ │ public │ │ newsletters ┴──────────────┘ (selfh.st, other RSS) ``` The catalog grows continuously — at the time of this writing, the catalog has 2,200+ verified recipes; main is updated via a stream of small bot-authored PRs (recently `batch 183 — owntracks-frontend, paaster, …`) plus occasional maintainer / AI-session PRs. The exact count is a [shields.io live badge](https://img.shields.io/github/directory-file-count/zhangqi444/open-forge/plugins/open-forge/skills/open-forge/references/projects?type=file&extension=md) in the README. ## The four actors | Actor | What they do | Files they touch | |---|---|---| | **The bot** (scheduled AI session) | Polls public newsletters (selfh.st, others) + GitHub issues. Triages by template type. Verifies against upstream per strict-doc-policy. Authors patches in batches. Bumps `plugin.json` version. Updates `progress/selfhst-progress.json` + `progress/issues-log.json`. | `references/projects/*.md`, `progress/*`, `plugin.json`, occasional `dist/` regen | | **Maintainer (zhangqi444)** | Strategic decisions, PR merges, repo-settings (About / topics / website / social preview), high-judgment edits | Repo settings (web UI), occasional manual PRs | | **AI sessions like Claude Code / Codex / Cursor / Aider / Continue / OpenClaw / Hermes / generic** | Maintainer-prompted: file issues, author specific PRs, audit catalog quality, generate distribution bundles for non-Claude-Code platforms | Anything except repo settings | | **End users** | File issues via the three templates: recipe-feedback / software-nomination / method-proposal. The skill drafts these automatically with sanitization at the end of each deploy. | GitHub issue templates only — no direct repo write access | End users do **not** open PRs. Per CLAUDE.md § *Issue-driven contribution model*, issues are the universal contribution channel; AI sessions process them centrally so the strict-doc-policy stays enforced and credentials don't leak into commit history. ## Data flow — how an issue becomes a recipe edit ``` 1. END USER files issue ──► uses one of three templates: (skill-drafted or manual) recipe-feedback / software-nomination / method-proposal OR — bot picks up signal directly from public newsletters (skips the issue step) 2. BOT or AI SESSION triages ──► labels: triaged / in-progress / applied / needs-info / out-of-scope (per CLAUDE.md § Processing incoming issues, state- machine via labels) 3. UPSTREAM VERIFICATION ──► re-fetch the recipe's cited upstream URLs (or for nominations, locate upstream's install-method index); apply strict-doc- policy; if fetch fails, stop and label 'needs-info' 4. PATCH AUTHORSHIP ──► apply per Recipe structure (must-have sections); cite upstream URL at top of every section; flag community-maintained methods with the ⚠️ blockquote; sanitize user-shared content 5. dist/ REGENERATION ──► if patch touches CLAUDE.md / SKILL.md / references/modules/, run ./scripts/build-dist.sh all (CI's dist-bundles-up-to-date check enforces this) 6. PR open on main ──► branch: bot/issue--; PR body cites originating issue + every upstream URL re-verified + version-bump rationale 7. CI guards ──► .github/workflows/dist-bundles.yml fails the build if dist/ drifted from canonical sources 8. Merge to main ──► marketplace publishes on next /plugin marketplace update (controlled by plugin.json version) 9. Issue label flips to ──► bot triage closes the loop; 'applied' label-state-machine documents what's done ``` ## State stores | Where | What | Owned by | |---|---|---| | **GitHub issues** | The input queue. Three templates encode the structured signals. State tracked via labels. | End users (drafts) + bot/maintainer (triage) | | **`progress/selfhst-progress.json`** | Per-app processing status — which selfh.st apps have been cataloged, which are pending or skipped. | Bot | | **`progress/selfhst-software.json`** | Cached source data fetched from selfh.st — app list with star counts. Rebuilt as needed. | Bot | | **`progress/issues-log.json`** | Per-issue triage history — issue number, status (addressed/skipped/waiting), commit ref, action taken. | Bot | | **`progress/sources.md`** | Catalog-growth source queue — which external lists / feeds the bot pulls from and in what order (selfh.st in progress, awesome-selfhosted-data queued, Self-Host Weekly newsletter continuous, GitHub issues continuous). Maintainer-curated; the bot reads it on each run to decide which source to pull from next. | Maintainer (curates queue) + bot (reads on each run) | | **`plugins/open-forge/skills/open-forge/references/projects/*.md`** | The catalog itself. ~2,200+ Tier 1 verified recipes. | Bot (mostly) + maintainer (strategic recipes like ghost.md) | | **`plugins/open-forge/skills/open-forge/references/{infra,runtimes,modules}/`** | Reusable orchestration layers — infra adapters, runtime modules, cross-cutting modules (preflight, dns, tls, smtp, inbound, tunnels, credentials, feedback, backups) | Maintainer mostly; bot adds modules when first-deploy-discipline surfaces gaps | | **`plugins/open-forge/skills/open-forge/references/bundles/*.md`** | Curated multi-software deployment bundles (AI homelab, privacy stack). Recipe-of-recipes that orchestrate existing Tier 1 recipes for goal-shaped requests. | Maintainer-direct authoring (per CLAUDE.md graduation criteria, bundles aren't speculative authoring — they pair existing recipes) | | **`dist/`** | Auto-generated multi-platform distribution bundles (codex / cursor / aider / continue / openclaw / hermes / generic). Concatenated from canonical sources via `scripts/build-dist.sh`. | Build script; CI enforces freshness | | **`docs/platforms/`** | Per-platform integration guides for the 7 supported platforms. | Maintainer; updated when a new platform is added | | **`.github/ISSUE_TEMPLATE/`** | The three input-channel forms (recipe-feedback / software-nomination / method-proposal) plus `config.yml` (disables blank issues) | Maintainer | | **`.github/workflows/dist-bundles.yml`** | CI gate: regenerates `dist/` and fails the build if it drifted from canonical sources | Maintainer | | **`~/.open-forge/deployments/.yaml`** (on user machines) | End-user deployment state. Phased-workflow checkpoints, inputs, outputs. Resume across sessions. | End user; never enters the repo | ## Quality gates What keeps the catalog from rotting: | Gate | Where it runs | What it catches | |---|---|---| | **Strict doc-verification policy** | At patch-authorship time (per CLAUDE.md) | Recipe content cited from training data / blog posts / search snippets — not upstream docs. Past failures (v0.7.0 Helm hallucination, v0.6.0 OpenClaw "every blessed path" 4-of-17 claim) traced back here. | | **Sanitization principles** | At issue-submission time (skill-side) AND at patch-acceptance time (issue-processing AI session re-checks) | User identifiers (domains, IPs, SSH keys, API keys, AWS account IDs, emails) leaking into the public repo. | | **CI: dist-bundles-up-to-date** | Every PR + push to main | Stale `dist/` bundles after CLAUDE.md / SKILL.md / module changes. Forces build-script regen. | | **Community-maintained flagging** | At patch-authorship time | Third-party install methods presented as upstream-blessed. ⚠️ blockquote required for community-maintained methods. | | **First-run discipline** | At deploy time (per CLAUDE.md) | Recipes that "look right" but fail in the wild. End-of-deploy feedback flow drafts a sanitized issue capturing the gotcha. | | **Versioning rules** | At commit time (in PR review) | User-visible behavior changes shipped without a version bump (breaks marketplace cache invalidation). | ## Cadence — what actually happens in main Observed from the commit log (2026-04 to 2026-05): - **Bot batches** — the bot ships small batches (3-5 recipes per commit), tagged `batch ` in the commit message. Catalog grew from ~180 to 2,200+ recipes over the observation window. - **Source queue** — bot pulls from external sources in priority order (see [`progress/sources.md`](../progress/sources.md)): currently working through selfh.st (1,274 apps, ~92% done); next source queued is awesome-selfhosted-data; Self-Host Weekly newsletter + GitHub issues processed continuously alongside. - **AI-session PRs** (this Claude Code session) — periodic batches authored on demand: catalog audits, architectural additions (issue-driven contribution model, multi-platform support, agent-platform support, backups module, monitoring module, bundles), bug fixes (#40 broken in-bundle links). - **End-user issues** — sparse but valuable; recent issues #24-27 (Windows setup gotchas), #40 (broken links), #41 (BookStack URL). - **Version bumps** — driven by user-visible changes. Recent: `0.17.1 → 0.17.2 → 0.19.x → 0.20.x → 0.21.0 (multi-platform) → 0.22.0 (agent platforms) → 0.23.0 (backups + bundles) → 0.24.0 (monitoring + gstack pointer)`. ## Three layers, one orchestration layer above The deployment model is a tuple of three independent axes (per CLAUDE.md § Architecture): 1. **What** to host — software → `references/projects/.md` 2. **Where** to host — infra → `references/infra//.md` 3. **How** to host — runtime → `references/runtimes/.md` Cross-cutting concerns (preflight / dns / tls / smtp / inbound / tunnels / credentials / feedback / backups) live as reusable modules under `references/modules/`. **A fourth orchestration layer** sits above these: **bundles** (`references/bundles/`). A bundle is a *recipe-of-recipes* — it pairs commonly-co-deployed software for goal-shaped user requests (*"set up an AI homelab"*) and ships the cross-software wiring (env vars / DNS / ports between the constituent recipes). Bundles don't replace single-recipe routing; they're an additional entry point for goal-shaped intents. ## Two-tier coverage Per CLAUDE.md § *Two-tier coverage model*: - **Tier 1** — verified recipes in `references/projects/`. Authored ahead of time, audited against upstream docs, kept current via first-run discipline + version bumps. Quality bar: every install method cites upstream URL; community methods flagged with blockquote. - **Tier 2** — derived live from upstream docs at request time. When the user names software not in the catalog, the skill announces the fallback, fetches upstream live, applies the strict-doc-policy on the fly, and reuses the runtime/infra/module layers. Best-effort, not authoritative. Tier 2 → Tier 1 graduation is **demand-driven**: 3+ feedback issues for the same software, repeat user, captured non-obvious gotcha, or maintainer-driven deploy. ## Multi-platform shape The skill is content-agnostic markdown that runs on 7+ AI platforms via per-platform integration: | Platform | Integration | |---|---| | Claude Code | Plugin marketplace (`/plugin marketplace add`); auto-loads via SKILL.md description match | | Codex | System-prompt embedding (ChatGPT custom instructions) or workspace files (Codex CLI) | | Cursor | `.cursor/rules/` bundle from `dist/cursor/` | | Aider | `--read` files + `CONVENTIONS.md` from `dist/aider/` | | Continue.dev | Context provider + slash command from `dist/continue/config.snippet.yaml` | | OpenClaw | Workspace skill at `~/.openclaw/workspace/skills/open-forge/SKILL.md` | | Hermes-Agent | User skill at `~/.hermes/skills/open-forge/SKILL.md` | | Generic | Single-file bundle from `dist/generic/open-forge-bundle.md` | The build script (`scripts/build-dist.sh`) generates each platform's bundle by concatenating canonical sources (CLAUDE.md + SKILL.md + the `credentials` and `feedback` modules) into the format that platform expects, plus a sed post-process step that rewrites in-bundle hyperlinks to point at the inlined sections (the original cross-references are valid for the Claude Code plugin install where files coexist, but break in single-file bundles — see issue #40). ## Where each subsystem's docs live | Subsystem | Doc | Purpose | |---|---|---| | Project policy | [CLAUDE.md](CLAUDE.md) | What's in scope; strict-doc rules; sanitization; issue-processing workflow | | AI-agent landing page | [AGENTS.md](AGENTS.md) | agents.md-standard pointer file for non-Claude-Code agents working on the repo | | User-facing project doc | [README.md](README.md) | Value prop, install, coverage | | End-user skill content | [SKILL.md](plugins/open-forge/skills/open-forge/SKILL.md) | Phased workflow, credential handling, post-deploy feedback | | Per-platform usage | [docs/platforms/*.md](docs/platforms/) | How to use open-forge on Codex / Cursor / Aider / Continue / OpenClaw / Hermes / generic agents | | Build tooling | [scripts/build-dist.sh](scripts/build-dist.sh) | Regenerates `dist/` bundles; documents itself with usage comment | | **System architecture** | **this file** | Actors, flow, state stores, quality gates — how it all fits together | ## What this doc deliberately doesn't cover - **Per-recipe content** — that's the catalog itself; browse `references/projects/` or [DeepWiki](https://deepwiki.com/zhangqi444/open-forge). - **Policy detail** — strict-doc-policy regex patterns, sanitization strip-list, recipe-structure must-haves are in [CLAUDE.md](CLAUDE.md). - **Per-platform integration nuance** — those live in [docs/platforms/](docs/platforms/) for Codex / Cursor / Aider / Continue / OpenClaw / Hermes / generic. - **End-user deployment instructions** — README.md and SKILL.md cover that; this doc is for contributors, maintainers, and AI sessions trying to understand the maintenance system.