[](https://app.railyard.run/deploy?repo=https://github.com/netgoat-xyz/netgoat) # NetGoat agent NetGoat is a self-hosted reverse proxy and traffic-policy agent written in Go. It can run from a local YAML configuration, consume snapshots from the companion control plane, and continue serving the last known-good configuration during an outage. > [!WARNING] > NetGoat is active alpha software (`v0.1.0-alpha.1`; git tags going > forward are `v*`, see [docs/release.md](docs/release.md)). The shipped > configuration contains no routes. Review every upstream, use strong > bootstrap credentials, and place administrative services behind > authenticated TLS before exposing a deployment to the internet. ## Feature status This table is the public roadmap surface. **Available** is what this agent ships today. **Planned** is designed work with open specs — not live behavior, not a package, and not an identity product. The shipped `config.yml` has `routes: {}`. A fresh default deployment returns `404` for every Host instead of proxying to a local service. ### Available | Capability | Status | Notes | | --- | --- | --- | | Domain and path routing | Available | Exact, wildcard, regex, and longest-prefix path routes; local routes can be overridden by streamed routes. | | Load balancing and failover | Available | Round-robin pools, bounded concurrent health checks, and safe-method retry/failover. | | WAF rules | Available | Precompiled expression rules with priorities, `BLOCK`/`ALLOW` actions, and request host/method/path/query/header/`stack_class` context. | | Traffic controls | Available | Global rate limiting, request queueing, bandwidth throttling, honeypot handling, and session-bound PoW with pinned Web Bot Auth skip. Challenge is JSON (not text/click/slider). Difficulty from load + `stack_class` mismatch (wired from terminate-only JA4/H2/ALPN), not User-Agent. Verify is bound to the terminated TLS session, not IP. | | Shared response cache | Available | Bounded LRU/TTL cache for explicitly public responses, with HTTP freshness and revalidation safeguards. | | Local authentication | Available | Cookie or Basic authentication, per-user zero-trust challenge flags, and explicit secure bootstrap users. | | TLS termination | Available | Static fallback, streamed per-domain/wildcard certificates, and atomic SNI selection. Termination is required for fingerprint v1 and for session-bound PoW. | | WebSocket proxying | Available | Upgrade connections are preserved by Go's reverse proxy. | | Metrics | Available | JSON and Prometheus endpoints for traffic, cache, block, latency, and proxy-error counters. | | AI request classifiers | Optional | Local GoatAI, Koda-WAF, and Koda-2 workers; model files and Python dependencies are required only when enabled. | | Control-plane recovery | Available | Polling with timeouts/backoff, atomic snapshot reconciliation, deduplication, and private on-disk recovery snapshots. | | Operational telemetry | Optional | Explicitly opt-in delivery to the companion telemetry server, with endpoint and ingestion-key configuration. | | Automatic certificate issuance/renewal | Available (opt-in) | Explicit ACME allow-list, HTTP-01 handler, encrypted persistent cache, and last-known-good certificate retention. | | JavaScript/TypeScript dynamic rules | Available (opt-in) | Isolated, bounded JS/TS request decisions with atomic last-known-good reload and fail-closed evaluation. | | Stack fingerprint v1 | Available | JA4 + Akamai-style H2 `SETTINGS` / `WINDOW_UPDATE` / `PRIORITY` / pseudo-header order + ALPN order, **only when this agent terminates TLS**. Opaque `stack_class` for bot clustering — not a user, hardware ID, or canvas hash. No JA4H. HTTP/3 / QUIC is a documented hole. Emit nothing for plaintext HTTP, TLS pass-through, or Cloudflare in front (`CF-Connecting-IP`). Spec: [#113](https://github.com/netgoat-xyz/netgoat/issues/113). | | Developer plugin catalog and middleware SDK | Available | Restart-only selections for exact compiled descriptors; v1 capability grants, lifecycle isolation, and no remote code/artifact loading. | | Cloudflare Access, DNS, and tunnel management | Available (opt-in) | Fail-closed Access JWT/JWKS verification plus bounded, dry-run-by-default startup reconciliation using an environment-only token. Cloudflare in front of this agent is **not** a JA4 source. | | Per-route cache/bandwidth policies | Available | Route policies inherit global defaults, isolate cache/bandwidth state, and support explicit per-route overrides. | Recent hardening (already merged): public plaintext HTTP is refused at startup unless `allow_insecure_public_http: true` is set ([#111](https://github.com/netgoat-xyz/netgoat/pull/111)). CI runs `go vet` and `go test -race` on pull requests and `main` ([#110](https://github.com/netgoat-xyz/netgoat/pull/110), [#112](https://github.com/netgoat-xyz/netgoat/pull/112)). The dashboard shown by the wider NetGoat project belongs to the control plane. This agent exposes metrics APIs but does not embed that dashboard. ### Next / roadmap Designed, not shipped. Specs live in the linked issues. Do not treat these rows as implementation claims. | Capability | Status | Notes | | --- | --- | --- | | VSA (Virtual System Administrator) | Planned | Out-of-band autonomous defense operator (not a hot-path classifier, not Kaseya VSA). Open: [#98](https://github.com/netgoat-xyz/netgoat/issues/98). | **Honesty** - Fingerprint is a client TLS/HTTP **stack class**. Chrome on a million laptops will collide. Do not persist it as identity. - JA4 is live **only** when this agent terminates TLS. If Cloudflare or any other terminator sits in front, this agent must not fingerprint that ClientHello and call it the browser. - Turnstile / reCAPTCHA are **not** NetGoat features. - Session-bound PoW and pinned Web Bot Auth skip **are live** when this agent terminates TLS. The pin list may be empty (skip lane empty until operators seed https `pinned_directories`). Unsigned agents take the PoW lane. **Gaps** - No terminate (plaintext, pass-through, or Cloudflare in front) → no fingerprint and no PoW. Same rule as [#113](https://github.com/netgoat-xyz/netgoat/issues/113). - Web Bot Auth registry is empty until operators seed a pinned https allowlist. - HTTP/3 / QUIC clients will not appear in fingerprint v1. - Unsigned LLM browsers / `python` / `go-http-client` / curl take the **PoW** lane — not a magic skip. **Hardening next** (docs only; not this change) - Keep CI race + vet on PRs and `main` as the verification baseline. - Leave alpha only when the written gate in [docs/release.md](docs/release.md) is checked off. Beta does not require Cloudflare parity, VSA, H3/QUIC fingerprint, default-on AI classifiers, or a control-plane MVP. ## Quick start Requirements: - Go 1.25 or newer - one or more reachable HTTP upstreams Clone the repository, add the routes you intend to expose to `config.yml`, then run: ```sh go test ./... go run . ``` The sample listener is `127.0.0.1:8080` (plaintext HTTP on loopback). A minimal local route looks like this: ```yaml listen: "127.0.0.1:8080" auth: enabled: false routes: app.localhost: type: domain targets: - url: http://127.0.0.1:3000 health_check: http ``` Then send a request with the configured host: ```sh curl -H 'Host: app.localhost' http://127.0.0.1:8080/ ``` A missing `config.yml` is fatal. Plaintext HTTP on a public address (`:8080`, `0.0.0.0`, `::`, or any non-loopback bind) is refused at startup unless `allow_insecure_public_http: true` is set. Enable TLS for public traffic. A route to a loopback or private-network target makes that target reachable through NetGoat, so keep a public listener behind TLS or enable authentication before adding such a route. The shipped `routes: {}` is intentionally empty: a fresh default deployment returns `404` for every Host instead of proxying to a local service. If the control plane is unavailable, NetGoat uses local routes and then the last valid recovery snapshot. Configure `api.url` as an empty string for a fully offline deployment. ## Docker ```sh docker build -t netgoat-agent . ``` The image is a distroless static runtime that runs as `nonroot`. It ships the repository's `config.yml` unchanged, which binds `127.0.0.1:8080` — inside a container that is only reachable from the container itself. Mount a config that listens on a container address before publishing a port: ```yaml # config.container.yml listen: "0.0.0.0:8080" # Only behind a TLS-terminating proxy you control. allow_insecure_public_http: true routes: {} ``` ```sh docker run --rm -p 8080:8080 \ -v "$PWD/config.container.yml:/app/config.yml:ro" \ -v netgoat-data:/app/database \ netgoat-agent ``` Mount `/app/database` so SQLite state, recovery snapshots, and ACME material survive a restart. The healthcheck is a TCP probe on port 8080 because the agent answers `404` for unconfigured hosts, so an HTTP status check would report a healthy server as failing. If your config moves the listener (`ssl.enabled`, a different `listen` address), point the probe at it: ```sh docker run --rm -p 8443:8443 \ -e NETGOAT_HEALTHCHECK_PORT=8443 \ -v "$PWD/config.tls.yml:/app/config.yml:ro" \ -v netgoat-data:/app/database \ netgoat-agent ``` Published images: `ghcr.io/netgoat-xyz/netgoat`. The image does not include Python, so the optional `koda_waf`, `koda_2`, and `anomaly` inference workers are unavailable in the container. Run those features from a source checkout, or build a derived image that adds a Python runtime and the `ai/` directory. ## Authentication bootstrap Fresh databases do not contain a default password. To enable local authentication, set both bootstrap variables before the first start: ```sh export NETGOAT_BOOTSTRAP_USERNAME=admin export NETGOAT_BOOTSTRAP_PASSWORD='replace-with-at-least-12-characters' ``` Then set `auth.enabled: true`. Bootstrap credentials are used only when the user table is empty; existing users are not overwritten. Basic authentication does not create persistent cookie sessions. ## Configuration highlights - `routes`: local fallback routes keyed by domain, wildcard/regex pattern, or path prefix; each route can override `policy.cache` and `policy.bandwidth`. - `api`: control-plane URL, key, poll interval, timeout, and maximum retry interval. - `health`: probe enablement, interval, timeout, and default path. - `cache`, `rate_limit`, `request_queue`, `bandwidth`: bounded global traffic defaults; cache and bandwidth can be overridden per route. - `metrics`: enables JSON at the configured path and Prometheus at `.prom`. - `listen`: plaintext HTTP bind address when TLS is off; the sample uses loopback. - `allow_insecure_public_http`: explicit opt-in for plaintext HTTP on a public address. - `ssl`: static fallback TLS, per-domain certificate selection, and optional ACME issuance/renewal. - `bot_auth`: pinned https Web Bot Auth directory allowlist; empty pins fail open to session-bound PoW. - `dynamic_rules`: bounded administrator-managed TypeScript/JavaScript request decisions. - `plugins`: restart-only catalog selections for middleware compiled into this exact agent build; `sha256` is the release descriptor fingerprint, not a downloaded artifact hash. - `cloudflare`: optional Access JWT enforcement and explicit DNS/tunnel startup reconciliation. Reconciliation defaults to dry-run. - `telemetry`: disabled by default; endpoint, shared ingestion key, and heartbeat interval. - `anomaly`, `koda_waf`, `koda_2`: optional local inference workers. Secrets may also be supplied through the environment. `API_STREAM_KEY` overrides the YAML control-plane key, `NETGOAT_ACME_CACHE_KEY` encrypts ACME state, `CLOUDFLARE_API_TOKEN` is required for Cloudflare reconciliation, and `NETGOAT_CHALLENGE_SECRET` (or `DiamondKey`) binds PoW commitments. Do not commit `.env`, private keys, model files, databases, recovery snapshots, or telemetry identifiers. See the [operations guide](docs/operations.md) for production deploy, metrics scrape, policy precedence, ACME setup, and dynamic-rule safety boundaries; the [release train](docs/release.md) for `v*` tags and the alpha→beta gate; the [middleware SDK guide](docs/middleware-sdk.md) for trusted compiled-in extensions; and the [developer plugin catalog guide](docs/developer-plugins.md) for the restart-time selection and publisher trust boundary. ## Architecture ```text client -> NetGoat agent -> healthy upstream pool | | | +-> SQLite state + recovery snapshot +------> stream-server (optional control plane) +------> telemetry-server (optional, opt-in) ``` The agent's hot request path optionally verifies Cloudflare Access, applies authentication and traffic controls, resolves a route, evaluates dynamic rules, precompiled WAF rules, and any selected compiled middleware, optionally runs enabled local classifiers, and proxies the request. Health checks and control-plane polling run in bounded background workers. The optional `docker-compose.yml` starts only a loopback-bound development MongoDB for `stream-server`; the Go agent itself does not require it. Export `MONGO_INITDB_ROOT_USERNAME` and `MONGO_INITDB_ROOT_PASSWORD` before running Compose so development credentials stay outside the repository. ## Development Run the full Go verification suite before submitting changes: ```sh go test ./... go test -race ./... go vet ./... ``` Python worker syntax can be checked without installing their model dependencies: ```sh python3 -m py_compile ai/*.py ``` See `CONTRIBUTING.md` for contribution conventions, `docs/release.md` for tagging and the alpha→beta gate, `docs/operations.md` for production deploy, and `SECURITY.md` for private vulnerability reporting. ## Project links - Community: [Discord](https://discord.com/invite/3aJ7MdJsZV) - First donor: [Cozy Critters Society](https://opencollective.com/cozy-critters-society) - License: [AGPL-3.0](LICENSE) Special thanks to **Cozy Critters Society** and **Snow** for being NetGoat's first donors.