WebBrain

Open-source AI browser agent for chatting with pages, automating tasks, and running multi-step workflows with your choice of LLM.

English · 中文 · Français · Docs · Website · Discord · GPL-3.0-or-later

` handoff. Details: [providers and models](docs/providers-and-models.md#local-providers). **Cloud APIs** — OpenAI, Anthropic Claude, Google Gemini, Azure OpenAI, AWS Bedrock, Mistral, DeepSeek, xAI Grok, MiniMax, Kimi, Qwen, z.ai GLM, Groq, Together, Cloudflare, Nvidia NIM, Hugging Face, Fireworks, OpenRouter, and more. The Providers page ships **117 built-in provider cards on Chromium** (116 on Firefox), including an endpoint-free local WebGPU option with the tested LFM2.5 2.6B preset and an experimental custom Hugging Face ONNX repository option — see the [full catalog](docs/providers-and-models.md#extended-provider-catalog). For a local or bring-your-own provider, the per-provider **Share queries for research** switch remains off by default. When enabled, it now shares a bounded, content-free diagnostic timeline for that provider's model attempts (including failed runs) alongside the existing scrubbed prompt/response share. The timeline includes tool names, outcomes, error codes, and timings, but not tool arguments, page content, or screenshots. If a run fails before a normal generation share, its bounded model-facing request and final blocker accompany the diagnostic record. No second sharing switch is required; turning the existing switch off also purges queued diagnostics. ## Features - **Reads any page** — text, links, forms, tables, PDFs, and interactive elements, via the accessibility tree rather than brittle selectors - **Acts on it** — click, type, scroll, navigate, upload, download, and verify forms, with per-site permission prompts before consequential actions - **Vision mode** (aka "Tesla" mode) — vision only, like a Tesla on cameras: with a model that takes images, Act runs see the page as screenshots and click, type and scroll at coordinates; only the changed part of the screen is sent after each action, and retina displays and browser zoom are handled ([details](docs/vision-mode.md)) - **Plan before Act** — Act and Dev can generate a structured plan, show it for approval, and pin the approved plan to the scratchpad before any tool runs - **Multi-step agent** — autonomous tool-use loop, configurable up to 195 steps (default 130), with a Continue button when it hits the limit - **Saved workflows** — turn a successful run into a reusable, value-free workflow you can re-run, export, and share - **Scheduled tasks and watches** — `/schedule` for later, `/watch` to poll a page and act when a condition is met - **Skills** — trusted instructions and tools that load only when relevant - **Smart context** — token-aware auto-compaction, tool-result limits, and emergency overflow recovery - **Per-tab conversations** — each tab keeps its own history; optional local user memory for stated preferences - **Optional cross-device recall** — connect MemCode with OAuth for read-only lookup while local memory stays the default ([setup and privacy](docs/memcode-recall.md)) - **Reading-first side panel** — streaming Ask replies, floating controls that keep your question in view as answers grow, copy buttons, a page-inspection banner, and a stop button that works mid-run - **Deterministic by default** — temperature `0.15` for browser-control decisions, `0.3` for Ask, `0` for vision screenshot descriptions ## Agent tools WebBrain separates **tier** from **mode**. Tier (`compact`, `mid`, `full`) is a per-provider setting controlling how many tools a model sees — Compact suits small local models, Full unlocks hover, drag-drop, frames, and shadow DOM. Mode (`ask`, `act`, `dev`) controls what the user is allowing. The full tool-by-tier matrix, WebMCP notes, and Dev-mode diagnostics are in [agent tools](docs/agent-tools.md). ## Slash commands Type `/help` in the panel for full signatures and flags. The most useful ones: | Command | What it does | | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | `/ask` · `/act` · `/dev` · `/plan` | Switch mode before sending | | `/schedule [prompt]` | Create a scheduled task | | `/watch [--keep] [--secs <30-120>] [--long \| --short] [/beep]` | Poll the current page and act when a condition is met | | `/workflow` · `/workflow --save ` | Manage saved workflows or compile the last successful run | | `/teach --start ` · `/teach --end` | Learn a reusable workflow from your demonstrated actions | | `/memory --add ` | Save a user preference | | `/screenshot [--full-page]` | Capture the tab, or the full scrollable page | | `/record [--transcribe]` | Record the current tab, optionally saving a transcript | | `/export [--traces \| --config]` | Download the conversation, tool chain, or a Settings snapshot | | `/compact` · `/reset` · `/verbose` | Compact context, clear the conversation, toggle tool detail | | `/allow-api` | Per-conversation override letting `fetch_url` mutate when the UI is failing | `/watch` runs its first check immediately, then polls every 60 seconds (`--secs` accepts 30–120). Relative conditions like "when a new commit appears" establish a baseline on the first check; `--keep` keeps the watch running and suppresses repeated alerts for the same stable event key. Full reference, including `/dangerously-skip-permissions` and the run-capture suffixes: [slash commands](docs/slash-commands.md). ## Keyboard Shortcuts Chrome side panel shortcuts work when the WebBrain side panel has focus. | Shortcut | What it does | | ------------------------------- | ---------------------------------------------------------------------------- | | `Ctrl+/` or `Cmd+/` | Focus the input | | `Ctrl+Shift+A` or `Cmd+Shift+A` | Start the message with `/ask` | | `Ctrl+Shift+X` or `Cmd+Shift+X` | Start the message with `/act` | | `Ctrl+Shift+D` or `Cmd+Shift+D` | Start the message with `/dev` | | `Escape` | Stop the active run, unless it is only dismissing slash-command autocomplete | | `Escape` twice | Stop an active recording from WebBrain or browser pages | ## Documentation | | | | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------- | | [Architecture](docs/architecture.md) | System overview, turn flow, subsystems | | [Agent tools](docs/agent-tools.md) | Tiers, modes, and the full tool matrix | | [Slash commands](docs/slash-commands.md) | Every command and flag | | [Providers and models](docs/providers-and-models.md) | All provider cards, local setup, tiers | | [Skills](docs/skills.md) | Bundled skills, importing, skill tools | | [Security model](docs/security-model.md) | Permissions, credentials, trust boundaries | | [Cloud Bridge approval](docs/cloud-bridge-browser-approval.md) | Opt-in browser registration and approval for backends | | [Prompt-injection defense](docs/prompt-injection-defense.md) | Defense layers and known gaps | | [Privacy and data flow](docs/privacy-and-data-flow.md) | What leaves the browser, and what doesn't | | [Accessibility tree and refs](docs/accessibility-tree-and-refs.md) | How pages are read and targeted | | [Site adapters](docs/site-adapters.md) | Per-site guidance and versioned workflow contracts | | [Export and workflow formats](docs/export-and-workflow-formats.md) | `webbrain-config/1`, `webbrain-workflow/1` | | [Adding a tool](docs/adding-a-tool.md) · [Localization](docs/localization.md) · [Test scenarios](docs/test-scenarios.md) | Contributor guides | | [Community](docs/community.md) | Discord server guide: channels, roles, rules, escalation | Also available in [中文](docs/zh-CN/) and [Français](docs/fr/). ## Community Chat about everything WebBrain — help, local and cloud model setups, site adapters, show-and-tell, and contributor coordination — on the [WebBrain Discord](https://discord.gg/cgC325ssfw). See [community](docs/community.md) for how the server is organized, and [discord-setup](docs/discord-setup.md) for the channel, role, and welcome-screen configuration. Bug reports and feature requests belong in [GitHub issues](https://github.com/webbrain-one/webbrain/issues), not Discord. ## Repository layout ``` src/chrome/ Manifest V3 build — service worker, chrome.scripting, sidePanel src/firefox/ Manifest V2 build — background page, executeScript, sidebar_action docs/ Design and reference docs (en, zh-CN, fr) mcp-server/ MCP server — delegate browser tasks from Claude Code, Codex, Cursor lmstudio-plugin/ Web tools + browser delegation as a standalone LM Studio plugin web/ Landing site and docs site test/ Node test suite, LLM scenario benchmarks, security corpora ``` Nearly all agent code is shared between the two builds. See [architecture](docs/architecture.md#chrome-vs-firefox-key-differences) for where they diverge. ## Known issues **Firefox is meaningfully weaker than Chrome.** Firefox has no equivalent to the Chrome DevTools Protocol via `chrome.debugger`, so the Firefox build has no shadow-DOM piercing, no real trusted mouse events (some React/Vue handlers won't fire), no closed-shadow-root traversal, no `resolveSelector` retry budget, no SPA-navigation-aware retry, and no CDP screenshots. It uses `tabs.captureTab` for viewport screenshots, including inactive run tabs, but still cannot provide Chrome's pixel-perfect or full-page CDP capture. Site adapters, vision detection, loop detection, the auto-screenshot loop, and the Compact prompt/tool set _are_ mirrored to Firefox. Some single-page apps may also fail to trigger content-script re-injection after client-side navigation. ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). To add a tool, follow the checklist in [adding a tool](docs/adding-a-tool.md). To add a provider, subclass `BaseLLMProvider`, implement `chat()` (and optionally `chatStream()`), and register it in `providers/manager.js` — mirroring both changes to `src/chrome/` and `src/firefox/`. All providers normalize to `{ content, toolCalls, usage }`; details in [providers and models](docs/providers-and-models.md#adding-a-provider). Recent changes are in [CHANGELOG.md](CHANGELOG.md). ## MCP server Let a coding agent use *your* browser. Claude Code, Codex, Cursor and OpenClaw can delegate a task to WebBrain running in the session you are already signed into — cookies present, SSO already passed. A headless framework starts logged out and stalls at the first login wall; this does not. ```bash claude mcp add --transport stdio webbrain -- npx -y @webbrain/mcp-server ``` Claude Code launches the server automatically when it starts an MCP session. To launch it yourself instead, run the following command and leave that terminal open (press `Ctrl+C` to stop it): ```bash npx -y @webbrain/mcp-server ``` Once the server is running, open **WebBrain → Settings → Bridge**, set the URL to `ws://127.0.0.1:17374/extension`, paste the pairing key printed by `npx -y @webbrain/mcp-server pairing-key` into **Pairing key**, and enable it. The server only trusts a socket that proves it holds that key (an HMAC over a fresh challenge; the key is never sent), so another local process cannot pose as the extension. The LM Studio plugin uses the same key file, `~/.webbrain/bridge-key`. Update the extension before upgrading the server: a version without the Pairing key field cannot pair. On Chromium browsers the bridge runs from the extension's off-screen document. Firefox hosts it in the background page and is configured under the same **Settings → Bridge** tab (not yet verified against the MCP server on a real Firefox install). A backend that needs to approve each browser before it can send commands can use the opt-in token handshake described in [Cloud Bridge browser approval](docs/cloud-bridge-browser-approval.md). If Settings reports **Connection error: WebSocket error**, nothing is normally listening at the configured URL. Start the MCP server, confirm that the URL uses port `17374`, and leave its process running. See the [`mcp-server` troubleshooting guide](mcp-server/README.md#troubleshooting) for a listener check and the other bridge ports. ``` webbrain_run(task: "open the Stripe dashboard and list last week's failed payments with amounts and customer emails", mode: "ask") ``` Use `webbrain_extract` with a JSON Schema when the caller needs predictable structured output instead of a prose summary. The server exposes six task-level tools: run, structured extraction, status, clarification response, abort, and connection diagnostics. `mode='ask'` is read-only. `mode='act'` can click and type, gated by the same in-browser approval prompts a human gets. The server exposes task delegation rather than the ~50 low-level browser primitives: WebBrain's permission gate lives in the agent loop, so per-primitive access over a socket would sit below the gate and bypass it. Details in [`mcp-server/`](mcp-server/). The complete client setup, tool arguments, run lifecycle, structured-output examples, safety boundaries, and troubleshooting guide live at [`web/docs/mcp/`](web/docs/mcp/). > The extension holds **one** bridge socket at a time — WebBrain Cloud (17373), > the MCP server (17374), or the LM Studio plugin (17375). Switch by changing > the URL under **Settings → Bridge**. ## LM Studio plugin A standalone [LM Studio](https://lmstudio.ai) plugin at [`webbrain/web-tools`](https://lmstudio.ai/webbrain/web-tools): ```bash lms clone webbrain/web-tools ``` `fetch_url` and `research_url` are pure Node HTTP — no browser needed, but also no cookies, no session and no JavaScript. With the extension installed on a Chromium browser, `browser_task` adds delegation to your real signed-in browser, reaching the authenticated and client-rendered pages plain HTTP cannot. It degrades with an actionable message when no extension is attached, and the HTTP tools keep working on Firefox. Source: [`lmstudio-plugin/`](lmstudio-plugin/). ## Contributors ## Citation ```bibtex @software{webbrain2026, author = {Sokullu, Emre}, title = {WebBrain: Open-source AI browser agent for chatting with pages}, year = {2026}, publisher = {GitHub}, url = {https://github.com/webbrain-one/webbrain} } ``` ## License WebBrain is licensed under [GPL-3.0-or-later](LICENSE) because the distributed browser extension bundles and integrates the GPL-licensed Xapian/libzim WebAssembly runtime. Built with ❤️ by [Emre Sokullu](https://emresokullu.com) and [open-source contributors](https://github.com/webbrain-one/webbrain/graphs/contributors).