BrowserControl
An MCP server that gives your agent a browser it can see.
One real session, twelve actions, nothing staged — navigate, scroll, type, select, check, click. Watch the numbers: they are rebuilt from scratch after every action, which is why the agent re-reads them each time.
## The idea
Most browser tools hand a model a DOM tree and hope it writes a working selector.
BrowserControl hands it a **picture**.
Every action returns a fresh screenshot with numbered red boxes drawn over the
interactive elements, plus the same list as text. The agent acts by number:
```text
click(5)
type_text(3, "hello world")
upload_file(7, "/path/to/resume.pdf")
```
No selectors to guess, none to break on the next redesign. This is the
[Set of Marks](https://adityasasidhar.github.io/browsercontrol/concepts/set-of-marks/)
pattern, used as the primary interface rather than a fallback.
## Quick start
```bash
uv add browsercontrol # or: pip install browsercontrol
```
Chromium installs itself on first run. If that fails, run
`python -m playwright install chromium` once.
Point your client at the `browsercontrol` command:
```json
{
"mcpServers": {
"browsercontrol": {
"command": "browsercontrol"
}
}
}
```
Where that file lives, per client
| Client | Location |
|---|---|
| **Claude Desktop** | `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) · `~/.config/Claude/claude_desktop_config.json` (Linux) · `%APPDATA%\Claude\claude_desktop_config.json` (Windows) |
| **Claude Code** | `claude mcp add browsercontrol -- browsercontrol` |
| **Cursor** | Settings → Model Context Protocol |
| **Cline** | Settings → MCP Servers |
| **Continue.dev** | `~/.continue/config.json`, under `mcpServers` |
| **Zed** | `~/.config/zed/settings.json`, under `context_servers` as `{"command": {"path": "browsercontrol"}}` |
No install at all, if you prefer: `"command": "uvx", "args": ["browsercontrol"]`.
Restart the client and ask it for something:
> *"Open Hacker News and tell me the top story."*
Full walkthrough: [Getting started](https://adityasasidhar.github.io/browsercontrol/getting-started/).
## The one rule
**Element numbers expire after every action.** The map is rebuilt from scratch on
every click, type, scroll, navigation, and screenshot — element `7` before a
click is rarely element `7` after it.
So an agent should never plan two clicks from one screenshot. It acts, reads the
new screenshot, finds the target again, then acts again. Almost every failure
worth debugging traces back to this.
Two corollaries:
- **Only what's in the viewport gets marked.** No number usually means "below the
fold" — scroll, then look again.
- **Cross-origin iframes are never marked.** Open shadow roots and same-origin
frames are traversed and numbered; third-party payment fields and embedded
widgets can't be. That's a boundary of the browser, not a bug.
## Tools
39 tools across seven categories. Everything that touches the page returns an
annotated screenshot plus the element map.
| Category | | Tools |
|---|:-:|---|
| [**Navigation**](https://adityasasidhar.github.io/browsercontrol/tools/navigation/) | 5 | `navigate_to` `go_back` `go_forward` `refresh_page` `scroll` |
| [**Interaction**](https://adityasasidhar.github.io/browsercontrol/tools/interaction/) | 7 | `click` `click_at` `type_text` `press_key` `hover` `scroll_to_element` `wait` |
| [**Forms**](https://adityasasidhar.github.io/browsercontrol/tools/forms/) | 3 | `select_option` `check_checkbox` `upload_file` |
| [**Content**](https://adityasasidhar.github.io/browsercontrol/tools/content/) | 5 | `get_page_content` `get_text` `get_page_info` `run_javascript` `screenshot` |
| [**Tabs**](https://adityasasidhar.github.io/browsercontrol/tools/tabs/) | 4 | `create_tab` `switch_tab` `close_tab` `list_tabs` |
| [**DevTools**](https://adityasasidhar.github.io/browsercontrol/tools/devtools/) | 11 | `get_console_logs` `get_network_requests` `get_page_errors` `run_in_console` `inspect_element` `get_page_performance` `get_cookies` `set_cookie` `delete_cookie` `clear_cookies` `set_viewport` |
| [**Recording**](https://adityasasidhar.github.io/browsercontrol/tools/recording/) | 4 | `start_recording` `stop_recording` `take_snapshot` `list_recordings` |
A few behaviours that aren't obvious from the names:
- `type_text` uses Playwright's `fill()` — it **replaces** the field, it doesn't append.
- `click` resolves the topmost element at the mark's centre, so a cookie banner
overlapping your target will eat the click. Dismiss overlays first.
- `upload_file` goes through `set_input_files`, which works on sites where
driving the file picker doesn't.
- `screenshot(annotate=True, full_page=True)` returns a *clean* image —
full-page captures can't be marked.
- `get_page_content` returns Markdown, capped at 30 KB.
- Recordings are Playwright traces. Open one with
`npx playwright show-trace ~/.browsercontrol/recordings/.zip`.
## Teach your agent to use it well
Connecting the server tells an agent *which* tools exist. The bundled
[agent skill](skills/browsercontrol/SKILL.md) tells it *how to use them well* —
that numbers expire, that a banner will swallow a click, that a missing number
means scroll.
```bash
mkdir -p ~/.claude/skills/browsercontrol
curl -fsSL https://raw.githubusercontent.com/adityasasidhar/browsercontrol/main/skills/browsercontrol/SKILL.md \
-o ~/.claude/skills/browsercontrol/SKILL.md
```
It loads only when a task actually involves a browser, so it costs one line of
context the rest of the time. Agents that would rather read the docs directly can
start at [`llms.txt`](https://adityasasidhar.github.io/browsercontrol/llms.txt).
## Configuration
Environment variables, read once at startup.
| Variable | Default | |
|---|---|---|
| `BROWSER_HEADLESS` | `true` | `false` to watch the browser work |
| `BROWSER_VIEWPORT_WIDTH` | `1280` | |
| `BROWSER_VIEWPORT_HEIGHT` | `720` | |
| `BROWSER_TIMEOUT` | `30000` | Navigation timeout, ms |
| `BROWSER_USER_DATA_DIR` | `~/.browsercontrol/user_data` | Profile directory |
| `BROWSER_EXTENSION_PATH` | — | Unpacked extension to load at launch |
| `BROWSER_EXECUTABLE_PATH` | — | Chromium binary to drive, when Playwright ships no build for your platform |
| `BROWSER_RECORDINGS_DIR` | next to the profile | Where traces are saved |
| `BROWSER_SNAPSHOTS_DIR` | next to the profile | Where snapshots are saved |
| `LOG_LEVEL` | `INFO` | |
```bash
BROWSER_HEADLESS=false BROWSER_VIEWPORT_WIDTH=390 BROWSER_VIEWPORT_HEIGHT=844 browsercontrol
```
## Why this one
The session is a real Chromium profile (`launch_persistent_context`), so cookies,
`localStorage` and logins survive restarts — log in once and your agent stays
logged in. DevTools are first-class: console, network timing, uncaught
exceptions, performance and computed styles are tools, not an afterthought. And
it runs entirely on your machine — no LLM API key, no cloud round-trip, no
per-action cost, nothing leaving the box.
Honest positioning: if you're happy driving the accessibility tree, Playwright
MCP is excellent. If you want an LLM planning the interactions for you, look at
Stagehand or Browser-Use. BrowserControl's niche is **vision-first, fully local
browser control with devtools built in**. Longer comparison
[in the docs](https://adityasasidhar.github.io/browsercontrol/concepts/comparison/).
## Development
```bash
git clone https://github.com/adityasasidhar/browsercontrol
cd browsercontrol
uv sync
uv run playwright install chromium --with-deps
uv run pytest # tests (Playwright is mocked — no real browser)
uv run ruff check . && uv run ruff format . # lint + format
uv run fastmcp dev browsercontrol/server.py # dev server with the MCP inspector
```
```
browsercontrol/
├── server.py # FastMCP instance, lifespan, tool registration
├── browser.py # BrowserManager: Playwright lifecycle, element detection, SoM rendering
├── config.py # env-var config
└── tools/ # one module per category, each exporting register_*_tools(mcp)
```
CI runs the suite on Ubuntu, Windows and macOS with `ruff`, `mypy --strict`,
`bandit` and `pytest`.
## Contributing
Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). New tools should
come with a test for the happy path *and* the error path; the existing
`tests/test_*.py` files show the pattern.
On the list, if you're looking for somewhere to start: Firefox and WebKit
support, mobile device presets, network mocking, DOM diffing between snapshots,
and accessibility auditing.
## License
[MIT](LICENSE). Built on [FastMCP](https://gofastmcp.com) and
[Playwright](https://playwright.dev).