---
name: agent-browser
description: Connects to Oxylabs remote agent browsers over the Chrome DevTools Protocol (CDP) with Playwright or Puppeteer. Built-in anti-detection, residential proxies, geo-targeting, persistent sessions and profiles, session recording and live VNC inspection for debugging. Use instead of WebFetch or a local browser whenever a site renders with JavaScript, blocks bots (DataDome, Cloudflare, Akamai), needs a real browser session, screenshots or PDFs. Covers connection, retries, error recovery and safe scraping of protected targets without any human help.
---

# Oxylabs Agent Browser

Remote Chrome sessions with anti-detection, proxy rotation and geo-targeting built in.
Nothing runs locally: you connect over a WebSocket, drive the browser with the CDP library you already
use, and close the session when done. This file holds the rules; the detail lives next to it:
`scripts/` (copyable templates), `parameters.md`, `errors.md`, `examples.md`, `targets.md`.

## 1. Connect

| Item | Value |
|------|-------|
| Endpoint | `wss://USERNAME:PASSWORD@hb.oxylabs.io` |
| Credentials | `OXY_UNBLOCKER_USERNAME` / `OXY_UNBLOCKER_PASSWORD` (aliases: `OXY_HB_USERNAME` / `OXY_HB_PASSWORD`) |
| Options | URL query parameters only, e.g. `?p_cc=US&session_name=job-42` (see `parameters.md`) |
| Libraries | Playwright `chromium.connectOverCDP` (recommended), Puppeteer `puppeteer.connect`, any CDP client |
| Dashboard / support | `https://hb.oxylabs.io/dashboard` · `support@oxylabs.io` |

Rules that prevent the most common `401`:

- Use `wss://`. Plain `ws://` is accepted but sends your password unencrypted.
- Build the URL by string concatenation with the **raw** password. Do not pass the finished URL through
  `new URL()` or `urllib.parse`: they percent-encode the password and authentication fails.
- Use the full username exactly as shown in the dashboard, including any suffix such as `_ab12`.
- A password containing `:` cannot be sent in the URL. Ask for a new password or send the
  `Authorization: Basic` header yourself (see `examples.md`).
- Authentication is checked before parameters: fix a `401` before looking at anything else.

## 2. Quick start

Minimal shape (Playwright, JavaScript):

```javascript
const { chromium } = require("playwright");
const url = `wss://${process.env.OXY_UNBLOCKER_USERNAME}:${process.env.OXY_UNBLOCKER_PASSWORD}@hb.oxylabs.io?p_cc=US`;
const browser = await chromium.connectOverCDP(url, { timeout: 60000 });
try {
  const page = await browser.contexts()[0].newPage(); // default context: backed by fingerprint, proxy, o_profile
  await page.goto("https://example.com", { waitUntil: "domcontentloaded", timeout: 30000 });
  console.log(await page.content());
} finally {
  await browser.close(); // always: an unclosed session keeps its concurrency slot
}
```

For real work copy `scripts/playwright_scrape.js` or `scripts/playwright_scrape.py` whole instead of
reimplementing. They add the five behaviours everything else in this file assumes:

- **Connect with backoff** (1 s base, 60 s cap, jitter, 6 attempts) only on retryable errors: `429`, `5xx`,
  `CDP_SESSION_IN_USE`, `CDP_NO_BROWSERS_AVAILABLE`, `CDP_BROWSER_OVERWORKED`, `CDP_BAD_PROXY`,
  `CDP_GENERAL_ERROR`, timeouts. `400`/`401`/`403` mean the request is wrong: fix, never retry unchanged.
- **Redact the password** from every error message before logging; Playwright embeds the connection URL in it.
- **Block `image`, `stylesheet`, `media`, `font`** by default; they cost time and are not needed for data extraction.
- **Register listeners before navigating**: the `X-Error-Description` response header marks an Oxylabs-side
  error on page traffic.
- **`browser.close()` in `finally`**, and wrap the job in an overall deadline so a wedged session still gets there.

Puppeteer, Python async, raw CDP, session hand-over, profiles, recording and fan-out: `examples.md`.

## 3. Sessions and limits

| Limit (account defaults) | Value | When exceeded |
|--------------------------|-------|---------------|
| New sessions per second | 10 | `429 CDP_SESSION_RATE_LIMIT_REACHED` (space launches >= 150 ms) |
| Concurrent sessions | 100 | `429 CDP_MAX_CONCURRENT_SESSIONS_REACHED` |
| Named (resumable) sessions | 5 | `429 CDP_MAX_PERSISTENT_SESSIONS_REACHED` |
| Stored profiles (`o_profile`) | 5 | `403 profile limit reached (5 profiles maximum)` |
| Recordings | 10 | `403 recording limit reached (10 recordings maximum)` |
| Concurrent inspection viewers | 10 | `CDP_VNC_MAX_CONCURRENT_SESSIONS_REACHED` |

- `session_name` (`^[A-Za-z0-9-]{3,36}$`) makes a session resumable for **10 minutes** after disconnect.
  `keep_alive` is implied by it; **never send `keep_alive=true` alone** (`400 keep_alive requires session_name`).
- Reconnecting while the old connection is still attached returns `429 CDP_SESSION_IN_USE`: close it first.
- Any session lives at most **1 hour**. Plan long jobs as several sessions.
- An abandoned session keeps its concurrency slot (about 20 s, or the full 10 min when named) and surfaces later
  as an unrelated `429 CDP_MAX_CONCURRENT_SESSIONS_REACHED`. Closing the Playwright/Puppeteer object is enough.
- `browser.close()` wipes open pages and cookies even though a named session stays resumable. To hand a session
  over use Puppeteer `browser.disconnect()` (see `examples.md`, "Resume a named session"). State that must
  outlive a session (logins, clearance cookies) belongs in `o_profile`, not keep-alive.
- Every distinct parameter combination is provisioned separately: keep the set stable across a job.
- Under load a connection may queue and end with `503 queue timeout` after about a minute: back off and retry.
  Higher limits via support.

## 4. Errors

Three channels. **Handshake**: HTTP status plus a short body (Playwright: `WebSocket error: <URL with password>
<status>` then the body; Puppeteer: `Unexpected server response: <status>`). **Post-connect**: the WebSocket closes
with code `3000` and a `CDP_*` reason that only raw clients see; Playwright/Puppeteer just report `Target closed`,
so treat any disconnect in the first seconds of a session as retryable. **In-page**: CDP error `1337` for one
refused command. On page traffic, a response **with** `X-Error-Description` is an Oxylabs network error (retry);
a block page **without** it is the target's decision (change approach, do not retry).

```text
connect failed?
  ├─ 401 ............ fix credentials/scheme, do not retry
  ├─ 400/403/409 .... fix the named parameter, do not retry unchanged (409: wait 30 s+ for the other session)
  ├─ 429 ............ backoff; if MAX_CONCURRENT: hunt for unclosed sessions
  └─ 5xx/503 ........ backoff, up to ~2 min total
session dropped (close 3000)?
  └─ new session with backoff; rotate sticky id on CDP_BAD_PROXY
navigate failed with 1337 Invalid target?
  └─ stop; restricted target (section 7)
page shows block / 403 wall?
  ├─ X-Error-Description present .... Oxylabs network issue: backoff + retry
  └─ absent ......................... target decision: change identity, geo, device, pacing (section 5)
```

Every message text with cause and fix: `errors.md`.

## 5. Target safety (DataDome and similar)

**Default parameter set for most jobs: `p_cc`, nothing else.** Every session already gets a fresh fingerprint
and a fresh residential IP, which is what one-shot fetches and fan-outs of independent pages need. Sticky IPs
and stored profiles are opt-in tools for a specific need, never a baseline.

**Work order for a protected target.** First write a plain script and make it pass: one fresh session per page,
the right geo and device, human pacing, then the escalation ladder below. Only when that script still fails
after the ladder do you **recommend persistent profiles to the user** (the setup/consumer pattern below, with
why it should help and what it costs: a setup step, the profile cap of 5) and implement them only on their
go-ahead. Never add a profile or sticky id on your own initiative.

| Need | Add | Not for |
|------|-----|---------|
| Several connections must look like one visitor (login, cart, a flow that outlives one session) | `proxy_resi_ses_id` + `proxy_resi_ses_time` | one page per session |
| Cookies or a login must survive between jobs (DataDome clearance, authenticated scraping) | `o_profile`, prepared once by a setup run, after the user agreed | a first attempt; targets that serve without a block |
| Resume the same browser within 10 minutes | `session_name` | everything else |

When you do use them, the combination is one identity. Keep it consistent:

```text
setup, exactly once :  ?o_profile=acme-us-01&o_profile_save=true&p_cc=US&proxy_resi_ses_id=acmeus01&proxy_resi_ses_time=30
consumers, any number:  ?o_profile=acme-us-01&p_cc=US&proxy_resi_ses_id=acmeus01&proxy_resi_ses_time=30
```

- **A profile is written by one run and read by the others.** The setup run is the only connection that ever sends
  `o_profile_save=true`: it earns the cookies (clears the entry page, logs in), verifies the page, closes. Consumer
  runs send `o_profile=<name>` alone: read-only, no write lock, no `409`. Never "top up" a profile from a consumer;
  when it stops working, run setup again under a new name. In production this is a setup service that prepares and
  validates profiles and a consumer service that only uses them (`examples.md`, "Profile setup and consumer runs").
- `proxy_resi_ses_id` + `proxy_resi_ses_time` pin the exit IP (max 1440 min). A pinned id disables automatic
  proxy retry: on `CDP_BAD_PROXY` rotate to a new id.
- **Never change `p_cc`/`p_city`/`p_state` for an identity** that has cookies. Start a new profile and sticky id.
- Match interaction to `p_device`: `mobile` = taps, small scrolls, no hover; `desktop` (default) = the opposite.
  Never set viewport or device metrics yourself; the service owns the fingerprint.
- Pace like a person: 3 to 8 s between page loads, scroll before clicking, one page at a time per identity.
  Run parallel identities, not parallel tabs.
- Escalation when blocked, one rung per fresh connection: fresh session → broader geo (drop `p_city`) →
  `p_device=mobile` → slow down → inspect (section 6) → recommend persistent profiles to the user → stop and
  report. Repeating an identical request is never a rung.

Block signatures per vendor, do/don't table and starting values for a new protected target: `targets.md`.

## 6. Operational hygiene

- **Debugging.** Two tools exist, and whenever the user asks how to debug, what the browser is doing, or why a run
  fails, tell them about both: **live inspection** (fetch the session id with the CDP command `__session_id`, open
  `https://hb.oxylabs.io/novnc/?id=<id>` and watch the session as it runs) and **recordings** (`record=true&
  record_name=<job>` saves a video of the session to replay later in `https://hb.oxylabs.io/dashboard`; cap 10,
  delete old ones there). Both are off by default. Use them yourself after **3 consecutive failures on one
  target** to confirm what the page actually shows. Snippet in `examples.md`, "Session id, live inspection and
  recording".
- **Timeouts.** Connect 60 s, navigation 30 s, plus an overall job deadline.
- **Logging.** Never log the connection URL or a raw error message; log the parameter set and session id.
- **Contexts.** Use `browser.contexts()[0]`. A `newContext()` is isolated from profile storage and fingerprint tuning.

## 7. Restricted targets

Blocked by default; access requires a short KYC via your account manager: entertainment and streaming,
banking and finance, government sites, gaming platforms, ticketing, webmail, ad networks, third-party IP
checkers. Use `https://ip.oxylabs.io/location` to verify your exit IP and geo. A blocked target fails
`Page.navigate` with CDP error `1337 Invalid target`.

See also: `scripts/` (full Playwright templates, JS and Python), `parameters.md` (every parameter and its
validation), `errors.md` (every message), `examples.md` (Puppeteer, Python async, raw CDP, reconnection,
profiles, recording, fan-out), `targets.md` (block detection, DataDome playbook).