---
name: anakin-web-data
description: Use when fetching content from the web with Anakin — reading a page, getting a site's URL structure, or bulk-collecting many pages. Covers scrape, map, and crawl, and routes to the right Anakin tool for a task. Triggers on scraping a URL, extracting page content as markdown, listing a site's pages, ingesting a site, building a corpus from a domain, or handling JS-heavy/SPA pages.
---

# Anakin: fetching web data

## Pick the right tool first

Anakin's tools fall into five groups. Choosing wrong is the most common and
most expensive mistake — check this table before calling anything.

| The task | Tool | Skill |
|---|---|---|
| Read one known URL | `scrape` | this one |
| Find what pages exist on a site | `map` | this one |
| Collect many pages from one site | `crawl` | this one |
| Find pages across the web for a query | `search` | `anakin-research` |
| Answer a question needing many sources | `agentic_search` | `anakin-research` |
| See what AI engines say about something | `ai_visibility_search` | `anakin-research` |
| Extract data from a *specific popular site* | `wire_discover` → `wire_read_action` | `anakin-wire` |
| Submit a form, add to cart, post content | `wire_discover` → `wire_write_action` | `anakin-wire` |
| Watch a page for changes over time | `monitor_create` | `anakin-monitoring` |
| Multi-step interaction no Wire action covers | `browser_task` | `anakin-browser` |
| Reach login-protected content | `session_list` → pass `sessionId` | `anakin-browser` |

**Check Wire before reaching for `scrape` or `crawl` on a well-known site.**
Amazon, Walmart, LinkedIn, Airbnb, Zillow and hundreds more have vetted,
pre-built extractors that return clean structured data in one call. Scraping
those sites by hand is slower, costs more, and gives you markdown you then have
to parse. See the `anakin-wire` skill.

**Escalate in cost order.** `scrape` → `scrape` with `useBrowser` → a Wire
action → `browser_task`. Each step is slower and more expensive than the last;
do not start at the end.

## scrape — one URL to markdown

Default behavior returns clean markdown. That is the right default; do not add
options reflexively.

- `generateJson: true` — also run AI extraction and return typed fields. Use for
  product pages, listings, articles — anywhere you want structured values rather
  than prose. Returns the JSON as the primary output.
- `useBrowser: true` — render with a stealth headless browser. **Slower and more
  expensive.** Only for SPAs and JS-rendered content. Try the default first; if
  the markdown comes back empty or is missing the content you can see in a
  browser, retry with `useBrowser: true`.
- `country` — two-letter proxy egress code, defaults to `"us"`. Set it when a
  page is geo-restricted or price/inventory varies by market (`"de"`, `"in"`, …).
- `forceFresh: true` — skip the cache. Results cache for roughly 24h, which is
  usually what you want. Only force fresh for genuinely live data (stock levels,
  prices you are about to act on).
- `sessionId` / `sessionName` — a saved browser session for login-protected
  pages. Pair with `useBrowser: true`.

## map — discover a site's URLs

Returns internal links, external links, and counts. Use it to understand
structure *before* crawling, so the crawl is scoped instead of blind.

- `limit` (default 100, max 1000) — total URLs returned.
- `depth` (default 2, max 5) — link-hops followed.
- `limitPerLevel` (default 100) — breadth cap per level.
- `search` — keyword filter on path/title. Cheap way to find the section you
  care about without crawling everything.
- `includeSubdomains`, `includeExternalLinks` — both default false.

## crawl — bulk markdown from a site

For catalog ingestion or building a RAG corpus. Returns an array of pages, each
with markdown and its own status — **check per-page status**, since a crawl can
partially succeed.

- `maxPages` (default 10, max 500) — a hard cap, and the main cost lever.
- `depth` (default 1, max 5).
- `includePatterns` / `excludePatterns` — glob/regex URL filters. Use these.
  An unscoped crawl burns credits on nav pages, tag archives, and pagination.

## Working pattern

For anything beyond a single page, `map` first, then `crawl` the subset:

1. `map` the domain with a `search` filter to see what exists.
2. Derive `includePatterns` from the URLs that matter.
3. `crawl` with those patterns and a deliberate `maxPages`.

This is consistently cheaper and cleaner than crawling from the homepage and
filtering afterward.

## Cost and failure notes

- `useBrowser` is the biggest cost multiplier — leave it off unless the page
  needs it.
- Every tool returns an error envelope rather than throwing. On failure, read
  the message: it names the tool and the cause.
- Authentication depends on the server. The hosted server (`mcp.anakin.io`,
  used by the Claude plugin) signs in with the user's Anakin account over OAuth,
  so there is no key to set. The local server (`@anakin-io/mcp`) reads
  `ANAKIN_API_KEY`. Either way an account comes from
  https://anakin.io/dashboard — free tier is 300 credits, no card.