# url-context-mcp

An MCP (Model Context Protocol) server that fetches web pages and extracts clean, AI-usable context from them. Runs over **stdio** transport.

It's designed for AI agents that need to:

- Read a web page as clean markdown (no boilerplate, no nav/footer noise)
- Discover links on a page (with filters) before deciding what to fetch next
- Search pages for specific text/code without reading the whole thing
- Do all of the above in a single call

## Tools

| Tool | What it does |
| --- | --- |
| `fetch_url` | Fetch a URL → readable article as markdown (or `text`/`html`/`readable`). Optional `includeLinks=false`, `maxChars`. |
| `extract_links` | Get all links from a URL or raw HTML. Optional regex `filter`, `sameDomainOnly`, `limit`. Resolves relative links correctly. |
| `search_content` | Regex search a URL's content; returns matches with surrounding context. |
| `fetch_and_search` | One-shot combo: returns content **and** links **and** search matches in a single call. The most token-efficient way to understand + navigate a page. |
| `fetch_image` | Fetch an image by URL (absolute or relative to a base page) and return it as base64 image content the model can see. Resolves relative URLs from page extraction (e.g. `images/logo.png` against the page that referenced it). Optional `maxSizeKB` guard (default 512 KB); set `compress=true` to have an oversized image automatically resized + re-encoded (JPEG, or PNG for transparency/animation) to fit within `maxSizeKB` instead of being rejected. Compress tuning: `compressQuality`, `compressMaxWidth`, `compressMaxHeight`. |

All tools accept either a `url` (server fetches it) or raw `html` (you provide the markup), so they're composable with other fetchers too.

## Installation (copy-paste into your MCP client config)

```json
{
  "mcpServers": {
    "url-context": {
      "command": "npx",
      "args": ["-y", "url-context-mcp"]
    }
  }
}
```

That's it. `npx -y` downloads and runs the server on first use; no separate install step needed. Requires Node.js 18+.

### Why it's the best design for an AI agent

- **Readability-first**: defaults to Mozilla Readability extraction so the agent gets the article, not the chrome. Reduces tokens and irrelevance.
- **One-call combo**: `fetch_and_search` returns content + links + matches together — fewer round trips when an agent wants to "understand this page and find X on it".
- **Link discovery before prefetch**: `extract_links` lets an agent survey a page's destinations and pick the right one to fetch, instead of guessing or fetching everything.
- **Regex search with context**: agents can locate an error string, an API endpoint, a version number, etc. on a page with bounded context windows.
- **No browser, no head**: pure HTTP + jsdom. Fast, sandboxed, works anywhere Node runs. No Playwright/Chromium dependency bloat.
- **Sane defaults + truncation**: `maxChars`, `limit`, `maxMatches` keep responses bounded so context windows don't blow up.

## Local development

```bash
npm install
node index.js          # run the stdio server
npm pack               # build the publishable tarball
```

## Changelog

### 1.3.0

- **`fetch_image`: optional compression.** Oversized images can now be
  shrunk to fit within `maxSizeKB` instead of being rejected. Set
  `compress=true` (default false) and the image is resized to
  `compressMaxWidth`×`compressMaxHeight` (defaults 1024×768, aspect
  preserved) and re-encoded as JPEG (or PNG when the source has an alpha
  channel or is animated), lowering `compressQuality` (1–100, default 75)
  in steps of 10 until under the limit. Supports tuning via
  `compressQuality`, `compressMaxWidth`, `compressMaxHeight`.
- **`fetch_image`: hardened DoS guards.** The response body is streamed
  with a hard byte cap (applied regardless of `compress`), so chunked/
  lying-header/infinite bodies can't OOM the server. Image decoding runs
  with `failOn: "error"`, an explicit `limitInputPixels` (≈26 MP) and
  rejects via a pre-decode dimension check, blocking decompression bombs.
  The compression encode loop is bounded by iteration count (6) and a
  15 s deadline.
- Server version bumps to 1.3; user-agent string updated accordingly.

### 1.2.0

- **New tool: `fetch_image`.** Resolves an image URL (absolute, or relative
  to a `baseUrl`) and returns it as an MCP `image` content block (base64
  with a `mimeType` derived from the response Content-Type header, falling
  back to the URL extension). Lets the model actually *see* images found in
  pages marked up in markdown. Optional `maxSizeKB` (default 512 KB) protects
  the context window from oversized payloads. MIME fallback map covers
  png/jpg/jpeg/gif/webp/svg/bmp/avif/ico.
- Server version bumps to 1.2; user-agent string updated accordingly.

### 1.1.0

- **Preserve dropped headings in extracted content.** Mozilla Readability
  strips the page `<h1>`/title from its `content` on minimal pages
  (e.g. `example.com`: the `# Example Domain` heading was never returned by
  `fetch_url`, and `search_content` therefore couldn't find "Example
  Domain"). We now detect headings present in the raw body but missing from
  Readability's cleaned HTML and re-prepend them, so the page title is
  always part of the returned markdown/text and is searchable.
- **`search_content` raw-HTML fallback.** When a search of the converted
  (markdown/text) content finds zero matches and raw HTML is available,
  the tool re-runs the search against the stripped raw text as a safety net.
  The result flag notes when the fallback path was used (`(raw-HTML
  fallback)` in the header).
- Misc: user-agent string bumped to `1.1`.

## License

MIT