# Firecrawl ↔ fastCRW Capability Matrix

**Date:** 2026-05-11
**Owner:** content/growth
**Re-verify within 48h of any page citing this doc.**
**Sources:** `docs.firecrawl.dev/sdks/rust` and `github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md` (Firecrawl repo on `main` as of 2026-05-11 — claims dependent on the exact Firecrawl self-host stack are stated as "as of 2026-05-11" and must be re-verified before any page citing this matrix ships); `firecrawl.dev/pricing`; fastCRW `crates/crw-server/src/routes/`, `crates/crw-search/src/`, `Cargo.toml`.

This is a **capability matrix**, not an API-shape compatibility matrix (which the parent plan's `COMPATIBILITY.md` already provides for Tavily). Capabilities here = what each product can actually do, not just whether endpoint names match.

> **Honest framing:** Firecrawl is a feature-richer cloud-first product with an OSS self-host story that has documented gaps. fastCRW is a single-binary Rust-native alternative with narrower scope but better self-host ergonomics and a Tavily-style search API on top. These products overlap on `/scrape`, `/crawl`, `/map`, `/search`; they diverge on LLM extraction, deep research, anti-bot depth, and deployment surface. We do NOT claim drop-in equivalence — claim is "Rust-native alternative for the overlap surface."

> **Supported compat target — Firecrawl v2 only (as of 2026-07-04):** Firecrawl drop-in compatibility is served under the dedicated **`/firecrawl/*`** namespace and targets **Firecrawl's v2 API/SDK**. `/firecrawl/v2/*` is verified end-to-end for `scrape`, `search`, `extract`, `map`, `crawl`, and `batch`. **How a Firecrawl SDK reaches it depends on the language:** the **JS/TS SDK** (`@mendable/firecrawl-js`) concatenates `apiUrl` + path, so `apiUrl: "…/firecrawl"` lands on `/firecrawl/v2/*` (verified via server access logs); the **Python SDK** (`firecrawl-py`) resolves endpoint paths against the host root via `urljoin`, dropping a `/firecrawl` suffix — so it lands on the equivalent root `/v2/*` (the same handlers). Point Python clients at the bare host. The **legacy v1 Firecrawl SDK is not a supported drop-in**: several v1 responses (`search`, `map`, crawl-status, `extract`) use fastCRW's own response shapes and don't deserialize under the strict v1 SDK models — `scrape` is the only fully v1-SDK-compatible route. fastCRW's own native API remains `/v1/*` at the root (what the public docs center on); root `/v2/*` is a deprecated alias of `/firecrawl/v2/*`, kept for backward-compat.

---

## 1. Endpoint coverage (high-level)

| Capability | Firecrawl Cloud | Firecrawl self-host (OSS) | fastCRW |
|---|---|---|---|
| `/v1/scrape` (single URL → markdown/html) | ✅ | ✅ (no Fire-engine) | ✅ |
| `/v1/crawl` (multi-page) | ✅ | ✅ | ✅ |
| `/v1/map` (URL discovery) | ✅ | ✅ | ✅ |
| `/v1/search` (web search → grounded results) | ✅ | ⚠️ (no Fire-engine; Cloud has stronger anti-bot) | ✅ (own search backend) |
| `/v1/extract` (LLM extraction) | ✅ (standalone route) | ⚠️ (requires LLM provider key + manual `.env`) | ✅ **Standalone `POST /v1/extract`** (async: returns a job id, poll `GET /v1/extract/{id}`). Accepts `urls: [...]` (multi-URL, capped by `limits.maxExtractUrls`). `/v1/scrape` with `formats: ["json"]` + `jsonSchema` also works for a single URL. Needs an LLM key (server-side or per-request). |
| `/v1/deep-research` | ✅ | ❌ (Cloud-only) | ❌ |
| `/firecrawl/v2/parse` (file upload → markdown) | ✅ (PDF/DOCX/XLSX/HTML/…) | ⚠️ (rolling out) | ✅ **PDF only** (multipart `file` + `options`; pure-Rust `pdf-inspector`, no OCR) |
| `/v1/agent` (Spark models) | ✅ | ❌ | ❌ |

**Source:** `github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md`. Capture commit hash and pin per page citing this row before publish.

### Document parsing (`parsers` + `/firecrawl/v2/parse`)

- **`parsers` on `/firecrawl/v2/scrape`** — Firecrawl-compatible. A URL returning
  `application/pdf` is auto-converted to markdown by default (no field needed,
  matches Firecrawl). Accepts both `parsers: ["pdf"]` and
  `parsers: [{ "type": "pdf", "mode": "auto"|"fast"|"ocr", "maxPages": N }]`.
  `parsers: []` disables conversion (raw bytes).
- **`mode`** — accepted for wire-compatibility. fastCRW has **no OCR**, so
  `mode: "ocr"` (and the OCR-fallback half of `auto`) degrades to text-layer
  extraction with a `pdf_ocr_unsupported` / `pdf_scanned` warning rather than an
  error. `fast` (text-only) is the native behavior.
- **`POST /firecrawl/v2/parse`** — multipart `file` + `options` (JSON string), same
  response envelope as `/firecrawl/v2/scrape`. fastCRW supports **PDF only** (the
  `pdf-inspector` engine); Firecrawl additionally parses DOCX/XLSX/RTF/ODT.
  Advertised at `GET /firecrawl/v2/capabilities` → `documents.parsers` / `fileUpload` so
  callers can detect support before sending.

---

## 2. Authentication

| | Firecrawl | fastCRW |
|---|---|---|
| Header | `Authorization: Bearer <key>` | `Authorization: Bearer <key>` (configurable; default `X-API-Key` for self-host) |
| Self-host auth bypass | Optional via env var | Optional via `[server.auth_required = false]` in `config.toml` |
| Per-key rate limits | Yes (Cloud) | Yes (per-key tier — see `crates/crw-server/src/routes/`) |

**Surface match:** Bearer style — yes. Param/header naming — divergent if user customizes.

---

## 3. Request/response shape (overlap surface — `/v1/scrape`, `/v1/crawl`, `/v1/map`, `/v1/search`)

> **Reference for full diff:** parent plan's `COMPATIBILITY.md` covers Tavily-shape comparison. For Firecrawl shape comparison: pin in next iteration when `crw-server/tests/` has the cross-vendor compat fixtures.

| Field | Firecrawl | fastCRW |
|---|---|---|
| Request `url` | string | string ✅ |
| Request `formats` | `["markdown", "html", ...]` | `["markdown", "html", …]` ✅ (`extract` / `llm-extract` are accepted aliases for `json`) |
| Request `onlyMainContent` | boolean | boolean ✅ |
| Request `waitFor` (ms) | number | number ✅ |
| Response `data.markdown` | string | string ✅ |
| Response `data.metadata` | object (title, description, language, sourceURL...) | object (similar) — **field-name divergence on a few keys; needs row-level diff** |
| Response `success` | boolean | boolean ✅ |
| Crawl `data.completed` polling | required | required ✅ |
| Error envelope | `{ success: false, error: "..." }` | similar; **divergence on error code naming** — needs row-level diff |

**Action item for next iteration of this doc:** add concrete field-by-field diff for `metadata` and error envelope before any page claims "drop-in" or "API-compatible." Today the page copy says "Rust-native alternative for the overlap surface" — defensible without exhaustive shape match.

---

## 4. Rust SDK status

| | Firecrawl | fastCRW |
|---|---|---|
| Official Rust SDK | ✅ `firecrawl` crate on crates.io | N/A (no SDK; HTTP API only — `reqwest` example in docs) |
| Self-host constructor | `Client::new_selfhosted(api_url, api_key)` per docs.firecrawl.dev/sdks/rust (verify in v1 — v2 split documented; constructor naming differs across doc versions) | N/A |
| Crate version pin (lock before any page cites) | **TODO** — capture exact crate version + verify constructor in that version + commit a CI demo before T8 spoke ships Path 1 copy. If unverifiable, T8 ships Path-2-only. | N/A |

**Plan iter-3 critical:** the `Client::new_selfhosted` signature is documented but the v1/firecrawl/v2 split means the Rust SDK lags v2 features. Before the T8 spoke claims "official Rust SDK works against self-hosted Firecrawl," a CI demo must succeed against the locked crate version. If CI infra isn't stood up (Firecrawl Docker stack: Postgres + Redis + workers), T8 collapses Path 1 to "official Rust SDK exists; CI demo TODO" pointer and ships Path 2 (fastCRW Rust-native alternative) as primary intent.

---

## 5. LLM extraction (`/extract`)

| | Firecrawl Cloud | Firecrawl self-host | fastCRW |
|---|---|---|---|
| Schema-based extraction (single URL) | ✅ via `/v1/extract` | ⚠️ (requires manual LLM key in `.env`) | ✅ via `POST /v1/extract`, or via `/v1/scrape` with `formats: ["json"]` + top-level `jsonSchema` (Firecrawl-compatible alias: `extract.schema`; LLM key configured in `[extraction.llm]` or passed per request) |
| Multi-URL `/extract` (one call → many URLs) | ✅ | ⚠️ | ✅ via `POST /v1/extract` with `urls: [...]` (capped by `limits.maxExtractUrls`, default 50) |
| Provider support | OpenAI, Anthropic, etc. | Same (manual config) | OpenAI, Anthropic, configurable via `[extraction.llm]` |
| Pricing | Per call + LLM token cost | Self-paid LLM tokens only | Self-paid LLM tokens only (self-host); cloud per-call pricing on managed plans |

**Shape note:** fastCRW's `/v1/extract` is async — it returns a job id you poll on `GET /v1/extract/{id}`, and it accepts a `urls` array. The per-request URL cap is advertised as `limits.maxExtractUrls` on `GET /v1/capabilities` (default 50). Single-URL extraction also works inline via `/v1/scrape` `formats: ["json"]` with a top-level `jsonSchema` (or the `extract.schema` alias for closer Firecrawl parity). Per-field `basis` attribution is supported: pass `basis: true` to get, for each top-level scalar field, an evidence object with its source URL, a hash of the exact text the model saw, a verbatim excerpt, a confidence, and an honest status (a field that cannot be grounded is marked `unverified`/`unsupported`, never fabricated). `perFieldAttribution` on `GET /v1/capabilities` reports whether this build ships it.

---

## 6. Deep research (`/v1/deep-research`)

| | Firecrawl Cloud | Firecrawl self-host | fastCRW |
|---|---|---|---|
| Multi-step web research | ✅ (Spark 1 backend) | ❌ (Cloud-only feature) | ❌ |

**Honest divergence:** Cloud-only Firecrawl feature. fastCRW does not match.

---

## 7. MCP server

| | Firecrawl | fastCRW |
|---|---|---|
| MCP server bundled | ✅ (`firecrawl-mcp-server`) | ✅ (built-in `crw-mcp` crate; `crw_scrape`, `crw_crawl`, `crw_check_crawl_status`, `crw_map`, `crw_search`, `crw_parse_file` tools) |

**Surface match:** both products ship MCP. Tool names differ (Firecrawl uses `firecrawl_*`, fastCRW uses `crw_*`); semantic mapping is straightforward.

**Structured output:** `crw_search` additionally emits MCP-2025-06-18 `structuredContent` shaped to whichever fastCRW `/v1/search` body served the call — `data.results` from a self-hosted engine, results directly in `data` from the hosted API — and **not** Firecrawl's `data.web` shape. It mirrors the body fastCRW clients already consume, so this is not a Firecrawl-shape parity claim. An `outputSchema` is advertised by the surfaces that produce the body themselves (embedded stdio and the engine's own HTTP `/mcp`), but not by the stdio proxy, where the body comes from a remote the MCP server does not author. The legacy text content block is retained for lenient clients.

---

## 8. Anti-bot / Fire-engine

| | Firecrawl Cloud | Firecrawl self-host | fastCRW |
|---|---|---|---|
| Fire-engine (Cloud-only proprietary) | ✅ | ❌ | ❌ |
| Browser fallback | ✅ (Playwright/Puppeteer) | ✅ (manual config) | ⚠️ (reqwest baseline; browser fallback path documented in `crates/crw-search/`) |
| Rotating IP / proxy | ✅ Cloud | ❌ (BYO proxy) | ❌ (BYO proxy) |

**Honest divergence:** Firecrawl Cloud has the best anti-bot story; self-host loses this; fastCRW is on par with Firecrawl self-host on this dimension. For high-anti-bot scenarios (cloudflare-protected, JS-heavy SPAs without API), Firecrawl Cloud is still the strongest choice.

---

## 9. Deployment surface

| | Firecrawl self-host | fastCRW |
|---|---|---|
| Stack | Docker Compose: API + workers + Postgres + Redis | Single Rust binary (or `docker compose up` with bundled search backend sidecar) |
| Memory baseline | ~1-2GB (full stack) | ~14 MB idle (binary); +search backend container if used |
| Cold start | ~5-15s (full stack warmup) | ~106ms (process start to first successful `/health`, binary) |
| Languages | TypeScript (workers), some Rust (`/parse` Apr 2026) | Rust |

**fastCRW wedge:** dramatically simpler deployment surface. Single binary vs multi-service stack. This is the primary "Rust-native, self-host friendly" claim for the T8 spoke.

---

## 10. License

| | Firecrawl | fastCRW |
|---|---|---|
| License | AGPL-3.0 | AGPL-3.0 |
| Self-host commercial use | Yes, with AGPL §13 obligations | Yes, same |

**Same license.** Counsel-reviewed §13 explainer is deferred per parent plan; pages cite neutral notice + link to license.

---

## 11. Pricing (managed cloud, for reference — re-verify within 48h)

| Plan | Firecrawl Cloud | fastCRW Cloud |
|---|---|---|
| Free | 1k credits/mo | (current pricing — see `/pricing`) |
| Hobby | $16 / 5k credits | — |
| Standard | $83 / 100k | $69/mo (per `src/lib/plans-client.ts:67`) |
| Growth | $333 / 500k | — |
| Scale | $599 / 1M | — |

**Source:** `firecrawl.dev/pricing` (verify within 48h before any page ships these numbers).

---

## 12. Recent moves (Mar–May 2026)

- **Firecrawl:** Lockdown Mode (Apr 30), Rust-based `/parse` engine (Apr 28), Spark 1 models on `/agent`, multiple status incidents Mar 21/24/31 + Apr 19-30 (`status.firecrawl.dev/incidents`), Series A $14.5M (Aug 2025).
- **fastCRW:** `/v1/search` on our own search backend (Q2 2026), bundled Docker sidecar, Tavily-cluster pages shipped 2026-05-09.

---

## What this matrix authorizes

- T8 spoke and T9 deep refresh may cite this doc by URL.
- Any "compatibility" or "drop-in" copy must reference a specific row that says ✅ in both columns.
- Divergence rows must be acknowledged in page copy with the same word ("not implemented," "not supported," etc.).
- Re-verify-within-48h rule applies to: pricing, recent moves, SDK constructor, SELF_HOST.md commit hash, and Apify announcement (separate doc).

## What this matrix does NOT authorize

- Latency claims (need own benchmark; cite parent plan Tavily benchmark or wait for Firecrawl-specific benchmark).
- Anti-bot success-rate claims (no benchmark).
- LLM extraction quality claims (the feature ships, but no benchmark backs a quality comparison).
- Drop-in API equivalence claims (request/response shape diff is incomplete in §3).