[](https://pypi.org/project/shadow-web/) [](https://pypi.org/project/shadow-web/) [](https://github.com/ulinycoin/shadow-web/actions/workflows/test.yml) [](https://pypi.org/project/shadow-web/) [](LICENSE) [](https://github.com/ulinycoin/shadow-web) [](https://lobehub.com/mcp/ulinycoin-shadow-web) # Shadow Web **Compress web pages 4–6× before your LLM sees them — then extract structured data.** Open-source Python SDK + MCP + standalone `shadow-web exec` agent: flattens Shadow DOM, builds a typed Action Map, indexes rendered text on demand, heals broken selectors, and turns tables/forms/lists into clean JSON — Camoufox by default, no cloud required. > **v0.4.4** — the extraction core moved out of the tree: tables/forms/lists are parsed by the standalone [`schemasnap`](https://pypi.org/project/schemasnap/) package (installed automatically by `pip install shadow-web`), and `shadow_web.schema_snap` is a 64-line facade over it — same imports, one implementation. Running this suite against the shared core surfaced four correctness bugs in it (phantom empty tables, a swallowed ARIA header row, dropped single-column grids, zero-padded ids rewritten as `integer`) — all fixed, all in 0.1.9. 0.4.3 was the verified-review pass — the scanner finally reaches `script`/`link`/`integrity` (`raw_html`), field classification no longer routes profile values into the wrong field, MCP sessions cannot recycle keys, and an `agent_run` timeout actually stops the work. ```python from shadow_web.compressor import process_html clean_html, actions, groups = process_html(raw_html) # ✅ actions = [{"id":"1","type":"button","label":"Buy Now","group":"Checkout"}, ...] # ✅ 164 → 46 tokens on a typical page from shadow_web.schema_snap import parse_page data = parse_page(clean_html) # ✅ tables: [{columns, types, rows, total_rows}] # ✅ forms: [{action, method, fields: [{name, type, required, label}]}] # ✅ lists: [{type, items, total}] ``` --- ## Pain point AI agents need to see web pages. But raw HTML is full of `