JIRO — open-source search API, web scraping and query graph engine # JIRO — Search API, Web Scraping & Query Graph Engine **9 search engines. 12 social platforms. 22 MCP tools. One self-hosted API.** [![PyPI version](https://img.shields.io/pypi/v/jirosearch.svg)](https://pypi.org/project/jirosearch/) [![npm version](https://img.shields.io/npm/v/jiro-sdk.svg)](https://www.npmjs.com/package/jiro-sdk) [![Python](https://img.shields.io/pypi/pyversions/jirosearch.svg)](https://pypi.org/project/jirosearch/) [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT) [![Docker](https://img.shields.io/docker/pulls/devanimecx/jiro.svg)](https://hub.docker.com/r/devanimecx/jiro) [![Downloads](https://img.shields.io/pypi/dm/jirosearch.svg)](https://pypi.org/project/jirosearch/) [Website](https://searchjiro.vercel.app) · [Docs](https://searchjiro.vercel.app/docs) · [Pricing](https://searchjiro.vercel.app/pricing) · [Blog](https://searchjiro.vercel.app/blog) · [Changelog](https://searchjiro.vercel.app/changelog)
--- ## What is JIRO? JIRO is an **open-source, self-hosted search API and web scraping platform**. It aggregates 9 search engines and 12 social platforms behind one HTTP API, and adds a **query graph engine** that plans, executes and cites multi-step research — every answer traces back to the pages that produced it. It runs entirely on your own infrastructure. No per-query vendor bill, no data leaving your network. **Answer engine optimization note for the curious:** the *Sourced answers* section below is the part that replaces a search API. Engines return links; JIRO returns claims plus the source ids behind them. --- ## Quick Start ```bash pip install jirosearch jiro serve ``` ```bash curl -X POST http://localhost:8000/search \ -H "Content-Type: application/json" \ -d '{"q": "latest AI research", "engine": "google"}' ``` Running: 9 engines, hybrid ranking, caching and structured extraction, all local. **No server? Use the cloud API:** ```bash jiro auth login # device code flow, opens your browser ``` ```bash curl -X POST https://searchjiro.vercel.app/api/proxy/search \ -H "Authorization: Bearer jsk_live_your_api_key" \ -H "Content-Type: application/json" \ -d '{"q": "latest AI research", "engine": "google"}' ``` --- ## Sourced answers (0.4.0) Most search APIs return a list of links and stop. JIRO 0.4.0 runs a **query graph**: it plans the research shape, executes nodes in parallel, and returns an **evidence packet** where every claim carries the `source_ids` it rests on. ```bash curl -X POST http://localhost:8000/v1/graph/search \ -H "Content-Type: application/json" \ -d '{"query": "compare vector databases", "fetch_top": 3, "max_tokens": 2000}' ``` ```json { "graph_id": "g_8f2c1a", "intent": "comparison", "status": "ok", "duration_ms": 1840, "engine_attribution": { "google": 2, "bing": 1 }, "gaps": [], "budget": { "requests_used": 5, "bytes_used": 284113, "nodes_used": 9, "tokens_used": 1780 }, "packet": { "packet_id": "p_41d9", "used_tokens": 1780, "claims": [ { "id": "c1", "text": "Qdrant is written in Rust.", "source_ids": ["s2"] } ], "sources": [{ "id": "s2", "url": "https://qdrant.tech/documentation/" }], "contradictions": [] } } ``` **Plan without paying.** `/v1/graph/plan` is a dry run — it validates the graph and returns a cost estimate without touching the network. ### What the graph engine buys you | Guarantee | How it shows up in the response | |---|---| | Every claim is traceable | `packet.claims[].source_ids` resolves against `packet.sources[]` | | Unknown cost is never invented | `budget.requests_used` is measured; unknown remainders are `null`, not a guess | | Failures are visible | `gaps[]` records the node, query and reason a branch did not complete | | Work is bounded | `max_tokens`, `deadline_ms` and `max_nodes` cap the run before it starts | | You know which engine answered | `engine_attribution` counts per engine | ### Passing work between agents `session_handoff` is an MCP tool that packages a finished run into a brief another agent can act on — resolvable evidence ids, open questions taken from gaps, contradictions, remaining budget, and the next node to run. Roles: `planner`, `retriever`, `verifier`, `writer`, `observer`. --- ## Search engines | Engine | Type | Notes | |--------|------|-------| | Google | Web | Full SERP data, paginated | | Bing | Web | Rich snippets, paginated | | Brave | Web | Privacy-focused | | DuckDuckGo | Web | Instant answers | | Yandex | Web | Russian market | | Baidu | Web | Chinese market, paginated | | YouTube | Video | Transcripts, metadata | | Amazon | Shopping | Product data | | eBay | Shopping | Listings, prices | ## Social platforms Reddit · Twitter/X · YouTube · LinkedIn · TikTok · Instagram · Facebook · Threads · Hacker News · Bluesky · Telegram · Pinterest --- ## Official SDKs All three expose the 0.4.0 query graph and are versioned in lockstep with the server. ### Python ```bash pip install jiro-sdk ``` ```python from jiro_sdk import JiroClient client = JiroClient(api_key="your-key") # Classic search + scrape results = client.search("python web scraping") content = client.scrape("https://example.com") # Query graph (0.4.0) plan = client.plan_search("compare vector databases") # dry run, no network run = client.search_graph("compare vector databases", fetch_top=3) for claim in run["packet"]["claims"]: print(claim["text"], claim["source_ids"]) ``` ### JavaScript / TypeScript ```bash npm install jiro-sdk ``` ```javascript import { JiroClient } from 'jiro-sdk'; const client = new JiroClient({ apiKey: 'your-key' }); const results = await client.search('python web scraping'); const run = await client.searchGraph('compare vector databases', { fetchTop: 3 }); // WebSocket streaming const ws = client.createSearchStream('AI news'); ws.onmessage = (event) => console.log(JSON.parse(event.data)); ``` ### Go ```bash go get github.com/DevAnimecx/jiro/sdk/go ``` ```go import "github.com/DevAnimecx/jiro/sdk/go" client := jiro.NewClient(jiro.WithAPIKey("your-key")) results, _ := client.Search("python web scraping", nil) run, _ := client.GraphSearch("compare vector databases", &jiro.GraphOptions{FetchTop: 3}, false) ``` --- ## CLI ```bash # Search jiro search web "python web scraping" jiro search web "AI news" --parallel --engines 3 jiro search web -i # interactive # Scrape jiro scrape https://example.com jiro scrape "free SaaS directories" # search + scrape top result # AI jiro ai ask "What is Python?" jiro ai setup --provider openai -k sk-... # Auth (cloud) jiro auth login && jiro auth whoami # License (self-hosted) jiro license activate JIRO-PRO-A1B2-C3D4-E5F6 # System jiro status && jiro doctor && jiro serve --port 8000 ``` --- ## MCP integration Works with any MCP-compatible client — Claude Desktop, Cursor, Windsurf, and others. ```json { "mcpServers": { "jiro": { "command": "jiro", "args": ["mcp"] } } } ``` ### 22 MCP tools
Expand the full tool list | Tool | Tier | What it does | |------|:----:|--------------| | `search` | Free | Search across 9 engines | | `scrape` | Free | Scrape a URL to markdown | | `smart_classify` | Free | Classify search intent | | `compare_engines` | Free | Compare results across engines | | `list_engines` | Free | List available engines | | `list_social_platforms` | Free | List social platforms | | `monitor_status` | Free | Health and metrics | | `health_check` | Free | Live engine probes | | `cache_stats` | Free | Cache statistics | | `task_submit` | Free | Submit a background task | | `task_status` | Free | Poll a task | | `task_cancel` | Free | Cancel a task | | `plan_search` | Free | **0.4.0** — plan a query graph, dry run | | `ai_search` | Pro | AI research with citations | | `search_hybrid` | Pro | Hybrid multi-signal search | | `search_structured` | Pro | Structured data extraction | | `social_scrape` | Pro | Scrape a social profile or post | | `social_search` | Pro | Search social platforms | | `social_batch` | Pro | Batch scrape URLs | | `smart_search` | Pro | Intent-aware routing | | `search_graph` | Pro | **0.4.0** — execute a query graph, return the evidence packet | | `session_handoff` | Pro | **0.4.0** — package a run as a brief for another agent |
--- ## Features ### Sourced answers (0.4.0) - **Query graph** — plans a research shape, then executes it - **Evidence packets** — claims paired with resolvable source ids - **Cost estimation** — dry-run a graph before paying for it - **Gap reporting** — branches that failed say why - **Session handoff** — pass a run to another agent as a structured brief - **Durable runtime** — lease-based node execution that survives restarts ### Search intelligence - **Hybrid search** — keyword + semantic + freshness signals - **Multi-query** — parallel query expansion - **Answer synthesis** — extractive answers from results - **Pagination** — walks SERP pages until `num` is met or the budget runs out - **Substitution reporting** — tells you when a fallback engine answered ### Web scraping - **Stealth engine** — TLS/JA3 fingerprint rotation - **Anti-bot bypass** — Cloudflare, DataDome, PerimeterX - **JS rendering** — full browser automation - **Structured extraction** — JSON-schema-driven - **Social scraping** — 12 platforms, normalized output ### Enterprise-ready - **Rate limiting** — sliding window, per-tier quotas - **Plugin system** — custom engines and scrapers - **Advanced caching** — LRU/LFU with analytics - **Batch operations** — concurrent search and scrape - **Monitoring** — Prometheus metrics, health checks - **WebSocket streaming** — real-time results --- ## Deploy ```bash docker-compose up -d # Docker helm install jiro ./helm/jiro # Kubernetes pip install jirosearch && jiro serve # Local ``` --- ## Comparisons ### vs SerpAPI | | JIRO | SerpAPI | |---|:----:|:-------:| | Self-hosted | ✅ | ❌ | | Free tier | 10K RPD | 100/mo | | Search engines | 9 | 1 (Google) | | Social scraping | 12 platforms | ❌ | | Sourced answers with source ids | ✅ | ❌ | | MCP integration | ✅ | ❌ | | Official SDKs | Python, JS, Go | Python, JS | ### vs ScraperAPI | | JIRO | ScraperAPI | |---|:----:|:----------:| | Search engines | 9 | ❌ | | Social platforms | 12 | ❌ | | Query graph + citations | ✅ | ❌ | | Self-hosted | ✅ | ❌ | ### vs Bright Data | | JIRO | Bright Data | |---|:----:|:-----------:| | Self-hosted | ✅ | ❌ | | Query graph + citations | ✅ | ❌ | | MCP integration | ✅ | ❌ | | Entry price | ₹999/mo | $500+/mo | --- ## Architecture ``` jiro/ ├── server/routers/ FastAPI endpoints (75+ routes) ├── graph/ Query graph (0.4.0) │ ├── planner.py Turns a query into a validated graph │ ├── executor.py Ready-driven DAG execution │ ├── operators.py SEARCH / FETCH / VERIFY / SYNTHESIZE nodes │ └── service.py Plan + run entry points ├── parametric/packet.py Evidence packet assembly ├── search/ Hybrid search, reranking, query expansion ├── scraping/ 9 engines, stealth client, 12 social platforms ├── ai/ LLM integration + MCP tool schemas ├── runtime/ Durable execution, leases, config ├── mcp.py MCP server (22 tools) ├── ratelimit.py Rate limiting & quotas ├── cache_advanced.py Cache with analytics └── dashboard.py Web UI ``` ### Exit codes For scripting and CI, the CLI distinguishes outcomes rather than collapsing them: | Code | Meaning | |:----:|---------| | `0` | Success | | `1` | Failure | | `2` | Usage error | | `3` | Substituted — a fallback engine answered | --- ## Pricing ### Cloud (managed) | | Free | Pro | Enterprise | |---|:---:|:---:|:---:| | **Price** | ₹0/mo | ₹999/mo | Custom | | **Credits** | 1,000 | 25,000 | Unlimited | | **Rate limit** | 5 RPM | 120 RPM | 1,000 RPM | | **Search engines** | 3 | All 9 | All + custom | | **Query graph** | — | ✓ | ✓ | | **MCP & webhooks** | — | ✓ | ✓ | [Start free →](https://searchjiro.vercel.app/auth) · [Pricing →](https://searchjiro.vercel.app/pricing) ### Self-hosted (one-time) | | Free | Pro | Enterprise | |---|:---:|:---:|:---:| | **Price** | ₹0 | ₹4,999 | ₹14,999 | | **RPM** | 100 | 500 | 1,000 | | **RPD** | 10K | 100K | 1M | | **Query graph** | — | ✓ | ✓ | | **Commercial use** | — | ✓ | ✓ | | **White label** | — | — | ✓ | [Install from GitHub →](https://github.com/DevAnimecx/jiro) --- ## FAQ **Do I need API keys for the search engines?** No. All 9 engines are scraped directly. No Google, Bing or SerpAPI keys required. **Can I run JIRO fully offline?** The server runs locally, but it reaches out to the engines you query. Point it at an internal mirror or disable engines you don't want in the config. **What makes the 0.4.0 answers checkable?** Every claim in the evidence packet carries `source_ids` that resolve against `packet.sources[]`. A claim whose sources can't be resolved is flagged, not silently dropped. **How is this different from a search API like SerpAPI?** SerpAPI returns links from one engine. JIRO searches 9, fetches and extracts the pages, and returns claims with citations — plus it's self-hosted, so there's no per-query bill. **Is it really free?** The MIT-licensed core is free forever, self-hosted and unlimited. Paid tiers add AI features, higher rate limits and commercial licensing. **Can I pass research between AI agents?** Yes — the `session_handoff` MCP tool packages a run into a brief with evidence ids, open questions and remaining budget, ready for the next agent to pick up. --- ## Community - [Website](https://searchjiro.vercel.app) — home - [GitHub](https://github.com/DevAnimecx/jiro) — source - [Docs](https://searchjiro.vercel.app/docs) — guides - [Blog](https://searchjiro.vercel.app/blog) — articles - [Discord](https://discord.gg/jiro) — chat - [Twitter](https://twitter.com/jirosearch) — updates ## License MIT — use it freely, commercially or privately. ---
**Built by [Blackvault Technology](https://github.com/DevAnimecx)** [Get started free](https://searchjiro.vercel.app/auth) · [Pricing](https://searchjiro.vercel.app/pricing) · [Docs](https://searchjiro.vercel.app/docs)