--- name: agentsop-query-routing version: 0.1.0 description: >- Routes heterogeneous queries to the appropriate index, tool, or answering engine before retrieval. Use when one endpoint serves multiple handlers, such as summary, vector retrieval, text-to-SQL, or tools, and query types require different paths. Covers LLM, semantic, and rule-based routers, confidence thresholds, fallbacks, and framework mappings. Do not use when one handler serves all queries or branching is fixed. when_to_use: - "a single answering endpoint fronts >=2 retrieval/answer handlers (summary index, vector index, SQL engine, tool) and queries differ in kind" - "users send a mix of lookup ('what is X'), summarize ('digest doc Y'), and compute ('how many Z') queries to one entry point" - "a monolithic VectorStoreIndex is being asked to also answer summary or aggregate queries and quality is uneven" - "designing the top-level shape of a RAG / agent system and deciding between one index vs a router over per-type engines" - "PR review of a router/classifier/conditional-edge that picks a downstream handler from the query" when_not_to_use: - "exactly one index/tool can correctly serve every query — routing is dead weight" - "the split is by tenant/permission, not by query kind — that is multi-tenant filtering, use [[agentsop-multi-tenant-rag]]" - "the branch is a fixed deterministic step (always A then B) with no query-dependent choice — that is a static edge, not a router" --- # Query-Type Routing · Enhancement Overlay > Third-person operating model for a coder agent that owns a multi-handler > answering surface. Audience is the LLM agent writing/reviewing the routing > code — not the end user. > **One sentence**: *A retriever is shaped by the query type it was built for; > a summary index, a vector index, and a text-to-SQL engine are not > interchangeable — so classify the query and route first, then retrieve.* This is an **ENHANCE overlay**. It distills the cross-framework *routing pattern* from three source skills. For the per-framework API, cross-link the base skill: `[[llamaindex]]` (`RouterQueryEngine`), `[[agentsop-dify]]` (Question Classifier node), `[[agentsop-langgraph]]` (conditional edges). --- ## 1. 何时激活 (Activation Rules) Activate when **any** of the following holds: 1. The system has **multiple indices / tools / engines** behind one entry point, and a query must be dispatched to exactly one (or a few) of them. 2. Inbound queries **differ in kind** — at least two of: *lookup* ("what does the contract say about termination"), *summarize* ("give me the gist of doc Y"), *compute/aggregate* ("how many tickets closed last week"), *compare* ("diff the 2024 vs 2025 policy"). 3. A `VectorStoreIndex` (or any single retriever) is being stretched to answer query types it was not built for, and quality is uneven across the mix. 4. The user names a routing primitive: `RouterQueryEngine`, `SelectorPromptTemplate`, `LLMSingleSelector`, Dify *Question Classifier*, LangGraph `add_conditional_edges`, "intent classifier", "text-to-SQL or RAG". 5. PR review touches a function that reads a query and returns *which handler* to call. Do **not** activate when: - One index/tool serves every query correctly — routing adds an LLM hop and a failure mode for no benefit (see §6 anti-pattern A1). - The split is by **who is asking** (tenant / ACL), not **what is asked** — that is filtering, route to `[[agentsop-multi-tenant-rag]]`. - The downstream choice is **fixed** (always retrieve then summarize) — that is a static edge / linear pipeline, not a router. --- ## 2. 核心心智模型 (Core Mental Model) Three principles. Violating any of them produces a router that misroutes silently or routes when it should not. ### Principle 1 — One retriever cannot serve all query types The index taxonomy is not cosmetic. From `[[llamaindex]]`: a `SummaryIndex` is a "small, fan-out synthesis" primitive — it reads *every* node to digest a doc; a `VectorStoreIndex` is top-k semantic lookup — it reads the *few* most similar chunks; a text-to-SQL engine answers *aggregate/compute* queries that no chunk contains the answer to. Ask a vector index to "summarize the whole document" and it returns 4 arbitrary chunks; ask a summary index "what is the late-fee clause" and it fans out over the whole corpus wastefully. **The query type names the correct primitive.** Routing is the act of recovering that name at runtime. > Operational corollary: `index.as_query_engine()` over a single > `VectorStoreIndex` answering a heterogeneous query mix is the symptom this > skill exists to fix. The fix is per-type handlers + a router on top. ### Principle 2 — Route first, retrieve second Routing is a **classification** step that runs *before* any retrieval. It reads only the query (and optionally light context) and emits a *destination*, not an answer. This ordering is what bounds latency and cost: you pay for the router once, then exactly one downstream handler, instead of fanning out to all of them and merging. LlamaIndex's `RouterQueryEngine`, Dify's Question Classifier node feeding IF/ELSE branches, and LangGraph's conditional edge over `state` are the *same shape* — a selector function `(query) -> handler_id` evaluated up front. The three frameworks differ only in *how* the selector is implemented (§7). ### Principle 3 — A router is only as good as its destinations' descriptions Every router — LLM, embedding, or keyword — picks among destinations described in words or examples. In LlamaIndex the signal is the `QueryEngineTool.description`; in Dify it is the class label + instruction; in LangGraph it is whatever the routing function reads off state plus the node names. From `[[llamaindex]]` Dilemma 3: *"invest in `QueryEngineTool.description` — it's the only signal the router/agent sees."* A misroute is, four times out of five, a **bad description**, not a bad model. Fix the description before swapping the router type. > The fourth case is genuinely ambiguous queries — for those, Principle of > Fallback (§3 Stage 4) applies: route to a safe default, never guess silently. --- ## 3. SOP 工作流 (Agentic Protocol) Five stages. Each gates the next. Stop and reconsider at the first "no". ### Stage 0 — Confirm routing is warranted Gate questions: - Are there genuinely **≥2 handlers** with different query-type fit? If no → single index, no router (anti-pattern A1). - Does real traffic actually **span query kinds**? Sample 50 real queries and label them. If >90% are one kind → build that handler well; skip the router. - Is the choice **query-dependent** (not fixed)? If fixed → static edge. If all three are yes, continue. ### Stage 1 — Enumerate the query types and their handlers Build the table *before* writing the router. One row per query kind: | Query kind | Example | Correct handler | Primitive | |---|---|---|---| | Summarize / digest | "summarize the Q3 report" | summary engine | `SummaryIndex` | | Fact lookup | "what is the late-fee clause" | vector engine | `VectorStoreIndex` + filters | | Compute / aggregate | "how many orders shipped in Q3" | text-to-SQL engine | NL2SQL over the DB | | Compare / multi-hop | "diff 2024 vs 2025 policy" | decomposition engine | `SubQuestionQueryEngine` | | Out of scope | "what's the weather" | default / refuse | fallback handler | This table is the spec for both the handlers and the router. Mapping mirrors `[[llamaindex]]` Stage 4 ("Compose for query heterogeneity"). ### Stage 2 — Build one handler per type, in isolation Implement and test each engine **independently** against its own query kind before wiring the router. A misroute is undebuggable if the handlers themselves are wrong. Author each handler's **description / label** here (Principle 3) — the destination metadata is part of the handler, not the router. ### Stage 3 — Write the router (pick the cheapest sufficient family) Three router families, cheapest to most capable: | Family | How it decides | Pick when | |---|---|---| | **Keyword / rule** | regex / substring / heuristic over the query | destinations are lexically distinct ("SELECT", "summarize", file extensions); latency-critical; cost-critical | | **Embedding / semantic** | embed query, nearest destination description | destinations semantically distinct but not lexically; no per-call LLM budget; deterministic-ish | | **LLM / selector** | LLM reads query + descriptions, returns choice (single or multi) | destinations need reasoning to disambiguate; multi-select needed; quality > latency | Default ladder: **start keyword if the types are lexically separable; else LLM selector**; reach for embedding when you want a middle point (no LLM hop, better than keyword). Always emit a **confidence / score**, never just a label. ### Stage 4 — Add the fallback default (non-negotiable) Every router must define behavior for the **unrouteable** query: - LLM selector returns low confidence / no match → route to a documented default handler (usually the broad vector index) **or** an explicit "I can't answer that" path. Never let an ambiguous query silently hit a random branch. - Set a **confidence threshold**: below it → fallback, not best-guess. - Log every routing decision `{query_hash, chosen_handler, score, fell_back}` so misroutes are observable, not anecdotal. A router with no fallback is the single most common production failure here (anti-pattern A2). --- ## 4. 操作模型 (Operation Models) Format: **Trigger / Action / Output / Evidence**. ### OP-01 EnumerateQueryTypes - **Trigger**: A single endpoint fronts heterogeneous queries; no type table exists. - **Action**: Sample ≥50 real queries, label each by kind (summarize / lookup / compute / compare / out-of-scope), map each kind to its correct handler + primitive (Stage 1 table). - **Output**: A query-type → handler spec table that drives both handler build and router design. - **Evidence**: `[[llamaindex]]` Stage 0 ("What is the query distribution?") + Stage 4 heterogeneity table. ### OP-02 BuildPerTypeHandler - **Trigger**: Type table exists; handlers not yet built. - **Action**: Implement one engine per type (`SummaryIndex` for digest, `VectorStoreIndex` for lookup, NL2SQL for compute, `SubQuestionQueryEngine` for compare). Test each against its own kind in isolation. Author its description/label. - **Output**: N independently-verified handlers, each with a destination description (Principle 3). - **Evidence**: `[[llamaindex]]` OP-06 RouteByQueryType; OP-07 DecomposeMultiHop. ### OP-03 KeywordRouter - **Trigger**: Destinations are lexically distinct; latency/cost-critical path. - **Action**: regex/substring rules over the query → handler id; explicit default for no-match. No model call. - **Output**: Sub-millisecond, deterministic, free routing for the separable cases. - **Evidence**: `[[agentsop-dify]]` IF/ELSE node over query; `[[agentsop-langgraph]]` conditional edge as a pure Python predicate (`add_conditional_edges`). ### OP-04 EmbeddingRouter - **Trigger**: Destinations semantically (not lexically) distinct; no LLM budget per call. - **Action**: Embed the query, cosine-match against pre-embedded destination descriptions / few examples per route; pick argmax; threshold for fallback. - **Output**: Cheap, low-latency semantic routing without an LLM hop. - **Evidence**: `[[llamaindex]]` `EmbeddingSingleSelector` family; router docs. ### OP-05 LLMSelectorRouter - **Trigger**: Disambiguation needs reasoning; or a query maps to multiple handlers. - **Action**: Use an LLM selector (single or multi) reading the query + each handler's description; return chosen id(s) + reasoning. In LlamaIndex: `RouterQueryEngine(selector=LLMSingleSelector.from_defaults(), query_engine_tools=[...])`. - **Output**: Highest-accuracy routing including multi-route fan-out; costs one LLM round-trip. - **Evidence**: `[[llamaindex]]` OP-06 + Dilemma 3; `SelectorPromptTemplate`. ### OP-06 ConfidenceThresholdFallback - **Trigger**: Any router that can be uncertain (LLM/embedding) on ambiguous queries. - **Action**: Read the selector's score; below threshold → route to documented default (broad vector index) or explicit refuse path. Never best-guess silently. - **Output**: Unrouteable queries land somewhere safe and auditable. - **Evidence**: `[[agentsop-langgraph]]` conditional edge can return a `"__default__"` branch; `[[agentsop-dify]]` Question Classifier has a built-in "other/else" class. ### OP-07 RouteDecisionLogging - **Trigger**: Any production router. - **Action**: Log `{ts, query_hash, chosen_handler, selector_score, fell_back}` on every decision; surface a misroute dashboard. - **Output**: Misroutes become observable metrics, not user-reported anecdotes. - **Evidence**: `[[agentsop-dify]]` 7-class trace incl. routing; `[[agentsop-langgraph]]` LangSmith trace of the conditional edge. ### OP-08 DescriptionFirstDebug - **Trigger**: Router misroutes a known query class. - **Action**: Before swapping router type, rewrite the destination description / add 1-2 disambiguating examples; re-evaluate on a labeled routing eval set. - **Output**: Most misroutes fixed at the description layer, no model change. - **Evidence**: `[[llamaindex]]` Dilemma 3 ("the only signal the router sees"). --- ## 5. 困境决策案例 (Dilemma Cases) ### Dilemma 1 — Router misroutes: improve the classifier or add a fallback? **困境**: A 3-way router (summary / lookup / SQL) misroutes ~12% of queries — some lookup queries land on the SQL engine and error out. The team is split: fine-tune / swap to a bigger LLM selector, vs. add a catch-all fallback. **约束**: - Bigger LLM selector adds latency + cost to *every* query to fix 12%. - A pure fallback masks the misroute but still sends those queries somewhere generic, lowering answer quality for the 12%. - The misroutes cluster: most are lookup queries phrased like questions the SQL engine "thinks" it can answer. **决策步骤**: 1. **Diagnose before fixing** — pull the routing log (OP-07); cluster the 12%. The clustering reveals it's a *description* problem, not a model problem (the SQL engine's description over-claims). 2. **Fix descriptions first** (OP-08) — tighten the SQL handler's description to "aggregate/count/sum over the orders table only"; add 2 negative examples. Re-run the routing eval set. Misroute drops to ~4%. 3. **Then add fallback** (OP-06) for the residual ambiguous tail — low-confidence → broad vector index, not SQL. 4. **Only if still bad** consider the bigger selector — but now on 4%, the cost/benefit usually says no. **结果**: Description fix + fallback, no model change. The two moves are complementary, not either/or: improve the classifier *signal* (descriptions) *and* add a fallback for the irreducible ambiguity. Mirrors `[[llamaindex]]` Dilemma 3's "fix the supervisor before switching paradigms" logic and `[[agentsop-langgraph]]`'s "hitting the limit means the logic is wrong" stance. **可提取的操作**: OP-08, OP-06, OP-07. ### Dilemma 2 — LLM router latency vs cheap keyword router **困境**: An LLM selector routes correctly but adds ~600ms + a token cost to every query. Traffic is high-QPS and most queries are lexically obvious ("summarize…", SQL-shaped, or a plain question). Is the LLM hop worth it? **约束**: - The LLM selector is accurate but is now the p95 latency bottleneck and a per-query cost line. - A keyword router is free and instant but brittle on the genuinely ambiguous tail. - `[[agentsop-dify]]`'s known per-node latency overhead and `[[agentsop-langgraph]]`'s "checkpoint serialisation adds overhead, latency budget <200ms" boundary both argue against an LLM hop on every request. **决策步骤**: 1. **Measure the separable fraction** — what % of queries a cheap keyword/regex router classifies with high confidence? Sample says ~80%. 2. **Tier the router**: cheap keyword/embedding router first; only the residual ~20% (no confident keyword match) escalates to the LLM selector. This is the routing analog of `[[llamaindex]]`'s "exhaust cheap knobs first" and `[[agentsop-langgraph]]`'s "promote upward only as needed" ladder. 3. **Set the keyword-confidence bar high** — a wrong cheap route is worse than a slow correct one; when in doubt, escalate. 4. **Re-measure p95 and cost**; the LLM hop now runs on 20% of traffic. **结果**: A two-tier (cascade) router — cheap router handles the obvious majority instantly, LLM selector handles the ambiguous minority. Cost and p95 drop ~5×; accuracy is preserved because the cheap tier only acts when confident. Reserve the LLM selector for where reasoning is actually required (Principle 2 + the §3 Stage 3 ladder). **可提取的操作**: OP-03, OP-04, OP-05, OP-06. ### Dilemma 3 — Route to one handler vs multi-select fan-out **困境**: Some queries legitimately need two handlers ("summarize the contract *and* tell me the late-fee clause"). A single-select router forces a wrong binary choice. **约束**: - Multi-select doubles downstream cost and needs a synthesis step to merge. - Most queries are single-intent; defaulting to multi-select wastes spend. **决策步骤**: 1. Quantify the multi-intent fraction from the routing log. If <10% → keep single-select + fallback; accept occasional follow-up. 2. If material → use a **multi-selector** (LlamaIndex `LLMMultiSelector` / Dify branching to multiple nodes / LangGraph `Send` fan-out to multiple handlers) and add a synthesis node to merge results. 3. Never make multi-select the default — it converts a routing problem into a fan-out + merge problem with the cost of both. **结果**: Single-select by default; multi-select only on the measured multi-intent slice, paired with explicit synthesis. Mirrors `[[agentsop-langgraph]]` "don't fan out with `Send` for fixed-cardinality work." **可提取的操作**: OP-05, OP-07. --- ## 6. 反模式与边界 (Anti-patterns & Boundaries) ### Top anti-patterns (red flags in code review) | # | Anti-pattern | Why it's wrong | Correct move | |---|---|---|---| | A1 | Adding a router when one index already serves every query | Pure overhead: an extra hop + a new failure mode for zero benefit | Single index; route only when ≥2 handlers differ in fit (Stage 0) | | A2 | Router with no fallback / default branch | Ambiguous or out-of-scope queries hit a random or erroring handler | Confidence threshold → documented default / refuse (OP-06) | | A3 | LLM selector on every query when keyword would do | p95 latency + per-query token cost for separable traffic | Tier: cheap router first, LLM only for the ambiguous tail (Dilemma 2) | | A4 | Vague destination descriptions | Router can't disambiguate; misroutes blamed on the model | Author precise descriptions + examples; fix here first (OP-08) | | A5 | No routing decision logging | Misroutes surface as user complaints, not metrics | Log `{query, chosen, score, fell_back}` per decision (OP-07) | | A6 | Routing by tenant/permission instead of query kind | That's access control, not query routing | Use `[[agentsop-multi-tenant-rag]]` filter at the store; route by *kind* only | | A7 | Multi-select as the default | Doubles cost + needs merge for mostly single-intent traffic | Single-select default; multi only on measured multi-intent slice (Dilemma 3) | | A8 | Best-guess on low confidence | Silent wrong route degrades the answer with no signal | Below threshold → fallback, never guess (OP-06) | | A9 | Tuning the router before the handlers work | A misroute is undebuggable atop broken handlers | Build + verify each handler in isolation first (Stage 2) | ### Boundaries — when query routing is **not** the move - **B1** One index/tool answers everything correctly → no router (A1). - **B2** The split is by *who asks* (tenant, ACL) → `[[agentsop-multi-tenant-rag]]`. - **B3** The downstream is a *fixed* sequence (always retrieve→summarize) → static edge / linear pipeline, no selector. - **B4** Hard real-time (<200ms) and an LLM selector is the only correct router → measure first; tier to a keyword/embedding front, escalate rarely (Dilemma 2). - **B5** The choice is *agentic/iterative* (retrieve→reflect→re-query in a loop) → that's an agent loop, not a one-shot router; use `[[agentsop-langgraph]]` cycles. ### PR review smells - A `RouterQueryEngine` / classifier with no `default` / `other` branch. - An LLM selector in a high-QPS hot path with no cheap pre-filter. - `QueryEngineTool(description="index")` — uselessly vague description. - Routing logic that reads `tenant_id` — that's filtering, not routing. - A router whose handlers were never tested independently. - No eval set of labeled `(query, expected_handler)` pairs gating router changes. --- ## 7. 跨框架对照 (Cross-Framework Reference) The same selector `(query) -> handler_id` surface across the three base skills. All verified against the source SKILLs (May 2026). Cross-link the base skill for the full API. ### 7.1 LlamaIndex — `RouterQueryEngine` + selectors → `[[llamaindex]]` ```python from llama_index.core.query_engine import RouterQueryEngine from llama_index.core.selectors import LLMSingleSelector # or LLMMultiSelector, # EmbeddingSingleSelector from llama_index.core.tools import QueryEngineTool tools = [ QueryEngineTool.from_defaults( query_engine=summary_engine, description="Useful for SUMMARIZING or digesting an entire document."), QueryEngineTool.from_defaults( query_engine=vector_engine, description="Useful for LOOKING UP a specific fact or clause."), ] router = RouterQueryEngine( selector=LLMSingleSelector.from_defaults(), # LLM selector family query_engine_tools=tools, # descriptions are the routing signal ) ``` The selector reads each `QueryEngineTool.description` (the only routing signal — Principle 3). Selector families: `LLMSingleSelector`, `LLMMultiSelector`, `EmbeddingSingleSelector`, `PydanticSingleSelector`. `SelectorPromptTemplate` customizes the LLM prompt. `RouterQueryEngine` over per-task indices is "often the correct top-level shape, not a single monolithic `VectorStoreIndex`" (`[[llamaindex]]` Principle 3 + OP-06). ### 7.2 Dify — Question Classifier node → `[[agentsop-dify]]` The **Question Classifier** node (an LLM node) takes the query and emits one of N declared classes; each class wires to a downstream branch (Knowledge Retrieval / LLM / Code / HTTP / SQL-via-Code). It is the visual analog of an LLM selector. The built-in "other" class is the fallback (OP-06). Routing logic that doesn't need an LLM uses the **IF/ELSE** node (keyword/rule router, OP-03). Node taxonomy: `Question Classifier`, `Parameter Extractor`, `IF/ELSE`. The classifier's class label + instruction is the routing signal (Principle 3). ### 7.3 LangGraph — conditional edges as routers → `[[agentsop-langgraph]]` ```python def route(state) -> str: # the selector function q = state["query"] if looks_like_sql(q): return "sql" # keyword tier (OP-03) if score := classify(q): return score.label # LLM/embedding tier (OP-04/05) return "vector_default" # fallback (OP-06) graph.add_conditional_edges("router", route, {"sql": "sql_node", "summary": "summary_node", "vector_default": "vector_node"}) ``` "Graph topology is just routing logic over state… a conditional edge reads state and picks a next node" (`[[agentsop-langgraph]]` Principle 3). The mapping dict's keys are the destinations; the `route` function is the selector — it can be keyword, embedding, or LLM, or a tier of all three (Dilemma 2). For genuine multi-route fan-out, return a list of `Send(...)` instead of one label. ### 7.4 Side-by-side | | LlamaIndex | Dify | LangGraph | |---|---|---|---| | Router primitive | `RouterQueryEngine` | Question Classifier node | `add_conditional_edges` | | Selector impl | LLM / Embedding / Pydantic selector | LLM classifier (or IF/ELSE for rules) | any Python fn (keyword/embed/LLM) | | Routing signal | `QueryEngineTool.description` | class label + instruction | node names + what `route()` reads | | Fallback | selector default / catch-all tool | "other" class | a `"__default__"` branch | | Multi-route | `LLMMultiSelector` | branch to multiple nodes | list of `Send(...)` | | Deep skill | `[[llamaindex]]` | `[[agentsop-dify]]` | `[[agentsop-langgraph]]` | The three are the **same pattern** — classify the query up front, dispatch to the structurally-correct handler, fall back when uncertain. Pick the framework your stack already uses; the routing discipline (Principles 1-3, Stages 0-4) is identical. --- ## References - `references/R1-source-evidence.md` — every cited claim resolved to its source SKILL line. - `intermediate/operation_candidates.json` — machine-readable operation list. ### Source skills (cited inline as `[[name]]`) - `[[llamaindex]]` — `llamaindex-sop-skill/SKILL.md` (`RouterQueryEngine`, selectors, OP-06 RouteByQueryType, Dilemma 3, Index taxonomy). - `[[agentsop-dify]]` — `dify-sop-skill/SKILL.md` (Question Classifier / IF-ELSE nodes, node taxonomy, per-node latency boundary). - `[[agentsop-langgraph]]` — `langgraph-sop-skill/SKILL.md` (conditional edges as routers, Principle 3 topology-as-routing, `Send` fan-out, latency boundary).