r"""Multilingual zero-shot classification benchmark (no English).

9 languages x 6 texts (2 positive, 2 negative, 2 neutral sentiment), same
sentence meanings across languages, labels in English (standard zero-shot
cross-lingual setup):

  popular : Spanish, French, Chinese (Simplified)
  medium  : Vietnamese, Turkish, Ukrainian
  rare    : Sinhala, Icelandic, Welsh  (Sinhala required by the owner)

Every system in the comparison answers the same 54 texts. Run one system
per invocation (results merge into the shared file):

    python bench_multilingual.py --system GLiNER2.5-base
    .venv-von/Scripts/python bench_multilingual.py --system von
    .venv-von/Scripts/python bench_multilingual.py --system JevK5-Lite
    .venv-von/Scripts/python bench_multilingual.py --system "MoJev 0.85B"
    .venv-von/Scripts/python bench_multilingual.py --system "Lumma-fev 0.15B"
    .venv-von/Scripts/python bench_multilingual.py --system "Lumma-fev 0.6B"
    .venv-von/Scripts/python bench_multilingual.py --system "Lumma-fev 4B"
    .venv-von/Scripts/python bench_multilingual.py --system "Intern-Decision 0.8B"
    .venv-von/Scripts/python bench_multilingual.py --system "Intern-Decision 2B"
    .venv-von/Scripts/python bench_multilingual.py --system "Intern-Decision 4B"
    .venv-von/Scripts/python bench_multilingual.py --system "K2-Type 0.9B (local)"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 2.0 Kai 0.6B"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 2.0 Eos 0.8B"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 2.0 Sol 2B"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 3.0 Edge 0.6B"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 3.0 Lite 0.85B"
    .venv-von/Scripts/python bench_multilingual.py --system "Decision 3.0 Nano 2B"

Jev is a paid API: it runs all 54 texts as one batched request.
Kev 0.8B needs its local server running first (System One contract):
    cd ../kev && uv run --extra serve python -m kev.serve \
        --run jaredpalmer/kev-0.8b --port 8009
AgentJev 0.6B likewise (own /api/evaluate contract):
    cd ../agent-jev && <python> -m jev_service.server \
        --checkpoint agentjev_v1.pt --model-path <Qwen3-0.6B snapshot> \
        --temperatures temperatures.json --port 8149 --device cpu
decider 0.8B and OpenThai 0.8B serve the System One contract as well
(shared agent-jev venv): DECIDER_MODEL=Mapika/decider-0.8b DECIDER_DEVICE=cpu
<py> -m uvicorn decider.serve:app --port 8018, respectively
OPENTHAI_SYSTEMONE_MODEL=iapp/OpenThai-SystemOne <py> -m uvicorn
openthai_systemone.server:app --port 8029.

OpenRouter-hosted systems need OPENROUTER_API_KEY in .env; their providers
are not ZDR - they may retain request data (Solar Decide additionally has
a ZDR Upstage endpoint, but the runs here use default routing):
    python bench_multilingual.py --system "Kev 4B (OpenRouter)"
    python bench_multilingual.py --system "Solar Decide (OpenRouter)"
    python bench_multilingual.py --system "Decider V1.1 27B (OpenRouter)"
    # Decisions family: also "Decider V1 27B (OpenRouter)",
    # "D1 (OpenRouter)", "Tev1 4B (OpenRouter)",
    # "Mercury Decide (OpenRouter)" (free tier, 20 req/min)
    python bench_multilingual.py --system "Span-01 Lite"
    python bench_multilingual.py --system "cohere-rerank-v3.5 (OpenRouter)"

Cloudflare's Clef models are hosted on Workers AI and need a one-time
`cf auth login` (the CLI's session token is used and auto-refreshed;
see engines/clef_client.py) or a Workers AI API token in .env as
CLOUDFLARE_AUTH_TOKEN plus CLOUDFLARE_ACCOUNT_ID:
    python bench_multilingual.py --system "Clef (Workers AI)"
    python bench_multilingual.py --system "Clef-flash (Workers AI)"

Fastino's hosted systems need FASTINO_API_KEY in .env (all ZDR per the
model catalog): GLiDE rides Fastino's /v1/systemone (input $0.15/M
tokens, thinking tokens free) and the hosted GLiNER twins the local
GLiNER rows line up against ride the chat-completions endpoint (input
$0.03/M tokens, output free):
    python bench_multilingual.py --system "GLiDE (Fastino)"
    python bench_multilingual.py --system "GLiNER2.5-base (Fastino)"
    python bench_multilingual.py --system "GLiNER2.5-multi (Fastino)"
    python bench_multilingual.py --system "GLiNER2.5-Decide (Fastino)"

TypeLLM's hosted type-safe generation API needs TYPELLM_API_KEY in .env
(input $0.05/M, thinking $0.50/M, answers free); both modes run, the
thinking one bills reasoning tokens per question:
    python bench_multilingual.py --system "TypeLLM (hosted)"
    python bench_multilingual.py --system "TypeLLM thinking (hosted)"

Ollaya systems need the local Ollaya daemon (System One contract on :11435;
`ollaya serve` after installing from https://ollaya.dev/download):
    python bench_multilingual.py --system "nli deberta-v3-large (Ollaya)"
    python bench_multilingual.py --system "winnow e4b (Ollaya)"

Ground-truth verification: sentences were blind-translated back to English
by one independent cold-context model instance (separate agent, no labels
in its instructions); it caught 8 errors (2 sentiment-flipping) which were
fixed and re-verified. The full sentence table is in the PR description
for human (native-speaker) review, Sinhala especially.

Output: results/bench_multilingual_results.json (rendered by the web UI's
Multilingual benchmark tab).
"""

from __future__ import annotations

import argparse
import json
import os
import statistics
import sys
import time

SENTIMENT_LABELS = {
    "positive": "Text expresses a clearly positive attitude",
    "negative": "Text expresses a clearly negative attitude",
    "neutral": "Factual text without a clear attitude",
}

# English templates (for the PR table only; not benchmarked):
# P1 "The food at this restaurant is delicious, and the staff are very friendly."
# P2 "I am very happy with my new phone; the battery lasts all day."
# N1 "The bus was crowded and slow, and I arrived very late."
# N2 "This movie was long and boring; I regret watching it."
# T1 "The train to the capital leaves at seven in the morning."
# T2 "This book was published in 2019 and has 300 pages."

LANGUAGES: dict[str, list[tuple[str, str]]] = {
    # ---- popular -----------------------------------------------------
    "Spanish": [
        ("La comida de este restaurante está deliciosa y el personal es muy amable.", "positive"),
        ("Estoy muy contento con mi teléfono nuevo; la batería dura todo el día.", "positive"),
        ("El autobús estaba lleno y lento, y llegué muy tarde.", "negative"),
        ("Esta película fue larga y aburrida; me arrepiento de haberla visto.", "negative"),
        ("El tren a la capital sale a las siete de la mañana.", "neutral"),
        ("Este libro fue publicado en 2019 y tiene 300 páginas.", "neutral"),
    ],
    "French": [
        ("La nourriture de ce restaurant est délicieuse et le personnel est très aimable.", "positive"),
        ("Je suis très content de mon nouveau téléphone ; la batterie tient toute la journée.", "positive"),
        ("Le bus était bondé et lent, et je suis arrivé très en retard.", "negative"),
        ("Ce film était long et ennuyeux ; je regrette de l'avoir vu.", "negative"),
        ("Le train pour la capitale part à sept heures du matin.", "neutral"),
        ("Ce livre a été publié en 2019 et compte 300 pages.", "neutral"),
    ],
    "Chinese": [
        ("这家餐厅的菜品很美味，员工也非常友好。", "positive"),
        ("我对新手机非常满意；电池能用一整天。", "positive"),
        ("公交车又挤又慢，我到得很晚。", "negative"),
        ("这部电影又长又无聊；我后悔看了它。", "negative"),
        ("开往首都的火车早上七点发车。", "neutral"),
        ("这本书于2019年出版，共300页。", "neutral"),
    ],
    # ---- medium ------------------------------------------------------
    "Vietnamese": [
        ("Món ăn ở nhà hàng này rất ngon và nhân viên rất thân thiện.", "positive"),
        ("Tôi rất hài lòng với chiếc điện thoại mới; pin dùng được cả ngày.", "positive"),
        ("Xe buýt vừa đông vừa chậm, và tôi đến rất muộn.", "negative"),
        ("Bộ phim này dài và nhàm chán; tôi hối tiếc vì đã xem.", "negative"),
        ("Tàu đi thủ đô khởi hành lúc bảy giờ sáng.", "neutral"),
        ("Cuốn sách này xuất bản năm 2019 và có 300 trang.", "neutral"),
    ],
    "Turkish": [
        ("Bu restoranın yemeği çok lezzetli ve personel çok güler yüzlü.", "positive"),
        ("Yeni telefonumdan çok memnunum; pil bütün gün dayanıyor.", "positive"),
        ("Otobüs kalabalık ve yavaştı, çok geç kaldım.", "negative"),
        ("Bu film uzundu ve sıkıcıydı; izlediğime pişman oldum.", "negative"),
        ("Başkente giden tren sabah yedide kalkıyor.", "neutral"),
        ("Bu kitap 2019'da yayımlandı ve 300 sayfası var.", "neutral"),
    ],
    "Ukrainian": [
        ("Їжа в цьому ресторані смачна, а персонал дуже привітний.", "positive"),
        ("Я дуже задоволений своїм новим телефоном; батареї вистачає на весь день.", "positive"),
        ("Автобус був переповнений і повільний, і я дуже запізнився.", "negative"),
        ("Цей фільм був довгим і нудним; я шкодую, що подивився його.", "negative"),
        ("Потяг до столиці відправляється о сьомій ранку.", "neutral"),
        ("Цю книгу опубліковано у 2019 році, і вона має 300 сторінок.", "neutral"),
    ],
    # ---- rare --------------------------------------------------------
    "Sinhala": [
        ("මෙම අවන්හලේ ආහාර රසවත් ය, සේවකයෝ ද ඉතා හිතවත් ය.", "positive"),
        ("මගේ අලුත් දුරකථනය ගැන මම ඉතා සතුටු ය; බැටරිය දිනය පුරා පවතී.", "positive"),
        ("බස් රථය පිරී සිටි අතර සෙමින් ගමන් කළේ ය; මම ඉතා ප්‍රමාද වීමි.", "negative"),
        ("මෙම චිත්‍රපටය දිගු හා කලකිරීම්බර ය; එය නැරඹූ බවට මම පසුතැවෙමි.", "negative"),
        ("අගනුවරට යන දුම්රිය උදේ හතට පිටත් වේ.", "neutral"),
        ("මෙම පොත 2019 දී ප්‍රකාශයට පත් වූ අතර පිටු 300 ක් ඇත.", "neutral"),
    ],
    "Icelandic": [
        ("Maturinn á þessum veitingastað er ljúffengur og starfsfólkið er mjög vinalegt.", "positive"),
        ("Ég er ánægður með nýja símann minn; rafhlöðan endist allan daginn.", "positive"),
        ("Strætóinn var fullur og hægur, og ég kom mjög seint.", "negative"),
        ("Þessi kvikmynd var löng og leiðinleg; ég iðrast þess að hafa horft á hana.", "negative"),
        ("Lestin til höfuðborgarinnar fer af stað klukkan sjö að morgni.", "neutral"),
        ("Þessi bók var gefin út árið 2019 og er 300 síðna.", "neutral"),
    ],
    "Welsh": [
        ("Mae bwyd y bwyty hwn yn flasus iawn a'r staff yn garedig iawn.", "positive"),
        ("Rwy'n falch iawn o'm ffôn newydd; mae'r batri'n para'r dydd cyfan.", "positive"),
        ("Roedd y bws yn llawn ac yn araf, a chyrhaeddais i'n hwyr iawn.", "negative"),
        ("Roedd y ffilm hon yn hir ac yn ddiflas; mae gennyf edifeirdrwch am ei gweld.", "negative"),
        ("Mae'r trên i'r brifddinas yn gadael am saith y bore.", "neutral"),
        ("Cyhoeddwyd y llyfr hwn yn 2019 ac mae ganddo 300 o dudalennau.", "neutral"),
    ],
}

TIERS = {
    "Spanish": "popular", "French": "popular", "Chinese": "popular",
    "Vietnamese": "medium", "Turkish": "medium", "Ukrainian": "medium",
    "Sinhala": "rare", "Icelandic": "rare", "Welsh": "rare",
}

RESULTS_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
                            "results", "bench_multilingual_results.json")

GLINER = {
    "GLiNER2.5-small": "fastino/gliner2.5-small-v1",
    "GLiNER2.5-base": "fastino/gliner2.5-base-v1",
    "GLiNER2.5-multi": "fastino/gliner2.5-multi-v1",
    "GLiNER2.5-Decide": "fastino/GLiNER2.5-Decide",
}
GLIFORMER = {
    "GLiFormer-base": "knowledgator/gliformer-base-v1",
    "GLiFormer-large": "knowledgator/gliformer-large-v1",
}
GLICLASS = {
    "gliclass-edge": "knowledgator/gliclass-edge-v3.0",
    "gliclass-modern-base": "knowledgator/gliclass-modern-base-v3.0",
    "gliclass-base": "knowledgator/gliclass-base-v3.0",
    "gliclass-large": "knowledgator/gliclass-large-v3.0",
}
RERANKERS = {
    "mxbai-rerank-base-v2": "mixedbread-ai/mxbai-rerank-base-v2",
    "mxbai-rerank-large-v2": "mixedbread-ai/mxbai-rerank-large-v2",
    "bge-reranker-v2-m3": "BAAI/bge-reranker-v2-m3",
    "GTE-rerank-ModernBERT-base": "Alibaba-NLP/gte-reranker-modernbert-base",
}
# NVIDIA's llama-nemotron-rerank-1b-v2 (openmdw-1.1 + Llama 3.2 Community
# License): remote-code bidirectional-attention Llama-3.2-1B cross-encoder
# with a trained "question:{q} \n \n passage:{p}" pair format - loads in
# the MAIN venv under the pinned transformers 4.57.6 (the remote code is
# version-adaptive, >=4.44 incl. 5.x). Its 26-card languages include this
# bench's rare tier (Sinhala is NOT one of them; Welsh/Icelandic aren't
# either - the card evals Thai, Hebrew, Bengali). Same measured request
# shape as bench_spectrum.py (raw text as query, the house label
# descriptions as passage; the house bare-label shape collapses it - see
# the NEMOTRON_RERANKERS notes there for the probe)
NEMOTRON_RERANKERS = {
    "nemotron-rerank-1b-v2": "nvidia/llama-nemotron-rerank-1b-v2",
}


def nemotron_prompt(query: str, passage: str) -> str:
    """The card's trained pair format, verbatim (same helper as
    bench_spectrum.py; the benches duplicate their shape helpers)."""
    return f"question:{query} \n \n passage:{passage}"
# hosted on OpenRouter (see bench_spectrum.py for the full id map and
# the Decisions-family notes); same names so results files line up
# across benchmarks
OPENROUTER_SYSTEMONE = {
    "Kev 4B (OpenRouter)": "jaredpalmer/kev-4b",
    "Solar Decide (OpenRouter)": "upstage/solar-decide",
    "Solar Decide Flash (OpenRouter)": "upstage/solar-decide-flash",
    "Span-01": "respan/span-01",
    "Span-01 Lite": "respan/span-01-lite",
    "Decider V1 27B (OpenRouter)": "perplexity/pplx-decider-v1-27b",
    "Decider V1.1 27B (OpenRouter)": "perplexity/pplx-decider-v1.1-27b",
    "D1 (OpenRouter)": "liquid/d1",
    "Tev1 4B (OpenRouter)": "togethercomputer/tev1-4b-experimental",
    "Mercury Decide (OpenRouter)": "inception/mercury-decide:free",
    "Microsoft Decision-1 (OpenRouter)": "microsoft/microsoft-decision-1",
    "Drex v1.5 (OpenRouter)": "nace-ai/drex-v1.5",
}
# the choice-question subset (the Span scorers answer noul only - their
# make_one() branch claims them first)
OR_SYSTEMONE_CHOICE = {name for name in OPENROUTER_SYSTEMONE
                       if not name.startswith("Span-01")}
# Cloudflare's Clef decision models on Workers AI (see bench_spectrum.py
# for the notes; engines/clef_client.py); same names across benchmarks so
# results files line up
CLEF_MODELS = {
    "Clef (Workers AI)": "clef",
    "Clef-flash (Workers AI)": "clef-flash",
    "Clef-omni (Workers AI)": "clef-omni",
}
# Fastino's hosted GLiDE decision model (see bench_spectrum.py for the
# notes; engines/fastino_client.py); same names across benchmarks so
# results files line up
FASTINO_MODELS = {"GLiDE (Fastino)": "glide"}
# TypeLLM's hosted type-safe generation API (see bench_spectrum.py for
# the notes; engines/typellm_client.py); single public model, so the
# value selects the variant: thinking off, or per-question reasoning.
# Same names across benchmarks so results files line up
TYPELLM_MODELS = {"TypeLLM (hosted)": False,
                  "TypeLLM thinking (hosted)": True}
# TypeSafe's Jev, the row pinned to the versioned jev-1.14.0 (jev-latest
# moves with TypeSafe's releases; see bench_spectrum.py's JEV_MODELS
# note for the 2026-10-17 story: gate miss 10/12, full-suite parity
# override — 54/54 here at ~2x 1.13's batch speed). TRIALS.md
JEV_MODELS = {"Jev": "jev-1.14.0"}
# Fastino's hosted GLiNER twins (see bench_spectrum.py for the notes;
# engines/fastino_client.py): same names as the local GLINER entries'
# hosted-available checkpoints, one API request per text like a local
# one-call-per-text run. No small twin - Fastino does not host it
FASTINO_EXTRACTORS = {
    "GLiNER2.5-base (Fastino)": "gliner2.5-base",
    "GLiNER2.5-multi (Fastino)": "gliner2.5-multi",
    "GLiNER2.5-Decide (Fastino)": "decide",
}
OPENROUTER_RERANKERS = {
    "qwen3-reranker-8b (OpenRouter)": "qwen/qwen3-reranker-8b",
    "voyage-rerank-2.5-lite (OpenRouter)": "voyageai/rerank-2.5-lite",
    "voyage-rerank-2.5 (OpenRouter)": "voyageai/rerank-2.5",
    "nemotron-rerank-vl-1b (OpenRouter)":
        "nvidia/llama-nemotron-rerank-vl-1b-v2:free",
    "cohere-rerank-4-pro (OpenRouter)": "cohere/rerank-4-pro",
    "cohere-rerank-4-fast (OpenRouter)": "cohere/rerank-4-fast",
    "cohere-rerank-v3.5 (OpenRouter)": "cohere/rerank-v3.5",
}
# served by the local Ollaya daemon (engines/ollaya_client.py holds the
# name -> tag map and the client)
from engines.ollaya_client import MODELS as OLLAYA_MODELS

from engines import (d1_client, d3_client, decision2_client, glinerx_client,
                     intern_decision_client, julia_client, k2type_client)

OLLAYA = OLLAYA_MODELS
# Knowledgator's GLiNER-X multilingual family (mT5 encoder backbone; see
# bench_spectrum.py for the notes; engines/glinerx_client.py holds the
# registry and the request-shape findings — NER-only checkpoints, the
# classification answers ride the vendor multitask prompt as census
# rows). Runs under .venv-glinerx/Scripts/python; same names across
# benchmarks so results files line up
GLINER_X = glinerx_client.MODELS
# FrontiersMind's Lumma-Fev typed-decision family, in-process via the
# lumma-fev package (.venv-von: transformers >=5.4,<6); same names as
# bench_spectrum.py so results files line up across benchmarks
LUMMA = {
    "Lumma-fev 0.15B": "FrontiersMind/Lumma-fev-0.1b",
    "Lumma-fev 0.6B": "FrontiersMind/Lumma-fev-0.6b",
    "Lumma-fev 4B": "FrontiersMind/Lumma-fev-4b",
}
# SupersonicLabs' Julia 1 typed-decision model, in-process via the `julia`
# package shipped inside its HF repo (engines/julia_client.py loads the
# local snapshot clone, JULIA_HOME, default ../Julia-1); same names as
# bench_spectrum.py so results files line up across benchmarks
JULIA = {"Julia 1 144M": julia_client.MODEL_ID}
# internlm's Intern-Decision typed-decision family, in-process via the
# runtime shipped inside each HF snapshot (engines/intern_decision_client.py
# loads the local C:\src\Intern-Decision-* snapshots); same names as
# bench_spectrum.py so results files line up across benchmarks
INTERN_DECISION = {"Intern-Decision 0.8B": "0.8B",
                   "Intern-Decision 2B": "2B",
                   "Intern-Decision 4B": "4B"}
# IFM's K2-Type-0.9B typed-decision model, in-process via the runtime
# shipped inside its HF snapshot (engines/k2type_client.py loads the local
# C:\src\K2-Type-0.9B snapshot; see bench_spectrum.py for the notes); same
# names across benchmarks so results files line up
K2TYPE = {"K2-Type 0.9B (local)": k2type_client.MODEL_ID}
# vLLM Semantic Router's Decision 2.0 typed-decision family (the line
# behind the Ollaya-served `decision` model), in-process via the
# self-verifying runtime shipped inside each HF snapshot
# (engines/decision2_client.py loads the local C:\src\Decision-2.0-*
# snapshots; see bench_spectrum.py for the notes); same names across
# benchmarks so results files line up
DECISION2 = {"Decision 2.0 Kai 0.6B": "kai-0.6b",
             "Decision 2.0 Eos 0.8B": "eos-0.8b",
             "Decision 2.0 Sol 2B": "sol-2b"}
# vLLM Semantic Router's Decision 3.0 family (Decision 2.0's successor,
# "Towards Open Multimodal Foundation Decision Models"), in-process via
# the self-verifying runtime shipped inside each HF snapshot
# (engines/d3_client.py loads the local C:\src\d3-* snapshots; see
# bench_spectrum.py for the notes); same names across benchmarks so
# results files line up
D3 = {"Decision 3.0 Edge 0.6B": "edge",
      "Decision 3.0 Lite 0.85B": "lite",
      "Decision 3.0 Nano 2B": "nano"}
# LiquidAI's open d1 (engines/d1_client.py loads the local C:\src\d1-3B /
# d1-omni-600M snapshots; see bench_spectrum.py for the notes — d1-3B
# lists 17 languages, d1-omni-600M 16); same names across benchmarks so
# results files line up
D1_LOCAL = {"D1 3B (local)": "d1-3b",
            "D1-omni 600M (local)": "d1-omni-600m"}
# CULLED holds the systems below the 12/12 trial bar or at/below the
# culled level — LFM2.5-RLCD and the nanodiff pair are gone from the
# lists entirely; the nine below are filtered out at the end. TRIALS.md
# carries their recorded data, and the dispatch branches stay, so
# removing a name from CULLED re-benches it.
CULLED = {"Certo 421M", "Verdict 151M (local)", "Julia 1 144M",
          "GLiNER-X-small", "GLiNER-X-base", "GLiNER-X-large",
          "nemotron-rerank-vl-1b (OpenRouter)", "so1 (Qwen2.5-0.5B)",
          "gliclass-edge"}
ALL_SYSTEMS = [n for n in (
    list(GLINER) + list(FASTINO_EXTRACTORS)
    + list(GLIFORMER) + list(GLICLASS) + list(GLINER_X)
    + list(RERANKERS) + list(NEMOTRON_RERANKERS)
    + list(OPENROUTER_SYSTEMONE)
    + list(OPENROUTER_RERANKERS) + list(CLEF_MODELS)
    + list(FASTINO_MODELS) + list(TYPELLM_MODELS) + list(JEV_MODELS)
    + ["Certo 421M", "MoJev 0.85B", "Laya Router",
       "Laya typed-decisions", "von", "JevK5-Lite",
       "so1 (Qwen2.5-0.5B)",
       "Kev 0.8B (local)", "AgentJev 0.6B (local)",
       "decider 0.8B (local)", "OpenThai 0.8B (local)",
       "Verdict 151M (local)", "K2-Type 0.9B (local)"]
    + list(OLLAYA) + list(LUMMA) + list(JULIA)
    + list(INTERN_DECISION) + list(DECISION2) + list(D3) + list(D1_LOCAL))
    if n not in CULLED]


def make_classifier(name: str, tracker=None):
    """Per-text sentiment classifier for encoder/classifier systems."""
    labels = list(SENTIMENT_LABELS)

    if name in GLINER:
        from gliner2 import AutoExtractor

        model = AutoExtractor.from_pretrained(GLINER[name],
                                              map_location="cpu")

        def one(text: str):
            return model.classify_text(text, {"task": labels})["task"]
    elif name in FASTINO_EXTRACTORS:
        # Fastino's hosted GLiNER twin: same classify_text surface over
        # the chat-completions endpoint (one API request per text, the
        # same decomposition as the local one-call-per-text run)
        from engines.fastino_client import gliner

        client = gliner(FASTINO_EXTRACTORS[name],
                        usage_sink=tracker.add if tracker else None)

        def one(text: str):
            return client.classify_text(text, {"task": labels})["task"]
    elif name in GLIFORMER:
        from gliformer import GLiFormer

        model = GLiFormer.from_pretrained(GLIFORMER[name],
                                          load_tokenizer=True).to("cpu").eval()

        def one(text: str):
            out = model.classify(text, labels, threshold=0.5)
            return out[0]["class_name"] if out else None
    elif name in GLINER_X:
        # Knowledgator's GLiNER-X (classic gliner package over an mT5
        # encoder, stanza word splitter; runs under .venv-glinerx) — see
        # bench_spectrum.py for the notes. NER-only checkpoints, so the
        # answers ride the vendor multitask prompt and land as census
        # rows (engines/glinerx_client.py). The word splitter lazily
        # builds a stanza tokenize pipeline per detected language, so an
        # untimed splitter pass over one text per bench language pays
        # every lazy pipeline load (and any first-use stanza download)
        # before the timed loop, like the other local engines' warm-up.
        from engines.glinerx_client import (CLASSIFICATION_THRESHOLD,
                                            classification_prompt,
                                            load_glinerx,
                                            reduce_classification,
                                            warm_splitter)

        model = load_glinerx(GLINER_X[name])
        warm_splitter(model, [items[0][0] for items in LANGUAGES.values()])

        def one(text: str):
            entities = model.predict_entities(
                classification_prompt(labels, text), labels,
                threshold=CLASSIFICATION_THRESHOLD)
            return reduce_classification(entities, labels)
    elif name == "Certo 421M":
        # calibrated non-generative decision model (vendored engines/certo_engine/):
        # score each label description against the state in one forward pass
        from huggingface_hub import snapshot_download

        from engines.certo_engine import DecisionModel

        model = DecisionModel.load(
            snapshot_download("altslate/certo-decision-model"), device="cpu")
        options = [{"id": label, "description": desc}
                   for label, desc in SENTIMENT_LABELS.items()]

        def one(text: str):
            return model.decide(text, options)["top"]
    elif name in RERANKERS:
        # cross-encoder reranker as decision engine: score one
        # (instruction, label) pair per label, pick the highest-scoring
        # label (same pair shape as bench_spectrum.py)
        from sentence_transformers import CrossEncoder

        model = CrossEncoder(RERANKERS[name], device="cpu")

        def one(text: str):
            pairs = [('What is the overall sentiment of this text: '
                      f'"{text}"', label) for label in labels]
            scores = model.predict(pairs)
            return labels[max(range(len(scores)), key=lambda i: scores[i])]
    elif name in NEMOTRON_RERANKERS:
        # NVIDIA's remote-code bidirectional cross-encoder: card-verbatim
        # load (trust_remote_code, left padding, pad token falling back to
        # eos) + the measured (text, described-label) shape from
        # bench_spectrum.py - the house (instruction, bare-label) pairs
        # the other rerankers score collapse it. One batched forward per
        # text, argmax = decision; fp32 on CPU (the card's bf16 example
        # targets GPU)
        import torch
        from transformers import (AutoModelForSequenceClassification,
                                  AutoTokenizer)

        repo = NEMOTRON_RERANKERS[name]
        tokenizer = AutoTokenizer.from_pretrained(
            repo, trust_remote_code=True, padding_side="left")
        if tokenizer.pad_token is None:
            tokenizer.pad_token = tokenizer.eos_token
        model = AutoModelForSequenceClassification.from_pretrained(
            repo, trust_remote_code=True).eval()
        if model.config.pad_token_id is None:
            model.config.pad_token_id = tokenizer.eos_token_id
        descriptions = dict(SENTIMENT_LABELS)

        def one(text: str):
            prompts = [nemotron_prompt(text, descriptions[label])
                       for label in labels]
            batch = tokenizer(prompts, padding=True, truncation=True,
                              return_tensors="pt", max_length=512)
            with torch.inference_mode():
                logits = model(**batch).logits
            scores = logits.view(-1).tolist()
            return labels[max(range(len(scores)), key=lambda i: scores[i])]
    elif name in OPENROUTER_RERANKERS:
        # OpenRouter-hosted rerankers: same (instruction, label) pair
        # mapping, one API request per text. Non-ZDR endpoints.
        from engines.openrouter_client import rerank_classify

        model_id = OPENROUTER_RERANKERS[name]

        def one(text: str):
            return rerank_classify(
                model_id,
                f'What is the overall sentiment of this text: "{text}"',
                labels, tracker)
    elif name in ("Span-01", "Span-01 Lite"):
        # OpenRouter-hosted behavior scorer: one noul (yes-probability)
        # question per label in one request, argmax = classification.
        # Non-ZDR endpoint.
        from engines.openrouter_client import noul_classify

        model_id = OPENROUTER_SYSTEMONE[name]

        def one(text: str):
            return noul_classify(model_id, text,
                                 "Does this text express {label} sentiment?",
                                 labels, tracker)
    else:  # gliclass
        from transformers import AutoTokenizer

        from gliclass import GLiClassModel, ZeroShotClassificationPipeline

        model = GLiClassModel.from_pretrained(GLICLASS[name])
        pipe = ZeroShotClassificationPipeline(
            model, AutoTokenizer.from_pretrained(GLICLASS[name]),
            classification_type="multi-label", device="cpu")

        def one(text: str):
            out = pipe(text, labels, threshold=0.0)[0]
            return max(out, key=lambda row: row["score"])["label"] if out else None
    return one


def make_decider(name: str):
    """Per-text sentiment decider for von / so1, label-head classifier
    for JevK5-Lite (same per-text call shape)."""
    labels = list(SENTIMENT_LABELS)
    if name == "von":
        from engines.von_client import load_von_decider

        decide = load_von_decider()

        def one(text: str):
            return decide(
                state=text, choices=dict(SENTIMENT_LABELS),
                instructions="What is the overall sentiment of this "
                             "text?").choice
    elif name == "JevK5-Lite":
        # in-process label-head classifier, run under the .venv-von python
        # (transformers 5.17 + jevk5)
        from jevk5 import JevK5Lite

        lite = JevK5Lite.from_pretrained("alibiserikbay/JevK5-Lite",
                                         threads=16)

        def one(text: str):
            out = lite.classify(text, {"task": labels})
            return out["task"]["labels"][0] if out["task"]["labels"] else None
    elif name == "LFM2.5-RLCD 350M":
        # in-process constrained-decision engine (vendored engines/rlcd_engine/),
        # run under the .venv-von python (transformers 5.17 + jsonschema)
        from engines.rlcd_engine.engine import Engine

        engine = Engine(device="cpu", dtype="float32")
        schema = {"type": "object",
                  "properties": {"sentiment": {"type": "string",
                                               "description": "The overall "
                                                             "sentiment of "
                                                             "the text",
                                               "enum": labels}},
                  "required": ["sentiment"],
                  "additionalProperties": False}

        def one(text: str):
            res = engine.constrained(text, schema)
            try:
                return json.loads(res["text"])["sentiment"]
            except (KeyError, ValueError):
                return None
    elif name == "MoJev 0.85B":
        # in-process packed one-pass decision scorer (vendored
        # engines/mojev_engine/), run under the .venv-von python (transformers 5.17
        # for the Qwen3.5 encoder)
        from engines.mojev_engine import load_engine

        score, _ = load_engine("cpu")

        def one(text: str):
            pred, _ = score(text, "sentiment",
                            "What is the overall sentiment of this text?",
                            labels)
            return pred
    elif name in LUMMA:
        # in-process Lumma-Fev typed-decision model via the lumma-fev
        # package (.venv-von: transformers >=5.4,<6). Same request shape
        # as bench_spectrum.py's probe: the text goes in the state AND is
        # restated in the instructions - the kev shape (empty state)
        # collapses onto one label, bare instructions trail.
        import lumma_fev

        model = lumma_fev.load(LUMMA[name])
        # one untimed decide() pays torch's first-pass init before the
        # timed loop, like the local servers' warm-up ask
        model.decide("good", {"w": {"type": "choice",
                                    "instructions": "What is the sentiment "
                                                    "of this text?",
                                    "criteria": dict(SENTIMENT_LABELS)}})

        def one(text: str):
            out = model.decide(text, {"q": {
                "type": "choice",
                "instructions": f'What is the overall sentiment of this '
                                f'text: "{text}"',
                "criteria": dict(SENTIMENT_LABELS)}})
            return out["q"].get("choice")
    elif name in JULIA:
        # in-process Julia 1 via the `julia` package shipped inside its HF
        # repo (engines/julia_client.py loads the ../Julia-1 snapshot under
        # .venv-von; the transformers 5.17 note lives there). Same request
        # shape as bench_spectrum.py's probe: bare instructions and the
        # restated text tie (3/8, identical picks), the house shape stays.
        engine = julia_client.load_engine()
        # one untimed predict() pays torch's first-pass init before the
        # timed loop, like the local servers' warm-up ask
        engine.predict("good", {"w": {"type": "choice",
                                      "instructions": "What is the sentiment "
                                                      "of this text?",
                                      "criteria": dict(SENTIMENT_LABELS)}})

        def one(text: str):
            out = engine.predict(text, {"q": {
                "type": "choice",
                "instructions": f'What is the overall sentiment of this '
                                f'text: "{text}"',
                "criteria": dict(SENTIMENT_LABELS)}})
            return out["answers"]["q"].get("choice")
    elif name in INTERN_DECISION:
        # in-process internlm Intern-Decision (Qwen3.5 fine-tune) via the
        # runtime shipped inside each HF snapshot
        # (engines/intern_decision_client.py loads the local snapshot with
        # its own per-checkpoint calibration temperature). Same request
        # shape as bench_spectrum.py's probe: bare instructions and the
        # restated text tie (8/8, identical picks), the house shape stays;
        # fp32 on CPU (bf16 is ~7x slower here, identical predictions) and
        # one predict() per text - the request is one dict with the state
        # and the questions.
        engine = intern_decision_client.load_engine(INTERN_DECISION[name])
        # one untimed predict() pays torch's first-pass init before the
        # timed loop, like the local servers' warm-up ask
        engine.predict({"state": "good", "questions": {"w": {
            "type": "choice",
            "instructions": "What is the sentiment of this text?",
            "criteria": dict(SENTIMENT_LABELS)}}})

        def one(text: str):
            out = engine.predict({"state": text, "questions": {"q": {
                "type": "choice",
                "instructions": f'What is the overall sentiment of this '
                                f'text: "{text}"',
                "criteria": dict(SENTIMENT_LABELS)}}})
            return out["answers"]["q"].get("choice")
    elif name == "nanodiff 350M":
        # diffusion-LM decision model: engines/nanodiff_engine vendors the NanoDiff
        # class (BY571/nanoDiff); runs in the MAIN venv (tiktoken),
        # slow (~10 s/text)
        from engines.nanodiff_engine.runner import QUESTION, load_model, predict

        model, _ = load_model("cpu")

        def one(text: str):
            pred, _ = predict(model, text, QUESTION["sentiment"],
                              labels, "cpu")
            return pred
    elif name == "nanodiff 350M v2":
        # the v2 retrain: same typed-decision format and architecture, new
        # weights (pngwn/nanodiff-350m-typed-decisions-v2 -- see
        # runner.CKPT_V2); like v1 otherwise
        from engines.nanodiff_engine.runner import (CKPT_V2, QUESTION,
                                                    load_model, predict)

        model, _ = load_model("cpu", CKPT_V2)

        def one(text: str):
            pred, _ = predict(model, text, QUESTION["sentiment"],
                              labels, "cpu")
            return pred
    else:
        from so1 import Choice, Decider

        decider = Decider.from_pretrained("Qwen/Qwen2.5-0.5B", backend="hf")

        def one(text: str):
            return decider.decide(state=text,
                                  questions=[Choice("sentiment", labels)],
                                  mode="separate")[0].choice
    return one


def run_laya_router(texts, lang_of):
    from laya import Router

    from engines.jev_client import choice

    router = Router(device="cpu")
    question = choice(
        "What is the overall sentiment of the text in the state: positive, "
        "negative, or neutral?",
        {label: None for label in SENTIMENT_LABELS})
    # Keep every checkpoint used by this pool resident. The Router's default
    # single-model cache otherwise reloads weights whenever scripts alternate.
    models = list(dict.fromkeys(
        router.route({"text": text}, {"q": question})["model"] for text in texts))
    router.preload(names=models)
    preds, lat = [], []
    routed: dict[str, dict[str, int]] = {}
    for text, lang in zip(texts, lang_of):
        t0 = time.perf_counter()
        out = router.predict({"text": text}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
        model = out.get("routing", {}).get("model") or "unknown"
        counts = routed.setdefault(lang, {})
        counts[model] = counts.get(model, 0) + 1
    return preds, statistics.mean(lat), routed


def run_laya_typed(texts):
    """Single English-only typed-decisions checkpoint, no routing."""
    import laya

    from engines.jev_client import choice

    agent = laya.load("convaiinnovations/laya-typed-decisions")
    question = choice(
        "What is the overall sentiment of the text in the state: positive, "
        "negative, or neutral?",
        {label: None for label in SENTIMENT_LABELS})
    preds, lat = [], []
    for text in texts:
        t0 = time.perf_counter()
        out = agent.predict({"text": text}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_jev(texts, tracker=None, model=None, meta=None):
    from engines.jev_client import DEFAULT_MODEL, JevClient, choice

    client = JevClient(model=model or DEFAULT_MODEL,
                       usage_sink=tracker.add if tracker else None)
    questions = {
        f"t{i}": choice(
            {"task": "sentiment classification of this text",
             "text": text}, SENTIMENT_LABELS)
        for i, text in enumerate(texts)
    }
    t0 = time.perf_counter()
    payload = client.ask({"task": "sentiment classification",
                          "labels": SENTIMENT_LABELS}, questions)
    dt = time.perf_counter() - t0
    if meta is not None:
        meta["model"] = payload.get("model") or client.last_model
    return ([payload["answers"][f"t{i}"].get("choice")
             for i in range(len(texts))], dt / len(texts))


def run_or_systemone(texts, model_id, tracker=None):
    """OpenRouter-hosted System One decision engine (kev-4b,
    solar-decide and the Decisions family): same per-text request shape
    as run_systemone (string instructions), no warmup - the endpoint is
    stateless. Non-ZDR endpoint unless the provider commits otherwise
    (see README daggers)."""
    from engines.jev_client import choice
    from engines.openrouter_client import systemone

    client = systemone(model_id, tracker)
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            {label: None for label in SENTIMENT_LABELS})
        t0 = time.perf_counter()
        # :free models (Mercury Decide) share a 20-requests/minute
        # account cap; JevClient.ask does not retry, so wait out the
        # reset window here instead of losing the run's 54th request
        for attempt in range(6):
            try:
                out = client.ask({"task": "sentiment"}, {"q": question})
                break
            except RuntimeError as exc:
                if "HTTP 429" not in str(exc) or attempt == 5:
                    raise
                time.sleep(20)
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_clef(texts, model_id, tracker=None):
    """Cloudflare Workers AI-hosted Clef model (clef / clef-flash): same
    per-text request shape as run_or_systemone (string instructions,
    stateless endpoint), one request per text so latency is comparable.
    Cloudflare commits to not reading/storing/training on requests."""
    from engines.clef_client import clef
    from engines.jev_client import choice

    client = clef(model_id, usage_sink=tracker.add if tracker else None)
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            {label: None for label in SENTIMENT_LABELS})
        t0 = time.perf_counter()
        out = client.ask({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_glide(texts, tracker=None):
    """Fastino-hosted GLiDE (engines/fastino_client.py): same per-text
    request shape as run_clef (string instructions with the text restated,
    stateless endpoint), one request per text so latency is comparable.
    ZDR per Fastino's model catalog; thinking tokens bill $0."""
    from engines.fastino_client import glide
    from engines.jev_client import choice

    client = glide(usage_sink=tracker.add if tracker else None)
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            {label: None for label in SENTIMENT_LABELS})
        t0 = time.perf_counter()
        out = client.ask({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_typellm(texts, thinking, tracker=None, meta=None):
    """TypeLLM's hosted API (engines/typellm_client.py): same per-text
    request shape as run_glide (house context state, text restated in the
    instructions, stateless endpoint), one request per text so latency is
    comparable. thinking=True bills per-question reasoning tokens at
    $0.50/M on top of input's $0.05/M."""
    from engines.typellm_client import enum_question, generate

    preds, lat = [], []
    for text in texts:
        question = enum_question(
            f'What is the overall sentiment of this text: "{text}"',
            list(SENTIMENT_LABELS), thinking=thinking)
        t0 = time.perf_counter()
        out = generate(json.dumps({"task": "sentiment"}), {"q": question},
                       timeout=90,
                       usage_sink=tracker.add if tracker else None)
        lat.append(time.perf_counter() - t0)
        preds.append(out["result"].get("q"))
        if meta is not None:
            meta["model"] = out.get("model")
    return preds, statistics.mean(lat)


def run_k2type(texts):
    """IFM's K2-Type-0.9B, in-process (engines/k2type_client.py loads the
    C:\\src\\K2-Type-0.9B snapshot on the CPU): same per-text request shape
    as run_clef (string instructions with the text restated, described
    criteria - the option texts the pointer head scores; the shape probe
    tied every variant at 8/8, see bench_spectrum.py), one predict() per
    text so latency is comparable with the other local models."""
    from engines import k2type_client
    from engines.jev_client import choice

    engine = k2type_client.load_engine()
    # one untimed predict pays torch's first-pass init before the timed
    # loop, like the other in-process engines
    engine.predict({"task": "warmup"},
                   {"w": choice('Sentiment of "good"?',
                                {"positive": None, "negative": None})})
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            dict(SENTIMENT_LABELS))
        t0 = time.perf_counter()
        out = engine.predict({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_decision2(texts, size):
    """vLLM Semantic Router's Decision 2.0 family, in-process
    (engines/decision2_client.py loads the local C:\\src\\Decision-2.0-*
    snapshots on the CPU; the loader also re-binds the vendored checkpoint
    fingerprint with POSIX path separators - see bench_spectrum.py for the
    notes): same per-text request shape as run_clef (string instructions
    with the text restated, described criteria - the option descriptions
    the candidate head scores; the shape probe tied every variant at 8/8,
    see bench_spectrum.py), one predict() per text so latency is
    comparable with the other local models."""
    from engines import decision2_client
    from engines.jev_client import choice

    engine = decision2_client.load_engine(size)
    # one untimed predict pays torch's first-pass init before the timed
    # loop, like the other in-process engines
    engine.predict({"task": "warmup"},
                   {"w": choice('Sentiment of "good"?',
                                {"positive": None, "negative": None})})
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            dict(SENTIMENT_LABELS))
        t0 = time.perf_counter()
        out = engine.predict({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_d3(texts, size):
    """vLLM Semantic Router's Decision 3.0 family, in-process
    (engines/d3_client.py loads the local C:\\src\\d3-* snapshots on the
    CPU; bf16 backbone, fp32 255-way code-readout head - see
    bench_spectrum.py for the notes): same per-text request shape as
    run_decision2 (string instructions with the text restated,
    described criteria - the option descriptions the answer codes are
    keyed to; Decision 2.0's shape probe tied every variant at 8/8),
    one predict() per text so latency is comparable with the other
    local models."""
    from engines import d3_client
    from engines.jev_client import choice

    engine = d3_client.load_engine(size)
    # one untimed predict pays torch's first-pass init before the timed
    # loop, like the other in-process engines
    engine.predict({"task": "warmup"},
                   {"w": choice('Sentiment of "good"?',
                                {"positive": None, "negative": None})})
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            dict(SENTIMENT_LABELS))
        t0 = time.perf_counter()
        out = engine.predict({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_d1_local(texts, model):
    """LiquidAI's open d1, in-process (engines/d1_client.py loads the
    local C:\\src\\d1-3B / d1-omni-600M snapshots on the CPU through the
    card's trust_remote_code AutoModel route): same per-text request shape
    as run_decision2 (string instructions with the text restated,
    described criteria - the option descriptions the decision head
    scores), one predict() per text so latency is comparable with the
    other local models."""
    from engines import d1_client
    from engines.jev_client import choice

    engine = d1_client.load_engine(model)
    # one untimed predict pays torch's first-pass init before the timed
    # loop, like the other in-process engines
    engine.predict({"task": "warmup"},
                   {"w": choice('Sentiment of "good"?',
                                {"positive": None, "negative": None})})
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            dict(SENTIMENT_LABELS))
        t0 = time.perf_counter()
        out = engine.predict({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_systemone(texts, port: int, model: str):
    """Local System One server (Kev/decider/OpenThai), one request per text
    so latency is comparable with the other local models. String
    instructions — the shape these servers and Laya both expect."""
    from engines.jev_client import JevClient, choice

    client = JevClient(base_url=f"http://127.0.0.1:{port}/v1/systemone",
                       model=model)
    # pay any lazy model loading before the timed loop; OpenThai's cold
    # load runs minutes, past ask()'s 120 s default
    client.ask({"task": "warmup"},
               {"w": choice('Sentiment of "good"?',
                            {"positive": None, "negative": None})},
               timeout=600)
    preds, lat = [], []
    for text in texts:
        question = choice(
            f'What is the overall sentiment of this text: "{text}"',
            {label: None for label in SENTIMENT_LABELS})
        t0 = time.perf_counter()
        out = client.ask({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_ollaya(texts, tag: str):
    """Ollaya-served decision model (local daemon on :11435, same System
    One contract as run_systemone), one request per text - and the state IS
    the text: these models' decision layers build their premise from the
    state, so embedding the text in the instructions (the shape the kev
    servers take) silently drops it for some of them. Criteria carry label
    descriptions - the option descriptions are what these models score
    (bare-label criteria measurably hurt the NLI pair encoder)."""
    from engines.jev_client import choice
    from engines.ollaya_client import systemone

    client = systemone(tag)
    # pay the first-request model load (winnow:e4b pulls 8 GB into memory)
    # before the timed loop, like the other local servers
    client.ask("good", {"w": choice("What is the sentiment of this text?",
                                    dict(SENTIMENT_LABELS))}, timeout=600)
    preds, lat = [], []
    for text in texts:
        question = choice("What is the overall sentiment of this text?",
                          dict(SENTIMENT_LABELS))
        t0 = time.perf_counter()
        out = client.ask(text, {"q": question}, timeout=600)
        lat.append(time.perf_counter() - t0)
        preds.append(out["answers"]["q"].get("choice"))
    return preds, statistics.mean(lat)


def run_verdict(texts):
    """Verdict 151M: in-process rlcd DecisionEngine (Verdict-open-jev
    checkout, agent-jev venv); abstention counts as no prediction."""
    home = os.environ.get("VERDICT_HOME", r"C:\src\verdict")
    sys.path.insert(0, home)
    from rlcd import Choice, DecisionEngine, Option

    engine = DecisionEngine(model_name_or_path=os.path.join(
        home, "artifacts", "v2"), device="cpu")
    options = [Option(id=l, description=d)
               for l, d in SENTIMENT_LABELS.items()]
    preds, lat = [], []
    for text in texts:
        query = Choice(id="q", question="What is the overall sentiment "
                                        "of this text?", options=options)
        t0 = time.perf_counter()
        res = engine.evaluate(context=text, queries=[query]).results[0]
        lat.append(time.perf_counter() - t0)
        preds.append(None if res.is_abstention else res.selected_id)
    return preds, statistics.mean(lat)


def run_agentjev(texts):
    """AgentJev-0.6B: own loopback contract (port 8149), one request per
    text, fixed per-label option descriptions (no per-question leakage)."""
    from engines.agentjev_client import ask

    options = {label: f"The text expresses {label} sentiment"
               for label in SENTIMENT_LABELS}
    preds, lat = [], []
    for text in texts:
        question = {"id": "q", "type": "choice",
                    "question": f'What is the overall sentiment of this '
                                f'text: "{text}"',
                    "options": options}
        t0 = time.perf_counter()
        out = ask({"task": "sentiment"}, {"q": question})
        lat.append(time.perf_counter() - t0)
        preds.append(out["q"]["value"])
    return preds, statistics.mean(lat)


def main() -> None:
    if hasattr(sys.stdout, "reconfigure"):
        sys.stdout.reconfigure(encoding="utf-8", errors="replace")
    parser = argparse.ArgumentParser()
    parser.add_argument("--system", required=True, choices=ALL_SYSTEMS)
    name = parser.parse_args().system

    texts, gold, lang_of = [], [], []
    for lang, items in LANGUAGES.items():
        for text, label in items:
            texts.append(text)
            gold.append(label)
            lang_of.append(lang)

    laya_routes = None
    tracker = None  # set by the hosted branches; locals stay untracked
    meta = None     # answering-model id off the response (Jev, TypeLLM)
    print(f"{name}: 54 texts (9 languages x 6) ...")
    t0 = time.perf_counter()
    if (name in GLINER or name in GLIFORMER or name in GLICLASS
            or name in GLINER_X
            or name in RERANKERS or name in NEMOTRON_RERANKERS
            or name in OPENROUTER_RERANKERS
            or name in ("Span-01", "Span-01 Lite")
            or name in FASTINO_EXTRACTORS
            or name == "Certo 421M"):
        if (name in OPENROUTER_RERANKERS or name in ("Span-01", "Span-01 Lite")
                or name in FASTINO_EXTRACTORS):
            from engines.openrouter_client import UsageTracker

            tracker = UsageTracker()
        one = make_classifier(name, tracker)
        preds, lat = [], []
        for text in texts:
            t1 = time.perf_counter()
            preds.append(one(text))
            lat.append(time.perf_counter() - t1)
        lat = statistics.mean(lat)
    elif (name in ("von", "so1 (Qwen2.5-0.5B)", "JevK5-Lite",
                   "MoJev 0.85B")
          or name in LUMMA or name in JULIA or name in INTERN_DECISION):
        one = make_decider(name)
        preds, lat = [], []
        for text in texts:
            t1 = time.perf_counter()
            preds.append(one(text))
            lat.append(time.perf_counter() - t1)
        lat = statistics.mean(lat)
    elif name == "Laya Router":
        preds, lat, laya_routes = run_laya_router(texts, lang_of)
    elif name == "Laya typed-decisions":
        preds, lat = run_laya_typed(texts)
    elif name == "Kev 0.8B (local)":
        preds, lat = run_systemone(texts, 8009, "kev-latest")
    elif name == "AgentJev 0.6B (local)":
        preds, lat = run_agentjev(texts)
    elif name == "decider 0.8B (local)":
        preds, lat = run_systemone(texts, 8018, "decider-0.8b")
    elif name == "OpenThai 0.8B (local)":
        preds, lat = run_systemone(texts, 8029, "openthai-systemone")
    elif name == "Verdict 151M (local)":
        preds, lat = run_verdict(texts)
    elif name in K2TYPE:
        preds, lat = run_k2type(texts)
    elif name in DECISION2:
        preds, lat = run_decision2(texts, DECISION2[name])
    elif name in D3:
        preds, lat = run_d3(texts, D3[name])
    elif name in D1_LOCAL:
        preds, lat = run_d1_local(texts, D1_LOCAL[name])
    elif name in OR_SYSTEMONE_CHOICE:
        from engines.openrouter_client import UsageTracker

        tracker = UsageTracker()
        preds, lat = run_or_systemone(texts, OPENROUTER_SYSTEMONE[name],
                                      tracker)
    elif name in CLEF_MODELS:
        from engines.openrouter_client import UsageTracker

        tracker = UsageTracker()
        preds, lat = run_clef(texts, CLEF_MODELS[name], tracker)
    elif name in FASTINO_MODELS:
        from engines.openrouter_client import UsageTracker

        tracker = UsageTracker()
        preds, lat = run_glide(texts, tracker)
    elif name in TYPELLM_MODELS:
        from engines.openrouter_client import UsageTracker

        tracker = UsageTracker()
        meta = {}
        preds, lat = run_typellm(texts, TYPELLM_MODELS[name], tracker,
                                 meta)
    elif name in JEV_MODELS:
        from engines.openrouter_client import UsageTracker

        tracker = UsageTracker()
        meta = {}
        preds, lat = run_jev(texts, tracker, JEV_MODELS[name], meta)
    elif name in OLLAYA:
        preds, lat = run_ollaya(texts, OLLAYA[name])
    else:
        # a new ALL_SYSTEMS entry without a dispatch branch must never
        # fall through to the paid Jev API
        raise SystemExit(f"unwired system {name!r} — add a dispatch "
                         "branch in main()")

    try:
        with open(RESULTS_FILE, encoding="utf-8") as fh:
            out = json.load(fh)
        if not isinstance(out, dict) or not all(
                isinstance(out.get(k), dict)
                for k in ("by_language", "by_tier", "latency")):
            raise ValueError
    except (OSError, ValueError):
        out = {"meta": {}, "by_language": {}, "by_tier": {}, "latency": {}}
    out.setdefault("meta", {}).setdefault("notes", []).append(
        f"{name} recorded {time.strftime('%Y-%m-%d %H:%M')}, "
        f"{time.perf_counter() - t0:.0f}s, CPU")

    by_lang = {}
    for lang in LANGUAGES:
        idx = [i for i, lg in enumerate(lang_of) if lg == lang]
        by_lang[lang] = round(sum(1 for i in idx if preds[i] == gold[i])
                              / len(idx), 4)
    by_tier = {}
    for tier in ("popular", "medium", "rare"):
        langs = [lg for lg, t in TIERS.items() if t == tier]
        by_tier[tier] = round(statistics.mean([by_lang[lg] for lg in langs]), 4)

    out["by_language"][name] = by_lang
    out["by_tier"][name] = by_tier
    out["latency"][name] = round(lat, 3)
    if tracker and tracker.requests:
        # measured provider accounting for the hosted run (54 texts)
        out.setdefault("usage", {})[name] = tracker.as_dict()
    if meta and meta.get("model"):
        # the versioned id that answered (Jev and TypeLLM repeat it on
        # every response; their aliases move with releases)
        out.setdefault("model", {})[name] = meta["model"]
    notes = out.setdefault("meta", {}).setdefault("notes", [])
    if notes and notes[0].startswith("All "):
        notes[0] = (f"All {len(out['by_language'])} systems answer "
                    "the same 54 texts.")
    # only the local System One servers, Ollaya's first-request model load,
    # and the first-pass init of Lumma, Julia, Intern-Decision, K2-Type,
    # the Decision 2.0 and Decision 3.0 families and the D1 pair get the
    # untimed warm-up; the hosted endpoints are stateless, so every
    # request is timed
    warmed = ((name.startswith(("Kev", "decider", "OpenThai", "K2-Type",
                                "Decision 2.0", "Decision 3.0"))
               or name in OLLAYA or name in LUMMA or name in JULIA
               or name in INTERN_DECISION or name in D1_LOCAL)
              and name != "Kev 4B (OpenRouter)")
    out.setdefault("timing", {})[name] = (
        "Model download and loading excluded; "
        + ("one untimed warm-up question pays the server's lazy weight "
           "load first - timed latencies are warmed." if warmed
           else "first forward pass included."))
    if name == "von":
        from engines.von_client import provenance

        out.setdefault("provenance", {})[name] = provenance()
    if laya_routes:
        out["laya_routing"] = laya_routes
    tmp = RESULTS_FILE + ".tmp"
    with open(tmp, "w", encoding="utf-8") as fh:
        json.dump(out, fh, indent=2, ensure_ascii=False)
    os.replace(tmp, RESULTS_FILE)

    overall = statistics.mean(by_lang.values())
    print(f"  overall {overall * 100:.1f}%  "
          f"(popular {by_tier['popular'] * 100:.0f}% / medium "
          f"{by_tier['medium'] * 100:.0f}% / rare "
          f"{by_tier['rare'] * 100:.0f}%)  "
          f"{lat:.3f} s/text -> {RESULTS_FILE}")
    for lang in LANGUAGES:
        print(f"    {lang:<12} {by_lang[lang] * 100:5.1f}%")


if __name__ == "__main__":
    main()