make demo: it boots Grafana + Prometheus + a synthetic LLM workload and deploys these dashboards for real in about a minute.scripts/.
Conceptual cost flow for native gateway spend and OTel GenAI token pricing. Not a Grafana screenshot..json` away, with no change to the blueprint code.
## Repo layout
```
SKILL.md # agent playbook (7-phase pipeline, doctrine, pitfalls)
scripts/
grafana_client.py # universal client: OSS/Cloud/Enterprise, legacy + K8s-style APIs
discover.py # capability map: real metric names, dialects, gaps
forge_dashboards.py # 7 blueprints × detected dialect, cost engine, alerts
visual_audit.py # render/Playwright capture + DOM pre-scan for vision review
references/
model_registry.json # 30+ models: $/1M in·out·cached, context, provider origin, GPAI
locale.fr.json # label translations (--locale fr)
ai_governance_frameworks.md # crosswalk: EU AI Act × ISO/IEC 42001 × NIST AI RMF
query_library.md # PromQL/LogQL/TraceQL per dialect, anti-patterns
dashboard_blueprints.md # panel-by-panel specs + optional extensions
instrumentation_guide.md # exact configs to close each gap
eu_ai_act_observability.md # article → signal → panel mapping, deployer checklist
visual_verification.md # vision checklist, failure signatures → fixes
grafana_api_compat.md # 3 API generations, editions matrix, schema v2 notes
demo/ # docker-compose stack: Grafana + Prometheus + synthetic LLM emitter
tools/package.py # builds the .skill reproducibly and verifies it matches sources
tests/live_query_check.py # runs every generated query against a real Prometheus
tests/value_invariants.py # checks the numbers agree with each other, not just exist
tests/audit_harness.py # offline checks across 4 instance topologies + regressions
```
## FAQ
**Why not Grafana Cloud's own AI Observability?** Use it if you're on Cloud and happy to instrument with Grafana's SDK, which does evaluations and conversation replay well. This project targets the other case: self-hosted OSS/Enterprise, telemetry you already emit (OTel, LiteLLM, vLLM, no SDK migration), plus cost attribution by provider origin and an EU AI Act evidence layer, which no vendor ships. Declared deployment locations are shown independently of provider origin. They compose fine: nothing here conflicts with the Grafana plugins.
**Does it overwrite my existing dashboards?** No. Everything lives in its own folder with `llmops-forge`-tagged, deterministically-UID'd dashboards. Re-running updates in place.
**Grafana Cloud?** Yes. Cloud is auto-detected; the image renderer is built in, so visual audit works out of the box.
**My models aren't in the registry.** They appear in an "unpriced models" panel instead of being billed wrong. Add a price or alias to `model_registry.json`, or explicitly opt in to `--pricing-fallback artificial-analysis`. The fallback accepts only a unique exact normalized ID, slug, name, or alias match. Ambiguous, absent, null, or failed responses stay unpriced.
**Can I publish the generated dashboards?** Yes: `--export-portable` emits JSON with `__inputs`/`${DS_PROMETHEUS}` placeholders, the format grafana.com/dashboards requires.
**Multiple Prometheus (prod + staging)?** Financial sources at the same priority are ambiguous: the forge refuses a silent choice. Pin discovery or forge with `--datasource `; costs from separate datasources are never added together.
**Multiple orgs or folders?** Pin the Grafana organization with `--org-id` and
give each independently managed folder a stable `--uid-scope`. The forge
refuses to overwrite an existing dashboard UID found in another folder.
**Is the governance dashboard legal advice?** No, and it says so on the dashboard. It is the *evidence layer* your counsel or auditor will ask for: the logging continuity, model inventory and incident trail that ISO 42001 Stage 2, NIST MANAGE 4.1 and AI Act Art. 12 all want to see. It proves monitoring happened; it does not prove a management system exists, and `references/ai_governance_frameworks.md` is explicit about both halves.
## Roadmap
**Make every AI change a decision you can defend.** The proposed direction is reproducible operational decisions for European enterprises on self-hosted Grafana, using their existing telemetry and evaluation tools.
These are planned horizons, with evidence gates rather than release dates:
- **H0 · Trusted foundations:** interpretable measurements, explicit gaps and read-only configuration diffs.
- **H1 · Compare a change:** cost per successful task at explicit quality and latency criteria, including a reasoned refusal when evidence is insufficient.
- **H2 · Ship with evidence:** portable promotion dossiers and checks consumed by the customer's existing CI and provisioning tools.
- **H3 · Rehearse the next migration:** executed migration tests or clearly labeled sensitivity scenarios, after the earlier workflow proves useful.
As of 2026-09-14, **v2.0.2 is the released baseline**; the five foundational corrections and budget-alert lifecycle fix in [PR #5](https://github.com/alebgl77/grafana-llmops-forge/pull/5) are integrated into main but not yet included in a release. The proposed comparison and dossier workflows are future work.
Read the [canonical roadmap](ROADMAP.md) or its [French translation](docs/ROADMAP.fr.md) for the future demonstration, scope and pass/fail criteria.
## Support
Issues and discussions are the place for bugs and questions; I read everything. If your organisation needs this wired into a real platform (multi-tenant Grafana, AI Act evidence pack for an audit, FinOps governance across business units), I do that professionally: [ia-b2b.fr](https://ia-b2b.fr).
## Contributing
Model prices drift quarterly, so **registry PRs are the most valuable contribution** and take 2 minutes ([guide](CONTRIBUTING.md)). Dialect additions (new gateway/engine signatures) are second. `python3 tests/audit_harness.py` must stay green.
Built with the Agent Skills open standard · works in Claude Code, Claude.ai, Cowork, and as a plain CLI.
If this saved your platform team some time, a ⭐ helps others find it.