**A security-first LLM and AI Agent gateway built in Go.**
Route, govern, and observe all LLM and MCP traffic through a single control point.
[](https://pkg.go.dev/github.com/NeuralTrust/TrustGate)
[](https://goreportcard.com/report/github.com/NeuralTrust/TrustGate)
[](go.mod)
[](LICENSE)
[](https://hub.docker.com/r/neuraltrust/trustgate)
[](https://github.com/NeuralTrust/TrustGate/actions/workflows/ci.yml)
[](https://github.com/NeuralTrust/TrustGate/actions/workflows/release.yml)
[Documentation](https://docs.neuraltrust.ai) |
[Quick Start](#-60-second-quick-start) |
[Examples](examples/) |
[Architecture](#%EF%B8%8F-architecture) |
[Community](https://join.slack.com/t/neuraltrustcommunity/shared_invite/zt-2xl47cag6-_HFNpltIULnA3wh4R6AqBg)
---
## Why TrustGate?
TrustGate is purpose-built for teams that need **enterprise-grade governance** over their LLM and agent traffic — not just routing and observability.
| | TrustGate | LiteLLM | Portkey | Helicone |
|---|:---:|:---:|:---:|:---:|
| **Security & Governance** | Per-consumer auth, policy stages, rate limiting | Basic API key proxy | API key management | Logging-focused |
| **MCP Aggregation Plane** | Native MCP gateway for AI agents (Cursor, Claude, etc.) | — | — | — |
| **Multi-Provider Routing** | 9+ providers, weighted load balancing, fallback | Multi-provider | Multi-provider | Proxy layer |
| **Deployment** | Single Go binary, no runtime deps | Python + Redis | SaaS / self-host | SaaS / self-host |
**TrustGate differentiators:**
1. **Security-first architecture** — API-key auth, per-consumer policies, plugin stages (rate limit, token rate limit, request size, semantic cache) that run before traffic hits providers.
2. **MCP aggregation plane** — A dedicated `:8082` plane that aggregates upstream MCP servers, so AI agents connect to one gateway instead of many tools. See the [MCP testing guide](docs/mcp/testing-guide.md) and [OpenAPI → MCP limits](docs/mcp/openapi.md).
3. **Single static binary** — No Python, no Node, no runtime dependencies. Deploy anywhere: Docker, Kubernetes, bare metal.
---
## 60-Second Quick Start
### Option A: One-line install (recommended)
```bash
curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash
```
This clones the repo, seeds `.env`, and starts the full stack. When Go is installed, it also builds the `trustgate` CLI.
### Option B: Docker Compose
```bash
git clone https://github.com/NeuralTrust/TrustGate.git && cd TrustGate
cp .env.example .env
make up
```
### Verify it's running
```bash
curl localhost:8080/healthz # Admin plane
curl localhost:8081/healthz # Proxy plane
curl localhost:8082/healthz # MCP plane
```
### Your first chat completion
Once running, make a request through the proxy (full setup in [examples/curl-first-request/](examples/curl-first-request/)):
```bash
# Assumes you've created a gateway, registry, and consumer (see examples/)
curl -X POST "http://localhost:8081/my-app/v1/chat/completions" \
-H "X-AG-Gateway-Slug: demo" \
-H "X-AG-API-Key: $CONSUMER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'
# 7. Embeddings (OpenAI, Azure OpenAI, Mistral, Vertex, Bedrock Titan, openai_compatible, Cohere)
curl -s -X POST "$PROXY/$CON_SLUG/v1/embeddings" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"text-embedding-3-small","input":["Hello from TrustGate"]}'
```
OpenAI-shaped clients always call `POST /{consumer}/v1/embeddings`. OpenAI, Azure, Mistral, and custom `openai_compatible` registries forward that payload to the upstream embeddings URL. Vertex uses Gemini `:embedContent` / `:batchEmbedContents`, and Bedrock Titan embed uses `InvokeModel` with `{inputText}`. A Cohere registry accepts the same OpenAI-shaped request and adapts it to Cohere `/v2/embed`:
```bash
curl -s -X POST "$PROXY/$CON_SLUG/v1/embeddings" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"embed-english-v3.0","input":["Hello from TrustGate"]}'
```
OpenAI-shaped clients call `/{consumer}/v1/files` for upload, list, retrieve, delete, and content download. OpenAI, Azure, OpenRouter, xAI, Mistral, and Anthropic registries that expose a Files API are forwarded as-is (Azure uses `{endpoint}/openai/files?api-version=…`; Anthropic uses `https://api.anthropic.com/v1/files` with `x-api-key` and `anthropic-version`). Providers without a Files store are filtered out of the pool. Retrieve, content, and delete then pin by file-id prefix: `file_` stays on Anthropic; any other files-capable provider is treated as OpenAI-compatible (`file-`). A 404 from one store is retried on the remaining backends in that family so a file uploaded to OpenAI is still found if the next request would otherwise land on Mistral. List and upload still load-balance across files-capable registries:
```bash
# 8. Files (OpenAI, Azure OpenAI, OpenRouter, xAI, Mistral, Anthropic)
curl -s -X POST "$PROXY/$CON_SLUG/v1/files" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-F purpose=assistants \
-F file="@notes.txt"
# 9. Audio speech (TTS) — raw audio bytes
curl -s -X POST "$PROXY/$CON_SLUG/v1/audio/speech" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"tts-1","input":"Hello from TrustGate","voice":"alloy"}' \
--output speech.mp3
# 10. Audio transcriptions (STT)
curl -s -X POST "$PROXY/$CON_SLUG/v1/audio/transcriptions" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-F model=whisper-1 \
-F file="@speech.mp3"
# 11. Images (OpenAI, Azure OpenAI, openai_compatible, OpenRouter)
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/generations" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"dall-e-3","prompt":"A minimal TrustGate logo","n":1,"size":"1024x1024"}'
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/edits" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-F model=dall-e-2 -F prompt="make it blue" -F image=@logo.png
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/variations" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
-F model=dall-e-2 -F image=@logo.png
# 12. Model discovery (OpenAI-compatible, gateway-owned)
curl -s "$PROXY/$CON_SLUG/v1/models" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY"
curl -s "$PROXY/$CON_SLUG/v1/models/gpt-4o-mini" \
-H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY"
```
OpenAI-shaped clients call `POST /{consumer}/v1/audio/speech` (TTS, raw audio bytes) and `POST /{consumer}/v1/audio/transcriptions` (STT, multipart file). OpenAI, Azure OpenAI, `openai_compatible`, OpenRouter, Groq, and Mistral registries that expose those APIs are forwarded as-is (Azure uses `{endpoint}/openai/deployments/{model}/audio/{speech|transcriptions}?api-version=…`). Mistral speech JSON `{audio_data}` is unwrapped to raw bytes so the gateway response stays OpenAI-shaped. `/v1/audio/translations` is not served yet. Providers without the matching audio capability are filtered out of the pool.
OpenAI-shaped clients call `POST /{consumer}/v1/images/generations` (JSON), plus multipart `POST /{consumer}/v1/images/edits` and `POST /{consumer}/v1/images/variations`. OpenAI, Azure, and `openai_compatible` registries forward the payload to the matching upstream images URL (Azure uses `{endpoint}/openai/deployments/{model}/images/{generations|edits|variations}?api-version=…`). OpenRouter registries map generations to `POST /api/v1/images` and keep edits/variations on `/api/v1/images/edits` and `/api/v1/images/variations`. Providers without an Images API are filtered out of the pool; pinning an incapable provider is a terminal 400.
`GET /{consumer}/v1/models` returns the union of native model ids the consumer can actually call (registries ∩ allowlists/policies ∩ provider capabilities). It is not an upstream `/v1/models` passthrough and not the full admin catalog.
Or use any OpenAI SDK — see [examples/openai-sdk/](examples/openai-sdk/).
---
## Features
- **High Performance** — Built in Go on [Fiber](https://gofiber.io), tuned for low latency and high concurrency.
- **Multi-Provider** — OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, Mistral, DeepSeek.
- **Smart Routing** — Round-robin, weighted, IP-hash strategies with health checks and fallback targets.
- **Plugin System** — Rate limiting, token rate limiting, request size guard, semantic cache, CORS.
- **Semantic Cache** — Embedding-based response caching for repeated prompts.
- **Multi-Tenancy** — Per-gateway consumers, API-key auth, scoped policies.
- **Observability** — Built-in metrics, OpenTelemetry (OTLP) export. See [Telemetry Configuration](#observability).
- **Independent Planes** — Admin (`:8080`), Proxy (`:8081`), MCP (`:8082`) scale separately.
---
## Architecture
TrustGate ships a **single binary** that boots one HTTP server per plane:
```bash
./trustgate # proxy (default)
./trustgate admin # admin
./trustgate mcp # MCP server
./trustgate run # admin + proxy together (single-node)
```
```mermaid
flowchart LR
subgraph Clients["Clients & Agents"]
APP["Apps / SDKs / Agents"]
end
subgraph AG["TrustGate"]
direction TB
ADMIN["Admin Plane :8080\nGateways · Registries · Consumers\nAuth · Policies · Catalog"]
PROXY["Proxy Plane :8081\nRouting · Load Balancing\nPolicy Stages · Plugins"]
MCP["MCP Plane :8082\nMCP targets & tools for agents"]
end
subgraph Plugins["Policy Plugins"]
RL["Rate Limit"]
TRL["Token Rate Limit"]
RS["Request Size"]
SC["Semantic Cache"]
CORS["CORS"]
end
subgraph Providers["LLM Providers"]
P1["OpenAI · Anthropic\nAzure · Bedrock"]
P2["Gemini · Vertex\nGroq · Mistral"]
end
subgraph Infra["Infrastructure"]
PG[("Postgres")]
RD[("Redis")]
KFK[["Kafka"]]
end
APP -->|API key| PROXY
APP -->|MCP| MCP
PROXY --> Plugins
PROXY -->|load balance| Providers
ADMIN -. config .-> PROXY
ADMIN -. config .-> MCP
ADMIN --- PG
PROXY --- PG
PROXY --- RD
MCP --- PG
PROXY -->|telemetry| KFK
```
| Plane | Port | Responsibilities |
|-------|------|------------------|
| **Admin** | `8080` | Gateway, registry, consumer, auth, policy management. DB migrations. |
| **Proxy** | `8081` | Request routing, load balancing, plugin execution, provider forwarding. |
| **MCP** | `8082` | Model Context Protocol server for AI agents. See [MCP Guide](docs/mcp/testing-guide.md) and [OpenAPI → MCP](docs/mcp/openapi.md). |
---
## MCP Plane for AI Agents
TrustGate's MCP plane (`:8082`) lets AI agents like **Cursor** and **Claude** connect to multiple MCP tool servers through a single gateway. Configure once, use everywhere.
```json
// Cursor mcp.json example
{
"mcpServers": {
"trustgate": {
"url": "http://localhost:8082/agent-client/mcp",
"headers": {
"X-AG-API-Key": "