# LLM Gateway

LLM Gateway is an open-source API gateway for Large Language Models (LLMs). It acts as a middleware between your applications and various LLM providers, allowing you to:

- Route requests to multiple LLM providers (OpenAI, Anthropic, Google Vertex AI, and others)
- Manage API keys for different providers in one place
- Track token usage and costs across all your LLM interactions
- Analyze performance metrics to optimize your LLM usage

## Features

- **Unified API Interface**: Compatible with the OpenAI API format for seamless migration
- **Usage Analytics**: Track requests, tokens used, response times, and costs
- **Multi-provider Support**: Connect to various LLM providers through a single gateway
- **Performance Monitoring**: Compare different models' performance and cost-effectiveness

## Getting Started

You can use LLM Gateway in two ways:

- **Hosted Version**: For immediate use without setup, visit [llmgateway.io](https://llmgateway.io) to create an account and get an API key.
- **Self-Hosted**: Deploy LLM Gateway on your own infrastructure for complete control over your data and configuration.

### Self-Hosted With Docker

Use Docker-managed volumes for the unified image. Do not bind-mount a host directory directly to `/var/lib/postgresql/data`, because PostgreSQL initialization inside the container needs to set permissions on that directory and that can fail depending on the host filesystem and ownership.

```bash
export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
./scripts/run-unified-container.sh
```

If you prefer a one-off `docker run`, use named volumes instead of `~/some-host-dir`:

```bash
docker volume create llmgateway_postgres
docker volume create llmgateway_redis

docker run -d \
  --name llmgateway \
  --restart unless-stopped \
  -p 3002:3002 \
  -p 3003:3003 \
  -p 3005:3005 \
  -p 3006:3006 \
  -p 3007:3007 \
  -p 4001:4001 \
  -p 4002:4002 \
  -v llmgateway_postgres:/var/lib/postgresql/data \
  -v llmgateway_redis:/var/lib/redis \
  -e AUTH_SECRET="$LLM_GATEWAY_SECRET" \
  -e GATEWAY_API_KEY_HASH_SECRET="$GATEWAY_API_KEY_HASH_SECRET" \
  ghcr.io/theopenco/llmgateway-unified:latest
```

### Using LLM Gateway API

```bash
curl -X POST https://api.llmgateway.io/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -d '{
  "model": "gpt-4o",
  "messages": [
    {"role": "user", "content": "Hello, how are you?"}
  ]
}'
```

## Development Setup

1. Install dependencies and set up the development environment:

   ```bash
   pnpm i && pnpm run setup
   ```

   This will install all dependencies, start Docker services, sync the database schema, and seed initial data.

   **Note for WSL2 users**: Ensure Docker Desktop is running with WSL integration enabled.

2. Start development servers:

   ```bash
   pnpm dev
   ```

3. Build for production:
   ```bash
   pnpm build
   ```

## Folder Structure

- `apps/ui`: Next.js dashboard frontend
- `apps/playground`: Lounge — consumer AI chat app (Next.js)
- `apps/code`: Next.js Dev Plans + coding tools landing & dashboard
- `apps/airside`: Airside — self-serve provider portal (Next.js)
- `apps/api`: Hono backend
- `apps/gateway`: API gateway for routing LLM requests
- `apps/docs`: Documentation site
- `ee/admin`: Internal admin dashboard
- `packages/db`: Drizzle ORM schema and migrations
- `packages/models`: Model and provider definitions
- `packages/shared`: Shared types and utilities

## Adding a provider

We do not accept pull requests that add new providers. Inference providers onboard through [Airside](https://airside.llmgateway.io), our self-serve provider portal, where you register and manage your model listings.

## License

LLMGateway is available under a dual license:

- **Open Source**: Core functionality is licensed under AGPLv3 - see the [LICENSE](LICENSE) file for details.
- **Enterprise**: Commercial features in the `ee/` directory require an Enterprise license; multi-organization administration requires a white-label license. See [ee/LICENSE](ee/LICENSE) for details.

### Enterprise features include:

- Advanced billing and subscription management
- Full request and response payload retention (Retain All Data); other plans are metadata only
- Custom provider key configurations
- Team and organization management
- Priority support
- And more to be defined

For enterprise licensing, please contact us at contact@llmgateway.io