---
title: Self Host LLM API
description: Simple guide to self-hosting LLM API using Docker.
icon: Server
---

# Self Host LLM API

LLM API is a self-hostable platform that provides a unified API gateway for multiple LLM providers. This guide offers two simple options to get started.

## Prerequisites

- Latest Docker
- API keys for the LLM providers you want to use (OpenAI, Anthropic, etc.)

## Option 1: Unified Docker Image (Simplest)

This option uses a single Docker container that includes all services (UI, API, Gateway, Database, Redis).

```bash
# Run the container
docker run -d \
  --name llmgateway \
  --restart unless-stopped \
  -p 3002:3002 \
  -p 3003:3003 \
  -p 3005:3005 \
  -p 3006:3006 \
  -p 4001:4001 \
  -p 4002:4002 \
  -v ~/llmgateway_data:/var/lib/postgresql/data \
  -e AUTH_SECRET=your-secret-key-here \
  ghcr.io/Spendbase/llmgateway-unified:latest
```

Note: it is recommended to use the latest version tag from here instead of `latest`: https://github.com/Spendbase/llmgateway/releases

### Using Docker Compose (Alternative for unified image)

```bash
# Download the compose file
curl -O https://raw.githubusercontent.com/Spendbase/llmgateway/main/infra/docker-compose.unified.yml
curl -O https://raw.githubusercontent.com/Spendbase/llmgateway/main/.env.example

# Configure environment
cp .env.example .env
# Edit .env with your configuration

# Start the service
docker compose -f docker-compose.unified.yml up -d
```

Note: it is recommended to replace the `latest` version tag in the image with the latest version from here: https://github.com/Spendbase/llmgateway/releases

## Option 2: Separate Services with Docker Compose

This option uses separate containers for each service, offering more flexibility.

```bash
# Clone the repository
git clone https://github.com/Spendbase/llmgateway.git
cd llmgateway

# Configure environment
cp .env.example .env
# Edit .env with your configuration

# Start the services
docker compose -f infra/docker-compose.split.yml up -d
```

Note: it is recommended to replace the `latest` version tag in all images in the compose file with the latest version from here: https://github.com/Spendbase/llmgateway/releases

## Accessing Your LLM API

After starting either option, you can access:

- **Web Interface**: http://localhost:3002
- **Documentation**: http://localhost:3005
- **API Endpoint**: http://localhost:4002
- **Gateway Endpoint**: http://localhost:4001

## Required Configuration

At minimum, you need to set these environment variables:

```bash
# Database (change the password!)
POSTGRES_PASSWORD=your_secure_password_here

# Authentication
AUTH_SECRET=your-secret-key-here

# LLM Provider API Keys (add the ones you need)
LLM_OPENAI_API_KEY=sk-...
LLM_ANTHROPIC_API_KEY=sk-ant-...
```

## Basic Management Commands

### For Unified Docker (Option 1)

```bash
# View logs
docker logs llmgateway

# Restart container
docker restart llmgateway

# Stop container
docker stop llmgateway
```

### For Docker Compose (Option 2)

```bash
# View logs
docker compose -f infra/docker-compose.split.yml logs -f

# Restart services
docker compose -f infra/docker-compose.split.yml restart

# Stop services
docker compose -f infra/docker-compose.split.yml down
```

## Build locally

To build locally, you can use the \*.local.yml compose file in the `infra` directory, which will build the images from the source code.

## All provider API keys

You can set any of the following API keys:

```text
LLM_OPENAI_API_KEY=
LLM_ANTHROPIC_API_KEY=
```

## Multiple API Keys and Load Balancing

LLM API supports multiple API keys per provider for load balancing and increased availability. Simply provide comma-separated values for your API keys:

```bash
# Multiple OpenAI keys for load balancing
LLM_OPENAI_API_KEY=sk-key1,sk-key2,sk-key3

# Multiple Anthropic keys
LLM_ANTHROPIC_API_KEY=sk-ant-key1,sk-ant-key2
```

### Health-Aware Routing

The gateway automatically tracks the health of each API key and routes requests to healthy keys. If a key experiences consecutive errors, it will be temporarily skipped. Keys that return authentication errors (401/403) are permanently blacklisted until restart.

### Related Configuration Values

For providers that require additional configuration (like Google Vertex), you can specify multiple values that correspond to each API key. The gateway will always use the matching index:

```bash
# Multiple Google Vertex configurations
LLM_GOOGLE_VERTEX_API_KEY=key1,key2,key3
LLM_GOOGLE_CLOUD_PROJECT=project-a,project-b,project-c
LLM_GOOGLE_VERTEX_REGION=us-central1,europe-west1,asia-east1
```

When the gateway selects `key2`, it will automatically use `project-b` and `europe-west1`. If you have fewer configuration values than keys, the last value will be reused for remaining keys.

## Next Steps

Once your LLM API is running:

1. **Open the web interface** at http://localhost:3002
2. **Create your first organization** and project
3. **Generate API keys** for your applications
4. **Test the gateway** by making API calls to http://localhost:4001