# Clients & coding agents

[← Back to README](../README.md) · [Documentation index](README.md)

- [OpenAI-compatible clients](#openai-compatible-clients)
- [Coding agents](#coding-agents)
- [Native Gemini clients](#native-gemini-clients)
- [Ollama clients](#ollama-clients)
- [Headerless clients](#headerless-clients)
- [MCP server](#mcp-server)
- [VS Code ghost-text autocomplete (Continue)](#vs-code-ghost-text-autocomplete-continue)
- [Context Handoff](#context-handoff)

## OpenAI-compatible clients

Any client that can target an OpenAI-compatible base URL can use FreeLLMAPI:

- **LangChain, LlamaIndex, official OpenAI SDKs**: set `base_url` to
  `http://localhost:3001/v1` and use the unified key from the dashboard.
- **Local GPU boxes**: add custom OpenAI-compatible endpoints for Ollama,
  llama.cpp, LM Studio, vLLM, or an internal gateway.

## Coding agents

Use the generator instead of hand-editing a client configuration:

```bash
export FREELLMAPI_API_KEY=<unified-key>   # or pass --api-key on each command
npx freellmapi setup-claude --url http://localhost:3001 --dry-run
npx freellmapi setup-claude --url http://localhost:3001
```

`--dry-run` prints a diff. Real writes merge with the existing configuration
and create a timestamped backup first. `--profile <name>` creates a named
Claude/Codex profile. The live `/v1/models` catalog supplies the model ids and
context windows.

| Agent | Automated command | Manual base URL | Wire |
| --- | --- | --- | --- |
| **Claude Code** | `setup-claude` or credential-free-on-disk `launch` | `http://localhost:3001` | Anthropic Messages |
| **Codex CLI** | `setup-codex` or `launch-codex` | `http://localhost:3001/v1` | Responses (`wire_api = "responses"`) |
| **Cline** | `setup-cline` | `http://localhost:3001/v1` | OpenAI Chat |
| **Continue** | `setup-continue` | `http://localhost:3001/v1` | OpenAI Chat / legacy Completions |
| **Aider** | `setup-aider` | `http://localhost:3001/v1` | OpenAI Chat |
| **OpenCode** | `setup-opencode` | `http://localhost:3001/v1` | OpenAI Chat |
| **Goose** | `setup-goose` | `http://localhost:3001/v1` | OpenAI Chat |
| **Qwen Code** | `setup-qwen` | `http://localhost:3001/v1` | OpenAI Chat (native Gemini also works) |
| **Roo Code** | `setup-roo` | `http://localhost:3001/v1` | OpenAI Chat |
| **Kilo Code** | `setup-kilo` | `http://localhost:3001/v1` | OpenAI Chat |
| **Crush** | `setup-crush` | `http://localhost:3001/v1` | OpenAI Chat |
| **Cursor** | `setup-cursor` prints the guide | public `https://…/v1` | OpenAI Chat |
| **Anything else** | `setup-generic` prints a ready block | `http://localhost:3001/v1` | OpenAI Chat |

The root-vs-`/v1` distinction matters: Claude Code expects the server root
because it appends the Anthropic Messages path. OpenAI-compatible clients in
this table—including Cline, Aider, Goose, Codex, Continue, OpenCode, Qwen,
Roo, Kilo, and Crush—expect their configured base URL to include `/v1`.

## Native Gemini clients

Gemini CLI and Gemini-lineage clients can speak Google's wire format directly:

```bash
export GOOGLE_GEMINI_BASE_URL=http://localhost:3001
export GEMINI_API_KEY=freellmapi-your-unified-key
gemini
```

The native surface implements `GET /v1beta/models`, model metadata,
`generateContent`, `streamGenerateContent` (including `?alt=sse`), and
`countTokens`. Authentication accepts `x-goog-api-key`, Bearer, or Gemini's
`?key=` fallback. Prefer the header: query credentials leak into history and
proxy logs.

The **Keys → Agents** tab maps Gemini Pro, Flash, and Flash-Lite family names to
Auto or a pinned catalog model.

## Ollama clients

Ollama emulation is off by default. Enable one of these modes on
**Keys → Agents**:

- `open-loopback`: no key on this machine only. The socket peer must be
  `127.0.0.1`/`::1`; enabling desktop LAN access does not widen it.
  **Docker note:** inside a container the socket peer is the Docker bridge
  IP, not loopback, so this mode refuses even host-local traffic through a
  published port — use `key-required` for Docker deployments.
- `key-required`: clients must send `Authorization: Bearer <unified-key>`.

The exact endpoints are `/api/tags`, `/api/chat`, `/api/generate`, `/api/show`,
`/api/version`, `/api/embed`, and legacy `/api/embeddings`. Streaming uses
newline-delimited JSON, not SSE. Point Zed, JetBrains AI Assistant, or another
Ollama-capable client at `http://localhost:3001`.

## Headerless clients

If a client cannot set headers, create a separately revocable token on
**Keys → Agents** and use:

```text
http://localhost:3001/v1/t/<token>/chat/completions
http://localhost:3001/v1/t/<token>/responses
http://localhost:3001/v1/t/<token>/models
```

The same prefix exposes `/api/chat` and `/api/tags`. Never put the unified API
key in a URL. URL tokens have independent hashes and immediate revocation
because URLs routinely leak into shell history, reverse-proxy logs, and
telemetry.

## MCP server

On top of inference, the router is an **MCP server**: agents can introspect it mid-session
(usable models and the params each one honors, provider health, usage and cache stats,
routing strategy). For Claude Code:

```bash
claude mcp add --transport http freellmapi http://localhost:3001/mcp \
  --header "Authorization: Bearer freellmapi-your-unified-key"
```

Any MCP client that speaks Streamable HTTP works the same way: point it at `/mcp` with the
unified key as a Bearer token.

FreeLLMAPI is local-first and single-user by design. Your provider keys stay in
your SQLite database, encrypted at rest, and requests go from your machine to the
upstream providers you enabled.

## VS Code ghost-text autocomplete (Continue)

FreeLLMAPI exposes `/v1/completions` for editor autocomplete clients that send legacy OpenAI prompt/suffix requests. Example Continue config:

```yaml
models:
  - name: FreeLLMAPI Autocomplete
    provider: openai
    model: auto
    apiBase: http://localhost:3001/v1
    apiKey: freellmapi-your-unified-key
    useLegacyCompletionsEndpoint: true
    roles:
      - autocomplete
```

## Context Handoff

When FreeLLMAPI falls over to a different model mid-conversation (quota, rate limit, cooldown), the new model has no idea it is picking up someone else's task. **Context handoff** adds a single compact `system` message to the outbound request that tells the new model exactly that:

```
FreeLLMAPI context handoff:
You are taking over an ongoing conversation from another model (groq:llama-3 → google:gemini-flash).
Continue the user's task using the conversation context already provided in this request.
Do not restart the task, re-ask already answered setup questions, or discard prior tool results.
Respect the user's latest message as the highest-priority instruction.

Recent session summary:
User: …
Assistant: …
```

**Enable it in `.env`:**

```env
FREELLMAPI_CONTEXT_HANDOFF=on_model_switch
```

**How it works:**

- Messages per session are stored in memory (TTL: 3 hours).
- Only injected when the selected model changes for a given session key.
- Not injected on the first request, on same-model continuations, or if a handoff message is already present.
- Session key: `X-Session-Id` header if present, otherwise SHA-1 of the first user message (same as sticky sessions).
- Storage is in-memory only. Nothing is written to disk or logged.

> **Important:** Context Handoff improves continuity for conversations routed through FreeLLMAPI. It cannot recover provider-internal hidden state or messages that were never sent to the proxy.