# Copy this file to .env before starting Docker Compose.
# .env is ignored by git so local keys and tuning changes stay out of commits.

# Speed-first single-user config for a local RTX 3090.
# This is the repo's recommended default for best speed when the card is yours alone.
SPEC=dflash2
# Prefix caching is on at 1, the launchers' default. 0 turns it off.
PREFIX_CACHE=1

# Optional toggles kept commented out for easy activation later.
# Uncomment ONLY when you need the behaviour below.

# DFLASH_TOKENS=15
# Use when the model is reproducing or quoting a long prompt/document/code block.
# This is the "copy/edit/quote" mode: much faster when the answer is reusing context,
# but it reduces request slots and context headroom compared with the default 7-token verify block.
# Typical use: long docs, code edits, RAG with quoting, prompt-copy workloads.

# CTX=long
# About 150k context, at ~95-100 tok/s instead of 127. Set SPEC=mtp with it: DFlash2
# past 64k is worth it only for context reproduction and loses to SPEC=mtp CTX=long
# roughly 2:1 on everything else. (The ~95-100 tok/s is the fp8 + FlashInfer arm; on a compute capability
# 8.6 GPU the default is PTH_FA=1 below.)

# PTH_FA=1
# With SPEC=mtp CTX=long or SPEC=dflash2 CTX=long only: the KV cache is int8 per token and head, read by
# the pth-flash-attn kernel (pthfa/, built into the image) on the target and the MTP drafter instead of
# fp8 on FlashInfer, or on the DFlash2 target and drafter instead of TRITON_ATTN.
# sm80/sm86/sm89 cards only; refused on any other SPEC/CTX and with a KV offload tier. Numbers, the
# hardware they come from and the limits: docs/long-context.md and docs/gotchas.md item 63.
# Default 1 on SPEC=dflash2 CTX=long and SPEC=mtp CTX=long on a compute capability 8.6 GPU when the
# package is installed and the setup is the measured one (else the arm's own default: FA2_FP8KV=1 on
# dflash2, fp8 on FlashInfer on mtp); 0 elsewhere. PTH_FA=0 opts out. It overrides the FA2_FP8KV default,
# and an explicit FA2_FP8KV=1 with it is refused.

# CTX=huge
# The KVarN 4/2-bit KV cache: 240k context (268k tokens of pool), and it combines with
# the SPEC=dflash2 above -- keep PREFIX_CACHE=1, since the first turn over a long document
# is the expensive one. Slower than the default profile: 67 tok/s across mixed tasks,
# 164 while reproducing. The container already has KVarN built in; a venv install needs
# bash kvarn/install.sh first.

# The launcher's own default is SPEC=mtp; this template sets dflash2 above because it is the
# faster single-user profile. The line below is enabled for Docker Desktop/WSL2, which needs
# it or the V2 runner aborts with UVA errors. On native Linux it is a no-op -- platforms/
# cuda.py reads it only inside the WSL-detected branch -- so leaving it on costs nothing.
VLLM_WSL2_ENABLE_PIN_MEMORY=1

# GPU_UTIL=0.93
# WSL2 fallback for startup memory headroom; default for this repo's single-user path is 0.93.
# Uncomment only if WSL/Docker is tight and the server refuses to boot on Windows/WSL2.

# MAX_SEQS=8
# Leave alone unless you understand the state-pool/capacity tradeoff.
# In this repo, the real limit is the resident-state pool, not MAX_SEQS; for local speed mode, keep the default.

# INT8_ACT=int8
# Only for extra prefill speed in some workloads; not the main speed knob for single-user decode.
# Best used if you care about prompt-side throughput and are okay with the tradeoff.

# PREFILL_ATTN=int8
# Companion to INT8_ACT for prefill optimization on a few long prompts.
# Useful for heavy prompt workloads; not generally needed for normal chat/coding sessions.

# SPEC_ATTN=1
# Split-KV attention for the MTP verify step on CTX=fast.
# Leave default unless you specifically want to tune the MTP attention path.

# LOOKUP=1
# DFlash2 lookup-augmented drafting.
# Leave enabled unless you are debugging DFlash2 drafting behavior.

# FA2_FP8KV=1
# Default 1 on SPEC=dflash2 CTX=long on a compute capability 8.6 GPU unless PTH_FA is or defaults to 1 (fp8 KV
# through the FlashAttention-2 plugin in fa2/), 0 elsewhere. Set 1 to opt in on a 4090 (sm89, unmeasured).
# 0 restores int8 + Triton. Native installs need `bash fa2/install.sh` (refuses to start without it).
# With KV_OFFLOAD_GB or a KV connector in EXTRA_ARGS an unset value resolves to 0; an explicit 1 refuses
# that and any other SPEC/CTX. Evidence: the T4 battery on the native 3090, issue #153.

# VLLM_API_KEY=
# Optional for local-only use; REQUIRED before the port is reachable by anything but this
# machine, because the server has no authentication at all without it. Generate one rather
# than inventing one -- this is the step docs/docker.md used to make unavoidable:
#   Linux/WSL:  echo "VLLM_API_KEY=$(openssl rand -hex 24)" >> .env

# FAST_VARIANT=0
# Uncomment only if you want to skip the extra fast-variant download during the first boot.
# This is a lighter first-time setup and can help on constrained hosts, but it gives up the faster variant.

# PORT=18020
# Change only if you need to avoid a local port conflict.

# BIND=127.0.0.1
# The host address compose publishes the port on. The default keeps the server reachable from this
# machine only, so a start with no VLLM_API_KEY is not open to the network. To serve other machines,
# set VLLM_API_KEY above AND BIND=0.0.0.0 (or this host's LAN address).

# MAX_LEN=65536
# Override only if you intentionally want a different context ceiling for the active profile.
# The defaults are chosen for the profile in [single-user/start_qwen.sh](single-user/start_qwen.sh).