# Copy this file to .env before starting Docker Compose.
# .env is ignored by git so local keys and tuning changes stay out of commits.

# Speed-first single-user config for a local RTX 3090.
# This is the repo's recommended default for best speed when the card is yours alone.
SPEC=dflash2
PREFIX_CACHE=1

# Optional toggles kept commented out for easy activation later.
# Uncomment ONLY when you need the behaviour below.

# DFLASH_TOKENS=15
# Use when the model is reproducing or quoting a long prompt/document/code block.
# This is the "copy/edit/quote" mode: much faster when the answer is reusing context,
# but it reduces request slots and context headroom compared with the default 7-token verify block.
# Typical use: long docs, code edits, RAG with quoting, prompt-copy workloads.

# CTX=long
# About 150k context, at ~95-100 tok/s instead of 127. Set SPEC=mtp with it: DFlash2
# past 64k is worth it only for context reproduction and loses to SPEC=mtp CTX=long
# roughly 2:1 on everything else.

# CTX=huge
# The KVarN 4/2-bit KV cache: 240k context (268k tokens of pool), and it combines with
# the SPEC=dflash2 above -- keep PREFIX_CACHE=1, since the first turn over a long document
# is the expensive one. Slower than the default profile: 67 tok/s across mixed tasks,
# 164 while reproducing. The container already has KVarN built in; a venv install needs
# bash kvarn/install.sh first.

# The launcher's own default is SPEC=mtp; this template sets dflash2 above because it is the
# faster single-user profile. The line below is enabled for Docker Desktop/WSL2, which needs
# it or the V2 runner aborts with UVA errors. On native Linux it is a no-op -- platforms/
# cuda.py reads it only inside the WSL-detected branch -- so leaving it on costs nothing.
VLLM_WSL2_ENABLE_PIN_MEMORY=1

# GPU_UTIL=0.93
# WSL2 fallback for startup memory headroom; default for this repo's single-user path is 0.93.
# Uncomment only if WSL/Docker is tight and the server refuses to boot on Windows/WSL2.

# MAX_SEQS=8
# Leave alone unless you understand the state-pool/capacity tradeoff.
# In this repo, the real limit is the resident-state pool, not MAX_SEQS; for local speed mode, keep the default.

# INT8_ACT=int8
# Only for extra prefill speed in some workloads; not the main speed knob for single-user decode.
# Best used if you care about prompt-side throughput and are okay with the tradeoff.

# PREFILL_ATTN=int8
# Companion to INT8_ACT for prefill optimization on a few long prompts.
# Useful for heavy prompt workloads; not generally needed for normal chat/coding sessions.

# SPEC_ATTN=1
# Split-KV attention for the MTP verify step on CTX=fast.
# Leave default unless you specifically want to tune the MTP attention path.

# LOOKUP=1
# DFlash2 lookup-augmented drafting.
# Leave enabled unless you are debugging DFlash2 drafting behavior.

# VLLM_API_KEY=
# Optional for local-only use; REQUIRED before the port is reachable by anything but this
# machine, because the server has no authentication at all without it. Generate one rather
# than inventing one -- this is the step docs/docker.md used to make unavoidable:
#   Linux/WSL:  echo "VLLM_API_KEY=$(openssl rand -hex 24)" >> .env

# FAST_VARIANT=0
# Uncomment only if you want to skip the extra fast-variant download during the first boot.
# This is a lighter first-time setup and can help on constrained hosts, but it gives up the faster variant.

# PORT=18020
# Change only if you need to avoid a local port conflict.

# MAX_LEN=65536
# Override only if you intentionally want a different context ceiling for the active profile.
# The defaults are chosen for the profile in [single-user/start_qwen.sh](single-user/start_qwen.sh).