# vllm-solar-plugin

Out-of-tree [vLLM plugin](https://docs.vllm.ai/en/latest/design/plugin_system.html) for the **Solar Open 100B** model by [Upstage](https://www.upstage.ai/).

This package packages the Solar-specific changes from the [UpstageAI/vllm](https://github.com/UpstageAI/vllm/tree/v0.12.0-solar-open) fork into a standalone plugin, so they can be used with an unmodified upstream vLLM installation.

## What's included

| Component | Class | Description |
|-----------|-------|-------------|
| Model | `SolarOpenForCausalLM` | Inference-only Solar Open (dense + MoE) model |
| Logits processor | `SolarOpenTemplateLogitsProcessor` | Enforces chat-template token order via state machine |
| Logits processor | `ParallelToolCallLogitsProcessor` | Prevents parallel tool calls when `parallel_tool_calls=False` |
| Tool parser | `SolarOpenToolParser` | Parses `<\|tool_call:*\|>` delimited tool calls (batch + streaming) |
| Reasoning parser | `SolarOpenReasoningParser` | Strips `<\|think\|>…<\|end\|>` reasoning blocks from responses |

## Installation

```bash
pip install vllm-solar-plugin
```

Or from source:

```bash
git clone https://github.com/mino-park7/vllm-solar-plugin
cd vllm-solar-plugin
pip install -e .
```

## Usage

Once installed, vLLM auto-discovers the plugin via the `vllm.general_plugins` entry point — no extra flags needed.

### Basic serving

```bash
vllm serve upstage/Solar-Open-100B \
  --tool-call-parser solar_open \
  --reasoning-parser solar_open \
  --enable-auto-tool-choice
```

### With logits processors

```bash
vllm serve upstage/Solar-Open-100B \
  --tool-call-parser solar_open \
  --reasoning-parser solar_open \
  --enable-auto-tool-choice \
  --logits-processors \
    vllm_solar_plugin.logits.parallel_tool_call:ParallelToolCallLogitsProcessor \
    vllm_solar_plugin.logits.solar_open_logits:SolarOpenTemplateLogitsProcessor
```

### Python API

```python
from vllm import LLM, SamplingParams

llm = LLM(
    model="upstage/Solar-Open-100B",
    reasoning_parser="solar_open",
)
outputs = llm.chat(
    messages=[{"role": "user", "content": "Explain quantum entanglement."}],
    sampling_params=SamplingParams(max_tokens=512),
)
print(outputs[0].outputs[0].text)
```

### Controlling reasoning budget

The thinking-phase token budget is configurable via environment variables:

```bash
# High-effort reasoning (default: max 32 768 tokens, min 8 192, 60 % of max_tokens)
export SOLAR_REASONING_BUDGET_HIGH_MAX=32768
export SOLAR_REASONING_BUDGET_HIGH_MIN=8192
export SOLAR_REASONING_BUDGET_HIGH_RATIO=60

# Medium-effort reasoning
export SOLAR_REASONING_BUDGET_MEDIUM_MAX=16384
export SOLAR_REASONING_BUDGET_MEDIUM_MIN=4096
export SOLAR_REASONING_BUDGET_MEDIUM_RATIO=30
```

## Plugin registration (internals)

The `vllm.general_plugins` entry point causes vLLM to call `vllm_solar_plugin.register()` at startup, which:

1. Registers `SolarOpenForCausalLM` into `ModelRegistry` (lazy, no overhead when unused).
2. Registers `SolarOpenReasoningParser` into `ReasoningParserManager` under the name `solar_open`.

The tool parser (`SolarOpenToolParser`) is discovered automatically by vLLM's `ToolParserManager` because the `solar_open_tool_parser` module is on the Python path; pass `--tool-call-parser solar_open` to activate it.

## Upstream

Source files are adapted from the [UpstageAI/vllm](https://github.com/UpstageAI/vllm) fork, branch `v0.12.0-solar-open`, under the Apache 2.0 license.

## License

Apache 2.0 — see [LICENSE](LICENSE).