# dsh-vision-any

[简体中文](README.md) | **English**

> Paste images into text-only DeepSeek Harness agents and analyze them with **any** OpenAI-compatible, Anthropic, or Gemini vision API.

<div align="center">

## 💖 Sponsor

I am a **college-graduate village official** with a low income.  
If this plugin helps you, feel free to support me.

**Every little bit counts — thank you for your kindness!**

<img src="assets/wechat-qr.png" width="220" alt="WeChat Pay QR code" />

</div>

---

`dsh-vision-any` is a standard [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (DSH) plugin that gives text-only models (DeepSeek V4, GLM, etc.) real image understanding.

It combines two things into one plugin:

1. **Paste-to-path admission** — pasted images are accepted even when the active model is text-only; the image is saved locally and replaced with a path hint.
2. **A `vision` tool** — the agent calls `vision` with the saved path (or any local path / URL / data URI) and gets a text description from the vision model you configure.

No model switching. No hard-coded provider. Bring your own vision API.

---

## Features

- ✅ Directly paste / drag images into the DSH chat box on text-only models
- ✅ Works with any OpenAI-compatible endpoint (OpenAI, OpenCode Go, Zhipu, Qwen, Ollama, vLLM, LM Studio, ...)
- ✅ Also supports Anthropic Messages API and Google Gemini `generateContent`
- ✅ Multi-image batch: up to 10 images in one tool call
- ✅ Local image storage with dedup, LRU cap, and stale-file cleanup
- ✅ No API keys in source code — keys come from config / env vars
- ✅ Standard DSH bundle: install / remove with `dsh plugin`

---

## Install

From a local checkout:

```bash
dsh plugin --profile web add /path/to/dsh-vision-any
dsh plugin --profile headless add /path/to/dsh-vision-any
```

Or from GitHub (once published):

```bash
dsh plugin --profile web add github:<your-name>/dsh-vision-any
```

Restart `dsh web` and refresh the browser.

---

## Quick start (OpenCode Go + MiMo V2.5)

Create `~/.config/dsh-vision-any/config.json`:

```json
{
  "provider": "openai",
  "openai": {
    "baseUrl": "https://opencode.ai/zen/go/v1",
    "model": "mimo-v2.5",
    "apiKeyEnv": "OPENCODE_GO_API_KEY"
  }
}
```

Make sure `OPENCODE_GO_API_KEY` is available in the environment (for example in `~/.dsh/.env` or exported in your shell).

Then start DSH:

```bash
dsh web
```

Paste an image into the chat. The model will see a path hint and call the `vision` tool automatically.

---

## Configuration

Configuration priority (highest wins):

1. Plugin row config in a profile patch / overlay
2. `~/.config/dsh-vision-any/config.json`
3. `VISION_ANY_*` environment variables
4. Built-in defaults (OpenAI-compatible)

### Simple config (recommended)

You can fill `provider`, `baseUrl`, `model`, and `apiKeyEnv` directly at the top level:

```json
{
  "provider": "openai",
  "baseUrl": "https://opencode.ai/zen/go/v1",
  "model": "mimo-v2.5",
  "apiKeyEnv": "OPENCODE_GO_API_KEY"
}
```

### Common model quick reference

| Want to use | `provider` | `baseUrl` | `model` | `apiKeyEnv` |
| --- | --- | --- | --- | --- |
| OpenCode Go / MiMo | `openai` | `https://opencode.ai/zen/go/v1` | `mimo-v2.5` | `OPENCODE_GO_API_KEY` |
| Qwen / DashScope | `openai` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `qwen-vl-plus` | `QWEN_API_KEY` |
| OpenAI | `openai` | `https://api.openai.com/v1` | `gpt-4o-mini` | `OPENAI_API_KEY` |
| Anthropic / Claude | `anthropic` | `https://api.anthropic.com` | `claude-sonnet-4-5` | `ANTHROPIC_API_KEY` |
| Gemini | `gemini` | `https://generativelanguage.googleapis.com/v1beta` | `gemini-2.0-flash` | `GEMINI_API_KEY` |
| Local Ollama | `openai` | `http://localhost:11434/v1` | `llava` | `ollama` (or any) |

Just change `model` to switch the vision model. If you use a different OpenAI-compatible gateway, keep `provider: "openai"` and fill its `baseUrl` + `model`.

### Advanced: per-provider block

You can also put settings inside `openai` / `anthropic` / `gemini` blocks for finer control:

```json
{
  "provider": "openai",
  "openai": {
    "baseUrl": "https://api.openai.com/v1",
    "model": "gpt-4o-mini",
    "apiKeyEnv": "OPENAI_API_KEY"
  }
}
```

### Plugin row config (alternative to config file)

In your profile `cordis.patch.yml`:

```yaml
- insert:
    - id: vision
      name: dsh-vision-any
      config:
        provider: openai
        baseUrl: https://opencode.ai/zen/go/v1
        model: mimo-v2.5
        apiKeyEnv: OPENCODE_GO_API_KEY
```

### Environment variables

| Variable | Meaning |
| --- | --- |
| `VISION_ANY_PROVIDER` | `openai`, `anthropic`, or `gemini` |
| `VISION_ANY_BASE_URL` | Base URL of the vision API |
| `VISION_ANY_MODEL` | Vision model id |
| `VISION_ANY_API_KEY` | API key (fallback) |
| `VISION_ANY_API_KEY_ENV` | Name of the env var that holds the API key |
| `VISION_ANY_TIMEOUT_MS` | Request timeout (default `120000`) |
| `VISION_ANY_MAX_TOKENS` | Max output tokens (default `4096`) |
| `VISION_ANY_MAX_IMAGES` | Pasted-image LRU cap (default `200`) |
| `VISION_ANY_SYSTEM_PROMPT` | `true` / `false` |

---

## Usage

### Paste an image

Just paste or drag an image into the DSH chat box. The plugin saves it and injects a hint like:

```
[Image #1 auto-saved to /tmp/dsh-vision-any/image1/xxxx.png]
```

The model will call `vision` with that path.

### Call the tool manually

You can also ask the model to analyze a local file or URL:

> Use the `vision` tool to read `C:\path\to\screenshot.png` and tell me what error is shown.

The tool accepts:

```json
{
  "paths": ["/tmp/dsh-vision-any/image1/xxxx.png", "https://example.com/a.png"],
  "question": "What does this chart show?"
}
```

---

## How it works

1. DSH normally rejects image attachments for text-only models.
2. `dsh-vision-any` wraps the API proxy prompt admission: image parts are saved to the local temp store and replaced with a text path hint.
3. The agent calls the `vision` tool.
4. The tool resolves the image (local path / URL / data URI), sends it to the configured VLM backend, and returns plain text.
5. The text-only main model continues the conversation with the description.

---

## Requirements

- Node.js 20+ (DeepSeek Harness requires 22.19+ in practice)
- DeepSeek Harness 0.1.x (developer preview)
- A vision-capable model endpoint + API key

---

## Troubleshooting

| Symptom | Fix |
| --- | --- |
| `The current model does not support images` | Restart `dsh web` and refresh the browser; make sure the plugin is installed |
| Pasted image accepted but model says it cannot see it | Make sure the model calls the `vision` tool; check the system prompt is enabled |
| `vision provider error (HTTP 401)` | Check `apiKeyEnv` / API key / base URL |
| `vision provider error (HTTP 404)` | Check `baseUrl` and `model`; for Anthropic use a base URL that resolves to `/messages` |
| Model does not support image input | Use a vision-capable model, e.g. `mimo-v2.5`, `gpt-4o-mini`, `kimi-k2.6`, `qwen3.6-plus`, `claude-sonnet-4-5`, `gemini-2.0-flash` |

---

## License

[MIT](LICENSE)