# DshOmniFile（dsh-omnifile）

> File adaptation plugin: attach local files (drag-drop / paste / multi-select / official `@file`) to a conversation; the main model resolves file content on demand via the `dshomnifile` tool.

> English | **[中文](README.md)**

## How it works (parse on demand)

1. Files become chips in the composer; sending serializes them as `文件：<absolute path>` references.
2. When needed, the main model calls the **dshomnifile tool**, which resolves each referenced path and returns Markdown.
3. A chat card above each user message shows the files: images as thumbnails (click to preview), other files as rows (click to open the source file with the OS default app).

## Supported parse routes

| Kind | Handling |
|---|---|
| Plain text (`.json/.txt/.md/.html/.shtml`) | Multi-encoding decode (UTF-8/16/32, GB18030, BOM detection), JSON pretty-print, HTML stripping; binary content rejected |
| Images | Describe via the configured multimodal model (PNG/JPEG/WebP/GIF natively; BMP/SVG/AVIF/TIFF auto-transcoded to PNG), content-hash cache + concurrency queue |
| Office docs / PDF / spreadsheets / EPUB / CSV | anydoc for text; **embedded images** unzipped and described, assembled in document order |
| **PDF: text-only** | anydoc extracts the text layer directly (no rasterization/OCR, cheapest) |
| **PDF: text + images** | Smart gate detects raster pages → render + per-page OCR |
| **PDF: pure scans** (incl. CCITT fax / JBIG2) | anydoc yields nothing → render + per-page OCR |

- PDF rendering is **in-process** (pdfjs-dist + @napi-rs/canvas with `wasmUrl` and related resource paths configured), cross-platform with zero local dependencies; CCITT-fax / JBIG2 1-bit scan pages are rendered and OCR’d automatically under `auto`.
- In a batch, a failure on one file only affects that file (error is inlined into its result); cancellation aborts immediately.

## Requirements

- DSH **0.1.2-alpha.1 or newer** (Desktop / Web).
- A multimodal model supporting image input, configured under **Settings → Models** (see Configuration).

## Installing from GitHub

The plugin is published as a ready-built package: the `lib/` bundles and `cordis.patch.yml` are committed, so installation needs no local build (no build script runs on install, hence no pnpm build-approval gate).

```bash
dsh plugin --profile desktop add github:<your-username>/dsh-omnifile
```

- DSH installs it with `pnpm add github:...` inside the profile directory (`$DSH_HOME/profiles/<profile name>`):
  - runtime dependencies (@napi-rs/canvas, pdfjs-dist, @firecrawl/anydoc) are installed automatically into the profile’s node_modules — nothing to do manually;
  - the plugin is auto-added to the profile layer — **restart DSH** to activate.
- The anydoc CLI / extract-zip used for parsing are NOT pre-installed: on the first document parse, `npx` fetches a pinned version from the npm registry on demand and caches it (network required; the first parse fails on an offline machine).
- Desktop ships its own pnpm; Web / CLI installs require `pnpm` on PATH.

## Configuration

After configuring a model under DSH “Settings → Models”, pick it in this plugin’s settings page as `providerRef`; without it, the current session model is used (must support images).

Overridable `settings.yaml` keys:

```yaml
omnifile:
  providerRef: llm-pi-ai/vision/general-model   # unique ref of the selected multimodal model
  describePrompt: '请按要求描述这张图片。'      # fixed describe prompt (tool may append the user’s question)
  timeoutMs: 60000            # multimodal call timeout (ms)
  maxFileBytes: 50            # single-file size cap (MB)
  maxBatchFiles: 20           # max files per upload (excess is truncated with a hint)
  concurrency: 10             # multimodal concurrency (excess waits in queue, never rejected)
  maxTokens: 16384            # max output tokens per describe call
  logLevel: warn              # debug / info / warn / error
  pdfScan: auto               # auto=smart (see table) / force=always render+OCR every PDF (most complete, priciest)
```

## Logging & troubleshooting

- 4 levels with prefix `[dsh-omnifile]`; host logs mirror to DSH `ctx.logger` (persisted to `userData/logs/dsh-*.log`) and `console.error`; browser logs go to DevTools Console (F12).
- Level control: settings page → env `DSH_OMNIFILE_LOG_LEVEL=debug` (highest priority, synced to the browser via `/api/omnifile/config`) → `omnifile.logLevel` in `settings.yaml`.
- `http://127.0.0.1:<port>/api/omnifile/config` returns the effective config and limits.

## Architecture

```text
src/core/          shared: constants / markers / utils / logger
src/core/host/     host: config, paths, http, file-kind (extensions), text decode, anydoc/npx,
                   PDF render & raster gate (pdfrender), parse pipeline (parse), vision (multimodal),
                   model catalog, tool registration
src/core/client/   client utils & logger adapter
src/host/index.ts  host root: settings namespace + /api/omnifile/* routes + dshomnifile tool
src/client/        client root: intake (upload/drag/paste/@), chip controller, chat file cards,
                   settings section, styles
build/ scripts/    build config & entry (vite: host / common / client)
test/              node --test regressions (CCITT render, raster gate, image transcode, encodings)
```

## Limitations

- `@` completion relies on the host’s workspace file-reference service; drag-drop/paste/upload still work without it.
- Image recognition requires the configured multimodal model; text-only models fail with an actionable error.
- BOM-less UTF-16 Chinese short texts are ambiguous and rejected as binary (use files with a BOM).
- Legacy `.doc/.xls/.ppt` depend on anydoc; AVIF/TIFF transcode depends on runtime decode support.
- Parsed Markdown is handed to the main model in full — the plugin never truncates content.
- Nothing is sent to third-party clouds; multimodal calls only go to endpoints you configured.

## License

MIT