# dsh-knowledge v0.4.1 — MinerU recovery, and the deferred audit items

A patch release. It fixes the dead end reported as issue #30, where a scanned PDF
that MinerU could not extract left a document that could not be rebuilt, and it
closes five of the six smaller findings that the 0.4.0 audit had deliberately
deferred.

The upgrade is additive and migration-free: existing 0.4.0 data directories,
documents, chunks, embeddings, sources and model configuration continue to work
without a migration, a forced reindex, or a model download.

## A failed MinerU import is no longer a dead end (issue #30)

- MinerU's extraction runs at import time, and when it failed the document fell
  back to local parsing. For a scanned PDF with no text layer and no local OCR
  available, that fallback failed too, leaving a placeholder row that held the raw
  PDF but no `rawText` and no chunks. Rebuilding that row had nothing to read, so
  it reported `has no source text to reindex` — and there was no way out of the
  state from the panel or the API.
- Import, single-document rebuild, and startup crash recovery now all go through
  one extraction chain. Rebuilding such a row retries MinerU, so a transient
  runtime failure (key, quota, network, API response) is recoverable instead of
  permanent.
- A PDF configured for MinerU keeps using MinerU when it is rebuilt, rather than
  silently degrading to local parsing; a document deleted mid-extraction passes
  the abort through instead of pointlessly running the local parser first; and a
  MinerU import interrupted by a crash can be resumed at startup like any other.
- MinerU's own failure reason is now part of the document error instead of only a
  log line. A double failure reports
  `MinerU extraction failed (...); local parsing failed (...)`, which is the
  difference between "your API key or quota is the problem" and a parser error
  that sends the user looking in the wrong place. Neither the MinerU client nor
  the import pipeline changed behaviourally since it was introduced, so this
  release fixes the missing recovery path rather than a regression.
- Recovery retries are bounded, so a permanently failing source cannot be retried
  forever.

## The deferred audit items

The 0.4.0 audit recorded six smaller findings as deliberately out of scope. Five
are fixed here:

- **`rawTextLimit` is clamped.** It accepted any integer, so a negative value
  reached `rawText.slice(0, rawTextLimit)` and silently dropped the tail of a
  document instead of capping the payload.
- **Unknown-base statistics answer 404.** `GET /knowledge/stats?baseId=<unknown>`
  used to answer with an all-zero summary that looked like a real, empty base.
- **Chunking no longer emits empty chunks**, which could happen when a window
  boundary landed on whitespace.
- **`context.ts` is no longer bundled twice.** The shared part moved to
  `context-protocol.ts`, which both host entries import — the previous pattern
  would have silently duplicated module state for a stateful helper.
- **The stress script is typechecked**, and the retrieval benchmark runs against a
  throwaway `DSH_HOME` instead of the developer's real profile.

The remaining deferred item — moving mupdf page rendering off the host thread — is
unchanged and is documented in `docs/audit-0.4-deferred-findings.md`, which is
updated to record what this release closed.

## Compatibility

- 0.4.0 data directories open as they are. No migration, no forced reindex, no
  re-embedding, no model download.
- The one behaviour change a client could notice is that an unknown knowledge base
  now answers 404 from the statistics route instead of a zero-filled 200. A caller
  relying on the old answer was relying on a wrong value.

## Quality

- 353 tests across 26 files, with a new `tests/mineru-recovery.spec.ts` covering
  the recovery path and the reported dual failure reason.
- The release preflight passes: typecheck, the full suite, the retrieval
  benchmark, the deterministic build, package verification, the production audit
  policy, and the worker-bundle protocol smoke.
- CI passes on Node 22.19, 24 and 26, on Ubuntu, Windows and macOS, with the
  packed-install smoke on Ubuntu and Windows.

## Credit

Thanks to [InfiniteScope](https://github.com/InfiniteScope) for PRs #27–#29 and
#31, which diagnosed the MinerU dead end and closed five of the deferred findings.