---
name: llama-stage-patch-changes
description: Use this skill when changing mesh-llm's patched llama.cpp Skippy ABI, runtime hooks, model introspection, tensor filtering, activation-frame execution, GGUF writer surface, upstream pin, or patch queue.
metadata:
  short-description: Maintain the llama.cpp Skippy patch queue
---

# llama-stage-patch-changes

Use this skill when changing the Skippy staged-runtime ABI carried in
`skippy/llama_cpp/patches`.

## Boundaries

- Keep durable llama.cpp-side changes in the ordered queue under
  `skippy/llama_cpp/patches`: top-level core patches first,
  `model_support/series` second, and `generated/series` last.
- Keep the upstream pin in `skippy/llama_cpp/upstream.txt`.
- Do not edit `.deps/llama.cpp` as the final artifact; regenerate the
  patch queue from commits.
- Keep mesh orchestration, protocol compatibility, lifecycle, model management,
  and API status behavior in Rust.
- Keep one functional boundary per patch. Patch numbers must be unique and
  contiguous within each queue lane.
- Put family-specific model support in a focused `model_support/` patch. Keep
  reusable staged-runtime machinery in the core lane and generated graph
  annotations in the generated lane.
- Keep public ABI declarations separate from independently reviewable model
  lifecycle, loading, and package implementation changes.
- The Skippy native ABI is an internal lockstep boundary, not a stable
  cross-version compatibility contract. It may change whenever the feature
  requires it; update the Rust FFI mirror and all callers in the same change.
- Do not preserve old native ABI signatures for compatibility. Bump the ABI
  version when the boundary changes so mismatches are diagnosable, and make
  sure the shipped Rust side and native runtime are built from the same queue.
- Do not add a terminal source-reorganization patch. A deliberate layout or
  ownership change must be represented in the recreated patches that own the
  affected capabilities.

## Native Source Layout

- `include/skippy.h` is an umbrella only. Put public C ABI declarations in
  standalone `include/skippy/<capability>.h` headers.
- Put implementations in `src/skippy/<capability>.cpp` and private C++
  declarations in narrowly named `src/skippy/*.h` headers.
- Use `snake_case` capability names. Keep exported symbols prefixed with
  `skippy_` and avoid generic `helpers`, `utils`, or expanded `common` modules.
- `src/skippy.cpp` is retired. Extend the owning capability module and keep new
  implementation files below 1,000 lines.
- Make every public header independently compilable as both C11 and C++17.
  Update explicit CMake source lists and installation rules with new modules.
- Do not preserve retired source include paths unless the task explicitly asks
  for compatibility. Continue to version and mirror any binary ABI change.

## Graph input capabilities

- Derive planned input modes from what the graph can execute, not merely from
  allocated input tensors. Shared builders may allocate an embedding input that
  a particular model's metadata forbids using.
- Carry capability declarations through both helper construction and direct
  input-bundle registration. Keep model restrictions at graph construction;
  do not add family-name dispatch to the generic planner.
- When adding an input-mode profile, cover both an accepting synthetic model
  and a rejecting variant. Preserve the graph's existing rejection of invalid
  direct input while ensuring metadata-only planning never probes that mode.

## Native API documentation

- Treat Doxygen-style comments in `include/skippy.h` and
  `include/skippy/*.h` as the source of truth for the public API reference.
  Every public header and exported `skippy_*` function must have an adjacent
  `@brief` describing what it is used for.
- When the public header surface changes, prepare the patched checkout and
  regenerate the website reference before finishing the change:

  ```bash
  scripts/prepare-llama.sh pinned
  python3 scripts/generate-skippy-api-doc.py
  python3 scripts/generate-skippy-api-doc.py --check
  ```

- Commit `mesh/website/src/docs/pages/skippy-api.md` alongside the native queue
  change. The generated page must not be hand-edited, and its inventory must
  include every public header and exported function in the prepared checkout.

## ABI PR documentation requirements

Every pull request that changes the Skippy ABI must include an explicit ABI
inventory in the PR description. Do not describe a changed function signature
as a newly added function.

The inventory must state, for each change:

- the exact symbol or declaration name and its complete signature or field
  change;
- whether it was added, changed, deprecated, deleted, or removed;
- the public header containing the declaration;
- the implementation source and Rust FFI mirror, when applicable;
- the ABI version before and after the change;
- why the change is required and what data or behavior it enables;
- that backward compatibility with older native runtimes is intentionally not
  required, and that the Rust FFI mirror and callers were updated in lockstep;
- the tests that exercise the native ABI boundary, including public-header
  compilation when a header changes.

Use this compact table in the PR description:

| Status | Symbol/declaration | Public header | Implementation / mirror | Reason | Lockstep update |
|---|---|---|---|---|---|
| Changed / Added / Removed | exact name and signature | `include/skippy/<capability>.h` | `src/skippy/<capability>.cpp`; Rust FFI path | behavior enabled | Rust mirror/callers updated; old ABI not supported |

For a changed function signature, call out that it is an ABI change even when
the symbol name is unchanged. List removed declarations explicitly as
“none” when no functions or fields were deleted; this prevents reviewers from
having to infer removals from a patch diff. Keep this inventory synchronized
with the ABI version constants in `include/skippy/common.h` and the mirrors in
`skippy/crates/skippy-ffi/src/lib.rs`. Do not add compatibility shims solely to
support an older native runtime; the acceptance criterion is a synchronized
Rust/native build and a clear version mismatch if the pieces are mixed.

## Local Flow

Prepare the pinned checkout and current patch queue:

```bash
scripts/prepare-llama.sh pinned
```

For llama-side editing, work in `.deps/llama.cpp` or another llama.cpp
checkout where commits can be named and inspected. Base the branch on the
pinned upstream, then carry core stage ABI commits, model-support commits, and
generated family commits in that order.

For an ordinary core capability change, emit one focused mail-format patch
after the current top-level core lane. Do not rewrite unrelated entries or put
the patch after `model_support/` or generated shards:

```bash
repo_root="$(pwd)"
llama_checkout="${LLAMA_CHECKOUT:-$repo_root/.deps/llama.cpp}"
last_patch="$(find skippy/llama_cpp/patches -maxdepth 1 -type f -name '*.patch' | sort | tail -n 1)"
last_number="${last_patch##*/}"
last_number="${last_number%%-*}"
next_number=$((10#$last_number + 1))
git -C "$llama_checkout" format-patch -1 \
  --start-number "$next_number" \
  --output-directory "$repo_root/skippy/llama_cpp/patches" HEAD
```

For a deliberate queue-boundary or source-layout change, rebuild the affected
series from the pinned upstream instead. Create capability-owned commits in
their intended order, place declarations and implementation in their final
modules from the first patch that introduces them, and format the complete
replacement series with contiguous numbering. Before replacing the durable
queue, verify both of these invariants:

```bash
# The reconstructed commit series has exactly the intended final tree.
git diff --exit-code <authoritative-final-commit> <reconstructed-series-head>

# No patch defers the structural change to the end of the series.
git log --reverse --oneline <pinned-upstream>..<reconstructed-series-head>
```

Move the old queue to an explicit temporary backup, generate the replacement
into a fresh `skippy/llama_cpp/patches` directory, and retain the backup
until clean application and native compilation pass. Never keep both series or
duplicate patch numbers in the durable directory.

## Validation

Validate patch application in a clean checkout:

```bash
tmp_root="$(mktemp -d /tmp/mesh-llama.XXXXXX)"
trap 'rm -rf -- "$tmp_root"' EXIT
LLAMA_WORKDIR="$tmp_root/llama.cpp" scripts/prepare-llama.sh pinned
LLAMA_WORKDIR="$tmp_root/llama.cpp" \
  MESH_LLM_LLAMA_BUILD_ROOT="$tmp_root/build" \
  LLAMA_STAGE_BACKEND=cpu \
  LLAMA_STAGE_LINK_MODE=static \
  scripts/build-llama.sh
```

### Re-pinning upstream

Advancing `skippy/llama_cpp/upstream.txt` can silently invalidate a patch
that depends on upstream's *ordering*, not just its symbols. The queue still
applies, everything compiles, and the behavior is broken. This happened with
upstream `1269cb1`, which moved `check_tensor_dims` ahead of `buft_for_tensor`
and left the stage tensor filter running too late; split serving was broken on
main because no test opened a real mid-stage artifact.

So on every re-pin, in addition to the checks above:

- Read `git log <old-pin>..<new-pin> -- src/llama-model-loader.* src/llama-model.*`
  for changes to load order, not just to signatures the patches touch.
- Prove the staged load path with a real artifact whose first block is not
  block 0. `cargo test -p skippy-package-builder` covers this via
  `mid_stage_artifact_opens_with_the_stage_filter_applied`.
- Confirm that test actually ran rather than skipped. It is gated on
  `SKIPPY_CORRECTNESS_MODEL`; without it the test prints
  `skipping mid-stage: SKIPPY_CORRECTNESS_MODEL is not set` and passes. Grep the
  CI log for `mid_stage_artifact_opens_with_the_stage_filter_applied ... ok`, or
  set the variable locally. A skipped gate reads identically to a pass.

Compile each new public header once as C11 and once as C++17 with warnings
treated as errors. For implementation moves, run the tests owned by the moved
capability in addition to the Rust fallout checks below.

For Rust fallout, run cargo commands serially:

```bash
cargo fmt --all --check
cargo check -p mesh-llm
cargo test -p skippy-runtime --lib
cargo test -p skippy-serving --lib
cargo test -p mesh-llm --lib
```

Patch files are mail-format artifacts. Do not hand-normalize them in a way that
breaks `git am`.