# Build System Reference — llama-cpp-pro 0.2.1

Complete consolidated guide covering build variants, package configuration, and release procedures for all platforms: iOS, Android, Web/PWA, and Desktop (Electron).

---

## 📖 Table of Contents

1. [Quick Start](#quick-start)
2. [Platform Overview](#platform-overview)
3. [Prerequisites](#prerequisites)
4. [Build Variants](#build-variants)
5. [Platform-Specific Build Instructions](#platform-specific-build-instructions)
6. [Package Configuration](#package-configuration)
7. [Release Process](#release-process)
8. [Pre-Release Checklist](#pre-release-checklist)
9. [Environment Variables](#environment-variables)
10. [Common Issues](#common-issues)
11. [FAQ](#faq)

---

## Quick Start

### One-Command Production Release

```bash
npm publish
```

This automatically runs the `prepublishOnly` hook → `npm run build:package`:
1. TypeScript compilation → `dist/`
2. iOS native framework → `ios/Frameworks/llama-cpp.framework/`
3. Android JNI library → `android/src/main/jniLibs/arm64-v8a/`
4. Web/PWA WASM worker → `dist/wasm/` + `dist/workers/`
5. Artifact verification
6. npm packaging and publish

**Result:** Users receive iOS + Android + Web/PWA + Desktop in one `~35-40 MB` package ✅

### Manual Build Steps

```bash
# 1. Clean all previous builds
npm run clean:all

# 2. Build minimal variant (iOS + Android + WASM + TypeScript)
./build-variants.sh --variant minimal

# 3. Verify all artifacts
npm run verify:pack:artifacts

# 4. Create and inspect the tarball
npm pack --ignore-scripts

# 5. Publish
npm publish
```

> **⚠️ Important:** `build-variants.sh` handles TypeScript compilation. Do NOT run `npm run build` separately afterward — it invokes `npm run clean` first and deletes WASM artifacts.

---

## Platform Overview

| Platform | Runtime | Build output | GPU |
|----------|---------|--------------|-----|
| **iOS** | Native Metal + CPU | `ios/Frameworks/llama-cpp.framework/` | ✅ Metal |
| **Android** | JNI + CPU/Vulkan | `android/src/main/jniLibs/arm64-v8a/libllama-cpp-arm64.so` | ✅ Vulkan |
| **Web / PWA** | WebAssembly (Emscripten) | `dist/wasm/` + `dist/workers/` | ❌ CPU only |
| **Desktop** | Electron + native sidecar | `sidecar/bin/<os>-<arch>` + WASM fallback | ✅ CUDA/ROCm/Metal/Vulkan |

All four platforms share the same TypeScript API surface (`src/definitions.ts`). Every method is implemented on every platform — see the [Feature Coverage Matrix](README.md#-full-api-feature-coverage-matrix) for the full breakdown.

---

## Prerequisites

### All Platforms

| Tool | Version | Install |
|------|---------|---------|
| Node.js | ≥ 18 | https://nodejs.org |
| npm | ≥ 9 | bundled with Node.js |
| CMake | ≥ 3.16 | `brew install cmake` / `apt install cmake` |

### iOS (macOS only)

| Tool | Version | Install |
|------|---------|---------|
| macOS | ≥ 13 Ventura | — |
| Xcode | ≥ 15 | App Store |
| Xcode CLI tools | — | `xcode-select --install` |

```bash
# Verify
xcode-select -p
cmake --version
```

### Android

| Tool | Version | Install |
|------|---------|---------|
| Android Studio | ≥ Flamingo | https://developer.android.com/studio |
| Android NDK | r26+ | Android Studio → SDK Manager → NDK |
| Java (JDK) | 17 | `brew install openjdk@17` |

```bash
# Required environment variables
export ANDROID_HOME=$HOME/Library/Android/sdk        # macOS
export ANDROID_HOME=$HOME/Android/Sdk               # Linux
export ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$(ls $ANDROID_HOME/ndk | tail -1)
export PATH=$PATH:$ANDROID_HOME/platform-tools

# Verify
adb version
$ANDROID_NDK_HOME/ndk-build --version
```

### Web / PWA (WASM)

| Tool | Version | Install |
|------|---------|---------|
| Emscripten | ≥ 3.1.50 | `brew install emscripten` or see [emsdk](https://emscripten.org/docs/getting_started/downloads.html) |
| Rust | ≥ 1.75 | `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \| sh` |
| wasm-bindgen-cli | matches Cargo.toml | `cargo install wasm-bindgen-cli` |

```bash
# Add wasm target
rustup target add wasm32-unknown-emscripten

# Verify
emcc --version
rustc --version
wasm-bindgen --version
cargo install wasm-pack   # optional, for standalone WASM builds
```

### Desktop (Electron sidecar)

| Tool | Needed for | Install |
|------|-----------|---------|
| CMake ≥ 3.20 | All sidecar builds | see above |
| NVIDIA CUDA Toolkit | `--cuda` variant | https://developer.nvidia.com/cuda-downloads |
| AMD ROCm | `--rocm` variant | https://rocm.docs.amd.com |
| Vulkan SDK | Vulkan backend | https://vulkan.lunarg.com/sdk/home |
| OpenBLAS | CPU-optimized Linux/Win | `apt install libopenblas-dev` |

---

## Build Variants

### Variant Comparison

| Variant | Size | iOS | Android | Web/WASM | Sources | Debug | Use |
|---------|------|-----|---------|---------|---------|-------|-----|
| **minimal** ⭐ | ~23-25 MB | arm64 stripped | arm64 stripped | ✅ | ❌ | ❌ | **Public npm** |
| **core** | ~33-40 MB | arm64 stripped | arm64 stripped | ✅ | ✅ | ❌ | Production + rebuild |
| **development** | ~55-70 MB | arm64+x86_64 | arm64+x86_64 | ✅ | ✅ | ✅ | Local dev/testing |
| **ios-only** | ~8-10 MB | arm64 stripped | ❌ | ❌ | ✅ | ❌ | iOS-only apps |
| **android-only** | ~25-30 MB | ❌ | arm64 stripped | ❌ | ✅ | ❌ | Android-only apps |
| **full** | ~75+ MB | All | All | ✅ | ✅ | ✅ | ❌ Not recommended |

### Build any variant

```bash
./build-variants.sh --variant [minimal|core|development|ios-only|android-only|full]
```

### Minimal Variant (Recommended for npm)

```bash
./build-variants.sh --variant minimal
```

**Size breakdown:**
```
iOS arm64 (stripped):         3–4 MB
Android arm64 (stripped):    15–16 MB
WASM + PWA worker:            2–3 MB
TypeScript dist:              1–2 MB
─────────────────────────────────────
Total:                      ~23–25 MB
```

---

## Platform-Specific Build Instructions

### iOS

The iOS build produces a universal `llama-cpp.framework` containing the C++ inference engine and exposes C symbols via `cap-ios-bridge.h`.

#### Build framework

```bash
# Device (arm64) — ships in the npm package
cd ios
cmake -B build-device -S . \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=16.0
cmake --build build-device --config Release -j$(sysctl -n hw.ncpu)

# Simulator (arm64 + x86_64) — needed for Xcode Simulator
bash scripts/ensure-llama-ios-xcframework.sh
```

#### What gets built

| Symbol | Purpose |
|--------|---------|
| `llama_init_context` | Load GGUF model, returns context ID |
| `llama_run_completion` | Run inference, returns JSON result |
| `llama_run_embedding_json` | Generate embeddings |
| `llama_rerank_json` | Rerank documents |
| `llama_bench` | Benchmark pp/tg performance |
| `llama_load/save_session_file` | KV-cache session persistence |
| `llama_apply/remove_lora_adapters` | LoRA adapter management |
| `llama_init/release_multimodal` | Vision/audio multimodal |
| `llama_init/release_vocoder` | TTS vocoder |
| `llama_get/decode_audio_tokens` | TTS token synthesis |
| `llama_get_context_gpu_info` | Metal GPU usage reporting |
| `cap_llama_server_start/stop` | In-process HTTP server |

All symbols are declared in `cpp/cap-ios-bridge.h` and implemented in `cpp/cap-ios-bridge.cpp`.

#### Swift integration

```swift
// LlamaNativeBridge.swift — dlsym-based dynamic loading
let fn = try LlamaNativeBridge.sym("llama_run_completion", RunCompletionFn.self)
```

The Swift layer in `ios/Sources/LlamaCppCapacitor/LlamaCpp.swift` wraps every symbol with error handling and background `DispatchQueue` dispatch.

#### GPU support

Metal is used automatically when `n_gpu_layers > 0` is passed to `initContext`. The `llama_get_context_gpu_info` symbol returns `{"gpu": true/false, "reasonNoGPU": "..."}`.

---

### Android

The Android build produces a JNI shared library loaded by `LlamaCpp.java` via `System.loadLibrary()`.

#### Build library

```bash
cd android
./gradlew assembleRelease

# Or via CMake directly
cmake -B build-android -S src/main \
  -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake \
  -DANDROID_ABI=arm64-v8a \
  -DANDROID_PLATFORM=android-26 \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-android --config Release -j$(nproc)
```

#### JNI file map

| JNI file | Methods |
|----------|---------|
| `jni.cpp` | `initContextNative`, `completionNative`, `tokenizeNative`, `embeddingNative`, `rerankNative`, `benchNative`, `loadSessionNative`, `saveSessionNative` |
| `jni-lora.cpp` | `applyLoraAdaptersNative`, `removeLoraAdaptersNative`, `getLoadedLoraAdaptersNative` |
| `jni-multimodal.cpp` | `initMultimodalNative`, `isMultimodalEnabledNative`, `getMultimodalSupportNative`, `releaseMultimodalNative` |
| `jni-tts.cpp` | `initVocoderNative`, `isVocoderEnabledNative`, `getFormattedAudioCompletionNative`, `getAudioCompletionGuideTokensNative`, `decodeAudioTokensNative`, `releaseVocoderNative` |
| `jni-chat-session.cpp` | (chat helper utilities, integrated into `jni.cpp` at link time) |

#### Completion param propagation

All 20+ sampling parameters from `NativeCompletionParams` are extracted in `completionNative` and mapped to `common_params::sampling`:

```
temperature, top_p, top_k, min_p, typical_p
penalty_repeat, penalty_freq, penalty_present, penalty_last_n
mirostat, mirostat_tau, mirostat_eta
dry_multiplier, dry_base, dry_allowed_length, dry_penalty_last_n
grammar, stop[], seed, n_probs, top_n_sigma, guide_tokens
```

#### GPU support

Vulkan layers are enabled when `n_gpu_layers > 0`. The context result reflects actual GPU availability. `n_gpu_layers=0` → CPU-only with explicit `reasonNoGPU` message.

---

### Web / PWA (WebAssembly)

The Web build compiles the C++ inference engine to WebAssembly via Emscripten + Rust FFI, running inside a dedicated Web Worker.

#### Build commands

```bash
# Standard WASM (single-threaded, JSPI async)
npm run build:wasm

# WASM with pthread (multi-threaded — requires COOP/COEP headers)
npm run build:wasm:pthreads

# Full WASM (JSPI + largest context support)
npm run build:wasm:full

# Build Web Worker bundle
npm run build:worker

# Copy WASM assets to dist/
npm run build:wasm:assets

# All three steps together
npm run build:pwa
```

#### Build output

```
dist/wasm/
├── llama_engine.js                   ← wasm-bindgen JS glue
├── llama_engine.wasm                 ← SIDE_MODULE (Rust FFI)
├── llama_engine_emscripten.mjs       ← ESM module (MAIN_MODULE with C++)
├── llama_engine_emscripten.wasm      ← WASM MAIN_MODULE binary
└── package.json

dist/workers/
└── llm.worker.js                     ← Web Worker bundle
```

#### Environment flags

```bash
LLAMA_WASM_JSPI=1    # Enable async/await JSPI (default: 1)
LLAMA_WASM_PTHREAD=0 # Disable pthreads (default: 0 — no SharedArrayBuffer needed)
LLAMA_WASM_PTHREAD=1 # Enable pthreads (requires COOP/COEP response headers)
```

#### Chat template support

On Web, `getFormattedChat` delegates to `provider.getFormattedChat` when available (WASM Jinja path). If not exposed by the WASM build, it falls back to four built-in client-side formatters: `chatml`, `llama3`, `mistral`, `gemma`.

#### OPFS model storage

Models are stored in the browser's Origin Private File System. Use `downloadModel` or `ensureModelInOpfs` to cache GGUF files:

```typescript
import { LlamaCpp } from 'llama-cpp-pro';
await LlamaCpp.downloadModel({ url: 'https://…/model.gguf', filename: 'model.gguf' });
```

#### PWA server headers (COOP/COEP)

Required only for the pthread build:

```http
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
```

Single-threaded WASM (default) does **not** require these headers.

---

### Desktop (Electron + native sidecar)

The Desktop build wraps a native llama.cpp sidecar process behind an IPC bridge, with WASM as a fallback.

#### Build sidecar

```bash
# macOS (Metal + CoreML ANE) — recommended for macOS
npm run build:sidecar:metal

# macOS or Linux (CPU with OpenBLAS)
npm run build:sidecar:cpu

# Linux / Windows (Vulkan + CPU)
npm run build:sidecar

# NVIDIA CUDA
npm run build:sidecar:cuda

# AMD ROCm
npm run build:sidecar:rocm

# Auto-detect host (Metal on macOS, Vulkan on Linux, CPU elsewhere)
./scripts/build-sidecar.sh
```

#### Stage for Electron packaging

```bash
# Copies sidecar binary + WASM to extraResources/
npm run build:desktop
```

#### Electron integration

```javascript
// Main process (main.js / main.ts)
const { registerLlamaDesktopIpc } = require('llama-cpp-pro/desktop');
registerLlamaDesktopIpc({ ipcMain, app });

// Preload script
require('llama-cpp-pro/desktop/preload')(contextBridge, ipcRenderer);

// Renderer / Capacitor web layer — auto-selected via LlamaCppDesktop
import { LlamaCpp } from 'llama-cpp-pro';
const ctx = await LlamaCpp.initContext({ contextId: 0, params: { model: '/path/to/model.gguf' } });
```

#### Electron Builder config

```javascript
// electron-builder.config.js
const llama = require('llama-cpp-pro/desktop/electron-builder');
module.exports = llama.merge({
  appId: 'com.yourapp.id',
  // ...your config
});
```

#### Sidecar variant table

| OS / arch | Variant | Accelerator | Script |
|-----------|---------|-------------|--------|
| macOS arm64 | `metal-coreml` | Metal + CoreML | `npm run build:sidecar:metal` |
| macOS x64 | `metal` | Metal | `npm run build:sidecar` |
| Linux x64 | `vulkan-openblas` | Vulkan + CPU | `npm run build:sidecar:linux` |
| Windows x64 | `vulkan-openblas` | Vulkan + CPU | `npm run build:sidecar:win` |
| Any | `cuda` | NVIDIA CUDA | `npm run build:sidecar:cuda` |
| Linux | `rocm` | AMD ROCm | `npm run build:sidecar:rocm` |
| Any | `cpu` | CPU only | `npm run build:sidecar:cpu` |

---

## Package Configuration

### `files` array in `package.json` (0.2.1 — current)

The current `package.json` includes all platforms:

```json
{
  "files": [
    "android/src/main/",
    "android/build.gradle",
    "build-native.sh",
    "cpp/",
    "dist/",
    "ios/CMakeLists.txt",
    "ios/CMakeLists-arm64.txt",
    "ios/CMakeLists-x86_64.txt",
    "ios/metal-embed.cmake",
    "ios/embed-metal-shaders.sh",
    "ios/Sources",
    "ios/Frameworks",
    "scripts/ensure-llama-ios-xcframework.sh",
    "scripts/embed-llama-ios-app-framework.sh",
    "Package.swift",
    "LlamaCpp.podspec",
    "LlamaCppCapacitor.podspec",
    "types/",
    "desktop/",
    "sidecar/",
    "cmake/",
    "extraResources/",
    "scripts/stage-desktop-resources.cjs",
    "scripts/ensure-desktop-sidecar-bundle.cjs",
    "scripts/build-sidecar.sh",
    "scripts/build-sidecar-linux.sh",
    "scripts/build-sidecar-win.bat"
  ]
}
```

### Minimal variant override (smallest package, no C++ sources)

```json
{
  "files": [
    "android/src/main/jniLibs/",
    "android/build.gradle",
    "dist/",
    "ios/Sources",
    "ios/Frameworks",
    "Package.swift",
    "LlamaCpp.podspec",
    "LlamaCppCapacitor.podspec",
    "types/"
  ]
}
```

**Size:** ~23-25 MB (no `cpp/`, no `sidecar/`, no desktop scripts)

---

## Release Process

### Step 1 — Bump version

```bash
# Edit package.json "version" to "0.2.1"
# Or use npm:
npm version patch   # 0.2.0 → 0.2.1
```

### Step 2 — Clean

```bash
npm run clean:all
```

### Step 3 — Build all platforms

```bash
# Option A — full automated (recommended)
./build-variants.sh --variant minimal

# Option B — build each platform separately
npm run build              # TypeScript
npm run build:native       # iOS + Android
npm run build:pwa          # WASM + Web Worker
npm run build:desktop      # Desktop sidecar + staging (optional)
```

### Step 4 — Verify artifacts

```bash
npm run verify:pack:artifacts
# Expected: iOS framework ✅, Android .so ✅, WASM ✅, dist/ ✅
```

### Step 5 — Create tarball

```bash
npm pack --ignore-scripts
# Creates: llama-cpp-pro-0.2.1.tgz
```

### Step 6 — Inspect tarball

```bash
# Check contents
tar -tzf llama-cpp-pro-0.2.1.tgz | head -40

# Check size
du -h llama-cpp-pro-0.2.1.tgz

# Verify platform artifacts are present
tar -tzf llama-cpp-pro-0.2.1.tgz | grep -E "(ios|android|wasm|sidecar)"
```

### Step 7 — Publish

```bash
npm whoami      # confirm logged in
npm publish
npm view llama-cpp-pro@0.2.1
```

---

## Pre-Release Checklist

- [ ] `package.json` version set to `0.2.1`
- [ ] `CHANGELOG.md` updated with 0.2.1 release notes
- [ ] `npm run clean:all` completed
- [ ] Build completed successfully (iOS + Android + WASM)
- [ ] `npm run verify:pack:artifacts` — all green
- [ ] `npm pack --ignore-scripts` — tarball created
- [ ] `tar -tzf llama-cpp-pro-0.2.1.tgz | grep ios/Frameworks` — iOS framework present
- [ ] `tar -tzf llama-cpp-pro-0.2.1.tgz | grep libllama-cpp` — Android .so present
- [ ] `tar -tzf llama-cpp-pro-0.2.1.tgz | grep wasm` — WASM assets present
- [ ] `npm whoami` — logged in to npm
- [ ] `npm publish` — published
- [ ] `npm view llama-cpp-pro@0.2.1` — live on registry

---

## Environment Variables

```bash
# Native builds
STRIP_SYMBOLS=true     # Strip debug symbols (default: true for production variants)
BUILD_JOBS=16          # Parallel CMake jobs (default: CPU count)
VERBOSE=1              # Verbose build output

# Android
ANDROID_HOME=/path/to/sdk
ANDROID_NDK_HOME=/path/to/ndk

# WASM
LLAMA_WASM_JSPI=1      # Enable JSPI async (default: 1)
LLAMA_WASM_PTHREAD=0   # Enable pthreads (default: 0)

# Desktop sidecar
LLAMA_CPP_UPSTREAM=/path/to/upstream  # Sync upstream llama.cpp GPU backends
OPENBLAS_ROOT=/path/to/openblas       # Custom OpenBLAS for Linux/Windows CPU builds

# Combined example
BUILD_JOBS=16 STRIP_SYMBOLS=true VERBOSE=1 ./build-variants.sh --variant minimal
```

---

## Clean Scripts

```bash
npm run clean          # TypeScript dist only
npm run clean:native   # iOS/Android build artifacts + Frameworks
npm run clean:wasm     # Rust target, WASM pkg, dist/wasm
npm run clean:all      # Everything above
npm run clean:test     # Test output and coverage
```

---

## Common Issues

### "Binary not found" after build

```bash
npm run clean:native
./build-variants.sh --variant minimal
```

### iOS build skipped on non-macOS

iOS builds require macOS + Xcode. On Linux/Windows, iOS is automatically skipped — this is expected. Build iOS on macOS, then copy the framework to `ios/Frameworks/`.

### Android NDK not found

```bash
export ANDROID_HOME=$HOME/Library/Android/sdk
export ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$(ls $ANDROID_HOME/ndk | sort -V | tail -1)
echo $ANDROID_NDK_HOME   # should print a path ending in a version number
```

### WASM build fails

```bash
# Check Emscripten is activated
source /path/to/emsdk/emsdk_env.sh
emcc --version

# Rebuild from scratch
npm run clean:wasm
npm run build:pwa
```

### Desktop sidecar not found

```bash
# Build for your host
npm run build:sidecar        # auto-detects OS
npm run stage:desktop        # copies to extraResources/
```

### Package larger than expected

```bash
# Check if C++ sources were accidentally included
tar -tzf llama-cpp-pro-0.2.1.tgz | grep "^package/cpp/" | wc -l
# Should be 0 for minimal variant
```

---

## FAQ

### Q: Why is there both `prepublishOnly` and `build:package`?

`prepublishOnly` runs automatically on `npm publish`. `build:package` is the same script you can call manually to test without publishing.

### Q: Can I build only one platform?

Yes:
```bash
npm run build:ios      # iOS only
npm run build:android  # Android only
npm run build:pwa      # Web/WASM only
npm run build:sidecar  # Desktop sidecar only
```

### Q: Which variant should I publish?

**Minimal** for the public npm package. It's ~23-25 MB, fully functional on all platforms, with no debug overhead.

### Q: How do I rebuild the iOS XCFramework for simulators?

```bash
bash scripts/ensure-llama-ios-xcframework.sh
npx cap sync ios
```

### Q: How do I add a new C++ API symbol to iOS?

1. Declare in `cpp/cap-ios-bridge.h`
2. Implement in `cpp/cap-ios-bridge.cpp` (add ABI alias at the bottom)
3. Add `typealias` + `trySymOpt` call in `ios/Sources/LlamaCppCapacitor/LlamaNativeBridge.swift`
4. Call from `ios/Sources/LlamaCppCapacitor/LlamaCpp.swift`
5. Rebuild iOS framework

### Q: How do I wire a new JNI method on Android?

1. Declare `private native ...` in `android/.../LlamaCpp.java`
2. Add `JNIEXPORT` function in `android/src/main/jni.cpp` (or relevant `jni-*.cpp`)
3. Call `nativeMethod(...)` from the Java method body
4. Rebuild Android

---

## Key Commands Reference

```bash
# Build all platforms
./build-variants.sh --variant minimal

# Build individual platforms
npm run build              # TypeScript
npm run build:native       # iOS + Android
npm run build:pwa          # Web/WASM + worker
npm run build:desktop      # Desktop sidecar
npm run build:ios:xcframework  # iOS Simulator XCFramework

# Verify and package
npm run verify:pack:artifacts
npm pack --ignore-scripts

# Release
npm version patch   # bump 0.2.0 → 0.2.1
npm publish

# Clean
npm run clean:all

# Inspect tarball
tar -tzf llama-cpp-pro-0.2.1.tgz | head -40
du -h llama-cpp-pro-0.2.1.tgz
```

## 🔬 Full API Feature Coverage Matrix

Verified against the complete `LlamaCppPlugin` interface. Every method in `src/definitions.ts` is accounted for across all four runtime targets.

**Legend:** ✅ Full native implementation · ⚠️ Partial / platform limitation · ❌ Not available (by design)

### Core Lifecycle

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `toggleNativeLog` | ✅ Metal framework | ✅ JNI | ✅ no-op | ✅ inherited |
| `setContextLimit` | ✅ | ✅ | ✅ no-op | ✅ |
| `modelInfo` | ✅ native GGUF scan | ✅ native GGUF scan | ✅ OPFS manifest | ✅ inherited |
| `initContext` | ✅ | ✅ | ✅ | ✅ sidecar |
| `releaseContext` | ✅ | ✅ | ✅ | ✅ |
| `releaseAllContexts` | ✅ | ✅ | ✅ | ✅ |

### GPU Reporting

| Field | iOS | Android | Web (PWA) | Desktop |
|-------|-----|---------|-----------|---------|
| `gpu` in `initContext` result | ✅ queries `llama_get_context_gpu_info` — reflects Metal usage | ✅ reflects `n_gpu_layers` request vs Vulkan availability | ✅ always `false` (correct — no WebGPU inference) | ✅ sidecar `gpuEnabled` flag |

### Inference & Chat

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `getFormattedChat` | ✅ native Jinja via `llama_get_formatted_chat` | ✅ native via JNI | ✅ provider delegate → 4-template client fallback | ✅ inherited |
| `completion` | ✅ full JSON param passthrough | ✅ all sampling params propagated | ✅ | ✅ sidecar HTTP |
| `stopCompletion` | ✅ | ✅ | ✅ | ✅ |
| `chat` | ✅ → `getFormattedChat` + `completion` | ✅ → `getFormattedChat` + `completion` | ✅ → `getFormattedChat` + `completion` | ✅ |
| `chatWithSystem` | ✅ | ✅ | ✅ | ✅ |
| `generateText` | ✅ | ✅ | ✅ | ✅ |

### Completion Sampling Parameters

All parameters in `NativeCompletionParams` are forwarded on every platform:

| Parameter group | iOS | Android | Web (PWA) | Desktop |
|-----------------|-----|---------|-----------|---------|
| `temperature`, `top_p`, `top_k`, `min_p` | ✅ | ✅ | ✅ | ✅ |
| `penalty_repeat`, `penalty_freq`, `penalty_present`, `penalty_last_n` | ✅ | ✅ | ✅ | ✅ |
| `mirostat`, `mirostat_tau`, `mirostat_eta` | ✅ | ✅ | ✅ | ✅ |
| `dry_multiplier`, `dry_base`, `dry_allowed_length`, `dry_penalty_last_n` | ✅ | ✅ | ✅ | ✅ |
| `grammar`, `json_schema`, `grammar_lazy` | ✅ | ✅ | ✅ | ✅ |
| `stop` array | ✅ | ✅ | ✅ | ✅ |
| `seed`, `n_probs`, `typical_p`, `top_n_sigma` | ✅ | ✅ | ✅ | ✅ |
| `guide_tokens` (TTS) | ✅ | ✅ | ✅ | ✅ |

### Session Management

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `loadSession` | ✅ `llama_load_session_file` (KV-cache restore) | ✅ `loadSessionNative` → `llama_state_load_file` | ✅ WASM VFS `/tmp/` | ✅ WASM VFS |
| `saveSession` | ✅ `llama_save_session_file` | ✅ `saveSessionNative` → `llama_state_save_file` | ✅ WASM VFS | ✅ WASM VFS |

### Tokenization

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `tokenize` | ✅ `llama_cap_tokenize` | ✅ `tokenizeNative` | ✅ | ✅ |
| `detokenize` | ✅ `llama_cap_detokenize` | ✅ `detokenizeNative` | ✅ | ✅ |

### Embeddings & Reranking

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `embedding` | ✅ `llama_run_embedding_json` | ✅ `embeddingNative` | ✅ | ✅ |
| `rerank` | ✅ `llama_rerank_json` → `completion::rerank()` | ✅ `rerankNative` → `completion::rerank()` | ✅ requires rank-pooling model | ✅ |

### Benchmarking

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `bench` | ✅ `llama_bench` → `completion::bench()` | ✅ `benchNative` → `completion::bench()` | ✅ | ✅ |

### LoRA Adapters

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `applyLoraAdapters` | ✅ `llama_apply_lora_adapters` | ✅ JNI `applyLoraAdaptersNative` → `jni-lora.cpp` | ✅ WASM VFS path mapping | ✅ |
| `removeLoraAdapters` | ✅ `llama_remove_lora_adapters` | ✅ JNI `removeLoraAdaptersNative` | ✅ | ✅ |
| `getLoadedLoraAdapters` | ✅ `llama_get_loaded_lora_adapters` | ✅ JNI `getLoadedLoraAdaptersNative` | ✅ | ✅ |

### Multimodal

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `initMultimodal` | ✅ `llama_init_multimodal` | ✅ JNI `initMultimodalNative` → `jni-multimodal.cpp` | ✅ VFS path | ✅ |
| `isMultimodalEnabled` | ✅ native query | ✅ native query | ✅ | ✅ |
| `getMultimodalSupport` | ✅ real `{vision, audio}` from native | ✅ real `{vision, audio}` from native | ✅ | ✅ |
| `releaseMultimodal` | ✅ | ✅ | ✅ | ✅ |

### TTS / Vocoder

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `initVocoder` | ✅ `llama_init_vocoder` | ✅ JNI `initVocoderNative` → `jni-tts.cpp` | ✅ VFS path | ✅ |
| `isVocoderEnabled` | ✅ native query | ✅ native query | ✅ | ✅ |
| `getFormattedAudioCompletion` | ✅ `llama_get_formatted_audio_completion` | ✅ JNI → `tts_wrapper` | ✅ | ✅ |
| `getAudioCompletionGuideTokens` | ✅ `llama_get_audio_completion_guide_tokens` | ✅ JNI → `tts_wrapper` | ✅ | ✅ |
| `decodeAudioTokens` | ✅ `llama_decode_audio_tokens` (float PCM) | ✅ JNI → `tts_wrapper` (Int16 scaled) | ✅ | ✅ |
| `releaseVocoder` | ✅ | ✅ | ✅ | ✅ |

### Model Download & Management

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `downloadModel` | ✅ `URLSession` streaming + `URLSessionDownloadDelegate` | ✅ `HttpURLConnection` background thread | ✅ OPFS streaming with progress events | ✅ OPFS |
| `getDownloadProgress` | ✅ real byte counters from delegate | ✅ polls native tracker | ✅ in-memory `activeDownloads` map | ✅ |
| `cancelDownload` | ✅ `URLSessionTask.cancel()` + session invalidate | ✅ native cancel | ✅ `AbortController.abort()` | ✅ |
| `getAvailableModels` | ✅ scans Documents + Downloads for `.gguf/.ggml/.bin` | ✅ scans internal + external storage | ✅ OPFS manifest | ✅ |

### Grammar Utilities

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `convertJsonSchemaToGrammar` | ✅ `llama_convert_json_schema_to_grammar` | ✅ `convertJsonSchemaToGrammarNative` | ✅ WASM | ✅ |

### Native HTTP Server

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `startNativeLlamaServer` | ✅ `cap_llama_server_start` | ✅ `startLlamaServerNative` | ❌ throws (TCP not available in browser) | ✅ sidecar `ensureSidecar` |
| `stopNativeLlamaServer` | ✅ | ✅ | ❌ no-op | ✅ |
| `isNativeLlamaServerRunning` | ✅ | ✅ | ❌ always `false` | ✅ |

### Events

| Method | iOS | Android | Web (PWA) | Desktop |
|--------|-----|---------|-----------|---------|
| `addListener` | ✅ | ✅ | ✅ | ✅ |
| `removeAllListeners` | ✅ | ✅ | ✅ | ✅ |

### Coverage Summary

| Platform | Methods fully implemented | Notes |
|----------|--------------------------|-------|
| **iOS** | 47 / 47 | Native Metal framework via `dlopen`; all symbols exposed through `cap-ios-bridge` ABI |
| **Android** | 47 / 47 | JNI wired through to `jni.cpp`, `jni-lora.cpp`, `jni-multimodal.cpp`, `jni-tts.cpp` |
| **Web (PWA)** | 44 / 47 | 3 native server methods correctly throw/return false — TCP binding not available in browsers |
| **Desktop** | 47 / 47 | Core inference via sidecar; auxiliary features (LoRA, TTS, multimodal) via WASM worker |

> The only `❌` entries are `startNativeLlamaServer`, `stopNativeLlamaServer`, and `isNativeLlamaServerRunning` on Web — these are intentionally unsupported because browsers cannot bind TCP sockets. All other methods are fully functional on all platforms.

### Build matrix

Quick reference for building plugin bundles and native artifacts. Run commands from the repo root.

| Target | Host / toolchain | Output | Command |
|--------|------------------|--------|---------|
| **Plugin (TypeScript)** | Node.js (any OS) | `dist/` (ESM, CJS, types) | `npm run build` |
| **iOS** | macOS + Xcode | `ios/Frameworks/llama-cpp.framework` | `npm run build:native` or `./build-native.sh` |
| **iOS XCFramework** | macOS + Xcode | `ios/Frameworks/LlamaCpp.xcframework` | `npm run build:ios:xcframework` |
| **Android** | macOS/Linux + Android NDK | `android/src/main/jniLibs/arm64-v8a/libllama-cpp-arm64.so` | `npm run build:native` |
| **Web / PWA (WASM)** | Emscripten | `dist/wasm/llama_engine.wasm` + worker | `npm run build:pwa` |
| **Web / PWA (pthread)** | Emscripten + COOP/COEP in app | same + pthread build | `npm run build:pwa:full` |
| **Desktop sidecar** | CMake on host OS | `sidecar/bin/<binary>` | see table below |
| **Desktop package staging** | WASM + sidecar | `extraResources/sidecar/` + WASM | `npm run build:desktop` |
| **Publish tarball (JS + native + WASM)** | macOS recommended | `dist/` + `ios/` + `android/` + `dist/wasm/` | `npm run build:package` |

**Desktop sidecar variants** (`SIDECAR_VARIANT` / npm script):

| OS / arch | Variant | Accelerator | Command | Binary name (example) |
|-----------|---------|-------------|---------|------------------------|
| macOS arm64 | `metal-coreml` | Metal + CoreML ANE | `npm run build:sidecar:metal` | `sidecar/bin/darwin-arm64` |
| macOS x64 | `metal` | Metal | `npm run build:sidecar` | `sidecar/bin/darwin-x64` |
| Linux x64 | `vulkan-openblas` | Vulkan + CPU (OpenBLAS) | `npm run build:sidecar:linux` | `sidecar/bin/linux-x64` |
| Windows x64 | `vulkan-openblas` | Vulkan + CPU | `npm run build:sidecar:win` | `sidecar/bin/win32-x64` |
| Linux / Win / macOS | `cuda` | NVIDIA CUDA | `npm run build:sidecar:cuda` | `…-cuda` suffix |
| Linux | `rocm` | AMD ROCm/HIP | `npm run build:sidecar:rocm` | `…-rocm` suffix |
| Any | `cpu` / `openblas` | CPU only | `npm run build:sidecar:cpu` | `…-cpu` or `…-openblas` |

`npm run build:sidecar` auto-picks the default variant for the current host (Metal on macOS, Vulkan on Linux, CPU elsewhere). Pass a variant explicitly: `./scripts/build-sidecar.sh <variant>`.

**Optional env (desktop GPU plugins):**

| Variable | Purpose |
|----------|---------|
| `LLAMA_CPP_UPSTREAM` | Path to upstream `llama.cpp` — also builds `libggml-*` GPU backend plugins bundled next to the sidecar |
| `OPENBLAS_ROOT` | Custom OpenBLAS install for CPU-optimized Linux/Windows builds |

**Verify / clean:**

| Task | Command |
|------|---------|
| Unit tests | `npm run test:unit` |
| Desktop bundle check | `npm run verify:desktop:bundle` |
| Clean all artifacts | `npm run clean:all` |

See also [docs/DESKTOP_ARCHITECTURE.md](docs/DESKTOP_ARCHITECTURE.md) (accelerator matrix) and [cpp/README.md](cpp/README.md) (syncing upstream llama.cpp).

### Desktop (Windows, macOS, Linux)

- **Architecture:** Electron + native sidecar + WASM fallback ([docs/DESKTOP_ARCHITECTURE.md](docs/DESKTOP_ARCHITECTURE.md))
- **Concurrent models:** up to **5** resident GGUF contexts on the sidecar (same as iOS/Android/Web); `DefaultModelScheduler` + OS memory admission in renderer; sidecar enforces limit via `/v1/internal/models/load`
- **Accelerators:** NVIDIA CUDA, AMD ROCm/Radeon, Intel OpenVINO, Apple Metal/CoreML, Vulkan (cross-vendor GPU), CPU (OpenBLAS/Accelerate)
- **Build sidecar:** `npm run build:sidecar` (macOS: `npm run build:sidecar:metal`; GPU: `build:sidecar:cuda` / `build:sidecar:rocm`)
- **Stage for packaging:** `npm run build:desktop` → copies sidecar + WASM to `extraResources/`
- **electron-builder:** merge config from the plugin:

```javascript
// electron-builder.config.js in your Electron app
const llama = require('llama-cpp-pro/desktop/electron-builder');
module.exports = llama.merge({
  appId: 'com.yourapp.id',
  // ...your existing config
});
```

- **Main process:** `registerLlamaDesktopIpc({ ipcMain, app })` from `llama-cpp-pro/desktop`
- **Preload:** `require('llama-cpp-pro/desktop/preload')(contextBridge, ipcRenderer)`
- **Capacitor API:** full `LlamaCppPlugin` surface via `LlamaCppDesktop` (auto-selected in Electron)

### PWA requirements

- **OPFS** for model storage (`navigator.storage.getDirectory`)
- **Dedicated worker** for inference (included in the package)
- **COOP/COEP** optional — enables pthread builds; single-threaded WASM works without it
- Rebuild WASM after updating: `npm run build:pwa:full`

## ✅ **Complete Implementation Status**

This plugin is now **FULLY IMPLEMENTED** with complete native integration of llama.cpp for iOS, Android, and Web/PWA platforms. The implementation includes:

### **Completed Features**

- **Complete C++ Integration**: Full llama.cpp library integration with all core components
- **Isomorphic Architecture**: Unified TypeScript API across iOS, Android, and Web with platform-specific optimizations
- **Native Build System**: CMake-based build system for iOS and Android, plus Rust/WASM compilation for Web
- **Web/PWA Support**: Complete WASM implementation with embedded llama.cpp via Rust FFI (src-rust)
- **Concurrent Model Management**: Support for up to 5 concurrent models with automatic memory admission control
- **Memory Scheduler**: Intelligent memory management across all platforms with per-device optimization
- **Platform Support**: 
  - iOS: arm64, x86_64 with Metal GPU acceleration
  - Android: arm64-v8a, armeabi-v7a, x86, x86_64 with Adreno GPU acceleration
  - Web/PWA: WebAssembly with SIMD acceleration and OPFS storage
- **TypeScript API**: Complete TypeScript interface matching llama.rn functionality with 40+ methods
- **Native Methods**: All 40+ native methods implemented with proper error handling
- **Event System**: Capacitor event system for progress and token streaming
- **Documentation**: Comprehensive README and API documentation

### **Technical Implementation**

- **C++ Core**: Complete llama.cpp library with GGML, GGUF, and all supporting components
- **iOS Framework**: Native iOS framework with Metal acceleration support
- **Android JNI**: Complete JNI implementation with multi-architecture support
- **Web/WASM Engine**: Rust FFI bridge to llama.cpp compiled to WebAssembly
- **Rust Integration**: `src-rust/` directory contains Rust FFI layer for WASM compilation
- **Build Scripts**: Automated build system for all three platforms
- **Error Handling**: Robust error handling and result types
- **Memory Management**: Automatic admission control with platform-specific tuning

### **Project Structure**

```
llama-cpp/
├── cpp/                    # Complete llama.cpp C++ library
│   ├── ggml.c             # GGML core
│   ├── gguf.cpp           # GGUF format support
│   ├── llama.cpp          # Main llama.cpp implementation
│   ├── rn-llama.cpp       # React Native wrapper (adapted)
│   ├── rn-completion.cpp  # Completion handling
│   ├── rn-tts.cpp         # Text-to-speech
│   └── tools/mtmd/        # Multimodal support
├── src-rust/               # Rust FFI layer for WASM
│   ├── src/
│   │   ├── lib.rs         # Main WASM exports
│   │   ├── ffi.rs         # FFI bridge to C/C++
│   │   ├── engine.rs      # State management
│   │   ├── model.rs       # Model registry & scheduler
│   │   ├── memory.rs      # Memory admission control
│   │   └── stream.rs      # Streaming support
│   ├── Cargo.toml         # Rust dependencies
│   └── build.rs           # C/C++ compilation script
├── ios/
│   ├── CMakeLists.txt     # iOS build configuration
│   └── Sources/           # Swift implementation
├── android/
│   ├── src/main/
│   │   ├── CMakeLists.txt # Android build configuration
│   │   ├── jni.cpp        # JNI implementation
│   │   └── jni-utils.h    # JNI utilities
│   └── build.gradle       # Android build config
├── src/
│   ├── definitions.ts     # Complete TypeScript interfaces
│   ├── index.ts           # Main plugin implementation
│   ├── web.ts             # Web/PWA implementation
│   ├── isomorphic/        # Platform-specific providers
│   │   ├── provider.web.ts      # Web provider with scheduler
│   │   └── wasmMemoryPolicy.ts  # Memory admission control
│   └── workers/           # Web Worker implementation
│       └── llm.worker.ts  # WASM worker with model registry
└── build-variants.sh      # Automated build script
```

### **Web/PWA Architecture**

The Web implementation features:
- **Rust WASM Engine** (`src-rust/`): Bridges JavaScript to C++ llama.cpp via FFI
- **Model Registry**: Maintains up to 5 concurrent models with automatic management
- **Memory Scheduler**: `DefaultModelScheduler` enforces 1.5 GB ceiling with 64 MB reserve
- **Web Worker**: Runs inference off main thread for responsive UI
- **OPFS Storage**: Efficient model storage using Origin Private File System
- **Streaming Support**: Real-time token generation with event callbacks

### **Concurrent Model Management**

All platforms support simultaneous operation of multiple models:

| Platform | Max Concurrent Models | Memory Management | Acceleration |
|----------|----------------------|-------------------|--------------|
| **Web/WASM** | 5 models | DefaultModelScheduler (1.5 GB pool, 64 MB reserve) | WASM SIMD |
| **Android** | 5 contexts | ModelAdmissionController (device RAM, 512 MB reserve) | Adreno GPU |
| **iOS** | 5 contexts | ModelAdmissionController (process memory, 512 MB reserve) | Metal GPU |

**Example: Load 5 Models Concurrently**
```typescript
const models = await Promise.all([
  initLlama({ modelPath: 'llama-7b.gguf', contextId: 0 }),
  initLlama({ modelPath: 'mistral-7b.gguf', contextId: 1 }),
  initLlama({ modelPath: 'tinyllama-1b.gguf', contextId: 2 }),
  initLlama({ modelPath: 'bge-small.gguf', contextId: 3, embedding: true }),
  initLlama({ modelPath: 'neural-vocoder.gguf', contextId: 4 })
]);
// All 5 run simultaneously with automatic memory management
```

## 📦 Installation

```sh
npm install llama-cpp-pro
```

**Package Size:** ~23-25 MB (minimal variant, no C++ sources, debug symbols stripped)
- Includes: iOS arm64 + Android arm64 + PWA/WASM + TypeScript
- For full build system details and variants, see [BUILD_GUIDE.md](BUILD_GUIDE.md)

## 🔨 **Building the Native Library**

The plugin includes a complete native implementation of llama.cpp. To build the native libraries:

### **Quick Start**

```bash
# Complete production build (iOS + Android + PWA/WASM)
npm run clean:all
./build-variants.sh --variant minimal
npm publish
```

### **Build Variants**

For complete build system documentation including all variants (minimal, core, development, ios-only, android-only, full), size optimization, and release procedures, see **[BUILD_GUIDE.md](BUILD_GUIDE.md)** and **[README_BUILD_SYSTEM.md](README_BUILD_SYSTEM.md)**.

```bash
# Production variant (~20 MB, no sources, stripped symbols)
./build-variants.sh --variant minimal

# Production with rebuild flexibility (~30-35 MB, with C++ sources)
./build-variants.sh --variant core

# Development variant (~50-60 MB, all architectures, debug symbols)
./build-variants.sh --variant development

# Platform-specific variants
./build-variants.sh --variant ios-only
./build-variants.sh --variant android-only
```

### **Prerequisites**

- **CMake** (3.16+ for iOS, 3.10+ for Android)
- **Xcode** (for iOS builds, macOS only)
- **Android Studio** with NDK (for Android builds)
- **Make** or **Ninja** build system

### **Automated Build**

```bash
# Build for all platforms (minimal variant)
./build-variants.sh --variant minimal

# Build TypeScript (included in variant builds)
npm run build

# Verify all artifacts exist
npm run verify:pack:artifacts

# Clean specific components
npm run clean          # Clean TypeScript dist
npm run clean:native  # Clean iOS/Android builds
npm run clean:wasm    # Clean WASM artifacts
npm run clean:all     # Clean everything
```

### **Manual Build**

#### **iOS Build**
```bash
cd ios
cmake -B build -S .
cmake --build build --config Release
```

#### **Android Build**
```bash
cd android
./gradlew assembleRelease
```

### **Web/WASM Build**

The Web implementation includes embedded llama.cpp compiled to WebAssembly via Rust FFI:

```bash
# Build Wasm with real llama.cpp (full build, ~30-120 seconds)
LLAMA_WASM_EMBED_CPP=1 npm run build:wasm:embed

# Copy WASM artifacts to dist/
npm run build:wasm:assets

# Run PWA smoke tests
npm run test:pwa:smoke
```

**Build Process:**
1. `src-rust/build.rs` compiles all C/C++ sources from `cpp/` directory
2. Rust code in `src-rust/src/` links against compiled llama.cpp
3. `wasm-pack` generates WebAssembly binary and JavaScript glue code
4. Final WASM module (~2-5 MB) includes complete llama.cpp inference engine
5. `wasm-bindgen` creates TypeScript definitions for type-safe bindings

**Build Outputs:**
- `dist/wasm/llama_engine.wasm` (~2-5 MB) - WebAssembly binary with embedded llama.cpp
- `dist/wasm/llama_engine.js` - wasm-bindgen JavaScript wrapper
- `dist/wasm/llama_engine.d.ts` - TypeScript type definitions

For detailed WASM build instructions, see [BUILD_GUIDE.md](BUILD_GUIDE.md).

### **Build Output**

- **iOS**: `ios/Frameworks/llama-cpp.framework/` (3-4 MB arm64 stripped)
- **Android**: `android/src/main/jniLibs/arm64-v8a/libllama-cpp-arm64.so` (15-16 MB stripped)
- **WASM**: `dist/wasm/llama_engine_emscripten.wasm` (~2-3 MB, included in builds)
- **TypeScript**: `dist/esm/`, `dist/plugin.cjs.js` (1-2 MB)

### **Updating the native source (e.g. for vision model support)**

The native `cpp/` layer is based on [llama.cpp](https://github.com/ggerganov/llama.cpp). To pull in a newer upstream version (e.g. for vision model support) **without overwriting** the Capacitor adapter code, use the **bootstrap script** (included in this repo):

```bash
./scripts/bootstrap.sh [branch-or-tag-or-commit]
# Example: ./scripts/bootstrap.sh master
```

This syncs upstream into `cpp/` and keeps project-specific files (`cap-*.cpp/h`, `tools/mtmd/`, etc.) intact. After running it, reconcile any API changes in the adapter code, then rebuild with `npm run build:native` or `./build-native.sh`. See [cpp/README.md](cpp/README.md) and [docs/IOS_IMPLEMENTATION_GUIDE.md](docs/IOS_IMPLEMENTATION_GUIDE.md).

### **iOS implementation (step-by-step)**

For a **step-by-step guide** on how methods are implemented on the iOS side (Swift bridge → native framework, adding/updating C symbols, and updating the native layer for vision), see **[docs/IOS_IMPLEMENTATION_GUIDE.md](docs/IOS_IMPLEMENTATION_GUIDE.md)**.

### iOS Setup

The iOS native framework is built as part of the build process and included in the package.

1. Install the plugin:
```sh
npm install llama-cpp-pro
```

2. Add to your iOS project:
```sh
npx cap add ios
npx cap sync ios
```

**Simulator builds** (macOS + Xcode + cmake): from your **Capacitor app root** (after `npm install`), build device+simulator slices once, then re-sync. Use the script shipped inside the plugin — do not copy it into your app:

```sh
bash node_modules/llama-cpp-pro/scripts/ensure-llama-ios-xcframework.sh
npx cap sync ios
```

When developing the plugin itself (git checkout): `npm run build:ios:xcframework`

3. Open the project in Xcode:
```sh
npx cap open ios
```

The plugin ships a pre-built **device** framework (`ios/Frameworks/llama-cpp.framework`, arm64, Metal + CPU fallback). `Package.swift` is Swift-only; the native binary is loaded at runtime via `dlopen`. For simulator slices, run the xcframework script above. See [BUILD_GUIDE.md](BUILD_GUIDE.md) for local rebuilds.

### Android Setup

The Android native library is built as part of the build process and included in the package.

1. Install the plugin:
```sh
npm install llama-cpp-pro
```

2. Add to your Android project:
```sh
npx cap add android
npx cap sync android
```

3. Open the project in Android Studio:
```sh
npx cap open android
```

The plugin includes pre-built Android libraries (`android/src/main/jniLibs/arm64-v8a/`) for arm64 devices. For details on building locally or adding additional architectures, see [BUILD_GUIDE.md](BUILD_GUIDE.md).

### Web/PWA Setup

The plugin includes complete Web/PWA support with WASM acceleration. For web applications:

```typescript
import { initLlama, LlamaContext } from 'llama-cpp-pro';

// Works seamlessly on Web/PWA without platform detection
const context = await initLlama({
  modelPath: '/models/model.gguf',
  n_ctx: 2048,
});

const result = await context.completion({
  prompt: 'Hello, how are you?',
  n_predict: 100,
});
```

**Web Features:**
- **WASM Inference**: Rust FFI layer (`src-rust/`) for llama.cpp compilation to WebAssembly
- **Model Registry**: Manage up to 5 concurrent models with automatic memory scheduling
- **Memory Admission Control**: Smart allocation with per-device optimization
- **Web Workers**: Runs inference off main thread for responsive UI
- **OPFS Storage**: Efficient model caching using Origin Private File System
- **Streaming Support**: Real-time token generation
- **SIMD Acceleration**: WebAssembly SIMD for faster inference

For complete Web/PWA build instructions, see [BUILD_GUIDE.md](BUILD_GUIDE.md).

## 🎯 Quick Start

### Basic Text Completion

```typescript
import { initLlama } from 'llama-cpp';

// Initialize a model
const context = await initLlama({
  model: '/path/to/your/model.gguf',
  n_ctx: 2048,
  n_threads: 4,
  n_gpu_layers: 0,
});

// Generate text
const result = await context.completion({
  prompt: "Hello, how are you today?",
  n_predict: 50,
  temperature: 0.8,
});

console.log('Generated text:', result.text);
```

### Chat-Style Conversations

```typescript
const result = await context.completion({
  messages: [
    { role: "system", content: "You are a helpful AI assistant." },
    { role: "user", content: "What is the capital of France?" },
    { role: "assistant", content: "The capital of France is Paris." },
    { role: "user", content: "Tell me more about it." }
  ],
  n_predict: 100,
  temperature: 0.7,
});

console.log('Chat response:', result.content);
```

### Streaming Completion

```typescript
let fullText = '';
const result = await context.completion({
  prompt: "Write a short story about a robot learning to paint:",
  n_predict: 150,
  temperature: 0.8,
}, (tokenData) => {
  // Called for each token as it's generated
  fullText += tokenData.token;
  console.log('Token:', tokenData.token);
});

console.log('Final result:', result.text);
```

## 🚀 **Mobile-Optimized Speculative Decoding**

**Achieve 2-8x faster inference with significantly reduced battery consumption!**

Speculative decoding uses a smaller "draft" model to predict multiple tokens ahead, which are then verified by the main model. This results in dramatic speedups with identical output quality.

### Basic Usage

```typescript
import { initLlama } from 'llama-cpp-pro';

// Initialize with speculative decoding
const context = await initLlama({
  model: '/path/to/your/main-model.gguf',         // Main model (e.g., 7B)
  draft_model: '/path/to/your/draft-model.gguf', // Draft model (e.g., 1.5B)
  
  // Speculative decoding parameters
  speculative_samples: 3,      // Number of tokens to predict speculatively
  mobile_speculative: true,    // Enable mobile optimizations
  
  // Standard parameters
  n_ctx: 2048,
  n_threads: 4,
});

// Use normally - speculative decoding is automatic
const result = await context.completion({
  prompt: "Write a story about AI:",
  n_predict: 200,
  temperature: 0.7,
});

console.log('🚀 Generated with speculative decoding:', result.text);
```

### Mobile-Optimized Configuration

```typescript
// Recommended mobile setup for best performance/battery balance
const mobileContext = await initLlama({
  // Quantized models for mobile efficiency
  model: '/models/llama-2-7b-chat.q4_0.gguf',
  draft_model: '/models/tinyllama-1.1b-chat.q4_0.gguf',
  
  // Conservative mobile settings
  n_ctx: 1024,                 // Smaller context for mobile
  n_threads: 3,                // Conservative threading
  n_batch: 64,                 // Smaller batch size
  n_gpu_layers: 24,            // Utilize mobile GPU
  
  // Optimized speculative decoding
  speculative_samples: 3,      // 2-3 tokens ideal for mobile
  mobile_speculative: true,    // Enables mobile-specific optimizations
  
  // Memory optimizations
  use_mmap: true,              // Memory mapping for efficiency
  use_mlock: false,            // Don't lock memory on mobile
});
```

### Performance Benefits

- **2-8x faster inference** - Dramatically reduced time to generate text
- **50-80% battery savings** - Less time computing = longer battery life
- **Identical output quality** - Same text quality as regular decoding
- **Automatic fallback** - Falls back to regular decoding if draft model fails
- **Mobile optimized** - Specifically tuned for mobile device constraints

### Model Recommendations

| Model Type | Recommended Size | Quantization | Example |
|------------|------------------|--------------|---------|
| **Main Model** | 3-7B parameters | Q4_0 or Q4_1 | `llama-2-7b-chat.q4_0.gguf` |
| **Draft Model** | 1-1.5B parameters | Q4_0 | `tinyllama-1.1b-chat.q4_0.gguf` |

### Error Handling & Fallback

```typescript
// Robust setup with automatic fallback
try {
  const context = await initLlama({
    model: '/models/main-model.gguf',
    draft_model: '/models/draft-model.gguf',
    speculative_samples: 3,
    mobile_speculative: true,
  });
  console.log('✅ Speculative decoding enabled');
} catch (error) {
  console.warn('⚠️ Falling back to regular decoding');
  const context = await initLlama({
    model: '/models/main-model.gguf',
    // No draft_model = regular decoding
  });
}
```

## 📚 API Reference

### Core Functions

#### `initLlama(params: ContextParams, onProgress?: (progress: number) => void): Promise<LlamaContext>`

Initialize a new llama.cpp context with a model.

**Parameters:**
- `params`: Context initialization parameters
- `onProgress`: Optional progress callback (0-100)

**Returns:** Promise resolving to a `LlamaContext` instance

#### `releaseAllLlama(): Promise<void>`

Release all contexts and free memory.

#### `toggleNativeLog(enabled: boolean): Promise<void>`

Enable or disable native logging.

#### `addNativeLogListener(listener: (level: string, text: string) => void): { remove: () => void }`

Add a listener for native log messages.

### LlamaContext Class

#### `completion(params: CompletionParams, callback?: (data: TokenData) => void): Promise<NativeCompletionResult>`

Generate text completion.

**Parameters:**
- `params`: Completion parameters including prompt or messages
- `callback`: Optional callback for token-by-token streaming

#### `tokenize(text: string, options?: { media_paths?: string[] }): Promise<NativeTokenizeResult>`

Tokenize text or text with images.

#### `detokenize(tokens: number[]): Promise<string>`

Convert tokens back to text.

#### `embedding(text: string, params?: EmbeddingParams): Promise<NativeEmbeddingResult>`

Generate embeddings for text.

#### `rerank(query: string, documents: string[], params?: RerankParams): Promise<RerankResult[]>`

Rank documents by relevance to a query.

#### `bench(pp: number, tg: number, pl: number, nr: number): Promise<BenchResult>`

Benchmark model performance.

### Multimodal Support

#### `initMultimodal(params: { path: string; use_gpu?: boolean }): Promise<boolean>`

Initialize multimodal support with a projector file.

#### `isMultimodalEnabled(): Promise<boolean>`

Check if multimodal support is enabled.

#### `getMultimodalSupport(): Promise<{ vision: boolean; audio: boolean }>`

Get multimodal capabilities.

#### `releaseMultimodal(): Promise<void>`

Release multimodal resources.

### TTS (Text-to-Speech)

#### `initVocoder(params: { path: string; n_batch?: number }): Promise<boolean>`

Initialize TTS with a vocoder model.

#### `isVocoderEnabled(): Promise<boolean>`

Check if TTS is enabled.

#### `getFormattedAudioCompletion(speaker: object | null, textToSpeak: string): Promise<{ prompt: string; grammar?: string }>`

Get formatted audio completion prompt.

#### `getAudioCompletionGuideTokens(textToSpeak: string): Promise<Array<number>>`

Get guide tokens for audio completion.

#### `decodeAudioTokens(tokens: number[]): Promise<Array<number>>`

Decode audio tokens to audio data.

#### `releaseVocoder(): Promise<void>`

Release TTS resources.

### LoRA Adapters

#### `applyLoraAdapters(loraList: Array<{ path: string; scaled?: number }>): Promise<void>`

Apply LoRA adapters to the model.

#### `removeLoraAdapters(): Promise<void>`

Remove all LoRA adapters.

#### `getLoadedLoraAdapters(): Promise<Array<{ path: string; scaled?: number }>>`

Get list of loaded LoRA adapters.

### Session Management

#### `saveSession(filepath: string, options?: { tokenSize: number }): Promise<number>`

Save current session to a file.

#### `loadSession(filepath: string): Promise<NativeSessionLoadResult>`

Load session from a file.

## 🔧 Configuration

### Context Parameters

```typescript
interface ContextParams {
  model: string;                    // Path to GGUF model file
  n_ctx?: number;                   // Context size (default: 512)
  n_threads?: number;               // Number of threads (default: 4)
  n_gpu_layers?: number;            // GPU layers (iOS only)
  use_mlock?: boolean;              // Lock memory (default: false)
  use_mmap?: boolean;               // Use memory mapping (default: true)
  embedding?: boolean;              // Embedding mode (default: false)
  cache_type_k?: string;            // KV cache type for K
  cache_type_v?: string;            // KV cache type for V
  pooling_type?: string;            // Pooling type
  // ... more parameters
}
```

### Completion Parameters

```typescript
interface CompletionParams {
  prompt?: string;                  // Text prompt
  messages?: Message[];             // Chat messages
  n_predict?: number;               // Max tokens to generate
  temperature?: number;             // Sampling temperature
  top_p?: number;                   // Top-p sampling
  top_k?: number;                   // Top-k sampling
  stop?: string[];                  // Stop sequences
  // ... more parameters
}
```

## 🎨 Advanced Examples

### Multimodal Processing

```typescript
// Initialize multimodal support
await context.initMultimodal({
  path: '/path/to/mmproj.gguf',
  use_gpu: true,
});

// Process image with text
const result = await context.completion({
  messages: [
    { 
      role: "user", 
      content: [
        { type: "text", text: "What do you see in this image?" },
        { type: "image_url", image_url: { url: "file:///path/to/image.jpg" } }
      ]
    }
  ],
  n_predict: 100,
});

console.log('Image analysis:', result.content);
```

### Text-to-Speech

```typescript
// Initialize TTS
await context.initVocoder({
  path: '/path/to/vocoder.gguf',
  n_batch: 512,
});

// Generate audio
const audioCompletion = await context.getFormattedAudioCompletion(
  null, // Speaker configuration
  "Hello, this is a test of text-to-speech functionality."
);

const guideTokens = await context.getAudioCompletionGuideTokens(
  "Hello, this is a test of text-to-speech functionality."
);

const audioResult = await context.completion({
  prompt: audioCompletion.prompt,
  grammar: audioCompletion.grammar,
  guide_tokens: guideTokens,
  n_predict: 1000,
});

const audioData = await context.decodeAudioTokens(audioResult.audio_tokens);
```

### LoRA Adapters

```typescript
// Apply LoRA adapters
await context.applyLoraAdapters([
  { path: '/path/to/adapter1.gguf', scaled: 1.0 },
  { path: '/path/to/adapter2.gguf', scaled: 0.5 }
]);

// Check loaded adapters
const adapters = await context.getLoadedLoraAdapters();
console.log('Loaded adapters:', adapters);

// Generate with adapters
const result = await context.completion({
  prompt: "Test prompt with LoRA adapters:",
  n_predict: 50,
});

// Remove adapters
await context.removeLoraAdapters();
```

### Structured Output

#### JSON Schema (Auto-converted to GBNF)
```typescript
const result = await context.completion({
  prompt: "Generate a JSON object with a person's name, age, and favorite color:",
  n_predict: 100,
  response_format: {
    type: 'json_schema',
    json_schema: {
      strict: true,
      schema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          age: { type: 'number' },
          favorite_color: { type: 'string' }
        },
        required: ['name', 'age', 'favorite_color']
      }
    }
  }
});

console.log('Structured output:', result.content);
```

#### Direct GBNF Grammar
```typescript
// Define GBNF grammar directly for maximum control
const grammar = `
root ::= "{" ws name_field "," ws age_field "," ws color_field "}"
name_field ::= "\\"name\\"" ws ":" ws string_value
age_field ::= "\\"age\\"" ws ":" ws number_value  
color_field ::= "\\"favorite_color\\"" ws ":" ws string_value
string_value ::= "\\"" [a-zA-Z ]+ "\\""
number_value ::= [0-9]+
ws ::= [ \\t\\n]*
`;

const result = await context.completion({
  prompt: "Generate a person's profile:",
  grammar: grammar,
  n_predict: 100
});

console.log('Grammar-constrained output:', result.text);
```

#### Manual JSON Schema to GBNF Conversion
```typescript
import { convertJsonSchemaToGrammar } from 'llama-cpp-pro';

const schema = {
  type: 'object',
  properties: {
    name: { type: 'string' },
    age: { type: 'number' }
  },
  required: ['name', 'age']
};

// Convert schema to GBNF grammar
const grammar = await convertJsonSchemaToGrammar(schema);
console.log('Generated grammar:', grammar);

const result = await context.completion({
  prompt: "Generate a person:",
  grammar: grammar,
  n_predict: 100
});
```

## 🔍 Model Compatibility

This plugin supports GGUF format models, which are compatible with llama.cpp. You can find GGUF models on Hugging Face by searching for the "GGUF" tag.

### Recommended Models

- **Llama 2**: Meta's latest language model
- **Mistral**: High-performance open model
- **Code Llama**: Specialized for code generation
- **Phi-2**: Microsoft's efficient model
- **Gemma**: Google's open model

### Model Quantization

For mobile devices, consider using quantized models (Q4_K_M, Q5_K_M, etc.) to reduce memory usage and improve performance.

## ⚡ Performance Considerations

### Memory Management

- Use quantized models for better memory efficiency
- Adjust `n_ctx` based on your use case
- Monitor memory usage with `use_mlock: false`

### GPU Acceleration

- iOS: Set `n_gpu_layers` to use Metal GPU acceleration
- Android: GPU acceleration is automatically enabled when available

### Threading

- Adjust `n_threads` based on device capabilities
- More threads may improve performance but increase memory usage

## 🐛 Troubleshooting

### Common Issues

1. **Model not found**: Ensure the model path is correct and the file exists
2. **Out of memory**: Try using a quantized model or reducing `n_ctx`
3. **Slow performance**: Enable GPU acceleration or increase `n_threads`
4. **Multimodal not working**: Ensure the mmproj file is compatible with your model

### Debugging

Enable native logging to see detailed information:

```typescript
import { toggleNativeLog, addNativeLogListener } from 'llama-cpp';

await toggleNativeLog(true);

const logListener = addNativeLogListener((level, text) => {
  console.log(`[${level}] ${text}`);
});
```

## 📦 Publishing

To publish the package to npm:

1. **Build** (runs automatically on `npm publish` via `prepublishOnly`): `npm run build` — produces `dist/` (plugin bundles, ESM, docs).
2. **Optional — include native libs in the tarball**: `npm run build:all` (requires macOS/NDK) — builds iOS framework and Android `.so` into `ios/Frameworks` and `android/src/main/jniLibs`.
3. **Verify pack**: `npm run pack` (JS only) or `npm run pack:full` (JS + native) — lists files that would be published.
4. **Publish**: `npm publish`.


---

## Document Info

- **Package:** llama-cpp-pro
- **Version:** 0.2.2
- **Updated:** 2025-07-07
- **Covers:** iOS, Android, Web/PWA, Desktop (Electron + sidecar)