v5.0.0 — Lossless Context Management

Process Unbounded Context
with Recursive LLMs

TypeScript/Go hybrid that recursively decomposes massive documents for LLM processing. Extract structured data from 10M+ tokens without degradation.

$ npm install recursive-llm-ts
84.8%
Context Savings
131KB
Package Size
10M+
Tokens
200+
Tests
0
Python Deps

Built for Production

Everything you need to process massive contexts with LLMs, from a single npm package.

⚡

Pure Go Backend

No Python runtime required. The native Go binary handles recursive decomposition, REPL execution, and LLM orchestration with minimal overhead.

🎯

Structured Outputs

Extract typed data using Zod schemas. Supports nested objects, arrays, enums, and constraints with automatic validation and retry logic.

🚀

Parallel Execution

Complex schemas are automatically decomposed and processed in parallel with goroutines. 3-5x faster extraction for multi-field schemas.

👁

Observability

Built-in OpenTelemetry tracing, Langfuse integration, and debug logging. Track every LLM call, REPL execution, and recursive step.

📁

File Storage Context

Process local directories or S3-compatible buckets as LLM context. Recursive traversal, filtering, and automatic context formatting.

🧠

Meta-Agent

Automatically optimize vague queries before processing. The meta-agent rewrites queries for better recursive decomposition.

🌐

Provider Agnostic

Works with OpenAI, Anthropic, Azure, Bedrock, Ollama, and any OpenAI-compatible API. Use LiteLLM proxy for 100+ providers.

📦

Simple Distribution

Single npm package with auto-installed platform binaries on supported targets, source-build fallback, and Docker-ready Linux container workflows.

⚙

Streaming

Async iterable streaming for real-time token delivery. Stream text or structured objects with AbortController support and backpressure handling.

💾

Caching

Built-in memory (LRU+TTL) and file-based cache backends. SHA-256 key generation, hit/miss stats, and automatic cache invalidation.

🛡

Retry & Resilience

Exponential, linear, and fixed backoff with jitter. Rate-limit aware retries, multi-provider fallback chains, and abort signal support.

⚠

Typed Errors

9 structured error classes with error codes, retryability flags, and actionable suggestions. Auto-classification from raw errors via classifyError().

🛠

Context Overflow Recovery

Automatic recovery when context exceeds model limits. 6 strategies: MapReduce, Truncate, Chunked, TF-IDF, TextRank, and Refine. Works with any model size.

🔒

Lossless Context Management

Immutable Store + Summary DAG ensures no context is ever lost. Soft/hard token thresholds with automatic compaction. Zero overhead below soft threshold.

🎯

BPE Tokenizer

Model-specific token counting via tiktoken (o200k_base for GPT-4o, cl100k_base for GPT-4). Cached counting with xxhash. Heuristic fallback for unknown models.

📚

Episodic Memory

Group messages into coherent episodes with auto-rotation, compaction, and budget-based retrieval. Parent chaining preserves conversation continuity.

⚙

LLM-Map & Agentic-Map

Parallel batch processing with schema validation per item. Agentic-Map spawns full sub-agent sessions. Worker pools with configurable concurrency.

💾

SQLite Persistence

Pure Go SQLite with FTS5 full-text search, WAL mode, and transactional writes. In-memory backend for zero-dependency usage. Indexed lookups by ID, role, timestamp.

🛡

Delegation Guard

Infinite recursion prevention with scope-reduction invariant. Sub-agents must declare delegated_scope and kept_work. Trivial delegation is rejected.

How It Works

Based on the 2025 MIT paper "Recursive Language Models" by Alex Zhang and Omar Khattab, RLM solves the fundamental context length limitation of LLMs.

Instead of truncating or failing on large documents, RLM recursively decomposes the problem:

  • Large context is split into manageable chunks
  • LLM writes JavaScript code to analyze each chunk
  • Code is executed in a sandboxed REPL (goja engine)
  • Results are recursively combined up the tree
  • For structured outputs, schemas are decomposed into subtasks
  • Parallel goroutines process independent fields simultaneously
  • Validation with retry ensures schema compliance
TypeScript API
RLM class, Zod schema conversion, file context building
↓ spawn binary, JSON via stdin
Go Binary
Recursive engine, JS REPL, meta-agent, observability
↓ OpenAI-compatible API
LLM Provider
OpenAI, Anthropic, Azure, Bedrock, local models
↑ JSON response + trace events
Typed Result
Structured data, stats, trace events

Proven Results

Reproducible benchmarks from the test suite. No LLM calls required — run them yourself.

84.8%
Context Savings
131KB
Package Size
5-Level
Summarization
BPE
Token Accuracy

Context Savings Pipeline

100 messages through episodic grouping + TF-IDF compaction:

StageTokensSavings
Raw messages50,491—
After TF-IDF compaction7,66984.8%
After budget selection (8K)7,66984.8%

Five-Level Summarization

5K token input → 2K token target (non-LLM levels):

LevelStrategyReduction
Level 3TF-IDF59.7%
Level 4TextRank60.0%
Level 5Truncate65.1%
Run benchmarks yourself

cd go && go test ./rlm/ -run "TestContextSavings_" -v

Quick Start

Get up and running in under a minute.

TypeScript
import { RLM } from 'recursive-llm-ts';

const rlm = new RLM('gpt-4o-mini', {
  api_key: process.env.OPENAI_API_KEY,
  max_iterations: 15,
});

// Process a 100K+ token document
const result = await rlm.completion(
  'What are the key findings in this research paper?',
  longResearchPaper
);

console.log(result.result);
console.log('Stats:', result.stats);
TypeScript
import { RLM } from 'recursive-llm-ts';
import { z } from 'zod';

const rlm = new RLM('gpt-4o-mini', {
  api_key: process.env.OPENAI_API_KEY,
});

const schema = z.object({
  sentiment: z.number().min(1).max(5),
  summary: z.string(),
  topics: z.array(z.enum(['pricing', 'features', 'support'])),
  keyPhrases: z.array(z.object({
    phrase: z.string(),
    weight: z.number(),
  })),
});

const result = await rlm.structuredCompletion(
  'Analyze the sentiment and extract key information',
  callTranscript,
  schema,
  { parallelExecution: true }
);

// result.result is fully typed!
console.log(result.result.sentiment);     // number
console.log(result.result.keyPhrases);    // Array<{phrase, weight}>
TypeScript
import { RLM } from 'recursive-llm-ts';

const rlm = new RLM('gpt-4o-mini', {
  api_key: process.env.OPENAI_API_KEY,
});

// Local files
const result = await rlm.completionFromFiles(
  'Summarize the architecture of this codebase',
  {
    type: 'local',
    path: '/path/to/project/src',
    extensions: ['.ts', '.tsx'],
    excludePatterns: ['*.test.ts'],
    maxFileSize: 100_000,
  }
);

// S3 bucket
const s3Result = await rlm.completionFromFiles(
  'What are the key findings?',
  {
    type: 's3',
    path: 'my-bucket',
    prefix: 'reports/2024/',
    extensions: ['.md', '.txt'],
    region: 'us-west-2',
  }
);
TypeScript
import { RLM } from 'recursive-llm-ts';

const rlm = new RLM('gpt-4o-mini', {
  api_key: process.env.OPENAI_API_KEY,

  // Meta-agent optimizes vague queries
  meta_agent: {
    enabled: true,
    model: 'gpt-4o',
  },

  // Full observability
  observability: {
    debug: true,
    trace_enabled: true,
    trace_endpoint: 'http://localhost:4318',
    langfuse_enabled: true,
    langfuse_public_key: process.env.LANGFUSE_PUBLIC_KEY,
    langfuse_secret_key: process.env.LANGFUSE_SECRET_KEY,
  }
});

const result = await rlm.completion('what happened?', transcript);

// Access trace events
const events = rlm.getTraceEvents();
for (const e of events) {
  console.log(`[${e.type}] ${e.name}`, e.attributes);
}
TypeScript
import { RLM } from 'recursive-llm-ts';

const rlm = new RLM('gpt-4o-mini', {
  api_key: process.env.OPENAI_API_KEY,
});

// Stream text tokens in real-time
const stream = await rlm.streamCompletion(
  'Summarize this document',
  longDocument
);

for await (const chunk of stream) {
  if (chunk.type === 'text') {
    process.stdout.write(chunk.text);
  }
}

// Or collect all text at once
const stream2 = await rlm.streamCompletion('...', doc);
const fullText = await stream2.toText();

// Stream structured objects
const objStream = await rlm.streamStructuredCompletion(
  'Extract data', context, schema
);
const result = await objStream.toObject();
TypeScript
import { RLM } from 'recursive-llm-ts';

// Fluent builder API
const rlm = RLM.builder()
  .model('gpt-4o-mini')
  .apiKey(process.env.OPENAI_API_KEY!)
  .maxDepth(5)
  .maxIterations(20)
  .enableMetaAgent('gpt-4o')
  .enableDebug()
  .enableCache({ maxEntries: 500, ttl: 3600000 })
  .enableRetry({ maxRetries: 3, backoff: 'exponential' })
  .build();

// Factory methods
const rlm2 = RLM.fromEnv();          // Auto-detect from env vars
const rlm3 = RLM.withDebug('gpt-4o'); // Quick debug setup
const rlm4 = RLM.forAzure(            // Azure-specific setup
  'my-deployment',
  'https://my.openai.azure.com',
  process.env.AZURE_API_KEY!
);

// Config validation
const { valid, issues } = rlm.validate();
for (const issue of issues) {
  console.log(`[${issue.level}] ${issue.field}: ${issue.message}`);
}
Go
package main

import (
    "fmt"
    "os"

    "github.com/howlerops/recursive-llm-ts/go/rlm"
)

func main() {
    engine := rlm.New("gpt-4o-mini", rlm.Config{
        MaxDepth:      5,
        MaxIterations: 30,
        APIKey:        os.Getenv("OPENAI_API_KEY"),
        MetaAgent: &rlm.MetaAgentConfig{
            Enabled: true,
        },
        Observability: &rlm.ObservabilityConfig{
            Debug: true,
        },
    })

    answer, stats, err := engine.Completion(
        "Summarize this document",
        largeDocument,
    )
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error: %v\n", err)
        os.Exit(1)
    }

    fmt.Printf("Answer: %s\n", answer)
    fmt.Printf("LLM calls: %d\n", stats.LlmCalls)
}

Use as a Go Module

The Go implementation is also available as a standalone library for Go projects. Import it directly for maximum performance without the TypeScript layer.

  • Full RLM engine with recursive decomposition
  • Structured output with JSON Schema validation
  • Meta-agent query optimization
  • OpenTelemetry and Langfuse observability
  • JavaScript REPL via goja engine
  • Cross-compile for any platform
Bash
go get github.com/howlerops/recursive-llm-ts/go

Ready to process unbounded context?

Install the package, set your API key, and start processing documents of any size.