Process Unbounded Context
with Recursive LLMs
TypeScript/Go hybrid that recursively decomposes massive documents for LLM processing. Extract structured data from 10M+ tokens without degradation.
Built for Production
Everything you need to process massive contexts with LLMs, from a single npm package.
Pure Go Backend
No Python runtime required. The native Go binary handles recursive decomposition, REPL execution, and LLM orchestration with minimal overhead.
Structured Outputs
Extract typed data using Zod schemas. Supports nested objects, arrays, enums, and constraints with automatic validation and retry logic.
Parallel Execution
Complex schemas are automatically decomposed and processed in parallel with goroutines. 3-5x faster extraction for multi-field schemas.
Observability
Built-in OpenTelemetry tracing, Langfuse integration, and debug logging. Track every LLM call, REPL execution, and recursive step.
File Storage Context
Process local directories or S3-compatible buckets as LLM context. Recursive traversal, filtering, and automatic context formatting.
Meta-Agent
Automatically optimize vague queries before processing. The meta-agent rewrites queries for better recursive decomposition.
Provider Agnostic
Works with OpenAI, Anthropic, Azure, Bedrock, Ollama, and any OpenAI-compatible API. Use LiteLLM proxy for 100+ providers.
Simple Distribution
Single npm package with auto-installed platform binaries on supported targets, source-build fallback, and Docker-ready Linux container workflows.
Streaming
Async iterable streaming for real-time token delivery. Stream text or structured objects with AbortController support and backpressure handling.
Caching
Built-in memory (LRU+TTL) and file-based cache backends. SHA-256 key generation, hit/miss stats, and automatic cache invalidation.
Retry & Resilience
Exponential, linear, and fixed backoff with jitter. Rate-limit aware retries, multi-provider fallback chains, and abort signal support.
Typed Errors
9 structured error classes with error codes, retryability flags, and actionable suggestions. Auto-classification from raw errors via classifyError().
Context Overflow Recovery
Automatic recovery when context exceeds model limits. 6 strategies: MapReduce, Truncate, Chunked, TF-IDF, TextRank, and Refine. Works with any model size.
Lossless Context Management
Immutable Store + Summary DAG ensures no context is ever lost. Soft/hard token thresholds with automatic compaction. Zero overhead below soft threshold.
BPE Tokenizer
Model-specific token counting via tiktoken (o200k_base for GPT-4o, cl100k_base for GPT-4). Cached counting with xxhash. Heuristic fallback for unknown models.
Episodic Memory
Group messages into coherent episodes with auto-rotation, compaction, and budget-based retrieval. Parent chaining preserves conversation continuity.
LLM-Map & Agentic-Map
Parallel batch processing with schema validation per item. Agentic-Map spawns full sub-agent sessions. Worker pools with configurable concurrency.
SQLite Persistence
Pure Go SQLite with FTS5 full-text search, WAL mode, and transactional writes. In-memory backend for zero-dependency usage. Indexed lookups by ID, role, timestamp.
Delegation Guard
Infinite recursion prevention with scope-reduction invariant. Sub-agents must declare delegated_scope and kept_work. Trivial delegation is rejected.
How It Works
Based on the 2025 MIT paper "Recursive Language Models" by Alex Zhang and Omar Khattab, RLM solves the fundamental context length limitation of LLMs.
Instead of truncating or failing on large documents, RLM recursively decomposes the problem:
- Large context is split into manageable chunks
- LLM writes JavaScript code to analyze each chunk
- Code is executed in a sandboxed REPL (goja engine)
- Results are recursively combined up the tree
- For structured outputs, schemas are decomposed into subtasks
- Parallel goroutines process independent fields simultaneously
- Validation with retry ensures schema compliance
Proven Results
Reproducible benchmarks from the test suite. No LLM calls required — run them yourself.
Context Savings Pipeline
100 messages through episodic grouping + TF-IDF compaction:
| Stage | Tokens | Savings |
|---|---|---|
| Raw messages | 50,491 | — |
| After TF-IDF compaction | 7,669 | 84.8% |
| After budget selection (8K) | 7,669 | 84.8% |
Five-Level Summarization
5K token input → 2K token target (non-LLM levels):
| Level | Strategy | Reduction |
|---|---|---|
| Level 3 | TF-IDF | 59.7% |
| Level 4 | TextRank | 60.0% |
| Level 5 | Truncate | 65.1% |
cd go && go test ./rlm/ -run "TestContextSavings_" -v
Quick Start
Get up and running in under a minute.
import { RLM } from 'recursive-llm-ts';
const rlm = new RLM('gpt-4o-mini', {
api_key: process.env.OPENAI_API_KEY,
max_iterations: 15,
});
// Process a 100K+ token document
const result = await rlm.completion(
'What are the key findings in this research paper?',
longResearchPaper
);
console.log(result.result);
console.log('Stats:', result.stats);
import { RLM } from 'recursive-llm-ts';
import { z } from 'zod';
const rlm = new RLM('gpt-4o-mini', {
api_key: process.env.OPENAI_API_KEY,
});
const schema = z.object({
sentiment: z.number().min(1).max(5),
summary: z.string(),
topics: z.array(z.enum(['pricing', 'features', 'support'])),
keyPhrases: z.array(z.object({
phrase: z.string(),
weight: z.number(),
})),
});
const result = await rlm.structuredCompletion(
'Analyze the sentiment and extract key information',
callTranscript,
schema,
{ parallelExecution: true }
);
// result.result is fully typed!
console.log(result.result.sentiment); // number
console.log(result.result.keyPhrases); // Array<{phrase, weight}>
import { RLM } from 'recursive-llm-ts';
const rlm = new RLM('gpt-4o-mini', {
api_key: process.env.OPENAI_API_KEY,
});
// Local files
const result = await rlm.completionFromFiles(
'Summarize the architecture of this codebase',
{
type: 'local',
path: '/path/to/project/src',
extensions: ['.ts', '.tsx'],
excludePatterns: ['*.test.ts'],
maxFileSize: 100_000,
}
);
// S3 bucket
const s3Result = await rlm.completionFromFiles(
'What are the key findings?',
{
type: 's3',
path: 'my-bucket',
prefix: 'reports/2024/',
extensions: ['.md', '.txt'],
region: 'us-west-2',
}
);
import { RLM } from 'recursive-llm-ts';
const rlm = new RLM('gpt-4o-mini', {
api_key: process.env.OPENAI_API_KEY,
// Meta-agent optimizes vague queries
meta_agent: {
enabled: true,
model: 'gpt-4o',
},
// Full observability
observability: {
debug: true,
trace_enabled: true,
trace_endpoint: 'http://localhost:4318',
langfuse_enabled: true,
langfuse_public_key: process.env.LANGFUSE_PUBLIC_KEY,
langfuse_secret_key: process.env.LANGFUSE_SECRET_KEY,
}
});
const result = await rlm.completion('what happened?', transcript);
// Access trace events
const events = rlm.getTraceEvents();
for (const e of events) {
console.log(`[${e.type}] ${e.name}`, e.attributes);
}
import { RLM } from 'recursive-llm-ts';
const rlm = new RLM('gpt-4o-mini', {
api_key: process.env.OPENAI_API_KEY,
});
// Stream text tokens in real-time
const stream = await rlm.streamCompletion(
'Summarize this document',
longDocument
);
for await (const chunk of stream) {
if (chunk.type === 'text') {
process.stdout.write(chunk.text);
}
}
// Or collect all text at once
const stream2 = await rlm.streamCompletion('...', doc);
const fullText = await stream2.toText();
// Stream structured objects
const objStream = await rlm.streamStructuredCompletion(
'Extract data', context, schema
);
const result = await objStream.toObject();
import { RLM } from 'recursive-llm-ts';
// Fluent builder API
const rlm = RLM.builder()
.model('gpt-4o-mini')
.apiKey(process.env.OPENAI_API_KEY!)
.maxDepth(5)
.maxIterations(20)
.enableMetaAgent('gpt-4o')
.enableDebug()
.enableCache({ maxEntries: 500, ttl: 3600000 })
.enableRetry({ maxRetries: 3, backoff: 'exponential' })
.build();
// Factory methods
const rlm2 = RLM.fromEnv(); // Auto-detect from env vars
const rlm3 = RLM.withDebug('gpt-4o'); // Quick debug setup
const rlm4 = RLM.forAzure( // Azure-specific setup
'my-deployment',
'https://my.openai.azure.com',
process.env.AZURE_API_KEY!
);
// Config validation
const { valid, issues } = rlm.validate();
for (const issue of issues) {
console.log(`[${issue.level}] ${issue.field}: ${issue.message}`);
}
package main
import (
"fmt"
"os"
"github.com/howlerops/recursive-llm-ts/go/rlm"
)
func main() {
engine := rlm.New("gpt-4o-mini", rlm.Config{
MaxDepth: 5,
MaxIterations: 30,
APIKey: os.Getenv("OPENAI_API_KEY"),
MetaAgent: &rlm.MetaAgentConfig{
Enabled: true,
},
Observability: &rlm.ObservabilityConfig{
Debug: true,
},
})
answer, stats, err := engine.Completion(
"Summarize this document",
largeDocument,
)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
fmt.Printf("Answer: %s\n", answer)
fmt.Printf("LLM calls: %d\n", stats.LlmCalls)
}
Use as a Go Module
The Go implementation is also available as a standalone library for Go projects. Import it directly for maximum performance without the TypeScript layer.
- Full RLM engine with recursive decomposition
- Structured output with JSON Schema validation
- Meta-agent query optimization
- OpenTelemetry and Langfuse observability
- JavaScript REPL via goja engine
- Cross-compile for any platform
go get github.com/howlerops/recursive-llm-ts/go
Ready to process unbounded context?
Install the package, set your API key, and start processing documents of any size.