# LLMSlim
**Long-term memory and context selection for AI agents and coding tools, with no LLM call.**
LLMSlim stores what users and agents say word for word, then returns the parts that answer a question within a
token budget you set. It runs as an MCP server, a Python library or a small HTTP server.
[](https://pypi.org/project/llmslim/)
[](https://pepy.tech/projects/llmslim)
[](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.Thanatos9404/llmslim)
[](LICENSE)
[](https://glama.ai/mcp/servers/Thanatos9404/llmslim)
[**Examples**](examples/) • [**Documentation**](https://www.llmslim.app/docs) • [**Live Studio Playground**](https://www.llmslim.app) • [**Sarvam Integration**](https://www.llmslim.app/integrations/sarvam) • [**Benchmarks**](#-benchmarks--radical-transparency) • [**Changelog**](CHANGELOG.md)
A real Python session with llmslim 0.11.1 from PyPI, typed by a script and recorded with asciinema (script, recording).
| Provider / Framework | What ships in Core | Pattern |
| :--- | :--- | :--- |
| **Sarvam AI** | `llmslim[sarvam]`: `SarvamProvider`/`AsyncSarvamProvider`, `SarvamLanguageTools`, `SarvamDocumentSource`, `annotate_languages()`/`normalize_scripts()`, `sarvam_messages()` | Local planning + official `sarvamai` Python SDK, guarded with dated INR prices |
| **Exa** | `llmslim[exa]`: `ExaSearchSource`, vendor-neutral `WebSearchSource` | Untrusted RAG context items with URL, date and rank provenance; spend capped in USD |
| **Neo4j** | `llmslim[neo4j]`: `Neo4jContextStore` | Explicit tenant-scoped temporal memory: `supersede()`, `recall(as_of=...)`, `history()`, `conflicts()` |
| **OpenAI** | `openai_compatible_request()`, `openai_responses_request()` | Execution-free request builders; your code sends the request |
| **Anthropic Claude** | `anthropic_messages_request()` | Execution-free request builder with optional prompt-cache markers |
| **Google Gemini** | `gemini_generate_request()`, `gemini_cached_content_request()` | Execution-free request builders |
| **vLLM / self-hosted** | `vllm_chat_request()` | Execution-free request builder |
| **OpenAI Agents SDK** | `llmslim[agents]`: `make_openai_agents_input_filter()`, `to_openai_agents_tools()` | Text-only model-input filter; the SDK keeps execution and tool ownership |
| **LangChain** | `llmslim[langchain]`: `LLMSlimCompressor` | `BaseDocumentCompressor` for `ContextualCompressionRetriever` |
| **LlamaIndex** | `llmslim[llamaindex]`: `LLMSlimPostprocessor` | `BaseNodePostprocessor` for query engines |
| **Haystack** | `llmslim[haystack]`: `LLMSlimDocumentCompressor` | Pipeline component between retriever and prompt builder |
| **gptme** | `llmslim[gptme]`: `LLMSlimContextProvider` | Context provider for gptme's compaction (entry point `llmslim`) |
| **Hermes Agent** | `LLMSlimContextEngine` (no extra) | `ContextEngine` plugin; no LLM call, cache-stable prefix |
| **Pydantic AI** | `llmslim[pydantic-ai]`: `llmslim_history_processor()` | History processor that keeps tool calls and returns paired |
For other frameworks there is no dedicated adapter. LLMSlim works with any framework
that can pass plain message lists (`{"role": ..., "content": ...}` dicts) or document
strings; you write the few lines of glue that pass LLMSlim's output to your client.
---
## 📜 License & Citation
LLMSlim is open-source software licensed under the **MIT License**. See [LICENSE](LICENSE) for full details.
If you use LLMSlim in your research or production systems, please cite:
```bibtex
@software{llmslim2026,
author = {Yashvardhan Thanvi},
title = {LLMSlim: Deterministic Context Compression and Contract-Safe Tool Schemas for LLMs},
year = {2026},
publisher = {GitHub},
url = {https://github.com/Thanatos9404/llmslim}
}
```
Built with precision for developers everywhere. Accepted into the Sarvam Startup Program.
---
## Agent Context Runtime (v0.7.0)
Version 0.7.0 added a per-turn context path for agent applications (0.7.0 was
not published to PyPI; install 0.7.1 or later).
The host supplies messages, retrieved documents, memories, tool results, and
authorized tool schemas; LLMSlim returns a quality-checked model input and an
explainable local trace. Model calls and tool execution stay with the host.
```python
from llmslim import ContextRuntime, generic_request
runtime = ContextRuntime(model="sarvam-105b", max_input_tokens=8_000)
prepared = runtime.prepare_sync(
user_input="When does Acme renew?",
messages=[{"role": "system", "content": "Use verified dates."}],
documents=[{"id": "contract", "content": "Acme renews on 9 November 2026."}],
)
if prepared.feasible:
request = generic_request(prepared) # the host sends this to its model
print(request["messages"])
print(prepared.trace.explain())
```
The runtime includes `ContextEnvelope`, `ContextGraph`, `ContextPolicy`,
progressive planning, quality gates, bounded in-memory sessions, and
execution-free adapters for generic, Sarvam, OpenAI-compatible, and text-only
OpenAI Agents applications. `ContextTrace` omits prompt bodies by default.
The existing `compress()` and `plan_context()` APIs continue to work.
See the [architecture](docs/runtime/ARCHITECTURE.md),
[security model](docs/runtime/SECURITY.md),
[integrations](docs/runtime/INTEGRATIONS.md),
[migration guide](docs/releases/v0.7.0-MIGRATION.md),
[benchmark](docs/releases/v0.7.0-BENCHMARK.md), and
[engineering report](docs/releases/v0.7.0-ENGINEERING-REPORT.md).
## Cache-Aware Context Runtime (v0.7.1)
Cache planning is an opt-in layer for repeated agent turns. LLMSlim identifies
stable provider-visible prefixes, keeps append-only conversation context when
safe, fingerprints cache-sensitive settings, and reports local cache estimates
separately from provider-reported usage. A tenant-scoped `CacheManager` stores
metadata and provider references, never prompt bodies or KV tensors.
```python
from llmslim import CacheManager, CachePolicy, ContextRuntime
runtime = ContextRuntime(
model="generic-128k",
cache_policy=CachePolicy(mode="auto", provider="openai", tenant_id="my-workspace"),
cache_manager=CacheManager(),
)
prepared = runtime.prepare_sync(
session_id="conversation-1",
messages=[{"role": "system", "content": "Use verified sources."}],
user_input="What changed?",
)
print(prepared.trace.to_dict()["cache"])
```
Execution-free request builders cover OpenAI Responses, Anthropic Messages,
Gemini cached resources, and self-hosted vLLM. An advanced in-process
Transformers adapter validates KV continuation state. OpenAI provider-stateful
continuation and Gemini cached-resource creation require explicit host opt-in.
Sarvam prompt caching remains unverified and is disabled by default. The
Context Inspector shows prefix segmentation and cache diagnostics without
calling a provider. See [caching guidance](docs/runtime/CACHING.md),
[multi-turn benchmark](docs/releases/v0.7.1-BENCHMARK.md), and
[engineering report](docs/releases/v0.7.1-ENGINEERING-REPORT.md).
## Open-source security tooling
[Snyk](https://snyk.io/) provides developer security tooling for open-source projects. Learn more about its [Secure Developer Program](https://snyk.io/open-source/).
LLMSlim is part of 15 startup programs. They provide credits, tools, and guidance;
none of them is an investor, and membership does not imply endorsement.
| Program | Company |
|---|---|
| [Sarvam Startup Program](https://www.sarvam.ai/startup-program) | Sarvam AI |
| [Zoho for Startups](https://www.zoho.com/startups/) | Zoho |
| [MongoDB for Startups](https://www.mongodb.com/startups) | MongoDB |
| [Claude for Startups](https://claude.com/programs/startups) | Anthropic |
| [OpenAI for Startups](https://openai.com/startups/) | OpenAI |
| [Auth0 for Startups](https://auth0.com/startups) | Okta (Auth0) |
| [Zendesk for Startups](https://www.zendesk.com/startups/) | Zendesk |
| [Mixpanel for Startups](https://mixpanel.com/startups/) | Mixpanel |
| [Sentry for Startups](https://sentry.io/for/startups/) | Sentry |
| [Descope Hello World Startup Program](https://www.descope.com/for-startups) | Descope |
| [Pulumi for Startups](https://www.pulumi.com/pulumi-for-startups/) | Pulumi |
| [Honeycomb for Builders](https://www.honeycomb.io/honeycomb-for-builders) | Honeycomb |
| [Exa Startup Grants](https://exa.ai/pricing) | Exa |
| [Neo4j Startup Program](https://neo4j.com/startup-program/) | Neo4j |
| [Cartesia Startups Grant Program](https://cartesia.ai/startups) | Cartesia |
Logos are trademarks of their owners and identify each program.
---
## Sarvam Startup Program and Sarvam integration
> ### ✦ Accepted into the Sarvam Startup Program
>
> LLMSlim has been accepted into the **Sarvam Startup Program**. This is program
> membership only: it is not a partnership, and it does not imply an endorsement
> by Sarvam AI.
Separately, LLMSlim Core ships an optional Sarvam integration (`llmslim[sarvam]`)
built on the official `sarvamai` Python SDK. You plan and compress context locally,
then send only the planned context to a Sarvam model such as `sarvam-105b`.
- **Local planning**: retrieved documents are planned and compressed in your Python process before any request is sent.
- **Local-first**: sensitive retrieved context stays local; only the planned context is sent to the model.
- **Your application stays in charge**: LLMSlim is a pre-processor. Your application owns API keys, prompts, and inference, and nothing is sent to Sarvam unless your code calls the provider.
### Quickstart: Adaptive Planning + Sarvam
```bash
pip install "llmslim[sarvam]"
export SARVAM_API_KEY="..." # your own Sarvam API key
```
```python
from llmslim import plan_context
from llmslim.integrations.sarvam import SarvamProvider
documents = [
{"content": "Q3 filing: new data-localisation rules take effect in April 2027."},
{"content": "Q3 filing: two pending licence renewals carry a regulatory risk of delay."},
]
plan = plan_context(
messages=[{"role": "system", "content": "Answer from verified context."}],
documents=documents,
query="Summarize the key regulatory risks.",
model="sarvam-105b",
max_input_tokens=8_000,
reserve_output_tokens=256,
)
provider = SarvamProvider.from_env(model="sarvam-105b", max_tokens=256)
answer = provider.chat([{"role": "user", "content": plan.final_context}])
print(answer, provider.last_usage)
```
👉 **[Read the Sarvam integration guide](https://www.llmslim.app/integrations/sarvam)** (also in [docs/integrations/SARVAM.md](docs/integrations/SARVAM.md))
---
## 🔮 What is LLMSlim?
