Extract structured data and insights from documents
using LLMs and decision models

Describe, don't prompt.

Quickstart · How it works · Agentic workflows · Changelog · Security · Documentation

You write a concept in plain language and make one call. You get every extracted fact with the line of the document it was taken from.
^{An illustrative contract. Every extracted fact is traced back to its source.}

> [!NOTE] > **New in 1.0: decision models.** Next to LLMs, ContextGem now works with decision models, which answer closed questions (yes/no, one of your labels, a level of a scale) with the probability of each possible answer, fast and at a low cost. A model group sends each aspect and concept to the model that suits it, and any model can fall back on others: see [LLMs and decision models](#llms-and-decision-models). > > **Upgrading from 0.x?** Some classes and parameters are renamed or removed, some defaults have changed, and Python 3.10 is no longer supported. Follow the [migration guide](https://contextgem.shcherbak.ai/migration_v1/), or point a coding agent at [llms.txt](https://contextgem.shcherbak.ai/llms.txt), the index of the documentation. ContextGem is a free, open-source framework that does the work around the models for you: the prompts, the data models, the source references and the justifications, with minimal code.

## Why ContextGem Reliable extraction from documents usually means writing prompts, designing validation models, mapping outputs back to the source, orchestrating multi-step pipelines and tracking usage across models. ContextGem has all of this built in, in one declarative pipeline that you can reuse and serialize.

No prompts to write Prompts are generated from your descriptions of what to extract.	Typed, validated output Text, numbers, dates, booleans, labels, ratings and JSON objects.	Every fact traced to its source Items map back to their paragraphs or sentences, with the reasoning on request.
Nested context Aspects, sub-aspects and concepts focus the model on the text that matters.	The right model for each question An LLM where it takes reasoning, a decision model where a fast, low-cost answer is enough.	Answers with probabilities A decision model gives the probability of each possible answer, and you set the threshold.

Read more on the [motivation](https://contextgem.shcherbak.ai/motivation/) behind ContextGem. ## LLMs and decision models ContextGem works with two kinds of models. You describe what to extract in the same way for both, and each aspect and concept goes to the model that suits it.

A model group with an LLM and a decision model. The open question, the notice period, goes to the LLM, which writes the answer: 30 days. Two closed questions go to the decision model, which gives the probability of each possible answer: termination for convenience is True with 0.98, above the decision threshold, and the notice form is Written with 1.00, the most probable of its labels.
^{An illustrative service agreement: the open question goes to the LLM, the closed ones to the decision model.}

| |

LLM |

Decision model | |---|---|---| | **How it answers** | Writes the answer, which ContextGem validates into typed items | Gives the probability of each possible answer, and writes no text | | **What it extracts** | Aspects and every type of concept, from text and images | Aspects, and Boolean, label and rating concepts, from text | | **With each item** | References to paragraphs or sentences, and a justification on request | The probabilities of all the possible answers, and a decision threshold that you set for the model, or for each aspect and concept | | **Use it for** | Open values, and answers that take reasoning | Closed questions in volume, where a fast, low-cost answer is enough | | **Models** | Any cloud or local LLM through [LiteLLM](https://github.com/BerriAI/litellm): OpenAI, Anthropic, Google, Azure OpenAI, xAI, Ollama, LM Studio and more | TypeSafe Jev, OpenAI Decisions API, Microsoft Foundry and more, cloud or local | | **Read more** | [Supported LLMs](https://contextgem.shcherbak.ai/llms/supported_llms/), [configuration](https://contextgem.shcherbak.ai/llms/llm_config/), [extraction methods](https://contextgem.shcherbak.ai/llms/llm_extraction_methods/) | [Decision models](https://contextgem.shcherbak.ai/decision_models/decision_models/) | > **Choosing a model.** ContextGem is tested with hosted LLMs such as `gpt-6-luna` (OpenAI) and `claude-haiku-5-5` (Anthropic), and with decision models such as Jev (TypeSafe). A smaller LLM, such as many that run locally, may not follow the detailed extraction instructions: check it on a sample of your documents, and see the [troubleshooting guide](https://contextgem.shcherbak.ai/optimizations/optimization_small_llm_troubleshooting/) if it falls short. Before you give questions to a decision model, compare it with an LLM on a sample of your documents: see [when to use a decision model](https://contextgem.shcherbak.ai/decision_models/decision_models/#when-to-use-a-decision-model).

## Quick start ```bash uv add contextgem # or: pip install -U contextgem uv add "contextgem[docling]" # with Docling, to convert PDF, DOCX and other files ``` ### Extract with an LLM An LLM extracts aspects and values of any type. Below, it extracts two things from a contract: an aspect, the **payment terms**, which comes back as the paragraphs on payments, and a concept, the **anomalies**, which span the whole document and which retrieval-based approaches can miss. Each anomaly comes with a justification and the sentence it was found in. ```python import os from contextgem import Aspect, Document, DocumentLLM, StringConcept doc = Document( raw_text=( "Consultancy Agreement\n" "This agreement between Company A (Supplier) and Company B (Customer)...\n" "The term of the agreement is 1 year from the Effective Date...\n" "The Supplier shall provide consultancy services as described in Annex 2...\n" "The Customer shall pay the Supplier within 30 calendar days of receiving an invoice...\n" "The purple elephant danced gracefully on the moon while eating ice cream.\n" # 💎 anomaly "Time-traveling dinosaurs will review all deliverables before acceptance.\n" # 💎 another anomaly "This agreement is governed by the laws of Norway...\n" ), ) doc.add_aspects( [ Aspect( name="Payment terms", description="Clauses on payments, fees and invoices", ) ] ) doc.add_concepts( [ StringConcept( name="Anomalies", description="Anomalies in the document", add_references=True, reference_depth="sentences", add_justifications=True, justification_depth="brief", ) ] ) llm = DocumentLLM( model="openai/gpt-6-luna", # or another LLM, cloud or local api_key=os.environ.get("CONTEXTGEM_OPENAI_API_KEY"), ) doc = llm.extract_all(doc) payment_terms = doc.get_aspect_by_name("Payment terms").extracted_items # 1 item, the paragraph: "The Customer shall pay the Supplier within 30 calendar # days of receiving an invoice..." anomalies = doc.get_concept_by_name("Anomalies").extracted_items # 2 items. The first one: # .value: "The statement that a purple elephant danced on the moon while eating # ice cream is anomalous and unrelated to the consultancy agreement." # .justification: "It is an implausible, fantastical statement in an otherwise # contractual document." # .reference_sentences: the sentence of the purple elephant ```

^{What the example above extracts: an aspect gives back the document's own text, a concept gives new values with their sources.}

### Extract with a decision model A decision model answers closed questions (yes/no, one of your labels, a level of a scale) with the probability of each possible answer. Below, it extracts an aspect from a contract, the **termination** clauses, by deciding for each paragraph whether it is one, and answers two questions about the contract, concepts, in one request. ```python import os from contextgem import ( Aspect, BooleanConcept, Document, DocumentDecisionModel, LabelConcept, ) doc = Document( raw_text=( "Service Agreement\n" "Either party may terminate this Agreement for any reason by giving 30 days' " "written notice to the other party.\n" "This Agreement is governed by the laws of the State of New York.\n" ), ) doc.add_aspects( [ Aspect( name="Termination", description="Clauses on termination of the agreement", ) ] ) doc.add_concepts( [ BooleanConcept( name="Termination for convenience", description="Either party may terminate the agreement without cause", ), LabelConcept( name="Notice form", description="Required form of the termination notice", labels=["Written", "Email", "Any form", "Not specified"], classification_type="multi_class", ), ] ) decision_model = DocumentDecisionModel( model="typesafe/jev-latest", # or another decision model, cloud or local api_key=os.environ.get("CONTEXTGEM_TYPESAFE_API_KEY"), ) # The aspect: a yes/no question for each paragraph. The concepts: both in one request. doc = decision_model.extract_all(doc) termination_clauses = doc.get_aspect_by_name("Termination").extracted_items # 1 item, the paragraph: "Either party may terminate this Agreement for any reason..." # .probabilities: {True: 0.99, False: 0.01} (rounded); the other two paragraphs # had 0.26 and 0.03, below the decision threshold of 0.5 convenience = doc.get_concept_by_name("Termination for convenience").extracted_items[0] # .value: True # .probabilities: {True: 0.98, False: 0.02} (rounded) notice_form = doc.get_concept_by_name("Notice form").extracted_items[0] # .value: ["Written"] # .probabilities: {"Written": 1.0, "Email": 0.0, "Any form": 0.0, "Not specified": 0.0} ```

^{What the example above extracts: a yes or no for each paragraph, and an answer for each concept, each with its probability.}

### Start from a file with Docling Your documents are files. [Docling](https://github.com/docling-project/docling) converts them, and [`Document.from_docling()`](https://contextgem.shcherbak.ai/documents/docling/) makes a document of the result, in which each paragraph keeps its place in the structure of the file, e.g. the column headers of its table row, and the pages it is on. ContextGem supports files in these formats:

_PDF	_{DOCX, DOC, RTF, ODT, Pages}	_{PPTX, PPT, ODP, Keynote}	_{XLSX, XLS, ODS, Numbers}	_CSV	_EPUB	_Email	_{Box Notes}
_Images	_Audio	_Video	_WebVTT	_{HTML, MHTML}	_Markdown	_AsciiDoc	_LaTeX
_XBRL	_{JATS XML}	_{USPTO XML}	_DocLang	_{Docling JSON}	_{METS GBS}	_EBCDIC	_AFP

Each format is tested with files of its own. The [supported formats](https://contextgem.shcherbak.ai/documents/docling/#supported-formats) give the extensions, and what some formats need, e.g. LibreOffice for DOC, PPT, XLS and RTF. Below, an LLM extracts an aspect and a concept from a [two-page service agreement](tests/other_files/docling/service_agreement.pdf), and each item is cited by its page. ```python import os from docling.document_converter import DocumentConverter from contextgem import Aspect, Document, DocumentLLM, JsonObjectConcept # Convert the file with Docling, e.g. PDF, DOCX, PPTX, XLSX or HTML result = DocumentConverter().convert("service_agreement.pdf") # The paragraphs keep their place in the structure of the file and their pages doc = Document.from_docling(result.document) doc.add_aspects( [ Aspect( name="Termination", description="Clauses on the term of the agreement and its termination", ) ] ) doc.add_concepts( [ JsonObjectConcept( name="Fees", description="The monthly fee of each service", structure={"service": str, "monthly_fee_eur": float, "billing": str}, add_references=True, ) ] ) llm = DocumentLLM( model="openai/gpt-6-luna", # or another LLM, cloud or local api_key=os.environ.get("CONTEXTGEM_OPENAI_API_KEY"), ) doc = llm.extract_all(doc) termination = doc.get_aspect_by_name("Termination") # .extracted_items: 3 items, the heading of section 3 and clauses 3.1 and 3.2 # .reference_page_numbers: [2] fees = doc.get_concept_by_name("Fees").extracted_items # 3 items, one for each row of the table of fees. The first one: # .value: {"service": "Hosting", "monthly_fee_eur": 1200.0, # "billing": "In advance"} # .reference_paragraphs: the table row, with the header of each column: # "Service: Hosting; Monthly fee: 1,200 EUR; Billing: In advance" # .reference_page_numbers: [1] ```

^{What the example above extracts: each fee from its row of the table, and the termination clauses, each cited by its page.}

### More examples

Documents	Images · Files from Docling · Paragraph context · Saving and loading
Aspects and concepts	Aspects · Sub-aspects · Concepts of a document · Concepts of an aspect · Aspects with concepts · JSON objects · Classification · References · Justifications · Reusable pipelines
LLMs	Cloud models · Local models · Extraction methods · Reasoning effort · Fallbacks · Usage and cost · Chat · Chat with tools
Decision models	Cloud models · Local models · Concepts · Aspects · Decision threshold · Fallbacks · Usage and cost
Model groups	LLMs with roles · LLM and decision model · Several documents
Agentic workflows	Metadata at ingestion · Extraction in a tool · A tool over MCP · A decision point · Choosing the tools

## How it works

A document with two aspects and four concepts, the type of each, and what is extracted: paragraphs for an aspect, and a typed value for a concept.
^{An illustrative contract: what you define, and what is extracted.}

1. **Wrap your content in a [Document](https://contextgem.shcherbak.ai/documents/document_config/).** Pass text, images or both, or **start from a file** that [Docling](https://github.com/docling-project/docling) converts: [`Document.from_docling()`](https://contextgem.shcherbak.ai/documents/docling/) keeps its structure and the page of each paragraph. 2. **Say what to extract.** [Aspects](https://contextgem.shcherbak.ai/aspects/aspects/) pull out the parts of the document on a topic, as paragraphs or sentences. [Concepts](https://contextgem.shcherbak.ai/concepts/supported_concepts/) extract typed values from the whole document or from an aspect, with optional references and justifications. Keep a set of both as a reusable [extraction pipeline](https://contextgem.shcherbak.ai/pipelines/extraction_pipelines/). 3. **Run a model.** One call to an [LLM](https://contextgem.shcherbak.ai/llms/llm_extraction_methods/), a [decision model](https://contextgem.shcherbak.ai/decision_models/decision_models/) or a [model group](https://contextgem.shcherbak.ai/llms/llm_config/#model-groups) of both extracts everything. A group sends each aspect and concept to the model of its role, e.g. the closed questions to a decision model and the rest to an LLM, and each model can have fallbacks. Read more on [how ContextGem works](https://contextgem.shcherbak.ai/how_it_works/) in the documentation. ## In agentic workflows An agent plans and works across documents and steps. ContextGem reads one document in depth and returns typed values with their sources, or the probabilities of the answers to closed questions. The two meet at three points:

^{Where ContextGem runs in an agentic workflow.}

- **At ingestion**: convert each new file with [Docling](https://contextgem.shcherbak.ai/documents/docling/) and extract its metadata once, so that agents find documents by their type, parties or dates. - **In a tool**: an agent calls an extraction and gets the values with the sentences they came from, without the document in its context. The tool is a Python function, for ContextGem's own chat, an agent framework or an MCP server. - **At a decision point**: a decision model answers a closed question with the probability of each answer, so that the workflow branches on a value that you defined and knows how sure the model is of it. Before the turn of an agent with many tools, it can also pick the groups of tools that the request needs. See [Agentic workflows](https://contextgem.shcherbak.ai/agentic_workflows/) in the documentation for an example of each. ## Every paragraph is read ContextGem is built for in-depth analysis of individual documents. There is no search step that picks the chunks that look relevant, as in RAG: the model reads every paragraph of the text it extracts from, all at once or in [batches of paragraphs](https://contextgem.shcherbak.ai/optimizations/optimization_long_docs/) of a size you set, so no fact is missed because a search ranked its passage low. This avoids the retrieval inconsistencies that make RAG [struggle with complex concepts and nuanced insights](https://www.linkedin.com/pulse/raging-contracts-pitfalls-rag-contract-review-shcherbak-ai-ptg3f). The trade-off: ContextGem does not query across documents or search a corpus. For these, RAG frameworks (e.g. LlamaIndex, Haystack) remain more appropriate. ## How it is tested Besides tests that replay recorded responses of real models, [Hypothesis](https://hypothesis.readthedocs.io/) property tests run whole extractions on generated model outputs, including what a model should never return: IDs that are not in the prompt, values of the wrong type or not valid JSON, and responses cut off at the max output tokens. They check that such an output is rejected and retried, or reported with an error or a warning, and that it never yields extracted items. A change that can affect the quality of the extraction is measured on evaluation cases, i.e. complex documents with reference answers, against a recorded baseline of quality and cost. What exact matching cannot compare (narrative values, justifications and text fields) is judged by a decision model, Jev: it answers one yes/no question at a time with a probability, and is checked on answers whose verdict is known before it is used. See [Testing](CONTRIBUTING.md#-testing). ## Documentation Full documentation: **[contextgem.shcherbak.ai](https://contextgem.shcherbak.ai)**

Get started	Define what to extract	Run models	Optimize
Installation Quickstart Advanced usage How it works Migrating to v1.0	Documents Files converted with Docling Aspects Concepts Extraction pipelines Serialization	Supported LLMs LLM configuration Extraction methods Decision models Chat with tools Agentic workflows	Accuracy Speed Cost Long documents Choosing the right LLM(s)

For LLMs and coding agents, [llms.txt](https://contextgem.shcherbak.ai/llms.txt) indexes the documentation, with a Markdown copy of each page. For visual architecture maps and natural language Q&A on the codebase, see [DeepWiki](https://deepwiki.com/shcherbak-ai/contextgem). ## Community

Need help or want to discuss? Start a thread in GitHub Discussions.	Found a bug or have a feature request? Open an issue.
Want to contribute? Contributions are welcome, from a fixed typo to a new feature: see the Contributor Guidelines.	Security The code is scanned with CodeQL and Bandit, and the dependencies are checked against known advisories: see the security policy.

## Acknowledgements ContextGem relies on these excellent open-source packages: [aiolimiter](https://github.com/mjpieters/aiolimiter), [colorlog](https://github.com/borntyping/python-colorlog), [docstring-parser](https://github.com/rr-/docstring_parser), [fastjsonschema](https://github.com/horejsek/python-fastjsonschema), [genai-prices](https://github.com/pydantic/genai-prices), [Jinja2](https://github.com/pallets/jinja), [langcodes](https://github.com/georgkrause/langcodes), [litellm](https://github.com/BerriAI/litellm), [pillow](https://github.com/python-pillow/Pillow), [pydantic](https://github.com/pydantic/pydantic), [python-ulid](https://github.com/mdomke/python-ulid), [tenacity](https://github.com/jd/tenacity) and [wtpsplit-lite](https://github.com/superlinear-ai/wtpsplit-lite), and, for converting files, [Docling](https://github.com/docling-project/docling). ## License Apache 2.0 License - see the [LICENSE](LICENSE) and [NOTICE](NOTICE) files for details. © 2025 [Shcherbak AI AS](https://shcherbak.ai) — Enterprise AI Engineering. We build AI agents to fit your enterprise — each verified, constrained, and governed.

## More from shcherbak_ai

	License compatibility checks for dependency trees, fast enough for CI.
	Runtime network egress control for Python.
	Catches .env secrets leaking into git: your own values, not lookalikes.

Questions or collaboration ideas? Reach out on [LinkedIn](https://www.linkedin.com/in/sergii-shcherbak-10068866/) or [X](https://x.com/seshch). Built with ❤️ in Oslo, Norway.

Extract structured data and insights from documentsusing LLMs and decision models

Extract structured data and insights from documents
using LLMs and decision models