# Pydantic AI Slim Architecture

Use this guide for non-trivial changes to `pydantic-ai-slim`: public APIs, provider behavior, models/profiles, capabilities, toolsets, tools/output, message history, streaming, UI adapters, or durable execution.

## Ownership

- `Agent` owns user-facing construction and run APIs. `Agent.iter()` is the graph-run facade: `_prepare_run()` resolves per-run inputs into a private `_PreparedAgentRun`, whose `open()` method owns resource entry, capability lifecycle, recovery, and cleanup. Prefer not to add constructor kwargs for behavior that can be modeled as a capability, toolset, model setting, or profile fact.
- `_agent_graph.py` owns loop orchestration: prompt assembly, model requests, tool/output processing, retries, usage checks, and finalization.
- `tool_manager.py`, `tools.py`, and `toolsets/` own tool discovery, validation, execution, retries, approval/deferral, wrapper composition, and stable tool identity.
- `output.py` is the public output API; `_output.py` owns internal output schemas, processors, output tools, and output validation/processing.
- `messages.py` owns the normalized protocol. Provider adapters, UI adapters, durable wrappers, and persisted histories should round-trip through this shape instead of encoding provider facts in strings or ad hoc fields.
- `models/` maps normalized requests/responses to provider wire formats. Put provider-specific request/response translation here rather than in graph/tool/output code.
- `providers/` owns authentication, clients, base URLs, HTTP lifecycle, and provider-level model/profile inference.
- `profiles/` owns model-family facts: structured output defaults, schema quirks, native tool support, thinking support, return-schema support, prompted-output templates, and intrinsic model-family behavior.
- `capabilities/` owns composable cross-cutting behavior: instructions, settings, toolsets, native tools, wrapper toolsets, and run/model/tool/output/event/history hooks.
- `durable_exec/` adapts agents, models, and toolsets to durable runtimes. Treat these integrations as compatibility tests for core semantics, not peripheral adapters.
- `ui/` adapters translate normalized messages/events for frontend protocols. Preserve message history and event semantics across round-trips.

## Compatibility Checks

Before editing, identify which contracts can change:

- Public API: constructor kwargs, decorators, settings, output types, tool/toolset APIs, imports, and documented names.
- Provider API compatibility: request parameters, response parts, streaming chunks, native tools, structured output, thinking/reasoning, prompt caching, token counting, usage, and provider metadata.
- Message/event protocol: persisted message history, partial responses, tool-call parts, output events, native tool parts, retry prompts, and UI event streams.
- Durable execution: context propagation, dependency serialization, tool ordering, retries, message replay, toolset lifecycle, activity/task boundaries, and deterministic behavior.
- Agent specs/config: whether new state is serializable, safe to load, and representable without runtime-only closures.

## Design Rules

- Keep provider facts in providers/profiles/model adapters. Do not scatter provider-name checks through graph, tool, output, or UI code.
- Preserve provider-specific data in structured metadata/provider-details fields. Do not overload normalized IDs, text content, or tool args.
- Prefer general primitives over one-off flags: capabilities for cross-cutting behavior, wrapper toolsets for tool collection behavior, profiles for capability facts, and typed settings for provider API knobs.
- Keep local tools and provider-native tools conceptually separate. If a feature can use either, make the fallback/selection behavior explicit and test the message history it produces.
- When changing tool/output execution, check ordering, retry semantics, deferred calls, output finalization, streaming events, and durable wrappers together.
- When removing deprecated APIs, distinguish public surface cleanup from persisted-data compatibility. Old constructors/imports may be removed in a major version; old serialized histories may still need to deserialize.

## Run Event Stream Internals

`GraphAgentState.event_stream_buffer` is the internal, run-scoped queue for framework events emitted outside the direct model/tool event generators. It's shared by reference into every `RunContext` this run as the private `_event_stream_buffer` field; framework code appends via `RunContext._emit_event(event)`.

`RunContext.emit` (and `AgentRun.emit` for code driving `agent.iter()`) is the public surface on top of that buffer, and the buffer's semantics are what it inherits. Application code emits a `CustomEvent` subclass; a capability emits a `CapabilityEvent` subclass, which `emit` resolves to its owning capability either from the hook context's `_capability` or from the executing `ToolManager`'s `tool_def.capability_id` — so a capability-contributed tool's events are attributed without the tool knowing. The families are open: subclasses register at class-definition time and the `AgentStreamEvent` union is rebuilt from those registries (`_event_registry.py`), with an unregistered tag degrading to `UnknownCustomEvent` / `UnknownCapabilityEvent` rather than failing. Anything memoizing an adapter over a hint containing `AgentStreamEvent` must key on `event_registry_version()`, since the union's choices are snapshotted when the schema is built.

Listeners run when the event's stream position is consumed, except for a `CapabilityEvent` family declared `dispatch='immediate'`, which is dispatched before `emit` returns so the emitter can read decision fields listeners set. Dispatch consults `AbstractCapability.listens_to(event)` before descending, so a capability is only woken for event classes one of its `@on_event` listeners named; a bare marker or an overridden `on_event()` widens that to everything, and a `CombinedCapability`/`WrapperCapability` reports the union of what it contains. Under Temporal, `emit` from a tool or event stream handler raises: those run in activities that can't reach the buffer. Under DBOS and Prefect the buffer is in-process, so it works, but an event emitted inside a durable unit is a side effect of running it — a replayed step or a cached task does not re-emit it. See `durable_exec/AGENTS.md` and #7971.

Node streams own draining the buffer into `wrap_run_event_stream` / `event_stream_handler`. `ModelRequestNode` delegates this to `AgentStream` (via `_event_stream_buffer_getter`), which drains before each pull from the model stream — events emitted while a pull is in flight surface on the next pull, or through the response-handling node's stream once the model stream is exhausted. `CallToolsNode` wraps its handle-response event iterator with `_with_event_stream_buffer`, which drains only at the start and end of the node stream (its trailing drain is what delivers events emitted after the last model/tool event of a step). While that stream is live, events reach consumers through `_iter_completed_or_buffered`, which interleaves them with tool completions as they land; draining them here as well could yield a buffered event ahead of an earlier one the stream is about to deliver, inverting emission order.

Feature code emits typed `AgentStreamEvent`s into the buffer once the public event semantics are true. Pending messages follow this pattern: `PendingMessageDrainCapability` emits one `EnqueuedMessagesEvent` per drained `enqueue` call (a single call can carry multiple messages) when it delivers that call's messages into history, describing the messages as delivered (indices are deliberately not carried, since `_clean_message_history` can merge adjacent requests across runs and stale them).

## Cancellation Internals

`_cancel.RunCancellation` is the run-scoped first-party cancellation controller, held on `GraphAgentDeps.cancellation` and shared by reference into every `RunContext` as the private `_cancellation` field (same never-`replace` invariant as `_event_stream_buffer` — see the comment in `build_run_context`). First-party cancellation works by cancelling the asyncio task driving the run, so it reuses the entire external-cancellation teardown; the `CancelledError` is classified exactly once, at the outer edge of `_PreparedAgentRun.open()`'s exit stack (`_translate_cancellation`), after all history-producing teardown has committed.

Three pieces of bookkeeping are load-bearing and easy to break from `_agent_graph.py` / `run.py`:

- **Issuance counting** (`_issued` + `Task.uncancel()` in `resolve()`): the controller consumes exactly the cancellations it issued; if `Task.cancelling()` is still positive afterwards, an external cancellation raced in and wins (never translated). The arbitration is deliberately baseline-free — conservative in the "external wins" direction.
- **Re-issue on `bind()`**: `bind()` runs at run start and every step boundary; it clamps the issued count to the task's live `cancelling()` (a caller that uncancelled took over the bookkeeping) and re-delivers a still-requested cancellation — this is what makes first-party cancellation sticky against hooks or callers that swallow/uncancel it, without per-site `cancel_requested` checks.
- **`release_issued()` in a `finally`** at the translation edge: any exit path (including a non-cancellation error overtaking a requested cancel) must release unresolved issuances, or the leaked `Task.cancelling()` count spuriously cancels unrelated later work on the same task.

`_utils.raise_if_cancelling()` (the level-triggered external backstop) and `RunCancellation.cancel_requested` (the first-party terminality flag) are deliberately separate checks — see the comment at `_finalize_result` in `agent/__init__.py`; collapsing them reintroduces the swallowed-cancellation bug each one exists to catch.

## Test Shape

- Use public agent/model/toolset behavior for most tests. Prefer snapshots for message history, event streams, provider request payloads, and protocol shapes.
- Use provider cassettes or real integration tests when behavior depends on provider APIs or SDK behavior.
- Use workflow-level tests when durable runtime behavior depends on serialization, replay, or activity/task boundaries.
- Update docs and examples where users discover the feature, not only docstrings.