--- description: "Fall back to other models when a Pydantic AI model request fails or its response is rejected, with the ModelFallback capability." --- # Model Fallback [`ModelFallback`][pydantic_ai.capabilities.ModelFallback] is a [capability](overview.md) that attempts other models when the one serving a request fails, or returns a response you reject. It's the way to fall back to other models in an agent; [`FallbackModel`](../models/overview.md#fallback-model) remains for [direct model requests](../direct.md) and standalone models (see [Migrating from `FallbackModel`](#migrating-from-fallbackmodel)). The chain is the model selected for the step, followed by the capability's models: ```python {title="model_fallback_capability.py"} from pydantic_ai import Agent from pydantic_ai.capabilities import ModelFallback # Try the OpenAI model first, then the Anthropic one. agent = Agent('{{ openai:medium }}', capabilities=[ModelFallback('{{ anthropic:medium }}')]) ``` Because the chain starts from the model selected for the step, `ModelFallback` composes with the agent's model, a model passed to `agent.run(model=...)`, and [`SelectModel`](select-model.md): whichever model they choose is attempted first. When the agent has no model and nothing else selects one, the first of `ModelFallback`'s models is used for the first attempt: ```python {title="model_fallback_without_agent_model.py"} from pydantic_ai import Agent from pydantic_ai.capabilities import ModelFallback agent = Agent(capabilities=[ModelFallback('{{ openai:medium }}', '{{ anthropic:medium }}')]) ``` A model that was already attempted in the step is skipped, whichever capability attempted it, so listing the agent's own model among the candidates doesn't cause a duplicate request. Falling back is not retrying: the next model usually can't use the prompt cache that earlier requests built on the first one, and can answer differently and at a different price. Leave a transient failure, like a dropped connection, to a retry of the same model, with a budget small enough that it doesn't hold up the fallback when the model stays down. See [Recovery scopes compared](../retries.md#recovery-scopes) and [Choosing one retry owner](../retries.md#choosing-one-retry-owner). ## When to fall back By default, `ModelFallback` moves on when a request raises [`ModelAPIError`][pydantic_ai.exceptions.ModelAPIError], the error a model raises when the provider's API fails, such as a 4xx or 5xx response, and when the model [refuses](#refusals): `fallback_on` defaults to `(ModelAPIError, ContentFilterError)`. The `fallback_on` parameter takes the same forms as [`FallbackModel`'s](../models/overview.md#fallback-model), whose default is `(ModelAPIError,)`: - a tuple of exception types - an exception handler, `(Exception) -> bool` - a response handler, `(ModelResponse) -> bool`, which rejects a response that was returned successfully - a sequence mixing any of the above Handlers may be sync or async. A handler whose first parameter is annotated as [`ModelResponse`][pydantic_ai.messages.ModelResponse] is a response handler; any other handler is an exception handler. See [Response-Based Fallback](../models/overview.md#response-based-fallback) for examples of handlers, which work the same for both. !!! warning "A response handler on its own replaces the default" Passing only a response handler as `fallback_on` replaces the default `(ModelAPIError, ContentFilterError)`, so API errors and refusals are raised rather than fallen back from. To keep falling back on them, pass both in a list, as below. Here the first model's response is cut off and rejected, the second model fails, and the third answers. The answering response lists both attempts it moved on from, and the rejected response's tokens count in the run's usage: ```python {title="model_fallback_mixed_policy.py"} from pydantic_ai import Agent, ModelAPIError, ModelMessage, ModelResponse, TextPart from pydantic_ai.capabilities import ModelFallback from pydantic_ai.models.function import AgentInfo, FunctionModel from pydantic_ai.usage import RequestUsage def truncated(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: return ModelResponse( parts=[TextPart('The capital of France is')], usage=RequestUsage(input_tokens=60, output_tokens=100), finish_reason='length', ) def unavailable(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: raise ModelAPIError('second', 'Service unavailable') def complete(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: return ModelResponse( parts=[TextPart('The capital of France is Paris.')], usage=RequestUsage(input_tokens=60, output_tokens=8), ) def was_truncated(response: ModelResponse) -> bool: return response.finish_reason == 'length' agent = Agent( # These deterministic models stand in for three providers. FunctionModel(truncated, model_name='primary'), capabilities=[ ModelFallback( FunctionModel(unavailable, model_name='second'), FunctionModel(complete, model_name='backup'), fallback_on=[ModelAPIError, was_truncated], ) ], ) result = agent.run_sync('What is the capital of France?') print(result.output) #> The capital of France is Paris. print(result.usage) #> RunUsage(input_tokens=120, output_tokens=108, requests=1) for attempt in result.response.failed_attempts or []: print(attempt.model_name, attempt.outcome, attempt.usage) #> primary rejected RequestUsage(input_tokens=60, output_tokens=100) #> second error None ``` When every model has failed or been rejected, `ModelFallback` raises [`FallbackExceptionGroup`][pydantic_ai.exceptions.FallbackExceptionGroup] with every exception, plus a [`ResponseRejected`][pydantic_ai.models.fallback.ResponseRejected] counting any rejected responses. !!! note Provider SDKs often retry failed requests themselves before raising, which delays falling back. Set [`ModelSettings['max_retries']`](../retries.md#setting-the-sdk-retry-count-per-request) to `0` on the agent, or on each candidate model but the last. As with `FallbackModel`, a candidate's own `settings` apply only to that candidate, and settings on the agent or run apply to every candidate. See [The layers](../retries.md#the-layers) in the retries guide for the full retry map. A run that reaches its `timeout=` stops with [`RunTimedOut`][pydantic_ai.exceptions.RunTimedOut] wherever it is, including in the middle of a candidate's request: the remaining candidates aren't attempted. Likewise, a request that exceeds the run's [usage limits](../agent.md#usage-limits) raises [`UsageLimitExceeded`][pydantic_ai.exceptions.UsageLimitExceeded] whatever `fallback_on` says, as another model wouldn't change that. ## Falling back when a model refuses {#refusals} A model sometimes refuses a request on safety grounds that another model would carry out, as a coding agent can find when a task touches on security. Pydantic AI reports a refusal as a successful response with a [`finish_reason`][pydantic_ai.messages.ModelResponse.finish_reason] of `'content_filter'`, and puts the provider's explanation, where it gives one, in [`provider_details['refusal']`][pydantic_ai.messages.ModelResponse.provider_details]: Anthropic's `stop_reason: 'refusal'` with its `stop_details`, the `refusal` that OpenAI sends in place of content, and Azure's content filter. `ModelFallback` moves on from a refusal by default, to the next model, which applies its own provider's safety policy. That's the capability doing what you add it for; the agent on its own keeps its content-filter behavior, so without `ModelFallback` an empty refusal raises `ContentFilterError` and one with text is the run's output. It does so whether the response is empty or has partial text: Anthropic [recommends](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback) treating any partial output of a refused response as incomplete, and pointing the retry at a fallback model rather than the same one, which usually refuses again. Naming [`ContentFilterError`][pydantic_ai.exceptions.ContentFilterError] in `fallback_on`, as the default does, is what turns this on: the agent raises that error only after it has accepted a response, too late to fall back from it, so in `fallback_on` it stands for the response it's raised for. ```python {title="model_fallback_on_refusal.py"} from pydantic_ai import Agent, ModelMessage, ModelResponse, TextPart from pydantic_ai.capabilities import ModelFallback from pydantic_ai.models.function import AgentInfo, FunctionModel def refuse(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: return ModelResponse( parts=[], finish_reason='content_filter', provider_details={'finish_reason': 'refusal', 'refusal': "I can't help with that request."}, ) def answer(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: return ModelResponse( parts=[TextPart("The input isn't escaped before it reaches the query; here's the fix.")] ) agent = Agent( # Deterministic stand-ins: an Anthropic model that refuses, and an OpenAI model that takes over. FunctionModel(refuse, model_name='anthropic-stand-in'), capabilities=[ModelFallback(FunctionModel(answer, model_name='openai-stand-in'))], ) result = agent.run_sync('Explain the vulnerability in this code and how to fix it.') print(result.output) #> The input isn't escaped before it reaches the query; here's the fix. for attempt in result.response.failed_attempts or []: print(attempt.model_name, attempt.outcome) #> anthropic-stand-in rejected ``` A few things to know about falling back on refusals: - **To have a refusal end the run instead, opt out** with `fallback_on=(ModelAPIError,)`. An empty refusal then raises `ContentFilterError`, and one with text is the run's output, as without `ModelFallback`; add [`RaiseContentFilterError`](raise-content-filter-error.md) to raise on those too. - **A [`FallbackModel`][pydantic_ai.models.fallback.FallbackModel] doesn't fall back on refusals by default.** Pass it `fallback_on=(ModelAPIError, ContentFilterError)`. - **It happens per request.** When a model refuses partway through a run, only that request moves to the next model, with the run's message history so far. The responses the first model gave earlier in the run stay in the history. Anthropic recommends staying on the fallback model for the rest of the conversation, which [`sticky=True`](#staying-on-the-model-that-answered) does. - **Without a fallback, an empty refusal raises `ContentFilterError`.** It's an [`UnexpectedModelBehavior`][pydantic_ai.exceptions.UnexpectedModelBehavior], not a `ModelAPIError`, and so is a thinking-only refusal. Its message starts with `Content filter triggered.` and then names the reason: `Refusal: '...'` with the text from `provider_details['refusal']` for an OpenAI refusal, or `Finish reason: 'refusal'` for an Anthropic one, whose refusal text is then only in the error's `body`, the serialized response. [`failed_attempts`][pydantic_ai.messages.ModelResponse.failed_attempts] records a refused attempt as `rejected`, without its `provider_details`. - **A [streamed](#streaming) refusal isn't fallen back from.** It's only known once it has been streamed, so it's accepted, without a warning, and an empty one then raises `ContentFilterError`. To fall back there, catch it and run again with another model passed as `model=`. - **Some refusals are caused by the prompt.** Anthropic's `reasoning_extraction` refusals come from prompts or tool schemas that ask the model to put its reasoning in its output, which Anthropic says to change instead (see [Refusals on repeated prompts](../models/anthropic.md#refusals-on-repeated-prompts)). Falling back to another provider still works for them. ## In agent specs {#agent-specs} `ModelFallback` can be declared in an [agent spec](../agent-spec.md), with its models as a list of model names: ```yaml {title="fallback_agent.yaml" test="skip"} model: {{ openai:medium }} capabilities: - ModelFallback: [{{ anthropic:medium }}] ``` As in Python, the spec's `model:` is attempted first, and a spec without one uses the first of `ModelFallback`'s models for the first attempt. To choose when to fall back, give the models and `fallback_on` as keyword arguments. In a spec, `fallback_on` takes the names of [`ModelAPIError`][pydantic_ai.exceptions.ModelAPIError] and its subclasses in [`pydantic_ai.exceptions`][pydantic_ai.exceptions], such as [`ModelRateLimitError`][pydantic_ai.exceptions.ModelRateLimitError] and [`ModelUnavailableError`][pydantic_ai.exceptions.ModelUnavailableError], and [`ContentFilterError`][pydantic_ai.exceptions.ContentFilterError]. Exception and response handlers can only be passed in Python: ```yaml {title="fallback_on_agent.yaml" test="skip"} model: {{ openai:medium }} capabilities: - ModelFallback: models: [{{ anthropic:medium }}, {{ google:medium }}] fallback_on: [ModelRateLimitError, ModelUnavailableError] ``` A spec can also set [`sticky: true`](#staying-on-the-model-that-answered) the same way. Model names are resolved when they're attempted, like those passed to `ModelFallback(...)`, so a spec can also list model IDs of your own that a [`ResolveModelId`](resolve-model-id.md) capability resolves, passed to [`Agent.from_spec()`][pydantic_ai.agent.Agent.from_spec] or [`Agent.from_file()`][pydantic_ai.agent.Agent.from_file] with `capabilities=`. ## How attempts work Each model attempted is an *attempt* within the same request step, not a new step: - `before_model_request` and `wrap_model_request` run once per step, and the model never sees a retry prompt. - [`prepare_model_request`][pydantic_ai.capabilities.AbstractCapability.prepare_model_request] runs again for every attempt, with `request_context.model` set to the model about to serve it. [`RunContext.model`][pydantic_ai.tools.RunContext.model] stays the model selected for the step, so a hook that depends on the serving model, its [profile](../models/overview.md#inspecting-a-models-profile) or its context window reads `request_context.model`. The model's own message preparation and token counting are redone for each candidate, and so is any preparation a capability does in `prepare_model_request`, rather than inherited from the previous one. The built-in compaction capabilities, including the Harness ones, still run in `before_model_request` for now, so they decide once per step for the step's model, and every candidate inherits that decision. See [Retrying and falling back](custom.md#model-request-attempts). - A [native tool factory](../native-tools.md#dynamic-configuration) is called again for each model after the first, with that model as `ctx.model`, so each model is sent the native tools it would be sent on its own, and a [provider-adaptive tool](overview.md#provider-adaptive-tools)'s local fallback is offered to a model its native tool doesn't apply to. A native tool a `before_model_request` hook removed stays removed. - `after_model_request` hooks of capabilities outside `ModelFallback` only ever see the response that was accepted. `ModelFallback` is positioned innermost, so it judges a response before them. - `usage.requests` counts the step once, so [`UsageLimits.request_limit`][pydantic_ai.usage.UsageLimits.request_limit] bounds how many steps the agent takes rather than how many models it tried. The tokens and cost of every attempt that produced a response, including rejected ones, are counted in the run's usage and checked against token and cost limits before the next attempt. ## Failed attempts and telemetry Failed attempts are recorded the same way [`FallbackModel` records them](../models/overview.md#failed-attempts-and-usage). The response that answers lists every attempt `ModelFallback` moved on from in [`failed_attempts`][pydantic_ai.messages.ModelResponse.failed_attempts], and when every model fails, the [`FallbackExceptionGroup`][pydantic_ai.exceptions.FallbackExceptionGroup] lists them in `attempts`. A rejected response's tokens and cost are counted once, in the run's [`RunUsage`][pydantic_ai.usage.RunUsage], whichever of the two you use. When the agent is [instrumented](../logfire.md), the model request (`chat`) span is named after the model that answered and describes it, with its usage. Each failed attempt gets its own `model request attempt ` child span with an ERROR status: an error is recorded on it as an exception, and a rejected response carries its usage, cost and finish reason. The span names the model it tried in its `gen_ai.request.model` and `gen_ai.provider.name` attributes, and its zero-based position among the request's attempts in `pydantic_ai.model_request.attempt`. When every model fails, the `chat` span is named after the last model attempted. See [Failed attempts and usage](../models/overview.md#failed-attempts-and-usage) for how these spans relate to the ones the tried model opens itself. ## Streaming A streamed request falls back if it fails before any output reaches you: `ModelFallback` waits for each candidate's stream to produce its first event before handing it over, so a connection error, an error status, or an error the provider reports right after accepting the request (like an overload) can still be handled. The first event is then replayed, so you see the same stream. Once output has been streamed, a failure is raised to you rather than retried. `FallbackModel` streams the same way. Response handlers aren't consulted for a streamed response, which is accepted as is with a `UserWarning`: it has already been streamed by the time it could be judged. A streamed [refusal](#refusals) is accepted for the same reason, without a warning, as it would be without `ModelFallback`. A request is streamed whenever something consumes the run's events: [`agent.run_stream()`][pydantic_ai.agent.AbstractAgent.run_stream], [`agent.run_stream_events()`][pydantic_ai.agent.AbstractAgent.run_stream_events] (which the [UI adapters](../ui/overview.md) use), [`agent.iter()`][pydantic_ai.agent.Agent.iter] with `node.stream()`, and [`agent.run()`][pydantic_ai.agent.AbstractAgent.run] or [`agent.run_sync()`][pydantic_ai.agent.AbstractAgent.run_sync] with an `event_stream_handler`, an [`agent.on_event`][pydantic_ai.agent.Agent.on_event] listener, or a capability that reacts to events. A capability of your own that raises [`RetryModelRequest`][pydantic_ai.exceptions.RetryModelRequest] from `after_model_request` on a streamed request gets a [`UserError`][pydantic_ai.exceptions.UserError]. ## Staying on the model that answered By default, every request step starts the chain from the model selected for it, so a conversation that fell back goes back to the first model at its next step. With `sticky=True`, it keeps using the candidate that answered instead. That keeps hitting the same provider's prompt cache, and doesn't make every step wait on a model that's failing: ```python {title="model_fallback_sticky.py"} from pydantic_ai import Agent, ModelMessage, ModelResponse, TextPart from pydantic_ai.capabilities import ModelFallback from pydantic_ai.exceptions import ModelAPIError from pydantic_ai.models.function import AgentInfo, FunctionModel def primary_response(messages: list[ModelMessage], _info: AgentInfo) -> ModelResponse: raise ModelAPIError(model_name='primary', message='Overloaded') def backup_response(messages: list[ModelMessage], _info: AgentInfo) -> ModelResponse: return ModelResponse(parts=[TextPart('Paris')]) # These deterministic responses stand in for calls to a primary model that's down and a backup model. primary = FunctionModel(primary_response, model_name='primary') backup = FunctionModel(backup_response, model_name='backup') agent = Agent(primary, capabilities=[ModelFallback(backup, sticky=True)]) result = agent.run_sync('What is the capital of France?') print([attempt.model_name for attempt in result.response.failed_attempts or []]) #> ['primary'] result = agent.run_sync('Are you sure?', message_history=result.all_messages()) print(result.response.model_name, result.response.failed_attempts) #> backup None ``` The second run goes straight to the backup model. Every response records the model that served it as [`model_id`][pydantic_ai.messages.ModelResponse.model_id], and the attempts that failed before it as [`failed_attempts`][pydantic_ai.messages.ModelResponse.failed_attempts], so this holds for any later run that continues the conversation, in this process or another. Only a fallback makes a conversation sticky: an answer from a candidate you picked for one run, with `agent.run(model=...)`, doesn't change which model later runs start from. The conversation stays on the candidate it fell back to until it fails. The chain then moves on to the models after it, and comes back around to the step's own model and the models before it, so no model the chain would have attempted is left out. Once the step's own model answers, the conversation starts from it again. `sticky` only replaces the agent's default model: a model passed to `agent.run(model=...)` or [`Agent.override`][pydantic_ai.agent.Agent.override], or selected by a capability such as [`SelectModel`](select-model.md), is still attempted first. A recorded candidate that's no longer among `ModelFallback`'s models is ignored. ## Continuing suspended responses Some providers suspend a response and continue it on a later request, like Anthropic's `pause_turn` or OpenAI's background mode. Every response records the model that served it as [`model_id`][pydantic_ai.messages.ModelResponse.model_id], so when a later run resumes a suspended response one of `ModelFallback`'s models started, the continuation goes to that model rather than to the first model in the chain. A suspended response started by a model outside the chain is continued by the step's model, as without `ModelFallback`. Response handlers judge the continued response like any other. If that continuation fails or is rejected, the suspended response is dropped and the turn is generated again by the next model in the chain after the one it was pinned to, as it would have been had the response failed in the run that started it. The chain doesn't start over: with [batching](../batch.md), the first model would only accept a new batch, and the request would never reach the models after it. A rejected response the provider is still working on, like a batch request that hasn't finished, has its job cancelled. ## Request deadlines A [`request_timeout`](../timeouts.md#model-request-deadlines) in the model settings bounds each model's attempt as a whole, across the provider SDK's own retries: each model `ModelFallback` tries gets a fresh deadline, and the [`ModelRequestTimeout`][pydantic_ai.exceptions.ModelRequestTimeout] it raises when the deadline passes moves on to the next model by default. A model's own `settings` can set a different `request_timeout` or `first_event_timeout` for it, which applies only when neither the agent nor the run sets one, as those take precedence. On a streamed request, [`first_event_timeout`](../timeouts.md#waiting-for-the-first-event) also bounds each model's wait for its first output, and whichever deadline passes first applies: a model that hasn't started streaming in time raises `ModelRequestTimeout` too, and `ModelFallback` moves on to the next model, which gets a fresh deadline. Once a response has started, the setting no longer applies, so a long response isn't cut off. ## Batching With [`BatchModelRequests`](../batch.md), every attempt is batched with the model that serves it, so when the first model's batch request fails, the request goes into a batch on the next model. Batching submits a request again itself after a transient failure, so `ModelFallback` only sees a batch failure once resubmitting is used up, or one resubmitting wouldn't fix. ## Durable execution Under a [durability capability](../durable_execution/overview.md), each attempt is a durable model operation of its own: on [Temporal](../durable_execution/temporal.md), an activity per model tried. The activity's retry policy retries a failed request before the error reaches `ModelFallback`'s `fallback_on`, and Temporal's default retry policy retries without limit: set `maximum_attempts` in the retry policy of `TemporalDurability`'s [`model_activity_config`](../durable_execution/temporal.md#activity-configuration) so that a failing model doesn't hold up the fallback. Each model's `request_timeout` deadline runs as a durable timer around its own activity. Its `first_event_timeout` deadline runs inside the activity, which is what sees the stream's events, so the `ModelRequestTimeout` it raises fails the activity there. `ModelRequestTimeout` is in [`non_retryable_model_errors`](../durable_execution/temporal.md#activity-retries) by default, so the activity isn't retried and `ModelFallback` moves on to the next model straight away. So are a rejected request and an exhausted quota. A streamed request is consumed inside its durable unit, where its events reach a durability capability's `event_stream_handler`, so a stream that fails after producing output there is raised rather than fallen back from, as outside durable execution, and what it produced counts towards the run's usage. A model `ModelFallback` moves to is rebuilt for its activity like any model a run uses: a model name string is built on the worker, and a model instance must be registered with the durability capability's `models`, then given to `ModelFallback` by that name: ```python {title="model_fallback_temporal.py" test="skip"} from pydantic_ai import Agent from pydantic_ai.capabilities import ModelFallback from pydantic_ai.durable_exec.temporal import TemporalDurability from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.providers.anthropic import AnthropicProvider backup = AnthropicModel('{{ anthropic:medium.name }}', provider=AnthropicProvider(api_key='...')) agent = Agent( '{{ openai:medium }}', name='resilient_agent', capabilities=[ ModelFallback('backup'), TemporalDurability(models={'backup': backup}), ], ) ``` A `FallbackModel` instead runs its whole chain inside one activity, along with the deadlines it gives each model, and when every model fails, its [`FallbackExceptionGroup`][pydantic_ai.exceptions.FallbackExceptionGroup] reaches the workflow without the attempts it lists, so the usage of the responses it rejected isn't recorded. ## Migrating from `FallbackModel` `ModelFallback` is the recommended way to fall back to other models inside an agent. [`FallbackModel`][pydantic_ai.models.fallback.FallbackModel] isn't deprecated and doesn't warn: like [`InstrumentedModel`][pydantic_ai.models.instrumented.InstrumentedModel] next to the [`Instrumentation`](instrumentation.md) capability, it remains the way to fall back with [direct model requests](../direct.md) or a standalone [`Model`][pydantic_ai.models.Model], outside an agent. To move an agent to the capability, keep the first model as the agent's model and pass the rest, with the same `fallback_on`, to `ModelFallback`: ```python {title="model_fallback_migration.py" test="skip" lint="skip"} # Before agent = Agent(FallbackModel(primary, backup, fallback_on=[ModelAPIError, was_truncated])) # After agent = Agent(primary, capabilities=[ModelFallback(backup, fallback_on=[ModelAPIError, was_truncated])]) ``` Everything a `FallbackModel` does carries over: - `fallback_on` takes the same exception types, exception handlers and response handlers, mixed in a list, sync or async. - Each model keeps its own `settings`, and settings on the agent or run apply to every model. - The answering response lists the attempts moved on from in [`failed_attempts`][pydantic_ai.messages.ModelResponse.failed_attempts], and when every model fails, [`FallbackExceptionGroup.attempts`][pydantic_ai.exceptions.FallbackExceptionGroup] lists them. - A rejected response's tokens and cost count in the run's usage and towards its limits, also when the run is cancelled, times out or ends on an error `fallback_on` doesn't cover. - Each model gets fresh `request_timeout` and `first_event_timeout` deadlines. - A stream that fails before any output falls back, and response handlers don't apply to a streamed response. - A suspended response is continued on the model that started it. - The same spans: a `chat` span for the model that answered, and a `model request attempt` span for each attempt moved on from. Behavior differences: - **Refusals fall back by default.** `ModelFallback`'s default `fallback_on` is `(ModelAPIError, ContentFilterError)`, so it moves on when a model [refuses](#refusals); `FallbackModel`'s is `(ModelAPIError,)`. Pass `fallback_on=(ModelAPIError,)` to keep `FallbackModel`'s behavior. And a few things only the capability can do: - [`prepare_model_request`][pydantic_ai.capabilities.AbstractCapability.prepare_model_request] runs for each model, so hooks see the model that will serve the request, its profile and context window, and prepare the request for it. The model's own message preparation and token counting are redone for each model too, so [`count_tokens_before_request`][pydantic_ai.usage.UsageLimits.count_tokens_before_request] works. - A [native tool factory](../native-tools.md#dynamic-configuration) sees the model each attempt is at as `ctx.model`, where under a `FallbackModel` it sees the `FallbackModel` itself. - The chain starts from the model selected for the step, so it composes with `agent.run(model=...)` and [`SelectModel`](select-model.md). - Under [durable execution](#durable-execution), each attempt is a durable model operation of its own. - With [batching](#batching), each attempt goes into a batch on its own model. - A failed or rejected continuation of a resumed suspended response moves on to the next model rather than starting the chain over.