Chat App
Ask me anything...
Error occurred, check the browser developer console for more information.
User: "My guess is 4" activate LLM Note over LLM: LLM decides to use
a tool LLM ->> Agent: Call tool
roll_dice() deactivate LLM activate Agent Note over Agent: Rolls a six-sided die Agent -->> LLM: ToolReturn
"4" deactivate Agent activate LLM Note over LLM: LLM decides to use
another tool LLM ->> Agent: Call tool
get_player_name() deactivate LLM activate Agent Note over Agent: Retrieves player name Agent -->> LLM: ToolReturn
"Anne" deactivate Agent activate LLM Note over LLM: LLM constructs final response LLM ->> Agent: ModelResponse
"Congratulations Anne, ..." deactivate LLM Note over Agent: Game session complete ``` ## Registering via Agent Argument As well as using the decorators, we can register tools via the `tools` argument to the Agent constructor. This is useful when you want to reuse tools, and can also give more fine-grained control over the tools. dice_game_tool_kwarg.py ```python import random from pydantic_ai import Agent, RunContext, Tool system_prompt = """\ You're a dice game, you should roll the die and see if the number you get back matches the user's guess. If so, tell them they're a winner. Use the player's name in the response. """ def roll_dice() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps agent_a = Agent( 'google-gla:gemini-1.5-flash', deps_type=str, tools=[roll_dice, get_player_name], # (1)! system_prompt=system_prompt, ) agent_b = Agent( 'google-gla:gemini-1.5-flash', deps_type=str, tools=[ # (2)! Tool(roll_dice, takes_ctx=False), Tool(get_player_name, takes_ctx=True), ], system_prompt=system_prompt, ) dice_result = {} dice_result['a'] = agent_a.run_sync('My guess is 6', deps='Yashar') dice_result['b'] = agent_b.run_sync('My guess is 4', deps='Anne') print(dice_result['a'].output) #> Tough luck, Yashar, you rolled a 4. Better luck next time. print(dice_result['b'].output) #> Congratulations Anne, you guessed correctly! You're a winner! ``` 1. The simplest way to register tools via the `Agent` constructor is to pass a list of functions, the function signature is inspected to determine if the tool takes RunContext. 1. `agent_a` and `agent_b` are identical — but we can use Tool to reuse tool definitions and give more fine-grained control over how tools are defined, e.g. setting their name or description, or using a custom [`prepare`](../tools-advanced/#tool-prepare) method. *(This example is complete, it can be run "as is")* ## Tool Output Tools can return anything that Pydantic can serialize to JSON. For advanced output options including multi-modal content and metadata, see [Advanced Tool Features](../tools-advanced/#function-tool-output). ## Tool Schema Function parameters are extracted from the function signature, and all parameters except `RunContext` are used to build the schema for that tool call. Even better, Pydantic AI extracts the docstring from functions and (thanks to [griffe](https://mkdocstrings.github.io/griffe/)) extracts parameter descriptions from the docstring and adds them to the schema. [Griffe supports](https://mkdocstrings.github.io/griffe/reference/docstrings/#docstrings) extracting parameter descriptions from `google`, `numpy`, and `sphinx` style docstrings. Pydantic AI will infer the format to use based on the docstring, but you can explicitly set it using docstring_format. You can also enforce parameter requirements by setting `require_parameter_descriptions=True`. This will raise a UserError if a parameter description is missing. To demonstrate a tool's schema, here we use FunctionModel to print the schema a model would receive: tool_schema.py ```python from pydantic_ai import Agent, ModelMessage, ModelResponse, TextPart from pydantic_ai.models.function import AgentInfo, FunctionModel agent = Agent() @agent.tool_plain(docstring_format='google', require_parameter_descriptions=True) def foobar(a: int, b: str, c: dict[str, list[float]]) -> str: """Get me foobar. Args: a: apple pie b: banana cake c: carrot smoothie """ return f'{a} {b} {c}' def print_schema(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: tool = info.function_tools[0] print(tool.description) #> Get me foobar. print(tool.parameters_json_schema) """ { 'additionalProperties': False, 'properties': { 'a': {'description': 'apple pie', 'type': 'integer'}, 'b': {'description': 'banana cake', 'type': 'string'}, 'c': { 'additionalProperties': {'items': {'type': 'number'}, 'type': 'array'}, 'description': 'carrot smoothie', 'type': 'object', }, }, 'required': ['a', 'b', 'c'], 'type': 'object', } """ return ModelResponse(parts=[TextPart('foobar')]) agent.run_sync('hello', model=FunctionModel(print_schema)) ``` *(This example is complete, it can be run "as is")* If a tool has a single parameter that can be represented as an object in JSON schema (e.g. dataclass, TypedDict, pydantic model), the schema for the tool is simplified to be just that object. Here's an example where we use TestModel.last_model_request_parameters to inspect the tool schema that would be passed to the model. single_parameter_tool.py ```python from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.test import TestModel agent = Agent() class Foobar(BaseModel): """This is a Foobar""" x: int y: str z: float = 3.14 @agent.tool_plain def foobar(f: Foobar) -> str: return str(f) test_model = TestModel() result = agent.run_sync('hello', model=test_model) print(result.output) #> {"foobar":"x=0 y='a' z=3.14"} print(test_model.last_model_request_parameters.function_tools) """ [ ToolDefinition( name='foobar', parameters_json_schema={ 'properties': { 'x': {'type': 'integer'}, 'y': {'type': 'string'}, 'z': {'default': 3.14, 'type': 'number'}, }, 'required': ['x', 'y'], 'title': 'Foobar', 'type': 'object', }, description='This is a Foobar', ) ] """ ``` *(This example is complete, it can be run "as is")* ## See Also For more tool features and integrations, see: - [Advanced Tool Features](../tools-advanced/) - Custom schemas, dynamic tools, tool execution and retries - [Toolsets](../toolsets/) - Managing collections of tools - [Builtin Tools](../builtin-tools/) - Native tools provided by LLM providers - [Common Tools](../common-tools/) - Ready-to-use tool implementations - [Third-Party Tools](../third-party-tools/) - Integrations with MCP, LangChain, ACI.dev and other tool libraries - [Deferred Tools](../deferred-tools/) - Tools requiring approval or external execution # Toolsets A toolset represents a collection of [tools](../tools/) that can be registered with an agent in one go. They can be reused by different agents, swapped out at runtime or during testing, and composed in order to dynamically filter which tools are available, modify tool definitions, or change tool execution behavior. A toolset can contain locally defined functions, depend on an external service to provide them, or implement custom logic to list available tools and handle them being called. Toolsets are used (among many other things) to define [MCP servers](../mcp/client/) available to an agent. Pydantic AI includes many kinds of toolsets which are described below, and you can define a [custom toolset](#building-a-custom-toolset) by inheriting from the AbstractToolset class. The toolsets that will be available during an agent run can be specified in four different ways: - at agent construction time, via the toolsets keyword argument to `Agent`, which takes toolset instances as well as functions that generate toolsets [dynamically](#dynamically-building-a-toolset) based on the agent run context - at agent run time, via the `toolsets` keyword argument to agent.run(), agent.run_sync(), agent.run_stream(), or agent.iter(). These toolsets will be additional to those registered on the `Agent` - [dynamically](#dynamically-building-a-toolset), via the @agent.toolset decorator which lets you build a toolset based on the agent run context - as a contextual override, via the `toolsets` keyword argument to the agent.override() context manager. These toolsets will replace those provided at agent construction or run time during the life of the context manager toolsets.py ```python from pydantic_ai import Agent, FunctionToolset from pydantic_ai.models.test import TestModel def agent_tool(): return "I'm registered directly on the agent" def extra_tool(): return "I'm passed as an extra tool for a specific run" def override_tool(): return 'I override all other tools' agent_toolset = FunctionToolset(tools=[agent_tool]) # (1)! extra_toolset = FunctionToolset(tools=[extra_tool]) override_toolset = FunctionToolset(tools=[override_tool]) test_model = TestModel() # (2)! agent = Agent(test_model, toolsets=[agent_toolset]) result = agent.run_sync('What tools are available?') print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['agent_tool'] result = agent.run_sync('What tools are available?', toolsets=[extra_toolset]) print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['agent_tool', 'extra_tool'] with agent.override(toolsets=[override_toolset]): result = agent.run_sync('What tools are available?', toolsets=[extra_toolset]) # (3)! print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['override_tool'] ``` 1. The FunctionToolset will be explained in detail in the next section. 1. We're using TestModel here because it makes it easy to see which tools were available on each run. 1. This `extra_toolset` will be ignored because we're inside an override context. *(This example is complete, it can be run "as is")* ## Function Toolset As the name suggests, a FunctionToolset makes locally defined functions available as tools. Functions can be added as tools in three different ways: - via the @toolset.tool decorator - via the tools keyword argument to the constructor which can take either plain functions, or instances of Tool - via the toolset.add_function() and toolset.add_tool() methods which can take a plain function or an instance of Tool respectively Functions registered in any of these ways can define an initial `ctx: RunContext` argument in order to receive the agent run context. The `add_function()` and `add_tool()` methods can also be used from a tool function to dynamically register new tools during a run to be available in future run steps. function_toolset.py ```python from datetime import datetime from pydantic_ai import Agent, FunctionToolset, RunContext from pydantic_ai.models.test import TestModel def temperature_celsius(city: str) -> float: return 21.0 def temperature_fahrenheit(city: str) -> float: return 69.8 weather_toolset = FunctionToolset(tools=[temperature_celsius, temperature_fahrenheit]) @weather_toolset.tool def conditions(ctx: RunContext, city: str) -> str: if ctx.run_step % 2 == 0: return "It's sunny" else: return "It's raining" datetime_toolset = FunctionToolset() datetime_toolset.add_function(lambda: datetime.now(), name='now') test_model = TestModel() # (1)! agent = Agent(test_model) result = agent.run_sync('What tools are available?', toolsets=[weather_toolset]) print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['temperature_celsius', 'temperature_fahrenheit', 'conditions'] result = agent.run_sync('What tools are available?', toolsets=[datetime_toolset]) print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['now'] ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. *(This example is complete, it can be run "as is")* ## Toolset Composition Toolsets can be composed to dynamically filter which tools are available, modify tool definitions, or change tool execution behavior. Multiple toolsets can also be combined into one. ### Combining Toolsets CombinedToolset takes a list of toolsets and lets them be used as one. combined_toolset.py ```python from pydantic_ai import Agent, CombinedToolset from pydantic_ai.models.test import TestModel from function_toolset import datetime_toolset, weather_toolset combined_toolset = CombinedToolset([weather_toolset, datetime_toolset]) test_model = TestModel() # (1)! agent = Agent(test_model, toolsets=[combined_toolset]) result = agent.run_sync('What tools are available?') print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['temperature_celsius', 'temperature_fahrenheit', 'conditions', 'now'] ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. *(This example is complete, it can be run "as is")* ### Filtering Tools FilteredToolset wraps a toolset and filters available tools ahead of each step of the run based on a user-defined function that is passed the agent run context and each tool's ToolDefinition and returns a boolean to indicate whether or not a given tool should be available. To easily chain different modifications, you can also call filtered() on any toolset instead of directly constructing a `FilteredToolset`. filtered_toolset.py ```python from pydantic_ai import Agent from pydantic_ai.models.test import TestModel from combined_toolset import combined_toolset filtered_toolset = combined_toolset.filtered(lambda ctx, tool_def: 'fahrenheit' not in tool_def.name) test_model = TestModel() # (1)! agent = Agent(test_model, toolsets=[filtered_toolset]) result = agent.run_sync('What tools are available?') print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['weather_temperature_celsius', 'weather_conditions', 'datetime_now'] ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. *(This example is complete, it can be run "as is")* ### Prefixing Tool Names PrefixedToolset wraps a toolset and adds a prefix to each tool name to prevent tool name conflicts between different toolsets. To easily chain different modifications, you can also call prefixed() on any toolset instead of directly constructing a `PrefixedToolset`. combined_toolset.py ```python from pydantic_ai import Agent, CombinedToolset from pydantic_ai.models.test import TestModel from function_toolset import datetime_toolset, weather_toolset combined_toolset = CombinedToolset( [ weather_toolset.prefixed('weather'), datetime_toolset.prefixed('datetime') ] ) test_model = TestModel() # (1)! agent = Agent(test_model, toolsets=[combined_toolset]) result = agent.run_sync('What tools are available?') print([t.name for t in test_model.last_model_request_parameters.function_tools]) """ [ 'weather_temperature_celsius', 'weather_temperature_fahrenheit', 'weather_conditions', 'datetime_now', ] """ ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. *(This example is complete, it can be run "as is")* ### Renaming Tools RenamedToolset wraps a toolset and lets you rename tools using a dictionary mapping new names to original names. This is useful when the names provided by a toolset are ambiguous or would conflict with tools defined by other toolsets, but [prefixing them](#prefixing-tool-names) creates a name that is unnecessarily long or could be confusing to the model. To easily chain different modifications, you can also call renamed() on any toolset instead of directly constructing a `RenamedToolset`. renamed_toolset.py ```python from pydantic_ai import Agent from pydantic_ai.models.test import TestModel from combined_toolset import combined_toolset renamed_toolset = combined_toolset.renamed( { 'current_time': 'datetime_now', 'temperature_celsius': 'weather_temperature_celsius', 'temperature_fahrenheit': 'weather_temperature_fahrenheit' } ) test_model = TestModel() # (1)! agent = Agent(test_model, toolsets=[renamed_toolset]) result = agent.run_sync('What tools are available?') print([t.name for t in test_model.last_model_request_parameters.function_tools]) """ ['temperature_celsius', 'temperature_fahrenheit', 'weather_conditions', 'current_time'] """ ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. *(This example is complete, it can be run "as is")* ### Dynamic Tool Definitions PreparedToolset lets you modify the entire list of available tools ahead of each step of the agent run using a user-defined function that takes the agent run context and a list of ToolDefinitions and returns a list of modified `ToolDefinition`s. This is the toolset-specific equivalent of the [`prepare_tools`](../tools-advanced/#prepare-tools) argument to `Agent` that prepares all tool definitions registered on an agent across toolsets. Note that it is not possible to add or rename tools using `PreparedToolset`. Instead, you can use [`FunctionToolset.add_function()`](#function-toolset) or [`RenamedToolset`](#renaming-tools). To easily chain different modifications, you can also call prepared() on any toolset instead of directly constructing a `PreparedToolset`. prepared_toolset.py ```python from dataclasses import replace from pydantic_ai import Agent, RunContext, ToolDefinition from pydantic_ai.models.test import TestModel from renamed_toolset import renamed_toolset descriptions = { 'temperature_celsius': 'Get the temperature in degrees Celsius', 'temperature_fahrenheit': 'Get the temperature in degrees Fahrenheit', 'weather_conditions': 'Get the current weather conditions', 'current_time': 'Get the current time', } async def add_descriptions(ctx: RunContext, tool_defs: list[ToolDefinition]) -> list[ToolDefinition] | None: return [ replace(tool_def, description=description) if (description := descriptions.get(tool_def.name, None)) else tool_def for tool_def in tool_defs ] prepared_toolset = renamed_toolset.prepared(add_descriptions) test_model = TestModel() # (1)! agent = Agent(test_model, toolsets=[prepared_toolset]) result = agent.run_sync('What tools are available?') print(test_model.last_model_request_parameters.function_tools) """ [ ToolDefinition( name='temperature_celsius', parameters_json_schema={ 'additionalProperties': False, 'properties': {'city': {'type': 'string'}}, 'required': ['city'], 'type': 'object', }, description='Get the temperature in degrees Celsius', ), ToolDefinition( name='temperature_fahrenheit', parameters_json_schema={ 'additionalProperties': False, 'properties': {'city': {'type': 'string'}}, 'required': ['city'], 'type': 'object', }, description='Get the temperature in degrees Fahrenheit', ), ToolDefinition( name='weather_conditions', parameters_json_schema={ 'additionalProperties': False, 'properties': {'city': {'type': 'string'}}, 'required': ['city'], 'type': 'object', }, description='Get the current weather conditions', ), ToolDefinition( name='current_time', parameters_json_schema={ 'additionalProperties': False, 'properties': {}, 'type': 'object', }, description='Get the current time', ), ] """ ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. ### Requiring Tool Approval ApprovalRequiredToolset wraps a toolset and lets you dynamically [require approval](../deferred-tools/#human-in-the-loop-tool-approval) for a given tool call based on a user-defined function that is passed the agent run context, the tool's ToolDefinition, and the validated tool call arguments. If no function is provided, all tool calls will require approval. To easily chain different modifications, you can also call approval_required() on any toolset instead of directly constructing a `ApprovalRequiredToolset`. See the [Human-in-the-Loop Tool Approval](../deferred-tools/#human-in-the-loop-tool-approval) documentation for more information on how to handle agent runs that call tools that require approval and how to pass in the results. approval_required_toolset.py ```python from pydantic_ai import Agent, DeferredToolRequests, DeferredToolResults from pydantic_ai.models.test import TestModel from prepared_toolset import prepared_toolset approval_required_toolset = prepared_toolset.approval_required(lambda ctx, tool_def, tool_args: tool_def.name.startswith('temperature')) test_model = TestModel(call_tools=['temperature_celsius', 'temperature_fahrenheit']) # (1)! agent = Agent( test_model, toolsets=[approval_required_toolset], output_type=[str, DeferredToolRequests], ) result = agent.run_sync('Call the temperature tools') messages = result.all_messages() print(result.output) """ DeferredToolRequests( calls=[], approvals=[ ToolCallPart( tool_name='temperature_celsius', args={'city': 'a'}, tool_call_id='pyd_ai_tool_call_id__temperature_celsius', ), ToolCallPart( tool_name='temperature_fahrenheit', args={'city': 'a'}, tool_call_id='pyd_ai_tool_call_id__temperature_fahrenheit', ), ], ) """ result = agent.run_sync( message_history=messages, deferred_tool_results=DeferredToolResults( approvals={ 'pyd_ai_tool_call_id__temperature_celsius': True, 'pyd_ai_tool_call_id__temperature_fahrenheit': False, } ) ) print(result.output) #> {"temperature_celsius":21.0,"temperature_fahrenheit":"The tool call was denied."} ``` 1. We're using TestModel here because it makes it easy to specify which tools to call. *(This example is complete, it can be run "as is")* ### Changing Tool Execution WrapperToolset wraps another toolset and delegates all responsibility to it. It is is a no-op by default, but you can subclass `WrapperToolset` to change the wrapped toolset's tool execution behavior by overriding the call_tool() method. logging_toolset.py ```python import asyncio from typing_extensions import Any from pydantic_ai import Agent, RunContext, ToolsetTool, WrapperToolset from pydantic_ai.models.test import TestModel from prepared_toolset import prepared_toolset LOG = [] class LoggingToolset(WrapperToolset): async def call_tool(self, name: str, tool_args: dict[str, Any], ctx: RunContext, tool: ToolsetTool) -> Any: LOG.append(f'Calling tool {name!r} with args: {tool_args!r}') try: await asyncio.sleep(0.1 * len(LOG)) # (1)! result = await super().call_tool(name, tool_args, ctx, tool) LOG.append(f'Finished calling tool {name!r} with result: {result!r}') except Exception as e: LOG.append(f'Error calling tool {name!r}: {e}') raise e else: return result logging_toolset = LoggingToolset(prepared_toolset) agent = Agent(TestModel(), toolsets=[logging_toolset]) # (2)! result = agent.run_sync('Call all the tools') print(LOG) """ [ "Calling tool 'temperature_celsius' with args: {'city': 'a'}", "Calling tool 'temperature_fahrenheit' with args: {'city': 'a'}", "Calling tool 'weather_conditions' with args: {'city': 'a'}", "Calling tool 'current_time' with args: {}", "Finished calling tool 'temperature_celsius' with result: 21.0", "Finished calling tool 'temperature_fahrenheit' with result: 69.8", 'Finished calling tool \'weather_conditions\' with result: "It\'s raining"', "Finished calling tool 'current_time' with result: datetime.datetime(...)", ] """ ``` 1. All docs examples are tested in CI and their their output is verified, so we need `LOG` to always have the same order whenever this code is run. Since the tools could finish in any order, we sleep an increasing amount of time based on which number tool call we are to ensure that they finish (and log) in the same order they were called in. 1. We use TestModel here as it will automatically call each tool. *(This example is complete, it can be run "as is")* ## External Toolset If your agent needs to be able to call [external tools](../deferred-tools/#external-tool-execution) that are provided and executed by an upstream service or frontend, you can build an ExternalToolset from a list of ToolDefinitions containing the tool names, arguments JSON schemas, and descriptions. When the model calls an external tool, the call is considered to be ["deferred"](../deferred-tools/#deferred-tools), and the agent run will end with a DeferredToolRequests output object with a `calls` list holding ToolCallParts containing the tool name, validated arguments, and a unique tool call ID, which are expected to be passed to the upstream service or frontend that will produce the results. When the tool call results are received from the upstream service or frontend, you can build a DeferredToolResults object with a `calls` dictionary that maps each tool call ID to an arbitrary value to be returned to the model, a [`ToolReturn`](../tools-advanced/#advanced-tool-returns) object, or a ModelRetry exception in case the tool call failed and the model should [try again](../tools-advanced/#tool-retries). This `DeferredToolResults` object can then be provided to one of the agent run methods as `deferred_tool_results`, alongside the original run's [message history](../message-history/). Note that you need to add `DeferredToolRequests` to the `Agent`'s or `agent.run()`'s [`output_type`](../output/#structured-output) so that the possible types of the agent run output are correctly inferred. For more information, see the [Deferred Tools](../deferred-tools/#deferred-tools) documentation. To demonstrate, let us first define a simple agent *without* deferred tools: deferred_toolset_agent.py ```python from pydantic import BaseModel from pydantic_ai import Agent, FunctionToolset toolset = FunctionToolset() @toolset.tool def get_default_language(): return 'en-US' @toolset.tool def get_user_name(): return 'David' class PersonalizedGreeting(BaseModel): greeting: str language_code: str agent = Agent('openai:gpt-4o', toolsets=[toolset], output_type=PersonalizedGreeting) result = agent.run_sync('Greet the user in a personalized way') print(repr(result.output)) #> PersonalizedGreeting(greeting='Hello, David!', language_code='en-US') ``` Next, let's define a function that represents a hypothetical "run agent" API endpoint that can be called by the frontend and takes a list of messages to send to the model, a list of frontend tool definitions, and optional deferred tool results. This is where `ExternalToolset`, `DeferredToolRequests`, and `DeferredToolResults` come in: deferred_toolset_api.py ```python from pydantic_ai import ( DeferredToolRequests, DeferredToolResults, ExternalToolset, ModelMessage, ToolDefinition, ) from deferred_toolset_agent import PersonalizedGreeting, agent def run_agent( messages: list[ModelMessage] = [], frontend_tools: list[ToolDefinition] = {}, deferred_tool_results: DeferredToolResults | None = None, ) -> tuple[PersonalizedGreeting | DeferredToolRequests, list[ModelMessage]]: deferred_toolset = ExternalToolset(frontend_tools) result = agent.run_sync( toolsets=[deferred_toolset], # (1)! output_type=[agent.output_type, DeferredToolRequests], # (2)! message_history=messages, # (3)! deferred_tool_results=deferred_tool_results, ) return result.output, result.new_messages() ``` 1. As mentioned in the [Deferred Tools](../deferred-tools/#deferred-tools) documentation, these `toolsets` are additional to those provided to the `Agent` constructor 1. As mentioned in the [Deferred Tools](../deferred-tools/#deferred-tools) documentation, this `output_type` overrides the one provided to the `Agent` constructor, so we have to make sure to not lose it 1. We don't include an `user_prompt` keyword argument as we expect the frontend to provide it via `messages` Now, imagine that the code below is implemented on the frontend, and `run_agent` stands in for an API call to the backend that runs the agent. This is where we actually execute the deferred tool calls and start a new run with the new result included: deferred_tools.py ```python from pydantic_ai import ( DeferredToolRequests, DeferredToolResults, ModelMessage, ModelRequest, ModelRetry, ToolDefinition, UserPromptPart, ) from deferred_toolset_api import run_agent frontend_tool_definitions = [ ToolDefinition( name='get_preferred_language', parameters_json_schema={'type': 'object', 'properties': {'default_language': {'type': 'string'}}}, description="Get the user's preferred language from their browser", ) ] def get_preferred_language(default_language: str) -> str: return 'es-MX' # (1)! frontend_tool_functions = {'get_preferred_language': get_preferred_language} messages: list[ModelMessage] = [ ModelRequest( parts=[ UserPromptPart(content='Greet the user in a personalized way') ] ) ] deferred_tool_results: DeferredToolResults | None = None final_output = None while True: output, new_messages = run_agent(messages, frontend_tool_definitions, deferred_tool_results) messages += new_messages if not isinstance(output, DeferredToolRequests): final_output = output break print(output.calls) """ [ ToolCallPart( tool_name='get_preferred_language', args={'default_language': 'en-US'}, tool_call_id='pyd_ai_tool_call_id', ) ] """ deferred_tool_results = DeferredToolResults() for tool_call in output.calls: if function := frontend_tool_functions.get(tool_call.tool_name): result = function(**tool_call.args_as_dict()) else: result = ModelRetry(f'Unknown tool {tool_call.tool_name!r}') deferred_tool_results.calls[tool_call.tool_call_id] = result print(repr(final_output)) """ PersonalizedGreeting(greeting='Hola, David! Espero que tengas un gran día!', language_code='es-MX') """ ``` 1. Imagine that this returns the frontend [`navigator.language`](https://developer.mozilla.org/en-US/docs/Web/API/Navigator/language). *(This example is complete, it can be run "as is")* ## Dynamically Building a Toolset Toolsets can be built dynamically ahead of each agent run or run step using a function that takes the agent run context and returns a toolset or `None`. This is useful when a toolset (like an MCP server) depends on information specific to an agent run, like its [dependencies](../dependencies/). To register a dynamic toolset, you can pass a function that takes RunContext to the `toolsets` argument of the `Agent` constructor, or you can wrap a compliant function in the @agent.toolset decorator. By default, the function will be called again ahead of each agent run step. If you are using the decorator, you can optionally provide a `per_run_step=False` argument to indicate that the toolset only needs to be built once for the entire run. dynamic_toolset.py ```python from dataclasses import dataclass from typing import Literal from pydantic_ai import Agent, RunContext from pydantic_ai.models.test import TestModel from function_toolset import datetime_toolset, weather_toolset @dataclass class ToggleableDeps: active: Literal['weather', 'datetime'] def toggle(self): if self.active == 'weather': self.active = 'datetime' else: self.active = 'weather' test_model = TestModel() # (1)! agent = Agent( test_model, deps_type=ToggleableDeps # (2)! ) @agent.toolset def toggleable_toolset(ctx: RunContext[ToggleableDeps]): if ctx.deps.active == 'weather': return weather_toolset else: return datetime_toolset @agent.tool def toggle(ctx: RunContext[ToggleableDeps]): ctx.deps.toggle() deps = ToggleableDeps('weather') result = agent.run_sync('Toggle the toolset', deps=deps) print([t.name for t in test_model.last_model_request_parameters.function_tools]) # (3)! #> ['toggle', 'now'] result = agent.run_sync('Toggle the toolset', deps=deps) print([t.name for t in test_model.last_model_request_parameters.function_tools]) #> ['toggle', 'temperature_celsius', 'temperature_fahrenheit', 'conditions'] ``` 1. We're using TestModel here because it makes it easy to see which tools were available on each run. 1. We're using the agent's dependencies to give the `toggle` tool access to the `active` via the `RunContext` argument. 1. This shows the available tools *after* the `toggle` tool was executed, as the "last model request" was the one that returned the `toggle` tool result to the model. *(This example is complete, it can be run "as is")* ## Building a Custom Toolset To define a fully custom toolset with its own logic to list available tools and handle them being called, you can subclass AbstractToolset and implement the get_tools() and call_tool() methods. If you want to reuse a network connection or session across tool listings and calls during an agent run, you can implement __aenter__() and __aexit__(). ## Third-Party Toolsets ### MCP Servers See the [MCP Client](../mcp/client/) documentation for how to use MCP servers with Pydantic AI. ### LangChain Tools If you'd like to use tools or a [toolkit](https://python.langchain.com/docs/concepts/tools/#toolkits) from LangChain's [community tool library](https://python.langchain.com/docs/integrations/tools/) with Pydantic AI, you can use the LangChainToolset which takes a list of LangChain tools. Note that Pydantic AI will not validate the arguments in this case -- it's up to the model to provide arguments matching the schema specified by the LangChain tool, and up to the LangChain tool to raise an error if the arguments are invalid. You will need to install the `langchain-community` package and any others required by the tools in question. ```python from langchain_community.agent_toolkits import SlackToolkit from pydantic_ai import Agent from pydantic_ai.ext.langchain import LangChainToolset toolkit = SlackToolkit() toolset = LangChainToolset(toolkit.get_tools()) agent = Agent('openai:gpt-4o', toolsets=[toolset]) # ... ``` ### ACI.dev Tools If you'd like to use tools from the [ACI.dev tool library](https://www.aci.dev/tools) with Pydantic AI, you can use the ACIToolset [toolset](./) which takes a list of ACI tool names as well as the `linked_account_owner_id`. Note that Pydantic AI will not validate the arguments in this case -- it's up to the model to provide arguments matching the schema specified by the ACI tool, and up to the ACI tool to raise an error if the arguments are invalid. You will need to install the `aci-sdk` package, set your ACI API key in the `ACI_API_KEY` environment variable, and pass your ACI "linked account owner ID" to the function. ```python import os from pydantic_ai import Agent from pydantic_ai.ext.aci import ACIToolset toolset = ACIToolset( [ 'OPEN_WEATHER_MAP__CURRENT_WEATHER', 'OPEN_WEATHER_MAP__FORECAST', ], linked_account_owner_id=os.getenv('LINKED_ACCOUNT_OWNER_ID'), ) agent = Agent('openai:gpt-4o', toolsets=[toolset]) ``` # Models # Anthropic ## Install To use `AnthropicModel` models, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `anthropic` optional group: ```bash pip install "pydantic-ai-slim[anthropic]" ``` ```bash uv add "pydantic-ai-slim[anthropic]" ``` ## Configuration To use [Anthropic](https://anthropic.com) through their API, go to [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys) to generate an API key. `AnthropicModelName` contains a list of available Anthropic models. ## Environment variable Once you have the API key, you can set it as an environment variable: ```bash export ANTHROPIC_API_KEY='your-api-key' ``` You can then use `AnthropicModel` by name: ```python from pydantic_ai import Agent agent = Agent('anthropic:claude-3-5-sonnet-latest') ... ``` Or initialise the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel model = AnthropicModel('claude-3-5-sonnet-latest') agent = Agent(model) ... ``` ## `provider` argument You can provide a custom `Provider` via the `provider` argument: ```python from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.providers.anthropic import AnthropicProvider model = AnthropicModel( 'claude-3-5-sonnet-latest', provider=AnthropicProvider(api_key='your-api-key') ) agent = Agent(model) ... ``` ## Custom HTTP Client You can customize the `AnthropicProvider` with a custom `httpx.AsyncClient`: ```python from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.providers.anthropic import AnthropicProvider custom_http_client = AsyncClient(timeout=30) model = AnthropicModel( 'claude-3-5-sonnet-latest', provider=AnthropicProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ... ``` # Bedrock ## Install To use `BedrockConverseModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `bedrock` optional group: ```bash pip install "pydantic-ai-slim[bedrock]" ``` ```bash uv add "pydantic-ai-slim[bedrock]" ``` ## Configuration To use [AWS Bedrock](https://aws.amazon.com/bedrock/), you'll need an AWS account with Bedrock enabled and appropriate credentials. You can use either AWS credentials directly or a pre-configured boto3 client. `BedrockModelName` contains a list of available Bedrock models, including models from Anthropic, Amazon, Cohere, Meta, and Mistral. ## Environment variables You can set your AWS credentials as environment variables ([among other options](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#using-environment-variables)): ```bash export AWS_BEARER_TOKEN_BEDROCK='your-api-key' # or: export AWS_ACCESS_KEY_ID='your-access-key' export AWS_SECRET_ACCESS_KEY='your-secret-key' export AWS_DEFAULT_REGION='us-east-1' # or your preferred region ``` You can then use `BedrockConverseModel` by name: ```python from pydantic_ai import Agent agent = Agent('bedrock:anthropic.claude-3-sonnet-20240229-v1:0') ... ``` Or initialize the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel model = BedrockConverseModel('anthropic.claude-3-sonnet-20240229-v1:0') agent = Agent(model) ... ``` ## Customizing Bedrock Runtime API You can customize the Bedrock Runtime API calls by adding additional parameters, such as [guardrail configurations](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) and [performance settings](https://docs.aws.amazon.com/bedrock/latest/userguide/latency-optimized-inference.html). For a complete list of configurable parameters, refer to the documentation for BedrockModelSettings. customize_bedrock_model_settings.py ```python from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel, BedrockModelSettings # Define Bedrock model settings with guardrail and performance configurations bedrock_model_settings = BedrockModelSettings( bedrock_guardrail_config={ 'guardrailIdentifier': 'v1', 'guardrailVersion': 'v1', 'trace': 'enabled' }, bedrock_performance_configuration={ 'latency': 'optimized' } ) model = BedrockConverseModel(model_name='us.amazon.nova-pro-v1:0') agent = Agent(model=model, model_settings=bedrock_model_settings) ``` ## `provider` argument You can provide a custom `BedrockProvider` via the `provider` argument. This is useful when you want to specify credentials directly or use a custom boto3 client: ```python from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel from pydantic_ai.providers.bedrock import BedrockProvider # Using AWS credentials directly model = BedrockConverseModel( 'anthropic.claude-3-sonnet-20240229-v1:0', provider=BedrockProvider( region_name='us-east-1', aws_access_key_id='your-access-key', aws_secret_access_key='your-secret-key', ), ) agent = Agent(model) ... ``` You can also pass a pre-configured boto3 client: ```python import boto3 from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel from pydantic_ai.providers.bedrock import BedrockProvider # Using a pre-configured boto3 client bedrock_client = boto3.client('bedrock-runtime', region_name='us-east-1') model = BedrockConverseModel( 'anthropic.claude-3-sonnet-20240229-v1:0', provider=BedrockProvider(bedrock_client=bedrock_client), ) agent = Agent(model) ... ``` # Cohere ## Install To use `CohereModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `cohere` optional group: ```bash pip install "pydantic-ai-slim[cohere]" ``` ```bash uv add "pydantic-ai-slim[cohere]" ``` ## Configuration To use [Cohere](https://cohere.com/) through their API, go to [dashboard.cohere.com/api-keys](https://dashboard.cohere.com/api-keys) and follow your nose until you find the place to generate an API key. `CohereModelName` contains a list of the most popular Cohere models. ## Environment variable Once you have the API key, you can set it as an environment variable: ```bash export CO_API_KEY='your-api-key' ``` You can then use `CohereModel` by name: ```python from pydantic_ai import Agent agent = Agent('cohere:command') ... ``` Or initialise the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel model = CohereModel('command') agent = Agent(model) ... ``` ## `provider` argument You can provide a custom `Provider` via the `provider` argument: ```python from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel from pydantic_ai.providers.cohere import CohereProvider model = CohereModel('command', provider=CohereProvider(api_key='your-api-key')) agent = Agent(model) ... ``` You can also customize the `CohereProvider` with a custom `http_client`: ```python from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel from pydantic_ai.providers.cohere import CohereProvider custom_http_client = AsyncClient(timeout=30) model = CohereModel( 'command', provider=CohereProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ... ``` # Google The `GoogleModel` is a model that uses the [`google-genai`](https://pypi.org/project/google-genai/) package under the hood to access Google's Gemini models via both the Generative Language API and Vertex AI. ## Install To use `GoogleModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `google` optional group: ```bash pip install "pydantic-ai-slim[google]" ``` ```bash uv add "pydantic-ai-slim[google]" ``` ______________________________________________________________________ ## Configuration `GoogleModel` lets you use Google's Gemini models through their [Generative Language API](https://ai.google.dev/api/all-methods) (`generativelanguage.googleapis.com`) or [Vertex AI API](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models) (`*-aiplatform.googleapis.com`). ### API Key (Generative Language API) To use Gemini via the Generative Language API, go to [aistudio.google.com](https://aistudio.google.com/apikey) and create an API key. Once you have the API key, set it as an environment variable: ```bash export GOOGLE_API_KEY=your-api-key ``` You can then use `GoogleModel` by explicitly creating a provider: ```python from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider provider = GoogleProvider(api_key='your-api-key') model = GoogleModel('gemini-1.5-flash', provider=provider) agent = Agent(model) ... ``` ### Vertex AI (Enterprise/Cloud) If you are an enterprise user, you can use the `google-vertex` provider with `GoogleModel` to access Gemini via Vertex AI. This interface has a number of advantages over the Generative Language API: 1. The VertexAI API comes with more enterprise readiness guarantees. 1. You can [purchase provisioned throughput](https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput#purchase-provisioned-throughput) with VertexAI to guarantee capacity. 1. If you're running Pydantic AI inside GCP, you don't need to set up authentication, it should "just work". 1. You can decide which region to use, which might be important from a regulatory perspective, and might improve latency. The big disadvantage is that for local development you may need to create and configure a "service account", which can be challenging to get right. Whichever way you authenticate, you'll need to have VertexAI enabled in your GCP account. To use Vertex AI, you may need to set up [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) or use a service account. You can also specify the region. #### Application Default Credentials If you have the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) installed and configured, you can use: ```python from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider provider = GoogleProvider(vertexai=True) model = GoogleModel('gemini-1.5-flash', provider=provider) agent = Agent(model) ... ``` #### Service Account To use a service account JSON file: google_model_service_account.py ```python from google.oauth2 import service_account from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider credentials = service_account.Credentials.from_service_account_file( 'path/to/service-account.json', scopes=['https://www.googleapis.com/auth/cloud-platform'], ) provider = GoogleProvider(credentials=credentials, project='your-project-id') model = GoogleModel('gemini-1.5-flash', provider=provider) agent = Agent(model) ... ``` #### Customizing Location You can specify the location when using Vertex AI: google_model_location.py ```python from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider provider = GoogleProvider(vertexai=True, location='asia-east1') model = GoogleModel('gemini-1.5-flash', provider=provider) agent = Agent(model) ... ``` #### Customizing Model You can access models from the [Model Garden](https://cloud.google.com/model-garden?hl=en) that support the generateContent API and are available under your GCP project, including but not limited to Gemini, using one of the following `model_name` patterns: - `{model_id}` for Gemini models - `{publisher}/{model_id}` - `publishers/{publisher}/models/{model_id}` - `projects/{project}/locations/{location}/publishers/{publisher}/models/{model_id}` ```python from google.oauth2 import service_account from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider credentials = service_account.Credentials.from_service_account_file( 'path/to/service-account.json', scopes=['https://www.googleapis.com/auth/cloud-platform'], ) provider = GoogleProvider( credentials=credentials, project='your-gcp-project-id', location='us-central1', # the region where the model is available ) model = GoogleModel('meta/llama-3.3-70b-instruct-maas', provider=provider) agent = Agent(model) ... ``` ## Provider Argument You can supply a custom `GoogleProvider` instance using the `provider` argument to configure advanced client options, such as setting a custom `base_url`. This is useful if you're using a custom-compatible endpoint with the Google Generative Language API. ```python from google.genai import Client from google.genai.types import HttpOptions from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel from pydantic_ai.providers.google import GoogleProvider client = Client( api_key='gemini-custom-api-key', http_options=HttpOptions(base_url='gemini-custom-base-url'), ) provider = GoogleProvider(client=client) model = GoogleModel('gemini-1.5-flash', provider=provider) agent = Agent(model) ... ``` ## Model Settings You can customize model behavior using GoogleModelSettings: ```python from google.genai.types import HarmBlockThreshold, HarmCategory from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel, GoogleModelSettings settings = GoogleModelSettings( temperature=0.2, max_tokens=1024, google_thinking_config={'thinking_budget': 2048}, google_safety_settings=[ { 'category': HarmCategory.HARM_CATEGORY_HATE_SPEECH, 'threshold': HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, } ] ) model = GoogleModel('gemini-1.5-flash') agent = Agent(model, model_settings=settings) ... ``` See the [Gemini API docs](https://ai.google.dev/gemini-api/docs/safety-settings) for more on safety settings, and [thinking config](https://ai.google.dev/gemini-api/docs/thinking). ## Document, Image, Audio, and Video Input `GoogleModel` supports multi-modal input, including documents, images, audio, and video. See the [input documentation](../../input/) for details and examples. ## Model settings You can use the GoogleModelSettings class to customize the model request. ### Disable thinking You can disable thinking by setting the `thinking_budget` to `0` on the `google_thinking_config`: ```python from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel, GoogleModelSettings model_settings = GoogleModelSettings(google_thinking_config={'thinking_budget': 0}) model = GoogleModel('gemini-2.0-flash') agent = Agent(model, model_settings=model_settings) ... ``` Check out the [Gemini API docs](https://ai.google.dev/gemini-api/docs/thinking) for more on thinking. ### Safety settings You can customize the safety settings by setting the `google_safety_settings` field. ```python from google.genai.types import HarmBlockThreshold, HarmCategory from pydantic_ai import Agent from pydantic_ai.models.google import GoogleModel, GoogleModelSettings model_settings = GoogleModelSettings( google_safety_settings=[ { 'category': HarmCategory.HARM_CATEGORY_HATE_SPEECH, 'threshold': HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, } ] ) model = GoogleModel('gemini-2.0-flash') agent = Agent(model, model_settings=model_settings) ... ``` See the [Gemini API docs](https://ai.google.dev/gemini-api/docs/safety-settings) for more on safety settings. # Groq ## Install To use `GroqModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `groq` optional group: ```bash pip install "pydantic-ai-slim[groq]" ``` ```bash uv add "pydantic-ai-slim[groq]" ``` ## Configuration To use [Groq](https://groq.com/) through their API, go to [console.groq.com/keys](https://console.groq.com/keys) and follow your nose until you find the place to generate an API key. `GroqModelName` contains a list of available Groq models. ## Environment variable Once you have the API key, you can set it as an environment variable: ```bash export GROQ_API_KEY='your-api-key' ``` You can then use `GroqModel` by name: ```python from pydantic_ai import Agent agent = Agent('groq:llama-3.3-70b-versatile') ... ``` Or initialise the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel model = GroqModel('llama-3.3-70b-versatile') agent = Agent(model) ... ``` ## `provider` argument You can provide a custom `Provider` via the `provider` argument: ```python from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel from pydantic_ai.providers.groq import GroqProvider model = GroqModel( 'llama-3.3-70b-versatile', provider=GroqProvider(api_key='your-api-key') ) agent = Agent(model) ... ``` You can also customize the `GroqProvider` with a custom `httpx.AsyncHTTPClient`: ```python from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel from pydantic_ai.providers.groq import GroqProvider custom_http_client = AsyncClient(timeout=30) model = GroqModel( 'llama-3.3-70b-versatile', provider=GroqProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ... ``` # Hugging Face [Hugging Face](https://huggingface.co/) is an AI platform with all major open source models, datasets, MCPs, and demos. You can use [Inference Providers](https://huggingface.co/docs/inference-providers) to run open source models like DeepSeek R1 on scalable serverless infrastructure. ## Install To use `HuggingFaceModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `huggingface` optional group: ```bash pip install "pydantic-ai-slim[huggingface]" ``` ```bash uv add "pydantic-ai-slim[huggingface]" ``` ## Configuration To use [Hugging Face](https://huggingface.co/) inference, you'll need to set up an account which will give you [free tier](https://huggingface.co/docs/inference-providers/pricing) allowance on [Inference Providers](https://huggingface.co/docs/inference-providers). To setup inference, follow these steps: 1. Go to [Hugging Face](https://huggingface.co/join) and sign up for an account. 1. Create a new access token in [Hugging Face](https://huggingface.co/settings/tokens). 1. Set the `HF_TOKEN` environment variable to the token you just created. Once you have a Hugging Face access token, you can set it as an environment variable: ```bash export HF_TOKEN='hf_token' ``` ## Usage You can then use HuggingFaceModel by name: ```python from pydantic_ai import Agent agent = Agent('huggingface:Qwen/Qwen3-235B-A22B') ... ``` Or initialise the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.huggingface import HuggingFaceModel model = HuggingFaceModel('Qwen/Qwen3-235B-A22B') agent = Agent(model) ... ``` By default, the HuggingFaceModel uses the HuggingFaceProvider that will select automatically the first of the inference providers (Cerebras, Together AI, Cohere..etc) available for the model, sorted by your preferred order in https://hf.co/settings/inference-providers. ## Configure the provider If you want to pass parameters in code to the provider, you can programmatically instantiate the HuggingFaceProvider and pass it to the model: ```python from pydantic_ai import Agent from pydantic_ai.models.huggingface import HuggingFaceModel from pydantic_ai.providers.huggingface import HuggingFaceProvider model = HuggingFaceModel('Qwen/Qwen3-235B-A22B', provider=HuggingFaceProvider(api_key='hf_token', provider_name='nebius')) agent = Agent(model) ... ``` ## Custom Hugging Face client HuggingFaceProvider also accepts a custom [`AsyncInferenceClient`](https://huggingface.co/docs/huggingface_hub/v0.29.3/en/package_reference/inference_client#huggingface_hub.AsyncInferenceClient) client via the `hf_client` parameter, so you can customise the `headers`, `bill_to` (billing to an HF organization you're a member of), `base_url` etc. as defined in the [Hugging Face Hub python library docs](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client). ```python from huggingface_hub import AsyncInferenceClient from pydantic_ai import Agent from pydantic_ai.models.huggingface import HuggingFaceModel from pydantic_ai.providers.huggingface import HuggingFaceProvider client = AsyncInferenceClient( bill_to='openai', api_key='hf_token', provider='fireworks-ai', ) model = HuggingFaceModel( 'Qwen/Qwen3-235B-A22B', provider=HuggingFaceProvider(hf_client=client), ) agent = Agent(model) ... ``` # Mistral ## Install To use `MistralModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `mistral` optional group: ```bash pip install "pydantic-ai-slim[mistral]" ``` ```bash uv add "pydantic-ai-slim[mistral]" ``` ## Configuration To use [Mistral](https://mistral.ai) through their API, go to [console.mistral.ai/api-keys/](https://console.mistral.ai/api-keys/) and follow your nose until you find the place to generate an API key. `LatestMistralModelNames` contains a list of the most popular Mistral models. ## Environment variable Once you have the API key, you can set it as an environment variable: ```bash export MISTRAL_API_KEY='your-api-key' ``` You can then use `MistralModel` by name: ```python from pydantic_ai import Agent agent = Agent('mistral:mistral-large-latest') ... ``` Or initialise the model directly with just the model name: ```python from pydantic_ai import Agent from pydantic_ai.models.mistral import MistralModel model = MistralModel('mistral-small-latest') agent = Agent(model) ... ``` ## `provider` argument You can provide a custom `Provider` via the `provider` argument: ```python from pydantic_ai import Agent from pydantic_ai.models.mistral import MistralModel from pydantic_ai.providers.mistral import MistralProvider model = MistralModel( 'mistral-large-latest', provider=MistralProvider(api_key='your-api-key', base_url='https://
(Union)") --- ModelRequest ModelRequest("ModelRequest(parts=list[...])") --- ModelMessage ModelResponsePart("ModelResponsePart
(Union)") --- ModelResponse ModelResponse("ModelResponse(parts=list[...])") --- ModelMessage("ModelMessage
(Union)") ``` ### FinishReason ```python FinishReason: TypeAlias = Literal[ "stop", "length", "content_filter", "tool_call", "error" ] ``` Reason the model finished generating the response, normalized to OpenTelemetry values. ### SystemPromptPart A system prompt, generally written by the application developer. This gives the model context and guidance on how to respond. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` ```python @dataclass(repr=False) class SystemPromptPart: """A system prompt, generally written by the application developer. This gives the model context and guidance on how to respond. """ content: str """The content of the prompt.""" _: KW_ONLY timestamp: datetime = field(default_factory=_now_utc) """The timestamp of the prompt.""" dynamic_ref: str | None = None """The ref of the dynamic system prompt function that generated this part. Only set if system prompt is dynamic, see [`system_prompt`][pydantic_ai.Agent.system_prompt] for more information. """ part_kind: Literal['system-prompt'] = 'system-prompt' """Part type identifier, this is available on all parts as a discriminator.""" def otel_event(self, settings: InstrumentationSettings) -> Event: return Event( 'gen_ai.system.message', body={'role': 'system', **({'content': self.content} if settings.include_content else {})}, ) def otel_message_parts(self, settings: InstrumentationSettings) -> list[_otel_messages.MessagePart]: return [_otel_messages.TextPart(type='text', **{'content': self.content} if settings.include_content else {})] __repr__ = _utils.dataclasses_no_defaults_repr ``` #### content ```python content: str ``` The content of the prompt. #### timestamp ```python timestamp: datetime = field(default_factory=now_utc) ``` The timestamp of the prompt. #### dynamic_ref ```python dynamic_ref: str | None = None ``` The ref of the dynamic system prompt function that generated this part. Only set if system prompt is dynamic, see system_prompt for more information. #### part_kind ```python part_kind: Literal['system-prompt'] = 'system-prompt' ``` Part type identifier, this is available on all parts as a discriminator. ### FileUrl Bases: `ABC` Abstract base class for any URL-based file. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` ```python @dataclass(init=False, repr=False) class FileUrl(ABC): """Abstract base class for any URL-based file.""" url: str """The URL of the file.""" _: KW_ONLY force_download: bool = False """For OpenAI and Google APIs it: * If True, the file is downloaded and the data is sent to the model as bytes. * If False, the URL is sent directly to the model and no download is performed. """ vendor_metadata: dict[str, Any] | None = None """Vendor-specific metadata for the file. Supported by: - `GoogleModel`: `VideoUrl.vendor_metadata` is used as `video_metadata`: https://ai.google.dev/gemini-api/docs/video-understanding#customize-video-processing - `OpenAIChatModel`, `OpenAIResponsesModel`: `ImageUrl.vendor_metadata['detail']` is used as `detail` setting for images """ _media_type: Annotated[str | None, pydantic.Field(alias='media_type', default=None, exclude=True)] = field( compare=False, default=None ) _identifier: Annotated[str | None, pydantic.Field(alias='identifier', default=None, exclude=True)] = field( compare=False, default=None ) def __init__( self, url: str, *, media_type: str | None = None, identifier: str | None = None, force_download: bool = False, vendor_metadata: dict[str, Any] | None = None, ) -> None: self.url = url self._media_type = media_type self._identifier = identifier self.force_download = force_download self.vendor_metadata = vendor_metadata @pydantic.computed_field @property def media_type(self) -> str: """Return the media type of the file, based on the URL or the provided `media_type`.""" return self._media_type or self._infer_media_type() @pydantic.computed_field @property def identifier(self) -> str: """The identifier of the file, such as a unique ID. This identifier can be provided to the model in a message to allow it to refer to this file in a tool call argument, and the tool can look up the file in question by iterating over the message history and finding the matching `FileUrl`. This identifier is only automatically passed to the model when the `FileUrl` is returned by a tool. If you're passing the `FileUrl` as a user message, it's up to you to include a separate text part with the identifier, e.g. "This is file
