Models and providers

ModelsPlugin provides a ModelRegistry that gives your systems access to LLMs through a provider-agnostic interface. Provider plugins register themselves with the registry, and your systems use a single API regardless of which provider is behind it.

For application code, treat polaris_ai::models as the public entrypoint. You do not need to wire polaris_models and polaris_model_providers manually unless you are working inside the Polaris workspace itself.

use polaris_ai::system::server::Server;
use polaris_ai::models::ModelsPlugin;
use polaris_ai::models::AnthropicPlugin;

let mut server = Server::new();
server
    .add_plugins(ModelsPlugin)
    .add_plugins(AnthropicPlugin::from_env("ANTHROPIC_API_KEY"));

Enable the provider you use

Provider implementations are feature-gated. For the Anthropic examples on this page, add the provider feature to the pinned dependency from Installation:

[dependencies]
polaris-ai = { git = "https://github.com/RadiantAILabs/polaris", rev = "5ff84646db63589d58bf87956976d749b8c754de", features = ["anthropic"] }

Use openai or bedrock instead for the corresponding provider. Register the provider before awaiting server.finish(). Supplying an API key without enabling the feature will not make its plugin available.

ModelsPlugin

ModelsPlugin inserts a ModelRegistry as a global resource. It does not provide any models on its own. You add one or more provider plugins alongside it, and they register their providers into the registry during startup.

The registry uses a two-phase lifecycle. During the build phase, provider plugins get mutable access to register themselves. During the ready phase, the registry is frozen into an immutable global resource available to all systems.

Provider plugins declare ModelsPlugin as a dependency. In normal code you should still register ModelsPlugin alongside them, but the server resolves the ordering for you during startup.

Provider plugins

Polaris ships three provider plugins.

AnthropicPlugin connects to the Anthropic Messages API. Create it with from_env, passing the name of the environment variable that holds your API key.

use polaris_ai::models::AnthropicPlugin;

server.add_plugins(AnthropicPlugin::from_env("ANTHROPIC_API_KEY"));

OpenAiPlugin connects to the OpenAI Responses API. It follows the same pattern.

use polaris_ai::models::OpenAiPlugin;

server.add_plugins(OpenAiPlugin::from_env("OPENAI_API_KEY"));

BedrockPlugin connects to AWS Bedrock. It uses the standard AWS credential chain by default, or you can pass a pre-configured SDK config.

use polaris_ai::models::BedrockPlugin;

// Default AWS credential chain
server.add_plugins(BedrockPlugin::from_env());

// Or with explicit SDK config
let sdk_config = aws_config::defaults(aws_config::BehaviorVersion::latest())
    .region("us-west-2")
    .load()
    .await;
server.add_plugins(BedrockPlugin::from_sdk_config(sdk_config));

Getting an Llm handle

In a system, you access the registry through Res<ModelRegistry> and call registry.llm() with a "provider/model" identifier. This returns an Llm handle bound to that specific provider and model.

use polaris_ai::system::system;
use polaris_ai::system::param::Res;
use polaris_ai::system::prelude::SystemError;
use polaris_ai::models::ModelRegistry;
use polaris_ai::models::llm::Llm;

#[system]
async fn summarize(registry: Res<ModelRegistry>) -> Result<String, SystemError> {
    let llm: Llm = registry
        .llm("anthropic/claude-sonnet-4-20250514")
        .map_err(|e| SystemError::ExecutionError(e.to_string()))?;

    let response = llm
        .builder()
        .system("You are a concise summarizer.")
        .user("Summarize the benefits of Rust's ownership model.")
        .generate()
        .await
        .map_err(|e| SystemError::ExecutionError(e.to_string()))?;

    Ok(response.text())
}

The model identifier follows the format "provider/model" where the provider name matches what the provider plugin registers (for example, "anthropic", "openai", or "bedrock").

LlmRequestBuilder

Llm::builder() returns an LlmRequestBuilder that uses typestate to ensure at least one message is present before you can send a request. You chain .system(), .user(), .assistant(), and .messages() calls, then call .generate() to send the request.

let response = llm
    .builder()
    .system("You are helpful.")
    .user("What is the capital of France?")
    .generate()
    .await
    .map_err(|e| SystemError::ExecutionError(e.to_string()))?;

The builder also supports .stream() for streaming responses, and .generate_structured::<T>() for typed output that conforms to a JSON schema derived from your type.

If you are using tools, the LlmRequestBuilderExt trait from polaris_ai::tools adds .with_registry(&tool_registry), which attaches all registered tool definitions to the request. See Tool calling for details.

LlmRequest and LlmResponse

LlmRequest holds the full payload sent to the provider: a system prompt, a list of Message values, optional tool definitions, and optional tool choice constraints. You rarely need to construct one directly since the builder handles it, but you can if you need full control.

LlmResponse contains the generated content blocks, token usage, and the reason generation stopped. The most common methods are:

• response.text() returns all text content blocks concatenated into a single string.
• response.has_tool_calls() returns true if the model called any tools.
• response.tool_calls() returns a Vec<&ToolCall> of all tool call blocks.
• response.stop_reason is a StopReason enum: EndTurn, MaxOutputTokens, StopSequence, ToolUse, ContentFilter, or Other(String).

Messages

The Message enum represents conversation turns. Each message is either User or Assistant and contains one or more content blocks.

Convenience constructors cover the common cases:

use polaris_ai::models::llm::{Message, ToolResultContent};

// Text messages
let user_msg = Message::user("Hello, what can you help me with?");
let assistant_msg = Message::assistant("I can help with many things.");

// Tool results (sent as user messages)
let tool_result = Message::tool_result(
    "tool_call_id",
    ToolResultContent::Text("72 degrees and sunny".to_string()),
);

// Tool errors
let tool_error = Message::tool_error(
    "tool_call_id",
    ToolResultContent::Text("location not found".to_string()),
);

User messages can contain text, images, audio, documents, and tool results. Assistant messages can contain text, tool calls, and reasoning blocks.

Reading tool calls from a response

When the model decides to call tools, the response’s stop_reason will be StopReason::ToolUse and the content will include ToolCall blocks. Each ToolCall has an id, a function.name, and function.arguments as a JSON value.

use polaris_ai::models::llm::{LlmResponse, Message, ToolResultContent, UserBlock};
use polaris_ai::tools::ToolRegistry;

async fn handle_tool_calls(
    response: &LlmResponse,
    tool_registry: &ToolRegistry,
) -> Vec<UserBlock> {
    let mut result_blocks = Vec::new();

    for tool_call in response.tool_calls() {
        let block = match tool_registry
            .execute(&tool_call.function.name, &tool_call.function.arguments)
            .await
        {
            Ok(value) => {
                let output = value
                    .as_str()
                    .map(String::from)
                    .unwrap_or_else(|| value.to_string());
                UserBlock::tool_result(&tool_call.id, ToolResultContent::Text(output))
            }
            Err(err) => {
                UserBlock::tool_error(
                    &tool_call.id,
                    ToolResultContent::Text(err.to_string()),
                )
            }
        };
        result_blocks.push(block);
    }

    result_blocks
}

You then wrap the result blocks in a Message::User { content: result_blocks } and append it to the conversation history for the next LLM call. The API requires every tool call in an assistant message to have a corresponding tool result in the following user message.

Swapping providers

Because systems interact with the ModelRegistry and Llm abstractions rather than provider-specific types, changing providers is a configuration change. Your systems remain identical.

use polaris_ai::models::ModelsPlugin;
use polaris_ai::models::AnthropicPlugin;
use polaris_ai::models::OpenAiPlugin;

let mut server = Server::new();
server.add_plugins(ModelsPlugin);

// Use Anthropic
server.add_plugins(AnthropicPlugin::from_env("ANTHROPIC_API_KEY"));
// Or swap to OpenAI — no system code changes
// server.add_plugins(OpenAiPlugin::from_env("OPENAI_API_KEY"));

The only thing that changes is the model identifier passed to registry.llm(). A system that calls registry.llm("anthropic/claude-sonnet-4-20250514") works exactly the same way as one calling registry.llm("openai/gpt-4o"), provided the corresponding provider plugin is registered. You can make the model identifier configurable through a resource so that swapping providers requires zero code changes.