Testing agents

Start with deterministic tests for your code. Add model evaluations after the graph, tools and plugin wiring behave as intended.

Test one graph without a server or model

In the project from Installation, create tests/graph.rs:

use polaris_ai::graph::{Graph, GraphExecutor};
use polaris_ai::system::param::{Res, SystemContext};
use polaris_ai::system::resource::LocalResource;
use polaris_ai::system::system;

struct Input(String);
impl LocalResource for Input {}

#[system]
async fn uppercase(input: Res<Input>) -> String {
    input.0.to_uppercase()
}

#[tokio::test]
async fn graph_transforms_input() {
    let mut graph = Graph::new();
    graph.add_system(uppercase);
    assert!(graph.validate().is_ok());
    let mut ctx = SystemContext::new().with(Input("hello".into()));
    let result = GraphExecutor::new()
        .execute(&graph, &mut ctx, None, None)
        .await
        .unwrap();
    assert_eq!(result.output::<String>().map(String::as_str), Some("HELLO"));
}

cargo test --test graph

The test provides a local input resource, validates the graph, runs it and checks its typed output. It needs no API key or network request. ValidationResult::is_ok() checks structural errors; inspect warnings separately when reviewing more complex graphs.

For a system-only test, use IntoSystem and a seeded SystemContext instead of constructing a graph. Test graph execution when ordering, branching, loops or error routes are the behaviour under test.

Choose the right level

Test	What it should establish
System	Known resources produce the expected output or error
Tool	Valid inputs work; malformed inputs and out-of-scope actions fail correctly
Plugin	Declared dependencies are sufficient and expected resources appear after startup
Graph	The intended branches, stopping conditions and recovery paths execute
Agent evaluation	Tasks succeed under a fixed protocol at acceptable cost and latency

SystemContext::new().with(value) inserts a local resource and requires LocalResource. For true globals, use Server::insert_global and create a context from that server, or construct a context with a global resource store. Do not add a global-only type through .with().

Keep model behaviour controlled

Use a deterministic provider or recorded responses when testing the model/tool boundary. The current LlmProvider contract includes required methods and types beyond a single generate method; implement the trait against the current provider reference rather than copying an older partial mock.

Cover at least a final answer, a tool request, a tool failure, an invalid response and exhaustion of the loop budget. Assert on the selected path and resulting state. For parallel work, assert on results without assuming completion order.

Evaluate changes separately

Run model evaluations on a versioned task set. Record model identifier, harness revision, prompt and tool configuration, task results, usage and latency. Use the same development tasks to compare a baseline and candidate, with held-out tasks reserved for validation. Keep unsuccessful runs as well as successful ones.

Real provider calls require credentials and may incur charges. Keep them separate from the default unit-test command. Radiant Platform can provide execution evidence for investigation; it does not replace the evaluation protocol.

The public testing reference explains Polaris’s test layers and contracts in more detail.