Trajectories
Multi-turn conversation simulation for LLM agents.
Trajectories is a framework-agnostic tool for generating multi-turn conversations. It allows you to simulate users, run agents, and record conversations for downstream evaluation in Tally.
Why use Trajectories?
- Simulate Users: Define a goal and persona for an LLM to play the role of a user.
- Test Robustness: Generate both "good" and intentionally "bad" (adversarial) trajectories.
- Step Graphs: Guide the conversation through specific milestones or flows.
- Framework Agnostic: Works with AI SDK, Mastra, or custom agent loops via wrappers.
- Export Formats: Seamlessly export to Tally
Conversationor JSONL formats.
Quick Start
import {
createTrajectory,
runTrajectory,
withAISdkAgent,
toConversation
} from '@tally-evals/trajectories';
import { google } from '@ai-sdk/google';
// 1. Wrap your agent (e.g., AI SDK)
const agent = withAISdkAgent(myWeatherAgent);
// 2. Define a trajectory
const trajectory = createTrajectory({
goal: 'Ask about weather in SF and NY',
persona: {
description: 'A helpful user who asks concise questions.',
},
steps: {
steps: [
{ id: 'sf', instruction: 'Ask for weather in San Francisco' },
{ id: 'ny', instruction: 'Follow up with weather in New York' },
],
start: 'sf',
terminals: ['ny'],
},
userModel: google('models/gemini-2.0-flash'),
}, agent);
// 3. Run and export
const result = await runTrajectory(trajectory);
const conversation = toConversation(result, 'weather-run-1');Check out the Step Graphs guide to learn how to design complex conversation flows.