Types
Reference for core data structures.
Tally uses a set of canonical types to represent conversations and evaluation data.
Conversation
The primary data structure for evaluation.
interface Conversation {
id: string;
steps: ConversationStep[];
metadata?: Record<string, any>;
}
interface ConversationStep {
stepIndex: number;
input: ModelMessage;
output: ModelMessage[];
timestamp: Date;
metadata?: Record<string, any>;
}ModelMessage
Standardized message format, compatible with AI SDK.
type ModelMessage = {
role: 'user' | 'assistant' | 'system' | 'tool';
content: string | Array<TextPart | ToolCallPart | ToolResultPart>;
};StepTrace
A richer version of a conversation step, generated by the Trajectories framework. It includes internal reasoning about step selection.
interface StepTrace {
turnIndex: number;
stepId: string;
instruction: string;
userMessage: ModelMessage;
agentMessage: ModelMessage;
selectionMethod: 'deterministic' | 'llm-ranked' | 'none';
// ...
}EvaluationReport
The final output of a Tally run.
interface EvaluationReport {
evalSummaries: EvalSummary[];
perTargetResults: PerTargetResult[];
metadata: RunMetadata;
}