Evaluations and benchmarks

A flow that works in your own test call can still fail on the tenth kind of caller. Two tools help you find out before your customers do.

Evaluations

An evaluation runs a flow against a set of scenarios and grades each conversation against what should have happened — was the appointment booked, was the right field collected, did the assistant stay on script. Scenarios can be written by hand or drawn from real conversations.

Graded checks also run during a conversation: an evaluation node on the canvas can judge what the caller said and branch on the result, using deterministic rules where an exact answer is required and the model where judgement is needed.

Benchmarks

A benchmark compares configurations — different models, prompts or settings — across the same scenarios, and scores the results side by side. Before it runs, a benchmark shows an estimate of what it will cost.

Simulated conversations

Automations can place AI-to-AI conversations against a flow and watch them as they happen, which turns a regression test into something you can run on every change. See Automations.