Agent Trajectory Evaluation
Agent trajectory evaluation scores the full sequence of steps, tool calls, and intermediate decisions an agent took to reach its final answer — not just whether the final answer was correct — since two agents can reach the same right answer through very different paths, only some of which are actually acceptable.
Final-answer-only evaluation misses a whole category of real bugs: an agent that gets the right answer by luck, by calling an expensive tool ten more times than necessary, or by taking an action a human reviewer would never have approved if they'd seen the intermediate steps. Trajectory evaluation exists specifically to catch these.
In practice this means scoring the transcript against a rubric — did it call the necessary tools, did it avoid unnecessary or unsafe ones, did it recover sensibly from an error instead of looping — often using an LLM-as-a-judge against a written rubric, since trajectories are too varied for simple string matching.
Because a "good enough" trajectory isn't unique, scoring usually grades against a set of acceptable-path criteria rather than one golden trajectory, and treats efficiency (tool calls, tokens, turns) as a separate axis from correctness rather than folding them into one score.
