Brief
New metric pinpoints the turn that broke your multi-turn AI agent
A new blog post introduces the Agent Evaluation Metric (AEM), a turn-level way to measure multi-turn agent quality. It aims to find the specific turn where a failure started, rather than just noting that the conversation went wrong.
Multi-turn AI agents can fail in ways that single-turn evaluation misses: one early mistake corrupts every later turn. The Agent Evaluation Metric (AEM) is a decomposable, turn-level approach to measuring agent quality. Its first dimension, correctness, is used to pinpoint the turn that caused a failure and separate it from the turns that inherited it. This means you can identify the root cause of a conversation breakdown instead of just seeing the final bad output.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by aws.amazon.com
- Multi-turn agents fail in ways single-turn evaluation misses: one early mistake corrupts every later turn.
- The Agent Evaluation Metric (AEM) is a decomposable, turn-level way to measure agent quality.
- AEM is applied to its first dimension, correctness, to pinpoint the turn that caused a failure and separate it from the turns that inherited it.
Sources
- AWS Machine Learning BlogText stored 15 September 2026
How this story was checked. Written from the 1 page listed above, stored 15 September 2026; claims checked against that stored text on 15 September 2026.
What that means
- 3 of 3 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.