How AI Agents Work
Evaluating Agents: SWE-bench, OSWorld and Your Own Tasks
How agent benchmarks work, what their failure analyses reveal and how to test your agent.
Advanced 20 min For: Developers / Researchers ✎ In progress
✎ This lesson is still being written. Below is the planned outline; it is not indexed by search engines yet.
What this lesson will cover
- SWE-bench family
- OSWorld and computer use
- τ-bench and tool use
- Trace-based evaluation