Open-Source Benchmarks for AI Coding Agents

SWE-bench gives an agent a real GitHub issue and a repository and checks whether its patch passes the project's tests. It became the standard baseline for coding agents.

Top models cluster near the ceiling on the original benchmark, and its tasks can leak into training data, so newer benchmarks are harder, more varied or built to resist contamination.

Terminal-Bench evaluates agents on tasks performed in a shell environment, which patch-based benchmarks do not cover.

ReviewBench is an open benchmark for AI code review agents; it is a research preview, so expect changes.

Match it to the capability you care about rather than the best-known name: SWE-bench for baseline issue resolution, Terminal-Bench for shell work, DeepSWE for contamination resistance and the Aider leaderboard for cost-aware editing.