SWE-bench — SWE-bench is a benchmark that tests whether AI can fix real problems in real open-source software projects. Each task comes from an actual reported issue and is checked by the project’s tests. Scores have risen quickly as coding agents improved. It is one of the most watched measures of AI coding ability.

Added October 21, 2026