Recovery-Bench: Evaluating LLMs' Ability to Recover from Mistakes Recent

Published: August 27, 2025 | Original Post

---

A benchmark measuring how well agents recover from errors and corrupted states.

Key Findings

• Performance gap -- Models show a 57% relative accuracy decrease on Recovery-Bench vs standard Terminal-Bench (11.2% vs 26.3%)
• Ranking reversal -- Best performers differ significantly between fresh and recovery scenarios
• Context pollution -- Leftover artifacts from failed attempts degrade performance

Surprising Results

Model	Standard Score	Recovery Score
Claude 4 Sonnet	34.8% (#1 standard)	#3 on recovery
GPT-5	20.2%	#1 on recovery

Providing full action histories yielded the worst performance -- detailed error information can actually hinder recovery rather than help it.

Evaluation Scenarios

1. Environment alone (no context about prior failures)
2. Environment + Action Summary (brief overview of previous attempts)
3. Environment + Full Action History (complete error record)