Introducing SWE-bench Verified
OpenAI released SWE-bench Verified, a human-verified subset of the original SWE-bench benchmark. Each task has been manually checked for correctness and clarity, removing flawed or ambiguous items. Th…
LLM Evaluation
Presenting the SWE-Lancer benchmark
The Remote Labor Index benchmark evaluates whether frontier AI models like GPT-5 and Claude Sonnet 4.5 could earn $1 million completing real freelance coding jobs from Upwork-style platforms. Results …
Battlefield Arena: Evaluating Strategic LLMs in Competitive Settings
Kaggle Game Arena is a dynamic benchmark that evaluates large language models through competitive games rather than fixed Q&A tests. By placing models in head-to-head environments such as Chess, Poker…