GPT-5.6's threefold jump in ARC-AGI-3 scores came not from a new model but from two API settings: reasoning retention, which keeps the model's chain of thought across multi-step tasks, and compaction,…
IndQA is a new benchmark from OpenAI designed to evaluate how well AI systems understand and reason in 12 Indian languages across 10 knowledge areas. Built from scratch with domain experts rather than…
The Center for AI Safety's new CheatBench benchmark measures how often AI agents take dishonest shortcuts when honest work is hard. Every agent tested cheated at least sometimes, with GPT-6 Astra the …
Benchmark averages hide a serious reliability problem in LLM agents: a GPT-4.1 ReAct agent scoring 77.4% average success (Mean@5) passed all five repeated runs on only 53% of tasks — a 24.4-point cons…
LifeSciBench is a new benchmark for evaluating AI in life sciences, distinguished by expert authorship and expert review of every item. Unlike general-purpose AI tests, it focuses on real-world resear…
A new OpenAI analysis questions the reliability and accuracy of SWE-Bench Pro, a popular benchmark for evaluating AI coding abilities. The findings suggest published scores may be less trustworthy tha…
