This article introduces reward misspecification in reinforcement learning: reward functions are only proxies for our true goals, and powerful optimizers exploit gaps between what we write and what we …
This article reviews OpenAI's early experiment fine-tuning a 774M-parameter GPT-2 model using human feedback. Labelers taught the model to match their preferences, but narrow instructions led to unint…
Researchers trained AI critique models to identify flaws in AI-generated summaries, such as missing details and unsupported claims. Human evaluators shown these critiques caught far more errors than w…
This article takes a closer look at sycophancy issues that were previously overlooked. It reviews the results of our analysis, candidly examines the missteps made along the way, and outlines the speci…
This article explains how OpenAI's investment in monitoring, alignment, and security shapes the release pace of its frontier AI models. It breaks down the three safety pillars, shows how pre-deploymen…
Recent research demonstrates that language models can be effectively guided toward specific behavioral values by fine-tuning on a small, carefully curated dataset. This approach highlights the power o…
OpenAI introduces InstructGPT, a family of language models now the default on its API. Built on GPT-3 but trained with humans in the loop, it follows user instructions more accurately, gives more trut…
OpenAI has introduced GPT-6 Astra, described as its most intelligent and best-aligned model to date. The flagship model leads in four key areas: computer use, coding, cybersecurity, and science. Beyon…
This article investigates how exposing language models to incorrect answers during training can cause misalignment to generalize into broader alignment failures. The researchers identify a specific in…
OpenAI has published new insights on AI safety for long-horizon models, explaining how extended tasks reveal failure modes invisible in short tests. The company advocates iterative deployment—gradual …
