Embedding-Space Directional Exploration with Anchored Noise for LLM Reinforcement Learning with Verifiable Rewards Goes Beyond Policy Rollouts
EDEN is an LLM math post-training method that adds anchored Gaussian noise to prompt embeddings during RLVR rollouts. This creates directional variation instead of repeated near-identical samples, giving group-relative learning more reward contrast. It filters and reweights noisy rollouts before training. On Qwen2.5-Math-1.5B and 7B, it improves average Pass@256 over GRPO, with a reported gain on AIME'25. The benefit is tied to the tested models and setup.
Key Takeaways
- EDEN improves LLM math post-training by injecting anchored Gaussian noise into embedding-space inputs during rollouts, enabling directional exploration beyond standard policy sampling.
- It targets a key RLVR bottleneck: repeated rollouts for the same prompt often collapse into similar trajectories, weakening group-relative learning signals.
- EDEN does not simply add noise; it uses a shared anchor embedding plus advantage-gated filtering and noise-aware reweighting to turn perturbed rollouts into useful training signal.
- On Qwen2.5-Math models, EDEN outperforms GRPO across English and Chinese math benchmarks, with average Pass@256 gains of 6.67 points and up to 13.33 points on AIME’25.
- The results suggest rollout diversity can be shaped at the embedding input level, though the evidence is currently tied to Qwen2.5-Math and GRPO-style setups rather than all models or RL settings.
Meta description: EDEN adds anchored noise to LLM math post-training rollouts, improving exploration and Qwen2.5-Math results over GRPO on Pass@256 in math reasoning.
Tags: EDEN, LLM post-training, math reasoning, RLVR, GRPO
Featured image: abstract friendly illustration of anchored Gaussian noise guiding LLM math exploration in embedding space, no real people, no logos, no text.
Why repeated rollouts can stall math post-training
For teams building or researching large language models for math, EDEN is a post-training method that injects anchored noise into rollout inputs in embedding space. That lets the model probe solutions its ordinary sampling would miss. It is aimed at RLVR setups where one prompt is sampled many times so outputs can be compared within a group.
The bottleneck is not a missing answer in principle. In RLVR, repeated rollouts for the same prompt often collapse into very similar trajectories. If the group does not vary, reward differences become too small for group-relative learning, even though the model has already sampled several outputs.
Anchored noise changes the input, not the sampling surface
EDEN addresses that problem before the model generates the answer. For a prompt, it first collects standard rollouts. It then creates another rollout batch from inputs whose embeddings have been shifted. Gaussian noise supplies the shift. A shared anchor embedding gives the whole batch a common conditioning direction.
The anchor is the key difference from arbitrary noise. The perturbed rollouts still vary, but they do not scatter without a reference point. The source describes the exploration as directional rather than random.
Filtering and reweighting turn perturbed rollouts into training signal
Variation alone is not enough for a useful update. EDEN selects the perturbed rollouts that carry the most useful signal and brings them into training through an advantage-gated filter. It also uses noise-aware advantage reweighting so the update accounts for the fact that some rollouts came from noisy inputs.
This makes the method a training filter as much as a sampling trick. The model explores wider, but the training update is still governed by which noisy samples are informative.
Reported gains on Qwen2.5-Math models
When tested on Qwen2.5-Math-1.5B and 7B, the source reports that EDEN beats GRPO on average across English and Chinese math benchmarks, from grade-school to olympiad level. The average Pass@256 rises by 6.67 points. At the 7B scale on AIME'25, the reported gain is 13.33 points.
The analysis also says EDEN gives more prompt groups a non-zero learning signal. It describes the noise and the anchor as complementary: noise creates variation, while the anchor keeps that variation aligned with a shared direction.
How to read this result
For readers running math post-training experiments, the practical implication is that rollout diversity can be shaped at the embedding input level. If many prompt groups produce nearly identical trajectories, the learning signal may be weak even when the model samples often. EDEN suggests one way to create useful variation at the input stage.
The evidence is tied to the Qwen2.5-Math models and the GRPO comparison described in the source. It points to a specific post-training mechanism and a set of reported math benchmark gains, not a universal fix for every model size, benchmark, or reinforcement learning setup.

Comments (0)