ReaDy uses recurrent states as dynamic context for speculative drafting in hybrid linear attention large language models.
ReaDy gives draft positions recurrent-state context for speculative decoding, reporting 3.16x speedup with Qwen3.8-27B and higher token acceptance.
Key Points
- ReaDy is a speculative decoding method that gives each draft position its own context lookup instead of relying only on the target model’s hidden features.
- It retrieves supplementary context from compact recurrent states in hybrid linear attention models, treating these states as an efficient source of position-specific history.
- The design uses learned queries, low-rank state reads, and fused operators to keep additional computation and input/output overhead small.
- With Qwen3.8-27B on a single NVIDIA B200 GPU, ReaDy reports an average 3.16× decoding speedup over autoregressive decoding across seven benchmarks.
- Compared with DSpark, ReaDy improves average throughput by 7.97% and acceptance length by 8.50%, suggesting more drafted tokens are accepted before correction.
Meta description: ReaDy uses hybrid linear attention recurrent states to retrieve position-specific context, achieving 3.16× average decoding speedup with Qwen3.8-27B.
Tags: ReaDy, speculative decoding, hybrid linear attention, GPU acceleration, language models
Featured image: abstract friendly illustration of recurrent states feeding position-specific queries into draft tokens, with no real people, logos, or text.
What ReaDy Does
ReaDy is a speculative decoding method aimed at systems that accelerate autoregressive generation by drafting multiple tokens in a single step. Its central change is to give each draft position its own context lookup instead of relying only on the target model's hidden features. The method is presented as the first speculative decoding approach, to the best of the authors' knowledge, to use recurrent states from hybrid linear attention models for position-specific context retrieval during drafting.
Speculative decoding speeds up autoregressive generation by drafting several tokens before the target model checks them. Feature-based drafters often rely on hidden features from the target model, but the source notes a possible mismatch: the emphasis carried by those hidden features may not match what each later draft position needs. Hybrid linear attention models store compact recurrent states that accumulate historical associations, which ReaDy treats as an efficient source of complementary context.
How It Works
ReaDy equips each draft position with a learned query that retrieves supplementary context directly from the target model's committed recurrent states. A lightweight state reader then integrates the retrieved context into draft representations through residual updates. To keep the extra work small, the design uses low-rank state reads and fused operators, limiting additional computation and input/output overhead.
The method builds on DSpark's parallel backbone and sequential prediction head. ReaDy uses a lightweight two-stage adaptation strategy. Its development also examined reader capacity, state sources, and residual placement through ablation studies, showing that these design choices are treated as tunable parts of the system rather than fixed assumptions.
Reported Results
Across seven benchmarks spanning mathematics, code generation, and dialogue, ReaDy achieves an average 3.16× decoding speedup over autoregressive decoding when paired with Qwen3.8-27B on a single NVIDIA B200 GPU. Compared with DSpark, ReaDy outperforms it on every benchmark in the reported set. On average, it improves throughput by 7.97% and acceptance length by 8.50%. Acceptance length measures how many drafted tokens are accepted in a speculative step, so the reported gain suggests the drafters are keeping more of their proposed tokens before the target model must correct them.
Why It Matters
For readers who care about generation speed, ReaDy points to a practical direction: the context needed by each draft position can be retrieved from a separate, compact history state instead of being squeezed from the target model's hidden features alone. The result is not a new model or a new benchmark platform, but a drafting mechanism that reports faster output while using an existing target model.
The reported setup is narrow enough to guide expectations: the headline speedup is measured with Qwen3.8-27B on one NVIDIA B200 GPU, and the comparison baseline is DSpark. The next point of interest is whether the same position-specific state retrieval helps across more target models, drafters, and serving environments without adding enough overhead to erase the gain.

Comments (0)