LoopFill Provides Pipelined Multi-Token Speculative Decoding for Looped Transformers
LoopFill pipelines suffix and shallow-loop drafts into one tree, reducing looped LLM decoding latency with 5.85× speedup on Ouro-2.6B.
LoopFill pipelines suffix and shallow-loop drafts into one tree, reducing looped LLM decoding latency with 5.85× speedup on Ouro-2.6B.
Dory and GLiDE show looped LLMs add depth by reusing layers, but gains cost compute and latency; cache choices decide real serving cost.