LoopFill Provides Pipelined Multi-Token Speculative Decoding for Looped Transformers
LoopFill pipelines suffix and shallow-loop drafts into one tree, reducing looped LLM decoding latency with 5.85× speedup on Ouro-2.6B.
LoopFill pipelines suffix and shallow-loop drafts into one tree, reducing looped LLM decoding latency with 5.85× speedup on Ouro-2.6B.