SparseEngine: An Inference Engine Built for Sparsity
SparseEngine makes sparse attention core for LLM agents, covering 15 methods, cutting KV cache costs, speeding long-context inference and decoding.
SparseEngine makes sparse attention core for LLM agents, covering 15 methods, cutting KV cache costs, speeding long-context inference and decoding.