LFM2.5-VL-DSpark: A faster approach to vision-language models
Liquid AI released DSpark draft models for its LFM2.5-VL-3B vision-language model, using speculative decoding to cut decoding time by up to 3.13x on Apple silicon and 2.66x on an H100 GPU. The lightweight ~280M-parameter drafter taps the target model's hidden states and was trained with a vision-language SFT mixture. Weights are on Hugging Face with day-one support for llama.cpp, MLX-VLM, and SGLang, and output remains exactly equivalent to the target model.
Liquid AI released DSpark draft models for its LFM2.5-VL-3B vision-language model, cutting decoding time by up to 3.13x on Apple silicon and 2.66x on an H100 GPU. The release includes day-one support for llama.cpp, MLX-VLM, and SGLang, and the weights are available on Hugging Face in Safetensors and GGUF formats.
If you run vision-language models on edge devices or GPUs, this post explains how the new drafter works, what speedups to expect on your hardware, and how to get started.
What speculative decoding adds to a vision model
The drafter follows the same design as Liquid AI's text-only LFM2.5-DSpark drafters. It taps the target model's hidden states at a fixed set of layers and uses them to propose a block of k candidate tokens, which the target model then verifies. Because image patches and text tokens are projected into a shared representation before those layers, the drafter always works with hidden-state vectors of the same dimensionality — so the inference algorithm is identical to the one used for text models.
The drafter is lightweight: roughly 280M parameters, adding just 8.9% on top of the 3B target model.
How it was trained
Liquid AI trained the drafter with its DSpark recipe, using a mixture of vision-language SFT data weighted toward expected workloads. After ablations across 3, 4, and 5 layers, the team settled on a simplified attention-only drafter with 4 layers and a block size of 9. Training ran for 10 epochs, with acceptance rates measured after each epoch — acceptance improved with more training tokens before hitting diminishing returns. At inference time, Liquid AI recommends a block size of 8 or 9 depending on the hardware.
Measured speedups on device and GPU
Both configurations used a DSpark block size of 8 and were evaluated on six vision-based tasks following the MMSpec benchmark: general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation.
On-device (Apple silicon):
- MLX on an M5 Max: decoding runs 2.30x to 3.13x faster by task; end-to-end latency improves 1.56x to 2.62x.
- llama.cpp on an M3 Ultra: decoding improves 1.57x to 2.14x; end-to-end improves 1.30x to 1.77x.
GPU (H100):
- Decoding runs 2.66x faster at best (the source also lists a 20.4x figure for decoding), with end-to-end improvements of 1.64x to 2.27x.
Why decode speedups don't fully translate to end-to-end gains
Speculative decoding only accelerates the decode phase — not the vision encoder or prefill. In vision-language models, an image first passes through a vision encoder, and the language backbone then processes hundreds of visual tokens alongside the text prompt. Edge devices have far less compute than datacenter GPUs, so prefill consumes a larger share of end-to-end latency there (though the M5's per-core GPU neural accelerators narrow this gap).
This is Amdahl's law in practice: when prefill already dominates wall time, even a large decode speedup yields only a modest end-to-end improvement.
How to run it
- SGLang: requires a build with DSpark support for LFM2 targets (PR #40651). Launch the target with the draft attached, then query the OpenAI-compatible endpoint at http://localhost:30000/v1. The block size is read from the draft's config.json.
- llama.cpp: requires the respective llama.cpp build (PR #29339).
- MLX-VLM: requires the respective build (PR #2280). The block size comes from sidecar metadata, with n-max clamped to it.
One important guarantee: speculative decoding here is exact. The target model verifies every proposed token, so greedy output is identical to running the target alone. Per-response timings report draft_n / draft_n_accepted.
Who should use it / who should skip it
Use it if you deploy LFM2.5-VL-3B on Apple silicon or H100 GPUs and your workloads are decode-heavy — multi-turn conversation, captioning, and long generations will benefit most. The open-weight, day-one support across llama.cpp, MLX, and SGLang makes adoption straightforward.
Skip it for now if your workloads are dominated by prefill — short prompts paired with large images on compute-limited devices — since the end-to-end gains will be modest in those cases.
Comments (0)