Use LFM3 to build specialized multimodal models

LFM3 is the newest generation of Liquid Foundation Models, and it is available on Hugging Face today. The release is aimed at developers and teams that need small, locally deployable models for text, vision, audio, omni input, and structured decision tasks. Instead of using one large general-purpose model, users can select from six 2B-class models built from LFM3-1.7B-Base, fine-tune the chosen model on a single GPU, and run it where the application operates.

The main practical change is that LFM3 places multiple modalities on one shared backbone while keeping the models small enough for edge devices. The text model fits in about 1 GB at 4-bit precision, and the text model has a 256K context window, which supports longer-running agents on-device.

What changes from LFM2

LFM3 introduces four changes compared with the previous LFM2 family.

First, the release says LFM3-1.7B outperforms LFM2.5-2.6B while using 35% fewer parameters. This gain comes from a new tri-hybrid architecture and a new training recipe.

Second, every modality now uses the same backbone. Six 2B-class models are trained from LFM3-1.7B-Base: text, vision, audio, omni, and two decision models. The omni model is the first in the family, and the vision model now accepts video input.

Third, the lineup is built for edge deployment. New in-house vision and audio encoders speed up processing, and the small text model can run on-device with a long context window.

Fourth, each model is designed for task-specific adaptation. Every model can be fine-tuned on a single GPU, allowing teams to adapt the model to their own data, terminology, and deployment environment.

Architecture built for agentic workloads

LFM2 used a hybrid of short convolutions and attention to run quickly on-device without sacrificing quality. For LFM3, the release says a hardware-in-the-loop architecture search redesigned that structure for agentic use cases, which need more capability per parameter and faster inference over long contexts.

The search covers 14 device and backend combinations, including phones, a Raspberry Pi 5, laptops, cloud CPUs, GPUs, and the Qualcomm Hexagon neural processing unit.

LFM3-1.7B-Base has 24 layers and four main architecture changes:

• A tri-hybrid layer mix contains 13 gated short convolutions, 5 sliding-window attention layers with a 256-token window, and 6 global attention layers. The convolution kernel size is 7, up from 3 in LFM2.
• Head-wise gated attention uses a sigmoid gate to scale each head's output, allowing a head to turn off when it does not contribute. This may help suppress activation outliers that hurt quantization.
• Partial Rotary Position Embedding rotates only the highest-frequency half of each head in global attention layers, while the other half matches on content regardless of distance. Sliding-window attention layers keep full Rotary Position Embedding.
• Sandwich norm normalizes both the input and output of each block to keep activations well-scaled.

The performance effect appears at long context. Per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices, and with 8K tokens in context it is about 25% faster.

LFM3-1.7B-Base was pre-trained on 22T tokens with the same 128K vocabulary as LFM2.5-8B-A1B. The final 3.5T tokens are a mid-training stage on curated data with a larger share of agentic traces and tool calls. This stage also extends the context window to 256K tokens.

The model lineup and what each one does

Every LFM3 model branches from LFM3-1.7B-Base. The release describes this as a way to choose the model that fits the inputs, customize it, and run it locally rather than relying on one large general-purpose model.

LFM3-1.7B

LFM3-1.7B is intended for local agents, tool calling, structured outputs, or as a base for domain-specific fine-tuning. It is post-trained to run in agentic harnesses such as Hermes Agent, Pi, and OpenClaw.

The model outperforms LFM2.5-2.6B with 35% fewer parameters, and the largest gains are on agentic tasks. On WebArena-Verified, it scores 38.42 against 3.02 for the predecessor, and it also beats Qwen3.5-9B on long tasks in browsers and business apps.

To run LFM3-1.7B, users install the latest version of transformers, then load and run the model from the model card. The release also points to an Agent Harnesses guide for local agent setup.

LFM3-VL-2B

LFM3-VL-2B is the vision model for on-device visual understanding. It is the first LFM vision model that takes video as input. It reasons step by step before answering, finds objects, and reasons about how objects are placed, which makes it a potential foundation for robotics.

Compared with LFM2.5-VL-3B, LFM3-VL-2B uses fewer parameters, 1.9B against 3.1B, yet raises the average across 62 vision benchmarks from 57.6 to 71.4. That average is on par with Gemma 4 12B, a model more than six times larger.

The model uses a new vision encoder that combines convolution and attention. The release says it uses 3x less compute than the SigLIP2 encoder in previous LFM VL models. The 164M-parameter encoder matches Qwen3.5-2B's vision encoder in accuracy while requiring 2.2x fewer floating-point operations, measured with the same LFM2.5-230M language backbone.

For video, the encoder embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length.

To run LFM3-VL-2B, users install transformers>=5.21.0 with its vision dependencies. The model card lists tool calling, tool search, and on-device inference with llama.cpp.

LFM3-Audio-2B

LFM3-Audio-2B is designed for voice chat, speech recognition, speech synthesis, and translation across 18 languages on a laptop or edge device. It takes text or audio as input and responds with text, speech, or both. It can also turn spoken requests into tool calls.

Compared with LFM2.5-Audio-1.5B, it uses a new in-house audio encoder and speech codec. The release says its speech quality is higher, scoring 3.83 on UTMOSv2 against 3.22. It also adds audio understanding, general-purpose audio reasoning, and tool calling.

The audio encoder mixes sliding-window attention and short convolutions and is fully causal, so the model can prefill while the user speaks, with time to first token as low as 50 ms. LiquidCodec, the new audio tokenizer, turns generated tokens into speech at about 87x real time on a single CPU thread on Apple M4 Max and supports streaming decoding.

To run LFM3-Audio-2B, users install transformers>=5.20.2 with its audio dependencies, then load the model and processor from the Hugging Face Hub. The model card covers live speech recognition, speech synthesis, audio understanding, and tool calling.

LFM3-Omni-2B

LFM3-Omni-2B is the first omni model in the family. It accepts text, vision, and audio input and produces text and speech output. The release positions it as a single on-device model that sees, listens, and talks back, replacing a separate speech recognition model, vision-language model, large language model, and text-to-speech model.

It can answer spoken questions about an image, process video with both visual and audio input, and turn what it sees and hears into tool calls. It keeps core capabilities from the standalone LFM3 models, including visual question answering, speech recognition, speech synthesis, and instruction following.

Like LFM3-Audio-2B, it speaks through LiquidCodec, which turns generated tokens into speech faster than real time, allowing spoken replies to stream as the model generates them. On audio-visual video understanding, the release says it beats models up to 10.7B parameters, such as Qwen2.5-Omni-7B, on Daily-Omni, WorldSense, and AV-SpeakerBench.

To run LFM3-Omni-2B, users install transformers>=5.20.2 with its vision and audio dependencies, then can ask a question about a photo by voice. The model card lists speech output, video with its soundtrack, and tool calling.

d1-2B and d1-omni-2B

The d1 models are for fast, structured decisions instead of generated text. Users provide a situation, such as a support ticket, invoice, app screen, photo, or voice note, along with a list of questions. The model answers all questions in a single forward pass, each with a calibrated probability.

Beyond yes or no answers, choices, and scores, the models can point at objects, read values from documents, estimate numbers with a range, and pick every option that applies.

d1-2B is for text, pictures, and video. It replaces d1-3B with a third fewer parameters and scores higher on all four decision suites, for example 75.5 against 64.0 on ImajevBench.

d1-omni-2B is for inputs that include audio. If the question is asked by voice instead of in writing, the release says it keeps nearly all of its accuracy.

To run d1-2B, users install transformers>=5.19 with its vision dependencies. The model cards describe audio clips and spoken questions for d1-omni-2B.

Fine-tuning and the robotics example

The release frames LFM3 as a family of small models that can become task-specific experts through short fine-tuning. Large frontier models are generalists, but they are not necessarily optimized for one team's data, terminology, or tools. LFM3 packs capacity into small, fast models that can be deployed from devices to private infrastructure to the cloud after adaptation.

Every model can be fine-tuned on a single GPU. Notebooks cover complete tasks for the text, vision, audio, and omni models, from data preparation to evaluation.

One example shows LFM3-VL-2B adapted into three complementary robotics models: orchestration, navigation, and manipulation.

The orchestrator operates a robot arm through tool calls. It grounds objects by pointing, measures them with depth tools, invokes motion planners, and checks outcomes in subsequent camera images.

The navigator follows route descriptions using recent red-green-blue observations. It predicts spatial targets and a waypoint trajectory in just five tokens.

The manipulation vision-language-action model maps images, language, and robot state to eight-step chunks of continuous actions in a single forward pass. Jointly trained across 15 embodiments, it achieves 96.2% average success across the four LIBERO suites and 45.7% on RoboCasa-GR1 tabletop tasks. It generates action chunks in under 50 ms on a Jetson AGX Orin with TensorRT FP16.

These results show a compact robotics foundation that can span high-level planning and low-level control while generating actions in real time on embedded hardware.

Availability and next steps

All LFM3 models are open-weight and available on Hugging Face today. The LFM3 collection includes LFM3-1.7B-Base for users who want their own post-training. Fine-tuning notebooks are provided for adapting the models to specific tasks, and a guide covers how to run local agents with OpenClaw, Hermes Agent, and Pi.

The release also asks users to cite the release blog if they use the work.