Build specialized multimodal models with LFM3

Back to Articles Build specialized multimodal models with LFM3 Team Article Published October 27, 2026 Upvote 11 +5 Leonie Monigatti iamleonie Follow LiquidAI Jimmy T.H.

Smith jimmysmith1919 Follow LiquidAI Anthony Maltsev amaltsev Follow LiquidAI Maxime Labonne mlabonne Follow LiquidAI Brandon Brandon3967 Follow LiquidAI Samuel Stevens samuelstevens Follow LiquidAI Ryan Shubert shubeydoo Follow LiquidAI Tianshu Yu tianshu-yu Follow LiquidAI Sina s-jse Follow LiquidAI Tim Seyde tseyde Follow LiquidAI Rouzbeh Amin Tafreshi rouzbehat78 Follow LiquidAI Alexander Amini alexanderamini Follow LiquidAI Saniya Karwa saniyakarwa Follow LiquidAI Marc Harkonen haerski Follow LiquidAI Anna Piunova orech Follow LiquidAI Calder Katyal CalderKat Follow LiquidAI Ishwarya Ananthabhotla ianant93 Follow LiquidAI Farris Atif farris-liquid Follow LiquidAI Tarek Dakhran tdakhran Follow LiquidAI Paul Pak paulpak58 Follow LiquidAI xx tugot17 Follow LiquidAI Mihir Bafna mihirbafna Follow LiquidAI Kyle Bruce Wheeler m3m0ryh0l3 Follow LiquidAI Aditya Tadimeti adityatadimeti Follow LiquidAI Alfred Eng alfredeng Follow LiquidAI Karthik Macherla karthikrmacherla Follow LiquidAI Khoi Nguyen KTLK Follow LiquidAI Aldo Gael Carranza agcarranza Follow LiquidAI Aurelien Lac Aurelien-Lac Follow LiquidAI Tyler Murray undfined Follow LiquidAI Raahul Vignesh Raahul26 Follow LiquidAI Song Duong sduong Follow LiquidAI Ramin Hasani raminmh Follow LiquidAI Mathias Lechner mlech26l Follow LiquidAI Akhilesh Deepak Gotmare akhileshdeepakgotmare Follow LiquidAI Fernando Fernandes Neto fernandofernandes Follow LiquidAI Jiahui Wang WWWjiahui Follow LiquidAI Yuri Khrustalev ykhrustalev Follow LiquidAI Alay Shah alay2shah Follow LiquidAI Simon Lee yungisimon Follow LiquidAI Mayank Agarwal mayankagarwal Follow LiquidAI Zetian Li zetianli Follow LiquidAI changyi yang changyiyang Follow LiquidAI Sinoué GAD GAD-cell Follow LiquidAI LFM3 is our new generation of Liquid Foundation Models (LFMs), built to be the starting point for specialized intelligence.

It brings text, vision, audio, omni, and decision models onto one shared backbone, all in the 2B parameter range.This means you can pick the model for your modality, fine-tune it on your own data on a single GPU, and deploy it where your application runs.

LFM3 brings four main changes compared to our previous LFM2 family: More capability per parameter: a new tri-hybrid architecture and training recipe allow LFM3-1.7B to outperform LFM2.5-2.6B while using 35% fewer parameters.

Every modality on one backbone: six 2B-class models (text, vision, audio, omni, and decision) trained from LFM3-1.7B-Base, including our first omni model and video input for our vision model.

Built for the edge: the text model fits in about 1 GB at 4-bit precision, new in-house vision and audio encoders speed up processing, and the text model’s 256K context window lets agents run on-device.

Made to be specialized: every model fine-tunes on a single GPU, so you can adapt it to your data, terminology, and deployment environment.How we built a more expressive backbone LFM2 showed that a hybrid of short convolutions and attention can run fast on-device without sacrificing quality.

For LFM3, we used our hardware-in-the-loop architecture search to redesign that architecture for agentic use cases, which need more capability per parameter and fast inference over long contexts.

The search now covers 14 device and backend combinations, from phones and a Raspberry Pi 5 to laptops, cloud CPUs, GPUs, and the Qualcomm Hexagon NPU.LFM3-1.

7B-Base has 24 layers and four main changes: Tri-hybrid layer mix: 13 gated short convolutions, 5 sliding-window attention (SWA) layers with a 256-token window, and 6 global attention layers.The convolutions use a kernel size of 7, up from 3 in LFM2.

Head-wise gated attention: a sigmoid gate scales each head's output, so a head can switch off when it has nothing to add.This can help suppress activation outliers that hurt quantization.

Partial RoPE: global attention layers rotate only the highest-frequency half of each head, so the other half matches on content regardless of distance.SWA layers keep full RoPE.Sandwich norm: each block normalizes both its input and its output to keep activations well-scaled.

The payoff shows at long context.Per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices, and with 8K tokens in context it is about 25% faster.LFM3-1.7B-Base was pre-trained on 22T tokens with the same 128K vocabulary as LFM2.5-8B-A1B.The final 3.

5T tokens are a mid-training stage on curated data with a larger share of agentic traces and tool calls, which also extends the context window to 256K tokens.One backbone for six models Every LFM3 model branches from LFM3-1.7B-Base.

Instead of running one large general-purpose model, you pick the one that fits your inputs, customize it, and run it on-device.Model Input Output Built on LFM3-1.7B Text Text LFM3-1.

7B-Base LFM3-VL-2B Text, images, video Text LFM3 backbone + vision encoder LFM3-Audio-2B Text, audio Text, speech LFM3 backbone + audio encoder, audio decoder, and LiquidCodec LFM3-Omni-2B Text, images, video, audio Text, speech LFM3 backbone + vision and audio encoders and audio decoder d1-2B Text, images, video Calibrated answers LFM3-VL-2B d1-omni-2B Text, images, video, audio Calibrated answers LFM3-Omni-2B The vision and audio models also get two new in-house encoders: Vision encoder: it combines convolution and attention and uses 3x less compute than the SigLIP2 encoder in previous LFM VL models.

Our 164M-parameter encoder matches Qwen3.5-2B's vision encoder in accuracy while requiring 2.2x fewer FLOPs (both measured with the same LFM2.5-230M language backbone).For video, it embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length.

Audio encoder and LiquidCodec: the audio encoder mixes sliding-window attention and short convolutions and is fully causal, so the model can prefill while you speak, with time to first token as low as 50 ms.

LiquidCodec, our new audio tokenizer, turns generated tokens into speech at about 87x real time on a single CPU thread on Apple M4 Max and supports streaming decoding.How to use the LFM3 models Each model runs in transformers.

The snippets below come from the model cards, which also list the other inference engines each model supports.LFM3-1.7B Reach for LFM3-1.7B when you need local agents, tool calling, structured outputs, or a base for domain-specific fine-tuning.

It is post-trained to run in agentic harnesses such as Hermes Agent, Pi, and OpenClaw.It outperforms its predecessor, LFM2.5-2.6B, with 35% fewer parameters, and the biggest gains are on agentic tasks: on WebArena-Verified, it scores 38.42 against 3.02, and it even beats Qwen3.

5-9B on long tasks in browsers and business apps.You can find more details in the LFM3-1.7B model card.To run LFM3-1.

7B, install the latest version of transformers: pip install -U transformers Then load and run the model: from transformers import pipeline generator = pipeline( "text-generation", model="LiquidAI/LFM3-1.7B", trustremotecode=True, ) messages = [{"role": "user", "content": "What is C.elegans?

"}] output = generator( messages, dosample=True, temperature=0.1, topk=50, repetitionpenalty=1.1, maxnewtokens=2048, ) print(output[0]["generatedtext"][-1]["content"]) For local agent setup, see the Agent Harnesses guide.LFM3-VL-2B Reach for LFM3-VL-2B when you need on-device visual understanding.

It is the first LFM vision model that takes video as input; it reasons step by step before it answers, and it finds objects and reasons about how they are placed, a strong foundation for robotics.Compared with LFM2.5-VL-3B, it has fewer parameters (1.9B against 3.

1B) yet raises the average across 62 vision benchmarks from 57.6 to 71.4, on par with Gemma 4 12B, a model more than six times larger.You can find more details in the LFM3-VL-2B model card.To run LFM3-VL-2B, install transformers>=5.21.0 with its vision dependencies: pip install "transformers>=5.21.

0" torch torchvision pillow Then load and run the model: import torch from transformers import AutoModelForImageTextToText, AutoProcessor modelid = "LiquidAI/LFM3-VL-2B" model = AutoModelForImageTextToText.frompretrained(modelid, dtype=torch.

bfloat16, devicemap="auto", trustremotecode=True) processor = AutoProcessor.frompretrained(modelid, trustremotecode=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://placecats.com/300/200"}, {"type": "text", "text": "Describe this image.

"}, ], } ] inputs = processor.applychattemplate( messages, addgenerationprompt=True, tokenize=True, returndict=True, returntensors="pt", ).to(model.device) with torch.inferencemode(): outputids = model.generate( inputs, dosample=True, temperature=0.1, topk=50, repetitionpenalty=1.

1, maxnewtokens=16384, # generate() needs an explicit limit; reasoning can be long ) generatedids = outputids[:, inputs["inputids"].shape[1] :] print(processor.

batchdecode(generatedids, skipspecialtokens=True)[0]) See the LFM3-VL-2B model card for tool calling, tool search, and on-device inference with llama.cpp.

LFM3-Audio-2B Reach for LFM3-Audio-2B when you need voice chat, speech recognition, speech synthesis, or translation across 18 languages on a laptop or edge device.It takes text or audio as input and responds with text, speech, or both, and it can turn spoken requests into tool calls.

Compared with LFM2.5-Audio-1.5B, it uses a new in-house audio encoder and speech codec, speaks with higher quality (UTMOSv2 3.83 against 3.22), and adds audio understanding, general-purpose audio reasoning, and tool calling.You can find more details in the LFM3-Audio-2B model card.

To run LFM3-Audio-2B, install transformers>=5.20.2 with its audio dependencies: pip install "transformers>=5.20.

2" torch accelerate librosa soundfile Load the model and processor directly from the Hugging Face Hub: import torch import soundfile as sf from transformers import AutoModelForMultimodalLM, AutoProcessor modelid = "LiquidAI/LFM3-Audio-2B" device = "cuda" if torch.cuda.

isavailable() else "cpu" dtype = torch.bfloat16 if device == "cuda" else torch.float32 model = AutoModelForMultimodalLM.frompretrained( modelid, dtype=dtype, devicemap=device, trustremotecode=True ).eval() processor = AutoProcessor.frompretrained(modelid, trustremotecode=True) codec = processor.

audiotokenizer.to(device=device).eval() Then ask a question aloud and get a reply in text and speech: conversation = [ {"role": "system", "content": "Respond with interleaved text and audio."}, {"role": "user", "content": [{"type": "audio", "audio": "question.

wav"}]}, {"role": "assistant", "content": "<|mixedstart|>"}, ] inputs = processor.applychattemplate( conversation, continuefinalmessage=True, tokenize=True, returndict=True, returntensors="pt", ).to(device) with torch.nograd(): outputs = model.generate( inputs, maxnewtokens=1024, audiotemperature=1.

0, audiotopk=128, audiorepetitionpenalty=1.1, audiorepetitionwindow=4096, returndictingenerate=True, ) text = processor.decode( outputs.sequences[0, inputs["inputids"].shape[1]:], skipspecialtokens=True, ) waveform = codec.decode( outputs.audiocodes[:1, :, :outputs.audiolengths[0]] ).

audiovalues[0] # 24 kHz print(text) sf.write("reply.wav", waveform.float().cpu().numpy(), 24_000) See the LFM3-Audio-2B model card for live speech recognition, speech synthesis, audio understanding, and tool calling.

LFM3-Omni-2B Reach for LFM3-Omni-2B when you need one on-device model that sees, listens, and talks back, in place of a separate ASR model, vision-language model, LLM, and TTS model.

It can answer spoken questions about an image, watch and listen to a video together, and turn what it sees and hears into tool calls.It is our first omni model, with text, vision, and audio input and both text and speech