PaperlayerView exact revision on arXiv

Source-linked AI summary

LFM2 Technical Report

Alexander Amini, Anna Banaszak, Harold Benoit, Arthur Böök, Tarek Dakhran, Song Duong, Alfred Eng, Fernando Fernandes, Marc Härkönen, Anne Harrington, Ramin Hasani, Saniya Karwa, Yuri Khrustalev, Maxime Labonne, Mathias Lechner, Valentine Lechner, Simon Lee, Zetian Li, Noel Loo, Jacob Marks, Edoardo Mosca, Samuel J. Paech, Paul Pak, Rom N. Parnichkun, Alex Quach, Ryan Rogers, Daniela Rus, Nayan Saxena, Bettina Schlager, Tim Seyde, Jimmy T. H. Smith, Aditya Tadimeti, Neehal Tumma

arXiv:2511.23404v1cs.LGcs.AI

TL;DR

LFM2 addresses the challenge of deploying capable generative models under strict edge latency and memory budgets. It co-designs a compact hybrid architecture, training pipeline, and model variants for language, multimodal, and retrieval tasks. Across sizes and applications, it reports strong quality-efficiency tradeoffs, including up to ∼2× CPU prefill and decode speedups, while remaining subject to small-model capacity limits and hardware-specific coverage.

  • Problem

    On-device generative applications require strong capabilities under strict latency, memory, energy, and privacy constraints.

  • Method

    LFM2 co-designs a hardware-searched hybrid backbone, small-model training and post-training objectives, and specialized language, multimodal, and retrieval variants.

  • Results

    Across sizes and applications, LFM2 achieves strong quality-efficiency tradeoffs, with up to ∼2× CPU prefill and decode speedups versus similarly sized baselines.

  • Takeaways & Limitations

    The released LFM2 family provides open-weight models and deployment packages spanning compact language models, multimodality, retrieval, and edge-oriented workflows.

  • Takeaways & Limitations

    The deployment recipes and architecture search focus on batch-size-one, low-latency inference on selected CPUs and mobile SoCs, not large-batch server settings or any particular accelerator family.

Abstract

from arXiv · show

We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under edge latency and memory constraints, we obtain a compact hybrid backbone that combines gated short convolutions with a small number of grouped query attention blocks, delivering up to 2x faster prefill and decode on CPUs compared to similarly sized models. The LFM2 family covers 350M-8.3B parameters, including dense models (350M, 700M, 1.2B, 2.6B) and a mixture-of-experts variant (8.3B total, 1.5B active), all with 32K context length. LFM2's training pipeline includes a tempered, decoupled Top-K knowledge distillation objective that avoids support mismatch; curriculum learning with difficulty-ordered data; and a three-stage post-training recipe of supervised fine-tuning, length-normalized preference optimization, and model merging. Pre-trained on 10-12T tokens, LFM2 models achieve strong results across diverse benchmarks; for example, LFM2-2.6B reaches 79.56% on IFEval and 82.41% on GSM8K. We further build multimodal and retrieval variants: LFM2-VL for vision-language tasks, LFM2-Audio for speech, and LFM2-ColBERT for retrieval. LFM2-VL supports tunable accuracy-latency tradeoffs via token-efficient visual processing, while LFM2-Audio separates audio input and output pathways to enable real-time speech-to-speech interaction competitive with models 3x larger. LFM2-ColBERT provides a low-latency encoder for queries and documents, enabling high-performance retrieval across multiple languages. All models are released with open weights and deployment packages for ExecuTorch, llama.cpp, and vLLM, making LFM2 a practical base for edge applications that need fast, memory-efficient inference and strong task capabilities.

1 Introduction

LFM2 is an edge-first family of Liquid Foundation Models that co-designs architecture, training, and deployment for strong quality under device latency and memory constraints. It spans compact language models, multimodal and retrieval variants, and open deployment packages for practical on-device use.

  • Motivation: LFM2 targets on-device applications where latency, memory, energy, and private or offline execution are hard constraints.The stated goal is to lead in quality, speed, and memory efficiency on CPUs and heterogeneous NPUs.
  • Edge-first backbone: A hardware-in-the-loop search produces a hybrid backbone with gated short convolutions and grouped-query attention for small, fast models.Dense models span 350M–2.6B parameters with 32K context, alongside an 8.3B MoE model with 1.5B active parameters.
  • Training: LFM2 combines 10–12T-token pre-training, long-context mid-training, decoupled Top-K distillation, and three-stage post-training for small-model quality.The distillation objective avoids support mismatch, and the post-training pipeline is designed to improve performance and robustness at small scales.
  • Applications: The portfolio extends the backbone to vision-language, audio-language, and multilingual retrieval through LFM2-VL, LFM2-Audio, and LFM2-ColBERT.These variants target tunable visual accuracy-latency trade-offs, low-latency speech applications, and high-performance multilingual retrieval.
  • Results and release: LFM2 dense checkpoints deliver up to ∼2× prefill and decode speedups versus similarly sized baselines while maintaining or improving benchmark accuracy.The family also includes an MoE model attaining 3–4B-class quality with ∼1.5B active parameters, and releases open weights plus deployment guides.

2 Architecture

LFM2 uses hardware-in-the-loop search to select a compact hybrid backbone, pairing gated short convolutions with a small number of GQA blocks for edge deployment. Across smartphone and laptop CPUs, the resulting models generally improve throughput over similarly sized baselines while covering dense and MoE configurations.

  • Architecture Optimization Process: The search optimizes quality, TTFT and decode latency, prefill throughput, and peak memory at 4K and 32K contexts, discarding candidates that violate device budgets.Remaining candidates are ranked by hypervolume improvement on the quality–latency–memory Pareto frontier.
  • Architecture Optimization Process: LFM2’s searched backbone uses mostly gated short-convolution blocks, interleaved with a minority of GQA blocks, plus SwiGLU position-wise MLPs.The search repeatedly selected this minimal hybrid under quality, latency, and memory constraints.
  • LFM2 Model Configurations: LFM2 releases dense models from 350M to 2.6B parameters and an 8.3B MoE model with 1.5B active parameters, while maintaining a 32K context window for dense checkpoints.The MoE model replaces dense MLPs with sparse MoE MLPs in most layers.
  • Inference Performance: 2–3× higher prefill throughput and 1.5–2.0× higher decode throughput are reported for LFM2-350M versus Granite-4.0-350M across 1K and 4K contexts.The comparison is measured on the Samsung Galaxy S25 CPU under the reported inference setup.
  • Inference Performance: 2.3–2.8× higher prefill throughput and 1.7–2.2× higher decode throughput are reported for LFM2-1.2B versus Granite-4.0-1B and Qwen3-1.7B across both context lengths.Across smartphone and laptop CPUs, the authors summarize substantial latency and throughput improvements over similarly sized open baselines.

3 Pre-Training

LFM2 pre-training combines large-scale text and code training with long-context mid-training and Top-K knowledge distillation. Its decoupled objective separates matching Top-K probability mass from matching relative probabilities within that set, avoiding support mismatch from naive tempering.

  • Pre-training Data: The training mixture contains roughly 75% English text, 20% multilingual text, and 5% code for dense models, with the MoE model using 60% English, 25% multilingual, and 15% code.The multilingual mixture prioritizes Japanese, Arabic, Korean, Spanish, French, and German, with additional support for Chinese, Italian, and Portuguese.
  • Training Stages: LFM2 dense checkpoints use 10T-token pre-training followed by 1T-token long-context mid-training at a 32,768-token context window.The MoE checkpoint uses 12T tokens in its initial phase before the same 1T-token mid-training stage.
  • Decoupled Top-K Knowledge Distillation: The decoupled Top-K objective uses the teacher’s Top-K=32 logits and decomposes distillation into a binary term for Top-K mass and a conditional term for relative probabilities within Top-K.The teacher is an internal LFM1-7B model, and the decomposition addresses support mismatch from naive KL divergence on truncated distributions.
  • Decoupled Top-K Knowledge Distillation: Temperature is applied only to conditional Top-K distributions, while the Bernoulli membership term and teacher Top-K mass remain untempered to prevent support mismatch.At τ = 1, the objective is a lower bound to the full forward KL because the unavailable teacher tail is omitted.

4 Post-Training

LFM2 uses a three-stage post-training pipeline to teach conversational behavior and improve downstream capabilities, supported by curated mixtures and direct alignment. The resulting models show competitive instruction-following, reasoning, multilingual, and throughput results.

  • Pipeline: The pipeline combines supervised fine-tuning, preference alignment, and model merging to improve capabilities and robustness.
  • SFT Data Mixture: The SFT mixture integrates 67 or 79 curated sources spanning mathematical reasoning, general-purpose data, instruction following, code, RAG, and tool use.
  • SFT Data Mixture: Multilingual samples are integrated across task categories, with 80% English and 20% distributed across seven other languages.
  • Evaluation: 79.56% on IFEval for LFM2-2.6B surpasses Llama-3.2-3B at 71.43% and SmolLM3-3B at 72.44%.
  • Evaluation: 82.41% on GSM8K is reached by LFM2-2.6B, while LFM2-8B-A1B adds +10.6 points on MATH 500 over the 2.6B model.
  • Evaluation: LFM2 models occupy the throughput-versus-average-evaluation Pareto frontier, pairing strong benchmark performance with superior inference speed.

5 Vision-Language LFM2

LFM2-VL extends the LFM2 language backbone with a SigLIP2 vision encoder and lightweight connector for efficient multimodal processing. Its variable-resolution and token-budget controls support accuracy-latency tradeoffs while maintaining strong benchmark performance across vision, OCR, multilingual, and language-only tasks.

  • Architecture: The architecture attaches a SigLIP2 image encoder through a PixelUnshuffle-plus-MLP connector that maps visual features into the LFM2 language token space.
  • Architecture: LFM2-VL supports single-frame inputs up to 512 × 512 pixels and dynamic tiling for larger images, with 2–10 tiles and optional thumbnails.
  • Evaluation: LFM2-VL-3B scores 76.55 on SEEDBench and 71.37 on RealWorldQA, outperforming all listed baselines including Qwen2.5-VL-3B.
  • Evaluation: LFM2-VL-1.6B reaches 46.35 on MM-IFEval versus 36.17 for InternVL3.5-1B, while LFM2-VL-3B reaches 51.83.
  • Evaluation: LFM2-VL-3B scores 75.84 on translated MMBench and 81.52 on translated MMMB, outperforming other open-source models under 4B parameters.
  • Accuracy-vs-latency: Reducing the vision-token budget causes minimal degradation on general vision benchmarks but larger drops on high-resolution perception tasks.

6 LFM2-Audio

LFM2-Audio extends the LFM2 language backbone with separate continuous audio input and discrete audio output pathways for speech interaction. It supports interleaved and sequential generation modes and performs competitively on conversational speech and ASR benchmarks.

  • Architecture: Separate audio input and output pathways extend the LFM2-1.2B backbone for speech interaction.Audio input uses an encoder and connector, while output uses discrete codes and a detokenizer.
  • Audio generation: Eight Mimi RVQ codebooks represent each audio frame, which is embedded for backbone feedback and streamed to the detokenizer.Each frame corresponds to 0.08 seconds of audio, and the eight code tokens are summed into one 2048-dimensional embedding.
  • Audio generation: A smaller RQ-Transformer generates the eight audio codes conditioned on one backbone output embedding, reducing the backbone’s sequential audio-generation burden.The RQ-Transformer runs for eight steps per frame, one per codebook.
  • Inference modes: The model offers interleaved generation for low-latency reasoning and speech, and sequential generation for explicit text-to-audio state transitions.Both modes share the backbone but differ in how text and audio outputs are scheduled.
  • Evaluation: LFM2-Audio-1.5B performs competitively against Qwen2.5-Omni-3B on conversational speech and approaches Whisper’s ASR error rates.The ASR evaluation covers eight datasets.

7 LFM2-ColBERT

LFM2-ColBERT-350M is a multilingual late-interaction retriever that combines efficient pre-computed document representations with fine-grained token matching. It delivers strong multilingual and cross-lingual retrieval while retaining competitive encoding throughput.

  • Retrieval design: The model uses late interaction: queries and documents are encoded independently, then matched through token-level MaxSim scoring.This permits document pre-computation and indexing while preserving fine-grained semantic matching.
  • Training: Knowledge distillation transfers cross-encoder relevance scores to the student by minimizing normalized score differences.The PyLate framework applies min-max normalization to stabilize gradients across teacher scales.
  • Multilingual evaluation: LFM2-ColBERT-350M achieves a mean monolingual NDCG@10 of 0.661 across languages and tasks.Reported language scores include 0.490 for Arabic, 0.557 for Japanese, and 0.52 for Korean.
  • Multilingual evaluation: LFM2-ColBERT-350M maintains more consistent multilingual retrieval than GTE-ModernColBERT-v1, especially in Arabic, Japanese, and Korean.LFM2-ColBERT-350M records 0.661 English and 0.490 Arabic NDCG@10, whereas the baseline records 0.680 and 0.309.
  • Cross-lingual evaluation: English-to-French retrieval reaches 0.551 NDCG@10 and Spanish-to-Portuguese retrieval reaches 0.547, corresponding to 97.7% and 100% of monolingual performance.These results show strong cross-lingual transfer for the reported European language pairs.
  • Efficiency: At batch size 64, query encoding reaches 1,420 queries per second versus 1,370 for GTE-ModernColBERT-v1, while document throughput is approximately equal at batch size 32.The reported document throughput is approximately 1,100 documents per second for both models.

8 Related Work

LFM2 adapts efficient sequence-modeling ideas to an edge-first objective, using a minimal hybrid of gated short convolutions and limited global attention. Its training contribution is a decoupled Top-K distillation objective designed for settings where tail logits are unavailable.

  • Edge-first modeling: LFM2 differs from broad foundation-model families by adapting decoder-only language modeling to explicit edge-side latency and memory constraints.The related-work framing contrasts this edge-first objective with families optimized primarily for datacenter serving.
  • Liquid-model inspiration: The LFM2 design draws on liquid networks’ input-conditioned temporal processing to adapt effective local and global context handling.The cited connection is inspiration for adaptive processing rather than a claim that LFM2 implements liquid time constants.
  • Hybrid architectures: Hybridization addresses the long-range ability degradation reported for pure sub-quadratic architectures such as SSMs, linear attention, and convolutions.Prior work motivates combining efficient local or sub-quadratic layers with global attention layers.
  • Hybrid architectures: LFM2 uses a minimal hybrid architecture in which gated short convolutions handle local mixing and a minority of GQA blocks provide global context.The design is selected through hardware-in-the-loop search for device latency and memory budgets.
  • Small-model training: The decoupled Top-K distillation objective separates binary membership and conditional Top-K KL terms, placing temperature only inside the conditional term.This avoids support mismatch when distilling from larger models with unavailable tail logits.

9 Conclusion

LFM2 co-designs architecture, training, and deployment for capable models under edge latency and memory constraints, extending the backbone to multimodal and retrieval applications.

  • Key Contributions: Up to 2× faster CPU prefill and decode than similarly sized attention-heavy baselines while matching or improving benchmark accuracy.The comparison uses identical quantization and runtimes.
  • Key Contributions: LFM2 combines gated short convolutions with a small number of GQA blocks, selected through hardware-in-the-loop architecture search.The family includes dense 350M–2.6B models and an 8.3B MoE model with 1.5B active parameters.
  • Key Contributions: A tempered, decoupled Top-K distillation objective reduces distillation storage and bandwidth while avoiding support mismatch and stabilizing losses.The training pipeline also uses a three-stage post-training recipe for small-model edge workflows.
  • Key Contributions: LFM2-VL, LFM2-Audio, and LFM2-ColBERT extend the backbone to vision-language, speech, and late-interaction retrieval tasks.These variants target flexible visual latency trade-offs, real-time speech interaction, and multilingual retrieval.
  • Limitations and Future Work: Deployment recipes were tuned for batch size 1 on a small set of CPU and mobile SoC configurations, not large-batch server settings or particular accelerator families.The authors leave broader hardware and quantization specialization as future work.
  • Limitations and Future Work: LFM2 is strongest for short- to medium-context, task-oriented edge workloads and remains capacity-limited relative to frontier-scale systems.The authors identify open-ended reasoning, knowledge-intensive, and highly compositional tasks as areas where small-scale models cannot match larger systems.
  • Limitations and Future Work: LFM2-VL lacks explicit grounding signals and native video support, limiting expected performance on fine-grained localization and temporal reasoning.The model supports images up to 512×512 directly and uses tiling plus token reduction for higher resolutions.
  • Limitations and Future Work: LFM2-Audio does not systematically evaluate non-speech audio or overlapping speakers, and its training is dominated by English and other high-resource languages.The authors identify low-resource languages and accents as likely robustness gaps.

10 Authors

The report lists its contributors alphabetically by last name.

  • The contributor list is presented alphabetically by last name.

A.1 Forward KL

The forward-KL derivation decomposes distillation into Top-K membership and within-Top-K shape terms, then tempers only the conditional distributions to avoid support mismatch.

  • Forward KL: The forward KL is split into a Top-K membership term and a conditional divergence over the Top-K tokens.The decomposition separates total probability mass assigned to the Top-K from relative probabilities within it.
  • Forward KL: The decoupled Top-K objective drops the unavailable teacher-tail term, yielding a lower bound to the full forward KL.The omitted term is non-negative and requires teacher logits outside the Top-K.
  • Forward KL: Temperature is applied only to the conditional Top-K distributions, while the Bernoulli membership term and Top-K mass remain untempered.This placement avoids tempering a truncated distribution over the full vocabulary.
  • Forward KL: Naively truncating the teacher and then applying temperature over the full vocabulary causes support mismatch and makes the tempered KL grow as τ^2.Conditioning both distributions on the Top-K makes them converge to the same uniform support as temperature increases.

A.2 Reverse KL variant

The reverse-KL variant decomposes the objective into Top-K and non-Top-K terms, then optimizes a computable lower bound when teacher logits outside Top-K are unavailable. A teacher-weighted surrogate avoids incentivizing the student to shrink the teacher probability mass on Top-K tokens, while the section also describes model-merging methods used elsewhere in the pipeline.

  • Reverse KL variant: Dropping the unavailable non-Top-K term yields a computable lower bound for reverse-KL distillation.The omitted term is non-negative, and temperature is applied only to the conditional Top-K term.
  • Reverse KL variant: The teacher-weighted surrogate removes the incentive to down-weight the student’s Top-K probability mass, but is no longer a strict lower bound.The outer binary term remains untempered while temperature applies only to the Top-K conditional term.
  • Model Merging Techniques: TIES-Merging sparsifies task vectors, elects a majority sign per parameter, and averages only parameters agreeing with that sign.The consensus mask and denominator normalize the merge over weights that agree with the majority sign.
  • Model Merging Techniques: DARE randomly drops task-vector parameters and rescales retained values by 1/(1 − p) to preserve expected update magnitude.DARE can be combined with standard task arithmetic or TIES-style consensus merging.
  • Model Merging Techniques: DELLA replaces uniform dropout with magnitude-aware sampling, assigning higher dropout probabilities to lower-magnitude parameters.Its rescaling preserves each parameter’s expected contribution, followed by TIES-style sign election and disjoint merging.

C Evaluation Details

The evaluation suite combines standardized benchmarks with custom parsing and scoring procedures tailored to each task. It covers language understanding, instruction following, mathematical reasoning, multilingual ability, and robustness under varied prompting and answer-order conditions.

  • Evaluation Harness: The internal evaluation harness extracts final answers with custom parsers designed to tolerate formatting variation beyond restrictive open-source evaluators.For text-only models, the harness reliably extracts answers from model outputs and applies task-specific scoring procedures.
  • Knowledge and Reasoning Benchmarks: MMLU uses 5-shot chat-formatted prompts and constrained A/B/C/D scoring, with fallback prediction when the top logit is not a valid choice.The implementation strips whitespace and uses the highest-probability valid choice when necessary.
  • Instruction Following: Instruction-following evaluation uses official IFEval, IFBench, and Multi-IF implementations with benchmark-specific strict or loose accuracy measures.Multi-IF averages accuracy across three turns, while IFEval and IFBench report prompt- and instruction-level variants.
  • Mathematical Reasoning: Mathematical benchmarks use task-specific few-shot settings, greedy generation limits, answer extraction rules, and strict or loose matching.GSM8K-derived evaluations use custom parsing, while GSM8K reports both strict and loose accuracy and uses loose scoring because strict scoring performs poorly for Gemma 3.
  • Multilingual Evaluation: Multilingual evaluations report averages across predefined language subsets, including seven languages for MMMLU and six for MGSM.MMMLU is evaluated zero-shot, whereas MGSM uses five-shot examples per language and reports average loose accuracy.

D Multilingual Vision Evaluations

The multilingual vision evaluation measures 2–4B vision-language models on translated MMBench and MMMB using VLMEvalKit. LFM2-VL-3B achieves the strongest reported results among open-source models under 4B parameters.

  • Multilingual Vision Evaluations: The evaluation covers 2–4B vision-language models on multilingual MMBench and MMMB using VLMEvalKit.The multilingual benchmarks were translated from English into Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, and Spanish.
Loading 2511.23404v1…