Source-linked AI summary
Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance
Kangsheng Duan, Ziyang Xu, Wenyu Liu, Xiaohu Ruan, Xiaoxin Chen, Xinggang Wang
TL;DR
Practical deployment of 10B-level inpainting models is limited by their computational and memory costs. Moebius combines compact LλMI blocks with adaptive latent distillation, matching or surpassing FLUX.1-Fill-Dev quality with 0.22B parameters and over 15× faster total inference.
Problem
10B-level inpainting models deliver strong quality but impose computational and memory costs that hinder deployment, motivating lightweight specialists that can rival their performance.
Method
Moebius combines LλMI blocks for compact spatial and semantic interaction modeling with adaptive multi-granularity latent distillation.
Results
0.22B parameters—less than 2% of FLUX.1-Fill-Dev’s 11.9B—and over 15× faster total inference accompany comparable or superior generation quality across natural and portrait benchmarks.
Takeaways & Limitations
Moebius demonstrates that a highly compact specialist can bridge the scale gap with 10B-level industrial generalists for high-fidelity image inpainting.
Takeaways & Limitations
Moebius can produce less plausible fine-grained geometry in tiny background regions with severely limited contextual textures than its 1B-parameter teacher.
Abstract
from arXiv · showhide
While 10B-level industrial foundation models have pushed the boundaries of image inpainting, their prohibitive computational costs severely hinder practical deployment. Constructing a highly optimized task-specific specialist offers a promising solution; however, extreme structural compression inevitably triggers a severe representation bottleneck. To conquer this, we propose Moebius, a highly efficient lightweight inpainting framework. We systematically reconstruct the diffusion backbone by introducing the Local-$λ$ Mix Interaction ($LλMI$) block. Comprising Local-$λ$ and Interactive-$λ$ modules, it elegantly summarizes spatial contexts and global semantic priors into fixed-size linear matrices, preserving complex latent interactions while drastically shedding parameters. Furthermore, to unlock the full representational capacity of this highly compact architecture, we synergistically pair it with an adaptive multi-granularity distillation strategy. Operating strictly within the latent space to avoid expensive pixel-space decoding, this strategy dynamically balances multiple gradient-based losses to achieve high-fidelity alignment. Extensive experiments across natural and portrait benchmarks demonstrate that this optimal synergy enables Moebius to rival or even surpass the generation quality of the 10B-level industrial generalist FLUX.1-Fill-Dev. Remarkably, Moebius achieves this using less than 2\% of the parameters (0.22B vs. 11.9B) while delivering a $>15\times$ acceleration in total inference time, setting a new efficiency standard for high-fidelity inpainting. Project page at https://hustvl.github.io/Moebius.
1 Introduction
Moebius addresses the deployment barriers of 10B-level inpainting models with a compact architecture and adaptive latent-space distillation. Experiments show a 0.22B-parameter specialist matching 10B-level generalists while substantially reducing inference cost.
- Motivation: 10B-level foundation models deliver strong zero-shot inpainting quality but impose computational and memory costs that hinder deployment on constrained or latency-sensitive devices.The introduction identifies this tension as motivation for a highly optimized, lightweight task-specific specialist.
- Motivation: Nearly one billion parameters leave PixelHacker prohibitive for edge deployment despite its efficient high-fidelity inpainting innovations.PixelHacker combines Latent Categories Guidance with Gated Linear Attention to reduce computational overhead.
- Method: Moebius reconstructs the diffusion backbone with Local-λ and Interactive-λ modules that summarize local spatial contexts and global semantic priors into compact representations.These modules are introduced to address architectural constraints in existing linear attention and alleviate the representation bottleneck.
- Method: Adaptive multi-granularity distillation dynamically balances gradient-based losses to align the lightweight specialist with a high-capacity teacher in latent space.The strategy avoids pixel-space decoding and is designed to compensate for the compact architecture’s representational capacity gap.
- Results: 0.22B parameters enable Moebius to match 10B-level generalist performance, while 26.01 ms/step and 0.154 TFLOPs support a > 15× total-inference speed-up over FLUX.1-Fill-Dev.Evaluations cover natural, portrait, and real-world object-removal scenarios and report a superior performance-parameter-latency trade-off.
2 Related Work
Related work pursues efficient vision architectures by reducing computation and parameters while preserving representation, and uses knowledge distillation to transfer capabilities from large teachers to lightweight students. Recent methods additionally target diffusion denoising dynamics with fewer sampling steps.
- Efficient Architectures: Compact vision architectures aim to reduce computational complexity and parameter count while preserving representational capacity.This goal motivates classical lightweight designs in computer vision.
- Efficient Architectures: DWConv and group convolutions efficiently extract local features by decoupling spatial and channel interactions.Low-rank FFNs and linear attention likewise improve transformer efficiency while maintaining representational ability.
- Knowledge Distillation: Knowledge distillation transfers knowledge from large, high-capacity teacher models to lightweight students through soft labels, feature maps, and perceptual metrics.These are the three supervision objectives described for classical KD techniques.
- Knowledge Distillation: Diffusion-specific distillation enables students to approximate teacher denoising dynamics with fewer sampling steps while preserving generation quality.These methods extend knowledge distillation to diffusion models.
3 Method
Moebius restructures a latent diffusion inpainting backbone with Local-λ Mix Interaction blocks, which address compression-induced representation bottlenecks through efficient local and semantic interactions. An adaptive multi-granularity distillation strategy transfers capabilities from a high-capacity teacher while dynamically balancing heterogeneous objectives for stable student optimization.
- 3.1 Baseline Latent Diffusion Architecture: Moebius adopts latent diffusion inpainting, encoding masked and clean images with a pretrained VAE and combining masked latents with downsampled mask information.The masked image produces zm = E(xm), while clean and masked latent representations construct the forward diffusion process.
- 3.2 LλMI Block: Naive lightweight substitutions cause catastrophic quality degradation, with FID deteriorating from 32.75 to over 43.58 because constrained operators create a representation bottleneck.The bottleneck motivates architectural synergy rather than directly inserting lightweight operators into the backbone.
- 3.2 LλMI Block: The LλMI block combines Local-λ and Interactive-λ modules, summarizing spatial contexts and semantic priors into fixed-size linear matrices for linear-complexity interactions.This design bypasses memory-intensive dot-product attention and integrates external semantic priors at lower computational cost.
- 3.2 LλMI Block: 0.22B parameters and 0.154T FLOPs place Moebius in the extreme lightweight regime while maintaining competitive generation quality under the fully equipped optimization scheme.The framework further uses Depthwise Residual Blocks and Mix-FFN compression to reduce the parameter budget.
- 3.3 Multi-Granularity Distillation: Multi-granularity distillation uses pretrained PixelHacker as teacher and Moebius as student to transfer semantic and generative capabilities across heterogeneous objectives.Adaptive gradient-based weighting addresses differing coarse- and fine-granularity loss magnitudes, stabilizing convergence and reducing manual hyperparameter tuning.
4 Experiments
Moebius is evaluated across natural and portrait inpainting benchmarks against academic specialists and industrial generalists, combining competitive quality with extreme architectural efficiency. Ablations and qualitative studies attribute its performance to holistic architectural integration and latent-space multi-granularity distillation.
- Efficiency: 0.226B parameters and 26.01 ms/step establish Moebius as an efficient alternative to diffusion competitors, using under 2–3% of 10B-level models’ parameter budgets.Compared with FLUX.1-Fill-Dev (11.9B) and SD3.5 Large-Inp. (8.05B), Moebius delivers 6× faster single-step latency.
- Natural Scenes: 0.92 FID and 0.091 LPIPS on Places2 (Small) surpass FLUX.1-Fill-Dev while remaining competitive across natural-scene mask conditions.Excluding PixelHacker, Moebius performs on par with FLUX.1-Fill-Dev across the natural-scene benchmarks.
- Portrait Scenes: 5.39 FID and 0.122 LPIPS on CelebA-HQ match MAT, while 8.15 FID and 0.231 LPIPS on FFHQ lead portrait-scene results.Moebius surpasses other diffusion models and eclipses the 10B-level industrial models in portrait inpainting.
- Human Evaluation: 31.76% average human preference for Moebius nearly matches PixelHacker at 32.18% across natural, portrait, and real-world scenarios.The double-blind study sampled 50 cases per scenario and included 22 expert and general-user participants.
- Architectural Ablation: Holistically integrating all structural modifications unlocks the optimal efficiency frontier, whereas distilling isolated lightweight operators fails to achieve optimal quality-efficiency balance.The comparison is conducted under the distillation framework through experiments 11–15 and identifies holistic integration as necessary for extreme compactness.
- Distillation Ablation: 74.20 FID from coarse-grained loss alone improves to 26.43 after adding fine-grained distillation and latent perceptual constraints.The ablation supports strictly latent-space multi-granularity optimization as the key mechanism for restoring compressed-network quality.
5 Conclusion
Moebius combines LλMI blocks with adaptive multi-granularity latent distillation to address the representation bottleneck caused by extreme structural compression. This synergy enables a 0.22B-parameter specialist to rival the generation quality of 10B-level industrial generalists.
- 5 Conclusion: Moebius combines LλMI blocks with an adaptive multi-granularity latent distillation strategy to overcome the representation bottleneck of extreme structural compression.The paper describes this combination as the framework’s central design synergy.
- 5 Conclusion: 0.22B parameters enable Moebius to rival the generation quality of 10B-level industrial generalists such as FLUX.1-Fill-Dev.The conclusion characterizes Moebius as a highly efficient specialist for high-fidelity image inpainting.
Supplementary Materials of Moebius
The supplementary materials reinforce Moebius’s performance through qualitative comparisons, failure analysis, CFG ablations, and OOD evaluation. They show strong fidelity and generalization, while documenting detail loss under extreme compression.
- Additional Showcases on Natural and Portrait Scenes: Moebius achieves superior contextual consistency on diverse Places2 natural scenes compared with 10B-level generalists FLUX.1-Fill-Dev and SD3.5 Large-Inp.It avoids semantic shifts, over-generation, structural confusion, blurriness, and artifacts across challenging backgrounds.
- Additional Showcases on Natural and Portrait Scenes: Moebius preserves facial symmetry, intricate skin textures, and plausible structure on CelebA-HQ and FFHQ portraits when critical facial components are masked.The comparisons report sharper restoration with fewer color discrepancies, artifacts, and semantic inconsistencies than 10B-level generalists.
- Comparison with Commercial Systems: Moebius delivers highly competitive visual fidelity against commercial systems Nano Banana and Qwen Image Edit despite their massive computational and parameter scales.The comparison highlights comparable quality while handling complex detail restoration.
- Failure Case Analysis: Extreme structural compression can cause minor detail loss or less plausible textures in tiny background regions with severely limited context.Moebius generally restores structured and natural contexts but struggles with fine-grained geometry in these cases compared with teacher model PixelHacker.
- Ablation of Classifier-Free Guidance (CFG): CFG ablations identify optimal guidance scales of 2.5 for natural Places2 scenes and 2.0 for portrait CelebA-HQ scenes.These values are used as the default settings after evaluating varying CFG scales on the 51k-step Places2 and 60k-step CelebA-HQ checkpoints.
- Evaluation of Out-of-Distribution (OOD) Performance: OOD evaluation shows strong results on natural and portrait scenes, demonstrating Moebius’s excellent generalizability and zero-shot capability.The evaluation samples 10k LVIS images for OOD natural scenes and 3k DeepFakeFace wiki images for OOD portrait scenes.