Source-linked AI summary
LoopViT: Scaling Visual ARC with Looped Transformers
Wen-Jie Shu, Xuerui Qiu, Rui-Jie Zhu, Harold Haodong Chen, Yexin Liu, Harry Yang
TL;DR
Feed-forward vision transformers bind reasoning depth to parameter capacity, limiting their fit for iterative visual induction. Loop-ViT uses weight-tied recurrence, a Hybrid Block, and entropy-based Dynamic Exit to adapt computation. On ARC-AGI-1, its 18M model reaches 65.8% and outperforms larger feed-forward baselines, supporting iteration as an efficient scaling axis.
Problem
Feed-forward vision transformers bind computational depth to parameter size, limiting their ability to capture the iterative, algorithmic nature of visual reasoning.
Method
Loop-ViT repeatedly applies a weight-tied Hybrid Block combining depthwise convolutions and self-attention, with parameter-free entropy-based Dynamic Exit.
Results
65.8%: the 18M Loop-ViT reaches this ARC-1 score, while the 3.8M Small model reaches 60.1% versus 54.5% for the 18M VARC baseline.
Takeaways & Limitations
Iterative computation is presented as a more effective scaling axis than increasing parameter capacity for abstract visual reasoning.
Takeaways & Limitations
The Hybrid Block is based on the hypothesis that ARC tasks require both local pattern matching and global rule induction.
Abstract
from arXiv · showhide
Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
1. Introduction
ARC visual reasoning requires inferring compositional transformations from few examples, but feed-forward ViTs struggle to allocate computation to puzzles with varying reasoning depth. Loop-ViT addresses this with weight-tied recurrence, adaptive halting, and improved accuracy–efficiency trade-offs.
- Motivation: ARC-AGI tasks require precise visual transformations inferred from only 2–4 demonstration pairs.Examples include recursive filling, object relocation, and gravity-like dynamics.
- Motivation: Feed-forward ViTs impose a fixed computation graph, causing diminishing returns when reasoning complexity increases.This creates a mismatch for puzzles requiring recursive or iterative processing.
- Approach: Loop-ViT uses a weight-tied recurrent core with a Hybrid Block combining depthwise convolutions and self-attention.The design learns a reusable state-transition operator and supports multiple reasoning iterations.
- Approach: Dynamic Exit uses predictive entropy to halt inference when predictions stabilize, reducing average computation without compromising accuracy on hard tasks.The mechanism is parameter-free and lets easier tasks use fewer iterations.
- Results: 60.1%: the 3.8M-parameter Loop-ViT Small surpasses the 18M VARC baseline at 54.5% with roughly one-fifth as many parameters.The 18M Large model reaches 65.8% and outperforms large feed-forward expert ensembles.
- Results: Loop-ViT improves the empirical Pareto frontier across accuracy, compute, and parameters through iterative computation.The contribution claims scaling through iteration is more effective than scaling parameters for abstract reasoning.
2. Related Work
Prior ARC systems often serialize grids into language or discrete sequences, sacrificing spatial topology, while pure ViTs preserve visual input but lack an iterative reasoning bias. Loop-ViT extends recurrence to modern vision transformers as a scalable primitive for abstract visual reasoning.
- Language vs. Vision: Language-based and recurrent ARC methods serialize 2D grids into 1D representations, which discards spatial topology important for visual puzzles.LLMs commonly use JSON or ASCII, while recurrent models process discrete grid tokens.
- Language vs. Vision: The VARC framework shifted ARC toward pure vision by treating reasoning as image-to-image translation with Vision Transformers and standard augmentations.Standard ViTs nevertheless lack an inherent bias for iterative algorithm execution on complex ARC tasks.
- Recurrence: Recurrence has been associated with improved parameter efficiency and generalization, and latent thinking models use it to adapt computation to task complexity.Earlier computer-vision applications mainly refined continuous signals or targeted narrow synthetic domains.
- Recurrence: Loop-ViT applies weight-tied loops to Vision Transformers as a scalable primitive for abstract visual reasoning across diverse tasks.This positions recurrence as a bridge between visual representations and iterative algorithm execution.
3. Method
Loop-ViT combines 2D visual processing with iterative, weight-tied state refinement, using hybrid local convolutions and global attention. Its Dynamic Exit mechanism adapts inference duration by halting when predictive entropy indicates a stable state.
- Loop-ViT Pipeline: The model combines 2D vision input with looped inference, repeatedly refining internal representations and predictions.This bridges spatial inductive bias with recurrent computation rather than processing the grid in a single forward pass.
- Global Recurrent Architecture: Loop-ViT reuses a weight-tied transformer core across multiple iterations, decoupling computational depth from parameter count.The recurrent state is updated through repeated applications of the same transition operator, with step-dependent embeddings indicating computation progress.
- Hybrid Encoder Block: The Hybrid Block fuses global self-attention with local depthwise convolution to support rule induction and spatial transformations.Attention broadcasts information across the grid, while convolution updates local neighborhoods in a cellular-automaton-like manner.
- Hybrid Encoder Block: ConvGLU applies depthwise convolution only to image tokens while task tokens bypass the spatial operator.This heterogeneous processing preserves task-level rules while modeling local image connectivity.
- Dynamic Exit: Dynamic Exit halts inference when average pixel-wise predictive entropy falls below τ = 0.05, or continues until Tmax.The halted state is frozen, and the mechanism adds no parameters.
- Training Strategy: Training unrolls the shared core for a fixed number of steps with final-output supervision, while evaluation additionally finetunes shared weights on task demonstrations.The fixed-depth protocol is intended to encourage convergence of a reusable transition rule before task-specific adaptation.
4. Experiments
Experiments evaluate Loop-ViT on ARC-AGI through global performance, space–time scaling, inductive-bias ablations, dynamic halting, and qualitative analyses of recurrent computation.
- Experimental Setup: Loop-ViT is evaluated on ARC-AGI-1 using Pass@2 accuracy, with ARC-AGI-2 included for out-of-distribution generalization.The training split is augmented with synthetic RE-ARC samples.
- Global Performance and Efficiency: 11.2M-parameter LoopViT outperforms a 73M-parameter ensemble, demonstrating strong parameter efficiency.The experiments compare LoopViT with vision and language-model baselines.
- Space-Time Joint Scaling: 63.9% is reached by a high-capacity core at the computational limit, while low-capacity B = 2 cores gain the most from increasing loop steps.Increasing T emulates expressive depth through weight-tied recurrence, and performance continues scaling with T for deeper cores.
- Inductive Bias: Hybrid vs. Vanilla: The Hybrid core maintains a consistent accuracy lead over Vanilla Transformer cores across depths, supporting local spatial priors for grid reasoning.The Hybrid block combines depth-wise convolution with multi-head self-attention.
- Impact of Dynamic Exit: Dynamic Exit achieves higher accuracy with lower average inference compute than a fixed 6-step baseline.Adaptive halting is evaluated with T ∈[4, 8], and recurrent inference compute scales approximately linearly with executed iterations.
- Efficiency vs. Difficulty: 83.33% accuracy occurs for B = 2 samples exiting at Step 5, compared with 45.80% for samples requiring Step 8.The results associate earlier exits with easier instances and deeper computation with harder ones.
- Prediction Crystallization: Prediction volatility and entropy decline across iterations as outputs progressively approach the ground truth and confidence stabilizes.The synchronized reductions are described as a prediction-crystallization process resolving logical ambiguities.
- Step-wise Attention Dynamics: Attention evolves from dense global scanning in early steps to sparse localized execution focused on grid transitions in later steps.This qualitative pattern is presented as a transition from integrating demonstrators to executing the predicted rule.
5. Conclusion
The conclusion presents Loop-ViT as a recurrent alternative to feed-forward visual reasoning, separating reasoning depth from model capacity through iterative computation. Its weight-tied Hybrid Block and entropy-driven Dynamic Exit support the reported efficiency and adaptive computation results.
- Conclusion: Loop-ViT decouples reasoning depth from model capacity through recurrent computation.The conclusion frames iterative computation as a more effective scaling axis than parameter width for abstract induction.
- Conclusion: Attention patterns shift from broad early processing to focused, sparse later execution across recurrent steps.This qualitative evolution is presented as resembling deliberative human visual reasoning.
- Conclusion: The architecture combines a weight-tied Hybrid Block with a predictive-entropy Dynamic Exit mechanism.The Hybrid Block matches the cellular nature of ARC transformations, while Dynamic Exit allows latent-state crystallization.
- Conclusion: Loop-ViT is reported to significantly outperform larger feed-forward baselines while using adaptive iterative computation.The conclusion positions the model as a baseline for future research on more complex reasoning tasks.