VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

Hossein Shahabadi; Niki Sepasian; Mahdieh Soleymani Baghshah

Source-linked AI summary

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah

arXiv:2608.22521v1Aug 23, 2026cs.CV

TL;DR

Visual autoregressive models still violate prompt-specified attribute bindings and spatial relations, and diffusion-oriented test-time alignment does not directly fit next-scale VAR sampling. VISTA addresses this gap by optimizing intermediate representations through a frozen transformer with extensible cross-attention objectives. Across two benchmarks and two model scales, it improves all targeted categories, with nearly 20% and almost 6% mean targeted-score gains on 2B and 8B backbones, respectively, while preserving image quality.

Problem
Next-scale VAR models exhibit persistent compositional failures, while existing diffusion alignment methods and VAR trajectory-selection methods do not provide a comparable gradient-based solution.
Method
VISTA optimizes intermediate representations in Infinity’s frozen generation process and uses a reusable registry of differentiable cross-attention objectives without updating parameters or requiring training.
Results
VISTA improves every targeted category across two benchmarks and backbones, raising mean targeted scores by 19.5% on Infinity-2B and 5.9% on Infinity-8B.
Takeaways & Limitations
VISTA lets Infinity-2B surpass Infinity-8B on most targeted categories and their average, while its outputs receive nearly 20% higher preference-model scores without image-quality degradation.
Takeaways & Limitations
Experiments cover two backbones from one model family, and objective coverage excludes numeracy and non-spatial relations.

Abstract

from arXiv · show

Visual autoregressive (VAR) models have emerged as a fast, high-quality alternative to diffusion for text-to-image generation, but like diffusion models they exhibit persistent compositional failures, producing images that violate the attribute bindings and spatial relations specified in the prompt. While a rich line of test-time alignment methods has developed for diffusion, no comparable approach exists for next-scale VAR generation, whose stateful, discrete, multi-resolution sampling process makes existing techniques inapplicable. We close this gap with \textbf{VISTA} (\textbf{Vi}sual Autoregressive \textbf{S}emantic \textbf{T}est-time \textbf{A}lignment), the first gradient-based test-time alignment framework for next-scale autoregressive image generation. Built on Infinity, VISTA intervenes directly in the generation process, optimizing intermediate representations through the frozen transformer to steer visual predictions toward compositional constraints, without modifying model parameters or requiring additional training. VISTA introduces the mechanisms needed to make such optimization stable across scales, together with an extensible objective space that any differentiable constraint on cross-attention can plug into. Across two benchmarks and two model scales, VISTA improves every targeted compositional category, raising the mean targeted score by nearly 20\% on a 2B backbone and almost 6\% on an 8B backbone, with the largest gains on spatial relations. Image quality is preserved: an independent preference model VISTA never optimizes scores its outputs nearly 20\% higher. Notably, the 2B model with VISTA surpasses a backbone four times its size, indicating that a substantial part of the compositional gap between model scales is recoverable at test time.

1 INTRODUCTION

VAR models suffer persistent compositional failures, while diffusion’s test-time alignment methods do not transfer directly to next-scale VAR generation. VISTA addresses this gap with gradient-based steering and improves targeted compositional performance without retraining or sacrificing image quality.

Next-scale VAR models systematically fail on attribute binding, spatial reasoning, and relational tasks, even as model scale increases.
Existing VAR test-time methods search sampled trajectories, so they are limited when satisfying candidates are rarely generated.
VISTA is the first gradient-based test-time framework for compositional alignment in next-scale VAR generation.
VISTA optimizes intermediate representations through a frozen transformer and supports differentiable cross-attention objectives for binding, planar relations, and depth ordering.
VISTA improves every targeted category across T2I-CompBench and GenEval while preserving image quality and requiring no additional training or task-specific data.On T2I-CompBench, it gains 84% relatively on 2D spatial relations and 12–20% on color, texture, and shape binding; on GenEval, it nearly triples object positioning accuracy.

2 PRELIMINARIES

Next-scale VAR generates images as increasingly fine discrete token maps, with each scale conditioned on previously generated coarser scales. Infinity extends this framework to free-form text-to-image generation using continuous per-scale inputs before discrete sampling.

VAR generates images as a sequence of discrete token maps with increasing spatial resolution, each conditioned on all coarser maps.
Infinity adapts next-scale VAR generation to free-form text-to-image synthesis using text-conditioning sequences from a frozen text encoder.
At each scale, Infinity upsamples prior quantized codes into a continuous embedding, processes it with self- and cross-attention, then discretely samples the next token map.Keys and values from each scale are cached for later scales.
Compositional failure means violating prompt constraints on entity cardinality, attributes, or relations, independent of overall visual quality.These constraints are later operationalized as differentiable cross-attention objectives.

3 METHOD

VISTA steers selected intermediate representations in Infinity so cross-attention satisfies prompt-derived compositional constraints. Its stable optimization mechanisms support reusable objectives for binding, planar relations, and depth ordering, including an attention-based occlusion formulation.

VISTA replaces the continuous input at selected scales with an optimized representation before frozen-transformer processing, allowing later scales to inherit the intervention.No parameters are updated and no training data is required.
VISTA reconstructs cross-attention with the optimized query and fixed keys to create a differentiable attention map over text tokens.
Cache restoration preserves the original stateful generation path after extra optimization passes, while per-token normalization makes updates effective across large multi-scale inputs.
VISTA combines active objective gradients by direction rather than summing incomparable loss magnitudes, and reallocates steps according to initial gradient severity.
Steering begins at coarse non-degenerate scales because coarse perturbations affect global structure whereas fine perturbations mainly alter local detail.
Prompt parsing supplies entity, attribute, and relation constraints that become differentiable objectives over cross-attention maps.
For depth ordering, VISTA uses occlusion geometry in planar attention rather than directly measuring depth or relying on an external estimator.The method tests whether a behind-object boundary wraps around the occluding object.

4 EXPERIMENTS

VISTA is evaluated on two Infinity backbones across T2I-CompBench and GenEval, with targeted compositional improvements, strong spatial gains, and adjustable quality–cost trade-offs. It also compares favorably with search-based test-time scaling and shows that a 2B model can outperform the 8B backbone on many targeted measures.

4.1 SETUP: VISTA is evaluated on Infinity-2B and Infinity-8B across T2I-CompBench and GenEval, using ScaleKV cache compression and four seeds.The experiments compare baseline and VISTA configurations, alongside prior systems where applicable.
4.2 MAIN RESULTS: 19.5% and 5.9%: VISTA raises the mean targeted T2I-CompBench score on Infinity-2B and Infinity-8B, respectively.Every targeted category improves on both backbones; the smaller 8B gain is attributed to its stronger starting point and lower headroom.
4.2 MAIN RESULTS: 84.2%: VISTA delivers its largest Infinity-2B gain on 2D Spatial, while Color, Texture, and Shape improve by 12–20%.The 3D Spatial category improves by 11.3%, whereas Complex improves by 4.2%.
4.2 MAIN RESULTS: Infinity-2B surpasses Infinity-8B on four of six targeted categories and on the targeted-category average, despite having one quarter as many parameters.The 2B model falls short only on Texture and Shape; coarse-scale correction propagates through later unsteered scales.
4.3 GENEVAL: 198% and 42.7%: GenEval Position improves on Infinity-2B and Infinity-8B, raising overall scores by 12.5% and 5.9%, respectively.Infinity-2B reaches 0.745 from 0.250 on Position and surpasses Infinity-8B’s 0.578.
4.3 GENEVAL: 19.8% versus 7.3%: VISTA raises the targeted average more than TTS-VAR and leads in every targeted category.The advantage is largest where the backbone systematically fails, such as 2D Spatial; selection helps when failures are stochastic.
4.4 QUALITY, FIDELITY, AND COST: Three steered scales are the default because compositional quality saturates while cost jumps at the fourth, with fidelity metrics showing different trends.Table 3 measures targeted score, ImageReward, aesthetic score, and generation time; the scale knob exposes a quality–cost trade-off.

5 LIMITATIONS

The evaluation is limited to two backbones from the Infinity family, so the mechanisms have not yet been verified on other next-scale VAR models. Objective coverage is also incomplete, and combined constraints are handled through isolated objectives.

5 LIMITATIONS: The mechanisms are verified only on Infinity, despite being designed around properties shared by next-scale VAR models.The experiments cover two backbones from one model family.
5 LIMITATIONS: VISTA covers attribute binding, planar relations, and depth ordering, but leaves numeracy and non-spatial relations at baseline.Numeracy fits the framework but is disabled, while no objective is offered for non-spatial relations.
5 LIMITATIONS: Complex prompts gain least because each objective treats one constraint in isolation rather than jointly optimizing multiple constraints.The parser recovered instances in the evaluated templated and tried free-form prompts, but parsing was not systematically evaluated beyond them.

6 CONCLUSION

VISTA extends test-time compositional alignment to next-scale VAR generation through a modular framework built around differentiable cross-attention objectives. Its results support transferring the alignment premise from diffusion while adapting mechanisms to VAR-specific generation dynamics.

VISTA improves targeted attribute-binding, planar-relation, and depth-ordering categories across two benchmarks and two backbones.
The framework separates general optimization mechanisms from a pluggable registry of compositional objectives.
VISTA’s premise transfers from diffusion, but its specific mechanisms must address stateful, multi-scale, discrete-token generation.
Unlike search-based VAR methods, trajectory steering is not bounded by how often the backbone already samples a satisfying candidate.
The framework’s virtual-point construction uses sharpened attention distributions, normalized grid coordinates, and differentiable perimeter-weighted sampling for boundary containment.

C PER-SEED RESULTS AND VARIANCE

Per-seed analysis shows that reported VISTA improvements are generally much larger than seed variation, while compression has negligible impact on the reference results.

0.008 is the maximum category change from cache compression in either direction, matching the order of seed-to-seed variation.
0.014 is the largest per-category standard deviation, while the targeted-average standard deviation is 0.002 on both backbones.
0.095 and 0.033 are VISTA’s targeted-average gains on the 2B and 8B backbones, respectively, far exceeding typical seed noise.
The 2B Complex gain is 0.014, making it the exception because it is only a few standard deviations above the noise floor.

D ADDITIONAL QUALITATIVE RESULTS

Qualitative examples show VISTA correcting simultaneous spatial and attribute-binding errors and reducing subject merging in a spatial prompt.

VISTA corrects both the reversed left-right relation and attribute leakage in a combined binding-and-spatial prompt.
The combined correction uses binding and spatial objectives within the same per-token update rather than trading one constraint against the other.
VISTA visibly reduces blurred and overlapping boundaries between a cat and dog in the staircase example.

E INFERENCE OVERHEAD: FULL BREAKDOWN

Inference overhead depends strongly on the number of steered scales and benefits from evaluating multiple objectives jointly, with three scales remaining the practical default.

3.09 seconds is the unmodified Infinity-2B baseline per image, with 10.95 GB peak memory.
At three scales and Nmax=5, binding, spatial, and depth objectives add 1.88, 1.78, and 1.85 seconds individually.
2.87 seconds is the measured overhead for all three objectives together, versus 5.51 seconds if their isolated costs stacked.
Joint objectives are cheaper because they share one forward and backward pass per optimization step.
Four steered scales cause disproportionate and less stable timing, while Infinity-8B memory rises from 22.7 GB to 36.1 GB.

F DETAILED COMPARISON WITH TTS-VAR

VISTA and TTS-VAR intervene in next-scale VAR generation through fundamentally different mechanisms: VISTA steers one trajectory, whereas TTS-VAR searches among sampled trajectories. Their intervention points and costs explain their different compositional behavior.

Approach: VISTA modifies a single trajectory, while TTS-VAR selects only images the unmodified backbone could have produced.This distinction separates gradient steering from candidate search.
Approach: TTS-VAR searches over sampled trajectories, using clustering for coarse-scale diversity and resampling-based selection at fine scales.Its published schedule keeps 8N candidates at the coarsest scales and narrows to 1N at the finest.
Compositional comparison: On 2D Spatial, selection reaches 0.269 against steering’s 0.442, a gain of 15% against 89% over the same baseline.This is the widest margin between the methods in the reported category comparison.
Intervention point: VISTA can steer at coarse scales where TTS-VAR’s pixel-space verifier is uninformative, while TTS-VAR selects at later scales where decoded-image scoring works.The methods are therefore described as orthogonal and potentially composable.
Cost: TTS-VAR incurs multiplicative runtime, larger coarse-scale memory, additional pretrained networks, and repeated decoding costs as candidate counts increase.VISTA is evaluated on both Infinity-2B and Infinity-8B, whereas the reported TTS-VAR run uses Infinity-2B only.
Provenance: Table 6 distinguishes adopting a motivating principle from adopting a prior loss function when attributing objective provenance.This distinction frames which components are borrowed versus developed in VISTA.

G BORROWED OBJECTIVES AND PROVENANCE

VISTA combines borrowed attention objectives with newly developed components, explicitly separating prior principles and functional forms. Its depth objective is the main objective-level contribution, using differentiable occlusion geometry instead of estimated depth.

Scope of contribution: The objective framework is presented as an extensible space over cross-attention rather than as a claim that every instantiated loss is novel.This provenance framing limits the contribution claim for borrowed terms.
Attribute binding: Attribute binding uses prior attention-based objectives that compare each attribute–noun pair against competing pairs.The maps are smoothed and normalized before the contrastive comparison.
Planar relations: Planar relations use PSG’s probability-of-superiority objective, with VISTA-specific temperature and relation-inversion choices.The planar loss is otherwise adopted unchanged, including its squared form.
Provenance: VISTA’s provenance analysis separates borrowing a principle from borrowing a functional form across its objective terms.The distinction is used to state precisely what is adopted from prior work and what is new.
Depth relations: VISTA’s depth objective tests occlusion-induced attention geometry rather than estimating metric depth.It distinguishes occlusion from adjacency, remains differentiable at coarse scales, and requires no model beyond the backbone.

H IMAGE FIDELITY

VISTA improves prompt-conditioned compositionality while largely preserving image quality under independent judges, though stronger intervention can reduce unconditional aesthetic quality. The fidelity study is limited by single-seed measurement and disagreement between aesthetic predictors.

Evaluation: The fidelity study evaluates VISTA with ImageReward, CLIPScore, and an aesthetic predictor that it does not optimize.All comparisons use paired base generations, and the table varies the number of steered scales.
Overall fidelity: Prompt-conditioned quality remains essentially unchanged from one to three steered scales and declines at four, while unconditional aesthetic quality declines throughout.The authors attribute the pattern to increasing displacement from the backbone’s sampled representation region.
Per-category fidelity: Texture gains +0.351 in ImageReward, followed by 3D Spatial +0.311, 2D Spatial +0.219, Shape +0.168, Color +0.149, and Complex +0.034.The default configuration improves ImageReward in every reported category.
Per-category fidelity: Complex prompts have a 49% per-prompt win rate versus 54–65% elsewhere, consistent with isolated objectives sometimes disturbing multi-constraint images.Complex prompts assert several constraints simultaneously.
Caveats: The study uses a single seed rather than the four seeds used for Table 1, so effect magnitudes retain the variance of one draw.The authors state that the direction of effects is supported, but their magnitudes remain seed-sensitive.
Caveats: The two aesthetic predictor versions disagree on magnitude, reporting −1.3% versus −2.2% at the default configuration.Both versions are reported instead of selecting one.

J ALTERNATIVE OBJECTIVE INSTANTIATIONS

Alternative objective experiments test whether VISTA’s optimization machinery generalizes beyond its default losses. Most isolated objectives improve their targeted category, with coarse-scale performance favoring objectives that summarize attention maps before comparison.

Objective independence: Sixteen of eighteen alternative objective configurations improve their isolated category over the unsteered backbone.All configurations use identical capture, normalization, combination, and budget machinery.
Evaluation design: T2I-CompBench categories isolate objectives: binding terms on Color, Texture, and Shape, planar terms on 2D Spatial, and depth terms on 3D Spatial.Complex prompts are omitted because they activate several terms simultaneously.
Planar objectives: Planar objectives that summarize attention maps before comparison gain between 112% and 176%, versus 31% for separation and 16% for attention IoU.The comparison suggests summary-based objectives transfer better to coarse scales than elementwise comparisons.
Binding objectives: Pointwise binding objectives are weakest, with JSD leaving all three categories exactly at baseline.At 4×4 to 8×8, pointwise gradients redistribute individual token activations rather than moving an object.
Depth objectives: The depth proxy is the only configuration that actively hurts, scoring 0.419 against a baseline of 0.439.This result marks a failure case within the otherwise generally positive objective sweep.
Objective choice: Relative-centroid outperforms PoS on 2D Spatial, 0.473 against 0.454, showing that VISTA’s objective space is not uniformly optimized by its default choices.The authors do not claim to have searched the full objective space.