PaperlayerView exact revision on arXiv

Source-linked AI summary

ScalingAR: Scaling Confidence for Autoregressive Image Generation

Harold Haodong Chen, Xianfeng Wu, Wen-Jie Shu, Rongjin Guo, Disen Lan, Harry Yang, Ying-Cong Chen

arXiv:2509.26376v3cs.CV

TL;DR

Test-time scaling for next-token-prediction autoregressive image generation remains underexplored, in part because partial image decodes are unstable. ScalingAR profiles confidence from visual token entropy and conditional signals, then uses it for pruning and guidance scheduling; it reports gains in quality, efficiency, and robustness.

  • Problem

    Test-time scaling for next-token-prediction autoregressive image generation remains underexplored, while holistic images, ambiguous objectives, and unstable partial decodes complicate existing approaches.

    Source: §1 · p. 1

  • Method

    ScalingAR combines intrinsic uncertainty and conditional signals into a confidence state for adaptive trajectory pruning and dynamic guidance scheduling.

    Source: §1 · p. 2

  • Results

    12.5% on GenEval and 15.2% on TIIF-Bench over base models, 62.0% less visual-token consumption while outperforming classic baselines, and 26.0% less performance degradation in challenging settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Evaluations on general and compositional benchmarks show improved generation quality, generalizability, and robustness for existing autoregressive models.

    Source: §6 · p. 8

    Confidence estimation may not fully capture uncertainty or semantic alignment, and entropy signals can vary with model calibration, training, and architecture.

    Source: F. Limitation and Future Works · p. 17

Abstract

from arXiv · show

Test-time strategies have shown remarkable success in improving large language models, but their application to next-token prediction (NTP) autoregressive (AR) image generation remains largely underexplored. Existing test-time scaling (TTS) methods for visual autoregressive models (VAR) rely on frequent partial decoding and external reward models, which are inefficient and often ineffective for NTP-based image generation due to the inherent instability of intermediate decoding results. To address these limitations, we propose ScalingAR, a novel test-time scaling framework tailored for NTP-based AR image generation. ScalingAR introduces token entropy as a confidence signal and operates at two complementary levels: (i) Profile Level, integrates intrinsic uncertainty and conditional utilization into a unified confidence state, and (ii) Policy Level, leverages this state for adaptive trajectory pruning and dynamic guidance scheduling. Without requiring early decoding or auxiliary rewards, ScalingAR achieves significant improvements across diverse benchmarks. Experiments show that ScalingAR (I) improves base models by $12.5\%$ on GenEval and $15.2\%$ on TIIF-Bench, (II) reduces visual token consumption by $62.0\%$ while outperforming baselines, and (III) enhances robustness, mitigating performance degradation by $26.0\%$ in challenging scenarios. These results establish ScalingAR as a robust and efficient test-time scaling solution for autoregressive image generation.

1. Introduction

ScalingAR addresses the limited progress of test-time scaling for next-token-prediction image generation with confidence profiling and adaptive control. Experiments report gains in quality, token efficiency, and robustness.

1. Introduction

  • Test-time scaling for autoregressive visual generation remains sparse because image outputs are holistic, lack a single ground-truth target, and have visually unstable partial decodes.Existing next-scale methods can score intermediate images, but predicting large residual token maps and frequent decoding are inefficient and less suitable for next-token prediction.Source: §1 · p. 1
  • ScalingAR profiles confidence from intrinsic visual-token entropy and conditional signals, then uses that state for trajectory pruning and dynamic guidance scheduling.The framework avoids frequent partial decoding and external reward models.Source: §1 · p. 2
  • The paper investigates token entropy in visual token generation and relies on intrinsic model signals rather than frequent early decoding or external rewards.It presents this as enabling a more efficient and reliable scaling process.Source: §1 · p. 2
  • 12.5% on GenEval and 15.2% on TIIF-Bench over base models; visual-token use falls 62.0% versus classic baselines, and performance degradation falls 26.0% in challenging settings.The reported comparisons cover LlamaGen and AR-GRPO as base models and Importance Sampling and Best-of-N as classic baselines.Source: §1 · p. 2

2. Related Work

Related work covers autoregressive image generation and test-time scaling, while identifying a gap in methods tailored to next-token-prediction image generation. ScalingAR applies visual token entropy without early decoding or external rewards.

Autoregressive Image Generation

  • Visual autoregressive models predict discrete image tokens sequentially, while next-scale models generate coarse-to-fine token maps and masked autoregressive models predict tokens in parallel.The field's mainstream approach remains next-token prediction, particularly as unified models handle textual and visual tokens.Source: §2 · pp. 2–3

Test-Time Scaling

  • Test-time scaling includes longer reasoning trajectories and parallel generation with aggregation, such as Self-Consistency and Best-of-N.Related work also uses token entropy to estimate confidence in reasoning traces.Source: §2 · p. 3
  • Test-time scaling for autoregressive image generation remains limited; Best-of-N and TTS-VAR use external rewards and frequent early decoding.TTS-VAR targets next-scale prediction, while the cited Best-of-N work applies external reward models to image generation.Source: §2 · p. 3
  • ScalingAR is presented as the first test-time scaling strategy tailored to autoregressive image generation, exploring visual token entropy without early decoding or additional rewards.The passage distinguishes this visual-generation use from prior entropy-based confidence estimation for reasoning traces.Source: §2 · p. 3

3. Preliminaries

The preliminaries describe autoregressive generation as sequential next-token prediction and introduce entropy and confidence as measures of predictive uncertainty.

3. Preliminaries

  • Autoregressive models generate sequences by predicting each next token from previously generated tokens.Past information informs future predictions, making the approach suitable for sequential data generation.Source: §3 · p. 3
  • Training typically maximizes the likelihood of observed sequences to learn their underlying distribution.The stated objective encourages coherent and contextually appropriate generation.Source: §3 · pp. 3–4
  • Token entropy quantifies unpredictability in the model’s output distribution, while token confidence is derived from that distribution.Low entropy indicates greater certainty; high confidence corresponds to a sharper predicted distribution.Source: §3 · p. 4

4. Methodology

ScalingAR uses a calibrated confidence state derived from model logits to guide test-time control in next-token-prediction image generation. Its two coupled levels profile generation health and use that profile for trajectory pruning and guidance modulation.

4. Methodology

  • ScalingAR operates without frequent partial decoding or external rewards, using a dual-channel confidence profile and policies for pruning trajectories and modulating guidance.The profile transforms logit statistics into a normalized stability signal; the policies act on that state.Source: §4 · p. 4

4.1. Dual-Channel Confidence Profile

  • The Dual-Channel Confidence Profile addresses local instability and semantic drift by tracking spatial anomalies and the marginal contribution of textual conditioning.The paper identifies high-entropy regions and fading prompt influence as failure modes.Source: §4.1 · p. 4

4.1.1. INTRINSIC CHANNEL

  • The intrinsic channel combines temporally smoothed token confidence with worst-block spatial stability, penalizing trajectories when any critical region fails.Token confidence combines normalized entropy and top-1/top-2 probability margin; worst-block stability focuses on the highest-entropy blocks.Source: §4.1.1 · p. 5

4.1.2. CONDITIONAL CHANNEL

  • The conditional channel measures prompt influence using the KL divergence between conditional and unconditional predictions, then normalizes it dynamically into a drift score.Low normalized values indicate that the model is ignoring the prompt and signal a need for intervention.Source: §4.1.2 · p. 5

4.1.3. UNIFIED CONFIDENCE STATE

  • The unified confidence state combines intrinsic and conditional signals with relative rebound, distinguishing temporary confidence dips from prolonged low-confidence failure.The state is defined as S_t = {C_t, R_t, I_t, bD_t} and serves as the basis for control policies.Source: §4.1.3 · p. 5

4.2. Confidence-Guided Policies

  • Confidence-guided policies use the calibrated state to prune unpromising trajectories and dynamically modulate classifier-free guidance to balance semantic alignment.Pruning reclaims computation, while the guidance scheduler adjusts CFG scale.Source: §4.2 · p. 5

4.2.1. ADAPTIVE TERMINATION

  • Adaptive termination uses a batch-quantile threshold updated by EMA, pruning relatively weak trajectories while a recovery window protects candidates with transient instability.A fixed cutoff could aggressively prune valid samples on difficult prompts; candidates below threshold are terminated only if they fail to rebound during probation.Source: §4.2.1 · p. 5

4.2.2. GUIDANCE SCHEDULER

  • The closed-loop guidance scheduler varies CFG scale with the confidence profile, strengthening guidance during drift or instability and relaxing it after a strong rebound.This compensatory adjustment is intended to navigate the quality-alignment trade-off more effectively than static baselines.Source: §4.2.2 · pp. 5–6

5. Experiments

ScalingAR is evaluated on general and compositional image-generation tasks, compared with conventional test-time scaling baselines, and assessed for component contributions, scalability, and robustness.

5.1. Experimental Settings

  • GenEval and TIIF-Bench assess general and compositional text-to-image generation, respectively, with LlamaGen and AR-GRPO as base models and Importance Sampling and Best-of-N as conventional baselines.SimpleAR and Janus-Pro results are reported in the appendix.Source: §5.1 · p. 6

5.2. Performance & Efficiency Comparison

  • ScalingAR outperforms Importance Sampling and Best-of-N on general and compositional generation quality, with stronger human-preference and qualitative results, while using fewer visual tokens.The comparison includes user-study dimensions and token consumption on TIIF-Bench.Source: §5.2 · pp. 6–7, Figure 4 · p. 7, Figure 5 · p. 8

5.3. Ablation Analysis

  • Removing Token-Level Confidence, Worst-Block Stability, or the Conditional Channel degrades TIIF-Bench performance, supporting the roles of intrinsic signal profiling and condition-state balance.The Policy Level is ablated separately in Appendix §D.Source: §5.3 · pp. 7–8, Table 2 · p. 7

5.4. Scalability & Robustness Analysis

  • Across varying sample numbers and token lengths, ScalingAR outperforms Importance Sampling and Best-of-N; it also shows robustness advantages under impossible prompts and challenging conditions.The impossible-prompt evaluation uses prompts for which none of the candidates is ideal.Source: §5.4 · p. 8, Figure 6 · p. 8, Figure 5 · p. 8

6. Conclusion

ScalingAR applies token entropy and a two-level confidence-and-policy design to NTP autoregressive image generation, and evaluations report improved quality, generalizability, and robustness.

6. Conclusion

  • ScalingAR uses visual token entropy as an intrinsic signal without partial decoding or external rewards.Its Profile Level calibrates confidence, while its Policy Level adaptively prunes trajectories and modulates conditioning.Source: §6 · p. 8
  • Evaluations on general and compositional benchmarks show improved generation quality for existing autoregressive models.Source: §6 · p. 8
  • The conclusion characterizes ScalingAR as a strong baseline for autoregressive image-generation test-time scaling, with generalizability and robustness improvements.Source: §6 · p. 8

Impact Statement

The impact statement presents ScalingAR as a potential contributor to sustainable AI through reduced generation overhead, while noting that it does not itself filter harmful content.

Impact Statement

  • Distribution-aware pruning eliminates unpromising generation trajectories to reduce computational overhead and energy consumption compared with standard test-time scaling strategies.Source: Impact Statement · p. 9
  • The stated societal benefit is promoting sustainable AI practices through more efficient autoregressive image generation.Source: Impact Statement · p. 9
  • ScalingAR inherits its base models’ safety profiles and biases and does not intrinsically filter harmful content.The authors recommend deployment with safety guardrails and content moderation.Source: Impact Statement · p. 9

A. Further Illustration of Entropy in AR Image Generation

The appendix motivates token entropy as a confidence signal: high-entropy regions often accompany uncertainty and poorer generation, although high entropy does not guarantee poor results.

A. Further Illustration of Entropy in AR Image Generation

B. Results of More Base Models

ScalingAR was evaluated on SimpleAR-1.5B and Janus-Pro-1B using the same hyperparameters as the main experiments. The authors report significant improvements on both models, with qualitative examples also provided.

B. Results of More Base Models

C. Implementation Details

ScalingAR uses one configuration across four AR models, with parameters organized around confidence-state construction and guidance policies. Its workflow updates confidence signals, applies dynamic guidance, and prunes trajectories; the appendices also describe evaluation settings and a user study.

C.1. Universal Configuration Strategy

  • One universal configuration is applied to LlamaGen, AR-GRPO, SimpleAR, and Janus-Pro, without model-specific hyperparameter tuning.The authors describe this as demonstrating structural robustness.Source: §C.1 · p. 13

C.2. Hyperparameters

  • Hyperparameters are grouped into confidence-state construction and guidance-policy execution, with sensitivity analyses reported separately.Listed settings include 0.75/0.25 intrinsic/conditional channel weights, a 0.2 pruning quantile, and a 12.5% warm-up period.Source: §C.2 · p. 13, §C.3 · p. 14

C.3. Algorithm Workflow

  • The workflow extracts signals, updates confidence states, adjusts guidance during sampling, then prunes candidates below an adaptive threshold unless they are recovering.The final output is decoded from the surviving candidate with the highest confidence.Source: §C.3 · pp. 13–14

D.1. More Details of Experimental Settings

  • Baseline comparisons use Importance Sampling and Best-of-N with ImageReward and N = 8; robustness is evaluated on filtered IPV-TXT prompts using GPT-4o binary judgments.The baseline CFG scales remain at each base model’s defaults.Source: §D.1 · pp. 14–15
  • The authors attribute outperforming Best-of-N to active rectification through closed-loop guidance, rather than selecting among static trajectories using an external reward proxy.They state that dynamic guidance stabilizes generation and can reach regions unavailable to static decoding strategies.Source: §D.1 · p. 15
  • A user study collected mean opinion scores from 15 volunteers evaluating generated images.The study instructions ask participants to score images on a 1–5 scale.Source: §D.1 · p. 15

D.2. More Analysis

The additional analyses examine policy ablations, computational costs, and sensitivity to confidence and guidance weights and thresholds. They report complementary policy contributions, reduced resource use, and parameter settings that balance generation quality and efficiency.

D.2. More Analysis

  • Combining termination and scheduling achieves the best ablation results, with each mechanism also improving performance independently.Termination prunes low-confidence trajectories; scheduling adjusts conditioning strength to balance semantic alignment and diversity.Source: §D.2 · p. 16, Table 4 · p. 16
  • 69.56s wall-clock time gives ScalingAR a 3.1× speedup over Best-of-N, while processing 2,350 tokens on average, a 71% reduction.ScalingAR uses 4.23 TFLOPs and 18.16 GB peak memory, compared with Best-of-N’s 39.12 TFLOPs and 48.72 GB; the comparison uses NVIDIA H200 GPUs on GenEval.Source: Table 5 · p. 16, §D.2 · p. 16
  • On TIIF-Bench, intrinsic/conditional weights of 0.75/0.25 perform best; scheduler weight α peaks at 0.3, while moderate β and λ improve results.Removing the conditional channel degrades performance, and overemphasizing α may reduce diversity.Source: §D.2 · p. 16, Figure 9 · p. 16
  • A pruning quantile of p = 0.20 and recovery threshold δrec = 0.05 provide the reported best trade-off between accuracy and token efficiency.Disabling recovery causes performance drops; extreme threshold values can either waste tokens or terminate viable trajectories early.Source: Figure 10 · p. 16, §D.2 · p. 17
  • Entropy/margin weights of 0.7/0.3 perform best overall, while token/block weights of 0.65/0.35 achieve the highest scores across most metrics.The analysis reports degradation when either margin or block-level stability is overemphasized.Source: §D.2 · p. 17, Figure 11 · p. 17

E. Exhibition Board

  • Additional comparison demonstrations are provided for AR-GRPO in Figure 12 and LlamaGen in Figure 13.The supplied passage does not specify outcomes from these demonstrations.Source: E. Exhibition Board · p. 17

F. Limitation and Future Works

The authors identify limits in ScalingAR’s confidence estimation and its dependence on model calibration and entropy signals. They propose finer-grained confidence measures and integrating entropy-based signals into training and test-time as future work.

F. Limitation and Future Works

  • Token entropy is a first step but may not fully capture uncertainty or semantic alignment, and calibration and entropy signals can vary across training and architectures.These factors constrain confidence estimation in AR image modeling.Source: F. Limitation and Future Works · p. 17
  • Future work will explore finer-grained confidence measures and integrate entropy-based signals into both training-time and test-time.The stated aim is a more unified pipeline.Source: F. Limitation and Future Works · p. 17

Related papers

Loading 2509.26376v3…