PaperlayerView exact revision on arXiv

Source-linked AI summary

CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generation

Yichen Wu, Haoxuan Qu, Yihang Lou, Hossein Rahmani, Jun Liu

arXiv:2608.30689v1cs.CV

TL;DR

Long-context autoregressive SVG generation can lose global consistency because local token probabilities do not reliably evaluate complete rendered graphics. CANVAS combines power-sharpened trajectory likelihood with rendered-future feedback, using stroke-wise adaptive sampling under limited budgets; experiments report consistent improvements across benchmarks and most metrics without additional training.

  • Problem

    Standard autoregressive decoding may fail to preserve geometric relationships, spatial layouts, occlusion ordering, and composition in complete long-context SVGs.

  • Method

    CANVAS is a training-free, render-aware inference framework that combines complete-trajectory likelihood with visual feedback and adaptively samples stroke-wise future values.

  • Results

    CANVAS yields consistent improvements across both HeisenVec and IntroSVG benchmarks and the large majority of reported metrics, including better global consistency than Best-of-N with N = 5.

  • Takeaways & Limitations

    CANVAS improves long-context SVG global consistency across multiple autoregressive generators and complementary benchmarks without additional training.

  • Takeaways & Limitations

    Power-sharpened model likelihood alone remains insufficient to determine rendered geometry, occlusion relations, and overall composition.

Abstract

from arXiv · show

Autoregressive large models have recently advanced Text-to-SVG generation from simple icons to complex, long-context graphics, yet standard autoregressive decoding often fails to maintain global consistency across geometry, layout, occlusion, and composition. We introduce CANVAS (Consistency-Aware Navigation via Visual Adaptive Sampling), a training-free, render-aware inference framework that combines power-sharpened trajectory likelihood with visual feedback from rendered futures and derives a stroke-wise navigation rule. It effectively estimates each candidate stroke's future value under a limited generation and rendering budget and adaptively allocates samples according to candidate uncertainty, decision influence, and rollout cost. Experiments across multiple autoregressive SVG backbones and complementary benchmarks demonstrate improvements in global consistency, which includes sound geometric relationships, spatial layouts, occlusion ordering, and overall composition, without additional training, demonstrating the effectiveness and generalization ability of our framework.

1. Introduction

Long-context autoregressive SVG generation can produce lengthy sequences without preserving global consistency, because local token probabilities may favor trajectories that render poorly. CANVAS addresses this at inference time by combining trajectory likelihood with rendered visual feedback and adaptive stroke-wise sampling.

  • Motivation: Longer SVG sequences do not guarantee global consistency in geometry, layout, occlusion ordering, or composition.These properties depend on many interdependent drawing decisions.
  • Motivation: Standard autoregressive decoding selects by current-position conditional probability, so locally likely choices can yield low-quality complete SVGs.Globally better trajectories may be eliminated early when their initial local probabilities are lower.
  • Approach: CANVAS is a training-free, render-aware framework that navigates generation trajectories in SVG program space rather than following local token decisions alone.It uses existing model capabilities without modifying model parameters.
  • Approach: CANVAS reformulates a complete-SVG objective combining trajectory preference and final-rendering quality into a stroke-wise future-aware navigation rule.Decisions occur at parser-identified complete drawing boundaries.
  • Efficient inference: Finite-sample rollouts estimate finite-horizon future values, while budgets adapt to candidate uncertainty, decision influence, and sampling cost.The framework targets reliable estimates under limited generation and rendering budgets.
  • Results: N = 5 unchanged-backbone samples still leave CANVAS with better global consistency than Best-of-N decoding.The comparison covers geometric relationships, spatial layouts, occlusion ordering, and overall composition.

2. Related Work

Prior Text-to-SVG research has mainly changed optimization, representation, training data, supervision, or inference procedures to handle increasingly complex graphics. CANVAS instead targets global consistency through training-free inference over autoregressive SVG generators.

  • Earlier approaches: Earlier Text-to-SVG methods optimized vector primitives through differentiable rasterization with vision-language or text-to-image diffusion guidance.
  • Autoregressive generation: Recent MLLM-based methods predominantly tokenize SVG code and predict tokens sequentially.Examples explore uniquely decodable path tokens, semantic stroke tokens, and colored graphic generation.
  • Scaling complexity: Methods for complex graphics commonly modify training data, representation, supervision, or the inference loop.Examples include large long-context datasets, domain-specific tokens, critique-and-refine loops, and chain-of-thought supervision.
  • CANVAS: CANVAS is presented as the first training-free inference framework intended to improve global SVG consistency across diverse autoregressive backbones.
  • Sequence-level sampling: Power-distribution sampling sharpens sequence distributions so inference can favor complete-sequence trajectories over only local token probabilities.Prior work applied this principle to reasoning and long-horizon robotic planning.

3. Method

CANVAS replaces local token-only decoding with a training-free, render-aware stroke-wise navigation rule that evaluates future SVG trajectories using model preferences and rendered visual quality. It approximates this objective with finite-horizon rollouts and adaptively allocates limited generation and rendering budgets.

  • Motivation: Standard autoregressive decoding can select locally probable strokes that produce globally inconsistent geometry, occlusion, and composition.The challenge arises because complex SVGs require many interdependent drawing decisions, while token probabilities do not anticipate the final canvas.
  • Complete-SVG target: CANVAS combines power-sharpened complete-trajectory likelihood with rendered visual quality while retaining the frozen base generator as a prior.Its target favors SVG programs that are both plausible to the generator and visually consistent after rendering.
  • Stroke-wise navigation: At parser-complete boundaries, CANVAS treats each renderable SVG element as a stroke and navigates among candidate next strokes rather than individual tokens.A stroke is a complete drawing unit such as a <path>, <circle>, or <rect>; incremental render potentials measure the visual change introduced by each stroke.
  • Stroke-wise navigation: The navigation rule separates each candidate stroke’s immediate model-and-visual contribution from the future value of complete continuations it may induce.The future value marginalizes model preferences and visual effects over suffixes generated after the candidate stroke.
  • Visual adaptive sampling: Finite-sample rollouts make future values estimable, while adaptive sampling concentrates budget according to candidate uncertainty, decision influence, and rollout cost.This design addresses the prohibitive cost of enumerating complete continuations and rendering them for every candidate.
  • Visual adaptive sampling: Under the same budget, the adaptive allocation is theoretically no worse than uniform allocation, with strict improvement unless candidates have identical decision-to-cost ratios.The comparison follows from a Cauchy–Schwarz inequality applied to the first-order decision-error surrogates.

4. Experiments

CANVAS is evaluated on two complementary Text-to-SVG benchmarks using benchmark-standard and additional consistency metrics. Across both benchmarks, it generally improves performance, while ablations examine decision units and rollout horizons under computational constraints.

  • Datasets and evaluation metrics: HeisenVec and IntroSVG provide complementary testbeds for evaluating CANVAS across different SVG generators and long-form generation settings.HeisenVec targets complex, long-context generation, while IntroSVG evaluates rendered-image feedback, visual fidelity, and semantic accuracy in multicolor SVGs.
  • Datasets and evaluation metrics: Six additional metrics assess long-range SVG geometry and composition beyond benchmark-standard measures.These metrics are LCI9×9, CVIU-C, SC, SL, CE, and LaDe.
  • Main results: Across both benchmarks and most reported metrics, CANVAS yields consistent improvements, supporting its effectiveness and generalization ability.The evaluation also compares CANVAS with temperature scaling, beam search, and Best-of-N.
  • Ablation studies: Ablation experiments analyze decision units and rollout horizons as central choices for effective and practical future-aware navigation.These experiments use vHector-8B with the HeisenVec evaluation protocol.
  • Ablation studies: Parser-complete strokes provide a practical decision unit by balancing visual meaning, generation quality, and inference cost.Token-wise decisions are prohibitively expensive, whereas fixed token blocks can be unstable when they end inside coordinates, commands, or XML attributes.
  • Ablation studies: Bounded future rollouts with visual adaptive sampling achieve a favorable computational cost–quality balance.The rollout study compares one-stroke, two-stroke, three-stroke, and valid-EOS horizons under a fixed inference-time budget.

5. Conclusion

CANVAS improves global consistency in long-context Text-to-SVG decoding through training-free, consistency-aware navigation in SVG generation space. It combines learned trajectory preferences with visual evidence from rendered futures within an acceptable inference-time budget.

  • Conclusion: CANVAS is a training-free framework for improving global consistency in long-context Text-to-SVG decoding.Its navigation operates through the SVG generation space.
  • Conclusion: The framework combines a generator’s learned preference over complete trajectories with direct visual evidence from rendered futures without additional training.These designs operate within an acceptable inference-time budget.
  • Conclusion: Extensive experiments demonstrate CANVAS’s effectiveness and generalization ability.

Supplementary Material

The supplementary material formalizes CANVAS’s complete-sequence power prior and its parser-aligned stroke-wise decomposition. It also explains how finite-horizon rollouts and budget-aware estimation approximate each candidate’s future value.

  • A. Complete-sequence power prior: The power-sharpened prior reinforces the base model’s preferences over complete SVG trajectories, with α = 1 preserving relative probabilities and larger α concentrating mass.Increasing α shifts probability toward programs the base model already considers more likely, without directly implying better rendering quality.
  • B. Block and stroke factorization: A deterministic boundary rule partitions each valid SVG into consecutive blocks, preserving the sequence likelihood while changing decision granularity.Token-wise, fixed-length, and parser-detected boundaries are unified; parser-detected boundaries yield the stroke-wise factorization used by CANVAS.
  • B. Parser-aligned decisions: Parser-detected candidates are first-hit blocks that complete a renderable drawing element or the entire SVG, while terminating candidates close remaining tags and emit EOS.This makes each candidate a parser-complete stroke rather than an arbitrary token fragment.
  • C. Stroke-wise future value: Each candidate’s unnormalized score combines its power-sharpened current-block probability with the aggregate likelihood and visual rewards of all valid future continuations.The future contribution sums over suffixes that complete the candidate into a valid, terminated SVG; normalization is required before obtaining probabilities.
  • E. Finite-horizon estimation: Finite-sample rollouts estimate future value by stopping continuations after at most H future strokes and averaging multiple rollouts.The exact untruncated value is recovered in expectation through a tail multiplier, while truncation error depends on the omitted multiplier remaining near one.
  • G. Adaptive allocation: The allocation objective minimizes an approximate squared decision error under a cost constraint for each candidate’s generation-and-rendering rollouts.This formalizes spending a fixed fresh-rollout budget according to candidate-level decision value and cost.

H. Correcting the Leading Bias from Random Normalization

This section analyzes bias introduced when unbiased candidate and rollout estimates are converted into probabilities through random normalization. CANVAS estimates and subtracts the leading ratio-bias term, while retaining assumptions and finite-sample qualifications.

  • H.1 Source of ratio bias: Random normalization makes the ratio of estimated numerator and denominator generally biased, even when both quantities are unbiased before normalization.The leading bias is identified as an O(L−1) term involving denominator variance and numerator–denominator covariance.
  • H.4 Guarantee: The correction is asymptotically justified under support, boundedness, moment, independence, and regularity conditions, but it is not exactly unbiased at finite sample sizes.The leading asymptotic term is removed; the finite-sample remainder remains.
  • H.1 Source of ratio bias: The estimator’s numerator and denominator are formed from candidate importance factors multiplied by fresh-rollout future-value estimates.Particle averages provide the estimated denominator mass and the corresponding statistic-weighted numerator.
  • H.2 Bias correction: CANVAS estimates the leading ratio-bias term from sampled variance and covariance statistics, then subtracts it from the ordinary self-normalized estimate.The corrected estimator is the main-paper Eq. 14 construction.
  • H.3 Unequal allocation: Unequal fresh-rollout counts remain covered by the correction, with candidate contributions scaling as σ_i^2/M_i and vanishing at O(M−1).The analysis explicitly retains adaptive allocation through the minimum rollout count M = min_i M_i.
  • H.5 Consistency check: When the statistic is identically one, the correction is zero and corrected candidate coordinates sum to one in exact arithmetic.Negative or nonfinite coordinates require the numerical fallback specified elsewhere.

I. Handling Failed Rollouts and Numerical Edge Cases

This section keeps failed rollouts and numerical failures inside a well-defined operational target rather than assigning zero weight or discarding them. It specifies smoothing, proposal correction, fallbacks, and the boundary between exact targets and approximations.

  • I.1 Failure smoothing: CANVAS assigns failed rollouts a fixed small positive reward ε, keeping capped outcomes in the probability space with nonzero weight while downweighting failures.The operational outcome space includes both valid complete programs and failure outcomes.
  • I.2 Target effect: Failure smoothing changes the valid-only target by a total-variation distance δ_ε that vanishes as ε → 0.Conditioning the operational target on valid programs exactly recovers the valid-only target.
  • I.3 Modified proposals: When rollout proposals are modified by pruning or rejection, importance weighting corrects sampling frequencies and requires coverage of every positive-target outcome.Using render-filter survivors without correction would estimate a survival-conditioned target instead.
  • I.4 Numerical fallbacks: The implementation uses corrected, uncorrected, or uniform valid-particle distributions when weights are zero, nonfinite, negative, or numerically invalid.These branches keep selection defined while separating fallback behavior from ordinary estimator error.
  • I.5 Stable implementation: Log-space accumulation, log-sum-exp normalization, shared rendering settings, and caching stabilize weights and keep visual scores comparable across candidates.The reported cost still includes generated tokens, renderer calls, and evaluator calls.

J. Complete CANVAS Procedure and Sources of Approximation

The complete CANVAS procedure samples parser-complete candidates, estimates and allocates rollout effort, combines current and future evidence, corrects normalization bias, and commits only valid strokes. Its guarantees are exact at the target and boundary levels but approximate in several inference components.

  • J. Procedure: At each parser-detected stroke boundary, CANVAS samples valid candidates, runs pilot futures, allocates integer fresh-rollout counts, and estimates future values independently.Pilot outcomes determine allocation, but are not reused for final future-value estimates.
  • J. Procedure: The method combines each candidate’s importance factor with its estimated future value, normalizes particle masses, and applies the leading ratio-bias correction before selection.The corrected coordinates form a signed vector used to construct the selection distribution.
  • J. Procedure: Fallback selection chooses a corrected, uncorrected, or uniform valid-particle distribution, and generation terminates only after all SVG/XML tags are closed and EOS is produced.Malformed or unrenderable hypothetical futures affect lookahead through ε but are never committed directly.
  • J. Sources of approximation: The complete-SVG target and boundary-wise marginalization are exact, while finite horizon, restricted support, smoothing, integer allocation, and fallback introduce separate approximations.The visual evaluator defines the optimized reward and does not guarantee agreement with unmeasured human preference.
  • J. Sources of approximation: The allocation result applies only to the stated first-order decision-error surrogate, while downstream metrics also depend on reward fidelity, horizon H, and proposal coverage.This separates the variance-reduction objective from broader evaluation outcomes.
  • J. Experimental configuration: Experiments keep backbone parameters frozen and change only inference, using multiple autoregressive SVG generators and benchmark-specific native tokenization and serialization.The reported configuration includes α = 2 and comparisons across HeisenVec and IntroSVG evaluation backbones.

K. Additional Ablation Studies

This section presents additional ablation studies of CANVAS’s main design choices, using vHector-8B and the HeisenVec evaluation protocol unless otherwise noted.

  • The ablations examine design choices that affect CANVAS’s performance and inference behavior.

K.1. Impact of the Rollout Horizon

CANVAS evaluates a bounded number of future strokes after each candidate, with H = 1 as the default horizon. H1 achieves the strongest overall quality–efficiency balance.

  • K.1. Impact of the Rollout Horizon: H1, H2, and H3 evaluate at most one, two, and three additional parser-complete future strokes, while HEOS continues to valid EOS under operational caps.
  • K.1. Impact of the Rollout Horizon: H1 achieves the best performance on most metrics at about 17.8 seconds per sample, whereas deeper rollouts improve some metrics at substantial runtime cost.

K.2. Evaluation on Decoding and Sampling Alternatives

The ablations compare CANVAS with conventional decoding, power sharpening, visual weighting, coefficient settings, and adaptive allocation choices. Across these comparisons, parser-aligned future-aware decisions provide the strongest overall quality–efficiency tradeoff.

  • K.2. Evaluation on Decoding and Sampling Alternatives: Best-of-N is the strongest conventional alternative, but CANVAS improves eight of its 12 metrics and achieves the best overall result on eight metrics.
  • K.2. Evaluation on Decoding and Sampling Alternatives: Power-only sampling improves eight of 12 metrics over native decoding but degrades FID, HPSv2, SSIM, and LPIPS; rendered visual weighting improves 11 of 12 metrics.
  • K.2. Evaluation on Decoding and Sampling Alternatives: With the remaining components fixed, α = 2 leads seven metrics, including all four global-consistency scores SC, SL, CE, and LaDe.
  • K.2. Evaluation on Decoding and Sampling Alternatives: The rollout-horizon comparison is reported in Table 5, which evaluates horizon choices alongside inference time.
  • K.2. Evaluation on Decoding and Sampling Alternatives: The decoding and sampling alternatives are reported in Table 6, while Tables 7 and 8 cover visual weighting and the power coefficient α.
  • K.2. Evaluation on Decoding and Sampling Alternatives: The complete adaptive estimator combines pilot rollouts, fresh rollouts, and analytic correction, while tested variants lead only on isolated metrics.
  • K.2. Evaluation on Decoding and Sampling Alternatives: Stroke-wise decisions take approximately 17.8 seconds and achieve the best result on most metrics, while token-wise and fixed-length decisions incur substantial additional runtime.

L. Benchmark Datasets and Official Evaluation Metrics

CANVAS is evaluated on the HeisenVec and IntroSVG Text-to-SVG benchmarks using their official protocols and metrics, supplemented by consistency measures and estimator ablations.

  • L. Benchmark Datasets and Official Evaluation Metrics: HeisenVec contains million-scale, richly captioned SVG data spanning diverse visual styles and sequence lengths, with samples extending to 32k tokens.
  • L. Benchmark Datasets and Official Evaluation Metrics: The reported benchmark measures include CLIP-S, FID, HPSv2, SSIM, LPIPS, DINO-S, average token count, and Render Success Rate.
  • L. Benchmark Datasets and Official Evaluation Metrics: The evaluation uses two complementary benchmarks: HeisenVec for complex long-context generation and IntroSVG’s unified held-out set of 1,400 samples.
  • L. Benchmark Datasets and Official Evaluation Metrics: The ablation tables evaluate decision units and adaptive allocation or estimator-reliability components.

M. Definitions and Qualitative Interpretation of the Consistency Metrics

The six complementary metrics assess SVG consistency through visual judging and deterministic image-edge rules, with qualitative examples showing what higher scores mean for structure, spatial relations, efficiency, and layered composition.

  • Evaluation Protocols: SC, SL, CE, and LaDe are VLM-judged, whereas LCI9×9 and CVIU-C are computed deterministically from image edges after a fixed raster-to-edge pipeline.The qualitative figures compare higher and lower outputs within same-prompt, same-sample-ID pairs rather than dataset-wide extrema.
  • Structural Coherence (SC): SC measures whether contours and parts join, close, and remain separate where required, penalizing fragmentation, gaps, fused parts, and unstable construction.Higher SC reflects coherent, recognizable objects rather than simply greater detail.
  • Spatial Logic (SL): SL evaluates whether the correct entities, counts, poses, directions, containment, and relative positions appear in the roles and locations specified by the caption.Higher SL indicates that requested arrangements and component relations are represented correctly.
  • Character Efficiency (CE): CE measures whether visible structure comes from purposeful elements rather than redundant code, repeated marks, visual noise, or local path spam.Higher CE outputs used three graphic elements, whereas lower-scoring counterparts used 152 and 167 elements for unnecessary grids or hatching.
  • LaDe Cross-Layer Consistency: LaDe scores whether ordered pseudo-layers agree in color, style, perspective, scale, occlusion, depth, and foreground/background relations on a 1–5 scale.Higher transferred LaDe outputs align components at compatible scales and establish clearer front-to-back order, while lower outputs contain conflicting overlays.
Loading 2608.30689v1…