PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Conditioning Enables Deep Sequential Reasoning in Flow Matching

Zeyang Zhang, Linxiang Gao, Meiqi Gu, Xingyan Chen, Minrui Luo, Kaifeng Lyu, Tianxing He

arXiv:2610.19628v1cs.CVcs.LG

TL;DR

Although flow-matching steps seem to offer sequential reasoning depth, the paper shows that vanilla sampling can be compressed to constant rounds under regularity conditions. It analyzes self-conditioning and finds that it improves reasoning performance and depth, with additional steps benefiting the self-conditioned sampler.

  • Problem

    It is unclear whether iterative flow-matching sampling truly provides sequential reasoning depth, despite its apparent fit for generating visual solutions.

    Source: §1 · p. 1

  • Method

    The paper uses circuit complexity analysis and controlled experiments to compare vanilla flow matching with self-conditioning, which supplies the model’s estimate of the final image at each step.

    Source: §1 · p. 2

  • Results

    Self-conditioning improves performance across diverse visual-reasoning tasks and benefits from more sampling steps, while vanilla flow matching cannot deepen sequential reasoning through additional steps.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Passing endpoint predictions between steps lets additional sampling steps translate into reasoning depth.

    Source: §6 · p. 10

    The theoretical guarantees rely on regularity assumptions, including a fairly strong Lipschitz restriction; the reported Lipschitz estimates are large and empirical rather than certified.

    Source: §6 · p. 10

Abstract

from arXiv · show

Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading. We prove that as long as the model outputs satisfy certain mild regularity conditions, the trajectory of the vanilla flow-matching sampler can be simulated by a constant number of sequential rounds of parallel model evaluations. Consequently, additional sampling steps cannot increase the expressive power of a constant-depth backbone model under the standard complexity-theoretic model of transformers. We then reveal that self-conditioning, which conditions the model on the previous step's estimate of the final image, plays a previously unrecognized role in removing this bottleneck. Controlled experiments on diverse visual-reasoning tasks show that flow matching with self-conditioning achieves higher accuracy and greater reasoning depth than vanilla flow matching, and its performance improves with more sampling steps. Theoretically, we prove that with self-conditioning, a constant-depth backbone can achieve expressive power equivalent to that of space-bounded Turing machines. Our study suggests a design principle for future visual reasoning research: carry more information between sampling steps, so that additional steps translate into deeper reasoning.

1 INTRODUCTION

The introduction challenges the idea that iterative flow matching automatically provides deeper visual reasoning: vanilla sampling remains constant-depth, while self-conditioning carries endpoint estimates between steps. It presents theoretical and experimental evidence that this changes the reasoning capacity of flow matching.

1 INTRODUCTION

  • Under matched training FLOPs, SELFCONDFM outperforms VANILLAFM by large margins on three synthetic tasks, keeps improving with more steps while VANILLAFM saturates, and improves on five broader tasks.The synthetic tasks are line tracing, long multiplication, and long text rendering.Source: Figure 1 · p. 2, §1 · p. 2
  • With mild regularity conditions, VANILLAFM’s trajectory can be simulated in a constant number of sequential rounds, so additional sampling steps do not increase a constant-depth backbone’s expressive power.The paper attributes this bottleneck to the small O(1/K) state updates of the discretized ODE.Source: §1 · pp. 1–2
  • SELFCONDFM feeds the model an estimate of the final image as an internal scratchpad that can change abruptly between steps, avoiding the constant-step simulation bottleneck.The endpoint estimate records and refines intermediate partial solutions.Source: §1 · p. 2
  • The theory shows that self-conditioning lifts flow matching from constant-depth circuits to space-bounded Turing-machine power, with AC0 deciding P/poly in polynomial steps and PSPACE/poly in at most exponential steps.The introduction also reports controlled experiments on tasks with known circuit complexity.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries review flow matching and circuit complexity, then establish the finite-precision model, regularity conditions, and acceptance conventions used to analyze sampler expressivity.

2.1 BACKGROUND

  • Flow matching learns a velocity field for a probability path from prior noise to data and generates samples by solving the resulting ODE, including with Euler’s method.For conditional generation, the velocity field also receives context.Source: §2.1 · p. 3
  • Circuit complexity represents a model as circuits deciding binary languages, with depth measuring sequential computation and size measuring total work.The background defines NC, AC, TC, P/poly, and PSPACE/poly as the complexity classes used in the paper.Source: §2.1 · p. 3
  • Constant-depth log-precision transformers lie in TC0, with some variants in AC0, motivating whether repeated flow-matching calls extend what such models can decide.The cited background says surpassing these bounds requires more layers or sequential calls such as chain-of-thought.Source: §2.1 · p. 3

2.2 THEORETICAL SETUP AND NOTATION

  • The theoretical setup models VANILLAFM on input w by setting context to w and running K Euler steps from a prior sample.The prior’s support is restricted to [-1, 1]^n, and the analysis uses finite-precision arithmetic.Source: §2.2 · pp. 3–4
  • The setup assumes bounded and Lipschitz model outputs and defines robust language acceptance by a fixed margin on every output coordinate.Precision, state dimension, and step count may grow with problem size.Source: §2.2 · p. 4

3 SELF-CONDITIONING INCREASES EXPRESSIVE POWER

Under mild regularity conditions, vanilla flow matching’s many sampler evaluations can be simulated in O(L) parallel rounds. Self-conditioning removes this constant-depth bottleneck by carrying endpoint predictions between steps, enabling substantially greater expressive power.

3.1 VANILLA FLOW MATCHING IS CONFINED TO CONSTANT DEPTH

  • Under mild regularity conditions, VANILLAFM’s K sequential evaluations can be simulated in O(L) parallel rounds, keeping constant-L models within their own circuit class.For log-precision softmax-attention transformers in TC0, and hard-attention or constant-precision variants in AC0, ODE iteration alone adds no expressive power.Source: §3.1 · pp. 4–5
  • Parallel Picard iteration updates all K trajectory positions together, making propagated error decay factorially so O(L) rounds suffice despite a naive e^O(L) step requirement.Source: §3.1 · p. 4

3.2 SELF-CONDITIONING

  • Self-conditioning feeds the model z_i, an endpoint estimate extrapolated from the previous state and velocity; the model learns to use this prediction from the preceding sampling step.The estimate can change abruptly between steps, unlike the O(1/K) state update, and requires one extra forward pass during training.Source: §3.2 · p. 5

3.3 SELF-CONDITIONING REACHES PSPACE/poly

  • With b=O(log m), n=poly(m), and K ≤ 2^poly(m), SELFCONDFM with an AC0 model accepts PSPACE/poly languages; the theorem also covers P/poly languages under condition (1).Source: §3.3 · p. 6

3.4 EXTENSIONS

  • The results extend to growing Lipschitz constants, ℓ_p norms, and fixed explicit Runge–Kutta solvers; the O(L) bound is tight for majority-circuit simulations.When L=poly(log m), VANILLAFM has an NC upper bound; when L=O(log^k m), it has a TCk upper bound.Source: §3.4 · p. 6

4 EXPERIMENTS

The experiments compare self-conditioned and vanilla flow matching under controlled compute and evaluate both expressivity-validation tasks and broader structured visual-reasoning tasks. They report stronger results for self-conditioning, including greater benefit from additional sampling steps, and analyze trained models’ inference behavior.

4.1 EXPERIMENTAL SETUP

  • Comparisons hold backbone, dataset, optimizer, and noise schedule fixed, match cumulative training FLOPs, and usually evaluate at 50 NFEs.SELFCONDFM uses two training forward passes per example versus one for VANILLAFM; NFE measures sequential computation and equals the sampling-step count.Source: §4.1 · p. 7

4.2 EMPIRICAL VALIDATION OF THE EXPRESSIVITY SEPARATION

  • At matched training compute, SELFCONDFM reached LCP 50 versus VANILLAFM near 8 on line tracing, exceeded 90% multiplication accuracy, and improved text rendering while VANILLAFM stayed near zero.On line tracing, SELFCONDFM reached LCP 50 at 1.10 × 10^18 training FLOPs; both methods were evaluated at 100 NFEs.Source: §4.2 · p. 7

4.3 EVALUATION ON STRUCTURED VISUAL-REASONING TASKS

  • SELFCONDFM raised average accuracy from 65.9% to 76.7% over VANILLAFM across the five structured visual-reasoning tasks.The comparison matches training FLOPs and NFE; SELFCONDFM improves on every task.Source: Table 1 · p. 8, §4.3 · p. 8

4.2 EMPIRICAL VALIDATION OF THE EXPRESSIVITY SEPARATION

  • LCP reached 49.84 for SELFCONDFM at NFE 10, while VANILLAFM reached 7.66 at NFE 6 and stayed below 8.0 through NFE 50.The methods used checkpoints with matched training FLOPs and were evaluated at different NFEs.Source: §4.2 · p. 8, Figure 3 · p. 8

4.3 EVALUATION ON STRUCTURED VISUAL-REASONING TASKS

  • 90.2% average accuracy for one jointly trained SELFCONDFM adapter matched task-specific adapters and exceeded unified DiffThinker (66.1%) and EndoCoT (76.6%).The 90.0% task-specific SELFCONDFM result outperformed DiffThinker (75.8%); EndoCoT is a reference, not a directly comparable baseline.Source: Table 2 · p. 8, §4.3 · p. 9

4.4 ADDITIONAL EMPIRICAL ANALYSES

  • Inference analyses estimate VANILLAFM’s Lipschitz constant near 400 for m ≥ 30; SELFCONDFM attends to endpoint predictions, forms parallel local results, and revises errors in place.The attention share for endpoint predictions is larger early in generation, when the current image is still noisy.Source: §4.4 · p. 9

5 RELATED WORK

Related work spans visual and video generation as reasoning media, theoretical analyses of generative models, and computational accounts of repeated model calls. This paper positions self-conditioning as a computational state, rather than only an empirical generation technique.

Generative visual reasoning

  • Visual reasoning work uses generated images, latent visual states, or learned representations as intermediate workspaces, while video-generation studies examine reasoning across denoising steps.Prior approaches include image-manipulating tool calls (Su et al., 2025a; 2026; Lai et al., 2026) and internal visual intermediates (Zhang et al., 2025; Qin et al., 2025).Source: §5 · p. 9
  • Prior computational analysis of visual generation includes a TC0 bound for optimal denoising diffusion, but requires exact score-function matching (Liu, 2025).The paper notes this assumption rarely holds for practical trained models.Source: §5 · p. 9

Self-conditioning

  • Prior self-conditioning studies emphasize generation quality and efficiency; this paper instead analyzes it as a computational state.Self-conditioning feeds an endpoint prediction into a subsequent denoising step and has been applied across discrete and continuous generation settings (Chen et al., 2022; Strudel et al., 2022; Jabri et al., 2022; Gupta et al., 2024; Stärk et al., 2023).Source: §5 · p. 9

Computational Expressiveness

  • Constant-depth transformers are bounded by AC0 or TC0 in studied settings, while chain-of-thought and repeated sampling calls provide sequential computation beyond single-evaluation bounds.For masked diffusion language models, Feng et al. (2025) prove that distribution matching can use constant sampling steps, whereas sequential reasoning requires steps scaling with sequence length.Source: §5 · p. 10

6 CONCLUSION AND DISCUSSION

The paper argues that self-conditioning lets additional flow-matching steps translate into deeper reasoning, unlike vanilla flow matching. It also identifies assumptions on theoretical guarantees and leaves empirical transfer beyond image-space reasoning open.

6 CONCLUSION AND DISCUSSION

  • Experiments across diverse visual-reasoning tasks show higher performance for self-conditioning and consistent benefits from more sampling steps.The conclusion attributes the change in information carried between steps to endpoint predictions.Source: §6 · p. 10
  • The theoretical guarantees rely on regularity assumptions, including a fairly strong Lipschitz restriction; reported Lipschitz estimates are large and empirical rather than certified.The paper notes that the dependence on the Lipschitz constant is tight up to constant factors and suggests exploring alternative regularity conditions.Source: §6 · p. 10
  • The theoretical analysis is modality-independent, but whether self-conditioning yields similar empirical gains in video, language, or visuomotor settings remains open.The paper focuses its experiments on image-space visual reasoning.Source: §6 · p. 10

AI DISCLOSURE

The disclosure lists generative-AI assistance in dataset creation, proof writing, coding, figure preparation, literature identification, and editing, alongside human review of the resulting work.

AI DISCLOSURE

  • Generative AI was used to generate synthetic datasets, assist with proofs, implement the training codebase, and suggest a proof-technique improvement.The authors state that these tools did not develop theoretical models or conceptual frameworks or design the experiments.Source: AI DISCLOSURE · p. 10
  • AI tools also assisted with figures, literature identification, reference formatting, and paragraph-level writing and editing.The disclosure says AI was not used to formulate mathematical claims or interpret results.Source: AI DISCLOSURE · p. 10
  • Authors reviewed AI-assisted work, including proof writing, language editing, the experimental codebase, datasets, figures, literature, and reference formatting.The authors state that they take responsibility for the final content and artifacts.Source: AI DISCLOSURE · p. 10

REPRODUCIBILITY STATEMENT

The appendices document the proof strategy and conventions behind the expressivity results. For VANILLAFM, a Picard iteration approximates the sampler in parallel, with its error controlled across rounds.

REPRODUCIBILITY STATEMENT

  • The reproducibility statement directs readers to appendices for proofs, task definitions, implementation and training details, and evaluation protocols.It also identifies the sections covering multiple-seed results and the NFE-scaling study.Source: REPRODUCIBILITY STATEMENT · p. 11

A PROOFS OF MAIN EXPRESSIVITY THEOREMS

  • Appendix A.2 proves Theorem 1 and Appendix A.3 proves Theorem 2, after Appendix A.1 fixes notation and encoding and collects preliminaries; Appendix B separately proves the §3.4 extension results.Source: A PROOFS OF MAIN EXPRESSIVITY THEOREMS · p. 18

A.1 CONVENTIONS AND PRELIMINARY FACTS

  • The setup fixes non-uniform circuit families, scalar encodings, finite-precision rounding, and a hardwired initialization to represent sampler runs as circuits.The table bound places arbitrary maps on constantly many O(log m)-bit scalars in depth two, with polynomial size.Source: §A.1 · p. 18

A.2 EXPRESSIVENESS UPPER BOUND FOR VANILLAFM

  • The proof overview replaces the K sequential Euler steps with parallel Picard rounds and approximates the sampler’s endpoint within constant error.The appendix separately establishes the error recurrence, the parallel simulation, and its circuit implementation.Source: §A.2 · p. 19

A.2.1 THE ERROR RECURRENCE

  • The error recurrence makes propagated initial error decay factorially across rounds, while additive errors accumulate to order eL.The bound uses a number of rounds C = ⌈eL + 4B⌉.Source: §A.2.1 · p. 19

A.2.2 PROOF OF LEMMA 1

  • For VANILLAFM, Picard iteration recomputes all K trajectory positions in parallel and reaches constant endpoint accuracy in O(L) rounds.Each round uses K parallel model calls and normalized prefix sums; the endpoint approximation preserves acceptance with margin.Source: §A.2.2 · p. 20

K 11: end function

The appendix shows how VANILLAFM’s K sequential updates can be approximated in O(L) parallel rounds, yielding a constant-depth upper bound when L is constant. It also gives the self-conditioned sampler a local-machine simulation, while noting that the upper-bound argument relies on constant-accuracy endpoint approximation.

K 11: end function

  • O(L) parallel rounds suffice to approximate VANILLAFM’s endpoint regardless of K, capping the sequential depth its sampler can exploit.The construction uses K parallel model calls per round; for constant L it yields constant-depth simulation.Source: K 11: end function · p. 21
  • The collapse argument requires only constant endpoint accuracy for acceptance with margin; exponentially small error can make even a Lipschitz ODE hard to solve.The cited ODE construction encodes a PSPACE-complete answer at precision 2^-poly(m).Source: K 11: end function · p. 22

A.2.3 APPROXIMATE NORMALIZED PREFIX SUMS IN AC0

  • Exact prefix sums lie beyond AC0, so the proof uses quantized summands and approximate counting to estimate normalized prefix sums.The approximation is sufficient for the Picard iteration’s constant-error requirement.Source: §A.2.3 · p. 22

A.2.4 PROOF OF THEOREM 1

  • Theorem 1’s circuit proof handles bounded K by unrolling the sampler and larger K with approximate prefix sums inside Picard iteration.The endpoint stays within the constant acceptance margin, and the resulting circuit has constant depth over model calls.Source: §A.2.4 · pp. 22, 24

A.3 EXPRESSIVENESS LOWER BOUND FOR SELFCONDFM

  • Self-conditioning simulates a space-bounded machine by storing its tape in endpoint predictions and applying a local cell-update rule at each sampling step.The tape freezes on its verdict, allowing the final state’s sign to encode acceptance or rejection.Source: §A.3 · p. 24

A.3.1 LOCAL SIMULATION OF SPACE-BOUNDED MACHINES

  • The machine is flattened into a cellular automaton whose cells update from themselves and their neighbors, then settle into an all-accept or all-reject tape.The construction uses polynomial tape length and a horizon at most 2^poly(m); for P/poly languages, the horizon is polynomial.Source: §A.3.1 · p. 24

A.3.2 PROOF OF THEOREM 2

The self-conditioned construction encodes a machine’s tape in endpoint predictions and advances its local update rule across sampling steps, then keeps the final verdict in the sampler’s output. The extensions show upper and lower bounds also hold under fixed ℓp Lipschitz conditions and examine growing Lipschitz constants for VANILLAFM.

A.3.2 PROOF OF THEOREM 2

  • Endpoint predictions encode tape bits in separated value bands, letting the model distinguish symbols while tolerating sampler perturbations.Each coordinate’s local update reads the input and the corresponding tape cell and its neighbors.Source: §A.3.2 · p. 25
  • With self-conditioning, each sampling step advances a space-bounded machine’s tape, and the verdict tape keeps the final output’s sign aligned with acceptance or rejection.The construction uses K = 16H steps and establishes acceptance with margin 1; the PSPACE/poly case permits exponentially many steps.Source: §A.3.2 · p. 26

B PROOFS OF THEORETICAL EXTENSIONS

  • The appendices extend the bounds to growing Lipschitz constants, fixed ℓp norms, explicit Runge–Kutta samplers, and majority-circuit lower bounds.The Runge–Kutta and majority-circuit extensions are identified in the appendix overview.Source: B PROOFS OF THEORETICAL EXTENSIONS · p. 27

B.1 THE GROWING LIPSCHITZ CONSTANT

  • For VANILLAFM with growing L(m), the parallel simulation yields majority circuits of depth O(L(m)); logarithmic-power L gives TCk, while polylogarithmic L gives NC.The result assumes polynomial state dimension and step count together with the stated precision bound.Source: §B.1 · p. 27
  • Both the VANILLAFM upper bound and self-conditioned lower bound extend to any fixed ℓp norm, with the latter using a Lipschitz constant that may depend on p.The self-conditioned construction remains unchanged; the P/poly case retains the stated precision condition.Source: §B.1 · pp. 27–28

B.3 RUNGE–KUTTA SAMPLERS

The appendix extends vanilla flow-matching sampler bounds to fixed explicit Runge–Kutta methods with constant stage counts, then shows how vanilla flow matching can match majority-circuit depth. The O(L) simulation rounds are not generally reducible without a consequence considered unlikely.

B.3 RUNGE–KUTTA SAMPLERS

  • For fixed explicit Runge–Kutta methods with constant stage count q and nodes in [0, 1], the sampler bounds persist with qK parallel calls per round.The precision condition depends on the method, and the O(L)-round simulation retains the theorem and corollary bounds.Source: §B.3 · pp. 29, 31
  • A parallel Picard iteration simulates the Runge–Kutta trajectory as a fixed-point system, with qK positions and error bounded by the same recurrence used for Euler.Each round evaluates the model in parallel at all stages; the analysis scales L by a constant depending on the tableau.Source: §B.3 · p. 29

B.4 MATCHING THE CIRCUIT DEPTH

  • The vanilla sampler resets its state, writes one target-circuit layer per step, then amplifies the verdict to achieve acceptance margin 1.Each layer's gate values persist in state coordinates after being written.Source: §B.4 · pp. 32–33
  • A TC0-computed vanilla-flow-matching model can accept languages decided by depth-d majority circuits with K = Θ(d) sampling steps.The construction uses polynomial n and b, and accepts with margin 1.Source: §B.4 · p. 32
  • Improving the O(L)-round simulation to o(L) rounds would collapse the majority-circuit depth hierarchy, which is widely conjectured to be infinite.The appendix presents this collapse as unlikely.Source: §B.4 · p. 33

C FORMALIZATION AND CIRCUIT COMPLEXITY OF THE TASKS IN §4.2

The appendix formalizes three image-output tasks and establishes their circuit complexity through associated symbolic functions. Line tracing is NC1-complete, while long multiplication and long text rendering are TC0-complete under the stated reductions.

C FORMALIZATION AND CIRCUIT COMPLEXITY OF THE TASKS IN §4.2

  • Line tracing is NC1-complete, while long multiplication and long text rendering are TC0-complete under AC0 Turing reductions.These classifications match the stated separation barriers for AC0 and TC0 backbones; the circuit classes considered are non-uniform.Source: C FORMALIZATION AND CIRCUIT COMPLEXITY… · p. 34

C.1 TASK DESCRIPTIONS

  • The tasks render five-polyline routing, complete long-multiplication worksheets, and monospaced text with greedy wrapping and paragraph spacing.Text rendering takes a word sequence with paragraph boundaries as input.Source: §C.1 · p. 34

C.2 PRELIMINARIES FOR THE COMPLEXITY ANALYSIS

  • Completeness is defined through output bits of each symbolic function, with image-space tasks connected to those functions by bidirectional AC0 Turing reductions.The complexity propositions describe asymptotic task families, while experiments instantiate them over finite problem sizes.Source: §C.2 · pp. 34–35
  • The experiments use finite instances of task families whose circuit-complexity classifications concern computation as problem size m grows.The appendix gives examples of the finite sizes and operand lengths used in experiments.Source: §C.2 · p. 35

C.3 LINE TRACING

  • Line tracing outputs the sequence of prefix permutations for the five polylines and is NC1-complete under AC0 Turing reductions.Hardness follows from the word problem for the non-solvable group A5.Source: §C.3 · p. 35

C.4 LONG MULTIPLICATION

  • Long multiplication outputs shifted partial products, successive partial sums, carries, and the final product, and is TC0-complete under AC0 Turing reductions.The proof establishes membership through parallel arithmetic and hardness by reducing exact counting to the worksheet function.Source: §C.4 · pp. 35–36

C.5 LONG TEXT RENDERING

  • For every fixed W ≥ 3 and fixed nontrivial monospaced raster font over an alphabet of at least two characters, long text rendering is TC0-complete.Its wrapping function tracks occupied width and computes word positions using prefix sums.Source: §C.5 · pp. 36–37

D.2 SETUP AND EVALUATION FOR THE TASKS IN §4.2

The evaluation uses matched task-specific procedures to compare vanilla flow matching and self-conditioned flow matching. On long text rendering, the reported character error rates favor self-conditioning at matched training compute.

Overall setup

  • Training FLOPs include VAE encoding and SELFCONDFM's second training pass and longer token sequence, so its extra computation is counted.Within each task, the methods share data, optimizer, noise schedule, and evaluation procedure.Source: §D.2 · p. 39

Line tracing

  • Line tracing uses 106 training instances with m = 50 and evaluates 20 held-out instances at 100 NFE using longest correct prefix.An instance's score is the longest stage prefix for which all five segments match the reference rendering.Source: §D.2 · p. 39

Long multiplication

  • Long multiplication trains on 2 × 10^5 problems with five-to-seven-digit operands and evaluates 100 test instances at 50 NFE using a worksheet judge.A sample is correct only if the final product and every intermediate row agree with the reference worksheet.Source: §D.2 · p. 39

Long text rendering

  • Long text rendering uses 3 × 10^5 TinyStories conditions and evaluates 100 held-out instances at 50 NFE by exact parsed-text match.The parser is prompted to preserve line breaks and whitespace.Source: §D.2 · p. 40
  • CER ends at 0.005 for SELFCONDFM; VANILLAFM is 0.046 at the matched budget of 4.70 × 10^19 FLOPs.The figure plots CER against cumulative training FLOPs; SELFCONDFM keeps improving while VANILLAFM plateaus.Source: Figure 5 · p. 40, §D.2 · p. 40

D.2.1 MULTI-SEED EVALUATION

Multi-seed evaluations show SELFCONDFM outperforming VANILLAFM on line tracing and long multiplication, while selected examples illustrate its successes on line tracing, multiplication, and text rendering. More sampling steps improve SELFCONDFM’s line-tracing results, and the appendix describes setups for five structured visual-reasoning tasks.

D.2.1 MULTI-SEED EVALUATION

  • 50.0±0.0 mean LCP vs 5.6±2.7, and 90.0±5.3% vs 35.0±8.9% accuracy: SELFCONDFM outperforms VANILLAFM on both replicated tasks at matched budgets.The line-tracing comparison uses matched training FLOPs; the long-multiplication comparison uses a smaller matched budget.Source: §D.2.1 · p. 41

D.2.2 GENERATED EXAMPLES

  • In held-out examples, SELFCONDFM preserves line colors across 50 stages, computes both products correctly, and renders text with zero CER; VANILLAFM fails each.The long-text examples show VANILLAFM CERs of 0.0106 and 0.0410.Source: Figure 8 · p. 43, Figure 9 · p. 44, Figure 10 · p. 45

D.3 NFE SCALING

  • At NFE 1, both methods have zero mean LCP; SELFCONDFM reaches 49.84 at 8.29 × 10^11 cumulative inference FLOPs, while VANILLAFM reaches 7.66 at 2.74 × 10^11 FLOPs and remains below 8.0 through NFE 50.Source: Figure 11 · p. 46

D.4 SETUP AND RESULTS FOR STRUCTURED VISUAL-REASONING TASKS

  • Across five tasks, controlled runs use matched FLOPs and 50 NFE; large-scale runs use task-specific or unified adapters, each trained for five epochs.The controlled comparisons use the shared task configuration; large-scale unified training combines VSP-Super, Maze, TSP, and Sudoku data.Source: §D.4 · p. 46

D.4.1 JIGSAW RESULTS

The appendix reports SELFCONDFM’s strongest average Jigsaw accuracy against the cited baselines and provides selected examples across five structured reasoning tasks. It also reports empirical Lipschitz estimates and attention allocation during sampling.

D.4.1 JIGSAW RESULTS

  • 96.0% average accuracy vs 92.3% for DiffThinker: SELFCONDFM attains the best Jigsaw result, with the largest gain at the hardest level.The table reports accuracy at difficulty levels 2 to 4 and defines average as their unweighted mean.Source: §D.4.1 · p. 46, Table 3 · p. 47

D.4.2 GENERATED EXAMPLES

  • Selected held-out examples illustrate SELFCONDFM succeeding where VANILLAFM fails across VSP-Super, Maze, TSP, Sudoku, and Jigsaw.The examples show disconnected or incomplete routes, suboptimal tours, Sudoku constraint violations, and incorrect tile permutations for VANILLAFM.Source: §D.4.2 · p. 47, Figure 12 · p. 48, Figure 13 · p. 49

D.5.1 EMPIRICAL LIPSCHITZ ESTIMATION

  • 580 maximum empirical Lipschitz estimate; estimates level around 400 for m ≥ 30 and show no continued growth over tested lengths, but are not certified bounds.The estimates empirically support the bounded-Lipschitz regime assumed in the theory, rather than establishing an upper bound.Source: §D.5.1 · p. 53

D.5.2 ATTENTION ALLOCATION

  • Endpoint prediction receives 20.0%–31.3% of attention throughout, with its share declining nearly monotonically and peaking early when noise dominates.Source: §D.5.2 · p. 53, Figure 18 · p. 54

D.5.3 PARALLEL AND REVISABLE REASONING

Both methods produce multiple correct local products in parallel, while SELFCONDFM more accurately revises outputs after a mid-trajectory change in the problem condition.

Parallel local computation

  • Above 98% partial-product accuracy at every NFE from 1 to 50 for both methods; even one evaluation yields multiple correct local products in parallel.Partial products are scored against the reference worksheet at each NFE.Source: Figure 19 · p. 54, §D.5.3 · p. 54

In-place error correction

  • Around 90% matching accuracy for SELFCONDFM on first-half swaps, versus VANILLAFM’s 65% at k = 1 and zero by k = 32; both revise in place, unlike autoregressive reasoning.SELFCONDFM’s matching accuracy falls to zero only once k exceeds 35.Source: Figure 20 · p. 55, §D.5.3 · p. 55

Related papers

Loading 2610.19628v1…