PaperlayerView exact revision on arXiv

Source-linked AI summary

Match the Canvas, Not Just the Answer: Predictability-Aware SFT for Diffusion Language Models

Miao Li, Hanyang Jiang, Hengyu Fu, Baihe Huang, Yating Liu, Yuhang Cai, Changkun Guan, Zhongzhu Chen, Jiecheng Lu, Michael A. Boateng, Pascal Van Hentenryck

arXiv:2610.18001v1cs.LGcs.AIcs.CL

TL;DR

Masked DLM SFT must choose which reference tokens are visible and how much canvas is allocated, choices that random masking makes independently of inference. PACS selects canvases and supervision states using model predictability along a reference-conditioned reveal process. Across reasoning and code-generation benchmarks, it achieves the highest average accuracy among compared SFT methods, with gains up to 12.84 percentage points over vanilla SFT.

  • Problem

    Masked DLM references fix target tokens but not the canvas context from which they should be predicted, while vanilla SFT typically masks randomly rather than following confidence-guided decoding.

    Source: §1 · pp. 1–2

  • Method

    PACS uses commitment-level reference predictability to select each example’s canvas and supervises the corresponding pre-commitment states along a reference-conditioned reveal trajectory.

    Source: §1 · p. 3, §5 · p. 6

  • Results

    Across four reasoning and code-generation benchmarks, PACS achieves the highest average accuracy among compared SFT methods on both models at every evaluated generation budget, with gains up to 12.84 percentage points over vanilla SFT.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Visible reference content and canvas extent can change the prediction problem induced by the same fixed reference.

    Source: §1 · p. 3

Abstract

from arXiv · show

Diffusion language models (DLMs) enable flexible parallel generation by resolving tokens according to the model's evolving predictability rather than a fixed left-to-right order. This flexibility, however, introduces training choices absent from standard autoregressive teacher forcing: which reference tokens are visible when a target is supervised and how much generation space is allocated. Standard supervised fine-tuning (SFT) typically makes these choices through random masking, independently of inference. Our analysis shows how this training-inference mismatch can misdirect supervision. Random masking can conceal prediction errors or penalize uncertainty that decoding would otherwise resolve, while the attention-visible input length can change which response formulations the model favors, even when the underlying solution is unchanged. Motivated by these findings, we propose Predictability-Aware Canvas Selection and Supervision (PACS), an SFT method. PACS uses the model's own predictability to select each example's attention-visible input length and supervises reference tokens immediately before their confidence-guided revelation. Across reasoning and code-generation benchmarks, PACS consistently achieves the highest mean accuracy among the compared SFT methods, with gains of up to 12.84 percentage points over vanilla SFT on individual benchmarks. Code will be released upon publication.

1 INTRODUCTION

DLM training must choose both which reference tokens are visible and how much attention-visible canvas is allocated, choices that standard autoregressive teacher forcing does not face. PACS uses model predictability to select canvases and supervise reference tokens before their confidence-guided revelation, achieving the highest average accuracy among compared SFT methods.

1 INTRODUCTION

  • A masked DLM reference fixes target tokens but not the canvas length, visible reference tokens, or mask pattern that define each prediction problem.Unlike autoregressive teacher forcing, the context for predicting each target is not fixed by preceding reference tokens.Source: §1 · pp. 1–2
  • Canvas choices determine both which reference tokens are visible when targets are supervised and how many input positions the model can attend to.The same fixed reference can therefore induce different prediction problems.Source: §1 · pp. 2–3
  • Random masking can omit context that decoding would reveal first or expose later content prematurely, respectively penalizing missing-context uncertainty or concealing selection-time errors.Figure 1 illustrates the distinction between visible reference content and commitment-time disagreement.Source: §1 · p. 2, Figure 1 · p. 2
  • Changing canvas extent can shift reference predictability and favor different response formulations even when the underlying solution is fixed.Longer canvases produce longer realized responses, and extent shifts reference loss particularly in later response blocks.Source: §1 · p. 3, Figure 2 · p. 3
  • PACS selects the canvas using commitment-level reference predictability and supervises the corresponding pre-commitment states along a reference-conditioned reveal trajectory.The reference itself is unchanged.Source: §1 · p. 3
  • Across four reasoning and code-generation benchmarks, PACS achieves the highest average accuracy among compared SFT methods on both models at every evaluated generation budget, with gains up to 12.84 percentage points over vanilla SFT.The models are LLaDA-8B-Instruct and Dream-7B-Instruct.Source: §1 · p. 3

2 PRELIMINARIES

Masked DLMs generate within a preallocated canvas whose positions are progressively revealed, while confidence-guided decoding determines which eligible predictions are committed. Vanilla SFT instead samples masks randomly over the non-prompt positions, including EOS padding.

2.1 MASKED DIFFUSION LANGUAGE MODELS AND CANVASES

  • The canvas is a prompt followed by a preallocated response region initialized with masks; full-canvas decoding makes all masked positions eligible, while semi-AR decoding restricts eligibility to the first unfinished block.Future semi-AR blocks remain attention-visible as masks.Source: §2.1 · p. 3, §2.2 · p. 4
  • A clean canvas contains the prompt and answer including its terminating EOS, followed by EOS copies filling any remaining attention-visible positions.The answer-ending EOS marks the reference boundary; subsequent EOS copies serve as padding.Source: §2.1 · pp. 3–4

2.2 CONFIDENCE-GUIDED DECODING

  • At each eligible position, confidence is the probability of the model’s most likely token; positions meeting threshold τ are revealed together, then confidence is recomputed.The commitment set contains eligible positions whose confidence is at least τ.Source: §2.2 · p. 4

2.3 TRAINING CANVASES IN SUPERVISED FINE-TUNING

  • Vanilla SFT independently masks each non-prompt clean token using a uniformly sampled masking probability, including EOS-padding positions, while leaving the prompt unchanged.Both masked and unmasked padding positions remain attention-visible, and masked padding positions are supervised to reconstruct EOS.Source: §2.3 · p. 4

3 RELATED WORK

Related work adapts DLM supervision through decoding structure, predictability, or response compatibility. PACS instead uses confidence-guided reference trajectories and extends compatibility to the attention-visible canvas extent.

3 RELATED WORK

  • PACS shares prior work’s use of predictability to shape supervision but recomputes confidence after each reference revelation to determine which positions to supervise next.Prior approaches include block conditioning, teacher-derived difficulty [Chen et al., 2026a], and model-estimated difficulty.Source: §3 · p. 4
  • Unlike on-policy trajectories that retain model-generated tokens, fixed-reference SFT changes later states through teacher forcing; PACS retains reference supervision while letting predictability guide reveals.Recent trajectory-aware methods include Wang et al. (2026b), Shao et al. (2026), and Ren et al. (2026).Source: §3 · pp. 4–5
  • On GSM8K with LLaDA-8B-Instruct, hiding preceding-step context makes 37.3% of initially correct predictions wrong, while leaking a future token makes 51.2% of initially wrong predictions correct.These context changes can respectively introduce uncertainty or weaken corrective supervision.Source: Figure 3 · p. 5
  • PACS extends response-compatibility approaches to canvas extent, adapting prediction conditions while preserving the reference answer.Related approaches select responses, rewrite references, or selectively use correct model-generated answers.Source: §3 · p. 5

4 HOW CANVAS SHAPES REFERENCE SUPERVISION

The section shows that both the reference content visible at supervision and canvas extent can change predictions for the same reference. These mismatches can misdirect supervision relative to the model’s errors when it commits to revealing a token.

4.1 THE CANVAS CAN CREATE OR CONCEAL COMMITMENT DISAGREEMENT

  • 37.3% of initially correct commitment predictions become wrong when the reference tokens revealed in the preceding step are hidden, and mean reference CE increases by 0.84 nats.The added supervision penalizes uncertainty from missing context rather than an error at the target’s original selection step.Source: §4.1 · p. 6
  • Revealing a future reference token early makes 51.2% of initially wrong commitment predictions correct and reduces mean reference CE by 1.08 nats, weakening corrective supervision for errors that remain.The analysis constructs reference-conditioned trajectories by revealing reference tokens in confidence order, then compares target predictions before and after changing surrounding context.Source: §4.1 · pp. 5–6

4.2 CANVAS EXTENT CHANGES REFERENCE PREDICTABILITY

  • Positive mean paired ∆CE appears at both shorter and longer canvas lengths, with excess CE particularly pronounced in later response blocks for K = 256 and K = 512.The references were generated at K0 = 384 and reevaluated at other lengths along each candidate’s own reference-conditioned trajectory; the added loss may reflect differences in response formulation rather than reasoning errors.Source: §4.2 · p. 6

5 PREDICTABILITY-AWARE CANVAS SELECTION AND SUPERVISION

PACS selects an attention-visible canvas extent using commitment-level predictability, then fine-tunes on the corresponding pre-commitment prediction states. Its selection and supervision use the same conditional prediction tasks.

A Average CE change

  • Figure 4A reports mean ∆CE across generation lengths, evaluated along each candidate’s own reference-conditioned trajectory.The change is relative to evaluation at source generation length K0 = 384.Source: Figure 4 · p. 7

B Blockwise CE change

5.1 COMMITMENT-BASED CANVAS SELECTION

  • PACS selects each fixed reference’s attention-visible canvas extent using commitment-level predictability, rather than inheriting it from a padding convention.Candidate lengths retain the same prompt and reference while varying the number of attention-visible positions beyond the reference; their reference-conditioned trajectories are compared using CEtr on the same scored reference span.Source: B Blockwise CE change · p. 7

5.2 COMMITMENT-LEVEL SUPERVISED FINE-TUNING

  • PACS applies SFT to the positions selected for commitment on the selected trajectory’s pre-commitment canvases, supervising each reference position in R once.The frozen model determines which positions are selected at each step, while the reference supplies the values revealed before the next prediction; this changes the context in which each token is taught rather than removing difficult targets.Source: §5.2 · pp. 7–8

5 PREDICTABILITY-AWARE CANVAS SELECTION AND SUPERVISION

  • Canvas selection and commitment-level SFT assess and train on the same conditional prediction tasks.The selected trajectory supplies the conditioning states used for fine-tuning.Source: §5.2 · p. 8

6 EXPERIMENTS

PACS performs best overall across the reported comparisons, while component and canvas-rule analyses show gains from both canvas selection and confidence-guided supervision. A compact search family captures most of the observed validation benefit.

6.1 EXPERIMENTAL SETUP

  • The evaluation covers LLaDA-8B-Instruct and Dream-7B-Instruct on GSM8K, MATH500, MBPP, and HumanEval at three generation budgets.SFT methods share training data from GSM8K, MATH, and OpenCoder; comparisons include vanilla random-mask SFT, semi-AR block SFT, LIFT, TRIMS, and PACS.Source: §6.1 · p. 8
  • PACS beats vanilla SFT in 22 of 24 model–benchmark–length settings and ties in two, and achieves the highest accuracy among compared methods in 19 settings.At K_eval = 256, PACS improves MATH500 by 2.00 and 5.00 percentage points and HumanEval by 9.15 and 6.71 points on LLaDA and Dream, respectively.Source: §6.1 · p. 8, Table 1 · p. 9, Table 2 · p. 9

6.2 COMPONENT ANALYSIS

  • Canvas selection improves both reveal-order variants, and confidence-guided ordering outperforms schedule-matched random ordering under either canvas rule.Combining both components achieves the highest accuracy on all four benchmarks.Source: §6.2 · p. 9, Table 3 · p. 9

6.3 CANVAS CHOICE AND REFERENCE PREDICTABILITY

  • With reference responses unchanged, PACS yields lower reference CE and higher test accuracy than batch-max, dataset-max, and random visible-length rules.Figure 5 compares target CE and post-SFT test accuracy on MATH500 and HumanEval at K_eval = 256.Source: §6.3 · p. 9, Figure 5 · p. 10

6.4 ABLATION ON CANVAS SEARCH EXTENT

  • Expanding the main candidate family Λ4 to Λ5 adds only 0.1 percentage points on Math and 0.2 points on Code in validation accuracy.The authors report that Λ4 captures most of the observed benefit while limiting per-example search.Source: §6.4 · p. 10, Figure 6 · p. 10

7 CONCLUSION

The conclusion frames training-canvas construction as integral to masked-DLM fine-tuning because visible reference content and canvas extent shape the learning problem even for a fixed answer. PACS uses commitment-level reference predictability to couple canvas selection with pre-commitment supervision.

7 CONCLUSION

  • The conclusion identifies revealed reference content and attention-visible extent as factors shaping the learning problem when the answer is fixed.It characterizes training-canvas construction as integral to masked-DLM fine-tuning.Source: §7 · p. 10
  • PACS couples canvas selection and reference-consistent pre-commitment supervision through commitment-level reference predictability.The conclusion describes this as bringing model decoding behavior into construction of its training tasks.Source: §7 · p. 10
  • Experiments across mathematical reasoning, code generation, and multiple generation lengths support a decoding-aware perspective on supervision.The authors identify future research on predictability-aware canvas construction across block-diffusion and hybrid diffusion–AR architectures.Source: §7 · p. 10

A.1 TRAINING CONFIGURATIONS

The appendix summarizes PACS training configurations, decoding behavior, and prompt setup. Its decoding configuration uses confidence-threshold unmasking and applies the same selection rule to reference-conditioned training trajectories.

A.1 TRAINING CONFIGURATIONS

  • The appendix points to Table A.1 for PACS hyperparameters and identifies the shared GSM8K, MATH, and OpenCoder training corpus and four-candidate canvas family with B = 32.Decoding settings and prompt formats are provided in Sections A.2 and A.3.Source: §A.1 · p. 13, Table A.1 · p. 13

A.2 DECODE CONFIGURATION

  • At temperature zero, decoding commits active-block masked positions with top-1 probability at least 0.9, or the single highest-confidence position if none qualifies.The same confidence-guided rule constructs reference-conditioned training trajectories: the frozen model selects positions and the reference supplies token values.Source: §A.2 · p. 13

A.3 GENERATION PROMPTS

  • All four benchmarks use a single user turn without added task instructions or few-shot exemplars.The appendix also labels a LLaDA-8B-Instruct source for HumanEval.Source: §A.3 · p. 14, §A.3.1 · p. 14

A.4 SELECTED-BIN DISTRIBUTIONS

PACS selects canvas extent heterogeneously across examples rather than collapsing to the shortest canvas or one fixed offset. The appendix also points to complete results across three generation budgets.

A.4 SELECTED-BIN DISTRIBUTIONS

  • Under the main family Λ4, PACS selects Lref for 53.0% of Math and 52.7% of OpenCoder examples; the remaining 47.0% and 47.3% span Lmin through Lmin + 2B.The selected bins show example-specific preferences for canvases extending beyond the exact reference length.Source: §A.4 · p. 15, Figure A.1 · p. 15
  • The newly added Lmin + 3B candidate in Λ5 is selected for 10.3% of Math and 12.3% of OpenCoder examples.The authors describe this as consistent with diminishing returns from searching substantially beyond Λ4.Source: §A.4 · pp. 15–16

A.5 COMPLETE RESULTS ACROSS GENERATION LENGTHS

  • Tables A.2 and A.3 report complete comparisons at K_eval values 128, 256, and 512; Δ is PACS’s relative percentage gain over the corresponding initial checkpoint.The supplied captions identify these as the complete results across generation lengths for LLaDA-8B-Instruct and Dream-7B-Instruct.Source: §A.5 · p. 16, Table A.2 · p. 16, Table A.3 · p. 17

B.1 ROBUSTNESS TO TRAINING DATASETS

The appendix tests PACS across alternative training data, reproduced baselines, matched compute budgets, and other supervision rules. PACS-selected canvases remain effective across these settings, including comparisons at matched budgets.

B.1 ROBUSTNESS TO TRAINING DATASETS

  • On Math14k and KodCode, PACS improves over vanilla SFT on all four benchmarks and achieves the highest accuracy on MATH500 and HumanEval.Math14k combines GSM8K and AQuA examples with model-generated rationales; KodCode is a synthetic coding dataset with verified solutions and unit tests.Source: Table B.1 · p. 18, §B.1 · p. 18

B.2 SANITY REPRODUCTION OF PRIOR BASELINES

  • LIFT and TRIMS reproductions fall within 0.61 percentage points of reported scores across four comparisons, providing a sanity check for the baseline implementations.Each reproduction follows the corresponding paper’s training and evaluation configuration.Source: Table B.2 · p. 18, §B.2 · p. 18

B.3 COMPUTATIONAL COST

  • At matched post-training budgets, PACS leads both benchmarks at 12 GPU-hours, beating the strongest baseline by 0.40 percentage points on MATH500 and 2.44 points on HumanEval.Budgets include method-specific preprocessing and optimization but exclude shared corpus construction.Source: Table B.4 · p. 19, §B.3 · p. 19

B.4 CANVAS SELECTION UNDER OTHER SUPERVISION RULES

  • PACS-selected canvases improve or match batch-max accuracy on all four benchmarks under each supervision rule; HumanEval gains are 0.61, 1.83, and 6.66 points for random masking, semi-AR block SFT, and confidence-guided revelation.Under semi-AR block SFT, selected canvases also outperform batch-max, dataset-max, and random lengths on MATH500 and HumanEval.Source: Table B.5 · p. 20, §B.4 · p. 20, Figure B.1 · p. 20

C DETAILS OF THE CANVAS DIAGNOSTICS

The diagnostics examine how revealed reference content and canvas extent affect prediction states, then use a schedule-matched random-order control to isolate confidence-based position selection. They also specify the PACS canvas-selection and fine-tuning procedure.

C DETAILS OF THE CANVAS DIAGNOSTICS

C.1 INTERVENTIONS ON REVEALED CONTENT

  • The revealed-content interventions withhold previously revealed reference tokens for targets already eligible at the prior step, or reveal one next-step reference token for initially incorrect targets.The two interventions use different target cohorts; the reported quantities are changes in reference agreement and paired cross-entropy.Source: §C.1 · p. 21

C.2 INTERVENTIONS ON CANVAS EXTENT

  • Changing attention-visible canvas extent changes pre-commitment cross-entropy, with position-wise changes increasing toward later response positions.The fixed-trajectory control keeps the reveal schedule unchanged, whereas candidate-trajectory comparisons recompute predictions and subsequent selections for each length.Source: §C.2 · p. 21

C.3 SCHEDULE-MATCHED RANDOM-ORDER CONTROL

  • The random-order control matches PACS’s per-step reveal counts and reference-consistent intermediate canvases, changing only whether eligible positions are selected by confidence or uniformly at random.This schedule-matched design isolates confidence-based ordering from differences in reveal rate, trajectory length, or visible reference context.Source: §C.3 · pp. 21–22

D ALGORITHM

  • Algorithm 2 constructs confidence-guided trajectories for candidate lengths, selects a canvas length using Equation 6, then fine-tunes on the resulting PACS dataset using Equation 7.Source: D ALGORITHM · p. 22

Related papers

Loading 2610.18001v1…