Source-linked AI summary
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Pengxiang Li, Shilin Yan, Joey Tsai, Renrui Zhang, Ruichuan An, Ziyu Guo, Xiaowei Gao
TL;DR
Static unconditional inputs may not track the changing uncertainty of iterative masked language generation. A-CFG re-masks low-confidence tokens to adapt CFG’s unconditional input, and improves performance over standard CFG across generation benchmarks, including a 3.9-point GPQA gain.
Problem
Standard CFG in iterative masked language models often uses a static unconditional input that may not adapt to uncertainty varying across tokens and denoising steps.
Source: §1 · p. 2Method
A-CFG temporarily re-masks currently generated tokens with low predictive confidence to construct a localized unconditional input for CFG.
Source: §1 · p. 2Results
A-CFG improves complex reasoning accuracy and prompt adherence over unguided LLaDA and standard CFG, including a 3.9-point absolute gain on GPQA over standard CFG.
Source: §1 · p. 2Takeaways & Limitations
The results support dynamically adapting CFG’s unconditional input to model confidence in iterative masked language generation.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Classifier-Free Guidance (CFG) significantly enhances controllability in generative models by interpolating conditional and unconditional predictions. However, standard CFG often employs a static unconditional input, which can be suboptimal for iterative generation processes where model uncertainty varies dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel method that tailors the unconditional input by leveraging the model's instantaneous predictive confidence. At each step of an iterative (masked) diffusion language model, A-CFG identifies tokens in the currently generated sequence for which the model exhibits low confidence. These tokens are temporarily re-masked to create a dynamic, localized unconditional input. This focuses CFG's corrective influence precisely on areas of ambiguity, leading to more effective guidance. We integrate A-CFG into a state-of-the-art masked diffusion language model and demonstrate its efficacy. Experiments on diverse language generation benchmarks show that A-CFG yields substantial improvements over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work highlights the benefit of dynamically adapting guidance mechanisms to model uncertainty in iterative generation.
1 Introduction
Standard CFG’s static unconditional input may not adapt to uncertainty that shifts across tokens and denoising steps. A-CFG dynamically re-masks low-confidence tokens to localize the unconditional input, and improves LLaDA results over standard CFG on language-generation benchmarks.
1 Introduction
- CFG steers conditional generation by interpolating conditional and unconditional predictions without requiring an auxiliary classifier.The paper applies this established guidance mechanism to iterative masked language models.Source: §1 · p. 1
- A static or generic unconditional input may not reflect uncertainty that varies across tokens and denoising steps, potentially misaligning CFG’s guidance.The introduction frames instantaneous predictive confidence as a signal for targeting guidance at ambiguous positions.Source: §1 · p. 2
- At each denoising step, A-CFG identifies low-confidence tokens and temporarily re-masks them to form a localized unconditional input for standard CFG.This adapts the unconditional prediction to the model’s current uncertainty.Source: §1 · p. 2
- A-CFG improves complex reasoning accuracy and prompt adherence over unguided LLaDA and standard CFG, including a 3.9-point absolute gain on GPQA over standard CFG.It also improves Sudoku task success by 8.0 points versus standard CFG.Source: §1 · p. 2
2 Related Work
Related work traces language diffusion from early discrete and embedding-space approaches to iterative Masked Diffusion Models, then describes CFG and the limits of static unconditioning in masked language generation.
2.1 Diffusion Models for Language Generation
- Early discrete state-space and embedding-space diffusion approaches showed controllability potential but often lagged autoregressive models in likelihood or efficiency.Autoregressive generation, meanwhile, can face error propagation and limits in bidirectional context modeling.Source: §2.1 · p. 3
- Masked Diffusion Models iteratively infill text, and systems such as LLaDA have shown competitive performance with strong autoregressive models, making them candidates for fine-grained guidance.These models learn to reverse a gradual masking procedure by refining a sequence.Source: §2.1 · p. 3
2.2 Classifier-Free Guidance in Generative Models
- CFG steers generation toward a condition by interpolating conditional and unconditional predictions, typically enabled by occasional conditioning dropout during training.This avoids separate classifier training.Source: §2.2 · p. 3
- In iterative masked language models, a fixed unconditional input may not adapt to certainty that varies across token positions and refinement steps, potentially misaligning guidance.A fully masked target sequence is one common static input.Source: §2.2 · p. 3
3 Methodology
A-CFG replaces static unconditioning with a localized input built by re-masking low-confidence tokens, then applies CFG to target guidance at uncertain positions.
3.1 Preliminaries
- Standard CFG combines conditional and unconditional logits using a guidance scale, while iterative masked models progressively infill masked positions.The paper identifies defining the unconditional logits effectively as a central challenge for applying CFG to iterative masked language models.Source: §3.1 · p. 4
3.2 Adaptive Classifier-Free Guidance (A-CFG)
- A-CFG re-masks low-confidence, already-generated tokens to form a dynamic unconditional input, focusing CFG on ambiguous positions.The standard CFG formula then uses this adaptive unconditional prediction alongside the conditional prediction.Source: §3 · pp. 3–4, Figure 2 · p. 4, §3.2 · p. 5
- A-CFG addresses static unconditioning by using the model’s changing uncertainty to select positions for localized re-masking.Unlike masking the whole prompt or using a null prompt, it targets positions with the greatest predictive ambiguity.Source: §3.2 · pp. 4–5
3.2.1 A-CFG Process
- At each step, A-CFG scores non-[MASK] tokens by maximum softmax probability and selects the lowest-confidence positions for re-masking.The re-masking proportion ρ sets the target number relative to the available non-[MASK] tokens.Source: §3.2.1 · p. 6
- The selected positions are replaced with [MASK] to compute unconditional logits, which are combined with conditional logits to produce guided logits for the next infilling step.If no positions are re-masked, the dynamic input remains identical to the current sequence.Source: §3.2.1 · pp. 6–7
4 Experiments
The experiments section introduces the evaluation, then reports benchmark comparisons, ablations, and qualitative examples of A-CFG.
4 Experiments
- The section first describes datasets, baseline models, evaluation metrics, and implementation details.This establishes the experimental setup before presenting results.Source: §4 · p. 7
- It then compares LLaDA with A-CFG against standard CFG, no guidance, and other state-of-the-art models using quantitative results from Table 1.Source: §4 · p. 7
- The section also analyzes A-CFG’s core hyperparameter through ablations and uses qualitative examples to illustrate the method’s behavior and benefits.Source: §4 · p. 7
4.1 Experimental Setup
The setup evaluates A-CFG across language understanding, reasoning, and planning benchmarks, with comparisons against guidance baselines and other models.
4.1.1 Datasets and Metrics
- The benchmark suite covers general language understanding, mathematics and science reasoning, and planning tasks.It includes MMLU, BBH, ARC-C, Hellaswag, TruthfulQA, WinoGrande, PIQA, GSM8K, MATH, GPQA, Countdown, and Sudoku.Source: §4.1.1 · p. 7
- Closed-form tasks select the candidate answer with the highest conditional log-likelihood, while open-ended tasks use generation and task-specific scoring.Likelihood-based evaluations approximate the conditional perplexity bound with Monte Carlo sampling; multiple-token tasks use 128 samples.Source: §4.1.1 · p. 7
4.1.3 Implementation Details
- Unless otherwise stated, generation uses 256 answer tokens and 256 reverse-diffusion steps, revealing one token per step.Source: §4.1.1 · p. 7
4.1.2 Baseline Models and Methods
- The primary comparison evaluates LLaDA 8B with no guidance, standard CFG, or A-CFG, and also tests A-CFG on Dream-7B against its baseline.Results are contextualized against reported autoregressive model scores; Table 1 notes LLaDA and Dream-7B baseline scores were re-evaluated under a consistent protocol.Source: §4.1.2 · p. 7, Table 1 · p. 8
4.1.3 Implementation Details
- For LLaDA, A-CFG uses 256 sampling steps and a re-masking proportion ρ of 0.7; guidance scale w is tuned and then fixed across benchmarks for each model.Standard CFG’s w is selected from {0.5, 1.0, 1.5, 2.0} using each task’s validation set; A-CFG’s confidence uses predicted-token softmax probability.Source: §4.1.3 · p. 8
4.2 Benchmark Results
Across the reported benchmarks, A-CFG improves LLaDA 8B over standard CFG, particularly on reasoning and planning, and shows preliminary benefits on Dream-7B. With A-CFG, LLaDA is competitive with autoregressive models on some reasoning tasks, though leaders retain advantages on some language-understanding benchmarks.
4.2 Benchmark Results
- 33.3 GPQA (+3.9 over Standard CFG's 29.4) and 42.0 Sudoku (+8.0 over 34.0) are reported for LLaDA 8B with A-CFG.The section also reports a +2.7-point gain on GSM8K and improvements across ARC-C, Hellaswag, and WinoGrande versus Standard CFG.Source: §4.2 · p. 8
- Dream-7B with A-CFG improves Sudoku by 8.0 points (80.0 vs. 72.0) and ARC-C by 1.0 point (60.8 vs. 59.8) in preliminary results.The authors suggest these observations indicate potential for other iterative masked diffusion models.Source: §4.2 · p. 8
- With A-CFG, LLaDA 8B scores 73.5 on GSM8K versus LLaMA3 8B's 53.1, and 42.0 on Sudoku versus 0.0.On GPQA, its 33.3 is above LLaMA3 8B's 25.9 and competitive with Qwen2 7B's 30.8; Qwen2 7B retains an advantage on some general language-understanding benchmarks.Source: §4.2 · p. 9
- The reported results characterize A-CFG as improving over standard CFG and enabling LLaDA to be competitive with, and sometimes outperform, strong autoregressive baselines, especially on sophisticated reasoning tasks.The authors report improvements over unguided generation as well as conventional CFG.Source: §4.2 · p. 9
4.3 Ablation Studies
The ablations examine how A-CFG performance on ARC-C varies with the adaptive re-masking proportion ρ and guidance scale w. Both parameters affect accuracy, with the reported best settings matching the main ARC-C result.
4.3 Ablation Studies
- The ablations target A-CFG's core components and sensitivity to hyperparameters, focusing on adaptive re-masking proportion ρ.The main ARC-C result uses ρ = 0.7 and reports 47.8 accuracy.Source: §4.3 · p. 9, §4.3.1 · p. 9
4.3.1 Impact of the Adaptive Re-masking Proportion (ρ)
- ARC-C accuracy rises from 45.9% at ρ = 0.1 to 47.8% at ρ = 0.7, then declines at ρ = 0.9.The intermediate values are 46.5% at 0.3 and 46.8% at 0.5; the authors suggest more substantial re-masking helps up to a point, while excessive re-masking may erase valuable context.Source: §4.3.1 · p. 9
4.3.2 Impact of the Guidance Scale (w)
- ARC-C accuracy peaks at 47.8% for w = 0.5 and w = 1.0, versus 45.5% at w = 0.0, then is slightly lower at 1.5 and 2.0.The respective scores at w = 1.5 and w = 2.0 are 47.5% and 47.6%; the main ARC-C result uses w = 0.5.Source: §4.3.2 · p. 9
4.4 Qualitative Analysis
The qualitative examples depict A-CFG refining uncertain parts of multi-step mathematical reasoning across iterations. Foundational tokens may appear early, while operators, intermediate values, and final quantities are resolved or corrected later.
4.4 Qualitative Analysis
- In the Natalia's clips example, foundational elements appear early, while arithmetic operators, intermediate results, or the final sum are often resolved or corrected later.The authors connect this pattern to A-CFG re-masking low-confidence positions for reconsideration.Source: §4.4 · pp. 9–10
- Darker shades in Table 3 mark tokens filled or corrected in later stages, often at points of initial uncertainty.The visualization concerns A-CFG's iterative refinement on math-reasoning tasks.Source: Table 3 · p. 10
- In the John's apples example, later steps refine the calculation, including the intermediate and final quantities in 6+12=18.The example is presented as an instance of refining a multi-step reasoning chain.Source: §4.4 · p. 10
5 Conclusion
The conclusion presents A-CFG as a confidence-responsive way to construct CFG's unconditional input for iterative masked language models. Experiments, particularly in LLaDA, report gains over standard CFG and unguided baselines, especially on reasoning and planning, and potential competitiveness with autoregressive models.
5 Conclusion
- A-CFG dynamically constructs CFG's unconditional input from confidence in already generated tokens, targeting guidance toward uncertain sequence regions.The conclusion frames this as a more targeted and responsive guidance mechanism for conditional generation in iterative masked language models.Source: §5 · p. 10
- Experiments, particularly within LLaDA, report that A-CFG outperforms standard CFG and unguided baselines across diverse benchmarks, especially in reasoning and planning.The conclusion also describes diffusion-based models as potentially more competitive with autoregressive counterparts.Source: §5 · p. 10