Stay on topic with Classifier-Free Guidance

Motivation

• Language models can hallucinate, degrade, or meander, while instruction-finetuning and reinforcement learning can carry compute and data costs that are inaccessible to some users.The paper hypothesizes that emphasizing user intent at inference time will improve alignment with expected behavior.Source: §1 · p. 1

Motivation and contributions

• The paper frames CFG as a way to make prompt intent more influential during generation, rather than relying only on training-based approaches to address generation issues.The introduction also identifies CFG’s use of negative prompts for more granular control and reports a 75% preference for GPT4All with CFG in a blind human evaluation.Source: §1 · pp. 1–2

Approach and illustration

• CFG adapts an inference-time guidance technique from text-to-image generation to improve prompt alignment across zero-shot, chain-of-thought, long-form, and chatbot-style prompting, without additional training in many situations.The table demonstrates GPT4All with CFG following both the system-level and user-level prompts where vanilla sampling ignores the system-level directive.Source: §1 · p. 2, Table 1 · p. 2

Contributions

• CFG brings significant gains across standard benchmarks spanning basic, chain-of-thought, long-text, and chatbot prompting, with LLaMA-7B achieving LAMBADA zero-shot SOTA over PaLM-540B.The introduction also reports similar performance to a half-size model at the same inference cost, 75% human preference for GPT4All with CFG, and reduced sampling entropy.Source: §1 · p. 2

2.1 Guidance in Text-to-Image Models

• Classifier Guidance uses an auxiliary classifier to steer generation, while Classifier-Free Guidance removes that classifier by using the generative model itself with conditioning dropout.Guidance strength γ controls the degree of conditioning emphasis; the text-to-image account describes γ > 1 as overemphasizing conditioning at a cost to diversity.Source: §2.1 · p. 3
• In latent-space terms, CFG steps away from an unconditional representation in the direction of the conditioning, with step size γ.The formulation can instead move away from a chosen negative prompt; the ordinary formulation follows when that negative conditioning is empty.Source: §2.1 · p. 3

2.2 Classifier-Free Guidance of Language Models

• The method treats the prompt as conditioning because a good continuation is expected to correlate with it, and extends the formulation to negative prompting.The paper introduces the prompt as context, instruction, or the beginning of text supplied for continuation.Source: §2.2 · pp. 3–4
• Language-model CFG reweights generation toward texts likely to begin with the prompt, approximating the reweighted distribution with the prompt-conditioned distribution raised to guidance strength γ.The method applies the guidance at the next-token level in logits space.Source: §2.2 · pp. 3–4
• Language models can provide both prompt-conditioned and unconditioned predictions because finite context windows let them naturally drop the prompt prefix, unlike conditioned diffusion models that need extra training.The method uses logits as its semantic space, avoiding network editing and remaining architecture agnostic.Source: §2.2 · pp. 3–4

3.1 Basic Prompting: Zero-Shot Prompts

• 81% accuracy on LAMBADA (OpenAI) was achieved by LLaMA 7B at γ = 1.5, exceeding PaLM-540B’s 77.9% zero-shot result.Figure 2 also places the LLaMA result above Chinchilla 70B and GPT-3 175B and identifies it as the June 2023 zero-shot SOTA.Source: §3.1 · p. 4, Figure 2 · p. 5

3.2 Deliberative Prompting: Chain-of-Thought

• On GSM8K and AQuA, low guidance increased valid, parseable chain-of-thought answers and improved performance, while large values degraded chain quality and overall performance.The authors leave testing alternative CFG weighting of prompt and reasoning-chain tokens for future work.Source: §3.2 · pp. 5–6, Figure 3 · p. 6

3.3 Text-to-Text Prompts: Generation

• The long-form generation experiments focus on code and also report machine-translation results in the appendix.These tasks test whether CFG helps maintain relevance across longer prompt and continuation sequences.Source: §3.3 · p. 6

3.3.1 Program synthesis evaluations

• Exploratory code-generation tests with GPT-J and CodeGen-350M-mono produced positive results, followed by a HumanEval evaluation.Source: §3.3.1 · p. 6

3.3.2 HumanEval benchmark

• Low CFG (γ ≤1.5) uniformly raises pass@1, while gains fade or reverse at high k as some previously nonzero task pass rates fall to 0%.For CodeGen-350M-mono at γ = 1.25, CFG outperformed on more tasks than it underperformed across temperatures 0.2, 0.6, and 0.8; the authors relate the high-k tradeoff to reduced variability and creativity.Source: §3.3.2 · pp. 6–7

3.4 Negative Prompting: Improving Assistants

• At γ = 3, 75% preferred the CFG system-prompt output, while user-prompt relevance remained 52%.The study paired CFG and baseline completions for 1,740 sampled system- and user-prompt combinations and evaluated them blindly.Source: Figure 5 · p. 7, §3.4 · p. 8

• CFG requires two network passes, motivating comparison against vanilla prompting with twice-larger models at matched inference cost.Source: §4 · p. 8
• Across 5 of 9 tasks, CFG and vanilla prompting with a twice-larger model differed insignificantly at p = .01; among significant tasks, two favored each.Source: §4 · p. 8
• Overall, a model using CFG can generally perform as well as a model twice its size, with implications for training budgets and inference latency under limited VRAM.Source: §4 · p. 8

5 Explaining the Success of Classifier-Free Guidance

• The exploratory analysis samples 32,902 P3 datapoints with Falcon-7b-Base and compares many results with Falcon-7b-Instruct.Source: §5 · p. 8

5.1 Classifier-Free Guidance’s Effect on Sampling Entropy

• CFG lowers generation entropy: its mean is 4.7 versus 5.4 for vanilla prompting, and the reordered tokens indicate an effect beyond temperature scaling.Source: §5.1 · p. 8, Figure 6 · p. 9

5.2 CFG’s Relation to Instruction Tuning

• Despite similar entropies, CFG and instruction-tuned vocabulary distributions are largely non-overlapping, so the hypothesized overall similarity is not supported.Source: §5.2 · pp. 8–9
• CFG–instruction-tuning agreement increases with prompt length (rs = .05); harder phrases for instruction-tuned models are also where the methods typically align.Source: §5.2 · p. 9

5.3 Visualizing Classifier-Free Guidance

• For the Paris-and-dragon prompt, CFG upweights tokens about dragons and Paris and downweights unrelated places, dates, and topics.Source: Table 3 · p. 10, §5.3 · p. 10
• The vocabulary-difference visualization can serve as a prompt-engineering tool by showing the differential impact of a candidate prompt on next-token predictions.Source: §5.3 · pp. 9–10

6 Conclusion

• CFG works out of the box in autoregressive language modeling and improves prompt adherence across basic, chain-of-thought, text-to-text, and chatbot prompting.Source: §6 · p. 10
• The authors leave CFG’s exact effects for future work and cannot conclude that the method is risk-free.Source: §6 · p. 10
• Doubling inference FLOPs with CFG can bring performance comparable to a model about twice the size, allowing smaller models that use smaller hardware and cost less to train.Source: §6 · p. 10

A Author Contributions

• The authors describe the work as a spontaneous EleutherAI collaboration involving Discord members, with contributions spanning experiments, writing, and supervision.Source: A Author Contributions · pp. 16–17

B.1 CFG

• Classifier-guided diffusion [28] informs CFG’s fidelity–diversity tradeoff, and prior image-generation experiments report greater prompt correlation as γ increases [68].Source: §B.1 · p. 17

B.2 Generative Guidance in NLP

• Related work traces controlled generation from CTRL [40] through PPLM [24] to expert–weak model logit differences [45].Source: §B.2 · p. 17

C Charts

• The appendix collects charts visualizing results from Sections 3.1, 3.3, and 5.Source: C Charts · p. 17

• Figures 8–10 plot accuracy against CFG strength for GPT-2, Pythia, and LLaMA model families, showing accuracy-to-γ curves across model sizes.Source: §C.1 · p. 17, Figure 8 · p. 18
• Figure 8 covers standard benchmarks over varying CFG strengths for GPT-2 models.Source: Figure 8 · p. 18
• TriviaQA uses substring rather than exact matching so quoted or sentence-form answers such as “Mark Twain” are not disqualified.Source: §C.1 · p. 18

C.2 Accuracy vs. FLOP

• At the same inference cost, CFG can generally match a model with twice the parameter count, with lower VRAM use and training cost noted as implications.The section describes ANCOVA comparisons between CFG and vanilla regression lines, using p = .01 as the significance cutoff.Source: §C.2 · pp. 18, 21
• The plots match models by inference FLOP per token, so similar accuracy does not represent equal training cost or parameter count.A 1.4B model using CFG can appear near a 2.8B vanilla model when their inference costs and performance are similar.Source: §C.2 · p. 21
• CFG improves the compute-accuracy curve for Lambada and SciQ, hurts WinoGrande, and has mixed effects on the remaining tasks.The charts compare models by inference FLOP per token; they do not reflect training cost.Source: §C.2 · p. 21

C.3 HumanEval benchmark

• The HumanEval appendix reports CodeGen-350M-mono, 2B-mono, and 6B-mono results across CFG strengths, temperatures, and pass@k curves.The accompanying tables and figures organize results by model size and temperature; the supplied passages do not state their numerical outcomes.Source: §C.3 · p. 21, Table 6 · p. 23, Table 7 · p. 23

C.4 Deliberative Prompting: Chain-of-Thought

• For AQuA CoT, small CFG strengths increase valid-answer structure and accuracy, while larger strengths retain few invalid answers but reduce accuracy.The appendix also reports CFG increasing the percentage of CoT chains yielding valid answers and boosting model performance.Source: §C.4 · p. 24, Figure 15 · p. 25

D.1 Machine translation

• In machine translation, γ values of 1.1–1.25 work best; gains hold across zero-shot prompts, but not Bloom-3B one-shot or significantly for mT0.The evaluation uses WMT14 French–English and reports BLEU scores.Source: §D.1 · p. 28

D.2 Prompting experiments for code generations

• 86% of GPT-J generations use the prompted programming language at γ = 1.25, up from 73% at γ = 1 (p-value 0.01).The experiment generated 100 samples across five runs and five prompts for each guidance strength.Source: §D.2 · p. 28, Table 12 · p. 29
• CodeGen-350M-mono completions were assessed for syntax, return type, output shape, and distance from a reference red-square image; return-type correctness improved most.The image-generation function returns a 32×32 RGB array, and the L2 comparison normalizes pixels to [0, 1].Source: §D.2 · p. 28

E.1 Continuations

• GPT2-medium with γ = 1.5 was preferred over γ = 1 and judged more satisfactory than GPT2-large with γ = 1, at comparable CPU latency for two medium passes.GPT2-large takes twice as long to evaluate and requires twice the memory, while also costing more to train.Source: §E.1 · p. 30
• The CoT examples contrast CFG-guided generations with baseline chains that diverge to incorrect answers or fail to preserve answer format.The examples cover GSM8K with WizardLM-30B and Guancano-65B.Source: Table 15 · p. 32, Table 14 · p. 32
• In the assistant example, CFG follows both the system-level directive and user prompt, whereas the baseline ignores the system directive.The example uses GPT4All with γ = 3 and a potentially out-of-distribution system instruction.Source: Table 16 · p. 33

F Further Comparison between CFG and Instruction-Tuning

• CFG and instruction-tuned outputs overlap more on longer, complex questions and least on vague, open-ended questions.The examples compare Falcon-7b-base with Falcon-7b-Instruct using top-p token overlap.Source: F Further Comparison between CFG and… · p. 34
• Across datasets, CFG and instruction tuning are mostly mixed in similarity; overall, vanilla prompting is more similar to instruction tuning than CFG.The comparison metrics are shown by overall word index of the generation.Source: F Further Comparison between CFG and… · p. 34

G.1 System prompts

• The system prompt directs the chatbot to identify whether an input is a question, task, or conversation and supplies varied response styles, including expert, poetic, code, and intentionally inappropriate responses.The section says these are the complete system prompts used in the chatbot experiment.Source: §G.1 · p. 34

G.2 User prompts

• The user prompt set includes 42 questions and tasks spanning general knowledge, advice, coding, creative writing, and personal recommendations.Examples range from explaining transformers and implementing Flask authentication to writing song lyrics and proposing campaign ideas.Source: §G.2 · p. 35
• Prompts 43–46 add time-travel science-fiction plots, a question about the Cambridge Analytica scandal, and a request about the band Halocene.Source: §G.2 · p. 36