PaperlayerView exact revision on arXiv

Source-linked AI summary

DIAL: Steerable Language Models with Distilled Instruction-Adherence Levels

Thanh Q. Tran, John Russell Himawan, Arun Verma, Kiwan Wong, Wei Xiao, Daniela Rus, Bryan Kian Hsiang Low

arXiv:2610.31438v1cs.AIcs.LG

TL;DR

Fine-grained instruction adherence is difficult to control reliably and efficiently across diverse instructions. DIAL distills guidance from a frozen LLM into a shared, text-conditioned steering adapter; evaluations report increasing concept expression across adherence levels and improvements in some instruction-following settings.

  • Problem

    Existing approaches do not jointly provide fine-grained, efficient, and scalable adherence control across general instructions; prompting is unreliable, while CFG requires two forward passes per generated token.

    Source: §1 · p. 2

  • Method

    DIAL distills classifier-free guidance from a frozen LLM into a lightweight, text-conditioned adapter that computes adaptive steering edits without per-instruction fitting.

    Source: §1 · pp. 2–3

  • Results

    Across LLM families, DIAL balances concept expression and fluency, with concept scores increasing across adherence levels; it also provides persona control and can improve verifiable instruction following when combined with prompting.

    Source: §1 · p. 3, §4.1 · p. 7, §5 · p. 9

  • Takeaways & Limitations

    A shared adapter trained from instruction and request text supports fine-grained control over new instructions without per-instruction fitting or labeled responses.

    Source: §1 · p. 3, §5 · p. 9

    On Qwen2.5-7B at α = 2, prompt+DIAL-IF follows local instructions better but global instructions worse than prompting, leaving overall accuracy lower; DIAL also does not improve every verifiable type.

    Source: §4.3 · p. 9, §5 · p. 9

Abstract

from arXiv · show

When a user says "Keep it short", how short should the LLM's response be? More generally, users and applications require different levels of instruction adherence, but prompting does not reliably provide fine-grained control over them. We therefore study "steerable language models" that provide fine-grained numerical control over the degree to which models follow general instructions specified in text. However, existing approaches do not jointly offer fine-grained control, efficiency, and scalability across diverse instructions. To address this gap, we introduce DIAL (Distilled Instruction-Adherence Levels) to achieve all three properties. For fine-grained control, we perform classifier-free guidance (CFG) with a frozen LLM to provide training signals for on-policy distillation of instruction adherence at different levels. For efficiency, we design a lightweight steering adapter that uses a classifier over the LLM's hidden states to compute a closed-form steering edit. The classifier adaptively sets the steering magnitude at each token based on the target adherence level, while accounting for the geometry of the LLM's hidden states to preserve response coherence. For scalability, the classifier is text-conditioned, allowing a shared adapter to support new instructions without per-instruction fitting, and training requires only instruction and request text. Evaluation across multiple LLM families shows that DIAL achieves a competitive balance between concept expression and response fluency, with concept scores increasing monotonically across adherence levels. We introduce SteerIF, an instruction-only dataset of 98, 625 concept, persona, verifiable, and descriptive instructions. When trained on SteerIF, DIAL provides fine-grained control over personas and, when combined with prompting, improves verifiable instruction following.

1 Introduction

Users and applications need different instruction-adherence levels, but existing approaches do not jointly provide fine-grained control, efficiency, and scalability. DIAL combines CFG distillation with a shared text-conditioned adapter to address these needs.

Motivation

  • Fine-grained adherence control addresses repeated prompting and regeneration when a response does not follow an instruction to the desired degree.The motivating applications include personalization, parental control, and company product-policy specifications.Source: §1 · pp. 1–2

Contributions

  • DIAL combines CFG-based on-policy distillation, a closed-form steering adapter, and text conditioning to target fine-grained control, efficiency, and scalability.The adapter adjusts steering magnitude at each token while accounting for hidden-state geometry; training uses instruction and request text without labeled responses, external generators, or judges.Source: §1 · pp. 2–3

Evaluation

  • Across multiple LLM families, DIAL’s concept score increases monotonically with adherence level while balancing concept expression and response fluency.On Gemma-2-9B, its 20.6M-parameter adapter retains 96–99% of base-LLM throughput in vLLM, with per-token latency approximately matching the base LLM.Source: §1 · p. 3
  • SteerIF contains 98,625 instruction-only examples spanning concept, persona, and verifiable instructions.Persona scores increase with adherence level across LLM families, and combining steering with prompting improves verifiable instruction following.Source: §1 · p. 3

2 Related Work

Prior steering approaches trade off fine-grained control, efficiency, and generalization across instructions. DIAL distills CFG distributions into a text-conditioned adapter intended to combine these properties.

Guided Inference

  • Guided inference offers fine-grained control, but CFG requires two forward passes per generated token.DIAL instead uses a single-pass adapter conditioned on adherence level to approximate CFG at multiple levels without changing LLM weights or prompts.Source: §2 · p. 3

Activation Steering

  • Activation steering is efficient and continuously adjustable, but its labeled binary training lacks intermediate adherence levels and often fits only one instruction.Increasing steering strength does not reliably increase adherence and can reduce fluency and coherence.Source: §2 · p. 3

Text-Conditioned Control

  • Text-conditioned control supports unseen instructions, but labeled training data, absent intermediate adherence supervision, or multi-step flows limit its scalability and practicality.Persona Dosing calibrates flow time against measured persona expression, whereas DIAL learns levels during training without measuring each new instruction.Source: §2 · p. 4

DIAL

  • DIAL distills the frozen LLM’s CFG distributions at multiple adherence levels into one adapter, using only instruction and request text.The approach is intended to provide fine-grained, efficient, and scalable instruction conditioning while keeping inference cost close to that of the base LLM.Source: §2 · p. 4

3 Method: DIAL (Distilled Instruction-Adherence Levels)

DIAL defines adherence levels through CFG, then trains a text-conditioned adapter to match CFG teacher distributions on student-generated prefixes. Its classifier adaptively edits hidden states at each token.

3.1 Instruction-Adherence Levels via Classifier-Free Guidance

  • At α = 1, the CFG teacher recovers the prompted distribution; lower α weakens instruction influence, while higher α strengthens it.CFG provides a next-token distribution at each level for any instruction text without labeled responses, external generators, or judges.Source: §3.1 · p. 4

3.2 On-Policy Self-Distillation across Adherence Levels

  • DIAL trains one adapter to match CFG teacher distributions across adherence levels while the frozen LLM generates responses steered at layer ℓ.The request enters the frozen LLM, while instruction text and adherence level enter the adapter.Source: §3.2 · p. 4
  • Training samples α from LogUniform[0.5, 2], covering levels weaker and stronger than prompting; beyond α = 2, CFG coherence degrades.The sampling distribution is symmetric in log α around α = 1.Source: §3.2 · p. 4
  • For each request–instruction pair, 24 adherence levels worked well across LLM families, and sampling α was important for fine-grained control.Training on more diverse instructions worked better than training longer.Source: Figure 2 · p. 5
  • On-policy training samples student responses and minimizes per-token Jensen–Shannon divergence between student and CFG teacher distributions.The loss restricts both distributions to the union of base, prompted, and student top-64 tokens, and adds a penalty on large relative edits.Source: §3.2 · p. 5

3.3 Lightweight and Adaptive Classifier-Based Steering Adapter

  • The adapter attends to instruction states and the 16 most recent edited states, then applies a linear map, LayerNorm, and GELU to produce a 512-dimensional feature.Its attention reuses the frozen layer’s normalization and query, key, and value projections.Source: §3.3 · p. 5
  • A text-conditioned classifier predicts a steering direction, an adherence-dependent threshold, and a norm weight to edit each token’s hidden state.States outside the classifier half-space receive a minimum-norm projection, then a norm adjustment; states already inside remain unchanged.Source: §3.3 · pp. 5–6
  • The norm weight interpolates between projection and rotation geometries, and the adapter’s learned norm weights and edit magnitudes differ across LLM families.On Gemma-2-9B, the adapter has 20.6M trainable parameters; cached instruction and memory states allow single-layer attention and a closed-form edit per token.Source: §3.3 · p. 6

4 Experiments

DIAL is evaluated across six LLM families on concept expression, persona expression, and verifiable instruction following. The experiments assess adjustable adherence alongside training and inference efficiency.

4 Experiments

  • Experiments train adapters across six LLM families and use SteerIF’s 98,625 instructions spanning concepts, personas, and verifiable and descriptive requirements.DIAL-IF uses one shared adapter per LLM; evaluation instructions are removed from both datasets through exact and near-match filtering.Source: §4 · p. 6

4.1 Controlling Concept Expression Levels

  • DIAL’s overall scores are 1.059 on Held-In and 1.036 on Held-Out concepts, versus FLAS’s 0.997 and 0.987.Prompt+DIAL scores 1.198 and 1.218, compared with prompting’s 1.049 and 1.060.Source: Table 1 · p. 7, §4.1 · p. 7
  • DIAL’s concept score rises from 0.20 at α = 0.5 to 1.42 at α = 2, spanning 1.21 points versus FLAS’s 0.66.It matches prompting’s concept score of 1.29 at α = 1.5 and surpasses it at α = 2.Source: §4.1 · p. 7, Figure 3 · p. 7
  • On Gemma-2-9B, DIAL retains 96–99% of base-LLM throughput in vLLM, versus 41–86% for CFG and 84–86% for FLAS.Its per-token inference latency approximately matches the base LLM’s.Source: Table 2 · p. 8, §4.1 · p. 8
  • For Gemma-2-9B, DIAL uses 4.65× fewer GPU-hours than FLAS and 25× fewer than HyperSteer, with 12.4× fewer trainable parameters than FLAS.DIAL trains using only instruction text, unlike FLAS and HyperSteer, which require labeled responses.Source: Table 3 · p. 8, §4.1 · p. 8

4.2 Controlling Persona Expression Levels

  • DIAL-IF’s persona score on Gemma rises from 12.0 without instructions to 76.4 at α = 1 and 91.4 at α = 2, near prompting’s 92.5.Combining prompting with DIAL-IF further raises persona scores, including from prompting’s 77.0 to 89.6 on Qwen at α = 2.Source: §4.2 · p. 8, Figure 4 · p. 9

4.3 Verifiable Instruction Following

  • At α = 1, prompt+DIAL-IF improves over prompting on five of six LLMs; on Qwen, it scores 71.9 versus prompting’s 72.3.On Gemma-2-9B, prompt+DIAL-IF reaches 76.9, compared with prompting’s 72.8.Source: §4.3 · p. 8, Table 4 · p. 9

5 Conclusion

DIAL combines a frozen LLM with a shared, text-conditioned steering adapter to control instruction adherence. Its token-level edit balances projection and norm-preserving rotation, though high adherence can reduce coherence and does not improve every verifiable instruction type.

5 Conclusion

  • DIAL offers fine-grained control over concept and persona expression, and combining it with prompting can improve verifiable instruction following; high adherence can reduce coherence, and gains do not cover every verifiable type.The conclusion proposes better classifier architectures and training losses, and filtering rollouts with verifiable rewards or judge feedback.Source: §5 · p. 9

A Extended Related Work

  • Related methods trade off fine-grained control, inference cost, and scalability, while DIAL distills CFG at every adherence level into one text-conditioned adapter rather than fitting per-instruction.The comparison spans methods that fine-tune model weights, steer activations, project hidden states, or condition control on text.Source: A Extended Related Work · pp. 20–21

B.1 Steering Adapter Architecture

  • The adapter reuses a frozen LLM layer’s normalization and query, key, and value projections, while training lightweight feature, normalization, prediction, and read-out components.It attends to instruction and memory states; inference caches instruction keys and values and stores recent edited states in a 16-entry buffer.Source: §B.1 · pp. 21–22

B.2 Geometry of the Steering Edit

  • At each token, DIAL projects states outside a classifier half-space, then interpolates between a norm-preserving rotation and minimum-norm projection; neither geometry guarantees instruction adherence.The norm weight λt sets the interpolation, with λt = 0 for rotation and λt = 1 for projection.Source: §B.2 · p. 22, Figure 5 · p. 22

B.3 Training Data

SteerIF is an instruction-text-only dataset assembled from concept, persona, verifiable, descriptive, and composite instructions. Its training setup pairs instruction types with requests and supplies CFG supervision even where responses cannot be checked automatically.

B.3 Training Data

  • SteerIF contains 98,625 instruction texts, including concept descriptions, persona instructions, single verifiable or descriptive requirements, and composite instructions.The concept pool has 14,880 descriptions, and the dataset includes 584 one-sentence persona instructions.Source: §B.3 · p. 23
  • The dataset adds 31,109 composites of two or three compatible requirements and 338 IFEval-type instructions, while retaining instruction text without responses or LLM outputs.Composites limit combinations to at most one requirement on length, case, language, structure, or punctuation.Source: §B.3 · p. 23
  • Verifiable and descriptive instructions are paired with 16–32 instruction-free IFEval questions, while concept and persona instructions use AxBench’s shared request corpus.Code verifies 36,014 of 82,823 verifiable and descriptive instructions; the remaining 46,809 receive CFG-teacher supervision without a response checker.Source: §B.3 · p. 23
  • Among 82,823 single and composite instructions excluding concepts and personas, 42.4% are positive, 29.9% negative, and 27.7% specify lower, upper, range, or exact requirements.Composite instructions have a larger share of negative requirements (44.5%) than single verifiable (14.4%) or descriptive instructions (28.3%).Source: §B.3 · p. 23, Table 6 · p. 23, Table 7 · p. 24

B.4 Training Configuration

DIAL and DIAL-IF are trained in separate one-epoch runs, with a shared adapter used across instruction types in DIAL-IF. Training samples adherence levels and uses CFG-guided student rollouts with a JSD objective; inference caches instruction and memory states for token-wise editing.

B.4 Training Configuration

  • DIAL-IF and DIAL-IF-Lite are trained for one epoch on SteerIF, sharing an adapter across concept, persona, and verifiable instructions and editing one layer near the LLM’s midpoint.DIAL-IF-Lite omits read-out paths; on Gemma-2-9B, the variants have 20.6M and 4.08M parameters, respectively.Source: §B.4 · p. 24
  • Training samples 24 adherence levels per instruction–request pair from LogUniform[0.5, 2] and uses student rollouts with temperature 1.The JSD loss uses β = 0.5 over the union of base, prompted, and student top-64 tokens, with a hidden-state-change regularizer.Source: §B.4 · p. 24
  • The CFG teacher sweep supports α = 2 as the upper training bound: its overall score falls from 1.32 at α = 2 to 1.23 at α = 2.5.Teacher fluency falls from 1.58 to 1.27 as α increases from 1 to 2.Source: §B.4 · p. 26

B.5 Pseudocode

  • The adapter attends to instruction and memory states to predict a classifier and apply a closed-form hidden-state edit; training matches CFG distributions on student rollouts while keeping the LLM frozen.At inference, instruction states are encoded once and cached, and generation uses one LLM inference pass per token.Source: Figure 6 · p. 26, Figure 7 · p. 27, Figure 8 · p. 27

C.1 Controlling Concept Expression Levels

On held-out concepts, DIAL provides a wide range of expression levels, while adding it to prompting extends control above prompting. DIAL remains below CFG at matched levels, but can preserve response fluency better in some comparisons.

C.1 Controlling Concept Expression Levels

  • On held-out concepts, prompt+DIAL raises AxBench’s prompt score from 1.049 to 1.198 on Concept500 and exceeds prompting by 0.158 on the held-out split.Prompt+DIAL is best on every split; its gain reflects higher concept incorporation with nearly unchanged instruction following.Source: §C.1 · p. 29
  • At α = 2, DIAL’s best overall score is 1.24 against FLAS’s 1.16, and it is clearly higher on both metrics than FLAS from α = 1.625 to 2.At a matched concept score near 1.05, DIAL also preserves more of the request (instruction scores 1.96 against 1.88) and fluency (1.46 against 1.33).Source: §C.1 · p. 29
  • DIAL’s held-out concept score rises from 0.20 at α = 0.5 to 1.42 at α = 2 and 1.47 at α = 2.5, while prompting with a stated level changes scores by only 0.10–0.21.At α = 1.5, DIAL’s concept score of 1.29 matches prompting; DIAL spans 1.18–1.40 points across levels, depending on the reported range.Source: §C.1 · p. 29
  • Prompt+DIAL at α = 0.5, 1, and 1.5 reaches concept scores of 1.35, 1.58, and 1.67, close to CFG at levels 1.5, 2, and 2.5.Here α controls the added edit, not an absolute adherence level; prompt+DIAL’s fluency is higher than CFG’s from α = 1.Source: §C.1 · p. 29
  • DIAL’s concept score rises in the same level order as CFG but stays below it at matched levels, reaching 82–90% of CFG’s concept score from α = 1 to 2.The text identifies limited capacity as a likely cause; editing four layers instead of one adds 0.057 on Gemma-2-2B.Source: §C.1 · pp. 29–30
  • On Gemma-2-2B, prompt+DIAL exceeds prompting by 0.231 and 0.245 on the two reported splits, while DIAL alone ties prompting.Prompt+DIAL also exceeds FLAS by 0.112 on held-out concepts and 0.044 on Concept500.Source: §C.1 · p. 30

C.2 Controlling Persona Expression Levels

Across held-out personas, DIAL-IF raises persona expression as adherence increases, while coherence can decline. On verifiable instructions, results vary by instruction category and model, with prompting and steering together improving accuracy in several settings.

C.2 Controlling Persona Expression Levels

  • DIAL-IF’s persona score rises monotonically across four levels on every evaluated LLM; on Gemma-2-9B it reaches 90.9 at α = 2, close to prompting’s 92.6.Prompt+DIAL-IF reaches 94.8 on Gemma at α = 2, while coherence falls to 68.1 against prompting’s 84.2.Source: §C.2 · p. 32
  • Adding a stated level to persona prompting changes persona scores by at most 6.6 points, while DIAL-IF spans 34–66 points from α = 0.5 to 2.On every LLM, prompting with a level stays within 5.4 points of prompting at the tabulated levels.Source: §C.2 · p. 32
  • On Llama-3.1-8B, full-vocabulary sampling produces low-coherence DIAL-IF responses; limited supervision of the low-probability tail is one possible cause.The training loss covers the union of the top-64 token sets rather than the full vocabulary.Source: §C.2 · p. 33

C.3 Verifiable Instruction Following

  • Prompt+DIAL-IF-Lite raises IFEval prompt-level strict accuracy over prompting on all five reported LLMs, including 72.8 to 77.6 on Gemma and 61.7 to 69.7 on Phi-4-14B.Source: §C.3 · p. 36
  • With the instruction in the prompt, DIAL-IF raises Gemma’s GLOBAL accuracy from 17.3 to 73.1 without prompting, but exact word-position instructions score 10.5.On top of the prompt at α = 0.5, accuracy across all 834 IFEval instructions rises from 80.2 to 83.7, while word-position accuracy falls from 89.5 to 86.8.Source: §C.3 · p. 37
  • At matched levels, prompt+DIAL-IF at α = 1 is within 0.4 points of CFG at α = 2 or above on four LLMs, but trails it on Phi-4-14B and Ministral-8B.CFG requires two LLM passes per token.Source: §C.3 · p. 38
  • On IFEval, prompt+DIAL-IF at α = 0.5 raises pooled GLOBAL accuracy from 81.1 to 84.6 on Gemma and from 83.1 to 86.8 on Llama, while matching Qwen at 82.0.Higher levels trade LOCAL for GLOBAL accuracy on Gemma and lower both relative to α = 0.5 on Llama.Source: §C.3 · p. 38
  • Without the instruction in the prompt, DIAL-IF accuracy on unseen IFBench instruction types rises from α = 1 to 2 on every LLM, including 10.7 to 15.9 on Gemma and 13.9 to 18.7 on Qwen.The filtered prompts cover 35 instruction types absent from SteerIF; all 25 IFEval types appear in SteerIF.Source: §C.3 · p. 38

C.4 Training and Inference Cost

DIAL approaches base-LLM throughput and uses fewer training resources than the compared methods. Paired comparisons and seed results support several reported gains, while layer ablations show that the default half-depth layer performs best in the tested settings.

C.4 Training and Inference Cost

  • In vLLM, DIAL retains 96–99% of base throughput, compared with 41–86% for CFG and 84–86% for FLAS; its per-token inference time matches the base LLM.DIAL’s first token arrives later than the base LLM’s but earlier than FLAS’s and CFG’s; in batched Hugging Face inference, DIAL retains 95–99% of base throughput.Source: §C.4 · p. 41
  • DIAL training requires 4.65× fewer GPU-hours than FLAS and 25× fewer than HyperSteer, with 12.4× fewer parameters than FLAS and no gpt-4o-mini completions.Training DIAL-IF on SteerIF takes 30.4–150.3 GPU-hours across six LLMs.Source: §C.4 · p. 41

C.5 Statistical Significance

  • On AxBench, prompt+DIAL and prompt+DIAL-IF improve over prompting and FLAS on both splits, with intervals well above zero; DIAL alone is on par with prompting.On IFEval, prompt+DIAL-IF at α = 1 gains more prompts than it loses on five of six LLMs, with intervals above zero on three.Source: §C.5 · p. 45
  • Across three seeds, the standard deviation is at most 0.013 except for DIAL at α = 1 (0.046); DIAL at α = 2 stays above FLAS on every seed.Prompt+DIAL at α = 1 also stays above prompting on every seed.Source: §C.5 · p. 45

D.1 Steering Layers

  • Both Gemma models score best at their default half-depth steering layer in the tested layer ablation; editing four consecutive 2B layers adds 0.057 on Concept500, while eight adds nothing over four.On Gemma-2-2B, every tested deeper single layer scores below the default layer.Source: §D.1 · pp. 45–46

D.2 Adapter Architecture

The adapter conditions edits on the instruction and recent response states, using a half-space parameterization that limits edits at lower adherence levels. Ablations find that an extra training pass does not help and direct edit prediction performs worse.

D.2 Adapter Architecture

  • The adapter attends to instruction states and a memory of the 16 most recent edited states, preserving word-level instruction detail rather than pooling the instruction into one embedding.At α = 0.5, it edits 12–67% of tokens, compared with 77–99% at α = 2.Source: §D.2 · p. 46
  • A second training pass with edited memory states does not improve results; the overall score falls from 1.134 to 0.939 for DIAL at α = 1.The second pass doubles training passes, and the other reported rows change by at most 0.03.Source: §D.2 · p. 46
  • Directly predicting an edit vector scores lower on all four rows than classifier-based steering, with the largest reported gap at prompt+DIAL α = 0.5 (1.409 against 1.558).Its edit size changes little from α = 0.5 to 2, unlike DIAL’s more level-responsive edit size.Source: §D.2 · p. 48

D.3 Steering Geometry

The learned norm weight varies across LLMs and changes quickly from its initial value. Compared with fixed rotation or projection, the learned geometry spans more persona-expression levels without reducing coherence.

Learned Norm Weight

  • The learned norm weight reaches 0.02–0.51 across six LLMs, and 44%–70% of positions are edited in the final training interval.It moves from its initial value of 0.95 within the first interval; Gemma-2-9B is close to norm-preserving, while the other five retain more projection norm change.Source: §D.3 · p. 48, Figure 12 · p. 50

Fixed Steering Geometry

  • Learned norm weighting spans the widest range of persona expression without reducing coherence, unlike fixed rotation or projection.Fixed rotation lowers persona score at every adherence level and can reduce matched-score coherence by up to 10 points; fixed projection scores 3–5 points lower without prompting at the same α.Source: §D.3 · pp. 48, 51, Table 41 · p. 51

Edit Analysis

  • From α=0.5 to 2, total intervention grows 5–13 times through more edited tokens, larger edits, or both, depending on the LLM.On five LLMs, the learned norm weight moves from projection at α=0.5 toward rotation at α=2; Gemma-2-9B rotates at every level.Source: §D.3 · p. 51

D.4 Training

Training ablations show that adherence-level supervision and a broad pool of distinct concepts matter, while extra epochs add little. In multi-turn conversations, DIAL retains concepts better when the instruction is given once, and prompting plus DIAL performs best when it is restated each turn.

D.4 Training

  • Training only at α=1 leaves the adapter insensitive to α: held-out concept score stays 0.95 from α=0.5 to 2, while DIAL rises from 0.25 to 1.58.A second or third epoch changes overall score by at most 0.06, and at equal compute 100 concepts score 1.137 at α=2 on Concept500 versus 1.408–1.463 for pools of 1,000 or more concepts.Source: §D.4 · pp. 51–52, Table 43 · p. 52

D.5 Multi-Turn Conversations

  • With the condition restated each turn, Prompt+DIAL scores 1.47 overall at turn 10, versus 1.36 for DIAL-LoRA and 1.34 for prompting.When specified only once, DIAL retains the most concept; long-context training keeps overall scores between 1.33 and 1.49 across ten turns, versus 0.75 for DIAL at turn 10, but takes 12.0 GPU-hours versus 4.06.Source: §D.5 · p. 53, Figure 13 · p. 54

E Ethics Statement

  • DIAL-IF raises scores for harmful personas and can partly bypass refusal behavior; the paper has not evaluated harmful-request benchmarks across adherence levels.The authors recommend validating steering instructions, restricting access to steering settings, and retaining output filtering; DIAL is not a verified safety mechanism and the evaluations omit long documents and real user interactions.Source: E Ethics Statement · p. 54

F Qualitative Examples

  • The qualitative examples were selected to illustrate changes across adherence levels, not sampled at random.Responses are verbatim excerpts shortened for readability.Source: F Qualitative Examples · p. 54

F.1 Example Generations for Personalization, Parental Control, and Product Policy

The examples illustrate adherence-level changes across personalization, concept, persona, and verifiable-instruction tasks. Across the 541-prompt comparison, Prompt+DIAL at α=0.5 both fixes some prompting failures and introduces some new failures.

F.1 Example Generations for Personalization, Parental Control, and Product Policy

  • As adherence increases, Gemma-2-9B’s examples become more Shakespearean, shorter, and more child-friendly, while its product-policy refusal remains intact.The examples cover personalization, parental control, and product policy, with one request per example and greedy decoding.Source: §F.1 · p. 55, Table 46 · p. 55, Table 47 · p. 55

F.2 Concept Steering

  • For a held-out superhero-narrative concept, higher adherence makes references more prominent in Gemma-2-9B’s response.The table holds the request and concept instruction fixed and reports concept ratings as the median of three judges.Source: §F.2 · p. 56, Table 50 · p. 57

F.3 Persona Steering

  • The humorous-persona example compares prompting, DIAL-IF alone, and Prompt+DIAL-IF across adherence levels, with persona and coherence scores from gpt-4o-mini.The example was selected for a large persona-score increase from α=1 to 2 among examples with coherence of at least 60.Source: §F.3 · p. 57, Table 51 · p. 57

F.4 Verifiable Instruction Following

  • Across 541 prompts, Prompt+DIAL at α=0.5 passes 40 prompts that prompting fails and fails 17 prompts that prompting passes.The examples are selected where strict-checker verdicts differ between methods, preferring shorter responses.Source: §F.4 · p. 65, Table 67 · p. 66
Loading 2610.31438v1…