Source-linked AI summary
How Instruction-Tuning Imparts Length Control: A Cross-Lingual Mechanistic Analysis
Elisabetta Rocchetti, Alfio Ferrara
TL;DR
Precise word-count control remains difficult for LLMs, despite its relevance to instruction-following applications. This study compares foundation and instruction-tuned Llama 3.1 8B in English and Italian, using CWA to analyze component contributions; instruction-tuning substantially improves control, with later-layer contributions varying by language.
Problem
Precise length constraints remain challenging for LLMs, even as applications increasingly require adherence to user instructions.
Source: §1 · p. 1Method
Results
Instruction-tuning substantially improves word-count control over BASE, which typically over-generates; later IT layers contribute positively, with attention especially influential in English and final-layer MLPs stronger in Italian.
Source: §7 · p. 9Takeaways & Limitations
The findings indicate that instruction-tuning reconfigures deeper layers for length adherence, with component-level strategies differing between English and Italian.
Source: §7 · p. 9Experiments used only single-digit target counts and Llama 3.1 8B, and examined only English and Italian.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Adhering to explicit length constraints, such as generating text with a precise word count, remains a significant challenge for Large Language Models (LLMs). This study aims at investigating the differences between foundation models and their instruction-tuned counterparts, on length-controlled text generation in English and Italian. We analyze both performance and internal component contributions using Cumulative Weighted Attribution, a metric derived from Direct Logit Attribution. Our findings reveal that instruction-tuning substantially improves length control, primarily by specializing components in deeper model layers. Specifically, attention heads in later layers of IT models show increasingly positive contributions, particularly in English. In Italian, while attention contributions are more attenuated, final-layer MLPs exhibit a stronger positive role, suggesting a compensatory mechanism. These results indicate that instruction-tuning reconfigures later layers for task adherence, with component-level strategies potentially adapting to linguistic context.
1 Introduction
Following user instructions is increasingly important as LLMs are used in diverse applications, yet precise word-count control remains challenging. This study compares foundation and instruction-tuned Llama 3.1 8B models in English and Italian and analyzes their component contributions.
1 Introduction
- LLMs are increasingly used for tasks such as rephrasing and summarizing within strict limits, increasing the need to follow instructions embedded in prompts.The paper proceeds from related work and preliminaries to methodology, experiments, findings, and limitations.Source: §1 · pp. 1–2
- The study investigates word-count control by comparing foundation and instruction-tuned Llama 3.1 8B models and analyzing their internal component contributions.The comparison also examines behavior in English and Italian.Source: §1 · p. 1
- Precise length constraints, including specified word, character, or syllable counts, remain challenging for LLMs.The study focuses specifically on word-count constraints.Source: §1 · p. 1
2 Related Work
Related work distinguishes explicit hard constraints such as word counts from softer controls such as style, and reports persistent difficulty with length control. The study connects that challenge to model mechanisms, instruction tuning, and the limited cross-lingual evidence available.
Controllable Text Generation
- Word-count control is an explicit hard constraint, unlike soft controls such as sentiment or style.Explicit controls are specified in natural-language prompts, while implicit control concerns quality standards and ethical guidelines.Source: §2 · p. 2
Length Control Capabilities
- Prior studies report that LLMs often miss requested lengths, struggle more as target counts rise, and remain imperfect with stricter prompts or few-shot examples.The cited literature also reports commercial models outperforming open-source models in length-controlled generation.Source: §2 · p. 2
- A proposed reason for poor word-count control is that token-based language models lack an explicit counting mechanism for words.The mismatch between tokens and words can make it harder to track word totals during generation.Source: §2 · p. 2
- The authors report no prior work comparing length-controlled generation across multiple languages.Source: §2 · p. 2
Approaches to Understanding LLM Control
- Mechanistic interpretability uses Direct Logit Attribution to identify component contributions, and this study applies it to word-count control across languages while comparing foundation and instruction-tuned models.Prior work has considered component contributions in linguistic tasks and the effects of instruction tuning on instruction-related behavior.Source: §2 · pp. 2–3
3 Preliminaries
The preliminaries introduce autoregressive language modeling and Transformer components, then explain Direct Logit Attribution as a way to estimate components’ direct effects on output logits.
Auto-regressive LLMs
- An autoregressive language model assigns probability to a token sequence, with each token conditioned on the preceding tokens.The sequence is denoted T, with t_i representing a token and t_<i its preceding tokens.Source: §3 · p. 3
- A Transformer LLM uses stacked blocks containing multi-head self-attention, a feed-forward network, and layer normalization to compute hidden states.Source: §3 · p. 3
- The final output layer projects the last hidden state into vocabulary space using the unembedding matrix.Source: §3 · p. 3
Direct Logit Attribution
- Direct Logit Attribution estimates a component’s direct effect on final logits using the additive structure of component outputs in the residual stream.Source: §3 · p. 3
4 Methodology
The methodology uses signed, stepwise Direct Logit Attribution to measure component contributions to word-count adherence, then compares those contributions across foundation and instruction-tuned models and across English and Italian.
4 Methodology
- Generation steps for the first N words count as correct, while premature EOS, post-N words, and delayed EOS count as failure steps.The EOS token immediately after the N-th word is also considered correct.Source: §4 · p. 4
- CWA aggregates component DLA contributions across generation steps, signing each step according to whether its token contributes to satisfying the N-word constraint.The study compares these component patterns between foundation and instruction-tuned models and across languages.Source: §4 · pp. 4–5, Figure 1 · p. 4
- Normalizing CWA by K enables comparisons across sequences of different lengths, while positive and negative values indicate contributions toward or against constraint satisfaction.CWA is a diagnostic of component contributions, not a generation-error metric, and is more sensitive to errors in shorter sequences.Source: §4 · p. 4
5 Experimental setup
The experiments compare base and instruction-tuned Llama 3.1 8B in English and Italian using prompt styles adapted to each model type, with cross-condition prompts to test phrasing robustness.
5 Experimental setup
- Four settings cover instruction-tuned and base models in both English and Italian, using direct-command prompts for instruction-tuned models and prefix prompts for base models.Each model type also receives one cross-condition prompt style to test robustness to phrasing.Source: §5 · p. 5
- Prompt templates vary the wording and language of word-count instructions; substituting N from 0 to 9 yields 80 unique prompts across the templates.The prompts include both instructional commands and prefix-style sentence completions.Source: §5 · p. 5
- The setup tests Llama 3.1 8B base and instruction-tuned models, which share a 32-layer architecture with 32 attention heads per layer; configurations were repeated 20 times.The shared architecture supports comparison under consistent architectural conditions.Source: §5 · p. 5
6 Results
Instruction-tuned models generally control word count better than foundation models, including under mismatched prompts. Their CWA profiles show stronger positive contributions from later-layer components, with different attention and MLP patterns across English and Italian.
Evaluating Length Control Performance
- Instruction-tuned models make fewer length errors than BASE models with matched prompts and generally outperform even optimally prompted BASE models when IT prompts are mismatched.The comparison includes both similarly mismatched prompts and the mixed condition of mismatched IT prompts versus matched BASE prompts.Source: Figure 2 · p. 6, §6 · pp. 6–7
- For template (b), MAE increases with target length in both languages, while Italian templates generally produce more errors than English ones.The IT model appears more accurate at N = 4 and N = 5 than at other target lengths.Source: §6 · p. 6, Figure 3 · p. 7
- With matched prompts, mean raw error was -0.07 for ENG-IT and -0.08 for ITA-IT.Raw error is generated word count minus target word count, so negative means are consistent with outputs that are correct or slightly shorter than requested.Source: §6 · pp. 6–7, Table 1 · p. 6
Analyzing Component Contributions to Length Control
- Attention-head CWA turns positive around layer 24 and strengthens in deeper IT layers, especially for ENG-IT; BASE attention is generally low or negative.Italian IT attention contributions are more attenuated than English IT contributions.Source: Figure 4 · p. 7, §6 · pp. 7–8
- Deeper BASE MLPs show strongly negative CWA, while ITA-IT shows stronger positive CWA in its final-layer MLP than ENG-IT.The authors suggest the Italian final-layer MLP may compensate for less positive attention contributions.Source: Figure 4 · p. 7, §6 · p. 8
- Across target lengths, ITA-IT attention CWA is generally lower than ENG-IT and dips notably at N = 3, while Italian later-layer MLP CWA is often more positive.The authors suggest later-layer MLPs may take on more processing when Italian attention contributes less positively.Source: Figure 5 · p. 8, §6 · pp. 8–9
7 Conclusions
Instruction-tuning substantially improves precise word-count control in Llama 3.1 8B, while the analysis associates this improvement with changed contributions in deeper layers and language-specific component patterns.
7 Conclusions
- Instruction-tuning substantially improves word-count control over BASE, which typically over-generates, though IT accuracy is slightly lower for Italian than English prompts.Source: §7 · p. 9
- CWA profiles show later-layer IT components consistently contribute positively to length adherence, while corresponding BASE components often contribute negatively.Source: §7 · p. 9
- Later-layer attention heads are especially influential in English, while Italian shows more attenuated attention contributions and a stronger compensatory role for final-layer MLPs.Source: §7 · p. 9
- The study tests only single-digit target counts in Llama 3.1 8B and English and Italian, limiting evidence about multi-token numbers, other model families, and broader language coverage.Source: §7 · p. 9
- Future work could use causal interventions to establish components’ functional roles and extend the framework to style transfer or sentiment control.Source: §7 · p. 9