PaperlayerView exact revision on arXiv

Source-linked AI summary

Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax

Zhenyan Lu, He Wang, Xiaohui Huang

arXiv:2609.29848v1cs.CL

TL;DR

Behavioral syntactic tests cannot distinguish absent encoding from encoded structure that fails to reach output. The paper compares behavioral deployment, LM-head readout, and probe recoverability on the same trilingual control items; the aggregate ordering places probe at or above LM-head and LM-head at or above behavior. The gap concentrates in subject-control, and activation patching localizes a layer shift under instruction tuning.

  • Problem

    Behavioral evaluation alone cannot distinguish whether a syntactic structure is absent from representations or encoded but not used at output.

    Source: §1 · p. 1

  • Method

    The paper evaluates behavioral deployment, the model’s LM-head readout, and probe recoverability on identical binary decisions in a trilingual control-dependency benchmark.

    Source: §1 · p. 1, §3.2 · p. 3

  • Results

    Across seven models and three languages, probe recoverability ≥ LM-head readout ≥ behavioral deployment in the aggregate, with the gap concentrated in subject-control.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support distinguishing what models encode from what they deploy, and show that instruction tuning can reshape deployment rather than encoding.

    Source: §6 · pp. 8–9

    Mechanistic evidence is concentrated in Qwen3, and cross-family evidence covers two text families rather than a broader architecture sample.

    Source: Limitations · p. 9

Abstract

from arXiv · show

A language model can fail a syntactic test in two distinct ways: by not encoding the relevant structure, or by encoding it but failing to use it at the output. Behavioral evaluation alone cannot tell these apart. We propose a three-level evaluation framework (behavioral deployment, LM-head readout, and probe recoverability) measured on the same items under the same binary decision. Using a compact trilingual (English, Chinese, German) control-dependency benchmark, we find that probe recoverability exceeds or equals LM-head readout, which in turn exceeds or equals behavioral deployment, across seven models and all three languages in the aggregate. The recoverability surplus is never negative across all 14 (model, task) conditions. The disconnect concentrates in subject-control, where a nearest-noun heuristic gives the wrong answer. The single largest gap (0.653) appears on Qwen3-0.6B Instruct in question answering. The gap persists at Qwen3-14B Instruct. Instruction tuning degrades deployment more than encoding in percentage terms. We rule out option-position bias, late-layer erasure, output-formatting artifacts, and probe-training variance. The pattern is consistent with decoding that favors surface shortcuts, and the behavior-probe gap measures the strength of that preference. Activation patching shows the gap is layer-localized. Under instruction tuning, the LM-head-decoded layer shifts approximately ten layers later than the probe-decoded layer. These findings argue that behavioral evaluation understates what models encode, while probing alone overstates what they deploy.

1 Introduction

Behavioral errors alone cannot show whether syntax was not encoded or was encoded but failed to reach the output. The paper measures behavioral deployment, LM-head readout, and probe recoverability on the same items, finding an aggregate ordering and a gap concentrated in subject-control.

1 Introduction

  • Behavioral failure leaves open whether the structure was never encoded or was encoded but failed to reach the output.Behavioral evaluation alone cannot distinguish these accounts.Source: §1 · p. 1
  • The three-level framework measures behavior, the model’s own LM-head readout, and linear-probe recoverability on identical minimal pairs under one binary decision.The benchmark covers English, Chinese, and German control dependencies.Source: §1 · pp. 1–2
  • Across seven models and three languages, probe recoverability ≥ LM-head readout ≥ behavioral deployment in the aggregate, with the gap concentrated in subject-control.The sharpest reported case is Qwen3-0.6B Instruct question answering: behavior 0.250 versus probe 0.903.Source: §1 · p. 2
  • Activation patching preliminarily localizes the gap: in instruction-tuned models, the LM-head-decoded layer shifts approximately ten layers later than the probe-decoded layer.The authors characterize the evidence as preliminary causal evidence.Source: §1 · p. 2

2 Related Work

Prior behavioral evaluation and representational probing measure different aspects of syntactic knowledge, and neither alone assesses both representation and output use. The paper positions LM-head readout as an intermediate measurement and uses control dependencies to examine the distinction.

2 Related Work

  • Behavioral evaluation measures preferences between matched strings, while probing tests what classifiers recover from hidden states; neither tradition alone measures both representation and output use.The cited behavioral examples include Linzen et al. (2016) and Warstadt et al. (2020); probing examples include Hewitt and Manning (2019).Source: §2 · p. 2
  • Prior studies report that syntactic probes do not reliably predict targeted evaluation and that hidden states can carry information not expressed in outputs.The cited work documents a recoverability-versus-usage separation, including in grammatical number.Source: §2 · p. 2
  • LM-head readout uses the model’s own output projection as an intermediate measurement, while control dependencies contrast linear proximity with verb-determined controller assignment.The section frames subject-control as the diagnostic case because it overrides the nearest-noun pattern.Source: §2 · p. 3

3 Methods

The methods use a small trilingual control-dependency benchmark and compare three kinds of access to the same binary decision. The design spans two task formats, multiple model families, and a probe ceiling that is distinguished from model deployment.

3.1 Benchmark design

  • The 48 hand-curated items are evenly divided across English, Chinese, and German and contrast object-control with subject-control using minimal pairs.Within each language, pairs share noun phrases and event semantics but differ in matrix-verb controller assignment.Source: §3.1 · p. 3
  • Question answering and paraphrase selection both score two explicit continuations by pairwise log probability, and the focal Qwen models also receive counterbalanced candidate ordering.The two formats test whether deployment failure depends on one template; counterbalancing tests option-position effects.Source: §3.1 · p. 3

3.2 Three levels of access

  • All three levels score the same binary forced choice, making their differences interpretable as access gaps rather than task- or label-format differences.Behavior scores the task-facing prompt; the LM head applies the model’s own output map to intermediate states without calibration, prompt, or training.Source: §3.2 · pp. 3–4
  • Intermediate-layer LM-head scores can reflect geometric incompatibility with the unembedding map, so they are not by themselves evidence of deployment failure.The behavior–probe contrast carries that claim; the LM-head readout anchors comparison in the model’s output geometry.Source: §3.2 · p. 4
  • Probe recoverability is the best leave-one-out accuracy across layers and feature modes, an estimate of what a linear decoder can extract rather than what the model deploys.The probe is logistic regression; three-seed checks report a maximum standard deviation of 0.064, while layer/mode selection carries optimism.Source: §3.2 · p. 4

3.3 Models

  • The model set centers on three Qwen3 variants, with Llama-3.2 and Gemma-4 checks and larger Qwen3-1.7B and Llama-3.2-3B pairs extending surplus-compression analyses.Llama-3.2 is the primary external replication target; Gemma-4 is used as an architectural contrast.Source: §3.3 · p. 4

4 Results

Across the aggregate benchmark, probe recoverability is at least as high as LM-head readout, which is at least as high as behavior, with the largest disconnect in subject-control. Cross-family checks, alternative-explanation tests, scaling analyses, and instruction-tuning comparisons qualify where the gap persists and how it changes.

4.1 Consistency of the three-level ordering

  • Across seven models and three languages, probe recoverability ≥ LM-head readout ≥ mean behavior; the recoverability surplus averages +0.210 across 14 model-task conditions.The aggregate ordering has bounded per-language exceptions: three of 21 cells invert, each within one item of per-language resolution except Gemma-4 E4B German.Source: §4.1 · pp. 4–5, Table 1 · p. 5, Figure 1 · p. 5
  • The largest gap is 0.653 on Qwen3-0.6B Instruct subject-control QA, where behavior is 0.250 and probe accuracy is 0.903.The three-level illustration gives LM-head readout as 0.667 between those values.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 Cross-family replication

  • Llama-3.2 preserves the ordering but narrows the surplus from +0.240 in the 1B base model to +0.115 in the 1B instruct model; Gemma-4 E4B marks an architectural boundary with surplus +0.042.In Llama-3.2-1B, instruction tuning lowers probe accuracy proportionally more than deployment; Gemma-4 differs on multiple architectural and pretraining axes.Source: §4.2 · pp. 5–6

4.3 Alternative explanations

  • Counterbalanced ordering, high late-layer probe accuracy, a prompt-free unembedding test, and probe-seed checks do not explain the three-level gap.A task-free third prompt preserves the ordering on five of seven models, with two exceptions where LM-head readout is at or below behavior.Source: §4.3 · p. 6

4.4 Scaling effects

  • At Qwen3-14B Instruct, the subject-control paraphrase gap remains 0.417 (behavior 0.583, probe 1.000), while the object-control gap approaches zero at 0.042.Scaling raises object-control deployment to ceiling but leaves the subject-control deficit open.Source: §4.4 · p. 6

4.5 Instruction tuning and the deployment gap

  • Instruction tuning lowers Qwen3-0.6B mean behavior by 14.1% and probe ceiling by 11.2%, narrowing the surplus from 0.344 to 0.323.Across Qwen3-1.7B and Llama-3.2-3B, the surplus also narrows, but the level taking the larger hit differs between pairs.Source: §4.5 · pp. 6–7
  • Activation patching places the effective controller-rescue layer approximately ten layers later in instruction-tuned models than in base models.The same-items comparison reports instructed Qwen3-0.6B LM-head accuracy of 0.667 against task-facing behavior of 0.510.Source: §4.5 · p. 7

5 Discussion

The discussion interprets the behavior–probe gap as a deployment failure concentrated in subject-control and persistent at 14B. Activation patching localizes the gap to a later readout layer under instruction tuning, while the content of probe recoverability changes with scale.

5 Discussion

  • At 14B, behavior and LM-head readout coincide at 0.792 while the probe reaches 1.000, and the ordering holds across seven models and three languages.The authors argue that the final decoding step fails to use controller information that is both encoded and projectable through the model’s own output geometry.Source: §5 · p. 7
  • Late-layer erasure and output-format artifacts are each ruled out as complete accounts, but a mixed account remains plausible.The authors report that probe recoverability stays high through final layers, debiased behavior remains below the LM-head peak, and a task-free prompt preserves the ordering on five of seven models.Source: §5 · p. 7
  • The failure concentrates in subject-control, where the nearest-noun heuristic gives the wrong answer despite probe-recoverable information.At 14B, the probe ceiling is perfect while behavior remains lower; instruction tuning also degrades deployment more than encoding in percentage terms.Source: §5 · pp. 7–8
  • The Gemma-4 E4B near-zero surplus supports two compatible interpretations, and the evidence does not distinguish them.The authors therefore base cross-family claims on Qwen3 and Llama-3.2.Source: §5 · p. 8
  • At 14B, subject-control probe recoverability generalizes across held-out verbs, whereas six smaller models remain leakage-dominant; object-control generalizes in every model.Thus, the highest-level subject-control readout shifts from verb-specific at small scale to verb-invariant at 14B.Source: §5 · p. 8
  • Activation patching finds the effective readout layer ten layers later in instruction-tuned models; patching there restores QA subject-control by +0.37.Linear steering along the probe-identified direction has no effect.Source: §5 · p. 8
  • LM-head readout prevents behavior-only and probe-only evaluation from collapsing deployment and recoverability into a single score.The authors describe it as the intermediate measurement that turns the gap into a localized diagnosis.Source: §5 · p. 8

6 Conclusion

The conclusion presents a three-level evaluation of behavioral deployment, LM-head readout, and probe recoverability on identical binary decisions. Across seven models and three languages, the levels diverge in a stable order, and instruction tuning can reshape deployment without reshaping encoding.

6 Conclusion

  • Encoding, readout, and deployment diverge in a stable order across all seven models and all three languages.The framework measures behavioral deployment, LM-head readout, and probe recoverability on identical binary decisions.Source: §6 · p. 8
  • The framework distinguishes what a model encodes from what it deploys, locating the gap at the output.Behavioral evaluation alone cannot distinguish absence of structure from encoded information that is not used at the output.Source: §6 · p. 8
  • Instruction tuning can reshape deployment without reshaping encoding.Source: §6 · p. 9

Limitations

The study’s diagnostic benchmark and mechanistic evidence have defined scope, while its measurements and interventions carry methodological limits. Probe recoverability is an upper bound on linear information extraction, not a measure of full syntactic competence.

Limitations

  • The 48 hand-curated items support targeted diagnosis, trading broad coverage for fixed item identity across measurement levels and languages.The benchmark spans English, Chinese, and German; template-generated benchmarks such as BLiMP contain tens of thousands of items.Source: Limitations · p. 9
  • Raw cross-language differences partly reflect tokenization and pretraining exposure, while non-English controller labels rely on matrix-verb semantics rather than explicit EUD propagation.The subject-control-gap-over-object-control contrast nevertheless replicates across seven models, three languages, and four task/control-type cells.Source: Limitations · p. 9
  • Mechanistic evidence centers on Qwen3, and evidence across text families is limited to Qwen3 and Llama-3.2; Gemma-4 supplies an architectural contrast, not a third replication.Qwen3-1.7B and Llama-3.2-3B add surplus-compression evidence but no mechanistic data.Source: Limitations · p. 9
  • Linear probes may understate recoverability, the fixed LM head is less clean for untied embeddings, and counterbalanced order does not address other prompt sensitivity.Single-layer activation patches support layer localization, not head-level attribution.Source: Limitations · p. 9
  • Probe recoverability bounds what a linear decoder can extract from its most informative layer, not syntactic competence or guaranteed deployment across conditions.The benchmark and experiments use public pretrained models; numerical drift may occur on near-tie items across GPU architectures.Source: Limitations · p. 9

A Descriptive Metric Definitions

This section defines descriptive summaries for behavioral deployment and the subject-control deficit, illustrates the benchmark’s controller contrast, and compares probe feature modes.

A Descriptive Metric Definitions

  • Mean behavior averages both task formats, while the behavior-to-recoverability ratio describes how much recoverable information reaches the behavioral interface.These measures are descriptive and do not replace per-task or per-type accuracies.Source: A Descriptive Metric Definitions · p. 12
  • A positive subject-control deficit means behavior is weaker on subject-control than object-control, independently of overall deployment level.Type-level behavior is summarized over 24 items of each control type.Source: A Descriptive Metric Definitions · p. 12

B Benchmark Examples and Prompt Templates

  • English told and promised, Chinese 让 and 答应, and German bat and versprach distinguish object-control from subject-control by which noun controls the embedded predicate.Object-control assigns control to the object; subject-control assigns it to the matrix subject.Source: B Benchmark Examples and Prompt… · p. 12

C Probe Feature Mode Comparison

  • Across layers, every feature mode exceeds 0.93 at 14B, while the default bv mode stays within 0.06 of the strongest mode in every model.For small models, av is weakest, consistent with subject-control being harder to encode at the matrix-subject position.Source: C Probe Feature Mode Comparison · p. 12, Table 3 · p. 12

D Probe Seed Robustness

Table 4 reports probe accuracy across three training seeds at the layer and feature mode selected by their mean; the accompanying text clarifies how this differs from seed-specific optima.

D Probe Seed Robustness

E Per-Language Breakdown

The per-language breakdown largely preserves the three-level ordering, with a localized exception and substantial language variation in surplus. The following order-sensitivity and intervention analyses distinguish behavioral readout effects from layer-localized patching effects.

E Per-Language Breakdown

  • The three-level ordering holds in every per-language row for five text-only models up to one-item noise, with larger inversions confined to Gemma-4 E4B German.There, behavioral accuracy is 0.594 versus LM-head accuracy of 0.500, a 1.5-item swing; Table 5 presents the per-language breakdown.Source: E Per-Language Breakdown · pp. 12–13, Table 5 · p. 12
  • For Qwen3-0.6B Instruct, Chinese recoverability surplus is +0.531 versus +0.125 in English; Qwen3-14B German behavior is 0.812 while probe ceiling is 1.000.Llama-3.2 Base also has higher German surplus (+0.312) than English surplus (+0.188).Source: E Per-Language Breakdown · p. 13

F Readout-Cleaning Order Sensitivity

G.1 Motivation and setup

  • The interventions test whether the gap reflects failed LM-head deployment of probe-readable information or probe geometry the LM head does not use.Directional steering produces no behavioral change; activation patching shows a layer-localized causal effect that differs between base and instruction-tuned models.Source: §G.1 · p. 13

G.2 Null directional steering at all tested magnitudes

  • Linear steering caused no candidate flips across tested strengths; at α = ±5, McNemar tests returned p = 1.000, while |α| ≥ 3 induced position-bias artifacts.At α = 0 the hook reproduced baseline behavioral scores exactly.Source: §G.2 · pp. 13–14

G.3 Layer-localized effects of activation patching

  • QA subject-control patching peaked near the probe layer in base models, but ten layers later in both instruction-tuned models.Llama-3.2-1B peaked at +0.24 at L9 (probe L7), Qwen3-0.6B Base at +0.21 at L18–L20 (probe L18), Qwen3-0.6B Instruct at +0.37 at L26 (probe L16), and Qwen3-14B Instruct at L27 (probe L17).Source: §G.3 · p. 14
  • Instruction-tuned models showed negative middle-layer QA object-control lifts, unlike the base models, before Qwen3-0.6B Instruct recovered to +0.35 at L26.Qwen3-14B Instruct reached −0.42 at L22 and L27; the authors interpret these negative lifts as a regime where a foreign correct geometry interferes more than a foreign wrong geometry.Source: §G.3 · p. 14

G.4 Joint interpretation of the layer shift

Activation patching localizes the deployment gap to a shifted LM-head-decoded layer under instruction tuning, while late-peak patching recovers only part of the controller variable. The accompanying verb-out analysis tests whether probe recoverability reflects generalizable control structure rather than verb-specific lookup.

G.4 Joint interpretation of the layer shift

  • Activation patching peaks at the probe layer in base models but ten layers later in instruction-tuned models, shifting the LM-head-decoded controller geometry.The intermediate window is probe-recoverable before the information is usable by output generation.Source: §G.4 · pp. 14–15, Figure 8 · p. 15
  • +0.37 at Qwen3-0.6B Instruct L26 and +0.19 at Qwen3-14B Instruct L27 are the QA subject-control late-peak patching lifts, both below ceiling.The late peak is causally relevant but does not realize all of the controller-assignment variable.Source: §G.4 · p. 15, Figure 7 · p. 15
  • Late-peak state geometry is not the only causal carrier; simultaneous multi-layer patching and finer component attribution remain open extensions.The authors describe their patching evidence as preliminary.Source: §G.4 · p. 15

H.1 Motivation

  • A probe ceiling of 1.000 could reflect memorization of clustered subject-control matrix verbs rather than an encoded syntactic relation.Verb-Out Cross-Validation tests this leakage hypothesis directly.Source: §H.1 · p. 16

H.2 Verb-Out CV Protocol

  • Verb-out CV holds out each unique matrix predicate as a fold; the benchmark has 32 unique predicates across 48 items.Training mixes subject- and object-control items, and holding out German versprach tests generalization across languages.Source: §H.2 · p. 16
  • The Qwen3-14B Instruct probe is retrained at its layer-17 peak across five feature modes, from candidate-only to verb-only exposure.Three random seeds match the main probing protocol.Source: §H.2 · p. 16

H.3 Focal results for Qwen3-14B Instruct

At Qwen3-14B Instruct’s probe peak, verb-out testing finds no measurable drop in default bv mode and only partial leakage in the stringent v-only mode. Across models, clean high-accuracy generalization across held-out verbs appears only at 14B, where it holds across a contiguous layer range.

H.3 Focal results for Qwen3-14B Instruct

  • +0.014 subject-control and −0.028 object-control are the default bv-mode item-out minus verb-out gaps at Qwen3-14B Instruct’s probe peak.In v-only mode, subject-control accuracy falls from 1.000 to 0.889 under verb-out CV, a +0.111 gap that remains in the partial-leakage range.Source: §H.3 · pp. 16–17, Table 8 · p. 16

H.4 Cross-model verification of scale-dependent emergence

  • Only Qwen3-14B Instruct reaches high subject-control probe accuracy with clean verb-out generalization; every smaller model is leakage-dominant at its probe peak.Three smaller models reach 0.903–0.917 item-out accuracy yet still fail verb-out; object-control passes cleanly at every model and layer.Source: §H.4 · p. 17, Table 9 · p. 18
  • Qwen3-14B Instruct’s clean subject-control window spans relative depth 0.30–0.62, or layers 12–25, with both accuracies above 0.9 and the gap below 0.10.The six smaller models never enter the clean band across their layer trajectories.Source: §H.4 · p. 17, Figure 9 · p. 18
  • The ordering holds across scales, but the highest-level subject-control content shifts from verb-specific lookup at small scale to a verb-invariant relation at 14B.At 14B, the relation generalizes across held-out matrix verbs and language boundaries; object-control is verb-invariant in every model.Source: §H.4 · p. 17

I Type-Level Gap and Three-Level Framework

  • Table 7 reports per-model, per-task, and per-control-type behavior and probe accuracies underlying the type-level analysis.The Qwen3-14B Instruct subject-control probe ceiling of 1.000 survives verb-out CV.Source: I Type-Level Gap and Three-Level… · p. 17

Related papers

Loading 2609.29848v1…