PaperlayerView exact revision on arXiv

Source-linked AI summary

DV-Lens: Revealing the Functional Organization of Language Model Parameters

Chenhang Cui, Jian Yu, Shuyi Miao, Xiaohao Liu, Rui Huang, Fei Shen, An Zhang, Tat-Seng Chua

arXiv:2610.04489v1cs.CLcs.LG

TL;DR

The paper asks how parameter directions from different model modules connect to verifiable output effects and how parameter organization relates to capability. It introduces DV-LENS to trace parameter columns into vocabulary responses and DV-COMPLEXITY to measure organization; its readouts receive intervention-based support, while complexity correlates with benchmark capability rankings.

  • Problem

    How native parameter directions affect outputs and how their functional organization relates to model capability remain insufficiently characterized.

    Source: §1 · p. 1

  • Method

    DV-LENS propagates native parameter columns through averaged downstream mappings into vocabulary-space readouts, while DV-COMPLEXITY groups parameters by those readouts and measures within-group weight variation.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    The joint-parameter DV-COMPLEXITY score correlates 0.904 with benchmark-based capability rankings across 48 language models, alongside intervention-based evidence supporting DV-LENS interpretations.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The results connect local parameter analysis with model-level comparison through a shared vocabulary-space representation.

    Source: §5 · p. 10

    The capability estimand is a cross-sectional association with a fixed capability measure, not a test of causal effects.

    Source: §3.4 · p. 6

Abstract

from arXiv · show

Understanding parameter functions helps elucidate the internal mechanisms of large language models (LLMs). However, how to connect parameters from different modules to verifiable output effects and further characterize the relationship between their functional organization and model capability remains to be explored. To this end, we introduce the downstream vocabulary lens (DV-Lens), a parameter-level interpretability framework that links native parameter directions to their downstream vocabulary responses. Specifically, we first estimate module-specific downstream Jacobians over a reference prompt set for attention query, key, value, and output (Q/K/V/O) projections and feed-forward networks (FFNs). Second, we use these mappings to project native parameter columns into the final vocabulary space, obtaining signed readouts that characterize their average local output responses. Third, we group parameter columns by their vocabulary readouts and introduce downstream vocabulary complexity (DV-Complexity), which quantifies within-group structural variation using normalized reconstruction residuals of the original weights. At the parameter level, randomized controls and finite-difference tests show that DV-Lens readouts capture non-random vocabulary structure and predict local logit changes with 98.0% coordinate-orientation agreement across 720 cases from nine models. These readouts further guide parameter ablation, steering, and swapping across 21 models, shifting target-token probabilities in the predicted directions under controlled conditions. At the model level, the joint-parameter score of DV-Complexity achieves a Spearman correlation of 0.904 with benchmark-based capability rankings across 48 language models. Together, these results provide intervention-based evidence for DV-Lens interpretations and reveal an association between DV-Complexity and model capability.

1 INTRODUCTION

The introduction asks how native parameter directions affect model outputs and whether their organization relates to model capability. It presents DV-LENS for tracing parameters into vocabulary space and DV-COMPLEXITY for measuring their organization.

1 INTRODUCTION

  • Prior work studies activations and parameters, but generally does not account for how parameter effects pass through later computations to model outputs.This gap motivates connecting downstream parameter effects to model capability.Source: §1 · pp. 1–2
  • DV-COMPLEXITY’s joint-parameter score correlates 0.904 with benchmark-based capability rankings across 48 language models.The introduction frames this as an association between parameter organization and model capability.Source: §1 · p. 2
  • DV-LENS maps native parameter columns through module-specific downstream mappings into a common vocabulary space, unlike direct projection through the LLM head alone.The framework covers attention Q/K/V/O projections and FFNs.Source: §1 · p. 2, Figure 1 · p. 2
  • Randomized controls, finite-difference tests, and direct parameter interventions provide complementary statistical and behavioral checks of DV-LENS readouts.The interventions assess predicted output changes under controlled settings.Source: §1 · p. 2

2 RELATED WORK

Related work interprets language-model internals through activation analysis, feature discovery, and parameter-oriented methods. DV-LENS instead traces parameter columns through downstream computation and tests predicted output changes with interventions.

2 RELATED WORK

  • Activation methods decode intermediate representations, patch activations, or characterize structure through sparse features and distinctive directions.Examples include Tuned Lens, Patchscopes, sparse autoencoders, transcoders, and ICA-based methods.Source: §2 · p. 3
  • DV-LENS focuses on parameter columns and uses module-specific mappings and prompt-averaged responses, then tests predicted output changes through direct interventions.This contrasts with methods that derive features from learned transcoder weights or extract vocabulary-aligned FFN directions without forward passes.Source: §2 · p. 3
  • Parameter-oriented studies interpret FFN weights as key-value memories, project gradients or weights into vocabulary space, and edit weights using localized factual associations.The cited examples include Backward Lens, WeightLens, ROTATE, causal tracing, and ROME.Source: §2 · p. 3

3 DOWNSTREAM VOCABULARY LENS

DV-LENS traces native parameter directions through downstream computation into vocabulary-space responses, which can guide interventions and organize parameters for analysis. DV-COMPLEXITY measures that organization and provides a loss-related interpretation of structural variation.

3 DOWNSTREAM VOCABULARY LENS

  • The section introduces DV-LENS for tracing learned parameter directions to model outputs, using vocabulary readouts to guide interventions and analyze parameter organization.It also introduces DV-COMPLEXITY to examine organization in relation to capability.Source: §3 · p. 3

3.1 MOTIVATION: TRACING PARAMETER EFFECTS TO THE OUTPUT

  • A parameter column specifies a direction written into a module output, while its input activation determines how strongly it participates in the current computation.A small activation perturbation changes the module output along that column, and the downstream Jacobian propagates the change toward final logits.Source: §3.1 · pp. 3–4

3.2 DOWNSTREAM VOCABULARY LENS

  • DV-LENS averages downstream mappings over a fixed prompt set, then propagates each parameter column through later computation before reading it out in the final vocabulary.The readout is a prompt-set average local response to a unit activation change, not an activation-weighted contribution on an individual prompt.Source: §3.2 · p. 4

3.3 PARAMETER SIGNALS FOR TARGETED TOKEN CONTROL

  • Targeted control represents concepts in vocabulary space, maps them to parameter signals, and uses those signals for ablation or steering.A parameter column’s association with a concept is evaluated after tracing it through downstream computation.Source: §3.3 · pp. 4–5

3.4 DV-COMPLEXITY AND MODEL CAPABILITY

  • DV-LENS readouts determine parameter clusters, while normalized reconstruction error evaluates how tightly the original weights group within those readout-defined clusters.The resulting model-level score aggregates matrix-level organization; the capability measure and cohort are fixed across scopes, making the estimand cross-sectional association.Source: §3.4 · pp. 5–6
  • The capability comparison ranks structural scores and capability scores across a fixed checkpoint cohort and reports their Spearman association.The same capability variable, cohort membership, and rank convention apply across six module scopes.Source: §3.4 · pp. 5–6

3.5 THEORETICAL INTERPRETATION OF DV-COMPLEXITY

  • DV-COMPLEXITY is interpreted as functional parameter structure that cannot be compressed into a small set of roles without affecting the predictive objective.The score quantifies normalized parameter energy removed by collapsing weights within roles, relating that removal to increased predictive loss.Source: §3.5 · p. 6

4 EXPERIMENTS

Across 48 models, DV-Complexity’s Joint rank is strongly associated with benchmark-based capability rank, and remains largely unchanged under complete-benchmark exposure. Parameter-level tests find non-random vocabulary structure, predict local responses, and guide behavioral interventions.

4.1.1 NON-RANDOM VOCABULARY STRUCTURE

  • +2.363 mean excess-kurtosis difference across 21 models, with the original prompt-level analysis also showing stronger heavy-tail and concentration structure than matched random directions.The prompt-level excess-kurtosis difference is +2.128 and the Top-1% ℓ2-energy difference is +0.0191; these analyses use different aggregation units.Source: §4.1.1 · p. 7, Table 1 · p. 7

4.1.2 FINITE-DIFFERENCE PREDICTION

  • 0.980 pooled coordinate-orientation agreement across 720 cases, compared with 0.407 exact Top-32 overlap and 0.613 Recall@128.Orientation is high across modules, while coordinate recovery is lowest for K and highest for O and FFN; wider-tail recall indicates many misses remain within the same high-impact region.Source: §4.1.2 · pp. 7–8, Figure 2 · p. 7, Table 2 · p. 8
  • From earliest to latest sampled layers, orientation agreement rises from 94.5% to 99.96%, exact Top-32 overlap from 21.4% to 67.2%, and Recall@128 from 38.3% to 86.3%.All nine models improve across these endpoints for each metric; early layers recover most coordinate signs but have less precise high-impact token rankings.Source: §4.1.2 · p. 8, Figure 3 · p. 8

4.2 BEHAVIORAL INTERVENTION

  • At Joint scope, peak conditional decoded success is 61/66 for ablation at 5%, 28/66 for steering at 0.1%–0.2%, and 23/66 for swapping at 5%.The conditional measure counts edited continuations satisfying an operation-specific answer-containment rule among concept cells whose clean continuation contains the original gold answer.Source: §4.2 · p. 8, Figure 4 · p. 9

4.3 MODEL-LEVEL CAPABILITY AND BENCHMARK-EXPOSURE DIAGNOSTICS

  • ρ = 0.904 for Joint DV-Complexity and capability ranks, the strongest of six module scopes; DV-Complexity ranks also remain largely unchanged under complete-benchmark exposure.The exposure test covers eight models from six families and five benchmark test splits; ability ranks improve while DV-Complexity ranks remain largely stable.Source: §4.3 · pp. 8–9, Figure 6 · p. 10, Figure 7 · p. 10

5 CONCLUSION

DV-LENS maps parameter columns to vocabulary-space readouts and connects parameter interpretation with targeted interventions. DV-Complexity summarizes functional organization and tracks broad capability across 48 models, with its ranks largely stable under benchmark exposure.

5 CONCLUSION

  • DV-Complexity ranks remain largely stable under benchmark exposure even when benchmark performance improves.The exposure comparison covers eight models from six families and five benchmark test splits.Source: §5 · p. 10

APPENDIX

  • The appendices document intervention and capability protocols, supporting checks, signal computations, DV-Complexity theory, limitations, and interpretation questions.Source: APPENDIX · p. 13

A.1 INTERVENTION SETUP

  • Intervention experiments vary model, concept, scope, operation, and intervention fraction, with Joint edits applied across Q, K, V, O, and FFN.Source: §A.1 · p. 13

A.2 CAPABILITY ANALYSIS

  • Capability analysis evaluates Joint, Q, K, V, O, and FFN on the same fixed set of 48 models and capability ranks.The six scopes yield 288 observed values; fitted lines summarize relations within this fixed model set.Source: §A.2 · p. 13

A.3 CAPABILITY SCORE AND FIXED MODEL SET

  • Capability is the unweighted mean of six Open LLM Leaderboard v2 benchmark scores, and the study ranks these scores against DV-Complexity across the same 48 models.The benchmarks are IFEval, BBH, MATH Level 5, GPQA, MuSR, and MMLU-Pro.Source: §A.3 · p. 14

A.4 CAPABILITY CONTROLS

The capability-correlation controls find that DV-Complexity exceeds matched random-label correlations and has higher point estimates than raw-weight clustering, while family-removal results remain positive.

Random Labels with Matched Cluster Sizes

  • All six DV-Complexity correlations exceed every matched random-label result across 128 permutations on the fixed 48-model set.The conditional randomization value reaches its minimum possible value.Source: Table 4 · p. 15, §A.4 · p. 15

Raw-Weight Baseline

  • DV-Complexity correlations are higher in all six raw-weight comparisons, but every bootstrap interval includes zero.This supports a consistent direction in point estimates, not a clear bootstrap difference.Source: §A.4 · pp. 15–16, Table 5 · p. 16

Removing One Model Family at a Time

  • All six correlations stay positive after removing each model family; Joint varies from 0.888 to 0.909, with maximum absolute change 0.016.Source: §A.4 · p. 16, Table 6 · p. 16

A.5 COMPLETE-BENCHMARK EXPOSURE

The benchmark-exposure experiment trains eight models on complete benchmark test splits and evaluates benchmark performance and DV-Complexity ranks separately. IFEval is omitted from this comparison because its exposure training was unstable and prone to collapse.

Exposure Setup

  • Eight models from six families are exposed separately to complete BBH, MATH, GPQA, MuSR, and MMLU-Pro test splits.Each model–benchmark arm starts from the original checkpoint and uses that benchmark’s reference answers as training targets.Source: §A.5 · p. 16

Rank Comparison

  • Rank advance is baseline rank minus post-exposure rank, and DV-Complexity rank changes of at most one count as stable.Each exposed checkpoint replaces only its own baseline score; other models retain their original scores.Source: §A.5 · p. 16

Why IFEval Is Omitted

  • IFEval is omitted from the exposure comparison because its training was unstable and prone to collapse, but remains in the six-task capability score.Source: §A.5 · p. 16

B.1 MATCHED-NULL COVERAGE AND VARIATION

Matched-null comparisons show positive vocabulary-structure differences across models, while fixed examples illustrate that vocabulary-tail readability varies. The merged material also specifies intervention setup and the scope of the complete decoded-success grid.

B.1 MATCHED-NULL COVERAGE AND VARIATION

  • Both excess kurtosis and Top-1% ℓ2-energy show positive mean differences across all 21 models, though effect sizes vary across models, depths, and modules.Q, K, O, and FFN are positive for both measures in all models; V is positive in 19 of 21.Source: §B.1 · p. 17, Table 8 · p. 18
  • After removing any one of 13 model families, excess-kurtosis differences remain 2.195–2.472 and Top-1% ℓ2-energy differences remain 0.0403–0.0432.Both measures remain positive for every retained model in each setting.Source: §B.1 · p. 17

B.2 FIXED FUNCTIONAL INSPECTION

  • Mistral-7B examples include a field-versus-complete token contrast and a terrain-versus-grade centroid, while other centroids contain hard-to-interpret fragments.The selected examples illustrate local readability, not population-wide readability or a single clean meaning for every parameter or cluster.Source: §B.2 · p. 19, Figure 9 · p. 20

C.1 AGGREGATE INTERVENTION SETUP AND DENOMINATORS

  • The aggregate intervention setup fixes full-Jacobian rankings and applies nested fractions of selected parameter columns across operations and scopes.Edits remain active through scoring and the 16-token continuation, then edited columns are restored exactly.Source: §C.1 · pp. 19, 21

C.2 COMPLETE CONDITIONAL DECODED SUCCESS-RATE GRID

  • Figure 10 extends the conditional decoded-success grid to all 21 models; grey cells have zero denominators and the bottom row is a matched random baseline.Source: Figure 10 · p. 21, §C.2 · p. 21

C.3 FUTURE-TOKEN PROBABILITY DIRECTION

The probability-direction event is frequent under the fixed protocol, particularly for steering at small fractions. It counts a movement in the required direction at any checked future answer-token position, not decoded success.

C.3 FUTURE-TOKEN PROBABILITY DIRECTION

  • The event counts any required-direction movement among checked future tokens and positions, without multiplicity correction, so it is not a decoded-success measure.All 3,024 rows are defined because every concept contributes to each aggregate row.Source: §C.3 · pp. 21–22, Figure 11 · p. 22
  • At Joint scope, future-token direction events reached 209/210 for ablation at 1%, 3%, and 5%; 210/210 for steering from 0.05% through 0.2%; and 198/210 for swapping at 0.05% and 0.2%.These are the highest pooled rates reported for each operation.Source: §C.3 · pp. 21–22, Figure 11 · p. 22
  • Rates are high across many models and scopes, especially for steering at small fractions, while differences across operations and scopes remain visible.Some settings approach saturation, consistent with the event being relatively easy to satisfy.Source: §C.3 · p. 22

C.4 MATCHED RANDOM-COLUMN AND SAME-NORM CONTROL

The matched random-column and same-norm control has much lower decoded success than the primary intervention. The merged material also describes qualitative intervention cases, computational details, and theoretical properties of role collapse.

C.4 MATCHED RANDOM-COLUMN AND SAME-NORM CONTROL

  • Across 144 matched settings, conditional decoded success averaged 30.82% for primary interventions versus 3.13% for controls; Figure 12 shows low control rates across scopes, with some nonzero cells.The overall difference is 27.69 percentage points from a single matched random draw, not an estimate of uncertainty across repeated controls.Source: §C.4 · p. 23, Figure 12 · p. 23

C.5 QUALITATIVE INTERVENTION CASES

  • The qualitative appendix presents 36 selected decoded cases across six models, covering Q, K, V, O, FFN, and Joint scopes.Cases retain their operation, fraction, and strength; ablation examples omit the source answer, while steering and swapping show the paired target.Source: §C.5 · p. 24, Table 11 · p. 24, Table 14 · p. 26

D ADDITIONAL DV-LENS COMPUTATION DETAILS

  • Vector–Jacobian products compute concept signals without forming the full Jacobian, and a precomputed concept-space mapping can recover repeated signals without another backward pass.Intervention scores combine concept sensitivity with input-dependent activations.Source: D ADDITIONAL DV-LENS COMPUTATION DETAILS · pp. 27–28
  • Each module shares one readout operator across its parameter columns, allowing all column readouts to be computed by matrix multiplication and then normalized for clustering.The operator is computed once per module.Source: D ADDITIONAL DV-LENS COMPUTATION DETAILS · p. 28

E THEORY OF THE DV-COMPLEXITY SCORE

  • DV-Complexity measures normalized parameter energy removed when each column is replaced by its role centroid, relating role-based compression to predictive loss.The paper presents this as a measure of loss-supported functional parameter richness.Source: E THEORY OF THE DV-COMPLEXITY SCORE · p. 28, §E.1 · p. 28

E.1 SETTING

  • The theory fixes the role labelling and centroids at trained weights and assumes finite losses, differentiability, and finite expectations for its loss analysis.The analyzed matrices are finite and nonempty.Source: §E.1 · p. 28

E.2 ENERGY REMOVED BY ROLE COLLAPSE

  • Role collapse orthogonally separates centroid weights from within-cluster residuals, yielding a normalized removed-energy score bounded by one.The residuals sum to zero within each cluster.Source: §E.2 · p. 29, These identities imply · p. 29

E.3 CROSS-ENTROPY UNDER A LOGIT CHANGE

  • The cross-entropy change under a logit change decomposes into KL divergence plus the inner product of the original prediction error and that logit change.The identity uses the original distribution p, the changed distribution q, and logit displacement h.Source: §E.3 · p. 29

E.4 EXACT LOSS DECOMPOSITION

The appendix derives exact and bounded loss consequences of replacing parameter columns by role centroids. It also characterizes the score, the conditions for a loss advantage, and properties of the readout-based labelling.

E.4 EXACT LOSS DECOMPOSITION

  • The exact loss change from role collapse is κ1 + D̄ + R, where κ1 = ω||∆||^2 − ⟨ρ,∆⟩ separates the regularization and stationarity contribution from KL divergence and the finite-logit remainder.The decomposition holds under the stated setting for every fixed labelling and ω ≥ 0.Source: §E.4 · pp. 29–30

E.5 THE SCORE AS A RATIO OF FIRST-ORDER COSTS

  • When εa < ω||Wa||F, the deletion derivative Da is positive, allowing the score to compare first-order role-collapse cost with deletion cost.The proposition bounds the matrix-level collapse-to-deletion ratio under this condition.Source: §E.5 · p. 30

E.6 FINITE ROLE-COLLAPSE COST

  • A strictly positive finite loss increase is certified when ε||∆|| + ρR is smaller than the regularization-weighted removed energy.The theorem provides a score certificate for finite role-collapse cost.Source: §E.6 · p. 31

E.7 LOSS ADVANTAGE REQUIRES ROLE-EXTERNAL ENERGY

  • A loss advantage over the best model with at most k distinct columns requires role-external energy when the advantage exceeds the stated remainder bound.The theorem’s condition is L(W) ≤ L⋆k − δ with δ > ρR.Source: §E.7 · pp. 31–32

E.8 WHAT THE DV-LENS LABELLING DETERMINES

  • Lloyd assignment and centroid updates weakly decrease the normalized-readout objective, and role centroids reproduce cluster-mean unnormalized readouts under the fixed map.For one column, the referenced scores are zero; the score is also bounded relative to the one-cluster value and a permutation reference.Source: §E.8 · pp. 32–33

F LIMITATIONS

DV-LENS examines parameter organization through downstream vocabulary behavior, which is only one view of model structure. The accompanying analyses report strong local response-direction agreement while noting differences in precision across modules.

F LIMITATIONS

  • Downstream vocabulary behavior provides only one view of internal structure, and the relation between representational structure and high-level behavior remains open across model families and training stages.Architecture, optimization, training data, and scale also shape learned representations.Source: F LIMITATIONS · p. 33

Q1. What does a DV-LENS readout describe for one parameter column?

  • A DV-LENS readout is the prompt-set average local logit response to a unit change in a parameter column’s activation coefficient.For a particular prompt, the realized first-order contribution also depends on the input-dependent coefficient.Source: Q1. What does a DV-LENS readout… · p. 33

Q2. Does the readout predict actual changes to the model output?

  • Across 720 cases from nine models, coordinate-orientation agreement is 0.980, while exact two-tail Top-32 overlap is 0.407 and Recall@128 is 0.613.The readout better captures response direction and the broader affected vocabulary region than the exact rank of every token.Source: Q2. Does the readout predict actual… · p. 34

Q3. Could the observed structure be a generic effect of random parameter directions?

  • Both excess kurtosis and top-1% energy exceed matched Gaussian-direction nulls on average in all 21 models, with the pattern persisting when each model family is removed in turn.These controls support structured vocabulary responses; finite-difference and intervention tests separately assess functional changes.Source: Q3. Could the observed structure be a… · p. 34

Q4. Do all modules provide equally precise vocabulary readouts?

  • Orientation agreement ranges from 0.947 for K to 0.997 for FFN, while exact Top-32 overlap ranges from 0.209 for K to 0.565 for FFN.Empirical coordinate precision differs across modules despite a shared local-response definition.Source: Q4. Do all modules provide equally… · p. 34

Related papers

Loading 2610.04489v1…