PaperlayerView exact revision on arXiv

Source-linked AI summary

Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs

Saurav Bhandari, Benjamin Wade

arXiv:2609.06473v1cs.LG

TL;DR

Whether post-training quantization preserves activation steering remains an open question despite the techniques’ convergence in deployment. The paper compares steering across weight-only precisions at matched behavioral effects and finds near-zero sentiment costs but asymmetric reasoning-length failures.

  • Problem

    Activation steering research typically assumes full-precision inference, leaving open whether quantization preserves steering-relevant representations, calibration, and matched-effect capability costs.

    Source: §1 · p. 2

  • Method

    The study compares activation steering across weight-only quantization conditions at matched behavioral effects and uses a censored construction for floor-bounded targets.

    Source: §3.1 · p. 4, §5 · p. 13

  • Results

    Pooled INT8 contrast was −0.010 (90% CI [−0.026, +0.007]); sentiment costs stayed near zero, while reasoning-length steering showed asymmetric failure.

    Source: §1 · p. 2, §5 · p. 13

  • Takeaways & Limitations

    At 7–9B scale, quantization is largely benign for sentiment steering costs, but length control requires monitoring per-alpha failure rates.

    Source: §5 · p. 13

    The study is a descriptive k = 3/k = 4 characterization of 7–9B models, limited to bitsandbytes weight-only quantization and one steering method and site per model.

    Source: §6 · p. 14

Abstract

from arXiv · show

Inference-time activation steering enables behavioral control of large language models without parameter modification, while post-training quantization reduces memory and compute costs for deployment. Despite their growing convergence in practice, the interaction between these two techniques remains uncharacterized. We systematically study activation steering under weight-only quantization (INT8 and NF4) across four open-weight 7-9B models and two behavioral targets: judged sentiment and judge-free reasoning length. Using an iso-effect framework that compares capability costs at matched behavioral effect, we find that sentiment steering survives quantization intact. After correcting a GSM8K parser artifact with a uniform v2.3.1 rescore, the pooled INT8 contrast is -0.010 (90% CI [-0.026, +0.007]), descriptively Equivalent under the preregistered three-label rule, while NF4 remains Inconclusive at -0.017 ([-0.067, +0.033]). In contrast, reasoning length exhibits a surprising asymmetric dose-response: lengthening is graded but terminates in cap-runaway and collapse, while shortening is a step function with only 12-30% shortening (model-dependent) before discontinuous failure. We expose a methodological pitfall: the naive iso-effect ladder anchors on the collapse floor for floor-bounded targets, and we introduce a censored construction that restores interpretable crossings. We also quantify a substantial baseline capability shift for Mistral-NF4 (0.545 to 0.365 GSM8K at alpha=0), demonstrating that compression can dominate the steering intervention. Despite this, steering vectors remain highly collinear with their FP16 siblings (cosine similarity 0.989-0.998 for INT8, 0.945-0.990 for NF4), confirming that the behavioral direction survives quantization even when the cost structure does not. All code and data are released.

1 Introduction

The paper asks whether post-training quantization preserves activation steering and characterizes capability costs at matched behavioral effects. Sentiment costs remain near zero in the current pool, while reasoning-length steering has asymmetric failure behavior.

1 Introduction

  • Quantization and activation steering increasingly intersect in deployment, but whether compression preserves steering-relevant representations and matched-effect capability costs remains unanswered.Quantization can alter the numerical computations that generate representations, potentially changing steering calibration or capability costs even when benchmark accuracy is retained.Source: §1 · p. 2
  • The study systematically characterizes activation steering under weight-only quantization across multiple models, quantization schemes, and behavioral targets.Activation steering injects vectors into intermediate hidden representations without modifying model parameters.Source: §1 · pp. 1–2
  • The pooled INT8 sentiment contrast is −0.010 (90% CI [−0.026, +0.007]), descriptively Equivalent, while pooled NF4 is −0.017 (90% CI [−0.067, +0.033]), Inconclusive; reasoning-length steering fails asymmetrically.The sentiment result follows a uniform v2.3.1 rescore correcting a parser artifact.Source: §1 · p. 2
  • The naive E∗ ladder can anchor on the failure floor for floor-bounded targets; the paper introduces a censored construction that excludes high-failure regions before deriving effect levels.Source: §1 · p. 2
  • Baseline capability shifts are measured separately from steering costs, and steering vectors remain highly similar between FP16 and quantized conditions.The paper argues that iso-effect costs must be interpreted relative to each quantization condition’s unsteered capability.Source: §1 · pp. 2–3
  • The reported sentiment pool includes three models, and length includes four; the broader preregistered K = 12 confirmatory equivalence result is not reported here.Sycophancy and truthfulness data collection is ongoing.Source: §1 · p. 3

2 Related Work

Related work treats activation steering and post-training quantization largely as separate areas, with limited systematic comparisons of steering across quantized and full-precision models. Mechanistic findings about influential internal structures motivate testing whether quantization preserves steerability.

2 Related Work

  • Steering research focuses on manipulating internal representations, while quantization research reduces computational cost while preserving predictive performance; the areas have largely developed independently.Source: §2 · p. 3

2.1 Activation Steering

  • ActAdd [1], RepE [2], CAA, and ITI exemplify activation-space steering, while prior work examines capability costs and steering-related adaptation.Task arithmetic extends the geometric view to weight space; other work frames steering as post-training adaptation and surveys mechanistic methods [19].Source: §2.1 · p. 3

2.2 Mechanistic Foundations

  • Massive activations [8], super weights [9], and outlier dimensions [4] identify influential internal structures; whether accuracy-oriented protection preserves behavioral steerability remains empirical.Source: §2.2 · p. 3

2.3 Post-Training Quantization

  • Post-training methods reduce memory or inference costs through distinct approaches, and this study restricts its experiments to weight-only quantization with higher-precision activations.SmoothQuant addresses weight-plus-activation quantization, while QuaRot [21] uses rotations to redistribute activation energy.Source: §2.3 · pp. 3–4

2.4 Activation Steering under Quantization

  • Prior studies of steering under quantization lack a matched full-precision comparison or examine weight steering; this study holds the steering method and target fixed across precision conditions.The comparison evaluates capability at matched behavioral effect.Source: §2.4 · p. 4

2.5 Benchmarks and Evaluation

  • GSM8K is the primary generative capability probe and MMLU the secondary dose-blind contrast; TruthfulQA and ARC are named for later work but not evaluated here.Source: §2.5 · p. 4

3 Methods

The study tests whether weight-only quantization changes inference-time activation steering, using controlled precision conditions and capability comparisons at matched behavioral effect. Each precision condition extracts and applies its own steering vector, while the iso-effect levels are anchored to the FP16 sibling.

3.1 Experimental Design

  • Models are evaluated across FP16, INT8, and NF4, with each deployed precision extracting and applying its own steering vector before capability is measured at matched behavioral effect.The FP16 sibling fixes the iso-effect arm and levels for the quantized runs.Source: §3.1 · p. 4

3.2 Models and Quantization Schemes

  • Four open-weight instruction-tuned models spanning three families and approximately 7–9B parameters use FP16, 8-bit W8A16, and 4-bit W4A16/NF4; Gemma participates only in the length target, leaving sentiment with three models.Gemma was excluded from the judged-sentiment campaign because matching behavioral effect required a steering-coefficient grid roughly an order of magnitude wider than for the other three models.Source: §3.2 · p. 6

3.3 Behavioral Targets

  • Sentiment uses a contrastive-pair CAA direction and RoBERTa scoring, while length uses verbose–terse pairs and survivor-median token counts under a fixed reasoning template.Length failures are excluded from the median but counted in a co-primary failure-rate endpoint; the length target uses about 10 prompts per α.Source: §3.3.2 · pp. 6–7, §3.3.1 · p. 6

3.4 Capability Probe

  • GSM8K capability is measured on 200 items with greedy decoding and a 512-token cap; MMLU is secondary because last-position steering made it dose-blind.Capability is probed at baseline and around an online efficacy crossing rather than at every efficacy-grid point.Source: §3.4 · p. 7

3.5 Iso-Effect Capability Cost and the E∗Ladder

  • The iso-effect cost compares unsteered and steered capability at a crossing where behavior reaches a preregistered fraction of its extreme, accepting the first reliable level in the 0.5, 0.4, 0.3 ladder.Levels are selected from the FP16 sibling, while each precision’s crossing is located on its own efficacy curve.Source: §3.5 · p. 7

3.5.1 E∗Censoring for Floor-Bounded Targets

  • For floor-bounded length, α values with failure rates above 0.10 are excluded before deriving levels or crossings, preventing the collapse floor from setting the target.The maximum surviving effect on the restricted FP16 curve sets the extreme; excluded regions and their failure rates are reported separately.Source: §3.5.1 · p. 7

3.6 Resampling and Aggregation

  • Five data-level resamples use fresh contrast pairs and prompt and capability samples; within-run uncertainty and between-run steering-vector variability are incorporated in the run-level combination.Confidence intervals use a t distribution with k−1 degrees of freedom, where k is five or four if a run is excluded.Source: §3.6 · p. 8, §3.6.1 · p. 8

3.6.2 Cross-Model Pooling; 3.7 Decision Rule; 3.8 Preregistration and Deviations

  • Cross-model pooling is gated by heterogeneity thresholds, and the three-label rule classifies pooled contrasts as Equivalent, Meaningfully Worse, or Inconclusive; this paper reports descriptive k=3 sentiment and k=4 length results.The preregistered confirmatory pool is K=12; the documented deviations include the GSM8K probe and censored length analysis.Source: §3.7 · p. 8, §3.6.2 · p. 8, §3.8 · p. 9

4 Results

Across sentiment and reasoning-length targets, quantization leaves matched-effect sentiment costs near FP16 but preserves a sharply asymmetric length failure curve. For length, the naive iso-effect ladder anchors on the collapse floor, while censoring restores cleaner crossings.

4.1 Sentiment: steering survives weight-only quantization at matched effect

  • −0.010 (90% CI [−0.026, +0.007]) is the pooled INT8 sentiment contrast, descriptively Equivalent; NF4 remains Inconclusive at −0.017 ([−0.067, +0.033]).All capability scores use the audited v2.3.1 parser; per-model contrasts are Inconclusive.Source: §4.1 · p. 9, Figure 2 · p. 10

4.2 Length: asymmetric dose–response with a termination cliff

  • Length steering is asymmetric: positive doses rise gradually into cap-runaway and failure, while negative doses yield only 12–30% shortening before abrupt empty-output collapse.Clean windows span ±40 for Qwen and ±200 for Gemma, but only roughly [−6, +8] for Llama and [−4, +4] for Mistral.Source: §4.2 · p. 10, Figure 3 · p. 11

4.3 The iso-effect rule breaks on a floor-bounded target

  • The naive E∗ ladder anchors on the collapse floor for three models, placing all f = 0.5 crossings across failure cliffs; censoring alphas above 10% failure restores clean crossings.At the 10% rule, FP16 acceptance rises to 3–5/5 resamples, but revised length IECC remains descriptive because acceptance is incomplete and some crossings are interpolation-limited.Source: §4.3 · pp. 11–12, Figure 4 · p. 12

4.4 Quantization moves the baseline itself – in one model

  • 0.545 (FP16) to 0.365 (NF4) is Mistral’s GSM8K accuracy shift at α = 0, an eighteen-point loss before steering.The shift also replicates in the sentiment campaign, where Mistral moves from 0.544 to 0.351 across FP16 and NF4.Source: §4.4 · p. 12, Figure 5 · p. 13

4.5 The steering direction survives quantization

  • Cosine similarity remains 0.989–0.998 for INT8 and 0.945–0.990 for NF4 against FP16 vectors, with Llama most and Gemma least quantization-sensitive.The cosine ordering across schemes held in all 21 comparisons; the vectors are unit-normalized, so the measure captures direction rather than magnitude.Source: §4.5 · p. 13

5 Discussion

Sentiment steering costs remain stable under quantization at matched effects, but reasoning-length steering has abrupt, asymmetric failure boundaries. The discussion also identifies a censored iso-effect construction and warns that compression can shift unsteered capability.

5 Discussion

  • At matched behavioral effect, INT8 and NF4 sentiment-steering costs are indistinguishable from FP16 after the parser-artifact correction.The pooled INT8 contrast is descriptively Equivalent under the three-label rule.Source: §5 · p. 13
  • Reasoning-length steering has an asymmetric dose-response: shortening fails discontinuously, while lengthening progresses to cap-runaway and collapse.Clean windows vary by nearly two orders of magnitude across models, and quantization does not rescue the cliff.Source: §5 · p. 13
  • For floor-bounded targets, a maximum-effect-fraction iso-effect anchor can land on the failure floor rather than the behavioral peak.The censored construction restricts to alphas with failure ≤10%, anchors on the maximum surviving effect, and reports the contaminated zone separately.Source: §5 · p. 13
  • Mistral-NF4’s baseline shift warns that compression can move unsteered capability enough to dominate the steering effect.Iso-effect comparisons are interpretable only when read jointly with the unsteered baseline.Source: §5 · p. 14

6 Limitations

The study is a descriptive characterization with limited model coverage, quantization methods, steering setup, and evaluation scope.

6 Limitations

  • The study covers k = 3 sentiment or k = 4 length model families in the 7–9B range, with imprecise cross-model heterogeneity estimates and the confirmatory K = 12 call deferred.The authors characterize the results as descriptive rather than the preregistered K = 12 confirmatory pool.Source: §6 · p. 14
  • Only bitsandbytes weight-only INT8 and NF4 are studied; activation quantization and outlier-aware weight algorithms are excluded and may behave differently.Source: §6 · p. 14
  • The study uses one steering method, CAA, at a single site and layer per model.Source: §6 · p. 14
  • Length efficacy resolution is coarse relative to its capability axis, length capability-cost reporting is one-armed, and all evaluation is in English.Source: §6 · p. 14

Reproducibility Statement

The paper releases its result files, steering vectors, preregistration materials, deviation notes, and analysis code.

Reproducibility Statement

Ethics Statement

The paper studies capability and reliability trade-offs of activation steering under deployment compression.

Ethics Statement

  • The authors report no direct dual-use concern beyond those already inherent to activation steering itself.Source: Ethics Statement · p. 14

A Deviation Ledger

The deviation ledger records dated design changes, with design choices ratified before the relevant data were seen or acted upon. These changes added a separately tracked reasoning-length target, revised the capability probe and analysis, and documented parser and model decisions.

A Deviation Ledger

  • The ledger states that deviations were dated when decided and design choices were ratified by both authors before relevant data were seen or acted upon.Source: A Deviation Ledger · p. 14
  • The preregistration began with weight-only and activation-quantization designs, four judged targets, and a K = 16 confirmatory pool, then dropped refusal and set K = 12 with length tracked separately.The revised pool covered four models and three judged targets.Source: A Deviation Ledger · p. 14
  • Reasoning length was added as a judge-free target, and its preregistered design set N = 200 per condition, δ = 3%, ∆min = 10%, a three-label H1 rule, and Rule A for H2.Pooling was required because single-cell equivalence power was approximately zero at realistic sample sizes.Source: A Deviation Ledger · p. 14
  • Single-pass MMLU was deemed dose-blind under site=last steering, so generative GSM8K exact-match with a 512-token cap became the primary capability probe.Single-pass MMLU remained as a secondary methodological contrast.Source: A Deviation Ledger · p. 14
  • Budget-driven changes specified adaptive capability-α selection and batch-16 GSM8K generation, alongside frozen Option C run-level aggregation using REML τ 2 and modified Knapp-Hartung.Source: A Deviation Ledger · p. 14
  • The GSM8K parser was corrected to v2.3 and later patched as v2.3.1, with the latter adding a non-finite-overflow guard and matching v2.3 on the shared domain.The v2.3 correction was applied to subsequent collection; the offline rescore of collected sentiment cells occurred later.Source: A Deviation Ledger · pp. 14–15
  • Gemma-2-9B-it was added as the fourth model for the length target only, instead of adding a 70B model.Source: A Deviation Ledger · p. 15
  • A later censored E∗ sensitivity re-analysis treated all-failure α values as missing rather than as zero-effect measurements and was disclosed as a new deviation.Source: A Deviation Ledger · p. 15

B GSM8K Parser Correction (v2.3.1 Rescore)

The v2.3.1 correction changed GSM8K answer parsing for specific patterns, and the offline audit validated its flip set and rescored results. The audit found no missing or extra flips and no downgrades of answers originally scored correct.

B GSM8K Parser Correction (v2.3.1 Rescore)

  • The v2.3 parser accepted a trailing bare final number when prior rules failed, while v2.3.1 added a non-finite-overflow guard; both changes were applied uniformly to all 45 sentiment cells.Scores are identical outside the specific patterns targeted by the changes.Source: B GSM8K Parser Correction (v2.3.1… · p. 15
  • The 1,644 items whose scores flipped were reviewed against original text, rescored text, and gold answers before acceptance.Source: B GSM8K Parser Correction (v2.3.1… · p. 15
  • The offline rescore matched the reference flip list exactly, with 0 missing and 0 extra flips, and every item originally scored correct re-verified without downgrades.Option C on the original files reproduced the first-look numbers to the fourth decimal.Source: B GSM8K Parser Correction (v2.3.1… · p. 15

D Additional Materials

The paper releases the materials needed to inspect its collected dose-response data and reproduce its analyses.

D Additional Materials

  • The release includes dose-response small multiples for all 105 collected cells, the extracted steering vectors, and code for the E∗ ladder, Option C combine, and DerSimonian-Laird pooling.The 105 cells comprise 45 sentiment and 60 length cells.Source: D Additional Materials · p. 15

Related papers

Loading 2609.06473v1…