Values as Style: Disentangling Values from Semantics with One-Way Mixing for Low-Damage LLM Steering

1 Introduction

• Dense activation edits can shift topical content or induce refusals along with the intended normative stance.The target is to emphasize a different value while retaining the people, facts, quantities, and constraints of the scenario.Source: §1 · p. 1
• The method separates scenario-conditioned semantic content from an editable value code to redirect value emphasis while retaining underlying facts and constraints.Semantic information grounds value recognition through one-way mixing, with stop-gradient blocking feedback through that pathway.Source: §1 · p. 1
• At inference, the semantic code stays fixed while editing the value code produces a reconstruction difference injected into the residual stream.This is the paper’s residual-delta editing path.Source: §1 · p. 1, Figure 1 · p. 2
• The evaluation combines split probes, a 2 × 2 mixing-by-gating ablation, prompting and steering comparisons, fidelity metrics, and human ratings.These tests connect representation selectivity to preservation at comparable alignment.Source: §1 · p. 2

2 Related Work

• Behavioral alignment methods use feedback-based training, constitutional filtering, or preference optimization, whereas this work keeps the backbone frozen and intervenes in hidden-state space.The authors frame the settings as complementary.Source: §2 · p. 2
• Prior representation-space work extracts steering directions, suppresses interference, edits localized knowledge, or learns sparse and explicit editing spaces.The paper positions itself closest to learned intervention interfaces, while focusing on separating value-related information from semantic content.Source: §2 · pp. 2–3
• Unlike purely stylistic attributes in content–style factorization, values are partly grounded in the scenario, making fully symmetric independence objectives less suitable here.The paper therefore motivates an explicitly asymmetric design.Source: §2 · p. 3
• Prior value research includes taxonomies, stance analysis, benchmarks, and internal value vectors; this work treats value control as an interface with low topic leakage and semantic damage.Its stated goal extends beyond measuring value stance to exposing an intervention interface.Source: §2 · p. 3

3.1 Setup and Goal

• The setup distinguishes scenario-conditioned propositions and constraints to preserve from normative priorities that guide recommendations.The goal is a factorization whose semantic code minimizes value leakage and whose value code minimizes topic shortcuts, while edits minimize semantic drift.Source: §3.1 · p. 3

3.2 Latent Interface with One-Way Semantic-to-Value Mixing

• A lightweight dual encoder forms semantic and value codes, with one-way semantic-to-value mixing so context can ground value representations without reverse value-loss gradients through the bridge.The figure depicts the frozen state feeding both encoders and the semantic code augmenting the value code through the mixing path.Source: §3.2 · p. 3, Figure 2 · p. 4

3.3 Training Objective

• Training combines reconstruction, swap consistency, and regularization to preserve reconstruction, support value discrimination, suppress topic shortcuts, and encourage code independence.The factorization is operationally assessed through matched leakage probes and preservation under edits, without assuming a unique latent decomposition.Source: §3.3 · p. 4

3.4 Inference-Time Editing

• At inference, the method compares a target value code with the original while holding the semantic code fixed, then applies the resulting value-induced residual update.For stronger semantic preservation, the update can be projected off a semantic subspace and gated by a separate value-relatedness classifier.Source: §3.4 · p. 5

3.5 Training Data

• Core experiments use 630 training quadruples, while the 10K resource is studied separately in scaling analysis.The quadruples pair paraphrases expressing target and contrasting values under the same scenario; the larger resource has separate human validation.Source: §3.1 · p. 3, §3.5 · p. 5

4.1 Experimental Setup

• Across two frozen instruction-tuned backbones, operating points are validation-selected; ValueBench measures value readout while interventions test alignment and semantic preservation.ValueBench includes Relatedness and Stance tasks; steering metrics also include fluency, capability loss, benign semantic similarity, and benign false refusal rate.Source: §4 · p. 5, §4.3 · p. 6, §4.4 · p. 6

4.7 Editing Results

• The full interface improves semantic preservation and reduces benign side effects versus residual- and latent-space baselines while maintaining strong alignment on SVQ-Test and held-out prompts.The comparison uses 630 training quadruples on LLaMA-3.1-8B-Instruct.Source: Table 2 · p. 7, §4.7 · p. 7

4.8 Separating Mixing from Edit Gating

• With gating fixed, one-way mixing raises alignment 0.719→0.744 and semantic similarity 0.856→0.871; gating lowers no-mixing FRR 0.085→0.058.The matched 2 × 2 study spans LLaMA-3.1-8B and Qwen2.5-7B.Source: §4.8 · pp. 7–8, Figure 3 · p. 8

4.9 Direct Prompting and Independent Assessment

• Across backbones, paired BERTScore gain was 0.014 [95% CI: 0.008, 0.020] and contradiction-rate difference −0.025 [−0.034, −0.016] versus validation-selected P2 at similar alignment.The reported measurements assess content preservation beyond generative value judges.Source: Table 3 · p. 8, §4.9 · p. 8
• In blinded ratings, the full interface scored 4.27 [95% CI: 4.15, 4.39] for semantic preservation, above no-mixing GatedNSI (4.12), P2 (3.99), and LinearAdd (3.72).The comparison covered 320 items across both backbones, with three raters per item.Source: §4.9 · p. 8

4.10 Ablations

• One-way mixing and adversarial de-confounding reduce topic leakage in zv, while orthogonality and swap objectives stabilize alignment–damage trade-offs under stronger edits.Table 4 isolates architectural and objective components.Source: Table 4 · p. 9, §4.10 · p. 9

4.11 Disentanglement Diagnostics

• Dimension-matched probes show Topic←zv 0.190 and Value←zs 0.210 for the full interface, versus 0.421/0.603 for a random split and 0.319/0.392 for value supervision.The full interface retains value accuracy 0.820 in zv and topic accuracy 0.680 in zs, with measurable residual cross-factor information.Source: §4.11 · p. 9
• Holding zs fixed, zv edits shift alignment 0.29→0.75 with high semantic similarity; under OOD topics, semantic similarity is 0.865 and FRR is 0.051.The OOD-topic comparison reports 0.781 semantic similarity and 0.141 FRR for LinearAdd; the low-damage pattern also holds under style shifts and adversarial phrasing.Source: §4.11 · p. 9, §4.12 · p. 9

5 Conclusion

• Holding the semantic code fixed and injecting a value-induced residual delta improves content preservation at comparable alignment.The interface connects representation selectivity to generated outputs through prompting comparisons, fidelity metrics, and human assessments.Source: §5 · p. 9
• The evaluation measures operational selectivity at frozen sites without assuming a unique factorization; longer interactions and broader scales remain open.The 7B/8B core study is complemented by compact taxonomy, three-turn, and backbone checks up to 14B.Source: §5 · p. 10

A.1 Complete Transfer and Robustness Results

• Tables 5 and 6 complete the operating-point metrics and distribution-shift comparison; Table 5 covers fluency, capability, and benign-semantic preservation at Table 2 operating points.Table 5 reports LLaMA-3.1-8B-Instruct results from 630 training quadruples, averaged over three seeds.Source: Table 5 · p. 13, Table 6 · p. 13, §A.1 · p. 13

A.2 Sensitivity to Layer and Token Position

• At LLaMA ℓ=20, t=last, Align is 0.750, SemSim 0.873, and FRR 0.043; at ℓ=20, t=first, SemSim is 0.816 and FRR is 0.121.Last-token interventions in mid layers deliver the best alignment–fidelity trade-off in the sweep; earlier-token edits increase semantic collateral damage and benign refusals.Source: §A.2 · p. 13

A.3 Efficiency

• Training requires 7.4±0.3 GPU-hours per backbone and 58M trainable parameters; inference uses one residual update at a single layer/token position, without iterative optimization.The backbone remains frozen during inference.Source: §A.3 · p. 13

A.4 Qualitative Examples

• Table 7’s edits regenerate completions from scratch while keeping topical content and factual premises comparable as normative framing shifts toward the target value.Edits are applied once at the last prompt token before decoding.Source: §A.4 · p. 13, Table 7 · p. 14

A.5 Cross-Backbone Replication

• Align 0.738±0.012, SemSim 0.868±0.008, and FRR 0.047±0.006 for the method versus LinearAdd’s Align 0.761±0.011, SemSim 0.786±0.011, and FRR 0.127±0.013 on Qwen2.5-7B-Instruct.The authors interpret this replication as evidence of generalization across backbones and an improved alignment–damage trade-off.Source: §A.5 · p. 14

A.6.1 Stronger Learned-Space Baselines

• Among learned-space operating points, the full interface has the highest semantic similarity and lowest benign refusal rate, with alignment in the same range.Table 8 uses the same extraction site, decoding protocol, and validation-only α∗ selection rule as the main experiments.Source: §A.6.1 · p. 14, Table 8 · p. 15

A.6 Additional Follow-up Controls

• Table 9 extends semantic-fidelity checks across BERTScore, NLI contradiction, entity recall, and constraint retention; Table 10 shows gradual degradation under smaller, noisier supervision.The measurement controls also test false refusal on benign prompts with value-adjacent keywords.Source: Table 9 · p. 15, §A.6.2 · p. 15, Table 10 · p. 15

A.6.4 Recomposer Sensitivity

• Direct replacement can slightly raise alignment but substantially reduces semantic preservation and worsens refusal behavior, supporting the practical importance of the delta formulation.Table 11 isolates the recomposer and delta update.Source: §A.6.4 · p. 15, Table 11 · p. 16

B.1 Notation

• The method uses a semantic code and a value code as its factorization, with the appendix directing readers to Table 12 for the symbols used in the method.The paper’s method-details appendix supplies definitions omitted from the main paper for clarity.Source: §B.1 · p. 16, B Method Details · p. 16

B.2 Architecture Details

• Lightweight MLP encoders map residual states to semantic and value codes, and a lightweight recomposer maps the code pair back to residual coordinates.One-way semantic-to-value mixing grounds value recognition; stop-gradient blocks value-loss feedback through that mixing path into the semantic encoder.Source: §B.2 · p. 16

B.3 Full Loss Definitions

• Reconstruction retains information across both codes, while swap and paraphrase constraints, value supervision, topic suppression, and decorrelation shape how information is distributed.The topic adversary discourages topic information in the value code, and cross-code decorrelation penalizes covariance between the codes.Source: §B.3 · pp. 16–17

B.4 Training Procedure and Stored Artifacts

• Training freezes the backbone and optimizes the interface and auxiliary parameters; deployment stores the interface, with optional value prototypes, projection statistics, and a GatedNSI classifier and threshold.The appendix says notation used in the main method and appendix is summarized in Table 12.Source: §B.1 · p. 16, §B.4 · p. 17

B.5 Inference-Time Editing Operators

• At inference, the default operator substitutes a target-value code while holding the semantic code fixed, then adds the resulting residual delta; α = 0 leaves the state unchanged.NSI projects the delta away from a semantic subspace, while GatedNSI activates editing only above a validation-selected value-relatedness threshold; conservative projection may reduce steering strength.Source: §B.5 · pp. 17–18

B.6 Practical Considerations and Scope

• The factorization is underconstrained, so swap consistency and regularizers encourage—but do not guarantee—a unique split; claims are operational, not identifiability claims.The recomposer’s capacity and reconstruction error can shape edit geometry; the default uses one mid-layer and the last prompt token, while multi-layer edits may add coupling.Source: §B.6 · p. 18

C.1 Generation Pipeline Overview

• The pipeline anchors a scenario and target value, generates a contrasting value statement in that same scenario, creates separate paraphrases, and filters quadruples with automatic checks.The illustrative Figure 5 shows Security and Achievement as contrasting value rows, each with an original and same-stance paraphrase; candidates enter the 10K collection only after all quality checks pass.Source: §C.1 · p. 18, Figure 5 · p. 19

C.2 Stages 1–2: Context Anchoring and Opposing Value Generation

• For each target, the generator selects an anti-value from the other nine Schwartz values and writes a contrasting statement that rejects the target motive while supporting the anti-value.The generation instructions require strict JSON and statements of 15–40 words that retain the scenario context.Source: §C.2 · pp. 19–20

C.3 Stage 3: Controlled Paraphrasing

• Controlled paraphrasing changes syntax and vocabulary while preserving the original scenario context, psychological intent, and value stance.The rewrite prompt also asks writers to avoid the core value keyword and preserve stance intensity.Source: §C.3 · p. 20, §C.4 · p. 21

C.4 Stage 4: Automatic Quality Control

• Candidates are discarded for invalid structure, inconsistent scenarios or contrasts, contradictory paraphrases, or insufficient semantic similarity; retained quadruples are balanced across the 10 Schwartz values.The resulting retained resource is named SVQ-EQ-10K and is subsequently assessed through human evaluation.Source: §C.4 · p. 21

C.6 Illustrative SVQ Examples

• The appendix’s examples contrast Power with Universalism in a lifeboat decision and Security with Stimulation in a risky business-venture scenario.Each example supplies pro-target and pro-anti statements plus paraphrases for the same scenario.Source: §C.6 · p. 21, Sample 1: Power vs. Universalism · p. 21

D Human Validation of the SVQ Data

• Fifteen annotators evaluated the retained 10,000 quadruples, with each quadruple rated by exactly three independent raters.The study primarily validates the final retained data rather than serving as its main filtering mechanism.Source: D Human Validation of the SVQ Data · p. 22
• Scenario Relevance and Target Alignment each yield 120,000 ratings, while Semantic Equivalence yields 60,000 pair-level ratings.The protocol pools scores across the four statements for the first two metrics and across the two paraphrase pairs for Semantic Equivalence.Source: D Human Validation of the SVQ Data · p. 23
• The rubric separately assesses whether statements fit the scenario, support the target value, and preserve meaning and stance across paraphrase pairs.The scenario rubric checks anchors, conflict, causal compatibility, and consistency; semantic-equivalence scoring penalizes changed facts, agents, targets, or conditions.Source: D Human Validation of the SVQ Data · pp. 24–25
• Mean scores exceeded 4 across all three validation metrics, and Target Alignment had Krippendorff’s α = 0.8951, indicating high rater consensus on the target–contrast distinction.The reported means and variances characterize score distributions; the agreement coefficients concern structured SVQ data validation, not edited completions.Source: D Human Validation of the SVQ Data · p. 25

E.1 ValueBench protocol

• ValueBench evaluation samples 2,000 prompts stratified over values and difficulty and reports macro-F1 for Relatedness and Stance.Relatedness tests value relevance to a response given the situation; Stance tests whether the response supports or opposes that value.Source: §E.1 · p. 25

E.2 Alignment judge specification and calibration

• Judge calibration reports stance Cohen’s κ of 0.61, 0.58, and 0.54 for the three judge pairs, with prompt-paraphrase stance flip rates of 3.6%, 5.1%, and 4.7%.The judges are GPT-4o, Kaleido, and ValueLlama; they produce relevance and stance labels under a shared rubric and structured output format.Source: §E.2 · pp. 25–27

E.3 Semantic fidelity metrics

• Preservation is assessed with BERTScore, NLI-based contradiction rate, and entity and constraint retention relative to the unedited response.A pair is marked contradictory when the NLI model’s contradiction probability exceeds 0.5; numeric constraints are counted as preserved within a 5% tolerance.Source: §E.3 · p. 27

E.4 Benign false refusal rate (FRR) and “hard benign” set

• The benign set samples general instruction prompts and filters for low value relevance, while a separate hard-benign subset uses value-adjacent keywords without requesting normative judgments.False refusal rate is the fraction of edited outputs judged to be refusals.Source: §E.4 · pp. 27–28

E.5 Human evaluation on edited outputs

• A human evaluation sampled 180 LLaMA-3.1-8B edited completions for ratings of value alignment, semantic preservation, and perceived refusal or unhelpfulness.The text characterizes output-level agreement as moderate or modest depending on criterion; the separate blinded 320-item comparison is in Appendix G.Source: §E.5 · p. 28, Table 14 · p. 28

F.2 Baseline implementation details

• The baseline implementations include logit steering, SAE-code steering, YaPO sparse steering, and a residual rerouter, evaluated under the shared protocol.SWAI’s steering strength is swept to match the Align range of other baselines.Source: §F.2 · pp. 28–29

F.1 Intervention-site sensitivity: layer and token position

• At LLaMA-3.1-8B’s default site (layer 20, last prompt token), Align is 0.750, SemSim is 0.873, and FRR is 0.043; first-token intervention lowers SemSim to 0.816 and raises FRR to 0.121 at comparable alignment.Across both backbones, last-token mid-layers yield the best reported alignment–fidelity trade-off; top-2-layer editing gains about 0.01 alignment but raises FRR about 0.01 and capability drop about 0.2 MMLU.Source: §F.1 · p. 28, Table 15 · p. 29

F.3 Gate activation statistics

• On LLaMA-3.1-8B, mean gate activation is 0.222 on value-relevant prompts versus 0.144 on value-irrelevant prompts, and gate mass correlates with value-relatedness at Spearman ρ = 0.53.At threshold 0.4, an average 0.122 of dimensions are active, rising to 0.179 on relevant prompts and falling to 0.093 on irrelevant prompts.Source: §F.3 · p. 30

F.4 Reconstruction error vs semantic drift

• On LLaMA-3.1-8B-Instruct, reconstruction error correlates with semantic drift at r = 0.51 on value-relevant prompts and r = 0.27 on value-irrelevant prompts.Reconstruction quality is not the sole driver of drift, but large errors can flag brittle edits.Source: §F.4 · p. 30

F.5 Interpreting Qualitative Edits

• A successful qualitative edit retains factual anchors and task constraints while changing normative framing; altered entities, quantities, causal premises, or unsupported refusals count as damage.The protocol edits before decoding and regenerates the full completion, so it does not imply invariance of a generated prefix.Source: §F.5 · p. 31, §G.1 · p. 31

G Controlled Comparisons and Transfer

• The controlled comparisons isolate interface and inference-operator roles, compare representation editing with direct prompting, and use complementary preservation measures and transfer settings.Source: G Controlled Comparisons and Transfer · p. 31

G.1 Data accounting and common comparison protocol

• Default controlled comparisons use 630 scenario quadruples; the 10,000-quadruple collection is a separate scaling condition, with scenario-disjoint splits.Preservation covers scenario-conditioned anchors—entities, facts, quantities, causal relations, constraints, and topic—while changes to those anchors or unsupported benign refusals count as damage; operating points are selected on validation data.Source: §G.1 · p. 31

G.2 Separating one-way mixing from inference gating

• With GatedNSI fixed, one-way mixing raises pooled alignment by 0.025 and semantic similarity by 0.015; inference gating reduces benign refusals.The full configuration combines these benefits on both backbones.Source: Table 17 · p. 32, §G.2 · p. 32

G.3 Matched controls for code selectivity

• The full interface retains high within-code predictability and reduces cross-code predictability versus matched controls, though off-diagonal accuracies remain above chance.Raw residual-state probes predict value/topic at .851/.744 on LLaMA and .842/.732 on Qwen.Source: §G.3 · p. 32, Table 18 · p. 33

G.4 Direct prompting at comparable alignment

• At comparable alignment (n = 2,000), the P2 comparison improves BERTScore by .014 and yields contradiction-rate difference −.025.The paired comparison reports a 95% CI of [.008, .020] for BERTScore and [−.034, −.016] for the contradiction-rate difference; preservation gains require no additional prompt tokens.Source: Table 19 · p. 33, §G.4 · p. 33, Table 20 · p. 34

G.5 Agreement across evaluation methods

• One-way-versus-no-mixing improvement is positive under each of three stance evaluators; BERTScore, NLI, and entity/constraint retention provide preservation measures independent of generative stance judgments.Source: §G.5 · p. 34, Table 21 · p. 34

G.6 Human output ratings and inter-rater agreement

• Semantic-preservation agreement ranges from .42 to .55, indicating meaningful disagreement; mean intervals quantify uncertainty, not high inter-rater agreement.The earlier 180-item output study is distinct from both the expanded output comparison and the 10K quadruple-validation study.Source: §G.6 · pp. 34–35
• On 240 held-out items with non-GPT prompts, the full method’s preservation score is 4.13 and its paired difference from LinearAdd is +0.44.The blinded three-rater comparison reports 95% CIs of [4.00, 4.26] and [0.26, 0.62], respectively.Source: §G.5 · p. 34, Table 22 · p. 35

• Increasing the fixed reference budget for six Moral Foundations labels improves alignment and preservation while reducing benign refusals, without retraining the frozen interface.The test includes 240 prompts, with target codes built from one, 20, or 50 reference prompts per label.Source: §G.7 · pp. 35–36
• In 150 three-turn dialogues, per-turn edit reapplication yields higher alignment than a first-turn-only edit and retains more semantic content than a persistent prompt at nearby alignment.The edit is reapplied at each turn; the results do not imply a persistent change to the frozen model.Source: §G.7 · pp. 35–36, Table 23 · p. 36
• On Mistral-7B-Instruct, the fixed-depth method scores alignment .728, semantic similarity .862, BERTScore .929, contradiction rate .061, and FRR .052; Qwen2.5-14B preserves favorable semantic-similarity and benign-refusal ordering.The Mistral check fixes the intervention near 0.6L; the Qwen2.5-14B comparison uses 1,000 SVQ-Test prompts.Source: Table 23 · p. 36, §G.7 · p. 36