PaperlayerView exact revision on arXiv

Source-linked AI summary

When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety

Tianyi Guan, Jianhui Chen, Liangming Pan

arXiv:2609.34771v1cs.AIcs.CLcs.LG

TL;DR

Behavioral safeguards and representation engineering address complementary needs, but their relative strengths are unclear because they are often evaluated under different settings. The paper compares them across safety control and monitoring and finds that their practical advantages depend on the task and operating conditions.

  • Problem

    The relative roles of behavioral safeguards and representation engineering remain unclear because the approaches are rarely compared side by side under matched safety settings.

    Source: §1 · pp. 1–2

  • Method

    The paper conducts matched evaluations of behavioral alignment and representation steering for control, text monitors and representation probes for monitoring, and monitor-guided interventions.

    Source: §1 · p. 2, §3 · p. 3, §5 · p. 8

  • Results

    DPO provides the strongest overall control and text monitors stronger overall detection, while steering can compete with limited high-quality data, probes offer lower marginal cost, and probe-guided interventions recover much DPO safety lost after benign fine-tuning.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Representation engineering is best treated as a complement to behavioral safeguards, with advantages in low-data control, efficient native monitoring, and monitor-guided intervention when model activations are available.

    Source: §1 · p. 2, §6 · p. 10

    The evaluated benchmarks and automated judgments cover only a limited set of deployment risks and do not certify any model as safe.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Reliable AI safeguards require both control mechanisms that reduce unsafe behavior and monitoring mechanisms that detect safety risks during model interactions. Established behavioral safeguards include alignment methods that optimize model outputs and text monitors that assess interaction text. Representation engineering instead reads or modifies internal model states, but the relative strengths of these approaches remain unclear because they are often evaluated under different settings. We present a matched evaluation across two tracks. For safety control, we compare DPO, a behavioral alignment method, with three representation steering methods across robustness, practicality, and granularity. DPO provides the strongest overall control and generally improves with increasing training data, although its safety can degrade after subsequent benign fine-tuning. Representation steering remains competitive primarily in low-data settings, particularly with high-quality contrastive data. For safety monitoring, we compare representation probes with fine-tuned and open-weight text monitors across full-response detection, early detection, and computational cost. Specialized text monitors achieve the strongest overall detection accuracy, while representation probes remain competitive at substantially lower marginal cost. Finally, monitor-guided interventions recover much of the safety lost by DPO after benign fine-tuning, with little additional over-refusal. Overall, representation engineering does not generally replace behavioral safeguards, but offers practical advantages under specific conditions and can provide complementary safety benefits.

1 INTRODUCTION

The paper asks whether internal representations can practically complement or substitute for behavioral safeguards, a question obscured by evaluations conducted under different settings. It compares these approaches side by side for safety control and monitoring, and finds representation engineering useful under particular conditions rather than as a wholesale replacement.

1 INTRODUCTION

  • The paper frames reliable safeguards as needing both control that reduces unsafe behavior and monitoring that detects risks, while most safeguards operate on model behavior rather than internal representations.Representation steering modifies activations; representation probes read them for safety-relevant states.Source: §1 · p. 1
  • Because steering, probes, and text monitors are usually evaluated separately or under different settings, their relative roles in safety systems remain unclear.The open questions include when internal methods can substitute for behavioral methods and when they can strengthen them.Source: §1 · pp. 1–2
  • In matched comparisons, DPO provides the strongest overall control and scales with training data, while flow-based steering can compete in low-data settings with high-quality contrastive data.Flow's efficiency can come with increased over-refusal and capability degradation.Source: §1 · p. 2
  • Text monitors achieve the highest overall detection accuracy, while representation probes remain competitive at lower marginal cost; rolling aggregation gives the most stable streaming detection.The comparison considers detection reliability, timeliness, and computational overhead.Source: §1 · p. 2
  • Probe-guided interventions recover much of the safety lost by DPO after benign fine-tuning, with minimal added over-refusal.Post-generation blocking is the most reliable integration strategy, while corrective regeneration is highly effective for larger, more capable models.Source: §1 · p. 2
  • The paper concludes that representation engineering offers targeted practical value in data-constrained control, efficient native monitoring, and internal signals that fortify existing alignment methods.It is not presented as a wholesale replacement for established behavioral safeguards.Source: §1 · p. 2

2 RELATED WORK

Prior work studies behavioral alignment and representation steering for control, and text monitors and representation probes for monitoring. The paper identifies a gap in controlled comparisons of these approaches and in evidence about whether internal signals can improve downstream control.

2 RELATED WORK

  • Prior work distinguishes behavioral alignment, which shapes outputs through post-training, from representation steering, which intervenes directly on internal activations.Studies cover activation-difference steering, refusal directions, side effects, activation transport, and comparisons with prompting or fine-tuning.Source: §2 · p. 2
  • Existing comparisons leave a gap: control studies do not jointly compare steering with behavioral alignment across key dimensions, and probes and text monitors are rarely compared in a shared safety setting.Whether representation-probe signals can improve downstream control remains underexplored.Source: §2 · pp. 2–3
  • Text monitors classify risks from visible interactions, whereas representation probes read internal activations and can approach larger monitors at low marginal cost when activations are reused.Related work also studies early prediction from chain-of-thought activations and probe trajectories.Source: §2 · p. 2

3 SAFETY CONTROL

In a matched evaluation of safety control, DPO generally outperformed representation steering, while Flow remained competitive with limited high-quality data. DPO’s safety could weaken after benign fine-tuning, and steering performance varied with data quality and evaluation scope.

3.1 METHODS AND EXPERIMENTAL SETUP

  • The comparison covered DPO and three steering methods—CAA, probe-based steering, and Flow—on three instruction-tuned models, using matched PKU-SafeRLHF data for robustness and practicality tests.The separate data-scaling study varied supervision source and amount.Source: §3.1 · p. 3

3.2 EVALUATION DIMENSIONS

  • Control was assessed for jailbreak and benign-fine-tuning robustness, capability and over-refusal costs, and transfer across general, cybercrime, physical-harm, and toxicity scopes.The evaluation also varied safety-data quantity and quality.Source: §3.2 · pp. 3–4

3.3 MAIN RESULTS

  • DPO had the lowest ASR under both attacks before and after benign fine-tuning, but its average ASR increase was 0.271.Flow degraded less than DPO under AIM, but not under refusal suppression; its persistence also depended on the steering layer.Source: §3.3 · p. 4
  • Flow had the highest over-refusal, rising from 0.281 to 0.308, while post-update capability macro-averages were comparable for Flow and Base (0.722 vs. 0.721).DPO’s over-refusal fell from 0.277 to 0.175 after fine-tuning, and its initial capability gaps moved closer to Base scores.Source: §3.3 · pp. 4–5, Table 2 · p. 4
  • DPO improved as training data grew across both sources, whereas steering gained little from quantity; Flow could match or outperform DPO with small, high-quality contrastive subsets.Filtered contrastive pairs produced stronger DPO results than original preference data.Source: Figure 1 · p. 5, §3.3 · p. 5
  • DPO showed the strongest cross-domain transfer, especially from domain-specific training to general safety; Flow transferred best from general safety to individual domains.CAA and probe steering transferred poorly and could reduce general-safety performance after domain-specific training.Source: Figure 1 · p. 5, §3.3 · p. 5
  • DPO provided the strongest overall control under matched supervision; Flow was a practical low-data option, while CAA and probe steering offered limited protection.The results do not support representation steering as a general replacement for behavioral alignment.Source: §3.3 · pp. 4–6

4 SAFETY MONITORING

This section compares representation probes with text monitors on matched trajectories, evaluating full-response detection, detection timeliness, and computational cost. Text monitors lead overall, while probes offer competitive detection at lower marginal cost when they reuse native activations.

4.1 METHODS AND EXPERIMENTAL SETUP

  • The matched evaluation compares four representation probes with two text monitors on held-out trajectories generated by Qwen2.5-32B-Instruct.Probes read layer-48 hidden states, while text monitors receive interaction text.Source: §4.1 · p. 6, §4 · p. 6

4.2 EVALUATION DIMENSIONS

  • The evaluation measures full-response discrimination, warning timeliness with realized safe-sequence FPR, and marginal computation for continuous monitoring.Timeliness metrics do not establish whether an alarm precedes harmful content; native probes reuse generation activations, whereas text-monitor cost includes an additional forward pass.Source: §4.2 · pp. 6–7

4.3 MAIN RESULTS AND INTERPRETATION

  • Qwen3Guard achieves 0.996 AUROC and ties FT-LLM at 0.995 AUPRC; mean pooling reaches 0.982 AUROC and 0.978 AUPRC.At Cal.-5%, mean pooling has 0.986 TPR with 0.190 FPR, versus Qwen3Guard's 0.968 TPR and 0.056 FPR.Source: §4.3 · p. 7
  • Qwen3Guard reaches 0.948 streaming recall and median first-alarm position 0.034, versus FT-LLM's 0.575 and 0.158; sequence-level FPR is 0.058 versus 0.163.Qwen3Guard's advantage over FT-LLM is most pronounced in streaming detection rather than full-response discrimination.Source: §4.3 · p. 7
  • Rolling reaches 0.913 streaming recall, median alarm position 0.143, and sequence-level FPR 0.017, offering the strongest overall balance among probes.Mean pooling has higher full-response AUROC and AUPRC than rolling (0.982 and 0.978 versus 0.971 and 0.966), but higher realized FPR at Cal.-5% (0.190 versus 0.122).Source: §4.3 · p. 8
  • Native probes reuse hidden states with lightweight aggregation; Figure 2 reports Qwen3Guard requiring approximately 7.6 × 10^5 times the rolling probe's marginal FLOPs.Replaying responses through a separate model to obtain activations requires an additional forward pass.Source: Figure 2 · p. 7, §4.3 · p. 8
  • Text monitors lead overall detection, while native-activation probes remain competitive at substantially lower marginal cost.Among the evaluated probes, rolling provides the strongest streaming performance and lowest sequence-level FPR.Source: §4.3 · p. 8

5 MONITOR–CONTROL INTEGRATION

The section tests whether monitor signals can guide interventions against unsafe responses before and after benign fine-tuning. Blocking consistently reduces attack success, while regeneration is more model-dependent and stronger Flow steering is not consistently beneficial.

5 MONITOR–CONTROL INTEGRATION

  • The study replays completed interactions through Qwen2.5-32B-Instruct for rolling-probe scoring and tests blocking, safe rewriting, and stronger Flow steering.The target models are drawn from the control experiments because training DPO on Qwen2.5-32B-Instruct is computationally expensive.Source: §5 · p. 8
  • At the 5% calibration threshold, blocking reduces post-update DPO mean ASR from 0.500 to 0.042; across conditions, blocking also lowers ASR for both DPO and Flow.Figure 4 compares mean ASR and over-refusal against unmonitored DPO and Flow baselines as the alarm threshold varies.Source: §5 · pp. 8–9, Figure 4 · p. 9
  • At the 5% threshold, corrective regeneration matches blocking's post-update DPO ASR of 0.042, with over-refusal 0.120 versus blocking's 0.124.Regeneration raises over-refusal from 0.108 to 0.120 and is more model-dependent than blocking, though effective in the larger-model setting.Source: §5 · pp. 8–9
  • Blocking lowers DPO ASR from 0.058 to 0.021 before fine-tuning and from 0.500 to 0.042 afterward; Flow falls from 0.157 to 0.038 and 0.639 to 0.056, respectively.Across additional target models, blocking consistently reduces ASR, whereas stronger Flow steering is non-monotonic.Source: §5 · p. 9

6 DISCUSSION AND CONCLUSION

The discussion concludes that representation engineering has conditional practical value rather than replacing behavioral safeguards. It highlights low-data control, efficient native monitoring, and monitor-guided recovery, while noting limits on seeds, attacks, and deployment-risk coverage.

6 DISCUSSION AND CONCLUSION

  • Probe-guided blocking and regeneration recover substantial DPO safety lost after benign fine-tuning, suggesting monitoring may preserve safety without another safety-alignment round.The replay-based integration study and native-activation results together indicate a potential low-cost route to monitor-guided intervention.Source: §6 · p. 9
  • Some control experiments use a single training seed, and the jailbreak evaluation uses non-adaptive attacks that do not assess tailored attackers.The authors also caution that the evaluated benchmarks and automated judgments cover only a limited set of deployment risks and do not certify model safety.Source: §6 · p. 9, ETHICS STATEMENT · p. 10
  • DPO provides stronger standalone control and text monitors stronger overall detection, while representation methods are advantageous in low-data control and efficient native monitoring.Steering is most useful when high-quality safety data are limited; probes are attractive when the generating model's activations can be reused.Source: §6 · pp. 9–10
  • The authors used generative AI for experimental-plan feedback, code assistance, literature candidates, figure and table formatting, and editing, while retaining responsibility for research decisions and interpretations.They report inspecting and testing AI-assisted code and manually validating a sample of automated judgments.Source: AI USE STATEMENT · p. 10
  • The appendices document configurations, preprocessing, seeds, monitor training and calibration, integration strategies, and supplementary results to support reproduction of the comparisons.The main paper specifies the evaluated models, datasets, methods, metrics, and operating protocols.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 TRAINING CONFIGURATION

The control methods use shared model-specific construction and safety data, with DPO treating safe and unsafe responses as preferences and steering methods using activation contrasts. The appendix also specifies inference interventions and a benign fine-tuning persistence protocol.

A.1 TRAINING CONFIGURATION

  • All control methods use the same 5,406 PKU-SafeRLHF examples per base model, with one safe and one unsafe response; DPO uses preferred and rejected completions, while steering uses activation contrasts.Flow adapts its formulation to learn transport from unsafe to safe activations; prompts and responses use each model's chat template, and the default seed is 42.Source: §A.1 · p. 15
  • CAA normalizes the difference between mean last-token activations of safe and unsafe sequences; Probe maps a binary probe weight to a unit-norm original-space direction; Flow uses a three-layer velocity MLP.Table 5 reports model-specific control settings, with CAA and Probe sharing an intervention layer and strength.Source: §A.1 · p. 16, Table 5 · p. 16
  • At inference, steering runs through a forward hook without updating base-model parameters; CAA and probe directions use α = 1.0, while Flow uses three Euler steps with T = 0.7.Persistence testing applies the pre-update safeguards after fine-tuning controlled models on Alpaca-Cleaned; DPO is not repeated and steering directions are not re-extracted.Source: §A.1 · p. 16

A.2 DATA-SCALING PROTOCOL

The appendices specify the data, scope, judging, and operating-point protocols behind the control evaluation, then report robustness, sensitivity, and practicality findings.

A.2 DATA-SCALING PROTOCOL

  • The scaling analysis varies data quantity and source on Qwen2.5-1.5B-Instruct, using fixed method-specific training schedules rather than a constant compute budget.The original and filtered sets contain 63,094 and 5,406 pairs, respectively; 100- and 200-pair results average three seeds, while larger settings use seed 42.Source: §A.2 · p. 16

A.3 GRANULARITY DATA CONSTRUCTION

  • The granularity evaluation defines cybercrime, physical harm, and toxicity scopes from benchmark categories and assigns evaluation prompts to exactly one scope.The evaluation sets have zero prompt overlap, although contrastive training pools can overlap because source records may carry multiple labels.Source: §A.3 · p. 17

A.4 SAFETY JUDGING

  • The harmful-compliance judge labels actionable assistance as harmful and safe redirections or non-actionable discussion as safe; binary-v3 achieves 98.6% accuracy on 624 manually reviewed responses.Validation reports 2.3% FPR among safe responses and 0.8% FNR among harmful responses.Source: §A.4 · pp. 17–18, Table 7 · p. 18

A.5 OPERATING-POINT SELECTION AND SENSITIVITY

  • Primary intervention layers are selected by a lightweight calibration sweep and then held fixed; expanded layer–strength sweeps characterize sensitivity without replacing those operating points.CAA and probe-based steering share the layer minimizing mean ASR, while Flow uses the layer with the lowest ASR.Source: §A.5 · pp. 18–19
  • Flow’s stronger initial safety–over-refusal trade-off at Layer 8 does not persist: after benign fine-tuning its AIM ASR rises from 0.077 to 0.712, while Layer 16 changes from 0.153 to 0.099.The layer comparison uses the full 5,406-pair, 10-epoch Flow configuration at T = 0.7.Source: §A.5 · p. 19, Table 10 · p. 20

B.1 ROBUSTNESS

  • DPO achieves the lowest ASR before and after benign fine-tuning in most settings, although ASR can increase substantially afterward, especially under AIM.At the primary operating points, Flow has smaller average AIM degradation, but that pattern does not hold uniformly across operating points.Source: §B.1 · p. 20

B.2.1 CAPABILITY PRESERVATION AND OVER-REFUSAL

  • Capability and over-refusal changes vary across models and tasks, and benign fine-tuning can improve capability scores or reduce over-refusal.DPO and Flow show model-dependent over-refusal, while CAA and probe-based steering remain closer to the corresponding base models.Source: §B.2.1 · p. 20

B.2.2 DATA EFFICIENCY

The results compare data quantity and quality, preference-pair sources, and transfer across safety scopes for behavioral alignment and representation steering.

B.2.2 DATA EFFICIENCY

  • DPO generally improves as preference-pair counts increase, but filtered contrastive data yields lower ASR at matched sample sizes; CAA and probe-based steering show little scaling benefit.Flow is also sensitive to pair quality, with small filtered subsets outperforming substantially larger samples from the original set.Source: §B.2.2 · pp. 20–21, Table 14 · p. 22

B.2.3 DATA-SOURCE COMPARISON ON LLAMA-3.1-8B

  • With the same 5,406 filtered contrastive pairs, DPO outperforms Flow while producing less over-refusal, and contrastive-pair DPO also outperforms original-data DPO.This comparison shows DPO performance is sensitive to preference-pair construction.Source: §B.2.3 · p. 21

B.3 GRANULARITY AND CROSS-SCOPE TRANSFER

  • DPO provides the strongest overall in-domain and cross-scope control; among steering methods, Flow more often transfers from general safety to individual domains.CAA and probe-based steering remain close to the base model in many cells and do not consistently improve matched-scope performance.Source: §B.3 · p. 22

C.1 TRAINING DATA

The monitoring setup uses prompt-level data splits, hidden-state probes and text monitors, held-out detection protocols, and estimates of marginal inference computation.

C.1 TRAINING DATA

  • Monitor labels use absolute safe or unsafe response annotations, with prompt-level splitting to keep responses from the same prompt out of multiple splits.Probes and the fine-tuned text monitor use the same data source, split, class balance, and training-set size.Source: §C.1 · pp. 22–23

C.2 REPRESENTATION PROBES

  • Representation probes read layer-48 hidden states from Qwen2.5-32B-Instruct and aggregate token information through mean, last-token, rolling-window, or learned-attention pooling.Positive scores indicate unsafe responses; rolling pooling takes the maximum score over 16-token means.Source: §C.2 · p. 23, Table 17 · p. 25

C.3 TEXT MONITORS

  • The fine-tuned text monitor is Qwen2.5-7B-Instruct adapted with LoRA to predict a binary safety label, while Qwen3Guard-Stream-4B is evaluated from released weights without task-specific fine-tuning.The LoRA monitor uses the same balanced training and validation examples as the probes.Source: §C.3 · p. 23

C.4 EVALUATION PROTOCOLS

  • Full-response evaluation uses held-out Qwen2.5-32B-Instruct trajectories and fixed thresholds calibrated to target 1% and 5% false-positive rates; streaming evaluation checks scores every 32 characters.The full-response calibration set contains only 51 safe trajectories, so realized test FPR may differ from its calibration target.Source: §C.4 · pp. 23–24

C.5 COMPUTATIONAL-COST ESTIMATION

  • Marginal-cost estimates count only pooling and probe-head operations for probes because target-model hidden states are treated as generation byproducts, but include an additional forward pass for text monitors.Costs use the measured mean sequence length for each method.Source: §C.5 · p. 25

D.1 REPLAY RESULTS

Replay experiments qualify probe advantages outside native activations, while hidden-reasoning and integration results examine early risk signals and safety recovery after fine-tuning.

D.1 REPLAY RESULTS

  • Under replay, text monitors generally achieve higher full-response accuracy than representation probes, and probes lose their native marginal-cost advantage because replay requires an extra Qwen2.5-32B-Instruct forward pass.The rolling probe is the strongest probe on both replayed source models in full-response AUROC and AUPRC, and retains comparatively strong early detection.Source: §D.1 · p. 25

D.2 HIDDEN-REASONING ACCESS

  • The rolling representation probe reaches 0.696 AUROC using replayed chain-of-thought activations, while prompt-only text monitors are near chance for distinguishing outcomes sampled from the same request.The oracle text condition performs similarly or better, so the experiment is evidence for reasoning access rather than a general probe advantage.Source: §D.2 · p. 26, Table 21 · p. 27

D.3 COMPUTATIONAL COST

  • The marginal-computation comparison reports estimated per-interaction monitor costs, with probe estimates excluding the target model’s generation forward pass.This exclusion reflects the assumption that native target-model activations are already available to probes.Source: §D.3 · p. 26, Table 22 · p. 27

E MONITOR–CONTROL INTEGRATION DETAILS

  • Integration evaluates blocking, corrective regeneration, and adaptive Flow steering after the complete response is generated and buffered.Blocking substitutes a fixed refusal; corrective regeneration makes one additional generation under a safety-editor instruction.Source: E MONITOR–CONTROL INTEGRATION DETAILS · pp. 26–27

E.1 INTEGRATION BEFORE AND AFTER BENIGN FINE-TUNING

  • For post-fine-tuning DPO, regeneration matches blocking in ASR while adding slightly less over-refusal; blocking provides similar safety recovery for Flow.The Qwen2.5-14B-Instruct results average AIM and refusal suppression at the threshold calibrated to 5% FPR.Source: §E.1 · p. 27

E.2 TEXT-MONITOR INTEGRATION BASELINES

Post-generation monitors reduce residual failures in DPO responses without increasing over-refusal at evaluated operating points, while threshold sweeps reveal model-dependent intervention trade-offs. The appendix also specifies corrective regeneration as a safety-editor instruction for revising harmful responses while preserving useful content when safe.

E.2 TEXT-MONITOR INTEGRATION BASELINES

  • Qwen3Guard achieves the lowest residual ASR at Cal.-5% while retaining lower realized FPR than the rolling probe; both monitor types reduce DPO failures without increasing over-refusal.All monitors score the same DPO responses using thresholds selected from corresponding calibration splits; Table 24 evaluates 538 safe responses, 313 responses per attack, and 250 benign prompts.Source: §E.2 · p. 27, Table 24 · p. 28
  • The representation probe has no computational-cost advantage in this integration setting because it uses Qwen2.5-32B replay.Source: §E.2 · p. 28

E.3 ADDITIONAL THRESHOLD SWEEPS

  • Across three target models, blocking lowers residual ASR as intervention strengthens, while corrective regeneration is model-dependent and stronger Flow intervention can be non-monotonic.In direct prompting, aggressive intervention can sharply increase over-refusal even when little residual harmful compliance remains; Figures 5–7 show threshold sweeps under direct prompting, AIM, and refusal suppression.Source: §E.3 · p. 28, Figure 5 · p. 29, Figure 6 · p. 30

E.4 CORRECTIVE-REGENERATION INSTRUCTION

  • The corrective-regeneration instruction treats requests and candidate responses as untrusted data, replacing materially harmful assistance with a concise refusal and, when useful, a safe alternative.Otherwise, it preserves useful content and prohibits role-play continuation or adding actionable harmful details.Source: §E.4 · p. 28

Related papers

Loading 2609.34771v1…