PaperlayerView exact revision on arXiv

Source-linked AI summary

LaVGuard: Selective Decoding for Contextual Value Drift

Jiale Dai, Liuxian Ma, Hongcan Deng, Xiaoke Niu, Wenjing Zhang, Zhaoxiang Liu, Shiguo Lian, Guojie Song

arXiv:2610.29135v1cs.AI

TL;DR

A response can be content-safe yet emphasize priorities contrary to its task, and existing approaches motivate the joint decision of when language departs from task priorities and how much to change it. LaVGuard separates an authorized profile from a frozen estimate of visible-prefix values and selectively tilts candidate continuations under position and KL budgets. Profile conditioning improves boundary detection, while selection achieves higher risk-safety than position-matched edits at the same mean intervention ratio.

  • Problem

    A task-relative controller must distinguish priorities expressed in a visible prefix from those authorized for the task and decide where finite intervention is warranted.

    Source: §1 · pp. 1–2

  • Method

    LaVGuard combines an authorized profile, a frozen prefix-value probe, profile-relative candidate scoring, and feasible logit tilts constrained by changed-position and cumulative-KL budgets.

    Source: §1 · p. 2

  • Results

    At 5% FPR, profile conditioning raises boundary TPR from .58 to .72; at a shared .12 mean intervention ratio, LaVGuard reaches .88 risk-safety versus .72 for position-matched selection.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The comparisons support treating where to intervene and how much to change as connected design choices while keeping the base decoder and shared content guard fixed.

    Source: §6 · p. 10

    The ten-dimensional profile is an operational representation that does not exhaust an utterance’s social meaning or resolve which priorities a task should authorize.

    Source: Limitations · p. 10

Abstract

from arXiv · show

A content-safe response can still promote priorities that conflict with its task: an assistant may demand compliance when asked to preserve autonomy. We study contextual value drift as a task-relative decoding problem with an explicit allowance for change. LaVGuard separates an authorized ten-dimensional profile from a frozen estimate of the values expressed in the visible prefix. A profile-relative density gate selects when to evaluate candidate continuations; their signed progress determines a logit tilt constrained by both changed-position and cumulative-KL budgets. We establish local surrogate improvement, pathwise budget feasibility, and a bound on response-distribution change. Profile conditioning raises boundary TPR at 5% FPR from .58 to .72. At the same .12 mean intervention ratio, the selection study reports risk-safety scores of .88 for LaVGuard and .72 for position-matched edits. The resulting interface makes task priorities actionable while keeping the extent of intervention explicit and bounded.

1 Introduction

Contextual value drift occurs when an answer departs from the priorities authorized for its task, even if it contains no prohibited content. LaVGuard separates task priorities from values estimated in visible language, selectively edits under two budgets, and evaluates profile conditioning, selection, and budget control.

1 Introduction

  • A content-safe answer can still privilege priorities contrary to the task, such as requiring compliance when the request is to preserve autonomy.Offering clear roles while letting team members choose respects both coordination and autonomy.Source: §1 · p. 1
  • Contextual value drift is departure from a task’s authorized priorities; a controller must distinguish values expressed in a prefix from those requested for the task.The intervention should also avoid spending changes at positions where ongoing language already fits the task.Source: §1 · pp. 1–2
  • LaVGuard uses a supplied profile and a frozen prefix-causal probe to select candidate continuations, score progress toward a mixture-derived reference, and choose a feasible tilt under position and cumulative-divergence budgets.The content guard is shared with the base decoder.Source: §1 · p. 2
  • Holding the language state fixed while changing the authorized profile can change whether intervention is warranted, without retraining the probe.The profile and observed prefix values remain distinct inputs to the task-relative decision.Source: §1 · p. 2
  • At 5% FPR, profile conditioning raises boundary TPR from .58 to .72; at a shared .12 mean intervention ratio, LaVGuard reaches .88 risk-safety versus .72 for position-matched selection.Removing the position budget raises the edit ratio to .34, with risk safety .89 and benign safety .73, compared with .88 and .81 for the full controller.Source: §1 · p. 2
  • The paper connects local candidate-surrogate improvement to the tilt, proves budget and response-distribution bounds, and evaluates profile conditioning, position selection, and the two allowances.Source: §1 · p. 2

2 Related Work

Related work addresses value estimation, steering, selective intervention, and safety or divergence control. LaVGuard combines task-relative triggering with explicit accounting for changed positions and conditional KL against its base decoder.

Values and task context

  • Prior work evaluates value priorities in language models and contextual normative judgments, while LaVGuard keeps the authorized profile separate from the observed prefix state.Source: §2 · p. 3

Attribute and reward guidance

  • Attribute-guidance methods use attribute models or auxiliary distributions, while LaVGuard uses candidate reweighting with a task-relative trigger and local prototype-progress score.FUDGE predicts an eventual attribute from a prefix; LaVGuard’s probe estimates values expressed in visible text.Source: §2 · p. 3

Value editing and semantic preservation

  • Representation and value-editing methods expose behavioral directions or semantic-value interfaces, whereas LaVGuard controls the sampling kernel and tracks its cumulative change from the base decoder.Source: §2 · p. 3

Selective and adaptive intervention

  • ConVA, MAT-Steer, and AIMES adapt activation edits to inputs or states; LaVGuard combines a profile-translated residual boundary with changed-position and conditional-KL allowances.Source: §2 · p. 3

Safety and divergence budgets

  • Safety and divergence methods address harmfulness, content safety, or trajectory-level divergence; LaVGuard instead accounts for changed positions and conditional KL against its base decoder while sharing the content guard.Source: §2 · p. 3

3 Task-Relative Value Control

LaVGuard represents task priorities with an authorized profile, estimates values from visible language, and defines drift relative to task-admissible expressions. It limits decoder change through per-response position and conditional-KL accounting.

3.1 Profile, language state, and drift

  • A frozen autoregressive model generates from the prompt and context, while an authorized user or policy owner supplies a ten-coordinate profile whose intensities need not sum to one.The probe reads visible language; numeric profile coordinates and annotation labels are excluded from its input.Source: §3.1 · p. 3
  • Contextual drift is an independently annotated value vector outside the task’s admissible set; online control instead uses a learned prefix estimate and fitted admissible set.Evaluation uses independent compatibility judgments, distinct from the controller’s estimate.Source: §3.1 · p. 4
  • A shared content-safety guard checks the initial and sampled prefixes, replacing and terminating a response when activated; base and controlled decoders use the same rule.This separates content safety from control over priorities among permissible continuations.Source: §3.1 · p. 4

3.2 An explicit allowance for decoder change

  • The allowance tracks changed positions and accumulated conditional KL for each generated response; guard-stop transitions cost zero, but a sampled token that triggers the guard retains its incurred cost.These quantities account for LaVGuard’s changes to the generation process.Source: §3.2 · pp. 4–5

4 The LaVGuard Controller

LaVGuard separates the task’s authorized value profile from a frozen estimate of values expressed in visible language, then uses profile-relative detection and candidate progress to guide decoding. Changed-position and KL allowances constrain the update; the analysis characterizes its local objective, pathwise feasibility, and response-distribution effect.

4.1 Read the visible value trajectory

  • The frozen probe estimates value intensities from visible prefixes, excluding profile coordinates and annotation labels from its input.It pools recent positions at four depth-quartile layer taps and is frozen before boundary fitting and decoding.Source: §4.1 · p. 5

4.2 Locate the state relative to the profile

  • Subtracting the authorized profile from the estimated state defines a task-relative residual; density determines when to act, while a mixture prototype guides candidate preferences.The same estimated state can be inside the boundary for one task and outside it for another.Source: §4.2 · p. 5

4.3 Evaluate likely next-token continuations

  • At active positions, LaVGuard scores likely next-token continuations by one-step progress toward the prototype and uses that score to set the logit direction.The score is zero-extended to unscored tokens, while boundary exceedance scales the proposal.Source: §4.3 · p. 6

4.4 Allocate the remaining change budget

  • A scalar search selects a feasible scale along the fixed candidate direction, subject to the remaining changed-position and conditional-KL allowances.The implementation samples from the tilted kernel and uses a feasible bisection endpoint; KL is computed between the actual sampling kernels.Source: §4.4 · p. 6

4.5 What the control rule guarantees

  • The tilt uniquely maximizes a local regularized surrogate, and the largest feasible scale gives the greatest expected surrogate score along that tilt path.The guarantee is local to a fixed prefix and prototype; the implemented rule uses a unit-scale cap and finite bisection.Source: §4.5 · pp. 6–7
  • Checking the integer counter and using a feasible endpoint enforce the position rule on every realized trajectory; KL composition and response mapping bound returned-response changes.The response-event bound follows from the trace-law KL chain rule, data processing, and Pinsker’s inequality.Source: §4.5 · pp. 6–7

5 Evaluation

In the matched-decoding study, LaVGuard improves risk outcomes over SafeInfer, while selection comparisons favor its profile-relative gate at matched mean intervention ratio. The evaluation also reports stronger profile-conditioned boundary detection and examines how removing either budget changes intervention costs and outcomes.

5.1 Setup and measurement roles

  • The benchmark contains 200 variants in 100 prompt families, and its reported metrics distinguish outcome roles; equal mean token ratios do not imply equal edit counts for every response.The family-grouped split assigns 70/15/15 families to training, validation, and testing.Source: §5.1 · p. 8

5.2 The profile improves boundary detection

  • At 5% FPR, task conditioning raises TPR from .58 to .72, and the complete controller reaches .76; at 95% TPR, FPR falls from .28 to .18 and .15, respectively.The decoder uses a threshold selected on validation families; the reported operating points characterize discrimination.Source: §5.2 · p. 8

5.3 Response outcomes under selective control

  • LaVGuard raises risk F1 from .58 to .84 and risk safety from .76 to .88 versus SafeInfer, while reducing over-refusal from .18 to .08 in matched decoding.Benign safety is .81 versus .79, and mean latency is 830 ms versus 760 ms in this comparison.Source: §5.3 · pp. 8–9

5.4 Selection concentrates useful changes

  • At a shared mean token ratio of .12, LaVGuard reaches .88 risk safety versus .72 for position-matched and .70 for random selection; gain per token is .105 versus .020 and .015.Mean KL is .04 in all three conditions.Source: Table 3 · p. 8, §5.4 · p. 9

5.5 The two budgets constrain different changes

  • Removing the position budget raises token ratio from .12 to .34 and mean KL from .04 to .10, while benign safety falls from .81 to .73.Removing the KL budget gives a .18 token ratio and .13 mean KL, with benign safety .74; the full controller has risk safety .88 and benign safety .81 at .12 ratio.Source: Figure 3 · p. 9, §5.5 · p. 10
  • In the independent efficiency study, full/fast mean and P95 latency are 920/1200 ms and 820/1050 ms; dialogue first detection is 2.0 versus 3.1 turns, with cumulative risk .18 versus .30.Transfer risk F1 and worst-group score are .78 and .74 for LaVGuard, versus .55 and .62 for SafeInfer; these are separate studies.Source: Figure 4 · p. 9, §5.5 · p. 10

6 Conclusion

LaVGuard turns an authorized task profile into a selective decoding rule, with candidate progress guiding local preferences and explicit allowances constraining the intervention. The paper’s analysis and comparisons support treating where to intervene and how much to change as connected design choices.

6 Conclusion

  • An authorized task profile and the observed prefix state remain separate inputs to a residual-based decoding decision.The profile determines how to interpret the observed value state.Source: §6 · p. 10
  • The analysis establishes what the local update optimizes and how its expenditure composes into a response-distribution bound.Source: §6 · p. 10
  • The profile, selection, and budget comparisons support treating where to intervene and how much to change as connected design choices.The resulting interface keeps intervention extent explicit and bounded while the base decoder and common content guard remain fixed.Source: §6 · p. 10

Limitations

The method’s value representation and control objectives have defined scope boundaries. Its calibration, human-judgment alignment, and computational demands constrain how its results should be interpreted and deployed.

Scope of the value representation

  • The ten-dimensional profile represents task priorities operationally, but does not exhaust an utterance’s social meaning or settle which priorities a task should authorize.Interpretation depends on the annotation rubric and supplied context; use across languages, communities, or domains requires appropriate profile design and evaluation.Source: Limitations · p. 10

Estimation and control objectives

  • Probe and residual-density calibration depends on the language distribution encountered during generation, while further contextual distinctions must be encoded in the task profile.The local improvement result concerns a candidate-score surrogate; human judgments of task compatibility remain a separate evaluation target, and a mixture prototype need not lie in the activation level set.Source: Limitations · p. 10

Access and computation

  • The method requires intermediate model states and next-token distributions, and candidate evaluation adds backbone computation at active steps.The supplied backend evaluates extended prefixes without KV-cache reuse, so a small intervention ratio does not by itself imply a proportional latency reduction; longer responses and other backbones require corresponding calibration and computation measurements.Source: Limitations · p. 10

Ethical Considerations

The paper makes task priorities explicit through authorized profiles and a control process that separates those profiles from observed language. It describes implementation and evaluation boundaries, as well as ethical safeguards for deployment.

Ethical Considerations

  • Task profiles should be authorized and disclosed; deployment should allow inspection, revision, and contestation while preserving shared content-safety precedence.Evaluation should distinguish task compatibility, completion, and refusal, including their distribution across affected groups; the authors also report generative-AI assistance and retain responsibility for the work.Source: Ethical Considerations · p. 11

A.1 Three objects with distinct roles

  • Annotations, predicted prefix values, and external task profiles are distinct objects; a high value coordinate is neither a universal violation nor authorization to act.The profile is fixed before generation; an explicit contract update can authorize a new profile for the next response, while within-response changes define a different control problem.Source: §A.1 · p. 14

A.2 Information available at each decision

  • Candidate extensions are evaluated hypothetically from the same prefix, so the controlled sampling distribution is available before the next token is sampled.This is an online-availability statement, not a claim about predicting a future response label.Source: §A.2 · p. 14, Table 5 · p. 14

B.1 Probe fitting and freezing

  • The probe estimates value intensities from projected representations and is frozen before residual-mixture fitting and threshold selection.The probe is selected by validation MAE; sigmoid outputs estimate conditional mean intensity, not calibrated probabilities of response-level drift without a separate readout and evaluation.Source: §B.1 · p. 15, Table 6 · p. 15

B.2 A complete decoder clock

  • At each step, the decoder checks the gate and budgets, evaluates candidate continuations, selects a feasible tilt, samples, updates counters, and checks the guard.The implementation specifies no-op handling, feasible-endpoint bisection, and accounting against the actual sampled kernels; an additional post-search truncation changes the kernel and requires recomputing KL.Source: §B.2 · pp. 15–16

B.3 Computational accounting

  • Candidate evaluation adds computation beyond probing, and the supplied backend evaluates observation and candidate branches without cache reuse.Batching affects elapsed time and memory; attempted candidate batches can occur more often than accepted interventions, so intervention frequency alone does not characterize cost.Source: §B.3 · p. 16

B.4 Execution and data records

  • Decoder traces record profile version, decisions, candidate scores, scales, KL, and counters, linking fitting choices to individual decoding decisions.Records also identify model and fitting artifacts; prefix sources should be distinguished because they induce different state distributions.Source: §B.4 · pp. 16–17

C.1 The level set and its context dependence

  • The estimated admissible set is a density superlevel set intersected with the intensity cube, and its contextual dependence is through the numerical task profile.Translating state and profile together leaves the residual score unchanged when both remain in the cube; contexts with the same profile induce the same estimated boundary.Source: §C.1 · p. 17

C.2 Density gradient and local curvature

The appendix distinguishes progress toward a prototype from density improvement and derives conditions under which local score progress certifies density reduction. It also characterizes the KL-constrained tilt, its feasibility, and the behavior of sparse updates on unscored tokens.

C.2 Density gradient and local curvature

  • Positive prototype progress is equivalent to a strict decrease in squared distance to the fixed prototype.The mixture’s posterior-mean prototype can lie outside the density level set: in the example, .2 and .8 are admissible while .5 is not.Source: §C.2 · pp. 17–18, Figure 5 · p. 18

C.3 A sufficient condition for density improvement

  • With a Lipschitz gradient on the segment, a negative descent-lemma right-hand side certifies local density decrease for a candidate.In the shared isotropic case before clipping, the condition can be expressed using the prototype-progress direction and candidate-step length.Source: §C.3 · p. 18

D.1 The variational meaning of the update

  • At an exact endpoint whose KL allowance binds, the exponential tilt uniquely maximizes expected direction reward within that KL ball, under the stated finite-support conditions.If the unit-scale cap binds instead, the update is best along the chosen tilt path but need not maximize reward over the larger KL ball; finite bisection returns a feasible approximation.Source: §D.1 · pp. 18–19

D.2 KL monotonicity and a correct no-op test

  • For a nonconstant direction, KL increases with tilt scale and the feasible scales form a closed interval; a constant shift changes no probabilities and is a no-op.Bisection retains the feasible endpoint, preserving the KL constraint at finite precision.Source: §D.2 · p. 19

D.3 Sparse updates preserve tail ratios, not tail probabilities

  • Sparse tilts preserve relative probabilities within the unscored tail but can change its total mass.The constructed signed and positive updates preserve tail-token odds while moving total tail mass in opposite directions.Source: §D.3 · p. 20, Table 7 · p. 20

D.4 What the clipped candidate score improves

The clipped candidate direction weakly improves expected prototype progress, while pathwise budgets constrain edits and KL. Selective gating is advantageous over a same-rate random gate when it concentrates contexts with greater benefit.

D.4 What the clipped candidate score improves

  • Every nonnegative accepted scale weakly increases expected zero-padded prototype progress, though it does not guarantee positive progress for every sampled token.The result allows signed candidate scores and clipping.Source: §D.4 · p. 20

D.5 Pathwise accounting and the shared guard

  • On every generated trajectory, changed positions and cumulative KL remain within their respective allowances.The integer budget limits whether another position can change; the KL budget limits each distribution shift and total response KL.Source: §D.5 · p. 21

E.1 The quantity a selective gate must concentrate

  • A selective gate’s benefit over a same-rate random gate is exactly its covariance with edit benefit; positive benefit also requires sufficient gate precision when relevant and other contexts have opposing benefits.The one-step prototype score is not, by definition, the base policy’s future-reward advantage.Source: §E.1 · p. 22

E.2 Profile and probe errors enter through the same residual

Profile and probe errors perturb the same residual, so a boundary decision is stable only with sufficient margin. The section also specifies ambiguity handling, validation and measurement conventions, and evaluation units.

E.2 Profile and probe errors enter through the same residual

  • A boundary sign remains stable when its absolute margin exceeds the error bound induced by profile and probe estimation errors.For the Gaussian mixture, sensitivity can increase with smaller variance floors, even when training residual likelihood fits closely.Source: §E.2 · p. 22

E.3 An explicit ambiguity band

  • An optional ambiguity band certifies the boundary sign outside the band under the stated error model; inside it, the bound alone does not certify the sign.The default decoder does not use this separate analytical criterion.Source: §E.3 · p. 22

E.4 Validation FPR and population coverage

  • The validation threshold maximizes family-level recall subject to empirical benign-family FPR at most .05, and individual-prefix deployment requires its own evaluation.The calibration bound assumes a fixed score and independent calibration units; validation-based probe selection needs an additional independent split.Source: §E.4 · p. 23

F.1 Benchmark construction

  • The benchmark uses 200 variants from 100 prompt families, split by family into 70/15/15 train, validation, and test families, with three annotators and adjudication.The profile comes from the pre-generation task specification, while prefix labels describe visible text.Source: §F.1 · p. 23

F.2 Diagnostic metrics and their units

  • TPR@5 is recall at FPR at most .05, distinct from recall at the deployed threshold; value-state diagnostics are also distinct from contextual-drift labels.Metric definitions retain their specified event, denominator, and aggregation unit.Source: §F.2 · p. 24

F.3 Response outcomes and denominators

  • Compatibility, completion, content harm, and refusal are separate response outcomes, and continuous safety scores are not interchangeable with binary pass rates.Result tables distinguish readouts such as risk safety, risk pass rate, benign safety, and benign utility.Source: §F.3 · pp. 24–25

F.4 Intervention, localization, and computation

  • Mean response intervention ratio and pooled token fraction weight variable-length responses differently, while cumulative response KL and the study’s Mean KL use distinct quantities.Localization and efficiency summaries also depend on their declared ratio, latency, hardware, and memory conventions.Source: §F.4 · p. 25

F.5 Transfer, dialogue, and uncertainty

  • Transfer scores require declared held-out dimensions and group partitions, while dialogue metrics require conventions for undetected cases, turn aggregation, and variable dialogue length.Statistical comparisons should retain prompt families as the paired resampling unit.Source: §F.5 · p. 25

G Complete evaluation results

The evaluation tables organize studies across boundary detection, generation, selection, components, transfer, dialogue, and efficiency. A separate analytical extension scores changes in the density boundary rather than the default controller’s prototype progress.

G Complete evaluation results

  • The evaluation tables separate prefix estimation and boundary detection from generation, selection, component, transfer, dialogue, and efficiency studies.Their captions retain each study’s own metric labels and readouts.Source: G Complete evaluation results · pp. 25–26

H.1 A separately specified correction objective

  • The density-boundary extension is analytically and numerically studied, while generation experiments evaluate the default prototype-progress controller.A positive certificate establishes expected one-step improvement in the learned boundary score, not task compatibility or completion.Source: §H.1 · p. 28
  • The positive-only density direction boosts candidates with measured density progress and leaves other unnormalized weights unchanged.The extension uses the same KL and token-budget machinery as the default controller.Source: §H.1 · p. 28

H.2 Verification cases for an equation-level implementation

Constructed checks validate the controller’s equation-level behavior, separately from language-model outcomes. The remaining sections specify task-profile inputs and updates, compose budgets across dialogue turns, and illustrate profile-dependent activation with a paired example.

H.2 Verification cases for an equation-level implementation

  • The reference controller passed 600 constructed-distribution checks and 100 finite-probability-tree checks, covering surrogate behavior, feasibility, accounting, and returned-response KL contraction.These equation-level checks are separate from empirical generated-response measurements.Source: §H.2 · p. 29

I.1 From a task contract to a control input

  • A task contract pairs a human-assessable natural-language statement with a complete ten-coordinate profile used in the residual calculation.The intensity scale is not a probability normalization, and unspecified coordinates require an owner-approved default or clarified contract.Source: §I.1 · p. 29

I.2 Version changes and the fixed statistical interface

  • A profile can change between responses without retraining when the coordinate system and learned components remain fixed; intended task distinctions must be represented in its authorized coordinates.A profile version is fixed for each generated response.Source: §I.2 · p. 30

I.3 Composing budgets across authorized dialogue turns

  • Adaptive dialogue profiles are compatible with a dialogue-wide KL bound when both systems share the same environment policy and response allowances respect the remaining balance.Resetting response counters alone does not create a fresh unlimited application budget.Source: §I.3 · pp. 30–31

J.1 One visible state under two authorized contracts

  • With the visible request and prefix held fixed, the workshop profile retained the base distribution while the replication profile activated candidate evaluation.The paired construction changes the external numeric reference, not the observed state or density model.Source: §J.1 · p. 31

Related papers

Loading 2610.29135v1…