Source-linked AI summary
Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifiers
Thiago Sandoval, Ufuk Topcu
TL;DR
Safety classifiers may follow training-time policies that diverge from deployers’ policies and degrade as traffic shifts. RCV estimates policy disagreement from classifier representations, selectively corrects likely errors, and monitors drift; across six classifier–dataset combinations, adherence rose in every case, with 0.29–0.81 of previously missed unsafe content caught.
Problem
Deployed safety classifiers may diverge from deployers’ policies and degrade as traffic distributions evolve.
Method
RCV uses verdict-conditioned probes of classifier representations to estimate policy disagreement, correct likely errors, and monitor drift without labels.
Results
Across six classifier–dataset combinations, adherence rose in every case, with 0.29–0.81 of previously missed unsafe content caught.
Takeaways & Limitations
RCV supports post-release policy adaptation and maintenance without treating every update as a classifier weight-update problem.
Takeaways & Limitations
Evidence is limited to English single-turn moderation, white-box classifier access, a deployer-supplied oracle, and simulated drift.
Abstract
from arXiv · showhide
Safety classifiers deployed with large language models often fail for two reasons: their decisions reflect the policy learned during training rather than the deployer's desired policy, and their performance degrades as deployment traffic evolves. We present Regime-Conditional Verification (RCV), a lightweight wrapper that adapts an off-the-shelf safety classifier without retraining it. RCV estimates, from the classifier's internal representations, the probability that each prediction disagrees with the deployer's policy, and selectively corrects predictions likely to be wrong. The same correctness estimates also provide a label-free signal for detecting distribution shift, enabling a maintenance loop that updates the correctness estimation layer and resorts to classifier fine-tuning only when necessary. Across three off-the-shelf safety classifiers and two benchmark datasets, RCV improves adherence to the deployer's policy in every classifier-dataset combination, catching up to 0.81 of previously missed unsafe content without modifying the underlying classifier. In a deployment study with ten attack campaigns, each a harm category held out of RCV's training, RCV detects every campaign in a dedicated injection panel; in the maintenance census most drift episodes are repaired without updating the classifier, and the fine-tune is reserved for the residual episodes that repair does not restore.
Introduction
RCV adapts off-the-shelf safety classifiers toward a deployer’s written policy by estimating verdict correctness from internal representations and selectively correcting likely errors. The same label-free estimates monitor deployment drift and drive maintenance that repairs the estimation layer before resorting to classifier fine-tuning.
- Problem formulation: Deployer-specific safety is operationalized by a written policy and an offline oracle, while live traffic is gated by the classifier’s binary safe/unsafe verdict.The oracle may use human annotation or an LLM judge applying the policy, but is impractical as a per-item live dependency.
- Method: RCV estimates whether each classifier verdict is wrong from its internal state, then corrects likely errors to steer decisions toward the deployer’s policy.The correctness estimate is conditioned on the verdict because safe and unsafe predictions have different error structures and require separate estimators and calibrations.
- Deployment monitoring: The correctness estimates also provide a label-free drift signal that monitors whether adherence remains justified as deployment traffic shifts.This addresses distribution shift as a reason safety-case claims become difficult to maintain during deployment.
- Deployment study: 100 of 100 runs detected all ten injected attack campaigns in a dedicated panel, with a median attack rate of 0.115 at first alarm.The campaigns were injected into traffic gated by an off-the-shelf Llama-Guard-3.
- Maintenance: 79 episodes were restored under a constrained label budget and 87 when the audit was bounded by data, reserving fine-tuning for residual episodes.These results cover one hundred drift episodes spanning ten held-out families across ten seeds.
Method: Regime-Conditional Verification
RCV uses regime-specific probes and calibration maps over a safety classifier’s internal representations to estimate verdict correctness, selectively correcting likely errors. The same estimates support label-free drift monitoring and a maintenance loop that updates probes using mixture-matched audit data.
- Correctness estimation: RCV equips each verdict-defined regime with its own probe and calibration map to estimate the probability that the classifier’s verdict is wrong.The method exploits features that can survive fine-tuning because specialization reshapes decisions more than representations.
- Decision correction: When the calibrated error estimate exceeds threshold τ, RCV flips the verdict: safe-regime passes become blocks, while unsafe-regime blocks become passes.τ defaults to one half.
- Drift monitoring: RCV estimates each regime’s error rate without labels by averaging calibrated per-item correctness estimates over deployed traffic.The traffic-weighted average of the two regime error rates gives deployed-system error, whose complement is adherence.
- Drift monitoring: Each regime is monitored with a sequential test on two nested event counts, alarming under a sustained increase above drift-free reference rates.Thresholds are calibrated on drift-free streams to a pre-specified false-alarm allowance.
- Maintenance: After an alarm, RCV records the traffic mixture, audits unrepresented items, splits labels between probe training and calibration, and evaluates candidate updates on held-out matching-mixture data.The candidate is accepted when it holds the pre-drift standard, recall and over-blocking criteria.
Experiments
Across three off-the-shelf safety classifiers and two benchmark datasets, RCV improves adherence to the deployer’s policy and catches previously missed unsafe content. In deployment experiments, its correctness estimates detect held-out attack campaigns and support maintenance that often repairs drift without classifier fine-tuning.
- Conditioning ablation: Per-regime routing raises adherence in all 60 paired per-seed comparisons across both datasets and raises the caught share in 59 comparisons.The routing ablation compares separate verdict-conditioned probes and calibrations with a pooled probe and calibration.
- Decision correction: RCV raises adherence in all six classifier–dataset combinations, with Beaver improving from 0.787 to 0.920 on PKU-SafeRLHF and from 0.898 to 0.933 on WildGuardMix.The largest gain on each dataset is observed for Beaver.
- Decision correction: Up to 0.81 of unsafe items passed by the underlying classifier are caught by RCV, while changes in the share of safe items blocked range from −5.3 to +7.0 percentage points.RCV also recovers 0.29 of the unsafe items WildGuard misses on its own training distribution.
- Maintenance loop: 100 of 100 held-out harm-family injection runs trigger the monitor before the 0.30 attack-rate cap, at a median attack rate of 0.115.The detection protocol uses ten seeds for each of ten held-out harm families.
- Maintenance loop: 79 of 100 drift episodes repair in place under the deployed budget, with 64 repairing on the first audit; median label cost is 600 per episode.The accepted update improves held-out recall over the pre-repair probe in 44 of the 79 repairs, by a median of 1.4 percentage points.
Related Work
Prior work estimates correctness, adapts frozen models, and monitors drift, but RCV combines verdict-conditioned correctness estimation with inference-time decision correction and deployment drift monitoring. Its probe uses the frozen classifier’s internal state, while its estimate supports both correction and monitoring.
- Correctness estimation: ConfidNet, P(IK), and judge-state probes estimate correctness or knowledge, but leave the original verdict in place.RCV is presented as a correctness-estimation approach associated with a frozen classifier.
- Post-hoc adaptation: Post-hoc adaptation methods add guardrails, train replacement classifiers, or probe policy-model activations; RCV predicts when the frozen classifier’s verdict is wrong.RCV’s probe reads the classifier’s internal state rather than replacing the underlying classifier.
- Drift monitoring: Existing drift-monitoring methods estimate performance, detect changes, localize representation drift, gate accuracy drops, or track evolving jailbreaks.The cited approaches include ATC, sequential risk tracking, DriftLens, the Suitability Filter, and JailbreaksOverTime.
- Verdict-conditioned estimation: RCV splits by the verdict rather than the true class and uses one correctness estimate for inference-time decision correction and deployment drift monitoring.The passage identifies top-label calibration as a prior for per-verdict calibration and class-conditional conformal prediction as a true-class split.
Limitations
The evaluation is limited in scope, language, interaction format, and deployment coverage, while adherence gains remain relative to a single deployer policy reference that may carry bias.
- Evaluation scope: Detection is evaluated only with Llama-Guard-3 across ten held-out harm families, and the end-to-end maintenance loop only on one WildGuardMix deployment.The loop’s audit, acceptance gate, and repair cycle are exercised only in that deployment.
- Evaluation scope: Both benchmark datasets are English and single-turn, limiting evidence across languages and interaction formats.Steering is measured on all six classifier–dataset cells, but detection and maintenance use narrower settings.
- Policy reference: Adherence measures agreement with Y⋆, so reported gains depend on one deployer-stated policy reference whose human or model judgment may carry bias.RCV is fit and calibrated on Y⋆, whereas the raw classifier does not use it; re-scoring against unused human labels bounds this concern.
Conclusion
RCV estimates safety-classifier correctness from the classifier’s internal state conditioned on its verdict, raising adherence to the deployer’s policy while keeping the classifier frozen. The same estimates support a label-free maintenance loop with audited in-place repair and fine-tuning reserved for residual episodes.
- RCV estimates correctness from the classifier’s own internal state when conditioned on its verdict.
- Per-regime probes raise adherence to the deployer’s stated policy while the underlying classifier remains frozen.
- The maintenance loop reuses the correctness estimates and monitors them without labels.
- An audited repair restores the deployment in place, with fine-tuning reserved for residual episodes.
Ethical Statement · Technical Appendix · A Formal Propositions
The appendix defines regime-conditional correctness and establishes when a single calibration map can represent both verdict regimes. It also formalizes regime-level monitoring while documenting data-release constraints and risks of verdict correction.
- Ethical Statement: The work uses potentially harmful prompt–response content but redistributes neither datasets nor harmful text.Rebuilder scripts reconstruct evaluation sets from official releases, while oracle labels are provided as a map keyed to public item identifiers.
- Ethical Statement: Verdict correction can over-block legitimate speech or pass content the classifier previously caught, subject to deployer-set controls.The ethical risk operates in both directions: false blocking and missed harmful content.
- Technical Appendix: The formal setup defines classifier verdicts, internal representations, oracle policy labels, agreement, adherence, and safe-versus-unsafe verdict regimes.The classifier emits Y, exposes Z, and agreement is A = 1{Y = Y ⋆}; adherence is expected agreement over traffic.
- A Formal Propositions: Proposition 1 states that a single ideal calibration map averages safe- and unsafe-regime correctness rates according to their score-specific regime shares.Its regime-specific miscalibration equals the other regime’s share multiplied by the gap between the two conditional agreement rates.
- A Formal Propositions: A single map is calibrated to both regimes only when correctness is conditionally independent of the verdict given the score.When regime correctness rates differ, the map is miscalibrated on both regimes; unequal shares make the minority regime’s error larger.
- A Formal Propositions: Proposition 2 formalizes a calibrated score as the probability that a verdict is correct, conditional on the representation and verdict regime.The score is calibrated within each regime on current traffic.
- A Formal Propositions: The calibrated score’s regime-conditional mean equals the regime’s current agreement rate, making a window mean an unbiased estimator when calibration holds.Adherence is the traffic-weighted average of regime agreement rates, while drift can erode the calibration condition.
- A Formal Propositions: Under calibration, one minus the mean estimates incorrect-verdict rates, and thresholded flip counts estimate the same rate but certify only the mean.The identity supports the alarm’s score interpretation, not the alarm’s behavior.
B Steering: Extended Results … B.4 The Confidence-Score Probe
The extended results evaluate RCV against human labels, test steering across clauses and probe families, and show that routing and internal representations drive its gains. A confidence-score probe nearly matches adherence but catches substantially fewer missed-unsafe items.
- B.1 Human-Label Evaluation Grid: The human-label grid re-scores all six classifier–dataset cells without refitting or changing thresholds, judging corrections by agreement with human labels.The calibrated scores and corrected verdicts remain functions of representations and per-regime probes trained on Y ⋆ labels.
- B.1 Human-Label Evaluation Grid: A fixed script re-derives every cell from per-seed steering records and aborts if any Y ⋆-axis value fails exact reproduction of Table 1.Paired p-values use two-sided paired t-tests across ten evaluation seeds, with movement counts specified after observing per-cell directions.
- B.2 Steering Toward the Added Clause: The flip rule recovers missed items in general, while the third classifier is excluded because the added clause is already included in its training.For remaining classifiers, the contrast compares caught shares on missed added-clause items with missed in-training-clause items.
- B.2 Steering Toward the Added Clause: +0.0090 is Beaver’s null contrast, whereas +0.0711 is Llama-Guard-3’s mildly positive contrast, positive at 47 of 47 operating points.The Beaver result is explained mechanically by its larger starting blind-spot gap; the contrast does not replicate on WildGuardMix.
- B.3 Routing Gain by Probe Family: +0.2 to +4.6 pp is the per-regime routing advantage over pooled routing in every PKU-SafeRLHF ablation cell.The advantage appears with linear probes, multilayer perceptrons, and gradient-boosted trees, indicating gains arise from verdict partitioning rather than probe capacity.
- B.4 The Confidence-Score Probe: A confidence-score probe recovers adherence to within 5.1 points of the deployed probe but catches fewer missed-unsafe items in all six cells.It replaces the internal representation Z with the classifier’s scalar decision score: Beaver’s continuous harm cost or the generative classifiers’ unsafe-token probability.
- B.4 The Confidence-Score Probe: 10.1 to 47.9 points is the confidence-score probe’s deficit in caught share on missed-unsafe items across all six cells.Thus, scalar confidence preserves much of adherence recovery while substantially reducing missed-unsafe-item capture.
B.5 The Escalation Fine-Tune
The escalation fine-tune is reserved for census episodes whose repair fails within budget, and is evaluated on a single disinformation episode after thirteen audits exhausted 3,720 fresh labels. On the held-out pool, combining fine-tuning with the probe passes in every seed, while the deployed gate block reproduces the recorded failure of base plus probe.
- Escalation trigger: The escalation runs once on a data-bound census episode from the disinformation family after thirteen audits exhaust 3,720 fresh labels and the held-out gate reaches 0.7255.The episode uses seed 8675309, and the escalation is reserved for cases whose repair does not pass the gate within budget.
- Escalation results: Fine-tune plus probe passes in five of five seeds at recall 0.7920 ± 0.0040 and over-block 0.0325 on the 3,420-item held-out pool.The pool excludes the fresh audit and gate block; the main text rounds over-block to 0.033.
- Escalation results: On the deployed 300-label gate block, base plus probe reproduces the episode’s recorded failure.The supplied passage identifies this gate-block result as part of the escalation evaluation.
B.6 The Text-Surface Control
The text-surface control adapts n-gram features to estimate classifier–deployer policy disagreement rather than input harmfulness. Its sparse TF-IDF logistic-regression probe uses fixed default regularization and optimizes a convex penalized objective.
- B.6 The Text-Surface Control: The control targets the classifier’s agreement with the deployer’s oracle, not input harmfulness, adapting the critique’s n-gram surface features to that target.This design does not reproduce the critique’s input-harmfulness result.
- B.6 The Text-Surface Control: Logistic regression fits the sparse text probe by optimizing the optimum of its convex penalised objective.Regularisation is fixed at the class default C = 1.0 rather than selected per regime on a held-out fold.
C Maintenance: Extended Results … D.2 The Judge Prompt and Agreement with Human Labels
The extended maintenance results show that most deployment episodes can be repaired in place, while drift monitoring, reproducibility details, and the judge-labeling procedure define how RCV is evaluated and maintained. Across the reported experiments, repair capacity, monitor comparisons, extraction recipes, data splits, and policy-oracle agreement are explicitly characterized.
- C.1 Census Outcomes by Family: 0.706 median pre-repair recall accompanies escalation, versus 0.792 where repair succeeds, indicating lower starting recall in escalated episodes.These medians are measured on each episode’s own held-out gate block.
- C.2 Repair Capacity by Family: 87 of 100 episodes repair under the data-bound audit, while 13 never pass the gate; 11 of 21 budget escalations repair and 10 do not.The data-bound audit expands through fresh material until the corpus is exhausted, producing different audit segments from the budget-bound design.
- C Maintenance: Extended Results: Most episodes repair in place across every held-out family, with most repairs occurring on the first audit; escalated episodes incur 1,500 labels.The census covers 100 episodes, with ten seeds per family.
- C.3 The Five Chain Records: Each maintenance chain sequences accepted repairs across cycles, with first-audit repairs costing 600 labels and escalated terminals costing 1,500 labels.After an accepted repair, its held-out gate reading becomes the next standard; a no-alarm terminal triggers no audit.
- C.4 Off-the-Shelf Drift Monitors Under a Matched Allowance: Under a matched 0.30-per-stream false-alarm allowance, the deployed monitor is compared with score-KS and ATC on 100 attack streams and 90 drift-free null instantiations.All monitors are calibrated on the same drift-free material; score-KS is within the allowance, whereas ATC is not.
- D.1 Recipes, Seeds, and Splits: The three classifiers use classifier-specific hidden-state extraction recipes: Llama-Guard-3 reads its own forward pass, while WildGuard-7B and Beaver use two-pass or wrapped-pair procedures.The extracted representations reproduce the study’s Z vectors at cosine 0.99996, 0.99997, and 0.99995 for Llama-Guard-3, WildGuard-7B, and Beaver, respectively.
- D.1 Recipes, Seeds, and Splits: The maintenance census and monitor comparison aggregate ten fixed seeds, while chain and fine-tune experiments use five; WildGuardMix steering uses 11,708 combined EVAL and POOL items.WildGuardMix measurements partly include items on which WildGuard was trained, potentially reducing misses and the probe’s correction opportunity; that reduction is not measured.
- D.2 The Judge Prompt and Agreement with Human Labels: The policy oracle is a stored GPT-5-Nano label map, and agreement with human labels is κ = 0.838 on PKU-SafeRLHF and κ = 0.766 on WildGuardMix.Raw agreement is 0.919 for PKU-SafeRLHF (n=16,422) and 0.930 for WildGuardMix (n=11,708); the rubric defines baseline harmfulness and commitment-to-assist tests.
D.3 Compute
RCV’s maintenance loop is compute-efficient: most alarms use CPU-based probing and monitoring while leaving the classifier unchanged, and fine-tuning is reserved for episodes that fail the gate. Representation extraction and escalation fine-tuning are the only GPU stages, run on separate single-GPU cloud instances.
- Maintenance compute: 21 of 100 census episodes require fine-tuning; most alarms are handled by a probe fit that leaves the classifier unchanged.Fine-tuning is reserved for episodes that do not pass the gate.
- Compute allocation: Representation extraction and fine-tuning are GPU stages, while probe fitting, calibration, flip-rule scoring, and monitor replays run on CPU.The recurring steering, added-clause, and monitoring compute is CPU-only.
- Hardware and runtime: Representation extraction runs on a single L40S GPU, while escalation fine-tuning runs on a single A100 80GB GPU.The GPU stages run on single-GPU cloud instances from a pinned container image using PyTorch 2.4.0 on Python 3.11 and CUDA.