PaperlayerView exact revision on arXiv

Source-linked AI summary

RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Models

Wenhan Chang, Tianqing Zhu, Ping Xiong, Shiyi Liao, Wanlei Zhou

arXiv:2609.00790v1cs.CR

TL;DR

Existing inference-time refusal methods may alter responses without checking whether the initial answer is already appropriate, while semantic variation and model differences complicate reliable verification. RISA inspects the initial response, combines contextual rules with a calibrated hidden-state probe, and applies model-specific selective actions without updating base-model weights. Experiments across three LLMs and multiple benchmarks report improved refusal reliability while largely preserving utility.

  • Problem

    Inference-time methods often intervene without inspecting already-appropriate responses, while fixed rules may miss semantic variations and require adaptation across base models.

  • Method

    RISA combines contextual rules, a calibrated final-layer hidden-state probe, model-specific thresholds, and initial refusal status to select preservation, enforcement, or regeneration actions.

  • Results

    Experiments across three instruction-tuned models and multiple safety benchmarks show improved refusal reliability on harmful prompts, while general utility is largely preserved.

  • Takeaways & Limitations

    Refusal calibration can be treated as runtime verification with asymmetric actions for harmful non-refusals and benign over-refusals.

Abstract

from arXiv · show

Reliable refusal behavior requires Large Language Models (LLMs) to reject harmful prompts with only answering benign ones. Incorrect refusal behavior can either expose users to harmful responses or prevent users from obtaining useful answers. Training-time alignment improves refusal behavior by updating model parameters with safety data, but requires additional computation and training. In contrast, inference-time alignment aims to modify LLM behavior during inference without updating the underlying model parameters. Existing inference-time methods mainly rely on in-context safety prompting, activation steering, or decoding control. However, most of them intervene without first determining whether the initial response is already appropriate, potentially altering a correct refusal or a useful answer. Effective selective intervention therefore requires identifying prompt intent beyond sensitive keywords, covering semantic variations that fixed rules may miss, and adapting the verifier to different base models. To address these challenges, we propose Response Inspection and Selective Actions (RISA), an inference-time framework that inspects the initial response and selectively corrects refusal errors without updating the base model. RISA first uses fixed contextual rules to assign refusal scores to clear cases. For unmatched cases, it derives a refusal score from the final-layer prompt hidden state using a calibrated linear probe. To adapt to different base models, RISA separately calibrates the probe score, representation-support boundary, and action thresholds. At runtime, RISA combines the prompt score with the initial refusal status and applies an action policy to intervene only when necessary. Experimental results demonstrate that RISA improves refusal reliability while largely preserving model utility, offering a practical solution for response-aware refusal calibration in LLMs.

I. INTRODUCTION

Reliable refusal behavior must reject harmful prompts while answering benign ones, but inference-time interventions can damage already-correct responses. RISA addresses this by inspecting initial responses and selectively applying calibrated corrections.

  • Motivation: Reliable refusal requires rejecting harmful prompts while answering benign ones, yet models can over-refuse benign-sensitive prompts or under-refuse harmful prompts.Sensitive keywords can appear in both harmful and harmless requests.
  • Motivation: Existing inference-time methods may intervene without checking whether the initial response is already safe and useful.Such intervention can turn a correct refusal into unsafe content or a helpful answer into an unnecessary refusal.
  • RISA: RISA combines contextual fixed rules with a calibrated linear probe to score clear cases and semantically varied prompts.The probe uses the final-layer hidden state of the last non-padding prompt token when rules do not resolve the case.
  • RISA: RISA calibrates probe scores, representation support, and action thresholds separately for each base model.Runtime intervention uses the initial refusal status together with calibrated evidence.
  • Contributions: RISA selectively corrects refusal errors without updating base-model weights while preserving responses that do not require intervention.The framework distinguishes refusal enforcement from safety-guided regeneration for the two correction directions.
  • Results: Experiments across three LLMs and multiple benchmarks show improved safety performance while preserving general utility.The reported evaluation supports response-aware refusal calibration as an inference-time approach.

A. Training-time Safety Alignment

Training-time safety alignment improves refusal behavior by updating model parameters with safety or preference data, but repeated retraining can be impractical. RISA instead keeps a trained base model fixed and calibrates refusal behavior at runtime.

  • Training-time Safety Alignment: Training-time alignment updates model parameters using preference data, safety data, or critique-and-revision methods.Representative approaches include reinforcement learning from human preferences, Constitutional AI, safety tuning, and Safe RLHF.
  • Training-time Safety Alignment: Safety tuning can improve safety with small data amounts but may produce exaggerated refusals of safe prompts when excessive.This illustrates the trade-off between stronger refusal behavior and over-refusal.
  • RISA: RISA targets settings where the base model is fixed and frequent policy updates make repeated model retraining impractical.It complements rather than replaces training-time safety alignment.
  • Inference-time Methods: Inference-time methods keep base-model weights fixed while using prompts, token controls, activation interventions, latent detection, or external guardrails.These methods avoid parameter updates and can be applied through additional components or serving-time controls.
  • RISA: RISA differs by comparing the initial refusal status with a refusal score before selecting an action.Its reliability depends on rule coverage, probe calibration, and the fixed model’s regeneration behavior.

C. Inference-Time Refusal Calibration

Inference-time refusal calibration treats refusal errors as a runtime verification problem for a fixed instruction-tuned LLM. RISA uses prompt evidence and the initial response to select whether to preserve, enforce, or regenerate the output.

  • Runtime Verification: RISA operates after the base model generates an initial response and verifies the prompt before returning a final response.The deployer controls the serving pipeline and can access the prompt, initial response, and verifier.
  • Action Policy: RISA changes the response only when the observed refusal status conflicts with strong verifier evidence.Unsupported representations cause the initial response to be preserved.
  • Inference-Time Refusal Calibration: Refusal miscalibration occurs when a model refuses a benign-sensitive prompt or answers a harmful prompt.These errors are respectively over-refusal and under-refusal.
  • Runtime Verification: The verifier can preserve the initial response, replace it with a fixed refusal, or request safety-guided regeneration.The user is not asked for a safety label or statement of intent.
  • Refusal Errors: Over-refusal and under-refusal are defined by whether the final response matches the target refusal policy for benign-sensitive and harmful prompts.Correct behavior answers safe prompts and refuses prompts requiring refusal.

B. Framework Overview

RISA separates prompt verification, model-specific verifier preparation, and runtime response selection. It combines contextual rules with a calibrated hidden-state probe, support checks, and thresholds tailored to each base model.

  • Framework components: RISA verifies the prompt and initial response, prepares verifier components before deployment, then selects a runtime action.The three components respectively inspect the initial response, calibrate the verifier, and decide whether to preserve or correct it.
  • Rule scoring: Contextual rules handle predefined patterns first, producing fixed refusal scores and resolving conflicts through priority ordering.Benign-context rules use 0.05, refusal-required rules use 0.95, and direct refusal patterns receive higher priority.
  • Hidden-state verification: For unmatched prompts, a model-specific linear probe maps the final-layer hidden state of the final non-padding prompt token to a refusal score.The probe uses logistic regression with balanced class weights and ℓ2 regularization; Platt scaling converts its raw logit into a bounded score.
  • Model-specific calibration: RISA separately trains probes for each base model because hidden-state dimensions and value distributions differ across models.A probe trained on one model cannot be applied directly to another.
  • Calibration: RISA calibrates scores and thresholds on separate non-overlapping data splits before deployment.Dscore fits Platt scaling and the support threshold, while Daction sets model-specific action thresholds.
  • Representation support: A support check allows probe use only when the prompt representation lies within the calibrated training-data boundary; otherwise, RISA preserves the initial response.The cutoff κ is the 99th percentile of score-calibration distances.

D. Threshold Calibration for Each Model

RISA calibrates separate allow and refusal thresholds for each base model using resolved verifier scores and class-conditional split-conformal calibration. The resulting ordered pair preserves the calibrated bounds while creating a strict no-action interval.

  • Model-specific thresholds: RISA derives a separate threshold pair for each base model because score ranges differ across models.The same score resolver is used during action calibration and runtime.
  • Score resolution: Only non-abstaining verifier scores enter calibration; abstentions provide no support for either intervention direction.Harmful and benign-sensitive scores are collected separately as V1 and V0.
  • Class-conditional calibration: The two action errors use separate class-conditional score tails and an allowed error rate ε, with smaller ε producing more conservative thresholds.Calibration requires k1 ≥ 1 and k0 ≥ 1.
  • Calibration bounds: The calibrated bounds limit harmful prompts entering the answer interval and benign-sensitive prompts entering the refusal interval.These guarantees rely on exchangeability within each class.
  • Threshold ordering: RISA orders reversed raw thresholds to guarantee τallow < τrefuse without expanding either raw intervention interval.The strict gap prevents one score from triggering both actions.
  • Calibration procedure: Algorithm 1 resolves scores, removes abstentions, sorts class-specific sets, derives raw thresholds, and constructs the ordered pair.The procedure preserves class-conditional bounds while ensuring a strict no-action interval.

E. Runtime Action Policy

At runtime, RISA combines the initial refusal status with the resolved verifier score. It preserves agreement or uncertain cases and intervenes only when strong score evidence conflicts with the initial response.

  • Runtime verification: RISA first generates an initial response, detects its refusal status, and resolves the verifier score.The score may be a rule score, probe score, or abstention.
  • Regeneration: A refused response with V(P) ≤ τallow triggers safety-guided regeneration, using the initial response if regeneration is empty.Regeneration changes the generation instruction without updating model weights.
  • Enforcement: A non-refusal response with V(P) ≥ τrefuse triggers enforcement of a fixed refusal response.This action addresses cases where the prompt evidence requires refusal.
  • Preservation: An abstaining verifier score causes RISA to preserve the initial response.This is one of the explicit preserve conditions in the runtime policy.
  • Preservation: When neither correction condition holds, RISA returns the initial response unchanged.The no-action interval and agreement between response behavior and verifier score avoid unnecessary rewriting.

F. Analysis of Correction Asymmetry

RISA separates corrective-action selection from correction success because refusal enforcement and over-refusal correction have different reliability properties. Under perfect action selection, refusal enforcement is guaranteed, whereas over-refusal correction remains dependent on regeneration behavior.

  • Correction mechanisms: Refusal enforcement returns the fixed refusal Yref, whereas over-refusal correction returns Yregen generated through safety-guided regeneration.The first action is determined by the policy; the second remains dependent on the fixed base model's generation.
  • Initial refusal errors: RISA defines under-refusal and over-refusal from the target policy and initial refusal status, before using the verifier score to select actions.Under-refusal covers harmful prompts initially answered without refusal; over-refusal covers benign prompts initially refused.
  • Action selection: Aunder selects refusal enforcement for an initial non-refusal when the verifier score reaches τrefuse, while Aover selects safety-guided regeneration for an initial refusal at or below τallow.RISA preserves the initial response when the verifier abstains or the score falls in the no-action interval.
  • Correction success: For under-refusal, η = 1 after refusal enforcement is selected, so the overall correction probability satisfies punder = qunder.The final refusal does not depend on another model generation.
  • Correction asymmetry: Under perfect action selection, punder = 1 but pover = ρ ≤1, because regenerated responses may still refuse or be empty.This asymmetry motivates separate reporting for the two directions, two action thresholds, and a cautious no-action interval.

V. EXPERIMENTS AND ANALYSIS

The experiments evaluate RISA across refusal behavior, utility preservation, verifier generalization, and component or action-policy contributions. These four research questions organize the empirical analysis.

  • Research questions: RQ1 evaluates how RISA changes refusal behavior on harmful and benign-sensitive prompts.
  • Research questions: RQ2 tests whether RISA preserves the general utility of the base model.
  • Research questions: RQ3 examines how well the verifier generalizes beyond the data used to build it.
  • Research questions: RQ4 analyzes how RISA's main components and action policy contribute to performance.

A. Experimental Setup

The evaluation uses multiple safety datasets, explicit harmful and benign-sensitive metrics, paired statistical tests, and comparisons against the base model and five inference-time or representation-level baselines. The protocol also fixes decoding and controls sampling noise in ablations.

  • Data and models: The probe is trained on 1,188 prompts, comprising 600 benign-sensitive and 588 harmful examples, using final-layer hidden states listed for each evaluated model.
  • Calibration: The 350-prompt calibration dataset is split into separate subsets for Platt scaling and support-threshold fitting versus model-specific action-threshold calibration.The split preserves source and label distributions.
  • Evaluation data: Four safety datasets contain 200 prompts each, with OR-Bench and XSTest-b assessing benign-sensitive behavior and XSTest-h, HarmBench, and Do-Not-Answer assessing harmful behavior.XSTest contains 100 benign-sensitive and 100 harmful prompts.
  • Metrics: The evaluation reports CRR and ASR for harmful prompts, ORR and SHR for benign-sensitive prompts, with higher CRR and SHR and lower ORR and ASR preferred.ASR averages judgments from GPT-OSS-Safeguard-20B and Qwen3Guard-Gen-8B, while CRR, ORR, and SHR use GPT-OSS judgments as specified.
  • Baselines and statistics: Paired comparisons use 95% confidence intervals from 10,000 bootstrap resamples and Holm-corrected exact McNemar tests against the base model and five baselines.All methods use a 256-token budget, and matched responses control sampling noise in ablations.

B. RQ1: Refusal Calibration Performance

RISA consistently improves refusal reliability on harmful prompts, while benign-sensitive outcomes are mixed and general utility is largely preserved. Probe quality is strong across models, though end-to-end outcomes vary by layer, calibration, and dataset.

  • Safety performance: RISA increases CRR across all nine harmful model–dataset settings, with gains from 0.12 to 0.44.Every paired 95% confidence interval favours RISA.
  • Safety performance: ASR remains at or below 0.07 in every harmful setting under RISA.Thus, higher CRR does not entail greater unsafe compliance.
  • Benign-sensitive behavior: Benign-sensitive results are mixed: ORR decreases on some OR-Bench settings but increases on Llama-3.2-3B and across XSTest-b.SHR also varies by model and dataset, with selected changes statistically significant after Holm adjustment.
  • Utility preservation: RISA preserves most general utility: GSM8K responses remain unchanged, while MMLU-STEM accuracy decreases by 0.008 for Qwen2.5-3B and 0.001 for the other models.Refusal enforcement changes 47, 5, and 2 MMLU-STEM responses, respectively.
  • Probe quality and scaling: With full probe training, AUROC reaches 0.98–0.99, Brier score falls to 0.04–0.06, and harmful coverage reaches 0.94–0.97.Most gains occur before the 50% training fraction, although the full dataset produces the most stable overall results.
  • Layer robustness: Probe discrimination does not determine end-to-end performance alone: useful refusal information appears at multiple layers, and no single layer is best across datasets.End-to-end outcomes also depend on calibration, support checks, thresholds, initial responses, and regeneration.

E. RQ4: Component and Policy Contributions

RISA’s probe supplies broad harmful-prompt coverage beyond explicit rules, while its two corrective actions serve different objectives. Refusal enforcement drives harmful-prompt gains, whereas regeneration primarily reduces over-refusal.

  • Component ablation: The probe-only variant matches full RISA in eight of nine harmful model–dataset settings.The rules-only variant is more dataset-dependent, reaching CRR 0.94 on HarmBench but 0.55 on Do-Not-Answer for Qwen3-4B.
  • Component ablation: Rules handle clear cases, while the probe provides broader coverage for cases beyond explicit rule patterns.The probe fallback is therefore the main source of harmful-prompt coverage.
  • Action contributions: Refusal enforcement matches full RISA in eight harmful settings, whereas regeneration-only stays close to the base model on harmful prompts.Regeneration-only yields the lowest ORR on both benign-sensitive datasets.
  • Action contributions: No single corrective action handles both goals: enforcement can add unnecessary refusals, while regeneration does not always reverse an initial refusal.The two actions therefore contribute asymmetrically to harmful-prompt safety and benign-sensitive utility.
  • Runtime routing: Safety-guided regeneration appears mainly on benign-sensitive prompts, reaching 0.42 for Qwen3-4B on OR-Bench and at most 0.02 on harmful datasets.Refusal enforcement is more common on harmful prompts, reaching 0.61 for Llama-3.2-3B on Do-Not-Answer.

3) RQ4.3: Action-Selection Reliability:

RISA usually selects appropriate actions for initial non-refusals, but correction of benign initial refusals is less consistent. Threshold changes reveal a safety–over-refusal trade-off, while regeneration is the main source of runtime cost.

  • Action-selection reliability: For initial non-refusals, action-selection accuracy ranges from 0.88 to 0.99 on benign prompts and reaches 0.96–0.98 on HarmBench harmful prompts.The lowest harmful accuracy is 0.74 for Qwen3-4B on XSTest-h, correcting 20 of 27 initial non-refusals.
  • Action-selection reliability: For initial refusals, benign action-selection accuracy ranges from 0.42 to 0.79 on OR-Bench and 0.00 to 0.30 on XSTest-b.Harmful refusals are preserved with accuracy 0.97–1.00 because regeneration on harmful prompts is rare.
  • Threshold sensitivity: As τrefuse increases, harmful CRR never increases and generally decreases, while benign-sensitive ORR reductions remain small.Do-Not-Answer is most sensitive, with CRR reductions of 0.10–0.17.
  • Threshold sensitivity: A higher refusal threshold can slightly reduce over-refusal but can also reduce harmful-prompt safety.For example, Qwen2.5-3B HarmBench ASR rises from 0.07 to 0.09.
  • Runtime cost: Latency overhead scales with regeneration rate: Qwen3-4B regenerates 0.11 of prompts and has a 0.10 relative latency increase.Peak GPU memory changes by at most 2 MiB.
  • Policy design: RISA’s asymmetric policy preserves, enforces, or regenerates responses by comparing initial refusal status with prompt refusal scores.Benign over-refusals may be regenerated, while harmful non-refusals may be replaced with fixed refusals.
Loading 2609.00790v1…