Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

Alejo López-Ávila; Iker García-Ferrero; Jezabel Garcia; Antonio Tiene; Román Orús

Source-linked AI summary

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

Alejo López-Ávila, Iker García-Ferrero, Jezabel Garcia, Antonio Tiene, Román Orús

arXiv:2609.04482v1Sep 3, 2026cs.CL

TL;DR

The paper asks how deployment-specific safety can refuse harmful requests within a topic without broadly refusing benign requests. It introduces an offline self-generated framework with controlled generation, coverage repair, compensation data, and harmful-benign pairs, finding that data composition improves refusal and boundary precision but can sharply increase over-refusal. The authors therefore evaluate both sides of the intended refusal boundary.

Problem
Standard topic-level safety policies do not capture deployment-specific boundaries that require refusing harmful political persuasion while preserving benign political assistance.
Method
The paper combines controlled topic-specific generation, coverage-oriented repair, in-distribution compensation data, and harmful-benign boundary pairs for training and evaluation.
Results
Refusal tuning improves target-domain and broader harmfulness outcomes, but the paper reports a simultaneous increase in benign over-refusal, while compensation and boundary-pair data reduce false refusals.
Takeaways & Limitations
Safety alignment should be assessed jointly by harmful refusal, benign compliance, and performance on both sides of the intended refusal boundary.
Takeaways & Limitations
Evidence is limited mainly to political persuasion with Qwen3-8B and does not establish generalisation across topics, model families, or training configurations.

Abstract

from arXiv · show

Safety alignment is usually posed as a topic-level question: is this subject harmful? Deployments ask a narrower one. A civics tutor and a public-sector assistant may share a base model yet need different boundaries inside the same topic, refusing targeted political manipulation while still answering factual questions about the same election. We formulate this as narrow-boundary safety and introduce an offline self-generated framework combining controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs for training and evaluation. Single-shot generation leaves 19.88% of prompts without accepted refusal traces, whereas escalating retries leave 0.20%. On political persuasion with Qwen3-8B, training on refusal data completed through Escalate increases target-domain refusal from 9.47% to 84.75% and reduces the mean unsafe-response rate across three broader harmfulness benchmarks from 26.26% to 0.14%, but increases XSTest over-refusal from 2.00% to 74.00%. In a separate matched comparison, replacing external responses with verified target-model responses reduces over-refusal from 15.20% to 5.20%. Boundary-pair data reduces comply-side over-refusal on held-out pairs from 32.94% to 4.16%, while harmful-side refusal decreases only from 91.88% to 87.72%. These results show that data composition controls the safety and usability trade-off, and that safety alignment should be evaluated on both sides of the intended refusal boundary.

1 Introduction

The paper frames safety as learning a deployment-specific refusal boundary within a broader topic, then proposes offline self-generated data construction and evaluation that measure both harmful refusal and benign compliance. Its results show that coverage repair and boundary-aware data improve refusal coverage and precision, while aggressive safety transfer can sharply increase over-refusal.

Problem framing: Narrow-boundary safety refuses a target-harmful political subset while preserving answers to factual, neutral, and non-persuasive prompts in the same topic.The paper contrasts deployment-specific boundaries with refusing all political content.
Framework: The framework combines controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs for training and evaluation.Generation controls include persona, length, style, and paraphrasing; evaluation separates harmfulness, over-refusal, and boundary behavior.
Coverage repair: 19.88% of the audited prompt pool is silently dropped by single-shot self-generated refusal data, compared with 0.20% after coverage-oriented completion.The framework repairs missing refusal supervision instead of discarding prompts that fail to elicit accepted refusal traces.
Boundary-aware evaluation: Boundary data reduces comply-side over-refusal from 0.3294 to 0.0416, while refusal on the refusal-worthy side decreases from 0.9188 to 0.8772.The paired evaluation measures both sides of the intended local boundary.
Safety-usability trade-off: 0.2626 to 0.0014 harmful_unsafe_avg accompanies a rise in XSTest over-refusal from 0.0200 to 0.7400 after refusal tuning.The paper therefore reports safety and over-refusal jointly rather than treating harmfulness alone as sufficient.

2 Related Work

Prior safety methods largely target broad harmful categories, while this paper focuses on deployment-specific refusal boundaries inside a topic. It combines controlled synthetic data, in-distribution responses, benign-preservation methods, and evaluations designed to detect false refusals and boundary errors.

Self-generated alignment: ThinkSafe elicits target-model refusal traces, filters them with a guard model, and can add forward-KL regularisation to benign responses.The paper adopts ThinkSafe as its reference recipe because it shares the target model, adapter configuration, and evaluation judges.
Scope: Existing alignment methods address broad harmful categories, whereas this work studies refusal boundaries within a broader topic universe.The paper positions political persuasion as a narrower deployment-specific problem than standard safety taxonomies.
Response provenance: Externally adopted safety responses may shift the target model away from its native distribution, whereas self-generated traces reduce this source shift.Benign regularisation is described as constraining changes outside the target harmful subset.
Calibration: XSTest, OR-Bench, SORRY-Bench, and related resources evaluate refusal calibration under benign prompts with harmful-looking lexical, topical, or contextual cues.These benchmarks motivate measuring false-positive refusal rather than interpreting higher refusal rates as uniformly better safety.
Data and evaluation: The paper’s controlled pipeline represents a target harmful subset, its benign complement, and harmful-benign boundary pairs rather than only broad safety categories.This extends topic-sensitive synthetic data construction toward local boundary evaluation.

3 Background and Problem Formulation

The paper formalizes narrow-boundary safety as refusing harmful prompts within a topic while preserving benign behavior, and identifies coverage gaps caused by self-generated supervision. It defines the data-generation process and the learned refusal function separately to analyze these failures.

3.1 Self-generated safety alignment via steering: Mθ is the target model, while M0 is the frozen reference model before safety tuning.The notation distinguishes the trainable policy from the baseline used for benign regularisation.
3.1 Self-generated safety alignment via steering: For harmful prompts, the pipeline samples refusal-steered traces and uses a guard model ϕ to verify accepted refusals; benign traces are routed through forward-KL to M0.Accepted harmful traces use cross-entropy, while benign responses preserve proximity to the frozen reference.
3.2 The boundary problem: Ω denotes the topic universe and H its target-harmful subset, so the intended policy refuses H while answering the benign complement Ω\H.In the main experiments, Ω is the space of political prompts.
3.2 The boundary problem: Bθ(x) is a refusal probability that approximates the ideal target but may extend beyond H and create false-positive refusal in Ω\H.The formulation makes over-refusal a boundary-shaping problem rather than only a harmful-refusal problem.
3.2 The boundary problem: The refusal boundary ∂H is operationalized with harmful-benign prompt pairs sharing a topic anchor but differing in requested intent.Held-out pairs test whether refusal changes at the local boundary between refusal-worthy and comply-worthy requests.
3.3 The data-coverage problem: f denotes the data-generation and labelling process, distinct from the language model, while f−1(Ref) and f−1(Com) contain prompts yielding verified refusal and compliance traces.These preimages capture only the observed portions of the desired safety data.
3.3 The data-coverage problem: The harmful drop set D = H\f−1(Ref) contains prompts that fail to produce accepted refusal traces, while compliance coverage may also be thinner or less reliable.Section 4 introduces constructions intended to reduce both coverage gaps.

4 Method: Data Pipeline and Training

The method builds controlled harmful, benign, and boundary data, repairs failed refusal generations, and routes harmful and benign examples through different training objectives. Its data variants target both coverage and false-positive refusal without treating the entire topic as unsafe.

Refusal supervision: Single-shot generation makes one refusal-steered attempt, and prompts without an accepted refusal form the drop set D.WildGuard verifies the resulting trace before it enters the refusal supervision.
Coverage repair: Graft pairs failed prompts with accepted topic-neutral refusals, while Escalate retries the same prompt through up to four tiers of resampling.These strategies are designed to recover failed refusal supervision rather than discard hard prompts.
Controlled generation: The harmful prompt hierarchy controls topic, subtopic, intent point, persona, style, paraphrasing, and prompt length.Length sampling is used to reduce the short-prompt bias of uncontrolled generation.
Boundary-aware data: PR is a harmful paraphrase that should be refused, while PB is a topically adjacent benign counterpart that should be answered.The generator varies refusal strength across clear, borderline, and mixed cases and rejects degenerate or off-topic pairs.
Boundary-aware data: PB2 supplies a separate comply-side build whose target-model responses pass WildGuard verification, while PR-OOD and PB-OOD use an earlier prompt source for out-of-distribution evaluation.The OOD pair set is not a held-out split of PR and PB.
Benign compensation: SC uses externally adopted compliance responses, whereas SC2 replaces them with verified target-model responses to the same prompt source.PB and PB2 provide local boundary compensation, and FakeHarm contains verified surface-dangerous benign prompts from an 18-type semantic grid.
Training objectives: Harmful examples use cross-entropy, while benign examples use forward-KL regularisation against the frozen reference model M0.This routing strengthens refusal within H while constraining changes on the benign complement Ω\H.

5 Experimental Setup

Experiments use Qwen3-8B with LoRA adapters, WildGuard-based refusal evaluation, broader harmfulness benchmarks, XSTest, and held-out boundary pairs. Figure 3 compares four coverage-repair strategies with and without SafeChain across matched epochs.

Qwen3-8B is the target model for the main experiments, using LoRA adapters with rank 32 and α = 16.
Evaluation covers political refusal, broader unsafe-response rates, XSTest over-refusal, and both sides of the held-out harmful-benign boundary.Higher refusal is desirable on harmful prompts, while lower values are desirable for harmful_unsafe_avg, XSTest, and PB-OOD over-refusal.
Figure 3 compares political refusal, harmful compliance, and over-refusal for four coverage-repair strategies at matched epochs 1, 4, and 12.The top row uses CE-only training, the bottom row adds SafeChain, and the untrained baseline is dashed.
The study compares its political refusal-tuning setup with ThinkSafe, which uses the same target model, adapter family, rank, and evaluation judges.ThinkSafe’s published broad-harmfulness results are stronger, but prompt subsets and decoding settings are not verified to be identical.

6 Results

Coverage repair strengthens harmful-request refusal and broader safety, but can sharply increase benign over-refusal. Compensation and boundary data address different failure modes, while metric coverage and checkpoint timing constrain interpretation.

Coverage repair and the safety–over-refusal trade-off: 0.20% residual generation failures under Escalate contrast with 19.88% discarded prompts under Single-shot generation.Coverage-oriented variants retain supervision for difficult prompts, without establishing a final performance ranking among generation strategies.
Coverage repair and the safety–over-refusal trade-off: 0.8475 political refusal and 0.0014 harmful_unsafe_avg at epoch 4 improve over 0.0947 and 0.2626, while XSTest over-refusal rises to 0.7400 from 0.0200.
Compensation data and distinct failure modes: 0.0520 versus 0.1520 XSTest over-refusal under Single-shot shows that verified target-model compliance responses reduce over-refusal, with a modest harmful_unsafe_avg increase.Under Graft, XSTest similarly falls from 0.2520 to 0.0440.
Compensation data and distinct failure modes: FakeHarm reduces both XSTest over-refusal and harmful_unsafe_avg relative to refusal-only training, while combining it with general compliance data further lowers over-refusal but raises unsafe responses relative to FakeHarm alone.
Evaluation caveat: The main broader-harm comparisons use LlamaGuard-3 unsafe-content scores because harmful_refusal_avg was not backfilled for all matched checkpoints.The study therefore does not report a dual-judge cross-check for omitted checkpoints.
Boundary behavior: 0.0416 comply-side PB-OOD over-refusal after adding PB improves on 0.3294, while PR-OOD refusal decreases from 0.9188 to 0.8772.The result supports improved local boundary precision with a smaller harmful-side refusal cost.
Checkpoint and prompt-construction effects: Later checkpoints can reduce over-refusal while worsening broader harmfulness, and length-controlled prompts yield higher political refusal than natural-length prompts across length buckets.

7 Conclusion

The paper frames safety as learning a deployment-specific refusal boundary within a topic, combining coverage repair, benign-behavior compensation, and boundary evaluation. Results show stronger refusal and safety can trade off against benign usability, so both sides of the boundary must be measured.

The framework learns a refusal boundary within a topic rather than imposing a topic-wide refusal policy.
The pipeline repairs missing refusal supervision, combines harmful cross-entropy with benign forward-KL preservation, and evaluates both sides of the held-out boundary.
SC2 compares verified target-model responses with externally adopted SafeChain responses, reaching lower XSTest over-refusal at a modest harmfulness cost.
FakeHarm reduces over-refusal while increasing harmful compliance from near zero, and is complementary when added on top of SC+PB.
Pairwise boundary data lowers comply-side over-refusal to 0.03–0.08 from 0.49, with harmful-side refusal at 0.88 against 0.92.
Compensation data reduces false-positive refusals, while pairwise benign data improves local boundary precision at a small recall cost.

Limitations

The evidence is limited in topic, model, and configuration scope, and broader harmfulness benchmarks may introduce judge-related uncertainty. Missing or incomplete refusal-score files further constrain some comparisons.

Evidence is limited to political persuasion, mainly Qwen3-8B with one LoRA configuration, so generalisation across topics, model families, and training setups is unestablished.Selected coverage-repair experiments also use DeepSeek-R1-Distill-Qwen-7B.
Broad harmfulness benchmarks measure general harmfulness rather than persuasion or deployment-specific neutrality, and their trade-offs may be affected by judge bias.The benchmarks are judged with LlamaGuard-3, with a complementary WildGuard aggregate reported separately.
Some harmful_refusal_avg comparisons are unavailable because required refusal-score files are missing for the external benchmarks.The paper reports results depending on this aggregate only when the required scores are present.

Ethical Considerations

The paper treats political safety data as sensitive because its generation pipeline could be repurposed for manipulation, while deployment boundaries remain normative choices requiring accountability. Reproducibility also depends on implementation details and model-specific evaluation components.

The political prompt-generation pipeline could be repurposed to produce manipulative content, although the data is constructed to train refusal rather than optimise persuasion.The authors plan to restrict dataset and code release to safety research.
Harmful and benign political boundaries are normative deployment choices, so policies should be transparent and accountable.Insufficient refusal can enable manipulation, whereas excessive refusal can restrict legitimate civic, educational, or public-information use.
Main experiments use Qwen3-8B, while DeepSeek-R1-Distill-Qwen-7B appears only in selected diagnostics, with model revisions not pinned in training scripts.Released checkpoints should record the downloaded revision.
Evaluation and training depend on specified model components, adapter settings, sampling parameters, and automated guard-model checks.The implementation uses vLLM for generation and evaluation, WildGuard for refusal-related checks, and LlamaGuard-3 for unsafe-content measurement.

A.5 Compute and Software

The appendix documents the compute environment, software versions, dataset-component naming, composition rules, and held-out evaluation-file construction. It also records audit and aggregation caveats relevant to reproducibility.

All reported training runs use one H200 GPU and one process, without distributed training; verified wall-clock time is not reported.The supplied logs do not provide verified training duration.
The principal software stack includes torch 2.7.1+cu128, transformers 4.55.0, peft 0.17.0, trl 0.21.0, accelerate 1.10.0, and vllm 0.10.1.The reported versions also include datasets, wandb, and flash_attn.
CE, PR, PB, PB2, SC, SC2, and FakeHarm denote distinct training components that differ in prompt sources, response sources, verification, or benign-content construction.SC2 replaces SafeChain responses with verified target-model generations, while PB2 tests response verification on a target-model-generated source.
Composite datasets concatenate named components in fixed order without resampling or truncation, so audited mixtures can be reconstructed from component counts.The component order is CE, SC or SC2, PB or PB2, PR, and FH.
The held-out political file contains 1,540 prompts from an earlier pipeline version, while PR-OOD and PB-OOD each contain 1,539 prompts generated from that source.A full near-duplicate audit between training and evaluation sources has not been completed.

B.6 Constructed Religion Dataset Inventory

The paper documents a separately constructed religion-domain dataset and its metadata, but no religion evaluation files or experiments are available. Consequently, the reported experimental claims remain restricted to politics.

The religion dataset is a constructed training artifact, not an experimental result, and its corresponding evaluation files have not been built.The missing files include in-distribution, held-out construction, PR-OOD, and PB-OOD tests.
Religion base builds contain 37,131 to 40,585 harmful refusal rows, with five persona buckets represented at similar rates.The PR artifact mixes politics and religion persona-bucket metadata, so it is not purely religion-persona data.
The controlled prompt pipeline expands topics into subtopics, generates intent points and persona-conditioned prompts, then flattens and deduplicates outputs while retaining metadata.Retained metadata covers subject, subtopic, point, persona, and target length.
The persona pool contains 38 entries, with only moderate personas mapped to the plain harmful label and other buckets mapped to the adversarial harmful label.The pool includes moderate, radical-right, radical-left, geopolitical, authoritarian, and other framings.
Single-shot generation retains harmful rows only when WildGuard identifies a refusal, while Graft repairs failed refusal prompts by attaching accepted neutral refusals.Graft is defined only for refusal data because compliant answers depend on each prompt's content; Escalate instead retries failed checks with stronger steering.
The v2 dropped file contains 8,009 failed rows, whereas the v3 residual contains 79 rows after retry escalation.The v2 count includes 8,008 ordinary refusal mismatches and one unparseable WildGuard output.
Boundary evaluation separates refusal-side recall from comply-side over-refusal across 1,539 prompts per side, while external harmfulness aggregates require complete component scores.Higher refusal-side recall and lower comply-side over-refusal represent different objectives.
Qwen3-8B political refusal rises from a baseline of 0.0947, but outcomes remain model- and recipe-dependent, and over-refusal measures are the least stable metrics.Across matched epochs, the largest seed differences are 0.065 for PB-OOD over-refusal and 0.080 for XSTest over-refusal.

F.8 Agreement Between Harmfulness and Refusal Metrics

The paper compares harmfulness and refusal aggregates across training runs to determine whether they track the same safety trade-off. Agreement is strong when training components provide enough metric variation, while weak correlations in CE-only and PB-only groups are consistent with floor and ceiling effects.

Pearson and Spearman correlations are computed within each run across evaluated epochs, then averaged within four training-data groups.The analysis tests whether unsafe-content and refusal aggregates describe the same training trade-off.
The two metrics agree closely for SC, SC2, and FakeHarm-containing groups.These components introduce enough variation for both judges to distinguish changes across epochs.
CE-only and PB-only groups show weaker, less consistent correlations alongside mean ranges below 0.012 for both metrics.The limited within-run variation constrains how informative the correlations can be.
The weak correlations are consistent with floor and ceiling effects rather than opposite safety trends.The aggregate file does not expose the contributing benchmark fields separately, and the exact coefficients assume the common three-benchmark composition.
Two-seed results support a sensitivity check, not a robust variance estimate or significance claim.The reported stability check uses mean ± sample standard deviation over seeds 42 and 123, with n = 2 for each recipe.