PaperlayerView exact revision on arXiv

Source-linked AI summary

Decoupled Safety Control: A Safety-Control Algorithm for Training-Free Safety Guidance

Huilin Zhou, Ruoxi Cheng, Yuhang Wang, Yuming Liu, Minghao Sun, Ruolong Ma, Lan Zhang

arXiv:2610.21569v1cs.CV

TL;DR

Existing training-free safeguards often tie multiple intervention decisions to one unsafe condition, which can be inadequate when harmful and benign semantics are entangled. DSC makes intervention an explicit state-dependent process during sampling. On standard safety benchmarks, it improves representative safeguards’ unsafe-content suppression while preserving benign-generation quality.

  • Problem

    Many training-free safeguards use one unsafe condition to specify avoidance, identify unsafe signals, construct interventions, and determine their duration, a coupling that is less reliable when harmful and benign semantics are entangled.

    Source: §1 · p. 2

  • Method

    DSC makes safety guidance an explicit state-dependent control process, selecting and executing interventions from the current generation state rather than inheriting them from a single unsafe condition.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across representative latent- and text-space safeguards, DSC strengthens unsafe-content suppression while preserving benign-generation quality; on MMA-Diffusion, ASR is reduced by 80.8% on SLD.

    Source: §4.1 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    Explicit runtime control can improve training-free safeguards without modifying the base model.

    Source: §5 · p. 10

    The evaluation uses released prompt sets with automatic safety evaluators supplemented by human annotation, and does not fully capture all deployment settings.

    Source: A Limitations and Broader Impacts · p. 14

Abstract

from arXiv · show

Diffusion models achieve strong performance in text-to-image generation, but they can still produce unsafe content. Training-free safety guidance offers a practical inference-time alternative that intervenes during sampling without modifying model weights. However, many existing safeguards still rely on a coupled safety-control chain. In this chain, the same unsafe condition simultaneously specifies what should be avoided, indicates the current unsafe signal, constructs the intervention, and determines when that intervention remains active. This coupling can work when unsafe factors are explicit, but becomes less reliable once harmful and benign semantics are entangled in the current generation state, often leaving residual unsafe content or unnecessarily degrading benign-generation quality or prompt fidelity. We address this problem with Decoupled Safety Control (DSC), a plug-in safety-control algorithm for training-free safety guidance. DSC reformulates training-free safety guidance as an explicit state-dependent control process, so the executed intervention is determined from the current generation state rather than inherited directly from a single unsafe condition. Experiments on standard safety benchmarks show that DSC improves representative latent- and text-space safeguards, achieving stronger unsafe-content suppression while preserving benign-generation quality and maintaining efficient inference.

1 Introduction

Training-free safety guidance avoids changing model weights, but many safeguards couple several control decisions to one unsafe condition. DSC makes intervention state-dependent, and benchmark results show stronger suppression while preserving benign-generation quality.

1 Introduction

  • Training-free safeguards intervene during inference without changing model weights, unlike training-based methods that require optimization or parameter modification and may harm benign quality.These methods provide a plug-and-play alternative to weight-level editing.Source: §1 · p. 1
  • Many safeguards use the same unsafe condition to define what to avoid, identify unsafe signals, construct corrections, and decide how long corrections remain active.This coupling can work when unsafe factors are explicit and aligned, but becomes inadequate when harmful and benign semantics are entangled.Source: §1 · p. 2
  • Entangled semantics can make coupled interventions either leave residual unsafe content or degrade benign quality and prompt fidelity through overly broad suppression.The paper connects this failure pattern with prior observations involving hard filtering, desired tasks, prolonged negative guidance, and complex prompts.Source: §1 · p. 2
  • DSC makes runtime evidence extraction, control-state estimation, intervention routing, update construction, and phase-aware execution explicit, with decisions based on the current generation state.It is instantiated with representative latent- and text-space safeguards while keeping the base model unchanged.Source: §1 · p. 2
  • Experiments on standard safety benchmarks show that DSC improves representative latent- and text-space safeguards, strengthening unsafe-content suppression while preserving benign-generation quality and efficient inference.The reported evaluation is under the evaluated backbones.Source: §1 · p. 2

2 Related Work

Prior work addresses safety through weight modification, inference-time latent or prompt interventions, and changes to the sampling trajectory. This paper focuses instead on how the safety-control chain is organized.

Training-based safety methods

  • Training-based approaches use concept erasure, model editing, or additional optimization, but require retraining or parameter changes and may degrade benign-generation quality.The paper distinguishes its inference-time focus from these weight-modification approaches.Source: §2 · p. 3

Training-free latent- and prompt-space safeguards

  • Training-free latent- and prompt-space safeguards intervene without modifying weights; SLD uses predefined safety prompts, while SAFREE filters unsafe prompt representations and adaptively reweights attention.The paper studies these as representative safeguards whose interventions derive from pre-specified unsafe concepts, subspaces, or references.Source: §2 · p. 3

Trajectory-level safety guidance

  • Trajectory-level safeguards change the denoising path using unsafe references, repulsive fields, or time-varying negative guidance; this work instead examines the organization of the safety-control chain.Examples include Safe Denoiser, Shielded Diffusion, and Safety-Guided Flow (SGF).Source: §2 · p. 3

3 Method

The method formulates training-free safe generation as state-dependent control: the unsafe condition remains fixed, while each intervention is resolved from the current generation state. DSC makes that process explicit and applies it to SLD and SAFREE without changing their underlying safeguard pipelines or model weights.

3.1 Problem Formulation

  • Training-free safe generation must suppress unsafe content while preserving unrelated benign prompt semantics, especially when harmful and benign semantics are entangled in the trajectory.The formulation treats intervention choice, what to suppress or preserve, and intervention strength as current-step control decisions.Source: §3.1 · p. 4

3.2 Decoupled Safety Control

  • At each denoising step, DSC extracts runtime evidence, estimates route-wise control states, selects a route, constructs its update, and executes it phase-aware.Runtime evidence is the control-relevant observation from the current generation state; it can include denoising-branch discrepancies, prompt-side filtering cues, predicted clean-sample relations, or support patterns.Source: §3.2 · p. 4
  • The three routes provide suppressive, preservation-aware, or release behavior, selected from current evidence rather than directly from the unsafe specification.Admissibility requires compatibility with benign preservation, limited suppressive-preservation leakage, and reliability on the current support scope; DSC then selects the admissible route with the strongest unsafe pressure.Source: §3.2 · p. 5
  • The selected route determines a safeguard-specific correction, while execution strength and release timing regulate how much correction is applied.A correction is stronger with high unsafe pressure and milder with greater preservation demand or suppressive-preservation leakage; the control-stage decomposition remains unchanged across safeguards.Source: §3.2 · p. 5, §3.3 · p. 6

3.3 Applied to SLD and SAFREE

  • For SLD, DSC uses prompt, unconditional, and safety-concept denoising branches as runtime evidence and realizes the selected route as a latent update.The prompt-preserving denoising direction supplies the preservation direction, and execution uses the shared phase-aware mechanism without replacing the SLD safeguard condition.Source: §3.3 · p. 6
  • For SAFREE, DSC uses prompt-side subspace and filtering cues with their induced denoising discrepancy, then applies a SAFREE-specific correction.The filtered benign prompt representation supplies the preservation direction; DSC leaves the underlying safeguard pipeline and model weights unchanged.Source: §3.3 · pp. 6–7

4 Experiments

Across the evaluated safety benchmarks, DSC improves safety for latent- and text-space safeguards while maintaining competitive benign-generation quality. Runtime analyses show how DSC routes interventions using the current generation state, and the SD-v3 and timing results assess compatibility and overhead.

4.1 Safe Generation against Nudity Prompts

  • On MMA-Diffusion, ASR falls by 80.8% for SLD and 72.2% for SAFREE; on UnlearnDiff, it falls by 90.5% and 61.2%, respectively.TR follows the same pattern across latent- and text-space safeguards.Source: §4.1 · pp. 7–8
  • Compared with SGF, DSC improves FID on both safeguards while matching or improving CLIP, preserving competitive benign-generation quality.For SAFREE, FID improves from 23.73 to 20.95 with CLIP at 30.36; for SLD, FID improves from 36.83 to 32.68 and CLIP rises from 28.13 to 28.65.Source: §4.1 · p. 8

4.2 Inappropriate Probability on CoPro Dataset

  • Avg. IP drops from 0.170 to 0.131 on SLD and from 0.153 to 0.088 on SAFREE across CoPro’s seven unsafe categories, with competitive CLIP scores of 29.02 and 28.38.Table 2 reports IP and CLIP on CoPro; lower IP and higher CLIP are better.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 Mechanism Analysis

  • Replacing any individual DSC control stage with its SAFREE-default behavior worsens ASR and TR, supporting the coordinated design of the stages.The ablation evaluates each stage replacement under the same SAFREE pipeline on MMA-Diffusion.Source: Table 3 · p. 8, §4.3 · p. 9
  • At runtime, DSC-selected intervention routes account for most sample-step decisions, while SAFREE-default selection remains small.Selection rates are computed over MMA-Diffusion samples at each denoising step.Source: Figure 3 · p. 8, §4.3 · p. 9
  • The routing margin is positive in 82.4% of explicit-control decisions, and update norms rise with unsafe pressure but fall with preservation demand.Route-conditioned update norms increase from 1.13 to 2.01 across unsafe-pressure quartiles and decrease from 1.95 to 1.15 across preservation-demand quartiles.Source: Figure 4 · p. 9, §4.3 · p. 9

4.4 Compatibility and Efficiency

  • On SD-v3, adding DSC to SAFREE reduces Ring-A-Bell ASR from 0.278 to 0.038 and TR from 0.298 to 0.073, while improving COCO-30K FID from 22.99 to 17.54 and keeping CLIP at 30.82.These results are reported for the SD-v3 evaluation setting.Source: Table 4 · p. 9, Compatibility with SD-v3 · p. 9
  • Under the shared COCO-3K protocol on one NVIDIA H100 80GB HBM3 GPU, adding DSC to SAFREE costs 0.158 seconds per image.Under the same setting, SAFREE+SGF runs at 5.181 seconds per image.Source: Table 5 · p. 9, Compatibility with SD-v3 · p. 9

5 Conclusions

The paper identifies excessive coupling in training-free safety guidance and proposes DSC as an explicit state-dependent control process. Across representative latent- and text-space safeguards, it reports reduced unsafe generation while preserving benign-generation quality under the evaluated settings.

5 Conclusions

  • The paper identifies an often-too-coupled safety-control chain as a mechanism-level limitation in training-free safety guidance.Source: §5 · p. 10
  • DSC makes training-free intervention an explicit state-dependent control process during sampling.Source: §5 · p. 10
  • Across representative latent- and text-space safeguards, DSC reduces unsafe generation while preserving benign-generation quality under the evaluated settings.The paper states this can improve training-free safeguards without modifying the base model.Source: §5 · p. 10

A Limitations and Broader Impacts

DSC is an inference-time safeguard for text-to-image generation, evaluated with released prompt sets, automatic safety evaluators, and human annotation. The authors caution that these protocols do not fully capture deployment settings.

Limitations

  • DSC is an inference-time safeguard for text-to-image generation, and the paper frames it as a practical layer of safety control.The work aims to improve safety without changing the base model.Source: A Limitations and Broader Impacts · p. 14
  • The empirical evaluation uses released prompt sets, automatic safety evaluators, and human annotation.The authors say these protocols provide a consistent basis for comparison but do not fully capture all deployment settings.Source: A Limitations and Broader Impacts · p. 14

Broader impacts

  • Responsible deployment still requires benchmark-aware evaluation and appropriate review because safety mechanisms may be probed or circumvented.The broader-impacts discussion also calls for careful handling of prompts and generated assets.Source: A Limitations and Broader Impacts · p. 14

B Experimental Details

The experimental setup follows released protocols for settings shared with SGF and Safe Denoiser. Main experiments apply DSC to SLD and SAFREE using SD-v1.4, with evaluation conditions held consistent across methods.

Experimental details

  • Main experiments use SD-v1.4 as the base model and evaluate DSC with SLD and SAFREE.For settings shared with SGF and Safe Denoiser, the authors follow their released protocols and implementations.Source: B Experimental Details · p. 14
  • Methods use the same prompt sources, checkpoints, samplers, and evaluators unless otherwise noted.Source: B Experimental Details · p. 14
  • The shared-protocol settings follow the released implementations of SGF and Safe Denoiser.Source: B Experimental Details · p. 14

B.1 Safe Generation against Nudity Prompts

The nudity benchmarks assess safety on adversarial prompts and generation fidelity on COCO, using shared model, sampling, and evaluator settings. The appendix also specifies benchmark subsets, compute conditions, and the sources and terms for shared assets.

B.1 Safe Generation against Nudity Prompts

  • ASR and TR measure safety on adversarial nudity prompts, while FID and CLIP score measure benign generation on COCO.Runs use SD-v1.4, DDPM sampling, and 50 inference steps; ASR uses a NudeNet probability threshold of 0.6, and TR is mean nude-class probability.Source: §B.1 · p. 14
  • Shared evaluation assets include released prompts, model checkpoints, repositories, and evaluators from the original public sources.The paper says these third-party assets are used under applicable repository terms or model licenses and are not redistributed.Source: §B.1 · p. 14
  • COCO-30K evaluation reports FID and CLIP on the first 10K prompt-image pairs in Table 1 and also reports full-set metrics in the appendix.COCO is used only to measure generation fidelity and prompt alignment.Source: §B.1 · p. 14
  • Inference-time overhead is measured on one NVIDIA H100 80GB HBM3 GPU under the shared COCO-3K protocol.Source: §B.1 · p. 15

B.1.1 Nudity in NSFW Prompt Datasets

  • Ring-A-Bell uses a curated 79-prompt red-team nudity subset aligned with recent training-free baselines.The authors follow the released prompt sources used in SGF and Safe Denoiser for comparison.Source: §B.1.1 · p. 15
  • MMA-Diffusion contributes 1,000 curated adversarial prompts that remain harmful without explicit unsafe expressions.The same safety metrics as Table 1 are reported.Source: §B.1.1 · p. 15
  • UnlearnDiff contributes a nudity-related split of 116 harmful prompts, with the protocol aligned to recent safeguard evaluations.The prompt protocol is kept consistent with SGF and Safe Denoiser.Source: §B.1.1 · p. 15

B.2 Inappropriate Probability on CoPro Dataset

The experiments specify a CoPro multi-category safety evaluation and a separate compatibility evaluation on SD-v3. Both use shared sampling and evaluator settings for their respective comparisons.

B.2 Inappropriate Probability on CoPro Dataset

  • CoPro-10K evaluates seven unsafe categories using Inappropriate Probability and CLIP score under the SD-v1.4, DDPM, 50-step protocol.The evaluation uses the shared prompt split and evaluator settings; Inappropriate Probability follows the Q16-based protocol.Source: §B.2 · p. 15
  • The CoPro evaluation uses the LatentGuard repository as its listed resource.Source: §B.2 · p. 15

B.3 Compatibility with SD-v3

  • SD-v3 compatibility compares SAFREE alone with DSC applied to SAFREE using Ring-A-Bell safety metrics and COCO-30K fidelity metrics.The evaluation uses the official SD3-medium checkpoint and the same sampling and evaluation settings for both conditions.Source: §B.3 · p. 15

B.4 Ablation Studies

The appendix details a controlled SAFREE ablation protocol and adds qualitative and full-prompt COCO-30K evidence on safety and benign-generation quality.

B.4 Ablation Studies

  • The five single-stage ablations replace one DSC control stage with SAFREE-default behavior while keeping the other stages and experimental settings fixed.They test runtime evidence extraction, control-state estimation, intervention routing, update construction, and phase-aware execution on MMA-Diffusion using the main-text ASR and TR protocol.Source: §B.4 · p. 15

C Additional Experimental Results

  • Figures C.1 and C.2 show examples in which the method avoids targeted unsafe content, consistent with lower ASR and TR in Table 1.The appendix describes these as additional qualitative examples beyond Figure 1.Source: §C.1 · p. 16, Figure C.1 · p. 16, Figure C.2 · p. 16

C.1 Safe Generation against Nudity Prompts

  • On the full COCO-30K prompt set, the method maintains competitive CLIP scores and low FID for both SLD and SAFREE.The appendix pairs Table C.1's FID and CLIP results with the first 300 generated images as additional evidence of benign-generation quality.Source: §C.1 · p. 16, Table C.1 · p. 16, Figure C.3 · p. 17

C.2 Inappropriate Probability on CoPro Dataset

The appendix adds CoPro and SD-v3 examples and documents how DSC constructs and configures its safeguard-specific updates.

C.2 Inappropriate Probability on CoPro Dataset

  • On selected CoPro illegal-activity prompts, adding DSC guides outputs toward safer alternatives while preserving broad scene context.The examples include animal crates, uniformed personnel, and open-field settings; Table 2 reports lower IP while maintaining competitive prompt alignment.Source: §C.2 · pp. 17–18, Figure C.4 · p. 17

C.3 Compatibility with SD-v3

  • On the shown SD-v3 Ring-A-Bell prompts, SAFREE + Ours avoids targeted unsafe content more consistently than the compared safeguard.The passage states this qualitative comparison is consistent with Table 4.Source: §C.3 · p. 18, Figure C.5 · p. 18

D.1 Evidence and update construction

  • For SLD, the preservation direction is the preservation reference; for SAFREE, it comes from the filtered branch.The preservation reference is formed from the raw text-conditioned update and baseline safeguard correction.Source: §D.1 · p. 19

D.2 Fixed configuration

  • DSC configurations are fixed per safeguard on a calibration split disjoint from test benchmarks and shared across benchmarks without per-benchmark tuning.The fixed parameters are listed in Table D.2.Source: §D.2 · p. 19, Table D.2 · p. 19

E Statistical Analysis and Supplementary Evaluation

Supplementary analyses report prompt-cluster intervals and additional evaluator and human comparisons for DSC. The checklist also records reproduction details and omissions in human-subject reporting.

E.1 Prompt-cluster confidence intervals

  • Both prompt-cluster bootstrap paired intervals for MMA-Diffusion ASR lie below zero, supporting lower unsafe generation for DSC than SGF under the evaluated protocol.Table E.1 defines paired Δ as DSC minus SGF within the same safeguard, with negative values favoring DSC.Source: §E.1 · p. 20, Table E.1 · p. 20

E.2 Additional automatic evaluators and human annotation

  • DSC has lower unsafe rates than SGF under each of five MMA-Diffusion evaluation protocols and lower CoPro seven-category averages under each reported protocol.The protocols include automatic evaluators and human annotation; Tables E.2 and E.3 report the comparisons.Source: §E.2 · p. 20, Table E.2 · p. 20, Table E.3 · p. 20

NeurIPS Paper Checklist

  • The paper and appendix document the assets, settings, evaluators, and protocols used for the main results, but include no code or data release package.The checklist reports that reproducibility is documented in the paper and appendix.Source: NeurIPS Paper Checklist · pp. 22–23
  • For human image annotation, full participant instructions and compensation details, as well as institutional-review status and participant risk disclosures, are not reported.The checklist notes that the appendix describes annotation coverage and method blinding.Source: NeurIPS Paper Checklist · pp. 26–27

Related papers

Loading 2610.21569v1…