PaperlayerView exact revision on arXiv

Source-linked AI summary

When Safety Collides: Resolving Multi-Category Harmful Conflicts in Text-to-Image Diffusion via Adaptive Safety Guidance

Yongli Xiang, Ziming Hong, Zhaoqing Wang, Xiangyu Zhao, Bo Han, Tongliang Liu

arXiv:2602.20880v4cs.CV

TL;DR

T2I safety guidance can fail when aggregating multiple harmful categories because their directions interfere, creating harmful conflicts. CASG dynamically identifies the category aligned with the current generative state and applies guidance only along that category, with experiments showing reduced harmful content and preserved benign image quality.

  • Problem

    Existing T2I safety methods often aggregate multiple harmful categories despite evidence that their safety directions can interfere and degrade safety.

  • Method

    CASG is a training-free, plug-and-play framework using CaCI to identify the dominant aligned category and CrGA to apply safety correction only along it.

  • Results

    Experiments across multiple benchmarks show that CASG significantly reduces harmful content while preserving image quality on benign prompts.

  • Takeaways & Limitations

    Category-consistent guidance provides a framework for resolving harmful conflicts in both latent- and text-space safeguards.

  • Takeaways & Limitations

    Prompt-category matching through classifiers or large language models requires additional training or external API calls and does not handle dynamically evolving denoising conflicts effectively.

Abstract

from arXiv · show

Text-to-Image (T2I) diffusion models have demonstrated significant advancements in generating high-quality images, while raising potential safety concerns regarding harmful content generation. Safety-guidance-based methods have been proposed to mitigate harmful outputs by steering generation away from harmful zones, where the zones are averaged across multiple harmful categories based on predefined keywords. However, these approaches fail to capture the complex interplay among different harm categories, leading to "harmful conflicts" where mitigating one type of harm may inadvertently amplify another, thus increasing overall harmful rate. To address this issue, we propose Conflict-aware Adaptive Safety Guidance (CASG), a training-free framework that dynamically identifies and applies the category-aligned safety direction during generation. CASG is composed of two components: (i) Conflict-aware Category Identification (CaCI), which identifies the harmful category most aligned with the model's evolving generative state, and (ii) Conflict-resolving Guidance Application (CrGA), which applies safety steering solely along the identified category to avoid multi-category interference. CASG can be applied to both latent-space and text-space safeguards. Experiments on T2I safety benchmarks demonstrate CASG's state-of-the-art performance, reducing the harmful rate by up to 15.4% compared to existing methods.

1. Introduction

T2I diffusion models offer powerful image generation but can produce harmful content, while existing multi-category safety guidance can create harmful conflicts. The paper introduces CASG, which dynamically selects and applies category-aligned guidance to address this problem.

  • Open-source T2I models combine advanced generation capabilities with broad accessibility, intensifying concerns about harmful content generation.
  • Existing safety-guidance methods concatenate harmful keywords into one direction, implicitly assuming different harm categories can be mitigated jointly.
  • Distinct category-specific safety directions can become misaligned, opposed, or mutually canceling when aggregated, producing harmful conflicts in latent and text space.
  • Mismatched guidance raises the harmful rate to 72.4%, compared with 3.2% for the correct sexual direction and 67.2% for unguided generation.
  • CASG is a training-free framework whose CaCI identifies the category most aligned with the evolving state and whose CrGA applies safety correction only along that category.
  • The paper identifies harmful conflicts as a previously overlooked safety issue and reports extensive benchmark evidence that CASG mitigates harmful concepts.

2. Related Work

Prior T2I safety approaches include diffusion foundations, latent-space generation, filtering, model editing, and directional safeguards. Existing safeguards remain limited by jailbreak vulnerability or naive multi-category keyword aggregation.

  • DDPMs generate images through a forward noising process and a learned reverse process that iteratively removes noise.
  • Latent diffusion models encode images into lower-dimensional representations, perform diffusion there, and decode the resulting latents into images.
  • Filtering methods remove unsafe concepts before or after generation but remain vulnerable to jailbreak attacks exploiting filtering weaknesses.
  • SLD uses latent-space directional guidance, whereas SAFREE uses text-space orthogonal projection to suppress harmful semantics.
  • These safeguards typically address single-category harmfulness or concatenate keywords naively, overlooking interactions among harmful categories.

3. Harmful Conflicts

Harmful conflict occurs when safety directions for multiple categories interfere in latent or text space, degrading safety control. The paper identifies directional inconsistency and directional attenuation as two mechanisms behind this degradation.

  • Safety Mechanisms: Latent- and text-space safeguards steer generation away from harmful semantics, but combining multiple categories can introduce harmful interference.
  • Empirical Evidence: Category-specific directions can intersect or oppose one another, and their relationships change across diffusion timesteps.
  • Harmful Conflict: Harmful conflict describes inconsistent, overlapping, opposing, or canceling safety directions applied jointly across harmful categories.
  • Empirical Evidence: Directional attenuation is measured with CDRR, where larger values indicate stronger alignment and smaller or negative values indicate attenuation or opposition.
  • Safety Degradation: Mismatched guidance produces safety misalignment degradation, with the correct sexual direction reducing attack success from 67.2% to 3.2%.
  • Safety Degradation: Aggregating all categories raises the harmful rate to 48.8%, while hate+sexual reaches 5.8% versus 3.2% for sexual guidance alone.
  • Safety Degradation: Consistent degradation patterns across base models, safety mechanisms, and harmful-keyword definitions show that unresolved conflicts reduce overall safety performance.

4. Conflict-aware Adaptive Safety Guidance

CASG resolves harmful conflicts by identifying the harmful category most aligned with the evolving generation state and applying safety correction only along that category. It supports both latent-space steering and text-space projection through CaCI and CrGA.

  • Framework overview: CASG is a plug-and-play framework that inserts conflict-aware correction into existing latent-space and text-space safeguards.Its two instantiations are CASG+SLD and CASG+SAFREE.
  • Latent-space guidance: CrGA applies SLD only along the selected latent harmful direction while retaining the remaining SLD mechanisms and hyperparameters.Replacing the aggregated direction makes the latent update targeted and reduces multi-category interference.
  • Latent-space guidance: For latent-space safeguards, CASG generates category-specific harmful directions and prompt guidance from conditional and unconditional noise predictions.Each harmful guidance is computed by subtracting unconditional noise from harmful-conditioned noise.
  • Latent-space guidance: CaCI identifies the dominant latent harmful category as the one with the highest cosine similarity to the prompt guidance.Higher cosine similarity indicates stronger alignment with the current generative trajectory.
  • Text-space guidance: For text-space safeguards, CASG selects the category with the smallest projection residual and applies SAFREE’s orthogonal projection only to that category’s harmful subspace.The residual measures how much prompt embedding remains after removing components aligned with a harmful concept.
  • Conflict resolution: Overall, CASG dynamically applies guidance from the most aligned harmful category instead of aggregating inconsistent multi-category signals.This design is intended to mitigate harmful conflicts during generation.

5. Experiments

Experiments evaluate CASG across four harmful-content benchmarks and benign COCO prompts against model-editing, guidance-based, and alignment-based baselines. CASG consistently improves safety while preserving competitive generation quality, and dynamic timestep updates outperform fixed LLM-assigned categories.

  • Experimental setup: CASG is evaluated on I2P, T2VSafetyBench, Unsafe Diffusion, and CoProv2, with benign quality assessed on 1,000 COCO captions.The evaluation covers diverse harmful categories and uses Q16, NudeNet, CLIP, and FID.
  • Experimental setup: The comparison includes model-editing methods, guidance-based methods, and the alignment-based SafetyDPO baseline.Experiments use official implementations or default hyperparameters on Stable Diffusion v1.5.
  • Safety results: CASG consistently enhances safety across all benchmarks while preserving competitive generation quality.The authors report robust safety control without compromising visual fidelity.
  • Safety results: CASG+SAFREE records harmful rates of 18.9%, 37.5%, 17.5%, and 11.8% across I2P, T2VSafetyBench, Unsafe-Diffusion, and CoProv2, respectively.These results are reported as improvements over SAFREE.
  • Safety results: CASG+SLD records harmful rates of 10.2%, 9.8%, 9.8%, and 3.9% on the same four benchmarks, achieving state-of-the-art safety among compared methods.The reported rates outperform the SLD baseline.
  • Quality results: CASG preserves near-baseline quality, with CASG+SAFREE reporting FID 46.3 and CLIP 30.4, and CASG+SLD reporting FID 52.0 and CLIP 29.4.These values remain close to the respective SAFREE and SLD baselines.
  • Additional analysis: LLM-assisted SLD variants show moderate or worse performance because category assignments can misclassify ambiguous prompts and remain fixed during generation.CASG+SLD updates the category dynamically at every timestep and achieves stronger safety performance.
  • Qualitative results: Qualitative comparisons show CASG removing weapons, blood, and explicit exposure while preserving semantic alignment across violence and sexual-content prompts.The comparison covers rows for violence and inappropriate-content prompts.

6. Conclusion

CASG identifies harmful conflicts in text-to-image safety and dynamically selects the category-aligned safety direction during generation. It applies this strategy across latent- and text-space safeguards, with analyses showing persistent directional divergence and attenuation across categories, prompts, and timesteps.

  • Contribution: CASG is a training-free, plug-and-play framework that dynamically identifies the dominant harmful category while preserving image quality on benign prompts.It integrates with existing text- and latent-space safeguards.
  • Conclusion: The extended analyses show harmful conflicts consistently across prompts, harmful categories, timesteps, and both latent-space and text-space safeguards.The visualizations complement the main-paper analyses across these settings.
  • Conflict Visualization: Category-wise safety directions diverge persistently across prompts and timesteps in latent space, indicating systematic rather than isolated inconsistency.The directions are computed from harmful- and prompt-conditioned noise predictions and visualized in the top three PCA dimensions.
  • Conflict Visualization: In text space, category-specific safety directions are represented as residuals after projecting prompt embeddings away from harmful subspaces.This provides the text-space counterpart to latent-space safety-direction analysis.
  • Directional Attenuation: Multi-category aggregation can sharply attenuate individual category contributions, with heterogeneous directions partially canceling one another.CDRR measures how much of each category-specific direction is retained or opposed in the aggregated direction across timesteps.

A.3. Safety Degradation

Additional experiments test whether safety degradation persists across keyword granularities, safeguard mechanisms, and base models. The results consistently attribute degradation to cross-category misalignment and aggregation rather than to a particular keyword definition or model setting.

  • Evaluation Scope: Safety degradation is evaluated across different harmful keyword definitions, safety-guidance methods, and base models using a consistent protocol.Harmful rates are reported across settings to assess degradation arising from harmful conflicts.
  • Keyword Definitions: Three keyword configurations test whether harmful conflicts arise from category interactions rather than wording or granularity.The configurations include default, coarse category-level, and fine-grained keyword sets.
  • Keyword Definitions: All three keyword settings exhibit both Safety Misalignment Degradation and Safety Averaging Degradation.The consistency across settings indicates that the patterns are independent of the particular harmful-keyword choice.
  • Different Safeguards: Safety degradation is examined under SAFREE, extending analysis beyond the latent-space SLD safeguard to text-space guidance.SAFREE removes harmful semantics through orthogonal projection of prompt embeddings.
  • Different Base Models: SDv3 exhibits the same cross-category misalignment and aggregation degradation observed in SDv1.5.Using a mismatched category weakens suppression, while aggregating categories weakens category-specific steering.

C.1. Datasets

The evaluation uses four safety datasets spanning diverse harmful categories, prompt sources, and severity levels, alongside benign-image quality assessment. Comparisons include model-modification and modification-free safeguards under consistent evaluation settings.

  • Evaluation Datasets: CASG is evaluated on I2P, T2VSafetyBench, Unsafe Diffusion, and CoProv2, covering diverse safety contexts.The datasets are designed to assess safety in image generation.
  • I2P: I2P contains 4,703 harmful prompts spanning seven categories sourced from real-world instances and language-model generations.Its categories include hate, harassment, violence, self-harm, sexual content, shocking images, and illegal activities.
  • T2VSafetyBench: T2VSafetyBench contributes 3,443 prompts across seven harmful categories aligned with the predefined evaluation categories.The selected categories include pornography, violence, gore, disturbing content, public figures, and illegal activities.
  • Unsafe Diffusion: Unsafe Diffusion contributes 904 user-contributed prompts spanning sexually explicit, violent, disturbing, hateful, and political content.The dataset covers five harmful categories.
  • CoProv2: CoProv2 contains more severe harmful prompts generated by language models, with 1,000 prompts randomly sampled for evaluation.It shares the same categorical structure as I2P.
  • Baselines: Comparisons include four model-modification methods and two modification-free methods using official implementations and consistent harmful keywords.The modification-free baselines are SLD and SAFREE.

D.1. Hyperparameter Analysis

The CASG algorithm selects the harmful category most aligned with the current prompt guidance at each timestep, then applies the corresponding latent- or text-space safeguard. Under stronger prompt guidance, the framework continues to improve safety while maintaining nearly unchanged generation quality.

  • CASG Algorithm: CASG processes harmful categories at each timestep and computes category-specific harmful directions before selecting the most aligned category.The latent-space branch computes harmful-conditioned noise and measures alignment with prompt guidance.
  • CASG Algorithm: The selected category determines whether CASG applies SLD steering in latent space or SAFREE projection in text space.Both branches return a safety-modified noise estimate or embedding.
  • Safety Results: CASG+SAFREE achieves harmful rates of 18.0%, 37.6%, 18.0%, and 10.8% across I2P, T2VSafetyBench, Unsafe-Diffusion, and CoProv2 under the analyzed setting.These results outperform the SAFREE baseline.
  • Safety Results: CASG+SLD achieves harmful rates of 12.3%, 13.3%, 14.5%, and 5.0% across the same four datasets.CASG+SLD is reported to achieve state-of-the-art safety among compared methods.
  • Generation Quality: Despite stronger safety control, CASG+SAFREE and CASG+SLD show only marginal FID and CLIP variations relative to their baselines.The passage reports nearly unchanged generation quality under the analyzed setting.

D.2. Robustness to Keyword Variants

CASG remains robust when harmful keywords are replaced with synonyms or varied in specificity, consistently reducing ASR across keyword settings. Category-wise results further show that its safety gains vary by harm category, reflecting differing harmful conflicts.

  • Keyword-variant robustness: CASG consistently reduces ASR across synonym, granularity, and completeness variants of predefined harmful keywords on T2VSafetyBench.The evaluation covers four keyword sets and indicates robustness to keyword selection and specificity.
  • Category-wise effects: CASG+SLD reduces harmful rates most substantially for sexual content (-61.2%) and illegal activity (-26.7%) relative to SLD.The category-wise analysis compares CASG+SLD and CASG+SAFREE with their respective baselines on I2P.
  • Category-wise effects: CASG+SLD shows stronger reductions in harmful cosine similarities across all categories than SLD as diffusion timesteps increase.Higher cosine indicates stronger alignment between prompt guidance and a harmful category; the heatmaps show greater suppression with CASG+SLD.
  • Qualitative validation: Additional qualitative examples cover diverse challenging scenarios and further validate CASG across different contexts.These examples are presented as supplementary qualitative evidence alongside the quantitative analyses.

D.6. Efficiency Analysis

CASG adds little computational overhead to text-space safeguards and scales linearly with the number of harmful categories for latent-space guidance. Overall, it preserves the lightweight character of training-free safety guidance while improving safety performance.

  • Inference efficiency: Latent-space CASG+SLD adds approximately 1 second per additional predefined harmful category per sample.Inference time grows linearly with the number of categories k.
  • Inference efficiency: With 7 predefined harmful categories, CASG+SLD reaches 10.2 seconds per sample, or 2.58 times the SLD inference time.This configuration follows the original SLD setting.
  • Overall assessment: CASG preserves the lightweight nature of safety-guidance frameworks while significantly improving safety performance.The paper summarizes this efficiency–safety trade-off across its text-space and latent-space integrations.

E. Impact of CASG on Content Shift

CASG largely preserves generation quality for benign requests while accepting content shifts for harmful requests to enhance safety. This trade-off is especially relevant when harmful intent is entangled with the core object or expressed adversarially.

  • Benign requests: For benign requests, CASG largely preserves generation quality, with CLIP Score and FID comparable to the base safeguards.The evaluation uses benign inputs such as the COCO dataset and reports negligible content shift.
  • Harmful requests: Figure 10 compares T2I safety methods on prompts containing different harmful content, including methods requiring tuning or model modifications.The figure marks methods that require parameter tuning or model modifications with an asterisk.
  • Benign requests: Qualitative COCO examples are provided to illustrate CASG's preservation of normal usability and user experience.Figure 11 compares CASG with the base safeguard on benign prompts.
  • Harmful requests: For harmful requests, CASG may induce content shifts while ensuring enhanced safety.The shift is particularly noted when harmful intent is entangled with the core object or presented in adversarial forms.
Loading 2602.20880v4…