SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

Zongrui Wang; Xiangyang Zhu; Sicheng Wang; Han Wang; Dingyi Rong; Zeyu Zhang; Chunyi Li; Yue Shi; Kaiwei Zhang; Zicheng Zhang; Yuan Tian; Qi Jia; Yan Teng; Wei Sun; Ning Liu; Guangtao Zhai

Source-linked AI summary

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

Zongrui Wang, Xiangyang Zhu, Sicheng Wang, Han Wang, Dingyi Rong, Zeyu Zhang, Chunyi Li, Yue Shi, Kaiwei Zhang, Zicheng Zhang, Yuan Tian, Qi Jia, Yan Teng, Wei Sun, Ning Liu, Guangtao Zhai

arXiv:2608.29098v1Aug 29, 2026cs.AIcs.CV

TL;DR

Multimodal safety moderation lacks a common ordered framework for comparing image, request, and response risks, while binary judgments obscure ambiguity. The paper introduces SafeAtlas-VL and SafeAtlas Guard, and reports that the 8B model achieves the strongest multimodal, text-only, and overall averages among evaluated guards. The work positions ordered supervision and continuous risk scoring as a basis for broader multimodal safety evaluation.

Problem
Existing safeguards are usually target-specific and binary, making it difficult to compare risk across images, requests, and responses or represent ambiguous cases.
Method
The paper builds a 1.5M-instance five-level dataset with disagreement-aware annotations and trains target-conditioned guards with soft cumulative ordinal learning.
Results
The 8B model achieves the strongest multimodal, text, and overall averages among evaluated guards across 11 external benchmark–task pairs.
Takeaways & Limitations
SafeAtlas-VL provides ordered supervision for multimodal safety judgments, while SafeAtlas Guard preserves safety-level order and produces scalar risk estimates.
Takeaways & Limitations
The annotations are shaped by policy, cultural context, and normative assumptions, and should be viewed as structured supervision rather than universal or immutable definitions.

Abstract

from arXiv · show

Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.

1 Introduction

SafeAtlas-VL addresses target-specific, mostly binary multimodal safety judgments by introducing a unified five-level ordered framework for images, requests, and responses. SafeAtlas Guard uses target-conditioned tuning and ordinal modeling to produce discrete and continuous risk estimates, with strong benchmark performance.

Motivation: Existing multimodal safeguards typically judge a single target using binary or categorical decisions, limiting common risk comparison across images, requests, and responses.A unified scale is motivated by the need to represent boundary cases and judge disagreement.
Dataset: SafeAtlas-VL assigns image-, request-, and response-level judgments to five ordered safety levels across 15 harm categories and 55 fine-grained subcategories.The dataset combines real and generated safety-relevant images with grounded requests and responses.
Guard model: SafeAtlas Guard combines target-conditioned instruction tuning with a soft cumulative ordinal head to output discrete safety levels and continuous risk scores.The design preserves the semantics of whether the image, request, or response is being evaluated while modeling ordered severity.
Resources: SafeAtlas-VL contains 1.5M training instances, and SafeAtlas-Bench provides 5,000 held-out instances for evaluation.The benchmark supports evaluation of five-way predictions and continuous risk scores.
Results: The 8B guard achieves state-of-the-art multimodal performance and competitive text-only results without training on pure-text data or external benchmark datasets.These results indicate generalization beyond the dataset’s own training distribution.

2 SafeAtlas-VL Dataset

SafeAtlas-VL is built from a large, filtered pool of real and synthetic images, grounded interactions, and disagreement-aware safety annotations. Its five-level labels are calibrated and human-validated, yielding a broad training set and a held-out benchmark.

Data collection: The dataset combines real-world photographs and taxonomy-guided synthetic images to cover diverse safety-relevant visual scenarios.Real images are collected across sources, cultures, and languages, while diffusion models generate additional taxonomy-guided scenes.
Data filtering: Filtering removes low-quality, redundant, and safety-irrelevant samples using quality checks, semantic deduplication, and taxonomy-guided relevance matching.CLIP and FAISS support deduplication, while textual anchors expand fine-grained risk categories into concrete visual contexts.
Interaction generation: Each image receives four generated requests and each request four responses, mixing standard and jailbreak conditions before independent verification.This yields up to 16 interaction types per image and checks visual relevance, category consistency, and request-response alignment.
Data labeling: Three heterogeneous judges produce 12 joint request- and response-level configurations, which are mapped to five ordered levels ranging from safe core to unsafe core.The two endpoint levels require unanimous judgments, while three intermediate levels represent disagreement or boundary cases.
Mapping calibration: Calibration groups configurations with similar empirical unsafe rates while preserving judge-output direction, producing a monotonic risk ordering across levels.The procedure is designed to retain disagreement information rather than treating it solely as annotation noise.
Human validation: 89.2% overall human non-reversal validates the intended five-level ordering, rising to 94.2% for comparisons separated by at least two levels.Lower agreement on adjacent levels reflects greater ambiguity in neighboring safety levels.
Dataset composition: The release contains 1,503,284 training instances over 746,895 unique images and a 5,000-instance held-out benchmark spanning all five levels.The training data include image, image-request, and image-request-response units; the benchmark supports five-way and continuous-risk evaluation.

3 SafeAtlas Guard

SafeAtlas Guard evaluates image, request, and response safety with structured target conditioning, ordered five-level predictions, continuous risk scores, and auxiliary safety-standard outputs. Its ordinal head preserves level ordering while refining risk within each level.

Guard outputs: SafeAtlas Guard predicts five-level safety labels, continuous risk scores, harm categories, and available external-judge labels.The outputs cover image, request, and response safety judgments.
Target conditioning: Target-conditioned instruction tuning distinguishes whether the image, user request, or assistant response is being judged.The model input is selected as an image, image–request pair, or image–request–response triple according to the target.
Ordinal risk modeling: The cumulative ordinal head models ordered five-level labels and produces both discrete predictions and continuous risk scores.It estimates threshold exceedance probabilities and converts the resulting distribution into a scalar score.
Ordinal risk modeling: Continuous risk scores preserve five-level ordering while resolving relative risk differences within the same discrete level.Figure 5 illustrates this behavior for image, request, and response targets.
Multiple safety standards: Separate simulation heads fit diverse safety judgment standards from Qwen3Guard, GuardReasoner-VL, and Llama Guard 4.The Qwen3Guard head is three-class, while the other two heads are binary; their losses are averaged.
Training procedure: Training proceeds in two stages: instruction-tune the multimodal backbone, freeze it, then train the ordinal and simulation heads.At inference, the guard returns five-way labels, continuous scores, and simulation outputs.

4 Experiments

Experiments evaluate SafeAtlas Guard across external multimodal, SafeAtlas-Bench, and text-only tasks, using unsafe-class F1 as the primary metric and ordinal-risk analyses. The 8B model achieves strong multimodal and text-only results, while human and ablation studies examine ordered scores, judgment simulation, and data scaling.

Experimental Setup: Evaluation covers seven external multimodal tasks, three SafeAtlas-Bench targets, and four text-only tasks, with unsafe-class F1 as the primary metric.AvgF1 is computed over common benchmark–task pairs; SafeAtlas-Bench additionally evaluates five-level predictions.
Main Results: SafeAtlas Guard-8B reaches 79.7% AvgF1 across seven multimodal tasks, exceeding the strongest complete baseline by 4.1 percentage points.Input and response averages are 80.6% and 77.7%, respectively.
Main Results: 83.7% is the 8B model’s best four-task text average, 1.2 percentage points above the strongest dedicated text guard despite no pure-text training.The 2B and 4B variants reach 83.0% and 82.1%, respectively.
Main Results: The 8B simulation heads exceed 80% agreement with three external judges, with Cohen’s κ ranging from 0.617 to 0.773.The heads reproduce Qwen3Guard, GuardReasoner-VL, and Llama Guard 4 from the same frozen representation.
Ablation Studies: Performance improves monotonically with training fraction at all three model scales, while gains from half to the full dataset diminish at higher coverage.The 8B model remains strongest throughout the scaling comparison.

5 Conclusion

The paper concludes that SafeAtlas-VL supplies ordered, fine-grained supervision for image, request, and response safety, while SafeAtlas Guard produces both categorical and continuous risk judgments. Across 11 external benchmark–task pairs, the 8B model leads the evaluated guards in multimodal, text, and overall averages, with uncertainty concentrated between adjacent levels.

Conclusion: SafeAtlas-VL contains 1.5M instances with five-level judgments for images, requests, and responses, alongside SafeAtlas-Bench for discrete and continuous evaluation.The dataset and benchmark support ordered safety assessment across interaction stages.
Conclusion: SafeAtlas Guard combines target-conditioned instruction tuning with soft cumulative ordinal learning to produce ordered labels and scalar risk estimates.The design preserves level order while supporting continuous scoring.
Conclusion: Across 11 external benchmark–task pairs, the 8B model achieves the strongest multimodal, text, and overall averages among evaluated guards with corresponding coverage.Five-class prediction and human alignment support global ordering, with uncertainty concentrated between adjacent levels.

6 Ethics and Impact

SafeAtlas-VL is intended to support multimodal safety research, but its harmful content and large-scale annotations create dual-use, privacy, copyright, and data-governance concerns. The authors also caution that the labels reflect policy, cultural context, and normative assumptions rather than universal definitions of harm.

Intended Impact: The dataset and guard models are intended for safety moderation, risk assessment, red-teaming, and safety alignment research.The stated scope is improving safer vision-language systems.
Risks: Unsafe, offensive, sensitive, and disturbing examples create dual-use risk because the resources could facilitate harmful-content generation, selection, or optimization.The authors condemn misuse and encourage legitimate safety research and development.
Risks: Large-scale image collection raises privacy, copyright, and data-governance concerns despite filtering and exclusions during dataset construction.Automated filtering cannot guarantee removal of every problematic instance.
Scope Boundary: Five-level annotations should be treated as structured safety supervision rather than universal or immutable definitions of harm.The authors attribute this boundary to policy, cultural context, and normative assumptions.

A Dataset Construction and Annotation Details

SafeAtlas-VL constructs a multimodal safety dataset by combining diverse real and synthetic images, taxonomy-guided filtering, generated interactions, and disagreement-aware annotations across image, request, and response targets.

Data Source: The image pool combines web images and model-generated images to broaden coverage of naturally occurring and rare safety scenarios.Web sources span multiple platforms and linguistic or cultural settings, while synthetic generation supplements infrequent risks.
Taxonomy Construction: The taxonomy defines 15 annotation categories and 55 fine-grained retrieval classes, assigning retained images a parent harm category.Textual anchors expand each retrieval class into concrete subjects, actions, settings, and visual contexts for image matching.
Taxonomy-Guided Relevance Filtering: CLIP-based relevance filtering retains an image only when its best retrieval-class match exceeds 0.3, keeping one highest-scoring class–anchor assignment.This provides explicit retrieval provenance while avoiding duplicate assignments from overlapping scene descriptions.
Request and Response Generation: Each image produces generated requests and responses grounded in its matched risk category, with standard, safety-directed, default, and jailbreak conditions.Four requests and up to four responses per request introduce variation in intent, language style, and assistant behavior.
Annotation and Mapping Calibration: Three heterogeneous safety judges support disagreement-aware request and response annotation, whose five aggregated levels show monotone unsafe rates.Unsafe rates progress from 2.7% to 96.5% for requests and from 6.6% to 96.6% for responses.
Construction-Stage Human Review: Human review reports 67.6% same-risk agreement, while strict accuracy increases from 61.8% for adjacent levels to 80.2% for gaps of at least two levels.Non-reversal rates are 84.2% and 94.2%, respectively, with most ambiguity concentrated between neighboring labels.

C.1 Benchmark Configuration

The evaluation uses 11 benchmark–task pairs spanning multimodal input and response safety, text-only moderation, and the three SafeAtlas-Bench targets, with continuous scores binarized per benchmark.

Benchmark Coverage: The common evaluation suite contains 11 benchmark–task pairs covering five multimodal input-safety, two multimodal response-safety, and four text-only tasks.SafeAtlas-Bench additionally reports image-, request-, and response-level targets.
Compared Models: The comparison includes text-only and multimodal guard models at multiple parameter scales where compatible evaluations are available.Multimodal baselines include GuardReasoner-VL, ProGuard, Llama Guard, LLaVAGuard, LLaVAShield, Nemotron, and SafeGuard-VL.
Score Binarization: Continuous risk scores are converted to binary predictions using operating thresholds selected on official validation splits when available.The selected threshold is fixed for the corresponding test split, while native binary models are evaluated directly.
Score Binarization: Qwen3Guard is evaluated in strict and loose modes, while SafeAtlas-Bench’s three higher-risk levels are mapped to unsafe for binary evaluation.Strict mode maps controversial and unsafe to unsafe; loose mode maps only unsafe to unsafe.

C.4 Training Hyperparameters

SafeAtlas Guard uses Qwen3-VL backbones with a two-stage training procedure: multimodal safety instruction tuning followed by frozen-backbone prediction-head training.

Safety Instruction Tuning: SafeAtlas Guard is trained with Qwen3-VL Instruct backbones at 2B, 4B, and 8B scales.Safety instruction tuning uses full-parameter supervised fine-tuning for one epoch with AdamW, cosine decay, and BF16 precision.
Prediction-Head Training: The second stage freezes the multimodal backbone and trains the cumulative ordinal, category, and three simulation heads for one epoch.The continuous risk score is normalized to [0, 100].

C.6 Ablation and Scaling Results

Ablations show that graded ordinal supervision drives most gains over binary supervision, while performance improves with more data but begins to flatten at high coverage.

Formulation Ablation: 77.9–78.7 average F1 results from five-way SFT, compared with 67.8–68.3 for binary SFT across model scales.Both variants use the same 11 benchmark–task pairs and equal-weighted average F1.
Formulation Ablation: 80.5, 80.4, and 81.2 are the soft ordinal results for the 2B, 4B, and 8B models, respectively.Soft ordinal training is strongest at every scale.
Data Scaling: Average F1 increases monotonically with training fraction for all backbone scales, with the 8B model strongest throughout.Curves flatten between one half and the full dataset, indicating diminishing marginal gains at higher data coverage.

C.7 Post-Training Human Evaluation Protocols

The post-training evaluation compares model predictions with human judgments for continuous risk ordering and five-way safety levels across image, request, and response targets.

Continuous-Score Ordering: 600 continuous-score pairs are constructed from 200 pairs per target and stratified by absolute score difference Δ.Five annotators choose the riskier instance or equal risk, with the unique modal response as reference.
Five-Way Class Alignment: 600 five-way instances are sampled from 200 instances per target for direct human assignment of the five ordered levels.Exact accuracy, Within 1, MAE, and QWK measure agreement with the unique modal human label.
Five-Way Class Alignment: 59.8% exact agreement, 91.7% Within 1, and 0.806 QWK are achieved by ordinal predictions, versus 52.5%, 83.8%, and 0.734 for original labels.MAE also decreases from 0.655 to 0.510.

C.8 Auxiliary Head Analyses

Auxiliary analyses examine simulated-judge fidelity, harm-category prediction, and taxonomy-guided retrieval examples used to construct the multimodal safety data.

Simulation Head Fidelity: 90.24%/.753 agreement/κ is obtained for Llama Guard 4, while GuardReasoner-VL is the most difficult standard to simulate.The reported 8B values are 86.84%/.773 for Qwen3Guard and 80.79%/.617 for GuardReasoner-VL.
Harm Category Prediction: 76.7% category accuracy is achieved on 3,900 non-none examples, with 2,990 predictions on the diagonal.Performance is strongest for risky financial practices, defamation, toxic content, unfair content, and security threats, and weakest for false beliefs, trade and compliance, and influence operations.
Taxonomy-Guided Retrieval: The 55 fine-grained retrieval classes are represented through keyword–scene anchors grouped under parent harm categories across six tables.The associated galleries show retained images, parent categories, retrieval classes, and matched textual anchors.
Taxonomy-Guided Retrieval: Figures A3 and A4 illustrate taxonomy-guided CLIP relevance filtering with one retained image for each of the 55 retrieval classes.Each example is paired with its parent harm category, retrieval class, and matched textual anchor.

F Dataset Label Distributions

The dataset distinguishes 15 predicted harm categories from 55 retrieval classes and reports separate label distributions for request-, response-, and image-level judgments.

Label Granularity: 15 annotation categories are predicted harm labels, while 55 child classes support image collection and provenance rather than an additional prediction task.Request and response tables use retrieval-class granularity, whereas image-level annotation and Table A14 use the coarser 15-category granularity.
Request and Response Labels: 528,916 request-level instances and 745,641 response-level instances are distributed by fine-grained category, including none.Tables A12 and A13 compute shares over all instances.
Image Labels: 228,727 image-level instances are distributed by harm category, including none.Table A14 reports shares over all image-level instances at the coarser annotation granularity.
Judgment Targets: Image-level judgment targets only the image, while request-level judgment evaluates the multimodal user request and response-level judgment evaluates only the assistant response in context.The corresponding formats output primary safety/category fields, with auxiliary judge labels for request- and response-level judgments.