PaperlayerView exact revision on arXiv

Source-linked AI summary

SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation

Junxiao Yang, Minghao Zhang, Xiaoce Wang, Haoran Liu, Shiyao Cui, Hongning Wang, Minlie Huang

arXiv:2606.03348v1cs.CVcs.AI

TL;DR

AI-generated artifacts can combine embedded claims with credibility-bearing formats, creating a visual misinformation threat. SYNCRED-Bench and FP450 assess this challenge, and evaluations find existing systems unreliable: at a 5% false-positive-rate constraint, 15 MLLMs average 10.5% TPR.

  • Problem

    Recent models can generate self-contained misleading artifacts with realistic embedded text and credible layouts, a visual-misinformation threat distinct from natural-image forgeries and paired image-text misinformation.

    Source: §1 · p. 1

  • Method

    The paper constructs SYNCRED-BENCH, a balanced benchmark of 600 AI-generated misinformation images across six credible-form categories and seven circulation styles, plus FP450, a real-image negative set.

    Source: §1 · p. 2

  • Results

    At a 5% false-positive-rate constraint, 15 MLLMs averaged 10.5% TPR, open-source AIGC detectors averaged less than 5% TPR, and commercial APIs reached 57.6% average accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Synthetic credibility is an exceptionally difficult detection challenge for both models and human annotators.

    Source: §1 · p. 2

    The benchmark measures detection of AI-generated images, not whether embedded claims are true, false, misleading, unsupported, or contextually misrepresented.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Recent generative models can now produce visual artifacts with realistic embedded text and layouts, creating a new misinformation threat: synthetic credibility. We introduce SYNCRED-Bench, a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles, together with FP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%, and commercial APIs reach 57.6%. Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR. These findings establish synthetic credibility as a severe and underexplored visual misinformation challenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.

1 Introduction

The paper defines synthetic credibility as misinformation embedded in realistic, credibility-rich visual artifacts and introduces SYNCRED-BENCH and FP450 to evaluate it. Results show that models and humans struggle to detect these artifacts.

1 Introduction

  • Synthetic credibility describes self-contained artifacts whose embedded misinformation derives persuasiveness from credibility cues and authoritative-looking formats, rather than claims superimposed on authentic photographs.Detection therefore focuses on identifying the artifact’s synthetic origin rather than fact-checking claims overlaid on otherwise authentic images.Source: §1 · pp. 1–2
  • At benchmark construction, GPT-image-2 was the only model the authors found capable of reliably generating the required layout-consistent, credible artifacts.This explains why the benchmark uses a single generator.Source: §1 · p. 1
  • Credible form imitates authoritative communication genres, while credible circulation imitates traces of genuine distribution such as scanning, screen photography, and online compression.The former makes an image look like an official or professional source; the latter makes it appear to have traveled through authentic channels.Source: §1 · p. 2
  • SYNCRED-BENCH balances 600 Chinese- and English-language AI-generated misinformation samples across six credible-form categories and seven fine-grained circulation styles, with FP450 providing 450 real-image negatives.The benchmark also covers four coarse circulation categories: native rendering, document reproduction, screen capture, and online circulation.Source: §1 · p. 2
  • Under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% average TPR on SYNCRED-BENCH.Without the FPR constraint, their average positive-class accuracy is 31.2%; open-source AIGC detectors achieve less than 5% average TPR under the same constraint, commercial APIs reach 57.6% average accuracy, and human majority voting reaches 63.0% TPR with 27.0% FPR.Source: §1 · p. 2

2 Related Work

Prior work addresses multimodal claim-image consistency, synthetic-image detection, and localized document tampering, whereas SYNCRED-BENCH centers false claims embedded in credible forms and circulation styles.

Multimodal misinformation detection and MLLM safety

  • MMD spans multimodal cue fusion, image-caption mismatch, retrieval, and safety vulnerabilities; SYNCRED-BENCH instead studies false claims natively embedded inside credibility-bearing images.Prior work verifies whether text and image jointly support a claim, while the benchmark focuses on credibility-bearing images themselves.Source: §2 · p. 3

Synthetic-image detection

  • Synthetic-image detectors evolved from CNN/GAN fingerprints to diffusion-era reconstruction, frequency, and foundation-model features, but remain scene- and object-centered rather than modeling rendered text, templates, institutional form, or circulation style.Prior benchmarks also test cross-generator and degradation robustness, and unified toolkits connect AIGC detection with broader forensics.Source: §2 · p. 3

Visual-text and document forensics

  • Scene-text and document-forensics work targets edited words or tampering in genuine materials, whereas SYNCRED-BENCH organizes categories around credible form and circulation rather than tamper type, generator family, or pixel-level masks.Related benchmarks cover English and Chinese documents, open-set and diffusion-based tampering, receipts, identity documents, and zero-shot document forensics.Source: §2 · p. 3

3 SYNCRED-BENCH Construction

SYNCRED-BENCH pairs AI-generated claim-bearing images with a real-image negative set to evaluate synthetic credibility detection. Its construction defines what counts as synthetic credibility and organizes samples by credible form and circulation style.

3 SYNCRED-BENCH Construction

  • SYNCRED-BENCH contains 600 AI-generated images, balanced across six credible-form categories at 100 each, alongside FP450, a 450-image real-image negative set.Source: §3 · p. 3

3.1 Task Scope and Label Semantics

  • An artifact qualifies as synthetic credibility when it is AI-generated, contains a plausible claim or evidence-like statement, and imitates a credibility-bearing visual convention.The benchmark treats synthetic images as untrustworthy visual evidence, without assuming that real images are truthful or synthetic images are false.Source: §3.1 · p. 3

3.2 Credible-Form Taxonomy

  • The six credible-form categories are Media Layout, Institutional Notice, Platform Interface, Credential Record, Analytical Display, and Assessment Material.Each category has multiple subtypes; images fitting multiple categories are assigned by their dominant social function.Source: §3.2 · p. 3

3.3 Credible-Circulation Taxonomy

  • Seven circulation styles span four coarse categories: Native Rendering, Scanned Copy, Camera Copy, Fax Copy, Screen Photograph, Cropped View, and Online Compression.These styles model plausible acquisition traces that can make artifacts seem realistic; the dataset covers all styles while preserving realistic form–style pairings rather than enforcing equal counts in every pairing.Source: §3.3 · p. 4

3.4 Image Generation

  • Authors manually create self-contained claims with assigned form, subtype, language, and circulation style, excluding visible synthetic labels and disclaimers.GPT Image 2 generates images with claim, style, and rendering instructions; candidates are retried or discarded if claims are missing or unreadable, form or circulation is inconsistent, or the API fails.Source: §3.4 · p. 5

3.5 Human verification and filtering.

  • Five authors verify candidate images for content fidelity, form consistency, circulation consistency, and benchmark validity.Validity checks exclude generation residues, severe layout failures, irrelevant content, generator marks, synthetic disclaimers, private personal information, and duplicates.Source: §3.5 · p. 5

3.6 FP450 Real-Negative Set

  • FP450 uses matched real images collected from web searches across the benchmark’s visual space, with manual edits to balance underrepresented styles and checks of source context when available.The set excludes duplicates and aims to include diverse real-world image origins.Source: §3.6 · p. 5

4 Experiments

Experiments show that synthetic credibility remains difficult for automated detectors and people, while credible layouts and circulation styles systematically shape detection judgments.

4.1 Experimental Setup

  • The evaluation covers 9 closed-source and 6 open-source MLLM judges, plus 3 commercial APIs and 3 lightweight open-source detectors, using TPR and FPR metrics.Confidence-aware TPR(λ) measures usable detection at an operating point whose FPR does not exceed λ%.Source: §4.1 · p. 5

4.2 Main Results

  • 31.2% average positive-class accuracy and 10.5% average TPR(5) across 15 MLLMs show weak detection; Claude Opus 4.6 is the strongest usable MLLM at 69.5% accuracy with 5.0% FPR.MLLMs perform relatively better on credential records, but average accuracy falls to 18.1% for screen-capture styles.Source: Table 3 · p. 6, §4.2 · p. 6
  • 48.3% average accuracy and 30.3% average TPR(5) show dedicated AIGC detectors also fall short; Hive AI reaches 75.2% accuracy with 0.9% FPR.AI-vs-Real has the highest raw accuracy, 78.7%, but 69.1% FPR; AI-vs-Human and Deepfake-vs-Real reach only 8.3% and 19.3% accuracy, respectively.Source: §4.2 · p. 6

4.4 Credible layout is frequently converted into authenticity evidence.

  • Structured layouts/templates and UI/screenshot conventions appear in 66.1% and 61.1% of false-negative explanations, respectively, ahead of typography and camera perspective/lighting at 46.9% and 46.0%.Model rationales vary, but judges often reinterpret credible layouts, coherent content, and plausible acquisition cues as authenticity evidence.Source: Figure 3 · p. 7, §4.4 · p. 7

4.3 Threshold sweeping reveals weak low-FPR operating points.

  • At 5% FPR, only Claude Opus 4.6 reaches about 69.5% TPR and Sonnet 4.6 reaches 55.3%, while most other MLLMs remain below 30%.Gemini 3.1 Flash, GLM-5V Turbo, and Qwen3.6 Plus improve mainly at looser FPR budgets, indicating poorly separated detection scores.Source: §4.3 · pp. 6–7, Figure 4 · p. 7

4.5 Circulation styles can systematically distort authenticity judgments.

  • Camera Copy, Screen Capture, and Cropped View shift MLLM detection by -6.5 pp, -11.1 pp, and -10.3 pp versus Native Rendering, while Scanned Copy and Fax-like Copy raise it by +4.1 pp and +9.4 pp.Generic Degradation has almost no average effect; semantically meaningful circulation cues can make synthetic defects seem like plausible acquisition evidence.Source: §4.5 · p. 7, Figure 5 · p. 8

4.6 Humans Also Struggle to Distinguish Synthetic Credibility

  • 63.0% TPR and 27.0% FPR under majority voting show that human annotators also achieved only moderate performance.Individual decisions reached 60.2% TPR with 35.8% FPR; the test used 100 balanced synthetic samples and 100 FP450 images, each judged independently by five annotators.Source: Table 4 · p. 8, §4.6 · p. 8

4.7 How to Handle Synthetic Credibility

5 Conclusion

The conclusion frames synthetic credibility as a visual misinformation threat that requires detection beyond conventional natural-image forgery methods, and introduces SYNCRED-BENCH and FP450 to study it.

5 Conclusion

  • Human annotators achieved only 63% success, while current MLLMs and AIGC detectors remained unreliable, especially under low false-positive-rate constraints.The conclusion also notes that models may mistake layouts, official-looking templates, typography, screenshots, lighting, and degradation artifacts for authenticity cues.Source: §5 · p. 8
  • SYNCRED-BENCH contains 600 AI-generated claim-bearing images across six credible-form categories and seven circulation styles, with FP450 for evaluating false positives on real images.Source: §5 · p. 8
  • Models may mistake structured layouts, official-looking templates, typography, screenshots, lighting, and degradation artifacts for evidence of authenticity.Source: §5 · p. 8
  • Synthetic credibility cannot be addressed by conventional natural-image forgery detection alone.The conclusion recommends combining provenance verification, watermarking, broader detector training data, and MLLM alignment that separates visual plausibility from image provenance.Source: §5 · p. 9

Limitations

The authors characterize SYNCRED-BENCH and its evaluation as preliminary, with limitations in dataset coverage, generator diversity, perceptual validation, scale, task scope, and interpretation of model results.

Limitations

  • The benchmark covers six credible-form categories and seven circulation styles but omits other formats, languages, cultural conventions, platforms, and domain cues, limiting transfer beyond English- and Chinese-language claim-bearing images.The authors describe it as an initial rather than exhaustive representation of credible visual misinformation.Source: Limitations · p. 9
  • The synthetic set uses only the gpt-image-2 pipeline, potentially embedding generator-specific artifacts, prompt regularities, or rendering biases.Future extensions could include other generation systems, editing pipelines, open-source generators, and adversarial post-processing.Source: Limitations · p. 9
  • Human verification checks claim presence, category, and circulation-style matches, but construction-assigned labels do not establish whether ordinary users find artifacts credible or persuasive.The authors call for independent-annotator studies of perceived credibility, persuasiveness, and potential harm.Source: Limitations · p. 9
  • The evaluation uses 600 generated images and 450 real-image false-positive samples, so limited scale may increase result randomness and leave evaluation-specific overfitting or benchmarking artifacts insufficiently ruled out.The authors recommend a larger, more diverse dataset for greater statistical stability and reduced sampling variation and evaluation-hacking risk.Source: Limitations · p. 9
  • The benchmark measures AI-generated-image detection rather than claim veracity: real images may contain misinformation, while generated images may contain true claims.It is intended for detecting credibility-bearing synthetic visual artifacts, not as a complete fact-checking or provenance-verification system.Source: Limitations · p. 9
  • Results from closed-source or API-based systems may shift with model updates, safety policies, prompts, preprocessing, or threshold calibration, and generated rationales are not direct causal evidence of internal decisions.The authors suggest controlled interventions, counterfactual edits, and systematic attribution methods to complement rationale analysis.Source: Limitations · p. 9

Ethical Considerations · A Dataset Details

SYNCRED-BENCH is a dual-use research resource that should be released with safeguards against deceptive use. Its deployment and real-image data also require protections against false-positive harms, privacy and governance risks, and limited cultural, linguistic, and geographic coverage.

Ethical Considerations

  • The benchmark is intended for measurement and safety research, not for producing misleading content, because its realistic artifacts could enable misinformation, impersonation, fraud, or deception.The paper recommends clearly labeling generated images as synthetic in documentation, metadata, and release materials.Source: Ethical Considerations · p. 9
  • Access should be research-only, with identity and intent verification, and released prompts or metadata should carry explicit restrictions and safety warnings.Recommended prohibited uses include deceptive use, political manipulation, harassment, impersonation, and public-facing misuse.Source: Ethical Considerations · p. 10
  • False-positive labels can harm journalists, institutions, document holders, platform users, and ordinary individuals, so detector outputs should not alone support accusations of fabrication.The paper cautions that authentic images may be incorrectly labeled as AI-generated.Source: Ethical Considerations · p. 10
  • Real-world assessment should combine predictions with provenance, watermark or content-credential checks, source verification, human review, and claim-level fact-checking.Low-confidence results should be treated only as signals for further investigation.Source: Ethical Considerations · p. 10
  • Releasing the real-image negative set requires respecting source licenses, avoiding redistribution of sensitive personal information, documenting sources where possible, and providing removal mechanisms.These measures address copyright, attribution, privacy, and data-governance concerns.Source: Ethical Considerations · p. 10
  • The benchmark may reflect construction-related cultural, linguistic, and institutional assumptions, so future expansions should broaden geographic, linguistic, and cultural coverage.The paper especially highlights high-stakes domains including public administration, education, healthcare, finance, and emergency communication.Source: Ethical Considerations · p. 10

A.1 Detailed Dataset composition. … B MLLM Judge Details

The supplementary sections detail SYNCRED-Bench’s composition, real-image negative set, metadata and release controls, and image-only MLLM evaluation protocol.

A.1 Detailed Dataset composition.

  • The positive split contains 600 AI-generated images, balanced at 100 each across six credible-form categories, with multiple subtypes and dominant-function labeling for ambiguous artifacts.Categories are Media Layout, Institutional Notice, Platform Interface, Credential Record, Analytical Display, and Assessment Material.Source: §A.1 · p. 12
  • Circulation styles are stratified for coverage rather than exact uniformity, with pairings chosen for visual plausibility across artifact forms.The styles include Native Rendering, Reproduced Style (Scanned Copy, Camera Copy, Fax-like Copy), Screen Photograph, and Circulated Content (Cropped View, Online Compression); Table 5 reports the final fine-grained distribution after duplicate-ID resolution.Source: §A.1 · p. 12, Table 5 · p. 12

A.2 FP450 Negative Set Details

  • FP450 is a matched real-image negative set collected through web searches spanning the same broad visual space as SYNCRED-BENCH.Its search coverage includes notices, screenshots, certificates, receipts, invoices, dashboards, scanned or photographed documents, cropped views, and compressed online images.Source: §A.2 · p. 12
  • Because relevant real-image corpora underrepresent several circulation styles and large-scale artifact collection raises practical concerns, the authors manually edited diverse, non-duplicate web images to emulate styles.Edits include reducing sharpness and saturation for Fax-like Copies, truncating Cropped Views, photographing screens to retain reflections and Moiré patterns, and downscaling then screenshotting Degradations; source-page context is inspected when available.Source: §A.2 · p. 13

A.3 Metadata Format

  • Each generated image has a metadata record containing its filename, semantic category labels, exact generation prompt, and compact grouping and audit fields.Metadata supports construction, stratified reporting, and error analysis; detectors are not shown those fields.Source: §A.3 · p. 13

A.4 Data Release and Access Protocol.

  • The benchmark follows a controlled release protocol: documentation, taxonomies, schema, fixed splits, evaluation code, aggregate statistics, and predictions will be public, while the full image set and prompts are research-use restricted.Research use is governed by a data-use agreement prohibiting deceptive use and other specified harms; samples carry synthetic-origin labels, version identifiers, and hashes, while FP450 redistribution depends on licensing.Source: §A.4 · p. 13

B MLLM Judge Details

  • MLLMs are evaluated as image-level detectors using only the image and a fixed instruction, with a unified prompt requesting a judgment, confidence, and reasons.The response format specifies an AI-generated true/false judgment, confidence from 0.0 to 1.0, and 2–4 concise sentences explaining visual evidence.Source: B MLLM Judge Details · pp. 13–14

C Details of Human Detection Experiment · D Examples of MLLM Failure

Human annotators struggled to identify synthetic credibility, while MLLM judges confidently mistook AI-generated images for real when they displayed familiar, plausible visual cues.

C Details of Human Detection Experiment

D Examples of MLLM Failure

  • MLLM judges incorrectly classified AI-generated images as real with high confidence in the representative false-negative cases.The examples included judge identity, confidence score, and the model’s original free-form rationale.Source: D Examples of MLLM Failure · pp. 14–15
  • MLLMs over-relied on plausible layouts, coherent typography, realistic camera artifacts, and familiar interface conventions when judging authenticity.Consistent text, recognizable UI elements, and naturalistic lighting or perspective could make synthetic images appear genuine.Source: D Examples of MLLM Failure · p. 15
  • The failures included synthetic screenshots, document-like images, presentation photos, and broadcast-style scenes mistaken for real-world captures.Source: D Examples of MLLM Failure · p. 15

E Models Used in Our Experiments · E.1 MLLMs · E.2 AIGC Detectors

The experiments used closed-source and open-source MLLMs and AIGC detectors, with the appendix listing the models and services included in each group.

E Models Used in Our Experiments

  • The experiments covered both MLLMs and AIGC detectors, with separate closed-source and open-source groups.Source: §E.1 · p. 15, §E.2 · p. 15

E.1 MLLMs

  • Closed-source MLLMs included GPT-5.4, GPT-4o, Grok-4.3, and Claude Opus 4.6.Source: §E.1 · p. 15
  • The closed-source MLLM roster also included Claude Sonnet 4.6, Gemini 3.1 Pro, Gemini 3.1 Flash-Lite, GLM-5V Turbo, and Qwen3.6 Plus.Source: §E.1 · p. 15
  • Other listed MLLMs were Qwen3.5-9B, Qwen3.5-35B-A3B, Qwen3-VL-8B, and Qwen3-VL-32B.Source: §E.1 · p. 15
  • The MLLM list also included Pixtral Large and Llama-3.2-11B-Vision.Source: §E.1 · p. 15

E.2 AIGC Detectors

  • Closed-source AIGC detectors were Sightengine, AI or Not, and Hive AI.Source: §E.2 · p. 15
  • Open-source detectors were AI-vs-Real, AI-vs-Human, and Deepfake-vs-Real.Source: §E.2 · p. 15

F Data Examples

The examples show AI-generated images that MLLM judges confidently misclassify as real, often relying on plausible layouts, text, and environmental cues. Additional examples span circulation styles, artifact types, and real versus AI-generated document images.

F Data Examples

  • MLM judges classify AI-generated Douban, news-site, and HEYTEA-interface screenshots as real with high confidence, citing consistent layouts, crisp text, coherent metadata, and few obvious artifacts.These are representative false negatives.Source: Figure 7 · p. 16
  • AI-generated conference-screen and GitHub-interface images are misclassified as real when judges treat projector glare, audience silhouettes, dim lighting, legible text, and standard web layouts as provenance evidence.The figure presents representative false-negative cases.Source: Figure 8 · p. 17
  • AI-generated document, notice, and broadcast scenes are misclassified as real; judges cite sharp text, familiar formats, paper texture, screen effects, and coherent graphics as signs of authenticity.The cases suggest judges conflate surface realism and recognizable visual conventions with photographic authenticity.Source: Figure 9 · p. 17
  • Examples span circulation-style variants and artifact types, while real and AI-generated documents vary across provenance, rendering, circulation, and capture styles.Shown variations include scanned, native digital, and compressed layouts, plus glare, cropping, perspective distortion, blur, and document degradation.Source: Figure 10 · p. 18, Figure 11 · p. 19, Figure 12 · p. 20

Related papers

Loading 2606.03348v1…