ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

ReMMDBench

• ReMMDBench evaluates multimodal misinformation under realistic conditions using multilingual, multi-image, varied-length samples and agent-oriented verification settings.It includes single-text, multi-image samples, three length tiers, five monolingual evaluations, and two cross-lingual transfer settings.Source: Introduction · p. 2
• ReMMDBench provides five-way veracity labels, eight distortion labels, evidence provenance, and rationales instead of reducing fact-checking to binary truth judgments.The benchmark is designed to capture graded verdicts and fine-grained attribution across textual, visual, and cross-modal distortions.Source: Introduction · pp. 1–2

ReMMD-Agent

• ReMMD-Agent decomposes posts into atomic claims and image bindings, retrieves multimodal evidence, and incrementally maintains a reusable persistent memory bank before judgment.A structured judge predicts L1 veracity, L2 distortion labels, and an L3 rationale from the accumulated evidence state.Source: Introduction · p. 2

ReMMD

• ReMMD combines ReMMDBench and ReMMD-Agent into a realistic multilingual multi-image agentic verification framework for multimodal misinformation detection.The evaluation spans closed-source and open-source agents across three backbone families and five total model sizes.Source: Introduction · p. 2

Evaluation

• 17.5% lower GPT-5.2 cost than MMD-Agent and 79.9% lower than T2-Agent accompany ReMMD-Agent’s best ReMMDBench result.The framework further analyzes performance by language, text length, distortion, transfer, and cost.Source: Introduction · p. 2

Datasets and benchmarks

• Multimodal misinformation benchmarks evaluate visual manipulations and cross-modal discrepancies, including image repurposing, unimodal bias, spatial localization, and semantic attribution.Source: Related Work · p. 2
• ReMMDBench constructs controlled multilingual multi-image samples by planning language, text length, visual provenance, and label conditions before evidence-based validation.Validation checks evidence, distortion annotations, text–image consistency, and image provenance before inclusion.Source: Figure 2 · p. 3

LVLMs and agentic verification

• Large vision-language models remain vulnerable to grounding errors, stale or adversarial evidence, and temporal contamination despite their suitability as multimodal verifiers.Source: Related Work · p. 3
• Agentic verification improves robustness by decomposing claims, asking targeted questions, and invoking retrieval or visual tools.Source: Related Work · p. 3

Data construction

• ReMMDBench transforms authentic posts from diverse trending topics and debunking sources into controlled samples with prescribed veracity, distortion, language, and visual configurations.A Planner coordinates specialized Text, Search, and Image Workers that retrieve evidence and manipulate visual assets.Source: Benchmark Design and Construction · pp. 3–4

Benchmark characteristics

• Image counts average 2.35 in short tiers and 10.05 in long tiers, while two cross-lingual settings migrate source material and visuals into different target languages.The pipeline deliberately engineers semantic misalignments and contextual displacement during multilingual migration.Source: Table 2 · p. 3, Benchmark Design and Construction · p. 4

Annotation scheme

• Each sample has hierarchical L1 veracity, L2 distortion, and L3 rationale annotations, distinguishing five ordered veracity conditions and textual, visual, and cross-modal errors.The taxonomy separates fabrication, factual-basis distortion, misleading context, synthetic or edited visuals, and semantic, contextual, or pragmatic inconsistency.Source: Annotation Schema · p. 4

Quality control

• Cohen’s κ reached 0.91 against two independent professional annotators across all 500 benchmark samples, validating the automated quality-control pipeline.A Validator agent checks whether L1 verdicts follow the central claim and whether L2 tags are physically grounded.Source: Quality Control · p. 4

Agent overview

• ReMMD-Agent outputs a five-way veracity label, an eight-dimensional distortion vector, and a concise rationale after reasoning over explicit evidence states.The system judges decomposed atomic units rather than the multimodal bundle directly.Source: ReMMD-Agent · p. 4

Parsing

• The parser converts text and images into atomic points containing minimal claims or visual observations, retrieval queries, visual cues, and point types.Predefined fields cover entities, temporal anchors, locations, events, and core claims.Source: Atomic Representation · p. 4

Evidence retrieval

• The retrieval stage maintains a sample-level memory bank asynchronously enriched by scheduled sweeps across news aggregates and fact-checking domains.During inference, targeted web and social calls retrieve evidence using parsed queries and visual cues.Source: Memory-Augmented Retrieval · pp. 4–5
• TopK is set to 20, enabling an auditable persistent evidence state that supports cross-claim evidence reuse.Evidence relevance is evaluated with cosine similarity over Qwen3-Embedding-8B representations.Source: Memory-Augmented Retrieval · p. 5

Structured judgment

• The structured judge assigns each atomic point a supported, contradicted, or unverified state before inferring the overall veracity from central claims and cross-modal bindings.Cross-lingual verification expands retrieval toward regional evidence, adds English or Chinese bridge queries, and audits translation-induced semantic discrepancies.Source: Structured Evidence Judgment · p. 5, Implementation Details · p. 5

Evaluation setup

• Evaluation covers the full 500-sample ReMMDBench split across Manus, ChatGPT, MMD-Agent, T2-Agent, and ReMMD-Agent.Model-backed agents use shared Serper retrieval and backbones including GPT-5.2, Gemma4-31B, Qwen3.6-27B, Qwen3.5-9B, and Qwen3.5-4B.Source: Experimental Setup · p. 5, Table 4 · p. 6

Main results

• ReMMDBench remains difficult for all systems, while ReMMD-Agent improves calibration of graded veracity across backbone families.General-purpose assistants are competitive on some L2 metrics but weaker on L1, and GPT-5.2 gives the best L1 performance.Source: Overall Results · p. 6
• Additional search alone is insufficient: MMD-Agent struggles with partial-truth labels, while T2-Agent’s expanded reasoning does not consistently improve veracity.MMD-Agent remains useful for distortion-oriented comparison, but evidence must be organized around the right claims.Source: Overall Results · p. 6

Ablation

• Atomic parsing and memory reuse reduce information noise, support provenance aggregation and cross-image evidence reuse, and improve both L1 and L2.Visual auxiliary analysis is especially important for L2 because visual edits and cross-modal mismatches can diagnose distortion before final veracity judgment.Source: Overall Results · p. 6, Table 5 · p. 6

Efficiency

• 17.5% lower per-sample cost than MMD-Agent and 79.9% lower than T2-Agent demonstrate architectural efficiency rather than increased computational expenditure.Evidence reuse across atomic points avoids repeated tool-augmented reasoning-path expansion, supporting dynamic benchmarks and high-concurrency deployments.Source: Table 6 · p. 6, Cost and Transfer · p. 7

Robustness

• ReMMD-Agent is more stable than baselines across text-length tiers and language slices, although low-level forensics and discourse-level support remain complementary challenges.Atomic parsing handles noisy long-context entities and references, while memory reuse reduces superficially related but temporally or geographically mismatched retrieval.Source: Fine-Grained Behavior · pp. 6–7, Figure 4 · p. 7

Evidence selection

• Realistic MMD is an evidence-selection problem requiring fine-grained labels because real evidence can support wrong conclusions.Retrieval helps only when each source is tied to the claim or image it verifies.Source: Discussion · p. 7
• Visual authenticity alone is insufficient because real images can be misused, while synthetic images do not automatically falsify accompanying text.Verification must assess the relationship among claims, evidence, and images rather than image authenticity in isolation.Source: Discussion · p. 7

Model scale and evidence

• Qwen3.5-9B outperforms Qwen3.6-27B on several metrics under the same ReMMD-Agent pipeline.The discussion attributes this pattern to evidence-provided schema following, partial-evidence calibration, and uncertainty handling rather than a general reversal of model scale.Source: Discussion · p. 7

Future directions

• Future systems should improve source-aware memory, temporal retrieval, multilingual entity linking, and metrics that separately evaluate visual edits, verdicts, and misleading mechanisms.Rationales should identify the claim, evidence, and image-text relation, while multilingual cases require local entity and source grounding beyond translation.Source: Discussion · p. 7

Conclusion

• ReMMDBench combines multilingual posts, multiple images, graded veracity, distortion labels, rationales, and evidence-centered analysis for realistic verification.ReMMD-Agent decomposes posts into checkable units, reuses retrieved evidence through memory, and judges veracity and distortion from explicit evidence.Source: Conclusion · p. 7

A Benchmark Examples

• Figure 5 presents non-sensitive benchmark samples featuring multilingual text, multiple images, hierarchical labels, rationales, and evidence-centered analysis.Source: A Benchmark Examples · p. 10

B Additional Benchmark Statistics

• The benchmark retained 500 samples after screening and controlled rewriting from an initial collection of 54,468 posts gathered across six listed sources.The collection period ran from March 1, 2026, through May 10, 2026.Source: B Additional Benchmark Statistics · p. 10
• The five L1 classes are close to balanced, while average L2-label counts rise monotonically from True to False, indicating that severe misinformation often accumulates distortions.Source: B Additional Benchmark Statistics · p. 10

C Label Boundary Notes

• L1 assigns exactly one veracity verdict, whereas L2 records every applicable distortion type.T2 captures altered claim scope or meaning, T3 captures misleading context, and V1–V2 may co-occur for edited real images containing generated content.Source: C Label Boundary Notes · p. 10

D Human–AI Annotation Agreement

• Human annotators achieved 0.96 exact agreement and 0.97 linear-weighted κ on L1 across the complete benchmark.The agreement analysis uses exact agreement for identical ordered verdicts and weighted κ for chance-corrected severity disagreements.Source: 500 ReMMDBench · p. 10
• The Annotator Agent achieved 0.90 exact agreement and 0.91 weighted κ against human consensus, while Agent–consensus L2 exact set match was 0.77.Human–human L2 exact set match was 0.83, reflecting the stricter multi-label evaluation.Source: 500 ReMMDBench · p. 10

E Agent Adaptation Details

• MMD-Agent and T2-Agent received the full post text unchanged without ReMMDBench gold labels or rationales during inference.MMD-Agent deduplicates and caps images at 10, while T2-Agent retains document-order images but uses the first image as its visual anchor for original MCTS and tool semantics.Source: E Agent Adaptation Details · p. 10

• The repository includes complete code and exact prompt templates for reproducing the experiments.Source: F Reproducibility Materials and… · p. 10
• Decoding configurations, failure-handling rules, and retrieval budgets are provided.Source: F Reproducibility Materials and… · p. 10
• Three runs are used to compute every experimental result reported in the paper.Source: F Reproducibility Materials and… · p. 10

Transfer setup

• The transfer experiment evaluates the official 10,000-instance MMFakeBench test split, distributed as 70% fake and 30% true.All compared agents use Qwen3.5-9B and Google Search by Serper API.Source: G MMFakeBench Transfer Setting · pp. 10, 13

Transfer results

• 0.824 accuracy and 0.871 Macro-F1: ReMMD-Agent outperforms MMD-Agent at 0.592 accuracy and 0.673 Macro-F1 and T2-Agent at 0.639 accuracy and 0.715 Macro-F1.The transfer task uses binary fake detection rather than five-way veracity and eight distortion labels.Source: G MMFakeBench Transfer Setting · p. 13

Analysis

• ReMMD-Agent’s advantage suggests its benefit is not limited to ReMMDBench-specific label definitions.T2-Agent performs more search, but its additional reasoning loop does not compensate for weaker evidence routing with a smaller open-source backbone.Source: G MMFakeBench Transfer Setting · p. 13

Short-text analysis

• ReMMD-Agent leads short-text L1 results, especially with GPT-5.2, while local visual cues keep assistant baselines competitive on L2.Exact match remains low across systems.Source: H Additional Fine-Grained Results · p. 13

Medium-text analysis

• ReMMD-Agent/GPT-5.2 gives the strongest medium-tier verdict performance, whereas ReMMD-Agent/Qwen3.5-9B achieves the best L2 macro-F1.Baseline L1 macro-F1 remains unstable because partial evidence must be assigned to the correct severity class.Source: H Additional Fine-Grained Results · p. 13

Long-text analysis

• ReMMD-Agent/Qwen3.5-9B achieves the highest long-tier L1 and L2 macro-F1, while ReMMD-Agent/GPT-5.2 records the highest accuracy.The long tier averages about ten images per sample and rewards binding claims, images, and retrieved sources.Source: H Additional Fine-Grained Results · p. 13

Language-slice analysis

• ReMMD-Agent improves verdict and distortion performance in every language, with the largest L1 gains in Japanese and French where MMD-Agent is weakest.T2Agent occasionally improves French L1 over MMD-Agent but drops sharply on L2.Source: H Additional Fine-Grained Results · p. 13

Distortion-label analysis

• ReMMD-Agent is strongest on evidence-alignment labels T2 Distortion, V2 Visual Editing, C1 Semantic Inconsistency, and C2 Contextual Inconsistency, but MMD-Agent leads V1 Synthetic Visual Content.The V1 exception indicates that low-level generation artifacts and forensic cues remain complementary to evidence retrieval.Source: H Additional Fine-Grained Results · p. 13, Table 15 · p. 16

Benchmark distributions

• ReMMDBench includes cross-lingual cases, frequent textual, visual, and cross-modal distortions, mixed image provenance, and multiple text-length tiers.These design choices prevent systems from optimizing for a single language, manipulation family, provenance type, or post length.Source: I Additional Benchmark Distributions… · p. 17, Figure 7 · p. 18

Image-count distribution

• The image-count distribution has an intentional long tail toward ten and eleven images, reflecting carousel-style posts with central, decorative, repeated, or weakly related images.Agents must identify which images support the claim rather than treating the entire image set as undifferentiated visual evidence.Source: I Additional Benchmark Distributions… · p. 17, Figure 8 · p. 19

Confusion matrices

• Direct prompting and T2-Agent avoid confident True predictions and concentrate probability mass around middle labels, indicating conservative bias under misinformation uncertainty.ReMMD-Agent reduces this drift by forcing judgments over supported, contradicted, and unverified atomic points.Source: I Additional Benchmark Distributions… · p. 17, Figure 10 · p. 20
• Errors concentrate among neighboring middle labels, where systems must assess the centrality of contradicted evidence rather than merely detect suspicious cues.The heatmaps show substantial diagonal mass, but partial-truth cases remain intrinsically ambiguous.Source: I Additional Benchmark Distributions… · p. 20, Figure 9 · p. 20, Figure 10 · p. 20