PaperlayerView exact revision on arXiv

Source-linked AI summary

Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources

Sahar Abdelnabi, Rakibul Hasan, Mario Fritz

arXiv:2112.00061v3cs.CVcs.CLcs.CYcs.LG

TL;DR

Out-of-context reuse of real images creates misleading image-caption claims, while multimodal fact-checking remains underexplored. The paper retrieves open-domain Web evidence from both modalities and uses CCN to assess consistency, reporting better detection than previous baselines and performance comparable to humans.

  • Problem

    Most automated fact-checking focuses on textual claims, leaving multimodal claim verification underexplored despite the effort required for manual fact-checking.

    Source: §1 · p. 2

  • Method

    The multi-modal cycle-consistency check retrieves text using the image and images using the caption, and CCN combines evidence-consistency memory networks with a CLIP image-caption consistency component.

    Source: §1 · p. 2

  • Results

    The evidence-augmented method significantly outperforms previous baselines and is reported as comparable to human performance.

    Source: §5.4 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    The paper offers an inspectable, automated tool intended to assist manual fact-checking and reduce its workload.

    Source: §5.4 · p. 8, §6 · p. 9

    The task remains challenging, and the authors caution that fully relying on automated tools might have dangerous consequences; the open-domain setting also has noisy evidence without relevance supervision.

    Source: §5.4 · p. 8

Abstract

from arXiv · show

Misinformation is now a major problem due to its potential high risks to our core democratic and societal values and orders. Out-of-context misinformation is one of the easiest and effective ways used by adversaries to spread viral false stories. In this threat, a real image is re-purposed to support other narratives by misrepresenting its context and/or elements. The internet is being used as the go-to way to verify information using different sources and modalities. Our goal is an inspectable method that automates this time-consuming and reasoning-intensive process by fact-checking the image-caption pairing using Web evidence. To integrate evidence and cues from both modalities, we introduce the concept of 'multi-modal cycle-consistency check'; starting from the image/caption, we gather textual/visual evidence, which will be compared against the other paired caption/image, respectively. Moreover, we propose a novel architecture, Consistency-Checking Network (CCN), that mimics the layered human reasoning across the same and different modalities: the caption vs. textual evidence, the image vs. visual evidence, and the image vs. caption. Our work offers the first step and benchmark for open-domain, content-based, multi-modal fact-checking, and significantly outperforms previous baselines that did not leverage external evidence.

1. Introduction

The introduction frames out-of-context image reuse as a consequential misinformation threat and multimodal fact-checking as an underexplored, labor-intensive task. It proposes open-domain evidence retrieval and CCN to automate inspectable checks of image-caption pairs.

1. Introduction

  • Out-of-context reuse of real images can make false narratives credible, while multimodal fact-checking remains underexplored and manual verification requires substantial effort.The paper identifies image-repurposing as an easy and effective misinformation tactic.Source: §1 · pp. 1–2
  • The multi-modal cycle-consistency check retrieves textual evidence using the image and visual evidence using the caption, then compares each with its paired modality.Evidence is retrieved automatically in an open-domain setting, without pre-identified or curated “golden evidence.”Source: §1 · p. 2
  • CCN uses memory-network components to assess claim-evidence consistency and a CLIP component to assess the image-caption pair itself.The architecture is intended to mimic layered human reasoning across same and different modalities.Source: §1 · p. 2
  • The paper evaluates evidence representations and memory components and conducts two user studies on detection performance and the usefulness of evidence and model attention.The studies examine whether collected evidence and model attention help people distinguish true from falsified pairs.Source: §1 · p. 2
  • The authors formalize multimodal fact-checking and report that their evidence-augmented method significantly improves detection over baselines.They present the work as a new task, an inspectable framework, and an evaluation including ablations and user studies.Source: §1 · p. 2

2. Related Work

Prior work addresses multimodal misinformation, out-of-context images, and textual fact verification through distinct tasks and evidence strategies. This paper focuses on verifying real-image, image-caption claims with both visual and textual Web evidence.

Multi-modal Misinformation

  • Khattar et al. classify multimodal fake news using image-caption representations, whereas this work verifies real news images that may be out of context.The images in Khattar et al.’s dataset could be edited.Source: §2 · p. 2
  • Zlatkova et al. [60] use Web-derived image features and claim-text features, while this work compares both visual and textual evidence with the claim.The cited approach does not compare the claim image’s actual content against evidence.Source: §2 · p. 2

Out-of-context image datasets

  • Aneja et al. detect whether two captions share an image’s context, but their unlabelled dataset does not support veracity-detection training or evaluation.Their task does not judge the veracity of a single image-caption claim.Source: §2 · p. 2
  • Earlier synthetic datasets used caption swaps or named-entity changes [23] [43], while NewsCLIPpings pairs real images with real captions to create convincing falsified examples [31].NewsCLIPpings reports that machine and human detection are limited, motivating the use of external Web evidence.Source: §2 · pp. 2–3

Open-domain QA and Fact-verification

  • This work tackles a large-scale, open-domain task involving automatic retrieval and comprehension, without assuming input is already labelled and identified as relevant or restricting evidence to a specific curated source such as fact-checking websites [54].Source: §2 · p. 3
  • Popat et al. [36] assess per-source credibility for textual claims, whereas this work learns aggregated consistency across sources for multimodal claims.The paper notes that, to its knowledge, prior work had not verified multimodal claims using both modalities.Source: §2 · p. 3

3. Dataset and Evidence Collection

The paper uses NewsCLIPpings as its out-of-context benchmark and gathers textual and visual Web evidence from image-caption queries. The resulting examples pair the query with retrieved evidence for pristine-or-falsified classification.

Dataset

  • The balanced NewsCLIPpings subset contains 71,072 training, 7,024 validation, and 7,264 test examples drawn from four news outlets.The dataset contains pristine and falsified images and is built on VisualNews.Source: §3 · p. 3

Textual evidence

  • For textual evidence, inverse image search returns associated entities and URLs, and a crawler collects image captions, page titles, text snippets, and domains.The crawler finds image tags by URL or perceptual-hash content matching and discards pages whose detected title language is not English.Source: §3 · p. 3

Visual evidence

  • For visual evidence, the caption is used to search for up to 10 images, with each result’s domain recorded.These results may be loosely related to the query image rather than exact matches.Source: §3 · p. 3

Dataset decomposition

  • An example includes the query image and caption alongside retrieved visual images, textual entities, and captions or sentences.Source: §3 · p. 3

Dataset

  • The task is to classify each query image-caption pair as pristine or falsified.Source: §3 · p. 3

4. The Consistency-Checking Network

CCN checks image-caption pairs by comparing each modality with evidence retrieved through the other, using memory networks to focus on relevant evidence. Its components combine visual and textual reasoning with image-caption consistency before producing a unified decision.

4. The Consistency-Checking Network

  • CCN uses memory networks to selectively compare claims with relevant evidence, while attention makes the evidence items most relevant to a decision inspectable.The architecture combines visual reasoning, textual reasoning, and a CLIP component; unrelated search results and the need for comprehension make evidence comparison challenging.Source: §4 · pp. 3–4

4.1. Visual Reasoning

  • The visual reasoning component compares query and evidence images using ResNet152 representations and overlapping labels, with a separate memory for ResNet50 scene representations.The overlap count is added as a feature, while the separate scene memory supports reasoning about place-based context discrepancies.Source: §4.1 · p. 4

4.2. Textual Reasoning

  • Textual reasoning represents the query caption and evidence with sentence embeddings, separate entity and sentence memories, and a named-entity-overlap feature.The paper tests a sentence transformer and BERT with an LSTM; evidence-domain embeddings are also concatenated with textual evidence representations, excluding entities.Source: §4.2 · pp. 4–5

4.3. CLIP

  • A fine-tuned CLIP model evaluates image-caption pairs without evidence, then contributes its joint embeddings to CCN’s final classifier.The fine-tuned CLIP is frozen when its joint embeddings are integrated into the classifier.Source: §4.3 · p. 5

4.4. Classifier

  • The classifier concatenates batch-normalized component outputs and predicts a falsified probability from the combined feature vector.The model is trained as a binary classifier with the backbone embedding networks frozen.Source: §4.4 · p. 5

5. Experimental Results

The experiments show that evidence-assisted detection outperforms the tested baselines, while user studies indicate that evidence and model-selected attention can help people assess image-caption pairs. The analyses also expose limitations tied to retrieval quality, noisy open-domain evidence, and unresolved risks from contradictory sources.

5.1. Quantitive Analysis

  • Removing evidence images or captions significantly reduced performance, while removing entities had less effect, possibly because captions already supplied redundant or generic entity information.The authors also report that batch normalization improved accuracy by nearly 11 percentage points and that a unified memory was less successful than separate memories.Source: §5.1 · pp. 5–6
  • Filtering exact same-site matches did not significantly reduce accuracy; CLIP recovered some filtering-related performance loss, and BERT embeddings and evidence domains also helped to some extent.CLIP had the largest reported effect among the other enhancements, increasing accuracy by around 1.5 percentage points.Source: §5.1 · p. 6
  • Nearly 19 percentage points over CLIP-only and ∼14 percentage points over the evidence baseline: the proposed model improves on both tested baselines.Table 2 compares classification performance on the test set.Source: Table 2 · p. 6, §5.1 · p. 6

5.2.1 Study 1: Human Performance Baseline

  • On 100 examples, workers averaged 65.6% detection with a 35% falsified detection rate; CCN reached 80% accuracy, on a par with average workers but below the best worker.On average, evidence contributed to decisions in 77.2% of examples versus 59.3% for the image-caption pair alone, and evidence was the only helpful cue in 28.3%. Evidence images were most helpful at 64%.Source: §5.2.1 · p. 7, Figure 5 · p. 7
  • Workers and CCN struggled with examples lacking obvious cues or with poor retrieval, including missing image matches and unrelated or inconclusive evidence.The authors suggest looser retrieval, such as caption-keyword searches or captions of similar images, might help in these cases.Source: §5.2.1 · p. 7

5.2.2 Study 2: Evaluating the Attention

  • Highest-attention evidence yielded higher worker performance and generally better helpfulness ratings than lowest-attention evidence, suggesting the model prioritized relevant items.The study compared the highest- and lowest-attention subsets of evidence.Source: §5.2.2 · p. 7

5.3. Qualitative Analysis

  • Qualitative examples show correct predictions despite dissimilar scenes or similar falsified topics, while one pristine example was misclassified amid weak caption-context links and dissimilar visual evidence.The displayed figure presents only a subset of collected evidence.Source: §5.3 · p. 8, Figure 6 · p. 8

5.4. Discussion and Limitations

  • The authors frame the system as an inspectable aid, not a replacement for humans, and note dependence on search-engine retrieval, noisy evidence without relevance supervision, and potentially contradictory web evidence.They did not observe contradictory evidence in the dataset; studying such cases is outside this work’s scope.Source: §5.4 · p. 8

6. Conclusion

CCN aggregates consistency signals from multimodal Web evidence and the image-caption pairing. The work reports improved performance over previous baselines and presents a benchmark and inspectable tool for multimodal fact-checking.

6. Conclusion

  • CCN aggregates consistency signals and consensus from multimodal Web evidence and the given image-caption pairing.Source: §6 · p. 8
  • The work reports that CCN significantly outperforms previous baselines for multimodal fact-checking.Source: §6 · p. 9
  • The paper offers a new multimodal fact-checking task and benchmark, plus an automated, inspectable tool to assist manual fact-checking.Source: §6 · p. 9

Supplementary Material

The supplementary material covers implementation details, additional experiments, user-study examples, qualitative examples, and societal aspects.

Supplementary Material

1. Implementation Details

The implementation details specify text and image representations, evidence-memory batching, and training configurations for CLIP and CCN.

1. Implementation Details

  • Captions are preprocessed to remove artifacts; BERT+LSTM uses bert-base-uncased with 768 dimensions and a 150-token maximum, while sentence-transformer uses paraphrasempnet-base-v2.Source: §1 · p. 11
  • Items in each image, entity, and caption memory are padded to the maximum number of evidence items in that memory's batch.Source: §1 · p. 11
  • The CLIP implementation uses pretrained ViT-B/32, with text truncated at 77 tokens.Source: §1 · p. 11
  • CLIP is trained for 100 epochs and CCN for 30; precomputed representations reduce CCN training to roughly five hours, versus roughly 30 hours with BERT without precomputing.Source: §1 · p. 11

2. Additional Experiments

Additional experiments test evidence-only classification and alternative model configurations, finding that query-evidence reasoning distinguishes classes and that intermediate fusion with concatenated memories performs best among tested options.

2. Additional Experiments

  • 53.4% accuracy from an evidence-only model is basically chance level, supporting the finding that reasoning against the query distinguishes pristine from falsified examples.Source: §2 · p. 11
  • Late fusion performed worse than the model's intermediate fusion, while concatenating visual and textual memories before CLIP had the highest performance among the tested strategies.Source: §2 · p. 11
  • Increasing the penultimate-layer dimension to 2048 increased accuracy by 0.3 percentage points, a relatively small effect.Source: §2 · p. 11

3. User Study: ‘Hard to Verify’ Examples

Hard-to-verify examples expose cases where image-caption evidence is incomplete or contextually similar, making falsified pairs difficult to distinguish.

3. User Study: ‘Hard to Verify’ Examples

  • Specific locations in captions can be hard to verify from the image alone, and missing textual evidence can leave context unconfirmed.Some images were not found by inverse image search, so no captions or titles were returned.Source: §3 · p. 11
  • Generic or unrelated entities and evidence visually similar to the query can make model performance depend on the evidence’s similarity to the image.The passage also notes falsified pairs whose context closely resembles the original and its evidence.Source: §3 · pp. 11–12
  • A falsified pair was misclassified when its caption and evidence shared the people and band name but did not strongly contradict one another.The caption did not clearly emphasize the band performing onstage, so the visual evidence may have remained generic.Source: §3 · p. 12

4. Qualitative Examples

Qualitative examples show CCN attending to relevant evidence while distinguishing contextual contradictions from mere overlap.

4. Qualitative Examples

  • CCN correctly predicted many examples despite imperfect evidence matching for pristine pairs and close evidence similarity for falsified pairs.The examples illustrate that exact one-to-one matching was not necessary in these cases.Source: §4 · p. 12
  • For three pristine examples, attention highlighted supporting names, topics, and events; one example also emphasized a same-scene image and a caption subset.The subset was “soon to be a Trump International Hotel.”Source: §4 · p. 12
  • Falsified examples were correctly identified despite shared people when location or scene details contradicted the claim; another showed a different building and place.Contrasting details included lighting, stage setup, or colours.Source: §4 · p. 12

5. Limitations and Societal Aspects

The authors see automation as useful for reducing manual fact-checking effort, but warn that model limitations and vulnerabilities make human oversight necessary.

5. Limitations and Societal Aspects

  • Manual fact-checking can take less than an hour to many days, motivating automation to alleviate the burden on fact-checkers and journalists.Source: §5 · p. 12
  • Over-reliance on automated tools can create false security, including flagging true examples as falsified and missing difficult falsified examples.Source: §5 · p. 12
  • Verification models may rely on dataset biases, struggle with multi-hop claims, or use outdated retrieved evidence and stored world knowledge.Source: §5 · p. 12
  • The model may be vulnerable to image-based adversarial attacks, and fact-checking tools could be misused to curate hard examples they misclassify.Source: §5 · p. 12
  • The authors recommend keeping humans in the loop; their framework can filter and select important evidence for quicker inspection.Source: §5 · p. 12

Related papers

Loading 2112.00061v3…