Source-linked AI summary
NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media
Grace Luo, Trevor Darrell, Anna Rohrbach
TL;DR
NewsCLIPpings addresses out-of-context image use in misinformation, where real images and captions are mismatched rather than manipulated. It automatically constructs mismatch datasets using multiple retrieval strategies and finds that these cases are difficult for models and humans while supporting automated image repurposing.
Problem
Out-of-context reuse of real images remains widespread, while existing inconsistency datasets often rely on text manipulation rather than mismatched, unmanipulated image-caption pairs.
Method
The paper constructs NewsCLIPpings from VisualNews using caption-image, caption-caption, person, and scene matching strategies, then benchmarks multimodal models across dataset splits and architectures.
Results
NewsCLIPpings provides a challenging benchmark: CLIP classifies the Merged split correctly 60% of the time, while human accuracy averages 66%.
Takeaways & Limitations
The dataset supports development of defenses against large-scale image repurposing and shows that multimodal models need entity and semantic knowledge to detect mismatches.
Takeaways & Limitations
The dataset-generation models may introduce social biases, including reported race- and gender-related disparities in CLIP and biases inherited by SBERT-WK from BERT.
Abstract
from arXiv · showhide
Online misinformation is a prevalent societal issue, with adversaries relying on tools ranging from cheap fakes to sophisticated deep fakes. We are motivated by the threat scenario where an image is used out of context to support a certain narrative. While some prior datasets for detecting image-text inconsistency generate samples via text manipulation, we propose a dataset where both image and text are unmanipulated but mismatched. We introduce several strategies for automatically retrieving convincing images for a given caption, capturing cases with inconsistent entities or semantic context. Our large-scale automatically generated NewsCLIPpings Dataset: (1) demonstrates that machine-driven image repurposing is now a realistic threat, and (2) provides samples that represent challenging instances of mismatch between text and image in news that are able to mislead humans. We benchmark several state-of-the-art multimodal models on our dataset and analyze their performance across different pretraining domains and visual backbones.
1 Introduction
The paper targets automated image repurposing, where real images and captions are mismatched to create convincing false news. NewsCLIPpings constructs such examples while requiring joint multimodal analysis because neither modality is manipulated.
- Motivation: Real images are reused out of context to misrepresent people, events, or locations and make misleading narratives seem credible.Adding an image can increase engagement and make false stories seem true.
- Problem: The task requires joint image-text analysis because both modalities remain real, unlike synthetic-media detection settings.The goal is to classify each image-caption pair as pristine or falsified.
- Contribution: NewsCLIPpings automatically constructs large-scale real and out-of-context news from VisualNews using semantic, entity, person, and scene matching strategies.The dataset also applies adversarial CLIP filtering to make mismatches more challenging.
- Evaluation: The dataset benchmarks multimodal models, examines pretraining domains and visual backbones, and includes human evaluation of pristine-versus-falsified discrimination.The authors also analyze model-discovered visual cues and release the dataset publicly.
2 Related Work
Prior multimodal misinformation datasets create mismatches through random image swaps, text entity manipulation, paired-caption consistency, or synthetic text. NewsCLIPpings instead studies automatically retrieved, non-random image-caption mismatches without assuming a reference package or two captions.
- Existing datasets: MAIM randomly matches images and captions, while MEIR swaps named entities and assumes an unmanipulated related package for verification.NewsCLIPpings does not assume availability of a perfect reference set.
- Text manipulation: TamperedNews swaps named entities in article text, but the resulting linguistic biases can let text-only models solve the task.NewsCLIPpings avoids textual manipulation to reduce unimodal bias.
- Paired captions: Aneja et al. analyze whether an image is consistent with two captions from distinct news sources, whereas NewsCLIPpings classifies individual image-caption pairs.The earlier dataset is also mostly unlabeled, with a manually annotated subset.
- Synthetic text: Synthetic-news work replaces articles or captions with generated text while retaining relevant images, making image analysis less central to detection.NewsCLIPpings focuses on automated retrieval of an image to match a given caption.
3 The NewsCLIPpings Dataset
NewsCLIPpings forms falsified pairs by retrieving real images for pristine captions across semantic, person, and scene-based threat scenarios. Its merged split combines these mismatch types, while adversarial filtering and balanced construction target challenging, unimodally unbiased evaluation.
- Construction: The dataset begins with a pristine pair (img1, cap1) and retrieves another pair (img2, cap2) to create the falsified pair (img2, cap1).The source corpus is VisualNews, containing image-caption pairs from four news agencies.
- Query for Semantics: Semantic retrieval uses CLIP text-image or text-text similarity to find images with related content while producing out-of-context pairings.The semantic split targets mislabeling the people or entities depicted in the image.
- Query for Person: Person retrieval targets images depicting a named person in a different context, using entity and person-detection constraints and SBERT-WK text similarity.Captions are filtered to reduce cases where the person is mentioned but unlikely to appear in the image.
- Query for Scene: Scene retrieval matches images with similar scene types while excluding captions containing PERSON entities, using Places365 ResNet similarity.This split is intended to mislabel the event rather than a particular person.
- Merged Split: The merged split contains equal numbers from all four splits, with captions and images disjoint across splits, to model varied mismatches at test time.It represents a more realistic mixture of out-of-context image-generation methods.
- Adversarial CLIP Filtering: Adversarial CLIP filtering reduces the initial score-distribution difference between pristine and falsified samples by retaining convincing falsified candidates.Candidates are compared using CLIP text-image similarity and split-specific retrieval rankings.
- Dataset Statistics: Exactly half of the samples are pristine and half falsified because each caption appears once in each type, eliminating unimodal text bias.The dataset reports both total samples including duplicates and unique text-image pairings.
4 Experiments
The experiments evaluate multimodal classifiers on NewsCLIPpings, testing unimodal bias, model architecture, pretraining domain, backbone size, and unified training. Results show that mismatch detection depends on both multimodal evidence and the type of mismatch, with strengths varying across models and splits.
- Experimental setup: The benchmark compares image-only CLIP, multimodal CLIP, VisualBERT-CC, and VisualBERT-VN using classification accuracy and ROC performance across NewsCLIPpings splits.The experiments also vary pretraining domain, visual backbone, and whether one model is trained jointly on all splits.
- Unimodal Model Performance: Image-only CLIP performs slightly above chance and substantially below the full image-text model, indicating that the task provides limited visual-only bias.The Person split is additionally balanced with respect to images, making image-only performance exactly 50% there in expectation.
- Multimodal Model Performance: CLIP over-predicts pristine labels, while the Person split is hardest and the Merged split reaches 60% accuracy versus 61% on Person.These results indicate that many falsified samples remain realistic and plausible for the multimodal classifier.
- Pretraining domain: VisualBERT-VN gains 3–5% from news-domain pretraining and exceeds CLIP on Person, whereas CLIP is strongest on named-entity understanding.VisualBERT-CC performs 10% worse than CLIP on the other entity-focused splits, while VisualBERT-VN approaches CLIP on Merged.
- ROC Curves: CLIP outperforms other models by a wide margin across false alarm rates on Semantics/CLIP Text-Text, while VisualBERT-VN slightly leads at higher false alarm rates on Person/SBERT-WK Text-Text.At low false alarm rates, VisualBERT-VN and CLIP have virtually identical Person performance.
- Comparing CLIP Models: RN50 performs slightly better than ViT-B/32 by at most 2%, while RN101 adds only 1–2% on most splits and 4% on Merged/Balanced.The dataset remains challenging despite changing the CLIP backbone or increasing model parameters.
- Evaluating A Single Unified Model: Unified training boosts Person accuracy by 2% and Merged/Balanced by almost 6%, but degrades Semantics/CLIP Text-Text by 5%.The authors hypothesize that joint training makes different mismatch types harder to distinguish, while pristine accuracy generally benefits more than falsified accuracy.
5 Additional Analysis
Additional analyses test whether NewsCLIPpings misleads humans and examine the visual cues used by CLIP. Human difficulty varies by mismatch type, while saliency examples reveal both useful recognition cues and characteristic failures.
- Caveat: Joint training can reduce Semantics/CLIP Text-Text performance by 5%, possibly because unknown pictured identities make semantic mismatches resemble Person mismatches.This is presented as a hypothesis about the interaction between the Person and Semantics samples.
- Human Performance: The human evaluation samples 200 Merged/Balanced examples equally across four mismatch types, with five workers answering questions for each image-caption pair.The evaluation is designed to estimate task difficulty and whether automatically matched examples can fool humans.
- Human Performance: 67% of the 31 samples misclassified by every worker appear answerable with additional person-identity or contextual knowledge.Humans are more accurate on familiar concepts, and reported search-engine use is associated with lower observed accuracy in the supplied results.
- Human Performance: Humans find Scene easiest and Person hardest, with Semantics/CLIP Text-Text and Semantics/CLIP Text-Image between them; this ordering aligns with automatic-model trends.The comparison spans all four mismatch types in the evaluation.
- Qualitative Analysis: Fine-tuning often shifts CLIP saliency toward caption-mentioned objects beyond the queried person, while existing sign-reading and logo-recognition abilities require little fine-tuning.Examples include distinguishing Tigerair Australia from Germanwings and expanding attention from David Cameron to an entourage.
- Qualitative Analysis: CLIP can fail when a caption mentions marches but the image lacks a crowd, because it focuses on foreground people and overlooks the missing contextual cue.Similar failures occur when captions mention multiple people but the image depicts only one, reflecting ambiguity in pristine news imagery.
6 Conclusion
NewsCLIPpings is an automatically constructed benchmark for classifying news image-caption pairs as real or out of context. The task is difficult for humans, while multimodal models and the dataset support research on detecting large-scale image repurposing.
- NewsCLIPpings classifies news image-caption pairs as real or out of context using an automatically constructed dataset.
- By design, unimodal models cannot solve the task, while multimodal models need named-entity and semantic knowledge on diagnostic splits.
- The dataset’s Merged set models the diversity of image-caption mismatches found in the wild.
- Average human accuracy was 66%, and a CLIP ViT model matched human performance on a small subset of the Merged / Balanced split.
- The training data can augment human-made falsified-news data and provide a challenging benchmark for defenses against large-scale image repurposing.
7 Ethical Considerations
The authors frame NewsCLIPpings as proactive training data for a societal threat while acknowledging dataset bias, misuse risks, and practical limits on targeted attacks. The dataset primarily represents Western English-language news, and its construction has documented computational costs and model-related bias concerns.
- The dataset targets image repurposing, a societal issue with costly human generation and annotation and limited training data.
- The dataset mainly covers events and people from Western countries, especially the US and UK, plus world news.
- The authors expect danger and detection-performance results to apply primarily to Western English-language news.
- Human evaluation found the generated matches challenging for humans, while construction and finetuning required estimated emissions of 1.28 kg and 0.97 kg CO2 eq, respectively.
- NewsCLIPpings can be misused to mass-generate repurposed images alongside synthetic or manually written inflammatory captions.
- Targeted attacks are not immediately supported because attackers need synthetic captions causing domain shift or manually tailored captions requiring time and money.
A Appendix to “NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media”
The appendix provides additional details on dataset construction, experiments, and examples.
- The appendix covers additional dataset-construction details, experimental results, and dataset examples.
A.1 Dataset Construction
Dataset construction filters VisualNews pairs, partitions them for parallel processing, computes multimodal features, runs the matching algorithm, and balances match quality.
- Pristine samples contain 5–30 words, at least two named entities, and non-corrupted images.
- The 509,730 pristine pairs are split into approximately 40k-sized train, validation, and test chunks, with matching restricted across disjoint chunks.
- Construction precomputes entity, text, image, object, and scene features before running the NewsCLIPpings matching algorithm.
- Low-quality samples are removed by balancing CLIP text-image scores between pristine and falsified pairs.
A.2 Additional Results
Additional experiments show that model performance depends on the dataset split, visual backbone, finetuning strategy, and model complementarity. CLIP dominates the split constructed with CLIP scoring, while combining VisualBERT and CLIP offers complementary predictions but only a modest ensemble gain.
- ROC Curves: CLIP vastly outperforms other models on the Semantics/CLIP Text-Image split, while performance is similar across models on Merged/Balanced.The split-specific difference is consistent with CLIP Text-Image being used as the matching score for the Semantics split.
- Finetuning CLIP Representation: Finetuning all CLIP layers benefits Semantics splits, whereas freezing some or all layers benefits Person and Scene splits.The authors associate this pattern with the larger training size of the Semantics splits.
- Ensembling Performance: The two best models have roughly 40% prediction overlap, but their union exceeds 80% accuracy, indicating complementary strengths.The comparison is between the best VisualBERT and CLIP-based models.
- Ensembling Performance: Averaging normalized VisualBERT and CLIP logits improves accuracy by only 1–2%, making effective combination non-trivial.The authors suggest that CLIP’s more disparate per-class logits may cause the ensemble to behave similarly to CLIP.
- Results on the Validation Set: Validation results show the same overall trends as the main-paper test results because the validation and test sets were generated with identical techniques.The authors describe the two splits as comparable in composition.
A.3 Additional Dataset Details
The additional dataset analysis finds limited overlap between splits and illustrates that the generated mismatches range from highly plausible semantic matches to cases detectable through identity or subtle scene cues. The accompanying tables document validation performance, backbone comparisons, aggregate training, and split overlap.
- Split Overlap: Falsified matches are relatively distinct across splits, with at most 11% overlap between the two Semantics splits.The reported overlap concerns exact matching across dataset splits.
- Qualitative Samples: Random samples from the Semantics splits are highly plausible, while identity information or subtle semantic cues can reveal some mismatches.Examples include mismatched national cues such as an American flag paired with a caption about a European person.
- Qualitative Samples: Scene-split samples show that captioned events can be plausibly illustrated by leveraging visual scene similarity.The passage characterizes these as randomly selected dataset samples.
- Additional Tables: The validation tables cover image-only and multimodal CLIP, VisualBERT variants, alternative CLIP backbones, aggregate training, and finetuning strategies.The listed tables organize the additional results by model comparison and training configuration.
- Split Overlap: Table 13 reports the ratio of exact overlap across dataset splits.This table corresponds to the reported split-overlap analysis.