Source-linked AI summary
Images and Misinformation in Political Groups: Evidence from WhatsApp in India
Kiran Garimella, Dean Eckles
TL;DR
Because WhatsApp’s semi-closed nature makes misinformation difficult to study systematically, the paper examines images shared in political groups in India. It labels image samples and finds misinformation prevalence and recurring categories, while noting that the data and context-dependent annotations constrain the findings.
Problem
WhatsApp’s semi-closed nature has limited systematic evidence on misinformation prevalence, types, and detectability, including for images.
Source: §2 · p. 1Method
The authors collected messages from political WhatsApp groups in India and used journalist annotations of image samples to study misinformation.
Source: §4.1 · p. 7, §4.2 · p. 8Results
Up to 13% of images shared in the studied groups are misinformation, with three main types: out-of-context images, photoshopped images, and memes.
Source: §1 · p. 1Takeaways & Limitations
The typology may help stakeholders, while addressing misinformation likely requires a mix of technical, social, and policy approaches.
Source: §2 · pp. 2–3The group sample is specific to public WhatsApp messages in India and may be biased, while image judgments can depend on context and timing.
Source: §4.1 · p. 8, §4.2 · p. 9
Abstract
from arXiv · showhide
WhatsApp is a key medium for the spread of news and rumors, often shared as images. We study a large collection of politically-oriented WhatsApp groups in India, focusing on the period leading up to the 2019 Indian national elections. By labeling samples of random and popular images, we find that around 13% of shared images are known misinformation and most fall into three types of images. Machine learning methods can be used to predict whether a viral image is misinformation, but are brittle to shifts in content over time.
1 Key Takeways
The paper documents image misinformation in political WhatsApp groups in India, categorizes its main forms, and tests machine-learning detection. The models can appear promising but are not robust to changes over time.
1 Key Takeways
- The study draws on a large dataset of images from thousands of public WhatsApp groups in India.The authors annotated a sample with help from journalists.Source: §1 · p. 1
- Up to 13% of all images shared in the studied WhatsApp public groups are misinformation.The estimate is based on journalist-assisted annotation.Source: §1 · p. 1
- The three main types are images taken out of context, photoshopped images, and memes.Source: §1 · p. 1
- Machine-learning models can predict image misinformation, but they are not robust to changes over time.The paper reports that results can sometimes appear promising.Source: §1 · p. 1
- The authors describe this as the first study of misinformation on WhatsApp and in image form at scale.They say the findings can help develop technology and policy.Source: §1 · p. 1
2 Argument & Implications
The paper examines the prevalence, forms, and detectability of image misinformation in a setting that is difficult to study systematically. It reports annotation and sampling constraints, limits in detection, and possible roles for technology, users, and platforms.
2 Argument & Implications
- The data cover public WhatsApp messages specific to India and may miss fringe or extreme content; the authors also note unsettled ethical norms for collection.Source: §2 · p. 2
- The estimate that about 13% of image shares are misinformation rests on annotations of only 0.1% of the dataset, despite oversampling highly shared images.Image judgments can depend on sharing context and timing, and require fact-checking expertise.Source: §2 · p. 2
- Almost 70% of misinformation images fall into three categories; out-of-context detection may be feasible, while manipulation tools confused photoshopped images with memes.Source: §2 · p. 2
- Content-based classifiers are not robust to training-data choice, with performance degrading across time periods, and the models are not close to deployable.Most models performed better than a random guess.Source: §2 · p. 2
- The typology and findings may inform regulators, technology companies, civic campaigners, media literacy, and political communication.Source: §2 · p. 2
- The authors propose repositories of old and fact-checked images, further study of users’ sharing motivations, and platform tools within a broader technical, social, and policy response.Source: §2 · pp. 2–3
3 Findings
The study finds image misinformation is prevalent in political WhatsApp groups, with most examples falling into three recurring types. Machine-learning detection shows promise in some settings but is brittle across time, while sharing patterns also differ across political groups.
Prevalence
- 13% of image shares were misinformation, including 10% obviously misinformation and 3% previously fact-checked misinformation, estimated using inverse probability weighting.The estimates cover image shares in monitored political groups, even though many images could not be conclusively labeled.Source: §3 · p. 3, Table 1 · p. 3
Types of misinformation
- Over 70% of misinformation images in the dataset fell into three categories: old images taken out of context (roughly 34%), memes with fake quotes or statistics (30%), and simple photoshopped images (roughly 10%).Source: §3 · p. 4
Detecting misinformation
- Metadata features gave higher accuracy for distinguishing misinformation from random images than from images labeled not misinformation, and combining features added little.The results suggest a weak signal in the groups and users where misinformation images spread.Source: §3 · p. 5, Table 2 · p. 6
- Content-feature accuracy was 0.57 for misinfo vs. not misinfo and 0.71 for misinfo vs. random, while pretrained ResNext exceeded 0.95 in some cases.Table 3 reports classification using Google Vision API features and, separately, a pretrained ResNext model.Source: §3 · p. 6, Table 3 · p. 6
Robustness over time
- Content-feature performance varied across time periods, falling as low as 0.55 from 0.95, while metadata features changed little across temporal splits.The authors use the variation to argue that robustness testing matters, especially for black-box models.Source: §3 · p. 6
Detecting misinformation types
- Web-domain features detected out-of-context images with 0.78 accuracy for misinfo vs. not misinfo, compared with 0.59 across all images.The authors attribute this pattern to out-of-context images' greater likelihood of appearing on low-quality or fact-checking websites.Source: §3 · p. 6
Political-group differences
- BJP groups had the highest misinformation-image fraction at 8%, compared with 3% for Congress groups.Source: §3 · p. 6
- Images were more likely to be shared across groups of the same party than across parties, a pattern extending to ideologically related groups but less precisely for misinformation images.BJP and Congress had 31% overlap among their top 100 images, while Hindu and Muslim groups had 21% overlap.Source: §3 · pp. 6–7, Figure 3 · p. 7
4 Methods
The study builds a dataset of images shared in public Indian political WhatsApp groups and uses expert annotation, image matching, and clustering to characterize misinformation. It then estimates prevalence and tests machine-learning models using image-spread and content features.
4.1 Data collection
- Researchers collected over 5 million messages from more than 5,000 political groups and 1.6 million images, but group sampling was difficult to estimate and overrepresented BJP-affiliated groups.The data span October 2018 to June 2019 and include over 250,000 users; 24.4% of groups apparently belonged to BJP or aligned religious groups, versus 5% for Congress.Source: §4.1 · pp. 7–8
4.2 Annotating the images for misinformation
- Three journalists annotated 2,500 images (500 random, 2,000 popular) using chat-context and reverse-search tools; after removing clear disagreements, 605 were labeled false.The two-class misinformation/not-misinformation Fleiss’ Kappa was 0.78.Source: §4.2 · pp. 8–9
4.3 Fact checked images
- Matching WhatsApp images against nearly 20,000 images scraped from fact-checking sites identified roughly 200 images fact-checked as false for inclusion in the misinformation set.Image matching used 256-bit perceptual hashes designed to detect near-similar images, including cropped or text-overlaid variants.Source: §4.3 · p. 9
4.4 Image clustering
- Facebook PDQ hashes and DBSCAN clustered image variants, yielding about 40,000 instances across 800 clusters shared by over 13,000 users.Variants of an annotated misinformation image were included as potential misinformation.Source: §4.4 · p. 9, Figure 5 · p. 10
4.5 Prevalence estimation
- Inverse-probability weighting combines the random and popular samples to estimate image prevalence and share-weighted prevalence, with post-stratification by share-count percentiles to improve precision.The combined sample has positive but heterogeneous inclusion probabilities.Source: §4.5 · p. 9
4.6 Machine learning models
- Random Forest models used spread, time, web, image-content, and adjacent-text features; manipulated-pixel features also flagged benign added text, and category-specific features were tested.The authors selected Random Forest for best accuracy and report means over 10-fold cross-validation.Source: §4.6 · pp. 10–11, Figure 6 · p. 11
5 Appendix
The appendix reports funding and competing interests, describes ethics approval and data-collection disclosure, and provides the article’s license and data-availability information.
Funding
- The research received support from a postdoctoral fellowship, an MIT Sloan award, and an Amazon Research Award that provided computing time.The named recipients were K.G. and D.E.Source: §5 · p. 11
Competing interests
- D.E. disclosed prior Facebook employment and significant financial interests in Facebook, Amazon, and Google, as well as research funding from Amazon and The Boston Globe.Source: §5 · p. 11
Ethics
- MIT’s human-subjects committee approved the research with a waiver of informed consent, and the WhatsApp profile disclosed the research and data collection.Source: §5 · p. 11
Copyright
- The article is distributed under a Creative Commons Attribution License permitting reuse with credit to the original author and source.Source: §5 · p. 11
Data Availability
- Annotated images and anonymized sharing patterns are available through the cited Zenodo record.Source: §5 · p. 11