Technology-Enabled Disinformation: Summary, Lessons, and Recommendations

1 Introduction

• Technology enables false information to be created realistically and spread with unprecedented speed and scale, with potentially serious societal consequences.The report cites potential effects on the 2016 U.S. election, vaccination distrust, murders in India, and individuals’ lives.Source: §1 · p. 1
• The report asks how technology and technical platforms enable mis/disinformation and how technical approaches might mitigate it or fall short.It approaches these questions from computer science while recognizing that the broader problem requires study and collaboration across disciplines.Source: §1 · p. 1
• The report surveys what is new in information pollution, current ecosystems, enabling technologies, and efforts to curb mis/disinformation, then presents lessons and recommendations.Its recommendations address technologists, researchers, platform designers, policymakers, and users.Source: §1 · p. 1
• Technologists should recognize that technology designs are not neutral, understand potential harms, and work with experts in other fields because solutions are not solely technological.The report calls for designing defenses or technologies that attempt to avoid the problem, alongside cross-field collaboration.Source: §1 · pp. 1–2

2 Understanding Mis- and Disinformation

• The report provides an overview of technology-enabled mis/disinformation, including what is new, what is happening, how technology enables it, and how technology may defend against it.It also recommends related summaries for readers seeking further coverage.Source: §2 · p. 2

2.1 What Is New?

• Anyone can publish content, and accessible tools such as Deep-Fakes lower the time and resource barriers to creating realistic fakes.The shift from a few major producers to many producers also makes it harder to distinguish true from false information.Source: §2.1 · p. 2
• Online content can reach across the world nearly instantaneously, while creators can use engagement feedback to iterate and A/B test campaigns.The report contrasts this with the six years it took the AIDS disinformation campaign to take off in the 1980s, compared with viral spread of Pizzagate during the 2016 election.Source: §2.1 · p. 2
• Filter bubbles can expose receptive audiences to mis/disinformation in limited context, and actors can deliberately target potentially vulnerable individuals.Research findings on the extremity of filter bubbles are mixed: some studies suggest they are less extreme than feared, while others find evidence of increased polarization.Source: §2.1 · p. 2
• Complex, nontransparent curation can obscure why users see particular content, its source, and how their online experience differs from others.The report notes criticism that YouTube’s recommender algorithms push people toward more extreme content.Source: §2.1 · p. 2
• Weak identity and account-management systems let actors create many accounts to pose as legitimate users or make content appear popular.Stricter verification can harm legitimate users, and bot accounts can also serve legitimate purposes.Source: §2.1 · p. 2
• Technology trends have changed the information ecosystem, but effective mis/disinformation still relies on consumer cognitive biases.Confirmation bias favors information consistent with prior beliefs, while the backfire effect suggests corrective facts can sometimes strengthen an original belief.Source: §2.1 · pp. 2–3

2.2 Measuring the Problem

• Understanding adversaries, targets, techniques, and effectiveness is important for developing defenses, and studies have examined particular campaigns as well as broader patterns.The section reviews case studies and more general studies or surveys.Source: §2.2 · p. 3, §2.2.1 · p. 3

2.2.1 Ecosystem Studies

• Disinformation campaigns use techniques ranging from spambots to supervised semiautomated and human-run accounts, including impersonation, bot promotion, and government-funded media.Some human-run accounts build followings before using the trust in their social links to spread content.Source: §2.2.1 · p. 3
• A study of Syrian White Helmets coverage found alternative news sources copied content without attribution and formed a densely linked cluster largely disconnected from reputable news.Differences in ecosystem structure may help detect mis/disinformation.Source: §2.2.1 · p. 3
• Fake-news consumption across U.S. states correlated with aggregate voting behavior during the 2016 election, but causality was unclear.Users may trust an article based on the friend who shared it rather than its source or upstream links; platform design can obscure sources and sharing links.Source: §2.2.1 · p. 3

2.2.2 Tools for Measurement

• Because measurement and story-by-story fact-checking require substantial manual effort, tools have been developed to track content sources and trajectories or facilitate fact-checking.Hoaxy tracks Twitter content and fact-checking; its creators found fact-checks were shared significantly less than original articles [84].Source: §2.2.2 · p. 3
• Hoaxy research identified social spambots’ central role in the ecosystem, suggesting that curbing them may be crucial; TwitterTrails and Hamilton 68 provide propagation and influence-operation information [85] [38].TwitterTrails visualizes content origin, burst characteristics, and propagation to help users investigate veracity.Source: §2.2.2 · p. 3
• Measurement tools are not a silver bullet: some are insufficiently automated, their surfaced information may be misunderstood, and their Twitter focus excludes closed platforms such as Facebook.Campaigns vary widely, automatic fake-news detection is hard, and measurement may illuminate ecosystem dynamics without directly providing a solution.Source: §2.2.2 · p. 4

2.3 Enabling Technologies

• False video, individualized tracking and targeting, and social bots are identified as key technologies underlying the rise of mis/disinformation at scale.Source: §2.3 · p. 4

2.3.1 False Video

• AI-based face reanimation can produce realistic false video, while tools such as FakeApp have made synthetic video creation possible for individuals in a growing set of cases.A typical deepfake reanimates a target video using facial expressions from a source video, often with generative adversarial networks and substantial training data.Source: §2.3.1 · p. 4
• Deep Video Portraits required only a few minutes of target video for realistic real-time reanimation, while another approach derived reanimation from audio but required substantial target training video.Source: §2.3.1 · p. 4
• Synthetic video can spread mis/disinformation and undermine legitimate information by allowing claims that authentic material has been faked.Source: §2.3.1 · p. 4

2.3.2 Tracking and Targeting

• Advertising-driven tracking supports targeted content, and the same systems can be exploited to pursue monetary or malicious aims, including influencing elections.Third-party trackers can re-identify users across websites and build browsing histories; no fully reliable way for users to avoid tracking is described.Source: §2.3.2 · pp. 4–5
• Targeting can expose private information or target sensitive attributes, while disinformation campaigns can exploit these ecosystems to reach vulnerable users and leverage filter bubbles.Source: §2.3.2 · p. 5

2.3.3 Social Bots

• Social bots amplify low-credibility content to create false impressions of support or intensify partisan debate; sophisticated bots build trust with context-relevant posts before spreading false or extremist content.Some operators combine automation with human control, a practice described as “cyborg” operation.Source: §2.3.3 · p. 5
• Bot detection draws on network structure, account and posting data, language analysis, crowdsourcing, and platform-observed behavior, but lacks reliable ground truth and faces evasion and arms-race challenges.Benign bots complicate detection, and platform bans may depress valuations even though platforms have useful behavioral visibility; campaigns also involve human operators, not only bots.Source: §2.3.3 · p. 5

2.4.1 Fake News Detection

• Fake-news detection is better approached through carefully defined subtasks than as an undifferentiated machine-learning problem, and a common definition requires information to be factually false and intentionally misleading.Source: §2.4.1 · p. 6
• End-to-end detection requires costly labeled examples, can lack explanations, and may inherit dataset biases that impair real-world performance.Source: §2.4.1 · p. 6
• Factual-verification approaches compare article claims with external evidence, but manual fact-checking is costly and automated knowledge-based verification struggles with formalization and novel information.Related subtasks include checking whether claims are supported by sources, using external web information, and classifying a text’s stance toward a headline.Source: §2.4.1 · p. 6
• Detection can also examine manipulative writing style and social metadata such as sharing, likes, user connectivity, sharer attributes, and propagation paths.Manipulative style may signal intent, but its absence does not establish good intentions, and such style may also relate to hyper-partisanship.Source: §2.4.1 · p. 6
• Skewed fake-news prevalence can mislead performance estimates, and detection systems may be deployed inaccurately or exploited by bad actors to improve fake content.Source: §2.4.1 · p. 6
• Even perfect detection does not guarantee defense because people’s psychological and cognitive patterns shape their response; evidence against a belief can trigger a backfire effect.Source: §2.4.1 · p. 6

2.4.2 UI/UX Interventions

• User-facing interventions include disputed-content labels, related articles, added context, political-ad disclosures, forwarded-message indicators, and tools for investigating images and videos.Facebook abandoned disputed labels after learning they did not change people’s minds; related articles were used instead.Source: §2.4.2 · p. 7
• Education, news-literacy cues, source labels, and feed-design changes are among the proposed interventions, but their effects are difficult to study without platform cooperation and accurate identification of unreliable content.Platform disclosures may not be replicable or generalizable, and some defenses still rely on human raters.Source: §2.4.2 · p. 7

3 Discussion

• This discussion shifts from summarizing the landscape and technology’s role to presenting the group’s broader lessons and stakeholder recommendations.Source: §3 · p. 7

3.1 Lessons

• Different motives require different responses: financial actors may be deterred through demonetization or review, while state-level actors may call for better detection and attribution.The report cites Facebook clickbait, fake news enterprises in Macedonia, and international legal proceedings as examples.Source: §3.1 · p. 8
• Mis/disinformation is difficult to define, so detection and moderation efforts should specify what counts as harmful and whether that definition is useful in practice.The report notes that some cases, such as human rights atrocities, may present clearer boundaries.Source: §3.1 · p. 8
• Automated detection alone is insufficient because the problem exploits human cognitive vulnerabilities and calls for broader approaches alongside technical defenses.The report argues that promising technical approaches still merit further research and implementation.Source: §3.1 · p. 8
• Attackers can cheaply test strategies and disappear, while defenders face a harder task of defining the problem and working against cognitive biases.The report describes this as an asymmetric arms race in which adversaries can use technology to worsen the problem.Source: §3.1 · p. 8
• Adversaries use both sophisticated targeted content and simple tactics, so detecting today’s methods does not ensure detection of tomorrow’s.Examples include tailored content, convincing fake videos, viral edited photos, and copied article text.Source: §3.1 · p. 8
• Platform and user incentives may conflict with mitigation, and responses should build legitimate trust rather than foster blanket skepticism.For-profit platforms may prioritize user numbers that affect valuations, while some campaigns seek to undermine institutions.Source: §3.1 · p. 8

There may be lessons to apply from traditional computer

• Interrupting users to decide what is good or bad is not, by itself, an effective strategy.Targeted spear phishing can undermine even the best technical systems by targeting human cognitive vulnerabilities.Source: There may be lessons to apply from… · p. 8

3.2.1 Recommendations for Technologists and Other Defenders

• Set specific, useful, achievable goals for technical solutions rather than treating a particular technology as the goal.The report contrasts stance detection in the Fake News Challenge with broader fake-news detection efforts, whose applicability and effectiveness may be limited.Source: §3.2.1 · p. 9
• Work on solvable technology-related subproblems, including defined detection tasks, information chain of custody, UI/UX interventions, bot detection, and user verification.The report suggests combining detection subproblems into larger pipelines and evaluating interventions through formative research.Source: §3.2.1 · p. 9
• Continue measurement studies because effective defenses require understanding the ecosystems that create, spread, and respond to mis/disinformation.Source: §3.2.1 · p. 9

Tailor solutions to specific types of users, content, and/or

• Tailor approaches to different actors and users, reconsider financial incentives behind online services, and develop and evaluate education that teaches technology awareness without promoting blanket distrust.The report also cautions that burdensome user interventions may be ignored and that tools requiring extra effort may have limited reach.Source: Tailor solutions to specific types of… · p. 9

3.2.2 Recommendations for Platform Designers

• Platform designers should examine how affordances, recommendation systems, transparency, and identity practices affect mis/disinformation, and consider changing organizational incentives.The report notes that likes, shares, and comment selection can confer credibility and encourage falsehoods; it also recommends studying ways to downweight extreme and misleading content.Source: §3.2.2 · p. 10
• Platforms should enable users and researchers to evaluate platform effects, including through experiments and research partnerships.Facebook’s SocialScienceOne is given as an example of providing platform data to researchers.Source: §3.2.2 · p. 10

3.2.3 Recommendations for Researchers

• Researchers should make findings accessible to the public, anticipate misuse of technologies, and collaborate across disciplines because binary fake-news labels may backfire.The report also recommends studying differences among user groups and notes that foundational science is needed to understand their effects.Source: §3.2.3 · pp. 10–11

3.2.4 Recommendations for Policymakers

• Policymakers should consider whether regulation can realign platform incentives while grounding proposals in technical realities.The report cautions that a proposed bot self-identification bill uses an imprecise definition and could be difficult or impossible to enforce.Source: §3.2.4 · p. 11

3.2.5 Recommendations for Users

• Users should be skeptical of suspicious content, seek corroborating or conflicting sources, and stay open to changing their minds.Source: §3.2.5 · p. 11
• Users should not assume everything is false; they can build trust by finding original sources, consulting trusted news sources, and seeking diverse sources.They should also watch for their own cognitive biases, including anger or confirmation of existing views.Source: §3.2.5 · p. 11

• The report calls for further efforts by researchers, technologists, platform designers, educators, and policymakers to study and combat misinformation and disinformation.It argues that the problem cannot be eliminated completely because human cognitive biases and their exploitation for persuasion predate the technologies discussed.Source: §4 · p. 11
• Technologists should understand potential harms, deploy defensive technologies for specific achievable goals, and collaborate with other fields.Source: §4 · p. 11
• The report presents an academic quarter’s investigation of the mis/disinformation landscape from a computer science perspective.Its summaries, lessons, and recommendations are intended to benefit interested readers.Source: §4 · p. 11