Source-linked AI summary
Manipulation and abuse on social media
Emilio Ferrara
TL;DR
Abuse on social media can have economic and societal effects, yet its harmful consequences have only recently drawn focused attention. This article reviews examples of abuse and recent advances, concluding that misinformation and fear can be amplified by social media’s reach and pervasiveness.
Problem
Research has only recently focused on the potentially harmful effects of social media abuse, despite its links to economic and societal risks.
Source: Introduction · p. 2Method
The article illustrates forms and consequences of social media abuse and discusses recent advances and possible solutions.
Source: Introduction · p. 2, Conclusions · p. 7Results
The article concludes that social media’s reach and pervasiveness magnify misinformation and abuse, with examples spanning economic and social harms and fear during emergencies.
Source: Conclusions · p. 7Takeaways & Limitations
The article urges computational frameworks that go beyond data analytics to support effective policies, decision-making, and timely interventions.
Source: Misinformation, fear, manipulation and… · p. 7The article notes that much remains to be done to make social media safer and healthier, despite recent advances in detecting bots, campaigns, and spam.
Source: Conclusions · p. 7
Abstract
from arXiv · showhide
The computer science research community has became increasingly interested in the study of social media due to their pervasiveness in the everyday life of millions of individuals. Methodological questions and technical challenges abound as more and more data from social platforms become available for analysis. This data deluge not only yields the unprecedented opportunity to unravel questions about online individuals' behavior at scale, but also allows to explore the potential perils that the massive adoption of social media brings to our society. These communication channels provide plenty of incentives (both economical and social) and opportunities for abuse. As social media activity became increasingly intertwined with the events in the offline world, individuals and organizations have found ways to exploit these platforms to spread misinformation, to attack and smear others, or to deceive and manipulate. During crises, social media have been effectively used for emergency response, but fear-mongering actions have also triggered mass hysteria and panic. Criminal gangs and terrorist organizations like ISIS adopt social media for propaganda and recruitment. Synthetic activity and social bots have been used to coordinate orchestrated astroturf campaigns, to manipulate political elections and the stock market. The lack of effective content verification systems on many of these platforms, including Twitter and Facebook, rises concerns when younger users become exposed to cyber-bulling, harassment, or hate speech, inducing risks like depression and suicide. This article illustrates some of the recent advances facing these issues and discusses what it remains to be done, including the challenges to address in the future to make social media a more useful and accessible, safer and healthier environment for all users.
Introduction
Despite extensive research on social media use, researchers have only recently recognized the potential harms of platform abuse, driven by social and economic incentives, weak policies, and inadequate detection and prevention tools.
- Social media abuse has become widespread amid social and economic incentives, weak policies against misbehavior, and insufficient tools to detect and hinder improper use.Source: Introduction · p. 2
- The article examines the perils of social media abuse and its effects on society.Source: Introduction · p. 2
- The article also discusses possible solutions to social media abuse.Source: Introduction · p. 2
Misinformation, fear, manipulation and abuse
Social media abuse can turn misinformation, automated manipulation, and fear-rich content into offline economic and societal harm, especially during crises. The section describes research on tracing information flows and emotional responses to inform targeted interventions and safer policy.
Misinformation, fear, manipulation and abuse
- A hacked Associated Press tweet claiming explosions at the White House sent the Dow Jones down 147 points in 3 minutes, erasing $136 billion before it recovered.The episode illustrates how quickly misinformation can spread and affect financial markets.Source: Misinformation, fear, manipulation and… · p. 2, Figure 1 · p. 3
- Social bots and synthetic accounts can manufacture grassroots political support, spread ISIS propaganda and recruitment, and distort social-media signals used for business analytics [Ratkiewicz et al. 2011; Berger and Morgan 2015; Ferrara et al. 2014; Yang et al. 2014].Synthetic activity can also manipulate stock performance and add noise to analytics that use social-media signals to assess business ventures.Source: Misinformation, fear, manipulation and… · p. 3
- Smearing attacks have targeted candidates in elections, and governments and other entities have tried to manipulate public perceptions; containing dangerous campaigns remains a challenge [Metaxas and Mustafaraj 2012; Ratkiewicz et al. 2011; Weng et al. 2013; 2014].The passage cites attacks during the 2009 Massachusetts and 2010 U.S. Senate elections, as well as attempts to shape perceptions on social issues.Source: Misinformation, fear, manipulation and… · pp. 3–4
- Although social media support crisis response, misinformation during Hurricane Sandy and the Boston Marathon bombings spread rumors, fake deaths, false alarms, and untrue stories that could provoke panic.The passage also notes fake donation campaigns and false claims about the bombing suspects during the ensuing manhunt.Source: Misinformation, fear, manipulation and… · p. 6
- The research approach analyzes crisis-related content by producer, sentiment, timing, and diffusion to distinguish awareness from fear and examine how media and official communications shape emotional responses.It aims to assess how language, timing, target-user demographics, and susceptibility affect whether fear and concern are triggered or mitigated.Source: Misinformation, fear, manipulation and… · p. 6
- During a one-hour Ebola discussion, information-seeking tweets spread from a central user group through retweets and mentions to thousands, reaching distant topical communities and news-agency audiences [Zhao and Mei 2013; Ferrara et al. 2013; JafariAsbagh et al. 2014].The network view uses node prominence, centrality, and topical communities to depict the diffusion.Source: Figure 2 · p. 4, Misinformation, fear, manipulation and… · p. 6
- Fear-rich content spread virally to large audiences, potentially reinforcing misinformation and panic; targeted intervention campaigns could interrupt this self-reinforcing cycle.The analysis of the same Ebola-discussion interval describes exposure to fear-based content as impairing judgment and discrimination between factual and exaggerated news.Source: Figure 3 · p. 5, Misinformation, fear, manipulation and… · p. 7
- The section calls for a computational framework that goes beyond analytics to design and implement effective policies for decision support and timely interventions, especially during crises.Source: Misinformation, fear, manipulation and… · p. 7
Conclusions
The article reviews social, economic, and emergency-related harms of social media abuse and recent efforts to detect it, while arguing that further technical and policy work is needed for safer platforms.
- Social media’s reach magnifies misinformation and enables economic and social abuse, including political manipulation, stock-market manipulation, propaganda, recruitment, and fearmongering during emergencies.Improper communication and fear-mongering can spread fear and panic during emergencies.Source: Conclusions · p. 7
- Recent advances address abuse through detection of social bots, campaigns, and spam, but much remains to be done to make platforms safer and healthier for all users.Source: Conclusions · p. 7
- Younger users are especially exposed to cyber-bullying, harassment, stigma, and hate speech, with increased risks of depression and suicide [Boyd 2008; 2014].Source: Conclusions · p. 7
- The community must address methodological and technical challenges in detecting abuse while reducing social and economic incentives through effective policy frameworks.Source: Conclusions · p. 7