Source-linked AI summary
Leveraging the Crowd to Detect and Reduce the Spread of Fake News and Misinformation
Jooyeon Kim, Behzad Tabibian, Alice Oh, Bernhard Schoelkopf, Manuel Gomez-Rodriguez
TL;DR
Misinformation reduction on social platforms must handle uncertain exposures, costly fact checking, and the trade-off between collecting flags and preventing exposure. The paper models this process, develops the online algorithm Curb through stochastic optimal control, and reports effective reduction of misinformation spread on Twitter and Weibo.
Problem
Social platforms need algorithms that decide which stories to fact check and when despite uncertain exposures, costly review, and a trade-off between flags and exposures.
Method
The paper represents crowd flagging and expert review with marked temporal point processes and solves a stochastic optimal control problem for SDEs with jumps to develop Curb.
Results
Experiments on real-world Twitter and Weibo datasets show that Curb may be able to effectively reduce the spread of misinformation and fake news.
Takeaways & Limitations
The work establishes a connection between stochastic optimal control, survival analysis, and Bayesian inference while providing an online approach to misinformation reduction.
Takeaways & Limitations
The paper assumes crowd members are equally good at flagging misinformation and treats stories as independent with common flag-based misinformation probabilities.
Abstract
from arXiv · showhide
Online social networking sites are experimenting with the following crowd-powered procedure to reduce the spread of fake news and misinformation: whenever a user is exposed to a story through her feed, she can flag the story as misinformation and, if the story receives enough flags, it is sent to a trusted third party for fact checking. If this party identifies the story as misinformation, it is marked as disputed. However, given the uncertain number of exposures, the high cost of fact checking, and the trade-off between flags and exposures, the above mentioned procedure requires careful reasoning and smart algorithms which, to the best of our knowledge, do not exist to date. In this paper, we first introduce a flexible representation of the above procedure using the framework of marked temporal point processes. Then, we develop a scalable online algorithm, Curb, to select which stories to send for fact checking and when to do so to efficiently reduce the spread of misinformation with provable guarantees. In doing so, we need to solve a novel stochastic optimal control problem for stochastic differential equations with jumps, which is of independent interest. Experiments on two real-world datasets gathered from Twitter and Weibo show that our algorithm may be able to effectively reduce the spread of fake news and misinformation.
1 Introduction
Online platforms are exploring crowd flags followed by expert fact checking, but uncertain exposures, costly review, and the flags–exposures trade-off require new algorithms. The paper models this process and develops Curb, an online fact-checking scheduler grounded in stochastic control and Bayesian inference.
- Motivation: Social platforms are experimenting with user flags, threshold-triggered expert fact checking, and disputed labels to reduce misinformation exposure.Stories identified as misinformation may also appear lower in users’ feeds.
- Challenges: Uncertain exposure counts require probabilistic models because information spread depends on content, user influence, and network structure.Exposure numbers can vary greatly across stories.
- Challenges: Fact checking is costly, so the procedure must choose which stories to review and when, rather than review all spreading stories.Only a small percentage of stories can be fact checked over time.
- Challenges: The central decision is balancing flag-based misinformation evidence against preventing additional exposures before verification.More exposures can increase confidence but also increase potential damage if the story is misinformation.
- Our approach: The paper represents the process with marked temporal point processes and solves a stochastic optimal control problem for SDEs with jumps.The control signal is a multidimensional survival process defined by conditional intensities for stories sent to fact checking.
- Our approach: The formulation incorporates posterior inference for unknown flagging probabilities, unlike previous work that did not consider application scenarios requiring posterior inference.The paper connects stochastic online optimal control, survival analysis, and Bayesian inference, and introduces Curb as a scalable online algorithm.
2 Problem formulation
The paper formulates crowd-assisted misinformation fact checking as a stochastic process, estimating each story’s misinformation rate from exposures and flags to schedule costly checks.
- 2 Problem formulation: The procedure sends stories with enough user flags to a third party, which verifies them and may reduce subsequent feed exposure if they are misinformation.Fact checking is assumed instantaneous for exposition, although verification delays could be modeled.
- 2.1 Data representation: The model distinguishes exogenous story publications from endogenous exposure-driven resharing and flagging events, recording their histories for each story.Endogenous events include whether exposed users reshare and/or flag a story.
- 2.2 Fact checking generative process: Exogenous and endogenous events are represented by multidimensional counting processes whose dimensions count events for each story and whose intensities describe their occurrence rates.The process uses separate counting processes for publications and exposure-related events.
- 2.2 Fact checking generative process: Exposure-related reshare and flag decisions are modeled with story-specific Bernoulli probabilities, while fact-checking uses a binary process that sends each story at most once.Fact-checking intensity u(t) governs sending decisions, and the process is stopped for a story after verification.
- 2.3 Endogenous and exogenous intensities: Endogenous exposure intensity combines effects from prior reshares and independent posts through an exponentially decaying triggering kernel, and becomes zero after verification.The representation can also be expressed as an SDE with jumps for algorithm design.
- 2.4 Estimated rate of misinformation: The algorithm estimates each story’s misinformation rate from observed exposures and flags rather than directly observing whether the story is fake.This estimate is used to determine which stories to fact check and when.
- 2.4 Estimated rate of misinformation: A Beta prior models unknown story-specific flagging probabilities, while the posterior estimate assumes the misinformation probability given a flag outcome is shared across stories.The shared probability can be estimated from historical exposure and flag data for fact-checked stories.
- 2.5 The fact checking scheduling problem: The scheduling problem chooses fact-checking intensities u(t) to minimize expected convex loss over posterior misinformation-rate estimates and checking intensity across a time window.The objective formalizes the allocation of limited fact-checking decisions.
3 Proposed algorithm
The paper formulates fact-checking scheduling as stochastic optimal control for jump SDEs, first solving the one-story case and then extending it to multiple stories. The resulting Curb algorithm updates and samples fact-checking intensities online using exposure and flagging information, with scalable parallel execution.
- Optimization framework: The fact-checking scheduling problem is solved as stochastic optimal control for jump SDEs, first for one story and then for multiple stories.The approach defines an optimal cost-to-go function, derives an HJB equation through Bellman’s principle, and solves it under quadratic loss and penalty functions.
- Optimizing for one story: The optimal fact-checking intensity has a linear relationship with the endogenous exposure intensity λe(t), with a coefficient determined by the numbers of exposures and flags.The coefficient can change over time as exposure and flag counts evolve.
- Optimizing for one story: The optimal intensity does not depend on the generally unknown exogenous intensity λp(t), while it uses a posterior estimate of the misinformation rate.This makes the control signal depend on observable story history rather than requiring the exogenous intensity.
- Curb algorithm: Curb implements the optimal intensity online by processing exposure events, updating exposure and flag counts, revising the intensity, and sampling fact-checking times.The algorithm uses thinning to sample from the updated intensity and accounts for resharing-driven changes in exposure intensity.
- Optimizing for multiple stories: For multiple stories, the optimal intensity is derived separately for each story under an independence assumption, with parameters that can trade off fact-checking effort against misinformation spread and favor some stories.The fact-checking times can be sampled by running one Curb instance per story, and the instances can run in parallel.
4 Experiments
Experiments on Twitter and Weibo evaluate Curb against an oracle and three baselines using precision and misinformation reduction. Curb performs comparably to the oracle, generally outperforming baselines and preventing misinformation from becoming viral.
- Dataset description and experimental setup: Experiments use Twitter and Weibo datasets containing posts, reshares, and manually annotated genuine and fake stories.After preprocessing, Twitter includes 7 fake and 39 genuine stories; Weibo includes 23 fake and 133 genuine stories.
- Dataset description and experimental setup: Because exposure timing and flags are unavailable, the experiments generate exposures from observed reshares and sample flags.The exposure model produces 313,357 Twitter exposures and 1,054,449 Weibo exposures under the stated parameter setting.
- Evaluation metrics and baselines: Curb is compared with an oracle and Flag Ratio, Flag Sum, and Exposure baselines using precision and misinformation reduction.Flag Sum uses a deterministic policy, while Curb, the oracle, and the other baselines use stochastic intensity functions.
- Solution quality: As fact-checking events increase, average misinformation reduction increases for all methods, while Curb and the oracle generally outperform the baselines.The oracle and Curb have comparable performance in both datasets and beat the threshold baseline by large margins.
- Solution quality: Curb and the oracle prevent misinformation from spreading before it becomes viral, unlike the other baselines.The methods use approximately 15 ± 4 fact-checked stories on Twitter and 64 ± 10 on Weibo in this comparison.
- Flagging behavior and misinformation evidence: Curb becomes more effective as the crowd’s true-positive flagging rate increases, while larger α values reduce the influence of individual observations on posterior estimates.The α analysis concerns posterior estimates of misinformation rate and flagging probability, including the rate used by the optimal intensity.
5 Conclusions
The paper introduces Curb, an efficient online algorithm for using crowd signals to detect and prevent misinformation spread. Experiments on Twitter and Weibo indicate that Curb may effectively reduce misinformation, while several modeling assumptions remain opportunities for future work.
- The paper introduces Curb as an efficient online algorithm that leverages the crowd to detect and prevent misinformation spread.
- Experiments on two real-world datasets gathered from Twitter and Weibo show that Curb may effectively reduce misinformation spread.
- Future work could model individual crowd trustworthiness, adversarial flagging, dependencies among stories, source-specific flagging probabilities, and alternative loss functions.The current formulation assumes equally capable crowd members, independent stories, common conditional flagging probabilities, and a quadratic loss.
Appendix
The appendix states a jump-diffusion model and an associated differential expansion for functions of its jump processes and state. The expansion uses Ito calculus and the zero-one jump law.
- The appendix defines a jump-diffusion state process with drift, exposure, propagation, and misinformation-related jump terms.Its differential combines f(x(t),t)dt with jump terms involving g, h, k, and the processes N^e, N^p, and M.
- The jump processes M(t), N^e(t), and N^p(t) are independent, while N^f(t) increments only when an exposure jump is marked by w(t).The mark variables w(t) and z(t) take values in {0,1}.
- The notation suppresses the arguments of the functions f, g, h, and k for simplicity.
- The proof sketch derives the differential of J using Ito calculus and zero-one jump-law relations among the counting processes.