PaperlayerView exact revision on arXiv

Source-linked AI summary

Community corrections have divergent downstream effects across corrected accounts

Yuwei Chuai, Thomas Renault, Nicolas Pröllochs, Gabriele Lenzini, Mohsen Mosleh

arXiv:2608.27526v1cs.SI

TL;DR

Whether community fact-checking changes the subsequent behavior of users whose content is corrected remains unclear. Using Difference-in-Differences to study Community Notes, the paper finds corrected accounts publish 2.9% more original posts over four weeks, with divergent trajectories across accounts.

  • Problem

    The effects of community fact-checking on the subsequent behavior of users who produce corrected content remain unclear.

    Source: Discussion · p. 18

  • Method

    Using Difference-in-Differences, the study examines corrected accounts’ subsequent posting activity and content, shifting the unit of analysis from corrected posts to their authors.

    Source: Introduction · pp. 4–5, Discussion · p. 18

  • Results

    Corrected accounts publish, on average, 2.9% more original posts during the subsequent four weeks, with the aggregate concealing divergent trajectories.

    Source: Discussion · p. 18

  • Takeaways & Limitations

    Success in correcting individual misleading posts does not imply success in changing their authors’ subsequent behavior.

    Source: Discussion · pp. 18, 21

    The quasi-experimental design cannot exclude unobserved time-varying confounding.

    Source: Discussion · p. 21

Abstract

from arXiv · show

Community-based fact-checking can reduce the spread of annotated misleading posts, but whether it produces lasting behavioral change among corrected authors remains unclear. Here, we conduct a large-scale quasi-experimental study of the Community Notes system on X (formerly Twitter), tracking four weeks of activity before and after note display for 19,854 accounts and 57,935 corrections (noted posts and matched controls), covering 11,909,591 original posts. Difference-in-Differences estimates show that note display is followed by an average 2.9% increase in corrected accounts' original-post activity. This aggregate conceals two divergent trajectories. Accounts corrected only once reduce their activity by 2.4% and subsequently publish less toxic and less misleading content. Repeatedly corrected accounts, which constitute 28.6% of corrected accounts but produce 73.4% of fact-checked posts, instead increase their activity by 4.4% after their first correction and show no detectable response to later ones. They exhibit no comparable content improvement, and instead publish more highly misleading posts, cite lower-quality domains, and post more political content. Community notes can thus constrain individual misleading posts without durably improving the behavior of the accounts most responsible for them, indicating that correcting content and changing its producers are distinct objectives for platform design.

Introduction

The introduction asks whether Community Notes change corrected authors’ subsequent activity and content, particularly for repeatedly corrected accounts. It reports divergent responses between once- and repeatedly corrected accounts.

Introduction

  • Prior research finds that visible community notes reduce engagement with annotated posts and alter their diffusion [4], but effects on authors’ later behavior remain largely unknown.The introduction distinguishes effects on corrected posts from whether authors subsequently post less or produce more reliable content.Source: Introduction · p. 3
  • Because misinformation is concentrated among a small minority of accounts, evaluation should distinguish once-corrected from repeatedly corrected accounts and measure both activity and content.Repeatedly corrected accounts may be especially consequential because they may continue or increase posting without improving content quality.Source: Introduction · p. 4
  • Notes may encourage accuracy through informational and reputational signals, but public corrections may provoke defensiveness; authors’ responses to repeated corrections remain unclear.The introduction frames these as competing possibilities for downstream author behavior.Source: Introduction · pp. 4–5
  • The Difference-in-Differences study tracks 19,854 corrected accounts for four weeks before and after corrections, covering 11,909,591 original posts.It compares accounts corrected once with those corrected repeatedly and examines both posting activity and content.Source: Introduction · p. 5

Results

The Results section examines how Community Notes corrections relate to corrected accounts’ posting activity and content over time, including how these patterns vary by correction frequency.

Overall changes in posting activity

  • Following community-note display, corrected accounts’ original-post activity increased by 2.9% relative to matched controls during post-display weeks (effect = 0.029, z = 7.351, p < 0.001; 95% CI [0.021,0.036]).Estimates for Weeks −4 to −2 were statistically indistinguishable from zero.Source: Overall changes in posting activity · p. 8, Figure 1 · p. 9

Changes in posting activity by correction frequency

  • 28.6% of corrected accounts received multiple notes and accounted for 73.4% of all noted posts.Source: Changes in posting activity by… · p. 10, Figure 2 · p. 12
  • −2.4%: Accounts receiving only one note reduced activity, while accounts receiving at least two notes increased it by 2.3%.Across cumulative thresholds, activity increases were 2.6% for accounts receiving at least three notes, 3% for at least four, and 3.4% for at least five.Source: Changes in posting activity by… · p. 10
  • 4.4%: Among accounts later observed to receive multiple notes, activity rose after the first correction, while changes after later corrections were not statistically detectable.Estimates from the second correction onward were statistically indistinguishable from zero.Source: Changes in posting activity by… · p. 11, Figure 2 · p. 12

Characteristics of repeatedly corrected accounts

  • The authors compare groups’ pre-correction account and posting characteristics, including audience size, political leaning, posting frequency, toxicity, URL quality, and misleadingness.The comparison also covers verified status, account age, misinformation exposure, sentiment, opinion confidence, and political content.Source: Characteristics of repeatedly… · p. 13
  • Compared with accounts corrected only once, repeatedly corrected accounts had more followers (coef. 0.835), were more likely to be verified (coef. 0.290), and had greater misinformation exposure (coef. 0.131).Source: Characteristics of repeatedly… · p. 13
  • Before correction, repeatedly corrected accounts posted more often and more misleading and political content, but their posts had lower toxicity and positive sentiment.They also expressed greater opinion confidence; the study found no significant difference in URL quality.Source: Characteristics of repeatedly… · p. 13
  • The increase in posting after a first correction was most detectable among relatively low-frequency accounts and accounts with higher-risk or politically oriented prior posting patterns.Basic account-profile characteristics did not clearly distinguish responses; higher-risk patterns included prior toxic, misleading, or political content.Source: Characteristics of repeatedly… · p. 14

Changes in post content

  • Repeatedly corrected accounts later cited lower-quality domains and posted a higher proportion of political content.The estimated effects were −0.047 for URL quality and 0.014 for political content; neither group showed detectable changes in positive sentiment, negative sentiment, or opinion confidence.Source: Changes in post content · p. 15
  • Accounts corrected once later post less toxic and misleading content, while repeatedly corrected accounts show no detectable reductions in either measure.For once-corrected accounts, toxicity fell by 0.034 and misleadingness by 0.064; repeatedly corrected accounts instead cited lower-quality domains and posted a higher proportion of political content.Source: Changes in post content · p. 15, Figure 4 · p. 16
  • Repeatedly corrected accounts produced more high-misleadingness posts after correction, while once-corrected accounts produced fewer high-toxicity and low-URL-quality posts.The estimated effects were 0.019 for high-misleadingness posts among repeatedly corrected accounts, and −0.031 for high-toxicity and −0.077 for low-URL-quality posts among once-corrected accounts.Source: Changes in post content · p. 15, Figure 5 · p. 17

Discussion

The discussion examines how Community Notes relate to corrected accounts’ posting activity and content, distinguishing accounts corrected once from those corrected repeatedly. It emphasizes that correcting misleading posts does not necessarily change their producers’ behavior.

Discussion

  • Accounts corrected only once reduce posting and subsequently publish less toxic and less misleading content.The pattern is consistent with corrective learning or reputational deterrence, though the discussion presents these as interpretations.Source: Discussion · pp. 18–19
  • 2.9% more original posts followed note display on average per correction event, an estimate shaped disproportionately by repeatedly corrected accounts.The event-level estimate is not the average effect on a unique account corrected once.Source: Discussion · p. 18
  • 4.4% more posting followed the first correction among repeatedly corrected accounts, while later corrections had no statistically detectable effect; this group made up 28.6% of corrected accounts but contributed 73.4% of fact-checked posts.The positive response was already apparent after the first observed correction, rather than emerging only after multiple corrections.Source: Discussion · pp. 18–19
  • Among repeatedly corrected accounts, first-correction increases are most detectable for those with low baseline activity and more toxic, misleading, or political prior content, and are more statistically significant for right-leaning than left-leaning accounts.These are subgroup patterns in this sample and platform context, not evidence that political leaning or other characteristics cause resistance to correction.Source: Discussion · p. 19
  • Repeatedly corrected accounts show no detectable reduction in toxicity or misleadingness, cite lower-quality domains, and post more political content and a greater volume of highly misleading posts.Their average misleadingness does not significantly increase; greater posting activity raises the volume of highly misleading posts.Source: Discussion · p. 20
  • Platforms should complement post-level diffusion measures with author-level outcomes and test whether first-correction interventions improve later behavior, while evaluating content correction and behavioral change as distinct objectives.Repeatedly displaying the same correction may be insufficient for frequently corrected accounts; differentiated interventions would need transparent appeals and safeguards against erroneous or coordinated corrections.Source: Discussion · pp. 20–21
  • The quasi-experimental design cannot exclude unobserved time-varying confounding, correction-frequency groups are defined retrospectively, and responses may differ across platforms and system designs.The study focuses on X; note selection, ranking, presentation, and associated enforcement may shape author responses elsewhere.Source: Discussion · p. 21
  • Cross-platform research is needed to assess generalizability, and future studies could examine behavioral effects of changes to contributor eligibility, ranking algorithms, or display practices.The Community Notes system continues to evolve, potentially altering who receives corrections and how authors respond.Source: Discussion · p. 21
  • Because authors’ noticing, acceptance, or defensive reactions to notes cannot be directly observed, surveys or field experiments could test learning, reputational pressure, reactance, and habituation.Source: Discussion · p. 21
  • Community Notes can limit individual misinformation cascades without uniformly changing the behavior of the accounts that generate them.The authors argue that distinguishing post correction from producer behavior is essential to designing systems that also promote durable improvements in the information ecosystem.Source: Discussion · pp. 21–22
  • Single-correction accounts subsequently post less and improve aspects of their content, while repeatedly corrected accounts increase activity after their first correction, respond little to later notes, and produce more highly misleading content.The authors suggest corrective learning or deterrence may explain the single-correction pattern.Source: Discussion · pp. 21–22

Methods

The methods link corrected misleading posts to matched controls and track authors’ original-post activity and content around note display. Fixed-effects Difference-in-Differences models estimate post-display changes, while separate univariate regressions characterize differences between once- and repeatedly corrected accounts.

Data and study design

  • Displayed-note fact-checked posts form the treatment group, matched posts form controls with assigned display times, and overlapping account windows are excluded while correction events remain the analysis unit.The design allows accounts to receive multiple corrections over time.Source: Data and study design · pp. 23–24
  • The study reports 3,244,360 daily and 463,480 weekly count observations, and successfully retrieves 11,909,591 original posts for analyses; the observation window runs 28 days before and after note display.Source: Data and study design · p. 23
  • The authors characterize accounts’ posting preferences from posts created before community corrections, using positive and negative sentiment, Detoxify toxicity, external-link URL quality, and a misleadingness framework.Source: Data and study design · p. 24

Difference-in-differences analyses

  • The Difference-in-Differences analysis combines daily and weekly leads-and-lags models with aggregated pre-post models; pre-treatment interactions assess parallel trends.Negative-binomial count models use post-level random effects to retain zero-count posts, and fixed-effects models provide robustness checks.Source: Difference-in-differences analyses · pp. 25–26
  • Fixed-effects linear DiD models estimate notes’ post-display ATT via β3 across sentiment, toxicity, URL quality, confidence, political-post ratio, and political leaning.Source: Difference-in-differences analyses · p. 27

Comparison of once- and repeatedly corrected accounts

Data and code availability

Upon publication, pseudonymized data, materials, and analysis scripts required to reproduce the study will be made publicly available.

  • Upon publication, pseudonymized data, materials, and analysis scripts required to reproduce the study will be made publicly available.Source: Data and code availability · p. 28

S1 Data overview

The dataset covers 57,935 fact-checked posts from 19,854 accounts, with most corrected accounts receiving one note while repeatedly corrected accounts produced most fact-checked posts.

S1 Data overview

  • 57,935 fact-checked posts came from 19,854 accounts: 29,049 posts from 10,799 treatment accounts and 28,886 from 12,823 control accounts.Source: S1 Data overview · p. 36
  • 21,336 fact-checked posts (73.4%) came from accounts receiving at least two notes, although most corrected accounts received only one.The study describes this as a disproportionate share of the misinformation flagged on X.Source: S1 Data overview · p. 36

S2 Robustness checks for changes in post count

Robustness checks assess changes in corrected accounts’ original-post and reply activity after Community Notes display, including under alternative specifications and trend analyses. Estimates generally support increased original-post activity, while reply activity shows no statistically detectable change in weekly estimates or HonestDiD results.

S2.1 Daily aggregated effect over four weeks

  • The aggregated daily effect was 0.027 and the aggregated weekly effect was 0.029; when correction windows did not overlap, the weekly estimate remained similar at 0.027.For the nonoverlapping-window estimate, the analysis included 244,912 observations across 30,614 events.Source: S2.1 Daily aggregated effect over four… · p. 38

S2.2 Post-level fixed effects

  • With post-level fixed effects, corrected aggregated effects were 0.022 over four days (z = 5.302, p < 0.001; 95% CI [0.014, 0.031]) and 0.026 over four weeks (z = 6.978, p < 0.001; 95% CI [0.019, 0.033]), consistent with main-analysis random-effects estimates.Source: S2.2 Post-level fixed effects · p. 41

S2.3 HonestDiD estimation

  • HonestDiD estimates retained a positive effect when allowing post-period deviations up to 1× the largest pre-trend violation for daily estimates and 1.5× for weekly estimates.The analysis used the Conditional Least Favorable Hybrid method to assess sensitivity to parallel-trend violations.Source: S2.3 HonestDiD estimation · p. 43, Figure S4 · p. 43

S2.4 DiD estimation with non-binary treatment

S2.5 Estimation for reply count

S3 Heterogeneity by correction frequency

The section identifies group sizes for the estimations in Figure 2.

S3 Heterogeneity by correction frequency

S4 Heterogeneity by account profiles and posting preferences

Among accounts observed to receive at least two displayed Community Notes, the first correction is followed by increased posting activity across many subgroups, but the response varies with baseline posting frequency and content characteristics. The analysis concerns which repeatedly corrected accounts change their activity after a first correction, not which accounts are more likely to be corrected repeatedly.

S4 Heterogeneity by account profiles and posting preferences

  • Posting activity increased significantly across subgroups defined by followers and followees, verification, account age, political leaning, and misinformation exposure.Although effect sizes varied, estimates were consistently positive and statistically significant, so the response was not confined to a particular profile level.Source: S4 Heterogeneity by account profiles… · p. 49
  • Low-frequency accounts increased posting 12.3% after their first correction, while the 1.2% estimate for high-frequency accounts was statistically indistinguishable from zero.The respective estimates were 0.123 (p < 0.001) and 0.012 (p = 0.128).Source: S4 Heterogeneity by account profiles… · p. 49, Figure S7 · p. 51
  • Among accounts with highly toxic prior posts, activity increased 5.4%, whereas the 2.2% estimate for accounts with low prior toxicity was not statistically significant.The respective estimates were 0.054 (p < 0.001) and 0.022 (p = 0.069).Source: S4 Heterogeneity by account profiles… · p. 49
  • Activity increased 5.8% for accounts with high pre-correction misleadingness, while the 0.8% estimate for low-misleadingness accounts was indistinguishable from zero.The respective estimates were 0.058 (p < 0.001) and 0.008 (p = 0.532).Source: S4 Heterogeneity by account profiles… · p. 50
  • Activity increased significantly among high-political-content accounts but not less-politically oriented accounts, and estimates were positive across subgroups of positive sentiment, negative sentiment, and opinion confidence.Political-content estimates were 0.058 (p < 0.001) and 0.008 (p = 0.561), respectively; sentiment and opinion-confidence features provided little evidence of meaningfully distinct responses.Source: S4 Heterogeneity by account profiles… · p. 50
  • Among high-URL-quality accounts, activity increased 5.1%, whereas the 1.0% estimate for low-URL-quality accounts was not statistically significant.The respective estimates were 0.051 (p < 0.001) and 0.010 (p = 0.489); URL quality measures the credibility of cited domains, distinct from a post’s accuracy or contextual integrity.Source: S4 Heterogeneity by account profiles… · p. 50

S5 Parallel-trends assessment for post content outcomes

Pre-display estimates for post-content outcomes are statistically indistinguishable from zero, supporting the parallel-trends assumption. Figure S8 reports estimates across content features, with Week −1 as the reference period except for daily misleadingness, which uses Day −1.

S5 Parallel-trends assessment for post content outcomes

Loading 2608.27526v1…