PaperlayerView exact revision on arXiv

Source-linked AI summary

The Rise of Guardians: Fact-checking URL Recommendation to Combat Fake News

Nguyen Vo, Kyumin Lee

arXiv:1806.07516v2cs.IRcs.SI

TL;DR

Fake news remains widely shared despite fact-checking systems, raising the problem of how to better utilize verified information. The paper analyzes guardians who correct misinformation and recommends fact-checking URLs using content and network information; the model outperformed four state-of-the-art methods by 11%∼33%.

  • Problem

    Fake news continues to spread despite existing detection and fact-checking systems, which are not fully utilized by online users.

  • Method

    The paper analyzes guardians who embed fact-checking URLs in online discussions and proposes a personalized recommendation model using URL content, social structure, and recent-tweet content.

  • Results

    11%∼33%: the proposed model outperformed four state-of-the-art recommendation algorithms.

  • Takeaways & Limitations

    Fact-checking URL recommendations provide a proactive way to engage guardians in spreading credible information.

  • Takeaways & Limitations

    The evaluation excludes a followee-based baseline because the authors could not collect all followees’ fact-checking tweets during the same period, limiting that comparison.

Abstract

from arXiv · show

A large body of research work and efforts have been focused on detecting fake news and building online fact-check systems in order to debunk fake news as soon as possible. Despite the existence of these systems, fake news is still wildly shared by online users. It indicates that these systems may not be fully utilized. After detecting fake news, what is the next step to stop people from sharing it? How can we improve the utilization of these fact-check systems? To fill this gap, in this paper, we (i) collect and analyze online users called guardians, who correct misinformation and fake news in online discussions by referring fact-checking URLs; and (ii) propose a novel fact-checking URL recommendation model to encourage the guardians to engage more in fact-checking activities. We found that the guardians usually took less than one day to reply to claims in online conversations and took another day to spread verified information to hundreds of millions of followers. Our proposed recommendation model outperformed four state-of-the-art models by 11%~33%. Our source code and dataset are available at https://github.com/nguyenvo09/CombatingFakeNews.

1 INTRODUCTION

Existing fact-checking systems have not prevented widespread misinformation sharing, motivating a framework that identifies guardians and recommends fact-checking URLs to encourage credible-information dissemination.

  • Research gap: Fake news remains widely disseminated despite expert checking, automated credibility systems, and detection models.A reported 86% of American adults do not fact-check articles they read.
  • Research gap: Verified information should reach users quickly and at scale because prior exposure to misinformation makes belief change more difficult.The paper argues that a large volume of verified social-media content can increase users’ chances of seeing legitimate information first.
  • Framework: The framework addresses identifying people willing to spread verified information and stimulating them to disseminate it.These are formulated as problems P1 and P2.
  • Framework: Guardians are online users who correct false claims in discussions by embedding fact-checking URLs, while personalized recommendations target their engagement.The paper contrasts this approach with bots and paid workers, which raise platform-policy or scalability concerns.
  • Contributions: The proposed model combines fact-checking-page content, social-network structure, and recent-tweet content to personalize URL recommendations.The paper presents this as a proactive way to help guardians spread credible information.
  • Contributions: 11%∼33%: the proposed model outperformed four state-of-the-art recommendation algorithms.The comparison evaluates the model against four competing recommendation methods.

2 RELATED WORK

Related work spans fake-news detection, rumor and misinformation analysis, user stances and spreaders, and URL recommendation using content, profiles, and auxiliary data.

  • Fake-news research: Prior fake-news research classifies content, applies deep learning, and studies misinformation in emergencies and natural disasters.This literature focuses primarily on identifying whether information is true or false.
  • User-centered analysis: Researchers also analyze users’ stances toward rumors and how those stances change over time.Related studies examine supporting or denying positions in addition to rumor content.
  • User-centered analysis: Other related work investigates content polluters, social bots, malicious campaigns, and people involved in spreading rumors.The paper identifies guardian analysis as especially closely related to these user-focused directions.
  • URL recommendation: URL recommendation methods use content, hashtags, topics, entities, Twitter lists, streaming data, external profiles, and SVM-based models.These approaches target fresh or personalized URL recommendation on social networks.

3 DATA COLLECTION

The authors collected fact-checking tweets from six websites through Hoaxy, filtered them into direct and sharing guardians, and constructed a dataset for recommendation experiments.

  • Dataset construction: The study analyzed guardians’ behavior and selected active users for recommendation-model training and evaluation.The broader collection included tens of thousands of D- and S-guardians before the smaller modeling subset was formed.
  • Collection: Hoaxy supplied 231,377 unique fact-checking tweets from six fact-checking websites collected over approximately one year and two months.The collection contained 161,981 direct tweets and 69,396 sharing tweets.
  • Filtering: Filtering removed tweets containing only base URLs or background pages because they might not express fact-checking enthusiasm or information.The resulting collection contained 225,068 fact-checking tweets and 7,295 unique URLs.
  • Dataset construction: The dataset distinguished D-guardians, who post direct fact-checking tweets, from S-guardians, who share fact-checking content.Some users played both roles.

4 CHARACTERISTICS OF GUARDIANS

Guardians mainly specialized in either correcting claims or sharing verified information, while both groups responded quickly and showed sustained, topic-focused activity.

  • 7,167 users (7%) performed both D-guardian and S-guardian roles, indicating that most guardians specialized in correcting claims or sharing credible information.
  • D-guardians were significantly more enthusiastic about their role than S-guardians (p-value<10−6).
  • 85.3% of guardians posted only 1∼2 fact-checking tweets, while some highly active guardians posted over 200.
  • 90% of D-tweets were posted within one day of an original claim, with mean and median response times of 2.26 days and 34 minutes.
  • 88.5% of S-tweets were posted within one day of the corresponding D-tweet, with mean and median response times of 3.1 days and 90 minutes.
  • S-guardians’ average inter-posting time was 5 minutes.
  • Fact-checking activity peaked at 25,000 tweets in November 2016, while D-tweets outnumbered S-tweets each month.
  • The most popular fact-checking websites were Snopes.com and politifact.com, used in 48.55% and 34.23% of referenced tweets, respectively.

5 FACT-CHECKING URL RECOMMENDATION

The paper proposes personalized fact-checking URL recommendations to help active guardians find relevant pages and spread verified information.

  • The recommendation model personalizes fact-checking URLs to help guardians access interesting pages, correct unverified claims, and disseminate verified information.

5.1 Problem Statement

The problem statement represents guardian–URL interactions in a binary matrix and learns low-dimensional latent representations for guardians and URLs.

  • X_ij = 1 when guardian u_i posts fact-checking URL ℓ_j, and X_ij = 0 otherwise.
  • The model learns guardian representations U and URL representations V in a shared latent dimension D.
  • The basic matrix-factorization objective uses observed interactions and Frobenius-norm regularization, with λ helping avoid overfitting.

5.2 Co-ocurrence model

The co-occurrence model enriches matrix factorization with URL–URL and guardian–guardian relationships derived from shared posting behavior and SPPMI-based embeddings.

  • The extension adds URL–URL and guardian–guardian co-occurrence matrices to regularize the interaction model.
  • SPPMI(i, j) is max{PMI(i, j) − log(s), 0}, where s is the number of negative samples.
  • For URLs, two URLs co-occur when the same guardian posts both, and their SPPMI values are computed from guardian-level co-occurrence counts.
  • For guardians, co-occurrence counts measure how many URLs two guardians commonly posted, producing a guardian SPPMI matrix G.
  • The joint factorization models URL similarity through shared guardians and guardian similarity through shared URLs.
  • The model omits bias matrices, factorizes G, and leaves K and L unregularized to reduce complexity and distinguish the formulation.

5.3 Integrating Auxiliary Information

The model enriches recommendation with auxiliary signals capturing guardian social structure, tweet-based interests, and fact-checking URL content. These signals are represented through graph and similarity-based regularization.

  • Social structure: 1,033,704 edges form the guardian graph, whose density is 0.013898 and 5.9 times higher than the density reported in.Edges connect guardians when either follows the other; the adjacency matrix S records these connections.
  • Social structure: Guardian social structure regularization encourages connected guardians to have similar latent representations.The model uses the adjacency matrix and a regularization term based on connected guardians.
  • Topical interests: Each guardian’s 200 recent tweets are aggregated into a document and encoded with Doc2Vec to represent topical interests.The resulting document representations are used to calculate pairwise cosine similarities between guardians.
  • URL content: Fact-checking URL content is added because semantically similar pages should have nearby latent representations.The model applies graph-based regularization to the similarity structure of URL contents.

5.4 Joint-learning fact-checking URL recommendation model

GAU jointly learns guardian and URL representations while incorporating auxiliary information and social structure into fact-checking URL recommendation. After optimization, guardian–URL preference scores rank candidate URLs for each guardian.

  • Model: GAU jointly models the Guardian-Guardian SPPMI matrix, auxiliary information, and URL-URL SPPMI matrix.Its objective function is denoted LGAU.
  • Optimization: The model is optimized by gradient descent with a fixed learning rate η = 0.001 after hyperparameters are tuned on a validation set.The learned representations are then used to estimate guardian preferences for URLs.
  • Recommendation: Guardian–URL preference is estimated as ˆri,j ≈ UiVj, and URLs are recommended by ranking these estimated preferences.The ranking produces an ordered list of URLs for each guardian.
  • Optimization: Algorithm 1 initializes U, V, K, and L with Gaussian values and iteratively updates them until convergence.The inputs include guardian–URL interactions, SPPMI matrices, social structure, Laplacians, and indication matrices.

6 EVALUATION

The evaluation tests GAU against variants and four collaborative-filtering baselines, examining auxiliary information, co-occurrence matrices, guardian activity levels, and hyperparameter sensitivity. GAU consistently achieves the strongest recommendation performance.

  • RQ1: Auxiliary information: 12.20%±1.31% Recall, 13.39%±0.34% NDCG, and 14.04%±0.76% MAP improved for BASIC+NW+UC over BASIC.The variant adds social network information and fact-checking URL content.
  • RQ1 and RQ2: Combined information: 7.90%±1.79% Recall, 6.58%±0.40% NDCG, and 5.53%±0.22% MAP improved over CSU+CSG when all auxiliary data and co-occurrence matrices were combined.The combined variant also exceeded BASIC+NW+UC by 9.10%±6.15% Recall, 7.92%±2.50% NDCG, and 7.75%±0.58% MAP.
  • RQ1: Auxiliary information: 25.13%±10.64% Recall, 28.64%±7.13% NDCG, and 32%±4.29% MAP improved over BASIC with GAU.Adding recent tweets’ content to the full auxiliary-information and co-occurrence variant produced statistically significant gains with p-value<0.001.
  • RQ3: Baseline comparison: GAU significantly outperformed all four baselines, with improvements of 11%∼33%.The comparison used MF, BPRMF, CoFactor, and CTR; the overall difference was statistically significant at p-value<0.001.
  • RQ4: Guardian activity levels: 6.5%∼10.0% Recall@15, 10.2%∼15.0% NDCG@15, and 12.8%∼20.1% MAP@15 improved over CTR across guardian activity levels.GAU consistently outperformed baselines for cold-start, warm-start, and highly active guardians.
  • RQ5: Hyperparameter sensitivity: Recall@15 generally increased as α and γ increased, peaking at α = 0.06 and γ = 0.06 when β = 0.08.When varying β and γ with α fixed, the largest Recall@15 was only 2.2% above the smallest; larger α, β, and γ generally improved performance.

7 DISCUSSION

The discussion limits recommendations to highly enthusiastic guardians and identifies additional data sources and activity information as directions for improving the recommender.

  • Guardian selection: The study recommends URLs only to guardians who posted at least 3 fact-checking URLs.These highly enthusiastic guardians may continue spreading fact-checked information in the future.
  • Future work: Temporal factors and guardian activeness remain potential data sources for future recommender improvements.The paper leaves integrating these sources for future work.

8 CONCLUSION

The paper constructs a guardian dataset and proposes personalized fact-checking URL recommendations to encourage credible-information sharing. GAU outperforms four baselines, while cold-start guardians and deeper models remain future-work concerns.

  • Dataset and guardians: The study collects guardians who embed fact-checking URLs to signal interest in information credibility.The dataset analysis reports interests mainly in politics, fauxotography, and fake news.
  • Recommendation model: The recommendation model personalizes fact-checking URLs to enhance guardians’ engagement and encourage credible-information posts.The model is designed to support the spread of fact-checked information.
  • Evaluation: GAU outperformed MF, CoFactor, BPRMF, and CTR.The conclusion identifies these four systems as the comparison baselines.
  • Future work: Future work will address guardians with fewer than 3 fact-checking URLs and investigate deep learning techniques.The paper identifies the former as a cold-start issue.
Loading 1806.07516v2…