Human and technological infrastructures of fact-checking

1 INTRODUCTION

• Automated systems often treat fact-checking as a technical problem, overlook stakeholder needs, and have limited applicability to real-world scenarios.Many focus on narrow tasks such as matching public statistics to official figures or identifying simple declarative claims.Source: §1 · pp. 1–2
• Interviews with 26 participants from 16 fact-checking teams and organizations across four continents informed the study.The researchers sampled broadly to uncover possible infrastructures, not to capture regional variability.Source: §1 · p. 2
• The study identifies six stakeholder groups and shows fact-checking spans short-term claim debunking and long-term research, policy, and advocacy work.These groups collaborate through tools and other infrastructures; fact-checkers also value explainable algorithms and human involvement in decisions.Source: §1 · p. 3
• The study identifies technical, policy, and informational challenges, including limited ways to monitor private messaging and investigate videos or local-language claims.In some countries, government and civic information needed for investigations is unavailable, difficult to obtain, or not updated periodically.Source: §1 · p. 3
• The authors argue that improving fact-checking requires both better tools and platforms and improved availability and quality of data and statistics globally.They present the work as a holistic account of collaboration among stakeholder groups within technological and social-informational contexts.Source: §1 · p. 3

2 RELATED WORK

• The paper is organized around study methods, two types of fact-checking, supporting infrastructures, stakeholder challenges, and implications.The authors use the final sections to address the infrastructure and barrier research questions.Source: §2 · p. 4, §2.4 · p. 6

2.1 Fact-checking: Definition, Origin, and Evolution

• Fact-checking developed from internal verification in journalism to post-publication analysis of public claims, but stakeholder roles and collaborations remain insufficiently understood.The section also notes the American Press Institute definition and the emergence of fact-checking as both part of news reporting and a separate activity.Source: §2.1 · p. 4

2.2 Human and Technological Infrastructures

• The paper treats human infrastructure as the actors and partnerships that must align, and technological infrastructure as tools and processes supporting their work.It uses these concepts to examine collaboration and coordination in fact-checking.Source: §2.2 · pp. 4–5

2.3 Invisible Work of Fact-checking

• Prior research largely frames fact-checking as debunking claims; this study also examines less-visible investigative, research, advocacy, and policy work.The paper connects this work to efforts to improve information availability and quality in a country.Source: §2.3 · p. 5

2.4 Current Landscape of Computational Research in Fact-checking

• Computational fact-checking research has developed diverse tools and methods, but their usefulness in practice and the limitations of current tools remain poorly understood.The study addresses this gap by interviewing stakeholders about tools they use and the challenges they face.Source: §2.4 · pp. 5–6

3 METHOD

• The researchers interviewed 26 people from six stakeholder groups, expanding recruitment after early interviews revealed workflows involving more roles than editors and fact-checkers.The study received Institutional Review Board approval.Source: §3 · p. 6

3.1 Participant Sampling Technique

• Convenience and snowball sampling yielded 26 participants from 16 fact-checking teams or organizations, with participants representing four continents.Roles were identified through qualitative analysis and could overlap or differ from participants’ formal job titles.Source: §3.1 · p. 6, Table 2 · p. 7

3.2 Interview Protocol and Data Analysis

• Interviews asked about roles, workflows, tools, collaboration, and challenges; two authors independently coded transcripts and notes using deductive and inductive analysis.Interviews lasted 60 to 125 minutes and averaged over 90 minutes.Source: §3.2 · pp. 7–8

4 TYPES OF FACT-CHECKING: SHORT-TERM CLAIMS AND LONG-TERM ADVOCACY

• Fact-checking includes short-term claims-centric work and long-term advocacy-centric work, with stakeholder roles and supporting tools represented in the ecosystem overview.The overview’s numbers indicate the sequence in which roles are performed.Source: §4 · p. 8, Figure 1 · p. 9

4.1 Short-term Claims Centric Fact-checking

• Short-term fact-checking moves from monitoring online content and pitching claims through investigation, labeling, reporting, editing, correction outreach, and social media publication.Investigations draw on online tools, experts, and authoritative public evidence; in-house fact-checkers also verify facts and quotes in newsroom articles.Source: §4.1 · p. 8

4.2 Long-term Advocacy Centric Fact-checking

• Long-term advocacy-centric work includes investigations of a country’s misinformation ecosystem, efforts to influence civic bodies and policymakers, training, and coalition building.One participant described the work as going beyond publishing fact-checks to influencing information producers, media, and public debate.Source: §4.2 · p. 10

5.1 News Desk Editors—Approving Claims and Guiding Fact-checkers

• Editors prioritize claims by reach, potential harm, social traction, and verifiability, then guide fact-checkers in gathering evidence and connecting with experts.Low-attention claims may be rejected to avoid amplifying them; claims likely to cause greater harm receive priority.Source: §5.1.1 · pp. 10–11, §5.1.2 · p. 11

5.2.1 Performing copy editing cycles

• Copy editors review fact-checks through two to three iterative stages, checking the premise first, then refining language and flow, and finally improving readability and engagement.Editors assess whether the claim and debunk hold, and return unresolved questions for further investigation.Source: §5.2.1 · p. 11, §5.2 · p. 11

5.2.2 Making fact-checks more engaging

• Copy editors and social media managers make fact-checks more engaging with concise, accessible writing, prominent verdicts, and visuals for complex or long-form explanations.They also use popular search terms to improve discoverability through search engines.Source: §5.2.2 · pp. 11–12

5.2.3 Use of collaborative systems, labelling and color coding schemes

• Google Docs and Trello support feedback and workflow tracking, while labels and color coding mark priorities, stages, and task assignments.Trello boards can track fact-checks from copy editing through publication and amplification.Source: §5.2.3 · p. 12

5.3.1 Monitoring online spaces

• External fact-checkers monitor online spaces reactively, in real time, and ahead of events, using tip lines, watch lists, manual searches, platform tools, and regional stringers.Search syntax and third-party tools help filter large volumes of content and find user-generated posts.Source: §5.3.1 · pp. 12–13

5.3.2 Making a decision to fact-check and extracting claim(s)

• After identifying claims, fact-checkers pitch them to editors, choose short- or long-form coverage, archive the content, and locate its original source and context.Archiving preserves claims that may later be deleted; Facebook partnership checks focus on one claim in short-form fact-checks.Source: §5.3.2 · p. 13

5.3.3 Researching

• Fact-checkers investigate claims using expert input, authoritative sources, verification tools, and comments that can reveal clues such as a link to the complete video.Image and video tools and reverse image searches help assess authenticity and trace digital material.Source: §5.3.3 · p. 13

5.3.4 Assigning veracity label and publishing fact-check

• Fact-checkers assign verdicts using varied labeling conventions, seek corrections when claims are false, and publish sources and evidence so readers can follow the investigation.Organizations use different scales, and there is no commonly accepted standard for labeling misinformation.Source: §5.3.4 · p. 14

5.4 In-house Fact-checkers—Gathering Sources and Verifying Claims

• In-house fact-checkers verify stories before publication, proposing changes to fix or remove inaccuracies without publicly calling attention to them.The role also helps publication houses ensure accuracy and reduce potential liability.Source: §5.4 · p. 14

5.4.1 Identifying and verifying claims

• In-house fact-checkers verify each line, modifying or removing vague claims without adequate sources and checking the context around opinions.They may scan top-down for central claims, prioritize time-consuming investigations under deadlines, or process claims in batches to give journalists time to respond.Source: §5.4.1 · pp. 14–15

5.4.2 Gathering sources for verification

• They rely on journalists’ primary sources or seek primary documents, reputable news, peer-reviewed research, and experts; unreliable sourcing may require three to four sources.Some use Nexis, a paid search service, to find news and documents.Source: §5.4.2 · p. 15

5.4.3 Organizing corrections

• In-house fact-checkers communicate questions and suggested changes using varied color-coded systems, such as marked claims in shared documents or claim-status spreadsheets.There is no standard convention for organizing corrections.Source: §5.4.3 · p. 15

5.5 Social Media Managers—Disseminating Fact-checks, Increasing Engagement

• Social media managers publish and disseminate fact-checks, monitor engagement, and update amplification strategies in response to audience feedback.Their work includes making content appealing to audiences.Source: §5.5 · p. 15

5.5.1 Disseminating fact-checks

• Managers distribute fact-checks and educational materials across platforms and use captions to make videos accessible to deaf audiences.One organization used captioned video because Twitter audio tweets lacked captioning features.Source: §5.5.1 · p. 15

5.5.2 Adopting strategies to increase engagement

• Managers use targeted advertisements and visual storytelling, including comics, videos, and regional-language versions, to promote fact-checks and engage audiences.Ad targeting may consider education, relevance to the fact-check, and interests; visual narratives respond to preferences for watching over reading.Source: §5.5.2 · pp. 15–16, Figure 2 · p. 16

5.5.3 Measuring engagement and updating strategies

• Click-through statistics help managers compare engagement across channels and adjust content, including reducing WhatsApp newsletter editions from five stories to three.The change followed observation that readers clicked mainly on the top two or three links.Source: §5.5.2 · p. 16, §5.5.3 · pp. 16–17

6.1 Investigators and Researchers—Conducting In-depth Research and Investigation

• Investigators and researchers examine persistently circulating campaigns through in-depth data and network analysis, rather than focusing on individual misinformation items.Teams may refer recurring cases for data sifting and account profiling.Source: §6.1 · p. 17

6.1.1 Conducting long term investigative projects.

• Long-term projects study misinformation campaigns, public responses, conspiracy theories, and how to communicate fact-checks effectively.Projects include surveys, analyses of influence operations, and research on fact-check design; findings are released as dossiers detailing methods and conclusions.Source: §6.1.1 · pp. 17–18

6.2 Advocators—Influencing Policy, Building Coalitions, Conducting Educational Workshops and Literacy Campaigns

• Advocators extend fact-checking into sustained campaigns to improve information quality, influence policy, build coalitions, and provide training.They use outreach programs, workshops, appeals, and training as informational and technological infrastructures.Source: §6 · p. 17, §6.2 · p. 18

6.2.1 Creating new generation of fact-checkers and fact-checking organizations.

• Advocators train people and organizations in fact-checking and in setting up their own fact-checking organizations.Training may support independent fact-checking desks in settings with limited press freedom.Source: §6.2.1 · p. 18

6.2.2 Pitching the importance of evidence based decision making to policymakers.

• Advocators engage policymakers and civil society organizations to promote evidence-based decision-making and seek policy changes.Examples include efforts to attribute online election and referendum materials to campaigners.Source: §6.2.2 · p. 19

6.2.3 Organizing literacy campaigns.

• Literacy campaigns teach people to critically examine information through community radio, courses, and tip sheets.Advocators also use partnerships and online learning platforms for training.Source: §6.2.3 · p. 19, §6.2.1 · p. 19

6.2.4 Spearheading initiatives to improve accessibility and quality of data and information.

• Advocators seek better access to reliable public data, including by engaging government agencies where data is old or inaccessible.In one example, census estimates were considered inaccurate, prompting outreach to statistical offices and other institutions.Source: §6.2.4 · pp. 19–20

6.2.5 Building collaborations and coalitions.

• Coalitions help organizations share resources and common concerns, including when making requests of internet companies.Some organizations also partner with Google and Facebook to increase fact-check visibility.Source: §6.2.5 · p. 20

• The section presents challenges faced by stakeholder groups, organized by themes.Source: §7 · p. 20

• Participants questioned AI’s ability to assess diverse political language, subtle truth differences, and the meaning of claims that require human discussion.They described fact-checking as a complex process involving claim extraction, interpretation, and decision-making.Source: §7.1 · pp. 20–21
• Some organizations use or are willing to use AI for low-stakes social-media monitoring, but not for high-stakes veracity labels.Participants described AI as useful for curating content to review, not deciding whether claims are true or false.Source: §7.1 · p. 21
• Fact-checkers said human review and explanations of how a tool reached its conclusions would support their trust in AI outputs.One participant said they would check an automated conclusion if its sources and reasoning were inspectable.Source: §7.1 · p. 21

7.2.1 Monitoring Social Media Platforms is Manual, Time Consuming and Difficult.

• Finding misleading content remains largely manual and time-consuming, while search-query generation often relies on trial and error.Participants described information overload and the need for better filtering; monitoring tools are also needed beyond platforms with existing support.Source: §7.2.1 · pp. 21–22

7.2.2 Limitations of platform affordances.

• Platform search limitations include restricted Facebook user-generated-content searches and difficulty finding duplicate YouTube videos posted under different keywords.Source: §7.2.2 · p. 22

7.2.3 Systems and tools needed to detect misleading claims on private message platforms.

• Privacy settings and end-to-end encryption constrain access to misleading content on private messaging platforms, while tool access raises ethical concerns about privacy.Source: §7.2.3 · p. 22

7.2.4 Overload of tools.

• Fact-checkers face tool overload, but a single tool may not meet the divergent needs of a multi-step process; purpose-built tool suites may be more useful.Source: §7.2.4 · p. 22

7.2.5 Need for specific tools.

• Video verification tools can check alteration or context, but claim extraction and verification in narrative videos remain manual, and regional-language transcription is difficult.Editors also described manual editorial work and accountability problems with Trello; social media managers reported difficulty gaining organic attention on some platforms.Source: §7.2.5 · pp. 22–23, §7.2.6 · p. 23

7.2.6 Getting organic engagement for fact-checks without advertisements.

• Getting organic engagement without advertising is easier on some platforms than others, according to participants.Source: §7.2.6 · p. 23

7.3 Issues around policy and information infrastructure

• Stakeholders regard data quality as essential both for investigating claims and for developing automated fact-checking tools.The section links data availability and quality to whether automated systems can work.Source: §7.3 · p. 23, §7.3.1 · p. 23

7.3.1 Need to improve information quality before automation.

• Fact-checkers need accessible, consistent, machine-usable data; missing or poorly formatted information limits automation and makes data integration difficult.Participants described data in PDFs or photographs and called for common formats and standards that support reuse.Source: §7.3.1 · pp. 23–24

7.3.2 Lack of information sources.

• In the Global South, information needed to investigate claims may be absent or outdated, leaving some claims uncovered.One participant said Kenya’s latest Demographic Health Survey was from 2014 and too old to use in 2020.Source: §7.3.2 · p. 24

7.3.3 Difficulty in getting information for research from civic organizations.

• Fact-checkers in African countries and the Balkans reported that research information is often not public and can be difficult to obtain from officials.Participants described requests to government as difficult because officials may not want to disclose information.Source: §7.3.3 · p. 24

7.3.4 Algorithmic bias against local content.

• Search algorithms can make local-language content from the Global South harder to find by favoring Global North content.A participant described regional-language material as taking longer to appear in Google searches and other platforms.Source: §7.3.4 · p. 24

7.4 Emotional cost of fact-checking

• Manually reviewing violent or conspiratorial content can take a toll on fact-checkers’ mental health.A participant described stress from examining violent content in detail alongside exposure to abuse.Source: §7.4 · p. 24
• Fact-checkers may face threats and online attacks, including after publishing their work.One participant described seeing their own photo in a post labeling them a censor.Source: §7.4 · p. 24
• The emotional cost of fact-checking has not been studied, leaving wellness interventions for its psychological effects as a future research direction.Source: §7.4 · p. 24

8.1 Rendering visibility to the human infrastructure of fact-checking

• Fact-checking includes hidden editorial, research, social-media, and advocacy work, extending beyond short-term claim debunking to long-term advocacy.The authors describe advocacy efforts to improve data and statistics as also improving claims-centered fact-checking.Source: §8.1 · p. 25

8.2 Collaborative efforts in the fact-checking process

• Collaboration spans internal stakeholder groups, outside experts, civic bodies, internet companies, policymakers, and social-media users.The discussion suggests fostering these connections, including through tools for user engagement and expert communication.Source: §8.2 · p. 25, §8.2.1 · pp. 25–26

8.2.2 Revealing the value of fact-checking work to internet companies.

• Fact-checking organizations provide fact-checks that internet companies use on their platforms, prompting the authors to raise whether organizations should receive greater economic benefits.Source: §8.2.2 · p. 26

8.2.3 Revealing power dynamics in collaborations.

• Power over fact-checking priorities and access is distributed unevenly among editors, civic bodies, and social-media companies.The discussion notes that platform tools can shape which claims are investigated and companies can remove fact-checks.Source: §8.2.3 · p. 26

8.3.1 Technical needs of fact-checking.

• Fact-checkers need better platform search and user reporting, tools to surface claims and credibility clues, shared verdict labels, and support for editors and social-media managers.Participants also described needs for multilingual reach, visible fact-checks, and machine-readable structured journalism.Source: §8.3.1 · p. 27

8.3.2 Values desired in fact-checking tools and systems.

• Fact-checkers are skeptical of black-box automation but open to low-stakes assistance, with explainability and human-in-the-loop design identified as desired values.Examples include monitoring platforms, detecting video alteration, and transcribing regional languages.Source: §8.3.2 · p. 28

8.3.3 Going beyond the technical: need for socio-technical solutions.

• Improving fact-checking quality requires changes beyond technology, including better availability and quality of public data and advocacy for policy change.The authors note that inaccessible or outdated information constrains manual and automated checking alike.Source: §8.3.3 · p. 28

8.3.4 Fact-checking in the Global South.

• Fact-checking research and tools need greater attention to the Global South, where regional-language resources and locally grounded knowledge are limited.The authors connect these gaps to the need for improved information infrastructure and research in the region.Source: §8.3.4 · pp. 28–29

9 CONCLUSIONS AND LIMITATIONS

• Interviews with 26 participants informed an account of six stakeholder groups, their work, supporting infrastructures, and challenges.The groups are editors, external and in-house fact-checkers, investigators and researchers, social-media managers, and advocators.Source: §9 · p. 29
• Most interviewed organizations were IFCN signatories or worked closely with them, so the described model and tools may not apply to every organization.The study also does not capture all regional variation, and not every organization includes every stakeholder role.Source: §9 · p. 29