Source-linked AI summary
Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories
Kevin Qinghong Lin, Batu EI, Yuhong Shi, Pan Lu, Philip Torr, James Zou
TL;DR
Data journalists must combine multidisciplinary reporting, analysis, design, and editing to turn raw data into trustworthy stories. Data2Story orchestrates these roles in a virtual newsroom with evidence traceability and multimodal storytelling, producing compelling, verifiable articles while human journalists retain advantages in editorial angle, creative design, and presentation.
Problem
End-to-end data journalism requires multidisciplinary skills that existing specialized agents do not combine into stories readers can trust.
Method
Data2Story orchestrates seven specialized roles, including an Inspector that traces article elements to upstream code or references, to produce generative multimedia articles.
Results
Across 18 paired samples, Data2Story received favorable human and computer-use-agent evaluations, with particular strength in claim-level auditability and transparency.
Takeaways & Limitations
Data2Story can augment newsroom workflows with creative multimedia assets and an auditability dimension that is rarely formalized.
Takeaways & Limitations
Data2Story recovers only about half of human journalists’ editorial angle because relevant causes may depend on expert testimony and outside knowledge absent from the data.
Abstract
from arXiv · showhide
Data tells stories that shape society; the data journalist's job is to turn raw information into stories non-experts can trust. A high-quality news feature takes a newsroom team weeks: hunting for context, running statistics, choosing an angle, and designing visuals. Recent agents handle individual steps well: data-science agents close the analysis loop, while design agents synthesize beautiful websites. But can an agent serve as a data journalist end to end? We introduce Data Journalist Agent (Data2Story), a multi-agent framework that orchestrates specialized roles into a single virtual newsroom. Data2Story contributes two innovations. (i) Claims are evidence-grounded: an Inspector links every number, angle, and asset back to data, code, or an external reference. (ii) Articles are multimodally generative: rather than defaulting to plain text and static charts, Data2Story reasons about what readers will want to see, then deploys multimodal tools, such as interactive maps for geography and audio for music. We evaluate Data2Story on 18 articles, each paired with the originally published expert piece, along four axes: (a) human-agent angle coverage; (b) rubric evaluation with 53 participants across five dimensions; (c) computer-use agents as judges, a cost-saving proxy for how readers navigate interactive articles; and (d) verifiability, where a coding verifier re-executes statements against the data and checks claims against references. Data2Story produces competitive, evidence-traceable multimedia stories, with particular strength in transparency and auditability. Human articles retain an edge in editorial angle, creative design, and presentation. We position Data2Story as a collaborator for journalists, enabling more evidence-based, transparent, and verifiable reporting. Code and demos are available at https://data2story.github.io.
1 Introduction
Data2Story is a multi-agent virtual newsroom that coordinates specialized roles to produce multimodal data stories with claim-level evidence traceability. Its evaluations indicate competitive, compelling, independently verifiable articles, while human journalists remain stronger in editorial angle, creative design, and presentation.
- Motivation: Data journalism requires weeks of multidisciplinary work, spanning context gathering, statistical analysis, narrative framing, asset design, webpage construction, and editing.Recent data-science agents can already profile datasets, run statistics, and return reproducible results, but end-to-end journalism requires skills that rarely coexist in one contributor.
- Evidence grounding: Nearly all statistics, visual assets, and factual claims are grounded in executable code or verifiable source URLs, making the reasoning chain auditable end to end.This directly addresses the verification and traceability gap in existing AI-generated news, where readers and editors may be unable to confirm sources, chart accuracy, or whether claims were hallucinated.
- Framework: Data2Story orchestrates seven specialized roles into a virtual newsroom, including Detective, Analyst, Editor, Designer, Programmer, Auditor, and Inspector.The Inspector traces elements of the final article back to their evidentiary basis.
- Evaluation: Data2Story produces multimodal articles that readers find compelling and that can be independently verified, with built-in claim-level evidence traceability.The framework is also demonstrated on new datasets, where it discovers original findings and presents them through user-friendly media such as interactive maps.
- Human–agent collaboration: Human journalists retain a clear edge in editorial angle, creative design, and informative presentation, while Data2Story’s greatest advantage is auditability.The system is positioned as a collaborator: humans provide perspective and editorial judgment, while agents handle labor-intensive computation and graphics design.
2 Related Work
Related work spans web-browsing research agents, data visualization and science agents, and data-journalism systems, while Data2Story is compared against representative works across these fields.
- Deep Search Agents: Deep Search Agents browse the web to produce retrieval-augmented text deliverables, using approaches such as query decomposition or end-to-end reinforcement learning.OpenAI’s Deep Research is a representative commercial demonstration; MindSearch decomposes queries into atomic sub-questions, while DeepResearcher trains browsing policy end-to-end.
- Data Visualization Agents: Data Visualization Agents compile fixed datasets into executable visualizations or narratives, but generally do not search for external evidence or produce interactive outputs.Examples include LIDA, DataNarrative, and MatplotAgent; their outputs are mostly static visual artifacts.
- Data Journalists: Data-journalism systems target general-audience communication through publishable artifacts, workflow studies, journalistic assistance, or multimodal formats such as animated data video.Recent work explores article planning, angle recommendation, and source identification; DataDirector combines Vega-Lite charts, TTS audio, and animation.
3 Data Journalist Agent
Data Journalist Agent transforms raw data into compelling, visually appealing, and verifiable articles through a virtual newsroom of specialised roles. The workflow combines contextual research, exhaustive coded analysis, editorial framing, multimodal design, HTML rendering, auditing, and evidence binding.
- Data Journalist Agent aims to produce articles that are narratively compelling, visually appealing, and verifiable from any raw dataset.
- The virtual newsroom coordinates specialised roles that gather context, analyse data, shape findings, create assets, render HTML, audit pages, and bind claims to evidence.
- Detective: The Detective enriches raw data with externally sourced context and reference media before analysis, helping downstream roles frame data without inventing claims.
- Analyst: The Analyst profiles every column, enumerates supported analyses, executes code, and produces findings with pointers to the scripts that generated them.
- Designer and Programmer: The Designer selects reader-appropriate media such as maps, audio, video, or interactive widgets, while the Programmer renders upstream artifacts into HTML without generating new facts or numbers.
- Auditor and Inspector: The Auditor flags visual and structural defects for repair, while the Inspector makes every visible claim evidence-traceable to source data, code lines, or external references.
4 Evaluation · 4.1 Setting
The evaluation examines how to assess Data2Story articles, compare them with human-written counterparts, and measure agreement between human and agent judges. Its setting pairs datasets with published reference pieces across diverse sources, domains, time periods, and data modalities.
- 4 Evaluation: The evaluation asks how to fairly measure article quality, compare Data2Story with human counterparts, and assess agreement and consistency among human and agent judges.
- 4.1 Setting: The evaluation set draws examples from three stylistically distinct sources selected to span contemporary data storytelling.
- 4.1 Setting: The curated examples span science, media, sports, politics, health, and other domains.
- 4.1 Setting: The set covers 2018–2026 and includes time-series and tabular data, among other modalities.
- 4.1 Setting: Examples come from The Economist, The Pudding, and TidyTuesday, representing analytical, artistically interactive, and community-oriented data storytelling.
- 4.1 Setting: Each evaluation example pairs underlying data with a published human-written piece for head-to-head comparison with Data2Story’s output.Table 2 notes that human articles rarely include complete code, with ✓ in Code indicating partial code such as data cleaning.
- 4.1 Setting: The authors cannot rule out pretraining contamination for well-known Economist and Pudding articles, but coverage and verifiability require capabilities beyond recalling article text.Coverage rewards both matching human angles and surfacing omitted claims, while verifiability depends on code and cross-family verification.
- 4.1 Setting: Data Journalist Agent articles are produced using Claude Code with claude-opus-4.7.Implementation details are provided in Appendix B.
4.2 Evaluation Metrics
Data2Story is evaluated through complementary protocols covering angle overlap, reader-rated quality, computer-use judging, and claim verifiability. The methods distinguish human and agent perspectives, assess five article-quality dimensions, and verify computational and reference-supported claims against their sources.
- Angle Coverage: Angle coverage uses P(Agent | Human) for the fraction of human claims also surfaced by the agent article.This measures whether the agent catches what a journalist would catch.
- Angle Coverage: P(Human | Agent) measures the fraction of agent claims appearing in the human article, indicating alignment with the human-curated angle.A gap between the two conditional measures reflects claims unique to one side, from divergence or broader coverage.
- Rubric Evaluation: Human reviewers assess Data2Story–human pairs across five rubric dimensions: visual design, narrative and pacing, data and method transparency, claim–data alignment, and insight value.The study recruits 53 Prolific reviewers, who also report an overall binary preference after viewing both versions.
- Computer-Use Agent as Judge: A computer-use agent, OpenAI’s browser-use gpt-5.5-xhigh, navigates rendered interactive pages and scores them on the same rubric dimensions as human readers.This protocol is intended as a cost-saving alternative to manual human evaluation.
- Verifiability: Verifiability replays each article with OpenAI’s Coder codex-GPT-5.4, checking computational claims by rerunning scripts and reference-supported claims by re-fetching cited URLs.For human-written articles, the verifier accesses the original dataset; for agent-written articles, it additionally receives the Inspector’s full reasoning trajectory.
4.3 Experiment Results
Across 18 paired articles, Data2Story matches human writing volume while using shorter, more granular statements, covers predictable angles more reliably than human narrative arcs, and robustly produces multimodal assets. Human evaluations favor Data2Story overall—especially for transparency—while provenance audits and Inspector analyses show strong machine-checkability and a concentrated transparency lift.
- Text and angle coverage: Data2Story produces 1305 words versus 1557 for humans, using 1.45× as many sentences that are 0.77× as long.Its articles are broken into shorter, more granular statements.
- Text and angle coverage: 50.4% of the human article angle appears in Data2Story articles, while 35.1% of the agent’s angle maps back to human articles.Coverage is strongest for predictable Economist briefings and weaker for Pudding’s creative long-form narratives and TidyTuesday’s diverse topics.
- Multimodal generation: Data2Story averages 13–14 multimedia assets per article and covers heading, interactive, audio, video, image, and chart modalities in similar proportions across sources.Human designers vary asset distributions substantially with editorial style.
- Human and agent evaluation: Data2Story scores 4.21 overall versus 3.38 for humans across 53 participants and leads on all five rubric dimensions, with the largest gap in Transparency (+1.49).The smallest gap is on Visual (+0.51); Economist and TidyTuesday favor the agent, while Pudding is a statistical tie.
- Human and agent evaluation: 39 of 53 reviewers preferred Data2Story, compared with 13 for humans and 1 tie, while agent-judge rankings correlate with human rankings at 𝜌=0.44, 𝑝<.01.The agent judge preserves the human ordering at a fraction of the cost, although it scores both article types higher in absolute terms.
- Provenance and Inspector effects: 93% of visible claims in Data2Story articles resolve to traceable upstream evidence, and opening the Inspector raises the overall mean from 4.60 to 5.10, concentrating on Transparency (4.28→5.94, Δ=+1.67).Role-level provenance coverage is Editor 99.3%, Detective 95.1%, Analyst 74.1%, and Designer 29.0%; 66% of reviewers found the Inspector helpful, while 25% found it unhelpful or distracting.
4.4 Qualitative assessment: where human did better?
Across paired examples, human-written articles retained advantages in editorial angle, creative design, and informative presentation. Data2Story’s analyses and evidence-traceable transparency remained strengths, but reporting beyond the data and hand-built storytelling craft remained difficult to reproduce.
- Overview: Human articles showed three recurring advantages: editorial angle, creative design, and informative presentation.These qualitative differences surfaced values that numerical experiments did not capture.
- Editorial Angle: Data2Story recovered only about half of the human editorial angle because expert testimony and outside knowledge were absent from the dataset.It could rank what breaks, but could not compute the accountability claim attributing failures to manufacturers.
- Creative Design: Human teams built bespoke interactions, while Data2Story used static charts and click-to-play or linked media for the same material.Examples include line-level laugh annotations for Ali Wong’s special and an animated audio-visual jukebox covering all 55 Internet Boy Band acts.
- Informative Presentation: Human-designed figures carried more meaning per frame by combining variables, annotations, and contextual explanations that Data2Story distributed across separate charts.The space-race chart encoded launch failures and satellite lifetimes, while the football chart aligned managers and star players with an impact annotation.
- Overall Assessment: Data2Story led on coverage, analysis, and auditable transparency, while reported angles and hand-built design or chart craft remained human strengths.The cases therefore qualify rather than overturn the agent’s broader strengths in evidence-grounded analysis.
5 Discussion
Data Journalist Agent is a multi-agent virtual newsroom for end-to-end data journalism, combining evidence-traceable claims with audience-aware multimodal storytelling. The authors position Data2Story as a collaborator that augments newsroom workflows and helps surface overlooked datasets as accessible, verifiable stories.
- Data Journalist Agent orchestrates specialised roles into a single virtual newsroom for end-to-end data journalism.
- Data2Story binds each number, quote, and asset to a specific code line or reference through an evidence-traceable Inspector.
- Data2Story generates multimodal stories by reasoning about audience needs before selecting sub-agents and tools suited to the data and reader.
- The authors frame agent-generated articles as collaborators that add creative multimodal assets and a rarely formalised auditability dimension to newsroom workflows.
- Data2Story can complement existing coverage by turning specialised or niche datasets that journalists rarely have time to investigate into accessible, verifiable stories.
Appendix · A Model Settings
The Data Journalist Agent uses Claude-code opus-4.7, with OpenRouter serving as the unified provider for generative models. Evaluation components use specialized OpenAI models for retrieval, matching, and computer-use judging.
- A Model Settings: Data Journalist Agent is based on Claude-code opus-4.7.
- A Model Settings: OpenRouter is the unified provider for all generative models.
- A Model Settings: The Designer role uses tools backed by generative models listed in Table 3.Table 3 summarizes each generative capability and its OpenRouter API model.
- A Model Settings: Human-agent angle coverage uses text-embedding-3-small to calculate retrieval similarity.
- A Model Settings: Human-agent angle coverage then uses gpt-4o-mini to decide matching.
- A Model Settings: Computer-use agent as judge experiments use OpenAI’s browser-use gpt-5.5-xhigh.
B Rubric Evaluation Scoring Standard
The rubric evaluates human and agent artifacts on five dimensions using detailed 1–7 scoring guidance, with 3 as the typical default. It emphasizes visual and narrative quality, transparency, claim–data alignment, and the value of readers’ cognitive updates, while examples illustrate trade-offs between human and agent designs.
- Scoring framework: Scores range from 1 to 7 across five dimensions, with 3 serving as the typical default for both human studies and agent judges.The dimensions are Visual Design, Narrative & Pacing, Data & Method Transparency, Claim–Data Alignment, and Insight Value.
- Visual Design: Visual Design assesses palette, typography, layout, whitespace, chart–claim fit, and polish, with top scores requiring optimal encodings and unusually well-targeted design choices.A score of 7 is indistinguishable from top editorial graphics work, while 5 denotes cohesive design with intentional palette, hierarchy, and chart selection.
- Narrative & Pacing: Narrative & Pacing evaluates the author’s guiding structure through the hook, ordering, rhythm, and ending, with higher scores requiring a strong arc and meaningful structural move.For non-narrative artifacts, the rubric scores whatever framing exists, and a bare dataset with only a title receives 1.
- Data & Method Transparency: Data & Method Transparency requires specific sources, described methodology, accessible data, and concrete limitations, with a 7 indicating fully replicable, audit-grade work.A score of 5 requires all four components, while a score of 3 reflects cited sources, briefly gestured-at methodology, no data access, and minimal caveats.
- Credibility and insight: Claim–Data Alignment measures whether claims are substantively supported, while Insight Value measures whether readers gain a non-trivial cognitive update rather than merely learning that a dataset exists.Top claim-alignment scores require bounded quantitative claims, explicit treatment of overinterpretation and uncertainty, unambiguous encodings, and no causal overclaiming; insight scores are capped at 2 for a dataset-only takeaway.
- Illustrative comparisons: Across examples, agent artifacts often increased interactivity or exposed additional structure, but human versions more consistently supplied annotations, explanatory context, polished multimedia integration, and immediate readability.Examples include interactive year-by-year figures without descriptive annotations, a static YouTube iframe replacing inline synchronized video, and denser charts that reveal between-class structure at the cost of legibility.
C Agent-as-Judge demonstration
The agent-as-judge demonstration shows a computer-use agent reading generated articles through visual interaction and preparing rubric judgments. With the Inspector enabled, it additionally verifies claims and visual assets against their source lineage before scoring.
- The judge loads the article and observes its introductory animation, reproducing a human reader’s first encounter with the page.
- It reads the article in natural order by repeatedly scrolling and capturing screenshots of prose, charts, and statistical callouts.
- With the Inspector open, the agent accesses annotated sentences with lineage badges and named rendered assets in two structured views.
- Before issuing scores, it navigates the Inspector to verify each rendered claim and visual asset against code lines, data tables, or external links.