Source-linked AI summary
Calliope: Automatic Visual Data Story Generation from a Spreadsheet
Danqing Shi, Xinyue Xu, Fuling Sun, Yang Shi, Nan Cao
TL;DR
Creating visual data stories is difficult because authors need data-analysis, visualization, and scripting skills, while existing tools still depend on manual insight extraction and organization. Calliope automatically explores spreadsheet data with a logic-oriented Monte Carlo tree search to generate and visualize logically ordered facts, and provides story editing. Evaluations comprising example stories, controlled experiments, and interviews with 10 experts report that the system benefits efficient visual data story generation.
Problem
Creating visual data stories requires difficult data analysis, visualization, and scripting, while existing authoring tools rely on users to manually extract, script, and organize insights.
Method
Calliope uses a logic-oriented Monte Carlo tree search to explore spreadsheet data, generate logically ordered data facts, measure their importance, and produce captioned charts with an online story editor.
Results
Calliope was evaluated through three example stories, two controlled experiments, and interviews with 10 experts, with results showing benefit for efficient visual data story generation.
Takeaways & Limitations
Calliope provides an automatic visual data story generation workflow that combines fact discovery, logical organization, chart visualization, captioning, and story revision.
Takeaways & Limitations
Experts identified rigid and grammatically imperfect captions, limited data-semantic understanding, simple visual encodings, and inability to handle hierarchical or relational datasets.
Abstract
from arXiv · showhide
Visual data stories shown in the form of narrative visualizations such as a poster or a data video, are frequently used in data-oriented storytelling to facilitate the understanding and memorization of the story content. Although useful, technique barriers, such as data analysis, visualization, and scripting, make the generation of a visual data story difficult. Existing authoring tools rely on users' skills and experiences, which are usually inefficient and still difficult. In this paper, we introduce a novel visual data story generating system, Calliope, which creates visual data stories from an input spreadsheet through an automatic process and facilities the easy revision of the generated story based on an online story editor. Particularly, Calliope incorporates a new logic-oriented Monte Carlo tree search algorithm that explores the data space given by the input spreadsheet to progressively generate story pieces (i.e., data facts) and organize them in a logical order. The importance of data facts is measured based on information theory, and each data fact is visualized in a chart and captioned by an automatically generated description. We evaluate the proposed technique through three example stories, two controlled experiments, and a series of interviews with 10 domain experts. Our evaluation shows that Calliope is beneficial to efficient visual data story generation.
1 INTRODUCTION
Calliope addresses the difficulty of creating visual data stories by automatically generating logically organized data facts from spreadsheets and supporting their visualization, narration, editing, and sharing.
- Visual data stories support comprehension and memorization but require data analysis, visualization, and scripting skills that are difficult for ordinary users.
- Existing authoring tools provide components for story creation, yet users still manually extract insights, write scripts, and organize story pieces.This involvement is inefficient, and resulting story quality remains dependent on user experience and skills.
- Automatic story generation must address informative fact discovery, fact importance measurement, fact visualization and narration, and logical story ordering.These challenges arise from large data spaces, complex data, and limited prior research on narrative logic from structured data.
- Calliope automatically generates visual data stories from spreadsheets using a logic-oriented Monte Carlo tree search algorithm.The system explores the input data space, generates potential data facts in logical context, and estimates their importance using information quantity and pattern significance.
- Calliope combines automatic generation with authoring and communication functions for editing the story logic and data-fact details, then publishing the story online.
2 RELATED WORK
Related work spans data-driven storytelling, automatic visualization, auto-insights, and natural language generation; Calliope combines these directions for automatic story generation from structured data.
- Data-Driven Storytelling: Narrative visualization is used in data-driven storytelling to enhance data understanding, information expression, and communication.Prior work identifies context definition, fact selection, modality selection, and order selection as key sequential story-creation actions.
- Data-Driven Storytelling: Existing authoring tools target general or specific narrative forms, but typically assume manually created story content or target expert users.Examples include annotated charts, infographics, slideshows, data videos, and time-oriented stories.
- Automatic Visualization: Automatic visualization research has progressed from chart recommendation to data-mapping generation and auto-insights.Calliope encodes different data-fact fields into charts using a rule-based method.
- Auto-Insights: Auto-insights systems extract data patterns and represent them in charts, with statistical significance used to quantify extracted insights.
- Natural Language Generation: Natural language generation methods for structured data include template-based captions and learned approaches, while Calliope uses fact-type-specific syntax to regulate readable captions.
- Design Guidance: Corpus statistics identify frequently used fact types, starting-point fact types, narrative relations, and preferred charts, guiding Calliope’s story-generation algorithm.Table 1 reports visualization-chart frequencies for 10 fact types.
3 DESIGN OF THE CALLIOPE SYSTEM
Calliope models a data story as meaningfully connected, data-backed story pieces and uses a preliminary survey to guide automatic generation. Its architecture combines a story generation engine with an editor supporting revision, representation, and sharing.
- 3.1 Data Story: A data story is a sequence of data facts connected by coherent relations to support a communication goal.Facts are visualized with charts and accompanied by annotations and narrations.
- 3.2 Preliminary Survey: 230 high-quality videos were segmented into 4186 story pieces, with 2583 data-related pieces labeled for fact types, charts, and coherence relations.The corpus contained 602 collected videos before selection.
- 3.2 Preliminary Survey: Survey statistics identify frequently used fact types, starting points, coherence relations, and preferred charts, guiding the story-generation algorithm.Table 2 summarizes coherence relations used after the 10 fact types.
- 3.3 System Design: Calliope targets successful, efficient, expressive, editable, and shareable stories through requirements covering logic, believable data, controllable time, multimodal expression, and flexible interaction.The requirements include adapting visual and textual representations across laptops, tablets, and smartphones.
- 3.3 System Design: The system has two modules: a story generation engine and a story editor.The engine progressively searches spreadsheet-defined data space, while the editor supports refinement of the generated storyline, narration, visuals, and facts.
4 STORY GENERATION ENGINE
The story generation engine defines structured data facts, scores their importance using self-information and pattern significance, and searches for logically connected story paths. A logic-oriented Monte Carlo tree search progressively explores spreadsheet data while using rewards and relations to guide generation.
- 4.1 Data Facts: A data fact measures data items in an input-dataset subspace using a measurable field, with optional grouping by a breakdown method.Calliope restricts fact fields and adds constraints to reduce redundancy, overlap, and ambiguity.
- 4.1 Data Facts: Calliope represents facts through type, subspace, breakdown, measure, and focus, supporting 10 fact types and aggregations such as count, sum, average, maximum, and minimum.Subspaces are defined by data-field filters, while measures retrieve or derive values from the selected scope or groups.
- 4.1 Data Facts: A fact’s importance combines self-information with pattern significance, because uncommon patterns and statistically meaningful patterns provide complementary evidence.Self-information increases as the fact’s occurrence probability decreases, while significance is estimated using auto-insight techniques.
- 4.1 Data Facts: The occurrence probability P(fi) factors the fact subspace, focus selection, and type-constrained choices of measure and breakdown.Focus probability is count(xi)/count(si), and unspecified focus assigns P(xi∣si) = 1.
- 4.2 Story Generation Algorithm: The algorithm constructs a search tree whose nodes are data facts, edges are logic relations, and paths represent candidate data stories scored by a reward function.Shared nodes receive the maximum reward among their paths.
- 4.2 Story Generation Algorithm: Given a spreadsheet and goal, the search iterates through selection, expansion, simulation, and back-propagation to generate a story progressively.Simulation explores several steps ahead within a time limit, and back-propagation updates relevant node weights.
13 end
Calliope generates stories by expanding logically related data facts through Monte Carlo tree search and selecting paths with high reward. Its reward balances diversity, logicality, integrity, and information entropy to guide coherent, data-covering stories.
- Story Generation Algorithm: The search expands each selected fact with logically relevant facts, simulates alternative directions, back-propagates rewards, and retains the highest-reward path until the goal is fulfilled.This process avoids exhaustive enumeration while progressively constructing the story.
- Logic-Oriented Node Expansion: Logic-oriented expansion uses relations such as similarity, temporal order, contrast, cause-effect, and elaboration to generate successive facts.For example, temporal relations change the temporal filter to a succeeding time, while similarity can modify fields without changing the subspace.
- Reward Function: Calliope’s reward function combines diversity, logicality, and integrity with the story’s information entropy.User-specified weights γ_i balance the three normalized criteria.
- Reward Function: Diversity measures both the number and evenness of fact types, while logicality averages the likelihood of coherent relations between successive facts.Higher logicality indicates a more coherent and easier-to-follow story.
- Reward Function: Integrity measures the story’s data coverage rate, with larger values indicating more comprehensive representation of the input data.The coverage calculation uses the total number of spreadsheet data items as its denominator.
5 DATA STORY EDITOR
The Calliope story editor supports configuration, fact and narrative-order editing, and visual story presentation. It converts facts into captioned charts and presents stories through multiple visualization modes.
- Story Editor Views: The editor has storyline, fact, and story visualization views for configuring generation, editing facts, and previewing or sharing stories.The storyline view also lets users upload a spreadsheet, set a generation goal, and adjust reward-function settings.
- Storyline View: Users can remove generated facts or change their narrative order according to their preferences.Generated facts appear in a row within the storyline view.
- Fact Visualization: Each data fact is represented by a chart and an automatically generated text caption.Captions follow the syntax specified for each fact type, and chart selection uses a rule-based approach.
- Story Visualization: The story visualization view first provides a textual summary including data coverage, the number of facts, and the generated narrative.Additional visualization modes support different application scenarios.
6 EVALUATION
Calliope was evaluated through example stories, controlled experiments, and expert interviews. The results indicate that its generated stories and logic were useful, human-consistent, and positively received by domain experts.
- Example Stories: Calliope generated stories from COVID-19, car sales, and startup-failure datasets, illustrating domain-specific data facts and trends.The datasets contained 903, 275, and 1234 rows, respectively.
- Example Stories: 21,921,768 cars were sold from 2007–2011 in the car-sales story, while sales generally decreased during the financial crisis.Ford, Toyota, and Honda were the top three sellers, and SUVs outsold MPVs by 6,764,065 cars.
- Controlled Experiments: Participants reported that Calliope’s ordered facts supported following and remembering stories, while comprehension, engagement, and dissemination were comparable to DataShot.The comparison used factsheets containing similar data facts and revised with the same visual style.
- Controlled Experiments: Calliope’s logic orders aligned with human-generated orders across datasets, with Kendall’s τb values of 0.487, 0.648, and 0.63.The random-order baseline was 0.015, and Calliope’s orders were much more consistent with humans than the baseline.
- Expert Interviews: Ten domain experts rated Calliope relatively highly and agreed that its generated stories expressed useful insights with clear, well-organized data facts.Experts also valued visualization modes, editing, efficient story generation, and support for users less skilled in data analysis or visualization.
7 Limitations and Future Work
The authors identify limitations in textual narrative quality, data-semantic understanding, visualization richness, and generation performance, outlining corresponding future work.
- Supporting a Better Textual Narrative: Generated captions were too rigid for data news and sometimes contained grammar errors, despite remaining useful for rapid preview and briefing.The authors propose more advanced natural language processing techniques for higher-quality narratives.
- Understanding Data Semantics: Calliope does not adequately understand data semantics, limiting its ability to generate meaningful and insightful story content and logic.The authors suggest advanced AI techniques or interactive feedback to involve users in steering generation.
- Enriching Visualization: Experts requested slides and dashboard modes, richer visual encodings, and support for hierarchical or relational datasets.Suggested encodings include point size in line charts and stacked bars for additional categorical fields.
- Performance Issues: Statistical computation of data-fact significance is slow, limiting facts explored per search iteration and affecting generation quality.This bottleneck concerns the calculation of S(fi) in Formula (1).
8 CONCLUSION
Calliope automatically generates visual data stories from spreadsheets by exploring data facts in logical order and visualizing them with charts and captions. Evaluation through examples, experiments, and expert interviews showed the system’s effectiveness while revealing limitations for future work.
- Conclusion: Calliope automatically generates spreadsheet-based visual data stories by progressively organizing data facts in logical order.Its logic-oriented Monte Carlo tree search explores the data space during generation.
- Conclusion: Fact importance is measured using information quantity and statistical significance, and each fact receives a chart and automatically generated caption.A story editor supports efficient editing of generated stories.
- Conclusion: Evaluation comprised three example cases, two controlled experiments, and interviews with 10 expert users from different domains.The evaluation both demonstrated the system’s power and revealed limitations for future work.