Source-linked AI summary
"Piecing Data Connections Together Like a Puzzle": Effects of Increasing Task Complexity on the Effectiveness of Data Storytelling Enhanced Visualisations
Mikaela Elizabeth Milesi, Paola Mejia-Domenzain, Laura Brandl, Vanessa Echeverría, Yueqiao Jin, Dragan Gašević, Yi-Shan Tsai, Tanja Käser, Roberto Martínez-Maldonado
TL;DR
Prior research had not established how data-storytelling enhancements affect task completion across cognitive complexity levels. The study compares conventional and DS-enhanced visualisations using Bloom-aligned tasks, finding stronger support for lower-order performance and improved efficiency on complex tasks, but not necessarily higher-order correctness.
Problem
Previous research had not examined how data-storytelling enhancements influence task completion across different levels of cognitive complexity.
Method
A within-subjects controlled experiment had 128 participants complete Bloom-aligned tasks using four line-chart and choropleth-map visualisations, with and without DS elements.
Results
DS-enhanced visualisations support lower-order tasks, while higher-order tasks are not necessarily completed more correctly but are completed more efficiently.
Takeaways & Limitations
DS enhancements can aid finding data points and understanding insights, but their benefits for complex tasks primarily concern efficiency rather than correct completion.
Takeaways & Limitations
The study focuses on line charts and choropleth maps within a defined DS-enhancement framework, limiting coverage of other visualisation types and storytelling components.
Abstract
from arXiv · showhide
The emerging concept of data storytelling (DS) suggests that enhancing visualisations with annotations and narratives can make complex data more insightful than conventional visualisations. Previous works found that DS-enhanced visualisations are more effective than conventional visualisations for simple tasks like identifying key data points or the main message. However, no previous work has explored the extent to which DS enhancements influence task completion across different levels of cognitive complexity. We address this gap by presenting the results of a study where 128 participants completed tasks based on four visualisations (two line charts and two choropleth maps, either with or without DS elements) spanning a range of complexity based on Bloom's taxonomy, which has been applied in data visualisation to categorise tasks hierarchically from lower to higher-order thinking. Results suggest that while DS-enhanced visualisations effectively support lower-order tasks (finding data points and understanding insights), they don't necessarily aid the correct completion of higher-order tasks (application, analysis, evaluation and creation). However, DS enhancements improve how efficiently participants complete complex tasks.
1 Introduction
Data interpretation is increasingly important yet difficult across tasks with different cognitive demands. This study tests whether data storytelling enhances visualisation effectiveness and efficiency across that range.
- Motivation: Data visualisation supports communication, exploration, and insight discovery, but users may struggle to interpret data when patterns are not readily apparent.
- Research gap: Prior theory argues that data storytelling improves the accuracy and speed of insight extraction, but empirical findings on comprehension, recall, and efficiency remain mixed.
- Study focus: The study examines DS-enhanced versus conventional visualisations across tasks spanning lower- to higher-order levels of Bloom’s taxonomy.The experiment used four visualisations and 128 participants, with tasks ranging from Remember/Identify and Understand to Apply, Analyse, Evaluate, and Create.
- Contribution: The paper introduces a Human-in-the-Loop-AI process that combines large-language-model task generation with human validation to create scalable Bloom-aligned questions.
- Key findings: DS-enhanced visualisations support lower-order tasks such as finding data points and understanding insights, but do not necessarily improve higher-order task completion.
- Key findings: For analysis, evaluation, and creation tasks, DS enhancements improve completion efficiency, while complex writing contains fewer evaluation-level sentences.
2 Background
Data storytelling extends visualisation with narratives, annotations, and visual emphasis to guide sensemaking and support users with different levels of visualisation literacy. Its effects may depend on the cognitive complexity of the task.
- 2.1 Data Visualisation & Data Storytelling: Sensemaking is the deliberate interpretation of complex data to uncover patterns, relationships, explanations, and insights for reasoning and problem-solving.
- 2.1 Data Visualisation & Data Storytelling: Data storytelling combines data, visual elements, and author-driven narratives to guide audiences toward key findings and reduce cognitive demands.
- 2.1 Data Visualisation & Data Storytelling: Narrative-enhanced visualisations commonly highlight important data points, use explanatory titles, and add annotations that provide context or explain takeaways.
- 2.2 Prior Evidence: Prior evidence on data storytelling is mixed: studies report differences in long-term recall and insight comprehension, but not consistently in memory retention or efficiency.
- 2.3 Data Storytelling for Supporting Tasks of Varying Complexity: Bloom’s taxonomy organises tasks from identifying and understanding information to analysing, evaluating, and creating, providing a hierarchy for studying visualisation effects.
3 Method
The study compares conventional and data-storytelling-enhanced visualisations through a within-subjects experiment using Bloom’s taxonomy to assess tasks of increasing cognitive complexity.
- Study Design: The study compares conventional and DS-enhanced visualisations across tasks aligned with Bloom’s taxonomy, covering lower- and higher-order cognitive skills.The taxonomy levels range from Remember/Identify and Understand to Apply, Analyse, Evaluate, and Create.
- Visualisation Materials: Participants viewed four visualisations—two line charts and two choropleth maps—with and without DS elements.The visualisations covered Colonialism, Territory Control, State Capacity, and Tax Revenue as a share of GDP.
- Visualisation Design and Transformation Process: The visualisation pairs used identical filtered data points, while DS versions added annotations, explanatory titles, text emphasis, colour, and a footnote.Conventional versions retained the original aesthetics and visualisation types, supporting a fair comparison between conditions.
- Task Generation: Tasks were generated through a Human-in-the-Loop-AI process combining GPT-4 generation, BloomBERT validation, and author refinement.Of 89 generated questions, 46 correctly categorised by BloomBERT were retained for further refinement.
- Study Design: The controlled within-subjects survey exposed participants to both visualisation conditions, used randomisation to minimise order effects, and included multiple-choice, open linking, and qualitative preference tasks.A pre-test assessed prior knowledge, and a priori power analysis determined a target sample of 128 participants.
4 Results
DS-enhanced visualisations improved overall effectiveness and were especially beneficial for understanding, while higher-order accuracy gains were inconsistent. They also made complex tasks more efficient and were generally preferred, although some participants valued conventional visualisations’ lower clutter.
- 4.1 RQ1: Effectiveness: 57% versus 53% overall effectiveness favored DS-enhanced over conventional visualisations, a statistically significant difference with a moderate effect size.The comparison covered the first five Bloom levels.
- 4.1 RQ1: Effectiveness: 15% higher effectiveness at Understand favored DS-enhanced visualisations, with mean scores of 86% versus 71%; Identify and Apply were 5% higher but nonsignificant.Identify and Apply showed small effect sizes and nonsignificant differences.
- 4.1 RQ1: Effectiveness: 4% and 1% higher effectiveness for conventional visualisations at Analyse and Evaluate, respectively, were not statistically significant.These higher-order comparisons had small effect sizes.
- 4.2 RQ2: Efficiency: Participants were significantly more efficient with DS-enhanced visualisations at Understand, Apply, Analyse, and Evaluate, despite no significant overall efficiency difference.Efficiency was measured as average correct answer time, with 95% confidence intervals shown in Figure 7.
- 4.3 RQ3: Quality of Complex Writing Creation Tasks: DS-enhanced participants wrote 21% more Understand and 24% more Create sentences, whereas conventional participants wrote 29% more Evaluate sentences; none of these differences was statistically significant.Overall sentence counts were also not significantly different between conditions.
- 4.4 RQ4: User Perception: DS-enhanced visualisations were preferred 60% of the time, mainly for contextual information beside data points, while conventional visualisations were preferred for minimal clutter.Contextual placement was cited by 84% of DS-enhanced preferences; minimal clutter by 56% of conventional preferences.
5 Discussion
The discussion section frames the study as a summary of research-question findings, implications, limitations, and future research directions.
- 5 Discussion: The section reports key findings by research question, discusses implications for research and practice, addresses study limitations, and identifies future research directions.It serves as the transition into the paper’s discussion of results and implications.
5.1 Summary of Results and Research Questions
DS-enhanced visualisations supported effectiveness mainly for lower-complexity tasks, while conventional visualisations performed slightly better on some higher-order accuracy measures. DS also improved efficiency at higher cognitive levels, but its persuasive guidance may reduce critical scrutiny.
- 5.1 Summary of Results and Research Questions: DS-enhanced visualisations significantly improved overall effectiveness, with higher scores for Identify, Understand, and Apply but slightly higher, nonsignificant scores for conventional visualisations at Analyse and Evaluate.The authors suggest narratives may structure simpler tasks but provide less value or distract during abstract thinking.
- 5.1 Summary of Results and Research Questions: DS-enhanced visualisations significantly improved efficiency at Understand, Apply, Analyse, and Evaluate, although overall efficiency did not differ significantly between conditions.Conventional visualisations were slightly quicker at Identify; the authors discuss task-order exposure as one possible explanation.
- 5.1 Summary of Results and Research Questions: DS-enhanced visualisations produced more Understand, Analyse, and Create sentences, while conventional visualisations produced more Identify and Evaluate sentences, without significant overall composition differences.The authors suggest emphasized key values may reduce the need for simple restatement in the DS condition.
- 5.1 Summary of Results and Research Questions: The authors caution that persuasive DS visualisations may reduce awareness of information limitations or biases and encourage over-reliance on guided interpretations.This concern is presented as a possible unintended consequence of narrative-driven persuasion.
- 5.1 Summary of Results and Research Questions: Participants preferred DS-enhanced visualisations for contextual placement and clear distinctions, but preferred conventional visualisations for simplicity and minimal clutter.This preference pattern aligns with prior findings that shorter, focused text segments are perceived as useful.
5.2 Implications for Research
The study extends data-storytelling research by evaluating cognitive tasks across Bloom’s taxonomy and showing that task complexity shapes DS effectiveness. It also indicates efficiency benefits for complex tasks and introduces a human-in-the-loop AI task-generation process.
- 5.2 Implications for Research: Task complexity significantly affects DS effectiveness, motivating future research on whether task nature, such as written versus verbal tasks, matters similarly.The implication is framed as a direction for future work rather than a settled conclusion.
- 5.2 Implications for Research: DS enhancements can significantly accelerate insight extraction from complex data, complementing prior work that found no speed improvement over conventional visualisations.The paper connects this efficiency result to work arguing that DS supports deeper understanding of the presented narrative.
- 5.2 Implications for Research: The study systematically evaluates DS support across the full Bloom taxonomy, extending evidence beyond lower-order tasks to Apply through Create.These higher-order levels are described as underexplored and important for communicating complex ideas.
- 5.2 Implications for Research: A human-in-the-loop AI process combines large language models with domain-specific validation to generate tasks for evaluating visualisation performance across cognitive complexity.This process is presented as an additional methodological contribution.
5.3 Implications for Design and Practice
DS enhancements can improve understanding and efficiency, especially for complex tasks, but may reduce critical evaluation in creation tasks. Design choices should therefore reflect task goals and cognitive demands.
- DS enhancements improve effectiveness for Understanding tasks and efficiency for higher-order tasks requiring interpretation and critical thinking.
- In Create tasks, DS-enhanced visualisations produced fewer Evaluate sentences, although the difference was not statistically significant.
- A single narrative may discourage critical thinking by reducing users’ inclination to question assumptions or consider information absent from the graph.
- For complex cognitive tasks, DS enhancements did not support correctness but tended to facilitate more efficient information retrieval.
- Conventional visualisations should be enhanced with DS elements when the primary objective is completing complex tasks more efficiently.
5.4 Limitations and Future Works
The study’s scope is constrained by its visualisation types, online setting, and open-ended Create task. Future work should broaden visualisation and storytelling coverage while improving experimental control and task specification.
- The study focuses on line charts and maps, limiting exploration of other visualisation types and variations in contextual annotations or DS enhancements.The authors propose examining formats such as bar charts and scatter plots in future studies.
- Future research should test broader storytelling components, including interactive narratives, narrative flow, and sequencing techniques.
- The online study could not fully exclude participants using search engines or GenAI despite screening efforts.The authors suggest controlled in-person studies to mitigate cheating and obtain richer qualitative data.
- The open-ended Create task left question interpretation and response content to participants, so demographic variation may have influenced RQ3 responses.
6 Conclusion
Across Bloom’s taxonomy tasks using line charts and choropleth maps, DS enhancements improved Understanding effectiveness and higher-order efficiency but reduced critical evaluation in creation tasks. The findings support adapting visualisation design to user preferences, task complexity, and goals.
- DS-enhanced visualisations improved Understanding effectiveness and efficiency in higher-thinking tasks from Understanding to Evaluating.
- DS-enhanced visualisations reduced Evaluate sentences in creation tasks, suggesting a possible decrease in critical thinking when users become overly confident in presented data.
- The study compared data-storytelling-enhanced and conventional visualisations across tasks varying in complexity according to Bloom’s taxonomy.
- Visualisation design should adapt to user preferences, task complexity, and task goals.
A Example Questions Aligned with Bloom’s Taxonomy
Table 3 presents example study questions aligned with Bloom’s taxonomy, ranging from multiple-choice items to open-ended Create tasks.
- Table 3 lists example questions from visualisation A across Bloom’s taxonomy levels, including multiple-choice and open-ended Create tasks.
B Create Task Labelling Rubric
Table 4 presents the rubric used to categorize participants’ Create-task responses according to Bloom’s cognitive levels.
- Table 4 categorizes sentences from participants’ Create-task responses using Bloom’s Taxonomy cognitive levels.