Source-linked AI summary
Why Authors Don't Visualize Uncertainty
Jessica Hullman
TL;DR
Uncertainty is rarely visualized in public-facing data communication, raising questions about why authors omit it despite its recognized value. The paper surveys and interviews visualization authors, then develops rhetorical and statistical accounts of omission. It finds a persistent tension between valuing uncertainty communication and treating uncertainty as a threat to a visualization’s signal.
Problem
Public-facing visualizations often omit uncertainty despite proposed communication techniques, motivating the question of why visualization authors choose not to show it.
Method
The study surveys 90 visualization authors, interviews 13 influential designers, develops a rhetorical model of omission, and adapts a graphical-inference model to communicative visualization.
Results
The study identifies a contradiction between authors’ stated appreciation of uncertainty and the norm of omitting its direct depiction, with uncertainty communication argued to reduce viewers’ inferential flexibility.
Takeaways & Limitations
Understanding omission requires addressing authors’ rationales, practical resource needs, and the beliefs and norms that make omission seem reasonable.
Takeaways & Limitations
The participants may overrepresent authors sympathetic to uncertainty visualization, and responses may be biased by retrospective recall of communication choices and rationales.
Abstract
from arXiv · showhide
Clear presentation of uncertainty is an exception rather than rule in media articles, data-driven reports, and consumer applications, despite proposed techniques for communicating sources of uncertainty in data. This work considers, Why do so many visualization authors choose not to visualize uncertainty? I contribute a detailed characterization of practices, associations, and attitudes related to uncertainty communication among visualization authors, derived from the results of surveying 90 authors who regularly create visualizations for others as part of their work, and interviewing thirteen influential visualization designers. My results highlight challenges that authors face and expose assumptions and inconsistencies in beliefs about the role of uncertainty in visualization. In particular, a clear contradiction arises between authors' acknowledgment of the value of depicting uncertainty and the norm of omitting direct depiction of uncertainty. To help explain this contradiction, I present a rhetorical model of uncertainty omission in visualization-based communication. I also adapt a formal statistical model of how viewers judge the strength of a signal in a visualization to visualization-based communication, to argue that uncertainty communication necessarily reduces degrees of freedom in viewers' statistical inferences. I conclude with recommendations for how visualization research on uncertainty communication could better serve practitioners' current needs and values while deepening understanding of assumptions that reinforce uncertainty omission.
1 INTRODUCTION
The paper investigates why uncertainty is rarely visualized in public-facing communication despite authors’ recognition that it can matter. It combines empirical characterization with rhetorical and statistical models to explain this omission.
- Only 14 of 612 data visualizations (3%) portrayed uncertainty visually, although 449 (73%) presented data intended for inference.The sample covered 121 online articles from data journalism, social science surveys, and economic estimates.
- Authors may omit uncertainty because they perceive it as psychologically complex, costly in viewers’ attention, or difficult to calculate and explain.The introduction also notes that adding information may make already complex displays harder to interpret.
- The study surveys 90 professional visualization authors and interviews 13 influential designers and journalists to characterize how communicative authors understand and use uncertainty.
- Authors report needs for resources that support text-based explanations and broadly understandable visual representations of uncertainty.
- A rhetorical model explains omission through beliefs that visualizations convey and produce signal while exposed uncertainty is thought to obfuscate it.
- The paper also adapts a graphical-inference model to argue that uncertainty visualization reduces flexibility in viewers’ processes for judging a visualization’s signal.
2 BACKGROUND
Prior work frames uncertainty communication as difficult for both authors and audiences, while this paper situates omission within a broader norm and examines its possible consequences for inference.
- Scientific and analytical communication can also fail when technical detail is difficult for decision-makers to decipher or leaves important assumptions implicit.
- Successful uncertainty communication requires authors to recognize its value to receivers and identify effective ways to communicate it.Threats can arise at either stage.
- Experts often prefer qualitative uncertainty language, whereas decision-makers and end-users typically prefer precise quantitative uncertainty.
- Risk communication can break down even after numbers are calculated, because communicating and explaining their meaning involves multiple developmental stages.
- Researchers have proposed that omission of uncertainty from data presentations may reflect an unstated norm, alongside evidence that data workers struggle to quantify and manage uncertainty.
- Uncertainty-visualization research has developed taxonomies and techniques for expressing quantified uncertainty to analysts, experts, and non-experts.
3 METHODS
The study uses formative survey and interview research with visualization authors recruited through convenience and social-media sampling. Responses were coded iteratively to characterize practices, rationales, and perceptions of uncertainty communication.
- The formative research surveyed visualization authors and interviewed a smaller group of authors viewed as influential among practitioners over seven months in 2018 and 2019.
- The online survey received 90 completed responses from people recruited as authors who regularly create visualizations for others.
- The survey asked how often and how respondents visualized uncertainty, why they sometimes omitted it, and what guidelines or considerations they associated with it.
- The convenience sample may overrepresent authors who have considered uncertainty or believe it should be expressed, limiting interpretation of response frequencies.
- Semi-structured interviews included 13 visualization influencers who did not self-identify as researchers.
- The analysis used open coding followed by repeated iteration until codes stabilized across survey and interview responses.
4 SUMMARY OF UNCERTAINTY ASSOCIATIONS & PRACTICES
Authors associate uncertainty with intervals, probabilities, variance, process, resolution, visual imprecision, and progressive views, yet communicate it infrequently. They cite viewer concerns, resource limits, authorial risk, and stakeholder resistance, even as most believe uncertainty should be represented more often.
- Associations: Intervals, ranges, or regions—often with error bars—were the most prevalent associations with uncertainty visualization, while possible outcomes and probabilities were also common.Roughly half of respondents mentioned intervals, ranges, or regions; roughly 15% mentioned possible outcomes or values not representing reality.
- Practices: 76% of surveyed authors said they had depicted uncertainty in the previous year, but uncertainty communication remained rare across their visualizations.Over one third reported uncertainty in 10% or less of their visualizations, while only one quarter reported it in 50% or more.
- Associations: Authors also associated uncertainty with process, data resolution, raw-data variance, intentional visual imprecision, and sequences of views ordered by probability.These associations extend beyond conventional quantitative probabilistic representations.
- Omission: Nearly half of respondents had attempted to communicate uncertainty but omitted it, most often to avoid confusing or overwhelming viewers.Other cited reasons included lacking uncertainty information, not knowing how to calculate it, and not wanting data to seem questionable.
- Omission: Authors described viewer effort, empathy, limited access or expertise, time and funding constraints, and fear of error or misplaced precision as omission pressures.Stakeholders sometimes questioned whether the effort required to represent uncertainty was worth the cost.
- Attitudes: Despite these pressures, a majority believed uncertainty should be represented more often, and nearly all interviewees described doing so as a goal or responsibility.Authors connected this belief to transparency, accurate presentation, and the educational potential of uncertainty representation.
5 A RHETORICAL MODEL OF UNCERTAINTY OMISSION
The rhetorical model explains uncertainty omission as a norm sustained by beliefs that visualizations should convey validated signals, while uncertainty threatens signal clarity, attention, and trust. Authors’ accounts reveal tensions between process-based validation, audience trust, and the perceived obfuscating effects of uncertainty.
- 5.2 Tenet 1: A Visualization Expresses a Signal: Tenet 1 frames communicative visualizations as representations of a signal or message that authors want audiences to receive clearly.Authors described signals as abstractions or crystallizations of more complex material.
- 5.3 Faith in Process: Tenet 2 holds that analytical processes validate signals for authors and viewers, although viewers generally trust processes they do not directly experience.Authors often relied on analysts’ or scientists’ processes, professional judgment, and assumed audience trust.
- 5.4 Tenet 3: Uncertainty Obfuscates Signal: Tenet 3 treats uncertainty as a question or seam that can threaten a signal through statistical invalidation, attentional distraction, or procedural exposure.Authors associated uncertainty with extra viewer work, possible doubt, and questions about how numbers were produced.
- 5.1 Premise: Uncertainty Omission is a Norm: Uncertainty omission is treated as a norm, supported by authors’ limited use of uncertainty visualization and repeated accounts of organizational indifference.The model uses this norm as its starting premise.
- 5 A RHETORICAL MODEL OF UNCERTAINTY OMISSION: The model explains why authors may value uncertainty in principle yet omit it in practice: omission preserves a clear signal, relies on trusted processes, and avoids exposing uncertainty as disruptive.Interview accounts also suggest that visualization design itself can tune signal strength, complicating the belief that signals exist independently of visualization.
6 MODELING INFERENCE IN COMMUNICATIVE VISUALIZATION
The paper models visualization viewing as an implicit statistical inference process in which viewers choose model specifications, reference distributions, and discrepancy functions. It argues that visualized uncertainty can constrain these choices, while also exposing inconsistencies between authors’ assumptions and viewers’ possible inferences.
- 6.1 A Formal Theory of Visualization-Based Inference: Viewers can judge a visualization’s signal by mentally comparing observed data with outcomes from a reference distribution generated by a model.The formalization uses posterior predictive distributions and discrepancy functions to represent this graphical inference process.
- 6.1 A Formal Theory of Visualization-Based Inference: Different assumptions about variance, fitted years, or model structure can lead viewers to different conclusions from the same visualization.Figures 4b–e illustrate how changing assumed variance or the year used to fit the model changes the viewer’s possible inference.
- 6.1.1 Contradictions Exposed by the Model: Authors appear to assume that viewers’ model specifications, discrepancy functions, and thresholds will closely match their own.This assumption conflicts with the multiple graphical inference processes that viewers may construct from the same display.
- 6.1 A Formal Theory of Visualization-Based Inference: Visualization choices themselves shape the reference distributions and discrepancy functions viewers may use to judge a signal.Removing a country or extending an axis can alter what outcomes seem plausible and therefore change the perceived disparity.
- 6.1.2 How Uncertainty Can Help: Explicitly visualizing uncertainty can reduce viewers’ degrees of freedom by supplying information about the intended reference distribution.Intervals can facilitate the intended inference, although they do not guarantee that all viewers construct the same model or interpretation.
7 DISCUSSION
The discussion frames uncertainty communication as an unresolved practical and theoretical problem: authors often value it but face challenges in calculating, explaining, and depicting it. The paper therefore calls for research and tools that address authors’ needs while making the consequences of omission more concrete.
- 7 DISCUSSION: Uncertainty communication remains an unresolved practical problem despite many authors’ desire to communicate it more frequently.Interviewees connected this tension to a long-term strategic challenge involving viewers’ tolerance for and ability to reason with uncertainty.
- 7 DISCUSSION: The rhetorical and graphical inference models explain how uncertainty omission can persist alongside authors’ stated belief that uncertainty matters.They connect omission to beliefs about signal clarity and to the inferential processes through which viewers interpret visualizations.
- 7 DISCUSSION: Researchers should develop tools that help authors calculate, visualize, and explain uncertainty in forms that meet practitioners’ needs.The discussion also motivates research on approximate communications that convey imprecision or problematize default aggregation.
- 7 DISCUSSION: Concrete examples, simulated reference distributions, and decision-theoretic tools could help authors consider the consequences of including or omitting uncertainty.Such tools could prompt consideration of possible viewer decisions under uncertainty omission versus inclusion.
- 7 DISCUSSION: Formalizing graphical inference could improve awareness of the cognitive and perceptual processes that produce perceived signal in communicative visualization.The paper suggests tools that help authors articulate target inferences and create uncertainty expressions aligned with them.
- 7 DISCUSSION: The study’s participants may not represent all visualization authors, because respondents may have been especially sympathetic to uncertainty communication and may have recalled past decisions imperfectly.The survey and interview samples also showed considerable heterogeneity in practices and perceptions.
8 CONCLUSION
The paper finds that uncertainty is usually absent from non-scientific visualizations and investigates why authors omit it despite recognizing its value. It combines evidence about authors’ practices and beliefs with rhetorical and graphical-inference models to explain omission and argue that uncertainty reduces, but does not eliminate, viewers’ inferential flexibility.
- 8 CONCLUSION: Most visualizations outside scientific journals do not explicitly represent uncertainty information.The paper examines authors’ perceptions, practices, challenges, and attitudes to understand this omission.
- 8 CONCLUSION: Many authors value uncertainty visualization but face challenges calculating, visualizing, and explaining uncertainty to viewers.These challenges alone do not fully explain why authors who acknowledge uncertainty’s benefits still omit it.
- 8 CONCLUSION: The rhetorical model summarizes associations and expectations that make a norm of uncertainty omission seem reasonable to authors.The model addresses the gap between authors’ positive attitudes toward uncertainty and their communicative practice.
- 8 CONCLUSION: The graphical statistical inference model shows that uncertainty reduces, though does not necessarily eliminate, degrees of freedom in viewers’ inferences.This provides a formal basis for analyzing how viewers determine the strength of a visualization’s signal.