Source-linked AI summary
In Automation We Trust: Investigating the Role of Uncertainty in Active Learning Systems
Michael L. Iuzzolino, Tetsumichi Umada, Nisar R. Ahmed, Danielle A. Szafir
TL;DR
Active learning query policies optimize information gain but may not account for how training affects analyst trust, which matters when analysts later rely on automated classifications. This study tests five query policies and seven uncertainty visualizations, finding that low-confidence-only policies are associated with more label changes, lower trust, and perceptions of greater deception.
Problem
Model-centric active learning selects uncertain examples for information gain but does not account for effects on analysts’ confidence and trust in classifiers they use after training.
Source: §1 · p. 1Method
The study paired five active-learning query policies with seven uncertainty visualizations and measured analysts’ label changes and reported perceptions of the classifier.
Source: §1 · p. 2, §5.2 · p. 5Results
Takeaways & Limitations
The findings offer preliminary considerations for aligning active-learning policies and visualizations with application needs for analyst trust and behavior.
Source: §8 · p. 8The study used simple query policies for binary image classification and a binary task requiring no expert knowledge.
Source: §7.1 · p. 8
Abstract
from arXiv · showhide
We investigate how different active learning (AL) query policies coupled with classification uncertainty visualizations affect analyst trust in automated classification systems. A current standard policy for AL is to query the oracle (e.g., the analyst) to refine labels for datapoints where the classifier has the highest uncertainty. This is an optimal policy for the automation system as it yields maximal information gain. However, model-centric policies neglect the effects of this uncertainty on the human component of the system and the consequent manner in which the human will interact with the system post-training. In this paper, we present an empirical study evaluating how AL query policies and visualizations lending transparency to classification influence trust in automated classification of image data. We found that query policy significantly influences an analyst's trust in an image classification system, and we use these results to propose a set of oracle query policies and visualizations for use during AL training phases that can influence analyst trust in classification.
1 INTRODUCTION
Active learning can use analyst labels to address sparse labeling, but conventional uncertainty-driven querying prioritizes model information gain without accounting for how training may shape analyst trust. This study tests query policies and uncertainty visualizations, finding that query policy is a prominent factor in trust and that high-confidence-only policies reduce label changes.
1 INTRODUCTION
- Analysts continue to rely on classifiers after training, yet model-centric query strategies optimize information gain without accounting for effects on analyst confidence and trust.Presenting only low-confidence examples may prime analysts to mistrust the system.Source: §1 · p. 1
- The controlled study examined how query policies and uncertainty information during active learning influence analyst agreement with and trust in autonomous classifications.It used five query policies paired with seven uncertainty visualizations.Source: §1 · p. 2
- Policies restricted to high-confidence images produced significantly fewer label changes, while low-confidence-only policies were associated with more label changes and lower trust.The paper also reports that low-confidence-only policies made analysts perceive the system as more deceptive.Source: §1 · p. 2
- Policies progressing from low to high confidence or high to low confidence may create anchoring effects that influence analysts’ trust.Source: §1 · p. 2
- Query policy may be the predominant factor affecting analyst trust when query policies and visualizations are compared for image classification.Source: §1 · p. 2
2 RELATED WORK
Prior work covers active learning, analyst reliance on automation, uncertainty visualization, and cognitive priming, but offers limited empirical understanding of how people’s interactions with classifiers shape use of their outputs. This study situates query policy and uncertainty displays within those lines of research.
2 RELATED WORK
- Visual analytics has communicated classifier state, performance, and debugging information, but empirical understanding of how interactions with classifiers influence analysts’ use of outputs remains limited.Source: §2 · p. 2
2.1 Active Learning and Visual-Interactive Labeling
- Active learning selects instances for algorithmic information gain, while Visual-Interactive Labeling lets analysts select instances using domain knowledge; prior comparisons found different efficiency and effectiveness strengths.Active learning outperformed VIL in effectiveness, while VIL improved classification efficiency and was particularly effective for simple tasks and bootstrapping.Source: §2.1 · p. 2
2.2 Automation Bias
- Automation bias is under- or over-reliance on automation, and trust can shift with valid or invalid recommendations; trust decrements may exceed increments.System errors can reverse an analyst’s correct assessment.Source: §2.2 · pp. 2–3
2.3 Uncertainty Visualization
- Uncertainty displays can affect analysts’ reasoning and decisions, motivating study of how visualizing classification uncertainty during active learning influences trust.Source: §2.3 · p. 3
2.4 Priming and Anchoring Effects
- Priming describes decisions influenced by a preceding perceptual stimulus, while anchoring is overreliance on an initial piece of information; prior visualization work found judgments depended on earlier exposure.Source: §2.4 · p. 3
3 MOTIVATION & RESEARCH QUESTIONS
The study asks whether confidence-based query policies and uncertainty visualizations affect trust in autonomous classification, and whether visualizations modify policy effects. It tests five query policies and seven displays spanning different degrees of classification transparency.
3 MOTIVATION & RESEARCH QUESTIONS
- The research questions ask whether system-confidence query policies affect analyst confidence, whether confidence visualizations influence trust, and whether visualizations alter policy effects.A two-phase study established human-derived confidence levels, then measured participant trust using agreement and self-reported perceptions.Source: §3 · p. 3
3.1 Query Policies
- Five policies sample images randomly, select only low- or high-confidence images, or order sampled images from low to high or high to low confidence.The ordered policies were designed to explore priming and anchoring effects, where early examples may shape later evaluation despite subsequent information.Source: §3.1 · p. 3
3.2 Visualizations
- Seven visualization conditions span no uncertainty information, numeric and bar-chart confidence, contextual scatter plots, and t-SNE displays of images in embedded feature space.The displays provide increasing transparency into classification certainty and, in some cases, the image’s context within the population.Source: §3.2 · pp. 3–4, Table 1 · p. 4
4 DATA COLLECTION STUDY
A preliminary human image-classification study measured confidence to create a ground-truth confidence dataset for simulating a classifier in the primary study. Participants classified dog-breed images at different downsampling rates.
4.1 Stimuli
- The stimuli came from Oxford-IIIT Pet Dataset dog breeds, with three visually challenging breed pairs and four max-pooling downsampling rates generating 2,400 images.Twenty-five original images per breed were sampled; rates were 2, 4, 7, and 10.Source: §4.1 · p. 4, Figure 2 · p. 4
4.2 Experimental Design
- The experiment varied six dog breeds between participants and four within-participant downsampling rates, and measured population accuracy as the proportion of correct labels.Source: §4.2 · p. 4
4.3 Experimental Task
- Participants learned two-breed labels in a tutorial, then labeled 25 images at each of four downsampling rates, with four full-resolution attention checks.They had to correctly label five tutorial images from each breed before proceeding; the task totaled 104 labeling trials.Source: §4.3 · p. 4
4.4 Participants
- Of 178 Mechanical Turk participants, 38 were excluded, leaving 120 valid participants.Exclusion criteria included labeling all images identically or failing four attention checks.Source: §4.4 · pp. 4–5
4.5 Results
- Mean confidence was 82% for dataset A, 71% for dataset B, and 81% for dataset C; the datasets were split into low- and high-confidence image pools.The mean split produced 104, 108, and 99 low-confidence images and 96, 92, and 101 high-confidence images for datasets A, B, and C, respectively.Source: §4.5 · p. 5
5 PRIMARY STUDY
The primary study tested how five active-learning query policies and seven uncertainty visualizations related to analysts’ trust in image classification. Participants trained and evaluated a simulated classifier in a Mechanical Turk experiment.
5.1 Stimuli
- Stimuli came from three breed-pair datasets, with human classification accuracy of at least 79% defining high confidence; the study used seven visualizations of classification uncertainty.The visualizations encoded breed with color; the two scatterplots and t-SNE also represented confidence.Source: §5.1 · p. 5
5.2 Experimental Design
- The full-factorial design crossed five query policies with seven visualizations and measured label disagreement, per-image trust, and post-survey trust, deception, and collaboration.ANCOVAs included question order and machine-learning experience as covariates, with Tukey HSD post-hoc tests.Source: §5.2 · p. 5
5.3 Experimental Task
- Participants trained on 25 policy-selected images with classification labels and visualizations, then evaluated 25 classifications without visualizations and rated their confidence and agreement.The task also included a tutorial, post-survey, and demographics phase.Source: §5.3 · p. 6
5.4 Participants
- The second study recruited 630 Mechanical Turk participants and retained 605 after exclusions for inverted training labels or inconsistent agreement responses.Source: §5.4 · p. 6
- Figure 4 reports significantly more disagreement with system classifications under low-confidence ALC than under other policies, including ALH.Source: Figure 4 · p. 6
6 RESULTS
Query policy shaped label changes and trust during and after evaluation, while visualization effects appeared through interactions with query policy. AHC yielded fewer label changes and higher trust than ALC.
6.1 Label Changing
- Policy–visualization interactions were significant, with several AHC–visualization combinations producing fewer label flips than selected ALC combinations.The comparisons include AHC paired with five visualization types and ALC paired with selected visualization types.Source: Figure 5 · p. 6, §6.1 · p. 7
6.2 System Trust: During Evaluation
- During evaluation, AHC yielded higher trust (µ = 5.89,σ = 0.029) than RAND (µ = 5.68,σ = 0.028), ALC (µ = 5.23,σ = 0.028), LtH (µ = 5.62,σ = 0.029), and HtL (µ = 5.51,σ = 0.030).Trust also differed significantly between query policies and visualizations interacted with policy.Source: Figure 6 · p. 7, §6.2 · p. 7
6.1 Label Changing
- Participants changed labels significantly more under ALC (µ = 22%,σ = 0.69%) than AHC (µ = 11%,σ = 0.70%).AHC also produced significantly fewer label changes than every other policy.Source: §6.1 · p. 7
6.3 System Trust and Deception: Post Evaluation
- Post-evaluation, ALC was rated more deceptive than AHC and HtL, and less trustworthy than RAND, AHC, LtH, and HtL.The post-evaluation policy effects were significant for both deception and trust.Source: Figure 8 · p. 7, Figure 7 · p. 7, §6.3 · p. 7
7 DISCUSSION
The discussion connects query policy with analysts’ behavior and trust, contrasting ALC’s algorithmic information-gain focus with human responses. It also identifies limits in the simulated, binary, confidence-visualization study.
7 DISCUSSION
- Seeing high-confidence classifications first may anchor evaluation trust; RAND’s higher trust than LtH and HtL is an exception to that proposed pattern.The authors suggest blocked trials may emphasize similarities, while interleaving may focus attention on differences.Source: §7 · p. 8
- Although ALC is the predominant active-learning policy for algorithmic information gain, it affected both objective behavior and subjective perceptions after training.The discussion identifies ALC’s effects on label-changing behavior and system perceptions.Source: §7 · p. 8
- Uncertainty visualizations affected behavior and perceptions only through secondary, policy-specific interactions; the authors judged query policy more influential overall.Some ALC pairings with Single Bar Plot, Scatter Plot with Density, or t-SNE may increase label changes.Source: §7 · p. 8
- AHC had positive effects on objective and subjective measures, but querying only high-confidence instances minimizes information gain; RAND, LtH, and HtL may better balance efficiency and long-term trust.The authors note this trade-off may matter in domains such as healthcare and aviation, where analysts work with automation over long periods.Source: §7 · p. 8
7.1 Limitations & Future Work
- The study uses simulated policies, a simple binary task without expert knowledge, and confidence-only visualizations rather than displays of algorithm state.The authors identify integrating effects into actual machine-learning algorithms and testing multiclass settings as future work.Source: §7.1 · p. 8
8 CONCLUSION
The conclusion links low-confidence querying with lower trust, more label changes, and greater perceptions of deception, offering preliminary considerations for active-learning systems seeking long-term analyst trust.
8 CONCLUSION
- Low-confidence query policies reduced analyst trust, increased classification label changes, and led analysts to perceive the system as more deceptive.The authors frame these findings as preliminary considerations for aligning trust and behavior with application needs.Source: §8 · p. 8