Source-linked AI summary
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Vivian Lai, Han Liu, Chenhao Tan
TL;DR
The paper addresses limited evidence that real-time explanations help humans understand unfamiliar patterns embedded in machine-learning models. It introduces model-driven tutorials before prediction and evaluates them through randomized human-subject experiments in deceptive review detection. Tutorials improve human performance with and without real-time assistance, while simple-model explanations are more useful to humans than deep-learning explanations.
Problem
Real-time explanations have shown limited effectiveness in improving human performance, motivating training that helps people understand unfamiliar model patterns before making decisions.
Method
The paper evaluates guidelines from scientific papers and explanation-bearing training examples in large-scale, randomized, pre-registered human-subject experiments using deceptive review detection.
Results
Tutorials improve human performance with and without real-time assistance, and simple-model tutorials and explanations are more useful to humans than deep-learning ones.
Takeaways & Limitations
Real-time assistance is crucial for further improving performance in challenging tasks, while simple models provide more interpretable tutorials and explanations for humans.
Takeaways & Limitations
The experiments use a random train-test split, so their generalization setting may differ from testing distributions that differ from training.
Abstract
from arXiv · showhide
To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans. While existing approaches focus on explaining machine predictions with real-time assistance, we explore model-driven tutorials to help humans understand these patterns in a training phase. We consider both tutorials with guidelines from scientific papers, analogous to current practices of science communication, and automatically selected examples from training data with explanations. We use deceptive review detection as a testbed and conduct large-scale, randomized human-subject experiments to examine the effectiveness of such tutorials. We find that tutorials indeed improve human performance, with and without real-time assistance. In particular, although deep learning provides superior predictive performance than simple models, tutorials and explanations from simple models are more useful to humans. Our work suggests future directions for human-centered tutorials and explanations towards a synergy between humans and AI.
INTRODUCTION
Existing real-time explanations often have limited effects on human decisions, especially when models discover patterns unfamiliar to people. This paper introduces model-driven tutorials before prediction and evaluates them in deceptive review detection, finding modest gains, stronger gains with real-time assistance, and greater human usefulness from simple-model explanations.
- Real-time explanations have shown limited effectiveness in improving human performance on challenging prediction tasks.
- The paper distinguishes model use for emulating human intelligence from discovering patterns in social processes, where explanations may enhance human knowledge.
- In deceptive review detection, patterns such as the association between “Chicago” and deceptive reviews can be difficult for humans to comprehend from real-time highlights alone.
- The authors introduce a training phase with model-driven tutorials, including scientific-paper guidelines and explanation-bearing examples selected from training data.
- Human performance after tutorials reaches approximately 60%, compared with over 85% for machine-learning models, although tutorials outperform no-training control conditions.
- Real-time assistance raises human performance above 70%, while simple-model tutorials and explanations outperform those from deep learning models.
- Participants found tutorials useful, but example-driven training could encourage memorization or pattern matching rather than understanding why features predict deception.
EXPERIMENT 2: HUMAN PERFORMANCE WITH VARYING REAL-TIME ASSISTANCE AFTER TUTORIALS
Experiment 2 tested whether real-time assistance improves human deceptive-review detection after tutorial training. Signed highlights improved performance, while additional priming added little, and gains remained below machine performance.
- Results: After tutorial training, real-time assistance generally improved human performance, except for unsigned highlights.The treatment effect was statistically significant overall (η2 = 0.23; p = 5.15×10−25).
- Results: Signed highlights outperformed unsigned highlights, indicating that directional information was useful for human decisions.The difference between unsigned and signed highlights was statistically significant (p < 0.001).
- Results: Additional information beyond signed highlights did not produce significant further gains.Comparisons between signed highlights and stronger-priming setups were not significant.
- Results: Human performance increased from approximately 60% without assistance to approximately 70% with real-time assistance, still below machine performance above 85%.The paper characterizes this improvement as substantial but limited relative to the models.
- Results: With effective assistance, 8.1% of participants exceeded machine performance, compared with 0.4% after tutorial training alone.This difference was statistically significant (p < 0.001).
EXPERIMENT 3: THE EFFECT OF MODEL COMPLEXITY AND METHODS OF DERIVING EXPLANATIONS
Experiment 3 examined whether model complexity and explanation methods affect human performance after tutorials. Explanations derived from simple SVM models consistently supported better human performance than explanations from BERT, while training helped across methods.
- Results: SVM-derived explanations consistently produced better human performance than explanations derived from deep BERT models.The comparison concerns highlighted important words used as real-time explanations.
- Results: Training improved human performance for every explanation method tested.The experiment compared training versus no training with SVM coefficients, BERT attention, and BERT LIME.
- Results: Tutorials and explanation methods both had statistically significant effects on human performance.The two-way ANOVA found an effect of tutorials (η2 = 0.049; p = 1.50×10−7) and explanation method (η2 = 0.13; p = 4.66×10−16).
- Results: Tutorials were useful but human performance remained approximately 70% after training with real-time assistance.The authors interpret this as improvement that is still limited relative to the task’s machine performance.
- Results: Participants particularly favored SVM-derived tutorials, with 85% finding them useful.SVM tutorials were rated more useful than both BERT attention and BERT LIME tutorials (p < 0.001 for each comparison).
- Discussion: The authors identify a need for tutorials that help humans form and test theories about why features matter, rather than only highlighting features.They also point toward interactive explanations, such as querying word importance or evaluating counterfactual reviews.
- Limitations: The study’s conclusions are bounded by its challenging deceptive-review task, random train-test split, and nonrepresentative participant samples.Participants were university students or U.S. Mechanical Turkers, and effectiveness may depend on the sample population and task choice.
PREVIEW OF THE SUPPLEMENTARY VIDEO
The supplementary video previews the tutorial interfaces and provides starting times for each tutorial type. It covers guidelines, random examples, explanation-based examples, spaced repetition, and BERT-based tutorials.
- Viewing guide: Viewers can skip tutorial-interface screenshots if they watch the supplementary video instead.The video’s timestamps are provided to support skimming.
- Video contents: The supplementary video includes tutorial types ranging from guidelines and random examples to SP-LIME, spaced repetition, and BERT-based tutorials.The listed starting times run from 00:08 for guidelines to 03:15 for BERT + LIME.
EXPERIMENT INTERFACES
The paper’s figures show interfaces for the three experiments: tutorial formats, real-time assistance conditions, and explanation highlights derived from SVM or BERT methods.
- Experiment 1: Experiment 1 interfaces show guidelines, random examples, SP-LIME examples, and selected examples combined with guidelines.These correspond to the tutorial conditions used for training.
- Experiment 2: Experiment 2 interfaces vary real-time assistance from no assistance to signed highlights, predicted labels, guidelines, and an accuracy statement.The figures represent progressively richer assistance configurations.
- Experiment 3: Experiment 3 interfaces highlight top features derived from SVM, BERT attention, and BERT LIME.These displays visualize the different explanation methods compared in the experiment.
Experiment Details
The experiments included participants with varied ages and educational backgrounds, excluded noncompliant submissions, and measured overall and prediction-phase time after outlier filtering.
- Participants: Participants spanned four age groups and multiple educational backgrounds in all three experiments.Experiment 1 included 480 participants; demographic counts for Experiments 2 and 3 are also reported.
- Participants: Participants who failed to complete the full task or submit a unique survey code were excluded.
- Timing: Timing outliers were identified and filtered separately for each experiment using an interquartile-range procedure.
- Timing: Average time was reported separately for each experimental setup and for the prediction phase across all three experiments.
TRUST ANALYSIS
Trust did not differ significantly across experiment 2 conditions, while participants trusted correct machine predictions more than incorrect ones.
- Differences in human trust across experiment 2 conditions were not statistically significant.The results suggest that guidelines and accuracy statements did not significantly increase trust in machine learning models.
- Human trust was significantly higher for correct machine predictions than for incorrect predictions.
Analysis of Free Responses from Turkers
Participants described the tutorial as useful for noticing linguistic nuances and potential deception, while reporting difficulty understanding why particular features indicated deceptive or genuine reviews.
- Participants felt the tutorial was useful but struggled to understand why certain features were deceptive or genuine.Free responses confirmed the qualitative-study findings.
HUMAN PERFORMANCE GROUPED BY DEMOGRAPHICS
Across the three experiments, the paper reports no clear demographic trend in human accuracy by gender, education background, review-writing frequency, or age.
- No clear trend linked human accuracy to gender, education background, review-writing frequency, or age across the experiments.
- Experiment 1: Experiment 1 grouped human accuracy by gender, age, education background, and review-writing frequency.
- Experiment 2: Experiment 2 grouped human accuracy by gender, age, education background, and review-writing frequency.
- Experiment 3: Experiment 3 grouped human accuracy by gender, age, education background, and review-writing frequency.
ATTENTION-CHECK DESIGN
The study revised its attention checks after a participant appeared unaware of the deceptive-review definition. The updated design lets participants confirm their answers, whereas the outdated design immediately disqualified wrong answers.
- ATTENTION-CHECK DESIGN: A participant’s confusion about whether reviews were generated by computers or written by people prompted changes to the attention checks.The interviewer directed the participant to dialogue-box instructions, which the participant said they had not noticed.
- ATTENTION-CHECK DESIGN: The outdated attention-check design did not allow participants to confirm their answers.Participants selecting the wrong answer were disqualified immediately.
- ATTENTION-CHECK DESIGN: The updated attention-check design allows participants to confirm their answers.
EXIT SURVEY
The appendix includes exit surveys for the experimental setups in Experiments 1, 2, and 3. These surveys cover control, guidelines, example-based, and training or non-training conditions.
- EXIT SURVEY: Figures 43–45 show exit surveys for the experimental setups in Experiment 1.The examples survey covers random, SP-LIME, and spaced-repetition conditions; question 7a changes for SR+guidelines.
- EXIT SURVEY: Figures 47 and 48 show exit surveys for the non-training and training experimental setups in Experiment 3.
- EXIT SURVEY: Figure 46 shows the exit survey for the experimental setup in Experiment 2.