Source-linked AI summary
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, Chenhao Tan
TL;DR
AI models can perform worse on out-of-distribution examples, while prior human-AI assistance approaches do not scale well to large volumes of lower-stakes decisions. This paper proposes conditional delegation, evaluates it through content moderation interfaces and randomized experiments across two data distributions, and finds improved model performance with distribution-dependent limits.
Problem
AI performance can drop on out-of-distribution examples, and individual-decision assistance does not scale well to large volumes of lower-stakes decisions.
Method
The paper uses conditional delegation, with humans identifying trustworthy model regions before deployment, and tests keyword-rule interfaces in content moderation across two data distributions.
Results
Conditional delegation improves performance over the model alone and achieves complementary performance in-distribution, but out-of-distribution gains do not surpass the manual rule-based approach.
Takeaways & Limitations
Conditional delegation is a promising alternative paradigm that gives users control over when to trust or distrust AI.
Takeaways & Limitations
The content-moderation experiment is an initial step because crowdworkers are not representative of experienced platform moderators.
Abstract
from arXiv · showhide
Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples. It remains an open question how such imperfect models can be used effectively in collaboration with humans. Prior work has focused on AI assistance that helps people make individual high-stakes decisions, which is not scalable for a large amount of relatively low-stakes decisions, e.g., moderating social media comments. Instead, we propose conditional delegation as an alternative paradigm for human-AI collaboration where humans create rules to indicate trustworthy regions of a model. Using content moderation as a testbed, we develop novel interfaces to assist humans in creating conditional delegation rules and conduct a randomized experiment with two datasets to simulate in-distribution and out-of-distribution scenarios. Our study demonstrates the promise of conditional delegation in improving model performance and provides insights into design for this novel paradigm, including the effect of AI explanations.
1 INTRODUCTION
The paper proposes conditional delegation, in which humans identify trustworthy regions before deployment and AI acts only within those regions. A randomized content-moderation study examines whether this approach improves precision and coverage across distribution settings and how interface explanations affect rule creation.
- Motivation: Distribution shift can substantially reduce AI performance on examples unlike the training data, motivating human-AI collaboration beyond full automation.The study contrasts in-distribution and out-of-distribution AI scenarios.
- Conditional delegation: Conditional delegation lets humans and AI identify trustworthy input regions before deployment, after which AI affects only instances within those regions.Other instances can receive manual review or another model.
- Content-moderation testbed: In content moderation, humans can operationalize trustworthy regions as keyword-based rules after inspecting model predictions on matching comments.The approach combines moderator control with AI-based decisions rather than relying solely on rigid manual rules or the model alone.
- Evaluation goals: Conditional delegation targets precision and coverage: precision limits mistaken or unnecessary moderation actions, while coverage captures how many toxic comments AI can identify.Recall is treated as a lower priority in this moderation workflow because moderators face massive comment volumes and limited time.
- Interface design: The study contributes interfaces supporting keyword search, predicted labels, local rationales, and global model explanations for creating delegation rules.These features are designed for settings where moderators lack labeled comments to quantify model performance directly.
- Findings: In a randomized experiment with 240 crowdworkers, conditional delegation achieved higher precision than the model alone and produced complementary performance for in-distribution AI.For out-of-distribution AI, performance improved but did not surpass the manual rule-based approach; explanations improved efficiency, with weak evidence of better user experiences.
2 RELATED WORK
Related work frames conditional delegation as a form of human-AI collaboration that gives humans proactive control over where AI operates. The paper connects this paradigm to automation, explanations, distribution shift, and scalable content moderation while emphasizing the need to test generalizability across data distributions.
- Human-AI collaboration: Existing human-AI collaboration includes interactive machine learning, AI-assisted decisions, and frameworks allocating subtasks among humans, AI, or both.These paradigms differ in how humans influence models and decisions.
- Human-AI collaboration: The paper introduces proactive human agency, where people control AI use before deployment to partially automate large volumes of decisions rather than assist every individual decision.Conditional delegation identifies trustworthy regions in the input space and is presented as a first controlled experimental study of this paradigm.
- Explainable AI: Local explanations describe rationales for particular predictions, whereas global explanations provide a high-level understanding of how the AI works.The study examines both explanation types as forms of assistance for creating delegation rules.
- Distribution shift: Distribution shift occurs when deployment data differs from training data and often causes performance drops, but its effects have been rarely examined in empirical human-AI collaboration studies.The paper uses distribution setting as a key lens for assessing collaborative performance.
- Distribution shift: Conditional delegation is motivated by distribution shift because humans can identify trustworthy regions, although strong in-distribution AI performance may be especially important for collaboration.The authors hypothesize that out-of-distribution regions are harder to identify because model behavior may be more spurious.
- Content moderation: In content moderation, rule-based tools provide moderator control and efficiency, but conditional delegation combines that control with AI while allowing precision requirements and false-positive tradeoffs to be tuned.The approach is intended for efficiently handling large numbers of comments.
3 AI MODEL
The study develops an interpretable rationale-style model for toxicity classification, evaluates it across in-distribution WikiAttack and out-of-distribution Reddit data, and uses conditional delegation to identify higher-precision regions. Performance drops under distribution shift, while selected words can improve precision relative to the model working alone.
- The model is trained using WikiAttack and evaluated on Reddit to simulate in-distribution and out-of-distribution scenarios.
- F1 drops from about ∼0.8 on WikiAttack to ∼0.6 on Reddit, validating Reddit as an out-of-distribution scenario.
- The rationale-style model identifies sufficient, parsimonious prediction rationales while retaining 15% of tokens for predicted-toxic comments and 2% for predicted-nontoxic comments on average.
- Global explanations list the 15 most frequent rationale tokens, with “cunt” and “retard” among Reddit’s top five but not common in WikiAttack rationales.
- At a 0.93 positive-class probability threshold on Reddit, precision reaches only 0.58, producing two false positives for every three true positives.
- The top-precision words improve over the model alone under conditional delegation, whereas Reddit global explanations exceed model precision for only six words.
- On Reddit, “retard” and “cunt” can yield higher reward through “report all” than conditional delegation because of their precision and coverage.
4 EXPERIMENTAL DESIGN
The experiment compares four interfaces that help participants create keyword-based conditional delegation rules or manual rules. A between-subject study with 240 participants evaluates rule quality, rewards, and user experience across WikiAttack and Reddit.
- Interface design: The tool lets participants search keywords and browse matching comments to determine whether each keyword is a suitable delegation rule.
- Experimental conditions: The four conditions are predicted labels, predicted labels plus local explanations, predicted labels plus local and global explanations, and a manual condition.
- Experimental conditions: Local explanations highlight rationale words in comments predicted toxic, while global explanations list words the model typically uses to determine toxicity.
- Experimental conditions: The manual condition simulates AutoModerator by allowing participants to browse comments and create “report all” rules without model-related information.
- Study procedure: Participants complete tutorials, attention checks, toxicity trials, and rule creation, with six participants removed for accuracy at or below 50%.
- Study procedure: The study uses a between-subject design with 30 participants per condition, totaling 120 participants for each distribution type and 240 overall.
- Study procedure: The bonus reward combines precision and coverage by paying for correctly reported toxic comments and penalizing mistakenly reported nontoxic comments.
5 RESULTS
Conditional delegation improved precision over the model working alone when participants created rules, especially for in-distribution WikiAttack, while gains were harder to achieve on out-of-distribution Reddit. Explanations produced mixed effects: global explanations improved rule-generation efficiency but could reduce coverage and precision, whereas subjective experiences were generally positive with limited differences between conditions.
- 5.1 Efficacy: p<0.001 differences showed that lay participants’ conditional-delegation rules achieved higher average and union precision than the model working alone on both WikiAttack and Reddit.On WikiAttack, the model already outperformed the manual condition, and conditional delegation further improved precision.
- 5.1 Efficacy: Significant effects of distribution type, experimental condition, and their interaction (p<0.001) showed clear performance differences between in-distribution WikiAttack and out-of-distribution Reddit.The distribution-type effect was the most salient in the two-way ANOVA.
- 5.1 Efficacy: On WikiAttack, participants achieved high precision and coverage with predicted labels, but on Reddit the lower-performing model made improvement over manual rules more challenging.Coverage was reflected by union precision and reward, and the WikiAttack outcome constituted complementary performance.
- 5.1 Efficacy: Explanations did not significantly improve conditional-delegation performance, while global explanations slightly reduced union precision and reward by encouraging high-coverage, lower-precision rules.The paper specifically identifies “fuck” as a global-keyword example that could draw participants toward lower-precision choices on Reddit.
- 5.2 Efficiency and Engagement: p=0.008 for rules per minute indicated an efficiency effect: global explanations produced the highest efficiency compared with predicted labels only.Global explanations also produced greater overlap between human-created rules and frequent words in model rationales, suggesting direct hints for possible rules.
- 5.2 Efficiency and Engagement: Participants were more engaged with conditional delegation than manual-rule creation, while explanations, especially global explanations, traded some efficiency and performance benefits against possible rule-selection errors.Across subjective measures, participants reported relatively positive experiences, with limited differences between conditions; local explanations were associated with the best experience when the model performed well.
6 DISCUSSION
The study finds conditional delegation promising for improving model performance, but its benefits depend on distribution and interface design. Explanations can improve efficiency while also biasing rule selection, and the study identifies important limits for generalization and future design.
- The promise of conditional delegation: Conditional delegation can outperform the model working alone, but it does not consistently outperform manual rules across in-distribution and out-of-distribution settings.The out-of-distribution model’s performance disadvantage was not adequately compensated by conditional delegation.
- The priming effect of explanations: Predicted labels were generally sufficient for participants to create delegation rules, while explanations improved efficiency but could encourage high-frequency, lower-precision rules.Global explanations slightly hurt performance with out-of-distribution AI when participants adopted explanation keywords without carefully examining model behavior.
- Implications and limitations: The study’s design space remains incomplete, including the need to examine input-and-output delegation, rule combinations, deployment monitoring, and updates after model changes.The experiment also used crowdworkers who created about 10 rules, so further experiments with expert users are needed.
- The effect of distribution shift: In-distribution results on WikiAttack were complementary, whereas results on Reddit did not show the same benefit.The WikiAttack–Reddit contrast also combines differences in model performance and comment characteristics, complicating its interpretation as distribution shift alone.
- The priming effect of explanations: Local explanations appeared less likely than global explanations to prime users, although they still improved efficiency.The authors suggest scattered highlights may reduce the salience of local explanations.
- Implications and limitations: The experiment is an initial step because crowdworkers are not representative of experienced moderators and practical moderation may use historical decision data.The authors identify participatory design and future work as ways to develop more suitable interfaces and features.