PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation

Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, Chenhao Tan

arXiv:2204.11788v1cs.AIcs.HCcs.LG

TL;DR

Decision-by-decision AI assistance is not scalable for large volumes of lower-stakes decisions, leaving open how people can collaborate effectively with imperfect models. This paper tests conditional delegation, in which people define trustworthy regions for AI, and finds improved performance in both distribution settings, with complementary human-AI performance in-distribution but not out-of-distribution.

  • Problem

    How to collaborate effectively with imperfect AI remains open, while decision-by-decision assistance is not scalable for tasks involving many decisions.

    Source: §1 · p. 1

  • Method

    The paper tests conditional delegation, where people define trustworthy AI regions before deployment, using content-moderation interfaces and a randomized experiment across in- and out-of-distribution datasets.

    Source: §1 · p. 1, §4.1 · p. 8, §5 · p. 11

  • Results

    Conditional delegation improved model performance in both settings and achieved complementary performance in-distribution, but did not overcome out-of-distribution AI's disadvantage versus manual rules.

    Source: §1 · p. 3, §6 · p. 15

  • Takeaways & Limitations

    The contrasting in- and out-of-distribution outcomes underscore the importance of accounting for distribution shift when evaluating human-AI collaboration.

    Source: §1 · p. 3, §6 · p. 15

    Because crowdworkers created only about 10 rules, the potential of conditional delegation with expert users still requires validation.

    Source: §6 · p. 15

Abstract

from arXiv · show

Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples. It remains an open question how such imperfect models can be used effectively in collaboration with humans. Prior work has focused on AI assistance that helps people make individual high-stakes decisions, which is not scalable for a large amount of relatively low-stakes decisions, e.g., moderating social media comments. Instead, we propose conditional delegation as an alternative paradigm for human-AI collaboration where humans create rules to indicate trustworthy regions of a model. Using content moderation as a testbed, we develop novel interfaces to assist humans in creating conditional delegation rules and conduct a randomized experiment with two datasets to simulate in-distribution and out-of-distribution scenarios. Our study demonstrates the promise of conditional delegation in improving model performance and provides insights into design for this novel paradigm, including the effect of AI explanations.

1 INTRODUCTION

The paper proposes conditional delegation, in which people define where an AI can be trusted before deployment, as a human-AI approach for high-volume moderation. It tests the approach across data-distribution settings and examines interfaces for creating delegation rules.

1 INTRODUCTION

  • Per-case AI assistance is poorly scalable for high-volume, low-stakes moderation; moderators already use manual automation rules, but rigid rules miss informal non-toxic language.Conditional delegation offers a way to use AI without relying on it blindly for every comment.Source: §1 · pp. 1–2
  • Before deployment, humans identify trustworthy regions; afterward, AI affects decisions only within those regions, while people retain control over how other cases are handled.Other cases can be routed to manual review or another model.Source: §1 · p. 1
  • In moderation, humans create keyword rules for trustworthy regions, and evaluation prioritizes precision and coverage over recall given volunteers’ limited capacity to handle massive comment volumes.The study assumes comments outside the keyword rules are ignored, as in the manual-rule workflow.Source: §1 · p. 2
  • The rule-creation interface supported keyword search and showed predicted labels, local rationales, and global model overviews; explanations improved rule-finding efficiency, with weak evidence of better user experiences.The interface was designed for moderators who may lack labeled comments to quantify model performance on their data of interest.Source: §1 · pp. 2–3
  • With in-distribution AI, conditional delegation exceeded both AI and human performance; with out-of-distribution AI, it improved the model but remained below manual rules.The in-distribution AI already outperformed the manual rule-based approach; the out-of-distribution result did not overcome AI’s performance disadvantage relative to manual rules.Source: §1 · p. 3
  • A randomized experiment with 240 Mechanical Turk participants compared conditional delegation with manual rules and standalone AI across community-specific and shifted-data models.The study examined whether people could improve model precision with good coverage and how performance and user experience varied across the two AI settings.Source: §1 · pp. 2–3

2 RELATED WORK

The related work positions conditional delegation as a mixed-initiative alternative for large-scale moderation: humans identify trustworthy regions before deployment and control where AI automates decisions.

2.1 Human-AI Collaboration

  • Unlike interactive learning requiring model updates, per-decision AI assistance, and task-allocation frameworks, this work gives humans proactive predeployment control to delegate trustworthy input regions.The study examines whether this approach can outperform AI-only and manual decision-making.Source: §2.1 · p. 3

2.2 AI explanations for human-AI interaction

  • Prior HCI studies find explanations can improve understanding but have mixed effects on trust, satisfaction, and adoption; this study tests local and global explanations as cues for keyword-rule creation.The authors hypothesize that prediction keywords can suggest rules and help people judge a rule’s effectiveness.Source: §2.2 · p. 4

2.3 Distribution Shift and Experimental Studies on Out-of-distribution Examples

  • Prior collaboration research rarely studies distribution shift, although studies report OOD changes in collaboration and reliance; here, strong in-distribution AI performance matters, while spurious OOD behavior may hinder identifying trustworthy regions.The cited studies found complementary performance more plausible OOD and, separately, greater human reliance on AI where its performance was worse.Source: §2.3 · p. 4

2.4 Content Moderation

  • For moderation’s high-volume workload, conditional delegation combines moderator-controlled rules with AI, letting moderators specify when to trust models and tune precision and false-positive–false-negative tradeoffs.This responds to concerns about hard-coded rule errors and full automation’s risks, including bias and differences in community values.Source: §2.4 · p. 4

3 AI MODEL

The section develops a rationale-producing moderation model trained on Wikipedia comments and applied to Reddit comments as an out-of-distribution test. Its analyses show that conditional delegation can improve precision with suitable keyword rules, while finding such rules is challenging out of distribution.

3.1 Model Development

  • The model was trained on 70k Wikipedia comments and evaluated on 23K held-out comments, then applied to 22K Reddit comments as an out-of-distribution dataset.Its rationale-style architecture produces predictions and token-level explanations by selecting rationales for a predictor; the authors chose recall weight 0.5 for competitive performance on both datasets.Source: §3.1 · p. 5, Figure 3 · p. 5
  • The model retained 2% of tokens for predicted-nontoxic comments versus 15% for predicted-toxic comments, while producing rationales the authors found sensible but that may not align with human reasoning.The rationales are sufficient for the model’s predictions, and conditional delegation can be used with other models.Source: §3.1 · pp. 5–6
  • At a 0.93 positive-class threshold on Reddit, precision reached only 0.58; model F1 also fell from about 0.8 on WikiAttack to about 0.6 on Reddit.Even this conservative Reddit setting produced two false positives for every three true positives.Source: §3.1 · pp. 5–6, Figure 4 · p. 6

3.2 Performance of Individual Words

  • For top-precision words, conditional delegation outperformed the model alone and generally beat “report all” on both datasets, though the advantage over “report all” was smaller on Reddit.These results suggest that well-chosen keyword rules can outperform both approaches, but depend on users finding suitable words.Source: Figure 5 · p. 6, §3.2 · p. 7
  • Most frequent rationale words exceeded model-alone precision for most WikiAttack global explanations, but for only six Reddit words: “cunt”, “retard”, “faggot”, “bitch”, “she”, and “her”.The top-15 rationale tokens differed between datasets; “cunt” and “retard” were uncommon in WikiAttack rationales but among the top five on Reddit.Source: §3.1 · p. 6, Figure 6 · p. 7, §3.2 · p. 7
  • On Reddit, “retard” and “cunt” produced high reward even under “report all,” but identifying such effective words relied on oracle access to fully labeled data.Reward combines the number of reported toxic comments and non-toxic comments; the advantage of conditional delegation over “report all” was smaller on Reddit and disappeared for these two words.Source: Figure 7 · p. 7, §3.2 · p. 7

4 EXPERIMENTAL DESIGN

The study tested whether interface support helps people create keyword-based conditional-delegation rules, using four experimental conditions across in- and out-of-distribution settings. It measured rule efficacy, engagement, efficiency, and participants’ subjective perceptions.

4.1 Experimental Conditions and Interface Design

  • The four conditions varied access to model information: predicted labels, local explanations, both local and global explanations, or a manual no-model baseline.Participants searched and browsed comments to assess candidate keywords, then added rules to a list; the interface displayed total and predicted-toxic matched comments.Source: §4.1 · p. 8

4.2 Study Design and Procedure

  • A 240-person between-subject experiment assigned 30 participants to each condition in each distribution type, with a target power of 0.95 to detect a 0.1 precision effect.Participants recruited from Amazon Mechanical Turk completed consent and instructions, an interface tutorial, the rule-making task, and an exit survey; they received incentives based on correctly reported toxic and mistakenly reported nontoxic comments. Most participants reported satisfaction with the task design and interface.Source: §4.2 · pp. 8–10

4.3 Evaluation Measures

  • Efficacy was assessed with average and union precision for rule sets, while logged actions measured engagement and elapsed time and rules per minute measured efficiency.The study also collected subjective workload, confidence, and perceived understanding on five-point Likert scales; precision was treated as a more reliable efficacy measure than the volatile incentive reward.Source: §4.3 · pp. 10–11

5 RESULTS

Conditional delegation improved moderation performance over the model working alone, but its advantage over manual rules depended on the model’s base performance. Participants were more engaged, explanations—especially global explanations—improved rule-creation efficiency, and subjective experience was generally positive without strong condition differences.

5.1 Efficacy

  • Conditional delegation improved precision over the model working alone in both datasets; it further improved WikiAttack performance, while gains over manual rules were less reliable on Reddit.For WikiAttack, manual rules performed worse than conditions with delegation support; on Reddit, global explanations performed worse than manual rules for union precision and reward.Source: §5.1 · p. 11, Figure 11 · p. 12
  • Explanations did not significantly improve efficacy, and global explanations could mislead participants toward high-coverage, lower-precision rules.Participants using delegation support selected “fuck” more often on WikiAttack, where it was high precision, but it was less precise on Reddit; conditional delegation could also achieve reward by trusting the model on terms beyond swearing words.Source: §5.1 · p. 12

5.2 Efficiency and Engagement

  • Conditional delegation increased engagement over manual rule creation, especially in actions, while rule counts stayed near the required minimum.Global explanations yielded 12.3 rules versus 10.6 for the manual condition, a statistically significant difference.Source: §5.2 · pp. 12–13, Figure 13 · p. 12

5.3 Subjective Perception

  • Participants reported generally positive subjective experiences, but workload, confidence, and perceived understanding showed few strong differences between conditions.On WikiAttack, local explanations were associated with trends toward lower workload and greater confidence, while explanation-related trends did not consistently hold on Reddit.Source: §5.3 · pp. 13–14

5.2 Efficiency and Engagement

  • Explanations improved rule-creation efficiency, with global explanations producing the highest rules-per-minute rate and greater overlap with rationale keywords.Predicted-label-only participants spent the most time and produced the fewest rules per minute; global explanations supplied keyword hints that helped participants create rules.Source: §5.2 · p. 13, Figure 14 · p. 14

6 DISCUSSION

The discussion presents conditional delegation as promising but context-dependent: it can outperform a model working alone, while distribution shift and explanation-induced priming limit its benefits. The authors frame content moderation as an initial testbed and call for further validation across users, models, and applications.

The promise of conditional delegation

  • Conditional delegation can outperform a model working alone even with non-expert crowdworkers, but whether it beats manual rules varies by distribution and out-of-distribution AI’s disadvantage is not adequately compensated.The study describes this as an initial demonstration of the paradigm’s promise, not a validated approach for every application.Source: §6 · pp. 14–15

The priming effect of explanations

  • Predicted labels with keyword search were generally sufficient; explanations improved efficiency, but global explanations slightly hurt out-of-distribution performance by priming adoption of frequent, not necessarily high-precision keywords, unlike local explanations’ weaker priming.The authors suggest investigating explanation methods that use a proxy for precision and supports such as reminders to check wrong predictions associated with chosen keywords.Source: §6 · pp. 15–16

Limitations

  • The study explored only input-based trustworthy regions and about 10 rules from non-experts; further work should test experts, more rules and rule combinations, and monitor or update conditions after deployment.The authors also call for empirical validation of alternative underlying models and explanation methods, and research into guarding against human bias that conditional delegation could introduce.Source: §6 · pp. 15–16

The effect of distribution shift

  • Complementary performance was achieved in-distribution on WikiAttack but not out-of-distribution on Reddit; differences in model performance and comment characteristics make distribution shift a non-monolithic experimental challenge.The authors stress that evaluating out-of-distribution contexts is important for understanding the generalizability and applicable scope of human-AI collaboration.Source: §6 · p. 15

The promise of conditional delegation

  • Conditional delegation could let users apply domain knowledge or fairness judgments to mitigate AI bias, but poorly conceived judgments may introduce human bias that requires safeguards.The authors encourage future work to understand and develop ways to guard against the impact of human biases.Source: §6 · p. 15

Implications on content moderation

  • Crowdworkers created keyword rules with greater precision than the model alone, but content moderation remains a testbed: moderators differ from crowdworkers, and real workflows involve historical decisions, rule updates, and broader policies.The authors propose integrating moderator feedback into models and interfaces while avoiding unnecessary interference with moderators’ work.Source: §6 · p. 16

Related papers

Loading 2204.11788v1…