PaperlayerView exact revision on arXiv

Source-linked AI summary

Jury Learning: Integrating Dissenting Voices into Machine Learning Models

Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeffrey T. Hancock, Tatsunori Hashimoto, Michael S. Bernstein

arXiv:2202.02950v1cs.HCcs.AIcs.LG

TL;DR

Supervised learning commonly resolves disagreement by aggregating labels into a majority-ground-truth voice, despite contested labels across societal groups. Jury learning models individual annotators so practitioners can specify, sample, and inspect juries whose weighted responses determine predictions. In toxicity detection, this produced more diverse juries, changed 14% of classifications, and improved both jury-learning and aggregate-label prediction.

  • Problem

    Majority-vote supervision implicitly makes the largest group’s viewpoint the ground truth, while many user-facing tasks contain substantial disagreement and lack mechanisms for explicitly choosing whose voices determine predictions.

  • Method

    Jury learning models every annotator, samples jurors according to practitioner-defined group or individual compositions, predicts their labels for unseen inputs, and aggregates those responses.

  • Results

    In toxicity detection, jury compositions contained 2.9 times the representation of non-White jurors and 31.5 times the representation of non-binary jurors, changed 14% of classifications, and achieved MAE=0.27 on aggregate labels versus MAE=0.41 for state-of-the-art classifiers.

  • Takeaways & Limitations

    Jury learning makes the selection and weighting of voices behind each prediction explicit while enabling uncertainty, dissent, counterfactual, and group-pattern exploration.

  • Takeaways & Limitations

    The paper demonstrates jury learning in a single application domain, toxicity detection, although it is designed for tasks with annotator disagreement, group-membership data, and high-quality item embeddings.

Abstract

from arXiv · show

Whose labels should a machine learning (ML) algorithm learn to emulate? For ML tasks ranging from online comment toxicity to misinformation detection to medical diagnosis, different groups in society may have irreconcilable disagreements about ground truth labels. Supervised ML today resolves these label disagreements implicitly using majority vote, which overrides minority groups' labels. We introduce jury learning, a supervised ML approach that resolves these disagreements explicitly through the metaphor of a jury: defining which people or groups, in what proportion, determine the classifier's prediction. For example, a jury learning model for online toxicity might centrally feature women and Black jurors, who are commonly targets of online harassment. To enable jury learning, we contribute a deep learning architecture that models every annotator in a dataset, samples from annotators' models to populate the jury, then runs inference to classify. Our architecture enables juries that dynamically adapt their composition, explore counterfactuals, and visualize dissent.

1 INTRODUCTION

Jury learning makes the choice of whose labels a classifier emulates explicit, replacing implicit majority aggregation with configurable juries of modeled annotators. In toxicity detection, this changed classifications and improved prediction of both jury and aggregate outcomes.

  • Majority-vote aggregation turns multiple annotations into a single ground-truth label representing the largest group’s viewpoint.
  • Jury learning lets practitioners specify which people or groups, and in what proportions, determine a classifier’s prediction.The architecture models each annotator, predicts their labels for unseen examples, and aggregates the selected jurors’ responses.
  • The architecture combines modern natural-language processing with recommender-system techniques to predict how individual annotators would label unseen examples.It also supports uncertainty visualization and expressive jury algorithms, including compositions that adapt to relevant stakeholders.
  • 2.9 times and 31.5 times: toxicity juries contained respectively more non-White and non-binary representation than implicitly created dataset juries.
  • 14% of toxicity items changed classification under the more diverse juries, while jury-learning predictions had MAE 0.62 versus 1.05 for an aggregated baseline.
  • The model achieved MAE=0.27 on aggregate test-set labels versus MAE=0.41 for contemporary state-of-the-art classifiers.

2 RELATED WORK

Related work identifies the limits of unexamined majoritarianism and motivates explicit stakeholder representation, disagreement modeling, and interactive mechanisms for selecting whose labels guide classifiers. Jury learning extends these directions by modeling annotators directly and tuning populations without recollecting large datasets.

  • Strict majoritarianism can exclude minority viewpoints, motivating governance-inspired mechanisms for more participatory decision-making.
  • Fairness and value-sensitive design research addresses stakeholder needs through mathematical fairness constraints, design guidance, and explicit stakeholder inclusion.
  • Conventional classifiers often speak as aggregated pseudohumans because majority aggregation or loss minimization favors the largest group’s opinions.
  • Annotator disagreement is substantial in user-facing tasks, including toxicity and misinformation classification, making a single learned voice potentially unsuitable across deployments.
  • Prior disagreement-aware models predict distributions of annotator labels, while argument-supporting systems can prompt users to review contentious cases.
  • Jury learning directly models disagreement and lets practitioners understand and re-weight whose voices are embedded in a classifier.
  • Unlike approaches requiring new large-scale datasets when stakeholders change, jury learning explores and tunes the represented population using existing annotator information.

3 JURY LEARNING

Jury learning treats annotators as a population of potential jurors whose modeled responses can be explicitly composed, sampled, aggregated, and inspected. The interface supports group-aware exploration, uncertainty analysis, and adaptive jury configurations for contested classification tasks.

  • 3 JURY LEARNING: Practitioners specify whose voices a classifier reflects and in what proportion, while the system models each annotator as a potential juror.
  • 3.2 Approach and interaction: A jury is a bounded set of sampled individuals whose predicted opinions aggregate into a classification decision or, for regression, an average score.
  • 3.2 Approach and interaction: Jury learning can create many classifiers from one labeled dataset when annotator group memberships are available for the voices practitioners want to include.
  • 3.2 Approach and interaction: For a new input, the system predicts every annotator’s response, repeatedly resamples juries matching the specification, and aggregates each jury’s decisions.
  • 3.2 Approach and interaction: The median-of-means estimator uses the median jury decision as the default verdict across sampled juries.
  • 3.2 Approach and interaction: Jury-sample outcome frequencies provide a direct uncertainty measure, such as the percentage of juries producing toxic versus non-toxic labels.
  • 3.2 Approach and interaction: The interface exposes juror annotations, backgrounds, group context, trends, and counterfactual juries to support inspection of disagreement.
  • 3.2 Approach and interaction: Jury compositions can adapt to the stakeholders relevant to an input, such as emphasizing women, LGBTQ+ people, or Black women for related topics.

4 TECHNICAL APPROACH

Jury learning predicts how individual annotators would label unseen inputs, then aggregates those predictions into a decision. Its architecture combines content embeddings with annotator and group representations so predictions work for cold-start examples while retaining modern classifier performance.

  • Technical approach: Jury learning predicts each annotator’s label for an unseen example and aggregates the individual classifications into a jury outcome.The approach treats annotators as potential jurors rather than merely inputs to a label-aggregation function.
  • Technical approach: The architecture is designed for labeled datasets with a few annotators per item and supports inputs represented by content embeddings, including text, images, and screens.The paper’s stated additional assumption is that characteristics used to select jurors must be present in the training data.
  • Technical approach: The model combines a content model with a deep recommender system to predict annotator opinions from input features alone.Content embeddings support unseen examples, while recommender-system components model individual and group differences.
  • Technical approach: The architecture jointly trains a content classifier and recommender system, preserving classification capability while modeling annotator-specific responses.The paper describes this as combining pre-trained deep learning models’ classification performance with recommender systems’ ability to model individuals’ opinions.
  • Dataset description: 35.9% of annotator pairs disagree on binarized toxicity labels in the evaluation dataset.The dataset contains 107,620 comments, each labeled by five annotators drawn from 17,280 unique annotators.

5 EXTENSIONS

Jury learning extends beyond a fixed jury by allowing composition to depend on the classified item and by searching for minimally changed juries that reverse a verdict. These extensions support context-sensitive decision rules and interpretability through counterfactuals.

  • Conditional juries: Conditional juries let practitioners represent different forms of expertise for decisions affecting different groups or topics.The paper gives larger representation of women for misogynistic comments as an example of context-dependent composition.
  • Conditional juries: Jury composition can condition on the item, selecting different jurors for topics such as #metoo, #blm, or vaccination.A default jury can remain fixed while the remaining members vary according to contextual conditions or alternative clustering and topic-modeling strategies.
  • Counterfactual juries: A counterfactual jury is the smallest edit to the current composition that changes the classification outcome.The paper formulates this search as a quadratic program that can be solved with off-the-shelf optimization solvers.
  • Counterfactual juries: Counterfactual juries provide an interpretability lens for assessing how dependent a classification is on jury composition.The interaction asks what jury composition would produce the opposite verdict and how different it would need to be.

6 MODEL EVALUATION

The evaluation tests whether jury learning can predict individual annotators and jury verdicts while preserving information about group and individual differences. On toxicity detection, it improves individual-label prediction over annotator-agnostic and group-only baselines, reduces disparities between example groups, and predicts jury verdicts more accurately than aggregate prediction.

  • Evaluation goals: Jury learning is evaluated at both individual-annotator and jury levels, asking whether it predicts jurors’ unseen responses and final jury verdicts.The toxicity evaluation compares individual annotator predictions and jury-level predictions, with attention to information borrowing and the risk of borrowing too much.
  • Individual annotator performance: 0.61 MAE versus 0.90 for the baseline shows more accurate individual annotator-response predictions than an annotator-agnostic classifier.The comparison uses Mean Absolute Error on unseen toxicity comments and annotations.
  • Individual annotator performance: 0.61 MAE for the full model versus 0.81 for the group-only model and 0.90 for the aggregate baseline indicates value from both individual and group features.The group-only ablation removes annotator IDs, leaving group-based and content-based features.
  • Group-based performance: Group disparities are smaller for jury learning: Asian versus Black annotator MAE is 0.62 versus 0.65, compared with 0.83 versus 1.12 for the aggregate baseline.The corresponding performance decrease is 4.9% for the proposed model and 35.0% for the standard aggregate model.
  • Jury-level performance: Over 550 ten-annotator juries, jury learning achieves a jury-level MAE of 0.27 versus 0.41 for aggregate prediction.The ten-annotator subset approximates a twelve-member jury, and the observed verdict is the mean of the ten annotations.

7 USER EVALUATION

The evaluation examined how content moderators compose juries and whether those juries alter toxicity classifications. Moderator-designed juries were more diverse, better modeled selected jurors’ views, and changed classifications especially for divisive comments.

  • Jury composition diversity: Sixteen analyzed moderators designed diverse juries, averaging 5.7 race values, 3.1 gender values, and 3.4 political-affiliation values.Their juries averaged 10.31 individuals with at least one non-majority attribute and 3.88 with at least two.
  • Jury composition diversity: Moderator-designed juries achieved greater diversity than the current algorithm’s implicit jury across all three demographic attributes.They generally raised representation for groups with the lowest implicit representation and narrowed gaps among groups.
  • Participant reasoning: Participants primarily prioritized diversity and equal representation, often increasing representation for groups targeted in the provided comments.Many anticipated that jury learning would better capture minority-group views and increase comments rated toxic.
  • Annotator-level modeling: Jury learning reduced the average error on selected jurors’ opinions by 41%, from MAE 1.05 to 0.62 versus the baseline aggregated model.The architecture modeled individual jurors and was evaluated by repeatedly sampling jurors from participant-provided compositions.
  • Classification changes: 13.6% of toxicity decisions flipped on average across the 16 moderator-provided juries, with a standard deviation of 4.1%.The changed classifications indicate different outcomes under community-customized juries than an off-the-shelf classifier.
  • Classification changes: Comments causing classification flips had a 46.4% annotator disagreement rate, versus 37.2% for comments that did not flip.The difference was significant in a two-proportion z-test (z=2.89, p<.01).

8 DISCUSSION

The discussion reflects on jury learning’s contributions, limitations, practical use by designers and product teams, and ethical considerations.

  • The discussion addresses jury learning’s contributions, limitations, practical use, and ethical considerations.

8.1 Implications for design

Jury learning treats annotator differences as meaningful value disagreements rather than label noise and offers tools for specifying and inspecting whose perspectives classifiers reflect.

  • Design implications: Jury learning preserves individual annotators in the ML pipeline instead of abstracting their differences into label noise.It asks practitioners to specify whose voices classifiers reflect and in what proportion.
  • Design implications: The approach adds an interpretability lens that explains predictions through annotator properties as well as item properties.This complements feature- or content-based explanations of model decisions.
  • Design implications: Users can interrogate which groups’ opinions influence automated moderation and call for greater representation.The paper connects this capability to transparency and debate over normative decisions about whose perspectives should be included.

8.2 Ethical considerations

Jury learning makes whose voices shape a classifier explicit, but the approach introduces ethical trade-offs involving fairness, transparency, accountability, privacy, and representation.

  • Fairness and transparency: Jury learning does not inherently enforce fairness and can intentionally or unintentionally produce biased decisions by selecting harmful or excluding relevant voices.The process can support diverse representation, but its outcomes depend on practitioners’ jury choices.
  • Fairness and transparency: Unlike post-hoc fairness approaches, jury learning emphasizes selecting whose voices a model should emulate and making that selection explicit.The authors distinguish process-oriented voice selection from methods that constrain model outputs statistically.
  • Fairness and transparency: Transparent jury compositions can enable conversations with stakeholders about whose voices a classifier emulates and the implications of those choices.The authors propose that organizations disclose jury composition to relevant stakeholders.
  • Decision-making and representation: The median-of-means aggregation resolves disagreement into a single decision but may still promote the ecological fallacy.The system takes the median jury from 100 sampled juries matching the practitioner’s composition while ignoring possible outliers.
  • Data and accountability: Model quality depends on datasets accurately representing annotators’ views and group memberships, requiring careful data collection and auditing by group.The authors recommend participatory audits and group-split performance metrics because misrepresentation can harm minority perspectives.
  • Accountability: Open-ended decisions about who controls jury outputs and how they enter moderation systems may let platforms evade responsibility for content decisions.The authors identify both the decisionmaker and the surrounding meta-policy as unresolved design aspects.
  • Privacy: Faithful representation may require collecting private attributes, creating privacy risks through data recovery or record-linkage attacks.The authors identify privacy-preserving disclosure and differential privacy as future directions.

8.3 Limitations and future work

The paper demonstrates jury learning in toxicity detection and identifies scope, data representation, privacy, and metaphor-related boundaries for future work.

  • Scope: The evaluation covers only toxicity detection, although the approach is intended for tasks with annotator disagreement, group-membership data, and strong item embeddings.The authors specifically suggest future applications in medical decision-making and design tasks.
  • Future applications: Applying jury learning to medical and design decisions could support juries reflecting patient preferences or different artistic sensibilities.These examples extend the proposed use beyond the demonstrated application domain.
  • Jury metaphor: The US legal-jury metaphor is deliberately partial, drawing on group voting and juror selection rather than deliberation or the full legal apparatus.The authors do not claim a complete correspondence with legal juries.
  • Data representation: The demonstrated dataset’s demographic categories create inclusion concerns, especially when gender interfaces inadequately represent non-binary identities.Possible remedies include inclusive options, open text fields, and selecting subsets of respondents as potential jurors.
  • Future methods: Current jury learning requires explicit information about annotators’ group memberships, motivating future unsupervised methods for discovering distinct voices.Such methods could make the approach usable with datasets lacking explicit group annotations.

8.4 Positionality statement

The authors situate jury learning within commitments to human oversight, increased control, and a North American norm of jury-based peer decision-making.

  • Author positioning: The authors acknowledge arguments for removing AI from sociotechnical systems and for substantially increasing human control, oversight, and auditing.They state an ideological commitment to cases where improving AI is preferable to removing it.
  • Normative commitments: The jury metaphor reflects a North American normative commitment to decisions made by a jury of peers, while recognizing that juries can produce progressive or regressive decisions.The authors distinguish this normative framing from a claim that legal juries are wholly beneficial.
  • Terminology: The paper uses “toxic” as a broad, non-specific term consistent with the process used to collect its dataset.The authors note that the term covers varied forms of content people do not wish to see online.

9 CONCLUSION

Jury learning addresses contested user-facing tasks by making whose voices models imitate explicit and enabling practitioners to reason about disagreement and value judgments.

  • Conclusion: Jury learning treats individual people as more than aggregated pseudo-humans, enabling reasoning about who disagrees and why.The approach is presented as a bridge between machine-learning pipelines and contested tasks.
  • Conclusion: The approach enables practitioners to make explicit value judgments about how models resolve disagreement.The authors hope this supports more informed and intentional classifier development and deployment.
Loading 2202.02950v1…