PaperlayerView exact revision on arXiv

Source-linked AI summary

Investigating Human + Machine Complementarity for Recidivism Predictions

Sarah Tan, Julius Adebayo, Kori Inkpen, Ece Kamar

arXiv:1808.09123v2cs.LGcs.AIcs.CYstat.ML

TL;DR

Although prior work found comparable accuracy for COMPAS and human recidivism predictions, it left open whether they were right or wrong on the same defendants. This paper builds Human scores, analyzes agreement and disagreement, and tests hybrid predictors; on this dataset, their differences did not yield significant ground-truth prediction gains.

  • Problem

    After comparable human and COMPAS accuracy, it remained unclear whether they made correct predictions on the same defendants, a distinction relevant to whether combining them could use complementary strengths.

    Source: Introduction · p. 1

  • Method

    The paper aggregates Mechanical Turk predictions into Human risk scores, characterizes COMPAS–Human agreement and disagreement, and tests hybrid models against single-score approaches.

    Source: Constructing Human risk score · p. 2, Hybrid models · p. 3

  • Results

    On this dataset, Human and COMPAS decisions differed but not in ways that significantly improved ground-truth prediction; wrong agreement covered 19% of the data, bounding maximal hybrid improvement.

    Source: Discussion · p. 5

  • Takeaways & Limitations

    The observed decision differences did not provide a way to improve ground-truth prediction on this dataset, and the 19% wrong-agreement region bounds maximal hybrid improvement.

    Source: Discussion · p. 5

    The recidivism label is a proxy for ground truth and is susceptible to racial or socioeconomic bias through differential policing.

    Source: Noisy ground truth labels · p. 6

Abstract

from arXiv · show

When might human input help (or not) when assessing risk in fairness domains? Dressel and Farid (2018) asked Mechanical Turk workers to evaluate a subset of defendants in the ProPublica COMPAS data for risk of recidivism, and concluded that COMPAS predictions were no more accurate or fair than predictions made by humans. We delve deeper into this claim to explore differences in human and algorithmic decision making. We construct a Human Risk Score based on the predictions made by multiple Turk workers, characterize the features that determine agreement and disagreement between COMPAS and Human Scores, and construct hybrid Human+Machine models to predict recidivism. Our key finding is that on this data set, Human and COMPAS decision making differed, but not in ways that could be leveraged to significantly improve ground-truth prediction. We present the results of our analyses and suggestions for data collection best practices to leverage complementary strengths of human and machines in the fairness domain.

Introduction

Risk assessment has long compared human judgment with mechanical prediction, while machine-learning use in bail and sentencing has renewed concerns about accuracy, fairness, and transparency. After Dressel and Farid (2018) found comparable human and COMPAS accuracy, this paper examines whether their errors and decisions differ in ways that could support hybrid prediction.

Introduction

  • Grove et al. (2000) found machine predictions averaged 10% greater accuracy across 136 studies, but recidivism studies in that analysis showed similar human and machine accuracy.Some studies showed no machine improvement, and a few found human predictions more accurate.Source: Introduction · p. 1
  • Dressel and Farid (2018) reported no significant accuracy difference: COMPAS reached 65.2%, humans without race information 67.0%, and humans with race information 66.5%.They sampled 1,000 defendants from ProPublica COMPAS data and asked Mechanical Turk workers to predict two-year recidivism.Source: Introduction · p. 1
  • The paper shifts the comparison from whether humans or COMPAS are superior to whether their decision differences can support a hybrid approach.Humans may use context unavailable to algorithms, while machines may avoid some human-judgment biases.Source: Introduction · p. 1

Related Work

Prior work examines selective labels and fairness perceptions in human decisions, hybrid performance across domains, and methods for diagnosing model errors. This paper draws on interpretable models to analyze where human and machine recidivism decisions differ and how they might be aggregated.

Humans and decision making

  • Selective labels complicate recidivism analyses based on human decisions because judges’ release decisions affect which outcomes are observed.Other work studies how people perceive different features as fair or unfair.Source: Related Work · p. 2

Hybrid models

  • Hybrid benefits vary by task: machine assistance reduced expert error in medicine, while complementing face-recognition experts with an inferior system could improve accuracy.Other studies report humans and machines working more effectively together in problem solving and image classification.Source: Related Work · p. 2

Diagnosing errors

  • Prior error-diagnosis methods include defining confident-but-wrong cases, interactive human correction, and distilling black-box decisions into interpretable models.This paper similarly uses interpretable models to examine recidivism reasoning, decision differences, and possible aggregation.Source: Related Work · p. 2

Approach

The approach constructs an aggregated Human risk score, partitions COMPAS and Human decisions by agreement and correctness, then tests hybrid models against single-score and random baselines.

Constructing Human risk score

  • The Human risk score averages 20 Turk workers’ binary predictions for each defendant and scales the mean to COMPAS’s 1–10 scale.On average, 80% of the workers assessing the same defendant agreed; scores were constructed both with and without race information.Source: Constructing Human risk score · p. 2

Partitioning by agreement and correctness

  • On the disagreement analysis, humans and COMPAS agree and are correct almost 50% of the time, while the remaining defendants include cases where one or both scores are wrong.The analysis groups eight combinations of two binary scores and ground truth into four partitions: both correct, both incorrect, Human correct, and COMPAS correct.Source: Partitioning by agreement and… · pp. 2–3
  • Clustering and decision trees characterize agreement and disagreement cases, and hybrid models are trained on defendants where COMPAS and Human scores disagree.When scores agree and are correct, either returns the same prediction; the models test whether disagreement can improve on single-score accuracy.Source: Partitioning by agreement and… · p. 3

Hybrid models

  • Hybrid approaches include score averages, learned weighted averages, direct ground-truth predictors, and indirect models that choose which score to use.Random forests are evaluated against random and single-score baselines using AUC across ten 80%-20% train-test splits.Source: Hybrid models · p. 3

Analysis and Results

Human and COMPAS scores differed in their error patterns and feature weights, but hybrid models made only marginal gains in predicting recidivism. Agreement was strongest when priors and age clearly distinguished defendants, and both scores failed when those features were insufficient.

COMPAS vs. Humans: Predictive Performance

COMPAS vs. Humans: Decision Making

  • Number of priors and age were the leading features for all scores, but Human scores placed more weight on priors and charge degree than COMPAS, while age had similar impact.When race was revealed, black defendants were assigned slightly higher Human risk; the reported effect was around +/- 0.15 on the 1–10 scale, with different workers in the two conditions.Source: COMPAS vs. Humans: Decision Making · p. 4, Figure 1 · p. 4

Easy calls: COMPAS and Humans agree, both correct.

  • Among defendants both scores correctly classified, priors and, to a lesser extent, age separated high-risk Case 1 from low-risk Case 2.Average priors were 7.9 in Case 1 and 0.34 in Case 2; average ages were 30.3 and 40.56, respectively.Source: Easy calls: COMPAS and Humans agree,… · p. 4

Unknown unknowns: COMPAS and Humans agree, but both incorrect.

  • When both scores agreed but were wrong, priors and age did not distinguish these defendants from others despite different ground-truth labels, and both scores relied too heavily on those features.The paper characterizes these Cases 7 and 8 as “unknown unknowns.”Source: Unknown unknowns: COMPAS and Humans… · pp. 4–5

Characterizing the space of disagreement.

  • Disagreement cases overlapped with other cases and were not cleanly separable by priors and age, limiting distinct feature profiles for hybrid decisions.For example, low-COMPAS, high-Human defendants who did not recidivate tended to have 1.5 to 5.5 priors and be younger than 32.5, but overlapped with several other cases.Source: Characterizing the space of disagreement · p. 5

COMPAS + Humans: Leveraging Disagreement to Build Hybrid Models

  • Hybrid models usually gained only slightly across racial groups, often within the margin of error; in disagreement cases, feature-using models tended to outperform weighted score averages.For all defendants, however, the best-performing hybrids were optimally weighted averages; for disagreement cases, models tended to prefer HNR over HWR, particularly outside the white group.Source: COMPAS + Humans: Leveraging… · p. 5
  • AUC 0.60 for a feature-plus-COMPAS-plus-Human random forest exceeded HNR (0.56), HWR (0.54), and COMPAS (0.49), but gains were marginal.It was comparable to a features-plus-HNR random forest (AUC 0.59); adding COMPAS to HNR improved AUC by 0.01, within the margin of error.Source: Table 4 · p. 5, COMPAS + Humans: Leveraging… · p. 5

Discussion

Human and COMPAS decision making differed, but the differences did not yield ground-truth prediction gains on this data set. The discussion identifies shared errors, limited inputs, label concerns, and sample size as boundaries on hybrid models.

Discussion

  • Human and COMPAS scores differed, but not in ways that improved ground-truth prediction; agreed-but-wrong cases were patternless and made up 19% of the data.That share bounds the maximal possible improvement from a hybrid model on this data.Source: Discussion · p. 5
  • Among the 32% of defendants with disagreeing decisions, hybrid models improved only within the margin of error, and single scores had 0.49–0.56 AUC.These cases were the most uncertain; allowing single scores some ground-truth labels could further improve them.Source: Discussion · p. 5
  • Number of priors and age were most important for determining Case 3–6 assignment, although the four cases were often not clearly separated.Number of priors also mattered for both COMPAS and Human decisions, which tended to agree and be correct at very high or low prior counts.Source: Discussion · pp. 5–6

Noisy ground truth labels

  • The ProPublica recidivism label is a proxy that can miss arrests without convictions and crimes without arrests, and may reflect racial or socioeconomic policing biases.Its primary definition includes reconviction, rearrest without conviction, or supervision revocation during the first two years back in the community.Source: Noisy ground truth labels · p. 6

Criminal justice expertise

  • Turk workers lacked criminal-justice experience, limiting ecological validity; the authors call for judge predictions and richer quantitative and qualitative data from real legal settings.They seek evidence about when judges and algorithms agree or disagree and what additional information informs judges’ decisions.Source: Criminal justice expertise · p. 6

Lacking evidence about the world

  • Recidivism may be inherently unpredictable, and the seven available features may be insufficient to assess risk adequately.The authors consider the limited feature set a likely explanation; real-world judges may use additional private information, whose value or harm remains uncertain.Source: Lacking evidence about the world · p. 6

Small sample size

  • Hybrid models trained on 340 defendants with disagreeing scores had improvements subsumed by large margins of error, including in race subgroups.The subgroup counts were 169 Black, 114 White, and 31 other-race defendants; the authors call for a larger sample.Source: Small sample size · p. 6

Conclusion

On this data set, differences between human and machine decisions did not improve ground-truth recidivism prediction. The authors identify real-world judge predictions and further study of bias as next steps.

Conclusion

  • On this data set, human and machine decisions differed but could not be leveraged to improve ground-truth recidivism prediction.The authors hypothesize that real-world context may provide better information for hybrid models and identify judge predictions as a next step.Source: Conclusion · p. 6
  • Race affected some aspects of the analyses but was not the primary focus; future work will examine fairness more closely as real-world data are gathered.The authors note that differing biases in humans and algorithms could give hybrid models potential to help overcome them.Source: Conclusion · p. 6

Appendix A: Details of Hybrid Models and Metrics

The paper describes indirect and direct hybrid models for combining COMPAS and Human scores. The indirect model selects which score to use when they disagree, while the direct model predicts recidivism from features and both scores.

Direct and indirect hybrid models.

Accuracy and error metrics.

The paper reports balanced accuracy and four error rates to assess classification performance. It notes that several fairness-related metrics cannot all be satisfied simultaneously.

Accuracy and error metrics.

  • Balanced accuracy is the mean classification accuracy across classes; the reported error rates are FPR, FNR, FDR, and FOR.The paper notes that Kleinberg, Mullainathan, and Raghavan (2017) and Chouldechova (2017) showed that several of these metrics cannot be satisfied simultaneously.Source: Accuracy and error metrics · p. 9
  • For a binary label and prediction, FP, FN, TP, and TN denote false positives, false negatives, true positives, and true negatives..Source: Accuracy and error metrics · p. 9

Appendix B: Detailed Characterization of the Space of Disagreement

Among cases where COMPAS and Human scores disagree, the decision-tree analysis identifies features associated with the two directions of disagreement and notes difficulty separating some outcomes. Additional figures describe the tree and score differences, and compare performance across score cutoffs.

Appendix B: Detailed Characterization of the Space of Disagreement

  • Cases 3 and 6 have high COMPAS and low Human scores, tend to have fewer than 0.5 priors, and differ in whether defendants recidivated; crime type is their key distinguishing feature.Some multiclass trees combine Cases 3 and 6, suggesting the features do not provide sufficient signal to distinguish them adequately.Source: Appendix B · p. 9
  • Cases 4 and 5 have low COMPAS and high Human scores, but differ in recidivism outcome; Case 4 defendants tended to have 1.5–5.5 priors and be older than 32.5.Case 5 was sometimes absent as a terminal node and was similar to Case 4 and agreement Cases 1 and 7.Source: Appendix B · p. 9

Appendix B: Additional Figures

  • Figure A3’s decision tree partitions features and reports sample counts and predicted cases across the eight COMPAS, Human-score, and ground-truth combinations.The caption describes node values including the feature partition, sample count, counts for each of the eight cases, and predicted case.Source: Figure A3 · p. 10
  • Figure A4 compares Human scores with and without race information and compares each version with COMPAS; Figure A5 plots accuracy, false positive rates, and false negative rates across score cutoffs..Source: Figure A4 · p. 11, Figure A5 · p. 11

Appendix C: Extended result tables for hybrid models for defendants whose COMPAS and Human scores disagree

Appendix C provides extended hybrid-model performance tables for defendants whose COMPAS and Human scores disagree, including subgroup results. The overall table reports test-set means and standard deviations across ten train-test splits.

Appendix C: Extended result tables for hybrid models for defendants whose COMPAS and Human scores disagree

  • Table A1 reports test-set performance for hybrid models on defendants whose COMPAS and Human scores disagree, alongside single-score and other baselines.Values are means and standard deviations across 10 train-test splits; best results are marked in cyan and bold.Source: Table A1 · p. 12
  • Tables A2, A3, and A4 report subgroup performance for African-American, white, and other-race defendants, respectively..Source: Table A2 · p. 12, Table A3 · p. 13, Table A4 · p. 13

Appendix D: Result tables for hybrid models for all defendants

Appendix D describes hybrid-model test-set comparisons for all defendants and provides subgroup tables for African-Americans, whites, and other races. Table A5 defines benevolent and adversarial oracle baselines, while Table A6 gives an extended version of the comparison.

Appendix D: Result tables for hybrid models for all defendants

  • Table A5 compares hybrid models on all individuals with single-risk-score and other baselines, using benevolent and adversarial oracles as upper and lower accuracy bounds.The benevolent oracle uses the risk score that best predicts ground truth; the adversarial oracle uses the one that predicts it worst.Source: Table A5 · p. 14
  • Table A6 extends the all-individuals comparison, reporting means and standard deviations over 10 train-test splits.Cyan and bold identify best results; asterisks mark rows labeled best in Table A5.Source: Table A6 · p. 14
  • Tables A7, A8, and A9 report subgroup performance for African-Americans, whites, and other races, respectively.These tables cover the models presented in Table A6.Source: Table A7 · p. 15, Table A8 · p. 15, Table A9 · p. 16
Loading 1808.09123v2…