PaperlayerView exact revision on arXiv

Source-linked AI summary

In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction

Caroline Wang, Bin Han, Bhrij Patel, Cynthia Rudin

arXiv:2005.04176v3stat.MLcs.LGstat.AP

TL;DR

Secret or opaque recidivism models raise concerns about understanding decisions and checking errors, while interpretable methods aim to retain predictive performance. The study evaluates interpretable models against black-box approaches and existing risk assessments, finding comparable performance and supporting location-specific models that are updated over time.

  • Problem

    Black-box recidivism models can perform well but are too complicated for humans to understand, making interpretable alternatives a central research aim.

    Source: Abstract of Main Results for Criminal… · p. 2

  • Method

    The study evaluates interpretable machine-learning models for predictive performance and fairness alongside black-box models and existing risk assessments.

    Source: Abstract of Main Results for Criminal… · pp. 2–3

  • Results

    Interpretable models perform approximately as well as black-box models, with no practical accuracy loss and gains in interpretability.

    Source: §10 · p. 28

  • Takeaways & Limitations

    Models should be constructed for distinct locations and periodically updated.

    Source: §10 · p. 28

    Fairness results for the “Other” race category are difficult to interpret because the race data have low resolution.

    Source: Abstract of Main Results for Criminal… · p. 3

Abstract

from arXiv · show

Objectives: We study interpretable recidivism prediction using machine learning (ML) models and analyze performance in terms of prediction ability, sparsity, and fairness. Unlike previous works, this study trains interpretable models that output probabilities rather than binary predictions, and uses quantitative fairness definitions to assess the models. This study also examines whether models can generalize across geographic locations. Methods: We generated black-box and interpretable ML models on two different criminal recidivism datasets from Florida and Kentucky. We compared predictive performance and fairness of these models against two methods that are currently used in the justice system to predict pretrial recidivism: the Arnold PSA and COMPAS. We evaluated predictive performance of all models on predicting six different types of crime over two time spans. Results: Several interpretable ML models can predict recidivism as well as black-box ML models and are more accurate than COMPAS or the Arnold PSA. These models are potentially useful in practice. Similar to the Arnold PSA, some of these interpretable models can be written down as a simple table. Others can be displayed using a set of visualizations. Our geographic analysis indicates that ML models should be trained separately for separate locations and updated over time. We also present a fairness analysis for the interpretable models. Conclusions: Interpretable machine learning models can perform just as well as non-interpretable methods and currently-used risk assessment scales, in terms of both prediction accuracy and fairness. Machine learning models might be more accurate when trained separately for distinct locations and kept up-to-date.

Abstract of Main Results for Criminal Justice Practitioners

The study evaluates machine-learning models for pretrial recidivism prediction across crime types and time horizons, comparing predictive performance and fairness with COMPAS and the Arnold Public Safety Assessment. It finds that interpretable models can match black-box models, while model performance may not transfer across locations and fairness results vary by group and measure.

Abstract of Main Results for Criminal Justice Practitioners

  • The evaluation covers six crime types over six-month and two-year horizons, alongside cross-location performance and race- and gender-based fairness analyses.Source: Abstract of Main Results for Criminal… · p. 2
  • Interpretable models perform approximately as well as black-box models and generally better than COMPAS or the Arnold Public Safety Assessment for their intended locations.Some can be represented in a one-page table; others use visualizations.Source: Abstract of Main Results for Criminal… · pp. 2–3
  • Models trained in one location tend to perform less well when applied in another, supporting local training and periodic updates.Source: Abstract of Main Results for Criminal… · p. 3
  • For the two-year general recidivism outcome in Kentucky, RiskSLIM and Explainable Boosting Machine approximately met calibration and balanced group AUC criteria across Black/White and male/female groups.Source: Abstract of Main Results for Criminal… · p. 3
  • The Arnold Public Safety Assessment’s New Criminal Activity score failed calibration at higher score values, while results for the “Other” race category were less consistent.The authors caution that low-resolution race data make the “Other” category results difficult to interpret.Source: Abstract of Main Results for Criminal… · p. 3

1 Introduction

The introduction motivates recidivism models that are interpretable and fair, and that account for differences across locations and over time. It presents a comparison of interpretable and black-box models with existing risk assessments, alongside analyses of generalization and fairness.

1 Introduction

  • Secret black-box justice algorithms can obscure how decisions were made or whether errors occurred, while raising concerns about due process and independent fairness or performance guarantees.Source: §1 · p. 3
  • Prior machine-learning work often omits data quality and prediction practicality, and the authors identify only one prior study jointly considering interpretability, fairness, and predictive performance.Source: §1 · p. 4
  • Risk assessments used across states may not achieve best performance because populations and data-generation processes differ across locations.Source: §1 · p. 4
  • Recidivism distributions can shift over time, motivating models trained for specific locations and retrained when distributions change.Source: §1 · p. 4
  • The study defines recidivism as a new charge resulting in conviction within six months or two years and examines whether models generalize across regions.Source: §1 · p. 4
  • Across the twelve recidivism problems, black-box models did not perform significantly better than interpretable models, which generally outperformed existing actuarial risk assessments.Source: §1 · p. 4

2 Contribution

The paper contributes interpretable, risk-calibrated models for criminal recidivism, alongside analyses of predictive performance, fairness, and geographic generalization. Its contributions include probability-based scores, comparisons with tools used in justice systems, and models designed for practical interpretation.

2 Contribution

  • The interpretable risk-calibrated models perform approximately as well as, and sometimes better than, existing actuarial risk assessments while predicting specific crime types.The study considers six crime types at six-month and two-year time scales, for 12 prediction problems.Source: §2 · p. 5
  • Models trained in one location do not tend to perform as well in another, supporting training models on data from their intended location.The analysis uses criminal-history data from Broward County and Kentucky to examine variation across locations.Source: §2 · p. 5
  • Some models can be represented as a one-page table, while others can be presented through visualizations.The authors describe the models as potentially useful in practice after careful evaluation.Source: §2 · p. 5
  • The paper provides an approach to evaluating interpretability and fairness in a high-stakes application, with potential relevance to other consequential decisions.The authors name financial lending and hiring as examples of settings to which this analysis could be ported.Source: §2 · p. 5
  • Unlike the cited prior work, the study produces probability-based risk scores, compares models with COMPAS and the Arnold PSA, and uses pretrial data from two locations.The study also assesses multiple fairness definitions and considers more recent data than the earlier work discussed.Source: §2 · p. 5

3 Background

This section situates pretrial recidivism forecasting within the broader use of criminal-justice risk assessments and debates about fairness, transparency, and their role. It also describes how interpretable models can clarify input factors and how competing fairness criteria require domain-specific judgment.

3.1 The Debate over Risk Assessments

  • Critics warn that risk assessments can exacerbate racial and socioeconomic disparities, while proponents argue they can help reduce violence and allocate resources.The authors emphasize that simple, interpretable risk scores may help judges and defendants understand predictions and reduce the risk of miscomputation.Source: §3.1 · pp. 6–7

3.2 Black-box and Interpretable Machine Learning for Predicting Criminal Recidivism

  • Although much recidivism research uses black-box models optimized for prediction, interpretable approaches remain less common and lack a consensus definition.The authors reject post-hoc explanations for this high-stakes setting, citing their unreliability and the absence of a clear accuracy benefit for black-box models.Source: §3.2 · p. 8
  • ProPublica’s COMPAS race-dependence allegation relied on flawed reasoning: a variable’s presence in an explanation does not establish that the black-box model uses it.Source: §3.2 · p. 8
  • Interpretable scoring systems can incorporate factors outside the database and show each input’s relative importance through points on a shared scale.The systems described are linear models with integer coefficients, assigning every input a number of points.Source: §3.2 · p. 9

3.3 Fair Machine Learning

  • Fairness is crucial for risk scores, but no single universal definition applies because fairness constraints can conflict and may trade off against performance.For example, calibration and balance for positive and negative classes cannot both be achieved in nontrivial cases; relaxing conditions can allow approximate satisfaction.Source: §3.3 · p. 9
  • The choice of a fairness definition should reflect model characteristics and domain expertise, informed by discussion among justice-system stakeholders.The paper uses fair-machine-learning definitions because they apply directly to machine-learning models and are more specific than general legal guidelines.Source: §3.3 · p. 9

4 Data

The study uses criminal-history data from Broward County, Florida, and Kentucky to build features and examine model performance across locations. The datasets differ in coverage, available features, and charge-label processing, which the authors identify as relevant to cross-location comparisons.

4 Data

  • The Broward dataset covers 11,757 people who received pretrial COMPAS scores in 2013–2014, but analysis was limited to 1,954 people for whom PSA scores could be computed.The data include criminal history, probation history, and demographics; features include prior arrests and charges.Source: §4 · pp. 9–10
  • Kentucky data combine pretrial PRIM records and CourtNet case information, totaling over 25 million tuples.PRIM records cover interviews conducted from July 2009 through June 2018.Source: §4 · p. 10
  • Features were made as similar as possible across regions, but Kentucky lacked Broward features such as age at first offense and prior juvenile charges.The authors note that Kentucky risk-assessment policies changed during data collection, possibly affecting collection consistency.Source: §4 · p. 10
  • Broward labels included non-convicted charges, whereas Kentucky labels used convicted charges; the authors chose this difference because restricting Broward to convictions would reduce its processed sample from 1,954 to 1,297.The paper notes that this processing difference might explain some cross-region performance discrepancies and that non-convicted charges can be problematic in bail or parole decisions.Source: §4 · p. 10

5 Methodology

The methodology compares machine-learning models with the Arnold PSA and COMPAS using fixed prediction windows and multiple recidivism labels. It retains nonbinary risk scores, evaluates them with AUC, and conducts comparisons within each dataset because the regions differ.

5 Methodology

  • The study compares its models with Arnold PSA NCA and NVCA scores, and with COMPAS general and violent scores on corresponding two-year tasks in Broward.COMPAS scores are not compared on Kentucky; the authors note that the scores may have been developed for parole but were applied to pretrial decisions in Broward.Source: §5 · pp. 10–11
  • The study uses fixed six-month and two-year windows so risk scores do not depend on jurisdiction-specific pretrial duration.The six-month window approximates a pretrial period, while two years provides more balanced labels; the authors note that AUC assesses relative rather than absolute risk.Source: §5 · p. 11

5.1 Prediction Labels

  • For Kentucky, recidivism means arrest within the window for a new crime later resulting in conviction; Broward instead uses any charge because later conviction information was unavailable.The authors say the Kentucky definition may alleviate a due-process concern associated with using non-convicted charges.Source: §5 · p. 11

5 Methodology

  • Comparisons of baseline and interpretable models are conducted within each dataset, not directly across Broward and Kentucky, because the regions differ in scale and other characteristics.Broward is county-level, while Kentucky spans 120 counties; the paper discusses regional performance gaps separately.Source: §5 · p. 12

5.1 Prediction Labels

  • The prediction labels cover six recidivism types—general, violent, drug, property, felony, and misdemeanor—at two time scales, yielding 12 tasks.The labels record a charge within the specified window; an individual can have multiple positive labels.Source: §5.1 · p. 12

5.2 Problem Setup

  • The models produce nonbinary probabilities or scores rather than binary predictions, enabling risk values across specific crime types and coarser categories.The study uses AUC to evaluate these predictions and reports that this choice also affects its fairness metrics.Source: §5.2 · pp. 12–13
  • The study trains models using nested cross-validation.Hyperparameters and software packages are reported in the appendix.Source: §5.2 · p. 13

6 Baseline Machine Learning Methods

The baseline comparisons show no consistent winner on Broward and only small performance differences among Kentucky models. The section introduces common machine-learning baselines, including linear models and ensembles, and notes that sparse models can still be difficult to interpret.

6 Baseline Machine Learning Methods

  • The baseline set includes penalized logistic regression, a linear-kernel SVM, Random Forest, and boosted decision trees, with ensembles treated as black-box models.Logistic regression and linear SVM models can still be difficult to interpret when they contain many nonzero terms; ensemble models obscure feature contributions or joint relationships.Source: §6 · p. 13
  • ℓ1-penalized logistic regression produced sparse Broward models with good predictive performance, but the best Kentucky models retained too many features for interpretation.The section frames sparsity as a trade-off with prediction performance.Source: §6 · p. 13

6.1 Broward Baseline Results

  • For two-year general recidivism prediction, ℓ2-penalized logistic regression and LinearSVM tie; XGBoost performs best in violent and property prediction, while ℓ1-penalized logistic regression leads drug and felony tasks, and ℓ2-penalized logistic regression leads misdemeanor prediction.The largest performance gap is 5.1%, from property recidivism prediction.Source: §6.1 · p. 14

6.2 Kentucky Baseline Results

  • On Kentucky, Random Forest and XGBoost are slightly ahead across tasks and time periods, with XGBoost best, while all algorithms perform slightly better at six months.The authors suggest Kentucky’s larger dataset—about 250K records versus 1,956 for Broward—may explain why complex, nonlinear baselines perform slightly better than linear models.Source: §6.2 · p. 14
  • No baseline consistently leads on Broward, while Kentucky baselines perform similarly despite XGBoost leading across tasks; the Kentucky comparison is visualized in Figure 1.The reported performance gaps across Kentucky prediction problems and time periods are very small.Source: Figure 1 · p. 14, §6.1 · p. 14, §6.2 · p. 14

7 Interpretable Machine Learning Methods

This section covers interpretable machine-learning methods—including scoring systems, trees, additive models, EBM visualizations, Additive Stumps, and RiskSLIM scoring tables—and their preparation and evaluation. It reports performance across Broward and Kentucky relative to benchmarks, location- and model-specific differences, and consistency across cross-validation folds for Kentucky models.

7 Interpretable Machine Learning Methods

  • Because traditional scoring-system constraints could reduce accuracy, the study considered interpretable approaches spanning scoring systems, decision trees, and additive models.The models include both simple tables and visual explanations, and the study also compared them with Arnold PSA and COMPAS.Source: §7 · p. 15

7.1 Preprocessing Features into Binary Stumps

  • For RiskSLIM and Additive Stumps, binary thresholds were chosen from EBM visualizations, with decreasing stumps for age and increasing stumps for criminal-history features.Summing mostly nonnegative stump contributions yields a monotonic feature function, supporting interpretability.Source: §7.1 · p. 17

7.2 Broward Prediction Results for Interpretable Models

  • Across considered prediction tasks, Additive Stumps and EBM performed similarly and outperformed Arnold PSA and COMPAS on most tasks; best interpretable-versus-baseline AUC gaps were 0.3–1.7% for two-year and 0.2–2.6% for six-month tasks.CART consistently performed worse than other algorithms.Source: §7.2 · p. 17

7.4 Tables and Visualizations of Interpretable Models

  • The models can be presented as a decision tree, scoring table, or set of visualizations, illustrated using Kentucky two-year general recidivism.The section presents visualizations for EBM and Additive Stumps and a table for RiskSLIM.Source: §7.4 · p. 18

7.4.1 EBM Models

  • EBM provides an interactive dashboard of feature visualizations and importance, but its models are not sparse in the number of features.Users can hover over bars or lines to see the corresponding scores.Source: §7.4.1 · p. 18
  • EBM feature contributions can be visualized separately, but their scores are not generally monotonic in each feature.For two-year general recidivism, the visualizations show greater prediction uncertainty at larger feature values, possibly because fewer data points occur there.Source: §7.4.1 · p. 18, Figure 3 · p. 19

7.4.2 Additive Stumps

  • The Kentucky two-year general recidivism Additive Stumps model used 28 stumps rooted under 14 original features.The method thresholds original features into binary stumps and applies ℓ1-penalized logistic regression, yielding a model sparse in original features.Source: §7.4.2 · p. 19

7.4.3 RiskSLIM

  • RiskSLIM models were often identical across Kentucky cross-validation folds, including all five folds for two-year drug and violent recidivism, suggesting robustness to the training subsample.For six-month predictions, all five misdemeanor models and four violent-recidivism models were identical; other prediction labels had four of five models in common. RiskSLIM assigns integer points to features and converts total scores to probabilities with a logistic function.Source: §7.4.3 · pp. 20–21

8 Recidivism prediction models do not generalize well across regions

Models trained in one region do not generalize well to another, and age–recidivism distributions differ between Broward and Kentucky. The findings support using location-specific models and updating them periodically.

8 Recidivism prediction models do not generalize well across regions

  • Models did not generalize well across regions, and the authors recommend location-specific models that are updated periodically.Their analysis also found that age and recidivism joint probability distributions vary across states.Source: §8 · p. 22

8.1 Training on One Region and Testing on the Other

  • Across the twelve prediction problems, models did not generalize across states; Kentucky-to-Broward transfer lowered two-year general baseline performance 3.5%–6.0%, with similar declines for interpretable models.Broward-to-Kentucky transfer produced even larger decreases; Figure 5 visualizes cross-region minus within-region AUC differences.Source: Figure 5 · p. 22, §8.1 · p. 22

8.2 Age-Recidivism Probability Distributions by Region

  • For general recidivism, Kentucky probabilities peak in the early-to-mid 30s, while Broward probabilities peak around ages 18–29; higher-age Broward estimates vary more where data are sparse.Broward probabilities can exceed 0.5, while Kentucky probabilities are all below 0.4.Source: §8.2 · p. 23
  • RiskSLIM models usually include age features for Broward, whereas none of the Kentucky models use age and almost all use prior-arrest features.The authors describe this contrast as reflecting greater reliance on prior criminal-history information in Kentucky.Source: §8.2 · pp. 23–24

9 Fairness

This section examines fairness in interpretable, probability-based recidivism prediction, assessing Kentucky models against calibration and balanced-group AUC criteria while discussing data scarcity and the difficulty of enforcing fairness.

9 Fairness

  • EBM and RiskSLIM had no observed violations of calibration or balanced group AUC; Arnold NCA raw had a high-score calibration violation.The broader assessment found no egregious violations among Arnold NCA, EBM, and RiskSLIM, but did identify small violations.Source: §9 · p. 24, Balanced Group AUC (BG-AUC) · p. 27
  • Fairness gaps were larger between the largest and smallest sensitive groups than between Black and White groups, and common enforcement techniques generally use non-interpretable transformations.Those transformations therefore do not work well with interpretable models.Source: §9 · p. 24

9.1 Selection of Fairness Metrics: Calibration, Balance for Positive/Negative Class, Balanced Group AUC

  • Because the study used real-valued risk scores rather than binary scores, it excluded some fairness definitions and selected calibration and balanced group AUC for evaluation.The authors explain that definitions such as statistical parity were not relevant to their setup.Source: §9.1 · pp. 24–25

9.2 Fairness Results

  • Fairness was evaluated only on Kentucky because conditioning on race or gender and true label or score produced noisy, undersized Broward subgroups.The comparison covered EBM, RiskSLIM, and Arnold PSA scores for the two-year general and violent recidivism problems.Source: §9.2 · p. 26

Calibration

  • Arnold NCA raw approximately met monotonic and group calibration below score 10, except for “Other,” while EBM and RiskSLIM met both for race and gender groups excluding “Other.”At higher Arnold NCA raw scores, calibration failed; the scaled Arnold NCA satisfied both calibration notions but had slightly worse predictive performance.Source: Calibration · pp. 26–27, Figure 7 · p. 26

Balanced Group AUC (BG-AUC)

  • AUC gaps between African-American and Caucasian groups ranged from 0.3% (RiskSLIM) to 2.1% (Arnold NCA raw); gender-group gaps ranged from 0.5% (Arnold NCA) to 1.3% (RiskSLIM).RiskSLIM had the smallest race-group AUC difference, while Arnold NCA had the smallest gender-group difference; the reported differences were small overall.Source: Balanced Group AUC (BG-AUC) · p. 27, Table 3 · p. 27
  • Prediction AUC is significantly lower for the “Other” race group, which comprised 2.49% of the sample; comparable group data may be unavailable in less-diverse states such as Kentucky.The authors recommend gathering comparable amounts of data for each sensitive group when possible.Source: Balanced Group AUC (BG-AUC) · p. 27

9.3 A Discussion on the Interaction between Fairness and Interpretability

  • Fairness methods are difficult to apply to interpretable models, and few fairness definitions or algorithms address nonbinary predictions.The authors note that most fairness research has focused on binary classification.Source: §9.3 · p. 27
  • The authors did not enforce fairness because transformations can undermine interpretability or probability-based assessments; modifying training loss is promising but requires new algorithms beyond this work’s scope.Preprocessing can alter the natural meaning of features, while some post-processing either changes predictions or requires threshold selection.Source: §9.3 · pp. 27–28
  • The authors argue that fairness deserves more study for regression and probability outputs, which are likely to inform decisions rather than make them.Source: §9.3 · p. 28

10 Discussion and Future Work

Interpretable models performed approximately as well as black-box models, with reported AUC differences around 1%—less than the uncertainty gap. The discussion argues for location-specific models that are periodically updated, while noting resource limits and the need to incorporate decision costs.

10 Discussion and Future Work

  • Interpretable models performed approximately as well as black-box models across the recidivism prediction problems discussed.On Broward, RiskSLIM, EBM, and Additive Stumps performed well; on Kentucky, EBM and Additive Stumps were highlighted.Source: §10 · p. 28
  • AUC differences around 1% were less than the uncertainty gap, and the authors report no practical accuracy loss from using interpretable models.They conclude that interpretability is gained without a practical loss in accuracy.Source: §10 · p. 28
  • Models for six-month outcomes generally outperformed those for two-year outcomes when conditioning on recidivism type.The authors suggest treatment or rehabilitation programs may have more time to affect recidivism over two years, but frame this as a possible explanation.Source: §10 · p. 28
  • The authors observed different age distributions in Kentucky and Broward County and hypothesized that this may help explain weak cross-region generalization.They also discuss policy, cultural, and temporal changes as possible sources of variation across jurisdictions.Source: §10 · p. 28
  • The authors propose separate models for different locations and periodic updates, while noting that local jurisdictions may lack resources to fit their own models.Their Kentucky analysis found gains even for state-level models compared with models trained on nationwide data such as the Arnold PSA.Source: §10 · pp. 28–29
  • The authors identify cost-benefit analysis as a future direction because decision contexts differ in how much risk predictions matter and involve costs to multiple stakeholders.They note that complete data on costs and risks to all stakeholders may never be available.Source: §10 · p. 29

11 Appendix

The appendix documents the study’s validation and dataset construction procedures, along with Kentucky data-processing choices, the rationale for focusing comparisons on COMPAS and the Arnold PSA, and limitations in available features and records.

11.1 Nested Cross Validation Procedure

  • Five-fold nested cross-validation tuned hyperparameters on each outer training set and evaluated the selected model on each held-out fold.The five holdout results were used to calculate average and standard deviation of performance.Source: §11.1 · p. 36

11.2 Broward Data Processing

  • Broward features were computed for each COMPAS screening date, with recidivism labels at six months and two years and 1,954 records across 41 features.Only observations with the required follow-up data after release were retained.Source: §11.2 · pp. 36–37

11.3 Kentucky Data Processing

  • Kentucky CourtNet and PRIM records were joined, using a two-year cutoff to separate criminal-history features from later recidivism labels.Features and labels used convicted charges, while the current charge could be non-convicted; the procedure therefore included people later found innocent of that charge without using it in criminal-history features.Source: §11.3 · p. 39
  • The processed Kentucky dataset contains 250,778 records and 40 features; recidivism labels mark a convicted charge occurring within two years or six months of the current charge or release date.Source: §11.3 · p. 40
  • A sensitivity analysis excluding sentence information produced predictive results almost exactly the same as the main analysis, despite uncertainty in sentencing records and processing choices.The authors note that shock probation and early release may make recorded sentence lengths unreliable, and that exploring alternative processing choices is left for future studies.Source: §11.3 · pp. 40–41

11.4 Why We Compare Only Against COMPAS and the PSA

  • Because the available features are static except for current age, the study could not compare against risk assessments using dynamic factors; its available criminal-history data were also limited to Kentucky.Arnold PSA features incorporated in the dataset used in- and out-of-state information, while the remaining features were computed from in-state records.Source: §11.3 · p. 40, §11.4 · p. 41
  • Static-factor machine-learning models can outperform COMPAS despite lacking dynamic factors, which may add intervention guidance but do not necessarily improve prediction and can introduce data-entry errors or feature bias.The authors state that COMPAS uses both static and dynamic factors, whereas their interpretable and standard machine-learning models use static factors only.Source: §11.4 · pp. 41–42
  • COMPAS comparisons used scores released by ProPublica for Broward County, and the study did not compare against COMPAS in Kentucky because the required data were unavailable.The authors examine COMPAS general and violent recidivism scores; they could compute only the Arnold PSA from their criminal-history and age variables among the listed tools.Source: §11.4 · pp. 41, 43

Baseline Models, CART, EBM

The section reports nested cross-validation for hyperparameter tuning and identifies the models covered by Table 8, with RiskSLIM and Additive Stumps treated separately.

Baseline Models, CART, EBM

  • Hyperparameters were tuned using nested cross-validation.The passage directs readers to Table 8 for parameter details.Source: Baseline Models, CART, EBM · p. 44
  • Table 8 covers hyperparameters for ℓ1- and ℓ2-penalized logistic regression, linear SVM, CART, Random Forest, XGBoost, and EBM.RiskSLIM and Additive Stumps are discussed separately.Source: Table 8 · p. 44
  • RiskSLIM and Additive Stumps are excluded from Table 8’s listed model set and discussed separately.The table caption identifies the other covered models as logistic regression, linear SVM, CART, Random Forest, XGBoost, and EBM.Source: Table 8 · p. 44

Additive Stumps

Additive Stumps are formed from feature-specific stumps and constrained to use at most 15 original features. Nested cross-validation searches a grid of ℓ1 regularization parameters above a lower bound.

Additive Stumps

  • Additive models were built from feature-specific stumps using ℓ1-penalized logistic regression, with no more than 15 original features.Multiple stumps corresponding to an individual feature could still be included.Source: Additive Stumps · p. 44
  • The 15-feature limit was chosen so that at most 15 plots would be needed to visualize the full model.The authors describe this as a reasonable number of visualizations.Source: Additive Stumps · p. 44
  • Nested cross-validation searched a grid of ℓ1 regularization parameters beginning at the smallest value that yielded at most 15 original features.That minimum parameter served as the lower bound for the search.Source: Additive Stumps · p. 44

RiskSLIM

RiskSLIM’s Kentucky calibration curves increase monotonically and approximately satisfy group calibration across race and gender. The section also describes how RiskSLIM was trained and the computational constraints that shaped model selection.

RiskSLIM

  • RiskSLIM’s mixed-integer nonlinear optimization can be slow to prove optimality, making nested cross-validation difficult, and it requires a CPLEX license.EBM training was also slow on Kentucky data, so the authors tested only one parameter set.Source: RiskSLIM · pp. 44–45
  • RiskSLIM training used ℓ1-penalized logistic regression to select stump features, then fit RiskSLIM with nested cross-validation on the selected features.When RiskSLIM reached an optimality gap below 5% within 1,000 seconds, the procedure tried a larger feature subset and repeated the fit.Source: RiskSLIM · p. 45
  • Training stopped when RiskSLIM could not reach a 5% optimality gap within 1,000 seconds or adding stumps failed to improve validation performance.The procedure generally stopped with 12–20 stumps; more did not improve validation performance.Source: RiskSLIM · p. 45

11.6 Figures

  • The Arnold NVCA Raw curves approximately satisfy group calibration, while EBM satisfies neither group nor monotonic calibration across race and gender.For Arnold NVCA Raw, African-American and male curves are consistently higher than their comparison groups, especially at larger scores.Source: §11.6 · p. 47, Figure 10 · p. 47
  • RiskSLIM’s calibration curves increase monotonically and roughly overlap across race and gender, approximately satisfying group calibration on Kentucky two-year violent recidivism.The African-American curve is slightly higher than the Caucasian and “Other” race curves; gender curves are close.Source: §11.6 · p. 47, Figure 10 · p. 47
Loading 2005.04176v3…