PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Additive Models: Interpretable Machine Learning with Neural Nets

Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, Geoffrey Hinton

arXiv:2004.13912v2cs.LGcs.AIstat.ML

TL;DR

Deep neural networks are accurate but difficult to interpret, limiting their use in high-stakes decisions. NAMs address this by assigning jointly trained neural networks to individual features within a generalized additive structure. They achieve accuracy competitive with GAMs while supporting composable multitask extensions and differentiable interpretable models.

  • Problem

    Deep neural networks are difficult to understand, limiting their applicability in high-stakes domains such as healthcare.

  • Method

    NAMs parameterize each generalized additive shape function with a jointly trained neural network attending to a single input feature.

  • Results

    NAMs achieve comparable performance to EBMs on classification and regression datasets, while multitask NAMs obtain mean squared error 34% lower than single task NAMs on average.

  • Takeaways & Limitations

    NAMs combine GAM interpretability with DNN expressivity and are more easily extended than existing GAMs through differentiability and composability.

  • Takeaways & Limitations

    Higher-order feature interactions may improve expressivity while worsening intelligibility, and NAMs may be only a first step toward trustworthy systems requiring broader safeguards and accountability.

Abstract

from arXiv · show

Deep neural networks (DNNs) are powerful black-box predictors that have achieved impressive performance on a wide variety of tasks. However, their accuracy comes at the cost of intelligibility: it is usually unclear how they make their decisions. This hinders their applicability to high stakes decision-making domains such as healthcare. We propose Neural Additive Models (NAMs) which combine some of the expressivity of DNNs with the inherent intelligibility of generalized additive models. NAMs learn a linear combination of neural networks that each attend to a single input feature. These networks are trained jointly and can learn arbitrarily complex relationships between their input feature and the output. Our experiments on regression and classification datasets show that NAMs are more accurate than widely used intelligible models such as logistic regression and shallow decision trees. They perform similarly to existing state-of-the-art generalized additive models in accuracy, but are more flexible because they are based on neural nets instead of boosted trees. To demonstrate this, we show how NAMs can be used for multitask learning on synthetic data and on the COMPAS recidivism data due to their composability, and demonstrate that the differentiability of NAMs allows them to train more complex interpretable models for COVID-19.

1 Introduction

NAMs restrict neural-network structure so each feature contributes through its own learned shape function, combining GAM-style intelligibility with neural expressivity. They are designed to retain accuracy while supporting exact visualization, hardware scalability, and extensions such as multitask learning.

  • Motivation: The motivation is to address neural networks’ difficult-to-understand predictions, which limit their use in high-stakes domains such as healthcare, finance, and criminal justice.Local explanation methods may lack a global view and may not faithfully represent the model.
  • Model design: GAMs combine a link-transformed expected target with an intercept and one shape function per input feature.Logistic regression is a special case in which every shape function is linear.
  • Model design: NAMs parameterize each univariate GAM shape function with a jointly trained neural network that attends to one input feature.Feature effects can be visualized independently, and the resulting graphs exactly describe how predictions are computed.
  • Extensions: NAMs are more easily extended than tree-based GAMs because neural networks are differentiable and composable.The paper highlights multitask, multiclass, and multi-label learning, as well as more complex interpretable COVID-19 models.
  • Interpretability: NAM graphs provide an exact description of prediction computation rather than only a heuristic explanation.This supports intelligibility in high-stakes settings where black-box neural networks are difficult to understand.
  • Scalability: NAMs can use GPU, TPU, and other specialized hardware for scalable training and inference, unlike current GAM implementations.The paper also contrasts their small ensemble of 2 - 100 neural nets with accurate GAMs requiring millions of decision trees per shape function.

2 Neural Additive Models

NAMs use jointly trained neural networks, one per feature, to learn complex shape functions while preserving exact, feature-wise interpretability. ExU units improve learnability for jagged functions, while regularization helps control overfitting.

  • Neural Additive Models: Standard ReLU networks struggle to fit highly jumpy one-dimensional functions under mini-batch gradient descent despite sufficient architectural expressivity.The authors associate this difficulty with a bias toward smoothness when fitting large local fluctuations.
  • Neural Additive Models: ExU hidden units learn weights in logarithmic space with shifted inputs, making steep slopes achievable with smaller weights and improving learnability for jagged functions.ExU units do not increase neural-network expressivity; they make sharp changes easier to learn during training.
  • Neural Additive Models: ExU networks require suitable initialization and strong regularization because realistic shape functions are usually smooth with only a few sharp jumps.The work uses initialization that produces jagged starting functions and methods including dropout, weight decay, output penalty, and feature dropout.
  • Neural Additive Models: Dropout makes ExU feature-net predictions smoother on average while individual dropped-out subnet predictions remain more jagged.The illustrated setting uses dropout = 0.2 for the age feature in MIMIC-II.
  • Intelligibility and Modularity of NAMs: NAM visualizations support modular interpretation by showing feature contributions that are centered across the training data and combined with a single bias term.For individual predictions, feature-net outputs act as contribution terms that are added before the link function.

3 Evaluating the Accuracy of NAMs

The evaluation compares NAMs with established intelligible models, boosted-tree GAMs, and unrestricted DNNs across regression and classification tasks. NAMs achieve performance comparable to EBMs while their learned graphs expose meaningful feature relationships in housing data.

  • Interpretations: California Housing graphs show house prices increasing linearly with median income in high-density regions and jumping around longitudes near San Francisco and Los Angeles.The learned NAM identifies median income and location as the two most important features in the illustrated analysis.
  • Evaluation Setup: The evaluation measures NAM accuracy against logistic or linear regression, CART decision trees, Explainable Boosting Machines, and unrestricted DNNs.Binary classification uses AUC, while regression uses RMSE, with 5-fold cross-validation.
  • Accuracy Results: NAMs achieve comparable performance to EBMs on both classification and regression datasets, making them a competitive alternative to boosted-tree GAMs.The paper then examines capabilities not available to EBMs or other tree-based learning methods.

4 Unique Capabilities of NAMs

NAMs extend interpretable additive modeling to differentiable multitask and treatment-benefit settings. Experiments show gains on synthetic multitask data and reveal task-specific patterns in COMPAS recidivism prediction.

  • 4.1 Intelligible Parameter Generation: Leveraging the Differentiability of NAMs: NAMs generate interpretable treatment-benefit maps by predicting mortality risk under each treatment from patient information.The model was trained on deidentified data from over 3000 COVID-19 patients.
  • 4.2 Multitask Learning: NAMs support multitask learning by combining multiple feature-specific subnets with task-specific weighted sums that produce each task’s shape functions.The number of subnets may be less than, equal to, or greater than the number of tasks.
  • 4.2.1 Experiments on Synthetic Multitask Data: 34% lower mean squared error was achieved by multitask NAMs than single-task NAMs across synthetic tasks, with at least 25% lower error on every task.Across 120 combined trials, multitask learning scored better in 119.
  • 4.2.2 Single and Multitask COMPAS Recidivism Prediction: Single-task NAM shape functions exposed a potential racial bias in COMPAS predictions and allowed the race contribution to be removed after training.The authors note that removing race would reduce AUC because race is discriminative, while retaining it during training helps detect bias.
  • 4.2.2 Single and Multitask COMPAS Recidivism Prediction: COMPAS multitask NAMs model men and women as separate outputs, revealing gender-specific relationships among race, charge degree, and recidivism risk.The multitask model achieved slightly higher overall accuracy than the single-task model.

5 Related Work

NAMs build on earlier additive neural and tree-based models while using deeper networks and backpropagation to improve flexibility, nonlinear modeling, and scalability.

  • Prior additive neural models: Earlier GANNs imposed additive neural architectures but used shallow networks, no backpropagation, and human-in-the-loop evaluation.The paper contrasts these limitations with NAMs’ deep-learning-based training and scalability.
  • NAM architecture: NAMs use many hidden units, multiple hidden layers, ExU units, and ensembles to learn complex nonlinear shape functions and provide uncertainty estimates.The ensemble also aims to improve accuracy and reduce variance from highly nonlinear function learning.
  • Tree-based GAMs: Tree-based GAM and GA2M models were prior state-of-the-art interpretable additive baselines, while NAMs can add pairwise interactions more flexibly.The paper compares NAM accuracy against these boosted-tree models.

6 Conclusion and Future Work

The paper presents NAMs as expressive, interpretable alternatives to black-box and tree-based models, while identifying higher-order interactions and broader applications as future directions.

  • Conclusion: NAMs combine GAM interpretability with DNN expressivity and are competitive with GAMs while offering accurate alternatives to shallow interpretable models.Their differentiability and composability also make them easier to extend than existing GAMs.
  • Future work: Higher-order feature interactions could improve NAM expressivity but may reduce intelligibility, motivating research on selecting a small number of crucial interactions.The authors identify this balance as an important future-work direction.
  • Future work: Improved activation functions or feature representations, including Fourier features, may help NAMs express complex or high-frequency functions.The authors cite Fourier features as effective for learning high-frequency functions with neural networks.
  • Future work: Future work includes extending NAMs beyond tabular data to high-dimensional tasks such as computer vision and language understanding.The paper also discusses multimodal inputs and a genomics application using a CNN-LSTM-based NAM extension.

Broader Impact

NAMs support interpretability-related goals including bias auditing, system improvement, regulatory compliance, and individual challenges to model outputs. However, their use must be considered within broader socio-technical safeguards and competing demands such as accuracy and fairness.

  • NAMs can help identify and correct biased or discriminatory uses of training data, including in recidivism risk prediction.
  • Their feature-level interpretations let developers investigate unexpected system behavior and uncover patient risks before deployment.
  • NAM explanations can support regulatory requirements and enable individuals to contest decisions such as unsuccessful loan applications.
  • Interpretability needs vary by context, and improving fairness can require trade-offs with predictive accuracy.
  • NAMs may be only one component of trustworthy systems that also require safeguards, audits, accountability, and socio-technical consideration.

Checklist

The checklist records affirmative responses for reporting claims, limitations, societal impacts, ethics guidance, reproducibility, training details, error bars, compute, and asset licensing. It also records that human-subject and crowdsourcing requirements were not applicable where indicated.

  • The authors report that they discussed the accuracy of their main claims, limitations, and potential negative societal impacts.
  • They report providing code, data, reproducibility instructions, training details, error bars, compute information, and asset citations or licenses.
  • New assets are marked not applicable.
  • They report that the paper conforms to ethics review guidelines and includes required consent and personally identifiable information discussions.
  • Crowdsourcing and human-subject requirements are marked not applicable, including participant instructions, risks, compensation, and hourly wages.

A.1 NAMs on MIMIC-II: Mortality Prediction in ICUs

NAM shape functions provide an interpretable view of ICU mortality predictions from MIMIC-II medical features, with most learned patterns consistent with medical knowledge but several anomalies requiring investigation. The plots describe the model’s predictions exactly, while not establishing causality.

  • NAM shape functions use ICU medical features to show their learned contributions to mortality risk, with lower y-values indicating lower risk.
  • Mortality risk generally increases with age, rising most rapidly above age 80, while small jumps near ages 18, 19, and 90 warrant study.
  • Bilirubin risk is low below 2–3, rises above 15–20, drops above 50, and includes an unexplained dip near 35.
  • Higher Glasgow Coma Index corresponds to lower risk, while severe renal failure corresponds to higher risk; these monotone shapes are data-driven rather than imposed.
  • Risk is lowest for normal heart rate, rises above 100, and increases with lymphoma, metastatic cancer, and abnormal CO2-related patterns.
  • A dip at PFratio = 332 reflects mean-imputed missing values, illustrating how shape plots can expose data-processing artifacts.
  • Most learned patterns align with medical knowledge, but NAMs are not causal models and their plots do not predict intervention effects.

A.2.1 FICO Score: Understanding Individual Predictions on Credit Scores

NAMs provide feature-level views of credit-score predictions, allowing individual applicant contributions to be inspected. For a high-scoring applicant, credit-history length and revolving-credit burden are identified as especially important features.

  • FICO Score: The FICO dataset contains anonymized credit applications with assigned FICO Scores, and NAM feature contributions are visualized for applicants with low and high scores.These visualizations support inspection of individual predictions rather than only aggregate model performance.
  • Interpretation: NAM feature plots expose how individual credit features contribute to predictions without relying on interactions with other features.This section applies that feature-level view to applicant-specific credit-score explanations.
  • High-scoring applicant: Average Months on File and Net Fraction Revolving Burden are the most important features for the high scoring applicant.Their values are 235 months and 0%, respectively.
  • High-scoring applicant: For Net Fraction Revolving Burden, using some credit is positively correlated with score at small values below 10, whereas excessive use is described as unfavorable.The interpretation is supported by the observed data density and similar shape functions across the NAM ensemble.
Loading 2004.13912v2…