PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning

Ziming Wang, Changwu Huang, Ke Tang, Yew-Soon Ong, Xin Yao

arXiv:2609.05946v1cs.LG

TL;DR

NN-based GAM research has emphasized accuracy while leaving explicit interpretability evaluation and the joint study of interpretability, fairness, and accuracy underexplored. The paper introduces MONBM, combining explicit interpretability metrics, evolutionary multi-objective learning, and partial retraining. It obtains models with varied trade-offs, reveals complex relationships among the objectives, and reports competitiveness with state-of-the-art methods.

  • Problem

    NN-based GAM research provides limited explicit measurement of interpretability and rarely studies fairness jointly with interpretability and accuracy.

  • Method

    MONBM jointly optimizes accuracy, interpretability, and fairness using evolutionary multi-objective learning, explicit smoothness and monotonicity metrics, and partial retraining of a pre-trained NBM.

  • Results

    MONBM obtains models with different trade-offs, reveals complex relationships among the dimensions, and is competitive with state-of-the-art self-interpretable and black-box models.

  • Takeaways & Limitations

    Combining evolutionary multi-objective optimization with self-interpretable models provides a way to analyze trade-offs and underlying relationships among trustworthiness objectives.

  • Takeaways & Limitations

    The current MONBM instantiation is designed for tabular datasets and is not directly applicable to raw pixel-level representations.

Abstract

from arXiv · show

Interpretability and fairness are two of the most emphasized dimensions in trustworthy artificial intelligence (AI). Various explainable AI methods have been introduced to improve interpretability. This paper focuses on neural network (NN)-based generalized additive models (GAMs), a class of self-interpretable models. While most existing research has prioritized improving the accuracy of NN-based GAMs, their interpretability remains largely underexplored. To address this gap, this paper introduces explicit quantitative metrics for evaluating the interpretability of NN-based GAMs, empirically examines their effectiveness, and explores strategies for improving interpretability within these models. In addition, the simultaneous and explicit optimization of both interpretability and fairness, along with their trade-offs and the underlying reasons, remains underexplored. To address this, we propose a multi-objective neural basis model (MONBM) framework based on multi-objective evolutionary learning to consider accuracy, interpretability, and fairness simultaneously. A partial retraining strategy is further developed to facilitate the practical application of evolutionary multi-objective optimization to deep model architectures. Based on MONBM, this paper reveals the complex relationships between these dimensions and the reasons behind these intricate relationships. This analysis demonstrates how multi-objective optimization can be combined with self-interpretable models to reveal relationships among trustworthiness objectives. In addition, MONBM obtains a set of models with different trade-offs between dimensions, and the competitiveness of the approach is validated by comparing it with state-of-the-art methods.

1. Introduction

The paper addresses limited attention to interpretability and fairness in NN-based GAMs by jointly optimizing accuracy, interpretability, and fairness through MONBM. It introduces explicit interpretability metrics, an efficient partial-retraining strategy, and multi-objective models with different trade-offs.

  • Motivation: Trustworthy AI emphasizes interpretability and fairness, but NN-based GAM research has primarily prioritized predictive accuracy.GAMs provide global and local explanations, yet explanation quality is not guaranteed.
  • Motivation: Existing fairness methods mainly target black-box models, while fairness in self-interpretable models such as GAMs remains rarely studied.The paper focuses on group fairness, requiring equal treatment of groups defined by sensitive attributes.
  • Research gap: Accuracy, interpretability, and fairness are often studied separately, with limited systematic analysis of their relationships and underlying mechanisms.The paper frames these dimensions as potentially conflicting objectives.
  • Approach: MONBM formulates trustworthy-model construction as multi-objective learning that simultaneously considers accuracy, interpretability, and fairness.The framework uses evolutionary multi-objective learning and supports multiple objective configurations.
  • Contributions: The paper introduces explicit smoothness and monotonicity metrics for NN-based GAMs, making interpretability properties quantifiable and independently optimizable.The metrics are experimentally evaluated for effectiveness and for obtaining NBMs with desirable global interpretability.
  • Contributions: MONBM produces NBMs with different accuracy–interpretability–fairness trade-offs and is reported as competitive with state-of-the-art self-interpretable and black-box models.The framework also uses explanations and solution-set diversity to examine relationships among trustworthiness objectives.

2. Background

GAMs explain predictions through additive feature-specific shape functions, supporting global and local views. NN-based GAM research improves scalability and accuracy, but fairness and interpretability are rarely studied together or explicitly optimized.

  • GAM foundations: GAMs sum independent nonlinear shape functions to model feature effects and target relationships.
  • GAM explanations: Global explanations visualize each shape function across feature values, while local explanations show feature contributions for a specific data point.
  • NN-based GAMs: NBM improves NN-based GAM scalability by using one shared basis network and feature-specific linear combinations instead of one deep network per feature.
  • Interpretability gaps: Local interpretability evaluates explanation complexity, but prior complexity metrics were largely developed for post-hoc explanations rather than NN-based GAMs.
  • Fairness and global interpretability: Existing NN-based GAMs lack explicit monotonicity metrics, while fairness methods are difficult to adapt from black-box models to additive structures.
  • Research positioning: Prior GAM studies usually address predictive performance, fairness, local interpretability, or global interpretability separately rather than jointly.

3. Multi-objective Evolutionary Learning to Construct Accurate, Interpretable, and Fair Models

This section defines explicit global-interpretability metrics and presents MONBM, a multi-objective evolutionary framework for constructing accurate, interpretable, and fair models. Its instantiation uses partial retraining and produces non-dominated models representing different objective trade-offs.

  • Smoothness and monotonicity metrics: Smoothness measures gradual shape-function changes, whereas monotonicity captures consistent increasing or decreasing trends; monotonicity is not universally required.
  • Smoothness and monotonicity metrics: The smoothness metric Ms and monotonicity metrics Mm↑ and Mm↓ explicitly quantify global interpretability from sampled shape-function values.
  • MONBM framework: MONBM uses evolutionary multi-objective learning to optimize accuracy, interpretability, and fairness simultaneously and obtain models with different trade-offs.
  • MONBM framework: The evolutionary process evaluates models, selects parents, generates offspring, partially trains them, and returns non-dominated solutions.
  • MONBM instantiation: The final models approximate a Pareto front, representing different trade-offs among the considered objectives.
  • MONBM instantiation: The instantiation pre-trains an NBM, freezes shared basis functions, and evolves only the feature-specific layer through partial retraining.
  • Evaluation objectives: The objectives include cross-entropy-based accuracy, group fairness measured by DP, local interpretability, and global interpretability.

4. Experimental Studies

The experiments examine how MONBM balances global and local interpretability with accuracy and fairness. Results show smoother, more monotonic global explanations, while moderate fairness or local-interpretability improvements preserve accuracy better than extreme optimization.

  • Global interpretability: GI improvements progressively smooth shape functions, reduce oscillations, preserve primary trends, and strengthen monotonic tendencies without guaranteeing perfect monotonicity.These visual changes align with GI metric values across datasets.
  • Fairness and accuracy: A 50% DP reduction decreases ACC by only 0.48% on average across six datasets, whereas reducing DP to zero decreases ACC by 17.06% on three datasets and 2.15% on the other three.The effect of extreme fairness varies substantially by dataset.
  • Fairness and accuracy: Moderate fairness primarily adjusts the sensitive attribute’s contribution with little accuracy impact, while extreme fairness can require broader feature changes and substantial accuracy losses.The authors report that the mechanisms underlying extreme fairness effects differ across datasets.
  • Local interpretability and accuracy: Moderate LI improvement reduces ACC by 2.03% on average, but extreme LI reduces it by 12.59%, exceeding the average impact reported for DP.The LI objective also shifts relative feature importance more strongly because it promotes sparsity across the feature set.
  • Local interpretability and accuracy: Extreme LI favors a small number of informative features and removes most other feature contributions, significantly reducing predictive power; local-importance rankings nevertheless remain generally consistent across trials.Variance is non-negligible at extreme fairness and local-interpretability points.

5. Discussion

The discussion scopes the paper’s conclusions across interpretability, fairness metrics, data modalities, and generalization. It reports limited overfitting but notes that some relationships and applications require further validation.

  • Scope of conclusions: The conclusions about interpretability are drawn from NN-based GAMs and may extend to other self-interpretable models with analogous local explanation structures.The paper relates NN-based GAM local interpretability to term counts, tree depth, and rule counts in other self-interpretable models.
  • Fairness scope: The fairness analysis primarily uses demographic parity, although sensitivity analyses indicate similar patterns under equal opportunity and equalized odds.The authors state that conclusions require further validation for other fairness categories despite consistency across DP, EOP, and EOD.
  • Data scope: MONBM is designed for tabular datasets and is not directly applicable to raw pixel-level inputs because isolated pixels lack meaningful semantics.Structured modalities may require transforming inputs into higher-level concepts before applying additive basis functions.
  • Empirical reliability: Training and test results were consistent in RQ2, suggesting limited overfitting, but non-dominance inconsistencies remain possible.The authors plan to use a validation set to mitigate this risk further.

6. Conclusion

The paper introduces explicit interpretability metrics and MONBM to jointly optimize accuracy, fairness, and interpretability in NN-based GAMs. MONBM exposes objective trade-offs and provides models with different trustworthiness compromises, while future work broadens validation and scope.

  • Contributions: The paper introduces smoothness and monotonicity metrics and a MONBM framework that jointly optimizes accuracy, fairness, and local and global interpretability.Partial retraining of the pre-trained NBM’s feature-specific layer makes evolutionary multi-objective optimization practical for large-scale networks.
  • Findings: Fairness, local interpretability, and global interpretability each trade off with accuracy, while their mutual relationships can be complementary or conflicting across datasets.The paper uses MONBM to examine these relationships and their underlying reasons.
  • Future work: Future work will test additional self-interpretable models, fairness metrics, high-dimensional problems, and other trustworthy AI dimensions.The planned extensions include localized smoothness control and analysis of robustness and security trade-offs.

Data availability

The experiments used publicly available or open-source datasets, with dataset details and access information provided in the manuscript.

  • Data availability: The experiments used publicly available or open-source datasets.Dataset details and access information are provided in the manuscript.

CRediT authorship contribution statement

The authors’ contributions span writing, methodology, data curation, validation, conceptualization, supervision, funding, project administration, and resources.

  • Contributions: Ziming Wang contributed to writing the original draft, methodology, data curation, and validation.
  • Contributions: Changwu Huang, Ke Tang, Yew-Soon Ong, Xin Yao contributed to writing or review, supervision, and funding-related roles, with Xin Yao also handling project administration and resources.

S-I. Dataset Under Consideration

The study uses six widely used fairness datasets, with dataset size, feature composition, and sensitive attributes documented in Table S-I.

  • Six datasets widely used in fairness research are included in the study.Their specific characteristics are reported in Table S-I.
  • Table S-I defines |D|, |X|, |Xdisc.|, and |Xnum.| as sample count, total features, discrete features, and numerical features.
  • S denotes the sensitive attribute considered for each dataset.

S-II. Algorithmic and Computational Properties of MONBM

MONBM combines multi-objective evolutionary optimization with partial parameter updates, producing diverse trade-off solutions while limiting per-model computation.

  • Baseline single-point methods were trained for 1,200 epochs, while FairEMOL used 200 generations and approximately 68,000 parameters.SNAM and GAMI-Net were trained for 1,200 and 5,000 epochs, respectively.
  • MONBM updates a population of 100 models per generation, but only approximately 2% of each NBM’s total parameters are updated.
  • Unlike single-point methods that produce one solution per run, MONBM generates diverse trade-off solutions in a single run.This supports downstream analysis and flexible deployment.
  • The MONBM instantiation analyzes reproduction, partial training, fitness evaluation, and environmental selection within each generation.
  • Per-generation time complexity is O(kτ^2 + τ · n · D log D), with quadratic dependence on population size.
  • Overall space complexity is O((τ + n)D), scaling linearly with feature dimension D because shared bases are frozen and shared.

S-II.3. Robustness and Generalization Analysis

The analysis links MONBM’s generalization and practical behavior to additive structure, restricted trainable parameters, smoothness regularization, and controllable accuracy–interpretability trade-offs.

  • Generalization behavior: As a GAM, MONBM limits high-order feature interactions, which can lower variance and mitigate overfitting.
  • Generalization behavior: Restricting evolutionary optimization to approximately 2% of total parameters reduces the effective hypothesis space relative to fully trainable deep models.
  • Generalization behavior: The smoothness objective Ms discourages oscillatory shape functions, helps avoid fitting high-frequency noise, and promotes stable global trends.
  • Generalization behavior: Across 15 independent trials, performance was consistently observed on both training and test sets, with relatively small training–test differences.
  • Practical scalability: MONBM execution time averaged approximately 12.4 times that of a single NBM training run across six datasets.The comparison used single-threaded experiments on a 64-core Linux server with 512GB RAM.
  • Practical scalability: Because one MONBM run yields diverse trade-off solutions, approximating a comparable Pareto front with scalarized optimization would typically require many independent runs.
  • Sensitivity analysis: Decreasing η makes the smoothness constraint stricter, shifts the Pareto front toward lower Ms values, and produces smoother shape functions.
  • Sensitivity analysis: Users can calibrate η to move between high-accuracy predictive modeling and smoother, more interpretable global explanations.

S-V. Required Monotonic Features

The paper specifies dataset-dependent monotonicity requirements and evaluates MONBM against other methods using dominance-based comparisons and training-set analyses.

  • Required monotonic features: Required monotonically increasing and decreasing features are specified separately for each dataset in Table S-IV.
  • Required monotonic features: For Adult, hours_per_week is constrained to have a monotonically increasing relationship with the probability of earning over $50,000 annually.The setting reflects the paper’s stated expectation that higher weekly hours increase this probability.
  • Comparative evaluation: The comparison uses mutual dominance relationships between MONBM and both population-based and single-solution methods.
  • Comparative evaluation: For population-based methods, 100 solutions from each method yield 10,000 pairwise comparisons.
  • Comparative evaluation: Larger Dominate values indicate that MONBM solutions dominate baseline solutions in more trials.
  • Comparative evaluation: The Incomparable metric measures the average proportion of MONBM solutions incomparable with baseline solutions, while Dominated measures the proportion dominated by baseline solutions.
  • Training-set analysis: Training-set analyses examine ACC, DP, LI, and GI for selected solutions across 15 trials and multiple MONBM instantiations.

S-VIII. Sensitivity Analysis to Fairness Metrics: Results on Equalized Odds (EOD)

The EOD sensitivity analysis reproduces the main accuracy–fairness trade-off observed with DP. Moderate EOD improvement has little accuracy impact, whereas extreme fairness can substantially reduce accuracy and alter feature explanations.

  • Results: 0.64% average ACC decrease accompanies a 50% EOD reduction across six datasets at the “middle EOD” point.Extreme fairness (EOD ≈0) causes substantial accuracy declines on Adult, COMPAS, and Dutch, but only moderate effects on Bank, Default, and LSAT.
  • Results: EOD produces a fairness–accuracy trade-off consistent with the pattern observed for DP.The analysis supports this conclusion across different fairness definitions rather than only for DP.
  • Global explanations: EOD-constrained models retain feature-explanation trends and feature-importance shifts highly similar to DP-optimized models.The comparison concerns global explanations from the MONBM-CE instantiation and the corresponding DP results.
  • Stability: Feature-importance variability increases away from Best CE solutions toward middle and extreme fairness or local-interpretability operating points.Across 15 trials, standard deviations remain relatively low near Best CE and rise at the other operating points.
  • Stability: Explanation fluctuations reflect both weaker optimization sharpness under strong constraints and variation in knee-point compromises across training samples.Strong fairness or interpretability constraints can reduce accuracy and permit multiple satisfying feature configurations, while middle points vary as Pareto fronts shift.
  • Interpretation: Per-feature importance variance does not by itself indicate framework unreliability because GAM explanations remain fully faithful to each underlying model.Differences in shape functions across runs primarily reflect training-sample variation.

S-XI. Ablation Analysis of Global Interpretability: Smoothness versus Monotonicity

The ablation separates global interpretability into smoothness and monotonicity, showing that both usually preserve accuracy better than fairness constraints but can conflict with local interpretability. Their fairness interactions differ across datasets, especially on Dutch.

  • Ablation results: Smoothness and monotonicity impose relatively limited accuracy degradation compared with fairness and local-interpretability constraints.Best GI models maintain relatively high predictive performance, whereas extreme DP often causes substantially larger accuracy reductions.
  • Ablation results: Global interpretability constraints mainly reshape local geometric properties of shape functions rather than fundamentally changing the overall decision boundary.The relevant geometric properties are smoothness and monotonicity.
  • Trade-offs: Local interpretability trades off persistently with both smoothness and monotonicity because sparse feature usage differs from globally regulated feature responses.These interpretability objectives operate through different mechanisms and cannot always improve simultaneously.
  • Dataset-specific behavior: On Dutch, the fairness–interpretability conflict is primarily associated with smoothness rather than monotonicity.Under Best Mm, fairness remains relatively stable at DP = 0.054, whereas smoothing suppresses local fluctuations and can make sex dominant.
  • Conclusion: Global interpretability is not homogeneous: smoothness and monotonicity interact differently with fairness depending on the underlying data distribution.The ablation therefore distinguishes interpretability constraints rather than treating global interpretability as one objective.

S-XII. Preliminary Exploration on Adaptive Selection Pressure

The adaptive selection-pressure study addresses weakened evolutionary search in four-objective MONBM optimization by increasing convergence pressure over generations. Its results generally improve fairness- and interpretability-oriented solutions while preserving competitive predictive performance, although many-objective trade-offs remain.

  • Motivation: Increasing objective count can weaken selection pressure and make promising solutions harder to distinguish, reducing convergence and final solution quality.This motivates the preliminary adaptive-selection-pressure study for MONBM-CDLG.
  • Adaptive strategy: The quadratic schedule starts with broader exploration and progressively increases convergence pressure during later generations.The schedule uses the current generation, maximum generations, initial probability, and target final probability.
  • Results: Adaptive selection pressure generally improves solution quality for fairness-, local-interpretability-, and global-interpretability-oriented solutions while maintaining competitive predictive performance.The comparison evaluates MONBM with adaptive selection pressure against the original fixed-probability configuration on Adult over 15 trials.
  • Limitations: Adaptive pressure does not completely eliminate the performance gap relative to lower-dimensional optimization settings.The remaining gap is attributed to inherent trade-offs in many-objective learning, and more advanced mechanisms remain future work.
  • Baseline comparison: Across four objectives, MONBM-CDLG solutions are never dominated by the ten baselines, with a 0.000 “Be-Do.” probability across datasets.It frequently dominates weaker baselines but rarely achieves complete dominance over most competitive methods.
Loading 2609.05946v1…