Source-linked AI summary
Personalized Counterfactual Fairness in Recommendation
Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, Yongfeng Zhang
TL;DR
Recommendation fairness must account for users’ different sensitive-feature concerns and move beyond association-based notions toward causal, counterfactual fairness. The paper proposes adversarial learning for feature-independent user embeddings, and experiments report fairer recommendations with desirable recommendation performance. The work remains an early step focused on recommendation, with broader extensions left for future improvement.
Problem
Existing recommendation-fairness work largely uses association-based notions and often applies the same sensitive features to all users, neglecting personalized causal fairness demands.
Method
The paper uses adversarial learning to generate feature-independent user embeddings, allowing users to select concerned sensitive features for personalized counterfactual fairness.
Results
Experiments on two real-world datasets with shallow and deep models show fairer recommendations while retaining desirable recommendation performance.
Takeaways & Limitations
Personalized counterfactual fairness can be incorporated into recommendation through adversarially learned embeddings while maintaining recommendation quality close to the original performance.
Takeaways & Limitations
The work is an early step toward personalized fairness under counterfactual notions and focuses on recommendation, leaving broader improvements and applications for future work.
Abstract
from arXiv · showhide
Recommender systems are gaining increasing and critical impacts on human and society since a growing number of users use them for information seeking and decision making. Therefore, it is crucial to address the potential unfairness problems in recommendations. Just like users have personalized preferences on items, users' demands for fairness are also personalized in many scenarios. Therefore, it is important to provide personalized fair recommendations for users to satisfy their personalized fairness demands. Besides, previous works on fair recommendation mainly focus on association-based fairness. However, it is important to advance from associative fairness notions to causal fairness notions for assessing fairness more properly in recommender systems. Based on the above considerations, this paper focuses on achieving personalized counterfactual fairness for users in recommender systems. To this end, we introduce a framework for achieving counterfactually fair recommendations through adversary learning by generating feature-independent user embeddings for recommendation. The framework allows recommender systems to achieve personalized fairness for users while also covering non-personalized situations. Experiments on two real-world datasets with shallow and deep recommendation algorithms show that our method can generate fairer recommendations for users with a desirable recommendation performance.
1 INTRODUCTION
The paper argues that recommendation fairness should reflect users’ different sensitive-feature concerns and move beyond association-based assessment toward personalized counterfactual fairness. It proposes adversarial learning to generate feature-independent user embeddings for fair recommendations.
- Motivation: User-side fairness is underexplored because different users can have different concerns about sensitive features.Some users may reject gender-influenced recommendations, whereas others may be more concerned about age.
- Motivation: Association-based fairness measures statistical discrepancies but cannot reason about causal relations between protected features and model outcomes.The paper motivates causal fairness partly through limitations illustrated by Simpson’s paradox.
- Objective: The paper pursues personalized counterfactual fairness, an individual-level causal notion suited to user-specific fairness demands.Counterfactual fairness compares outcomes in factual and counterfactual worlds.
- Method: The proposed framework uses adversarial learning to remove sensitive-feature information from user embeddings and thereby produce feature-independent recommendations.It controls dependence between sensitive features and embeddings within the recommendation pipeline.
- Evaluation: Experiments on two real-world datasets with shallow and deep models evaluate whether the framework improves recommendation fairness.The stated contribution includes testing the approach across both model types.
2 RELATED WORK
Related work distinguishes association-based from causal-based fairness and explains why recommendation requires attention to users’ personalized demands alongside its multi-stakeholder setting. The paper adopts counterfactual fairness as a fine-grained individual-level causal notion.
- Fairness foundations: Fairness research spans group and individual fairness, but individual fairness is harder to define because similarity metrics vary across tasks.The distinction concerns whether protected groups or similar individuals receive similar treatment.
- Association-based Fairness: Association-based fairness seeks discrepancies in statistical metrics across individuals or sub-populations, including in classification and ranking.Ranking work often evaluates exposure disparities between protected item groups.
- Association-based Fairness: Recommendation fairness must address user-side demands because personalization is fundamental and users may require different fairness criteria.This distinguishes recommendation from prior item-side fair-ranking work.
- Causal-based Fairness: Causal-based fairness models relations between inputs and outcomes, supporting reasoning about how sensitive-feature changes propagate through a system.The related work contrasts this with association-based notions that cannot represent such causal relations.
- Causal-based Fairness: Causal fairness includes total effect, treatment on the treated, disparate treatment, and path-specific effects, which constrain different causal pathways.These notions respectively address overall effects, counterfactual changes, treatment use, and selected paths.
- Causal-based Fairness: Counterfactual fairness is a fine-grained treatment-on-the-treated variant requiring equal outcome distributions in factual and counterfactual worlds for every individual.The paper applies this individual-level notion to recommendation.
- Fair Recommendation: Recommendation fairness is more complex than fair classification or ranking because recommendation systems involve multiple stakeholders and diverse fairness objectives.Prior work covers group recommendation, collaborative filtering, ranking, post-processing, and two-sided marketplaces.
3 PRELIMINARIES AND NOTATIONS
The preliminaries define recommendation inputs and outputs, introduce counterfactual reasoning, and formalize counterfactually fair recommendation. Fairness requires recommendation distributions to remain unchanged when only specified sensitive features are altered.
- Notation: The notation distinguishes variables from values and uses bold symbols for sets of variables or values.Capital letters denote variables, lowercase letters denote specific values, and bold forms denote sets.
- Recommendation task: A recommendation task contains users, items, and a binary interaction matrix whose entries indicate user-item interactions.The model learns preference scores and produces top-N recommendation lists.
- Counterfactuals: Counterfactual reasoning asks what would have happened under an alternative value while conditioning on the observed world.The formulation distinguishes observed and counterfactual worlds.
- Counterfactual fairness: Counterfactual fairness requires each individual’s predicted result to remain the same after intervening on sensitive features while keeping non-dependent features unchanged.This is an individual-level causal fairness notion.
- Counterfactually fair recommendation: For recommendation, fairness requires the Top-N list’s probability distribution to be equal across factual and counterfactual sensitive-feature values.The definition holds non-sensitive features X fixed while changing sensitive features Z.
4 FAIR RECOMMENDATION FRAMEWORK
The framework targets personalized counterfactual fairness by removing selected sensitive-feature information from user embeddings while preserving recommendation functionality. It uses adversarial learning, with separate or combinational filters, and provides a theoretical independence guarantee under stated assumptions.
- Problem formulation: Counterfactual fairness requires recommendation results to remain unchanged when specified sensitive features change in a counterfactual world.
- Problem formulation: Sensitive features can affect recommendations indirectly through user histories and learned user embeddings, even when models do not directly use those features.
- Personalized filtering: Users select the sensitive-feature set for which recommendations should be fair, supporting personalized demands rather than a single fixed fairness configuration.
- Adversarial framework: The framework filters sensitive information from user embeddings, while an adversarial classifier predicts protected features and the recommender optimizes the main task.
- Personalized filtering: The Separate Method trains filters for feature combinations, whereas the Combination Method trains one filter per feature and combines the resulting embeddings.
- Theoretical guarantee: Under sufficient capacity and optimization assumptions, Theorem 4.1 gives r_u⊥Z_u as λ→∞.
5 EXPERIMENTS
The experiments compare baseline models with Separate and Combination fairness models, then analyze recommendation performance and fairness through ablation studies.
- The experimental section introduces datasets, baselines, and setup before presenting recommendation-performance and fairness comparisons.
- The study includes comparisons between baseline models and the two fairness models.
- Ablation studies further analyze the algorithm after the main experimental comparisons.
5.1 Dataset Description
Experiments use two real-world public datasets representing movie and insurance recommendation, with user-sensitive features defined for each domain. Dataset splits use 80% training, 10% validation, and 10% testing.
- The experiments use MovieLens-1M and a Kaggle insurance recommendation dataset to cover movie and insurance recommendation scenarios.
- MovieLens1: MovieLens-1M uses gender, age, and occupation as sensitive features, with age grouped into seven ranges and occupation represented by 21 classes.
- Insurance: The insurance dataset uses gender, marital_status, and occupation as sensitive features.
- Insurance: Minority classes in the insurance data are grouped into three classes, and users with fewer than four interactions are removed.
- Each dataset is split into train (80%), validation (10%), and test (10%) sets shared by all baseline models.
5.2 Evaluation Methods
Evaluation measures recommendation quality with NDCG and Hit rate and measures sensitive-feature leakage with attacker AUC. The evaluation uses dataset-specific top-N metrics and reports lower attacker AUC as better.
- Recommendation quality: Recommendation quality is evaluated with NDCG@N and Hit@N, using NDCG@5 and Hit@5 for MovieLens and NDCG@3 and Hit@3 for Insurance.
- Recommendation quality: Sampled negative interactions are used for evaluation instead of scoring every user-item pair across the full item space.
- Fairness evaluation: Attackers matching discriminator structure and capacity classify sensitive labels from filtered user embeddings to assess residual sensitive information.
- Fairness evaluation: Attacker performance is reported with AUC, where scores range from 0.5 to 1 and lower values are better.
- Fairness evaluation: For multi-class evaluation, AUC is macroaveraged across all feature pairs to reduce sensitivity to class imbalance.
5.3 Baselines
The experiments compare shallow and deep recommendation baselines, including matrix-factorization and neural models, across two real-world datasets. Dataset statistics and attacker AUC results are reported for the evaluation setup.
- Baseline Models: The study evaluates the proposed framework alongside PMF, BiasedMF, DeepModel, and DMF baselines.PMF and BiasedMF are matrix-factorization methods, while DeepModel and DMF are deep recommendation models.
- Datasets: The dataset statistics are summarized in Table 1.
- Evaluation Targets: Table 2 reports attacker AUC scores for sensitive features including gender, age or marital status, and occupation.MovieLens uses gender, age, and occupation; Insurance uses gender, marital status, and occupation.
5.4 Experimental Settings
The evaluation uses standard top-N recommendation metrics and compares baseline, Separate Method, and Combination Method configurations on MovieLens and Insurance. Training uses specified optimization and regularization settings across the models.
- Optimization: The baseline models use BPR recommendation loss with learning rate 0.001, ℓ2 regularization coefficient 0.0001, dropout rate 0.2, and early stopping.
- Optimization: Training uses ReLU and LeakyReLU activations, with batch normalization applied for classifier training.
- Compared Methods: Tables 3 and 4 compare baseline, Separate Method, and Combination Method performance on MovieLens and Insurance, respectively.Orig. denotes the baseline model, and the fair methods are evaluated across sensitive-feature combinations.
5.5 Main Results
The main results show that adversarially trained methods substantially reduce sensitive-feature predictability while retaining recommendation quality close to the original models. Separate Method generally outperforms Combination Method on recommendation metrics, especially for single sensitive features.
- Fairness Improvement: Baseline attackers achieve AUC scores significantly above 0.5, whereas both Separate Method and Combination Method produce AUC scores around 0.5.This indicates that adversarial training makes sensitive features difficult to distinguish from the filtered user embeddings.
- Fairness Improvement: Separate Method usually achieves lower attacker AUC than Combination Method, indicating greater removal of sensitive information in most cases.
- Recommendation Performance: Both fair methods retain high recommendation quality, with performance remaining very close to the original models despite a small sacrifice.The authors attribute the trade-off to filtering sensitive-feature information from user embeddings.
- Method Comparison: Separate Method performs better than Combination Method on most metrics across the two datasets, especially for single-feature cases.
(f) AUC-O
The ablation examines how the adversarial coefficient λ changes recommendation performance and attacker AUC. Increasing λ lowers recommendation performance while reducing AUC, reflecting the reported fairness–performance trade-off.
- Ablation Setup: Figure 6 plots Hit@5 recommendation performance and attacker AUC as λ varies for Biased-MF on MovieLens.Subfigures cover gender, age, and occupation as sensitive features.
- Combination Method: Combination Method can unintentionally remove information about sensitive features beyond the feature targeted by an individual filter.The method trains all filter functions together, which can reduce information retained for recommendation.
- Limitations: Separate Method may become infeasible with many sensitive-feature combinations and usually requires more training epochs to converge.
- Method Selection: Separate Method is recommended when the number of sensitive features is small, while Combination Method handles settings with too many feature combinations.
- Coefficient Interpretation: λ controls the trade-off between recommendation quality and fairness by increasing the discriminator loss contribution.As λ grows, the fairness requirement becomes stricter and may require sacrificing more recommendation performance.
- Observed Trade-off: Increasing λ causes recommendation performance to decline while attacker AUC decreases, indicating increasingly fairer representations.The same trend is reported across other models, datasets, metrics, and sensitive-feature combinations.
6 CONCLUSION
The paper addresses user-side fairness in recommender systems by combining causal, counterfactual, and personalized fairness through feature-independent user embeddings. Experiments show desirable recommendation performance, while the work remains an early step with substantial room for improvement.
- The paper studies personalized counterfactual fairness for users in recommender systems using causal-based fairness notions.It proposes individual-level fairness rather than relying only on traditional association-based fairness notions.
- Its adversary-learning method generates feature-independent user embeddings to satisfy counterfactual fairness requirements.The method is designed for recommendations and targets independence from sensitive features.
- Experiments on two real-world datasets with shallow and deep recommendation algorithms produce counterfactually fair recommendations with desirable recommendation performance.
- The work is an initial step toward personalized fairness under counterfactual notions, with substantial room for future improvement.