PaperlayerView exact revision on arXiv

Source-linked AI summary

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

Chongming Gao, Shiqi Wang, Shijun Li, Jiawei Chen, Xiangnan He, Wenqiang Lei, Biao Li, Yuan Zhang, Peng Jiang

arXiv:2204.01266v2cs.IR

TL;DR

Repeated exposure to a user’s preferred items can create filter bubbles, while static approaches do not directly model overexposure. CIRS combines causal inference with offline RL; experiments report that it bursts filter bubbles and increases cumulative satisfaction.

  • Problem

    Overexposure can reduce satisfaction even for items users like, and existing static approaches do not directly model this effect.

    Source: §1 · p. 2

  • Method

    CIRS combines causal inference with offline RL to address filter bubbles in interactive recommendation.

    Source: §1 · p. 3, §6 · p. 22

  • Results

    Experiments show that CIRS bursts filter bubbles and increases users’ cumulative satisfaction.

    Source: §6 · p. 22

  • Takeaways & Limitations

    KuaiEnv supports faithful evaluation because each user’s preference toward all items is known.

    Source: §6 · p. 22

    The authors treat recommender overexploitation as the main cause of filter bubbles and use overexposure as its evaluation proxy.

    Source: §1 · p. 2

Abstract

from arXiv · show

While personalization increases the utility of recommender systems, it also brings the issue of filter bubbles. E.g., if the system keeps exposing and recommending the items that the user is interested in, it may also make the user feel bored and less satisfied. Existing work studies filter bubbles in static recommendation, where the effect of overexposure is hard to capture. In contrast, we believe it is more meaningful to study the issue in interactive recommendation and optimize long-term user satisfaction. Nevertheless, it is unrealistic to train the model online due to the high cost. As such, we have to leverage offline training data and disentangle the causal effect on user satisfaction. To achieve this goal, we propose a counterfactual interactive recommender system (CIRS) that augments offline reinforcement learning (offline RL) with causal inference. The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction. It then uses the learned causal user model to help the planning of the RL policy. To conduct evaluation offline, we innovatively create an authentic RL environment (KuaiEnv) based on a real-world fully observed user rating dataset. The experiments show the effectiveness of CIRS in bursting filter bubbles and achieving long-term success in interactive recommendation. The implementation of CIRS is available via https://github.com/chongminggao/CIRS-codes.

1 INTRODUCTION

Repeatedly recommending a user’s dominant interests can reduce satisfaction and create filter bubbles, motivating interactive recommendation that models both intrinsic interest and overexposure. The authors propose combining causal inference with offline RL to plan for long-term satisfaction and evaluate policies in an interactive environment.

1 INTRODUCTION

  • Prior filter-bubble strategies are mostly heuristic and static, and do not directly model overexposure as a cause of bubbles.Source: §1 · p. 2
  • Because online training is costly and can hurt users’ satisfaction, the authors use offline logs to disentangle interest and overexposure effects.Source: §1 · pp. 2–3
  • CIRS learns a causal user model of interest and overexposure, then uses counterfactual satisfaction to plan an offline RL policy.The framework has three steps: learn the model, use its rewards for policy planning, and evaluate cumulative satisfaction in real environments.Source: §1 · p. 3
  • KuaiEnv supports offline evaluation with a fully filled user-item matrix and a boredom-triggered exit mechanism.The matrix has 99.6% density; the exit mechanism is also added to VirtualTaobao.Source: §1 · p. 3
  • CIRS captures the overexposure effect in offline learning and achieves larger cumulative satisfaction than other baselines.Source: §1 · p. 4
  • The work frames filter bubbles as an interactive-recommendation problem and claims to be the first to combine causal inference with offline RL for interactive recommenders.Source: §1 · p. 4

2 RELATED WORK

The related work reviews filter bubbles in interactive recommendation, causal inference for modeling their effects, and offline RL for optimizing recommendations over multiple rounds. It positions CIRS as combining model-based offline RL with causal inference, distinguishes its goal from prior approaches, and clarifies dimensions for comparing recommender systems.

2.1 Filter Bubbles in Recommendation

  • Filter bubbles can stem from users’ intentions or recommender overexploitation, while repeated familiar items can undermine satisfaction because users value novelty and serendipity.The paper focuses on overexposure as a symptom or proxy of filter bubbles and addresses it in interactive recommendation rather than static recommendation.Source: §2.1 · pp. 4–5
  • Prior bubble-mitigation strategies emphasize diversity, serendipity, fairness, or debunking content [1, 49, 79, 59, 93, 53, 78], but address static recommendations rather than dynamic exposure.CIRS instead captures overexposure in interactive recommendation.Source: §2.1 · p. 5

2.2 Causality-enhanced Recommendation

  • Causal recommendation methods include direct/error-imputation estimators, inverse propensity scoring, and doubly robust estimators, but precise imputations or propensities are difficult and IPS has high variance.The paper applies a causal-inference framework to explicitly model recommender overexposure.Source: §2.2 · pp. 5–6

2.3 Offline RL for Recommendation

  • Whereas single-round methods focus on recommendation performance under an I.I.D. assumption, recommendations may have future rather than instantaneous impacts; online learning of an interactive policy is impractical because it is slow and hurts user experience.Source: §2.3 · p. 6
  • OPE-based offline RL can suffer high variance, whereas model-based methods plan from estimated transitions and rewards but risk bias from model estimation.CIRS specifies the relevant recommender-system bias as whether the model accounts for users feeling bored during interaction.Source: §2.3 · p. 7
  • The comparison in Table 1 classifies six recommender types by user modeling, debiasing, and use of an RL-based policy.Two other works share CIRS’s category, but are not designed to address filter bubbles; their use of “unbiasedness” concerns estimation accuracy rather than addressing a particular recommendation bias.Source: §2.3 · p. 7, Table 1 · p. 7

3 PREREQUISITES

The section formulates interactive recommendation as an RL problem and outlines how CIRS learns a causal user model from historical data to train its policy offline. Kuaishou field-study observations show lower satisfaction with greater exposure to similar items and inform the simulated user behavior.

3.1 Problem Definition

  • Interactive recommendation is framed as RL: the system recommends one item per action, receives user feedback as reward, and updates its state after each interaction.A Transformer-based state tracker models the transition; the policy chooses actions conditioned on the state.Source: §3.1 · p. 8
  • CIRS trains a causal user model on historical interactions to estimate interest and overexposure, then uses its counterfactual satisfaction rewards to train the policy offline.The model shrinks rewards when the policy has made similar recommendations before.Source: §3.1 · p. 8

3.2 Field Study of Overexposure Effect on User Satisfaction

  • The study groups collected views by same-tag videos watched within one hour or time since the last same-tag view, then averages two satisfaction indicators: comments-section duration and video watching ratio.Source: §3.2 · p. 9
  • Satisfaction drops as more similar items appear in recent recommendations and as the interval between similar items increases.The authors report that these observations empirically support the hypothesis that repeated short-term exposure harms satisfaction.Source: §3.2 · pp. 9–10

4 PROPOSED METHODS

CIRS augments offline RL with a causal user model to account for item overexposure, using counterfactual satisfaction to plan recommendations and evaluate a policy aimed at preventing filter bubbles.

4.1 Offline RL-based Framework

  • CIRS uses offline data in three stages: supervised pre-learning of the user model, RL policy planning with counterfactual satisfaction as reward, and policy evaluation in a real or realistic simulated environment.The framework is based on offline RL to train the interactive recommender using historical data.Source: §4.1 · p. 10

4.1.1 Causal User Model.

  • The causal user model learns interest from historical data and provides counterfactual satisfaction, with separate modules for intrinsic-interest estimation and overexposure-aware satisfaction estimation.It is designed to disentangle how item overexposure affects user satisfaction.Source: §4.1 · p. 10, §4.1.1 · p. 10

4.1.2 Transformer-based State Tracker.

  • A Transformer state tracker encodes user features, recommended-item history, and feedback; a gate filters action and feedback inputs, and masking prevents future information leakage.The implementation uses a two-layer encoder.Source: §4.1.2 · p. 10

4.1.3 RL-based Interactive Recommendation Policy.

  • The interactive recommendation policy uses PPO to maximize cumulative long-term satisfaction, with counterfactual satisfaction from the causal model replacing immediate feedback in its reward term.The advantage estimate uses generalized advantage estimation [69].Source: §4.1.3 · pp. 11–12

4.2 Causal Inference-based User Satisfaction Disentanglement

  • CIRS models overexposure as a cause of satisfaction alongside intrinsic interest, unlike traditional recommenders that fit satisfaction from user and item features alone.The causal model separates the path from user and item to intrinsic interest and satisfaction from the path through item overexposure, which negatively affects satisfaction.Source: §4.2 · pp. 12–13
  • The causal user model estimates intrinsic interest from user and item features, then subtracts an overexposure effect that grows for similar items recommended recently.The exposure effect aggregates contributions from previously recommended items and incorporates user sensitivity and item unendurableness.Source: §4.2 · p. 13
  • In RL planning, CIRS intervenes by cutting the overexposure-to-satisfaction path, estimates counterfactual satisfaction on the new trajectory, and uses it as the policy reward.The method retains a model of the correct overexposure effect during intervention rather than simply removing it.Source: §4.2 · pp. 13–14
  • The method aims to guarantee large user satisfaction by preventing filter bubbles, that is, item overexposure.Source: §4.2 · p. 14

5 EXPERIMENTS

The experiments evaluate interactive recommendation and ask how CIRS performs across comparison methods, interaction horizons, environments, and parameter choices.

5 EXPERIMENTS

  • The experiments assess the interactive recommender system and investigate four research questions.They cover comparisons with static and RL-based methods, limited interaction rounds, varying user tolerance of filter bubbles, and key CIRS parameters.Source: §5 · p. 14
  • One question compares CIRS with state-of-the-art static recommendation methods and RL-based interactive methods.Source: §5 · p. 14
  • The remaining questions examine CIRS under limited interaction rounds, differing user tolerance, and changes to key parameters.Source: §5 · p. 14

5.1 Experimental Setup

The experimental setup evaluates recommendation policies interactively in VirtualTaobao and KuaiEnv, using an exit mechanism for repeated recommendations and cumulative satisfaction over interaction trajectories. It also describes the baseline comparison, including CIRS’s DeepFM user model and Transformer-based state tracker.

5.1.1 Evaluation in the Interactive Recommendation Setting.

  • Interactive evaluation accumulates rewards along trajectories rather than comparing top-k outputs with fixed test answers, capturing dynamically changing satisfaction.The paper evaluates in VirtualTaobao and KuaiEnv because offline evaluation of interactive models is difficult.Source: §5.1.1 · pp. 14–15

5.1.2 Recommendation Environments.

  • VirtualTaobao simulates Taobao users with MAIL, whereas KuaiEnv uses KuaiRec’s fully observed historical feedback and video-watching ratio as reward.VirtualTaobao supplies 100,000 logged interactions for offline RL training; KuaiEnv uses the fully observed matrix for policy evaluation and additional sparse interactions to pre-learn the user model.Source: §5.1.2 · p. 15
  • Both environments add a “feel bored then quit” mechanism that ends interaction after sufficiently similar recent recommendations.VirtualTaobao uses a distance threshold against the recent N items; KuaiEnv exits when more than n_Q of the N recent items share an attribute with the current recommendation.Source: §5.1.2 · p. 16

5.1.3 Evaluation Metrics.

  • Evaluation reports average cumulative satisfaction across 100 interaction sequences, accumulating rewards until an exit interrupts the trajectory.The metric reflects a trade-off between higher single-round satisfaction and maintaining a longer interaction sequence.Source: §5.1.3 · p. 16

5.1.4 Baselines.

  • KuaiEnv’s static-model baselines are DeepFM, IPS, PD, and DICE, with IPS, PD, and DICE implemented using DeepFM as their backbone.The deterministic models sample from final logits rather than selecting top-scoring items, which the authors say would cause immediate overexposure in the interactive setting.Source: §5.1.4 · p. 17
  • Other baselines include Random, ε-greedy, and UCB in KuaiEnv, plus sampled-Softmax MLP, ε-greedy, and PPO in VirtualTaobao.PPO is trained in the same offline RL framework without causal inference and is labeled CIRS w/o CI.Source: §5.1.4 · p. 17

5.1.4 Baselines

  • Unlike Huang et al., CIRS uses DeepFM rather than MF; unlike Huang et al., it uses a Transformer-based state tracker rather than many sequential models.Source: §5.1.4 · p. 17

5.2 Overall Performance Comparison

Across VirtualTaobao and KuaiEnv, CIRS reaches the highest cumulative satisfaction after sufficient training, while comparisons with its no-causal-inference variant support the causal module’s effectiveness. Other methods show trade-offs between single-round satisfaction and interaction length, and heuristic methods do not overcome overexposure.

5.2 Overall Performance Comparison

  • CIRS achieves the maximal average cumulative satisfaction in both VirtualTaobao and KuaiEnv after several epochs.It also beats CIRS w/o CI in both environments.Source: Figure 7 · p. 18, §5.2 · pp. 18–19
  • In VirtualTaobao, CIRS balances interaction length and single-round satisfaction, while CIRS w/o CI becomes unstable and its performance degenerates over epochs.The authors attribute the contrast to causal inference capturing overexposure and helping avoid repeated recommendations.Source: §5.2 · pp. 18–19
  • In KuaiEnv, CIRS achieves the largest cumulative satisfaction after enough epochs, while CIRS and CIRS w.o. CI reach maximum interaction length 100 after about 160 epochs; CIRS satisfaction rises further at about 180 epochs.Source: §5.2 · p. 19
  • Heuristic and static methods do not overcome overexposure, while UCB ends with the lowest interaction length but the maximum single-round satisfaction.The authors describe UCB as getting stuck in a filter bubble after exploration; CIRS and CIRS w/o CI are deep RL policy-based methods.Source: §5.2 · p. 19

5.3 Results with Limited Interaction Rounds

With interaction rounds limited, CIRS outperforms its no-causal-inference counterpart in KuaiEnv and performs similarly in VirtualTaobao. The results also highlight improved single-round performance in KuaiEnv and continued gains with more training.

5.3 Results with Limited Interaction Rounds

  • CIRS outperforms CIRS w/o CI in KuaiEnv, while both reach the same level of single-round performance in VirtualTaobao.In VirtualTaobao, both exceed 4.0 after approximately 150 epochs.Source: §5.3 · p. 19
  • CIRS shows a great improvement in single-round performance in KuaiEnv, supporting its suitability for real-world scenarios with limited interaction rounds.The comparison is between the limited-round and former settings.Source: §5.3 · p. 19
  • CIRS’s performance continues to increase at epoch 200 in the longer-round settings, indicating potential given enough training time.The results also attribute gains to correct user preference being distilled from the causal user model to the RL policy.Source: §5.3 · p. 19
  • The results indicate that integrating causal inference lets CIRS outperform its counterpart through the model-based RL framework.The reported mechanism is that correct user preference is distilled from the causal user model to the RL policy.Source: §5.3 · pp. 19–20

5.4 Results with Different User Sensitivity

CIRS performs best when users are less sensitive to overexposure, but its performance declines as sensitivity rises. For extremely picky users, it may match or fall below other baselines.

5.4 Results with Different User Sensitivity

  • CIRS outperforms all baseline methods for less-sensitive users and achieves the best cumulative satisfaction by avoiding repeated recommendations of highly similar items.The less-sensitive conditions are small d_Q in VirtualTaobao and small N in KuaiEnv.Source: §5.4 · p. 20
  • CIRS’s performance decreases as users become more sensitive, though it can still beat CIRS w/o CI; at d_Q ≥ 4 or N ≥ 4, it may match or underperform other baselines.For extremely picky users, causal inference cannot alleviate dissatisfaction caused by overexposing any item.Source: §5.4 · p. 20
  • When users are too picky, RL-based methods cannot beat the DeepFM or MLP teacher models because they lack an opportunity to explore and exploit.Other baselines perform similarly across sensitivity levels, and users may still quit despite greater tolerance to filter bubbles.Source: §5.4 · p. 20

5.5 Effect of Key Parameters

The learned user and item parameters track activity and popularity, respectively. Suitable choices of the two time parameters improve performance over CIRS without causal inference.

5.5 Effect of Key Parameters

  • User sensitivity α_u is proportional to user activity, while item unendurableness β_i is proportional to item popularity.The authors relate active users’ greater susceptibility to boredom and popular videos’ lower endurance under overexposure to these patterns.Source: §5.5 · pp. 20–21, Figure 9 · p. 21
  • Suitable τ-τ* pairs improve performance compared with CIRS w/o CI.The τ=0, τ*=0 setting degenerates to CIRS w/o CI.Source: §5.5 · p. 21, Figure 10 · p. 21
  • In KuaiEnv, the τ and τ* axes use different time units: seconds in the log data and steps in RL planning and evaluation.Source: §5.5 · p. 22

6 CONCLUSION AND DISCUSSION

This work studies filter bubbles in interactive recommendation and proposes CIRS, which uses a causal user model to support offline RL. The paper presents KuaiEnv for evaluation and reports that CIRS achieves optimal cumulative satisfaction by balancing per-round satisfaction with lasting interaction.

6 CONCLUSION AND DISCUSSION

  • The authors frame interactive recommendation as a practical way to track and estimate filter bubbles, and report field studies where satisfaction drops as similar content increases.Source: §6 · p. 22
  • CIRS uses a causal user model to disentangle intrinsic user interest from item overexposure and provide counterfactual rewards for RL policy learning.Source: §6 · p. 22
  • KuaiEnv is an RL evaluation environment based on a real-world, fully observed user-rating dataset, where each user’s preferences toward all items are known.The paper presents cumulative reward as the evaluation criterion for recommendation policies.Source: §6 · p. 22
  • CIRS obtains optimal cumulative satisfaction by balancing high single-round satisfaction with maintaining a long-lasting interaction.Source: §6 · p. 22
  • Interactive evaluation reflects settings without standard answers in users’ minds, where companies care whether recommendations satisfy users over the long term.Source: §6 · p. 22
  • The authors identify exploring other biases in interactive recommendation and combining causal inference with RL as future directions.They suggest causal inference can provide an explicit guide for optimizing models and introduce explainability in RL.Source: §6 · p. 22
Loading 2204.01266v2…