Source-linked AI summary
Deep Session Interest Network for Click-Through Rate Prediction
Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, Keping Yang
TL;DR
CTR prediction needs to capture users’ dynamic, evolving interests, but existing sequence models overlook the session structure of behavior sequences. DSIN extracts session interests with biased self-attention, models their evolution with Bi-LSTM, and adaptively aggregates them for the target item; it outperforms state-of-the-art models on advertising and production recommender datasets.
Problem
Existing CTR models use sequential behaviors but overlook that behavior sequences consist of sessions with homogeneous within-session and heterogeneous cross-session interests.
Method
DSIN divides behavior sequences into sessions, extracts each session’s interest with self-attention and bias encoding, models session-interest evolution with Bi-LSTM, and aggregates interests relative to the target item.
Results
DSIN outperforms other state-of-the-art CTR models on both advertising and production recommender datasets.
Takeaways & Limitations
Modeling multiple historical sessions provides an effective approach for representing users’ dynamic and evolving interests in CTR prediction.
Abstract
from arXiv · showhide
Click-Through Rate (CTR) prediction plays an important role in many industrial applications, such as online advertising and recommender systems. How to capture users' dynamic and evolving interests from their behavior sequences remains a continuous research topic in the CTR prediction. However, most existing studies overlook the intrinsic structure of the sequences: the sequences are composed of sessions, where sessions are user behaviors separated by their occurring time. We observe that user behaviors are highly homogeneous in each session, and heterogeneous cross sessions. Based on this observation, we propose a novel CTR model named Deep Session Interest Network (DSIN) that leverages users' multiple historical sessions in their behavior sequences. We first use self-attention mechanism with bias encoding to extract users' interests in each session. Then we apply Bi-LSTM to model how users' interests evolve and interact among sessions. Finally, we employ the local activation unit to adaptively learn the influences of various session interests on the target item. Experiments are conducted on both advertising and production recommender datasets and DSIN outperforms other state-of-the-art models on both datasets.
1 Introduction
DSIN treats user behavior sequences as collections of time-separated sessions, reflecting homogeneous within-session behavior and sharply changing interests across sessions. It models session interests and their evolution, then adapts their influence to the target item; experiments report superiority on advertising and production recommender datasets.
- Motivation: Existing CTR models use sequential behaviors to represent users’ dynamic interests but overlook that behavior sequences are composed of sessions.A session is a list of interactions occurring within a given time frame.
- Motivation: User behaviors are highly homogeneous within sessions but heterogeneous across sessions, with interests potentially changing sharply when a new session begins.The introduction illustrates separate browsing interests across three sessions and divides sessions at time gaps exceeding 30 minutes.
- DSIN: DSIN uses self-attention with bias encoding to extract each session’s interest, Bi-LSTM to model interactions and evolution across sessions, and a local activation unit to aggregate interests for the target item.The local activation unit accounts for different influences of session interests on the target item.
- Contributions: The paper highlights session-aware modeling as DSIN’s contribution to effectively modeling users’ multiple sessions for CTR prediction.Its stated contributions emphasize both the within-session and cross-session structure of user behavior.
- Experiments: DSIN outperforms other state-of-the-art CTR models in comparative experiments on advertising and production recommender datasets.The paper reports two groups of comparative experiments across both dataset types.
2 Related Work
Related CTR research models feature interactions and sequential user behaviors, while session-based recommendation research models within-session dynamics. The paper positions sessions as comparatively uncommon in CTR prediction.
- CTR prediction: Recent CTR models improve feature representations by learning linear, higher-order, residual, or attention-weighted feature interactions.Examples include Wide&Deep, DeepFM, DCN, and AFM.
- CTR prediction: Sequential-behavior CTR models represent users’ dynamic interests using pooling, target-item attention, or attention-based modeling of heterogeneous behaviors.The cited examples include YoutubeNet, DIN, and ATRANK.
- Session-based recommendation: Session is common in sequential recommendation but rare in CTR prediction, where session-based methods model the evolution of users’ interests within sessions.Existing session-based approaches include pooling-based and RNN-based methods.
3 Deep Session Interest Network
DSIN models user behavior as multiple sessions, extracting session interests, modeling their sequential interactions, and activating them relative to the target item before final prediction.
- BaseModel: DSIN uses User Profile, Item Profile, and User Behavior as three groups of sparse features for CTR prediction.Item side information can be concatenated to represent the item.
- Session Division Layer: The behavior sequence is divided into sessions, with a new session beginning when adjacent behaviors are more than 30 minutes apart.Each retained session contains a fixed number of user behaviors represented as embeddings.
- Session Interest Extractor Layer: Within each session, multi-head self-attention with bias encoding captures behavior relationships and extracts session-interest representations.Bias encoding represents session, within-session position, and embedding-unit biases before self-attention.
- Session Interest Interacting Layer: Bi-LSTM models sequential relations and contextual interactions among the extracted session interests.Its hidden states combine forward and backward recurrent representations.
- Session Interest Activating Layer: A local activation unit reallocates the influence of session interests and contextual hidden states with respect to the target item.The activated representations are concatenated with user and item profile embeddings before entering the MLP for final prediction.
4 Experiments
Experiments evaluate DSIN against established CTR models on advertising and recommender datasets, using AUC and additional analyses of its technical designs. DSIN achieves the best results on both datasets, while session partitioning, session interaction, and bias encoding each improve performance.
- Datasets and competitors: Experiments compare DSIN with YoutubeNet, Wide&Deep, DIN, DIN-RNN, and DIEN on advertising and recommender datasets.The evaluation uses AUC as the ranking metric.
- Datasets and competitors: AUC measures the model’s ranking ability using positive and negative examples and their predicted scores.The metric averages whether each positive example receives a higher score than each negative example.
- Overall results: DSIN gets the best results on both the advertising and recommender datasets.The comparison attributes this to extracting session interests, modeling their evolution, and adaptively weighting them relative to the target item.
- Effect of Multiple Sessions: Partitioning behavior sequences into multiple sessions improves DSIN performance because within-session behaviors are homogeneous and session interests follow a sequential pattern.The analysis contrasts DSIN-BE with DSIN-BE-NO-SIIL and DIN-RNN with DIN, finding the session-aware design beneficial in the reported comparisons.
- Effect of Session Interest Interacting Layer: The session interest interacting layer improves DSIN by mixing session interests with contextual information and making them more expressive.DSIN-BE performs better than DSIN-BE-NO-SIIL in the comparative experiment.
- Effect of Bias Encoding: Bias encoding improves DSIN over positional encoding by capturing session order information.The reported comparison is between DSIN-BE and DSIN-PE.
- Visualization: Figure 3 shows self-attention emphasizing trouser-related behaviors and the local activation unit emphasizing session interests related to the target item.The visualization uses darker lines for greater weights, with self-attention shown in the lower half and activation-unit behavior in the upper half.
5 Conclusion
DSIN treats user behavior sequences as multiple historical sessions, extracting session interests with self-attention and bias encoding, modeling their relations with Bi-LSTM, and aggregating them for the target item. Experiments demonstrate DSIN’s effectiveness on advertising and recommender datasets.
- DSIN models sequential behaviors as multiple historical sessions rather than one undifferentiated sequence.
- Self-attention with bias encoding extracts each session’s user-interest representation.
- Bi-LSTM captures sequential relations among contextual session interests.
- A local activation unit aggregates session-interest representations with respect to the target item.
- DSIN demonstrates effectiveness on both advertising and recommender datasets.