PaperlayerView exact revision on arXiv

Source-linked AI summary

Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation

Wenjing Meng, Deqing Yang, Yanghua Xiao

arXiv:2006.06922v1cs.IRcs.LG

TL;DR

Session-based recommenders predict a user's next item from recent session behavior, but item-only models overlook user operations and item knowledge that can inform preferences and sparse transitions. MKM-SR models item-operation micro-behaviors and learns item knowledge as an auxiliary task; the authors report superiority over state-of-the-art models in experiments on two datasets.

  • Problem

    Item-only session modeling overlooks operations that distinguish user intentions and item knowledge that could provide side information for sparse interactions.

    Source: §1 · p. 1

  • Method

    MKM-SR models sessions as item-operation micro-behavior sequences and learns item knowledge embeddings as an auxiliary task alongside next-item prediction.

    Source: §1 · p. 2

  • Results

    Experiments on two realistic datasets justify MKM-SR's reported superiority over state-of-the-art recommendation models.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The evaluations support incorporating user micro-behaviors and item knowledge through multi-task learning into session-based recommendation.

    Source: §1 · p. 2, §5 · p. 10

    The objective specifies a zero-mean Gaussian prior with standard deviation 0.1 for Φ and a Bernoulli-product likelihood for Y.

    Source: §3.5 · p. 6

Abstract

from arXiv · show

Session-based recommendation (SR) has become an important and popular component of various e-commerce platforms, which aims to predict the next interacted item based on a given session. Most of existing SR models only focus on exploiting the consecutive items in a session interacted by a certain user, to capture the transition pattern among the items. Although some of them have been proven effective, the following two insights are often neglected. First, a user's micro-behaviors, such as the manner in which the user locates an item, the activities that the user commits on an item (e.g., reading comments, adding to cart), offer fine-grained and deep understanding of the user's preference. Second, the item attributes, also known as item knowledge, provide side information to model the transition pattern among interacted items and alleviate the data sparsity problem. These insights motivate us to propose a novel SR model MKM-SR in this paper, which incorporates user Micro-behaviors and item Knowledge into Multi-task learning for Session-based Recommendation. Specifically, a given session is modeled on micro-behavior level in MKM-SR, i.e., with a sequence of item-operation pairs rather than a sequence of items, to capture the transition pattern in the session sufficiently. Furthermore, we propose a multi-task learning paradigm to involve learning knowledge embeddings which plays a role as an auxiliary task to promote the major task of SR. It enables our model to obtain better session representations, resulting in more precise SR recommendation results. The extensive evaluations on two benchmark datasets demonstrate MKM-SR's superiority over the state-of-the-art SR models, justifying the strategy of incorporating knowledge learning.

1 INTRODUCTION

Session-based recommendation predicts a user's next item from recent session behavior, but item-only modeling omits operations that distinguish user intentions and item knowledge that can inform sparse transitions. MKM-SR models item-operation micro-behaviors and learns item knowledge as an auxiliary task, with evaluations reported on two datasets.

1 INTRODUCTION

  • Session-based recommendation predicts the next item a user will interact with from recent behavior in a session.The introduction frames sessions as reflecting a user's current preference and intention.Source: §1 · p. 1
  • Item-only session models can treat sessions with the same items as identical even when users' operations reveal different intentions and preferences.The music example contrasts two sessions with the same first two songs but different operations and subsequent items.Source: §1 · pp. 1–2
  • Item attributes, called item knowledge, can provide side information to alleviate data sparsity and indicate semantic relationships between session items.The paper illustrates this with shared song-artist attributes connecting successive songs.Source: §1 · p. 2
  • MKM-SR combines item-operation micro-behaviors with item knowledge learned as an auxiliary task for next-item recommendation.It uses different mechanisms to learn item and operation embeddings before forming session representations.Source: §1 · p. 2
  • The authors report evaluations on KKBOX and JDATA to justify MKM-SR's superiority over state-of-the-art recommendation models.They also investigate modeling algorithms for micro-behaviors and training strategies for multi-task learning.Source: §1 · p. 2

2 RELATED WORK

Related work reviews sequential recommendation and knowledge-based recommendation, identifying gaps in how prior approaches model micro-behaviors and incorporate item knowledge. MKM-SR is positioned as using distinct mechanisms for item and operation sequences and a multi-task learning approach for knowledge.

2 RELATED WORK

  • The related-work section situates the paper among session-based, sequential, and knowledge-based recommendation approaches.It begins with an overview of the research related to the work.Source: §2 · p. 2, §2.1 · p. 2, §2.2 · p. 3

2.1 Session-based and Sequential Recommendation

  • Prior sequential models use Markov chains, RNNs, or GGNNs, while the cited micro-behavior approaches either restrict behavior chains or do not distinguish item and operation transitions.MKM-SR adopts RNN and GNN mechanisms together to account for those differences and preserve operation order.Source: §2.1 · p. 3

2.2 Knowledge-based Recommendation

  • Knowledge-based recommenders use item attributes and knowledge-graph embeddings, but cited sequential approaches model behavior at the item level rather than the micro-behavior level.The paper distinguishes MKM-SR's auxiliary-task multi-task learning from approaches that use knowledge embeddings as pre-trained item embeddings.Source: §2.2 · p. 3

3 METHODOLOGY

The methodology section introduces MKM-SR by formalizing its target problem and then outlining the model pipeline and component descriptions. It also establishes notation conventions for vectors and sets, matrices, or tensors.

3 METHODOLOGY

  • The methodology first formalizes the problem, then summarizes MKM-SR's pipeline before describing its components.The section uses bold lowercase for vectors and bold uppercase for sets, matrices, or tensors.Source: §3 · p. 3
  • The section's notation distinguishes vectors from sets, matrices, and tensors using bold lowercase and uppercase letters, respectively.Source: §3 · p. 3
  • The model description includes the algorithms involved in MKM-SR and proceeds from formal problem statement to component details.Source: §3 · p. 3

3.1 Problem Definition

MKM-SR represents a session as a sequence of item-operation micro-behaviors and incorporates item knowledge as triplets. Given a session and candidate item, it produces a matching score used to rank next-item candidates.

3.1 Problem Definition

  • A micro-behavior is a session event combining an item with its corresponding operation, and the session is represented by a sequence of these events.The sequence is denoted {m1, m2, ..., mL}, where L is its length.Source: §3.1 · p. 3
  • Item knowledge is represented as triplets <i, r, a>, where a is the value of item i's attribute r.The example <Sugar, song-artist, Maroon 5> encodes Maroon 5 as the singer of Sugar.Source: §3.1 · p. 3
  • For a session and candidate item, the model produces a matching score that supports a top-k ranking of next-item candidates.The highest-scoring item is predicted as the next interaction.Source: §3.1 · p. 3
  • The framework extracts item and operation sequences, learns their embeddings, and combines them into micro-behavior embeddings for a session representation.The final score is computed by an MLP followed by softmax, with TransH knowledge embeddings included in the multi-task loss.Source: Figure 2 · p. 4

3.2 Model Overview

MKM-SR combines micro-behavior-based session modeling with item-knowledge learning as an auxiliary task. Item and operation embeddings form micro-behavior embeddings, which are aggregated into session representations for candidate scoring.

3.2 Model Overview

  • MKM-SR concatenates item and operation embeddings into micro-behavior embeddings, then aggregates them into session representations used to score candidate items.The score is computed by an MLP over the session representation and candidate-item embedding, followed by Softmax.Source: §3.2 · p. 4
  • Item knowledge is learned as an auxiliary task in a multi-task learning paradigm, using TransH loss and alternating training.The knowledge task is intended to discover semantic correlations among session items and assist session recommendation.Source: §3.2 · p. 4

3.3 Encoding Session Information

MKM-SR represents each session as item-operation micro-behaviors, learning item and operation embeddings separately before concatenating them. It uses graph modeling for item transitions, sequential modeling for operations, and aggregates micro-behaviors to represent the session.

3.3 Encoding Session Information

  • A micro-behavior combines an item with the operation performed on it; MKM-SR learns their embeddings separately and concatenates them.The paper reports that this is better than directly learning a micro-behavior embedding as a whole.Source: §3.3 · p. 4

3.3.1 Intuitions of Embedding Learning.

  • GGNN models item sequences because their transitions can depend on both preceding and subsequent items, while GRU models sequential operation patterns.The authors note that repeated operation types can yield similar graph neighborhoods, whereas successive operations can exhibit patterns such as adding a product to cart after reading comments.Source: §3.3.1 · p. 5

3.3.2 Learning Item Embeddings.

  • The item sequence becomes a weighted directed graph whose nodes are unique items and whose edge weights normalize successive-item transition frequencies.An item that recurs in a session is represented by one graph node; repeated items can produce self-loops.Source: §3.3.2 · p. 5

3.3.3 Learning Operation Embeddings.

  • GRU produces operation embeddings from the operation sequence, so the same operation can have different embeddings at different positions.The paper illustrates this with operation o3, whose third- and fifth-position embeddings differ.Source: §3.3.3 · p. 5

3.3.4 Generating Session Representations.

  • The model concatenates each item embedding with its operation embedding, allowing sessions with the same items but different operations to have different representations.It represents a session using its most recent micro-behavior for local preference and soft attention over all micro-behaviors for global preference.Source: §3.3.3 · pp. 5–6, §3.3.4 · p. 6
  • An MLP scores each candidate item from its embedding and the session representation, and binary cross-entropy trains the next-item prediction task.Training examples label the next interacted item as positive and other candidate items as negative.Source: §3.3.4 · p. 6

3.4 Learning Knowledge Embeddings

MKM-SR learns knowledge embeddings from item-attribute triplets using a TransH-based loss. The model uses these embeddings to represent semantic relations among items.

3.4 Learning Knowledge Embeddings

  • Knowledge triplets capture semantic correlations among session items, which motivate using knowledge embeddings to enhance session recommendation.The paper gives shared singer or genre attributes as examples of relations between items.Source: §3.4 · p. 6
  • The model adopts TransH because its hyperplane-based embedding approach handles many-to-many and many-to-one relations.For a triplet, it projects the item and attribute embeddings onto a relation-specific hyperplane and evaluates their translated distance.Source: §3.4 · p. 6
  • A knowledge-learning loss is defined over the set of observed item-knowledge triplets.The passage identifies K as the set of all knowledge triplets.Source: §3.4 · p. 6

3.5 The Objective of Multi-task Learning

MKM-SR treats knowledge embedding learning as an auxiliary task for session recommendation rather than as separate pretraining. Its empirical comparisons favor alternating training over joint training.

3.5 The Objective of Multi-task Learning

  • Unlike approaches that pretrain knowledge embeddings before recommendation, MKM-SR learns knowledge embeddings as an auxiliary task within multi-task learning.The knowledge task supplies item-embedding hints to the session-recommendation task.Source: §3.5 · p. 6
  • The multi-task objective combines likelihoods for knowledge triplets and session-recommendation training data, with a Gaussian prior on model parameters.The prior has zero mean and standard deviation 0.1; the loss also includes a regularization term and tuned control parameters.Source: §3.5 · pp. 6–7
  • The objective is optimized using either alternating training or joint learning, and the reported comparison favors alternating training.The paper presents both strategies before reporting the empirical preference for alternating training.Source: §3.5 · p. 7
  • Alternating training is empirically a better strategy for MKM-SR than joint training.The paper reports this conclusion from empirical comparisons.Source: §3.5 · p. 7

4 EXPERIMENTS

The experiments evaluate whether MKM-SR improves session-based recommendation by incorporating micro-behaviors and item knowledge, and compare modeling and training choices.

4.1.1 Datasets.

  • The study evaluates on KKBOX music logs and JDATA e-commerce actions, using four item attributes as knowledge and a later 10% of user behaviors for testing.It also retains test-only cold-start items and creates Demo from the early 1% of JDATA behaviors to examine sparse interactions.Source: §4.1.1 · p. 7

4.1.2 Compared Models.

  • Comparisons include established SR baselines and variants that remove knowledge or operations, directly learn micro-behaviors, or change how item and operation sequences are modeled.The baselines include Markov- and neural-network-based models; the variants test the contributions of knowledge and micro-behaviors.Source: §4.1.2 · pp. 7–8

4.1.3 Evaluation Metrics.

  • Performance is measured with Hit@k, the share of samples whose next item ranks in the top k, and MRR@k, the mean reciprocal rank within the top k.MRR@k is zero when the correct item ranks below k.Source: §4.1.3 · p. 8

4.1.4 Hyper-parameter Setup.

  • For fair comparisons, all models use the same item and operation embedding dimensions, and MKM-SR uses alternating training with Adam at learning rate 0.001 and batch size 128.The reported results use 100-dimensional embeddings; λ1 is 0.0001 and λ2 is 10^-5.Source: §4.1.4 · p. 8

4.2 Global Performance Comparisons

MKM-SR outperforms the baselines and variants across datasets, with stronger advantages for knowledge-aware models in cold-start settings and for operation-aware models on KKBOX.

4.2 Global Performance Comparisons

  • MKM-SR outperforms all baselines and variants across the evaluated datasets, with particularly marked advantages for MKM-SR and KM-SR on cold-start and Demo data.The authors interpret these results as support for incorporating item knowledge to address cold-start sparsity.Source: §4.2 · p. 8
  • Operation-aware RIB has a larger advantage over GRU-based baselines on KKBOX than on JDATA, where KKBOX has more unique operations.The authors interpret this pattern as support for incorporating micro-behaviors.Source: §4.2 · p. 8
  • SR-GNN has greater advantage over GRU-based models on JDATA, where users are more likely to interact with the same item within a session.The authors attribute this pattern to GGNN capturing transitions between successive items in this setting better than GRU.Source: §4.2 · pp. 8–9

4.3 Ablation Study

The ablations support including both micro-behaviors and item knowledge, while experiments favor alternating training and show that the auxiliary-task weight should remain limited.

4.3 Ablation Study

  • MKM-SR outperforms variants without micro-behaviors or knowledge, and separately learning item embeddings with GGNN and operation embeddings with GRU outperforms direct micro-behavior sequence learning.The comparison also favors learning operation sequences with GRU rather than GGNN.Source: §4.3 · p. 9

4.4 Strategies of Incorporating Knowledge Learning

  • Alternating training outperforms joint training and knowledge-based pre-training in the session-recommendation setting.Pre-training starts with better MRR@20 learning curves but is overtaken at convergence; joint training can bias frequent-item embeddings toward the auxiliary knowledge task and shrink learning on the main SR task.Source: §4.4 · p. 9, Figure 3 · p. 9
  • MTL-learned song embeddings distinguish most genres while keeping different genres close, reflecting both semantic and cross-session co-occurrence correlations.The authors state that capturing both correlations simultaneously is useful for improving SR.Source: §4.4 · p. 9, Figure 4 · p. 9

4.5 MTL’s Control Parameter Tuning

  • On KKBOX(N), MKM-SR’s MRR@20 varies by about 1% as λ1 ranges from 0.00001 to 0.1, with the best score at λ1 = 0.0001.The authors suggest that assigning more weight to auxiliary knowledge embedding learning can disturb the main SR task.Source: §4.5 · pp. 9–10, Figure 5 · p. 10

5 CONCLUSION

MKM-SR combines user micro-behaviors and item knowledge for session-based recommendation, using distinct mechanisms to learn item and operation embeddings for fine-grained session representations.

5 CONCLUSION

  • MKM-SR incorporates user micro-behaviors and item knowledge simultaneously into session-based recommendation.The model uses distinct mechanisms for item and operation embeddings to generate fine-grained session representations.Source: §5 · p. 10
  • The paper compares the significance of knowledge embedding learning and alternative training strategies through experiments.Source: §5 · p. 10
  • Experiments justify MKM-SR’s superiority over state-of-the-art SR models, and the authors identify this as a promising direction for improving SR.Source: §5 · p. 10

Related papers

Loading 2006.06922v1…