Source-linked AI summary
Deep Coevolutionary Network: Embedding User and Item Features for Recommendation
Hanjun Dai, Yichen Wang, Rakshit Trivedi, Le Song
TL;DR
Existing point-process approaches make strong parametric assumptions that may not capture nonlinear co-evolution of user and item features. DeepCoevolve uses RNNs over evolving interaction networks to model those features and their interaction intensities. The authors report superior performance on time and item prediction.
Problem
Prior point-process methods use strong parametric assumptions that may not capture the complex nonlinear co-evolution of user and item features.
Source: §1 · p. 1Method
DeepCoevolve uses RNNs over evolving interaction networks to model co-evolving user and item embeddings and parameterize interaction intensities.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The model provides a framework for nonlinear co-evolution modeling and stochastic training on co-evolving user–item networks.
Source: §9 · p. 9The embeddings are event-driven and piecewise constant, changing only when an interaction occurs.
Source: §3.2 · p. 3
Abstract
from arXiv · showhide
Recommender systems often use latent features to explain the behaviors of users and capture the properties of items. As users interact with different items over time, user and item features can influence each other, evolve and co-evolve over time. The compatibility of user and item's feature further influence the future interaction between users and items. Recently, point process based models have been proposed in the literature aiming to capture the temporally evolving nature of these latent features. However, these models often make strong parametric assumptions about the evolution process of the user and item latent features, which may not reflect the reality, and has limited power in expressing the complex and nonlinear dynamics underlying these processes. To address these limitations, we propose a novel deep coevolutionary network model (DeepCoevolve), for learning user and item features based on their interaction graph. DeepCoevolve use recurrent neural network (RNN) over evolving networks to define the intensity function in point processes, which allows the model to capture complex mutual influence between users and items, and the feature evolution over time. We also develop an efficient procedure for training the model parameters, and show that the learned models lead to significant improvements in recommendation and activity prediction compared to previous state-of-the-arts parametric models.
1 INTRODUCTION
User and item features co-evolve through interactions, making timely recommendation depend on their compatibility and reciprocal influence. DeepCoevolve uses RNNs over evolving interaction networks to model this nonlinear evolution without fixed parametric forms.
1 INTRODUCTION
- User interests and item features influence one another over time, and their compatibility helps predict future interactions.Interactions can shape item features, while evolving items can also change users’ interests.Source: §1 · p. 1
- Prior point-process methods make strong assumptions about generative forms that may not capture real-world nonlinear co-evolution, while epoch-based factorization misses fine-grained timing.The latter methods also cannot answer queries about interaction time.Source: §1 · p. 1
- DeepCoevolve defines point-process intensities with RNNs over evolving interaction networks, avoiding fixed parametric forms for user and item feature evolution.Its embedding updates include self evolution, co-evolution, interaction context, and self drift.Source: §1 · p. 1, Figure 1 · p. 2
- The authors report significant improvements in user behavior prediction over state-of-the-art methods and identify precise time prediction as a novel capability.They also describe an efficient stochastic training algorithm for the co-evolving network.Source: §1 · p. 2
2 BACKGROUND ON TEMPORAL POINT PROCESSES
Temporal point processes represent events localized in time and characterize event timing through a conditional intensity given the history. Their intensity functions can take different forms, including constant and history-dependent forms.
2 BACKGROUND ON TEMPORAL POINT PROCESSES
- A temporal point process is a random process realized as discrete events localized in time, and can equivalently be represented by a counting process N(t).N(t) records the number of events before time t.Source: §2 · p. 2
- The conditional intensity λ(t) gives the conditional probability rate of an event in a small interval, given the history up to t and no earlier event.It is also expressed as the conditional expectation of dN(t).Source: §2 · p. 2
- The conditional probability of no event during [0,t) is represented by the survival function S(t), and p(t) denotes the conditional density of an event at t.The passages introduce these as complementary ways to characterize event timing.Source: §2 · p. 2
- Intensity forms encode different event patterns: Poisson processes use a constant rate, while Hawkes processes model history-dependent mutual excitation.In the Hawkes example, historical events raise intensity according to an exponential triggering kernel.Source: §2 · p. 2
3 DEEP COEVOLUTIONARY FEATURE EMBEDDING PROCESSES
DeepCoevolve represents interactions as time-ordered user-item events and uses recurrent updates to model user and item embeddings as they influence one another. Those event-driven embeddings determine pairwise interaction intensities in a temporal point process.
3.1 Deep coevolutionary network
- DeepCoevolve models each user-item interaction as an event with a time and context, and uses recurrent updates to evolve low-dimensional user and item embeddings.The context can include text such as reviews or static profile and item-category features.Source: §3 · pp. 2–3, §3.1 · p. 3
- Each embedding update combines temporal drift, self-evolution, user-item coevolution, and interaction features.Drift allows basic features to change smoothly over time; interaction context, such as posts or reviews, can also influence embeddings.Source: §3.1 · p. 3
3.2 Understanding coevolutionary embeddings
- An embedding change can propagate beyond the interacting user-item pair as the modified item or user affects later interactions elsewhere in the network.The update functions are event-driven, so embeddings remain piecewise constant between interaction events.Source: §3.2 · p. 3
3.3 Intensity function as the compatibility between embeddings
- Each user-item pair is a dimension of a temporal point process whose intensity depends on the compatibility of the pair’s instantaneous embeddings.The model uses a Rayleigh process: piecewise-constant embeddings yield piecewise-linear intensity, with time intervals modeled by a Rayleigh distribution.Source: §3.3 · pp. 3–4
- The intensity uses the embeddings at the most recent time either member changed, and its time-lapse term supports modeling event timing without discretizing time into epochs.The passage describes this Rayleigh form as suited to fads, where event likelihood rises to a peak and then drops rapidly.Source: §3.3 · p. 4
4 EFFICIENT LEARNING FOR DEEP COEVOLUTIONARY NETWORK
DeepCoevolve learns parameters by maximizing the likelihood of observed interactions while accounting for survival across user-item pairs. Its training procedure uses globally ordered sliding-window mini-batches and sampling to address dependencies and computation over a sparse interaction graph.
4.1 Objective function
- The negative log-likelihood rewards observed interactions through their intensity terms and penalizes non-occurrence across possible user-item pairs through survival probabilities.The objective therefore accounts for both why an event happens and why another possible interaction does not.Source: §4.1 · p. 4
4.2 Efficient learning algorithm
- Because events are linked through the time-varying user-item graph, the model’s recurrent dependencies make parameter learning difficult to decompose.The authors note that conventional truncated backpropagation through time is difficult to apply directly in this setting.Source: §4.1 · p. 4, §4.2 · p. 4
- Training globally orders events and uses sliding-window mini-batches, truncating the computational graph along the global timeline rather than independent sequences.Each window builds its graph from consecutive events, a strategy contrasted with segmenting independent sequences.Source: §4.2 · p. 4
- For each mini-batch, the method updates embeddings and intensities for observed events, then computes survival contributions over intervals where embeddings remain constant.The survival integral is split at timestamps associated with events involving either member of a user-item pair.Source: §4.2 · p. 5
- To reduce survival-term computation, the method randomly samples user-item dimensions without events in the current mini-batch.This sampling addresses the cost of monitoring every pair in a sparse interaction graph; the intensity and survival terms resemble softmax classification.Source: §4.2 · p. 5
5 PREDICTION WITH DEEPCOEVOLVE
DeepCoevolve supports next-item and event-time prediction from user-item interaction intensities. The two tasks respectively rank items at a given time and estimate when a user will next interact with a given item.
5 PREDICTION WITH DEEPCOEVOLVE
- For a given user and time, next-item prediction ranks items by their conditional density, enabling time-sensitive recommendations.The best prediction is the item with the largest conditional density.Source: §5 · p. 5
- For a given user-item pair, event-time prediction estimates the expected time of their next interaction, computed in closed form.The expected next event time is computed under the pair’s conditional density.Source: §5 · pp. 5–6
6 COMPLEXITY ANALYSIS
The analysis describes model size, training cost, and prediction complexity in terms of user, item, event, and mini-batch counts, treating latent dimensionality as constant.
Model Size
- With one-hot user and item profiles, basic feature embeddings use O(#user + #item) parameters; interaction-feature and RNN parameters are dataset-size independent.The authors characterize the resulting model as as scalable as traditional matrix factorization methods.Source: §6 · p. 6
Training Complexity
- With mini-batch budget M, embedding updates cost O(M); NCE reduces intensity and survival-probability computation to C × M, and each stochastic training step is summarized as O(M).Without NCE sampling, the ideal cost for these computations is O((n + m) × M).Source: §6 · p. 6
Test/Prediction Complexity
- Item prediction costs O(N) for N items and may use fast inner-product search; closed-form event-time prediction costs O(1).The O(N) item comparison uses the current user embedding.Source: §6 · p. 6
7 EXPERIMENTS
DeepCoevolve was evaluated on three real-world datasets for item and time prediction, with performance compared against epoch-based and point-process baselines. It consistently outperformed competitors overall, and point-process methods benefited more on recurring events.
7 EXPERIMENTS
- IPTV, Yelp, and Reddit provide three real-world interaction datasets; Yelp was expected to be most challenging because it combines shorter user histories with greater item diversity.The experiments used IPTV watching histories, Yelp reviews, and Reddit discussions.Source: §7 · p. 6
7.1 Competitors
- Competitors included epoch-based factorization and rating methods, point-process models, and STIC, which is designed only for time prediction.The point-process baselines differ in their assumptions: LowRankHawkes models interactions independently, while Coevolving uses linear coevolutionary embeddings.Source: §7.1 · p. 6
7.2 Overall Performance Comparison
- DeepCoevolve significantly outperformed epoch-based and point-process baselines on item and time prediction across five dataset splits.The reported comparisons cover three real-world datasets; Figure 5 presents the results.Source: Figure 5 · p. 7, §7.2 · p. 8
7.3 Refined Performance Comparison
- DeepCoevolve performed consistently better across ten consecutive test-time windows, while all point-process methods showed stable performance with small variance.For recurring events, all algorithms performed better than on the overall test set, and point-process models benefited more; the authors relate this to improved intensity estimates with more observations for a user-item pair.Source: Figure 6 · p. 7, §7.3 · p. 8, Figure 7 · p. 8
8 RELATED WORK
Related work includes static and epoch-based latent-feature models, point-process approaches, and neural recommendation methods. DeepCoevolve differs by using RNNs to model nonlinear coevolution in a multivariate point-process recommendation setting.
8 RELATED WORK
- Epoch-based latent-feature methods require choosing an epoch length and typically cannot answer time-sensitive questions at finer resolution.Low-rank point processes address these issues but do not capture heterogeneous coevolutionary user-item properties.Source: §8 · p. 8
- Compared with Wang et al., DeepCoevolve uses RNNs rather than linear user and item embeddings and has model size linear in the number of users and items.The authors describe the RNN approach as nonparametric and more expressive for complex dynamics.Source: §8 · p. 8
- Unlike RMTPP’s one-dimensional point-process setting, DeepCoevolve applies RNNs and multivariate point processes to coevolving user-item features over a temporal network.Other reviewed neural methods include session-based item-to-item recommendation, user-history-dependent factors, and models with separate temporal granularities.Source: §8 · pp. 8–9
9 CONCLUSIONS
The conclusion presents DeepCoevolve as an efficient framework for nonlinear coevolution of user and item embeddings, modeled by RNNs and trained with a stochastic algorithm. The authors report superior time- and item-prediction performance and identify group dynamics in message services as a future extension.
9 CONCLUSIONS
- DeepCoevolve models nonlinear coevolution of user and item embeddings with RNNs and uses an efficient stochastic training algorithm.The authors report superior performance on both time and item prediction, tasks that most prior work could not address together.Source: §9 · p. 9
- The authors identify extending the framework to social applications, including group dynamics in message services, as future work.Source: §9 · p. 9