Source-linked AI summary
SwapRec: Warming Up Cold Items Through Training-Time Swaps
Marta Moscati, Jan Malte Lichtenberg, Davide Abbattista, Antonio De Candia, Laura Boggia, Matteo Ruffini
TL;DR
Cold-item interactions can degrade real-time personalization in ID-based sequential recommenders, while ignoring them prevents updates. SwapRec applies similarity-based item swaps during training, and experiments across three domains show more accurate cold-item recommendations and greater cold-item exposure. The authors limit evaluation to last-item swaps and a single most-similar warm neighbor.
Problem
Cold-item interactions can degrade user preference estimates, whereas excluding them from profiles prevents real-time recommendation updates in ID-based systems.
Method
SwapRec trains ID-based sequential recommenders with the same side-information-based similarity swaps used to replace cold items at inference.
Results
Across online shopping, movie, and music domains, SwapRec substantially improves recommendations with cold-item interactions and increases the percentage of cold items recommended.
Takeaways & Limitations
SwapRec offers a simple, architecture-preserving way to improve cold-item robustness and surface more cold items in ID-based sequential recommendations.
Takeaways & Limitations
Evaluation limits swaps to the last sequence item and the most similar warm item, leaving multiple swaps, positions, and maps unstudied.
Abstract
from arXiv · showhide
Interactions with cold items negatively impact real-time personalization of ID-based recommender systems. This is because the use of such interactions degrades user preference estimates, whereas excluding cold items from the user profile prevents real-time recommendation updates. In industrial scenarios, one heuristic often applied to address this shortcoming at inference time is to replace, i.e., "swap", cold-start items by their most similar "warm" neighbor, where similarity is inferred from the items' side information. In this paper, we demonstrate that sequential models, most often used for real-time personalization, are not robust to such swaps, and propose SwapRec, an approach to address this issue. SwapRec relies on using the same swap heuristics already at training time. We apply SwapRec to state-of-the-art models for sequential recommendation and analyze its impact by means of quantitative experiments in three recommendation domains (online shopping, movie, music). The experimental results show that, irrespective of the underlying sequential architecture, our easy-to-implement SwapRec approach allows for substantially more accurate recommendations when in presence of interactions with cold items, simultaneously leading to a larger percentage of cold items in the recommendation lists.
1. Introduction
Item cold-start harms real-time personalization because cold-item clicks can degrade preference estimates, while ignoring them prevents recommendation updates. SwapRec trains ID-based sequential recommenders with the same similarity swaps used at inference, improving cold-item robustness and exposure without architectural changes.
- Cold-item clicks can produce inaccurate recommendations, whereas discarding those clicks prevents real-time recommendation updates.
- Industrial systems often swap a cold item with its most similar warm item using side-information similarity, preserving compatibility with deployed ID-based recommenders.The example uses audio similarity to replace “Good Day” with “Dark Heart News.”
- Inference-time swaps can reduce accuracy because semantically similar items may occur in different users’ sequences and are not treated similarly by behavioral ID models.
- SwapRec introduces inference-style swaps during training to make ID-based recommenders robust to interactions with cold items.In the motivating example, SwapRec includes the next interacted item in its top-5 list, unlike SASRec.
- SwapRec integrates into ID-based recommenders without changing the architecture or training additional external components.
- Across movie, music, and online-shopping domains, SwapRec substantially improves recommendations with cold-item interactions and surfaces more cold items, regardless of sequential architecture.
2. Related Work
Prior cold-start solutions use multimodal or semantic item representations, but can add complexity or resources. SwapRec instead preserves ID-based recommenders and applies content-based similarity swaps during training to improve robustness and catalog exposure.
- Item cold-start concerns accurate recommendations for items with no prior user interactions and is identified as a major recommender-system challenge.
- Multimodal recommenders supplement behavioral data with multiple item representations to address item cold-start.
- Other methods infer behavioral IDs from side information or replace them with semantic IDs representing item content.
- Industrial systems commonly use simple cold-item proxies because ID-based sequential recommenders support real-time personalization.
- Behavior-only models struggle to treat semantically similar items similarly, causing performance deterioration when swaps are applied at inference.
- SwapRec retains the ID-based architecture and uses content-similarity swaps during training, improving swap robustness while surfacing a larger portion of the catalog.
3. Methodology
SwapRec constructs nearest-neighbor item swaps from side information and applies them stochastically to sequential training data. The swapped sequences train the backbone so embeddings combine behavioral patterns with item semantics and remain robust at inference.
- 3.1. Preliminaries: Sequential recommenders represent each user’s interaction history as a chronologically ordered sequence of catalog items.
- 3.1. Preliminaries: Items are classified as cold or warm according to their number of training interactions, with zero interactions constituting strict cold start.
- 3.1. Preliminaries: Side-information similarity induces a nearest-neighbor map that associates each item with its most similar neighbor.
- 3.2. SwapRec: SwapRec randomly replaces sequence items with their nearest neighbors during training, using a swap probability and a maximum swap count.
- 3.2. SwapRec: Swaps affect both training inputs and targets, and the resulting augmented sequences train the backbone sequential recommender.
- 3.2. SwapRec: Because cold items occur less often unswapped, swapping provides additional embedding updates for cold items.
- 3.2. SwapRec: The swap-count limit controls modification of true behavioral data and concentrates swaps earlier in sequences, further increasing cold-item updates.
- 3.2. SwapRec: At inference, SwapRec swaps cold input items with nearest neighbors while leaving the target item unchanged.
4. Experimental Setup
The experiments evaluate SwapRec across music, shopping, and movie recommendation datasets using temporally split sequential data, nearest-neighbor swaps, and transformer-based backbones alongside collaborative-filtering baselines.
- Datasets: Experiments cover music streaming, online shopping, and movie recommendation domains.Music4All-Onion uses audio representations, Amazon uses product text, and ML-20M uses movie plots as item side information.
- Preprocessing: A global temporal split assigns 20% of interactions to testing, using each sequence’s final item as target and preceding interactions as input.The procedure is designed to prevent future-data leakage and better reflect real-world usage.
- Preprocessing: Sequences are capped at length 40, with longer sequences partitioned into 40-item chunks to retain available training interactions.All chunks before the global split timestamp are used for training.
- Cold-start evaluation: The nearest-neighbor map uses cosine similarity after iterative 2-core filtering of training interactions.The last input item is evaluated unchanged, swapped with its nearest neighbor, randomly initialized, or dropped.
- Evaluation: Cold-item robustness is evaluated by splitting sequences according to the training-interaction count of the last input item, including cases with n_train ≤10.The study measures HR@10 and the empirical CDF of top-k recommendations over n_train; drop comparisons require at least two input items.
- Models and baselines: SwapRec is applied to BERT4Rec and SASRec, while MultVAE, Item-kNN, and ALS provide non-sequential collaborative-filtering baselines.BERT4Rec uses bidirectional self-attention, SASRec uses causal self-attention, and ALS is a matrix-factorization baseline.
5. Results
SwapRec improves sequential recommendation robustness to cold-item interactions while also increasing exposure to less popular and cold items. It can preserve or improve overall accuracy, including in standard warm scenarios.
- SwapRec reaches the best performance on two datasets, improving recommendation accuracy even in a standard warm scenario.
- All models suffer performance deterioration after inference-time swaps, but the deterioration is less pronounced when trained with SwapRec.
- For sequences ending with cold items, performance generally declines as the last item becomes colder, while inference-time swapping often causes additional deterioration.
- SwapRec improves robustness to cold-item interactions irrespective of the swapped item’s training frequency or the strategy used to handle those interactions.
- SwapRec recommends less popular items, increasing both the percentage of cold items and overall catalog coverage compared with SASRec.
- Training-time swaps produce more frequent updates to cold-item ID embeddings, explaining SwapRec’s broader catalog exposure.
6. Conclusions and Future Work
SwapRec applies cold-item swaps during training to reduce item-cold-start deterioration and increase cold-item representation in recommendation lists. The evaluation is limited to specific swap settings, leaving broader swap strategies and embedding-space effects for future work.
- SwapRec applies training-time swapping to transformer-based sequential recommenders.
- SwapRec mitigates performance deterioration in item cold-start and increases the percentage of cold-item occurrences in recommendation lists.
- Evaluation limited item swaps to the last sequence item and mapped each cold item exclusively to its most similar warm item.
- The study did not investigate multiple inference-time swaps, different swap positions, alternative swap maps, or SwapRec’s impact on ID embedding-space structure.
- These extensions are left for future research.
Declaration on Generative AI
The authors used Claude Code during SwapRec development and figure generation, then reviewed and edited the code as needed and accepted responsibility for the produced content.
- Claude Code was used during SwapRec development and to generate the paper’s figures.
- The authors reviewed and edited the code as needed and took full responsibility for the produced content.