PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-efficient Fine-tuning for LLM-based Recommendation

Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, Tat-Seng Chua

arXiv:2401.17197v2cs.IR

TL;DR

Rapidly expanding recommendation data makes frequent LLM fine-tuning costly, while randomly sampled few-shot data may lack representativeness. The paper formulates data pruning for LLM-based recommendation and proposes DEALRec, which uses influence and effort scores to select samples efficiently; experiments on three real-world datasets demonstrate effectiveness in both efficiency and accuracy.

  • Problem

    Frequent fine-tuning is costly as recommendation data rapidly expands, and randomly sampled few-shot data may not sufficiently represent new items and user behaviors.

  • Method

    DEALRec identifies influential samples using influence and effort scores, estimating sample-removal effects on empirical risk and using a surrogate model for efficient pruning.

  • Results

    Experiments on three real-world datasets demonstrate DEALRec's effectiveness in achieving both high efficiency and accuracy.

  • Takeaways & Limitations

    The proposed data-pruning task opens a research direction for efficient LLM-based recommendation and supports fine-tuning with limited resources.

  • Takeaways & Limitations

    Influence-score computation remains costly because stochastic HVP estimation still requires independent estimations for each sample.

Abstract

from arXiv · show

Leveraging Large Language Models (LLMs) for recommendation has recently garnered considerable attention, where fine-tuning plays a key role in LLMs' adaptation. However, the cost of fine-tuning LLMs on rapidly expanding recommendation data limits their practical application. To address this challenge, few-shot fine-tuning offers a promising approach to quickly adapt LLMs to new recommendation data. We propose the task of data pruning for efficient LLM-based recommendation, aimed at identifying representative samples tailored for LLMs' few-shot fine-tuning. While coreset selection is closely related to the proposed task, existing coreset selection methods often rely on suboptimal heuristic metrics or entail costly optimization on large-scale recommendation data. To tackle these issues, we introduce two objectives for the data pruning task in the context of LLM-based recommendation: 1) high accuracy aims to identify the influential samples that can lead to high overall performance; and 2) high efficiency underlines the low costs of the data pruning process. To pursue the two objectives, we propose a novel data pruning method based on two scores, i.e., influence score and effort score, to efficiently identify the influential samples. Particularly, the influence score is introduced to accurately estimate the influence of sample removal on the overall performance. To achieve low costs of the data pruning process, we use a small-sized surrogate model to replace LLMs to obtain the influence score. Considering the potential gap between the surrogate model and LLMs, we further propose an effort score to prioritize some hard samples specifically for LLMs. Empirical results on three real-world datasets validate the effectiveness of our proposed method. In particular, the proposed method uses only 2% samples to surpass the full data fine-tuning, reducing time costs by 97%.

1 INTRODUCTION

LLM-based recommendation requires frequent fine-tuning on rapidly expanding data, but full-data tuning is costly. The paper frames data pruning as selecting representative samples for few-shot fine-tuning while balancing accuracy and pruning efficiency.

  • Motivation: 160 million new videos and 942 billion interactions emerge on TikTok daily, motivating frequent LLM fine-tuning for updated recommendation data.The paper links frequent updates to incorporating current item information and user behavior.
  • Motivation: Fine-tuning LLMs on large-scale recommendation data requires substantial computational resources and time, reducing practical applicability.
  • Data Pruning: Few-shot fine-tuning can reduce training costs, but randomly sampled data may not represent new items and user behaviors sufficiently.The paper therefore targets representative samples tailored to LLM few-shot fine-tuning.
  • Related Work: Existing coreset methods use heuristic metrics or costly subset optimization, and heuristics may select samples suboptimally because they ignore empirical-risk influence.The cited related-work discussion identifies these as two main categories of existing approaches.
  • Objectives: Data pruning must achieve high accuracy by selecting low-risk samples while remaining efficient enough to avoid dependence on well-trained LLMs using full data.
  • Proposed Method: DEALRec combines influence and effort scores, using a surrogate model to estimate sample-removal influence and account for the surrogate–LLM capability gap.Its influence calculation extends influence functions and uses a symmetric property for one estimation across samples.
  • Evaluation: Experiments on three real-world datasets demonstrate DEALRec's effectiveness in achieving both high efficiency and accuracy.

2 TASK FORMULATION

LLM-based recommendation requires fine-tuning on recommendation data, but its resource demands and continuously changing data motivate efficient data pruning. The task seeks representative samples while addressing accuracy, efficiency, and limitations of existing coreset selection methods.

  • LLM-based recommender models: LLM-based recommender models fine-tune language models on user interaction sequences to predict the next interacted item.Each sample contains a chronological interaction history x and next item y.
  • Motivation: Fine-tuning LLMs is costly and difficult to sustain as recommendation data continuously changes.These costs hinder practical application and motivate more efficient fine-tuning.
  • Data pruning task: Data pruning selects representative samples for LLM few-shot fine-tuning, preserving information about changing user behavior and items.Random subsets may omit crucial information such as trending items.
  • Coreset selection: Heuristic coreset methods use intuitive metrics such as prediction entropy or representation clustering without explicitly estimating a sample’s influence on empirical risk.This may produce suboptimal subsets and lower performance.
  • DEALRec: DEALRec targets high accuracy and high efficiency by using influence scores, effort-score regularization, and a surrogate-model pipeline to identify influential samples.The method first trains a surrogate model on full training samples, then calculates and regularizes influence scores.
  • Coreset selection: Optimization-based coreset methods seek subsets through optimization, but complex solving limits their use on large-scale recommendation datasets.Prior methods also commonly require training over the original data D, which is infeasible for LLM-based recommenders.

3 DEALREC

DEALRec identifies influential samples for LLM-based recommendation by combining influence estimates from a surrogate model with an effort score that accounts for differences in LLM learning ability. It further uses coverage-enhanced sampling to select representative few-shot data efficiently.

  • 3 DEALREC: DEALRec combines an influence score for empirical-risk impact with an effort score that regularizes the gap between surrogate models and LLMs.The method uses a small surrogate model for efficient influence estimation and adds LLM-specific effort signals.
  • 3.1 Influence Score: Influence scores estimate how removing each training sample affects empirical risk without brute-force leaving-one-out retraining.The method extends influence functions through chain rules and uses second-order optimization techniques for approximation.
  • 3.1 Influence Score: Calculating influence scores independently for all samples remains costly, motivating stochastic Hessian-vector-product estimation and shared computation.Stochastic HVP estimation reduces computation burdens, while a single estimation can support influence scores across samples under the stated reformulation.
  • 3.2 Gap Regularization: The surrogate model reduces pruning costs, but differing learning abilities can make samples influential for LLMs differ from those identified by the surrogate.The effort score supplements surrogate-based influence by emphasizing samples significant specifically to LLM learning.
  • 3.2 Gap Regularization: The effort score measures each user sequence’s learning effort for LLMs, with larger values indicating harder samples to learn.It uses the gradient norm of the LLM parameters and reflects discrepancies between encoded knowledge and recent item knowledge or user behavior.
  • 3.3 Few-shot Fine-tuning: Coverage-enhanced selection stratifies samples by overall score and samples from the group with the fewest remaining samples to improve distributional coverage.This avoids selecting highly similar high-scoring samples while maintaining sampling budgets across groups.

4 EXPERIMENT

The experiments evaluate DEALRec against coreset and full-data baselines, examine its components and surrogate-model generalizability, and assess its performance under varying conditions.

  • 4 EXPERIMENT: Experiments on three real-world datasets compare DEALRec with coreset-selection baselines and models trained on full data.This comparison addresses whether pruning can preserve recommendation performance while reducing data.
  • 4 EXPERIMENT: The study evaluates how influence score, gap regularization, and stratified sampling affect DEALRec’s performance.The experiments also test whether the method generalizes to different surrogate models.
  • 4 EXPERIMENT: The experiments investigate DEALRec’s performance across the research questions posed for efficient LLM-based recommendation.The listed questions include comparisons, component effects, surrogate-model generalizability, and additional performance analyses.

4.1 Experimental Settings

The experiments evaluate DEALRec against random sampling and coreset-selection baselines under few-shot and full fine-tuning settings, using two LLM-based recommender backends across three datasets.

  • Datasets: Experiments use Games, MicroLens-50K, and Book datasets, split chronologically into training, validation, and testing sets with an 8:1:1 ratio.Games and Book retain interactions rated at least 4.
  • Fine-tuning settings: Few-shot fine-tuning uses fixed-size subsets such as 1024 samples, whereas full fine-tuning uses all samples without pruning.
  • Baselines: DEALRec is compared with random, difficulty-based, diversity-based, and recommendation-specific coreset-selection methods.The baselines include GraNd, EL2N, CCS, TF-DCon, and RecRanker.
  • Backend models: The study instantiates methods on BIGRec and TIGER, two competitive LLM-based recommender models with different item-sequence representations.BIGRec uses LLaMA-7B and item titles, while TIGER learns extra item tokens from item features.
  • Evaluation: Accuracy is measured with Recall@K and NDCG@K, using K=10 and 20 for Games and K=20 and 50 for MicroLens-50K and Book.Fine-tuning experiments use four NVIDIA RTX A5000 GPUs; BIGRec uses LoRA and TIGER is fully fine-tuned.

4.2 Overall Performance (RQ1)

DEALRec consistently outperforms coreset-selection methods across three datasets and two LLM-based recommender backends. Against full-data BIGRec fine-tuning, it achieves higher performance while substantially reducing fine-tuning time.

  • Overall comparison: BIGRec generally performs better than TIGER, which the authors attribute to its larger LLaMA-7B backbone and use of item titles.
  • Overall comparison: DEALRec significantly outperforms all coreset-selection methods across the three datasets and both backend models.The authors attribute this to accurate and efficient estimation of sample influence together with adaptation to LLM-specific learning gaps.
  • Surrogate-model robustness: Different surrogate models preserve DEALRec's advantage over full-fine-tuned BIGRec, demonstrating robustness across BERT4Rec, SASRec, and DCRec.
  • Comparison with full fine-tuning: 97.11% reduction of fine-tuning costs is achieved by DEALRec compared with full-data BIGRec training on average.DEALRec also achieves higher performance than the model trained on the full data.

4.3 In-depth Analysis

In-depth analyses show that DEALRec benefits from all three selection components, remains effective with different surrogate models and selection ratios, and balances influence with effort through λ.

  • Ablation Study (RQ2): Removing either the influence score or effort score causes performance drops, validating the contribution of both components.Replacing coverage-enhanced selection with greedy high-score selection is also evaluated in the ablation.
  • Different surrogate models (RQ2): DEALRec consistently outperforms full-fine-tuned BIGRec with BERT4Rec, SASRec, and DCRec as surrogate models.SASRec has the lowest surrogate-training time and competitive performance among the three models.
  • Effect of selection ratio (RQ3): Accuracy rapidly improves as the selection ratio increases from 0.2% to 1%, surpasses full training at r=1%, and gains little from increasing r beyond 2%.Fine-tuning cost reduction remains above 94% across the tested ratios; the authors recommend r=1%.
  • User group evaluation (RQ3): DEALRec evaluates user sequences grouped from easy to difficult according to loss under a randomly sampled few-shot model, supporting influence-aware selection across groups.
  • Effect of λ: Increasing λ generally improves accuracy, but excessively large values are not necessarily beneficial because influence-driven and LLM-specific sample priorities must be balanced.

5 RELATED WORK

Related work frames DEALRec as a data-pruning approach connected to LLM recommendation and coreset selection. Existing methods commonly rely on heuristic difficulty or diversity metrics to choose representative subsets.

  • LLM-based recommendation: LLM recommendation research spans CTR prediction, sequential recommendation, explainable recommendation, and in-context learning, but domain-specific fine-tuning remains important.
  • Coreset selection: Coreset selection seeks a small representative subset that maintains comparable model performance across data-efficient learning and related tasks.
  • Coreset selection: Heuristic coreset methods select difficult or diverse samples using predefined metrics rather than explicitly estimating each sample's influence on empirical risk.The paper identifies this omission as a source of potentially suboptimal selection.

6 CONCLUSION

The paper defines data pruning for LLM-based recommendation and introduces DEALRec to identify representative samples for few-shot fine-tuning. It concludes that the approach achieves high efficiency and accuracy while motivating broader applications and future research.

  • DEALRec identifies representative samples tailored for LLMs’ few-shot fine-tuning.The method targets efficient data pruning for LLM-based recommendation.
  • DEALRec uses influence and effort scores to select influential samples efficiently.The influence score estimates sample-removal effects, while the effort score addresses the surrogate-model–LLM gap.
  • Empirical results validate DEALRec’s effectiveness in achieving both high efficiency and high accuracy.
  • Future work includes applying DEALRec across more recommender models and cross-domain datasets with limited resources.
  • Future directions include selecting informative interacted items under limited LLM context windows and improving inference efficiency for deployment.
Loading 2401.17197v2…