PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization

Yilun Qiu, Xiaoyan Zhao, Yang Zhang, Yimeng Bai, Wenjie Wang, Hong Cheng, Fuli Feng, Tat-Seng Chua

arXiv:2503.02450v3cs.CL

TL;DR

General LLMs’ one-size-fits-all approach may not reflect distinct user preferences, and finding useful inter-user differences requires choosing comparison users and guiding extraction. DPL selects representative users and structures task-relevant difference extraction; experiments report improved personalization and state-of-the-art performance on real-world review-generation data.

  • Problem

    Personalized generation must reflect users’ distinct preferences, but identifying useful inter-user differences requires selecting whom to compare and guiding what is extracted.

    Source: §1 · pp. 1–2

  • Method

    DPL clusters users to select representative comparison users and structures extraction of task-relevant differences as instructional context for generation.

    Source: §1 · p. 2, §3.2.1 · p. 4

  • Results

    DPL enhances personalization on review generation and achieves state-of-the-art performance on real-world datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    DPL treats inter-user differences as key preference information for personalizing LLM generation.

    Source: §1 · p. 2

    Validation focuses on review generation, and the three preference aspects used there may not suit other tasks, which may require additional factors.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Personalizing Large Language Models (LLMs) has become a critical step in facilitating their widespread application to enhance individual life experiences. In pursuit of personalization, distilling key preference information from an individual's historical data as instructional preference context to customize LLM generation has emerged as a promising direction. However, these methods face a fundamental limitation by overlooking the inter-user comparative analysis, which is essential for identifying the inter-user differences that truly shape preferences. To address this limitation, we propose Difference-aware Personalization Learning (DPL), a novel approach that emphasizes extracting inter-user differences to enhance LLM personalization. DPL strategically selects representative users for comparison and establishes a structured standard to extract meaningful, task-relevant differences for customizing LLM generation. Extensive experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization. We release our code at https://github.com/SnowCharmQ/DPL.

1 Introduction

General LLMs may not serve users’ distinct preferences, and existing personalization approaches overlook comparisons that can identify preference-shaping differences. DPL selects representative users and structures difference extraction; experiments report state-of-the-art performance for personalized text generation.

1 Introduction

  • General LLMs’ “onesize-fits-all” paradigm is described as ineffective for users with distinct preferences who expect personalized generation.The paper frames personalization as adapting generation to individual preferences.Source: §1 · p. 1
  • Existing personalization methods overlook inter-user comparison, which is described as essential for identifying differences that shape preferences.The paper argues that these differences matter because individual uniqueness shapes preferences.Source: §1 · p. 1
  • The paper proposes difference-aware user modeling, treating differences among users as key preference information for enhancing LLM personalization.The approach is motivated by the view that differences make users unique.Source: §1 · pp. 1–2
  • The approach addresses the cost of comparing a target user with everyone and the risk that unguided LLM extraction yields irrelevant differences.It therefore selects comparison users and guides extraction toward task-relevant differences.Source: §1 · p. 2
  • DPL selects representative users and uses structured, task-relevant extraction to capture differences as instructional contexts for personalization.The method clusters users by data similarity, compares cluster-center users, and focuses on dimensions such as writing style, emotional tone, and semantic content.Source: §1 · p. 2
  • DPL achieves state-of-the-art performance on real-world datasets, and experiments on review generation show enhanced LLM personalization.The reported validation task is personalized review generation.Source: §1 · p. 2

2 Preliminary

Personalized generation uses a user’s historical texts as evidence of preferences, with this paper focusing primarily on review generation. A conventional memorize-then-inject approach extracts preference information from user history to guide generation.

Problem Formulation

  • The task is to generate text for a target user and item in a way that reflects the user’s preferences, using historical texts they wrote or preferred.The formulation represents each historical sample as a user, item, and text tuple.Source: §2 · p. 2
  • The paper primarily studies personalized review generation for items such as movies or products, aiming to align generated reviews with true user reviews.Source: §2 · p. 2

Memorize-then-Inject Solution

  • A memorize-then-inject approach stores user history and retrieves or summarizes key information as preference context for generation.The formulation is presented as requiring no LLM retraining and potentially supporting lifelong historical data in memory (Zhang et al., 2024c).Source: §2 · p. 2

3 Methodology

DPL forms personalization context by combining a user's key history with differences extracted through comparison to other users. It selects representative users and guides difference extraction with predefined dimensions.

3.1 Overview

  • DPL combines extracted inter-user differences with the target user’s key history to form an instructional preference context for LLM generation.The process summarizes differences across key historical elements together with the user’s own key history.Source: §3.1 · p. 3

3.2 Selective User Comparison and Structured Difference Extraction

  • For each key historical element, DPL compares the target user’s text with other users’ texts on the same item, then extracts and structures the differences.Selective comparison reduces complexity, while a predefined standard is intended to keep differences task-relevant and meaningful.Source: §3.1 · p. 3, §3.2 · p. 4

3.2.1 Selective User Comparison

  • DPL clusters users with texts on the relevant item into K+1 groups and selects K central users outside the target user’s group for comparison.Clustering is based on the users’ texts and is intended to retain a global view while limiting comparisons to representatives.Source: §3.2.1 · p. 4

3.2.2 Structured Difference Extraction

  • The extraction standard guides the LLM to identify differences across writing, emotional, and semantic style dimensions.These dimensions address lexical and syntactic patterns, sentiment and affective tone, and the depth, clarity, and coherence of meaning.Source: §3.2.2 · p. 4

4 Experiments

Experiments evaluate DPL on real-world review generation, comparing it with baselines and examining its components, hyperparameters, user uniqueness, and qualitative outputs. DPL leads across the evaluated datasets and metrics, while ablations and analyses probe where its gains arise and how they vary.

4 Experiments

  • The experiments address benchmark performance, component effects, hyperparameters, uniqueness levels, and qualitative personalization through case analysis.Source: §4 · p. 4

4.1 Experimental Setup

  • Experiments use Amazon Reviews 2023 review-generation data from Movies & TV, CDs & Vinyl, and Books, with lexical-overlap and LLM-based personalization metrics.DPL uses Qwen2.5-14B-Instruct as its extractor, summarizer, and generator.Source: §4.1 · p. 5

4.2 Main Results (RQ1)

  • DPL ranks first across all three datasets and evaluation metrics; Non-Perso is weakest, summarization methods beat RAG, and CICL and Persona-DB lag behind DPL.On CDs & Vinyl, DPL averages 4.29% relative improvement over the best baselines, with six metrics showing significant gains; Movies & TV and Books have six and four significant metrics, respectively.Source: Table 1 · p. 6, §4.2 · p. 6

4.3 Ablation Studies (RQ2)

  • Replacing cluster-based user selection with random selection or SimRank reduces performance, and removing extraction dimensions generally lowers performance on Books.The extraction dimensions are writing style, emotional style, and semantic style.Source: §4.3 · p. 7, Table 2 · p. 7

4.4 In-depth Analyses (RQ3 & RQ4)

  • The in-depth analyses examine performance across hyperparameter values and levels of user uniqueness.The uniqueness comparison uses absolute improvement over the non-personalization method.Source: §4.4 · p. 7, Figure 3 · p. 7

4.4.1 Hyper-parameter Analysis

  • With temperature fixed at 0.8, increasing K generally yields slight-to-moderate performance gains, with the best results at K = 4.K is the number of selected representative users for comparison.Source: §4.4.1 · pp. 7–8, Figure 2 · p. 7

4.4.2 Impact of User Uniqueness

  • Users in the top 50% of historical-review embedding distance from the mean show significantly greater improvement over Non-Perso than users in the bottom 50%.The paper derives these unique and non-unique labels from review data because they do not inherently exist.Source: Figure 3 · p. 7, §4.4.2 · p. 8

4.5 Case Study (RQ5)

  • In one Movies & TV case, DPL’s review has more sections consistent with the user’s review and is more personalized and concise than LLM-TRSR’s.The paper also describes DPL’s review as better aligned with the user’s writing style and semantic preferences.Source: §4.5 · p. 8, Figure 4 · p. 8

5 Related Work

Prior personalization methods use user-specific information through retrieval or summarization, while DPL focuses on inter-user differences as preference signals. Related work also uses reviews as a personalization testbed and explores data from other users.

5 Related Work

  • Personalized text generation spans short-form evaluation with LaMP and long-form evaluation with LongLaMP, alongside work in recommendation and image generation.The related-work passage describes personalized text generation as foundational to LLM personalization.Source: §5 · p. 8
  • Item review writing is a key personalized text-generation task because reviews reflect individual preferences and can test personalized content generation.The cited work presents personalized text generation as foundational to LLM personalization.Source: §5 · pp. 8–9
  • Retrieval-based methods select useful user information, while summarization-based methods produce profiles or personalized embeddings from user histories.Examples include ROPG and HYDRA for retrieval, and ONCE and PPlug for summarization.Source: §5 · p. 9
  • Persona-DB enriches relevant knowledge from other users rather than capturing uniqueness, while contrastive learning depends on meaningful negative examples that are difficult to obtain here.The authors state that DPL addresses this limitation.Source: §5 · p. 9

6 Conclusions

DPL uses inter-user differences as preference information and combines selective user comparison with structured extraction. Experiments on personalized review generation show significant personalization gains, while the method's validation and context-length assumptions remain bounded.

6 Conclusions

  • DPL significantly enhances LLM personalization on the personalized review generation task.The authors limit current validation to review generation and plan to extend it to other tasks.Source: §6 · p. 9
  • DPL identifies inter-user differences as preference information through selective user comparison and structured extraction of task-relevant differences.The extraction is intended to capture meaningful differences for personalization.Source: §6 · p. 9
  • The method's effectiveness depends on LLM long-context processing, and the study assumes its context window can handle the data.The authors plan to extend the method to long-context cases.Source: §6 · p. 9

Limitations

The reported review-generation criteria and computational costs bound the current approach: other tasks may need different preference factors, and inter-user comparison adds overhead.

Limitations

  • The study focuses on writing style, emotional style, and semantic style for reviews; other tasks may need different factors, such as personality traits.The authors note that these standards may not be as suitable in other tasks.Source: Limitations · p. 9
  • Comparing users adds computational overhead and time costs.The authors suggest precomputing and storing difference-aware representations in user memory, then retrieving them online and updating periodically.Source: Limitations · p. 9
  • The approach uses LLMs to extract inter-user differences, summarize difference-aware representations, and generate personalized reviews.The paper says these processes require a highly powerful model to analyze personalized information and align reviews with user preferences.Source: Limitations · p. 9

Ethical Statements

Deployment raises privacy and misuse concerns because the method stores and retrieves interaction histories and integrates information across users. The study also reports compliance with the dataset's MIT license.

Ethical Statements

  • Integrating information across users raises privacy concerns, so deployment calls for anonymization, secure storage, and explicit user consent.The authors also flag risks of surveillance or targeted manipulation from detailed user profiles.Source: Ethical Statements · p. 9
  • Storing and retrieving users' historical interaction data requires strict safety protocols and ethical guidelines.The authors place responsibility on developers to handle user data with integrity and respect.Source: Ethical Statements · p. 9
  • The research uses an open-source dataset under its MIT license and states that it prioritizes data integrity, privacy, and ethical use.Source: Ethical Statements · p. 9

A Dataset Details

The paper builds a review-generation benchmark from Amazon user reviews and item metadata, then compares DPL with baselines that use no user data, retrieve or summarize history, or incorporate other users’ reviews.

A Dataset Details

  • The Amazon dataset joins user reviews to item metadata and retains review, rating, timestamp, and item-description fields for personalized review generation.The main dataset uses each user’s 10th- to 3rd-most-recent reviews for training, the 2nd-most-recent for validation, and the 3rd-most-recent for testing; Figure 5 shows examples of the main and item datasets.Source: A Dataset Details · pp. 13–14, Figure 5 · p. 13
  • Data curation filters for complete long-form reviews and item descriptions, multiple reviewers per item, and sufficient user histories and shared-item reviewers for evaluation.Users are limited to 18–500 reviews, and validation and test users must have at least eight historical records linked to items reviewed by at least five users.Source: A Dataset Details · pp. 13–14

B Baselines Details

  • Baselines range from no user-specific input and BM25-retrieved history to dynamically summarized histories and recurrent summaries over history blocks.LLM-TRSR updates its summary with newer blocks and includes the final summary and retrieved reviews in the generator prompt.Source: B Baselines Details · pp. 14–15
  • Other baselines use contrastive examples from the same item or add reviews from a similar user alongside the target user’s summary and history.CICL contrasts a target-user review with another user’s review; Persona-DB selects a similar user using embeddings.Source: B Baselines Details · p. 15

C Evaluation Metrics

The evaluation combines text-overlap metrics with LLM-based scoring against reference reviews. Experiments use a consistent inference setup, including a 2048-token maximum output length and top_p of 0.95.

C Evaluation Metrics

  • ROUGE-1 measures unigram overlap, while ROUGE-L measures the longest common subsequence and METEOR accounts for exact matches, stemming, synonyms, and word order.BLEU measures n-gram precision; the experiments use the standardized SacreBLEU implementation.Source: C Evaluation Metrics · p. 15
  • S-72B and S-GPT use LLM evaluators to score generated reviews against reference reviews, with S-72B averaging two runs and S-GPT averaging five evaluations.S-72B uses Qwen-2.5-72B-Instruct-AWQ; S-GPT uses GPT-4o-mini.Source: C Evaluation Metrics · pp. 15–16
  • The evaluation prompt supplies the generated and reference text plus review and item metadata, with italicized angle-bracketed text marking variable placeholders.The prompt is shown in Figure 6.Source: C Evaluation Metrics · p. 16, Figure 6 · p. 16

D.1 Running Environments

  • Experiments use vLLM and Sentence-Transformers in Python 3.11.11 on one NVIDIA H100 NVL GPU with 96GB memory, a 2048-token output limit, and top_p of 0.95.The output-length and top_p settings are consistent across experiments.Source: §D.1 · p. 16, §D.2 · p. 16

E Efficiency Analysis

DPL’s comparison-user selection adds little measured runtime, while most components have stable cost as dataset size grows; K-means selection is the size-dependent step.

E Efficiency Analysis

  • 0.016s per generation is the reported additional cost of selecting comparison users for eight key historical reviews.Selection averages 0.002s per key historical review using K-means with precomputed review embeddings.Source: E Efficiency Analysis · p. 16
  • Under 0.3 seconds is the reported time to cluster 10,000 samples, while K-means selection has complexity O(u′ × K × t) ≈ O(u′).Here K = 5, and u′ is the number of users who interacted with the item; t is the number of iterations.Source: E Efficiency Analysis · p. 16
  • Most component costs remain stable as data size increases, and the authors characterize DPL’s additional overhead as minimal and its scaling as comparable to baselines.Comparison-user selection is the stated exception because its cost depends on the number of users who interacted with an item.Source: E Efficiency Analysis · pp. 16–17

F.1 Yelp Dataset

On Yelp, DPL reportedly outperforms all baselines across four evaluation metrics with statistically significant improvements. Additional experiments examine model size and review-history volume, while case studies illustrate DPL’s representations and component outputs.

F.1 Yelp Dataset

  • DPL achieves the best performance on Yelp and statistically significant improvements over all baselines across four evaluation metrics.The Yelp results are reported in Table 6.Source: Table 6 · p. 17, §F.1 · p. 17

F.2 Model of Different Size

  • With a 32B model, DPL retains state-of-the-art performance on Books, while CICL and Persona-DB drop on some metrics and can underperform other baselines.The results compare the 32B and 14B models; Table 7 reports the Books-category results for the 32B model.Source: §F.2 · p. 17, Table 7 · p. 18

F.3 Number of Retrieved Reviews

  • With eight retrieved reviews, DPL outperforms all other methods across the three Amazon categories; across history sizes, it is comparable to or better than the strongest baselines.Performance metrics generally rise as the number of retrieved reviews increases; the results are reported in Table 8.Source: Table 8 · p. 18, §F.3 · p. 18

G Overview of Templates & Prompts

H Case Studies

H.1 Case Study: Comparison between Difference-aware Representation and User Profile Summary

  • DPL’s difference-aware representation better captures writing, emotional, and semantic style than LLM-TRSR’s profile summary, which includes irrelevant information.The comparison is illustrated in Figures 8 and 4.Source: §H.1 · pp. 19–20, Figure 8 · p. 19

H.2 Case Study: Outputs of DPL Components

  • The difference extractor, summarizer, and review generator each produce outputs described as well-structured and meaningful.Figure 11 presents the component outputs, with the generated review as the final targeted review.Source: §H.2 · p. 20, Figure 11 · p. 21
Loading 2503.02450v3…