PaperlayerView exact revision on arXiv

Source-linked AI summary

CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generation

Yutong Song, Jiang Wu, Weijia Zhang, Chengze Shen, Shaofan Yuan, Weitao Lu, Jian Wang, Yu Wang, Nikil Dutt, Amir M. Rahmani

arXiv:2601.06352v3cs.AI

TL;DR

Personalized generation must balance fine-grained user preferences with scalable deployment, while existing approaches face parameter-cost and preference-supervision challenges. CARD combines shared cluster adapters with implicit individual-style learning and lightweight decoding-time steering. Experiments on LaMP and LongLaMP report improved personalization quality alongside generalization and efficiency.

  • Problem

    Personalized generation must balance individual fidelity with scalable deployment, while per-user parameters are costly and high-quality preference supervision is scarce.

    Source: §1 · p. 1

  • Method

    CARD clusters users for shared adapters, learns individual stylistic deviations by contrasting user text with cluster generations, then steers decoding with lightweight logit corrections while the backbone stays frozen.

    Source: §1 · p. 2, §2.4 · p. 3

  • Results

    Across LaMP and LongLaMP, CARD consistently improves personalization quality while maintaining strong generalization and efficiency.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Decoding-time steering supports rapid user switching with minimal per-user storage, improving deployment scalability while maintaining personalization fidelity.

    Source: §1 · p. 2

    Unsupervised K-means may miss complex user relationships, one preference vector may limit diverse or evolving preferences, and noisy histories may reduce personalization quality.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Adapting large language models to individual users remains challenging due to the tension between fine-grained personalization and scalable deployment. We present CARD, a hierarchical framework that achieves effective personalization through progressive refinement. CARD first clusters users according to shared stylistic patterns and learns group-specific LoRA adapters, enabling robust generalization and strong low-resource performance. To capture individual differences within each cluster, we propose an implicit preference learning mechanism that contrasts user-authored text with cluster-level generations, allowing the model to infer user-specific style preferences without manual annotation. At inference time, CARD injects personalization exclusively at decoding via lightweight user preference vectors and low-rank logit corrections, while keeping the base model frozen. Experiments on the LaMP and LongLaMP benchmarks show that CARD achieves superior generation quality compared to baselines, while significantly improving efficiency and scalability for practical personalized text generation.

1 Introduction

CARD addresses the tension between scalable group adaptation and fine-grained personalization by combining shared cluster priors, implicit user-style supervision, and decoding-time steering.

1 Introduction

  • Existing personalization methods face scaling and supervision challenges: per-user PEFT parameters are costly, while preference data is scarce and can confound style with topic (Rafailov et al., 2023; Shenfeld et al., 2025).The paper frames the goal as retaining individual fidelity without prohibitive per-user costs or reliance on sparse user histories.Source: §1 · p. 1
  • CARD clusters users to learn shared adapters that capture group preferences, amortize adaptation costs, and establish priors for low-resource users.The framework treats broad preferences as group-level priors and fine-grained nuances as stable individual differences.Source: §1 · p. 2
  • CARD learns individual stylistic deviations through implicit preference learning that contrasts user texts with cluster baselines, reducing semantic confounding without manual preference annotation.This provides supervision for individual preferences from user-authored text and cluster-level generations.Source: §1 · p. 2
  • At inference, CARD steers generation with lightweight reward-guided logit editing while the backbone and cluster parameters remain frozen, enabling rapid user switching with minimal per-user storage.This places individual personalization at decoding time rather than in per-user model parameters.Source: §1 · p. 2

2 CARD Model Design

CARD personalizes generation by combining shared cluster-level priors with user-specific decoding adjustments. It constructs same-prompt preference pairs to learn individual stylistic differences while keeping the backbone and cluster parameters fixed.

2.1 Task Formulation

  • Personalized generation targets each user's authentic style using historical context and combines cluster-level adaptation with user-level personalization.Source: §2.1 · p. 2

2.2 Overall Framework

  • CARD uses group-conditioned logits as a shared prior, then applies a compact user preference vector at decoding without updating the backbone or cluster parameters.Source: §2.2 · p. 2, Figure 1 · p. 3

2.3 Group-level Adaptation: Clustering and PEFT

  • Users are clustered by profile-embedding similarity, and each cluster's LoRA adapter is trained on aggregated instances to provide a shared prior for low-resource users.Source: §2.3 · p. 3

2.4 Preference Pair Construction

  • Preference pairs contrast a user's response with the cluster-LoRA response to the same prompt, isolating stylistic differences while holding semantic context constant.Source: §2.4 · p. 3

2.5 User-level Personalization

  • A Bradley–Terry objective learns compact user vectors that scale stylistic dimensions and add reward-guided low-rank logit corrections only to Top-k candidates, without changing base or LoRA weights.Source: §2.2 · p. 2, §2.5 · p. 4

2.6 New User Adaptation.

  • For a new user, CARD assigns a profile embedding to a cluster and estimates a user preference vector from historical data while keeping the backbone and LoRA fixed.Source: §2.6 · p. 4

3 Experiment Settings

The experiments evaluate CARD on short- and long-form personalized generation using LaMP and LongLaMP, compare it with baselines from several personalization paradigms, and describe a shared implementation setup.

3.1 Benchmarks and Evaluations

  • LaMP covers short-form and LongLaMP long-form personalized generation; both use the usersplit setting and ROUGE-1 and ROUGE-L, with GPT-5.2 and human evaluations also reported for LaMP.Further evaluation details are in Appendices B and C.Source: §3.1 · p. 4

3.2 Baselines

  • CARD is compared with retrieval augmentation (RAG and PAG), decoding alignment (PAD), and PEFT methods OPPU and PROPER.The comparison results cover LaMP and LongLaMP; Table 1 marks best and second-best results.Source: §3.2 · p. 4, Table 1 · p. 5

3.3 Implementation Details

  • CARD and all base models use Qwen/Qwen3-8B, and retrieval baselines retrieve the top 4 history items under matched historical-context limits.Additional settings, training details, and evaluations across 0.6B-to-32B model scales are reported in Appendix D.8.Source: §3.3 · p. 5

4 Results and Analysis

The section evaluates CARD’s benchmark performance, component contributions, low-resource behavior, robustness, and efficiency. Across these analyses, it reports strong results alongside trade-offs in user-vector settings and history quality.

4.1 Performance Results

  • CARD ranks first in 10 of 12 task–metric settings across LaMP and LongLaMP, with the other two results near-best.In the LLM and human evaluations, personalized methods generally rank above non-personalized baselines, though the rankings do not perfectly match; on LaMP-5, CARD’s LLM score exceeds the reference answer by 10.5%.Source: §4.1 · pp. 5–6, Figure 2 · p. 5

4.2 Ablation Study

  • Removing either group-level adaptation or the user vector degrades performance across tasks, and removing the user vector causes the largest reported ROUGE drop.On LaMP-4, R-1 falls from 0.218 to 0.148 without the user vector; Group LoRA’s advantages are more pronounced in LLM and human judgments than in ROUGE alone.Source: §4.2 · p. 6, Table 2 · p. 6

4.3 User Vector Analysis

  • Moderate user-vector strength, dimensionality, and representation depth yield the strongest personalization; extremes can overwhelm content or introduce noise.Too few representation layers limit richness, while too many can dilute user-specific information; larger vectors may add noise or overfit, particularly with limited data.Source: §4.3 · p. 6, Figure 5 · p. 7, Table 3 · p. 7

4.4 Case Study

  • The LaMP-7 case study reports that CARD retains central themes while matching users’ expressive style more closely than group-level LoRA alone.The described stylistic cues include informal wording, heightened emotional cues, and light emojis.Source: §4.4 · pp. 6–7, Figure 3 · p. 7

4.5 Low-Resource Users Analysis

  • With five histories, CARD reaches approximately 0.216 R-1 on LaMP-4 versus approximately 0.146 for the non-personalized baseline.CARD approaches its LaMP-4 peak of approximately 0.219 R-1 around ten histories; the authors suggest noisy long histories may weaken its user vector on LaMP-7.Source: §4.5 · p. 7, Figure 4 · p. 7

4.6 Robustness Across Model Scales and Families

  • CARD maintains personalization efficacy across Qwen model capacities from 0.6B to 32B parameters, while absolute generation quality improves with larger backbones.The performance comparison is presented in Table 5.Source: §4.6 · p. 8, Table 5 · p. 8

4.7 Efficiency Analysis

  • For each new user, CARD encodes profile items for cluster assignment and optimizes a compact 128-dimensional preference vector while freezing the backbone and LoRA modules.The authors report that this design is efficient in computation and storage, can run on-device, and offers stronger privacy protection for user-specific preferences.Source: §4.7 · p. 8

5 Related Work

The section surveys personalized LLMs, PEFT approaches, and decoding-time alignment with user preferences. It also defines efficiency-comparison notation used in the discussion.

5.1 Personalized LLMs

  • Personalized LLM work includes retrieval and profile summarization, user-representation injection or memory editing, and implicit supervision from user content.LaMP is identified as a standard benchmark for personalized language generation.Source: §5.1 · p. 8

5.2 PEFT for Personalization

  • PEFT for personalization ranges from per-user adapters to shared adapter pieces, continuous prompts, and group-level or hierarchical designs.The cited work also includes factorization-based approaches for black-box alignment.Source: §5.2 · p. 8

5.3 Decoding-time Alignment with User Preferences

  • Decoding-time alignment adjusts outputs from frozen language models at inference, and related work uses preference-vector steering and lightweight user-interaction signals.Source: §5.3 · p. 8
  • The efficiency notation defines training-free as preprocessing cost; H and L denote model hidden size and Transformer-layer count, while k and j denote selected components and projection dimension.The notation also sets D=128.Source: §5.3 · p. 8

6 Conclusion

CARD combines group-level adapters with lightweight user-specific logit modulation for personalized text generation. On LaMP and LongLaMP, it improves personalization quality while maintaining generalization and efficiency.

6 Conclusion

  • CARD combines group-level adapters with lightweight user-specific modulation at the logit level for coarse-to-fine personalized text generation.Source: §6 · p. 9
  • CARD achieves fine-grained personalization without per-user model fine-tuning or long-context history retrieval.Source: §6 · p. 9
  • Experiments on LaMP and LongLaMP show that CARD consistently improves personalization quality while maintaining strong generalization and efficiency.Source: §6 · p. 9

Limitations

CARD's limitations include constraints in clustering and user representation, pipeline coordination costs, and sensitivity to weak or noisy histories.

Limitations

  • Offline unsupervised K-means clustering may not capture complex user relationships or latent personalization structure.Source: Limitations · p. 9
  • A single decoding-time preference vector may limit expression of diverse or evolving preferences, and its learned dimensions are not directly interpretable.Source: Limitations · p. 9
  • Despite the framework's lightweight design, its multi-stage pipeline requires coordination across several components.Source: Limitations · p. 9
  • Noisy or weakly relevant histories may degrade the learned user vector and reduce personalization quality.History filtering, relevance weighting, or noise-robust profile selection are proposed as future extensions.Source: Limitations · p. 9

Ethical Considerations

The paper describes privacy considerations for its public, anonymized data and presents an evaluation prompt and scoring rubric for personalization. A separate table reports relative improvements over a non-personalized baseline.

Ethical Considerations

  • The LaMP and LongLaMP datasets are public and anonymized; the authors state that users can construct personal representations locally without uploading history.The paper says this design reduces data-leakage risk relative to retrieval-based or user-specific fine-tuning methods.Source: Ethical Considerations · p. 9

A Relative Improvements of CARD

  • Table 7 presents CARD's relative improvement percentages over the non-personalized baseline.Source: Table 7 · p. 13

Persona Score evaluation prompt

Score Rubric

  • The rubric distinguishes five personalization levels, from generic or impersonal responses to excellent tailoring of profile details and tone.Source: Score Rubric · p. 13

C Dataset Statistics and Task Details

This appendix describes the task data and the reproduced personalization baselines, including retrieval, profile, embedding, and adapter approaches. LongLaMP adaptations use task-specific histories and queries, while baseline comparisons keep core model and decoding conditions aligned.

C Dataset Statistics and Task Details

D Baseline Details

  • BM25 indexes each history entry as a document and retrieves using the current task input; when context exceeds limits, history items and profiles are trimmed before retrieved-item count is reduced.The retrieved items retained are the top-scored ones.Source: D Baseline Details · p. 14
  • LongLaMP covers abstract, topic, and review generation, using prior papers, posts, or reviews as profiles and task-specific input cues for retrieval.Abstract queries use title and keywords; topic-writing queries use a post summary; review queries use product description and rating.Source: D Baseline Details · pp. 14–15

Baseline prompt skeleton

  • Across baseline prompts, user information may appear as a profile summary, retrieved history, or both; task instructions and the current input remain in the prompt.The shared prompt skeleton specifies that the model generates only the answer text.Source: Baseline prompt skeleton · p. 15

D.1 Non-personalized (Non-pers.)

  • Non-pers. removes user-specific signals, while RAG retrieves up to four BM25-ranked history entries using the current task input as its query.When fewer than four entries exist, RAG uses all available items and serializes input-output history pairs for generation tasks.Source: §D.1 · p. 16, §D.2 · p. 16

D.3 PAG (Profile-Augmented Generation)

  • PAG adds a once-per-user, cached profile summary to a prompt that also includes BM25-retrieved history, reducing profile-generation runtime costs.Profiles are generated offline with GPT-5.2 at temperature 0.Source: §D.3 · pp. 16–17

D.4 PPLUG (Persona-Plug User Embedding)

  • PPlug aggregates encoded user histories into an input-aware personal embedding, attaches it as a continuous prefix, and trains the embedder while freezing the backbone LLM.The history representations are weighted with respect to the current input; a learned projection maps user embeddings to the LLM representation space.Source: §D.4 · p. 17

D.5 OPPU (One PEFT Per User, LoRA)

  • OPPU uses personal LoRA modules (rank 8 in this reproduction) and, at inference, combines each user’s module with a profile summary and BM25 top-4 retrieved histories; CARD uses K-means and rank-16 cluster-LoRA.During this stage, the base model parameters remain frozen.Source: §D.5 · p. 18, §D.6 · p. 18

D.7 Implementation Details for PAD

PAD is reproduced as a decoding-time steering baseline: a separately trained personalized reward model adjusts the output distribution of a frozen language model. Its implementation uses preference-conditioned reward features and a vectorized decoding strategy.

D.7 Implementation Details for PAD

  • PAD steers a frozen base language model at decoding by combining its likelihood with a reward signal from a separately trained personalized reward model.The reward model evaluates compatibility between a user preference and a candidate token at each step.Source: §D.7 · pp. 18–19
  • PAD trains preference-agnostic features first, then freezes the backbone and optimizes the mapping from user preference to its reward-model embedding.Training triples contain user preference, input, a ground-truth positive response, and a greedily generated negative response.Source: §D.7 · p. 18
  • To reduce inference latency, PAD computes full-vocabulary logits in one batch but applies reward aggregation only to the base model’s top-k indices.The reproduced candidate pool uses k = 10, and the penalty coefficient β is tuned on validation data.Source: §D.7 · p. 19

D.8 Implementation Details and Hyperparameters

Table 10 summarizes CARD implementation details and hyperparameters.

D.8 Implementation Details and Hyperparameters

  • Table 10 lists CARD implementation details and hyperparameters, including λu for the new-user preference-learning stage.The supplied passage gives the table’s scope but not its hyperparameter values.Source: Table 10 · p. 19

E Alignment between LLM and Human Judgments

The authors assess LLM-as-a-judge ratings against human judgments on LaMP and report strong positive correlations and agreement. Alignment varies by task, with stylistic tasks showing somewhat lower agreement than objective, structure-heavy tasks.

E Alignment between LLM and Human Judgments

  • LLM scores show strong positive correlations with human judgments, and rank correlations indicate reliable preservation of relative personalization-quality rankings.The evaluation reports Pearson, Spearman, Kendall, Cohen’s Kappa, and Quadratic Weighted Kappa metrics.Source: E Alignment between LLM and Human… · p. 19
  • Agreement is highest on objective, structure-heavy LaMP5 and slightly lower, yet robust, on stylistic LaMP7, where human aesthetic preferences diverge.The authors attribute the task variation to factual-fidelity capture on LaMP5 and differences in aesthetic preferences for informal text on LaMP7.Source: E Alignment between LLM and Human… · p. 19
  • Average QWK is 0.618, while standard Cohen’s κ is around 0.38; the authors conclude the automated protocol is a trustworthy proxy for human evaluation.QWK applies penalties proportional to squared score differences on the ordinal 1–5 scale.Source: E Alignment between LLM and Human… · p. 19, Table 11 · p. 20

Related papers

Loading 2601.06352v3…