PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs

Yutong Song, Jiang Wu, Shaofan Yuan, Chengze Shen, Jian Wang, Yu Wang, Nikil Dutt, Amir M. Rahmani

arXiv:2601.06362v3cs.AI

TL;DR

Explicit style instructions can undermine user-specific characteristics that personalization methods aim to preserve. PsPLUG models the user-specific signal as a style-conditioned residual and uses an inference-time control to balance personalization with style adherence. It achieves best or second-best performance in over 80% of style–task–metric settings.

  • Problem

    Existing personalization methods can suffer persona degradation under explicit style constraints, leaving the challenge of balancing task directives with implicit user preferences.

    Source: §1 · p. 1

  • Method

    PsPLUG models personalization as a residual from neutral base behavior and uses style-conditioned preference training in a soft-prompt plug-in with inference-time scaling.

    Source: §1 · p. 2

  • Results

    Over 80% of style–task–metric settings show best or second-best performance, with gains over prior baselines of up to 2–4 ROUGE points.

    Source: §4.1 · p. 6

  • Takeaways & Limitations

    PsPLUG offers scalable persona injection without per-user fine-tuning and reports strong alignment and style compliance with minimal inference overhead.

    Source: §6 · p. 9

    Evaluation uses four predefined styles and specific backbones within LaMP; robustness across architectures, multilingual settings, and long-context domains remains unverified.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Personalized large language models are often expected to follow explicit style instructions, yet we find that such instructions can undermine the user-specific characteristics that personalization methods aim to preserve. We call this failure mode personalization collapse: explicit style control can conflict with implicit user preferences. To address this challenge, we propose PsPLUG, a lightweight plug-in that learns a user-specific residual after accounting for the requested style. PsPLUG also allows us to tune personalization strength at inference time. Our experiments show that explicit style instructions can diminish personalization in existing methods, whereas PsPLUG better preserves user preferences while providing precise control over the balance between personalization and style adherence.

1 Introduction

Explicit style instructions can override implicit user preferences, creating a challenge for personalized generation. PsPLUG models personalization as a residual and provides a tunable way to balance it with style adherence.

1 Introduction

  • Existing personalization methods can suffer severe persona degradation when explicit system instructions dominate generation and override user-specific traits.The paper identifies balancing explicit task directives with implicit user priors as the central challenge.Source: §1 · pp. 1–2
  • PsPLUG frames personalization as the deviation between a user's linguistic distribution and a neutral base-model distribution under identical inputs.A user-authored response reflects personalized preferences, while a zero-shot response reflects a generic population-level prior.Source: §1 · p. 2
  • Its style-conditioned preference objective contrasts user-authored texts with style-following negatives that lack user-specific nuances.This is intended to isolate persona signals from dominant instruction and style effects.Source: §1 · p. 2
  • PsPLUG is a lightweight soft-prompt module that prepends learned prefix embeddings to a frozen LLM, avoiding per-user fine-tuning.The prefix is trained as a plug-in for the frozen backbone.Source: §1 · p. 2
  • On LaMP, PsPLUG consistently outperforms state-of-the-art baselines in preserving persona alignment under strong instructions.Its inference-time scaling coefficient α adjusts the trade-off between instruction adherence and personalization strength.Source: §1 · p. 2

2 Method

PsPLUG models personalization as a user-specific residual relative to generic behavior under the same context, then uses a modular prefix and style-conditioned preference training to balance user preferences with explicit instructions.

2.1 Task Formulation

  • The task pairs an input with an optional style instruction and seeks a policy that reproduces user behavior while balancing that instruction; experiments use four predefined styles.The reference model represents a non-personalized, population-level distribution.Source: §2.1 · p. 2

2.2 PsPLUG Framework

  • A compact prefix combines system-instruction, user-history, and current-input vectors to condition a frozen LLM and support rapid user-context switching.Scaling the user vector provides continuous inference-time control over personalization without changing the system-instruction or input vectors.Source: §2.2 · pp. 2–3, §2.3 · p. 4

2.3 PsPLUG Training: Persona–Style Balancing

  • Style-conditioned preference pairs favor user-authored responses over style-only outputs, anchoring personalization to the reference model’s style-following behavior.This trains the user-specific residual beyond style compliance.Source: §2.3 · p. 4

2.4 Special Case: Personalization without Style Instructions

  • Without a style instruction, the style-conditioned reference reduces to a generic baseline, and user information enters through the user vector as the personalization residual.This special case trains PsPLUG without style guidance.Source: §2.4 · pp. 4–5

3 Experimental Settings

Experiments follow the LaMP benchmark protocol with task-specific metrics and auxiliary personalization and style scores, using Qwen3-8B, automated LLM judging, and matched personalization baselines.

3.1 Datasets and Evaluation

  • LaMP-1, LaMP-2, and LaMP-3 use classification or regression metrics, while LaMP-4, LaMP-5, and LaMP-7 use ROUGE-1, ROUGE-L, and METEOR.Controlled customization is additionally assessed with LLM-judged personalization and style scores, with human validation on a subset.Source: §3.1 · p. 5

3.2 Implementation Details

  • All tasks use Qwen/Qwen3-8B, with user profiles encoded by a frozen sentence encoder and GPT-5.2 PRO used as an automated generation-quality judge.The experiments run on eight NVIDIA H100 GPUs.Source: §3.2 · p. 5

3.3 Baselines

  • Baselines include a non-personalized model, BM25 retrieval of user-history demonstrations, and three personalization methods: PAG, OPPU, and PPlug.The backbone and decoding configurations are matched for a fair comparison.Source: §3.3 · p. 5

4 Results and Analysis

Across LaMP, PsPLUG retains personalization under explicit styles and allows strength adjustment. Results also show that styles differ in their interference with personalization, while PsPLUG’s inference cost is independent of user-history size.

4.1 Main Results

  • Without explicit styles, PsPLUG consistently outperforms existing personalization baselines, though it remains competitive with PPlug on a few tasks.The reported comparison covers LaMP personalization benchmarks.Source: §4.1 · p. 5, Table 2 · p. 6
  • Warm and critical styles tend to disrupt personalization baselines more than concise, which can improve scores when aligned with task structure and user preferences.For the non-personalized baseline on LaMP-5, R-1 rises from 0.426 to 0.441 under concise.Source: §4.1 · p. 6
  • On LaMP-7, PsPLUG achieves the highest style scores across styles; human and LLM judgments broadly align but diverge more under elaborative style.Concise receives the highest style scores for all methods, while critical and elaborative are generally harder to control.Source: §4.1 · pp. 6–7, Figure 3 · p. 7
  • PsPLUG ranks best or second-best in over 80% of style–task–metric settings and beats prior baselines by up to 2–4 ROUGE points.These are personalization results under explicit styles.Source: §4.1 · p. 6, Table 3 · p. 7

4.2 Strength sensitivity test

  • Concise remains most stable across strength settings, while warm and elaborative are more sensitive; excessive elaborative strength sharply degrades LaMP-5 title generation.PsPLUG’s strength parameter provides a controllable trade-off.Source: §4.2 · p. 7, Figure 5 · p. 8

4.3 Efficiency and scalability

  • Medium-sized models often perform competitively and can outperform larger models on some metrics, although larger models generally score better on some tasks.The comparison covers LaMP tasks, including LaMP-5 and LaMP-7.Source: §4.3 · p. 7, Table 4 · p. 8
  • PsPLUG encodes each user profile once as a static embedding and prepends it at inference, keeping prompt length and computational cost independent of history size.The fixed-vector design also keeps inference overhead constant as |Pu| changes.Source: §4.1 · p. 5, Table 5 · p. 8, §4.3 · p. 8

5 Related Works

Prior personalization work adapts models through user-specific training, retrieved histories, or lightweight modules. These approaches leave open how to separate persona signals from neutral behavior and compose them with explicit style instructions.

5.1 Personalized LLMs

  • Training-based personalization uses user data, feedback, edits, or mined preferences, but can incur per-user adaptation costs and often leaves persona–style composition unspecified.Examples include PUGC preference pairs and DPL’s contrastive learning across users.Source: §5.1 · p. 8

5.2 Retrieval-based Personalized LLMs

  • Retrieval-based methods inject user histories into frozen models, but retrieval can increase latency and consume context length.The paradigm includes top-k history retrieval for diverse tasks and extensions to long-form generation.Source: §5.2 · p. 8

5.3 PEFT for Personalization

  • Personalization PEFT methods use lightweight user-specific modules, while existing formulations can under-specify persona disentanglement and lack inference-time control over persona–task composition.Related approaches include modular composition, hierarchical grouping, low-rank adaptation, and post-hoc model merging.Source: §5.3 · pp. 8–9

6 Conclusion

The paper frames personalization as a residual from a neutral policy and introduces PsPLUG, a soft-prompt plug-in with an inference-time scalar to balance personalization and style adherence. Experiments on LaMP show superior alignment and strong compliance with minimal inference overhead.

6 Conclusion

  • PsPLUG frames personalization as a residual from a neutral policy and jointly models persona and style.It avoids fitting a user-specific output distribution.Source: §6 · p. 9
  • The soft-prompt plug-in injects persona without per-user fine-tuning, while inference-time α balances personalization preservation and style adherence.The approach is described as scalable, with minimal inference overhead.Source: §6 · p. 9
  • Experiments on LaMP demonstrate superior alignment and strong compliance.The conclusion reports minimal inference overhead.Source: §6 · p. 9

Limitations

The study uses four predefined style categories and specific backbone models within LaMP; robustness across architectures, multilingual settings, and diverse long-context domains remains unverified.

Limitations

  • Style conditions cover four predefined categories, while experiments use specific backbones within LaMP; broader styles and settings remain unverified.The authors identify open-ended and compositional stylistic demands, other architectures, multilingual settings, and diverse long-context domains as future scope.Source: Limitations · p. 9

7 Ethical Considerations

The study uses publicly accessible, strictly anonymized LaMP and LongLaMP benchmarks and obtains datasets through official APIs. Its architecture separates personalization signals from raw histories, allowing local user representations and transmission of lightweight modules instead of sensitive logs.

7 Ethical Considerations

  • LaMP and LongLaMP are public and strictly anonymized, and datasets were obtained through official APIs without proprietary or non-open-source data.The authors say no personally identifiable information is exposed.Source: §7 · p. 9
  • User representations can be constructed locally, with only lightweight personalization modules transmitted instead of sensitive history logs.The authors say this design significantly minimizes data-leakage risk compared with retrieval-augmented generation or user-specific fine-tuning.Source: §7 · p. 9

A Dataset Statistics and Task Details

The appendix points to tables covering six LaMP tasks' statistics, task formats, and experimental hyperparameters. It also specifies a personalization evaluation prompt and a five-level scoring rubric.

A Dataset Statistics and Task Details

Persona Score evaluation prompt

  • The persona evaluation prompt asks for rubric-based feedback on personalization and an integer score from 1 to 5.It directs evaluators to consider alignment with the user's preferences, interests, and background, not unrelated general quality.Source: Persona Score evaluation prompt · p. 13

Score Rubric

  • The rubric ranges from generic or impersonal responses at score 1 to highly tailored, preference-aligned responses at score 5.Intermediate scores describe increasing use of relevant user details and tone matching.Source: Score Rubric · p. 13

A.3 Hyperparameter Settings

  • Table 8 lists the hyperparameter settings used across experiments.The passage provides no individual settings.Source: Table 8 · p. 14

B Detailed Baseline Implementations

The appendix specifies implementation protocols for the baselines, including zero-shot generation, retrieval- and profile-based approaches, and a personalized plug-in. The methods differ in how they use user history and train personalization components.

B Detailed Baseline Implementations

B.1 Non-Personalized Zero-Shot (Zero-Shot)

  • Zero-Shot uses Qwen3-8B without user-specific context and wraps the style instruction and task input in the model’s chat template.The baseline uses greedy decoding to minimize variance.Source: §B.1 · p. 14

B.2 Retrieval-Augmented Generation (RAG)

  • RAG retrieves up to four history documents with BM25, falls back to Zero-Shot when there is no lexical overlap, and allocates context through dynamic truncation.Retrieval queries use the current task input but exclude the style instruction; retrieved title-and-abstract examples are formatted as demonstrations.Source: §B.2 · pp. 14–15

B.3 Profile-Augmented Generation (PAG)

  • PAG compresses user history into a task-specific natural-language profile using a two-step summarize-then-generate pipeline.Historical examples are formatted before an instruction-tuned Qwen3-8B-Instruct model extracts the profile.Source: §B.3 · p. 15

B.4 Personalized Plug-in (PPlug)

  • PPlug encodes and aggregates history with a frozen BERT-base encoder and an input-aware attention mechanism, then projects it to the LLM embedding dimension.Unlike PsPLUG, PPlug is trained with standard causal language-modeling loss rather than preference optimization or a style-balancing residual loss.Source: §B.4 · p. 15

B.5 One-PEFT-Per-User (OPPU)

This appendix describes OPPU, a separate per-user adapter baseline, and the system prompts used to evaluate four target styles. The prompts assess style matching against style-specific criteria.

B.5 One-PEFT-Per-User (OPPU)

  • OPPU trains a separate LoRA adapter for each user and is evaluated only without style instructions because its user-specific parameters overfit historical style and resist conflicting instructions.The authors describe it as a theoretical upper bound for personalization fidelity.Source: §B.5 · pp. 15–16

C System Prompts

  • The evaluation prompts provide a task description, response, reference answer, and style-specific rubric, directing the judge to assess style rather than factual correctness.The appendix lists Warm, Concise, Critical, and Elaborative as the four evaluated styles.Source: C System Prompts · p. 16

C.1 Warm and Humorous Prompt

  • The Warm and Humorous rubric rewards a friendly persona with gentle jokes and a soft, uplifting tone, while penalizing robotic or overly serious responses.Its score-5 criterion calls for warmth, empathy, and gentle humor without harshness, sarcasm, or coldness.Source: §C.1 · p. 17

C.2 Concise and Formal Prompt

  • The Concise and Formal rubric prioritizes professional, objective text, precise language, and avoiding unnecessary elaboration or colloquialisms.Its highest score requires concise, well-structured, technically accurate text without slang or unnecessary elaboration.Source: §C.2 · pp. 17–18

C.3 Sharply Critical Prompt

  • The Sharply Critical rubric rewards directly identifying flaws and discourages hedging or overly balanced phrasing, without permitting abusive or disrespectful language.The highest-score criterion emphasizes clear, firm, unambiguous criticism.Source: §C.3 · pp. 18–19

C.4 Reflective and Elaborative Prompt

  • The Reflective and Elaborative rubric rewards detailed reasoning, concrete examples, and consideration of multiple perspectives.Its highest score describes a deep reflection supported by relevant examples and balanced consideration of perspectives.Source: §C.4 · p. 19

D Alignment Between Human Evaluation and LLM Judgment

The authors compare human judgments with LLM scores for style and persona assessment on LaMP-7. They report strong alignment across both dimensions, while noting residual subjectivity in personalization judgments.

D Alignment Between Human Evaluation and LLM Judgment

  • Style evaluation correlations with human judgments were Pearson r = 0.858, Spearman’s ρ = 0.864, and Kendall’s τ = 0.725; persona evaluation correlations were Pearson’s r = 0.704 and Spearman’s ρ = 0.712.The reported system-level correlations aggregate mean scores across five evaluated models and four target styles on LaMP-7.Source: D Alignment Between Human Evaluation… · pp. 19–20, Table 10 · p. 20
  • The authors report that human and LLM evaluators both identify PsPLUG as state of the art across stylistic contexts, while remaining score variance is attributed primarily to personalization’s inherent subjectivity.They note that human annotators sometimes weight surface fluency and verbosity more than strict profile use, whereas the LLM follows the persona-grounding rubric.Source: D Alignment Between Human Evaluation… · p. 20

Related papers

Loading 2601.06362v3…