PaperlayerView exact revision on arXiv

Source-linked AI summary

HYDRA: Model Factorization Framework for Black-Box LLM Personalization

Yuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang, Qifan Wang, Chao Zhang, Bo Dai

arXiv:2406.02888v3cs.CLcs.AIcs.LG

TL;DR

Black-box LLM personalization must adapt outputs without access to model parameters, while prompt-based approaches can have difficulty capturing shared user knowledge. HYDRA factorizes reranking and adaptation into shared base models and user-specific heads; it reports a 9.01% average relative improvement over RAG and PAG baselines across five personalization tasks.

  • Problem

    Black-box LLMs are difficult to personalize through parameter updates, and prompt-based approaches may struggle to capture shared knowledge across users.

    Source: §1 · pp. 1–2

  • Method

    HYDRA trains a personalized reranker and adapter, each decomposed into a shared base model and multiple user-specific heads.

    Source: §1 · p. 2

  • Results

    9.01% average relative improvement over state-of-the-art RAG and PAG baselines across five diverse personalization tasks.

    Source: §5 · p. 10

  • Takeaways & Limitations

    HYDRA offers a learning-based alternative for black-box personalization without requiring access to model parameters.

    Source: §5 · p. 10

    Prompt-based methods may be distracted by irrelevant retrieved information, and user-specific prompts lack access to global knowledge.

    Source: §2 · p. 3

Abstract

from arXiv · show

Personalization has emerged as a critical research area in modern intelligent systems, focusing on mining users' behavioral history and adapting to their preferences for delivering tailored experiences. Despite the remarkable few-shot capabilities exhibited by black-box large language models (LLMs), the inherent opacity of their model parameters presents significant challenges in aligning the generated output with individual expectations. Existing solutions have primarily focused on prompt design to incorporate user-specific profiles and behaviors; however, such approaches often struggle to generalize effectively due to their inability to capture shared knowledge among all users. To address these challenges, we propose HYDRA, a model factorization framework that captures both user-specific behavior patterns from historical data and shared general knowledge among all users to deliver personalized generation. In order to capture user-specific behavior patterns, we first train a reranker to prioritize the most useful information from top-retrieved relevant historical records. By combining the prioritized history with the corresponding query, we train an adapter to align the output with individual user-specific preferences, eliminating the reliance on access to inherent model parameters of black-box LLMs. Both the reranker and the adapter can be decomposed into a base model with multiple user-specific heads, resembling a hydra. The base model maintains shared knowledge across users, while the multiple personal heads capture user-specific preferences. Experimental results demonstrate that HYDRA outperforms existing state-of-the-art prompt-based methods by an average relative improvement of 9.01% across five diverse personalization tasks in the LaMP benchmark. Our implementation is available at https://github.com/night-chen/HYDRA.

1 Introduction

Personalizing black-box LLMs is difficult because their parameters cannot be modified, while prompt-based approaches can struggle to represent useful history and shared knowledge. HYDRA factorizes a reranker and adapter into shared base models and user-specific heads, and its LaMP evaluation reports improvements over baselines.

1 Introduction

  • Fine-tuning and RLHF-based personalization require model-parameter access, restricting their use to white-box LLMs.The passage notes that white-box models tend to be less capable than black-box models.Source: §1 · pp. 1–2
  • Prompt-based RAG methods can struggle to capture shared patterns across users, while using complete profiles risks context-length limits and substantial costs.Randomly selected records may also fail to represent users’ patterns effectively.Source: §1 · p. 2
  • HYDRA combines a personalized reranker and adapter with shared base models and user-specific heads to capture both shared and individual behavior patterns.The reranker prioritizes useful information from retrieved histories; the adapter aligns black-box outputs with user preferences without access to internal parameters.Source: §1 · p. 2
  • HYDRA improves over the best-performing baselines by an average of 4.8% across five LaMP tasks.The tasks include three text classification and two text generation tasks.Source: §1 · p. 2
  • HYDRA integrates global knowledge from a base model with local preferences from user-specific heads to support generalizable personalization.The paper presents this factorization as a learning-based approach for black-box LLMs.Source: §1 · p. 2

2 Related Works

Prior prompting approaches personalize LLMs by sampling user histories, summarizing them into profiles, or retrieving relevant records. The discussion identifies limits involving irrelevant information, retrieval usefulness, and the absence of shared global knowledge.

2 Related Works

  • In-context learning uses randomly sampled user histories as few-shot examples, and longer histories may improve performance.Examples include ratings, interaction histories, and reviews.Source: §2 · p. 3
  • Profile-augmented prompting summarizes user preferences and behavior patterns into natural-language profiles for query augmentation.The cited examples generate summaries of user histories or browsing interests.Source: §2 · p. 3
  • Retrieval-augmented prompting selects relevant historical records to provide evidence while managing limited context length.Examples retrieve behavioral history or user-authored documents.Source: §2 · p. 3
  • Prompting methods may retrieve information that is relevant but not useful for a new query, and user-specific prompts lack global knowledge.PAG may also be distracted by irrelevant records, especially when user behavior shifts.Source: §2 · p. 3

3 HYDRA: Model Factorization for Black-Box LLM Personalization

HYDRA formulates black-box personalization as learning shared knowledge alongside user-specific preferences, using retrieved history to adapt generation without modifying the black-box LLM. Its pipeline reranks history, scores candidate generations with personalized adapters, and represents personalization through a shared base and user-specific heads.

3.1 Problem Formulation: Black-Box LLM Personalization

  • Personalization conditions target generation on a user's query and history, but black-box parameters are inaccessible, motivating a shared-plus-personal model decomposition.The formulation separates general knowledge from user-specific knowledge.Source: §3.1 · pp. 3–4

3.2 Retrieve-then-Rerank

  • At inference, HYDRA retrieves top-N query-relevant histories and reranks them for usefulness to select top-k; training candidates also include histories from sampled prior queries.Reranker labels reflect whether history-assisted generation matches the target, with a Rouge threshold for generation tasks.Source: §3.2 · pp. 4–5

3.3 Black-box LLM Adaptation

  • The adapter is trained on query-plus-candidate-generation examples labeled against personalized targets, then scores black-box LLM proposals so the highest-scoring response can be selected.The LLM supplies candidate responses; the adapter evaluates them for alignment with user preferences.Source: §3.3 · p. 5

3.4 Model Factorization for Personalization

  • Each user model combines shared base σ with personal head τ(u); for a test user, HYDRA freezes the base, fits a new head from history, then uses both for inference.The paper describes the test-user head fitting process as requiring minimal computational resources.Source: §3.4 · p. 6

4 Experiments

Experiments evaluate HYDRA on five LaMP tasks and examine its components, scalability, shared knowledge, and performance under behavior shifts. HYDRA improves over the best-performing baselines, with further analyses exploring performance as user data and history vary.

4.1 Experimental Setup

  • Evaluation uses 100 training users and 50 test users, compares black-box prompting baselines, and reports task-specific classification, rating, and generation metrics.Source: §4.1 · pp. 6–7

4.2 Main Results

4.3 Ablation Studies

  • HYDRA-Reranker and HYDRA-Adapter improve RAG and PAG separately, outperforming either alone; removing personalized training reduces performance across all tasks.Source: Table 2 · p. 7, §4.3 · p. 8

4.4 Scale-up Analysis

  • More user history improves classification and generation across metrics; HYDRA leads across selected-history counts, though noisy or irrelevant records may disrupt RAG on LaMP-3.Source: §4.4 · pp. 8–9, Figure 3 · p. 8
  • HYDRA maintains its advantage over baselines as training users increase from 100 to 1,000, with 500 users used for testing.Source: Table 3 · p. 9, §4.4 · p. 9

4.5 Empirical Personalization Analysis

  • More training-user data improves results over relying only on test-user history, while updating only user-specific heads outperforms updating the full model.Source: §4.5 · pp. 9–10, Figure 4 · p. 9
  • HYDRA outperforms state-of-the-art baselines when queries are unrelated to some historical records, with the reranker selecting by utility and the adapter aligning outputs to preferences.Source: §4.5 · p. 10, Figure 5 · p. 10
  • HYDRA still beats baselines under extreme user-interaction imbalance, although performance falls relative to random user selection.Source: §4.5 · p. 10

5 Conclusions

HYDRA combines user-specific behavior patterns with shared knowledge for black-box LLM personalization. It outperforms state-of-the-art RAG and PAG baselines across five LaMP tasks.

5 Conclusions

  • 9.01% average relative improvement over state-of-the-art RAG and PAG baselines across five diverse LaMP personalization tasks.The conclusion also notes that Figure 6 examines significant disparities in user behavior history.Source: §5 · p. 10, Figure 6 · p. 10
  • HYDRA captures and leverages both user-specific and shared behavior patterns from historical data.The framework uses a reranker and black-box LLM adapter without requiring access to model parameters.Source: §5 · p. 10
  • HYDRA presents a learning-based alternative to mainstream prompt-based methods for black-box LLM personalization.The conclusion describes the framework as a potential foundation for targeted user-experience improvements and human-centric intelligent systems.Source: §5 · p. 10

A.1 Limitations

The appendices describe practical constraints, societal considerations, related personalization approaches, black-box adaptation challenges, and HYDRA's training procedure.

A.1 Limitations

  • Experiments were limited to Azure OpenAI's gpt-3.5-turbo(1106) because of API access and budget constraints.HYDRA-Adapter does not share training data with third parties via APIs, but retrieved histories or queries may contain private information; the black-box model's size may also preclude on-device inference.Source: §A.1 · p. 15

A.2 Broader Impacts

  • Personalization may improve user experience and accessibility, but privacy, security, bias, and filter-bubble risks require attention.The paper identifies fairness, diversity, and inclusivity as priorities for future work and proposes exploring multimodal vision-language personalization.Source: §A.2 · p. 15

A.3 Ethical Statements

  • The study used publicly available datasets, followed Azure data-use guidelines, and withdrew from human review to prevent potential information leaks.The authors report submitting the Azure OpenAI Additional Use Case Form.Source: §A.3 · p. 16

B.1 LLM Personalization

  • Table 4 compares personalization methods across user-specific and global knowledge, retrieval, learning-based approaches, and black-box availability.Its listed aspects also distinguish retrieval for relevance from retrieval for usefulness.Source: Table 4 · p. 16

B.2 Learning-based Personalization for White-Box LLMs

  • Learning-based personalization approaches such as RLHF often require white-box access and explicit preference information, while HYDRA incorporates shared knowledge alongside user-specific preferences.The cited discussion notes that collecting explicit preference pairs can require additional human annotation effort.Source: §B.2 · p. 17

B.3 Black-Box LLM Adaptation

  • Black-box fine-tuning APIs raise transparency, privacy, and cost challenges, motivating HYDRA-Adapter's approach without direct parameter or API-service access.The passage notes limited visibility into trainable layers and weights, privacy concerns from uploading data, and higher tuning costs.Source: §B.3 · p. 17

C Algorithm Details

  • HYDRA trains shared parameters and user-specific heads, then freezes the shared base and updates user-specific heads using history and query candidates.The algorithm constructs labeled reranker training data and uses reranked history candidates with the query for adapter training.Source: C Algorithm Details · p. 18

D Dataset and Task Details

The dataset appendix describes the LaMP tasks used for personalization evaluation and notes several exclusions from the empirical study.

D Dataset and Task Details

  • LaMP-2N, LaMP-2M, and LaMP-3 evaluate personalized categorization, movie tagging, and product rating, respectively.LaMP-2N predicts one of 15 news categories, LaMP-2M one of 15 movie tags, and LaMP-3 a rating from one to five.Source: D Dataset and Task Details · p. 18
  • LaMP-4 and LaMP-5 evaluate personalized headline and scholarly-title generation using users' historical article-title pairs.LaMP-4 focuses on authors' stylistic nuances; LaMP-5 generates titles for research articles.Source: D Dataset and Task Details · pp. 18–19
  • LaMP-6 was excluded because its dataset was not publicly available, while LaMP-1 and LaMP-7 were excluded because history and query formats were inconsistent.Table 6 is described as providing statistics for the five included personalization tasks.Source: D Dataset and Task Details · p. 19, Table 6 · p. 19

E Baseline Details

The baseline setup compares non-personalized and personalized prompting approaches using a shared black-box backbone and retrieval configuration.

E Baseline Details

  • The experiments use gpt-3.5-turbo(1106) with the LaMP prompt template and BM25 as the default retriever.The comparisons include both non-personalized and personalized baselines.Source: E Baseline Details · p. 19
  • The non-personalized baselines are zero-shot gpt-3.5-turbo and ICL-Random, which adds random history items to the query.ICL-Random uses k randomly selected behavior-history items.Source: E Baseline Details · p. 19
  • RAG adds the top k retrieved history items, while PAG adds a generated profile summary and can also include retrieved history.PAG's profile summary is generated by gpt-3.5-turbo.Source: E Baseline Details · p. 19

F Implementation Details

The experiments used specified generation settings for black-box LLMs and common training hyperparameters across experiments.

F Implementation Details

  • Experiments ran on an AMD EPYC 7702 CPU and an NVIDIA A100-SXM4-80GB GPU using Python 3.10.13.Source: F Implementation Details · p. 20
  • Black-box LLM generations used a maximum solution length of 512 tokens, temperature 1.0, and b = 8 intermediate candidates during adapter adaptation.Other baselines used temperature 0.Source: F Implementation Details · p. 20
  • Training used learning rate 2e−5, global batch size 64, two epochs, AdamW, and weight decay 0.01.Source: F Implementation Details · p. 20

G.1 Additional Analysis of Baselines in the Main Experimental Results

The analysis finds that adding user history often helps black-box LLM performance, but retrieval-based prompt methods remain behind HYDRA.

G.1 Additional Analysis of Baselines in the Main Experimental Results

  • HYDRA outperforms RAG and PAG, which may be limited by noisy retrieved behavior patterns and lack of shared knowledge across users.Source: §G.1 · p. 20
  • Randomly selected history or profiles improve performance over zero-shot GPT-3.5-turbo in most tasks, though LaMP-3 has less optimal cases.The authors hypothesize that noise and irrelevant behavior patterns may complicate user-behavior understanding.Source: §G.1 · p. 20
  • Increasing the number of retrieved items is positively correlated with performance, while RAG and PAG provide only marginal improvements over random selection.Source: §G.1 · p. 20

G.2 Additional Experimental Details for Sacle-up Analysis

The additional analyses examine history volume, behavior shifts, retriever choice, and adapter design in HYDRA.

G.2 Additional Experimental Details for Sacle-up Analysis

  • Figure 7 reports accuracy and F-1 across different amounts of selected user history, with results indicating robustness to additional history.Source: §G.2 · p. 20, Figure 7 · p. 20

G.3 Additional Experimental Details for User Behavior Shift

  • Under user behavior shift, HYDRA still outperforms state-of-the-art baselines; its reranker prioritizes usefulness, while prompt methods can suffer from mismatches between demonstrations, histories, and queries.The passage attributes the performance improvement as possibly stemming from the reranker and adapter.Source: §G.3 · pp. 20–21

G.4 Effect of Retrievers

  • Dense semantic retrievers such as Contriever perform similarly to sparse retrievers such as BM25 in black-box LLM personalization.Source: §G.4 · p. 21, Table 7 · p. 21

G.5 Effect of Black-Box LLM Adapters

  • HYDRA-Adapter outperforms BBox-Adapter under the same personalization settings, and is described as capturing global and user-specific knowledge.Source: §G.5 · p. 21, Table 8 · p. 21

G.6 Additional Datasets

The paper reports results on two additional datasets, gives efficiency-analysis details, and illustrates how its reranker handles retrieval order.

G.6 Additional Datasets

G.7 Efficiency Analysis

  • The stated complexity of each stage is proportional to transformer attention, O(dL^2); training uses T epochs, while inference requires one pass.Table 10 is described as summarizing time complexity and running time on LaMP.Source: §G.7 · p. 21, Table 10 · p. 22
  • Reranker training forms M^2 + 1 samples per user with M = 20 by default, while adapter training forms H̄k samples per user, with k = 8 by default.H̄ denotes the average number of histories per user.Source: §G.7 · pp. 21–22

H.1 Case Studies for Effectiveness of Reranker

  • The reranker can preserve a retriever's correct relevance ordering or assign higher scores to more relevant terms when retrieval order is wrong.The cases illustrate usefulness as an additional measurement dimension beyond relevance.Source: §H.1 · p. 22

H.2 Error Analysis

The appendix describes recurring reranker errors and documents prompt designs and demonstrations for RAG/PAG baselines and HYDRA’s RAG workflow.

H.2 Error Analysis

  • The reranker’s common errors are saturated scores that obscure distinctions, incorrect scores that favor less relevant terms, and divergent scores that exaggerate differences among inputs.Divergent scores can occur on hard samples where none is closer to the target than the others.Source: §H.2 · p. 22
  • Table 11 defines Target as ground-truth categorization, Gen as retrieved-text categorization, Score as the reranker score, and Order as the retriever’s ranking.Source: Table 11 · p. 23

I Prompts

Loading 2406.02888v3…