Source-linked AI summary
Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs
Ziyi Zhao, Chongming Gao, Yang Zhang, Haoyan Liu, Weinan Gan, Huifeng Guo, Yong Liu, Fuli Feng
TL;DR
Foundation-model upgrades can invalidate user-specific soft prompts and require expensive retraining. PUMA migrates them with a parameter-efficient adapter and representative user selection, matching or surpassing retraining while reducing computation by up to 98%.
Problem
Foundation-model changes break the alignment of user-specific soft prompts, making them obsolete and requiring costly retraining.
Method
PUMA uses an end-to-end parameter-efficient migration adapter and group-based user selection based on prompt embeddings and output variance.
Results
PUMA matches or surpasses full retraining across three large-scale datasets while reducing computational cost by up to 98%.
Takeaways & Limitations
PUMA supports scalable personalization across diverse architectures, chained migrations, and aggregated multi-source migration.
Takeaways & Limitations
Migration transfers existing knowledge but does not create new knowledge, and weaker intermediate models can cause information loss.
Abstract
from arXiv · showhide
Personalization in Large Language Models (LLMs) often relies on user-specific soft prompts. However, these prompts become obsolete when the foundation model is upgraded, necessitating costly, full-scale retraining. To overcome this limitation, we propose the Prompt-level User Migration Adapter (PUMA), a lightweight framework to efficiently migrate personalized prompts across incompatible models. PUMA utilizes a parameter-efficient adapter to bridge the semantic gap, combined with a group-based user selection strategy to significantly reduce training costs. Experiments on three large-scale datasets show our method matches or even surpasses the performance of retraining from scratch, reducing computational cost by up to 98%. The framework demonstrates strong generalization across diverse model architectures and robustness in advanced scenarios like chained and aggregated migrations, offering a practical path for the sustainable evolution of personalized AI by decoupling user assets from the underlying models.
Introduction
Personalized soft prompts are efficient user assets but become obsolete when their foundation model changes. PUMA addresses this migration problem with an adapter and representative user sampling, matching retraining performance at far lower cost.
- Soft prompts enable user-specific customization without changing the foundation model, forming “1+N” systems with one model and many user representations.
- Foundation-model replacement breaks prompt–model semantic alignment, making existing personalized prompts obsolete and forcing costly retraining.
- PUMA separates migration into semantic incompatibility and migration efficiency, addressing both cross-model interpretation and scaling to tens of thousands of users.
- PUMA combines an end-to-end parameter-efficient adapter with group-based sampling by prompt embeddings and output variance to create a representative training subset.
- Up to 98% lower computation accompanies performance that matches or surpasses from-scratch retraining across three large-scale datasets and model changes.
- Chained and aggregated migrations extend PUMA beyond single transfers, with aggregated multi-source migration improving knowledge integration and post-migration performance.
Related Work
Prior personalization and prompt-transfer methods remain coupled to their base models and generally address task-level transfer. This work instead targets large-scale migration of private user prompts while using efficient coreset selection.
- Personalization research includes retrieval augmentation, prompt engineering, reinforcement learning, and parameter-efficient methods such as personalized PEFT modules and selective prompt tuning.
- Existing PEFT personalization methods are coupled to their base models and require costly retraining when models are updated.
- Coreset Selection: Coreset selection seeks a small representative subset that approximates training on the full dataset, using score-based or optimization-driven approaches.
- Coreset Selection: Per-sample gradient or uncertainty evaluations are computationally infeasible at the target scale, motivating a more efficient selection strategy.
- Prior soft-prompt transfer studies focus on general task-level knowledge transfer across downstream tasks or models.
- This work addresses the “N-to-N” migration of many user-personalized prompts, preserving private personalization across foundation-model changes.
Methodology
The methodology formalizes migration as mapping source-model user prompts into functionally equivalent target prompts, then trains a lightweight adapter while keeping models and source prompts frozen. PUMA also uses chained or aggregated migration designs and representative user selection based on preference diversity and output complexity.
- Cross-Model Migration: PUMA formalizes cross-model migration as learning a function that maps each source prompt to a functionally equivalent target-model prompt.The source and target models may differ in architecture or embedding dimensionality, creating a semantic gap.
- Adapter Training: The migration adapter is trained by minimizing target-model task loss while the target model and source prompts remain frozen.Only the migration function is optimized during this process.
- Advanced Migration: PUMA supports chained migration across sequential models and aggregated migration by concatenating prompts from multiple source models before mapping them to one target.Aggregated migration is intended to synthesize a richer user representation from diverse sources.
- User Selection: To reduce training cost, PUMA trains on a compact user subset selected to represent both prompt-embedding diversity and the spectrum of user output complexity.The subset is drawn from the full training data using users selected into U′.
- User Selection: The two-stage selection process clusters source prompts with K-means, then samples users within variance strata using greater weight for medium-variance groups.This combines preference-profile diversity with output-variance coverage.
Experiments
The experiments evaluate PUMA against retraining and sampling baselines, test generalization across models, and examine robustness in chained and aggregated migration settings.
- Research Questions: The experiments address effectiveness, selection efficiency, cross-architecture generalization, and robustness to chained and aggregated migration topologies.These four objectives correspond to RQ1 through RQ4.
Experimental Setup
The experimental setup spans three recommendation datasets, multiple model migrations, task-specific metrics and losses, and comparisons against retraining, random initialization, and alternative sampling strategies.
- Datasets: The evaluation uses Amazon and Yelp rating prediction datasets and MIND clickthrough-rate prediction data.Amazon and Yelp contain 1-to-5 star ratings, while MIND uses implicit click feedback.
- Metrics: Performance is measured with RMSE and MAE for rating prediction, and AUC and user-weighted AUC for clickthrough-rate prediction.User-weighted AUC averages per-user AUC scores on MIND.
- Model Migrations: The study migrates prompts from Llama-2-1B-Instruct to Llama-2-3B-Instruct and tests broader migrations across five model configurations.The setup targets both model-size changes and architectural or family diversity.
- Training: The experiments freeze foundation-model parameters, pre-train source prompts for 15 epochs, and train the PUMA adapter for 4 epochs.Adapter training uses FusedAdam, a learning rate of 1 × 10^-4, and batch size 32.
- Losses: Rating prediction uses 0.8·LMSE+0.2·LCE, while MIND uses binary cross-entropy on the “yes” token logit.The rating objective combines continuous regression and classification losses.
- Baselines: PUMA is compared with full retraining and random initialization, while efficiency experiments compare group-based selection with random, variance-, loss-, clustering-, and hybrid sampling methods.The full-retraining baseline retrains all user prompts on the target model.
Performance Results
PUMA improves personalized-prompt migration across models, using efficient user selection and adapter-based transfer to preserve or enhance performance in standard and advanced migration settings.
- Effectiveness: PUMA outperforms full retraining across Amazon, Yelp, and MIND, including RMSE reductions and a uAUC increase from 0.5289 to 0.6552.Reported RMSE values are 0.9135 vs. 0.9414 on Amazon and 1.1073 vs. 1.1994 on Yelp.
- Efficiency: A 2,000-user PUMA selection achieves Amazon RMSE 0.9315, beating Random (6k) at 0.9320 while using one-third as many users.The fixed-budget comparison uses 2,000 users for Amazon/Yelp and 1,500 for MIND, whereas Random (6K) uses 6,000 users.
- Efficiency: Approximately 5,000 randomly selected users are needed merely to match full-retraining RMSE on Amazon, motivating group-based selection.The figure is presented as evidence that unguided sampling requires substantially more data for comparable migration quality.
- Generalization across Architectures: Across architectures and model families, PUMA often matches or exceeds full retraining, although migration quality remains bounded by source-prompt quality.Weaker sources may not reach target-model retraining performance, while high-performing sources such as Phi-3 can surpass it.
- Advanced Migration Scenarios: In five-model chain migration, RMSE remains strong from 0.9348 initially to 0.9277 finally, despite successive migrations without original-data retraining.A minor dip occurs when migrating from Gemma, but the final result exceeds direct Gemma-to-StableLM migration.
- Advanced Migration Scenarios: Aggregating Llama and StableLM prompts for Phi-3 achieves RMSE 0.9217, outperforming single-source migrations from Llama at 0.9293 and StableLM at 0.9380.The reported result is attributed to complementary knowledge from multiple source models.
Conclusion & Discussion
PUMA migrates personalized soft prompts across foundation models with a lightweight adapter and efficient user selection, matching or surpassing full retraining while reducing computational costs by up to 98%. The paper also identifies extensions toward broader asset migration and more complex personalization settings.
- Conclusion & Discussion: PUMA combines a lightweight adapter framework with an efficient user selection strategy for migrating personalized soft prompts across foundation models.The approach addresses both semantic incompatibility and migration efficiency.
- Conclusion & Discussion: Up to 98% lower computational costs accompany performance that matches or surpasses full retraining across three large-scale datasets.
- Conclusion & Discussion: The framework generalizes across diverse model architectures and remains robust in chained migrations while supporting knowledge fusion from multiple source models.
- Conclusion & Discussion: Future work includes learning-based user selection, simultaneous migration of user and item embeddings, and cold-start personalization for new target users.The authors propose reinforcement-learning-based selection and using the migration adapter to initialize prompts for users absent from the source system.