PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters

Mind Lab, :, Vin Bo, Song Cao, Vic Cao, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Kaixuan Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Hongquan Gu, Aaron Guan, Nolan Ho, Mutian Hong, Hailee Hou, Peixuan Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Qiuyu Jin, Fancy Kong, Andrew Lei, Kyrie Lei, Alexy Li, Lucian Li, Ray Li, Theo Li, Wenhao Li, Zhihui Li, Allen Lin, Jiayi Lin, Kairus Liu, Kieran Liu, Logan Liu, Xiang Liu, Irvine Lu, Maeve Luo, Runze Lv, Pony Ma, Verity Niu, Anson Qiu, Vincent Wang, Rio Yang, Maxwell Yao, Carrie Ye, Regis Ye, Wenlin Ye, Josh Ying, Danney Zeng, Yuhan Zhan, Anya Zhang, Di Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhang, Wei Zhao, Ada Zhou, Adrian Zhou, Yuhua Zhou, Xinyue Zhu, Murphy Zhuang

arXiv:2606.02437v2cs.LGcs.CL

TL;DR

Strong assistants do not automatically preserve continuity with one person over time. This paper frames PEFT adapters as persistent local state atop shared foundation models and examines scaling up, down, and out, finding that they provide a compact unit for scalable persistent individuality.

  • Problem

    Existing assistants lack persistent, adaptive state that preserves continuity with one person over time.

  • Method

    The paper frames adapters as local adaptive state atop shared models and organizes PEFT around Scale Up, Scale Down, and Scale Out.

  • Results

    Across the three axes, the evidence supports PEFT as a practical unit of persistent individuality that is small enough to scale.

  • Takeaways & Limitations

    PEFT can support many persistent personal-model instances rather than serving only as a cheaper substitute for full fine-tuning.

  • Takeaways & Limitations

    Most experiments use controlled benchmarks and simulations, while large-scale empirical validation on personal-model deployments remains limited.

Abstract

from arXiv · show

Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.

1 Introduction

The introduction frames PEFT as a practical substrate for persistent personal model instances, where shared foundation models provide broad intelligence and adapters carry local individuality. It organizes this agenda around dependent Scale Up, Scale Down, and Scale Out problems.

  • Personal model architecture: PEFT represents part of a personal model’s persistent local state while the shared base model supplies broad intelligence.Adapters can carry preferences, skills, tool habits, and some memory-like behavior, while raw facts and documents may remain external.
  • Motivation: A small fraction of variation can produce individuality, motivating many persistent adapted models built on shared foundation-model priors.The introduction uses the analogy that any two humans share approximately 99.9% of their genome while accumulating distinct experience.
  • Three scaling axes: The three scaling axes are dependent: Scale Up strengthens shared priors, Scale Down minimizes reliable local state, and Scale Out supports many persistent instances.The paper argues that omitting either prerequisite yields adapters that are too expensive, too weak, or too disposable.
  • Scope and calibration: PEFT contributes a practical unit of individuality that is local, persistent, efficient, and manageable rather than a complete solution for personal memory or identity.Adapters can be changed and repeatedly updated without rewriting the shared base model, at lower cost than a full checkpoint.
  • Contributions: The paper’s central contribution is a framework for scaling from one shared foundation model to many persistent personal model instances through algorithmic and infrastructural work.The stated contributions include the three-axis framework and demonstrations of trillion-scale LoRA reinforcement learning with scale-induced failure modes.

2 The Three Scaling Axes of PEFT

PEFT scales through three coupled axes: stronger shared priors increase adapter leverage, smaller stable adaptive states reduce repeated-learning costs, and low marginal cost enables persistent populations. The paper argues that PEFT becomes transformative when Scale Up, Scale Down, and Scale Out reinforce one another.

  • Three Scaling Axes: Scale Up, Scale Down, and Scale Out jointly frame PEFT as both a learning problem and a systems object.The axes address shared-model strength, local-state size and stability, and the management of many persistent instances.
  • Scale Up: Scale Up gives each small adapter more latent capability to redirect by strengthening the shared base model.This makes small local updates higher leverage.
  • Scale Down: Scale Down lowers the marginal cost of repeated learning through a smaller, stabler adaptive state.The axis asks how small, stable, and repeatedly writable local state can become.
  • Scale Out: Scale Out expands personalization from isolated adapters to populations of persistent model instances when marginal cost is low.Persistence includes training, serving, evaluation, and governance over time.
  • Axis Dependencies: The three axes are mutually dependent: omitting any one yields costly adaptation, limited specialization, or many weak and disposable variants.The paper’s thesis is that PEFT becomes transformative only when all three axes reinforce one another.
  • Long-Term Memory: For long-term memory, adapters carry behavioral consequences of repeated experience while editable facts and documents remain in external memory.This defines PEFT as local adaptive state rather than a raw archive of user history.

3 Scale Up: Scaling Model Capacity for a Stronger Shared Prior

Scale Up argues that stronger shared priors make small adapters more useful by expanding the behaviors reachable during post-training. Trillion-scale LoRA RL shows that these priors can be made operationally accessible, but reliable adaptation requires aligning algorithms with distributed execution and sparse-model semantics.

  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: Scale Up makes stronger shared base models repeatedly adaptable under realistic learning budgets, so small local updates can redirect broad capabilities.The section frames model capacity as a prerequisite for persistent personal model instances, not a replacement for personalization.
  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: Larger models expand the trajectories available to post-training, allowing RL to select, sharpen, and stabilize reasoning, tool-use, and long-horizon behaviors.The stronger prior changes which behaviors exploration can discover and credit assignment can reinforce.
  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: Under broadly comparable RL budgets, larger LoRA-adapted models achieved larger headroom-normalized gains than a much smaller model trained with full-parameter RL, despite fewer trainable parameters.The comparison varies model size and training method simultaneously, so it illustrates rather than cleanly isolates prior strength from trainable parameter count.
  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: Trillion-scale LoRA RL requires coordinated handling of MoE routing, rollout–training alternation, adapter sharding, synchronization, merging, and serving.In MoE systems, numerical differences can change expert routing and therefore the effective computation.
  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: 10% of conventional full-parameter RL is the approximate GPU budget required by LoRA RL on the Kimi K2 model class, with smooth improvement and preserved general capabilities.Held-out evaluations suggest task-specific improvement without catastrophic divergence or loss of the base model’s general capabilities.
  • Scale Up: Scaling Model Capacity for a Stronger Shared Prior: Training–inference mismatch becomes an algorithmic failure when rollout and training paths activate different experts, while GLM5-style support adds sparse-attention, MTP, and adapter-semantics risks.At scale, the RL algorithm cannot be defined independently of the execution path that realizes the policy.

4 Scale Down: The Operating Regime of Efficient Adaptation

Scale Down identifies the smallest reliable persistent adaptive state, showing that middle-rank adapters are practical while extremely low-rank adapters retain attainable performance but lose across-seed reliability. Geometry-aware minor-subspace initialization, especially OLoRA-tail, makes tiny adapters more usable and robust without increasing trainable weights.

  • Scale Down: The Operating Regime of Efficient Adaptation: Scale Down seeks an adaptive state that is expressive, stable, cheap to update and store, and practical to serve across many persistent instances.The section frames efficient adaptation as a deployment regime rather than a fixed minimum rank.
  • 4.1 Standard LoRA: Ranks 16 and 32 are the current deployment default, combining the highest mean gains, relatively low downside risk, and strong token efficiency.The middle-rank band is identified as the strongest practical region under the current recipe.
  • 4.1 Standard LoRA: Ranks 1 to 4 retain best-seed performance close to ranks 16 and 32, but their across-seed means dip and seed-to-seed spread widens.The results indicate an optimization and initialization bottleneck rather than a hard capacity limit.
  • 4.1 Standard LoRA: Ranks 64, 128, and 256 increase parameter, optimizer-state, checkpoint, and memory costs without extending the observed best-run frontier.The section therefore warns against treating larger adapters as the default direction.
  • 4.2 Geometry-aware Initialization: On Qwen3-8B, OLoRA-tail delivers a consistent ∼+20% gain across batch sizes, while standard LoRA falls from +15% at bs16 to −18% at bs128, with collapse risk reaching 67%.OLoRA-tail uses minor singular vectors, which keep KL divergence near zero and training stable under RL updates.
  • 4.2 Geometry-aware Initialization: Initialization determines whether tiny adapters’ available directions are usable, and OLoRA-tail improves usable capacity without increasing trainable weights or optimizer state.The method chooses comparatively inert minor-subspace directions instead of increasing rank.

5 Scale Out: From Individual Adaptation to Population-Scale Personalization

Scale Out frames personal adapters as persistent policy-bearing state: they specialize memory, tools, action tendencies, and interaction behavior for individual users. At population scale, many such adapters can produce heterogeneous, behaviorally distinct agents rather than interchangeable instances of one shared model.

  • Individual adaptation: Personal adaptation becomes a policy layer governing what to remember, which tools to prefer, when to act, and how to recover for a particular user.Personalization therefore determines how a shared base model becomes useful, rather than merely changing response tone.
  • Individual adaptation: LoRA memory has a capacity ceiling: across 263 runs, accuracy stays near one below roughly 10−3 memory tokens per trainable parameter and collapses beyond about 10−2.Accuracy degrades between 10−3 and 10−2, indicating a sharp transition in usable memory capacity.
  • Individual adaptation: MLP LoRA provides the best parameter efficiency, maintaining higher accuracy at larger capacity-efficiency values than attention-only, combined, or unembedding adapters.This makes MLP-only training the most efficient design in the benchmark for maximizing memory capacity per trainable parameter.
  • Memory policy: Scale Out should store persistent behavioral adaptations and reusable skills in LoRA, while editable facts remain in retrieval and inspectable reality remains in tools.Context Learning acts as the write policy that selects which temporary experience becomes stable adapter state; RAG2LoRA illustrates this transfer.
  • Population-scale simulation: Per-user LoRA preserves identity-level heterogeneity, with supportive-user stance dispersion 2.18×–2.45× and skeptical-user dispersion 1.32×–2.01× that of the shared-base condition.The population also exhibits richer action environments, including broader stance dispersion, lower within-community side-homophily, and a heavier attention long tail; results are limited to one community, recommender mechanism, and seed per cell.

6 Infrastructure for PEFT Populations

PEFT populations require infrastructure that treats adapters as persistent, identifiable policy state rather than disposable files. MinT illustrates this lifecycle across repeated adaptation, compact handoff, and bounded serving residency.

  • Policy identity and lifecycle: MinT manages adapted behavior as policy records and fixed adapter revisions, separating durable identity from temporary training and serving residency.Records connect a compatible base, adapter shape, checkpoints, rollout records, exported versions, evaluation, and placement.
  • Policy identity and lifecycle: Persistent personal models need revision boundaries so memories, skills, preferences, and permissions can accumulate while remaining evaluable, serveable, rollbackable, and retireable.The policy record makes local adaptive state persist as one population member across time.
  • Scale Up: Resident dense or MoE bases let repeated LoRA RL restore only selected adapter and training state, while provenance mechanisms preserve semantic continuity from rollout through serving.The same adapted behavior must remain associated with a compatible base across trajectory generation, training probability scoring, export, and deployment.
  • Scale Down: 252 MiB versus 8.061 GB on Qwen3-4B and 1.692 GB versus 61.084 GB on Qwen3-30B show why adapter-only revisions are practical handoff artifacts.The measurements depend on rank, target modules, dtype, and tensor layout, so they illustrate a systems invariant rather than a universal ratio.
  • Scale Out: Scale Out controls many revisions through catalog addressability, CPU caching, bounded GPU diversity, scheduled cold activation, and readiness gates rather than simultaneous residency.MinT audits a 106-entry catalog, while tested active diversity is bounded at 64 distinct adapters; admission prevents warm-tenant stalls but delays new cold requests.

7 Conclusion

The paper frames PEFT as a population architecture in which strong shared base models provide general capability while lightweight adapters provide persistent local adaptive state. Its thesis depends jointly on scaling up shared priors, scaling down update cost and instability, and scaling out to persistent populations of adapted instances.

  • Conclusion: Strong shared base models provide general capability, while millions of lightweight adapters provide persistent local adaptive state rather than separate trillion-parameter checkpoints.The base model carries reasoning, world knowledge, language competence, and tool-use priors; the adapter carries local adaptive state.
  • Conclusion: The thesis requires all three axes: Scale Up makes bases worth adapting, Scale Down makes repeated updates cheap and stable, and Scale Out creates persistent populations.Removing any axis breaks the thesis: weak bases limit learning, expensive adapters prevent continuous adaptation, and without multiplicity PEFT remains local optimization.
  • Scale Up: Open Scale Up problems include a unified RL-native PEFT account and reliable tiny adapters under seed, batch, and task variation.The proposed theory must connect adapter geometry with training stability, including rank, scaling, initialization, KL drift, and off-policiness.
  • Scale Down: Open Scale Down problems include determining when stateful adapters outperform LoRA, retrieval, or full context and measuring durable gains from repeated interaction updates.δ-mem needs controlled comparisons, while Context Learning needs benchmarks for extracted improvement and its degradation or compounding.
  • Scale Out: Open Scale Out problems concern durable experience selection, long-horizon population heterogeneity, and how routing, voting, debate, and distillation scale across diverse adapters.The paper also notes that evidence remains mostly from controlled benchmarks and simulations, with limited large-scale validation on personal-model deployments.
  • Conclusion: PEFT enables a progression from one shared foundation model to persistent personal instances, user simulation, and collective intelligence from diversity among adapted models.The paper presents this as a population architecture rather than one universal assistant with increasingly centralized control.
Loading 2606.02437v2…