Source-linked AI summary
LoRA on a Budget: Function-Aware Post-Hoc Compression for Adapter Fleets
Shenghui Li, Thiemo voigt
TL;DR
Post-hoc LoRA fleet compression must distribute rank across layers and adapters because spectral redundancy alone does not reveal how much behavioral change an adapter tolerates. FRA constructs measured functional frontiers and allocates a shared budget across them. Across three public fleets, it improves tail utility in constrained-budget regimes.
Problem
LoRA fleets need rank allocated across layers and adapters under a shared capacity budget, while spectral redundancy alone does not establish functional tolerance to compression.
Source: §1 · p. 2, §3.1 · p. 3Method
FRA searches adapter-level allocations using isolated end-to-end truncation measurements, remeasures complete candidates, combines them with a spectral proposal, and allocates the shared budget across constructed frontiers.
Source: §1 · p. 2Results
Takeaways & Limitations
Cross-proposal surrogate miscalibration supports retaining complementary spectral candidates alongside functional search.
Source: §5 · p. 7The constructed frontiers approximate the ideal and may miss better allocations; isolated-effect interactions can cause search error, though complete candidates are remeasured before their risks enter fleet allocation.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
Low-Rank Adaptation (LoRA) has become a key mechanism for efficiently specializing foundation models into many task- and user-specific variants. Serving many LoRA adapters under a shared resource budget turns post-hoc compression into a two-level allocation problem: rank must be distributed across layers within each adapter and across adapters within the fleet. Although trained LoRAs contain substantial spectral redundancy, spectral reconstruction alone does not determine how much compression an adapter can functionally tolerate. We formulate fleet compression through a functional rank-distortion frontier for each adapter, followed by fleet-level minimax allocation under a shared rank budget. Given exact frontiers, this decomposition is equivalent to joint optimization over all layer ranks. We introduce Functional Rank Allocation (FRA), an unlabeled method for constructing practical frontiers. FRA measures isolated module truncations end to end, uses these measurements as an additive search surrogate solved by dynamic programming, remeasures every complete candidate, and takes the lower-risk envelope with a complementary spectral proposal before exact minimax allocation over the constructed frontiers. Across 78 adapters from three public fleets, FRA improves worst-case and tail utility over uniform spectral compression in constrained-budget regimes. Fixed-budget probes show that isolated costs rank candidates well, but their calibration shifts across proposal shapes; end-to-end verification and the spectral proposal therefore limit functional-search error.
1 Introduction
LoRA fleet compression must allocate rank both within adapters and across a shared fleet budget while preserving model behavior. FRA constructs measured functional frontiers for this allocation, and improves tail utility in constrained-budget regimes.
1 Introduction
- LoRA’s compact task-specific updates reduce the cost of specialization, while shared-base serving makes adapters compete for cache capacity, transfer bandwidth, and rank-dependent resources.Source: §1 · pp. 1–2
- At τ = .90, the median layer retains only 31.2% of nominal rank, yet compression must preserve behavior while allocating rank across layers and adapters.Source: §1 · p. 2, Figure 1 · p. 2
- FRA searches layer-rank allocations using isolated end-to-end truncation costs, remeasures complete candidates, and keeps a complementary spectral proposal in its measured frontiers.Source: §1 · p. 2
- Across 78 adapters from three public fleets, function-aware allocation substantially improves tail utility under constrained budgets.Source: §1 · p. 2
- The formulation allocates rank within each adapter’s functional rank–distortion frontier, then distributes the shared fleet budget across adapters; exact frontiers make this equivalent to joint layer-rank optimization.Source: §1 · p. 2
2 Background
LoRA represents an adapter as low-rank updates to frozen base weights. Post-hoc compression truncates those updates, while fleet budgets constrain the total retained rank.
2 Background
- Each LoRA module adds a product of low-rank factors to a frozen pretrained weight matrix, and an adapter comprises such updates across modules.Source: §2 · p. 2
- Keeping the leading k singular directions gives a Frobenius-optimal rank-k approximation that can be refactored and served in the original LoRA form.Source: §2 · p. 3
- The fleet budget counts retained rank-direction slots rather than exact bytes; deployments can instead weight directions by each module’s input and output widths.Source: §2 · p. 3
3 Method
The method formulates LoRA compression as allocating rank within adapters and across a fleet under a shared budget, using functional risk to measure behavioral change. FRA constructs practical adapter frontiers, then performs minimax allocation across them.
3.1 Problem Formulation
- Exact adapter frontiers make fleet-level minimax allocation equivalent to joint optimization over all layer ranks under the shared budget.The minimax objective is chosen to prevent a small subset of adapters from absorbing disproportionate compression damage.Source: §3.1 · p. 3
3.2 Functional Rank Allocation
- FRA measures isolated module truncations end to end on unlabeled calibration prompts and uses their summed costs as a search surrogate.Dynamic programming solves the surrogate allocation exactly; each complete proposed adapter is then remeasured end to end, and its risk is combined with a spectral proposal using the lower measured value.Source: §3.2 · p. 4
- The measured adapter frontier may not attain the ideal minimum-risk allocation at every rank budget.Isolated effects can interact, creating search error; the complete-candidate risk is remeasured, and a spectral proposal supplies an independent candidate.Source: §3.2 · p. 4
3.3 Fleet-Level Minimax Allocation
- FRA applies monotone closure to each measured frontier, then finds the smallest feasible worst-case risk across adapters under the fleet budget.For a candidate risk level, it sums the minimum ranks each adapter needs to stay below that level and selects the smallest level whose total fits the budget.Source: §3.3 · p. 4
4 Experiments
Across three LoRA fleets, the experiments compare allocation strategies at matched rank-direction budgets and measure both held-out utility and calibration cost. FRA improves worst-case retention in the binding-budget examples, while fixed-budget probes reveal proposal-shape-dependent surrogate miscalibration.
Evaluation Setup
- The evaluated fleets contain 12 LoRA Land, 25 Lots-of-LoRAs, and 41 LoRARetriever adapters, spanning reasoning, generation, domain, and classification tasks.Lots-of-LoRAs’ 25 adapters are the full reachable pool: three controlled-pool members lack public datasets and five have held-out splits too small to score.Source: §4 · p. 4
Calibration and its cost
- FRA’s 512 single-layer probes on LoRA Land are paid once per adapter at registration and take about 8 minutes on one GH200.The calibration prompts are disjoint from scored examples, and no additional cost is added to the serving path.Source: §4 · p. 5
Metric and budgets
- Retention is reported as fleet mean, p10, and minimum at budgets matching uniform truncation thresholds τuni = .90, .70, and .50.These thresholds define moderate, binding, and severe regimes; all methods share rank-direction budgets, not necessarily parameter counts.Source: §4 · pp. 5–6
Measured Functional Frontiers Matter
- The fleet optimizer needs end-to-end risk values for candidate frontiers, so every complete spectral or functional candidate is remeasured before optimization.The dynamic program solves the surrogate allocation problem exactly, but the fixed-budget probes cover only evaluated allocations, not the full feasible set.Source: §4 · p. 6
Ablating the FRA Design
- At the binding budget, FRA raises worst-case retention to 0.879 on LoRA Land, 0.910 on Lots-of-LoRAs, and 0.500 on LoRARetriever.The comparison is against 0.800 for FuncDP on Lots-of-LoRAs and 0.250 for E2E-Spec on LoRARetriever; LoRA Land remains at 0.879 after adding the spectral envelope.Source: Table 2 · p. 5, §4 · p. 6
When Does Inner Allocation Help?
- Functional search favors functional proposals in 73.7%, 58.0%, and 28.3% of matched-cost comparisons, yet median measured risk favors spectral proposals for 1/12, 9/25, and 39/41 adapters.Across the three fleets, surrogate-to-measured spectral-to-functional risk ratios are 2.74/1.73, 3.02/1.15, and 1.76/0.56, respectively.Source: §4 · p. 6, Figure 2 · p. 6
5 Conclusion
FRA allocates rank within and across LoRA adapters using approximate functional frontiers built from isolated truncation probes, end-to-end remeasurement, and a spectral envelope. Across three public fleets, it improves tail utility in constrained-budget regimes, while fixed-budget probes expose cross-proposal surrogate miscalibration.
Conclusion
- FRA combines dynamic-programming candidate generation, end-to-end remeasurement, and a spectral envelope for approximate functional frontiers, then allocates across adapters.Fleet allocation is exact over the constructed frontiers, not over all possible layer-rank allocations.Source: §5 · p. 7
- Across three public fleets, FRA improves tail utility in constrained-budget regimes and identifies cross-proposal surrogate miscalibration as a reason to retain complementary spectral candidates.Source: §5 · p. 7