PaperlayerView exact revision on arXiv

Source-linked AI summary

FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning

Reeyansh Shah, Maram Hasan, Subhasis Chaudhuri, Biplab Banerjee

arXiv:2610.21720v1cs.LGcs.AIcs.CL

TL;DR

Federated LLM unlearning must remove targeted knowledge when forget and retain data are distributed across private clients and inaccessible to the server. FIRM combines Fisher-normalized merging with a retention-constrained projection in a single-round framework. Experiments show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU.

  • Problem

    Federated LLM unlearning must aggregate locally trained updates with opposing forget and retain objectives while the underlying client data remain inaccessible to the server.

    Source: §1 · p. 1

  • Method

    FIRM uses client-local Fisher statistics for a single-round normalized base merge and a second-order projection that constrains a diagonal-Fisher retention proxy.

    Source: §1 · p. 2, §4.2.3 · p. 4

  • Results

    Experiments show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU.

    Source: §7 · p. 9

  • Takeaways & Limitations

    Controlled comparisons support benefits of the combined Fisher-based design beyond global rescaling and additional retention correction at selected operating points.

    Source: §7 · p. 9

    FIRM constrains a second-order empirical Fisher proxy rather than exact retain-loss change, and its fidelity depends on how well the diagonal Fisher represents local retain-loss curvature.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Federated LLM unlearning addresses targeted knowledge removal when forget and retain data remain distributed across private clients and are inaccessible to the server. The resulting locally trained adapters encode conflicting objectives, making their aggregation fundamentally different from conventional federated optimization. A key failure mode is magnitude explosion, whereby unnormalized cross-client aggregation increases the merged update magnitude and degrades retained utility under stronger unlearning. We propose FIRM (Fisher-Informed Retention Merging), a single-round framework for stable and retention-aware server-side aggregation. FIRM uses client-local Fisher statistics as a unified sensitivity signal to construct a Fisher-normalized base merge and impose explicit control over retain-sensitive parameters. Its second-order retention projection admits a unique minimum-distance solution, while the corresponding dual scales with the number of retain clients rather than the model dimension. Experiments across WMDP, TOFU, and MUSE-HP using Zephyr-7B and Llama-3-8B show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU. Controlled comparisons support benefits of the combined base merge and additional retention correction at selected operating points.

1 Introduction

Federated LLM unlearning must combine updates with opposing forget and retain objectives while client data remain private. FIRM uses Fisher-informed merging and retention control to address aggregation scale and retain-side damage.

1 Introduction

  • Centralized unlearning methods commonly assume access to forget and representative retain data, limiting their fit to distributed, data-local settings.Retraining billion-parameter models is costly, making targeted post-hoc removal practically important.Source: §1 · p. 1
  • Federated unlearning requires the server to arbitrate between forget and retain updates without access to either client dataset.Unlike conventional federated learning, client objectives have opposing functional roles.Source: §1 · p. 1
  • Unnormalized aggregation can amplify merged updates and degrade retained utility under stronger unlearning, a failure mode termed magnitude explosion.Sign-based aggregation also lacks explicit control over parameter importance to retain clients.Source: §1 · pp. 1–2
  • FIRM uses client-local Fisher information for a single-round, Fisher-normalized merge that controls update scale and retain-sensitive parameters without server access to client data.Its second-order retention projection has a unique minimum-distance solution and a dual with one variable per retain constraint.Source: §1 · p. 2

2 Related Work

Prior work addresses centralized unlearning, adapter merging, and federated unlearning, but the paper identifies a gap in aggregating LLM updates with opposing forget and retain roles under data-locality constraints.

2 Related Work

  • Centralized LLM unlearning methods use approaches such as gradient ascent, gradient difference, KL penalties, task-vector subtraction, and representation redirection, but assume access to forget and retain data.TIES, KnOTS, and Fisher-weighted merging address adapter or model combination, while FIRM applies Fisher sensitivities to opposing forget and retain roles.Source: §2 · p. 2
  • FedOSD avoids forget–retain gradient interference through iterative client-gradient access, whereas FIRM targets one-shot server-side aggregation of LLM adapters.FIRM requires one task vector and Fisher summary from each client.Source: §2 · p. 2
  • FULM combines locally trained unlearning and retention adapters through hierarchical TIES merging and inter-group summation, but its aggregation is unnormalized and lacks an explicit retain-side constraint.FIRM addresses this gap with a single-round Fisher-normalized merge and retention-constrained projection without server access to client data.Source: §2 · p. 3

3 Problem Setup

The setup places a shared pretrained LLM at a central server while forget and retain data remain with separate clients. Clients adapt local copies and send task vectors and Fisher information rather than raw data.

3 Problem Setup

  • A central server hosts the shared initial LLM, while separate forget and retain clients hold knowledge to remove or preserve.The server seeks an updated model that forgets targeted knowledge while maintaining retained utility, with no raw data transmitted.Source: §3 · p. 3
  • Clients locally adapt copies with LoRA and form task vectors from the resulting weight updates; forget clients optimize unlearning and retain clients optimize preservation.The LoRA update is W + sBA, with rank r much smaller than the weight dimensions.Source: §3 · p. 3
  • Each client estimates a diagonal empirical Fisher at the shared pretrained reference, providing nonnegative parameter sensitivities while task vectors provide signed directions.For TOFU, the reference precedes the expert adaptation used to initialize client training.Source: §3 · p. 3

4 Method

FIRM uses a single-round client-summary protocol and a server-side pipeline that pools forget updates by domain, normalizes Fisher information, forms a base merge, and optionally projects it to control retain-side perturbations. The projection has a unique minimum-distance optimum and a dual whose variable count depends on retain clients, although evaluation still depends on tensor dimension.

4 Method

  • Each client locally adapts the pretrained model and transmits one task vector and raw empirical Fisher diagonal; the server computes the final model without further client participation.Only Fisher coordinates aligned with the task vector enter aggregation, and raw client data remain local.Source: §4 · p. 3, §4.1 · p. 3

4.2.1 Intra-Domain Aggregation

  • When multiple forget clients share a domain, FIRM forms a domain-level centroid by mean aggregation, preserving opposing signed contributions for the later Fisher-normalized merge.Retain summaries remain separate to preserve client-specific constraints.Source: §4.2.1 · p. 3
  • The server independently normalizes and sparsifies pooled forget and retain Fisher tensors before aggregation.The retained fraction is set by α, with threshold ties kept.Source: §4.2.1 · p. 3, Figure 1 · p. 4

4.2.2 Fisher-Weighted Base Merge

  • Fisher-weighted base merging separately combines forget and retain updates, with self-normalization preventing the merge from growing proportionally with client count.Global scaling γ controls the forget–retain trade-off.Source: §4.2.2 · p. 4

4.2.3 Retention-constrained Projection

  • For each selected tensor, retention projection finds the closest update to the scaled base merge subject to per-retain-client bounds on a second-order Fisher perturbation proxy.The proxy uses squared update coordinates weighted by each retain client's sparsified Fisher sensitivity, so directional cancellation does not reduce the measure.Source: §4.2.3 · p. 4

4.3 Theoretical Guarantees

  • The per-tensor program has a unique global minimum-distance optimum because its constraints are nonnegatively weighted sums of squares and it satisfies Slater’s condition.The optimum satisfies every retain-client proxy bound; its dual has m variables per tensor, independent of dimension d, while evaluation costs O(md).Source: §4.2.3 · p. 4, §4.3 · p. 5

4.2.3 Retention-constrained Projection

  • The projection shrinks parameters more strongly when retain sensitivity is larger, while leaving insensitive parameters closer to the base merge.The server applies the correction by adjusting coordinates using the retain-client multipliers.Source: §4.2.3 · p. 5

4.3 Theoretical Guarantees

  • For m > 1, the described sequential multiplier increases check feasibility but do not certify minimum-distance optimality for the computed output.The theoretical guarantees characterize the exact per-tensor optimum rather than every numerical solver output.Source: §4.2.3 · p. 5, §4.3 · p. 5

5 Experiments

Experiments compare FIRM with merging baselines across WMDP, TOFU, and MUSE-HP, then test its aggregation and retention-control components. FIRM shows utility gains at comparable forgetting on WMDP and MUSE-HP, while TOFU results vary by metric.

5.1 Setup

  • Experiments use Zephyr-7B on WMDP and TOFU and Llama-3-8B on MUSE-HP, with local adapter training and controlled baseline conditions.WMDP uses six forget clients and one retain client; TOFU uses five forget clients and one retain client.Source: §5.1 · pp. 5–6

5.2 Main Results: WMDP (Zephyr-7B)

  • At comparable WMDP Bio forgetting (≈43%), FIRM retains 48.29% Utility versus FULM’s 45.88%, with similar Cyber accuracy (31.91% versus 31.76%).This is a 2.41 pp Utility improvement at comparable forgetting depth.Source: Table 1 · p. 6, §5.2 · p. 6

5.3 Near-iid Unlearning: TOFU (Zephyr-7B)

  • At comparable forgetting, FIRM at γ = 16.0 lowers forget ROUGE-L (70.00% vs. 71.85%) and raises real-author ROUGE-L by 2.50 points, while FULM leads MMLU and real-world ROUGE-L.FIRM maintains retain ROUGE-L above 94% across evaluated scales; at these settings FULM has higher MMLU (37.35% vs. 36.44%) and real-world ROUGE-L (75.93% vs. 72.76%).Source: §5.3 · pp. 6–7, Table 3 · p. 7

5.4 Robustness: MUSE-HP (Llama-3-8B)

  • At zero forget-QA ROUGE-L, FIRM reaches 58.57% MMLU versus FULM’s 57.52%, a 1.05 pp gain at equal measured forgetting.The comparison is on MUSE-HP with Llama-3-8B; the paper cautions that method-specific γ values do not establish reduced scaling sensitivity.Source: Table 4 · p. 7, §5.4 · p. 7

5.5 Ablation Studies

  • At γ = 1.0, raw-Fisher weighting gives 63.00% Bio accuracy and 54.37% MMLU utility, while uniform weighting gives 24.82% Bio accuracy and 24.54% MMLU utility.These fixed-scale comparisons also alter update calibration, support, or magnitude; the authors therefore caution against attributing the differences to Fisher weighting alone.Source: §5.5 · pp. 7–8
  • Against scale-tuned averaging on shared WMDP checkpoints, FIRM gains 2.756 MMLU points over dense uniform averaging and 0.612 over FULM-MEAN at the reported matches.These comparisons use separate γ values and support the combined FIRM design beyond global rescaling at the evaluated operating points.Source: Table 6 · p. 8, §5.5 · p. 8

5.6 Retention Control and Dual Scalability

  • At selected matched-forgetting WMDP operating points, retention projection improves MMLU by 0.669 or 1.723 points over FIRM-base.With the tighter tolerance, utility gains are 0.249–1.453 points in three comparisons; these are local improvements, not uniform dominance.Source: Table 7 · p. 8, §5.6 · p. 8
  • Multi-client checks report positive multipliers for all retain clients, reduced per-tensor Fisher damage, higher utility, and modest server-side cost.These measurements assess constraint satisfaction and overhead, but do not certify minimum-distance optimality for m > 1.Source: §5.6 · p. 8, Table 8 · p. 9

6 Analysis

FIRM’s base merge normalizes heterogeneous forget-domain updates so their contribution at each coordinate does not grow merely from adding more centroids. In the measured WMDP configuration, merging is faster than FULM, and the retention correction adds little runtime.

How Normalization Controls Update Magnitude

  • FIRM’s coordinate-wise weighted average bounds each role-specific term by the largest contributing update, avoiding norm growth caused solely by summing more centroids.As heterogeneous centroid count increases, FULM’s norm rises more rapidly and utility collapses, while FIRM’s norm stabilizes and utility remains substantially higher.Source: §6 · p. 9, Table 9 · p. 9

Computational Cost

  • In the measured WMDP configuration, server-side merging takes 217 seconds for FIRM versus 547 seconds for FULM, while retention correction adds 1.2 seconds.Fisher estimation takes 15.96 minutes per client as a one-time client-side step.Source: §6 · p. 9

7 Conclusion

FIRM combines local forget and retain updates in a single-round framework without server access to client data. Across two model families, it improves utility at comparable forgetting on WMDP and MUSE-HP, while TOFU shows metric-dependent trade-offs.

7 Conclusion

  • FIRM combines local forget and retain updates without server access to client data, using Fisher-normalized merging and retention control based on a diagonal-Fisher perturbation proxy.Its per-tensor projection has a unique minimum-distance optimum and an m-variable dual.Source: §7 · p. 9
  • Experiments across two model families show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU.Scale-tuned comparisons support benefits of the combined Fisher-based design beyond global rescaling, while matched-forgetting comparisons show additional retention-correction benefits at selected operating points.Source: §7 · p. 9

Limitations

FIRM controls a second-order empirical Fisher proxy rather than exact retain-loss change, and its evaluation and protocol leave several settings for future work.

Limitations

  • FIRM constrains a second-order empirical Fisher proxy rather than exact retain-loss change, and its fidelity depends on how well diagonal empirical Fisher represents local retain-loss curvature.Tighter bounds between the proxy and true loss change remain an important theoretical direction.Source: Limitations · p. 9
  • The evaluation covers two model families across WMDP, TOFU, and MUSE-HP, including cross-model tests on WMDP and MUSE-HP; validation beyond 8B and on additional families remains future work.Broader client and domain heterogeneity also remains future work.Source: Limitations · pp. 9–10
  • The current protocol assigns each client a single role per request, leaving mixed-role clients with both deletion-requested and retained data for future work.Source: Limitations · p. 10
  • The protocol assumes a trusted central server and honest-but-curious clients; robustness to manipulated Fisher statistics, poisoned task vectors, client dropout, and fully decentralized aggregation remains open.Source: Limitations · p. 10

A Notations

A.1 Cross-Model Robustness

Cross-model tests report utility gains at selected forgetting levels for MUSE-HP on Zephyr-7B and WMDP on Llama-3-8B. These comparisons have nonzero matching gaps, and the sweeps evaluate the base merge without projection.

Cross-model robustness caveat

  • The γ sweeps use an inactive QP (ϵ = 999), so they evaluate the combined base merge without projection and do not isolate continuous Fisher weighting.Numerical γ interval widths are not comparable measures of robustness; the evidence comes from measured forgetting–utility pairs.Source: §A.1 · p. 11

MUSE-HP / Zephyr-7B

  • On MUSE-HP/Zephyr-7B, FIRM at γ = 0.30 has 50.98% MMLU versus FULM’s 47.35% at γ = 0.07, with HP ROUGE-L of 0.06% versus 0.81%.At zero HP ROUGE-L, FIRM retains 49.34% MMLU versus FULM’s 35.37%; zero ROUGE-L concerns evaluated answers, not proof of complete knowledge removal.Source: Table 11 · p. 11, §A.1 · p. 11

WMDP on Llama-3-8B

  • On WMDP/Llama-3-8B, FIRM retains 36.58% MMLU versus FULM’s 29.65%, a 6.93 pp utility gain, with weaker Bio forgetting by 2.83 points and stronger Cyber forgetting by 0.65 points.The comparison has nonzero matching gaps rather than equal forgetting.Source: §A.1 · p. 11, Table 12 · p. 12

B Dual Derivation and Numerical Retention Correction

The per-tensor retention projection is a strictly convex program with a unique optimum characterized by its KKT conditions; its dual uses one variable per retain client. The numerical multi-client procedure checks proxy-budget feasibility but does not certify optimality, and implementation details specify Fisher estimation and processing.

B Dual Derivation and Numerical Retention Correction

  • For each tensor, the constrained update is the closest update to the base merge satisfying each retain client's quadratic proxy budget.The summed proxy across T constrained tensors is bounded by Tϵ, not ϵ.Source: B Dual Derivation and Numerical… · p. 12
  • The per-tensor projection has a unique global optimum, and its KKT conditions fully characterize that solution under strong duality.The dual uses one variable per retain client; objective and gradient evaluation cost O(md) for m retain clients and d coordinates.Source: B Dual Derivation and Numerical… · pp. 12–13
  • For one retain client, scalar bisection finds the unique multiplier at the active proxy boundary, recovering the projection to numerical precision.The proxy decreases smoothly and monotonically from its base-merge value toward zero as the multiplier increases.Source: B Dual Derivation and Numerical… · p. 13
  • The numerical routine initializes multipliers at zero and uses bracketing and 60 bisection steps for each violated constraint, while holding other multipliers fixed.It stops when computed damages are within ϵ + τ or when an iteration or multiplier-change limit is reached; stopping at a limit alone does not certify feasibility.Source: B Dual Derivation and Numerical… · p. 13
  • For multiple retain clients, the implemented routine only increases multipliers, so satisfying proxy budgets does not certify minimum-distance optimality or KKT conditions.The reported multi-client results give empirical constraint satisfaction and cost, not a KKT certificate; the exact optimum also does not bound exact retain-loss change.Source: B Dual Derivation and Numerical… · p. 13

C Implementation Details

  • Fisher diagonals are estimated from 20 single-sequence batches, then forget tensors are pooled before normalization and Top-αd sparsification.For TOFU, Fisher estimation uses the original backbone without the expert adapter, although client training starts from the expert-adapted model.Source: C Implementation Details · pp. 13–14

C.1 Efficiency Analysis

FIRM adds client-side Fisher estimation, while its server-side merge and retention projection have comparatively light computational cost. The reported memory and communication figures have specific measurement and payload-estimation boundaries.

C.1 Efficiency Analysis

  • The FIRM merge takes 217 s versus 547 s for FULM, and the constrained dual projection adds 1.2 s.Fisher estimation with 20 batches is the dominant client-side cost, but clients can run it independently and in parallel.Source: §C.1 · p. 14
  • The Fisher pass reaches 38.4 GB peak GPU memory versus 19.7 GB for local adapter training, with both model copies resident during measurement.An isolated Fisher pass was not measured; the server uses transmitted task vectors and Fisher summaries, not client data or full client models.Source: §C.1 · p. 14
  • Each client sends a task vector and raw Fisher summary once, and projection requires no further communication; the profiled task-vector artifact is 336 MiB.The 336 MiB dense Fisher payload is a derived estimate, not measured network traffic; Top-10% sparsification occurs after pooling and does not reduce upload payload.Source: §C.1 · p. 14

D.1 Scale-Tuned Uniform Baseline

On WMDP, the observed-step envelopes favor FIRM over dense uniform merging across the reported common accuracy ranges, though the summaries are grid-dependent. Other controls report a local FULM-MEAN utility gain, no consistent advantage for continuous Fisher weighting alone, and qualified projection comparisons.

D.1 Scale-Tuned Uniform Baseline

  • 1.268 and 2.326 percentage points: range-normalized observed-step areas favor FIRM over uniform for Bio- and Cyber-constrained MMLU envelopes, respectively.The common accuracy ranges are 25.22–61.59% for Bio and 25.97–39.41% for Cyber; these grid-dependent summaries establish neither statistical significance nor uniform dominance.Source: §D.1 · pp. 14–15, Table 14 · p. 15

D.2 FULM with Inter-Cluster Averaging

  • The only base comparison with both Bio and Cyber gaps at most 0.5 percentage points shows a 0.612-point utility gain over FULM-MEAN.This is the γ = 0.35/0.60 pair; for fixed centroids, MEAN at γ equals SUM at γ/3 in real arithmetic.Source: §D.2 · p. 15

D.3 Support- and Norm-Matched Weighting Control

  • At γ = 1.1, Bio accuracy is identical at 42.97%, while Fisher/uniform Cyber accuracy is 32.76/32.56% and MMLU is 49.76/49.94%.Neither method dominates the observed frontiers, and the control does not establish a consistent advantage for continuous Fisher weighting alone.Source: §D.3 · p. 15

D.4 Retention-Tolerance Sensitivity

  • In the fixed-γ tolerance sweep, tighter budgets improve utility while weakening forgetting; its no-QP reference differs from the separately recorded main-table result.The records do not establish the exact source of the discrepancy, and projection effects are compared within the tolerance sweep.Source: §D.4 · p. 15, Table 15 · p. 15

D.5 Matched Projection Comparisons

  • Matched projection comparisons require both Bio/Cyber accuracy gaps to be at most 0.5 points and an active QP/base update, but do not establish uniform dominance.Reference selection uses forgetting gaps, not MMLU; finite ϵ is applied separately to each tensor.Source: §D.5 · pp. 15–16, Table 16 · p. 16
Loading 2610.21720v1…