Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

Motivation

• Proprietary records provide both domain knowledge and private information, while fine-tuning pretrained LLMs offers an alternative to often-impractical training from scratch.Organizations adapt models to domains including legal analysis, financial reporting, and medical question answering.Source: §1 · p. 1
• Fine-tuned models can disclose memorized PII, so organizations must reduce leakage without erasing useful domain knowledge.Scrubbing and differential privacy can compromise utility, while privacy-neuron editing lacks a PII-free reference and training-time defenses are difficult to apply after deployment.Source: §1 · p. 1

Approach

• JEPA-PRUNE trains a public model on surrogate and masked views of the same records, providing a reference that captures context without original PII.The public model uses a JEPA objective, while the private model is fine-tuned on raw records; both use a shared frozen backbone.Source: §1 · p. 2
• A cross-model probe measures representation differences on leaked records, and integrated gradients plus a scale-invariant specificity score identify PII-responsive neurons for pruning.The framework prunes identified neurons without retraining.Source: §1 · p. 2

Results

• Pruning 1.6% of neurons reduces PII inference accuracy by up to 7.4× and extraction recall by up to 13.9×, outperforming random pruning.At a 3.2% budget, JEPA-PRUNE matches data scrubbing with lower perplexity, while DEPN achieves lower leakage at the cost of at least 26× higher unseen-text perplexity.Source: §1 · p. 2

Mitigating PII Leakage

• Data scrubbing can miss entities or remove useful context, while differential privacy may reduce utility and requires training-time intervention (Lukas et al., 2023; Abadi et al., 2016).Post-training unlearning methods include shard retraining and optimization-based approaches; forgotten information can resurface after later model transformations (Jang et al., 2023; Zhang et al., 2024; Hu et al., 2025; Zhang et al., 2025; Łucki et al., 2024).Source: §2 · p. 2

Neuron Localization & Privacy Editing

• Neuron-localization methods can identify unrelated memories, and existing neuron defenses lack a PII-free reference for distinguishing identity-specific representations from useful context (Geva et al., 2021; Dai et al., 2022; Chang et al., 2024; Wu et al., 2023b; Wu et al., 2024).The paper positions JEPA-PRUNE as the first method, to the authors’ knowledge, to use a JEPA-trained PII-free reference for post-hoc pruning without retraining.Source: §2 · pp. 2–3

Reference Models & Public–Private Contrast

• Other public–private contrasts detect memorization, edit task vectors, or add runtime controls, whereas JEPA-PRUNE removes selected neurons after training (Carlini et al., 2022a; Ilharco et al., 2022; Ginart et al., 2022; Vinod et al., 2026; Shaaban and Elmahallawy, 2026).The paper distinguishes its approach from access control, which restricts queries rather than removing memorized data.Source: §2 · p. 3

3.1 Background and Problem Setup

• Fine-tuning minimizes next-token cross-entropy, and a model may memorize PII entities and reproduce their values verbatim.LoRA freezes pretrained weights and learns a low-rank update, allowing adapters to share a backbone while differing in their updates.Source: §3.1 · p. 3
• A neuron is an input coordinate of an attention or feed-forward output projection; privacy-neuron editing ranks neurons by PII-token likelihood and deactivates the highest-scoring ones.Without a PII-free reference, this approach cannot distinguish PII-specific neurons from those encoding context.Source: §3.1 · p. 3
• JEPA uses a predictor to map a context-view embedding toward a related target-view embedding, and JEPA-PRUNE adapts the objective to train a public PII-free reference.The embedding is the final-token hidden state.Source: §3.1 · p. 3

3.2 Threat Model

• The data owner has white-box access to models, records, and PII annotations, while an external adversary has black-box queries and seeks extraction, attribute inference, or membership inference.White-box attacks are out of scope; before deployment, the owner uses inference attacks to identify leaked records for attribution.Source: §3.2 · p. 3

4.1 Private and Public Model Training

• On a shared frozen backbone, the private LoRA adapter learns from raw records while the public adapter uses surrogate-context and masked-target views with language modeling and JEPA.Because the paired views differ at entity spans, JEPA encourages representations that retain record context while reducing dependence on entity identity.Source: §4.1 · pp. 4–5

4.2 Reference-Guided Neuron Attribution

• For attack-successful leaks not ranked correctly by the pretrained base model, JEPA-PRUNE compares private record and entity-type embeddings with a public type-view reference and attributes discrepancy to neurons.The entity-type view replaces each entity with a type placeholder; a control term captures PII-independent differences such as syntax and domain vocabulary.Source: §4.2 · p. 5

4.3 Pruning PII-Responsive Neurons

• A normalized record-versus-control contribution score flags neurons whose positive contribution to reference distance is greater for original records than entity-type views.The score supports scale-invariant joint ranking of attention and feed-forward neurons.Source: §4.3 · pp. 5–6
• Across leaked records, the method selects neurons flagged most frequently, breaks ties by summed flagged scores, and zeroes selected output-projection columns without retraining.The pruned private model replaces the original at deployment; the public reference remains unchanged and is not deployed.Source: §4.3 · p. 6

5.1 Evaluating JEPA-PRUNE vs. Baselines

• JEPA-PRUNE lowers inference accuracy more than random pruning in every setting and budget, while its utility cost varies by model.On Gemma-2-2B and ai4privacy at 0.8%, accuracy falls from 40.86% to 9.30% with unseen PPL× of 0.809; random pruning leaves 31.23%.Source: §5.1 · p. 6, Table 1 · p. 7
• At 1.6%, JEPA-PRUNE reduces extracted names on ai4privacy from 474 to 59 and on ECHR from 126 to 9.Recall falls from 24.01% to 2.99% and from 10.59% to 0.76%, while precision changes little; random pruning leaves 489 and 80 names.Source: §5.1 · p. 6, Table 2 · p. 7
• Across budgets of 0.8%, 1.6%, and 3.2%, JEPA-PRUNE lowers reference-attack AUC from 0.952 to 0.725–0.827, versus 0.873–0.932 for random pruning.The selected neurons also carry much of the record-level membership signal.Source: §5.1 · p. 7
• JEPA-PRUNE reaches similar inference accuracy to a public model while keeping perplexity close to the private model.The public-model alternative has much higher unseen perplexity.Source: §5.1 · p. 7

5.2 Ablation Studies

• At equal budgets, joint ranking achieves the lowest inference accuracy on Llama-3.2-1B and Mistral-7B: 5.32% and 9.63%.It outperforms FFN-only pruning at similar unseen perplexity; neuron-type contributions vary by model.Source: §5.2 · pp. 7–8, Table 3 · p. 8
• Pruned neurons span all layers, but distribution depends on dataset: the first 16 layers contain 56.8% and 73.6% on ai4privacy, while the last 16 contain 77.7% and 67.0% on ECHR.The dataset-dependent pattern appears in both model families, and attention accounts for a larger share on ai4privacy than ECHR.Source: §5.2 · p. 8, Figure 2 · p. 8
• Additional ablations find most leakage reduction in the first 0.08% of neurons, with budgets above 0.61% mainly harming utility.Realistic surrogates increase public-model leakage without improving pruning; entity-type views match masked-view leakage reduction with lower perplexity.Source: §5.2 · p. 8

• JEPA-PRUNE prunes neurons that distinguish leaked-record representations from those of a public model trained on PII-free views.It reduces inference, extraction, and membership-inference attacks more effectively than random pruning while largely preserving utility without retraining.Source: §6 · p. 8

• Entity types missed by the named-entity recognizer reach the public model, and person-name-based pruning leaves other PII leakage largely unchanged.Protecting other PII types requires leaked records containing those types; pruned content may also resurface after further fine-tuning or quantization.Source: §7 · p. 9

A.1 Full Results

• At 0.8%, extracted names are 68 on ai4privacy and 20 on ECHR, rising to 29 and 5 at 3.2%; random pruning extracts 361–489 and 80–91 at every budget.Source: §A.1 · p. 12

A.2 Additional Models

• At 1.6%, JEPA-PRUNE lowers Mistral-7B inference accuracy from 49.17% to 6.64% on ai4privacy and from 59.47% to 33.89% on ECHR.It also lowers Llama-3.1-8B accuracy from 12.29% to 5.98% and from 36.21% to 28.24%; DEPN lowers accuracy further only with unseen PPL× of at least 26.0.Source: §A.2 · p. 12

A.3 Membership Inference

• Across budgets of 0.8%, 1.6%, and 3.2%, JEPA-PRUNE lowers LOSS-attack AUC to 0.649–0.737 and reference-attack AUC to 0.725–0.827.Random pruning reaches 0.786–0.867 and 0.873–0.932, respectively.Source: §A.3 · p. 12

A.4 Pruning Budget

• On Llama-3.2-1B, pruning 125 to 1,000 neurons lowers inference accuracy from 11.96% to 6.98–5.32%, with the largest drop from the first 125 neurons.Unseen PPL× stays at or below 1.099 over this range, while larger budgets sharply degrade utility for little further accuracy reduction.Source: §A.4 · p. 12

A.5 Leaked-Record Selection

• All three record-selection rules lower inference accuracy from 11.96% to 4.98–5.65% at similar unseen PPL× costs of 1.064–1.099.Inference-selected records are disjoint from evaluation records by construction; 71 extraction-guided and 15 DEPN-style records overlap evaluation.Source: §A.5 · pp. 12–13

A.6 Leaked-Record Count

• At 1.6%, 50 leaked records bring accuracy close to the 100-record result, while unseen PPL× remains 0.853–0.902 across record counts.At 3.2%, 10 and 25 records leave accuracy at 9.30% and 8.97%, versus 4.98% with 100; additional records bring diminishing returns.Source: §A.6 · p. 13
• Even 10 leaked records lower accuracy from 40.86% to 12.62% at 1.6% and 9.30% at 3.2%.The results are for Gemma-2-2B; the 153 records comprise all records found by the inference attack.Source: §A.6 · p. 13

A.7 Surrogate Generator

• Pseudo-word surrogates keep the public reference closest to PII-free, while realistic Faker and Hybrid surrogates raise its inference accuracy without improving pruning.The public model’s accuracy is 1.33% with pseudo-words, 2.99% with Faker, and 4.32% with Hybrid; pruned-model accuracy is 5.32–6.31% at similar perplexity cost.Source: §A.7 · pp. 13–14

A.8 LoRA Rank

• For LoRA ranks r ≥8, pruning reduces inference accuracy by 56–64% at unseen PPL× of 1.039–1.099; the reduction rises from 25% at r = 4 to 64% at r = 32.At r = 4, unpruned accuracy is 6.64% and falls to 4.98%; at r = 32, seen PPL× reaches 2.037.Source: §A.8 · p. 14

A.9 Candidate Count

• With 50, 100, and 500 candidates, JEPA-PRUNE reduces accuracy by 56%, 45%, and 67%, respectively, removing about half or more of leakage at each count.The unpruned model’s accuracy falls from 11.96% to 1.99% as candidate count grows from 50 to 500.Source: §A.9 · p. 14

A.10 Other Entity Types

• Pruning guided by person-name leaks leaves organization accuracy nearly unchanged and lowers location accuracy only slightly; the authors conclude other entity types require leaks of their own.The reported organization accuracies are 33.89% and 33.55%, and location accuracy falls from 15.47% to 13.26%.Source: §A.10 · p. 14

A.11 JEPA versus Scrubbed Reference

• At every budget, the JEPA reference leaves lower inference accuracy than the scrubbed reference, while its non-PII PPL× stays below the unpruned model’s.The JEPA reference reaches 8.64% at 1.6%, which the scrubbed reference reaches only at 3.2%; the authors conclude JEPA makes attribution more efficient.Source: §A.11 · pp. 14–15

A.12 Entity-Type versus Masked View

• Using the same entity-type view for the reference embedding and control term yields similar accuracy reductions at the lowest unseen PPL× cost among the tested variants.The entity-type view preserves entity types while replacing PII values; the authors conclude matching the views helps target identities rather than general entity-type knowledge.Source: §A.12 · pp. 15–16, Table 16 · p. 18

A.13 Pruning versus Public Models

• At 3.2%, JEPA-PRUNE reaches 4.98% inference accuracy with unseen all-token and non-PII PPL× of 1.038 and 0.824, respectively, compared with higher unseen PPL× for public models.The two public models have unseen PPL× of 1.617 and 2.200; JEPA-PRUNE also has the lowest membership-inference AUC among the compared options.Source: §A.13 · p. 16, Table 17 · p. 19

B.1 Reproduction Hyperparameters

• Private and public adapters use the same records and optimization settings, with LoRA rank r = 8 and 10 training epochs.The base model is 4-bit NF4-quantized; the appendix gives a batch size of 32 and learning rate of 5 × 10^-5, reduced for Mistral-7B and Llama-3.1-8B.Source: §B.1 · p. 16

B.2 DEPN Adaptation

• The DEPN adaptation attributes negative log-likelihood of PII tokens given preceding text and scales activations at all positions; it uses the same leaked records as JEPA-PRUNE.DEPN ranks feed-forward neurons using integrated gradients and may select fewer than the pruning budget’s k neurons.Source: §B.2 · p. 17

B.3 Attack Settings

• The extraction attack counts generated PERSON mentions absent from pretrained-model samples, while inference ranks 50 candidate names by perplexity on completed records.Membership inference uses 1,000 training members and 1,000 test non-members, with loss-based and reference-based attack scores.Source: §B.3 · p. 17

B.4 Surrogate Generators

• The surrogate ablation compares pseudo-words, Faker values, and a Hybrid generator; after 50 redraws, some Faker and Hybrid values still share words with training PII, while no pseudo-word does.Repeated mentions within a record receive the same surrogate, and each record uses its own seeded generator.Source: §B.4 · p. 18

B.5 Leaked-Record Selection Rules

• Inference-selected records are the only selection rule disjoint from evaluation records by construction; 71 extraction-guided and 15 DEPN-style records overlap evaluation.Inference selection counts a record as leaked when the private model ranks the true masked name first among 50 candidates and the pretrained base model does not.Source: §B.5 · p. 19

B.6 Evaluation Metrics

• Privacy metrics include extracted PII, extraction precision and recall, inference accuracy, and membership-inference AUC and TPR@1%FPR.Inference accuracy is the fraction of attacked records where the true name is the lowest-perplexity candidate among 50.Source: §B.6 · p. 19
• Perplexity is exponentiated mean negative log-likelihood; non-PII PPL excludes entity tokens, and PPL× is the post-pruning to pre-pruning ratio.Perplexity is computed on test records and, in ablations, also on training records.Source: §B.6 · pp. 19–20

B.7 Computing Environment; B.8 Implementation Details

• The appendix reports experiments on NVIDIA H100 PCIe GPUs and accesses neurons through hooks on layer output projections; pruning zeroes neuron inputs at every position.For the private model, this is equivalent to zeroing the corresponding column of the merged projection.Source: §B.7 · p. 20, §B.8 · p. 20

C.1 Feed-Forward and Attention Neurons

• A feed-forward neuron is a coordinate of the activation vector, and zeroing it corresponds to zeroing its output-projection column.The appendix defines attention neurons as coordinates of the concatenated attention-head output, indexed jointly with feed-forward neurons.Source: §C.1 · p. 20

C.2 LLM-JEPA Objective

• LLM-JEPA adds an embedding-space term that predicts a target-view representation from a related context-view representation.A predictor maps between the views, and λ balances the embedding-space and language-modeling terms.Source: §C.2 · p. 20