PaperlayerView exact revision on arXiv

Source-linked AI summary

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA

Fei Wang, Baochun Li

arXiv:2506.20856v1cs.LGcs.CLcs.CR

TL;DR

Fine-tuning on limited data can leave LLMs vulnerable to extraction, while LoRA’s memorization risks have been less explored. The paper compares fine-tuning strategies using similarity-based evaluation and finds that LoRA reduces plagiarism-based memorization relative to full fine-tuning while retaining strong task performance.

  • Problem

    Fine-tuning datasets may contain private or proprietary material, but the extraction risks of fine-tuning—particularly LoRA—have been less systematically examined than pre-training memorization.

    Source: §1 · p. 1

  • Method

    The study evaluates LoRA, full, and head-only fine-tuning across model scales and duplication levels using plagiarism- and similarity-based memorization evaluation.

    Source: §1 · p. 1, §2.2 · p. 2, §4.2 · p. 4

  • Results

    LoRA achieves near-zero plagiarism-based memorization and significantly reduces memorization risks relative to full fine-tuning while maintaining strong task performance.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    LoRA is a computationally efficient alternative to full fine-tuning and a potential privacy-preserving approach within the evaluated settings.

    Source: §1 · p. 2, §6 · p. 8

    Whether advanced data extraction attacks could make fine-tuning data more extractable remains unclear.

    Source: §7 · p. 9

Abstract

from arXiv · show

Memorization in large language models (LLMs) makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tuning, a widely adopted parameter-efficient method. In this work, we re-examine memorization in fine-tuning and uncover a surprising divergence from prior findings across different fine-tuning strategies. Factors such as model scale and data duplication, which strongly influence memorization in pre-training and full fine-tuning, do not follow the same trend in LoRA fine-tuning. Using a more relaxed similarity-based memorization metric, we demonstrate that LoRA significantly reduces memorization risks compared to full fine-tuning, while still maintaining strong task performance.

1 Introduction

Fine-tuning on limited or private data raises extraction concerns, but its memorization risks—especially for LoRA—have been less systematically studied. This work compares fine-tuning strategies with similarity-based evaluation and finds that LoRA reduces memorization while retaining strong task performance.

1 Introduction

  • Fine-tuning on private or proprietary data raises concerns that training sequences could be exposed through data extraction attacks.Examples include personal medical records, internal company documents, and domain-specific knowledge bases.Source: §1 · p. 1
  • Prior fine-tuning studies examined full, head-only, and bottleneck adapter methods, but their metrics may under-detect memorized data.A separate task-level study found higher memorization for summarization and dialogue than for classification and translation.Source: §1 · p. 1
  • This work re-examines full and head-only fine-tuning and adds LoRA, whose memorization and extraction risks were largely unexplored.It first uses plagiarism-based evaluation, then compares methods across scales, duplication levels, and hyperparameter configurations using similarity-based metrics.Source: §1 · pp. 1–2
  • Across GPT-2 and Llama 3 models, LoRA significantly reduces memorization relative to full fine-tuning, with near-zero plagiarism-based memorization and strong task performance.The authors also report similarity scores below extraction thresholds and describe LoRA as a potential privacy-preserving approach.Source: §1 · p. 2

2 Preliminaries

The paper defines fine-tuning memorization in terms of extractability from the fine-tuning data and outlines the extraction setup. It also distinguishes full, head-only, and LoRA training, with LoRA updating low-rank matrices while freezing pretrained weights.

2.1 Memorization in Pre-Training vs. Fine-Tuning

  • In the extraction setup, a generated sequence counts as memorized when it is extractable and appears in the fine-tuning data.The paper notes that differences in memorization and extractability criteria can lead to different assessments of risk.Source: §2.1 · p. 2

2.2 Model Fine-Tuning Methods

  • Full fine-tuning updates all parameters, head-only tuning updates the final projection layer, and LoRA adds trainable low-rank matrices to attention layers while freezing the rest.For a pretrained weight matrix, LoRA uses W = W0 + α·BA; only A and B are updated, reducing trainable parameters from d × k to r × (d + k).Source: §2.2 · p. 2

3 Related Work

Prior work on fine-tuning memorization is limited and has used different evaluation approaches. The reviewed studies report different risks across fine-tuning strategies and task types, with limitations in metrics and model coverage.

3 Related Work

  • Mireshghallah et al. (2022b) compared full-model, head-only, and bottleneck adapter tuning, reporting the highest memorization for head-only tuning using MIA recall and exposure.The paper notes that MIA recall can have a high false-negative rate and that these metrics may underestimate memorization.Source: §3 · p. 3
  • Zeng et al. (2024) found higher memorization for summarization and dialogue than for classification, reading comprehension, and translation, and reported that larger models amplify memorization in susceptible tasks.Their models were mainly BART and T5, with GPT-Neo-125M also evaluated; the authors characterize these as limited in relevance to modern large-scale models.Source: §3 · p. 3

4 A First Look at Memorization of LLM Fine-Tuning

This section tests plagiarism-based memorization across fine-tuning methods, model scales, and data duplication levels. It reports near-zero plagiarism for head-only and LoRA tuning even where full fine-tuning exhibits higher plagiarism, while both parameter-efficient methods retain competitive task performance.

4.1 A Tentative Metric for Fine-Tuning Memorization

  • The section motivates a more permissive memorization criterion because exact-wording definitions may miss semantic memorization, and describes a retrieval-and-alignment plagiarism detector classifying verbatim, paraphrase, and idea plagiarism.The preliminary evaluation follows the detector’s original setup and default hyperparameters.Source: §4.1 · p. 3

4.2 Minimal Plagiarism-Based Memorization in Parameter-Efficient Fine-Tuning

  • The study asks whether model scale and duplication amplify memorization similarly across fine-tuning strategies, as prior work found for training more broadly.The reported increase with scale persists in full fine-tuning, unlike the near-zero rates for head-only and LoRA tuning.Source: §4.2 · pp. 3–4
  • The experiments use GPT-2 models from 124M to 1.5B parameters and Arxiver papers published after GPT-2 training, with ρ controlling repeated samples at fixed dataset size.Models were fine-tuned for 10 epochs; the extraction evaluation generated 1,000 samples per model, and ROUGE-L approximated task utility.Source: §4.2 · p. 4
  • Head-only and LoRA tuning maintain near-zero plagiarism across model sizes and duplication levels, while full fine-tuning plagiarism rises with scale and exceeds 50% for paraphrase at ρ = 20.Figure 1 presents verbatim, idea, and paraphrase plagiarism across methods, varying model size at ρ = 10 and duplication levels for GPT-2 Small.Source: §4.2 · p. 4, Figure 1 · p. 4
  • Full fine-tuning achieves the highest ROUGE-L scores but also the highest plagiarism rates; head-only and LoRA tuning offer competitive ROUGE-L with substantially less plagiarism.LoRA incurs minimal performance loss relative to full fine-tuning; Figure 2 reports ROUGE-L across model sizes and duplication levels.Source: §4.2 · p. 4, Figure 2 · p. 4
  • Under plagiarism-based evaluation, head-only and LoRA tuning show zero detected plagiarism despite prior reports of greater head-only memorization under MIA recall and exposure metrics.The authors contrast this result with earlier findings and note that the methods freeze most model parameters.Source: §4.2 · pp. 4–5

5 Expanding the Lens: A More Permissive View of Memorization in Fine-Tuning

This section extends plagiarism-based detection with sentence-level semantic similarity to capture outputs that resemble training data without direct extraction. Across model scales, duplication levels, and ChatDoctor, LoRA generally stays below the 0.8 threshold while retaining utility; its hyperparameters shape the utility–similarity trade-off.

5. Expanding the Lens: A More Permissive View of Memorization in Fine-Tuning

  • Sentence similarity complements plagiarism-based detection by measuring how closely generated outputs semantically resemble training samples, including when they are not directly extractable.The analysis uses cosine similarity to each generated sequence’s closest training sample; scores above 0.8 count as close alignment.Source: §5 · p. 5, §5.1 · p. 5

5.1 What A Looser Metric Reveals About Fine-Tuning

  • Across model sizes, full fine-tuning shifts similarity scores rightward with more samples above 0.8, whereas head-only and LoRA remain relatively stable with few above threshold.Under duplication, full fine-tuning develops near-perfect similarities when ρ ≥5; at ρ = 20, most head-only and LoRA samples remain below 0.8.Source: Figure 3 · p. 5, §5.1 · pp. 5–6, Figure 4 · p. 6

5.2 LoRA Mitigates Memorization Even in High-Memorization Tasks

  • On ChatDoctor, full fine-tuning achieves the highest ROUGE content relevance, while LoRA keeps similarity below 0.8 and retains competitive utility with minimal extractability.The 10k full-fine-tuning samples have a median similarity around 0.7; LoRA maintains a broader distribution with its highest score below 0.8.Source: Figure 5 · p. 6, §5.2 · p. 7, Table 1 · p. 7

5.3 Impact of LoRA Fine-Tuning Hyperparameters on Memorization

  • All tested LoRA settings improve utility and lower median similarity versus the pretrained model, but increasing α alone can degrade utility and yield samples above 0.8; dropout further mitigates similarity with little utility loss.The authors recommend moderate α with moderate dropout to balance utility and memorization mitigation.Source: §5.3 · pp. 7–8, Figure 6 · p. 8
  • On Llama 3.1 8B Instruct, LoRA’s mitigation remains pronounced, with fewer than 5 generated samples exceeding the 0.8 threshold.The larger model consistently achieves higher scores than the 1B model after fine-tuning.Source: Figure 8 · p. 8, §5.3 · p. 8

6 Concluding Remarks

LoRA fine-tuning achieves performance comparable to full fine-tuning while reducing memorization risks across the evaluated settings.

6 Concluding Remarks

  • LoRA achieves performance comparable to full fine-tuning while reducing memorization risks, with near-zero plagiarism-based and effectively prevented strict verbatim memorization.Across model sizes, data duplication levels, and hyperparameter configurations, it consistently yields lower maximum similarity scores.Source: §6 · p. 8
  • Across model sizes, data duplication levels, and hyperparameter configurations, LoRA consistently produces lower maximum similarity scores.Source: §6 · p. 8
  • The findings contrast with prior work and present LoRA as both a computationally efficient alternative and a potential privacy-preserving approach.Source: §6 · p. 8

7 Limitations

The authors identify a need for stronger theoretical explanation and further testing against advanced data-extraction attacks.

7 Limitations

  • A more rigorous theoretical analysis is needed to explain why LoRA stays below the threshold of extractable memorization, including across rank, scaling-factor, and dropout settings.Source: §7 · p. 9
  • Whether advanced data-extraction attacks could enhance fine-tuning-data extractability remains unclear, motivating investigation of their interaction with LoRA.The authors note that further study is needed to determine whether additional safeguards are needed.Source: §7 · p. 9

A Additional Experiments

The GPT-2 Arxiver experiments show that maximum similarity stays below the extraction threshold as hyperparameters trade off content alignment and memorization risk.

A Additional Experiments

  • ROUGE-L reaches 0.5921 with r = 32, α = 32, and dropout = 0.05, while lower rank and scaling factors generally have lower mean similarity scores.Higher ranks and scaling factors tend to have slightly higher maximum similarity values, still well below extraction thresholds.Source: A Additional Experiments · p. 10
  • Increasing rank and scaling factor enhances content alignment but slightly increases memorization risk, while dropout reduces extreme similarities and maintains task performance.Source: A Additional Experiments · p. 10
  • All maximum similarity scores remain below 0.8, supporting the conclusion that LoRA minimizes extractable memorization compared with full fine-tuning.Source: A Additional Experiments · p. 10

B Computational Resources

The appendix reports separate compute environments for GPT-2 and Llama experiments and identifies the scope of the GPT-2 hyperparameter-results table.

B Computational Resources

Loading 2506.20856v1…