PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning New Facts with QLoRA: An Acquisition-Retention Frontier

Estelle Zheng, Sébastien Warichet, Emmanuel Helbert, Christophe Cerisara

arXiv:2608.25677v1cs.CLcs.AIcs.LG

TL;DR

The paper asks whether parameter-efficient fine-tuning reliably preserves pretrained capabilities when adapting models to genuinely new facts. Using a controlled anonymized OpenStreetMap benchmark, it compares FFT with rank-varied QLoRA and finds that adapter rank governs an acquisition–retention frontier, with higher acquisition accompanied by greater disruption of pretrained behavior.

  • Problem

    Parameter-efficient fine-tuning is assumed to preserve pretrained capabilities, but its capacity may limit acquisition of genuinely new factual associations.

  • Method

    The study compares FFT with QLoRA ranks r ∈{8, 16, 32, 64} on anonymized OpenStreetMap geographic associations, measuring acquisition, paraphrase generalization, OOD retention, and model drift.

  • Results

    QLoRA rank creates a clear acquisition–retention frontier: higher ranks improve factual acquisition and paraphrase generalization but increasingly degrade OOD performance, while FFT remains conservative.

  • Takeaways & Limitations

    PEFT is not inherently safe for knowledge injection because adapter rank controls both newly installed factual knowledge and disturbance of pretrained behavior.

  • Takeaways & Limitations

    The benchmark uses 1,938 examples from 14 small cities and anonymized synthetic identifiers, so generalization to larger, diverse, and realistic knowledge-injection settings remains unclear.

Abstract

from arXiv · show

Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends strongly on adapter capacity. We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must acquire anonymized geographic associations while retaining unrelated capabilities. Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition--retention frontier. Low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at an increasing cost in performance on unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime. Distributional, weight-space, and spectral diagnostics mirror this behavioral trade-off, with higher-rank QLoRA moving farther from the pretrained model. A separate math adaptation experiment shows a weaker frontier, suggesting that the effect is most pronounced when adaptation must install new factual associations rather than reinforce skills already supported by pretraining. Code and data are available at https://github.com/zhngstl/new_facts_forgetting.

1 Introduction

The paper examines whether parameter-efficient fine-tuning preserves pretrained capabilities when models must acquire genuinely new factual associations. It introduces a controlled benchmark and shows that adapter rank creates an acquisition–retention frontier.

  • 1 Introduction: Parameter-efficient fine-tuning can preserve prior capabilities but may limit what models acquire.LoRA and QLoRA freeze pretrained weights while learning low-rank updates.
  • 1 Introduction: The study isolates novel factual acquisition from broader adaptation by using a single controlled batch of anonymized geographic associations.This differs from benchmarks focused on editing known facts or sequential continual learning.
  • 1 Introduction: The OpenStreetMap-derived benchmark compares FFT with QLoRA ranks r ∈{8, 16, 32, 64} across memorization, paraphrase generalization, and OOD retention.The study also measures KL, dense-update norms, and SVD-based spectral changes.
  • 1 Introduction: QLoRA rank induces an acquisition–retention frontier: higher ranks improve factual acquisition but increasingly degrade unrelated capabilities.Lower ranks preserve OOD performance while acquiring fewer facts.
  • 1 Introduction: FFT retains general capabilities relatively well but does not reach the highest factual-acquisition regime observed with higher-rank LoRA.Higher-acquisition LoRA runs also move farther from the pretrained model in drift diagnostics.

2 Methodology

The methodology constructs a controlled, anonymized OpenStreetMap benchmark to test acquisition of novel geographic associations while limiting pretrained world-knowledge shortcuts. Base-model diagnostics assess prior knowledge and answer-type biases before adaptation.

  • 2 Methodology: The benchmark uses a naturally coherent OpenStreetMap graph rather than researcher-designed synthetic topology.Its facts link POIs, roads, and cities across five relation types.
  • 2 Methodology: The training split contains 1,938 instruction-style examples, while evaluation uses 900 held-out examples with disjoint surface templates.This tests factual-association acquisition and generalization across phrasing variations.
  • 2 Methodology: Synthetic identifiers and small source cities reduce direct reliance on pretrained world knowledge.Examples use anonymized names such as C-TRAIN-001 and POI-TRAIN-000001.
  • 2 Methodology: The prior diagnostic compares anonymized and real-entity versions using exact-match generation and teacher-forced gold-versus-distractor preferences.It reports gold-preferred pairs and mean log-probability margins.
  • 2 Methodology: Anonymization sharply reduces exact match and answer-likelihood margins, while slightly-above-chance preferences indicate weak structural or answer-type biases.Higher paraphrase EM partly reflects a larger share of yes/no questions and response-format effects.

3 Experimental Setup

The experiments compare FFT and rank-varied QLoRA, with an additional unquantized LoRA control, and evaluate factual acquisition, OOD retention, and model drift. These axes distinguish learned factual associations from preservation of general capabilities.

  • 3 Experimental Setup: The main experiments use Qwen3-4B and compare FFT with QLoRA ranks r ∈{8, 16, 32, 64}.All runs use five random seeds and apply autoregressive loss only to answer tokens.
  • 3 Experimental Setup: A standard-LoRA sweep on Qwen3-1.7B at ranks r ∈{8, 16, 32} tests whether rank trends persist without quantization.The control changes model scale as well as quantization.
  • 3 Experimental Setup: Factual acquisition is measured through training EM and paraphrase EM on same-fact examples with held-out templates.Paraphrase accuracy tests robustness to phrasing variation, not unseen OSM knowledge.
  • 3 Experimental Setup: OOD retention is evaluated on HumanEval, IFEval, TruthfulQA, MMLU-Redux-2.0, and BBH.These cover code generation, instruction following, truthfulness, general knowledge, and reasoning.
  • 3 Experimental Setup: Model drift is assessed with KL divergence, teacher-forced negative log-likelihood, RMS-normalized dense weight drift, and SVD-based intruder dimensions.These diagnostics capture movement from the pretrained model beyond behavioral accuracy alone.

4 Results

QLoRA rank controls a clear acquisition–retention trade-off: higher ranks improve OSM paraphrase acquisition but reduce OOD retention, while model drift increases alongside forgetting. The weaker math frontier indicates this pattern is most pronounced for novel factual associations.

  • 4.1 QLoRA rank controls the acquisition–retention trade-off: Higher QLoRA ranks improve OSM same-fact paraphrase accuracy but move the model toward lower OOD retention.Figure 1 reports final-checkpoint means with standard deviations over five seeds.
  • 4.2 Higher acquisition requires greater adaptation capacity: Target-acquisition comparisons select the evaluated checkpoint nearest each target paraphrase accuracy for every method and seed.OOD retention is reported relative to base-model OOD performance.
  • 4.2 Higher acquisition requires greater adaptation capacity: FFT and QLoRA r = 8 retain OOD performance well but do not reach the highest paraphrase accuracy.
  • 4.3 Model drift is associated with forgetting: Higher-rank QLoRA checkpoints show larger KL divergence, effective dense updates, and spectral shifts from the pretrained model.The strongest forgetting regime, QLoRA r = 64, has the largest SVD intruder excess.
  • 4.3 Model drift is associated with forgetting: The behavioral and diagnostic results jointly associate stronger OSM acquisition with larger distributional and weight-space shifts.Higher-acquisition LoRA runs move farther from the pretrained model.
  • 4.3 Model drift is associated with forgetting: Math adaptation shows nearly identical average performance for FFT and QLoRA, with averages of 42.50, 42.60, and 42.03 for FFT, r = 16, and r = 32.The corresponding OOD drops are 1.71, 2.23, and 1.58 points, respectively.

5 Conclusion

The paper concludes that QLoRA rank governs a plasticity trade-off in factual acquisition: greater acquisition comes with more disturbance to pretrained behavior. This conclusion is bounded by the weaker math frontier and the study’s novel-association setting.

  • 5 Conclusion: QLoRA rank controls an acquisition–retention trade-off, with low ranks preserving general capabilities and higher ranks improving paraphrase generalization at increasing OOD cost.
  • 5 Conclusion: Higher-rank QLoRA produces larger KL divergence, effective dense updates, and SVD intruder effects.
  • 5 Conclusion: The weaker math frontier limits the conclusion to adaptation that installs new factual associations rather than fine-tuning in general.

Limitations

The paper’s conclusions are bounded by the benchmark’s limited factual diversity, model coverage, OOD evaluation, adaptation-method comparisons, and narrow math ablation.

  • Benchmark scope: The OSM benchmark contains 1,938 examples from 14 small cities, so generalization to larger or more diverse factual corpora remains unclear.Anonymized synthetic identifiers also may not reflect real-world knowledge injection, and the benchmark does not fully separate entity association from abstract relation learning.
  • Model coverage: The main experiments use Qwen3-4B, while the standard-LoRA control uses Qwen3-1.7B, leaving persistence of rank effects at scale unresolved.Different model sizes, pretraining mixtures, and architectures may exhibit different frontier shapes.
  • OOD benchmark coverage: Retention is measured on five OOD benchmarks, so effects on long-context reasoning, multilingual tasks, and other capabilities are not assessed.The selected suite is described as reasonable but not exhaustive.
  • Adaptation-method coverage: The standard-LoRA control supports a within-adapter rank effect but does not isolate every method-level difference because it changes model scale and lacks matched baselines.The main QLoRA–FFT comparison also differs in quantization and optimization.
  • Math experiment scope: The math comparison tests only QLoRA ranks 16 and 32 for one epoch, so its similarity between FFT and QLoRA rests on a narrow hyperparameter sweep.A fuller rank ablation analogous to the OSM experiments would strengthen this conclusion.

Ethical Considerations

The benchmark uses anonymized, city-level OpenStreetMap records to study factual associations while limiting direct exposure of real-world entities and user-level information.

  • Data and licensing: The dataset uses public OpenStreetMap records and geometries under the ODbL 1.0 license, rather than rendered map tiles.The records describe real places, so anonymization does not guarantee protection against geographic reidentification.
  • Dataset construction: The dataset contains 1,938 instruction examples from 14 city-level extracts, covering POI category, city membership, nearest-road and nearest-POI relations, and road-length buckets.These relations are derived from named points of interest and roads with locally unique names.
  • Task design: Examples include direct queries, same-fact paraphrases, locality probes, spatial-compositional questions, and inverse city-signature examples.Fixed seeds and relation-balanced quotas reduce dominance by common POI categories.
  • Evaluation design: The held-out paraphrase set uses disjoint lookup, slot-query, and predicate templates to test robustness to phrasing variation rather than exact prompt memorization.The set contains 900 examples constructed from facts represented in training.
  • Anonymization: Cities, POIs, and roads are replaced with synthetic identifiers, and instances contain no source coordinates or user-level data.Smaller cities with populations between 5,000 and 80,000 further reduce likely reliance on prior knowledge.
  • Evaluation caveat: The paraphrase split has a higher yes/no-question proportion than training, raising approximate chance EM from 6.68% to 10.36%.Yes/no questions comprise 13.3% of paraphrases versus 6.2% of training examples.

B Per-benchmark OOD Results at Final Checkpoints

At final checkpoints, average OOD degradation is distributed across several benchmarks rather than driven by one benchmark alone.

  • Final-checkpoint OOD results: HumanEval, IFEval, MMLU-Redux, and BBH decline with increasing QLoRA rank, whereas TruthfulQA remains comparatively stable.The benchmark-level pattern complements the aggregate OOD results.

C Details on metrics

The paper measures behavioral and parameter-space drift using symmetric KL, dense RMS updates, SVD intruder dimensions, and answer-likelihood diagnostics. These metrics quantify how adapted checkpoints differ from the pretrained model and how strongly they prefer gold answers over distractors.

  • Symmetric KL: Symmetric KL compares pretrained and adapted next-token distributions under teacher forcing, excluding padding and averaging over non-padding tokens and batches.It symmetrizes both directional KL terms and emphasizes changes in high-probability regions.
  • Dense RMS drift: Dense RMS drift measures the effective dense weight update, using Wθ − W0 for FFT and the merged LoRA update for QLoRA.For QLoRA, the update depends on factors A and B, rank r, and scaling α.
  • SVD intruder dimensions: The SVD diagnostic compares top adapted singular vectors with top pretrained singular vectors and reports intruder rates at ϵ = 0.8, with excess plotted relative to FFT.The implementation uses k = 10 adapted vectors and K = 64 pretrained vectors.
  • Answer likelihood: Answer log-probability scores only answer continuation tokens under teacher forcing, while the distractor margin compares gold and sampled distractor likelihoods.A positive margin means the model assigns higher teacher-forced likelihood to the gold answer.

D Additional dataset examples

The additional examples instantiate anonymized OpenStreetMap relations in direct, paraphrased, graph, multiple-choice, and inverse-signature formats. Held-out validation changes surface templates while preserving the underlying factual relations.

  • Training examples: Training examples cover atomic category, nearest-POI, road-length, spatial multiple-choice, and inverse-city-signature queries.Examples link anonymized POIs, roads, and cities through concrete relation-answer pairs.
  • Held-out validation: Held-out paraphrase validation uses disjoint surface templates, including slot-style category queries, to test generalization across wording.The validation example asks for a place type from a city-key-slot format.
  • Training examples: Additional training formats include nearest-road lookup, road-graph predicates, paraphrased road-length queries, and validation multiple-choice questions.These examples require outputs such as road aliases, yes/no predicates, length buckets, and matching POI choices.

E.1 OpenStreetMap task

The OpenStreetMap experiments tune learning rates separately for QLoRA and FFT, selecting settings by training loss. The appendix tables document the hyperparameters for the main OSM and additional math experiments.

  • Learning-rate selection: QLoRA uses a learning-rate sweep over {2×10^-5, 5×10^-5, 2×10^-4}, while FFT uses {2 × 10^-5, 2 × 10^-4}.The best learning rate for each method is selected using the lowest training loss.
  • Experiment documentation: The main OSM experiments are documented in Table 6, while the additional math adaptation experiments are documented in Table 7.The appendix separates hyperparameter reporting for the two experimental settings.
Loading 2608.25677v1…