PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

arXiv:2604.10079v4cs.CL

TL;DR

Supervised fine-tuning can leave some of its own training examples unlearned, while existing approaches offer limited tools for diagnosing failures at the sample or pattern level. This paper categorizes their causes and evaluates targeted diagnostic and mitigation strategies, finding that incomplete learning is widespread and heterogeneous and can persist despite aggregate improvements.

  • Problem

    After SFT convergence, models may still fail on supervised examples, while existing approaches provide limited tools to diagnose which examples remain unlearned and why.

    Source: §1 · pp. 1–2

  • Method

    The paper identifies five recurring causes of incomplete learning and uses observable training and inference signals to diagnose samples and examine targeted mitigation strategies.

    Source: §1 · p. 2

  • Results

    Incomplete learning is prevalent and heterogeneous across evaluated models and tasks; no single intervention resolves all failures, and aggregate improvements can mask unlearned subsets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support shifting SFT evaluation toward fine-grained diagnosis of which supervised knowledge remains unlearned and why.

    Source: §1 · p. 2

    Conflict detection depends on high-quality annotations and reliable external tools, while noisy or inconsistent data may reduce its accuracy and leave conflicts only partly resolved.

    Source: §6 · p. 9

Abstract

from arXiv · show

Supervised Fine-Tuning (SFT) is the standard approach for adapting large language models (LLMs) to downstream tasks. However, we observe a persistent failure mode: even after convergence, models often fail to correctly reproduce a subset of their own supervised training data. We refer to this behavior as the Incomplete Learning Phenomenon(ILP). This paper presents the first systematic study of ILP in LLM fine-tuning. We formalize ILP as post-training failure to internalize supervised instances and demonstrate its prevalence across multiple model families, domains, and datasets. Through controlled analyses, we identify five recurrent sources of incomplete learning: (1) missing prerequisite knowledge in the pre-trained model, (2) conflicts between SFT supervision and pre-training knowledge, (3) internal inconsistencies within SFT data, (4) left-side forgetting during sequential fine-tuning, and (5) insufficient optimization for rare or complex patterns. We introduce a diagnostic-first framework that maps unlearned samples to these causes using observable training and inference signals, and study several targeted mitigation strategies as causal interventions. Experiments on Qwen, LLaMA, and OLMo2 show that incomplete learning is widespread and heterogeneous, and that improvements in aggregate metrics can mask persistent unlearned subsets. The findings highlight the need for fine-grained diagnosis of what supervised fine-tuning fails to learn, and why.

1 Introduction

The paper frames incomplete learning as a persistent failure to reproduce some supervised training examples after SFT, despite convergence. It identifies recurring causes and proposes diagnosing unlearned samples rather than relying only on aggregate performance.

1 Introduction

  • Incomplete learning is failure to correctly answer some supervised training examples after convergence, even with extensive tuning and across seeds and evaluation settings.The failures occur on the SFT dataset itself, not only on held-out or out-of-distribution data.Source: §1 · p. 1
  • Unlearned cases can reduce the utility of costly expert-domain datasets, affect robustness and reliability, and remain hidden by aggregate metric improvements.They tend to involve rare cases, compositional patterns, or knowledge-intensive instances.Source: §1 · pp. 1–2
  • The paper identifies five recurring contributors: missing prerequisite knowledge, conflicts with pre-training knowledge, inconsistent SFT data, left-side forgetting, and insufficient optimization for complex patterns.The causes include earlier supervised instances being overwritten during sequential fine-tuning and rare or compositional structures receiving inadequate training signal.Source: §1 · p. 2
  • A diagnostic framework links unlearned samples to hypothesized causes using training and inference indicators, while targeted interventions test those explanations across Qwen, LLaMA, and OLMo2.The reported findings are heterogeneous: no single intervention resolves all failures, and aggregate improvements can mask persistent unlearned subsets.Source: §1 · p. 2
  • The paper argues for shifting SFT evaluation from performance-centric metrics toward fine-grained diagnosis of which supervised knowledge remains unlearned and why.Its contributions include formalizing ILP, providing a taxonomy and diagnostic framework, and showing that different sources require different remedies.Source: §1 · p. 2

2 Related Works

Related work covers SFT applications and methods for improving data, training schedules, and model capabilities. The paper distinguishes its focus on why supervised examples themselves may remain unlearned.

Supervised Fine-Tuning of Large Language Models

  • SFT is used across instruction following, domain-specific tasks, and multi-task training, with prior work examining both generalization and degradation or forgetting.The cited literature also describes large instruction datasets such as FLAN and their improved zero-shot generalization.Source: §2 · p. 3
  • The paper focuses on why SFT data itself is not fully learned, a question distinct from the related work’s emphasis on improving task performance and fine-tuning behavior.The related-work overview describes reasoning demonstrations injected through SFT as a common thread across several research directions.Source: §2 · pp. 2–3

Data Quality and Multi-Stage Fine-Tuning

  • Prior approaches improve data quality and training through augmentation, active learning, prompt engineering, distillation, curriculum learning, and revisiting earlier tasks.These strategies address diversity, informativeness, training order, and repeated exposure to samples.Source: §2 · pp. 3–4

Scaling Laws in Large Language Models

  • Scaling-law research studies how performance changes with scale across pre-training, transfer learning, preference modeling, and mathematical reasoning.Source: §2 · p. 4

3 Methods

The methods detect supervised instances that remain unlearned after SFT convergence, probe their likely causes, and apply targeted processing strategies rather than treating all failures alike.

3.1 Unlearned Sample Detection

  • After convergence, the framework converts responses to multiple-choice evaluations and uses repeated sampling to identify instances that fail to reproduce the supervision consistently.The converted format is for detection and analysis; it does not alter the original SFT objective.Source: §3.1 · p. 4, §3.1.1 · p. 4, §3.1.2 · p. 5

3.1.4 Empirical Prevalence of Unlearned Samples

  • 15.3% ± 2.1% of supervised instances remain unlearned after SFT convergence on average across ten benchmark datasets and multiple model families and domains.The most severe cases are selected for fine-grained diagnosis.Source: §3.1.4 · p. 5

3.1.5 Knowledge-State Probing for Diagnostic Preparation

  • The framework probes each candidate in the base model before SFT and compares base and fine-tuned predictive distributions to characterize knowledge gaps, conflicting priors, or unstable updates.The base-model probe tests zero-shot correctness; the distribution comparison uses Jensen–Shannon divergence.Source: §3.1.5 · p. 5

3.2.1 Base Model Knowledge Limitations

  • For missing prerequisite knowledge, the method extracts factual triplets, probes them with BoN and pass@N, then uses retrieved knowledge mixed with general data for continued pre-training.The paper reports consistent downstream improvements across medical, legal, and financial benchmarks after continued pre-training and reapplying SFT.Source: §3.2.1 · p. 6

3.2.2 Conflicts Between SFT and Base Model

  • For conflicts with base-model beliefs, high-confidence incorrect answers identify cases for which curated verified information is used in continued pre-training before subsequent SFT.The conflict flag compares the model’s answer confidence with a threshold and the ground-truth SFT label.Source: §3.2.2 · p. 6

3.2.3 Knowledge Conflicts Within SFT Data

  • For contradictory labels on semantically similar SFT inputs, the method removes samples judged incorrect or separates mutually correct conflicts into different mini-batches.The buckets are periodically reassessed, and reduced error rates after bucketing are reported as evidence that data conflict was the primary cause for those samples.Source: §3.2.3 · pp. 6–7

3.2.4 Left-side Forgetting

  • For left-side forgetting during sequential training, the method shuffles the SFT data and dynamically upweights subsets whose validation accuracy drops.The diagnosis tracks per-dataset accuracy while reversing dataset order to examine whether earlier samples are overshadowed.Source: §3.2.4 · p. 7

3.2.5 Insufficient Training

  • For insufficient optimization, training starts with a minimal epoch count and increases it until validation performance ceases to improve.The strategy is intended to accommodate datasets of varying complexity while avoiding premature stopping due to noise.Source: §3.2.5 · p. 7

4 Results Analysis

The results address five sources of incomplete learning with targeted interventions, including continued pre-training, conflict-aware data handling, resampling, and adaptive training duration. The interventions improve domain-specific accuracy, preserve early-learned performance, and reduce learning gaps across the reported settings.

4.1 Base Model Knowledge Enhancement

  • CPT raises accuracy by 9.4%–14.1% across reported domain benchmarks, while extending SFT epochs yields only marginal gains.The reported gains include +12.5% on MedQA and are also observed for OLMo2-7B; careful corpus balancing mitigates negative effects on generalization.Source: §4.1 · p. 7, Figure 4 · p. 8

4.2 Knowledge Conflict Calibration

  • CPT-based calibration improves accuracy across models and benchmarks, with gains from +1.6% to +2.8% and a marked reduction in high-confidence SFT conflicts.Case studies on OLMo2-7B show recalibration where pre-trained knowledge had overridden SFT supervision.Source: §4.2 · pp. 7–8, Table 1 · p. 8

4.3 SFT Knowledge Conflict Resolution

  • Dynamic bucketing improves mixed-domain results, including Qwen-7B from 82.3% to 85.1%, while outperforming removal of conflicting samples.Bucketing separates conflicting examples into distinct batches while preserving valid supervision; removing flagged examples can discard informative data and reduce learning capacity.Source: §4.3 · p. 8

4.4 Alleviating Left-Side Forgetting

  • Global shuffling with dynamic resampling raises Qwen-7B accuracy from 78.5% to 79.8% and improves first-decile summarization ROUGE-L by 29%.The first 10% of summarization data rises from 0.41 to 0.53, with minimal interference in later-stage task acquisition.Source: §4.4 · p. 8, Table 2 · p. 9

4.5 Alleviating Insufficient Learning

  • Adaptive epoch increases raise Qwen-7B accuracy on underlearned tasks from 88.2% to 90.1%, with gains of +1.0% to +1.9% across other reported models.The validation-driven schedule adds epochs when marginal gains appear and stops to prevent overfitting, without reported broader-generalization or unnecessary-compute costs.Source: Table 2 · p. 9, §4.5 · p. 9

5 Conclusion

The paper identifies five contributors to incomplete learning and integrates targeted mitigation strategies into a unified framework. Experiments across multiple LLMs and datasets report improvements in both SFT-specific knowledge mastery and standard-benchmark generalization.

5 Conclusion

  • The paper identifies five contributors to incomplete learning: knowledge limitations, conflicts, dataset inconsistencies, left-side forgetting, and insufficient optimization.The framework integrates pre-training enhancement, conflict-aware processing, dynamic bucketing, resampling, and adaptive epoch augmentation.Source: §5 · p. 9
  • The conclusion frames incomplete learning as heterogeneous, with different underlying causes addressed through targeted rather than one-size-fits-all strategies.The paper identifies distinct contributors and describes a unified framework combining different interventions.Source: §5 · p. 9
  • Across multiple LLMs and diverse datasets, the combined strategies mitigate incomplete learning and improve mastery of SFT-specific knowledge and standard-benchmark generalization.Source: §5 · p. 9

6 Limitations

The section identifies limitations in conflict detection, knowledge supplementation, and computational feasibility, and notes that fine-grained calibration remains underexplored.

6 Limitations

  • Conflict detection depends on high-quality annotations and reliable external tools, while noisy sources can reduce accuracy and leave conflicts only partially resolved.The method also assumes supplementary pre-training data can bridge knowledge gaps; noisy or biased corpora may propagate errors into later fine-tuning.Source: §6 · p. 9
  • At multi-billion-parameter scale, pre-training enhancement and knowledge resampling increase resource demands, potentially limiting feasibility for organizations with constrained hardware or budgets.The section also describes fine-grained calibration as underexplored.Source: §6 · p. 9

A.1 Experimental Datasets

  • The supplement evaluates knowledge-enhancement methods using MedQA, LegalBench, TechFAQ, and FinanceBench, spanning medical, legal, technical, and financial domains.Table 3 is described as reporting SFT dataset statistics and corresponding evaluation metrics.Source: §A.1 · p. 18, Table 3 · p. 18

A.2 Experimental Baselines

  • The baseline comparison uses Qwen-7B, Qwen-14B, Llama2-8B, and Llama2-13B to assess effects across model scales.The passage frames parameter-scale variation as supporting a broader assessment of adaptability and enhancement effects.Source: §A.2 · p. 18

A.3 Algorithm of Knowledge-Enhanced Continue Pre-training

  • Knowledge-enhanced continued pre-training identifies blind knowledge areas, augments them with external material, mixes that corpus with general pre-training data, then validates through SFT.The described mixture is 0.8 general data and 0.2 augmented data; the table compares increased SFT epochs with CPT + SFT across four datasets.Source: §A.3 · p. 18, Table 4 · p. 19

A.4.1 Experimental Design

The experiment tests whether external knowledge enhancement addresses base-model knowledge blind spots. TechFAQ coverage rises after enhancement, while additional SFT epochs alone do not reliably teach the missing knowledge.

A.4.1 Experimental Design

  • TechFAQ knowledge coverage increased from 68.1% to 83.6% after knowledge enhancement.The passage presents this as an example of improved coverage of base-model blind spots.Source: §A.4.1 · p. 19
  • Additional SFT epochs did not reliably help the model master knowledge missing from its base-model blind spots.The passage characterizes these gaps as exhibiting a degree of “unlearnability” under additional SFT alone.Source: §A.4.1 · p. 19
  • The authors conclude that addressing these blind spots requires external knowledge introduced through continued pre-training, rather than relying solely on SFT.The experiment is framed around whether knowledge enhancement improves coverage of blind spots.Source: §A.4.1 · p. 19

B.1 Experimental Datasets

This block covers datasets and strategies for addressing conflicts between base-model knowledge and SFT data, as well as conflicts within SFT datasets.

B.1 Experimental Datasets

  • The knowledge-conflict evaluation includes ARC, CommonsenseQA, SocialIQA, and MedMCQA, covering science, commonsense, social, and medical reasoning.The datasets are described as testing distinct knowledge and reasoning capabilities.Source: §B.1 · pp. 19–20

B.2 Algorithm for Resolving Calibration Conflicts Between SFT and Base Model

  • The strategy flags high-confidence incorrect predictions, retrieves relevant external knowledge, mixes it with general pre-training data, and continues pre-training before SFT validation.A sample enters the error set when confidence in an incorrect prediction exceeds a predefined threshold.Source: §B.2 · p. 20, §C.1 · p. 21
  • Continued pre-training significantly reduced SFT data conflict rates across all models and four datasets, with the largest reduction on MedMCQA.Qwen 14B had the greatest conflict-rate reduction among the models; the authors suggest its larger parameter scale may contribute.Source: §B.2 · p. 20, Table 6 · p. 20

C.1 Experimental Datasets

  • The SQuAD dataset evaluates passage-based reading comprehension, including the ability to resolve potential conflicts in context-specific information.It is listed among question-answering datasets used to evaluate performance in knowledge-conflict scenarios.Source: §C.1 · pp. 20–21

CoQA (Conversational Question Answering

  • The broader QA dataset list includes CoQA, TriviaQA, and Natural Questions, covering conversational context, general trivia, and open-domain queries.The descriptions emphasize maintaining context across dialogue turns and integrating information from diverse sources.Source: CoQA (Conversational Question Answering · p. 21

C.2 Algorithm of Knowledge Conflicts Between SFT Data

  • The within-SFT conflict strategy checks semantically similar sample pairs, removes incorrect items, and distributes conflict-group samples across buckets.The description says Sentence-BERT computes similarity and GPT-4 assesses correctness.Source: §C.2 · p. 21

C.3 Deleting and Grouping Conflicting Data

The experiments report accuracy gains from detecting and grouping conflicting SFT examples, with dynamic grouping described as producing superior learning outcomes.

C.3 Deleting and Grouping Conflicting Data

  • The experiment evaluates conflict detection and conflict grouping as strategies for handling sample-level contradictions during SFT.The reported rationale is to reduce interference from conflicting knowledge while preserving high-quality samples.Source: §C.3 · pp. 21–22
  • 1–5% accuracy increases followed conflict-data removal, while detection and grouping consistently improved accuracy over directly merging datasets.The authors also report that periodically re-evaluating conflicts through dynamic grouping yields superior learning outcomes.Source: Table 8 · p. 22, §C.3 · p. 22

D.1 Experimental Datasets

The section describes datasets used in the experiments, a dynamic resampling procedure for left-side forgetting, and results showing its strongest gains on early training data.

D.1 Experimental Datasets

  • Table 9 catalogs dataset statistics and evaluation indices; examples include CNN/DailyMail and XSum summarization datasets, with XSum testing highly abstractive summaries.OpenWebText is described as an open-source text dataset derived from Reddit submissions.Source: Table 9 · p. 22, §D.1 · p. 22

D.2 Algorithm of Left-side Forgetting

  • Dynamic resampling tracks accuracy changes by dataset and, when accuracy falls beyond a threshold, adds resampled examples from that dataset to the current training batch.The procedure shuffles the data and checks each dataset after fixed training-step intervals.Source: §D.1 · p. 22, §D.2 · p. 23

D.3 Analysis of Alleviating Left-sided Forgetting

  • ROUGE-L for the first 10% of training data rose 29% (0.41 to 0.53); the last 10% decreased 1.6%, while middle-stage data gained 3.5%.The authors report that forgetting was most pronounced during the initial stage of training.Source: §D.3 · p. 23

E.1 Experimental Datasets

The experiments describe the datasets and adaptive epoch strategy, then assess OLMo2-7B against SFT data and its Dolma pre-training corpus. The analysis reports non-existence and conflict rates, especially for professional-knowledge data.

E.2 Algorithm of Insufficient Training

  • The Epoch Increment Strategy progressively trains and evaluates on a validation set, stopping when performance no longer improves and recording the best epoch.The strategy starts at one epoch and increases the training round when validation performance improves.Source: §E.2 · p. 23, §E.1 · p. 23

F Experiment and Analysis with Olmo2-7B

E.1 Experimental Datasets

  • The section’s dataset inventory includes AG News, IMDB, MultiNLI, and Quora Question Pairs, with Table 12 presenting dataset statistics and evaluation indices.The datasets cover classification, sentiment analysis, natural-language inference, and duplicate-question detection.Source: §E.1 · p. 23, Table 12 · p. 24

F.1 Experimental Setup

  • The expanded evaluation measures general, reasoning, professional-knowledge, and multilingual abilities using standard benchmarks and each benchmark’s primary accuracy metric.The listed benchmarks include MMLU and AGIEval, BBH, GPQA and NQ, and MMLU-Multi.Source: §F.1 · p. 24

F.2 Analysis of SFT Data in Relation to OLMo2 Pre-training Corpus

  • The Dolma comparison retrieves thematically relevant snippets and uses semantic matching and GPT evaluation to assess whether SFT knowledge is absent or conflicts with pre-training data.The retrieval step selects the top 100,000 snippets for each SFT dataset’s concepts.Source: §F.2 · pp. 24–25
  • 14.5% overall Conflict Rate; Professional Knowledge reached 27.4% Knowledge Non-Existence Rate and 18.4% Conflict Rate in OLMo2’s Dolma comparison.The analysis compares SFT knowledge items with information retrieved from the pre-training corpus.Source: Table 13 · p. 25, §F.2 · p. 25

F.3 CPT Performance and Knowledge Relationship Analysis on OLMo2-7B

On OLMo2-7B, CPT decreased performance across the listed broad benchmarks, while case studies show that it could steer specific conflicting outputs toward SFT knowledge. The results point to a trade-off between targeted correction and broad capability retention.

F.3 CPT Performance and Knowledge Relationship Analysis on OLMo2-7B

  • -6.8% on GPQA and -8.1% on MMLU-Multi; CPT lowered performance across all listed OLMo2-7B general, reasoning, professional-knowledge, and multilingual benchmarks.The authors describe these broad benchmark declines as potentially reflecting representational adjustment, not as evidence that targeted corrections failed.Source: §F.3 · pp. 25–26, Table 14 · p. 26
  • The authors identify a potential trade-off between targeted knowledge injection and preserving broad generalization, with CPT’s broader impact requiring careful calibration.They suggest subsequent fine-tuning or alignment may be needed to re-optimize general capabilities after specialized CPT.Source: §F.3 · p. 26, §F.5 · p. 27

F.4 Case Studies of Knowledge Conflict Resolution in OLMo2-7B

  • CPT shifted OLMo2-7B toward updated SFT facts when its pre-training knowledge was outdated, as illustrated by a current-president query.Before CPT, responses often defaulted to older pre-training knowledge.Source: §F.4 · p. 26
  • For disciplinary controversies and evolving terminology, CPT shifted responses toward SFT-emphasized perspectives and vocabulary, including Loop Quantum Gravity and LayerScale.The examples contrast these with String Theory and Dropout, respectively.Source: §F.4 · pp. 26–27
  • CPT improved responses to multilingual and geospecific queries, including recognizing the Chinese name for Cupertino rather than defaulting to English-based processing.The passage describes a prior tendency to default to English processing or fail to link the Chinese entity correctly.Source: §F.4 · p. 27
  • CPT also made responses more likely to reflect SFT-provided cultural and regional legal specifics, such as gesture meanings and jurisdiction-dependent privacy ages.Source: §F.4 · p. 27

F.5 Summary and Implications of OLMo2 Experiments

  • Across OLMo2 experiments, substantial SFT knowledge was absent from or conflicted with pre-training, while CPT steered specific conflict cases toward SFT knowledge despite complex broad-capability effects.The authors characterize CPT as useful for fine-grained correction but say its wider effects warrant context-dependent application and evaluation.Source: §F.5 · p. 27

Related papers

Loading 2604.10079v4…