PaperlayerView exact revision on arXiv

Source-linked AI summary

Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization

Qian Kou, Xiaofeng Shi, Xiaosong Qiu, Hua Zhou

arXiv:2608.20281v1cs.CLcs.AI

TL;DR

Retrieval-free document question answering asks whether a model can answer held-out questions from a bounded corpus without seeing source documents at inference time. IAR separates document exposure, QA alignment, and general-capability recovery, improving the domain-general frontier across corpora and model families.

  • Problem

    Document knowledge internalization asks whether models can answer questions about a bounded corpus without receiving its source documents at inference time.

  • Method

    IAR separates structured document exposure, answer-only QA alignment, and post-hoc recovery through model merging as three post-training stages.

  • Results

    IAR improves the domain-general frontier in most settings; on Qwen3-4B CC, domain accuracy reaches 50.5% versus 42.4% for Vanilla SFT.

  • Takeaways & Limitations

    The results support evaluating document exposure, QA accessibility, and general-capability recovery separately rather than collapsing them into one fine-tuning comparison.

Abstract

from arXiv · show

Large language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time. We study this setting as document knowledge internalization: converting a fixed corpus into usable parametric knowledge for retrieval-free question answering. We propose IAR (Inject, Align, and Recover), a three-stage post-training framework that separates structured document knowledge injection, QA behavior alignment, and general ability recovery. Unlike conventional continued pretraining, Inject converts source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives. Align then adapts the injected model with answer-only QA supervision, while Recover merges the domain-adapted model with the base instruction model to recover general capabilities. Across Common Corpus (CC) and CCI, and across Llama, Phi, Qwen, and SmolLM model families, IAR improves the domain-primary domain-general frontier for retrieval-free document internalization. In the main comparison, IAR improves over Vanilla SFT on all four reported metrics in 7 of 8 dataset-model settings, with average gains of 3.6 percentage points in domain QA accuracy and 12.1 percentage points in mean general performance across IFEval, MMLU, and MSBench. Extended CC baselines show that LoRA and FAPM can win individual general metrics, but among methods that also reach leading or near-leading domain internalization, IAR retains one of the strongest general profiles.

Introduction

The paper studies retrieval-free question answering over fixed document collections, where models must answer document-derived questions without seeing source documents at inference time. It proposes IAR, a three-stage framework that injects structured document knowledge, aligns it to QA, and recovers general capability through model merging.

  • Problem setting: Retrieval-free document internalization requires answering held-out questions from a bounded corpus without receiving source documents at inference time.The problem is evaluated on Common Corpus (CC) and CCI.
  • Limitations of existing routes: Vanilla SFT provides the test-time QA format but sparse factual supervision, whereas CPT exposes models to more document text without directly teaching question answering.Both approaches also face a trade-off involving general capability.
  • IAR framework: IAR separates post-training into Inject, Align, and Recover: structured reconstruction objectives, answer-only QA supervision, and post-hoc merging with the original instruction model.Inject uses continuation, rewrite, and instruction-conditioned reconstruction tasks; Recover produces checkpoints trading domain accuracy against general capability.
  • Empirical claim: IAR improves the domain-general frontier by increasing retrieval-free domain accuracy over Vanilla SFT while restoring general capability damaged by domain adaptation.Against BudgetMatch QA-only SFT, both CCI settings are all-metric wins for IAR, with Qwen3-4B CCI showing the largest separation.
  • Evaluation scope: Experiments cover CC and CCI, multiple model families, and comparisons with direct SFT, LoRA, SDFT, Replay, FAPM, and continued-pretraining baselines.The paper reports both frontier improvements and boundary cases clarifying when initialization strength and data-related factors matter.

Related Work

Prior work studies parametric knowledge, retrieval-augmented memory, knowledge editing, and document-focused acquisition, but published comparisons use different protocols. IAR instead presents a controlled decomposition of structured exposure, answer-only accessibility, and post-hoc capability recovery.

  • Parametric knowledge and retrieval: Parametric models store factual associations but recall and updating are unreliable, motivating retrieval-augmented systems and localized knowledge editing.Retrieval-augmented systems externalize document memory through retrievers, whereas knowledge editing targets localized factual changes.
  • Document knowledge acquisition: Document-acquisition methods vary in interface and training design, including reading-comprehension conversion, reversed training order, self-supervised tasks, and simulated dialogues.AdaptLLM converts raw corpora into reading-comprehension texts; PIT trains on questions before documents; SELF-TUNING adds memorization, comprehension, and reflection tasks; KiDG simulates dialogues.
  • Document knowledge acquisition: IAR separates structured document exposure from answer-only accessibility and treats post-hoc capability recovery as an explicit third variable.The paper frames this separation as a controlled empirical decomposition rather than a claim to introduce a new family of reconstruction losses.
  • Forgetting and recovery: Task adaptation can cause catastrophic forgetting, while prior recovery approaches include post-hoc pruning and weight-space model merging.Recover selects among existing SLERP, task-arithmetic, TIES, and DARE operators rather than introducing a new merge algorithm.

Methodology

IAR treats retrieval-free document knowledge internalization as a staged process: dense document supervision is injected, QA accessibility is aligned, and general capabilities are recovered. The framework evaluates domain QA accuracy while using general instruction-following metrics as retention guardrails.

  • Problem setup: The task derives training and held-out QA sets from a target document collection and requires answering test questions without retrieved passages at inference.The primary domain metric is correctness-based QA accuracy.
  • Framework rationale: IAR decomposes document adaptation into Inject, Align, and Recover to separately test corpus exposure, QA accessibility, and general capability retention.Inject increases document-level supervision, Align makes acquired information accessible through question answering, and Recover moves the adapted model toward the original instruction model.
  • Inject: Inject uses continuation, rewrite, and instruction-formatted reconstruction objectives to provide denser supervised document targets than QA-only training without raw-stream CPT loss.Prompt and user tokens are masked, so supervision is applied only to the assistant document target; this distinguishes the recipes from raw continued pretraining.
  • Align: Align fine-tunes the injected checkpoint on domain QA pairs with answer-only supervision, whereas Vanilla SFT applies the same objective from the original instruction model.The resulting Inject+Align checkpoint is denoted IA; BudgetMatch preserves the QA-only objective while matching the two-stage pipeline’s token budget with setting-specific epochs.
  • Recover: Recover evaluates SLERP, task arithmetic, TIES, and DARE merges, selecting candidates by a domain-primary validation frontier with general metrics as guardrails.The selection retains candidates within τ = 1.0 percentage point of Vanilla SFT in domain accuracy, requires no lower mean general performance, and requires at least two general metrics to remain within tolerance.

Experiments

Experiments evaluate IAR’s retrieval-free domain-general operating points through cross-method comparisons, ablations, and Qwen scaling on CC and CCI. The study uses question-only test inputs, adaptive LLM judging for domain QA, and controlled general-evaluation decoding.

  • Research questions: The experiments test four questions spanning IAR’s domain-general frontier, extended QA-only training, pre-Recover Inject+Align performance, and scaling across larger Qwen3 models.CC-only stress tests are grouped under RQ1, while Qwen scaling remains separate from cross-family comparisons.
  • Datasets: 10,926 CCI training and 575 test pairs complement 14,258 CC training and 750 test QA pairs, with question-only test inputs.Both datasets cover Llama-3.2-3B, Phi-4-mini, Qwen3-4B, and SmolLM3-3B; Qwen3-8B/14B/32B are CC scaling ablations.
  • Baselines: The comparison includes Vanilla SFT, BudgetMatch, SDFT, LoRA, Replay, CPT+SFT, FAPM, and the original instruction model to probe distinct adaptation and recovery strategies.The main comparison focuses on rows supporting the central domain-general claim, while CPT+SFT diagnostics serve as dense-document baselines rather than frontier-defining rows.
  • Baselines: CPT+SFT uses released Base checkpoints, unlike IAR’s Instruct initialization, so it does not by itself isolate the Inject objective.Instruct-initialized CPT+SFT diagnostics are separately available for selected Llama and Phi settings; Phi-4-mini lacks a corresponding non-instruction Base release.
  • Evaluation: Domain QA uses two adaptive LLM judges, adds a third only for disagreement, and aggregates the final ordinal score by the judges’ median.Per-judge outputs are stored before aggregation, and the archived panel includes gpt-oss-120b, MiniMax-M2.5, and DeepSeek-V3 variants.
  • Evaluation: Domain QA uses one generation pass with temperature .7, top_p=.95, repetition penalty 1.1, and a 2,048-token limit; general evaluations use task-specific greedy decoding limits.MSBench judge calls use temperature .1, and sampler seeds are unspecified for inference and judge APIs, so raw judge votes are preserved for audit.

Results

Across model families and corpora, IAR generally improves the domain–general frontier over Vanilla SFT, while its gains depend on the corpus, model, and training recipe. Token-budget controls, stage-level analyses, and scaling results indicate that Inject+Align supplies domain gains and Recover restores general performance with limited domain loss.

  • Main comparison: IAR exceeds Vanilla SFT on domain accuracy and all three general metrics across CC models and most CCI models.This holds for all four CC model families and for Llama, Qwen3-4B, and SmolLM3-3B on CCI; Phi is the boundary case.
  • Main comparison: 50.5% versus 42.4% domain accuracy is the clearest CC Qwen3-4B example, alongside improvements on all three general metrics.The comparison illustrates IAR’s domain–general frontier rather than uniform dominance.
  • Extended baselines: IAR is domain-best for Phi, Qwen3-4B, and SmolLM3-3B, and second only to SDFT for Llama in the CC-only extended baseline test.LoRA and FAPM often preserve or recover stronger individual general metrics than IAR, but usually trail on domain internalization.
  • Token-budget ablation: IAR wins 14 of 16 token-budget comparisons, while CC Llama remains a domain–general trade-off.BudgetMatch is competitive on CC but does not transfer uniformly to CCI; Figure 2 reports a 3.9-point domain loss and 11.0-point mean-general gain for CC Llama.
  • Stage-level effect: Best IA improves domain accuracy over Vanilla SFT in all eight settings, with gains of 2.8–7.7 points on CC and 0.4–6.1 points on CCI.The only near-tie is CCI Qwen3-4B at 0.4 points, which begins from a 70.6% base domain score.
  • Scaling and recovery: Across Qwen3 8B/14B/32B, IAR stays within 1.1 points of Best IA domain accuracy while raising mean general performance by 14.9–24.1 points.Relative to Best IA, IAR raises IFEval/MMLU/MSBench by 11.6/8.2/25.0 points at 8B, 14.0/26.9/31.5 at 14B, and 14.0/11.5/28.0 at 32B, while domain decreases by 0.7, 0.9, and 1.1 points.

Discussion

The discussion frames IAR as a decomposition of adaptation budgets rather than a universally dominant recipe, emphasizing separate evaluation of domain acquisition and general retention. It also describes frontier-based checkpoint selection and individually reported general benchmarks to keep deployment choices auditable.

  • Adaptation budgets: IAR should be interpreted as a decomposition of adaptation budgets, not a recipe that dominates every baseline.The compared methods intervene through QA repetition, data recipes, parameter-efficient adaptation, pruning-based recovery, or raw-document modeling.
  • Evaluation: Domain acquisition and general retention must be evaluated separately rather than collapsed into one score.The discussion presents this separation as necessary for interpreting the differing intervention points of the controls.
  • Checkpoint selection: Recover selects a deployable checkpoint from a small domain-general frontier because the highest-domain IA checkpoint may lose instruction-following or general-benchmark performance.Domain accuracy is the primary objective, while IFEval, MMLU, and MSBench serve as guardrails.
  • Evaluation: IFEval, MMLU, and MSBench are reported individually so recovery differences remain visible across instruction following, factual reasoning, and judged response quality.The component metrics make the selected operating point auditable instead of reducing recovery to a single composite score.

Conclusion

IAR separates document exposure, QA alignment, and post-hoc recovery for retrieval-free document knowledge internalization. Inject+Align provides domain gains across all eight main settings, while Recover strengthens general performance while retaining most domain gains.

  • Framework: IAR separates document exposure, QA alignment, and post-hoc recovery for retrieval-free internalization.The framework’s three stages are Inject, Align, and Recover.
  • Domain internalization: Before Recover, Inject+Align contributes domain gains across all eight main settings.The passage reports gains across all eight settings but does not quantify their sizes here.
  • General performance recovery: Recover moves adapted checkpoints toward stronger general performance while retaining most of their domain gain.The passage also notes a repeated Qwen3 scaling pattern.
  • Method dependence: The best Inject mixture remains dependent on the model and corpus.This dependence is reported before Recover and Align’s domain gains are assessed across the main settings.

A Dataset and Task Examples · B Prompt Templates and QA Accounting

CC and CCI evaluate retrieval-free document knowledge internalization: models train on document-derived QA pairs but receive only questions at test time. The paper specifies anchor-aware QA construction, explicit prompt and scoring contracts, and accounting caveats for the resulting datasets.

  • A Dataset and Task Examples: CC derives from Common Corpus and CCI from the CCI dataset, with document-generated training examples and held-out test examples.Both datasets evaluate models without source documents or retrieved passages at inference, making the setting stricter than RAG-style QA.
  • B Prompt Templates and QA Accounting: The paper reports English prompt schemas for reproducibility, while the implementation uses Chinese instruction templates across QA construction, post-training, and evaluation.These schemas preserve operative constraints, placeholders, and output contracts.
  • B Prompt Templates and QA Accounting: QA construction extracts referable anchors, selects question types, generates self-contained questions, validates them, and produces answers grounded in the same document chunk.The schemas are designed to prevent deictic questions that require access to the original document.
  • B Prompt Templates and QA Accounting: CC and CCI stage rates describe data flow rather than directly comparable dataset-quality scores because their source-sampling and prefiltering paths differ.This limits interpretation of cross-dataset comparisons based solely on pipeline rates.
  • B Prompt Templates and QA Accounting: All domain evaluations use question-only inference, and the V2 correctness judge compares the model answer with the reference answer for semantic equivalence while ignoring source attribution.General benchmarks use task-specific contracts: rule-based scoring for IFEval, option-letter output for MMLU, and judged JSON correctness and quality for MSBench.
  • A Dataset and Task Examples: At evaluation, models receive only the question and must answer from internalized parameters rather than retrieved source passages.The same checkpoint is also evaluated on IFEval, MMLU, and MSBench.
  • B Prompt Templates and QA Accounting: Table 6 counts retained and considered files, chunks, questions, and QA records, with Experiment QA representing the train-plus-test total after deduplication and fixed chunk-group sampling.CC counts come from trusted run statistics, whereas CCI counts were recovered from validated timestamped logs and resume artifacts after stale persisted statistics were identified.
  • B Prompt Templates and QA Accounting: Inject uses three instruction-conditioned document-generation objectives with assistant-target loss, while Align and Vanilla use question-only prompts with loss applied only to answer spans.The Inject mixture samples recipe datasets in stated integer ratios, though tokenizer-specific length filtering can alter realized shares.

C Training and BudgetMatch Details

Training uses masked assistant-target optimization with DeepSpeed-based full-parameter or adapter-only runs, while BudgetMatch compares realized non-padding token volumes across Inject and Align. Reported checkpoints are single final-epoch runs with fixed seeds, and scaling-run provenance is incomplete beyond the retained Recover settings.

  • Training configuration: All objectives mask prompt tokens and optimize only the assistant target span; full-parameter runs use DeepSpeed ZeRO-2, while LoRA updates adapters under the same optimizer schedule.Stage-specific settings and exceptions are recorded separately in Table 9.
  • Computing environment: The archived main runs used eight NVIDIA A100-SXM4-40GB GPUs, 128 physical CPU cores, and approximately 1 TiB of host memory.The software stack included CUDA 12.4, PyTorch 2.6.0, Transformers 4.57.1, DeepSpeed 0.14.3, PEFT 0.12.0, vLLM 0.8.4, and mergekit 0.1.3.
  • BudgetMatch accounting: BudgetMatch defines IA total as Inject plus Align and compares it directly with realized non-padding training-token volumes under each model’s tokenizer.The reported integer schedules are 14, 17, 21, and 11 epochs for CC Llama, CC Qwen3-4B, CCI Llama, and CCI Qwen3-4B, respectively.
  • Run protocol: Every reported checkpoint is a single run using seed=42 and data_seed=42, with data shuffling seed 1234 and an internal split seed of 42.DeepSpeed saves once per epoch without selecting a validation-best checkpoint; final_model is the final-epoch model.
  • Scaling-run provenance: Scaling-run archives retain Qwen3-8B/14B/32B Recover configurations with TIES density .3, but lack original Inject/Align arguments, logs, and node manifests needed to establish other settings.The available archive does not support inferring learning rate, batch size, sequence length, precision, GPU or node counts, or ZeRO/offload settings from launcher defaults.

D Evaluation and Judge Reliability · E Recover Selection Protocol

Evaluation uses domain QA as the primary retrieval-free internalization metric, with IFEval, MMLU, and MSBench as general-capability guardrails and judge-based audits for non-answer-key metrics. Recover selects an operating point on validation data using domain feasibility and general-performance constraints, while final CC and CCI scores come from held-out test sets.

  • D Evaluation and Judge Reliability: Domain QA uses two judges per answer, adds a third on disagreement, and aggregates scores by the median of available scores.Domain scores are drawn from {0, .5, 1}.
  • D Evaluation and Judge Reliability: MSBench uses two judges first, adds a third on binary disagreement, aggregates correctness by majority vote, and averages available quality scores.Both MSBench metrics are judge-produced rather than answer-key labels.
  • D Evaluation and Judge Reliability: The judge-agreement audit covers 343 result files and 242,255 valid model-answer records, with ten malformed JSON lines skipped.Agreement is measured on the first two judges, while the third-judge rate measures arbitration frequency.
  • D Evaluation and Judge Reliability: Each result file receives 2,000 example-level bootstrap resamples with seed 20260706 and percentile 95% intervals for domain accuracy and mean correctness.These intervals quantify evaluation-sample uncertainty but not training-seed robustness.
  • D Evaluation and Judge Reliability: Domain QA is the primary retrieval-free internalization metric, while IFEval, MMLU, and MSBench are general-capability guardrails.IFEval and MMLU use rules or answer keys; MSBench uses an LLM judge.
  • E Recover Selection Protocol: The selected Recover row is an operating point on the domain-primary frontier rather than necessarily the maximum-domain checkpoint.Recover candidates are compared on IFEval, MMLU, and MSBench after meaningful domain accuracy is established relative to Vanilla SFT.
  • E Recover Selection Protocol: Recover selection uses an internal validation split, while the published CC and CCI files with 750 and 575 examples are held-out test sets.This separation prevents final reported scores from being used for Recover-candidate selection.
  • E Recover Selection Protocol: Recover first filters candidates by D(c) ≥ D(v) − τ with τ = 1.0 percentage point, then applies general-performance guardrails against Vanilla SFT.The guardrails require G(c) ≥ G(v) and at least two of IFEval, MMLU, and MSBench to be no more than τ below Vanilla SFT.

F Qwen3-4B CCI Diagnostic

Qwen3-4B on CCI is a boundary case because its instruction checkpoint already achieves 70.6% domain accuracy, leaving limited headroom for document-internalization training. Recover therefore matters mainly for preserving this strong prior while restoring general capability, rather than for producing a large domain gain.

  • High-base-prior setting: 70.6% domain accuracy is achieved by the original Qwen3-4B instruction checkpoint before document-internalization training, unusually high relative to the other model families.This high-base-prior setting changes how subsequent CCI results should be interpreted.
  • Main interpretation: 76.3% is reached by the selected IAR checkpoint, compared with 75.1% for Vanilla SFT and 75.5% for the best pre-recovery Inject+Align row.The selected IAR checkpoint also substantially improves IFEval, MMLU, and MSBench over Vanilla SFT.
  • Source-text diagnostic: 100% of documents favor Qwen over Llama in BPB, versus 99.0% over Phi and 86.8% over SmolLM3.Qwen3-4B has lower equal-domain macro BPB than every peer, with all paired 95% bootstrap intervals below zero.
  • Diagnostic limitation: Phi’s lower BPB than Llama but slightly lower CCI QA accuracy shows that source-text fit is not a sufficient explanation of QA behavior.The diagnostic compares complete checkpoints rather than controlling model capacity or other confounds.
  • Recover sweep: 76.3% domain performance is reached by Task Arithmetic w = 0.7 and TIES d = 0.3 in the Qwen3-4B Recover sweep.Several merge operators recover to the mid-70s domain range, supporting Recover’s role in restoring general capability while retaining the high CCI prior.

G Extended Baselines and Recipe Analyses

Extended analyses show that baseline behavior depends on initialization and recovery method, while Inject recipe preferences vary by model and corpus. FAPM can restore general capability but may sacrifice document internalization, whereas Qwen3 CCI represents a high-base-prior boundary case.

  • Initialization diagnostics: Instruct-initialized CPT+SFT is diagnostic rather than a second main baseline matrix, with incomplete coverage across the available archived runs.The diagnostics do not support extrapolation to Qwen3 or SmolLM3.
  • Initialization diagnostics: CPT is initialization-sensitive, so the results do not support a uniform ordering between structured Inject and raw-document CPT.The reported CC Llama comparison is 52.9 versus 58.6.
  • Qwen3 boundary case: Qwen3-4B CCI is a high-base-prior boundary case: multiple merge operators remain in the high-domain range, and Task Arithmetic and TIES reach the selected value.The setting is interpreted primarily as preserving and recovering a strong prior rather than creating a large new domain gain.
  • Inject recipe sensitivity: Mixed 1:1:2 is best for most model–corpus combinations, while CC Qwen and SmolLM favor Mixed 1:1:1 and CCI Qwen favors reconstruction-only 1:0:0.These preferences come from the full pre-recovery Inject recipe grids.
  • Recovery baselines: FAPM often improves general benchmarks but can sacrifice internalized document knowledge, making it unsuitable as a drop-in replacement for IAR.The comparison uses pruning-based recovery at sparsity 0.9.
Loading 2608.20281v1…