PaperlayerView exact revision on arXiv

Source-linked AI summary

Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge

Xiaofeng Shi, Xiaosong Qiu, Wenxin Ma, Qian Kou, Yiming Pan, Longbin Yu, Ying Liu, Haiping Wang, Hua Zhou

arXiv:2608.01862v1cs.AI

TL;DR

Enterprise QA must acquire proprietary knowledge while retaining general capabilities, but evidence is limited on how post-training choices interact after SFT fixes easy examples. Wnuan combines document-to-QA supervision, general-data replay, and residual-error reinforcement learning, raising WnuanBench AAR from 52.76% to 91.51% while showing residual-error sampling outperforms alternatives under a matched budget.

  • Problem

    Enterprise QA requires models to learn proprietary knowledge, retain general instruction-following ability, and use limited post-training budgets efficiently, while interactions among pipeline choices remain less understood.

  • Method

    Wnuan combines document-to-QA supervision, general-data replay, and residual-error reinforcement learning in an end-to-end closed-book enterprise QA pipeline.

  • Results

    91.51% AAR was achieved after RL from 52.76% before adaptation, while residual-error sampling outperformed full-pool and size-matched random sampling under matched 100-update training.

  • Takeaways & Limitations

    After SFT resolves many easy examples, concentrating a fixed update budget on remaining errors is effective under the tested protocol.

  • Takeaways & Limitations

    The study covers one enterprise and benchmark sets sharing the authorized source corpus, so its uncertainty estimates do not extend to new organizations or open-world documents.

Abstract

from arXiv · show

Enterprise question answering requires models to acquire proprietary knowledge without discarding general capabilities. We present Wnuan, a three-stage pipeline that constructs task-oriented supervision from documents, performs supervised fine-tuning with general-data replay, and applies reinforcement learning to residual errors. On the 707-question WnuanBench, the primary 32B route raises acceptable-answer rate (AAR) from 52.76% before adaptation to 80.06% after SFT and 91.51% after RL. Under a matched 100-update protocol, residual-error sampling outperforms full-pool and size-matched random sampling by 3.11 and 2.97 points, respectively. Source-cluster bootstrap intervals remain above zero for both contrasts, and a same-domain validation set preserves the ordering. The general-benchmark average decreases by 5.17 points across the route, concentrated in instruction following. The automatic evaluation ensemble agrees with an authoritative domain expert on 90.5% of a stratified Wnuan-Inst response sample. These results characterize both the gains and the general-capability cost of staged enterprise adaptation.

1 Introduction

Wnuan addresses enterprise QA by adapting general models to proprietary knowledge while seeking to preserve instruction following and use limited post-training efficiently. Its three-stage pipeline combines document-derived supervision, general-data replay during SFT, and residual-error reinforcement learning, evaluated through WnuanBench and controlled studies.

  • Motivation: Enterprise QA requires learning internal policies, technical standards, and procedures absent from public pretraining while retaining general instruction-following ability.The setting also imposes a limited post-training budget.
  • Motivation: Prior methods address document adaptation, generalization, instruction following, replay, retrieval, and data selection separately, leaving their interaction in one enterprise QA pipeline less understood.The gap is especially relevant after SFT has corrected most easy examples.
  • Method: Wnuan uses three stages: document-to-question–answer supervision, full-parameter SFT with general-data replay, and semantic-reward GRPO on residual errors.Retrieval is evaluated separately rather than used to train a retrieval-aware generator.
  • Evaluation: The paper evaluates the complete enterprise-model training pipeline with WnuanBench, measuring SFT contributions, replay trade-offs, residual-error RL, and fixed-budget sampling alternatives.Additional evaluation includes public general benchmarks, a same-domain validation set, and a training-side validation signal.
  • Contributions: The contributions include an end-to-end proprietary-document adaptation pipeline, a general-data replay operating point, residual-error RL, and the WnuanBench benchmark.The evaluation program includes automatic correctness assessment, configuration studies, controlled GRPO comparisons, and source-cluster sensitivity analysis.

2 Related Work

Wnuan situates its approach among domain-QA supervision, retention, post-training data selection, retrieval, and model-based evaluation precedents. It assembles these techniques as a recipe rather than introducing a new synthesis method or retention objective.

  • Domain adaptation and task-oriented supervision: Domain-QA supervision spans reading-comprehension reformulation, pre-instruction tuning, and knowledge- or coverage-aware generation.The cited precedents include AdaptLLM, pre-instruction tuning, KEFT, and DS2-Instruct.
  • Domain adaptation and task-oriented supervision: Wnuan combines these domain-QA supervision precedents as a recipe, not a new synthesis method.This positioning is stated explicitly in the related-work discussion.
  • Retention during specialization: Replay is a documented mitigation for specialization-related changes in generalization and instruction following.Prior work mixes general QA examples during fine-tuning, while Wnuan measures an SFT replay grid and selects an observed domain–general operating point.
  • RL and data selection: PPO and GRPO provide the optimization basis for modern language-model post-training, while prior methods prioritize influential, filtered, or difficult prompts.The cited examples include LESS, DAPO, difficulty-aware alignment, and fixed-budget GRPO studies.
  • Retrieval and model-based evaluation: RAG and RAFT motivate retrieval-augmented comparisons, but Wnuan is not retrieval-aware and instead uses a fixed retrieval-concatenation baseline.Open-form enterprise answers are evaluated with multi-model judging calibrated against a domain expert because exact match is unreliable.

3 Method

Wnuan constructs target-aligned supervision from proprietary documents, selects a 50% nominal general-data replay setting for SFT, and initializes residual-error RL from the resulting model. Its main pipeline combines filtered QA generation, answer rewriting, and a controlled data-selection comparison.

  • Supervision construction: 231,662 rows spanning 221,825 unique questions and 5,648 source paths form the pre-rewriting QA inventory, filtered through semantic, answerability, faithfulness, and quality checks.Documents are OCR-normalized and segmented before named anchors, six task-form questions, and supporting-chunk answers are generated.
  • Supervision construction: 221,294 examples comprise the SFT domain set after target-aligned answer rewriting, which replaces originals only after semantic-similarity and format gates.Among final SFT examples, 164,793 pass the similarity gate; filtering leaves 164,744 rewritten targets and 56,550 retained original targets.
  • SFT replay: 50% nominal replay is selected by the highest unweighted average of MMLU, IFEval, and C-Eval, defining Wnuan-Inst for Stage III.The selected setting contains 106,950 general examples, or an actual ratio of 48.3%.
  • Data-selection comparison: 100 updates compare residual-error, full-pool, and size-matched random selection under shared initialization, prompts, scoring, rollouts, optimizer settings, and schedule.The direct-answer prompt omits format tags, making rformat = 0 for every arm and using the common 0.6racc + 0.3rquality signal.

4 Experimental Setup

The experiments evaluate enterprise QA on a 707-question benchmark spanning three question types and eight business domains, using independent references and same-domain validation. Evaluation combines ordered multi-judge scoring, expert agreement checks, bootstrap confidence intervals, and matched data-selection arms under controlled training protocols.

  • Evaluation data: 707 questions cover 160 general-knowledge, 370 operational-scenario, and 177 standards/specification items across eight business domains.Internal personnel independently curate questions and references; each record includes a question, reference answer, source, and domain label.
  • Evaluation protocol: Two primary judges assign ordered correctness labels on {0, 0.5, 1}, with a third vote on disagreement and the ordered median retained.The primary judges are gpt-oss-120b and MiniMax-M2.5, while DeepSeek-V3.2 supplies the third vote.
  • Evaluation data: 90.5% of 147 valid binary labels agree with the authoritative domain expert.The 95% CI is 85.7–94.6%, with κ = 0.796; hallucination detection has precision 0.868, recall 0.657, and F1 0.748.
  • Statistical analysis: 2,000 question-level bootstrap resamples provide primary paired confidence intervals, with source-document resampling used for sensitivity analysis.Paired binary comparisons use McNemar tests with Holm adjustment for three planned residual/full/random contrasts.
  • Controlled comparisons: 56,147 rows comprise each residual and random arm, versus 230,183 rows in the full arm.All arms start from Wnuan-Inst; residual versus random controls pool size, while residual versus full tests sampling efficiency under a common update budget.

5 Results

Wnuan’s staged adaptation substantially improves enterprise QA, with SFT and RL raising WnuanBench AAR from 52.76% to 91.51%. Residual-error sampling further outperforms full-pool and size-matched random sampling, while replay and retrieval expose trade-offs with general capability and adaptation stage.

  • Primary 32B trajectory: 91.51% AAR follows 80.06% after SFT and 52.76% for Wnuan-Base, with gains of 11.45 and 27.30 points, respectively.On validation, the Inst-to-RL transition increases AAR from 76.89% to 89.00% (+12.11 points).
  • Stage-III sampling: 3.11 points separate residual-error from full-pool sampling, while 2.97 points separate it from size-matched random sampling under the common protocol.Residual-error sampling reaches 89.39% AAR versus 86.28% for full-pool and 86.42% for random sampling; source-cluster intervals remain above zero.
  • Stage-III sampling: 81.33% AAR for residual-error sampling exceeds 78.67% for random and 77.78% for full-pool sampling on the same-domain validation set.Residual versus full gains 3.56 points, while residual versus random gains 2.67 points after correction.
  • Training configuration: 31.12 points improve AAR from 52.33% to 83.45% with Document-to-QA training, while target-aligned rewriting retains 82.04% and restores the general average from 79.21% to 82.15%.The Document-to-QA comparison uses 27.8% more estimated FLOPs and is configuration-level evidence.
  • Training configuration: 2.49 points gained on the general-benchmark average accompany a 1.98-point AAR sacrifice at the selected 48.3% replay operating point relative to no replay.The operating point was selected for the highest measured general average, not the highest domain AAR.
  • Error analysis and retrieval: 101 Wnuan-Inst errors are repaired and 20 previously acceptable answers regress after RL, for a net reduction of 81 errors.Remaining failures often involve exact numbers, dates, responsible departments, document names, closed lists, and omitted conditions; retrieval changes AAR from 91.51% to 81.75% for Wnuan-RL under the fixed pipeline.

6 Discussion

The discussion assigns distinct roles to document-to-QA supervision, general-data replay, and residual-error RL, while emphasizing measurable trade-offs and limitations. RL improves domain performance but reduces instruction following, and retrieval should be gated independently rather than assumed additive.

  • Pipeline roles: Document-to-QA supervision organizes enterprise knowledge, general-data replay selects retention, and residual-error GRPO concentrates updates on remaining mistakes.The evidence hierarchy treats Stage I as an unequal-compute study, Stage II as a finite operating-point search, and Stage III as the matched data-selection experiment.
  • Design trade-offs: 27.8% more estimated FLOPs accompanies task-oriented Document-to-QA training’s advantage over fixed-window training, while replay exposes a domain–general trade-off.Target-aligned answer rewriting approximately preserves domain AAR while recovering part of the general-benchmark loss; the selected replay setting is determined by the measured general-benchmark average.
  • Limitations: 91.37% is the AAR after regression-aware continuation versus 91.51% for Wnuan-RL, while hallucination rises from 15.70% to 20.93% and IFEval falls from 80.00% to 76.00%.Without an otherwise identical unbucketed control, the experiment does not isolate the partition rule; it nevertheless shows that another residual-focused continuation is not automatically beneficial.
  • Inference-time intervention: Retrieval remains an inference-time intervention whose value depends on the checkpoint and retrieved context, so it should be gated independently of the training recipe.The discussion rejects treating retrieval as an automatically additive fourth stage.

7 Limitations and Responsible Use

The study’s limitations include restricted evaluation scope, incomplete controlled comparisons, retrieval and reproducibility constraints, and the need for human verification in intended use. The authors disclose AI assistance for editing and consistency checks while retaining responsibility for the submitted material.

  • Evaluation scope: Evaluation covers one enterprise and in-domain data, with no source-, time-, enterprise-, or open-world split.WnuanBench is QA-record-disjoint from training and residual selection but shares the authorized source corpus; its bootstrap intervals quantify uncertainty only within this fixed benchmark.
  • Training evidence: 91.51% and 89.39% are not directly comparable endpoints because their response formats, batch sizes, sequence lengths, hardware, and schedules differ.Stage I is an unequal-budget configuration study, while the residual, full, and random arms share a common 100-update protocol.
  • Training evidence: Only three sampling arms were completed; uncertainty-, loss-, influence-, and online zero-advantage selectors were not tested.Source-cluster intervals provide a sensitivity analysis.
  • Retrieval and reproducibility: The RAG diagnostic uses one retrieval pipeline, lacks gold Recall@5 labels, and uses unmatched generation seeds.These constraints limit retrieval and reproducibility assessment.
  • Retrieval and reproducibility: Private documents and the complete benchmark cannot be released, although the evaluation flow and supplementary reference materials are documented.The supplement provides the correctness prompt, a synthetic fixture, and reference statistics; data construction and training used locally deployed models in a controlled environment.
  • Responsible use: The intended use is internal knowledge assistance with human verification, not automated personnel, compliance, safety, or other high-impact decisions.Generative AI tools supported language editing and consistency checks, while the authors verified the submitted material and take responsibility for it.

8 Conclusion … A.1 Benchmark Composition

Wnuan combines staged supervision, general-data replay, and residual-error reinforcement learning to improve closed-book enterprise QA, while incurring capability-retention costs and remaining limited to represented enterprise knowledge. The appendices document benchmark composition, evaluation conventions, implementation analyses, and validation details.

  • 8 Conclusion: 91.51% AAR follows RL, up from 52.76% before adaptation and 80.06% after SFT on WnuanBench for the primary 32B route.The pipeline combines document-to-QA supervision, general-data replay, and residual-error RL.
  • 8 Conclusion: General-data replay trades some domain accuracy for broader capability retention, while RL reduces IFEval performance.Future work targets instruction-following retention and transfer-oriented or retrieval-aware evaluation.
  • 8 Conclusion: WnuanBench measures mastery of represented enterprise knowledge rather than source-held-out or cross-enterprise transfer.This defines a boundary on the conclusion’s scope.
  • Appendix Overview: Appendices A–B cover WnuanBench and judge calibration, while Appendices C–F describe data construction, training, and stage-wise experiments.Appendices G–I analyze capability retention, business domains, and retrieval; J–L document Wnuan-Plus, unsuccessful RL-2, reproducibility, and responsible use.
  • Appendix Overview: AAR is the primary metric: the fraction of answers labeled correct or partially correct, also named Accuracy in evaluation exports.The paper uses AAR to distinguish this measure from strict full correctness.
  • A.1 Benchmark Composition: 707 questions comprise WnuanBench, including 160 general-knowledge, 370 operational-scenario, and 177 standards/specification questions grounded in authorized enterprise documents.Each record includes a question, reference answer, source identifier, and domain label; a separate taxonomy assigns eight business domains.

A.2 Construction and Data Roles … F.4 Aggregate GRPO Signal Diagnostics

The paper builds disjoint enterprise QA evaluation and training assets, then stages target-aligned construction, replay-based SFT, and residual-error GRPO. Across diagnostics, adaptation improves domain AAR while exposing semantic-evaluation, compute-matching, provenance, and generalization limits.

  • A.2 Construction and Data Roles; A.4 Aggregation and Statistical Tests: WnuanBench is QA-record-disjoint from fitting and development data, with no exact-question overlap against validation data, although 1,479 training rows cover 698 benchmark questions before filtering.After exact removal, median nearest-neighbor cosine similarity is 0.9295, and 458 benchmark questions retain a training neighbor at or above 0.90.
  • A.2 Construction and Data Roles; A.3 Evaluation Dimensions: 76.89% and 89.00% AAR are obtained by Wnuan-Inst and Wnuan-RL on validation, a 12.11-point gain with non-negative AAR changes across all nine domains.AAR is the primary outcome; correctness, completeness, faithfulness, and hallucination provide supporting dimensions, while MMLU, IFEval, and C-Eval guide replay selection.
  • B.1 Sampling and Binary Agreement; B.2 Ordinal and Auxiliary Agreement; B.3 Selection, Training, and Evaluation Judge Roles: 90.5% agreement is achieved between automatic acceptable/unacceptable decisions and an authoritative expert on 147 valid, stratified responses.The estimate corresponds to 133 of 147 responses, with Cohen’s κ = 0.796 and post-stratified agreement of 90.4%; ordinal disagreements mostly occur at adjacent boundaries.
  • C.1 Data Inventory and Provenance; C.2 Reference Implementation; C.3 Answer Rewriting: 164,744 rewritten targets and 56,550 retained originals result from target-aligned answer rewriting, while the final SFT-domain file contains 221,294 rows after consolidation and exact-match filtering.The rewriting gate accepts 164,793 candidate generations, and the controlled evidence does not show an independent domain-AAR gain from knowledge induction.
  • D.1 Supervised Fine-Tuning; D.2 Main and Controlled GRPO Runs; D.3 Retrieval-Augmented Inference: The GRPO setup initializes both RL experiments from Wnuan-Inst, samples five responses per prompt, and combines accuracy, semantic quality, and format rewards.Residual-error sampling is selected after 100 updates using the highest training-side validation accuracy reward; retrieval evaluates Base, Inst, and RL with identical saved contexts.
  • E.1 Stage-I Configurations; E.2 Stage-II Replay Grid: 27.30–34.94 points is the 95% CI for Document-to-QA’s gain over fixed-window training, whereas answer rewriting changes AAR by −1.41 points and replay candidates are not compute-matched.The selected replay setting uses the public-general average with validation as a secondary measure; validation and WnuanBench rankings have Pearson r = 0.12 and Spearman ρ = −0.05.
  • F.1 Data Arms and Endpoints; F.2 Reference-Target Sensitivity Diagnostic: 91.69% of rows retain changed-target membership under the reference-target audit, while the incorrect rate shifts from 25.15% to 24.28%.The audit is a fixed-prediction sensitivity diagnostic: it cannot isolate a causal rewriting effect or replace human semantic-equivalence validation.
  • F.3 Source-Cluster Bootstrap Sensitivity; F.4 Aggregate GRPO Signal Diagnostics: At update 100, residual, full, and random arms change AAR by +5.66, −0.71, and +1.13 points, respectively, with residual-versus-control contrasts remaining above zero under source-cluster resampling.Update-100 AAR intervals are 87.1–91.5 for residual, 83.7–88.8 for full, and 84.0–88.8 for random; the monitor’s mean endpoint discrepancy is 1.19 points.

G Capability Retention and Residual Errors

Capability retention losses are concentrated in instruction following for the 32B route, while the Wnuan-Plus route declines across all three reported public-benchmark components. Residual-error reinforcement learning also shows lower on-policy accuracy reward and larger aggregate update statistics under the matched protocol.

  • General-benchmark retention: 32B public-benchmark decline is concentrated in IFEval rather than shared uniformly across MMLU, IFEval, and C-Eval.Figure 8(a) provides the full trajectory, while Table 21 reports endpoint components for both routes.
  • General-benchmark retention: 7.82 points is the Wnuan-Plus C-Eval decrease after LoRA-SFT, compared with 5.29 points for MMLU and 5.00 points for IFEval.All three Wnuan-Plus components decrease after LoRA-SFT.
  • General-data replay: 48.3% replay maximizes the public-general average and is selected before WnuanBench evaluation.Replay labels are nominal display labels relative to 221,294 domain examples.
  • Residual-error reinforcement learning: The residual arm combines lower on-policy accuracy reward with larger aggregate update statistics under the matched protocol.Figure 6 also reports validation reward components, entropy, and absolute PPO-KL as centered 15-update moving averages for readability.
  • Residual errors: WnuanBench lacks mutually exclusive expert error labels, limiting the interpretation of residual-error categories.This limitation is stated directly in the supplied passage.

H Within-Enterprise Domain Analysis … L Reproducibility and Responsible Use

The extended analyses show that retrieval and domain matching produce strongly stage-dependent AAR changes, while domain-level effects remain limited and the attempted Stage-III continuation does not improve AAR. API results provide contextual rather than controlled comparisons, and several analyses qualify causal and reproducibility claims.

  • H Within-Enterprise Domain Analysis: Only engineering management and departmental responsibilities remain significant after Holm correction among six raw checkpoint-trajectory McNemar tests.The complete descriptive breakdown contains six raw tests below 0.05, but only these two domains survive correction.
  • H Within-Enterprise Domain Analysis: None of the domain-level controlled contrasts is significant, with production technology and sales the two numerical exceptions relative to full-pool sampling.Small slices and ceiling effects make aggregate comparisons primary.
  • L Reproducibility and Responsible Use: The reported analyses identify limitations in domain-slice power, retrieval causal interpretation, unmatched API settings, and the absence of an otherwise identical control for Wnuan-RL-2.The supplied passages do not provide a separate result for Reproducibility and Responsible Use.
  • I.1 Same-Question Comparisons: RAG changes Wnuan-Base from 52.76% to 72.98% AAR, Wnuan-Inst from 80.06% to 76.24%, and Wnuan-RL from 91.51% to 81.75%.Improved/regressed item counts are 198/55, 76/103, and 29/98, with exact McNemar p-values < 0.0001, 0.0517, and < 0.0001.
  • I.1 Same-Question Comparisons: Under common retrieval traces, SFT and RL stage gains remain +3.25 and +5.52 points, smaller than their no-RAG counterparts.The comparison uses one retrieval system and does not randomize retrieval by training.
  • I.3 Post Hoc Retrieval-Trace Slices: In same-domain questions, RAG changes Base, Inst, and RL by +35.80, +6.47, and −2.54 points, versus −4.38, −20.07, and −21.17 points in the proxy-negative slice.A domain match need not contain the answer and may correlate with question difficulty; the association motivates confidence gating but does not establish causality.
  • J API Context and Wnuan-Plus Configuration: The API systems provide same-question context rather than controlled model comparisons because decoding, serving configuration, and compute are unmatched.Wnuan-Plus starts from DeepSeek-V3.1-Terminus and receives LoRA-SFT but no RL.
  • K Unsuccessful Extension Beyond Stage III: Wnuan-RL-2 changes AAR from 91.51% to 91.37%, while hallucination changes from 15.70% to 20.93% and IFEval from 80.00% to 76.00%.AAR is statistically unchanged (−0.14 points, McNemar p = 1); without an identical unbucketed control, the partition rule is not isolated.

L.1 Releaseable Evaluation Capsule … L.4 Intended Use

The paper releases a non-proprietary evaluation capsule and reproducibility materials while keeping enterprise data and provenance controlled. Wnuan is intended for human-verified internal assistance, not high-impact automated decisions, with evidence limited to one enterprise and specified validation and training settings.

  • L.1 Releaseable Evaluation Capsule: The evaluation capsule releases the primary correctness prompt, 0/0.5/1 rubric, JSON contract, supporting-dimension definitions, and statistical evaluation procedures.The capsule also provides fictional fixtures and scripts for reproducing endpoint tables, source-cluster sensitivity analyses, and selected training diagnostics.
  • L.1 Releaseable Evaluation Capsule: Correctness is scored independently by gpt-oss-120b and MiniMax-M2.5, with a DeepSeek-V3.2 tie-breaking vote and median aggregation when ordered labels disagree.Scores are mapped to correct, partially correct, and incorrect, followed by AAR computation, paired bootstrap intervals, McNemar tests, and Holm correction.
  • L.1 Releaseable Evaluation Capsule: The capsule includes planned comparisons, controlled-update records, public-benchmark components, and RAG diagnostics, while noting that some slices are descriptive.Table 19 records matched training controls; Figure 9 marks positive RL-minus-Inst changes and reports no significant controlled domain contrast; Table 21 treats cross-route values as descriptive.
  • L.2 Data Availability: Private source documents and complete WnuanBench cannot be distributed because they are governed enterprise materials.Released materials expose prompts, aggregation, schemas, statistical transformations, configurations, data roles, endpoint counts, paired tests, and snapshot digest prefixes, while row-level provenance remains controlled.
  • L.3 Local Processing and Data Governance: Document processing, QA construction, filtering, target rewriting, SFT, residual selection, and GRPO ran on locally deployed models inside a controlled environment without external provider APIs.No source document or evidence excerpt was transmitted outside that environment.
  • L.4 Intended Use: Wnuan is intended as an internal knowledge assistant whose outputs require human verification, not automated personnel, compliance, safety, or other high-impact decisions.The study uses authorized internal documents under data-minimization and deidentification procedures.
  • L.4 Intended Use: The evidence is limited to one enterprise, one same-domain validation set, one final in-domain benchmark, one completed run per configuration, and the reported retrieval and optimization budgets.These limitations define the scope of the intended-use evidence.
Loading 2608.01862v1…