Source-linked AI summary
How to Train a Real-World Silicon Concierge? Internalizing Complex Business Workflow to Only OneModel
Chang Liu, Chaoyang Ning, Dayi Jiang, Enrui Gu, Fang Ran, Hongyan Xue, Huaqing Li, Hui Cai, Jia Liu, Jiang-Ming Yang, Jianshe Li, Jiawei Luo, Jin Zhou, Leshen Zhu, Lihui Chen, Liying Ma, Lyuxin Xue, Mengjian Ji, Ruijia Xu, Wei Ren, Wei Wu, Xiaoling Qu, Xiaoyun Feng, Xin Zhang, Xixie Zhou, Xuanwei Hu, Yan Chen, Yichao Wang, Yongqi Tong, Yu Liu, Yuhong Zhou, Zemin Sun, Zhenwen Xu, Zhiling Liu, Zifan Wang
TL;DR
Existing industrial agents rely on fragmented modular workflows that introduce cascading errors, latency, and difficulty leveraging fluid model capabilities. OneModel internalizes domain knowledge and business procedures through hierarchical training, including CPT and logic-compilation SFT, within a unified attention space. In deployment, it reduced latency by 57.2% while raising IRR from 64.3% to 83.3%.
Problem
Modular industrial workflows fragment business queries into patched serialized stages, limiting emergent-model use and complicating error attribution.
Method
OneModel internalizes domain knowledge and business logic through hierarchical knowledge management, CPT, SFT logic compilation, and unified attention.
Results
57.2% latency reduction accompanies an increase in Overall Intelligent Resolution Rate from 64.3% to 83.3% in real-world deployment.
Takeaways & Limitations
OneModel demonstrates a deployed single-model approach for replacing fragmented workflow engineering with internalized financial SOPs and domain knowledge.
Takeaways & Limitations
The user simulator’s 95.8% human-likeness contrasted with a 0.0446 Kappa coefficient against real-user satisfaction behavior, limiting behavioral fidelity.
Abstract
from arXiv · showhide
Traditional industrial agents rely on modular pipelines, including Router, Retriever, Planner, Executor, Responder, Reviewer, and other components. These systems often fracture into a labyrinth of ad-hoc patches, leading to cascading errors and high latency. We propose OneModel, an applicable paradigm shift from external workflows to internalized knowledge representation. Unlike modular systems that slice fluid user intents into static steps, OneModel consolidates complex business logic and SOPs directly into the model parameters. Through Continual Pre-training (CPT) and logic-compilation SFT, we transform fragmented business rules into intuitive model reasoning within a unified attention space. Deployed in our global financial service system, OneModel effectively breaks the trade-off between latency, accuracy, and complexity. Online A/B testing demonstrates an end-to-end latency reduction of more than 50 percent, from 18.7 seconds to 8.0 seconds, while the Intelligent Resolution Rate (IRR) increases from 64.3 percent to 83.3 percent. The results show that OneModel can replace brittle engineering logic with internalized cognitive intuition, offering a scalable blueprint for transitioning industrial agents from complex, error-prone workflows to unified model architectures.
1 Introduction
Traditional industrial agents decompose complex service queries into serialized modular workflows that accumulate patches, cascading errors, and latency. ONEMODEL instead internalizes business knowledge and consolidates interaction intelligence within a unified attention space.
- Traditional PEER-style workflows route, retrieve, plan, execute, express, and review queries through serialized modules.
- Modular fragmentation creates cascading errors, difficult error attribution, and limited use of emergent foundational-model capabilities.
- ONEMODEL shifts from external workflow orchestration to internalized knowledge by consolidating intent reasoning, tool usage, and SOP adherence in one attention space.
- CPT establishes static domain principles, while SFT compiles semi-static business logic and response scripts into model reasoning.
2 Real-World Applications
ONEMODEL is deployed in a large-scale financial merchant service setting using hierarchical knowledge internalization and multi-stage training. The system combines knowledge injection, reflection, logic compilation, and reinforcement learning with real-world evaluation and operational infrastructure.
- 2 Real-World Applications: Online A/B testing evaluates latency, resolution rate, and interaction quality against a standard PEER modular baseline.
- 2 Real-World Applications: 90.75% peak resolution rate is achieved by the 8B ONEMODEL, exceeding Claude-3.5-Haiku, Gemini-2.5-Pro, and GPT-5.2 baselines.
- 2 Real-World Applications: 83.3% Overall Intelligent Resolution Rate rises from 64.3%, while latency falls by 57.2% through the unified architecture.
- Knowledge Management: Hierarchical knowledge management separates static axioms, semi-static procedural logic, and dynamic transactional context according to volatility and complexity.
- Knowledge Injection and Reflection: Knowledge Injection builds parametric foundations from manuals, FAQs, online Q&A, and real-world exceptions.
- Knowledge Injection and Reflection: Knowledge Reflection activates dormant parametric memory through scenario probes, synthesized CPT data, self-reflection rationales, and compiled SOP reasoning.
- Reinforcement Learning: RL addresses long-tail failures, advanced function-calling reasoning, robotic phrasing, and multi-turn compliance through simulated interactions.
4 Experiments
The experiments evaluate staged knowledge internalization, business-logic compilation, and reinforcement learning across exams, held-out sets, human-likeness, and trajectory-level grounding. Results report strong parameter-only performance, knowledge-reflection gains, and robust RL improvements across model scales.
- Phase I: Domain Knowledge Injection: 34.9% of the CPT sampling distribution is core internal knowledge, while domain knowledge and FAQs each represent 25.6%.Structured QA pairs form the smallest segment at 4.7%; these are training-time sampling ratios, not raw dataset composition.
- Phase I: Domain Knowledge Injection: 68.28 KI for ONEMODEL approaches the human professional score of 70.00, whereas SFT alone reaches 60.35.The result is reported for Qwen3-80B-A3B in a strictly parameter-only setting without a notebook.
- Phase I: Domain Knowledge Injection: 90.15 in-domain KI (+32.79) follows from adding KR to Qwen3-80B-A3B using only 72 original seed kernels, exceeding the human open-book benchmark of 89.36.The study also reports broader generalization from the KR mechanism.
- Phase I: Domain Knowledge Injection: 75.71 is the 80B model’s out-of-domain score, with higher fidelity and transferability than smaller models despite the 8B model’s larger raw in-domain gain.The authors associate this pattern with weakly supervised distillation projecting sparse knowledge kernels into generalized reasoning space.
- Phase II: Business Logic Compilation: 91.96 is achieved by the 8B integrated ONEMODEL in a parameter-only setting, improving over the 49.13 baseline and exceeding the cited professional-human comparisons.The final evaluation covers the SSQ Exam and held-out Closed Sets, and the paper attributes the gain to combining Phase I and Phase II.
- Phase III: Alignment and Refinement: 95.05 KGA for 8B and 96.13 KGA for 32B follow RL, up from 81.41 and 85.11 respectively.Both RL variants exceed 95 KGA, while scaling contributes a smaller post-RL gap than for the base models.
5 Conclusions
ONEMODEL shifts industrial agents from brittle, fragmented workflows to internalized cognitive intuition by consolidating financial SOPs and domain axioms into a unified attention space. Its real-world deployment reports lower latency, higher resolution rates, and improved reasoning on complex cases.
- ONEMODEL shifts industrial agent architecture from brittle, fragmented workflows to internalized cognitive intuition.
- 57.2% latency reduction coincides with an 83.3% Intent Resolution Rate and doubled reasoning performance on complex cases.
- A progressive multi-stage pipeline consolidates complex financial SOPs and domain axioms into a unified attention space.
Limitations and Future Work
The paper frames ONEMODEL’s transition as producing non-trivial insights and highlighting tensions between academic metrics and industrial viability in high-stakes financial environments.
- ONEMODEL’s transition from a modular pipeline yielded several non-trivial insights.
- The reported lessons expose tensions between academic metrics and industrial viability in high-stakes financial environments.
1. Style Fidelity̸ = Behavioral Fidelity (The Turing Trap)
The user simulator’s linguistic human-likeness did not translate into realistic financial-customer behavior. Real-user behavioral alignment was near random, revealing a gap between style fidelity and behavioral fidelity.
- 95.8% Human-likeness contrasted with a 0.0446 Kappa coefficient between simulator satisfaction and real-user behavior.
- The simulator underrepresented irrational persistence and escalation, with Real Transfer Rate ≈10% versus Simulator 1.7%.
- Robust RL training requires explicit negative behaviors, including impatience, complaints, and demands for human intervention.
2. Reward Modeling is Data Cleaning, Not Just Training
The RL phase exposed human-annotation inconsistency as a bottleneck, motivating a shift from reward-model training toward an iterative data-cleaning flywheel.
- Noisy human labels caused reward-model training to degrade policy quality.
- An iterative reward model filtered and corrected its own training data by identifying bad cases in human annotations.
- A weaker model trained on rigorously cleaned, model-assisted data consistently outperformed larger models trained on raw annotations.
3. Global Optimization Over Modular Debuggability
ONEMODEL prioritizes globally coherent reasoning over modular white-box debuggability. Its knowledge-governance strategy assigns different business knowledge types to different training methods.
- Global optimization over modular debuggability: ONEMODEL sacrifices white-box debugging for performance by consolidating reasoning within a unified attention space.Modular handoffs caused semantic discontinuities and logic jumps that individual modules could not resolve.
- Global optimization over modular debuggability: Local improvements to individual Routers or Retrievers can produce global sub-optimization.The paper links this limitation to the lack of synchronized evolution among understanding, reasoning, and expression.
- Knowledge governance: CPT is reserved for immutable Domain Axioms, while semi-static Procedural Logic is delegated to SFT.This hierarchy addresses the risk that volatile business parameters become difficult to update without catastrophic forgetting.
5. The Helpfulness vs. Compliance Conflict
Financial-service agents must balance user helpfulness with regulatory adherence. The paper proposes adversarial training, process supervision, and surgical editing as responses to compliance, reasoning, and knowledge-volatility challenges.
- The helpfulness vs. compliance conflict: Standard RL can drift toward sycophancy, so financial agents require principled refusal and compliance violations as a hard veto.The paper describes excessive helpfulness, such as bypassing KYC, as a severe compliance risk.
- Future directions: Adversarial self-play will reward user simulators for inducing compliance violations or exposing logic loops.The intended result is greater resilience against irrational persistence and better coverage of rare corner cases.
- Future directions: Fine-grained Process Reward Models will score intermediate logical transitions and tool invocations in Chain-of-SOP reasoning.The proposed dense rewards target structured and interpretable reasoning without relying only on final outcomes.
- Future directions: ROME and parametric memory separation are proposed to update volatile facts without causing catastrophic forgetting of broader dialogue policies.Target examples include daily exchange-rate spreads and flash campaign dates.
A Related Work
The paper situates OneModel within work on hierarchical business-knowledge injection, user simulation, reinforcement learning, and production serving infrastructure. These strands address knowledge organization, realistic interaction, behavioral optimization, and deployment efficiency.
- Business knowledge management and injection: OneModel organizes business intelligence into Fundamental Domain Axioms, Procedural Business Logic, and Volatile Transactional Knowledge.The hierarchy is presented as a way to balance parameter rigidity with operational fluidity.
- User simulation: LLM-based user simulators use prompting, fine-tuning, or persona constraints, while task-oriented environments impose stricter trajectory-alignment requirements.τ 2-Bench couples user actions with environmental tools to preserve state adherence.
- Reinforcement learning: Reinforcement learning expands interactive capabilities by exploring trajectories beyond the initial training distribution.The related-work passage frames RL as a mechanism for discovering sophisticated behaviors and problem-solving strategies.
- Deployment infrastructure: Production deployment uses NVIDIA H200 GPUs, vLLM serving, an OpenAI-compatible API, and monitoring with dynamic autoscaling.These components support high-throughput, low-latency inference and adaptation to diurnal traffic patterns.
D CPT Data Construction and Ablation for Phase I: Knowledge Injection
Phase I constructs a high-density financial corpus and injects its domain knowledge through targeted sampling and training. Ablation results indicate that this targeted injection substantially improves zero-shot performance on core business exams.
- CPT data construction: The corpus comprises approximately 60% Core Internal Knowledge, ∼15% Customer-Facing Knowledge, and ∼25% Structured QA Pairs.The three tiers cover internal axioms, user-accessible procedures, and interrogative formats.
- CPT data construction: Dynamic up-sampling heavily weights scarce internal training manuals to reinforce immutable domain rules.The manuals serve as ground truth for business axioms and implicit logic.
- Ablation for Phase I: Over 20 percentage points of zero-shot accuracy improvement raised core business exam performance from ∼62% to >82%.The ablation attributes this gain to targeted knowledge injection and high-frequency repetition of dense domain material.
- CPT data construction: Data cleaning removes HTML tags and visual noise while document chunking preserves semantic structure.The corpus combines internal manuals, customer-facing documentation, and RAG-retrieved FAQ pairs.