PaperlayerView exact revision on arXiv

Source-linked AI summary

Salesforce Koa: An Enterprise Language Model for Agentic Tool Use

Zixiang Chen, Sufeng Niu, Yingchi Liu, Wenting Zhao, Akshara Prabhakar, Shubham Mehrotra, Bin Bi, Zhujun Lan, Katherine Tan, Mohammad Ramezanali, Tulika Manoj Awalgaonkar, Monojit Banerjee, Jielin Qiu, Shiva Kumar Pentyala, Zhepeng Cen, Anupam Tripathi, Ali Ziaei, Regunathan Radhakrishnan, Darvish Lee Shadravan, Shelby Heinecke, Sitaram Asur, Jayesh Govindarajan, Silvio Savarese, James Zhu, Phil Mui, Huan Wang

arXiv:2609.15066v2cs.CLcs.AIcs.LG

TL;DR

Enterprise agents need reliable tool use across structured information and multi-step workflows, while organizations seek control over deployment and customization. Salesforce Koa uses specification-driven RL to specialize an open-weight model, reaching 87% Task Success Rate on CRMAgentBench versus GPT-4.1 at 82% and the untuned base at 79%.

  • Problem

    Enterprise agents must retrieve structured information, invoke business APIs, and complete multi-step workflows, while organizations often need more deployment and customization control than one proprietary provider allows.

    Source: §1 · p. 1

  • Method

    Koa post-trains open-weight Nemotron-120B with GRPO using public and synthetic data, expanding declarative Agent Script specifications into persona-conditioned environments with grounded tool-use rewards.

    Source: §1 · p. 1

  • Results

    87% Task Success Rate on CRMAgentBench, versus GPT-4.1 at 82% and the untuned Nemotron base at 79%; Koa also outperforms both on the human-labeled production set and preserves general tool-use capability.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    Production Salesforce specifications yield the in-domain training distribution used to specialize Koa, connecting agent configuration to the model's enterprise training tasks.

    Source: §3.4 · p. 5, §6 · p. 10

    Results are bounded by an already RL-post-trained base and modest task counts in some enterprise evaluations; the RL-over-SFT conclusion may differ from a pre-RL checkpoint.

    Source: §6 · p. 10

Abstract

from arXiv · show

We present Salesforce Koa, an enterprise language model built by post-training the open-weight Nemotron-3-Super-120B foundation model with reinforcement learning using Group Relative Policy Optimization (GRPO), and deployed in FP8 for production. Koa is trained only on public and synthetically generated data, and specialized for the agentic tool use that enterprise workflows demand: routing a request to the correct action, invoking the right tool with valid arguments, and completing multi-turn business tasks. The distinctive component of our pipeline is specification-driven task construction: declarative Agent Script specifications are expanded into persona-conditioned multi-turn environments whose rewards are grounded in successful tool use. Applied to enterprise CRM specifications, the same pipeline produces the in-domain training distribution on which Koa is specialized. On CRMAgentBench and the human-labeled production tool-calling set, Koa outperforms both its untuned open-weight base and GPT-4.1 and is competitive with the strongest frontier models. It reaches 87% Task Success Rate on CRMAgentBench (vs. GPT-4.1 at 82% and the base at 79%), is at or near the top of every metric on the human-labeled portion of an internal production benchmark, and preserves the base model's general capability on public benchmarks (Tau2Bench, BFCL). A controlled comparison with architecture and RL recipe held fixed shows that the additional in-domain RL stage improves argument accuracy and full tool-call success on the human-labeled enterprise benchmark.

1 Introduction

Enterprise agents must handle structured information, external tools, and multi-step workflows, while organizations seek more control than a single proprietary provider allows. Salesforce Koa applies specification-driven RL to this setting, improving enterprise tool use while preserving general capability.

1 Introduction

  • Enterprise agents must retrieve structured information, invoke business APIs, and execute multi-step workflows over organization-specific schemas and policies.Unlike standalone question answering, enterprise tool use can involve overlapping tools and actions that affect live business records.Source: §1 · p. 1
  • Koa post-trains open-weight Nemotron-3-Super-120B with GRPO using public and synthetic data, then serves the model in FP8.Its specification-driven pipeline expands declarative Agent Script specifications into persona-conditioned multi-turn environments with rewards grounded in successful tool use.Source: §1 · pp. 1–2
  • With architecture and RL recipe fixed, additional in-domain Salesforce RL improves argument accuracy and full tool-call success on the human-labeled enterprise benchmark.RL on public and synthetic data yields modest public-benchmark gains; the deployed FP8 model has comparable aggregate benchmark performance.Source: §1 · p. 2
  • Enterprise evaluation leads with CRMAgentBench and production tool calling, while public tool-calling benchmarks check whether general-purpose capability is preserved.This separates changes on the deployment distribution from changes on generalist tasks.Source: §1 · p. 2
  • 87% Task Success Rate on CRMAgentBench, versus GPT-4.1 at 82% and the untuned Nemotron base at 79%; Koa also leads evaluated open-weight baselines on production tool-name accuracy.On the human-labeled production set, it achieves the highest non-hallucination rate of any evaluated model.Source: §1 · p. 2

2 Related Work

Related work establishes open-weight models, tool-use systems, and post-training methods as foundations for enterprise agent adaptation. Koa focuses on RL specialization of an already post-trained open-weight model for enterprise workflows.

Open-weight foundation models

  • Open-weight model families including Llama [7], Gemma [8], Qwen [27], Mistral [11], and Nemotron [13] provide foundations for downstream adaptation and greater deployment and customization control.The paper builds on an open-weight Nemotron foundation model and studies RL for enterprise specialization.Source: §2 · p. 2

Enterprise language models and tool use

  • Toolformer [24], Gorilla [16], and ToolLLM demonstrate tool use, while APIGen [12] and APIGen-MT [18] synthesize verifiable function-calling and multi-turn agent data.Other related efforts include xLAM [31], ActionStudio [30], and ToolLibGen [29].Source: §2 · p. 2
  • Enterprise agents must act over organization-specific schemas and policies, retrieving structured information and executing multi-step workflows rather than answering standalone questions.The paper evaluates RL specialization on both public function-calling benchmarks and enterprise CRM tasks.Source: §2 · p. 3

Post-training for LLMs

  • Prior post-training work spans SFT [15], RLHF and Constitutional AI [3], policy-gradient methods such as PPO [25] and GRPO [26] [9], and preference optimization [22].The paper compares SFT and RL for enterprise adaptation and studies additional training on the deployment distribution.Source: §2 · p. 3

3 Method

Salesforce Koa uses declarative agent specifications to construct training environments and grounded rewards, then applies GRPO across public/synthetic and in-domain Salesforce regimes. The deployed checkpoint uses in-domain-calibrated FP8 quantization.

3.1 Spec-driven environment and task construction

  • Declarative Agent Script specifications define enterprise workflows, while public tool-use domains use synthesized structures; both become typed workflow graphs for shared simulation and reward machinery.The graph captures tool argument schemas, state effects, routing conditions, and loop or termination configuration.Source: §3.1 · p. 3
  • Scenario- and persona-conditioned sessions become RL examples with opening requests or generated dialogue prefixes, and NeMo Gym executes their trajectories online.Examples carry the environment reference and initial rollout context, separating workflow/task construction from rollout execution and policy optimization.Source: §3.1 · pp. 3–4

3.2 Online rollout and grounded reward

  • A frozen helper model simulates customers, emulates tools, and judges coverage; the reward uses resolved intent sub-questions and gates consecutive duplicate calls.For customer- or system-specific answers, the judge requires grounding in a successful relevant tool call; the deterministic sandbox instead rewards reproducing the reference end state.Source: §3.2 · p. 4

3.3 Optimization

  • GRPO samples groups of complete trajectories, uses a leave-one-out group baseline without a value network, and applies token-level truncated importance sampling.Dynamic sampling discards zero-variance groups; malformed tool-call or thinking syntax receives fixed negative token advantages, while over-length or inconsistent trajectories are masked.Source: §3.3 · pp. 4–5

3.4 Training-data regimes

  • Koa first trains on public and synthetic multi-domain environments, then continues on production Salesforce specifications and schemas with the RL recipe held fixed.The added in-domain stage produces the deployed checkpoint and matches its synthesized sessions to the production distribution's tools, argument conventions, and workflows.Source: §3.4 · p. 5

3.5 FP8 quantization for deployment

  • The deployed checkpoint uses post-training quantization with in-domain calibration data drawn from the production distribution for FP8 inference.The paper reports Koa (BF16) as the unquantized checkpoint and Koa (FP8) as the deployed model.Source: §3.5 · p. 5

4 Experiments

Experiments compare Koa with open and proprietary baselines, leading with enterprise benchmarks and using public benchmarks to check whether general tool-use capability is preserved.

4.1 Setup

  • The evaluation compares Koa with GPT-4.1, GPT-5.5, and its untuned base; production Koa is served in FP8, while comparison checkpoints are generally BF16.GPT-4.1 and the untuned base are identified as the main references for isolating post-training effects.Source: §4.1 · pp. 5–6

4.2 Enterprise benchmark results

  • Enterprise evaluation spans multi-turn task success, production tool calling with human-labeled and synthetic splits, and single-turn CRM assistant behavior.CRMAgentBench pools 97 tasks across five CRM domains and uses strict all-or-nothing TSR checks.Source: §4.2 · p. 6, §4.2.1 · p. 6

4.2.1 CRMAgentBench

  • Koa reaches 87% overall TSR on CRMAgentBench, above GPT-4.1 (82%) and its untuned base (79%), and matching Nemotron-Ultra-550B.GPT-5.5 and Gemini 3.7 Flash score 91%.Source: §4.2.1 · p. 6, Table 1 · p. 6

4.2.2 Production tool-calling benchmark

  • On the human-labeled production set, Koa beats GPT-4.1 on all four metrics, leads NoHall at 0.989, and scores 0.851/0.848 on Args/Succ; frontier models lead synthetic-set success.Koa has higher tool-name accuracy than all evaluated open-weight baselines, at 0.966.Source: §4.2.2 · p. 7, Table 2 · p. 7

4.3 Public benchmark results

  • Tau2Bench task-weighted average is 69.33, above GPT-4.1 (54.48) and the base (68.64); BFCL is 63.87%, above GPT-4.1 (53.96%) and near the base (64.73%).Koa trails GPT-5.5 on both overall averages; BFCL category movements are small and offsetting, with no category collapse.Source: Table 4 · p. 8, §4.3 · p. 8, Table 5 · p. 8
  • On CRM Bench, Koa scores 0.87 overall, above GPT-4.1 (0.81) and the base (0.84); its 0.85 function-call accuracy ties GPT-4.1 and exceeds the base (0.71).Across the three benchmarks, the paper concludes that enterprise specialization preserves general tool-use capability.Source: §4.3 · p. 8

5 Analysis

The analysis isolates how in-domain RL, the choice between SFT and RL, and FP8 deployment affect enterprise and general tool-use performance. It finds enterprise gains alongside trade-offs on public benchmarks and across FP8 categories.

5.1 Effect of in-domain adaptation

  • 0.825/0.825 to 0.842/0.842: the added in-domain RL stage improves human-labeled enterprise success and argument accuracy; Koa reaches 0.848/0.851.The comparison holds the base and RL recipe fixed, with the additional in-domain stage as the only difference.Source: §5.1 · pp. 8–9

5.2 SFT versus RL from an already post-trained base

  • 59.50: RL improves BFCL multi-turn tool-use accuracy, while SFT lowers it from 54.12 to 53.25; SFT instead leads modestly on Tau2Bench and CRM Bench.Tau2Bench scores are SFT 70.04, RL 69.41, and base 68.64; CRM Bench scores are SFT 0.88 and RL 0.86, from a base of 0.84.Source: §5.2 · p. 9

5.3 General tool use across our checkpoints and FP8 deployment

  • 66.63: public/synthetic RL raises BFCL overall accuracy above the base’s 64.73, while the in-domain stage brings it back near the base at 64.31.The public/synthetic stage improves multi-turn tool use from 54.12 to 59.50 and web search from 71.50 to 73.50; the in-domain stage lowers web search to 67.00.Source: §5.3 · p. 9, Table 7 · p. 9
  • FP8 retains similar aggregate results: Tau2Bench changes from 69.70 to 69.33, CRM Bench rises from 0.86 to 0.87, and BFCL moves from 64.31 to 63.87.On BFCL, lower Multi-Turn and Memory scores are offset by gains in Web Search and relevance detection; on the production benchmark, calibrated FP8 is at or above BF16 on every human-labeled metric.Source: §5.3 · pp. 9–10, Table 7 · p. 9

6 Conclusion

Salesforce Koa is an FP8-deployed Nemotron-120B model trained with GRPO and a specification-driven RL pipeline. The paper reports enterprise benchmark gains from in-domain RL, while noting limits to the evaluation and to generalizing the RL-over-SFT comparison.

6 Conclusion

  • Koa combines Nemotron-120B post-training with GRPO and FP8 deployment; its specification-driven pipeline uses Agent Script specifications to structure rollouts and grounded rewards.Applied to enterprise CRM specifications, the pipeline produces the in-domain distribution used to specialize Koa.Source: §6 · p. 10
  • With architecture and RL recipe held fixed, an additional in-domain RL stage improves argument accuracy and full tool-call success on the human-labeled enterprise benchmark.Table 8 summarizes environment interaction and verification mechanisms.Source: §6 · p. 10, Table 8 · p. 14, §A.1 · p. 14
  • The RL-over-SFT preference may not hold from a pre-RL checkpoint, and some enterprise evaluations use modest task counts.Source: §6 · p. 10

A.2 Training rollout

Figure 1 traces a training rollout and distinguishes the trained policy from frozen helpers. The reward comes from a coverage judge at the end.

A.2 Training rollout

  • The policy is the only trained component in the rollout.Source: Figure 1 · p. 14
  • Solid arrows mark the trained or rollout path, while dashed arrows mark frozen-helper responses.Source: Figure 1 · p. 14
  • A coverage judge produces the reward at the end of the rollout.Source: Figure 1 · p. 14

B Training and evaluation details

The training details describe reward-maximizing GRPO updates from sampled trajectories and a separate mode for constrained decision turns. Validity rules mask or penalize malformed and unusable trajectories.

B.1 GRPO optimization

  • The objective maximizes expected trajectory reward under the policy and environment.Source: §B.1 · p. 15
  • Each prompt yields a group of complete trajectories; zero-reward-variance groups are discarded and resampled, while advantages compare each trajectory with the others.Advantages use leave-one-out reward statistics, with clipping when the comparison standard deviation is zero.Source: §B.1 · p. 15
  • Training performs one update per rollout batch, using truncated token-level importance weights to correct the train/generation mismatch.The policy ratio is forced to one, with no clipped-ratio term or reference-KL penalty.Source: §B.1 · p. 15
  • Malformed tool-call or thinking syntax receives a fixed negative token advantage; over-length and log-probability-inconsistent trajectories are masked.Source: §B.1 · p. 15

B.2 Constrained decision turns

  • For constrained decision turns, a needed tool is withheld so the agent must communicate or recover later; a procedural gate penalizes premature state-changing calls.The single-step reward also scores whether the withheld tool is correctly emitted at a later recovery turn.Source: §B.2 · p. 15

B.3 Distributed training and offline evaluation

Training alternates rollout generation and optimization on distributed actor workers, with a separate frozen environment helper. Offline evaluation independently scores tool-name, argument, and full-call correctness.

B.3 Distributed training and offline evaluation

  • Rollout generation and optimization alternate on actor workers distributed with TP=8, EP=8, and PP=1.Single-env and multi-env GRPO use four actor nodes plus one dedicated helper node.Source: §B.3 · p. 16
  • The frozen environment helper runs on a separate inference engine to avoid a coordination failure observed when split across replicas.Source: §B.3 · p. 16
  • Offline checkpoints use greedy decoding and schema-aware parsing; evaluation scores tool-name, argument, and full-call correctness, with an independent LLM judge for argument equivalence.Training-time validation instead replays held-out environment examples and reports aggregate and per-environment reward.Source: §B.3 · p. 16

C Additional post-training experiments

The preliminary study tested conservative supervised fine-tuning of Nemotron on synthetic function-calling data. Its essentially unchanged BFCL multi-turn score motivated applying RL directly to the base model.

C.1 Preliminary SFT setup

  • SFT used a masked next-token loss restricted to assistant and tool-call target tokens, with a fully synthetic corpus built from public function-calling resources.Schemas and executable environments came from BFCL and Tau2Bench Airline tool collections.Source: §C.1 · p. 16
  • 54.1% → 53.2%: the SFT-only checkpoint’s BFCL multi-turn score was essentially unchanged, motivating RL applied directly to the base model.The comparison is reported in Table 10.Source: §C.1 · p. 16
  • The study full-parameter fine-tuned Nemotron-120B over 32 H200 GPUs using a conservative recipe intended to avoid catastrophic forgetting.Training used sequence length 8192, global batch size 32, a 5×10^-8 peak learning rate, cosine decay, gradient clipping, and early stopping.Source: §C.1 · p. 16

C.2 Post-training and precision comparisons

The appendix tables report results across public benchmarks and internal CRM, with checkpoint lineage distinguishing SFT-only, public/synthetic RL, enterprise-data RL, and FP8 deployment. Their captions specify benchmark scope and evaluation precision.

C.2 Post-training and precision comparisons

  • Koa (BF16) continues the public/synthetic RL checkpoint with enterprise-data RL, while Koa (FP8) is its quantized deployment version.The SFT-only and public/synthetic RL checkpoints were trained separately from the same Nemotron-3-Super-120B base.Source: §C.2 · p. 16
  • Tau2Bench results cover customer-service domains, and its weighted average accounts for the number of tasks per domain.The table evaluates checkpoints in BF16 except Salesforce Koa (FP8).Source: Table 9 · p. 16
  • The BFCL comparison includes the SFT-only checkpoint alongside the RL-pipeline checkpoints; all are evaluated in BF16 except Salesforce Koa (FP8).The table provides post-training and precision comparisons.Source: §C.2 · p. 16, Table 10 · p. 17
  • The internal CRM benchmark table reports checkpoint results, with all checkpoints in BF16 except Salesforce Koa (FP8).Source: Table 11 · p. 17
Loading 2609.15066v2…