Source-linked AI summary
Decision Model Factory: Building Specialized Sovereign Decision Models
Yuma Ichikawa
TL;DR
Decision applications need task- and deployment-specific models, while free-form generation and repeated processing of shared evidence add unnecessary work. The paper combines a recipe-driven factory with a decision-specific runner; across five models, specialization improves decision quality, and the runner improves serving efficiency.
Problem
Method
Results
Across five models, specialization improves decision quality; DF-Long outperforms its 31-billion-parameter backbone on Japanese tasks, while compact distilled models gain more than 20 points.
Source: §5.3 · p. 5, §7 · p. 6Takeaways & Limitations
The runner makes specialized models practical to deploy, with shared-prefix execution especially effective for multiple questions about the same evidence.
Source: §5.3 · p. 5
Abstract
from arXiv · showhide
Decision needs vary with each application, language, and deployment environment. The Decision Model Factory turns those needs into specialized, sovereign models on demand. A brief recipe guides the training, calibration, verification, and evaluation of a selected backbone, delivering a deployable decision model under the operator's control. The same workflow creates a new specialist whenever the requirements change. Across Japanese decision tasks, compact deployment, and long-context passage ranking, five factory-built models improve upon their backbones; three surpass the strongest results in a fixed public Japanese benchmark comparison. A dedicated runner computes only permitted-answer probabilities and reuses shared context, more than doubling serving speed on matched multi-question workloads with essentially unchanged decision scores. Together, the factory and runner make fast, task-specific decision models practical to build and deploy whenever needed.
1 Introduction
Applications need decision models tailored to changing tasks, languages, and deployment constraints, but repeating separate training and deployment projects is difficult to sustain. The Decision Model Factory turns specialization into a reproducible build, demonstrated with five models spanning Japanese, compact, and long-context needs.
1 Introduction
- Decision models return probabilities over permitted outcomes in one forward pass, unlike free-form generation that adds decoding and parsing.Applications can act on confident predictions and refer uncertain cases for review.Source: §1 · p. 1
- Applications vary in language, policy, answer labels, document length, and serving budget, while operators may need control of training data, model weights, and deployment.The paper calls a model sovereign when the operator can build and serve it under its own control.Source: §1 · p. 2
- The Factory runs a recipe-driven workflow for training, calibration, export verification, and evaluation, producing a package for GPU or CPU serving.When requirements change, the recipe changes rather than the pipeline.Source: §1 · p. 2, Figure 2 · p. 2
- Three factory-built models surpass the strongest score in the fixed public Japanese comparison, across a five-model family of Japanese, compact, and long-context specialists.Table 1 covers Japanese Task Avg before and after specialization.Source: §1 · p. 2, Table 1 · p. 2
2 From a Language Model to a Decision Model
The paper converts a language model into a decision model by scoring only permitted answer options and calibrating their probabilities. It adapts the backbone with low-rank updates, then selects and verifies the exported model.
2 From a Language Model to a Decision Model
- A decision request pairs a state with a task question and K possible answers, categorized as Noul, Choice, or Score.A fixed template presents the state first and assigns each answer a single-token code.Source: §2 · p. 2
- The option-based readout produces calibrated probabilities over allowed answers, reusing the backbone tokenizer and output layer without an additional classification head.For Score answers, the model also reports the expected value over the ordered ratings.Source: §2 · p. 3
- LoRA keeps selected backbone weights frozen and learns a low-rank update; training fits a target distribution that may mix ground-truth labels with a teacher distribution.For Score tasks, the loss additionally penalizes expected-value errors with weight λ.Source: §2 · p. 3
- The Factory evaluates candidate update strengths on held-out decisions, fits a separate calibration temperature for each answer type, and verifies the exported readout.Export verification checks that the package reproduces the selected readout within a predefined tolerance.Source: §2 · p. 3
3 Decision Model Factory
The Factory uses a common, traceable build workflow for model variants with distinct recipes. Its packages include the weights and serving components needed for deployment.
3 Decision Model Factory
- Each recipe specifies a backbone, decision dataset, optional teacher, adapter configuration, and deployment formats; preparation validates answer types and sets aside development examples.Training learns the low-rank update, followed by update-strength selection and calibration.Source: §3 · p. 3
- Export merges the selected update into the backbone and verifies consistent training and serving readouts; MoE verification runs separately for each expert.The final package contains model weights, prompt template, option codes, and calibration temperatures.Source: §3 · p. 3
- The pipeline can run unattended, resume after interruption, and trace results to a build through recorded recipes, code, data hashes, and selected configurations.Japanese-focused, distilled compact, and long-context variants use distinct recipes but share the build pipeline and serving interface.Source: §3 · p. 3
4 A Purpose-Built Decision Runner
The decision runner targets wasted computation from full-vocabulary output and repeated processing of shared evidence. It computes only permitted-answer probabilities and reuses state across questions.
4 A Purpose-Built Decision Runner
- The runner addresses two sources of redundant work: full-vocabulary logits for small answer sets and independent processing of questions about the same state.The CPU implementation also accesses only the output-matrix rows required for permitted answers.Source: §4 · pp. 3–4
- The runner projects onto output rows for allowed answers rather than the full vocabulary, so output-projection cost scales with answer count while probabilities remain equivalent up to floating-point rounding.It applies the backbone output transformation and exported calibration temperatures.Source: §4 · p. 4
- For questions sharing a state, the runner computes attention keys and values once and reuses the cached prefix across questions.For the MoE backbone, it captures prompt processing in a GPU execution graph to reduce kernel-launch overhead.Source: §4 · p. 4
5 Experiments
Across the evaluated Japanese benchmarks, specialization improved Task Avg for every backbone, and three models exceeded the strongest score in the fixed public comparison. DF-Long also performed well on passage ranking and long-context tasks, while the decision runner reduced multi-question serving latency with little change in decision scores.
5.1 Evaluation Setting
- The evaluation covered 8,067 Japanese S1MB judgments across 27 benchmarks and 26,269 English judgments across 137 benchmarks, with task-specific trivial-baseline adjustment and equally weighted answer-type averages.The comparison used matched backbone readouts and 8-bit packages, plus a fixed snapshot of 34 complete public evaluations retrieved on 18 October 2026; serving measurements used a B200 GPU or 16 Xeon CPU cores.Source: §5.1 · p. 4
5.2 Results
- Specialization consistently improves Japanese Task Avg across all backbones; reported scores of 67.63, 66.98, and 63.57 each surpass the fixed public snapshot’s highest score of 60.05.DF-Small and DF-JA-Small rose from 31.97 to 52.96 and 52.49, respectively, both exceeding Saihai 8B’s published 49.77. English scores improved for DF-Long from 52.14 to 63.55 and DF-Small from 36.49 to 47.30.Source: §5.2 · p. 4
- DF-Long outscored both retrieval baselines on all three Japanese passage-ranking benchmarks and ignored 899 of 900 embedded instructions across three languages.It supports inputs up to 262,144 tokens, had an ECE of 0.024 on development decisions used for temperature fitting, and answered a question about a 27K-token document in 0.41 s on the evaluated GPU.Source: §5.2 · p. 4, §5.1 · p. 4, Table 2 · p. 5
- For eight questions sharing a state, runner latency fell from 71.7 ms to 29.4 ms, a 2.44× speedup; eight independent states saw a 2.23× speedup.With four concurrent states and eight questions per state, throughput reached 570 decisions per second.Source: §5.2 · p. 5, Table 3 · p. 5
- DF-Long’s eight-question batch took approximately 28 ms, about 40% faster than DF-Large; tuned throughput reached approximately 2,328 decisions/s for E4B and 380 decisions/s for DF-Large on one GPU.On 16 CPU cores, E4B achieved 11.2 decisions per second. Using identical DF-Long BF16 weights, runner and standard-path scores differed by at most 0.2 points across S1MB English, S1MB Japanese, and JQaRA.Source: §5.2 · p. 5
5.3 Discussion
- DF-Long outperformed its unspecialized 31-billion-parameter backbone on Japanese decision tasks while activating about 4 billion parameters per token; compact distilled models gained more than 20 points without increasing backbone size.Long-context and relevance-judgment data extended DF-Long’s supported applications, and shared-prefix execution and option-only output address repeated evidence processing and unnecessary vocabulary computation, respectively.Source: §5.3 · p. 5
6 Related Work
Prior work addresses reusable model-development components and probabilistic decision interfaces, including structured answer scoring and open decision models. The Decision Model Factory brings reproducible specialization, verification, and deployment together around a shared decision interface.
6 Related Work
- Model factories aim to streamline development through reusable components and versioned experiments.Source: §6 · p. 5
- Prior decision-model work includes a probability-based interface for predefined answers, structured answer-suffix scoring [20], and option-token logits combined with low-rank fine-tuning [5].Source: §6 · pp. 5–6
- For Japanese decision tasks, Tateno presents a specialized model and benchmark evaluation; this paper describes a unified workflow for reproducible specialization, verification, and efficient deployment.Source: §6 · p. 6
7 Conclusion and Outlook
The Decision Model Factory provides an operator-controlled workflow for specialized decision models as requirements evolve. Five variants demonstrate support for Japanese-focused, distilled, and long-context specializations.
7 Conclusion and Outlook
- Five DF models show that one workflow supports Japanese-focused, distilled, and long-context specializations without separate training or serving systems.The recipe-based workflow is presented as a path toward additional tasks and languages while keeping training data, weights, and deployment under operator control.Source: §7 · p. 6
- Improved decision quality and the dedicated runner’s efficiency gains demonstrate the practical value of combining model specialization with decision-specific inference.The runner serves multiple judgments over shared evidence more than twice as fast as the general-purpose path in matched workloads.Source: §7 · p. 6
- Integrating PHOTON is future work that could support more memory-efficient processing of many parallel judgments over large shared documents.The paper leaves this integration for future work.Source: §7 · p. 6
A Main Hyperparameters
The appendix reports shared training settings and implementation details for the evaluated Gemma 4 backbones and runner.
A Main Hyperparameters
- All models train one-epoch LoRA adapters on attention and feed-forward projections, use 5% warm-up, fix Score loss weight at λ = 1, and test Σ = {0.5, 0.75, 1}.DF-Long’s paired hyperparameter values correspond to its initial and long-input training stages; Gemma 4 26B-A4B training includes fused expert weights.Source: A Main Hyperparameters · p. 9
- 17 of 18 DF-Long BF16 output-projection comparisons are bit-identical; the remaining differs by one rounding step, with probability differences below 2 × 10−8.Evaluated Gemma 4 backbones use logit soft-capping with c = 30, which the runner applies to selected option logits.Source: A Main Hyperparameters · p. 9
- For Gemma 4 E4B, the final 18 layers reuse earlier attention states, enabling the prompt-position optimization.Source: A Main Hyperparameters · p. 9