PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers

Ziyi Chen, Yan Zhang, Jianhui Wei, Jianing Yin, Daoan Zhang

arXiv:2610.15501v1cs.AI

TL;DR

Autoregressive decisions incur decoding costs and do not explicitly expose decision probabilities, while task-specific classifiers are tied to predefined labels. The paper trains single-pass Large Decision Models with diverse schema supervision and reasoning-derived targets; accuracy improves over a step-matched baseline, and confidence routing can selectively retain reasoning rollout.

  • Problem

    Autoregressive rollout incurs latency and computational costs without explicitly exposing decision probabilities, while task-specific classifiers tie output heads to predefined labels.

    Source: §1 · p. 1

  • Method

    Large Decision Models use one-pass readout over run-time labels, trained with diverse schema supervision and reasoning-teacher answer distributions, with routing to reasoning for uncertain cases.

    Source: §1 · pp. 1–2, §5.2 · p. 5, §5.3 · p. 5

  • Results

    1.8 points on DecisionVista-Synth and 1.5 points on DecisionVista-Human: the combined recipe improves accuracy over a step-matched baseline, and tuned readout is 182× faster than reasoning rollout on Gemma-4-31B.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Confidence routing offers a deployment pattern that accepts confident readout answers and invokes reasoning rollout for uncertain cases.

    Source: §5.3 · p. 5, §9 · p. 8

    The work does not establish suitability for consequential decisions without oversight, and some evaluation sets are written and labelled by language models whose conventions can differ from human judgement.

    Source: Impact Statement · p. 9

Abstract

from arXiv · show

Large language models make decisions through parallel prefill and sequential decoding. Although effective, this approach incurs decoding costs and does not explicitly expose decision probabilities. Existing task-specific classifiers require per-task training and offer limited generalization to unseen decision schemas. We propose Large Decision Models (LDMs), repurposing language models to return explicit probabilities over labels specified at run time using one prefill forward pass. We first adapt models to this interface through Diverse Decision Supervision (DDS), covering varied domains, rules and labels. To further narrow the performance gap to a reasoning model, Self-Posed Reasoning Distillation (SPRD) adds tasks generated by a frozen teacher, filters them for answer consistency, and trains on label distributions obtained from repeated teacher reasoning. We evaluate on DecisionVista, pairing new-schema tests with held-out human-labelled sources. Across three base models, the recipe improves accuracy over a step-matched baseline by 1.8 and 1.5 percentage points, respectively, achieving a 182-fold speedup in latency over reasoning rollout on Gemma-4-31B. Reasoning remains stronger on the new-schema test, motivating a confidence router for uncertain decisions. On development sets, routing recovers a median 92% of reasoning's accuracy advantage while generating 27-58% of its tokens. Code is available at https://github.com/JianhuiWei7/LDM.

1. Introduction

The paper proposes single-pass Large Decision Models to provide fast decisions with explicit probabilities over run-time labels, addressing the cost and limited confidence visibility of autoregressive rollout. It combines broad schema supervision and reasoning-derived distillation, evaluates transfer on DecisionVista, and explores confidence-based routing.

1. Introduction

  • Large Decision Models return probabilities over labels specified at run time in one forward pass, avoiding autoregressive decoding and task-specific output heads.The paper identifies decoding cost and lack of explicit decision probabilities as limitations of rollout, while conventional classifiers tie output heads to predefined labels.Source: §1 · p. 1
  • Diverse Decision Supervision broadens schema coverage, while Self-Posed Reasoning Distillation trains readout toward answer distributions from reasoning on new self-posed tasks.The recipe varies domains, rules, and label meanings and retains single-pass inference.Source: §1 · p. 2
  • Confidence routing accepts confident readout answers and sends uncertain decisions to the base model for reasoning rollout, with its threshold fixed on validation data.This is presented as a deployment strategy combining fast readout with selective reasoning rollout.Source: §1 · p. 2
  • DecisionVista pairs new-schema tests with human-labelled tasks from eight sources excluded from fine-tuning and method design, using a protocol that separates development from pre-registered testing.The paper evaluates DDS and SPRD separately and together across three base models, with three seeds each.Source: §1 · p. 2
  • Averaged across bases, the recipe improves accuracy over a step-matched baseline by 1.8 points on DecisionVista-Synth and 1.5 on DecisionVista-Human; on Gemma-4-31B, tuned readout delivers a 182× median-latency speedup.On validation sets, routing recovers a median 92% of the rollout–readout gap using 27–58% of full reasoning’s generated tokens.Source: §1 · pp. 2–3
  • The contribution list frames the work around schema generalization, a combined training recipe, a paired benchmark suite, and selective reasoning at deployment.The list also identifies a protocol separating model development from pre-registered testing.Source: §1 · p. 2

2. Large Decision Models

A Large Decision Model maps a context and a run-time decision schema to probabilities over its allowed outcomes in a single forward pass. The same model can address new decisions without a task-specific output head or further training.

2. Large Decision Models

  • A Large Decision Model maps context and a run-time schema to a probability distribution over the schema’s allowed outcomes in one forward pass.A schema comprises a question or decision rule and a finite label set with natural-language descriptions.Source: §2 · p. 3
  • The predicted outcome is the highest-probability label, and schemas can specify categorical choices, binary judgements, or ordinal ratings.Label meanings and any ordering are supplied at run time.Source: §2 · p. 3
  • The interface allows the same model to address new decisions without a task-specific output head or further training.The paper says a pretrained model can use the interface zero-shot and identifies schema-transfer accuracy, calibration, and cost as evaluation priorities.Source: §2 · p. 3

3. Evaluating schema transfer

The evaluation separates development and public comparisons from sealed, pre-registered tests, pairing newly constructed decision schemas with held-out human-labelled sources. Validation data support model selection and calibration, while primary test sets are reserved for confirmation.

3. Evaluating schema transfer

  • Public comparisons include Nimble’s 13-subset benchmark and a typed-decisions test set of 2,000 decisions across 400 cases.The typed-decisions authors report 73.5% repeatability of teacher labels, and no primary-study model trains on those sources.Source: §3 · p. 3
  • Table 1 distinguishes validation sets used for development from sealed test sets first read after models were frozen, and notes that primary-study models train on neither.Later analyses are marked post hoc; its caption defines retention for most sets as agreement by at least two of three blind non-author labellers with the author.Source: Table 1 · p. 3
  • Validation sets guide recipe development, and the human-labelled validation suite supports model selection, temperature calibration, and routing thresholds.Because these sets inform development, their gains require confirmation on untouched tests.Source: §3 · p. 3
  • The primary tests were constructed after models were frozen, sealed, and first read under pre-registered analyses; no primary-study model trains on these sets.The evaluation distinguishes public comparisons, model selection, and exploratory diagnosis from independent confirmation.Source: §3 · p. 3
  • DecisionVista-Synth tests 35 newly constructed schema families across 18 operators, while DecisionVista-Human contains 2,299 human-labelled items from eight sources absent from training mixtures and earlier evaluation sets.Synthetic labels represent model agreement rather than human judgement, and screened non-overlap does not establish freedom from pretraining exposure.Source: §3 · p. 3
  • Primary statistics average accuracy equally across task families on DecisionVista-Synth and across sources on DecisionVista-Human, with paired bootstrap intervals.Calibration is measured using NLL, Brier score, and 15-bin ECE; a two-level sensitivity analysis also resamples training seeds.Source: §3 · p. 4

4. Diagnosing schema transfer

The diagnosis finds that single-pass readout often uses knowledge already present in pretrained representations, but transfer to new schemas is uneven and reasoning rollout remains more accurate on the examined validation sets. These findings motivate broader schema supervision and reasoning-derived targets.

4. Diagnosing schema transfer

  • The diagnosis frames uneven schema transfer and the readout’s unused accuracy as reasons to broaden schema supervision and train toward reasoning-derived targets.The rollout–readout gap also motivates retaining selective reasoning at deployment; reasoning costs hundreds of generated tokens per decision.Source: §4 · p. 4
  • 76–97%: across Qwen3.5 sizes from 0.8B to 27B, untuned single-pass readout reaches this share of the same model’s decision-tuned accuracy.The tuning gain shrinks from 15.8 to 2.3 points as the base grows, and 1/16 of tuning data recovers most of it; LoRA updates under 0.5% of parameters, while training only the frozen-feature option-letter head recovers almost none of the gain.Source: §4 · p. 4
  • At 27B, training on 20k records from 36 public datasets raises validation accuracy from 79.3 to 81.8, yet neither tested recipe consistently improves new-schema accuracy.The public-data and source-augmented recipes score 88.1/87.5 and 87.8/88.0 on DV-Dev-A/B, against the untuned model’s 89.1/87.5; the passage cautions that transfer depends on scale and evaluation set.Source: §4 · p. 4
  • 96.9 vs. 89.1 on DV-Dev-A and 96.8 vs. 87.5 on DV-Dev-B: reasoning is more accurate than readout for the same untuned 27B model.On DV-Dev-B disagreements, reasoning is correct 90% of the time; reasoning can lose on NLI, while single-sample gains are largest on exact multi-step computation.Source: §4 · p. 4

5. Training LDMs for schema transfer

The training recipe adapts a single-pass readout to unfamiliar decision schemas through diverse supervision and reasoning-derived targets. At inference, a confidence router can defer uncertain decisions to reasoning rollout.

5. Training LDMs for schema transfer

  • The LDM readout normalizes logits for single-token option identifiers, with LoRA training and shuffled option order.Supported formats include choice, yes/no, and ordinal scores; their bounds are implementation choices, not part of the LDM definition.Source: §5 · p. 4

5.1. Diverse Decision Supervision (DDS)

  • DDS varies domains, rules, and label meanings in synthetic decisions, complemented by public datasets and categorical instruction tasks.Its fixed readout objective is intended to encourage interpretation beyond the original training tasks.Source: §5.1 · p. 4

5.2. Self-Posed Reasoning Distillation (SPRD)

  • SPRD distils answer distributions from repeated reasoning on teacher-generated tasks, without training the student on reasoning traces.Four teacher samples are used per item; a top-vote share of at least 0.75 retains 14,829 items, while SPRD-C keeps 10,331 whose majority agrees with the constructed label—a consistency check, not a correctness check.Source: §5.2 · p. 5
  • Joint training combines supervised labels from baseline and DDS records with teacher distributions from pool records.The enabled data streams are shuffled together and trained with mean per-record loss; inference remains one forward pass.Source: §5.2 · p. 5

5.3. Confidence routing to reasoning

  • The confidence router uses the top-two margin to accept confident readout answers or send uncertain items to the untuned base for reasoning rollout.Its threshold is fixed using a validation-suite quantile, so the routed fraction on an unseen set can differ from validation coverage.Source: §5.3 · p. 5

6. Results

The combined recipe improves pre-registered transfer accuracy over a step-matched baseline, with gains varying by comparator and model. Results also cover component effects, released benchmarks, serving efficiency, and confidence routing.

Primary transfer results

  • +1.8 points on DecisionVista-Synth and +1.5 on DecisionVista-Human are the combined recipe’s gains over the step-matched baseline.Both primary tests succeed; the 95% confidence intervals are [0.6, 3.1] and [0.9, 2.2], respectively.Source: §6 · p. 5, Table 2 · p. 6

Dependence on the baseline comparator

  • +0.8 points on DecisionVista-Synth, +0.8 on DecisionVista-Human, and +2.1 on typed-decisions are the gains with validation-selected baselines.The synthetic interval includes zero (−0.1, 1.7), while the human-labelled and typed-decisions intervals are [0.2, 1.4] and [1.2, 3.0].Source: §6 · p. 5

Component effects and validation results

  • The combined recipe’s component comparisons are mixed, while its gains on the development transfer suite and DV-Dev-A/B are +3.9, +2.3, and +2.5 across the three bases.On DecisionVista-Synth, the pre-registered combination-versus-DDS test is not significant; the component-testing chain stops there. The pooled test gains are 52–62% of the validation gain.Source: §6 · pp. 5–6

Comparison with released decision models

  • 76.2 (±0.6) versus Nimble-9B’s 74.8 is the source-free 9B recipe’s score on Nimble’s 13-subset benchmark, but larger source-free recipes do not lead it.Adding benchmark training splits gives DDS scores of 80.8 at 9B and 81.7 at 27B, a different comparison regime.Source: §6 · p. 6

Measured serving efficiency

  • 40.6 ms versus 7.38 s is the tuned readout’s median latency compared with reasoning rollout on Gemma-4-31B, with 56.2× higher throughput.These BF16 serving measurements use two H100 GPUs and exclude confidence routing.Source: §6 · p. 7, Table 4 · p. 7

Deployment through confidence routing

  • At a 30% validation routing budget, confidence routing recovers a median 92% of the rollout–readout accuracy gap while generating 27–58% of reasoning’s tokens.Random routing recovers 20–35%; routed items also require a second prefill, so end-to-end savings are smaller than token counts imply.Source: §6 · pp. 6–7, Figure 2 · p. 6

7. Analysis

The analysis tests what reasoning-derived labels add, how calibration changes confidence quality, and how rollout–readout gaps vary across model sizes and architectures.

What the labels contribute

  • 1.3–2.4 points on validation and development sets, and 1.2–1.9 on human-labelled and typed-decisions sets, are lost when reasoning targets are replaced by single-pass labels.This comparison holds the 10,331 filtered items and training steps fixed; hard and soft reasoning targets perform similarly.Source: §7 · p. 8

Teacher identity and selection

  • At 9B, SPRD-C’s advantage over an equal-sized random pool with generator labels is only +0.7 to +1.2 points, with intervals touching zero.At 27B, the model’s own reasoning provides stronger labels than readout self-training; the comparison does not isolate teacher family as the explanation for Gemma’s smaller gains.Source: §7 · p. 8

Calibration

  • After temperature scaling, every recipe retains a lower Brier score than the step-matched baseline on every base, while Gemma’s raw readout remains strongly overconfident.The Qwen recipes’ transfer-suite ECE is 8–12, compared with 23.8 at 9B and 18.5 at 27B for longer-trained baselines; Gemma requires calibration.Source: §7 · p. 8

Mixture-of-experts bases

  • The largest tested MoE has the dense base’s 5.2-point transfer-suite rollout–readout gap, while tested expert-routing changes do not close that gap.The suggested explanation that reasoning tokens engage different experts is not isolated by these comparisons and does not establish a general scaling law.Source: §7 · p. 8

8. Related work

Related work situates LDMs among label-aware decisions, reasoning distillation, and selective reasoning systems, while distinguishing their interface and supervision choices.

Decision interfaces and transfer

  • Prior label-aware classifiers support unseen classes, and next-token scores can also provide class decisions.The paper positions its typed, run-time decision interface alongside these approaches.Source: §8 · p. 8

Reasoning supervision

  • SPRD distils final-answer distributions over run-time label sets at one readout position, using offline teacher supervision without student-generated trajectories.The passage relates it to System-2 distillation, implicit chain-of-thought, on-policy distillation, and readout self-training.Source: §8 · p. 8

Selective reasoning

  • The router combines selective prediction with a same-base reasoning fallback, using a validation-fixed margin threshold; temperature scaling separately calibrates probabilities.The paper relates this acceptance-and-escalation pattern to LLM cascades.Source: §8 · p. 8

9. Conclusion

The conclusion frames LDMs as schema-transfer decision models and identifies confidence routing as a deployment pattern when a rollout–readout gap remains.

Conclusion

  • DDS broadens schema supervision, SPRD trains readout toward reasoning-derived decisions, and DecisionVista tests their combined effect after method development.The conclusion notes that pre-registered transfer gains are concentrated in Qwen and synthetic-schema labels measure model agreement rather than human judgement.Source: §9 · p. 8

Impact Statement

This material details the evaluation’s scope and safeguards, the training and benchmark comparisons, and the construction of its synthetic and human-labelled test sets. It also reports limitations on interpreting model-labelled results and on extending findings beyond the studied settings.

Impact Statement

  • The study does not establish suitability for consequential decisions without oversight, and model-labelled synthetic evaluations may differ from human judgement.The authors provide a human audit protocol, but its pre-registered audit is not part of this paper.Source: Impact Statement · p. 9, A. Limitations · p. 11

B. Supervised recipes behind the diagnosis

  • Halving the learned baseline update raises DV-Dev-A accuracy from 88.1 to 89.9, motivating update-scaled baseline comparisons.The passage attributes the diagnostic to halving the learned baseline update after training.Source: B. Supervised recipes behind the… · p. 12

C. Statistics

  • The reported statistics pair each recipe seed with its same-seed baseline, while test-set intervals resample the units defining novelty.Synthetic DecisionVista sets resample task families; DecisionVista-Human resamples items within each of its fixed eight sources.Source: C. Statistics · p. 12

D. Protocol chronology

  • The time-stamped planning log records decisions before their corresponding test reads, while DV-Pilot and DecisionVista-Synth serve as confirmatory reads for different recipe stages.The log, including the DV-Pilot file hash, is among the released artifacts.Source: D. Protocol chronology · p. 12

E. Cost

  • The 16k-item teacher pass generated about 59M tokens and took about 8 H100 GPU-hours, while inference uses one prefill and no generated tokens.The reasoning rollout used for comparison generated a median of 335–513 tokens per sample.Source: E. Cost · p. 12

H. Training configurations

  • The controlled study’s test comparisons are distinguished from validation evidence, descriptive analyses, and post hoc comparisons.The experiment summary maps configurations to data, budgets, seeds, and analysis status; Table 9 identifies which sets were sealed or informed design and selection.Source: H. Training configurations · p. 13, Table 9 · p. 14, Table 11 · p. 15

I. Benchmark-source training comparisons

  • 80.8 at 9B and 81.7 at 27B versus 74.8 for Nimble-9B and 77.7 for OpenJev on the 13-subset benchmark after adding its training sources.Adding these sources makes the benchmark in-distribution for the source-augmented runs; adding SPRD-C does not improve it further.Source: §I · p. 14

J. The DDS corpus

  • The DDS corpus ablation attributes most gain to synthesised decision tasks; public data and Super-NaturalInstructions add little on new schemas but help the knowledge-heavy transfer suite.Removing any single listed field changes neither the average nor that field’s held-out benchmark beyond noise.Source: J. The DDS corpus · pp. 14–15, Table 15 · p. 17

N. Reproducibility details

The reproducibility details describe released artifacts, model and training configurations, and construction of the self-posed pool. Examples show how reasoning votes can differ from single-pass readout and how the evaluation items are structured.

N. Reproducibility details

  • The released artifacts include the self-posed pool, audit decisions, code, adapters, and per-item predictions for every evaluation set.Development and stress sets are also released with their generator and verifier labels.Source: N. Reproducibility details · p. 19
  • The self-posed pool retains 14,829 items meeting a 0.75 top-vote threshold, with 10,331 also agreeing with construction-time intent.The pool has 16,040 items from 696 schemas, each sampled four times by the reasoning teacher.Source: N. Reproducibility details · pp. 19–20
  • The pool spans 62 domains, 20 skills, and three formats, while evaluation-family nearest-neighbour similarities remain below the mean similarity between pool schemas.Mean nearest-pool-schema cosine similarity ranges from 0.732 to 0.760 across four evaluation sets, versus 0.793 between pool schemas and their nearest neighbours.Source: N. Reproducibility details · p. 20

O. Examples

  • For one negation-scope item, reasoning votes and the external labeller select local_denial while the single-pass readout selects global_denial.The item illustrates a disagreement between the readout and reasoning rollout on the same model.Source: O. Examples · p. 21

A DV-Stress item

  • The DV-Stress medication item presents a medication list, a fluconazole prescription, and the options safe, caution, or avoid.The item includes “avoid” as its reference label.Source: A DV-Stress item · p. 21

P. What reasoning solves that the readout does not

Reasoning most clearly outperforms single-pass readout on multi-step computation, while helping on several other task types and hurting on some pragmatic language judgments. A margin-based router sends uncertain decisions to reasoning; a learned rejection option performs similarly on most reported sets.

P. What reasoning solves that the readout does not

  • On date arithmetic, numeric comparison, and legal deadlines, readout scores 56–66% at every scale, while reasoning scores 98–100%.These schemas are the largest fixes across the three tested models.Source: P. What reasoning solves that the… · p. 22
  • Reasoning also helps with chained rules, implicit commonsense, and knowledge-option elimination, with gains varying by model and dataset.Reported examples include +31 for the 9B model on chained conditions, WinoGrande gains of +11 for Gemma and +22 for Qwen3.8-27B, and transfer-suite gains of +10 on MedQA, +9 on LogiQA, and +6 on MMLU.Source: P. What reasoning solves that the… · p. 22
  • Reasoning hurts on some pragmatic entailment and rating tasks, including CommitmentBank, MultiNLI-fiction, VitaminC-synthetic, and SummEval.CommitmentBank loses 7 points for Gemma and 20 for Qwen3.8-27B; SummEval relevance and consistency fall by 7 and 6 points.Source: P. What reasoning solves that the… · p. 22

Q. Routing uncertain decisions to reasoning

  • A 30%-budget confidence router sends items below a validation-fixed top-two-log-probability margin threshold to reasoning rollout, while confident items use readout.Table 20 compares the router with readout, reasoning, and random routing at the same rate.Source: Q. Routing uncertain decisions to… · p. 23, Table 20 · p. 23
  • The learned rejection option is trained on pool items where the DDS readout errs but reasoning labels agree, then used to route uncertain cases.Evaluation routes when its predicted “needs step-by-step reasoning” probability crosses a validation-set threshold or the option is the argmax.Source: Q. Routing uncertain decisions to… · p. 23
  • The learned rejection score is within 0.7 points of margin routing on four reported sets, while scoring 2.1 points higher on sealed DecisionVista-Synth and routing more items.The comparison on DecisionVista-Synth is 48% routed versus 43% for margin routing; the paired interval includes the reported +1.2 to +2.9 range.Source: Q. Routing uncertain decisions to… · p. 23

R. Mixture-of-experts models as LDMs

Mixture-of-experts models show a larger reasoning-to-readout gap than dense models at smaller tested scales, but the penalty shrinks at larger scale. Additional analyses find comparable Gemma results with its own reasoning labels, limited transfer from head-only training, and diminishing tuning gains as model size grows.

R. Mixture-of-experts models as LDMs

S. Gemma with its own reasoning labels

  • Gemma’s own reasoning labels give comparable results to Qwen-27B labels in these post-hoc runs, but the control changes both the retained subset and training-step count.The authors state that this is not a formal equivalence test and does not show teacher family is the sole explanation for Gemma’s smaller gains.Source: S. Gemma with its own reasoning labels · pp. 25–26, Table 27 · p. 26

T. Tuning as adaptation of the readout

  • The untuned readout reaches 76% of tuned accuracy at 0.8B and 97% at 27B, while the tuning gain shrinks with size.Using 1/16 of the data recovers most of the gain at every tested size; scaling the learned update down to 0.65 helps by +0.3 to +0.8 at 4B and above.Source: T. Tuning as adaptation of the readout · p. 26, Table 28 · p. 26

U. Is a classification head enough?

  • Training only the 26 option-letter output rows recovers little of LoRA’s transfer gain, though it substantially lowers raw transfer-suite ECE.On the DDS mixture, transfer changes by +0.1 for head-only training versus +5.6 for LoRA, and temperature scaling removes the head’s ECE advantage.Source: U. Is a classification head enough? · p. 26, Table 29 · p. 27

V. Further analyses

Further analyses identify task-specific reasoning gains and harms, probe label validity and transfer robustness, and compare related approaches to schema transfer and reasoning supervision. Serving measurements report a 182× median-latency advantage for tuned readout over reasoning rollout under the stated Gemma setup.

V. Further analyses

  • Reasoning helps most on WinoGrande, MedMCQA, and irony, but hurts on CommitmentBank; the 27B SPRD student remains within one item of baseline on CommitmentBank.The student’s largest validation gains occur on WinoGrande (+2.8) and irony (+3.2).Source: §V · p. 27
  • On earlier two-model-labelled sets, a third model family disputes 10–13% of items, and much of the 27B SPRD gain lies on contested items.On items all three families confirm, the gain is +0.8 with interval [−0.3, 2.0]; the authors say correctness versus shared model convention requires human judgment.Source: §V · pp. 28–29
  • Exploratory single-seed DDS ablations suggest a few thousand audited synthetic tasks provide most of the gain, while removing any one field changes little.DDS is tested as a package, not as independently validated components.Source: §V · p. 29

W. Extended related work

  • Prior work includes synthetic instruction-task generation and reasoning distillation; SPRD instead uses self-posed schemas, final-answer vote distributions, and an offline frozen teacher.Unlike on-policy methods, SPRD uses a fixed pool without student-generated trajectories; the primary shared teacher is the student’s own base only at 27B.Source: W. Extended related work · p. 30

X. Serving efficiency measurement details

  • On Gemma-4-31B, tuned readout has 40.6 ms median latency versus 7.38 s for reasoning rollout, a 182× speedup in the measured serving setup.The measurements exclude prompt preparation, model loading, and compilation, and do not measure confidence-router latency.Source: §X · p. 31
Loading 2610.15501v1…