LAVOIR: Teaching a Single-Pass Decision Encoder When and What to Ask with Amortized Value of Information

1 Introduction

• LAVOIR predicts each candidate slot’s expected gain in correct-decision probability alongside the decision distribution, asking about the best slot only when its value exceeds a question cost.The model chooses the question content, while a slot-attached template supplies its wording.Source: §1 · p. 1
• A zero-initialized segment embedding preserves Laya’s logits when no slots are present, while a VOI head and Gini cap bound predicted question value.The cap limits predicted value to the maximum expected gain of a calibrated model.Source: §1 · p. 1
• Its label-free targets pair each message with multiple profiles and use schema-rule gold decisions, while an LLM verbalizes messages and answers and another-family model checks the text.Regression on realized gains estimates expected value without human labels.Source: §1 · p. 1
• Evaluation against exact references finds decisions at the Bayes ceiling, oracle-level question policy, and real-conversation questions directed where answers help.The introduction reports these as evaluation contributions, without giving detailed metrics here.Source: §1 · p. 1
• In the ecommerce_returns example, LAVOIR asks about seller and problem, skips three decision-irrelevant slots, and routes correctly, while Laya’s unseen-schema guess is wrong.Laya’s confidence remains 0.12, below its 0.85 handoff threshold.Source: Figure 1 · p. 2

2 Related Work

• Jev and Laya return typed decisions with probabilities, while GLiNER, GLiClass, and SCX Router encode or score labels; none of these systems asks for missing information.Jev is API-only, and Laya offers an open Jev-compatible interface.Source: §2 · p. 2
• Clarification work ranks questions using generated answers, studies search or information-gain questions, or trains models to clarify; LAVOIR instead predicts every candidate’s value in one encoder pass.It does not generate or simulate answers at inference.Source: §2 · p. 2
• VOI, expected information, adaptive design, and dialogue managers provide related decision frameworks, while selective classification, deferral, and conformal methods address whether to answer.The paper also keeps labels out of the LLM and uses another-family model checks because generated text can carry exploitable artifacts.Source: §2 · p. 2

3 Problem Formulation

• A schema defines units, decisive and non-decisive slots, slot-value priors, and ordered rules mapping decisive values to a gold unit; observed evidence sets specify what is known.Evidence can represent a stated value, a partial answer, or “I don’t know.”Source: §3 · p. 2
• The exact posterior is computed by enumeration, and the Bayes ceiling is the mean test-set maximum posterior probability, which no text-only model can exceed in expectation.This gives an exact reference for decisions under the information present in the text.Source: §3 · p. 3
• Oracle VOI is the expected increase in correct-unit probability under the simulator’s slot-answer distribution, and it cannot exceed posterior Gini impurity.For a calibrated model, the bound implies question value vanishes as certainty rises; the learned head does not discover this bound on its own out of distribution.Source: §3 · p. 3
• The policy asks the highest-valued unasked slot when its predicted value exceeds the question cost and the question limit is not reached; otherwise it hands off when risk is high or decides.A slot is never asked twice.Source: §3 · p. 3

Input

• A slot block follows the options, and option and slot orders are reshuffled while the state remains a list of turns.This keeps the target snapshot and deployed model in the same input format.Source: §4 · p. 3

Architecture

• The VOI head combines each slot-marker representation with detached features of the decision distribution; its MLP has 0.27M parameters.The features include the largest probability, top-two margin, normalized entropy, and option count divided by 255.Source: §4 · p. 3
• The Gini cap uses detached G, so confident predictions yield structurally small predicted values.The uncapped variants output the head’s raw value instead.Source: §4 · p. 3

Losses

• The VOI target is the change in the gold unit’s predicted probability after a probed answer, relative to a frozen, temperature-scaled snapshot.The target uses the gold unit rather than the model’s own risk and is bounded in [−1,1].Source: §4 · p. 3

Training

• After warm training and temperature fitting, the model runs two joint decision-and-VOI epochs, recomputes targets, then runs one more joint epoch.This final phase regresses on values from a decision model that has stopped moving.Source: §4 · p. 3

Schemas

• Twelve customer-service schemas use five or six units, three or four decisive slots, and one non-decisive slot; eight schemas train the model and four are held out.Pilots indicated that semantic independence of slots, rather than rule complexity, governs data quality.Source: §5 · p. 3

Cases and answers

• Each case pairs a partial profile with two to four complete profiles consistent with it, typically differing in hidden values and gold unit.Pairing prevents regression from learning only a deterministic mapping; examples include message-only cases, question–answer turns, and slot probes.Source: §5 · pp. 3–4

Input and asking loop

• The input places the conversation state beside answer-option and candidate-slot marker blocks; the option markers feed the option scorer and slot markers feed the VOI head.The asking loop appends a fixed question and its answer to the conversation and re-encodes it while the best slot’s value exceeds the cost and budget remains.Source: Figure 3 · p. 4, Figure 2 · p. 4

Generation and two-way checking

• Qwen3.6-35B-A3B writes messages and answers, while Gemma-4-26B-A4B checks decisive-slot guesses and evidence for leakage or fidelity.The final run produced 21,601 training examples, 5,379 seen-schema test examples, and 5,432 zero-shot test examples.Source: §5 · p. 4

General single-turn data

• The broad model also uses general decision, tool-choice, and open-source data, for 103,685 examples in total.Controlled models use schema data, the first 19,000 general examples, and tool-choice data, totaling 46,048 examples.Source: §5 · p. 4

Real conversations

• The real-conversation tests use 918 ABCD dialogues and 3,812 SGD first user turns, including 2,766 from services unseen in training.ABCD units are ten flows; SGD service arguments serve as slots and cannot change the intent.Source: §5 · p. 4

Models

• The final LAVOIR is the broad warm model with joint phases rerun using the Gini cap; all models use ModernBERT-large and a single seed.Controlled variants vary the decision-loss weight and temperature grouping, while the broad model is uncapped.Source: §6 · p. 4

Controlled study

• Table 1 covers seen and zero-shot schemas, reports accuracy-minus-Bayes-ceiling gaps and VOI-policy measures, and contrasts group temperatures with Laya’s per-type temperatures.Its listed measures include predicted-versus-oracle VOI agreement, AUC, and best accuracy at ≤0.5 questions.Source: Table 1 · p. 5

Policies

• Policies share the decision head and allow at most two questions; baselines include never asking, random-slot asking, and highest-VOI asking under a prediction-set condition.VOI asks for the highest-value slot, while ORACLE greedily selects the slot with the largest exact VOI and still uses the model’s decision head.Source: §6 · pp. 4–5

Metrics

• The evaluation reports accuracy, Bayes-ceiling gap, ECE, KL, VOI-ranking agreement, accuracy–questions AUC, best accuracy at ≤0.5 questions, and unnecessary-question share.AUC spans zero to two questions; accuracy differences are expressed in percentage points.Source: §6 · p. 5

7.1 Controlled study

• Seen-schema accuracy was 0.006 below the Bayes ceiling with an interval containing zero; held-out accuracy was 0.51–0.52 versus a ceiling of 0.667.Source: §7.1 · p. 5
• Group-temperature VOI reached AUC 0.799–0.800 versus ORACLE’s 0.800–0.803; with at most 0.5 questions, it beat never asking by 12.9–14.4 accuracy points.Source: §7.1 · p. 5

7.2 Real conversations and the Gini cap

• On ABCD, the real exchange raised accuracy by 8.3 points where LAVOIR asks and changed it by −0.3 where it does not; uncapped gains were +2.7 in both groups.Source: Table 2 · p. 5, §7.2 · p. 6
• Without the cap, the broad model would ask on 93% of SGD and 92% of ABCD conversations at c = 0.02; the cap reduced asking on confident cases to zero.Source: §7.2 · pp. 5–6

7.3 External benchmarks

• LAVOIR exceeded Laya on seven of twelve benchmarks, tied on spam, and fell below on four; their reported latencies use different GPUs.Source: Table 3 · p. 6, §7.3 · p. 6

7.4 Lessons about Laya’s recipe

• Removing the policy-gradient term improved VOI top-1 agreement, while per-group temperatures avoided the tenfold schema-KL increase from a shared temperature.RL0 versus RL1 top-1 agreement was 0.92 vs. 0.88 on seen schemas and 0.75 vs. 0.56 on zero-shot schemas; shared-temperature KL rose from 0.004 to 0.04–0.06.Source: §7.4 · p. 6

• LAVOIR predicts each candidate question’s value in the same forward pass as the decision, using rule-defined gold, multiple profiles per message, cross-family checks, and a Gini cap.Source: §8 · p. 7
• On seen schemas, decisions are Bayes-optimal and question selection is oracle-level; on real conversations, questions go where answers help.Source: §8 · p. 7
• The cap addresses out-of-distribution overvaluation of fields that seem important, while zero-shot schema performance, calibration, and evaluation with human participants remain future work.Source: §8 · p. 7

Limitations

• Controlled dialogues use a chosen simulator and LLM-written text that may be more cooperative than real users; real-data tests cover single exchanges, not full human-in-the-loop conversations.Source: Limitations · p. 7

Ethics Statement

• For sensitive routing, the authors recommend conservative hand-off thresholds and review of questions to avoid requesting unnecessary personal information.Source: Ethics Statement · p. 7

Code and Data Availability

• The code is available under Apache-2.0 and model weights under CC BY-NC 4.0; third-party data are not redistributed.Source: Code and Data Availability · p. 7, Ethics Statement · p. 7

A Schemas

• Automatic gates checked schema reachability, unit masses, multi-slot needs, and dry-run posterior and VOI ranges before text generation.A validator also enforced an else rule and excluded non-decisive slots from rules.Source: A Schemas · p. 9
• Pilot revisions addressed context leakage, contradictory answers, rejected examples, and invented specifics associated with non-decisive slots.The final run replaced the non-decisive date with the customer’s name and forbade invented specifics.Source: A Schemas · p. 9

B Training Data

• The training data spans typed decisions, intent selection, entailment, review scoring, and first-turn service-intent routing.General v1 totals 19,000 examples, while the tool-choice set uses a user message, candidate tools, and the first assistant-turn call as the gold label.Source: B Training Data · pp. 9–10
• Table 4 describes twelve schemas using profile counts, multi-slot prior mass, mean maximum posterior, and the share of unknown slots with oracle VOI above 0.05.Table 5 reports per-schema model-minus-posterior gold-unit probability and accuracy-minus-ceiling differences with case-clustered bootstrap intervals.Source: Table 4 · p. 10, Table 5 · p. 10
• Additional sources include Laya benchmark-builder data, Enron, and open-diversity choice, yes/no, and score datasets.Laya-source texts overlapping its benchmark builder were removed; open-diversity data draws on 27 sources.Source: B Training Data · p. 10

Leakage

• In leakage tests, the gold unit received slightly less probability than the posterior in insurance, while argmax excess arose from nearly tied posteriors.Banking showed noise in the opposite direction, and high-confidence examples contained no hidden clue, only partial answers ambiguous between two values.Source: C Additional Results · pp. 10–11

Policy gradient

• At full scale, the policy-gradient to VOI-gradient norm ratio on shared parameters was 1.5–7×.Tables 6 and 7 cover miniature-model gradients and full-scale controlled training, respectively.Source: C Additional Results · p. 11, Table 6 · p. 11, Table 7 · p. 11

Temperatures

• Per-group temperatures were 1.015–1.017 for the schemas and 3.78–7.51 across general-data task types, whose KL fell from 0.70–0.94 to 0.28–0.31.The reported general-data ranges are T = 3.78–4.34 for choice, 5.52–7.51 for score, and 2.04–4.69 for yes/no.Source: C Additional Results · p. 11

Effect of the broad data

• AUC remained 0.799 →0.795 against an oracle of 0.797, while accuracy was 0.652 →0.648 against a ceiling of 0.659 after broadening data.VOI rank correlation fell from 0.831 to 0.763 and zero-shot b0.5 from 0.579 to 0.535; the Gini cap restored b0.5 to 0.581.Source: C Additional Results · p. 11

D Implementation

• The internal module’s 149 tests cover logits, marker placement, posterior and VOI behavior, and gradient isolation; the release is a standalone package with its own tests.Laya’s act/escalate head is not used because its notebook trains it with a zero-weighted loss.Source: D Implementation · p. 11