PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

Yinheng Li, Justin Wagle

arXiv:2610.02076v1cs.CL

TL;DR

Jev-style systems can act on categorical decisions directly, but it is unclear how much of this capability general-purpose LLMs already have and when fine-tuning is needed. LLM2Jev extracts decisions from bracketed numeric continuations and offers training-free inference or targeted fine-tuning. Qwen3.5-4B performs strongly without training, while fine-tuning’s benefits depend on model and task.

  • Problem

    The paper asks how much decision-making capability pre-trained LLMs already possess and under what conditions fine-tuning yields practical gains.

    Source: §1 · p. 1

  • Method

    LLM2Jev preserves causal LLM architectures and uses bracketed numeric candidate continuations for training-free decisions or listwise fine-tuning with KL anchors.

    Source: §1 · p. 1, §3.2 · p. 4, §3.4 · p. 5

  • Results

    Without training, Qwen3.5-4B matches same-backbone community Jev-style models, outperforms letter-logit readouts, supports arbitrary candidate counts, and handles multimodal visual decisions.

    Source: §1 · p. 2, §6 · p. 11

  • Takeaways & Limitations

    Fine-tuning is most valuable for compact backbones and specialized domains, while capable models show diminishing returns and KL anchors preserve conversational generation.

    Source: §6 · p. 11

    JevBench evaluations in §4.3.3 are exploratory because later training-data iterations were motivated by error analysis on public JevBench samples.

    Source: §3.5 · p. 6

Abstract

from arXiv · show

Jev-style decision models return categorical probability distributions over predefined options without generating free-form text, enabling software systems to act on their outputs directly. In this work, we investigate the extent to which general-purpose LLMs already possess this capability out of the box, and when fine-tuning is actually necessary. We present LLM2Jev, an architecture-preserving framework that extracts calibrated decisions directly from next-token probabilities over bracketed numeric identifiers. LLM2Jev provides both a training-free inference recipe and a fine-tuning objective that optimizes candidate selection via a tree-factorized listwise loss while anchoring auxiliary predictions to the base model using KL divergence penalties. Evaluating on Qwen3.5-4B and Qwen3-0.6B, we find that modern LLMs are inherently effective decision models: without training, the 4B model matches community Jev-style models built on the same backbone, outperforms letter-logit readouts, supports arbitrary option counts, and natively handles multimodal decisions over images. Fine-tuning provides targeted rather than universal benefits -- substantially improving weaker models and specific tasks (such as many-option intent routing), but offering diminishing returns for strong backbones. Crucially, our KL anchors prevent behavioral degradation in conversational text generation, with LoRA delivering the strongest performance on capable models.

1 Introduction

The paper asks whether general-purpose LLMs already make effective decisions over predefined options and when fine-tuning is worthwhile. LLM2Jev uses an architecture-preserving interface for training-free decisions or targeted tuning, and Qwen3.5-4B performs strongly without training.

1 Introduction

  • Jev-style outputs provide categorical probabilities over predefined options that software can consume directly, avoiding parsing generated text.Parsing can add latency and risk malformed outputs.Source: §1 · p. 1
  • The study asks how much decision-making capability pre-trained LLMs already possess and when fine-tuning yields practical gains, amid varied performance from adapted and frozen models.Examples include 89.6% accuracy for Plumb and 82.3% and 79.2% for Hopper and reflex, versus 81.0% for SemIf on public JevBench items.Source: §1 · p. 1

Our approach: leveraging off-the-shelf LLMs

  • LLM2Jev turns causal LLMs into Jev-style decision models without changing their architectures, tokenizers, or vocabularies.Unlike approaches with custom heads or reserved tokens, the design avoids backbone-specific retraining.Source: §1 · pp. 1–2
  • The architecture-preserving interface applies across causal LLMs, inherits improvements in base-model capabilities, and supports image classification and visual question answering without multimodal fine-tuning.New models can be evaluated immediately, with targeted adaptation used when required.Source: §1 · p. 2

Two complementary recipes

  • The training-free recipe numbers options, prefills “Best answer: [”, and scores bracketed candidate continuations by probability without supervision or added parameters.The fine-tuning recipe uses the same interface, a tree-factorized listwise loss, and KL penalties to keep other predictions close to the base model.Source: §1 · p. 2

Findings

  • Without training, Qwen3.5-4B matches same-backbone community Jev-style models, outperforms letter-logit readouts, scales to arbitrary candidate counts, and supports zero-shot multimodal decisions.The model also exhibits strong probabilistic calibration.Source: §1 · p. 2

2. When does fine-tuning help?

Fine-tuning helps most when training data addresses a base model’s specific deficiencies. Its value varies by model strength and task, while KL anchors and different regularization strengths matter for preserving conversational behavior.

2. When does fine-tuning help?

  • Fine-tuning markedly improves smaller models and targeted domains such as high-cardinality intent routing, but yields diminishing returns or negative transfer on capable models and broad benchmarks.Targeted adaptation incurs negligible degradation in general model abilities.Source: §2 · pp. 2–3
  • KL anchors are crucial for preventing degenerate generation outputs and preserving conversational integrity.The reported protection concerns generation across conversational contexts.Source: §2 · p. 3
  • Stronger models benefit most from LoRA with tight anchor constraints, whereas smaller models perform better with milder regularization.Source: §2 · p. 3

2 Background and Related Work

The background defines Jev decision formats and benchmark scope, then distinguishes frozen probability readouts from fine-tuned systems. It also motivates assessing calibration alongside top-choice accuracy.

Jev and JevBench

  • Jev represents decisions as categorical distributions over discrete candidates in choice, binary noul, and ordinal score formats.Source: §2 · p. 3
  • JevBench covers intelligence, calibration, latency, and computational cost; this study uses 231 public items and reports diagnostic accuracies, not official leaderboard standings.Held-out and sealed test suites remain private.Source: §2 · p. 3

Community Jev-style models

  • Frozen readouts use output-token probabilities without parameter updates, while approaches include option-letter logits, temperature fitting, and cyclic permutations with batch-level calibration.Source: §2 · p. 3
  • Fine-tuned readouts adapt parameters for task alignment, often using LoRA or teacher-generated synthetic queries; decider instead encourages replay predictions to match the parent model’s distribution.Source: §2 · p. 3

Probabilistic calibration

  • The study reports raw-distribution ECE and Brier score because probabilities matter alongside top-1 choices, including for deferring low-confidence cases to human review.Lower values indicate better performance for both metrics.Source: §2 · p. 3

3 The LLM2Jev Framework

LLM2Jev frames decisions as probability distributions over candidate options and provides both training-free scoring and fine-tuning through the same bracketed-number interface. Its fine-tuning objective trains candidate selection while anchoring other predictions to the base model.

3.1 Problem Formulation

  • A decision instance maps a state, candidate set, and target choice to a probability distribution over the candidates.The formulation also covers yes/no verification and ordinal scoring by expressing them as candidate options.Source: §3.1 · p. 3

3.2 Answer Interface

  • Bracketed numeric suffixes represent options, allowing prefix-free candidate identifiers and arbitrary option counts while preserving standard text generation.The answer prefix is “Best answer: [”, and selecting option k completes the suffix as “k]”.Source: §3.2 · p. 4

3.3 Training-Free Recipe

  • Candidate scores sum next-token log-probabilities conditioned on the full prompt and are normalized across options, enabling joint comparison of the candidate slate.The paper contrasts this with methods that score options independently; jev-local reaches 74.9% accuracy with frozen Qwen3.5-9B, versus 81.4% for listwise readouts on Qwen3.5-4B.Source: §3.3 · p. 4

3.4 Fine-Tuning Recipe

  • Fine-tuning combines a gold-candidate tree-factorized listwise loss with KL anchors that keep legal-token mass, illegal-token distributions, and unrelated next-token predictions close to the reference model.The listwise loss follows only the gold path, making it tractable for high-cardinality tasks such as K = 151.Source: §3.4 · p. 5
  • Anchor penalties use sampled trie nodes and context positions, with top-64-token truncation yielding a lower bound on full KL; these sampling choices were heuristic and not systematically ablated.The paper reports that changing the sampled counts has virtually no impact on training throughput or wall-clock time.Source: §3.4 · pp. 5–6

3.5 Training Data Construction

  • Training examples come from public human-annotated datasets with labels preserved, while n-gram overlap and duplicate validation inputs are filtered to limit contamination.Later training mixtures were motivated by public JevBench error analysis, so those evaluations are treated as exploratory; JevBench was excluded from tuning and checkpoint selection.Source: §3.5 · p. 6

4 Experiments

Across two Qwen scales, frozen LLMs already make useful structured decisions, while fine-tuning helps selectively and requires regularization to preserve generation behavior.

4.1 Setup

  • Experiments use Qwen3.5-4B (thinking mode disabled) and Qwen3-0.6B to assess JevBench, general, multimodal, and conversational tasks; four open Jev-style systems benchmark 231 public JevBench items, with GPT-5.6 Sol included for broader context in standard text-generation mode.Training configurations vary data mixtures and step budgets; Reason+Long-12k is retrained across anchor weights under full-parameter tuning and LoRA.Source: §4.1 · pp. 6–7

4.2 An LLM Is Already a Decision Model

  • Without adaptation, Qwen3.5-4B matches or exceeds same-backbone community systems on JevBench and beats SemIf by 3.2 percentage points in external-benchmark macro accuracy.It scores 69.0% on 77-way Banking77 and has JevBench ECE 0.057, below SemIf’s 0.061 and reflex 4B’s 0.070.Source: §4.2 · p. 7
  • Zero-shot Qwen3.5-4B scores 82.4% on MMBench and 64.6% on MMStar, and text-only fine-tuning preserves its multimodal decision abilities.With visual inputs withheld, performance falls to 15.2% and 28.8%, respectively.Source: §4.2 · p. 8

4.3.1 Performance Gains Track Supervised Task Coverage

  • Fine-tuning gains concentrate on tasks represented in training data, while knowledge-intensive benchmarks absent from the mixtures show negligible gains.Banking77 improves across configurations; commonsense data also boosts 4B WinoGrande, whereas lexical overlap with reasoning data does not ensure gains on MMLU or MMLU-Pro.Source: §4.3.1 · p. 8

4.3.2 Intent Supervision: Domain Gains Coupled with Heuristic Shortcuts

  • Intent-only supervision improves Banking77 but produces JevBench shortcuts, including yes bias, fallback suppression, lexical matching, and overconfidence.Accuracy falls from 81.4% zero-shot to 79.2% at 100 steps and 76.2% after one epoch despite improving in-distribution loss; KL anchors do not prevent this candidate-probability redistribution.Source: §4.3.2 · pp. 8–9

4.3.3 Balancing Verification Formats and Sequence Lengths

  • Reason-9k curbs affirmative bias and matches zero-shot JevBench accuracy, but scaling short-context reasoning data harms long-context performance until long-context corpora are added.Reason+Long-12k restores long-sequence performance to base-model levels.Source: §4.3.3 · p. 9

4.4.1 KL Anchoring Safeguards Generative Stability

  • KL anchors preserve generation behavior even when benchmark scores barely change: modest penalties (λ ≥ 0.01) eliminate virtually all format-breaking generations.Without anchors, the 4B model continues after identifiers on 69 of 231 JevBench queries, while the 0.6B model has severe conversational termination failures.Source: §4.4.1 · p. 10

4.3.4 Model Scale Governs Adaptation Utility

  • At 4B, LoRA with λ = 1 reaches 84.0% JevBench accuracy; full-parameter tuning yields no net JevBench gain, while compact models gain broader structural improvements.At 0.6B, LoRA and full-parameter tuning are within a three-item JevBench margin; unregularized LoRA still has text-mode and conversational failures.Source: §4.3.4 · p. 9, §4.4.3 · pp. 10–11

5 A Practical Recipe

The practical recipe is to begin with a capable model and a training-free numbered interface, then adapt only for verified weaknesses. Training data should reflect test conditions, and evaluation should check both transfer and generation behavior.

5 A Practical Recipe

  • Start with a training-free baseline: a capable foundation model and numbered prefix-free interface provide a strong, calibrated starting point, including zero-shot visual decisions for multimodal backbones.Newly published LLMs can be evaluated immediately without training delays.Source: §5 · p. 11
  • Reserve fine-tuning for clear base-model weaknesses with domain-specific annotated data; do not expect universal accuracy gains across heterogeneous decision tasks.Training mixtures should reflect production answer formats and prompt lengths, including verification and fallback options, while avoiding shallow keyword shortcuts.Source: §5 · p. 11
  • Keep KL anchors to limit behavioral drift; λ = 1 is a robust default, while λ = 0.1 can balance plasticity and stability for compact models under full-parameter tuning.A few hundred gradient steps typically suffice.Source: §5 · p. 11
  • For capable backbones, prefer LoRA and audit out-of-distribution transfer and autoregressive completions, since in-distribution loss and standard likelihood metrics can miss degradation.Open-ended completion checks can reveal syntax failures and generation runaway.Source: §5 · p. 11

6 Conclusion

LLM2Jev extracts calibrated decisions from causal LLMs without changing their architecture, with or without adaptation. Qwen3.5-4B performs strongly without training, while fine-tuning helps selectively and KL anchors prevent generative degradation.

6 Conclusion

  • LLM2Jev extracts calibrated Jev-style decisions from general-purpose causal LLMs through structured prefix-free conditioning, with or without parameter adaptation.Source: §6 · p. 11
  • Without training, Qwen3.5-4B matches same-backbone community Jev-style models, supports arbitrary candidate counts, and handles multimodal visual decisions.Source: §6 · p. 11
  • Fine-tuning brings targeted rather than universal improvements, benefiting compact backbones and specialized domains most, while KL anchors prevent generative degeneration across conversational contexts.Source: §6 · p. 11

A Implementation Details

The implementation uses bracketed numeric options and scores candidate suffixes from a shared prompt. It also evaluates generated answers and reports training, anchoring, and latency details.

A Implementation Details

  • The prompt lists options with bracketed numeric identifiers and instructs the model to answer only with the selected identifier.Scoring appends “Best answer: [” and computes candidate-suffix log probabilities; text mode greedily generates and counts malformed or unfinished answers as parse failures.Source: A Implementation Details · p. 14
  • Before training, the frozen base model supplies cached anchor statistics, including legal mass, selected illegal-token mass, and top-token distributions.Caching avoids keeping a second model in memory; the top-64 truncation and three off-path nodes are heuristic settings.Source: A Implementation Details · pp. 14–15
  • The reference implementation reports 194 ms median latency for 4B and 76 ms for 0.6B on one RTX A5500, versus 46 ms for SemIf in BF16.The implementation uses FP32 and is not speed-optimized; latency optimization is outside the work’s scope.Source: A Implementation Details · p. 15
  • Training supports full-parameter updates or LoRA on decoder linear layers, with LoRA merged into the base checkpoint after training.The listed configuration uses AdamW, FSDP, BF16 compute, and FP32 optimizer states.Source: A Implementation Details · p. 15

B Evaluation Details

The evaluation spans external classification and reasoning, public JevBench, general capabilities, image decisions, and conversational behavior. The appendix also details tokenizer-dependent candidate tries and dataset handling.

B Evaluation Details

  • External evaluation uses seven datasets, JevBench uses 231 public items, and the General average covers four tasks; Banking77 is assessed separately as a 77-way benchmark.The External macro average covers the first six datasets; General evaluation also includes WikiText-2 for perplexity.Source: B Evaluation Details · p. 15
  • Image evaluation uses MMBench and MMStar, while behavior evaluation covers 560 replies per model from Dolly-15k and MT-Bench.Image questions use rotated options for CircularEval; behavior testing includes short-answer and long-form prompts.Source: B Evaluation Details · pp. 15–16

C Tokenizer Differences

  • All three tested tokenizer families yield prefix-free tries for identifiers 1–1,000, with trie depth and scoring steps determined by digit segmentation.Digit-level tokenizers can share an initial token for options 1 and 10; tokenizers grouping up to three digits give a depth-two trie, and full-string suffix tokenization avoids SentencePiece’s extra word-start token.Source: C Tokenizer Differences · p. 16

D Datasets

  • Training and evaluation dataset tables report dataset licenses, and training examples whose inputs appear in official validation splits are removed.The dataset table notes non-commercial-only use for ReClor and no located license statement for αNLI and ProofWriter data releases.Source: D Datasets · p. 16

E Similarity Between Training and Evaluation Data

The appendix checks training–evaluation overlap and compares lexical similarity between training mixtures and evaluation sets. It reports that relative similarity patterns align more closely with shared task formats than shared vocabulary.

E Similarity Between Training and Evaluation Data

  • No 13-word window overlaps between the 9,000 reasoning examples and 7,311 External and JevBench items after filtering a shared passage.Intent data shares some label names with evaluation items but no user utterances; two incidental overlaps are noted for Reason+Long-12k.Source: E Similarity Between Training and… · pp. 16–17
  • Lexical similarity is measured with content-word TF-IDF cosine scores, weighted by each dataset’s share of the training mixture.The table supports relative comparisons rather than absolute judgments of similarity.Source: E Similarity Between Training and… · p. 17
  • The reasoning set is more lexically similar to External reasoning benchmarks, while the intent set is more similar to Banking77 and JevBench.Together with §4.3.1, these comparisons suggest shared task formats explain observed gains better than shared vocabulary.Source: E Similarity Between Training and… · p. 17

F Full Results

Fine-tuning does not significantly reduce image-benchmark performance, and reasoning configurations improve MMBench accuracy. The appendix also records dataset-license information.

F Full Results

  • 85.1% and 84.6% MMBench accuracy are reached by Reason-30k and Reason+Long-12k, respectively, with no significant decline on either image benchmark.MMStar remains within 1.1 points of the original model.Source: F Full Results · p. 17
  • 42 of 46, 52 of 54, and 43 of 44 newly correct MMBench questions came from questions the original answered correctly under some option rotations.Fine-tuned models choose the same option across rotations more often, at 89.2–91.4% versus 87.6%.Source: F Full Results · p. 17
  • MMStar ECE falls from 0.090 to 0.051–0.077, while MMBench ECE remains between 0.011 and 0.016.Coarse perception improves significantly for all three fine-tuned models.Source: F Full Results · p. 17
  • The dataset tables note that ReClor permits only non-commercial research use, no license statement was found for αNLI or ProofWriter, and no license statement was found for MMStar.Table 4 covers training-dataset licenses; Table 5 covers evaluation-dataset licenses.Source: Table 4 · p. 18, Table 5 · p. 18

G Details of the Data Comparisons

The comparisons show broad gains for the weaker 0.6B model but targeted returns for the 4B model. Data composition and input length shape how fine-tuning transfers across tasks.

G Details of the Data Comparisons

  • The comparison table covers intent, reasoning, and JevBench composition; its lexical-similarity scores support relative comparisons, not absolute similarity judgments.The General average is mean accuracy across four tasks, and the fine-tuning table defines Long as the 37 items exceeding 1,500 tokens.Source: Table 6 · p. 18, Table 7 · p. 18, Table 8 · p. 19
  • The intent set contains 95,461 instances; for 4B, full-epoch intent training (2,984 steps) lowers JevBench accuracy from 81.4% to 76.2%, while Bitext development loss falls from 0.65 to 0.002.Error analysis found increased yes predictions (36 to 50), lower mean probability for catch-all options (0.056 to 0.008), and higher ECE (0.057 to 0.116); the data has no yes/no or verification questions, and “None of these” is correct in only 250 of 95,461 examples.Source: G Details of the Data Comparisons · pp. 18–19
  • Reason-9k combines six reasoning datasets with three intent datasets to address identified failure modes, including verification bias, keyword matching, and missing two-option questions.ProofWriter supplies balanced true/false/unknown examples, while CosmosQA adds “none of the above” answers.Source: G Details of the Data Comparisons · p. 19
  • Reason-9k improves 4B calibration and reduces “yes” predictions while keeping JevBench accuracy on par; adding more reasoning data does not improve JevBench accuracy.Reason-9k has 80.5% JevBench accuracy and 0.046 ECE; Reason-21k and Reason-30k score 78.4% and 76.6%.Source: G Details of the Data Comparisons · p. 20
  • Reason-21k and Reason-30k answer fewer long JevBench items than Reason-9k, but adding long-input corpora restores performance near the original model’s level.Reason+Long-12k answers 22 long items, versus 16 for Reason-30k and 24 for Reason-9k; it uses 375 rather than 938 steps, which may contribute to recovery.Source: G Details of the Data Comparisons · pp. 20–21
  • Banking77 accuracy rises by 35.0–42.6 points for 0.6B, while the 4B model’s JevBench accuracy remains below its original 81.4%.For 0.6B, Reason-9k reduces CLINC150 development loss from 6.91 to 0.89; ARC-Challenge gains 4.0–7.0 points.Source: G Details of the Data Comparisons · p. 21
  • 4B Banking77 gains 4.7–6.9 points and WinoGrande gains 2.0–4.2, while External macro accuracy stays within ±0.9 points of the original.No full-parameter configuration beats the original 4B model on JevBench.Source: G Details of the Data Comparisons · p. 21
  • No fine-tuned model significantly declines on the General average, and WikiText perplexity remains stable for both model sizes.The only significant per-task change is a 6.8-point GSM8K gain for 0.6B Reason+Long-12k.Source: G Details of the Data Comparisons · p. 21, Table 10 · p. 21

H Ablation Details

The ablations show that LoRA performs best on the capable 4B backbone, while the preferred anchor strength depends on model scale. They also show that removing anchors can destabilize generation even when aggregate benchmark scores appear stable.

H Ablation Details

  • No anchor weight significantly changes JevBench accuracy relative to λ = 1, but an early listwise-only run collapsed and cannot isolate anchor removal as the cause.That run also used a constant 10^-5 learning rate, no warmup, and batch size 3.Source: H Ablation Details · p. 22, Table 12 · p. 24
  • Without anchors, generation can run past the decision: 69 of 231 4B JevBench text-mode answers fail to parse, while 176 0.6B answers reach the token limit.At 4B, text accuracy falls to 63.2% while scoring accuracy remains 79.7%; λ = 0.01 reduces failures substantially.Source: H Ablation Details · p. 22, Table 14 · p. 25
  • At 0.6B, falling anchor weights increase chat replies that fail to end, often on long-form Dolly prompts, while the General average can rise by 1.0 point.The benchmark scores do not reveal this conversational change.Source: H Ablation Details · p. 22, Table 14 · p. 25
  • At 0.6B, λ = 0.1 gives the best JevBench accuracy (59.7%) and calibration (ECE 0.101), while at 4B, λ = 1 gives the highest JevBench accuracy with both methods.Lighter anchors also significantly raise 0.6B Banking77 accuracy; at 4B, λ = 1 significantly exceeds lighter weights on Banking77 with full-parameter training.Source: H Ablation Details · p. 22, Table 12 · p. 24
  • 84.0% JevBench accuracy makes 4B LoRA with λ = 1 the best trained 4B model, beating full-parameter training by 2.2–3.5 points at every anchor weight.All four LoRA runs exceed the training-free baseline of 81.8–84.0%.Source: H Ablation Details · p. 22, Table 12 · p. 24
  • At every anchor weight, small-scale LoRA and full-parameter training are within three JevBench items, and a higher LoRA learning rate fits development data better without raising benchmark accuracy.Without anchors, LoRA does not by itself protect generation: 9 4B answers fail to parse and 108 0.6B answers reach the 32-token limit.Source: H Ablation Details · p. 23, Table 12 · p. 24, Table 13 · p. 24
  • For 4B full-parameter training at λ = 1, option 10 is selected on 19 of 658 ten-option MMLU-Pro questions, versus 45 before training; the cause is unidentified.The displaced answers go to option 1, and 15 of the model’s 17 net MMLU-Pro losses occur on these questions.Source: H Ablation Details · p. 23

I Community Jev-Style Models

Related papers

Loading 2610.02076v1…