The Hitchhiker's Guide to System 1 Models: Fast Decision-Making Beyond LLMS

What You Will Gain

• The guide equips readers with a five-family taxonomy, calibration theory, training methods, production deployment guidance, and evaluation practices for System 1 decision models.It assumes familiarity with neural networks and basic probability, but not prior knowledge of decision models, RLCD, or non-autoregressive architectures.Source: What You Will Gain · p. 9, Who This Guide Is For · p. 9

Positioning Relative to Existing Work

• The guide integrates five model families through a common decision interface, evaluates decision quality, calibration, latency, cost, and robustness, and explains composition in agent workflows and cascades.Its extended coverage adds the open-source RLCD ecosystem, calibration phenomena including IIA violations and probability contracts, and application domains such as cybersecurity and low-resource languages; the taxonomy is proposed as a useful framing, not a consensus structure.Source: Positioning Relative to Existing Work · p. 10

Chapter I Foundations

• Chapter I establishes dual-process theory, calibration and proper scoring rules, and a five-family taxonomy as groundwork for the guide’s later model, method, and application coverage.The broader guide spans 18 sections and two appendices across seven chapters, with an open-source model zoo and quiz questions in the appendices.Source: How This Guide Is Organized · pp. 9–10, Chapter I Foundations · p. 13

1 Introduction

• LLMs typically incur 500ms–5s latency versus specialized models’ 10–70ms and cost 40–400× more per decision, motivating fast, structured System 1 models.The passage cautions that the magnitude of these gaps depends on model size, input length, hardware, batch size, and serving infrastructure.Source: §1 · p. 13

2.1 Dual-Process Theory in Cognitive Science

• The System 1/System 2 distinction organizes AI architectures by computational profile, but critiques caution against treating the cognitive systems as sharply bounded modules.The guide uses the analogy for single-pass versus multi-step computation and structured versus free-form output, not as a claim that neural networks mechanistically implement human dual-process cognition.Source: §2.1 · pp. 14–15

2.2 LLMs as System 2 Machines

• Autoregressive generation mismatches structured decisions: even a binary safety gate requires token generation, wasting compute and creating failure modes absent from single-pass models.LLM next-token probabilities are optimized for language modeling rather than calibration, while autoregressive latency has a sequential floor proportional to output length.Source: §2.2 · p. 15
• Sequential text generation is an architectural mismatch for single calibrated judgments, and this guide complements Li et al. [147] by focusing on System 1 decision models.The passage also identifies prompt fragility as a limitation of LLM decisions [74].Source: §2.2 · p. 15

2.3 The Calibration Problem

• At ±0.10 precision System 2 needs ∼100 retries, and at ±0.01 it needs ∼10,000, yielding a three-to-six-order efficiency gap over System 1 probability estimates.The comparison assumes a System 2/System 1 per-call cost ratio typically of 10–100×; the table uses 50×.Source: §2.3 · p. 16, Table 2 · p. 16
• Proper scoring rules incentivize truthful probability reports, whereas accuracy ignores probability quality; calibration must also be paired with sharpness to make predictions informative.Log loss penalizes confident errors more harshly than the Brier score.Source: §2.3 · pp. 17–18
• Post-hoc calibration maps raw scores using temperature scaling, Platt scaling, or isotonic regression, but fitting and evaluation require separate data and cannot optimize internal representations.Temperature scaling preserves ranking but applies a global correction; isotonic regression can fit complex curves but typically needs >1,000 calibration samples.Source: §2.3 · p. 19
• LLMs show systematic miscalibration, including near-chance confidence–accuracy correlation and overconfidence on out-of-distribution inputs; prompt phrasing and option order can also flip decisions [40] [61] [137] [74].These failures motivate System 1 models trained to estimate calibrated probabilities directly.Source: §2.2 · p. 15, §2.3 · p. 20

2.4 Defining System 1 Decision Models

• A System 1 model directly maps an input to a bounded structured decision without autoregressive generation; calibration is an evaluated quality, not a membership requirement.A fast autoregressive model producing short JSON does not qualify because it still generates the decision token by token.Source: §2.4 · p. 20

Taxonomy

• Five families cover encoder-based classifiers, reward models, non-autoregressive decision engines, contrastive decision models, and emerging extraction models for typed-value output.Extraction models share architectural DNA with Family III but target value extraction rather than decisions.Source: §3 · p. 21

Taxonomy scope and overlaps

• The families are partially overlapping clusters organized by historical lineage and deployment role, not mutually exclusive architectural categories.Examples include encoder backbones in reward models and contrastive pretraining in non-autoregressive decision engines.Source: §3 · p. 21, Figure 3 · p. 21

Calibration

• Family III is calibrated by construction through RLCD with proper-scoring-rule rewards, while Families I, II, and IV require post-hoc calibration.Post-hoc calibration depends on a held-out set matching deployment and needs recalibration after distribution shifts; RLCD calibration holds within its training scope.Source: §3 · pp. 21–22

Latency and scale

• Family IV reaches 5–30ms latency despite up to 8B parameters, while Family V takes 1–10ms at 150M–1B parameters.The comparison attributes Family IV's latency to cached candidate embeddings and Family V's speed and size to its narrow task scope.Source: §3 · pp. 21–22, Table 3 · p. 21

Outputs and integration

• Families I and III offer structured outputs suited to threshold logic, whereas Family II yields ordinal scores and Family IV yields candidate-relative similarity scores.Family V instead outputs typed values, and how to define and measure its calibration remains an open question.Source: §3 · pp. 21–22

Chapter II Model Families

• The chapter proceeds from encoder classifiers through reward models and non-autoregressive decision engines to contrastive models and emerging extraction models.It covers architecture, training, inference, practical trade-offs, and code examples where applicable.Source: Chapter II Model Families · p. 23

4 Family I: Encoder-Based Classifiers

• Encoder classifiers use bidirectional transformer encoders to produce fixed-dimensional representations in one forward pass, a pattern refined into production decision architectures.The family includes BERT, RoBERTa, DeBERTa, cross-encoder rerankers, SPLADE, and ColBERT.Source: §4 · p. 23

4.1 BERT and the [CLS] Paradigm

• BERT fine-tunes a bidirectional encoder pretrained with masked language modeling and next sentence prediction, using the final [CLS] state for classification.MaxPoolBERT aggregates across tokens and layers to address the [CLS] bottleneck, improving accuracy particularly on low-resource tasks without additional pretraining.Source: §4.1 · pp. 23–24, Figure 4 · p. 23

4.2 Successor Encoders: RoBERTa and DeBERTa

• RoBERTa improved BERT by removing NSP and training longer with larger batches and more data, while DeBERTa separates content and relative position in attention.DeBERTa-v3 further uses replaced-token detection pretraining.Source: §4.2 · pp. 24–25

4.3 Cross-Encoder Rerankers

• Cross-encoders train with pointwise, pairwise, or listwise objectives; pairwise margin loss stops updating pairs once their score gap reaches the margin.A bi-encoder caches independently computed embeddings but has shallow interactions; a cross-encoder jointly processes each pair.Source: §4.3 · p. 25, Figure 5 · p. 25
• BERT-based cross-encoders have been deployed at Walmart scale using intermediate representations, operator fusion, and vectorization to achieve production-viable latency.Joint query-document encoding enables rich token-level interactions but requires a full forward pass per pair.Source: §4.3 · p. 25, Figure 5 · p. 25

4.4 Learned Sparse Retrieval: SPLADE

• SPLADE predicts sparse term-importance weights from BERT encoder outputs, enabling inverted-index retrieval with semantic term expansion.Its sparse vector spans the vocabulary.Source: §4.4 · p. 26

4.5 Late Interaction: ColBERT

• ColBERT independently encodes query and document tokens, then scores relevance through MaxSim late interaction with O(|q| · |d|) scoring complexity.This falls between dual-encoder and cross-encoder scoring complexity; the chapter also describes encoder classifiers for clothing-size prediction across 286 classes.Source: §4.5 · pp. 26–27

5 Family II: Reward Models

• A reward score represents learned preference or utility, not necessarily the probability an answer is correct, so calibrated routing or safety gating may need an added calibration layer.In Bradley-Terry models, reward differences predict pairwise preferences; absolute reward values have no intrinsic probabilistic meaning.Source: §5 · p. 27

5.1 Bradley-Terry Reward Models

• Bradley-Terry models train a scalar reward so preferred responses outrank dispreferred ones; only reward differences matter, making the model invariant to additive shifts.They typically use a pretrained LLM with a scalar projection [102]; gradients emphasize similarly scored preferred and dispreferred pairs, while reward scales range from 350M to 70B parameters [102].Source: §5.1 · pp. 27–28

5.2 Plackett-Luce Listwise Reward Models

• Plackett-Luce extends pairwise preferences to full rankings over K responses, and its listwise objective naturally fits GRPO-style training with ranked response sets [116].It models sequential selection from the remaining pool using a softmax over rewards.Source: §5.2 · p. 28

5.3 Process Reward Models (PRMs)

• PRMs score reasoning steps and provide a compositional trajectory signal; they significantly outperform outcome-level reward models for mathematical reasoning, particularly with beam search [81].Each step receives a score for correctness given the prompt and prior steps [81, 125].Source: §5.3 · p. 28

5.4 Outcome Reward Models (ORMs)

• Outcome reward models [17] score a complete problem–solution pair without evaluating intermediate reasoning, trading less informative feedback for simpler training and cheaper inference.They require only final-answer correctness labels, unlike step-level PRMs.Source: §5.4 · p. 28

5.5 Generative Reward Models

• Generative reward models use LLM critiques mapped to scores [146]; CE-RM [13] uses two-stage contrastive evaluation, while RubricARROW [114] structures assessment around rubric criteria.Because they generate text, these models are technically System 2; RubricARROW partially recovers System 1 properties through structured assessment.Source: §5.5 · p. 28

5.6 Rule-Based Rewards

• DeepSeek-R1 showed that rule-based rewards can effectively guide RL with deterministic, reproducible supervision, zero latency, and no learned parameters.Their reliability depends on rule correctness and coverage; they are limited to mechanically verifiable criteria and do not inherently provide calibrated probabilities.Source: §5.6 · p. 29

6.2.1 Architecture

• 70–500ms latencies and vendor-reported ∼200× speedup and ∼400× cost reduction distinguish Jev’s shared-encoder parallel sampler from sequential LLM-based decisions.Jev uses a transformer encoder with typed decision heads; the sampler reuses one encoded state across multiple decisions. Jev 1.14 adds image support, longer context, and a Box head for bounding boxes. The passage reports these as vendor figures and notes a lack of controlled independent benchmarks. Typical encoder decisions require ∼3 × 10^11 FLOPs, while generating even a short autoregressive JSON response requires sequential passes, increasing FLOPs and latency.Source: §6.1.4 · p. 32, §6.2.1 · p. 33

6.2.2 RLCD: Reinforcement Learning for Calibrated Decisions

• RLCD optimizes calibrated probabilities with proper scoring rules on structured decisions, unlike RLHF’s learned reward model and subjective preference targets [102, 124].For binary Brier loss, the sigmoid derivative attenuates updates for confident predictions, whereas the cross-entropy gradient can encourage extreme logits and overconfidence [42].Source: §6.2.2 · pp. 33–34

6.2.3 RLCR: Reinforcement Learning with Calibration Rewards

• For α < 1, RLCR’s expected reward is uniquely maximized when stated confidence equals true accuracy; at α = 1 it provides no calibration signal [18].The guarantee assumes optimization reaches the global optimum under the true data distribution; finite data, capacity, imperfect optimization, and distribution shift can degrade deployed calibration.Source: §6.2.3 · p. 34

6.3 Laya / Lia

• Laya combines specialized task-cluster checkpoints, a lightweight router, multilingual encoding, and ∼33ms GPU inference.Its training includes multilingual pretraining, decision-head and router training on ∼500K structured tasks, then RLCD fine-tuning; checkpoint loading is near-zero latency when GPU-cached.Source: §6.3 · p. 35, §6.3.1 · p. 35, §6.3.2 · p. 35

6.4 THX-01

• THX-01 uses weight-tied decision heads and teacher-generated synthetic labels with calibration-aware fine-tuning to target compact, production-oriented deployment.Sharing the heads’ first-stage projection reduces head parameters by ∼60%; HAL-X AI reports approximately 10 ms latency.Source: §6.4 · p. 36, §6.4.1 · p. 36, §6.4.2 · p. 36

6.5 System One Gemma

• System One Gemma converts a pretrained small LM into a non-autoregressive decision model by replacing its LM head, using bidirectional attention, and fine-tuning with LoRA.The recipe uses rank r = 16 LoRA on query and value projections, with heads trained from scratch; it presents conversion as an alternative to pretraining a dedicated model.Source: §6.5 · p. 36, §6.5.1 · p. 37

6.6 OpenJev-RLCD: The First Open RLCD Implementation

• OpenJev-RLCD [159] provides an open-data, documented reproduction of Jev’s training pipeline using a Qwen3-1.7B backbone.Its implementation uses SFT on structured decisions with temperature scaling for post-hoc calibration, rather than PPO with Brier rewards; it performs full-parameter fine-tuning during RLCD.Source: §6.6 · p. 37

6.7 Tachyone: Local-First Multilingual Decision Engine

• Tachyone is a local-first multilingual decision engine with hot-swappable task adapters and 54–59 ms p50 latency on consumer GPUs.It uses a ModernBERT backbone [190] with multilingual embeddings and offers a Jev-compatible REST API and Apache-2.0 licensing.Source: §6.7 · p. 38
• Tachyone supports 20+ languages and achieves ECE < 6% on Turkish HakemBench tasks versus ECE > 12% for English-only models transferred to Turkish [170].Typical latency is 54–59 ms for local Tachyone inference versus 70–200 ms for the Jev API.Source: §6.7 · p. 39

6.8 DeepOpen: Non-Autoregressive Decisions with Language Routing

• DeepOpen routes inputs to language-specific encoder modules, achieving 3–5% higher accuracy on low-resource languages than Laya with negligible English degradation.This trades memory for specialization and avoids cross-lingual interference associated with a single multilingual encoder.Source: §6.8 · p. 39

6.10 LLM-as-Jev: Bridging Generative and Decision Models

• LLM-as-Jev [78] packs the input and candidate options into one prompt and uses option-token softmax probabilities for zero-shot decisions without autoregressive decoding.It gives existing LLM systems a migration path to structured decisions, but lacks the latency advantage of true non-autoregressive models.Source: §6.10 · pp. 39–40

6.12 Additional Open-Source Models

• Decision 2.0’s Sol, Lux, and Vega score 52.1, 68.1, and 74.0 on JevArena, with median single-GPU latencies from 7.2 ms (Sol) to 71.4 ms (Vega).The family spans 2B to 27B parameters and supports Choice, Yes/No, and Score in a single forward pass.Source: §6.12 · p. 40
• Other open-weight models include Strands Decider 2B, Lev, Kev, Clef, Julia, and GLiNER2.5-Decide, which adapts NER for schema validation [145].The passage says these models are released under Apache 2.0 and can be served through llama.cpp’s /v1/systemone endpoint or framework bindings.Source: §6.12 · p. 40

7.1 CLM-8B: Contrastive Language Models

• CLM-8B uses a frozen bidirectional Qwen3-8B encoder with separate state and action projection heads, then selects by cosine similarity.It is trained with bidirectional InfoNCE, which pulls matching state–action pairs together and pushes non-matching pairs apart.Source: §7.1 · pp. 41–42
• CLM-8B achieves up to 9× lower latency than Jev, state-of-the-art verifier results on DeepSWE and Terminal-Bench 2.1, and zero-shot transfer to unseen tasks.Cached action embeddings reduce each decision to one state encoding and K dot products; verifier speedups over generative verifiers are 4–5×.Source: §7.1 · p. 42

7.2 Foundational Contrastive Models

• Contrastive models extend an embed-then-compare approach from sentence similarity and passage retrieval to vision-language alignment and modern text embeddings.Sentence-BERT established the dual-encoder paradigm; DPR uses query–passage similarity, CLIP scales contrastive learning to vision-language pairs, and E5/GTE embeddings support retrieval or zero-shot classification.Source: §7.2 · pp. 42–43

7.3 Contrastive vs. Cross-Encoder Trade-offs

• Cross-encoders model token-level state–action interactions, while dual-encoders compress each input into a vector and compare them with a dot product or cosine similarity.The dual-encoder bottleneck can lose subtle cross-input dependencies, but reusable embeddings enable caching and approximate nearest-neighbor search.Source: §7.3 · p. 43
• For small candidate sets, cross-encoders typically yield higher accuracy; for large sets, cached dual-encoder embeddings reduce computation and enable approximate search.Dual-encoder training also depends on batch size and negative sampling, while production systems may use contrastive models for candidate generation followed by cross-encoder reranking.Source: §7.3 · pp. 43–44

8 Emerging Family V: Extraction Models

• Extraction models form an emerging family adjacent to, but distinct from, decision models because they return typed values rather than decisions.The section covers Clairic, Jev’s Box primitive, and architectures that combine decision and extraction outputs.Source: §8 · p. 44

8.1 Decision vs. Extraction

• Decision models choose or score from a finite option set, while extraction models return values from potentially open-ended spaces such as dates or entity names.Their confidence semantics differ: decision confidence concerns whether the selected option is correct, whereas extraction confidence concerns whether the value matches ground truth.Source: §8.1 · p. 44

8.2 Clairic

• Clairic is a production-scale extraction model that returns typed structured values from web content, such as ISO-8601 dates and amount–currency pairs.It differs from NER models that identify entity spans.Source: §8.2 · p. 45

8.3 The Box Primitive

• Jev 1.14’s Box head takes a state and typed schema, then returns an extracted value with confidence trained against exact-match indicators.The passage describes this as applying RLCD calibration to extraction for downstream threshold-based logic.Source: §8.3 · p. 45, Figure 11 · p. 45

8.4 Advanced Architectures

• Verdict reuses one document encoding across multiple decision heads, while VX-One combines an action, calibrated confidence, threat category, and severity in one typed output.Verdict reports 1.3× the latency of a single decision on a benchmark of 10-decision pipelines, versus 10× for independent calls.Source: §8.4 · pp. 45–46

• The chapter treats calibration and confidence estimation as properties that determine whether reported probabilities can be trusted in threshold-based downstream logic.Training methodologies are covered as the means by which those calibration properties are achieved.Source: Chapter III Methods · p. 47
• Calibration and confidence estimation are presented as tightly linked topics.The chapter pairs them with training methodologies rather than treating confidence assessment alone.Source: Chapter III Methods · p. 47
• The chapter’s scope is both evaluating probability trustworthiness and examining how training produces calibration properties.Source: Chapter III Methods · p. 47

9 Calibration and Confidence Estimation

• Calibration supports confidence-gated escalation, safety rejection thresholds, and prioritization of human review; miscalibration can cause silent failures or unnecessary escalation.The guide contrasts this with LLM calibration challenges, including RLHF-related degradation and unreliable verbalized confidence.Source: §9 · p. 47

9.1 Measuring Calibration

• Proper scoring rules provide a binning-free calibration decomposition, whereas ECE depends on arbitrary bins and conflates calibration with discrimination.The guide says the Brier-derived calibration term isolates miscalibration and is preferable to ECE for routing and safety gating.Source: §9.1 · p. 48

9.2 Calibration in Encoder-Based Models

• Encoder models may be overconfident despite cross-entropy’s proper-scoring-rule property, and temperature scaling offers a simple post-hoc calibration method.Guo et al. [42] showed that modern deep networks can become overconfident after training.Source: §9.2 · p. 48

T NLLval(T). (56)

• After training encoder-based classifiers, temperature scaling on a held-out set typically reduces ECE from 0.10–0.15 to 0.02–0.04 with zero accuracy loss because the argmax is unchanged.Platt scaling learns both a scale and shift for binary classification. Multiclass matrix scaling introduces C squared plus C parameters and typically overfits unless the validation set is large.Source: T NLLval(T). (56) · pp. 49–50

9.3 Calibration in Reward Models

• Pairwise-preference reward scores preserve rankings but have no calibrated absolute probability scale, and post-hoc mappings face costly labels, prompt dependence, and policy drift.Platt scaling maps rewards to probabilities with labeled validation data; isotonic regression can capture nonlinear mappings but needs more data to avoid overfitting.Source: §9.3 · pp. 50–51

9.4 Calibration via RLCD

• RLCD uses proper-scoring-rule training signals for calibration, while whether that calibration persists under domain shift remains an open question.Reported held-out results include ECE of 0.081 after temperature scaling and ECE = 0.246 for Jev in the same evaluation.Source: §9.4 · p. 51

9.5 Calibration in LLMs

• LLM confidence estimates use logits, verbalized confidence, or response consistency; verbalized estimates can correlate nearly randomly with accuracy, while consistency costs N× more.Logit-based probabilities can be distorted by RLHF training.Source: §9.5 · p. 51

9.6 Recalibration with Few Labels

• As few as 50 labeled target-domain examples can recover most calibration quality through lightweight temperature or Platt recalibration, even when confidence magnitudes shift.The reported approach relies on ranking ability transferring across distributions.Source: §9.6 · p. 51

9.7 Probability Contracts

• Probability contracts specify calibration metric and threshold, population, refresh cadence, and fallback behavior, making calibration a runtime-enforceable service-level objective.The framework supports downstream teams building cascade thresholds and safety gates against a published contract.Source: §9.7 · p. 52

9.8 IIA Violations and Option-Set Dependence

• Adding a decoy option can shift existing option scores by 3–12%, making unchanged routing decisions sensitive to tool-catalog changes.The text attributes this IIA issue to shared softmax probability competition and notes contrastive models are less affected.Source: §9.8 · pp. 52–53

9.9 Ordinal-Scale Bias

• On a 1–5 scale, Score heads overproduce s=3 by 8–15% and underproduce s=1 and s=5 by 5–12% despite uniform labels.The reported boundary gap is 0.04–0.08 for full ordinal heads versus 0.10–0.18 for simplified sigmoid heads.Source: §9.9 · pp. 53–54

10.1 Supervised Fine-Tuning (SFT)

• Cross-entropy is the standard training loss for encoder classifiers and the first stage for most reward-model training, while its proper-scoring-rule property also encourages calibration.The passage describes minimizing cross-entropy as maximizing the likelihood of training labels.Source: §10.1 · p. 54

10.2 Reward Model Training via Preference Learning

• Reward models learn scalar scores from human preference pairs using the Bradley–Terry loss and can serve as scoring oracles during policy optimization.Their inference speed directly bounds training throughput when invoked repeatedly during optimization.Source: §10.2 · p. 54

10.3 RLCD: Reinforcement Learning for Calibrated Decisions

• RLCD replaces preference optimization with proper-scoring-rule rewards to optimize calibrated decision-making.The section contrasts this training paradigm with RLHF in Table 9, whose caption identifies the comparison but supplies no cell values.Source: §10.3 · p. 55, Table 9 · p. 55

10.4 Contrastive Learning

• Contrastive models learn shared state-action embeddings; hard negatives improve learning efficiency, and frozen-backbone CLM-8B training reduces cost by >10× versus full fine-tuning.In-batch negatives offer a simpler scalable alternative, and annealing the InfoNCE temperature from warm to cool improves convergence.Source: §10.4 · p. 55

10.5 Knowledge Distillation

• Distillation combines hard-label cross-entropy with teacher soft-label KL divergence, and LLM-to-classifier distillation can reach >90% of teacher decision accuracy at lower cost and latency.Soft labels carry inter-class similarity information discarded by hard labels; exact gains depend on task complexity and teacher-decision quality.Source: Figure 14 · p. 55, §10.5 · pp. 55–56

10.6 Multi-Task and Curriculum Training

• Multi-task training across decision types can improve generalization, while an easy-to-hard curriculum is reported to improve convergence and transfer confidence from clear-cut cases.The described curriculum progresses through high-agreement decisions, moderately ambiguous tasks, and hard edge cases.Source: §10.6 · p. 56, Figure 15 · p. 56

10.7 Practitioner Training Recipes

• The recipes cover compact-encoder fine-tuning and causal-LLM conversion, but conversion requires attention-mask changes beyond a configuration flag.The text recommends DeBERTa fine-tuning or a managed API for most practitioners, reserving conversion for cases needing a large model’s domain knowledge.Source: §10.7 · pp. 56, 58

• The chapter surveys deployment domains, System 1 roles in agentic workflows, and cascades combining fast System 1 decisions with slower System 2 reasoning.Source: Chapter IV Applications and Integration · p. 59

11.1 Agent Routing and Tool Selection

• System 1 agent routing adds 600ms of routing overhead over a 20-step trajectory versus 10 seconds with LLM routing, a 16× reduction.Confidence can gate fallback to an LLM or skip verification; precomputed action embeddings let ContrastiveLM route via one state encoding plus K dot products. Search systems similarly compose retrieval and reranking stages, with the final cross-encoder scoring about 50 documents in 10–50ms.Source: §11.1 · p. 59, §11.2 · p. 60

11.3 Content Moderation and Safety

• A zero-shot Jev detector reached median AUROC 0.886 across ten misalignment categories, surpassing most supervised baselines at 63× lower cost than LLM-judge scorers.In hate-speech evaluation, the best hosted decision model came within 1.6 macro-F1 points of the best commercial LLM on a diagnostic set at approximately 97% lower inference cost; explicit definitions changed up to 28% of predictions without consistently improving classification.Source: §11.3 · p. 61

11.4 Knowledge Organization and RAG Alternatives

• CROW replaces query-time retrieval with classifier-routed navigation over an organized knowledge base, while universal NLI classifiers handle varied label sets through hypothesis templates and calibrated probabilities.CROW assigns classifiers filing, relevance, merge, and retrieval decisions, trading ingestion-time organization for lower per-query overhead; NLI framing also enables zero-shot classification without task-specific fine-tuning.Source: §11.4 · p. 61, §11.7 · p. 63

11.5 Reward Scoring in RL Training

• System 1 models also target repeated reward scoring, millisecond-scale financial decisions, and simultaneous customer-support triage through typed decisions and calibrated scores.Reward models may be called 4 × 10^8 times in the stated GRPO example; parallel Choice, Score, and Noul decisions can jointly classify, prioritize, and escalate support tickets. Visual decision models separate perception from decision heads, with neuroscience-informed fine-tuning improving accuracy while matching human behavioral patterns.Source: §11.5 · p. 63, §11.9 · p. 64, §11.10 · p. 64

11.6 Code Verification

• CLM-8B achieved 48.2% on DeepSWE and 72.3% on Terminal-Bench 2.1, outperforming generative verifiers while running 4–5× faster.Its contrastive verifier compares the problem specification and candidate solution in embedding space, producing a scalar for agent continue-or-submit decisions.Source: §11.6 · p. 63

11.8 Medical and Clinical Decision Support

• On 978 held-out sentences, Jev achieved 7.5% WER versus 7.8% for OPT-6.7B and Qwen2.5-7B in speech-neuroprosthesis rescoring.With decoder-weight retuning, Jev reached 6.9% versus 7.2% and 7.4%; it requires no GPU and costs $0.07 per thousand sentences.Source: §11.8 · p. 64

11.11 Recommendation and Ranking

• Jev’s structured Choice formulation maintained strong NDCG@10 effectiveness against recommendation-specific models and Qwen 2.5 7B rerankers across three e-commerce domains.As candidate sets grew from K=20 to K=200, Jev showed more gradual latency growth, occupying a quality–latency regime between faster specialized models and slower LLM rerankers.Source: §11.11 · p. 64

11.12 Wireless and Telecom Network Control

• Jev reduced median response latency by up to 8.5× in receive-antenna selection and by more than 3.5× in intent-conditioned RAN slicing versus evaluated LLMs.In RAN slicing, utility was comparable to LLMs; receive-antenna selection retained a quality gap versus task-specific alternatives, and edge-service orchestration showed faster decisions need not lower end-to-end service latency.Source: §11.12 · pp. 64–65

11.13 Multi-Modal Decision-Making

• Multimodal System 1 models extend structured decisions to vision, audio, and GUI agents through shared execution, candidate-conditioned probabilities, order calibration, and fast action selection.Visual Jev is 8.9× faster at 32 questions per image, and answer-supervised training raises macro accuracy from 70.6% to 76.1%; PixelJev’s 64-shot adaptation raises Pets accuracy from 60.1% to 92.4% [143] [149]. AudioJev reaches 68.9%/55.3% mean accuracy on MMAU/MMAR and reduces order-sensitivity by 43.8%/60.5%; Jev-Mobile completes 79% of AndroidWorld tasks while cutting time by 32.7% and API cost by 73.4% [86] [148].Source: §11.13 · p. 65

11.14 Cybersecurity and Perimeter Triage

• Perimeter triage and prompt-injection screening show how fast System 1 classifiers can protect slower systems, but missed attacks remain a serious limitation.VX-One uses typed action, confidence, threat-category, and severity outputs; its 0.7–1.7 ms CPU inference matches fine-tuned DeBERTa accuracy on 50K events [162]. Prompt-injection detection reaches AUROC 0.871 on 47,611 examples at 8 ms, but misses 64.9% of attacks at the default threshold [166].Source: §11.14 · p. 66

11.15 Theory of Mind

• Jev-ToM reaches 73.2% accuracy on 11,117 theory-of-mind questions, above 50% chance but below frontier LLMs at about 85%, indicating limits on complex mental-state reasoning [163].The evaluation covers false-belief, knowledge-attribution, and intention-recognition questions; the authors suggest fast but imperfect assessment may still suit some social AI applications.Source: §11.15 · p. 66

11.16 Political Science

• Across seven political text-classification replications, zero-shot Jev Choice achieves 78–92% of fine-tuned classifier accuracy, with latency advantages but no significant cost advantage at batch API prices [172].For three studies, it outperforms the original reported classifier, likely because those classifiers used fewer than 2K training examples; calibrated confidence can support uncertainty-weighted estimates.Source: §11.16 · p. 66

11.17 Text Quantification (Learning to Quantify)

• Text quantification estimates class prevalence over document bags; transformer-based classifiers improve aggregative methods, while end-to-end neural architectures predict prevalence directly.QuaNet combines embeddings, posterior probabilities, and classical quantifier estimates; HistNetQ outperformed all methods in LeQua 2022, and GMNet reports state-of-the-art results over HistNetQ and classical methods [184] [185] [186] [188]. Evaluation uses bags with varying prevalence, including UPP and NPP protocols; the multi-label Concordance Ratio is symmetric to over- and under-estimation and equals 1 for perfect estimates [187, 183].Source: §11.17 · pp. 66–67

11.18 Low-Resource Languages

• On four tabular benchmarks, serialized-row System 1 models are competitive with gradient-boosted trees in low-data settings, but tree models surpass them at about 64 labeled rows [173].Their calibrated outputs can feed threshold-based decisions directly without the post-hoc calibration step required by tree ensembles.Source: §11.18 · p. 67

11.19 Low-Resource Languages

• Low-resource language tests reveal calibration and transfer gaps, while a unified audio decision model handles diverse tasks without task-specific fine-tuning.On Turkish tasks, English-trained models have ECE > 12% despite accuracy > 80%, while Tachyone has ECE < 6% [170]. Zero-shot transfer loses 15–20% accuracy for Balinese and Minangkabau but less than 5% for Javanese and Sundanese [171]; a unified audio pipeline reaches 71.3% mean accuracy across eight benchmarks at 45 ms per decision [179].Source: §11.19 · p. 68

12.1 The Agent Harness as a Decision Architecture

• A production agent harness has tools, middleware, skills, and memory; System 1 models serve as middleware at its decision points.The agent loop is formalized as an MDP with a frozen LLM policy.Source: §12.1 · p. 68

12.2 System 1 Decision Points in Agent Loops

• System 1 models can route before the policy acts, gate actions during execution, and verify outcomes afterward.Verification can use Score models for solution quality and Noul models for postconditions; CLM-8B is reported to verify code at 4–5× lower latency than generative verifiers.Source: §12.1 · p. 68, §12.2 · p. 69

12.3 System 1 Models as Loop Infrastructure

• A 20-iteration agent loop makes 60 millisecond-scale decisions across routing, gating, and verification, so keeping each fast is key to limiting overhead.These control-flow decisions contrast with the generative policy’s System 2 role.Source: Figure 18 · p. 68, §12.3 · p. 69
• System 1 routing and safety-check behaviors can be distilled into a generative model, raising the question of whether dedicated decision models remain necessary at deployment.The passage presents this as an open question, not a resolved outcome.Source: §12.3 · p. 69

12.4 System 1-Controlled Agentic Memory

• Jev-Mem uses typed System 1 decisions to control memory writing and retrieval, reserving the LLM for final answer synthesis.On LoCoMo, it scores 0.777 overall, 11.0% above the strongest baseline, while reducing construction time to 158 s and average query latency to 0.93 s.Source: §12.4 · pp. 69–70

13.1 Confidence-Based Routing

• Confidence-based routing sends routine, high-confidence cases to System 1 and ambiguous or complex cases to System 2 using a threshold tuned on validation data.The threshold may adapt to task difficulty and compute budget; illustrative 10–30% escalation rates have not been independently verified across diverse applications.Source: §13.1 · p. 70, §13 · p. 70
• Near-chance System 1 accuracy can drive 90–100% escalation, while over 80% correlated errors between models leaves little cascade benefit.The passage recommends measuring base accuracy and error correlation before building a cascade.Source: §13.1 · p. 72

13.2 System 1 Models as Judges

• Jev-style judging reports 70–80% lower cost with less than 2% accuracy loss versus full LLM judging by escalating uncertain responses.The System 1 model scores candidates in parallel at approximately 10 ms each.Source: §13.2 · p. 72
• CLM judging is often near chance and trails same-size autoregressive judges, although temperature scaling reduces ECE to at most 0.062 and answer-order flips are rare.At a 0.97 retention bar, the cascade escalates 92.3–100% of items; calibration alone does not make this judge effective for cascading.Source: §13.2 · p. 72
• Jev’s most confident judging errors were repeated by LLM judges about 96% of the time, limiting oracle-threshold cascade gains to at most 2.7 accuracy points.The passage concludes that effective cascading requires complementary errors, not just a fast judge followed by a stronger one.Source: §13.2 · p. 72

13.3 Speculative Decision-Making

• Speculative decision-making accepts high-confidence proposals, rejects low-confidence ones to a safe default, and sends intermediate cases to System 2 for verification.The passage relates this pattern to speculative decoding and multi-step agent trajectories.Source: §13.3 · pp. 72–73

13.4 Multi-Agent System 1 Orchestration

• Jev-LiteMAS assigns inter-agent dispatch, priority, and capability checks to System 1 models, reducing routing overhead to under 50 ms per hop for teams of 10+ specialists.Its routing primitives use Choice, Score, and Noul models, respectively.Source: §13.4 · p. 73

13.5 System 1-Controlled Memory Compaction

• Prune the Summary uses Noul relevance decisions and Score importance weights to compact memory, reaching 94% of oracle-compacted performance at 12× lower cost than LLM-based summarization.It evaluates this result on three long-horizon benchmarks.Source: §13.5 · p. 73

14 Benchmarks and Evaluation

• No single benchmark captures the full System 1 profile: existing suites leave gaps in calibration, latency, cost, or structured-decision evaluation.Practitioners must combine results across benchmarks that may use incompatible protocols.Source: §14 · p. 74

14.1 Existing Benchmarks

• Prism-1 finds near-zero position and rival sensitivity for contrastive models but 2–8% position sensitivity for Family III; HakemBench reports ECE above 12% on Turkish inputs despite acceptable accuracy.Both findings motivate evaluating properties beyond accuracy alone.Source: §14.1 · p. 75
• Jev leads Qwen on 27 of 37 evaluated datasets and Gemma on all 37, while performance drops for all three systems on under-resourced languages, fine-grained or noisy labels, and rubric-style assessment.On UNFAIR-ToS, tuning the binary Noul threshold raises micro-F1 from 0.50 to 0.75.Source: §14.1 · p. 75

14.2 Evaluation Dimensions

• A comprehensive evaluation should jointly measure five dimensions, and reported typical costs range from $0.50–$5 per million decisions for Family I to $50–$500 for LLM alternatives.The five dimensions are decision accuracy, calibration, latency, cost efficiency, and robustness.Source: §14.2 · pp. 75–76

15 Robustness of System 1 Decision Models

• System 1 decisions can fail silently when a scalar or categorical output flips, making robustness important as models move into production middleware.The chapter examines position bias, framing sensitivity, and fallback reliability.Source: §15 · p. 77

15.1 Position and Ordering Bias

• Family III models show 2–8% accuracy shifts on reordered inputs, versus 12–22% for autoregressive judges, while contrastive models are largely immune by construction.The reported comparison concerns position bias under option reordering.Source: §15.1 · p. 77

15.2 Adversarial Framing Sensitivity

• Appending one opinion sentence reverses 12% of predictions and pushes 38% of confident predictions below the 0.8 escalation threshold; a Noul injection detector reports 94.2% accuracy at 8 ms.The passage connects confidence degradation to unnecessary escalation in cascades and describes System 1 defense-in-depth.Source: §15.2 · pp. 77–78

15.3 Fallback Option Reliability

• Fallback options are inconsistently selected: models may overcommit to poor candidates or choose “none” when a valid option exists, so evaluation should measure false negatives when none is correct.This boundary matters in routing, where clarification may be preferable to selecting a tool, and in safety gating, where uncertainty supports escalation.Source: §15.3 · pp. 78–79

Chapter VI Open Challenges; 16 Open Challenges and Future Directions

• The chapter groups open challenges into architectural, methodological, and deployment categories and surveys current status and promising research directions across them.Topics include expressiveness, composability, multimodal coverage, distribution shift, scaling, theory, privacy, fairness, continual learning, and energy efficiency.Source: §16 · p. 80, Chapter VI Open Challenges · p. 80

16.1 The Expressiveness–Speed Trade-off

• System 1 models trade expressiveness for speed, while the moderately complex 50–200 ms regime remains largely unexplored.Proposed directions include adaptive compute, lightweight explanations, and hierarchical routing, but tighter System 1/System 2 integration remains open.Source: §16.1 · p. 80, Figure 20 · p. 80

16.2 Calibration Under Distribution Shift

• Under label shift, Table 13 reports ECE worsening 15× (0.01 → 0.15) while accuracy drops modestly; 50 labeled target examples can recover most lost calibration quality.The chapter notes that calibration can degrade faster than accuracy, silently undermining cascades, safety gates, and other systems that rely on confidence scores.Source: Table 13 · p. 81, §16.2 · pp. 81–82

16.3 Standardized Decision Interfaces; 16.4 Extending Multi-Modal Coverage

• Choice, Score, and Noul may not cover every decision type, though NumericJev composes Choice decisions to achieve 1.84% range-normalized MAE versus 5.18% for direct selection.The chapter also identifies missing formal interface specifications, interoperability certification, and standardized serialization, while structured-data coverage and cross-modal calibration remain unresolved.Source: §16.3 · p. 82, §16.4 · p. 82

16.5 Continual Learning and Forgetting

• Continual learning must preserve calibration as well as accuracy: accuracy can remain high while ECE rises, and adapting existing methods across heterogeneous decision heads remains open.Post-hoc calibration and dynamic gradient calibration are described as complementary approaches; LPC jointly calibrates logits and parameters for sequential NLU tasks.Source: §16.5 · pp. 82–83, Figure 22 · p. 83

16.6 Privacy and Fairness

• Global calibration can mask subgroup miscalibration, and the interaction between fairness constraints and overall calibration—and whether RLCD inherits or amplifies training-data bias—remains unknown.A separate security study reports prompt-injection, poisoning, and inference attacks against Jev deployments, motivating dedicated defenses for System 1 decision models.Source: §16.6 · pp. 83–84

16.7 Scaling Laws for Decision Models; 16.8 Theoretical Foundations

• Analogous scaling laws for System 1 models do not yet exist, and sparse preliminary evidence leaves their scaling behavior uncertain.The chapter also identifies open theoretical questions about capacity for calibrated decisions, RLCD convergence, contrastive–Bayesian connections, and sample complexity.Source: §16.7 · p. 84, §16.8 · p. 84

16.9 Composability and Decision Chaining; 16.10 Energy Efficiency and Environmental Impact

• Verdict encodes a document once for multiple decision heads, making ten chained decisions cost roughly 1.3× a single decision rather than 10×; whether joint training improves results remains unexplored.The chapter calls for joint optimization of decision sequences and for energy comparisons that include both inference and amortized training costs.Source: §16.9 · p. 85, §16.10 · p. 85

17.1 Common Pitfalls

• Uncalibrated confidence and distribution shift can silently undermine cascades and escalation, so measure ECE on held-out data and monitor calibration on fresh data.The checklist also advises testing correlated cascade errors and treating inputs to safety gates as untrusted, since a single unverified opinion can flip 12% of decisions.Source: §17.1 · p. 86
• Extraction confidence measures the likelihood that an extracted value is correct, not decision-level uncertainty; do not assume it is suitable for routing or safety gates.The checklist also recommends starting with an API before adding custom training or deployment optimization, and evaluating local-first options when latency, privacy, or cost matter.Source: §17.1 · p. 86

17.2 Production Monitoring

• The production monitor tracks ECE on a rolling window and alerts when it exceeds a threshold; the example uses a 10,000-item window and 0.05 threshold.The implementation waits for at least 100 observations before reporting ECE.Source: §17.2 · p. 87

17.3 Production Deployment Checklist

• The deployment checklist recommends quantization, batching, input truncation, cached embeddings, and runtime export to improve inference efficiency.It also calls for monitoring P99 latency, rolling ECE, escalation rate, throughput, and errors, plus monthly IIA tests and local-first evaluation at high volume.Source: §17.3 · p. 87

18 Conclusion

• System 1 models provide fast, structured decisions that complement the slower reasoning of autoregressive LLMs.The conclusion describes calibration as a differentiator and presents System 1/System 2 cascades as a deployment pattern, with escalation rates dependent on application and quality requirements.Source: §18 · pp. 87–88
• The open-source ecosystem includes an RLCD training pipeline, local-first inference, and approximately 70 reproductions within two weeks of Jev’s launch.Commercial offerings include OpenAI’s Decisions API, Microsoft-Decision-1, and Clairic; the conclusion also notes expanding applications and architectural approaches.Source: §18 · p. 88

A Open-Source Decision Model Zoo

• Table 14 is a practitioner reference of open-source decision and extraction models, organized by parameter count; its listed models use permissive Apache-2.0 or MIT licenses.The table identifies RLCD training and supported primitives: Choice, Score, Noul, and Box.Source: A Open-Source Decision Model Zoo · p. 89, Table 14 · p. 89

B Quiz Questions and Detailed Answers

• The quiz provides self-assessment questions and detailed answers spanning foundational concepts, model families, methods, applications, evaluation, and practice.It is intended to reinforce understanding of the guide’s concepts and trade-offs.Source: B Quiz Questions and Detailed Answers · p. 90

B.1 Chapter I: Foundations

• System 1 models produce bounded structured decisions through dedicated, non-autoregressive or few-pass heads; calibration, latency, and cost are evaluated properties, not membership criteria.Strictly proper scoring rules incentivize truthful probabilities; calibration alone is insufficient because a base-rate predictor can be calibrated but uninformative, so sharpness also matters.Source: §B.1 · p. 90

B.2 Chapter II: Model Families

• The five families are encoder classifiers, reward models, non-autoregressive decision engines, contrastive models, and extraction models, with calibration strategy their sharpest differentiator.Encoders use classification heads and typically post-hoc calibration; reward models need an added layer; Family III emphasizes end-to-end proper-scoring-rule training; contrastive models need calibration layers; extraction confidence is edit-distance based, with boundaries to Family III still fluid.Source: §B.1 · p. 91
• Family III uses typed Choice, Score, and Noul outputs, while contrastive and extraction models address candidate similarity and typed-value retrieval, respectively.CLM-8B’s dual encoder allows cached action embeddings and efficient scoring of large candidate sets, but its information bottleneck can lose token-level interactions; Jev 1.14’s Box primitive applies RLCD calibration to extraction confidence.Source: §B.2 · pp. 92–93

B.3 Chapter III: Methods

• ECE depends on arbitrary bins and conflates calibration with discrimination; temperature scaling is simple and preserves ranking but assumes uniform miscalibration and requires matching validation data.Brier-score calibration components offer a binning-free alternative, with reliability diagrams for visualization; distribution shifts require recalibration, while RLCD puts proper scoring rules into training.Source: §B.3 · p. 93

Aspect RLHF RLCD

• RLHF optimizes human preference and can degrade calibration, whereas RLCD uses strictly proper scoring rules to incentivize truthful probabilities.Distillation transfers a teacher’s soft-label class similarities alongside hard labels, enabling a 350M-parameter encoder student to inherit decision quality from a 70B-parameter LLM teacher.Source: Aspect RLHF RLCD · pp. 93–94

B.4 Chapter IV: Applications and Integration

• A 20-step trajectory adds 10 seconds of routing with LLMs at about 500 ms per decision, versus 600 ms with System 1 routing at about 30 ms.A System 1 + System 2 cascade can avoid 80% of System 2 cost when it handles 80% of decisions, but near-chance accuracy or correlated errors can erase the benefit.Source: §B.4 · pp. 94–95
• System 1 classifiers support prevalence estimation by aggregating predictions, while CROW uses classifier decisions to organize and navigate a structured knowledge base instead of RAG retrieval.Quantification depends on calibrated posteriors; CROW makes routing, relevance, and modify-versus-new decisions during ingestion, then scores folders and notes during retrieval, falling back to an LLM when uncertain.Source: §B.4 · p. 95

B.5 Chapter V: Evaluation and Robustness

• ∆pos = 2–8% for Family III models, while Family IV models achieve near-zero ∆pos and ∆rival on Prism-1.Dual-encoder models score candidates independently, supporting position and rival independence; cross-encoders process options jointly, enabling deeper interactions but making scores sensitive to option position and competitors.Source: §B.5 · pp. 95–96
• Softmax-based Choice heads violate IIA because adding a decoy option can shift probability mass between existing options and change the top choice.This can make production routing unpredictable and push confident cascade predictions below escalation thresholds; contrastive models or independent Noul heads avoid the shared-simplex issue.Source: §B.5 · p. 96
• System 1 evaluation spans accuracy, calibration, latency, cost, and robustness, but no comprehensive benchmark jointly measures all five dimensions.Most benchmarks emphasize accuracy; JevBench and JevArena add calibration, but latency and cost remain unstandardized.Source: §B.5 · p. 96

B.6 Chapter VI: Open Challenges

• Continual learning must preserve calibration on old decision types as well as accuracy; preserving 95% accuracy while ECE rises from 3% to 12% can silently break downstream systems.LPC, dynamic gradient calibration, and continual post-hoc calibration are proposed approaches, but preserving calibration across heterogeneous Choice, Score, and Noul heads remains unsolved.Source: §B.6 · p. 97
• JevAdvBench found that one appended opinion sentence reversed 12.1% of predictions and pushed 38% of confident predictions below the 0.8 escalation threshold, versus a 10.1% reversal rate for the strongest injected-command attack.Recommended cascade defenses include input validation, a Noul pre-filter with 94.2% detection accuracy at 8 ms, threshold margins, and robustness audits.Source: §B.6 · p. 97