PaperlayerView exact revision on arXiv

Source-linked AI summary

GUT: Quantifying and Optimizing the Reasoning Uncertainty of LLMs via Graph Complexity

Shuang Liang, Xin-Yu Hu, Xiang-Jun Ou, Shao-Qun Zhang

arXiv:2609.05284v1cs.AI

TL;DR

LLM reasoning is uncertain because stochastic generation produces variable and sometimes nonsensical reasoning branches. GUT models these branches with directed acyclic graphs, quantifies uncertainty through graph complexity, and optimizes it with reinforcement learning; across four LLMs and five datasets, GUT-Q outperformed 45 uncertainty-quantification contenders in selective generation, while GUT-O reduced uncertainty and improved accuracy.

  • Problem

    Stochastic token sampling makes LLM reasoning variable, while representing reasoning chains merely as sequences fails to capture their inherent uncertainty.

    Source: §1 · p. 1, §2 · p. 4

  • Method

    GUT represents potential reasoning branches as directed acyclic graphs, estimates uncertainty with graph complexity, and optimizes it using negative uncertainty as a reinforcement-learning reward.

    Source: §1 · p. 3

  • Results

    Across 4 LLMs and 5 datasets, GUT-Q outperformed 45 uncertainty-quantification contenders on selective generation, while GUT-O reduced uncertainty and improved accuracy.

    Source: §1 · p. 3

  • Takeaways & Limitations

    GUT provides a graph-complexity framework for quantifying and reducing uncertainty in LLM reasoning within the evaluated tasks.

    Source: §1 · p. 3

    Prompt-based generation remains fundamentally limited by inherent stochasticity, which induces reasoning variability.

    Source: §1 · p. 2

Abstract

from arXiv · show

Recent years have witnessed great advances in the reasoning ability of Large Language Models (LLMs). However, the reasoning processes of LLMs often exhibit uncertainty, where LLMs often produce a proliferation of divergent branches at each reasoning step even when fed the same prompting inputs, and certain branches exhibit evidently incredible, even nonsensical, reasoning chains and results. In this paper, we propose the Graph-complexity-based UncerTainty (GUT) method for investigating the reasoning uncertainty of LLMs. The key idea of GUT is to characterize the potential branches of each reasoning chain with a directed acyclic graph, thereby ensuring that all potential branches are comprehensively covered within the graph space. Building upon this recognition, we further build two modules of GUT, that is, a Quantification (GUT-Q) module and an Optimization (GUT-O) module, for quantifying and reducing the reasoning uncertainty of LLMs, respectively. GUT-Q measures LLM reasoning uncertainty by approximating the reasoning space complexity with graph complexity. GUT-O implements uncertainty optimization by treating negative uncertainty as the reward function in reinforcement learning. Experimental results conducted on four LLMs and five datasets validate the effectiveness of GUT.

1. Introduction

GUT addresses LLM reasoning uncertainty caused by token-level stochasticity and sampling by modeling potential reasoning branches as graphs. It provides GUT-Q for quantification and GUT-O for optimization through reinforcement learning.

Motivation

  • Temperature-based token sampling induces variable reasoning chains, while sequence-based uncertainty methods overlook potential branches at individual reasoning steps.Self-judgment methods are also limited because stochastic generation recursively amplifies uncertainty.Source: §1 · pp. 1–2
  • Graph-based uncertainty quantification is needed because precise branch characterization must account for both token-level distributions and stochastic sampling.Existing graph methods model branches effectively but rely heavily on self-prompting, creating an effectiveness bottleneck.Source: §1 · p. 2

Proposed method

  • GUT-Q estimates reasoning-space complexity with graph complexity, integrating token-distribution statistics and graph topology to characterize branches from multiple samples merged using NLI.This approach aims to trace uncertainty to both token-level distributions and stochastic sampling while avoiding self-prompting bottlenecks.Source: §1 · pp. 2–3

Empirical evidence

  • GUT-Q outperforms 45 uncertainty-quantification contenders across 4 LLMs and 5 datasets on selective generation measured by AUROC, AUPRC, and PRR.The introduction reports this empirical comparison without providing metric values.Source: §1 · p. 3

Optimization

  • GUT-O uses a negative uncertainty proxy as the reward in reinforcement learning to optimize LLM reasoning uncertainty.Across 4 LLM scales and 5 datasets, GUT-O reduces reasoning uncertainty and improves accuracy.Source: §1 · p. 3

2. Related Work

Prior work quantifies reasoning uncertainty through self-prompting, sequence-level statistics, and graph topology, but remains limited in modeling branches induced by stochastic generation. Reasoning uncertainty optimization has mainly used uncertainty as a signal, leaving direct reduction largely unexplored.

Reasoning Uncertainty Quantification

  • Self-prompting methods use multiple-choice [20] and reasoning-decomposition [47] templates, but token-level stochasticity recursively amplifies uncertainty and limits their effectiveness.The generation process is inherently stochastic because of token-level distributions and stochastic sampling.Source: §2 · p. 3
  • Sequence-based methods quantify uncertainty using entropy [25], likelihood [22], eccentricity [23], or semantic-cluster entropy, but sequence modeling misses branches emerging at each reasoning step.These approaches rely on statistical and information-theoretic measures over single or multiple sampled sequences.Source: §2 · pp. 3–4
  • Graph-based methods model reasoning spaces and quantify uncertainty with graph distance [8] or path counts [54], yet rely heavily on self-prompting despite more precise branch modeling.The reasoning space is defined as the set of all possible reasoning chains of a problem [5].Source: §2 · p. 4

Reasoning Uncertainty Optimization

  • Uncertainty-optimization studies use logits, entropy [14], top-1/top-2 probability gaps [57], or consecutive-position logit variation to explore or terminate uncertain reasoning, rather than reduce uncertainty.Consequently, uncertainty optimization in LLM reasoning remains largely unexplored.Source: §2 · p. 4

3. Uncertainty Quantification · Appendix A.2 lists the procedure of GUT-Q-H for graph complexity estimation.

GUT-Q quantifies LLM reasoning uncertainty by representing potential reasoning branches as a merged DAG, measuring step-level stochasticity, and estimating graph complexity. It includes width, height, and uncertainty-propagation estimators, with GUT-Q-H procedure details listed in Appendix A.2.

3.1. Graph Construction

  • GUT-Q constructs a DAG from sampled CoT chains, merging semantically equivalent reasoning-step nodes to expose the reasoning space’s potential branches.The graph is rooted at the problem, connects successive reasoning steps, and terminates paths at final-answer leaf nodes.Source: §3.1 · p. 5
  • GUT-Q uses bidirectional NLI entailment as its example node-merging method because it provides deterministic judgments with a small-scale NLI model.Embedding-based merging can require task-specific models and threshold tuning, while prompting an LLM incurs inference cost and stochasticity.Source: §3.1 · pp. 5–6

3.2. Step-level Uncertainty Quantification

  • GUT-Q captures step-level uncertainty by computing token-level uncertainty, aggregating local token groups, and averaging selected positions into node uncertainty.It uses four token-level uncertainty measures, and larger U(t_j) indicates greater uncertainty for token t_j.Source: §3.2 · p. 6

3.3. Graph Complexity Estimation

  • GUT-Q estimates graph complexity through three alternatives—uncertainty-weighted width, uncertainty-weighted height, and GUT-Q-UP—combining topology with node uncertainty.Width reflects branch divergence at a step, whereas height captures reasoning depth and token-level stochasticity along a longest path.Source: §3.3 · pp. 7–8
  • GUT-Q-W finds the maximum number of nodes at one topological level and sums their node uncertainties to quantify branch divergence.Topological sorting supports the width calculation over the DAG.Source: §3.3 · p. 7
  • GUT-Q-H selects a longest root-to-leaf path and sums node uncertainties along it, thereby capturing reasoning-space depth and stochasticity.The height is measured by the number of edges, and the path is identified across connected topological levels; Appendix A.2 lists its procedure.Source: §3.3 · pp. 7–8, Appendix A.2 · p. 8

Appendix A.2 lists the procedure of GUT-Q-H for graph complexity estimation.

  • GUT-Q-UP propagates node uncertainty through the DAG as an FNN, connecting initial reasoning steps to an auxiliary node receiving all final-answer nodes.Its output incorporates uncertainty propagated throughout the reasoning space, with node uncertainties initializing the FNN biases.Source: Appendix A.2 · p. 8

Appendix A.2 lists a variant that initializes this FNN’s weights with U(v). · 4. Uncertainty Optimization

The appendix contrasts graph-complexity estimators by explainability and selective-generation performance, while Section 4 introduces GUT-O to optimize non-differentiable uncertainty indirectly through a differentiable surrogate. MTLP’s consistently positive correlation with complexity-based uncertainty supports this optimization strategy, which uses negative MTLP as the GRPO reward.

Appendix A.2 lists a variant that initializes this FNN’s weights with U(v).

  • GUT-Q-W and GUT-Q-H provide greater explainability than GUT-Q-UP by characterizing the intrinsic topology of the reasoning space.GUT-Q-W captures the maximum potential branches at one step, whereas GUT-Q-H integrates the longest reasoning-chain length.Source: Appendix A.2 · p. 8
  • GUT-Q-UP typically outperforms GUT-Q-W and GUT-Q-H on the downstream selective-generation task [34].Thus, estimator choice depends on whether intrinsic topology explainability or downstream selective generation is the priority.Source: Appendix A.2 · p. 8

4. Uncertainty Optimization

  • GUT-O formally optimizes LLM reasoning uncertainty U(x) as measured by GUT-Q.The module treats uncertainty as an optimization target while addressing the discrete node-merging structure of the reasoning graph.Source: §4 · p. 9
  • Because node merging makes U(x) non-differentiable with respect to LLM parameters, GUT-O uses the law of large numbers to optimize differentiable conditional uncertainty U(x, c).The method approximates U(x) through an expectation over contexts, so optimizing U(x, c) also optimizes U(x).Source: §4 · p. 9
  • MTLP [26] serves as a differentiable example of U(x, c), enabling direct optimization with respect to LLM parameters.Its validity is evaluated through Pearson correlation with complexity-based uncertainty across mathematical-reasoning datasets, averaged across four Qwen3 LLMs.Source: §4 · p. 9
  • PCCs are consistently positive across all investigated mathematical-reasoning datasets, validating MTLP as the optimization target; Table 1 reports averages across four Qwen3 LLMs.GUT-O then applies gradient-based GRPO [36], using negative MTLP as the reward, following Prabhudesai et al. [31] and Zhao et al. [56].Source: §4 · p. 9, Table 1 · p. 9

5. Experiments … B. Additional Experimental Details on GUT-Q

This section details the experimental setup and procedures for evaluating GUT-Q and GUT-O across four Qwen3 models and five datasets spanning mathematical, first-order logic, and long-form question-answering tasks. It examines uncertainty quantification and optimization using graph-complexity estimation, uncertainty-weighted variants, selective-generation evaluation, and reinforcement-learning optimization.

5. Experiments

  • The experimental suite uses Qwen3 models at 8B, 4B, 1.7B, and 0.6B across mathematical reasoning, first-order logic reasoning, and long-form QA datasets.The datasets are GSM8K, MATH-500, AMC2022-2024, FOLIO, and MMLU-Pro.Source: §5 · p. 10

5.1. Verifications on GUT-Q

  • The experiments evaluate selective-generation uncertainty quantification with AUROC, AUPRC, and PRR, where higher values indicate better discrimination of correct from incorrect generations.They compare 45 white-box and black-box contenders across diversity-based, information-based, self-reflexive, reasoning-enhanced, and graph-based categories.Source: §5.1 · p. 10
  • GUT-Q-UP outperforms the best uncertainty-quantification contenders by 11.79% in PRR, 13.33% in AUROC, and 9.66% in AUPRC across four LLMs and five datasets.GUT-Q-UP consistently leads across the three metrics and datasets spanning three difficulty levels, with comparable runtime complexity to methods such as SE and Eig-C.Source: §5.1 · p. 10
  • Node merging is more important than node uncertainty for GUT-Q-UP, while both components are essential for GUT-Q-W and GUT-Q-H.The ablation compares these components on Qwen3-4B across mathematical-reasoning datasets.Source: Figure 2 · p. 12, §5.1 · p. 12
  • The recommended GUT-Q settings are K = 9, T = 1.0, and F = 5, balancing computational efficiency with UQ performance.Larger K increases computational costs, while the sensitivity analyses evaluate sample count, temperature, and shot count.Source: §5.1 · pp. 12–13

5.2. Verifications on GUT-O

  • GUT-O reduces reasoning uncertainty by 13.95% and improves accuracy by 1.93% on average across four LLMs and five datasets, whereas GRPO typically leaves uncertainty comparable or higher.The results attribute GUT-O’s benefit primarily to the MTLP optimization target rather than task-specific fine-tuning.Source: §5.2 · pp. 13–14

A.1. Prompt Templates

  • The experiments use few-shot prompt templates with seven dataset-specific examples, a 3-shot default for extracting reasoning and final answers, and #### as the stopping string.Seven examples are drawn from GSM8K, MATH, and AMC training data, with the AMC examples removed from the evaluation set; shot-count sensitivity is analyzed elsewhere.Source: §A.1 · p. 16

A.2. Details of GUT-Q Algorithm

  • GUT-Q estimates reasoning uncertainty from graph complexity, with GUT-Q-H outputting uncertainty-weighted height H_u for this purpose.The appendix also gives pseudocode for constructing the directed acyclic graph and describes an auxiliary-node weighting variant initialized from node uncertainties and a bias hyperparameter ρ.Source: §A.2 · pp. 16–17
  • GUT-Q-UP and GUT-Q-UP-W achieve comparable UQ performance across all evaluated LLM scales, datasets, and uncertainty-quantification metrics.The comparison uses Algorithms 2 and 4, respectively, as reported in Table 4.Source: §A.2 · p. 17, Table 4 · p. 18

A.3. Details of GUT-O Formulation

  • GUT-O defines reward as negative uncertainty and optimizes the policy to favor reasoning paths with lower MTLP while clipping updates and penalizing KL divergence from a reference model.The clipped objective promotes higher-reward paths, whereas the KL regularizer limits policy deviation to mitigate reward hacking and preserve language capabilities.Source: §A.3 · p. 18

B. Additional Experimental Details on GUT-Q

  • The appendix supplies additional experimental results and implementation details for the proposed GUT-Q method.This material constitutes the additional GUT-Q experimental-details portion of the appendix.Source: B. Additional Experimental Details on… · p. 20

B.1. Details on Datasets · B.2. Configurations of Hyperparameters · B.3. UQ Contenders

The experiments span mathematical reasoning benchmarks of increasing difficulty, plus FOLIO and MMLU-Pro to assess generality. The appendix specifies GUT-Q sampling and generation settings, analyzes max_new_tokens trade-offs, and evaluates 45 UQ contenders.

B.1. Details on Datasets

  • The mathematical benchmarks comprise GSM8K, MATH-500, and AMC2022-2024, containing 1319, 500, and 128 test problems, respectively.The datasets cover grade-school, high-school, and competition mathematics; all tasks are fill-in-the-blank with numeric or vector answers.Source: §B.1 · p. 20
  • Accuracy decreases sequentially from GSM8K to MATH-500 to AMC2022-2024 across all evaluated LLM scales, establishing the intended difficulty ordering.Table 5 reports evaluated-model accuracy on the three datasets.Source: §B.1 · p. 20, Table 5 · p. 20
  • Reported accuracy may differ from [48] because the study uses a different prompt template to extract reasoning steps and final answers for UQ and a smaller token generation budget due to computational-resource constraints.The work mainly focuses on quantifying and optimizing reasoning uncertainty rather than improving accuracy, making accuracy differences from [48] acceptable.Source: §B.1 · p. 20
  • FOLIO and MMLU-Pro extend evaluation to first-order logic and challenging long-form question answering, using multiple-choice answers from task-specific label sets.FOLIO answers are True, False, or Uncertain, whereas MMLU-Pro answers are A, B, C, or D.Source: §B.1 · p. 21

B.2. Configurations of Hyperparameters

  • GUT-Q follows Fu et al. [14] for step-level uncertainty hyperparameters and uses different sampling setups for evaluating correctness and quantifying uncertainty.Source: §B.2 · p. 21

B.3. UQ Contenders

  • The study compares 45 UQ contenders, implementing CoT-UQ [52] and TopoUQ [8] from official repositories and the remainder through LM-Polygraph.Table 7 lists the contenders’ full names and abbreviations.Source: §B.3 · p. 22, Table 7 · p. 23

B.4. Runtime Complexity Analysis of GUT-Q and Its Contenders

GUT-Q’s runtime is primarily determined by LLM inference and is comparable to representative diversity-based and graph-based UQ methods. On MATH-500 with Qwen3-4B, its wall-clock time is comparable to SE, Eig-E, and Topo-UQ, while batch-parallel prompting improves practical efficiency.

Complexity analysis

  • The analysis covers eight representative methods—Ppl, MCSE, SE, P(True), CoT-UQ-ME, Eig-E, CoT-UQ-SP, and Topo-UQ—and parameterizes complexity by sampling, reasoning-step, graph, vertex, and edge counts.For SE and GUT-Q, auxiliary-model-guided merging is cheapest when chains or steps share one cluster and most expensive when each forms a distinct cluster.Source: §B.4 · p. 24

Runtime components

  • UQ runtime is dominated by the number of LLM inferences because LLM inference is more expensive than auxiliary-model execution or other computations.The analyzed runtime components are LLM inference, auxiliary-model execution, and computations such as token-level statistics and eigenvalue derivation.Source: §B.4 · p. 24

Complexity comparison

  • GUT-Q has runtime complexity comparable to the investigated diversity-based and graph-based UQ methods.It uses the same number of LLM prompts as SE and Eig-E with slightly more auxiliary-model executions, while matching Topo-UQ more closely on both quantities.Source: §B.4 · pp. 24–25

Runtime comparison

  • On MATH-500, GUT-Q’s average wall-clock time is comparable to existing state-of-the-art methods such as SE, Eig-E, and Topo-UQ, according to Figure 4.Its sampling time is only around 2.5 times that of Ppl, far less than K = 10 times, because batch-parallel processing can be performed.Source: Figure 4 · p. 25, §B.4 · p. 25

B.5. UQ Performance Evaluations · C. Additional Experimental Details on GUT-O · C.1. Implementation Details

Across five datasets, GUT-Q with the UP algorithm generally outperforms all contenders across three evaluation metrics and difficulty levels. The appendices provide GUT-O implementation details, training configuration, and hyperparameter settings.

B.5. UQ Performance Evaluations

  • The UQ evaluations cover Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B across five datasets.Tables 9–12 identify the corresponding model-specific comparisons.Source: §B.5 · p. 25, Table 9 · p. 26, Table 10 · p. 27
  • GUT-Q with the UP algorithm generally outperforms all contenders across three evaluation metrics, five datasets, and three difficulty levels, validating GUT-Q.Tables 9–12 report comparisons for Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B.Source: §B.5 · p. 25, Table 9 · p. 26, Table 10 · p. 27

C. Additional Experimental Details on GUT-O

C.1. Implementation Details

  • GUT-O was implemented with the open-r112 repository, with its hyperparameter configuration listed in Table 13.Table 13 summarizes the GUT-O hyperparameters.Source: §C.1 · p. 30, Table 13 · p. 30
  • The authors fine-tuned four Qwen3 models on the training sets of GSM8K [6], MATH [18], FOLIO, and MMLU-Pro.The models were Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B.Source: §C.1 · p. 30
  • For AMC2022-2024, the training set used a random selection of 20% of synthetic instances.Appendix B.1 provides further details about AMC2022-2024.Source: §C.1 · p. 30

C.2. UO Performance Evaluation · C.3. Validation of the MTLP optimization target · D. Additional Ablation and Sensitivity Analyses

GUT-O reduces reasoning uncertainty while improving accuracy across Qwen3 scales, datasets, and uncertainty metrics, and MTLP is validated as an optimization target for GUT-Q uncertainty. Additional analyses examine Qwen3 variants on FOLIO and MMLU.

C.2. UO Performance Evaluation

  • GUT-O reduces uncertainty and improves accuracy across Qwen3 scales, datasets, and UQ evaluation metrics.Table 14 compares UO performance for Qwen3-1.7B and Qwen3-0.6B across five reasoning datasets.Source: §C.2 · p. 30, Table 14 · p. 31
  • Table 14 reports percentage changes in mean uncertainty and absolute changes in accuracy for Qwen3-1.7B and Qwen3-0.6B.A plus sign indicates the applied fine-tuning method, while parenthesized values indicate percentage changes.Source: §C.2 · pp. 30–31, Table 14 · p. 31
  • The UO evaluation covers five reasoning datasets and compares performance across Qwen3 model families.The continued table presents additional comparisons for the Qwen3 families.Source: Table 14 · pp. 31–32

C.3. Validation of the MTLP optimization target

  • MTLP generally has significant positive PCCs with GUT-Q-derived uncertainty across datasets and all four Qwen3 scales.The evaluation considers Qwen3-0.6B, 1.7B, 4B, and 8B across five datasets.Source: §C.3 · pp. 32–33, Table 15 · p. 33
  • These positive correlations support MTLP as a valid optimization target for optimizing GUT-Q uncertainty U(x).Table 15 reports the PCCs between the MTLP target and GUT-Q-derived uncertainties.Source: §C.3 · p. 32, Table 15 · p. 33

D. Additional Ablation and Sensitivity Analyses

D.1. Additional Ablation Analyses

Additional ablations show that node merging is more important than node uncertainty for UP, while both components are essential to GUT-Q-W and GUT-Q-H width and height calculations.

D.1. Additional Ablation Analyses

  • Node merging is more important than node uncertainty in the UP algorithm across Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B.The ablation comparisons are shown in Figure 5a–d.Source: §D.1 · p. 33, Figure 5 · p. 34
  • Both node uncertainty and node merging are essential for width and height calculations in GUT-Q-W and GUT-Q-H.For both methods, the red and green bars are substantially shorter than the blue bars.Source: §D.1 · p. 33

D.2. Additional Sensitivity Analyses

Sensitivity analyses evaluate UQ performance across sample count, temperature, shot count, and node-merging criteria for four Qwen3 models. They recommend model-specific hyperparameters and find GUT-Q robust to node-merging criterion choices.

Number of Samples K

  • The recommended sample counts are K = 9 for Qwen3-4B, K = 12 for Qwen3-1.7B, and K = 11 for Qwen3-8B and Qwen3-0.6B.These settings balance efficiency and performance because larger K increases computational cost.Source: §D.2 · p. 35, Figure 6 · p. 36

Temperature T

  • The recommended temperatures are T = 1.6 for Qwen3-8B, T = 1.0 for Qwen3-4B, and T = 0.7 for Qwen3-1.7B and Qwen3-0.6B.These configurations are selected based on UQ performance.Source: Figure 7 · p. 37, §D.2 · p. 37

Number of Shots F

  • The recommended shot counts are F = 5 for Qwen3-8B and Qwen3-4B, F = 3 for Qwen3-1.7B, and F = 7 for Qwen3-0.6B.These settings are selected based on UQ performance.Source: Figure 8 · p. 38, §D.2 · p. 38

Node Merging Criterion

  • GUT-Q remains robust across bi-entailment, uni-entailment, and non-contradiction node-merging criteria, which yield comparable UQ performance across model scales, datasets, and evaluation metrics.The bi-entailment criterion is the original Algorithm 5 criterion.Source: Figure 9 · p. 39, §D.2 · p. 40

E. Case study of the GUT

The case study applies GUT to MATH-500 instances using Qwen3-4B, showing that graph complexity distinguishes incorrect from correct reasoning and that GUT-O produces shorter reasoning chains.

Case study overview

  • Figure 10 shows that incorrectly answered instances typically have larger graph-complexity metrics than correctly answered instances, supporting GUT-Q’s discrimination capability.The comparison uses two MATH-500 test instances with Qwen3-4B.Source: E. Case study of the GUT · p. 40, Figure 10 · p. 40
  • GUT-O tends to reduce the number of reasoning steps used by the LLM for the same question.This behavior is observed in the illustrated optimized Qwen3-4B reasoning chains.Source: E. Case study of the GUT · p. 40

Illustrative reasoning graphs

  • For the binomial-expansion instance, the pretrained Qwen3-4B reasoning includes a final answer of 23, while another shown reasoning path gives 20.The constructed DAG exposes divergent intermediate calculations and correction attempts, including values a = 11, b = 9, c = 3 and a + b + c = 23, as well as a + b + c = 20.Source: E. Case study of the GUT · p. 42
  • For the two-digit-prime instance, the correct reasoning identifies 17, 53, and 71 as primes whose digits sum to 8, yielding 3.The optimized Qwen3-4B is explicitly described as correctly answering this test instance.Source: E. Case study of the GUT · pp. 44–45

Related papers

Loading 2609.05284v1…