PaperlayerView exact revision on arXiv

Source-linked AI summary

Computational Thinking Reasoning in Large Language Models

Kechi Zhang, Ge Li, Jia Li, Huangzhao Zhang, Jingjing Xu, Hao Zhu, Lecheng Wang, Jia Li, Yihong Dong, Jing Mai, Bin Gu, Zhi Jin

arXiv:2506.02658v2cs.SE

TL;DR

LLMs often lack computational-thinking paradigms for complex reasoning, and tool use may remain auxiliary rather than part of a cohesive reasoning process. CTM interleaves language reasoning with live code execution and trains computational-thinking behaviors; the paper reports gains across code and mathematical benchmarks, including two code benchmarks.

  • Problem

    LLMs lack computational-thinking paradigms for complex reasoning, while external tools may supplement reasoning without integrating into a cohesive thinking process.

    Source: §I · p. 2

  • Method

    CTM interleaves natural-language reasoning with live code execution and uses supervised fine-tuning followed by reinforcement learning to develop computational-thinking strategies.

    Source: §I · pp. 1–2

  • Results

    CTM outperforms conventional reasoning and tool-augmented baselines across code and mathematical benchmarks; with reinforcement learning, it scores 66.5 on LiveCodeBench and 28.5 on CodeContests.

    Source: §I · p. 2, A. RQ1: Performance on Code Problems · p. 7

  • Takeaways & Limitations

    CTM frames complex reasoning as computational workflows that are intended to be robust and verifiable.

    Source: §VIII · p. 11

    The benchmark evaluation may not fully capture real-world tasks involving long-term planning or multimodal inputs.

    Source: C. Threats to Validity · p. 10

Abstract

from arXiv · show

While large language models (LLMs) have demonstrated remarkable reasoning capabilities, they often struggle with complex tasks that require specific thinking paradigms, such as divide-and-conquer and procedural deduction, \etc Previous researches integrate external, reliable tools to alleviate logical inconsistencies and hallucinations in LLMs' problem-solving processes. However, we argue that the root challenge is more profound: LLMs lack the complex thinking paradigms (\ie, computational thinking) during reasoning. In this paper, we propose Computational Thinking Model (CTM), a novel framework that incorporates computational thinking paradigms into LLMs. This framework enables LLMs to reformulate complex problems through decomposition, abstraction, reduction, and simulation, among other techniques. Specifically, live code execution is seamlessly integrated into the reasoning process, allowing CTM to think by computing. CTM directly instills computational thinking objectives into LLMs through tailored reinforcement learning rewards, which encourages problem simplification, modular planning, and iterative verification. We conduct extensive evaluations on multiple code generation and mathematical benchmarks. The results demonstrate that CTM outperforms conventional reasoning models and tool-augmented baselines in terms of accuracy, interpretability, and generalizability. We hope this study offers valuable insights for AI reasoning, where LLMs can transform problems into robust, verifiable, and scalable computational workflows, much like computer scientists do.

I. INTRODUCTION

The introduction argues that LLMs lack foundational computational-thinking paradigms, leaving complex reasoning without systematic verification. CTM integrates those paradigms into an interactive, code-executing framework and is reported to outperform conventional and tool-augmented reasoning models.

I. INTRODUCTION

  • LLMs lack computational-thinking paradigms such as decomposition, reduction, abstraction, and simulation, which computer scientists use to tackle complex problems.The paper characterizes computational thinking as transforming complex problems into manageable forms through methods including decomposition, enumeration, and verification.Source: §I · pp. 1–2
  • CTM makes computational thinking a primary reasoning strategy by interleaving natural-language reasoning with live code execution and its feedback.The interactive process supports decomposition, abstraction beyond textual limits, and explicit intermediate computation.Source: §I · pp. 1–2
  • Across code and mathematical benchmarks, CTM outperforms conventional reasoning LLMs and tool-augmented baselines.The paper also reports higher accuracy, interpretability, and generalizability across diverse benchmarks.Source: §I · p. 2
  • CTM uses supervised fine-tuning on constructed structured-reasoning data as a warm start, followed by reinforcement learning with a custom reward function.The approach is intended to help the model learn adaptive reasoning strategies aligned with problem-solving objectives.Source: §I · p. 2

II. MOTIVATING EXAMPLES

The motivating examples identify two weaknesses in natural-language reasoning: self-correction is rare, and symbolic calculations can violate formal rules. These cases illustrate why complex reasoning may need dynamic revision and structured computation.

1) The Rare ”Aha Moment” with Self-Edit Capabilities:

  • Chain-of-thought models often follow linear reasoning paths, allowing early errors to propagate without robust mechanisms to identify and revise them.The motivating examples frame dynamic revision as particularly relevant to complex reasoning tasks.Source: 1) The Rare ”Aha Moment” with… · pp. 2–3
  • Self-editing can occur during training, but such “aha moments” are rare and difficult to elicit consistently.The authors describe these occurrences as evidence of latent self-editing potential, while noting that systematic self-correction remains challenging to train.Source: 1) The Rare ”Aha Moment” with… · pp. 2–3

2) Symbolic Calculation in Intermediate Steps:

  • Current LLMs treat mathematical symbols as text patterns, leading to invalid derivations and errors in intermediate calculations, even on simple tasks.The paper reports these weaknesses in tasks such as polynomial factorization and modular arithmetic, including examples from DeepSeek-R1.Source: 2) Symbolic Calculation in… · p. 3

B. Computational Thinking as a Foundational Remedy

The paper presents computational thinking as a remedy for opaque text generation: CTM interleaves language reasoning and code execution so intermediate steps can be computed and checked. Its interactive framework uses execution feedback and persistent state to support iterative refinement.

B. Computational Thinking as a Foundational Remedy

  • CTM turns reasoning into an iterative, executable process: live-code outputs act as validation checkpoints, while symbolic operations are grounded in formal computation.The authors associate this shift from text generation to computational workflows with more reliable, scalable, and verifiable problem-solving.Source: B. Computational Thinking as a… · p. 3

III. METHOD

A. Interactive Language-Code Reasoning Framework

1) Code Execution Integration in Computational Thinking Models:

  • Persistent sandbox state lets CTM build on variables, functions, and intermediate results across code executions.This supports modular, iterative problem-solving; the process continues until the model signals completion and gives a final solution.Source: 1) Code Execution Integration in… · p. 4

2) Execution Environment and Result Integration:

The execution environment preserves notebook state and returns execution results to the model as textual feedback. This loop allows code to be revised and retested, while the implementation uses an isolated interpreter and trims verbose error messages.

2) Execution Environment and Result Integration:

  • A notebook-style environment persists state across code cells, capturing errors and standard outputs while exposing newly defined variables and functions to later blocks.The implementation uses an isolated Jupyter mode through Sandbox Fusion and retains only the final line of verbose execution errors to limit context length.Source: 2) Execution Environment and Result… · p. 4
  • CTM appends execution results to its context and can revise and retest code when an error arises before producing a sufficiently correct answer.The described loop captures outputs and returns them to the model as feedback.Source: 2) Execution Environment and Result… · p. 4

B. Training Strategy For Computational Thinking Models

CTM uses a cold-start supervised fine-tuning phase followed by reinforcement learning to teach reasoning through mixed natural-language and code traces. The phases establish structured problem-solving patterns, then optimize rollouts using task-specific correctness rewards.

1) Cold Start Supervised Fine-tuning:

  • SFT trains on curated traces that interleave natural-language explanations and executable code, modeling iterative problem solving across math and programming tasks.The dataset samples 13k code questions and 11.9k math questions and uses traces that may include hypothesizing, implementation, debugging, refinement, and validation.Source: 1) Cold Start Supervised Fine-tuning · pp. 4–5
  • SFT masks sandbox execution outputs so the model learns reasoning patterns rather than predicting specific computational results.The paper says this reduces reliance on memorization and improves training stability while laying groundwork for feedback-driven refinement.Source: 1) Cold Start Supervised Fine-tuning · p. 5

2) Reinforcement Learning with NL-Code-Mixed Rollouts:

  • RL samples mixed natural-language/code rollouts and uses DAPO with rule-based rewards to optimize reasoning efficiency and reliability.Programming rewards reflect passing sandbox test cases; math rewards reflect exact or equivalent answers verified against ground truth.Source: 2) Reinforcement Learning with… · p. 5
  • RL uses only batches containing both successful and failed rollouts, providing a gradient signal for iterative solution refinement.The model writes code, evaluates results, and adjusts generation patterns.Source: 2) Reinforcement Learning with… · p. 5

C. Inference Enhanced with Code Execution Environment

At inference, CTM alternates natural-language reasoning with sandboxed code execution and revises its trajectory using execution outcomes. The evaluation asks how the model performs on code and math benchmarks, how its reasoning trajectories behave, and how design choices affect performance.

C. Inference Enhanced with Code Execution Environment

  • CTM appends natural-language reasoning and code to a shared context, executes code in a persistent sandbox, and uses outputs to refine its solution.The paper says this iterative correction can reduce the likelihood of logical or computational failures and makes reasoning steps verifiable.Source: C. Inference Enhanced with Code… · p. 6

A. Research Questions

  • RQ1 evaluates code problem-solving across six benchmarks, from function-level generation to programming contests.The benchmarks include LiveCodeBench, CodeContests, HumanEval, HumanEval+, MBPP, and MBPP+.Source: A. Research Questions · p. 6
  • RQ2 assesses generalization beyond code using AIME 2024, AIME 2025, and MATH-500 mathematical reasoning benchmarks.The benchmarks include elementary and advanced problems requiring complex reasoning and precise intermediate computation.Source: A. Research Questions · p. 6
  • RQ3 examines whether model trajectories reflect decomposition, abstraction, and iteration through subproblem solving, intermediate computation, and error correction..Source: A. Research Questions · p. 6
  • RQ4 varies reward design, training-data curriculum, and interactive mixed-language environments to assess the impact of CTM's design choices.The environment variation includes Jupyter-style execution.Source: A. Research Questions · p. 6

B. Benchmarks and Metrics

CTM is evaluated on nine code and non-code benchmarks, with code generation and mathematical reasoning measured using task-specific correctness criteria. Comparisons cover proprietary, open-source, and tool-augmented systems.

B. Benchmarks and Metrics

  • The evaluation covers nine benchmarks spanning code generation and mathematical reasoning.The study frames this breadth through its Software Engineering for AI motivation.Source: B. Benchmarks and Metrics · p. 6
  • Six programming benchmarks cover functional code generation and contest tasks, including HumanEval(+), MBPP(+), CodeContests, and LiveCodeBench.LiveCodeBench contains 880 contest problems released between May 2023 and January 2025; CodeContests has 165 questions.Source: B. Benchmarks and Metrics · p. 6
  • Three math benchmarks assess problem solving: AIME 2024, AIME 2025, and MATH-500.The paper describes MATH-500 as requiring long-horizon reasoning.Source: B. Benchmarks and Metrics · p. 6
  • Code performance is measured by hidden-test pass rate, while math performance is measured by answer accuracy using math-verify..Source: B. Benchmarks and Metrics · p. 6

C. Compared Baselines

  • CTM is compared with proprietary models, open-source systems, and tool-augmented agents, with Qwen2.5-32B-Instruct included as its training starting point.The paper states that comparisons use identical evaluation settings.Source: C. Compared Baselines · pp. 6–7

D. Experimental Setups

CTM's RL method reports the best results on the two complex competitive-programming benchmarks, while the paper also reports gains over leading reasoning models. Experiments start from Qwen-2.5-32B-Instruct and use deterministic greedy decoding for evaluation.

D. Experimental Setups

  • Experiments initialize CTM from Qwen-2.5-32B-Instruct, train with SFT and RL, and use a Jupyter-style execution environment on 32 A100-80GB GPUs.Training typically completes within one week; Sandbox Fusion is selected for stability despite slightly higher latency.Source: D. Experimental Setups · p. 7
  • Greedy search is used at inference for deterministic, consistent evaluation results..Source: D. Experimental Setups · p. 7

A. RQ1: Performance on Code Problems

B. RQ2: Performance on Math Problems

CTM performs strongly on mathematical benchmarks, and its reasoning behavior features more code-inspired reasoning than the comparison models. Reinforcement learning training curves also show rising reward and code use.

B. RQ2: Performance on Math Problems

  • Scores of 76.6 on AIME 2024, 66.6 on AIME 2025, and 96.8 on MATH-500 accompany superior-or-comparable performance to powerful proprietary and open-source models.The authors report that these results generalize across code-specific and general-domain tasks.Source: B. RQ2: Performance on Math Problems · p. 8
  • The RL variant gains +33.1 on LiveCodeBench and +70.0 on AIME 2024 over the Qwen2.5-32B-instruct base model.The comparison evaluates training stages against a same-architecture base model without computational-thinking integration.Source: B. RQ2: Performance on Math Problems · p. 8

C. RQ3: Reasoning Behavior in Our CTM

  • Across four annotated behavior categories, CTM has a higher proportion of code-inspired reasoning, although overall behavior distributions are relatively similar to other reasoning models.GPT-4o assigns one primary behavior per output; the analysis samples 500 questions, and the authors say executable interaction guides more accurate, grounded reasoning.Source: C. RQ3: Reasoning Behavior in Our CTM · pp. 8–9, Figure 4 · p. 8
  • Reward rises steadily and the proportion of rollouts containing code tags increases sharply during reinforcement learning.The authors define the code-reasoning ratio as the share of rollouts containing <code>...</code> tags.Source: Figure 5 · p. 8, C. RQ3: Reasoning Behavior in Our CTM · p. 9

D. RQ4: Ablation Study: Effect of Code Integration

Ablations indicate that code integration matters in both training and inference. A case study illustrates iterative validation, while the framework's rollout-focused design supports compatibility with varied reinforcement-learning algorithms.

D. RQ4: Ablation Study: Effect of Code Integration

  • Removing code integration during either training or inference consistently reduces reasoning ability, indicating its importance in both phases.The ablations retain the same training questions when code is removed during training; the results are summarized in Table IV.Source: D. RQ4: Ablation Study: Effect of Code… · p. 9, Table IV · p. 9

A. Case Study of Computational Thinking Pattern

  • In a coding case study, CTM tests a brute-force solution, then iteratively optimizes it while using the reference implementation to maintain correctness.The authors describe this self-imposed validation pattern as helping detect potential errors and enhance solution robustness.Source: A. Case Study of Computational… · p. 9, Figure 6 · p. 9

B. Compatibility with Popular RL Algorithms

  • CTM modifies rollout behavior during decoding without changing model architecture or training-loss formulation, supporting compatibility with offline and online RL methods.Examples include Reject Sampling, DPO, PPO, GRPO, and advanced variants.Source: B. Compatibility with Popular RL… · pp. 9–10

C. Threats to Validity

The paper identifies benchmark, execution, training, and metric boundaries on its validity claims. It situates CTM among chain-of-thought and tool-integrated approaches, concluding that interleaved language and executable code can support robust, verifiable workflows.

C. Threats to Validity

  • Evaluation benchmarks may miss real-world long-term planning and multimodal tasks; Python-only execution and accuracy-focused metrics also limit scope and omit reasoning efficiency, clarity, and readability.The authors additionally flag possible bias from training configuration, execution-environment design, and curated traces that overrepresent problem types or tool patterns.Source: C. Threats to Validity · p. 10

A. Large Reasoning Models

  • Chain-of-Thought prompting articulates intermediate steps in natural language, while Program-of-Thought [4] uses external computation to simplify and validate operations.The paper distinguishes CTM by its interactive natural-language and code-mixed reasoning pattern.Source: A. Large Reasoning Models · p. 10

B. Tool-Integrated Models

  • CTM extends tool-integrated reasoning by applying it beyond mathematics and using tools to guide reasoning trajectories, with computational-thinking principles trained through SFT and RL.The paper contrasts this with concurrent work such as ReTool [6], which focuses on mathematical reasoning and relatively simple tool-use patterns.Source: B. Tool-Integrated Models · p. 10

VIII. CONCLUSION

  • CTM interleaves natural-language and executable-code reasoning to transform problems into computational workflows described as robust and verifiable.The conclusion frames this as integrating computational-thinking principles into LLM reasoning.Source: §VIII · p. 11, B. Tool-Integrated Models · p. 11
Loading 2506.02658v2…