Source-linked AI summary
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, Brian Ichter
TL;DR
Language models can struggle when semantic reasoning must be expressed as executable code, limiting code-driven reasoning beyond arithmetic and logic. Chain of Code combines generated code with interpreter execution and LMulator simulation for non-executable semantic operations. It reports broad gains over baselines, including 84% on BIG-Bench Hard, while requiring additional computation and context and having documented implementation and task-scope limitations.
Problem
Semantic tasks such as sarcasm detection can be difficult to express as executable functions, raising whether code-driven reasoning can combine executable computation with language-based reasoning.
Method
Chain of Code generates code or pseudocode, executes executable lines with an interpreter, and uses an LMulator to simulate non-executable operations while updating program state.
Results
84% on BIG-Bench Hard is CoC’s reported state-of-the-art result, and CoC outperforms baselines across challenging numerical and semantic reasoning tasks.
Takeaways & Limitations
CoC extends code-driven reasoning to problems outside the scope of executable code, including semantic reasoning, while retaining code’s computational structure.
Takeaways & Limitations
CoC requires additional context length and computation time, does not help every semantic task, and cannot currently modify custom Python objects during simulated execution.
Abstract
from arXiv · showhide
Code provides a general syntactic structure to build complex programs and perform precise computations when paired with a code interpreter - we hypothesize that language models (LMs) can leverage code-writing to improve Chain of Thought reasoning not only for logic and arithmetic tasks, but also for semantic ones (and in particular, those that are a mix of both). For example, consider prompting an LM to write code that counts the number of times it detects sarcasm in an essay: the LM may struggle to write an implementation for "detect_sarcasm(string)" that can be executed by the interpreter (handling the edge cases would be insurmountable). However, LMs may still produce a valid solution if they not only write code, but also selectively "emulate" the interpreter by generating the expected output of "detect_sarcasm(string)". In this work, we propose Chain of Code (CoC), a simple yet surprisingly effective extension that improves LM code-driven reasoning. The key idea is to encourage LMs to format semantic sub-tasks in a program as flexible pseudocode that the interpreter can explicitly catch undefined behaviors and hand off to simulate with an LM (as an "LMulator"). Experiments demonstrate that Chain of Code outperforms Chain of Thought and other baselines across a variety of benchmarks; on BIG-Bench Hard, Chain of Code achieves 84%, a gain of 12% over Chain of Thought. In a nutshell, CoC broadens the scope of reasoning questions that LMs can answer by "thinking in code".
1. Introduction
Chain of Code addresses the difficulty of expressing semantic reasoning in executable code by combining code execution with language-model simulation. Across challenging numerical and semantic tasks, it reports stronger performance than popular baselines, including 84% on BIG-Bench Hard.
- CoT improves semantic reasoning but struggles with numeric or symbolic questions, while executable code offers precise algorithmic computation.Code supplies a general structure for complex programs and can use interpreters for computations such as multiplication and sorting.
- Semantic subtasks such as sarcasm detection can be difficult or nearly impossible to express as executable functions, motivating a method that combines code and language.The paper asks whether reasoning restrictions imposed by interpreter-executable traces can be relaxed.
- Chain of Code writes programs containing flexible pseudocode and uses an LMulator to generate outputs for lines that an interpreter cannot execute.Executable lines are handled by the interpreter, while undefined semantic operations are simulated by the language model and incorporated into program execution.
- CoC applies to numerical and semantic reasoning and outperforms popular baselines across challenging tasks.The experiments report performance on BIG-Bench Hard that exceeds average human raters overall and even the best human raters on an algorithmic subset.
2. Chain of Code: Reasoning with an LMulator
Chain of Code generates reasoning in code, pseudocode, or natural language and executes it with Python when possible or an LMulator when execution fails. This interleaving preserves code’s computational structure while extending it to semantic tasks and supporting improved reasoning performance.
- CoC uses in-context learning and prior intermediate-step methods as foundations for combining natural-language reasoning, code execution, and program-state simulation.Chain of Thought uses natural-language substeps, ScratchPad tracks program state, and Program of Thoughts generates executable code.
- 2.2. Chain of Code: CoC proceeds in two steps: generation produces code-like reasoning, and execution uses an interpreter when possible or an LM when code is not executable.The generated reasoning may be explicit code, pseudocode, or natural language.
- 2.2. Chain of Code: When execution fails, the LMulator simulates the result, updates the shared program state, and lets execution continue.The system interweaves Python execution and language-model simulation, including within control flow such as loops and conditionals.
- 2.3. Chain of Code Implementation: In the sarcasm example, Python initializes answer, the LMulator handles the non-executable semantic function, and Python completes the arithmetic updates to return 2.The example illustrates how semantic predictions can be inserted into an otherwise executable computation.
- 2.4. Chain of Code Abilities: CoC combines semantic language-model knowledge with code’s formal structure and computational tools, while retaining intermediate-step interpretability.The paper identifies these properties as enabling expressive pseudocode and broader code use.
- 2.4. Chain of Code Abilities: The paper reports significant reasoning improvements across challenging tasks from combining these benefits.
3. Experimental Evaluation
The evaluation tests Chain of Code across diverse BIG-Bench Hard reasoning tasks, problem types, model sizes, prompting settings, instruction-tuned models, prompt variations, and robotics. CoC generally outperforms baselines, with its strongest results arising from combining Python execution and LM-based simulation.
- Evaluation Setup: The evaluation uses challenging BIG-Bench Hard tasks spanning semantic, numerical, and combined reasoning, with few-shot and cross-task prompting settings.The study also examines ablations, scaling, instruction-tuned models, prompt robustness, and applications beyond language reasoning.
- Overall Performance: 84% overall performance makes CoC state of the art to the authors’ knowledge, exceeding human baselines on more tasks and by a larger overall margin.Combining CoC with gpt-4 reaches 91%.
- Problem Type: CoC performs particularly well on algorithmic tasks and on par with CoT on natural-language tasks.LMulator simulation preserves semantic reasoning flexibility when code cannot directly express the task.
- Problem Type: When code is repeated and executable by Python, CoC reaches nearly 100%, while the other task categories remain more challenging but still benefit over baselines.The results distinguish repeated versus new code and Python-only versus Python-plus-LM execution.
- Ablations: Ablations indicate that Python execution alone performs poorly on non-algorithmic tasks, whereas combining interpreter execution with LM simulation performs well.Maintaining program state improves performance, and simpler fallback variants incur only minor degradations.
- Scaling: CoC improves with model size and outperforms direct question answering even for smaller models, unlike CoT, which benefits only from the largest model.For some algorithmic tasks, CoC also exceeds the best human raters.
- Cross-task Prompting: Under cross-task prompting, performance drops for all methods, but CoC still outperforms CoT and direct prompting at scale and nearly reaches average human performance.This setting supplies examples from different problem families rather than similar tasks.
- Generality: The authors report noticeable benefits with instruction-tuned models, robustness to prompt variation, and applicability to robotics tasks combining semantic and algorithmic reasoning.In robotics, CoC can use Python APIs for perception and control while handling semantic subtasks inline.
4. Related Work
Prior work improves language-model reasoning through structured prompting, tool use, and code-based program synthesis. Chain of Code builds on these directions by combining code reasoning with language-model simulation for tasks beyond executable code.
- Chain of Thought decomposes complex tasks into intermediate reasoning steps, while ScratchPad maintains intermediate results for interpreting code.
- Chain of Code extends these lines of work by using an LM-augmented code emulator to combine executable code with simulated execution.
- Language models can use prompted tools such as calculators, code interpreters, and databases to answer queries and receive feedback from novel modalities.
- Language models have been applied to program synthesis and code-based reasoning in programming, robotics, and embodied-agent settings.
5. Conclusions, Limitations, and Future Work
Chain of Code combines executable code with language-model simulation so reasoning can cover both numerical and semantic problems. The paper reports gains over baselines and outlines computational, task-level, implementation, and deployment limitations alongside future extensions.
- CoC executes code with an interpreter when possible and uses an LMulator to simulate non-executable code, extending code-based reasoning to semantic problems.
- CoC outperforms baselines across challenging language and numeric reasoning problems and exceeds the best human raters on some tasks.
- The method requires additional context length and computation time, sometimes provides little benefit, and cannot currently modify custom Python objects during simulation.
- Future work includes unified code-language interpreters, LMulator fine-tuning, multi-pathway reasoning, and integration with external modalities such as vision and databases.
Impact Statement
Chain of Code executes language-model outputs with the Python interpreter as though they were benign code. Deployment therefore requires safeguards against potentially harmful code generated through malicious prompting.
- CoC runs language-model outputs with Python as though they were benign code, creating a deployment concern for maliciously prompted outputs.
- The authors state that real-world deployment will require additional safeguards before executing generated code.
A.1. Quantitative results on language reasoning tasks
The appendix reports full per-task BIG-Bench Hard results across ablations, task types, and execution types. Its notation distinguishes algorithmic and NLP tasks and records whether code is repeated or changed and whether Python or the LM executes it.
- Table A1 reports per-task BIG-Bench Hard results across ablations, task types, and execution types.
- The notation uses λ for algorithmic tasks and κ for NLP tasks.
- The symbols + and − indicate repeated or changed code executed by Python, while / and ∗ indicate repeated or changed code executed by the LM.
A.2. Quantitative results on the GSM8K Benchmark
On GSM8K, Chain of Code generally outperforms Chain of Thought and direct prompting. Because these tasks are primarily algorithmic, Python-based CoC variants achieve the same performance.
- CoC generally outperforms CoT and Direct prompting on GSM8K.
- Python-based CoC variants achieve the same performance because GSM8K tasks are primarily algorithmic and solvable with Python alone.
A.3. Qualitative results on language reasoning tasks
The qualitative examples illustrate CoC interleaving Python execution with LMulator simulation across reasoning tasks. These examples focus on the method’s execution pattern rather than reporting a separate quantitative result.
- CoC interleaves Python interpreter execution with LMulator simulation in BIG-Bench Hard reasoning examples.
- The GSM8K results table reports performance comparisons for direct prompting, Chain of Thought, and Chain of Code under single-task and cross-task prompting.
A.4. Instruction Tuned Models
The paper evaluates CoC with instruction-tuned models using zero-shot and few-shot prompting. CoC remains competitive with stronger models, although gains narrow when the base model already performs well or when prompting conditions differ.
- Zero-shot prompting: With gpt-3.5-turbo, CoT and CoC (Python) improve over direct prompting, while CoC (Interweave) outperforms both.
- Zero-shot prompting: With gpt-4, CoC (Interweave) still outperforms despite the model’s strength advantage over text-davinci-003, although the gap is narrower.
- Zero-shot prompting: The zero-shot comparison uses generic prompts without few-shot examples for gpt-3.5-turbo and gpt-4, while CoC (Interweave) is prompted with three examples.
- Few-shot prompting: Few-shot prompting asks instruction-tuned models to follow completion-model patterns using examples from the same or different task domains.
- Few-shot prompting: CoC retains a sizable performance gain over Direct and CoT, while the gap narrows with gpt-4 because the base model leaves less room for improvement.
A.5. Robustness of Chain of Code
CoC is reported as robust to prompt variation and applicable to robotics tasks combining semantic reasoning, algorithmic reasoning, and robot APIs. Robotics examples show one-shot generalization and interleaved execution between Python and LMulator.
- Robustness of Chain of Code: CoC is reported to be robust against prompt variation and not to require extensive prompt engineering.
- Robotics applications: Robotics tasks combine semantic reasoning, algorithmic reasoning, code-based API interaction, and natural-language interaction with users.
- Robotics applications: The robot setup provides perception through detect_objects(), control through pick_place(obj1, obj2), and communication through say(sentence).
- Robotics applications: A single example teaches the expected structure and available robot APIs for the few-shot robotics evaluation.
- Robotics applications: The seven test instructions include dietary preparation, multilingual cooking, object placement, recycling, composting, and user assistance.
- Robotics results: With one example, CoC generalizes to new objects, languages, and task domains in the robotics tasks.
- Robotics results: For robotics, CoC (Interweave) is described as the only capable approach because robot APIs and commonsense subtasks require line-by-line interplay between Python and LMulator.
- Robotics results: The robotics figures visualize one-shot prompts, model outputs, execution highlighting, and a trajectory for sorting objects into compost and recycle bins.