PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agents

Raj Jaiswal, Dhruv Jain, Harsh Parimal Popat, Avinash Anand, Abhishek Dharmadhikari, Atharva Marathe, Rajiv Ratn Shah

arXiv:2412.00821v1cs.AI

TL;DR

Open-source LLMs struggle to solve complex physics problems that require mathematical and domain-specific reasoning and may involve several kinds of errors. MoRA uses an advanced model to identify errors and guide iterative refinement agents. It improves performance across multiple datasets, including PhysicsQA, where it raises accuracy over CoT-generated solutions for both evaluated models.

  • Problem

    Open-source LLMs struggle with complex physics problems requiring integrated mathematical and domain-specific reasoning, and reliably correcting multiple kinds of solution errors remains challenging.

    Source: Introduction · p. 1

  • Method

    MoRA uses an advanced model to identify and score solution errors, then routes refinement agents to address them iteratively.

    Source: Introduction · p. 1

  • Results

    MoRA improves Llama-3-70B and Gemma-2-27B across multiple datasets; on PhysicsQA, accuracy over CoT-generated solutions rises by 13.38% and 16.03%, respectively.

    Source: Introduction · pp. 1–2, Conclusion · p. 7

  • Takeaways & Limitations

    PhysicsQA provides diverse high-school physics problems for evaluating and analyzing LLM physics reasoning.

    Source: Dataset: PhysicsQA · p. 2

    Problem-miscomprehension errors occur in very few LLM-generated solutions, constraining the cases available for evaluating refinement of that error type.

    Source: Miscomprehension Refinement · p. 4

Abstract

from arXiv · show

Large Language Models (LLMs) demonstrate remarkable capabilities in various reasoning tasks. However, they encounter significant challenges when it comes to scientific reasoning, particularly in physics, which requires not only mathematical reasoning but also factual and conceptual understanding. When addressing complex physics problems, LLMs typically face three key issues: problem miscomprehension, incorrect concept application, and computational errors. While each of these problems can be addressed individually, there is a need for a generalized approach that can tackle all three issues simultaneously. To address this, we introduce Mixture of Refinement Agents (MoRA), a novel agentic refinement framework that iteratively refines the LLM generated base solution by correcting the aforementioned errors, resulting in a significant performance improvement for open-source LLMs. Our approach aims to bridge the gap between opensource LLMs and GPT-4o by utilizing the latter as error identifier to guide these refinement agents. We evaluate our approach on the SciEval and MMLU subsets along with our own physics dataset (PhysicsQA). MoRA significantly improves the performance of Llama-3-70B and Gemma-2-27B on these datasets, achieving up to a 16% increase in final answer accuracy.

Introduction

Open-source LLMs struggle with physics problems that require integrated mathematical and domain-specific reasoning and can make multiple kinds of errors. MoRA iteratively identifies and routes those errors for refinement, and improves performance across datasets, including PhysicsQA.

Introduction

  • Physics solutions can contain objective misalignment, incorrect formula application, and computational mistakes, sometimes together in multihop problems.Open-source LLMs also struggle to identify their own reasoning mistakes, while correcting conceptual and computational errors requires mathematical reasoning and question-specific context.Source: Introduction · p. 1
  • MoRA uses an advanced model to identify and score solution errors, then routes prioritized refinement agents to address them iteratively.The process progressively refines the generated solution.Source: Introduction · p. 1
  • PhysicsQA adds a diverse set of intermediate-level high-school physics problems for evaluating open-source LLMs.The paper evaluates MoRA across four datasets, including PhysicsQA.Source: Introduction · p. 1
  • MoRA improves PhysicsQA accuracy over CoT-generated solutions by 13.38% for Llama-3-70B and 16.03% for Gemma-2-27B.The experiments cover four datasets, and further analyses examine error distributions and individual agents' refinement rates.Source: Introduction · pp. 1–2

Related Works

Prior work extends LLM reasoning through structured reasoning, verification, code execution, and external knowledge, while addressing challenges in complex scientific and mathematical tasks.

LLM Reasoning

  • CoT, Auto-CoT, and Complex-CoT generate intermediate reasoning, while Iter-CoT and ToT extend it but remain constrained by training knowledge and designed structures.In-context learning improves performance, but hallucinations and limited reasoning flexibility persist.Source: LLM Reasoning · p. 2

LLMs for Scientific Reasoning

  • Scientific reasoning methods include structured strategies for chemistry problems and agents that plan, retrieve functions, and execute actions using natural language and Python.These approaches address tasks requiring domain knowledge and step-by-step reasoning.Source: LLMs for Scientific Reasoning · p. 2

Self Verification with LLMs

  • Some error-detection methods require extra training data or domain-specific exemplars, while LLM self-verification checks conditional correctness step by step.Other work defines tasks for evaluating mathematical error recognition and correction.Source: Self Verification with LLMs · p. 2

LLMs for Mathematical Reasoning

  • Code generation and execution have been used to improve mathematical reasoning accuracy, with execution outcomes informing dynamic solution refinement.The passage describes this approach in GPT-4 Code Interpreter and MathCoder.Source: LLMs for Mathematical Reasoning · p. 2

LLM Reasoning with external database

  • Retrieval systems and knowledge graphs augment LLM reasoning with external information and map entity relationships to support retrieval.The cited work reports improved performance and more robust reasoning from comprehensive external knowledge in knowledge graphs.Source: LLM Reasoning with external database · p. 2

Dataset: PhysicsQA

PhysicsQA is a curated collection of high-school physics problems paired with step-by-step solutions, designed to support evaluation and analysis across diverse topics and levels of complexity.

Dataset: PhysicsQA

  • PhysicsQA comprises 370 carefully selected high-school physics questions sourced from online resources.Source: Dataset: PhysicsQA · p. 2
  • The questions often require multiple concepts, intricate computations, and multihop reasoning, and each has a comprehensive step-by-step solution.The solutions support evaluation and fine-tuning of LLMs for physics reasoning.Source: Dataset: PhysicsQA · p. 2
  • Table 1 presents the topic-wise distribution of PhysicsQA questions.Source: Table 1 · p. 2, Dataset: PhysicsQA · p. 2

Mixture of Refinement Agents

MoRA combines error identification with specialized refinement agents to address multiple kinds of mistakes in physics solutions. It routes agents in a prioritized order and repeats identification and refinement until errors are resolved or an iteration limit is reached.

Motivation

  • Physics solutions may combine miscomprehension, concept-application, and computational errors, motivating one framework rather than isolated fixes.Source: Motivation · p. 3

Error Identification

  • GPT-4o checks objective and variable alignment, scores conceptual errors by their first-error stage, and uses Code Interpreter to evaluate arithmetic and algebra with 0.1 tolerance.Source: Error Identification · pp. 3–4

Refinement Agents

  • Each specialized refinement agent targets a specific error type and uses the same LLM that generated the original solution.Source: Refinement Agents · p. 4

Miscomprehension Refinement

  • Miscomprehension errors are corrected through instructions that refocus the solution on the question objective or correct variable values.Source: Miscomprehension Refinement · p. 4

Concept Refinement

  • For conceptual errors, the LLM identifies the earliest faulty concept, forms a retrieval thought, and uses GraphRAG to retrieve relevant physics context before revising the solution.Source: Concept Refinement · p. 5

Computational Refinement

  • For computation errors, the LLM generates and executes Python code for the faulty step, then incorporates the returned computation into the refined solution.Source: Computational Refinement · p. 5

Agent Routing and Iterative Refinement

  • Agents run in order—miscomprehension, concept, then computation—and the framework repeats error identification and refinement until scores and flags are resolved or the iteration limit is reached.Source: Agent Routing and Iterative Refinement · p. 5

Experiments

Experiments compare baseline prompting and MoRA across four physics-related datasets, using final-answer accuracy to evaluate performance. MoRA improves results on most reported model–dataset comparisons, with the PhysicsQA gains particularly notable.

Datasets

  • Experiments use SciEval-Static, PhysicsQA, MMLU High School, and MMLU College, with six listed models evaluated using the same prompts.SciEval-Static contains 164 physics questions; the MMLU subsets contain 173 high-school and 118 college multiple-choice questions.Source: Datasets · p. 5

Baselines

Evaluation

  • Physics reasoning is evaluated by comparing final answers and calculating accuracy on each dataset.Source: Evaluation · p. 5

Results

  • MoRA raises PhysicsQA accuracy from 59.29% (3-shot) to 70.14% for LLaMA-3-70B and from 59.45% (3-shot) to 70.62% for Gemma-2-27B.Table 3 compares baseline approaches with MoRA across four datasets using final-answer accuracy.Source: Table 3 · p. 6, Results · p. 6
  • On SciEval-Static, MoRA raises LLaMA-3-70B accuracy from 82.23% (CoT) to 86.58% and Gemma-2-27B accuracy from 79.26% (CoT) to 88.76%.On MMLU High School, LLaMA-3-70B rises from 73.66% (3-Shot) to 78.81%, while Gemma-2-27B changes from 77.11% (CoT) to 75.88%.Source: Results · p. 6, Table 3 · p. 6
  • On MMLU College, MoRA raises LLaMA-3-70B accuracy from 71.76% (3-Shot) to 78.82% and Gemma-2-27B accuracy from 73.52% (CoT) to 82.20%.The paper states that the models achieve competitive performance without extensive fine-tuning.Source: Results · p. 6, Table 3 · p. 6

Analysis

The analysis examines errors in physics Chain-of-Thought solutions and tests the refinement agents individually. Open models show conceptual and computational errors, while refinement rates vary by error type and dataset.

Error Analysis

  • All models show strong problem comprehension, though the open models make more errors in PhysicsQA and SciEval-Static than GPT-4o.GPT-4o has near-perfect comprehension on SciEval-Static and both MMLU subsets, with only minor PhysicsQA errors.Source: Error Analysis · p. 6
  • GPT-4o averages 96.4% accuracy on conceptual errors and 95.64% on computational errors across the four datasets.Source: Error Analysis · pp. 6–7
  • On PhysicsQA, Llama-3-70B and Gemma-2-27B average 18.11% conceptual errors and 21.62% computational mistakes.Table 4 covers errors in incorrect physics CoT solutions across models and four datasets.Source: Error Analysis · pp. 6–7, Table 4 · p. 7

Ablation

  • Instruction prompting yields 100% miscomprehension-error refinement for both open models on MMLU High School and College.On PhysicsQA, Llama-3-70B reaches 66.7% and Gemma-2-27B 62.5%.Source: Table 5 · p. 7, Ablation · p. 7
  • Conceptual-mistake and retrieval-thought refinement is mixed: Gemma-2-27B reaches 16.7% on MMLU High School and 48.7% on PhysicsQA.Llama-3-70B reaches 46.9% on PhysicsQA, while Gemma-2-27B reaches 62.5% on SciEval-Static.Source: Table 5 · p. 7, Ablation · p. 7
  • Code-driven refinement reaches 72.6% for Llama-3-70B and 73.3% for Gemma-2-27B on PhysicsQA.Gemma-2-27B reaches 100% on MMLU College but 33.3% on MMLU High School.Source: Table 5 · p. 7, Ablation · p. 7

Conclusion

The conclusion presents MoRA as an iterative framework using GPT-4o to identify and score errors that guide refinement agents. It also reports a curated high-school physics dataset and improved performance for Llama-3-70B and Gemma-2-27B across multiple datasets.

Conclusion

  • MoRA uses GPT-4o to identify and score solution errors, then iteratively guides refinement agents until the errors are mitigated.The conclusion describes this as the framework's process for addressing three critical errors in complex physics problem solving.Source: Conclusion · p. 7
  • PhysicsQA is a curated dataset of diverse high-school-level physics problems for comprehensive evaluation.Source: Conclusion · p. 7
  • Experiments and in-depth analysis across multiple datasets show improved performance for Llama-3-70B and Gemma-2-27B.Source: Conclusion · p. 7
Loading 2412.00821v1…