PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration

Marta Knežević, Seyed Parsa Neshaei, Jibril Frej, Vinitra Swamy, Tanja Käser

arXiv:2610.21604v1cs.AI

TL;DR

Prior work typically evaluates LLM inquiry capabilities in isolation, leaving unclear how models coordinate exploration, hypothesis refinement, and reasoning over observations. This paper introduces an interactive mathematical modeling framework and evaluates LLM agents with and without structured inquiry guidance. The experiments report limited inherent inquiry capabilities and improved success with guided scaffolding.

  • Problem

    Prior evaluations typically isolate LLM capabilities, leaving unclear how models coordinate exploration, hypothesis refinement, and causal reasoning across an inquiry cycle.

    Source: §1 · p. 1

  • Method

    The paper evaluates inquiry in an interactive mathematical environment where agents explore variables and infer relationships, with a guided approach structuring inquiry steps.

    Source: §1 · pp. 1–2, §3.2 · p. 4

  • Results

    Experiments find limited inherent inquiry capabilities and low numbers of correctly discovered expressions, while structured GIL scaffolding consistently improves performance.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Structured scaffolding can support more systematic exploration and modeling by LLM-based agents in the studied inquiry setting.

    Source: §6 · p. 9

    The study examines a particular set of guidance types, and its analysis covers only a subsample of trajectories and a limited range of model families.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex systems. Large language models (LLMs) have demonstrated strong capabilities in several components of this process, including inductive reasoning, hypothesis generation, and data analysis. However, prior work typically evaluates these abilities in isolation, leaving open how models integrate them across a full inquiry cycle. In this work, we introduce a framework for evaluating LLMs' inquiry capabilities in a unified, interactive setting. We extend the Countdown task to a mathematical environment where an agent probes the system by manipulating variables, collecting observations, and inferring relationships between them. This setup enables analysis of exploration strategies, causal discovery, and hypothesis refinement during inquiry. Across seven LLMs and multiple prompting strategies, our experiments reveal limited inherent inquiry capabilities and low success rates in discovering the underlying expressions. However, structured guidance inspired by educational scaffolding significantly improves performance. Our findings highlight current limitations of LLMs in strategic exploration and iterative hypothesis refinement, and show how simple guidance strategies developed for human learners in inquiry learning can support more effective inquiry behavior.

1 Introduction

Scientific inquiry combines exploration, hypothesis generation, and evidence-based reasoning, but prior LLM evaluations typically isolate these abilities. The paper introduces an interactive mathematical task to study their coordination and reports limited inherent inquiry performance, with guided inquiry learning improving success.

1 Introduction

  • Prior evaluations typically isolate inquiry capabilities, leaving unclear how LLMs coordinate exploration, revise hypotheses from observations, and use causal relationships to guide exploration.The paper frames these as interconnected parts of inquiry learning.Source: §1 · p. 1
  • The interactive task extends Countdown by letting agents choose variable values, observe their effects on a target, and infer the underlying mathematical expression.Its full interaction trajectory records explored configurations and intermediate conclusions for hypothesis refinement and model discovery.Source: §1 · p. 2
  • Across seven instruction-following LLMs and prompting strategies, experiments find limited inherent inquiry capabilities and few correctly discovered expressions, while GIL improves success rates.The framework evaluates strategic exploration and iterative inference of relationships between variables.Source: §1 · p. 2
  • The paper contributes an interactive benchmark, structured in-context scaffolding for exploration and reasoning, and quantitative and qualitative evaluation of inquiry behavior.The evaluation examines exploration, hypothesis refinement, and reasoning.Source: §1 · p. 2

2 Related Work and Background

Prior research examines LLM planning, hypothesis generation, inductive reasoning, and inquiry learning, but has not established whether LLM agents can carry out inquiry cycles or which guidance they benefit from.

2 Related Work and Background

  • LLM agent research spans reasoning-and-action, reflection over prior trajectories, and structured multistep planning across interactive settings.Examples include ReAct (Yao et al., 2023), Reflexion (Shinn et al., 2023), and multistep planning.Source: §2 · p. 2
  • Hypothesis-generation and symbolic-regression work mainly studies producing candidate hypotheses, rather than iterative refinement and causal reasoning over numerical observations.The cited symbolic-regression approaches use external tools for fitting, verification, or execution.Source: §2 · p. 2
  • Inductive-reasoning studies infer rules from fixed examples or active environmental interaction, while prior mathematical interaction work includes univariate equations with elementary functions.The paper also extends findings on semantically rich examples to mathematical modeling, reporting that domain priors influence exploration and modeling behavior.Source: §2 · pp. 2–3
  • Inquiry learning is commonly organized around exploration, hypothesizing, and reasoning, with prior work proposing structured and metacognitive guidance for learners.Its applicability to LLM agents—whether they can engage in inquiry and what guidance they benefit from—remains unexplored in the cited discussion.Source: §2 · p. 3

3 Methodology

The methodology models inquiry as an agent’s iterative interaction with mathematical environments, then adds guided scaffolding and verification to structure that process. The environments use abstract variables and mathematical expressions, with additional datasets for scientific laws and more complex equations.

3 Methodology

  • The methodology combines iterative exploration, hypothesizing, and reasoning, with a guided variant that structures inquiry and provides instructions for systematic exploration and reasoning.GIL builds on the inquiry approach with an IL guide and a verifier that checks hypotheses against ground truth.Source: §3 · p. 3, §3.2 · p. 4

3.1 IL-based Exploration

  • An inquiry trajectory alternates environment-changing Actions, resulting Observations, internal Thoughts, and Hypotheses, ending with a returned model or the observation limit.The set Action explores, Hypothesis states a proposed model, and Thought reasons over collected data.Source: §3.1 · p. 3

3.2 Guided Inquiry Learning

  • GIL conditions each next step on the inquiry phase, guiding Actions during exploration, Thoughts after observations, and Hypotheses after the observation limit.Guidance prompts the agent to change one variable at a time and analyze how dependent-variable changes relate to independent variables.Source: §3.2 · p. 4

3.3 Mathematical Environment

  • The environment replaces Countdown’s fixed numbers with changeable abstract variables, whose effects on a target variable help agents infer expressions built from +, −, ·, and ÷.The dataset includes three-variable equations, subsampled four- and five-variable equations, equations corresponding to scientific laws, and extensions with coefficients and intercepts.Source: §3.3 · pp. 4–5

4 Experimental Settings

Experiments evaluate seven LLMs on multivariable equations under multiple prompting settings, using exact-match function correctness and trajectory annotations to assess inquiry behavior.

4.1 Experimental Setup

  • The experiments use six open-source LLMs and GPT-4o on 174 equations with three to five independent variables, across zero-shot, one-shot, and three-shot settings.Few-shot examples are human-created exploration and modeling trajectories for two-variable equations.Source: §4.1 · p. 5

4.2 Evaluation Metrics

  • On-task performance is the share of trajectories with correctly modeled functions, while behavioral analysis annotates exploration, hypothesis refinement, and reasoning.Two researchers independently annotated 11 trajectories with Cohen’s Kappa of 0.94; they then annotated 10,926 steps from selected trajectories.Source: §4.2 · p. 5

5 Results

The results show that guided inquiry learning improves equation discovery for most models, while unguided performance and inquiry behaviors reveal limitations in exploration, hypothesis refinement, and reasoning.

5.1 IL-based Exploration

  • In IL, open-source models scored up to 12.6% at zero-shot, while Llama-70B reached 24.1% with examples; accuracy also fell sharply as independent variables increased.In IL 3-shot, accuracy fell from 34.5% to 12.1% for Qwen-32B, from 34.5% to 10.3% for Llama-70B, and from 31% to 13.8% for GPT-4o.Source: §5.1 · p. 6, Figure 2 · p. 6
  • Removing the Thought step decreased performance for most models, while controlled exploration improved both Qwen models, Gemma-27B, Llama-70B, and GPT; adding a verifier consistently helped Qwen-14B, Qwen-32B, and GPT-4o.Explicit hypothesizing showed no clear overall pattern except for consistent benefits to Qwen-32B.Source: §5.1 · p. 6

5.2 Guided Inquiry Learning

  • GIL solved more equations correctly than IL for most models, with exceptions for Qwen-14B at 1-shot and Vicuna-13B at 0-shot; three-variable 3-shot accuracy reached 44.8% for Qwen-32B and 43.1% for Llama-70B.For three-variable equations, the GIL-over-IL performance gap was larger at 0-shot than at 3-shot.Source: Table 2 · p. 7, §5.2 · p. 7
  • GIL balanced trajectory step distributions by reducing over-exploration in some models and increasing exploration in under-exploring models; all models benefited from more inquiry cycles.For example, it reduced over-exploration for Llama-70B at 3-shot and increased engagement for Qwen-14B at 0-shot.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 Trajectory Analysis of IL Behaviors

  • Under GIL, strategic exploration averaged 91.64% of steps at 0-shot and 85.21% at 3-shot, compared with 21.63% under IL at 0-shot.Models also repeated hypotheses in up to 44% of 0-shot IL trajectories for Qwen-32B, and Qwen models favored numerical-factor refinements while Llama-70B favored variable-term changes.Source: §5.3 · pp. 7–8, Table 3 · p. 8
  • Thought steps with explicit cause–effect relations reached at most 0.44 in IL and 1.00 in every GIL condition; reasoning guidance also increased identification of quantitative relationships.Llama-70B showed fewer repeated hypotheses and numerical modifications than Qwen models, while Qwen-32B more often identified quantitative relations in IL and Qwen-14B performed best on that task under GIL.Source: §5.3 · p. 8, Table 4 · p. 9

6 Discussion & Conclusion

The study finds that LLM agents struggle to conduct systematic inquiry without guidance, while structured scaffolding improves exploration and modeling. Their difficulties parallel those reported in human learners, motivating inquiry strategies informed by learning sciences.

6 Discussion & Conclusion

  • Structured scaffolds through GIL consistently improved performance by promoting systematic exploration and modeling.The work studies agents in an interactive environment of simple multivariable equations.Source: §6 · p. 9
  • Without guidance, agents often explore unsystematically, commit early to hypotheses, and struggle to infer quantitative dependencies despite identifying qualitative relationships.They may change multiple variables simultaneously, making causal relationships difficult to isolate.Source: §6 · p. 9
  • Several inquiry difficulties resemble those reported in human learners, including challenges with causal isolation, hypothesis reconsideration, and quantitative model formulation.The authors relate these behaviors to confirmation bias and difficulties in scientific reasoning.Source: §6 · p. 9
  • The findings suggest that learning-sciences insights can guide strategies for more systematic exploration, reasoning, and hypothesis refinement in LLM agents.The authors describe the agents’ inquiry capabilities as emerging and their performance as benefiting from structured guidance.Source: §6 · p. 9

Limitations

The study identifies boundaries in model coverage, trajectory annotation, tool-use evaluation, and the guidance types considered. It also notes remaining GIL performance gaps tied to iterative hypothesizing and cause-effect reasoning.

Limitations

  • The evaluation did not extend to more LLMs from different model families.Manual annotation covered only a subsample of trajectories because the experiments produced many trajectories.Source: Limitations · p. 9
  • The study isolated LLM reasoning from external tool use, leaving their interplay in more complex or computationally demanding scenarios for future work.Source: Limitations · p. 9
  • The study focused on guidance for exploration, cause-effect reasoning, and structured scaffolding, leaving other forms of guidance for future research.Source: Limitations · p. 9
  • GIL retains performance gaps associated with LLM limitations in iterative hypothesizing and identifying cause-effect relationships.Source: Limitations · p. 9

A.1 Domain Priors

In a benchmark of 26 scientific equations, variable naming and contextual domain information affected equation induction. Acronyms paired with full variable names in context produced the best results.

A.1 Domain Priors

  • With full-name context, Llama-70B and Qwen-14B each achieved 100% accuracy, and all models performed best with acronyms plus that context.Source: §A.1 · p. 13
  • Without full-name context, acronym abbreviations raised accuracy from 7.7% to 30.8% for Vicuna-13B, from 30.8% to 57.7% for Qwen-14B, and from 23.1% to 34.6% for Gemma-27B.Source: §A.1 · p. 13
  • Without full-name context, acronym abbreviations raised accuracy from 53.8% to 76.9% for Qwen-32B, from 38.5% to 65.4% for Llama-70B, and from 57.7% to 84.6% for GPT-4o.Source: §A.1 · p. 13
  • The authors used abstract variable names to eliminate contextual information and isolate exploration and mathematical modeling abilities.Source: §A.1 · p. 13

A.2 Distribution of steps in trajectories

Trajectory analyses show that GIL changes how agents distribute their steps, reducing over-exploration and encouraging hypothesis generation in smaller models. Adding a verifier also increased hypothesis steps for both Qwen models.

A.2 Distribution of steps in trajectories

  • Under IL, Llama-8B and Vicuna-13B rarely hypothesized, while total steps increased from 0-shot to 1-shot and 3-shot for four models.The increase was reported for Llama-8B, Qwen-14B, Gemma-27B, and GPT-4o.Source: §A.2 · p. 13
  • GIL produced a more uniform step distribution across shot conditions, reduced over-exploration, and encouraged hypothesizing in smaller models.Figure 4 encodes approach by color, example count by shading, and step type by pattern.Source: Figure 4 · p. 13, §A.2 · p. 13
  • In the 0-shot setting, verifier addition raised mean hypothesis steps from 3.43 ± 0.32 to 11.76 ± 0.68 for Qwen-14B and from 3.28 ± 0.33 to 13.74 ± 0.69 for Qwen-32B.The reported values are means ± SEM.Source: §A.2 · p. 13
  • GIL prompted hypotheses in Llama-8B and Vicuna-13B, which did not hypothesize during IL, but did not resolve this for Vicuna in 0-shot.The authors note the importance of in-context examples for Vicuna.Source: §A.2 · pp. 13–14

A.3 Equations with coefficients

Adding numerical coefficients and intercepts makes equation discovery substantially harder for LLMs, with low accuracy and high variability. Longer GIL exploration helps some coefficient settings but can also hurt simpler ones.

A.3 Equations with coefficients

  • 10.3% accuracy at the low end contrasts with 44.8% for Qwen-32B on the coefficient-free setting.The expanded equation space and difficulty modeling more complex functional relationships contribute to the degradation.Source: §A.3 · p. 14
  • Equations with only an intercept are generally easier than equations with full coefficients, and free-form inquiry slightly outperforms GIL.Qwen-32B under inquiry is the exception to the intercept-versus-full-coefficients pattern; the authors hypothesize that GIL’s early hypothesis proposal becomes harder as equations and coefficient search spaces grow.Source: §A.3 · p. 14
  • For Qwen-32B on coefficient equations, extended GIL raises accuracy from 0%, 5%, and 7% to 3%, 7%, and 10% across 0-, 1-, and 3-shot settings.The benefit is not consistent on intercept-only equations; in some Qwen-32B settings, longer exploration lowers the percentage of induced equations, potentially through repetitive patterns or misleading long context.Source: §A.3 · p. 14
  • More complex equations with numerical terms remain a major challenge, with low overall performance and frequent failure to induce a correct equation.The tested variants add an intercept alone or both scaling coefficients and an intercept.Source: §A.3 · p. 14

A.4 Hyperparameters for GIL method

These analyses examine how GIL exploration settings relate to performance and describe exploration and verification behaviors, evaluation procedures, and example trajectories. The findings include model-dependent benefits from exploration schedules, performance plateaus, and inconsistent explicit hypothesis verification.

A.4 Hyperparameters for GIL method

  • Best observation schedules vary by model and shot setting: Qwen-14B favors shorter uniform exploration, while Qwen-32B and Llama-70B often favor a medium initial cycle followed by shorter cycles.Llama-70B also peaks with uniform short exploration and with a medium initial cycle followed by refinement without new data in 0-shot.Source: §A.4 · p. 14
  • A medium initial cycle (7-3-3) yields the highest accuracy for Llama-70B and Qwen-32B in 3-shot, while a long initial cycle reduces accuracy by 5% and 8%, respectively.Shorter subsequent cycles outperform no additional datapoints, while longer subsequent cycles reduce performance; gains plateau after model- and shot-dependent cycle counts.Source: §A.4 · p. 15

A.5 Exploration Analysis

  • Guidance increases strategic exploration over IL, but inherent strategic exploration varies by model and does not necessarily track overall task performance.Across annotated models, repeated exploration averages 2.67% and decreases under GIL, though it reaches 13% for Qwen-14B in 3-shot IL; random exploration also decreases with examples and guidance.Source: §A.5 · p. 16, Figure 5 · p. 16, Table 10 · p. 16

A.6 Behavioral Analysis of Verification

  • Models verify hypotheses at varying rates, but do not consistently substitute observed values into equations; computational errors are not the primary source of model-induction failure.Qwen-14B has the highest 0-shot verification rate at 58%; incorrect computations account for 3.06%, 2.93%, and 0.66% of total trajectory steps for Qwen-14B, Qwen-32B, and Llama-70B.Source: §A.6 · p. 17, Table 11 · p. 17

A.7 Evaluation Criterion; A.8 Evaluation Metrics

  • Correctness requires an exact match after symbolic normalization, while behavior annotations separately classify exploration, hypotheses, and reasoning content.Exploration labels include one-variable changes versus non-strategic actions; hypothesis labels include repetition and numerical or term modification; reasoning labels distinguish qualitative, quantitative, and observational statements.Source: §A.7 · p. 17, §A.8 · pp. 17–18

A.9 Trajectory Structure; A.13 Prompt Instructions

  • Trajectories alternate among Action, Observation, Thought, and Hypothesis steps, with set() assigning variable values and finish() returning the equation.The prompt instructions require thoughts to be followed by actions and hypotheses to be revised when unconfirmed.Source: §A.9 · p. 18, §A.14 · p. 19

A.10 Trajectory Sampling; A.11 Models; A.12 Computational Resources

  • Behavior analysis samples 288 trajectories from 24 equations across three models, two prompting settings, and IL/GIL; the broader evaluation spans open- and closed-source models.The appendix also reports computational resources, including two A100 GPUs and approximately $1000 spent on GPT-4o API experiments.Source: §A.10 · p. 18, §A.11 · p. 18, §A.12 · p. 18

A.14 Examples of IL Behaviors

  • Example trajectories contrast strategic single-variable probes with non-strategic multi-variable changes and show hypotheses being refined from observations.The examples also illustrate observation-only reasoning and a trajectory that reaches Z = Y - X.Source: §A.14 · pp. 18, 20, 21
Loading 2610.21604v1…