PaperlayerView exact revision on arXiv

Source-linked AI summary

ScienceWorld: Is your Agent Smarter than a 5th Grader?

Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, Prithviraj Ammanabrolu

arXiv:2203.07540v2cs.CLcs.AI

TL;DR

ScienceWorld addresses whether models that answer science questions can reason with those concepts in novel, grounded contexts. It introduces an interactive text benchmark of elementary science tasks and finds that current agents perform poorly overall, while interactively trained smaller agents are more sample- and parameter-efficient than offline-trained large language models. The benchmark’s conclusions are bounded by simplifying aids and abstractions in both agents and environment.

  • Problem

    Current models produce science answers and explanations but struggle to reason about and use fundamental science concepts in grounded, reusable ways.

  • Method

    The paper builds SCIENCEWORLD, an interactive text environment and benchmark, and evaluates learning agents on elementary science tasks using sequential actions.

  • Results

    Overall performance on unseen science tasks is low, while interactively trained agents are more sample- and parameter-efficient than large language models trained offline from static sources.

  • Takeaways & Limitations

    Grounded interactive training is associated with better efficiency than static training for this benchmark, but the best performance remains modest.

  • Takeaways & Limitations

    Most benchmarked agents use SCIENCEWORLD’s valid-action detection aid at test time, and the environment uses shortened descriptions and simplified elementary-science simulations.

Abstract

from arXiv · show

We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum. Despite the transformer-based progress seen in question-answering and scientific text processing, we find that current models cannot reason about or explain learned science concepts in novel contexts. For instance, models can easily answer what the conductivity of a known material is but struggle when asked how they would conduct an experiment in a grounded environment to find the conductivity of an unknown material. This begs the question of whether current models are simply retrieving answers by way of seeing a large number of similar examples or if they have learned to reason about concepts in a reusable manner. We hypothesize that agents need to be grounded in interactive environments to achieve such reasoning capabilities. Our experiments provide empirical evidence supporting this hypothesis -- showing that a 1.5 million parameter agent trained interactively for 100k steps outperforms a 11 billion parameter model statically trained for scientific question-answering and reasoning from millions of expert demonstrations.

1 Introduction

ScienceWorld reframes science question answering as interactive experimentation, testing whether agents can use scientific knowledge procedurally rather than merely retrieve answers. It introduces an environment and benchmark spanning elementary science, where current agents perform poorly on tasks children can perform easily.

  • Motivation and task: ScienceWorld targets the gap between fluent science answers or explanations and meaningful, reusable use of the underlying knowledge.
  • Motivation and task: ScienceWorld asks agents to complete elementary science experiments in a simulated text environment instead of simply answering questions.The task requires combining declarative scientific knowledge with procedural knowledge to complete experiments.
  • Benchmark: The benchmark includes simulation engines for thermodynamics, electrical circuits, chemistry reactions, and biological processes.
  • Benchmark: 30 benchmark tasks span 10 topics across the elementary science curriculum, including phase changes and pollination.
  • Results: Five state-of-the-art reinforcement learning and language-model agents perform poorly on tasks such as melting ice that fifth-grade students can perform easily.

2 Related Work

Prior science QA systems answer many questions and generate explanations, but explanation evaluation and interactive-environment modeling remain difficult. ScienceWorld addresses these gaps by executing agent action sequences in a richer simulator and benchmarking current agents on elementary science tasks.

  • Science-domain inference: Large language models can answer more than 90% of standardized multiple-choice science questions correctly, typically using large pretrained models.
  • Science-domain inference: Comparing generated explanations with one gold explanation can underestimate explanation-generation performance by up to 40%.Adding alternate gold explanations is expensive and has only been demonstrated on a small scale.
  • Science-domain inference: ScienceWorld treats agent action sequences as structured explanations whose correctness can be tested directly by running them in the simulator.This evaluation is independent of variations in how agents solve a task.
  • Simulators: Existing text-environment tooling primarily supports simpler simulations, making autonomous physical processes such as thermodynamics difficult to model.The authors therefore build a novel simulator for physical processes in text environments.
  • Agents: ScienceWorld benchmarking finds elementary science tasks difficult for current agents, while smaller agents can outperform billion-parameter language models trained on gold sequences.

3 SCIENCEWORLD

SCIENCEWORLD is a complex interactive text simulation with diverse locations, objects, actions, and elementary-science process engines. Its modeled domains include thermal, electrical, chemical, biological, force, and container behavior, with deliberate simplifications for tractability.

  • Environment: SCIENCEWORLD provides an English interactive text environment with many objects, actions, and simulation engines, implemented in 40k lines of Scala with a Python interface.
  • Environment: The environment has 10 interconnected locations, up to 200 object types, and approximately 200k possible action-object combinations per step.
  • Simulation engines: Its science processes include automatic thermodynamics and action-coupled devices or chemical mixing, alongside ordinary text-environment actions.
  • Simulation engines: Thermodynamics models heat transfer through material-dependent conduction coefficients, phase transitions, and combustion points.
  • Simulation engines: Electrical circuits use explicit terminal-to-terminal connections, allowing agents to test conductivity by observing whether a bulb illuminates with different objects.
  • Design scope: For tractability, the simulation has elementary-science fidelity, including simplified thermal transfer, staged biological changes, and only simple series circuits.
  • Simulation engines: The simulator models elementary chemistry, life stages, genetics, inclined-plane friction, and container visibility or heat effects.

4 Experiments

The experiments benchmark reinforcement-learning and language-model agents in SCIENCEWORLD, testing zero-shot generalization across varied elementary science tasks. Results show that current models struggle especially with open-ended procedures and unseen experimental contexts.

  • Agents: The benchmark compares DRRN, KG-A2C, CALM, behavior cloning, and Text Decision Transformer agents using reinforcement learning, language-model action generation, or both.DRRN selects relevant actions from text observations; KG-A2C uses dynamically constructed knowledge graphs; CALM, behavior cloning, and TDT use expert demonstrations in different ways.
  • Tasks: SCIENCEWORLD evaluates agents on diverse elementary science tasks in an interactive text environment with 10 broad curriculum topics.Tasks include changes of state, temperature measurement, electrical circuits, friction, object classification, chemical mixtures, and plant-related topics.
  • Tasks: The benchmark varies critical objects, starting locations, and environment contents across 7200 variations to encourage generalization.Each subtask has 10–1400 parametric variations, split into training, development, and test sets with critical unseen variations reserved for development and testing.
  • Environment and supervision: Agents receive normalized rewards for required and optional subgoals, while hand-coded oracles provide canonical solution trajectories for imitation learning.Optional subgoals reduce reward sparsity, and oracle trajectories represent canonical methods rather than every possible route to a goal state.
  • Results: 0.17 average score: DRRN is the best model across all 30 subtasks, but overall performance remains low.Models using valid-action detection generally outperform models that must generate valid actions while also selecting actions for task progress.
  • Results: All models perform poorly on open-ended state-change tasks, while even stronger agents struggle to complete the final pickup-and-placement steps in an easier task.Paired tasks also contrast named-material retrieval with experimentally determining conductivity for randomly generated substances.

5 Discussion

SCIENCEWORLD remains difficult for current agents, exposing weaknesses in both science-domain procedures and commonsense interaction. Smaller interactively trained agents can be more efficient than much larger offline-trained language models, while the environment and explanations retain important simplifying limitations.

  • Elementary science tasks are challenging for text agents: 0.17 normalized average score was achieved by top-performing agents across tasks, indicating that elementary science tasks remain challenging.Performance was comparable to agents on medium-difficulty interactive fiction games.
  • Elementary science tasks are challenging for text agents: Agents struggle with both science procedures, such as heating substances or growing seeds, and commonsense skills, such as navigation and liquid storage.The observed failures span domain inference and basic interaction competencies.
  • Larger models are not necessarily better: The 1.5-million-parameter DRRN outperforms larger T5 models despite receiving the same SCIENCEWORLD updates and less pretraining.The comparison suggests that state-space and action-sequence modeling can matter more than pretraining scope in this benchmark.
  • Larger models are not necessarily better: Online interactive training enables DRRN and KG-A2C to handle long action sequences more efficiently in both samples and parameters.
  • Limitations of agents and environments: At test time, all but two benchmarked agents rely on valid-action detection, which substantially simplifies action-space search.This aid is needed because agents frequently generate plausible but invalid actions.
  • Explanations as action sequences: Action-sequence explanations describe how to complete tasks but omit high-level reasons connecting scientific goals to the required actions.The paper suggests explanatory scaffolds could improve task performance and human interpretability.

6 Conclusion

The paper introduces SCIENCEWORLD to test whether agents can use elementary scientific knowledge in grounded, reusable ways. Across evaluated agents, performance on unseen science tasks remains low, while interactive training appears more sample- and parameter-efficient than offline learning from static text.

  • 6 Conclusion: SCIENCEWORLD is an interactive text environment derived from an elementary school curriculum, covering tasks from electrical conductivity to Mendelian genetics.
  • 6 Conclusion: Five agents are evaluated, including three reinforcement-learning text-game agents and two transformer-based agents trained for scientific reasoning.
  • 6 Conclusion: Overall performance on unseen tasks requiring science-domain knowledge is low across all agents.The best performance is on-par with medium-difficulty interactive fiction environments such as Zork.
  • 6 Conclusion: Interactive grounded learning is more sample- and parameter-efficient than offline learning from static sources in the reported results.The findings highlight the need to integrate declarative scientific and world knowledge with procedural action sequences.

7 Broader Impacts

SCIENCEWORLD uses an abstract interactive text interface to support lower-cost, more interpretable sequential planning, but its exploratory value does not extend directly to real-world advice. Its simulated world includes varied objects, actions, and processes designed for elementary science tasks.

  • 7 Broader Impacts: Interactive text environments offer a faster and cheaper alternative to 3D environments for teaching sequential decision-making.Their abstraction level can range from high-level movement to detailed physical door manipulation.
  • 7 Broader Impacts: Abstract-language planning is presented as simpler and more interpretable than fine-grained physical action sequences.
  • Risks and scope: ScienceWorld is exploratory and should not be used to provide real-world advice; its environment is safer than the real world.For example, agents cannot accidentally burn themselves while boiling a substance on a campfire.
  • SCIENCEWORLD environment: The framework supports creating new science-domain tasks and evaluating solution correctness through a Scala simulator with a Python interface.
  • SCIENCEWORLD environment: Randomly varying contextual items discourage agents from memorizing one fixed environment.

A.4 Simulation Engines

SCIENCEWORLD simulates elementary thermodynamic, electrical, chemical, biological, and mechanical processes through actions and automatic state changes. Its simulation fidelity is intentionally simplified to keep the environment tractable.

  • Simulation engines: Simulation engines model processes including thermodynamics, chemistry, life stages, reproduction and genetics, friction, containers, electricity, and devices.
  • Thermodynamics: Thermodynamics transfers heat among objects in thermal contact using material-dependent conduction, with heat sources, sinks, and ambient exchange also modeled.Thermal conductors and insulators are distinguished through conduction coefficients.
  • Simulation scope: The simulator deliberately uses elementary-curriculum fidelity, including simplified thermal transfer, staged biological change, and only series electrical circuits.Resistance, inductance, and other advanced electrical concepts are excluded.
  • Electricity: Simple series circuits connect powered devices to energy sources through conductors, and non-electrical objects can serve as testable conductors.A light bulb can reveal conductivity differences between materials such as plastic and metal forks.
  • Chemistry: Chemical reactions produce resultant substances when specified substances are mixed in containers.Examples include salt and water producing salt water, along with modeled rust, food, water, and paint reactions.
  • Biology: Plants and animals progress through discrete life stages when their needs are met, and plants can model inherited traits through Mendelian reproduction.Unmet needs such as insufficient water, soil, or suitable temperature can cause death.

B Tasks and Competencies

SCIENCEWORLD organizes elementary science reasoning into varied interactive subtasks that require intentional actions, generalization, and adaptation to task difficulty.

  • Task structure: SCIENCEWORLD covers 10 broad science curriculum topics, each subdivided into 2–4 subtasks testing specific reasoning capacities.Some subtasks mask critical object names to distinguish experimental reasoning from direct lookup.
  • Intentional action: Agents must accomplish tasks intentionally, often by focusing on the specific object before performing the requested operation.This adds preliminary intent-signaling subgoals to the task procedure.
  • Task formats: Tasks can be performed directly for a measurable end-state or solved through forced-choice placement after an inference.Conductivity classification is an example of the forced-choice format.
  • Task structure: 7200 parametric task variations change critical objects, starting locations, and environment contents across the 30 subtasks.Each subtask contains between 10 and 1400 variations to prevent overfitting and encourage generalization.
  • Difficulty: Environment simplifications, including teleportation and open containers, provide adjustable difficulty settings.The environment offers five simplifications, with easy mode enabling all of them.

B.1 Scoring and Evaluation Protocol

The benchmark evaluates science competencies through concrete environment changes, measurements, classifications, and procedures that also depend on commonsense interaction skills.

  • Scoring: Required goals define task success, while 2–15 optional subgoals provide denser rewards toward canonical solutions.Scores for all tasks are normalized between 0 and 1.
  • Science competencies: State-change tasks require finding a substance and using heating or cooling devices to produce melting, boiling, freezing, or another phase change.Variations can disable common devices, requiring alternate heating or cooling methods.
  • Science competencies: Measurement tasks require thermometer use for object temperatures or melting points, with forced choices based on a predetermined threshold.The agent continually monitors temperature when measuring a substance’s melting point.
  • Science competencies: Circuit tasks require building series circuits to power components, compare energy sources, or test known and unknown substances for conductivity.Variations alter components, wire colors, and required objects.
  • Science competencies: Other tasks test classification, plant growth and pollination, chemical mixtures, animal life spans, life stages, forces, and Mendelian genetics.The genetics task grows two plant generations and counts traits to determine dominance or recessiveness.
  • Supporting competencies: Successful completion also requires commonsense knowledge, object affordances, navigation, and opening containers to inspect or use contents.Examples include obtaining water from a sink, carrying liquids in cups, and traversing rooms through doors.

B.4 Scoring and Evaluation Protocol

The evaluation combines shaped rewards, held-out task variations, repeated model runs, and test-time action selection to measure interactive performance across SCIENCEWORLD subtasks.

  • Scoring: Required and optional subgoal completion increases normalized subtask scores, with each task containing 2–15 optional reward-shaping goals.Required goals are method-agnostic, while optional goals nudge agents toward canonical solutions.
  • Evaluation splits: Variations are split 50% training, 25% development, and 25% test, with unseen critical substances, animals, or plants reserved for later splits.This design evaluates generalization to critical variations absent from training.
  • Evaluation settings: Environment simplifications such as teleportation, self-watering pots, and open containers create easier evaluation settings.Five simplifications can be enabled individually or together in easy mode.
  • Training protocol: Reinforcement-learning models use 8 environment threads for 100k steps per thread, with episodes resetting after success, failure, or 100 steps.KG-A2C and CALM also reset after 100 steps of invalid actions when stuck.
  • Evaluation protocol: Test evaluation samples 10 test variations and runs one episode per variation at each evaluation period, reporting the average score.Evaluation occurs every 1000 environment steps per thread.
  • Evaluation protocol: Figure 3 plots DRRN, KGA2C, and CALM episode rewards on unseen test sets against training environment interactions.The curves summarize learning across all 30 subtasks.

C.4 Impact of Model Size and Pre-training Methodology on Performance

SCIENCEWORLD results indicate that larger models do not necessarily perform better, while interactive training can be more parameter- and sample-efficient than offline approaches.

  • Model size: SCIENCEWORLD’s average performance is evaluated against model parameter size across tasks and random seeds.Table 5 marks a 131M-parameter count that includes a pretrained GPT-2 action generator, while only 6.9M policy parameters are updated in RL.
  • Pre-training: T5-Large and Macaw-Large, each with 770M parameters, outperform the larger 11-billion-parameter models by approximately a factor of two.Scientific-QA pretraining also outperforms general-language-model pretraining in the reported comparison.
  • Interpretation: The apparent inverse scaling pattern remains suggestive rather than conclusive because hyperparameter differences cannot be ruled out.The authors defer verification to future work because training and inference costs are high.
Loading 2203.07540v2…