Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning

Sample, Simulate, Update Model (SSUP)

• SSUP combines object-based sampling, noisy physics simulation, and updating from simulated and real experience to guide physical problem-solving search.The framework is presented as a general modeling framework, with this paper instantiating a minimal model for the Virtual Tools game.Source: Sample, Simulate, Update Model (SSUP) · pp. 3, 5
• The object-based prior samples actions near movable objects and selects among the game’s tools when proposing candidate interventions.The prior restricts proposals to actions with potential to affect the environment.Source: Sample, Simulate, Update Model (SSUP) · p. 3
• The noisy physics engine evaluates proposed actions through a small set of stochastic forward simulations of possible outcomes.The model uses four simulations per action and acts when average reward exceeds a threshold or after five proposals without acting.Source: Sample, Simulate, Update Model (SSUP) · pp. 3, 5
• The update mechanism represents promising actions with a Gaussian mixture policy and revises it using observations from thought and action.Epsilon-greedy exploration occasionally samples from the prior to propose novel internal simulations.Source: Sample, Simulate, Update Model (SSUP) · p. 5
• Across matched levels, participants’ actions and required solution attempts differed, indicating sensitivity to subtle scene or goal differences.First actions were differentiable for four matched pairs but not for Falling or Towers, while solution actions differed for every level pair.Source: Human results · pp. 5–6

• The SSUP model matches human solution behavior across levels, using similar numbers of attempts and achieving similar accuracy.Human and model attempts correlate at r = 0.71, with mean attempts of 4.48 and 4.24, respectively; accuracy correlates at r = 0.86.Source: Model results · p. 6
• SSUP often captures the evolution of human solutions and particular action choices, although it does not systematically explain every discrepancy.Figure 6 compares cumulative solution rates and action distributions across selected levels.Source: Model results · p. 6, Figure 6 · p. 7
• Removing the prior, simulation, or updating component significantly worsens performance because each guides search toward useful actions.All reduced models show significant performance decreases, with bootstrapped ps < 0.0001.Source: Model comparisons on Virtual Tools · p. 7
• DQN + Updating performs worst among plausible alternatives, using the most actions and solving at a rate barely above chance.This result suggests generalized action policies are difficult to learn from repeated play on similar levels.Source: Model comparisons on Virtual Tools · p. 7

Validation on novel levels

• Without tuning parameters, SSUP generalizes to 10 additional levels, correlating with human performance at r = 0.85 for placements and r = 0.95 for accuracy.The additional levels included six novel level types and four variants of original levels.Source: Validation on novel levels · pp. 7–8, Figure 7 · p. 8

• The model is a first approximation because its simple priors and forward-only planning do not capture the full richness of human physical reasoning.The authors identify context-specific human priors and backward reasoning from goal states as important limitations.Source: Model limitations · pp. 8–9

Related cognitive systems

• The game combines prior knowledge of dynamics with iterative refinement, rather than relying solely on dynamics learning or a single insight.This positions Virtual Tools between evolving-situation problem solving and immediately solvable insight tasks.Source: Related cognitive systems · p. 9
• SSUP exemplifies “learning by thinking” by training a policy with data generated from an internal physical model.The cited passage frames this as transferring knowledge from internal physics models to level-specific action–outcome mappings.Source: Related cognitive systems · p. 9
• Preliminary model-free reinforcement learning showed limited generalization and inefficient learning across almost all Virtual Tools levels.These findings held despite significant experience with related levels.Source: Virtual Tools as an AI Challenge · p. 9
• Virtual Tools studies intuitive but non-obvious physical problems, unlike benchmarks that depend on precise world knowledge.Its focus is on problems people find challenging while retaining intuitive solutions.Source: Virtual Tools as an AI Challenge · p. 9
• Virtual Tools contributes to physics-reasoning benchmarks while emphasizing human-challenging problems that do not require precise environmental knowledge.The paper contrasts this emphasis with concurrently developed and other physics-based environments.Source: Virtual Tools as an AI Challenge · p. 9

Future empirical directions

• Future work should examine when and how people update internal physical models to support planning.The paper notes that the present task offers little benefit from additional precision, while unusual object properties could make updating useful.Source: Future empirical directions · p. 10
• Expanding Virtual Tools to include tool creation could study how people develop and transmit novel tools, which is more challenging than using existing objects.The paper relates this direction to evidence that children discover uses for existing objects before making novel tools.Source: Future empirical directions · p. 10

Conclusion

• SSUP explains rapid trial-and-error learning through rich prior world knowledge, hypothetical-action simulation, and learning from simulated and observed actions.The conclusion presents these three components as the framework’s account of efficient human tool use.Source: Conclusion · p. 10