Can LLMs Uncover Hidden Physical Laws through Free Exploration?

• Scientific inquiry can begin with an unexpected observation, but autonomous LLMs must choose what to investigate and test their own explanations.The paper frames this ability as potentially useful for investigating phenomena scientists might otherwise overlook.Source: §1 · p. 1
• Unlike physical-law benchmarks that assign discovery tasks, LongChang asks whether LLMs will notice phenomena worth investigating and formulate their own research questions.The paper asks whether they can then experiment to distinguish explanations and discover altered laws.Source: §1 · p. 1
• LongChang is introduced as a framework for generating simulated worlds with altered physical laws.Source: §1 · p. 1

• The base scene set includes ordinary-physics Scene 01 and 14 scenes, each with one altered physical law.Table 1 directs readers to Appendix A.1 for the laws.Source: Table 1 · p. 2
• 16.7% discovery rate for GPT-6-Astra under free exploration, while the other 11 evaluated LLMs score 0%.Some models leave observed anomalies uninvestigated; others experiment but only partially identify the laws, leaving conditions or applicability unresolved.Source: §06 · p. 2
• LongChang contributes a complete discovery-cycle framework and an empirical study of autonomous physical inquiry.Model comparisons, controlled interventions, and exploration records are used to examine how LLMs form questions and test explanations.Source: §06 · p. 2
• LongChang is compared with PhysGym, NewtonBench, and DiscoverPhysics across task assignment, interactive 3D scenes, tool design, and scene evolution.Source: Table 2 · p. 2

2.1 Autonomous Scientific Discovery

• Earlier discovery benchmarks specify goals before experimentation, while other open-ended protocols provide an anomaly, a research role, or modeling guidance.LongChang instead tests whether unusual physical observations prompt LLMs to pose their own questions and establish laws experimentally without assigned tasks or advance hints.Source: §2.1 · pp. 2–3

2.2 Tool Use and Design

• LongChang ties tool construction to the explorer’s own investigation: the LLM decides whether a tool is needed, what to build, and how to test a hypothesis with it.Constructed tools interact physically with other objects in the simulated world.Source: §2.2 · p. 3

2.3 Environment Generation and Evolution

• Separate LLM agents use unresolved questions and corrected errors in exploration records to generate, refine, and compose scenes for further exploration.Explorers choose what to investigate, while other agents review tools and assess reports.Source: §2.3 · p. 3

3 LongChang

• Separate LLM agents design and review scenes, review apparatus, and evaluate reports, while the full discovery cycle runs without human intervention.Exploration records guide subsequent scene evolution.Source: §3 · p. 3

3.1 Environment Construction

• The base set contains a normal control and 14 scenes with one altered law each, whose target objects, conditions, and observable effects are recorded for evaluation.Scene designers can generate, refine, or compose scenes using exploration summaries; automated tests and an LLM reviewer check proposed scenes before later use.Source: §3.1 · p. 4

3.2 Autonomous Exploration

• Explorers choose observations, formulate questions, and test explanations without a research task or hints about the altered physics.They can manipulate objects and build apparatus; observations and rationales are recorded across turns, and final reports must cite supporting turns and preserve recorded uncertainty.Source: §3.2 · p. 4

3.3 Discovery Evaluation

• Discovery requires both identifying altered laws correctly and supporting every report claim and its stated certainty with recorded experiments.Mechanisms may differ if they yield the same observable predictions under allowed interventions; correct guesses can pass correctness alone, while supported ordinary observations or partial findings can pass evidence support alone.Source: §3.3 · pp. 4–5

3.4 Experimental Setup

• The main evaluation runs 12 LLMs on the same 15 scenes from fixed initial states, allowing up to 100 turns per run.GPT-6.1-Sol judges reports, and GPT-6-Astra reviews apparatus and generates and reviews scenes; metrics cover discovery checks, control-report rejections, exploration, actions, and construction.Source: §3.4 · p. 5

4 Results and Analysis

• 16.7% discovery for GPT-6-Astra versus 0% for the other 11 LLMs; Astra’s evidence support rate is 97.6%, but law correctness is 19.0%.Claude-Opus-5 identifies altered laws in 16.7% of runs, but none of its reports passes the evidence-support check.Source: §4 · p. 5
• Gemini-3.7-Flash stops after an average of 10.9 turns, while GLM models use about 90 turns and build apparatus yet make no discoveries.GLM models fail in more than 60% of build attempts, and failed actions account for 16.8–20.2% of turns.Source: §4 · p. 5

4.1 Forming Research Questions

• In Scene 15, GPT-6-Astra investigates red’s unexpected restart and finds that pushes or throws pause its northward force for ten counts; Gemini-3.7-Flash leaves it unexamined.Figure 5 compares the two explorations.Source: §4.1 · p. 5, Figure 5 · p. 6
• With a task, altered-law warning, and scene-specific clue, discovery rises from 16.7% to 28.6%, while evidence support is unchanged and mean exploration falls from 39.3 to 30.9 turns.The other settings are fixed.Source: §4.1 · pp. 5–6

4.2 Validating Scientific Claims

• Testing two red balls distinguishes color exchange from independent red–blue flipping: both remain red, supporting exchange and ruling out flipping.Under ordinary physics, 36.1% of reports are rejected for false altered-law claims or unsupported explanations.Source: §4.2 · p. 6

4.3 Using Experimental Resources

• Raising the turn limit from 100 to 500 does not improve discovery: GPT-6-Astra uses about 40 turns under either limit and sometimes leaves questions unresolved.Extra turns provide little value when explorers stop early.Source: §4.3 · p. 7
• With construction enabled, discovery rises from 4.8% to 16.7% and law correctness from 4.8% to 19.0%, although action failures also increase.The explorer chooses whether to build, so the comparison concerns access to construction, not tool use in each run; Figure 7 shows a weighing platform redesigned for stable readings.Source: §4.3 · p. 7, Figure 7 · p. 7

5 Scene Evolution

• Corrected errors from successful explorations guide scene designers to generate, refine, and compose new challenges while retaining the original pause mechanism.The three variants respectively change the force source, add a spatial condition, or combine the pause with delayed motion copying.Source: §5 · pp. 7–8
• The evolved scenes test distinct unresolved rules: position-dependent repulsion, a spatial blocker, and interactions between separate timing rules.Each design includes a planned experiment to distinguish the altered mechanism or trigger.Source: §5 · p. 8
• 33.3% discovery in Scene 15 and 66.7% in Scene 14 contrast with 0% in all three evolved scenes; evolved-scene evidence support remains 100%, but no report passes law correctness.The generated, refined, and composed scene reports each leave key conditions, dependencies, or timing rules unresolved.Source: §5 · p. 8, Table 6 · p. 8

6 Conclusion

• LongChang examines autonomous inquiry into altered physical laws, with object manipulation and tool building but no research tasks or hints about the laws.Separate LLM agents evolve scenes using unanswered questions and corrected errors in exploration records, completing the cycle without human intervention.Source: §6 · pp. 8–9
• Only one of twelve evaluated LLMs makes discoveries, underscoring the difficulty of turning observations into complete, experimentally supported laws.Source: §6 · p. 8

7 Limitations

• The study is limited to simulated worlds with altered laws whose discoveries can be checked; real-world questions with unsettled answers require further research on evaluating exploration and findings.Source: §7 · p. 9

• Because LongChang uses no human annotation or testing, its evaluation raises no participant privacy or informed-consent concerns.Source: §8 · p. 9

A.1 Scene Laws

• The base scene set includes ordinary-physics controls and altered-law scenes, with each of Scenes 02–15 changing one physical law.Scene 01 has mass-independent downward gravity, rigid-body motion, collisions, and friction; its decorative gray tile does not affect motion.Source: §A.1 · p. 10

A.2 Action Interface

• The action interface supports object manipulation and optional custom apparatus construction, with action targets and parameters specified in the appendix.Available operations include PUSH, THROW, WAIT, GRASP, MOVE, RELEASE, and MARK; constructed tools can have bodies, joints, and instrumented sites.Source: §A.2 · p. 10, §A.5 · pp. 10–11

A.3 Prompts

• Prompts guide exploration, final reporting, and discovery evaluation, while task guidance supplements the original exploration prompt.The exploration prompt asks models to identify and experimentally verify any laws they find.Source: §A.3 · p. 10, §A.5 · p. 12

A.4 AI Assistance

• AI assistants helped improve manuscript wording and clarity and assisted with code implementation and debugging; GPT-6.1-Sol judged report correctness and experimental support.The authors remain responsible for the manuscript, code, and findings.Source: §A.4 · p. 10

A.5 Potential Risks

• Repeated LLM inference and simulation consume energy and have associated environmental costs.Source: §A.5 · p. 10
• Exploration reports must distinguish observed results from plans and assumptions, cite supporting turns, and omit claims unsupported by observations.The reporting instructions require experiments for reported turns, including construction and failed operations, and prohibit inventing results or measurements.Source: §A.5 · p. 11
• The evaluator checks whether reports identify altered laws and whether cited evidence supports the claims, while allowing supported partial findings to differ from full identification.The appendix warns that LLM judges may endorse unsupported findings and mislead subsequent research; physical use of generated experiments or apparatus could cause harm without independent validation and expert safety review.Source: §A.5 · pp. 10, 12