PaperlayerView exact revision on arXiv

Source-linked AI summary

SCOPE: Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets

Chaoyue Li, Zeyu Gao, Yanfeng Lu, Fan Luo

arXiv:2610.31518v1cs.RO

TL;DR

High-quality TAMP fine-tuning data is scarce, and automated generation can produce repetitive scenes or unsolvable tasks through hallucinated scene graphs. SCOPE combines diversity-aware scene generation with planner-feedback correction; SCOPE-generated data yields a 12.33% absolute success-rate gain over DELTA-generated data when used for LLM fine-tuning.

  • Problem

    High-quality TAMP fine-tuning data is scarce, while automated data generation can produce repetitive scenes and unsolvable tasks through hallucinated scene graphs.

    Source: §I · p. 1

  • Method

    SCOPE combines multi-level diversity metrics with planner-execution feedback to retroactively correct logical inconsistencies in scene graphs.

    Source: §I · p. 2, 2) Autoregressive Error-Correction and… · p. 4

  • Results

    12.33% absolute gain in success rate over DELTA-generated data was achieved by the downstream TAMP model fine-tuned on SCOPE-generated data.

    Source: §I · p. 2

  • Takeaways & Limitations

    SCOPE-generated data provides a higher-quality data source for LLM fine-tuning and supports more robust long-horizon planning in realistic simulations.

    Source: 2) Autoregressive Error-Correction and… · p. 5, §V · p. 8

Abstract

from arXiv · show

Task and Motion Planning (TAMP) stands as a fundamental challenge in robotics, essential for enabling autonomous agents to execute complex, long-horizon tasks in diverse environments. While Large Language Models (LLMs) introduce a new paradigm for TAMP by leveraging extensive common sense, their inherent reasoning limitations hinder direct application, and fine-tuning strategies to enhance these capabilities are currently bottlenecked by the scarcity of high-quality datasets. Existing automated generation methods often suffer from homogeneity, limiting model generalization, and unsolvability, where hallucinated scene graphs create physical mismatches with task goals. To address these issues, we propose Scene-graph Construction Optimized by Plan Error-correction (SCOPE), an automated framework designed to generate diverse and executable TAMP datasets. SCOPE incorporates multi-level diversity metrics to maximize the information entropy of scenarios. Crucially, it introduces a Plan Feedback Error-correction pipeline that utilizes planner execution failures as feedback signals to retroactively rectify logical inconsistencies in scene graphs, ensuring physical feasibility. Experimental results demonstrate that SCOPE exhibits greater robustness to erroneous scene graphs and generates higher-quality data. When utilized for LLM fine-tuning, SCOPE-generated data yield a downstream TAMP model that achieves an absolute 12.33% gain in success rate over its counterpart fine-tuned with data generated by DELTA, a state-of-the-art PDDL-based planner.

I. INTRODUCTION

LLMs offer a route to TAMP, but their planning limitations and the scarcity and quality of fine-tuning data constrain their use. SCOPE targets homogeneous, physically unsolvable generated data with diversity-aware generation and planner-feedback correction; its data yields a 12.33% absolute success-rate gain over DELTA-generated data.

I. INTRODUCTION

  • LLMs bring common-sense knowledge to TAMP, whereas traditional rule-based and heuristic approaches often lack sufficient generalization.The introduction identifies LLM reasoning limitations as a challenge to direct application.Source: §I · p. 1
  • Manually annotated TAMP datasets offer high quality but incur prohibitive costs, while low-cost LLM-generated data often has low quality.A standard fine-tuning sample contains a scene graph, task instruction, and solution plan.Source: §I · p. 1
  • LLM-generated data can be repetitive and physically unsolvable when hallucinated scene graphs omit interactive objects or encode incorrect attributes.These mismatches introduce noise into training data and limit model generalization.Source: §I · pp. 1–2
  • SCOPE combines multi-level diversity metrics with plan-feedback correction to generate diverse, physically executable TAMP datasets.Planner execution results serve as feedback to identify and correct scene-graph inconsistencies.Source: §I · p. 2
  • 12.33% absolute gain in planning success rate over data generated by DELTA shows higher end-to-end performance for models fine-tuned on SCOPE data.Experiments were validated in AI2-THOR.Source: §I · p. 2

II. RELATED WORK

LLM-based TAMP includes symbolic and direct planning approaches, while instruction-tuning outcomes depend on training-data quality and diversity. Existing automated data-generation approaches face both limited scene diversity and task solvability concerns.

A. Task and Motion Planning with LLMs

  • Both symbolic and direct planning remain constrained by the underlying planning capabilities of their LLMs.Fine-tuning is presented as a strategy to internalize planning logic and enhance temporal reasoning.Source: A. Task and Motion Planning with LLMs · p. 2

B. Instruction Tuning and Automated Data Generation

  • Human-annotated datasets can offer high-quality examples but are costly and limited in scale; LLM-synthesized data is scalable but can introduce noise.PDDL-based planners can improve logical consistency over purely LLM-generated data.Source: B. Instruction Tuning and Automated… · p. 2
  • Automated PDDL-based pipelines often produce structurally simple, homogeneous scenes and do not guarantee that task goals are solvable from initial states.The passage attributes invalid samples to persistent hallucinations in task formulation.Source: B. Instruction Tuning and Automated… · p. 2

III. METHODS

SCOPE uses statistical feedback and dynamic sampling to diversify scenes, then constructs scene graphs through structured instructions. Its scene-generation mechanisms address both repeated room and asset configurations and structural errors in generated graphs.

III. METHODS

  • SCOPE addresses automated-data homogeneity and physical unsolvability through a closed-loop framework that refines environment configurations to align with task goals.The framework overview lists diverse scene generation, CoT instruction generation, and planning-feedback scene-graph correction.Source: §III · p. 2, Figure 2 · p. 3

A. Diverse Scene Graph Generation via Statistical Feedback and Dynamic Sampling

  • A room-composition tracker uses historical room-type frequencies to estimate diversity, with lower entropy indicating a more concentrated and repetitive distribution.An inverse-frequency weighting mechanism dynamically adjusts selection probabilities; α is empirically set to 3.0.Source: A. Diverse Scene Graph Generation via… · p. 3
  • Dynamic sampling penalizes frequently selected room types to encourage exploration of underused configurations.The mechanism applies exponential weighting to room-type selection probabilities.Source: A. Diverse Scene Graph Generation via… · p. 3
  • For each room type, SCOPE samples a restricted furniture and object subset to vary asset combinations across scene generations.Furniture samples contain 5–10 assets and object samples contain 20–30.Source: A. Diverse Scene Graph Generation via… · p. 3
  • A lightweight DSL builds scene graphs coarse-to-fine, establishing room and agent state before adding furniture and placing objects.The sequence is designed to avoid structural errors such as unsupported objects and inconsistent containment relationships.Source: A. Diverse Scene Graph Generation via… · p. 3

B. Chain-of-Thought Driven Task Instruction Generation

SCOPE generates task instructions from scene entities through a CoT sequence that constructs plausible action chains and renders them in natural language. It then verifies entity consistency and uses solver outcomes to diagnose and correct scene-graph errors.

B. Chain-of-Thought Driven Task Instruction Generation

  • Instructions are generated from furniture and object nodes in the scene graph to create semantically rich, difficulty-graded TAMP tasks.The generation strategy is scene-entity-oriented.Source: B. Chain-of-Thought Driven Task… · p. 4
  • The instruction pipeline proceeds through target sampling, action-chain construction, and natural-language embodiment.LLMs first select candidate assets with interaction potential, then construct a causal chain toward the target state.Source: B. Chain-of-Thought Driven Task… · p. 4
  • Action chains are constrained by embodied configurations and sampled assets to support logical coherence and physical executability.The LLM uses scene-graph attributes and spatial relations to filter candidate assets.Source: B. Chain-of-Thought Driven Task… · p. 4
  • Task descriptions vary in how explicitly user intent is expressed, producing simple, complex, and vague difficulty levels.Source: B. Chain-of-Thought Driven Task… · p. 4
  • A set-matching verification module retains only instructions whose target objects and locations exist among the scene entities.It checks whether parsed target objects and locations are contained in the scene entities.Source: B. Chain-of-Thought Driven Task… · p. 4

C. Plan Generation and Error-Correction

  • SCOPE uses a PDDL solver to decompose tasks and generate executable plans, treating solver failures or inconsistent plans as diagnostic signals.The signals can indicate missing support or containment relations, invalid initial states, or unreachable target entities.Source: C. Plan Generation and Error-Correction · p. 4

1) Task Decomposition and Formalization:

SCOPE formalizes instructions as PDDL sub-goals and uses planner failures to identify and repair scene-graph deficiencies. It repeats planning and repair until it obtains valid sub-plans for a complete executable task sequence.

1) Task Decomposition and Formalization:

2) Autoregressive Error-Correction and Optimization of Scene Graphs via Planning Feedback:

  • When planning fails because a scene graph omits needed attributes, SCOPE uses the failure as a trigger for reverse repair rather than discarding the task.The passage gives a missing openable state as an example of a deficiency that can block reaching a target.Source: 2) Autoregressive Error-Correction and… · p. 4
  • SCOPE traces failure to the target action and infers the missing structural state needed to make that action valid.The deficiency can correspond to absent static physical attributes or initial dynamic states.Source: 2) Autoregressive Error-Correction and… · p. 4
  • SCOPE adds the inferred missing elements to entity nodes, producing an updated state and scene graph that satisfies action prerequisites.The repaired scene state is returned to the planner for verification.Source: 2) Autoregressive Error-Correction and… · p. 4
  • A Symbolic Rule-LLM module identifies missing preconditions through a rule mapping, using LLM reasoning for long-tail actions not covered by the table.For example, a failed power_on plan can prompt addition of a powerable attribute.Source: 2) Autoregressive Error-Correction and… · p. 4
  • The planner-repair loop continues until each sub-task has a valid sub-plan, which SCOPE concatenates into a complete task sequence.The system processes the sub-tasks autoregressively.Source: 2) Autoregressive Error-Correction and… · p. 4
  • The resulting data tuple pairs the corrected scene graph and instruction with a valid planning path, yielding physically executable training data.The tuple is E = ⟨SGfinal, I, Π⟩.Source: 2) Autoregressive Error-Correction and… · p. 5

D. Overview of the SCOPE Framework and Pipeline Design

SCOPE generates scene-instruction pairs and converts them into executable planning data through scene generation, instruction synthesis, and planning with repair. The experiments ask whether it improves robustness to erroneous scene graphs and the quality and downstream value of generated data.

D. Overview of the SCOPE Framework and Pipeline Design

  • The pipeline samples long-tail scene configurations, compiles scene graphs, and synthesizes task instructions through target sampling, action-chain construction, and language embodiment.A set-matching check verifies that the generated instructions are valid for the scene.Source: D. Overview of the SCOPE Framework and… · p. 5
  • SCOPE converts scene-instruction pairs into serialized PDDL sub-problems, repairs scene graphs after planning failures, and replans until it obtains valid sub-plans.It concatenates the sub-plans into a final tuple containing the corrected scene graph, instruction, and plan.Source: D. Overview of the SCOPE Framework and… · p. 5

IV. EXPERIMENTS

  • The experiments examine planning robustness on logically erroneous scene graphs and whether generated-data quality affects fine-tuned LLM performance on TAMP tasks.Source: §IV · p. 5

A. Experimental Setup

The setup uses separate datasets for testing planning under synthetic scene-graph errors and fine-tuned model generalization in grounded simulator states. Tasks are categorized by difficulty and reasoning demands.

1) Dataset Construction:

  • The study constructs two datasets, each containing 100 scene graphs and 300 instructions across three difficulty levels.Source: 1) Dataset Construction · p. 5
  • Simple tasks test basic action mapping, complex tasks require long-horizon dependencies and physical preconditions, and vague tasks assess intent understanding under ambiguity.Source: 1) Dataset Construction · p. 5
  • The LLM-Synthetic Noisy Dataset tests planner robustness and feedback-driven repair, while the AI2-THOR Grounded Dataset tests fine-tuned model generalization in logic-flaw-free simulator environments.Source: 1) Dataset Construction · p. 5

2) Evaluation Metrics and Baselines:

The evaluation defines physical executability and task success metrics, compares SCOPE with four planning baselines, and includes noisy-scene robustness and grounded-data ablation evaluations.

2) Evaluation Metrics and Baselines:

  • Executability measures successfully executed actions, Average Quality Score grades trajectories from 0–4, and Success Rate counts only perfect-scoring tasks.Executability does not require overall task relevance; Success Rate requires Score = 4.Source: 2) Evaluation Metrics and Baselines · pp. 5–6
  • Table I evaluates planning robustness on an LLM-synthetic noisy dataset under erroneous scene graphs, comparing SCOPE with baseline models.Source: Table I · p. 6
  • Table II covers fine-tuned model generalization on AI2-THOR grounded data, with an ablation varying scene diversity and error-correction.Source: Table II · p. 6
  • The baselines span open-loop LLM action generation, LLM-to-PDDL planning, executable-code planning, and hierarchical autoregressive planning.These correspond to LLM-As-Planner, LLM+P, LLMGenPlan, and DELTA.Source: 2) Evaluation Metrics and Baselines · p. 6

B. Planning Robustness and Repair Capabilities under Erroneous Data

On identical noisy scene graphs without fine-tuning, SCOPE outperforms the tested planning baselines in success and executability, while repairing scene-graph errors enables strong performance on complex tasks.

B. Planning Robustness and Repair Capabilities under Erroneous Data

  • LLMGenPlan reaches 6.33% overall success and 36.67% executability; retries raise executability to 61.33% but success only to 23.00%.LLM-As-Planner achieves 27.00% overall success and 29.67% executability on noisy graphs with missing attributes.Source: B. Planning Robustness and Repair… · p. 6
  • Retry-based text validation filters invalid action expressions but does not repair scene-graph logic, leaving physically executable plans that can be semantically incorrect.LLMGenPlan Replan success falls to 14.00% on complex tasks involving long-horizon dependencies.Source: B. Planning Robustness and Repair… · p. 6
  • LLM+P and DELTA achieve only 30.00% and 31.00% success, respectively, as rigid initial-state dependence limits executability to 48.00% and 67.00%.Their average scores on complex tasks remain below 1.94.Source: B. Planning Robustness and Repair… · p. 6
  • 70.67% overall success, 92.67% executability, and a 3.28 average score; SCOPE also reaches 94.00% executability on complex tasks.Its success rates on complex and vague tasks are 55.00% and 71.00%, respectively, over 24% above DELTA on those tasks.Source: B. Planning Robustness and Repair… · pp. 6–7
  • SCOPE reverse-traces missing preconditions to repair scene graphs when the solver halts, supporting physically grounded plan execution.The reported complex-task executability is 94.00%.Source: B. Planning Robustness and Repair… · p. 7

C. Impact of Scene Diversity and Error-correction on Model Fine-Tuning

The fine-tuning study compares datasets generated with conventional or entropy-aware diverse scenes, then evaluates downstream planning performance on AI2-THOR.

C. Impact of Scene Diversity and Error-correction on Model Fine-Tuning

  • The study fine-tunes Qwen2.5-7B-Instruct on separate 10,000-triplet datasets generated by LLM-as-Planner, DELTA, and SCOPE configurations.All models use identical LoRA settings and are evaluated on the AI2-THOR Grounded Dataset.Source: C. Impact of Scene Diversity and… · p. 7

1) Impact of Scene Diversity on Fine-Tuning Performance:

  • With diverse scenes, DELTA success rises from 50.67% to 59.67%, executability reaches 87.00%, and complex-task success increases by nearly 12%.The authors associate these improvements with broader coverage of long-tail environmental distributions.Source: 1) Impact of Scene Diversity on… · p. 7

2) Impact of Error-correction on Fine-Tuning Performance:

Fine-tuning results show gains from plan feedback error-correction and from combining it with diverse scenes; simulator tests and failure analysis qualify the remaining scope.

2) Impact of Error-correction on Fine-Tuning Performance:

  • On conventional scenes, SCOPE-generated data yields 58.00% success and 81.00% executability, while LLM-as-Planner data yields 25.00% success and 26.00% executability.The authors describe plan feedback error-correction as rectifying hallucinated, initially inexecutable trajectories into physically constrained training samples.Source: 2) Impact of Error-correction on… · p. 7
  • 63.00% overall success, 85.33% executability, and a 3.08 average score are reported when error-correction is combined with diverse scenes.With conventional scenes, SCOPE reaches 58.00% success and 81.00% executability, versus DELTA's 50.67% success; complex-task success is 41.00% versus 28.00%.Source: 2) Impact of Error-correction on… · p. 7
  • Simulator execution shows the fine-tuned model producing physically compliant action sequences and completing multi-step long-horizon tasks.This is an interactive execution validation in AI2-THOR.Source: 2) Impact of Error-correction on… · p. 7
  • Among 111 failed cases, about 60% are cognitive errors, including 33 instruction omissions and 34 logic or semantic errors.The remaining cases include 23 physical limitations and 21 perceptual hallucinations.Source: 2) Impact of Error-correction on… · p. 7

V. CONCLUSION

  • The conclusion identifies future work in more complex, dynamic environments.The authors characterize SCOPE as combining entropy-aware scene generation with plan feedback error-correction for diverse, physically executable datasets.Source: §V · p. 8

Related papers

Loading 2610.31518v1…