PaperlayerView exact revision on arXiv

Source-linked AI summary

FaSTA$^*$: Fast-Slow Toolpath Agent with Subroutine Mining for Efficient Multi-turn Image Editing

Advait Gupta, Rishie Raj, Dang Nguyen, Tianyi Zhou

arXiv:2506.20911v2cs.CV

TL;DR

Multi-turn image editing requires long sequences of heterogeneous tool calls, whose quality and cost make planning challenging. FaSTA* learns reusable subroutines from successful paths and tries them before invoking local A* search when needed. Compared with CoSTA*, it reports 49.3% cost savings with 3.2% quality degradation.

  • Problem

    Multi-turn image editing requires sequences of heterogeneous tool calls, but tool quality and cost vary across tasks and samples, making toolpath exploration expensive.

    Source: §1 · p. 1, §4.1 · p. 5

  • Method

    FaSTA* uses LLMs to mine reusable symbolic subroutines from explored successful toolpaths, selects them for planned subtasks, and falls back to local A* search when fast planning fails.

    Source: §1 · pp. 2–3, §4.2 · p. 6

  • Results

    FaSTA* saves 49.3% cost versus CoSTA* with 3.2% quality degradation.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Reusing learned subroutines for future tasks reduces the need to repeat costly exploration of recurring toolpaths.

    Source: §1 · p. 3, §4.1 · p. 5

    Performance on entirely new tasks may be suboptimal until sufficient experience has been gathered to learn effective subroutines.

    Source: §5.4 · p. 10

Abstract

from arXiv · show

We develop a cost-efficient neurosymbolic agent to address challenging multi-turn image editing tasks such as ``Detect the bench in the image while recoloring it to pink. Also, remove the cat for a clearer view and recolor the wall to yellow.'' It combines the fast, high-level subtask planning by large language models (LLMs) with the slow, accurate, tool-use, and local A$^*$ search per subtask to find a cost-efficient toolpath -- a sequence of calls to AI tools. To save the cost of A$^*$ on similar subtasks, we perform inductive reasoning on previously successful toolpaths via LLMs to continuously extract/refine frequently used subroutines and reuse them as new tools for future tasks in an adaptive fast-slow planning, where the higher-level subroutines are explored first, and only when they fail, the low-level A$^*$ search is activated. The reusable symbolic subroutines considerably save exploration cost on the same types of subtasks applied to similar images, yielding a human-like fast-slow toolpath agent ``FaSTA$^*$'': fast subtask planning followed by rule-based subroutine selection per subtask is attempted by LLMs at first, which is expected to cover most tasks, while slow A$^*$ search is only triggered for novel and challenging subtasks. By comparing with recent image editing approaches, we demonstrate FaSTA$^*$ is significantly more computationally efficient while remaining competitive with the state-of-the-art baseline in terms of success rate. Our code and data can be accessed at https://github.com/tianyi-lab/FaSTAR.

1 INTRODUCTION

Multi-turn image editing requires planning sequences of heterogeneous tool calls, but estimating tool quality and cost makes toolpath exploration expensive. FaSTA* learns reusable subroutines from successful paths and combines LLM-based fast planning with local A* fallback; it reports lower cost than CoSTA* with a small quality reduction.

1 INTRODUCTION

  • Long-horizon editing requires heterogeneous operations, while tool quality and cost vary across tasks and samples, making toolpath exploration expensive.The paper contrasts this challenge with text-to-image models’ difficulty handling long-horizon tasks directly.Source: §1 · p. 1
  • FaSTA* mines symbolic rules for recurring tool-call subroutines from successful explored paths and reuses them on future tasks.The rules include identifiable activation conditions.Source: §1 · pp. 2–3
  • FaSTA* condenses explored paths into reusable subroutines rather than retaining all prior tools and paths as context for future exploration.The paper frames this as an improvement to in-context reinforcement learning.Source: §1 · pp. 2–3
  • LLMs plan subtasks and select reusable subroutines first; local A* search is invoked only when fast planning fails.A subroutine can fail when its output does not pass VLM quality checks.Source: §1 · p. 3, Figure 2 · p. 3
  • 49.3% cost savings versus CoSTA* come with 3.2% quality degradation.The paper presents this as a better cost–quality trade-off than most baselines.Source: §1 · p. 3

2 RELATED WORK

Prior work spans generative and controllability methods for image editing, as well as agentic systems that decompose instructions for tool use. Efficient execution of recurrent operations and low-level tool sequences remains a challenge.

Multi-turn Image Editing

  • Generative models support synthesis and single edits, while multi-turn composite instructions remain challenging for iterative editing without sophisticated planning.Controllability methods improve precision on short sequences, and unified multimodal models have shown promise for native generation and editing.Source: §2 · p. 4
  • Agentic systems decompose editing goals for tool use, but efficient execution of recurrent operation patterns remains a challenge.The passage identifies lack of learning from past patterns as a source of redundant computational effort.Source: §2 · p. 4

Tool-use Agent and Planning

  • LLM-planned A* search improves toolpath planning, but intensive search costs persist without experience reuse.The related-work passage identifies robust learning and symbolic reuse of successful tool sequences as largely missing.Source: §2 · p. 4

3 PRELIMINARIES: COSTA∗AND EFFICIENT TOOLPATH SEARCH

CoSTA* maps natural-language editing tasks to toolpaths using task and tool representations, then applies cost-sensitive A* search over the resulting tool subgraph.

Foundational Components of CoSTA*

  • CoSTA* uses a Tool Dependency Graph, Model Description Table, and Benchmark Table to represent tool dependencies, tool capabilities, and cost/quality data.Together, these structures map high-level requirements to tool sequences and predicted performance.Source: §3 · p. 4

CoSTA* Planning Stages

  • An LLM turns the instruction and image into a subtask tree, which identifies relevant tools and dependencies for a task-specific Tool Subgraph.The paper notes that a single branch suffices in practice and reduces CoSTA* complexity.Source: §3 · pp. 4–5
  • Cost-sensitive A* searches the Tool Subgraph using accumulated actual cost plus a Benchmark Table-derived estimate of cost-to-go.VLM checks after tool executions enable dynamic updates and path corrections upon failure.Source: §3 · p. 5

4 FAST-SLOW TOOLPATH AGENT (FASTA∗)

FaSTA* addresses CoSTA*’s costly repeated A* planning by learning symbolic tool-call subroutines online and reusing them for later subtasks. It first attempts these learned routines and activates localized A* search when they are missing or fail quality checks.

4.1 FROM COSTA∗TO FASTA∗

  • FaSTA* augments CoSTA* with online symbolic-subroutine learning and reuses learned routines on later tasks, triggering A* search only when a routine fails.This targets repeated exploration of recurring subroutines in CoSTA* toolpaths.Source: §4.1 · p. 5

4.2 ONLINE LEARNING AND REFINEMENT OF REUSABLE SUBROUTINES

  • LLM inductive reasoning analyzes execution traces to identify effective tool-call sequences and their activation conditions, then validates proposed rule changes before acceptance.Subroutine reuse rates and fast-plan success increase with experience; induction uses diverse held-out tasks excluded from the benchmark.Source: §4.2 · pp. 5–6

4.3 ADAPTIVE FAST-SLOW PLANNING

  • An LLM selects context-matching subroutines for each subtask; missing routines or failed quality checks trigger localized low-level A* search.The fast plan is the default, with the more expensive fallback invoked only for subtasks where it fails.Source: §4.3 · pp. 6–7

5 EXPERIMENTS

FaSTA∗ substantially reduces execution cost while keeping quality close to CoSTA∗, and its cost-quality frontier is superior across trade-offs. Experiments also show that verification and low-level fallback matter for reliable performance, while cold-start performance and dependence on LLM capabilities remain limitations.

5.1 EXPERIMENTAL SETUP

  • The evaluation uses 121 image-prompt pairs with 1–8 subtasks per image, human-evaluated quality, and execution time including retries as its cost measure.CoSTA∗ is the primary baseline; both methods use α = 1 for the balanced cost-quality comparison.Source: §5.1 · pp. 7–8, Table 3 · p. 8

5.2 MAIN RESULTS

  • Over 49.3% lower average execution cost accompanies an approximately 3.2% accuracy drop versus CoSTA∗; Complex-Edit evaluation shows ∼30% lower cost with comparable quality.The comparison is reported for the main benchmark; the independent Complex-Edit result supports generalization beyond that benchmark.Source: §5.2 · p. 8, Table 4 · p. 8
  • FaSTA∗ achieves a superior cost-quality frontier to CoSTA∗ and other evaluated baselines, with roughly half CoSTA∗’s execution cost at high quality.Its frontier becomes increasingly optimal as more subroutines are mined; qualitative examples report results identical to CoSTA∗ and better accuracy and coherence than other baselines.Source: §5.2 · pp. 8–9, Figure 5 · p. 8, Figure 4 · p. 9

5.3 ABLATION STUDIES

  • Verified subroutines yield far fewer fallbacks to low-level A∗ search than directly inducted proposals, avoiding wasted execution on flawed rules.The comparison tests subroutine changes either directly or after the full verification process.Source: §5.3 · p. 9, Table 5 · p. 9
  • High-level-only planning causes a significant quality drop, while FaSTA∗’s low-level A∗ fallback can recover from failed subroutines or missing selections.The high-level-only variant fails when a subroutine fails its VLM check or none is selected, rather than activating the low-level graph.Source: §5.3 · pp. 9–10
  • CoSTA∗’s slow-planning-only variant achieves marginally higher output quality than FaSTA∗, but at significantly higher average execution cost.Source: §5.3 · p. 10
  • With Qwen 2.5VL 3B selecting subroutines, incorrect selections occur ∼40% of the time and total execution cost rises ∼20%, while final quality remains consistent.The increased cost is attributed to additional low-level A∗ fallback despite negligible API costs.Source: §5.3 · p. 10

5.4 SUMMARY AND LIMITATIONS

  • Performance on entirely new tasks may be suboptimal until sufficient experience is gathered, and nuanced complex-task rules remain dependent on evolving LLM capabilities.Source: §5.4 · p. 10

6 CONCLUSION

FaSTA* combines LLM-based online subroutine mining with adaptive fast-slow execution for complex, multi-turn image editing. Experiments show comparable image quality to CoSTA* at substantially reduced computational cost.

6 CONCLUSION

  • FaSTA* prioritizes LLM-selected reusable subroutines and falls back to localized A* search for novel or challenging subtasks.Subroutines are learned and refined from experience using LLM-based inductive reasoning.Source: §6 · p. 10
  • Experiments show image quality comparable to CoSTA* with substantially reduced computational cost.The authors identify reduced search cost as addressing a key bottleneck in prior methods.Source: §6 · p. 10
  • The authors present learned symbolic shortcuts combined with principled search as a direction for more agile, cost-sensitive agents.They describe this direction for complex tasks and continual improvement.Source: §6 · p. 10

A.1 QUALITATIVE GENERALIZATION TO MAGICBRUSH DATASET

FaSTA* completes the shown MagicBrush examples using learned subroutines without invoking low-level A* search. The CoSTA* dataset is used for the main evaluation because its complex, multi-turn and multimodal tasks better match FaSTA*’s efficiency goals.

A.1 QUALITATIVE GENERALIZATION TO MAGICBRUSH DATASET

  • All shown MagicBrush tasks were completed entirely through Fast Plans, without fallback to low-level A* search.Subroutines learned from different data sources handled these examples, offering positive evidence of generalization to new instances.Source: §A.1 · pp. 14–15

B BENCHMARK DATASET RATIONALE: COSTA* VS. MAGICBRUSH

  • The CoSTA* dataset was selected because its longer, multi-step tasks better test the cost-saving and adaptive-planning goals of FaSTA*.The authors note that many MagicBrush tasks involve simpler, more direct edits.Source: B BENCHMARK DATASET RATIONALE: COSTA*… · p. 15
  • CoSTA* includes multimodal text-in-image editing and 550 manipulation turns across 121 image-prompt pairs, with 1–8 subtasks per image.The paper contrasts this with MagicBrush’s primary focus on visual edits and limited coverage of text-in-image scenarios.Source: B BENCHMARK DATASET RATIONALE: COSTA*… · p. 15
  • In Figure 8 examples, learned rules select paths that avoid potentially failing alternatives, with average execution cost of 15.21s versus CoSTA*’s 25.32s.The caption describes results identical to CoSTA* at lower average cost for these examples.Source: Figure 8 · p. 16

C DETAILED QUALITATIVE ANALYSIS AND SUBROUTINE EFFECTIVENESS

FaSTA*’s cost falls as it learns subroutines while maintaining quality comparable to CoSTA*, and its fast plan can avoid costly exploration. The appendix also reports its fallback behavior, restricted-tool comparisons, and the underlying CoSTA* planning and search process.

C DETAILED QUALITATIVE ANALYSIS AND SUBROUTINE EFFECTIVENESS

  • In the small-ball example, FaSTA* uses SD Search&Recolor to avoid an SD Inpaint path that can fail on small targets, reaching the same high-quality edit more directly.For the examples shown, average execution cost is 15.21s for FaSTA* and 25.32s for CoSTA*; learned rules minimize trial-and-error exploration.Source: C DETAILED QUALITATIVE ANALYSIS AND… · pp. 16–17

C.1 COST/QUALITY TRADE-OFF W.R.T. SUBROUTINE MINING

  • Table 7 reports rapidly decreasing execution cost as FaSTA* learns subroutines, while quality remains comparable to CoSTA*.At cold start, cost and quality match CoSTA*; efficiency improves once an effective subroutine is learned and applied.Source: Table 7 · p. 17, §C.1 · p. 17
  • 49.3% to ∼45.2%: amortized over 2,000 tasks, the average cost reduction adjusts only slightly after accounting for LLM overhead.The authors state that inductive-reasoning overhead is amortized over the system’s lifespan and becomes negligible in long-term deployment after roughly 200 tasks.Source: §C.1 · p. 18

C.2 FASTA∗FAILURE CASE ANALYSIS

  • The fast plan handles 91% of subtasks, with A* fallback for the remaining 9%; 7% of subroutine failures typically involve novel tasks with unforeseen image complexity.Fallback uses A* search, which the authors say preserves final output quality.Source: §C.2 · p. 18

D FAIR COMPARISON WITH RESTRICTED TOOLSET AND SUBTASKS

  • With restricted toolsets and subtasks, FaSTA* performs comparably to CoSTA* and significantly outperforms the baselines.The authors attribute the observed differences primarily to planning and execution strategy rather than tool breadth.Source: D FAIR COMPARISON WITH RESTRICTED… · p. 19

E.1 FOUNDATIONAL KNOWLEDGE STRUCTURES IN COSTA∗

  • CoSTA*’s Model Description Table catalogs tool capabilities, inputs, and outputs; its Tool Dependency Graph captures valid tool-to-tool dependencies, and its Benchmark Table stores time and quality data.The MDT information supports mapping subtasks to tools and constructing the TDG; the BT supplies data for the A* heuristic.Source: §E.1 · pp. 19–20

E.2 COSTA∗PLANNING AND EXECUTION STAGES

  • CoSTA* decomposes an instruction into dependent subtasks, builds a task-relevant tool subgraph, then searches it for a cost-quality-balanced toolpath.The tool subgraph combines prerequisite tools and dependencies; the A* search uses a user-defined trade-off parameter α.Source: §E.2 · p. 20, §1 · p. 20, §2 · p. 20

3. Cost-Sensitive A∗Search for Optimal Toolpath

  • CoSTA* prioritizes search nodes using cumulative path cost-quality score plus an admissible estimate of cost-to-go; VLM checks and retries can redirect search after poor outputs.The actual path score uses measured execution time and VLM-validated quality, while the heuristic draws on Benchmark Table values.Source: §3 · pp. 20–21

F RATIONALE FOR USING SUBTASK CHAIN GENERATION

A single-path subtask chain reduces CoSTA*’s average execution time, while FaSTA*’s larger reported savings are attributed primarily to its adaptive fast-slow strategy. The appendix describes how execution traces inform subroutine proposals that are tested before adoption.

F RATIONALE FOR USING SUBTASK CHAIN GENERATION

  • A chain replaces CoSTA*’s potentially costly branching subtask tree with one plausible sequence, adopted after a preliminary comparison found similar output quality.The paper says the chain prompt aims to reduce planning complexity.Source: F RATIONALE FOR USING SUBTASK CHAIN… · p. 21
  • 58.2 to 46.8 seconds: CoSTA*’s average execution time falls with the single-path chain prompt, while FaSTA*’s 29.5s cost is attributed primarily to adaptive fast-slow execution.On 50 diverse tasks, final output quality differed by an average of only 1.08% between tree and chain prompts.Source: F RATIONALE FOR USING SUBTASK CHAIN… · pp. 21–22, Table 12 · p. 22

G.1 STEP 1: DATA LOGGING

  • FaSTA* logs traces containing subtask, executed toolpath, image and object context, path cost and quality, and failure information.Context features can include object size, mask properties, and higher-level semantic features.Source: §G.1 · p. 22

G.2 STEP 2: PERIODIC REFINEMENT TRIGGER

  • After a predefined number K of task inferences, the system retrieves the K most recent traces from its buffer for refinement.The passage gives K = 20 as an example trigger interval.Source: §G.2 · p. 22, §G.1 · p. 22

G.3 STEP 3: INDUCTIVE REASONING BY LLM

  • An LLM compares recent successful and failed traces with the current rule table, then proposes adding or modifying a subroutine or its activation conditions.The proposals are based on patterns linking context features, tool sequences, and observed outcomes.Source: §G.3 · pp. 22–23

G.4 STEP 4: VERIFICATION AND SELECTION OF NEW/MODIFIED SUBROUTINES

  • Proposed rules are tested against a baseline on subtask-specific samples and retained only when B(∆) < 0, with up to Nretries refinement attempts.If a proposal misses the net-benefit criterion, the LLM receives failure feedback and may refine it before a final decision.Source: §G.4 · p. 23

H EFFICACY OF ONLINE SUBROUTINE LEARNING

FaSTA*’s fast-plan success on held-out benchmark subtasks improves as its rule table is refined using external task executions. The learned rules focus on subtasks with multiple viable toolpaths and are updated online.

H EFFICACY OF ONLINE SUBROUTINE LEARNING

  • Learning traces come from diverse external tasks, separate from the benchmark tasks used to monitor Fast Plan success.This separation is intended to assess generalization rather than overfitting to benchmark data.Source: H EFFICACY OF ONLINE SUBROUTINE LEARNING · p. 24
  • The analysis tracks the share of applicable held-out subtasks solved by Fast Plan across successive updates based on 40 to 200 external task evaluations.It focuses on subtask types where multiple toolpaths are typically possible; the final table is used for the main benchmark evaluations.Source: H EFFICACY OF ONLINE SUBROUTINE LEARNING · p. 24
  • Fast Plan success on unseen benchmark tasks increases with more external task executions and reduces fallback to A* search for subtasks suited to subroutine learning.The evaluated Fast Plan success rate tracks successful use of learned subroutines without Slow Path fallback.Source: H EFFICACY OF ONLINE SUBROUTINE LEARNING · p. 24

I COMPLETE SUBROUTINE RULE TABLE

  • The rule table stores subroutines, context-based activation rules, associated subtasks, and empirical execution cost and quality, and is dynamically updated.Mining generally targets subtasks with multiple viable tool sequences; fixed-path subtasks are generally not mined into complex rules.Source: I COMPLETE SUBROUTINE RULE TABLE · p. 24

J.1 WITH DIRECT CACHING

The direct-cache comparison uses a tight context-feature match, while the end-to-end comparison evaluates FaSTA* against Gemini and GPT-4o. The supplied passages report high cache fallback but provide no table values for either comparison.

J.1 WITH DIRECT CACHING

  • Direct caching reuses a toolpath only when a new task’s context features closely match a cached example, a condition described as rare.Object size is given as an example of a context feature.Source: §J.1 · pp. 24–25
  • Table 13 lists the mined subroutines, their activation rules, and performance metrics used to generate Fast Plans.Source: Table 13 · p. 25
  • Even with a 5% context-feature tolerance, direct caching has an extremely high fallback rate, while FaSTA*’s learned symbolic rules handle a broader range of unseen tasks.Table 14 compares fallback rates against explored tasks; the passage supplies no numerical rates.Source: §J.1 · p. 25, Table 14 · p. 26

J.2 WITH END-TO-END MODELS

  • FaSTA* is qualitatively compared with Gemini 2.0 Flash Preview Image Generation on a few tasks and quantitatively compared with Gemini and GPT-4o on the benchmark.Table 15 is identified as a comparison of quality scores; no comparative values are supplied here.Source: §J.2 · p. 26, Table 15 · p. 26

K HUMAN EVALUATION METHODOLOGY FOR ACCURACY

Accuracy is assessed through human scoring of individual subtasks, aggregated first into task-level accuracy and then into overall benchmark accuracy. The section also introduces the procedure for verifying proposed subroutine changes.

K HUMAN EVALUATION METHODOLOGY FOR ACCURACY

  • Human evaluation measures accuracy because automated metrics may miss nuanced errors, semantic inconsistencies, and critical local changes; evaluator-score variance was 0.07.Evaluators manually score each subtask within a larger task.Source: K HUMAN EVALUATION METHODOLOGY FOR… · p. 26, §K.1 · p. 26

K.1 SUBTASK-LEVEL ACCURACY SCORING

  • Subtasks receive a score of 1 when fully correct, 0 when failed or unusable, and a predefined intermediate score when partially correct.Partial-correctness scores follow operation-specific rules adapted from Gupta et al. (2025).Source: §K.1 · p. 26

K.2 TASK-LEVEL ACCURACY CALCULATION

  • Task-level accuracy is the arithmetic mean of the correctness scores for all constituent subtasks.This aggregation is intended to reflect performance across all required steps.Source: §K.2 · pp. 26–27

K.3 OVERALL SYSTEM ACCURACY

  • Overall accuracy is the average of task-level accuracies across all evaluated benchmark tasks.The dataset’s total number of tasks defines the averaging set.Source: §K.3 · p. 27

L SUBROUTINE VERIFICATION: DATASETS AND EVALUATION PROTOCOL

L.1 SUBTASK-SPECIFIC TEST DATASETS (Ds∆)

Subroutine changes are tested on datasets tailored to each target subtask, using sampled tasks to compare quality and execution time. The process also analyzes execution traces for inductive reasoning.

L.1 SUBTASK-SPECIFIC TEST DATASETS (Ds∆)

  • Each target subtask has a specialized test dataset built from CoSTA* benchmark images: image-based datasets use all 121 images, while text-related datasets use the 40 images containing relevant text.Source: §L.1 · p. 27
  • Prompts target one subtask type per dataset, and task complexity varies; Figure 10 shows manipulation counts while Figure 11 illustrates generation from a shared base image.Image-based datasets can have different manipulation totals because images contain different numbers of relevant objects or regions and prompts vary in complexity.Source: §L.1 · pp. 27–28, Figure 10 · p. 28, Figure 11 · p. 29

L.2 EVALUATION PROTOCOL FOR SUBROUTINE CHANGES

  • Each proposed change is evaluated on a random sample of 25 image-based tasks or 20 text-based tasks, with automated VLM checks measuring quality and average execution time measuring cost.Quality is averaged across subtasks within each task, then across sampled tasks.Source: §L.2 · p. 28
  • The specialized datasets and evaluation protocol assess proposed changes on the specific subtask type they target.Source: §L.2 · p. 28

M TRACE DATA FOR INDUCTIVE REASONING

M.1 TRACE COMPOSITION AND DATA GATHERING

This section describes FaSTA∗’s trace-based composition and data gathering, covering subtask ordering, image-conditioned subroutine selection, and the use of execution traces to infer and refine activation rules for reusable paths.

M.1 TRACE COMPOSITION AND DATA GATHERING

  • Each trace records the subtask, executed tool path, image and object context, path cost and quality, and tool or quality-check failures.Features include object size and mask or color properties, text-box size, and LLM-inferred context such as clarity, background type, and target overlap.Source: §M.1 · pp. 29–30

M.2 EXAMPLE TRACE FOR AN OBJECT RECOLORATION SUBTASK

  • In the recoloration example, the initial Grounding DINO–SAM–SD Inpaint path failed quality checks, while a retraced SD Search&Recolor path passed.When the selected path lacks detection or segmentation tools, an LLM estimates context such as object size and color for the trace.Source: §M.2 · p. 30

N SUBROUTINE REUSE RATE

  • Subroutine reuse rate measures successful selections among applicable subtasks, and Figure 13 reports this rate for all 16 learned subroutines.The rate excludes unrelated subtask types; higher values indicate a subroutine is often selected when relevant.Source: N SUBROUTINE REUSE RATE · p. 31, Figure 13 · p. 31

O USE OF LARGE LANGUAGE MODELS (LLMS)

  • GPT-4o converts image-and-prompt instructions into an ordered subtask chain, while GPT-o1 selects cost-effective rules and VLM checks can trigger A∗ fallback.When a suitable subroutine is unavailable, or a quality check fails, planning falls back to localized A∗ search for that subtask.Source: O USE OF LARGE LANGUAGE MODELS (LLMS) · p. 32, §1 · p. 34

P.1 FASTA∗ONLINE SUBROUTINE INDUCTION

  • LLMs analyze successful and failed execution traces to induce reusable symbolic subroutines and activation rules, with FaSTA∗ updating its rule table explicitly.After every K = 20 task executions, recent traces are analyzed; proposed changes are validated for net cost-quality benefit and may be refined for up to two retries.Source: O USE OF LARGE LANGUAGE MODELS (LLMS) · p. 32, §7 · p. 33, §1 · p. 33

P.2 FASTA∗ADAPTIVE FAST-SLOW EXECUTION

  • The adaptive executor attempts a subroutine for each planned subtask, then uses localized A∗ search if no rule applies or execution fails.It proceeds to the next subtask after successful completion.Source: §1 · p. 34

Q LLM PROMPT FOR GENERATING SUBTASK CHAIN

  • The subtask-chain prompt requires logical dependency ordering and distinct object-based labels, and restricts plans to the supported subtask types.For example, an operation on a replacement object must follow the replacement that creates it.Source: Q LLM PROMPT FOR GENERATING SUBTASK… · p. 35

Q.7 EXAMPLE INPUTS & EXPECTED OUTPUTS

  • The chain-generation examples show a prompt decomposed into ordered, labeled editing steps, including pedestrian detection followed by car removal.The supplied example output begins with those two subtasks and their parent relationship.Source: §Q.7 · p. 36
  • Subtask chains should respect operational dependencies rather than prompt order, placing dependent edits after the operations that determine their inputs.For example, recoloring a car follows replacing a truck with a car, and text detection follows text replacement.Source: §Q.7 · p. 37

R LLM PROMPT FOR SUBROUTINE SELECTION

  • Subroutine selection checks each subtask’s relevant image conditions against rule observations, allows multiple matches, and returns None when no rule applies.The prompt calls for inspecting local context around the relevant object rather than the entire image, with None leaving the subtask for A* search.Source: R LLM PROMPT FOR SUBROUTINE SELECTION · pp. 38–39

S LLM PROMPT FOR INDUCTIVE REASONING ON SUBROUTINES

  • Trace-derived activation rules should capture conditions associated with success, avoid known failure conditions, and be inferred separately for each path–subtask pair.The induction prompt asks for positive activation rules, including negated conditions when a condition consistently predicts failure.Source: S LLM PROMPT FOR INDUCTIVE REASONING… · pp. 40–42

Related papers

Loading 2506.20911v2…