Automating the Design of Embodied Agent Architectures

Jian Zhou; Sihao Lin; Jin Li; Shuai Fu; Gengze Zhou; Qi Wu

Source-linked AI summary

Automating the Design of Embodied Agent Architectures

Jian Zhou, Sihao Lin, Jin Li, Shuai Fu, Gengze Zhou, Qi Wu

arXiv:2606.30111v2Jun 29, 2026cs.ROcs.AIcs.LG

TL;DR

Embodied agent architectures are usually designed by hand despite a growing space of choices. This paper introduces a simulator-grounded architecture-search framework and finds confirmed success-rate gains across several embodied executors, while exposing important evaluation and attribution limits.

Problem
Embodied-agent architecture design involves many choices that are usually fixed manually, motivating the question of whether architecture design can be automated.
Method
The paper combines AgentCanvas, an editable typed-graph simulator runtime, with KDLoop and two ported AAS variants to search graph-level modifications across embodied executors.
Results
Several searched graphs obtain confirmed success-rate gains over seeded baselines across four executors and three embodied task families.
Takeaways & Limitations
Architecture-level search can produce functional, deployable improvements for embodied agents when key failure modes lie within the editable graph.
Takeaways & Limitations
The study is an initial characterization limited to workflow-shaped executors, while noisy rollouts and local search basins can obscure architecture quality and exploration.

Abstract

from arXiv · show

Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules. This modularity exposes a large architectural design space, but current systems still rely on researcher intuition to choose where information is stored, how observations are processed, and how model calls are connected. Agent Architecture Search (AAS) automates such design for text-domain agents, but has not been systematically evaluated on perceptual embodied agents through simulator rollouts. We study this transfer. We introduce AgentCanvas, a typed-graph runtime that hosts embodied executors as editable node-and-wire programs with simulator-aware execution and episode-level logs, and KDLoop, a coding-agent search procedure that cycles through proposal, critique, experiment, and distillation, with triggered reflection after stalls. We evaluate three AAS variants across four embodied executors spanning vision-language navigation, embodied question answering, and language-conditioned manipulation. The resulting 3x4 matrix shows that architecture-level search can produce deployable and directional success-rate gains on embodied tasks, while one apparent high-scoring candidate is rejected as leak-bearing. At the same time, the experiments expose constraints that are muted in text-domain AAS: optimization signals can be masked by rollout noise, search can become trapped in local edit basins, and episode-level credit assignment only partially emerges even when detailed logs are available. These results characterize both the promise and the current limits of automated architecture search for embodied agents.

1 Introduction

Embodied agents expose a growing hand-designed architectural search space, motivating automated architecture search beyond text-domain agents. This study introduces a simulator-integrated graph substrate and method-seeded search harness, finding embodied success-rate gains alongside rollout-noise, local-search, and credit-assignment constraints.

Motivation: Embodied agents compose perception, mapping, memory, planning, and control modules across navigation, question answering, and manipulation tasks.Their structured pipelines transform observations and internal state into model-based reasoning and actions.
Motivation: Hand-designed choices about representations, memory, prompts, planner topology, model placement, and action interfaces create an architectural space too large for manual iteration.The proliferation of foundation models and embodied tools intensifies this design problem.
Prior work and gap: Agent Architecture Search automates alternative LLM-agent workflows in text domains, but its transfer to embodied agents requires evaluation through embodied execution.The introduction frames embodied AAS as the next setting for testing whether architecture design itself can be automated.
Method: AgentCanvas represents embodied agents as editable typed node-and-wire graphs that execute and log inside a simulator, while the harness compares architecture-search optimizers from published seed architectures.The method searches nearby graph-level modifications rather than assembling agents from scratch.
Findings: Across four executors and three embodied task families, architecture search produces confirmed success-rate gains over seeded baselines, but rollout noise, local edit basins, and partial credit assignment constrain optimization.The results indicate functional behavioral changes while exposing evaluation-regime limitations specific to embodied AAS.

2 Related Work

Related work spans embodied agents that explicitly connect perception, language, memory, tools, geometry, and action, alongside Agent Architecture Search (AAS) for automatically designing LLM-agent workflows. Extending AAS to embodied systems requires coding-agent harnesses and typed graph substrates beyond single-completion text-domain search.

Embodied Agent Systems: Embodied AI includes vision-language navigation, embodied question answering, and instruction-following household tasks such as ALFRED.These systems connect perception, language, and action in interactive environments.
Embodied Agent Systems: Existing embodied systems explicitly compose frozen LLMs and VLMs with memory, tools, geometric modules, and low-level controllers, leaving module choices hand-designed for each benchmark.Explicit structure exposes a search space, but current systems still rely on researcher-designed architectures.
Agent Architecture Search: AAS uses an external optimizer at development time to search candidate LLM-agent architectures, unlike self-improving agents that adapt during deployment.Prior work includes ADAS, AFlow, AgentSquare, and MaAS, which explore stronger optimizers, structured workflow spaces, and multi-agent compositions.
Coding Agents and Graph Substrates: Embodied AAS requires coding-agent harnesses and execution substrates because single-completion candidate generation is insufficient for multi-file embodied-agent code.The cited coding-agent systems edit codebases, run tests, and iterate; typed graph substrates support the required execution structure.

3 Method

The method separates task-performing Executors from external Optimizers and searches editable, simulator-runnable embodied-agent graphs from published method seeds. AgentCanvas provides typed graph execution and episode logs, while three search variants share the same harness and differ in proposal and memory mechanisms.

Executor–Optimizer separation: Executors are typed graphs of perception, memory, planning, and action modules that run in simulators, while Optimizers propose edits and trigger evaluations without acting directly.The Optimizer is a separate LLM-agent system operating across development-time iterations.
Search space: Candidates are method-seeded from published embodied agents because task-specific interfaces lack a comparable library of generic, runnable operators.The setting tests whether AAS can improve real executors rather than synthetic workflow sketches.
Executor substrate: AgentCanvas represents Executors as typed node-and-wire JSON graphs backed by Python modules, applying type-checked structured patches before expensive rollouts.Supported edits include adding edges, swapping nodes, and replacing subgraphs.
Shared harness: The shared harness runs proposer, implementer, evaluator, validation, logging, and file-access stages, so variants differ only in proposer logic and persistent memory.All variants use the same Executor substrate, evaluation suite, and iteration budget.
KDLoop: KDLoop cycles through THINK, CRITIC, EXPERIMENT, and distillation phases while tracking intervention axes and mechanism-level evidence to reduce repeated local sampling.Its coverage bias can leave a productive basin before deeper exploitation is complete, so it is evaluated both as an optimizer and as a probe of search structure.

4 Experiments

Experiments evaluate architecture-level search across a 3×4 optimizer–executor matrix of embodied agents validated through simulator rollouts. Search yields directional improvements in several cells, but rerun variance, local edit basins, and incomplete episode-level credit assignment limit certification and interpretation.

Experimental scope: The study systematically evaluates AAS-style graph edits across four published embodied executors spanning navigation, question answering, and manipulation.Each cell starts from its corresponding method baseline and uses task success rate (SR) as search-time fitness.
Results: Several selected candidates improve after reruns, with MapGPT reaching ∼54% SR from a 46.9±3.1 baseline under both AFlow and KDLoop.The gains arise through different stopping, revisitation, and action-gating edits; variance-overlapping deltas remain directional rather than certified.
Search behavior: No optimizer dominates: ADAS emphasizes topology and model configuration, AFlow often edits observation processing, and KDLoop spans observation, prompt, control, and state-memory axes.ADAS finds no improving graph-level lever on ExploreEQA, while KDLoop identifies a shared VoxPoser logging fault and ends with a below-graph diagnosis.
Robustness and certification: 43/32/41% reruns produce 38.7±5.9% SR for AFlow–SmartWay, while KDLoop’s 32/35/26% produce 31.0±4.6% SR, showing why single-pass selection requires repeated evaluation.Lower-variance improvements remain separated from baselines after rerun confirmation, including KDLoop–MapGPT at 54.0±2.3 versus 46.9±3.1.
Search limitations: Search frequently remains within local edit basins: the most-used axis averages 60%/43%/38% for ADAS/AFlow/KDLoop, and ADAS repeats SR=0.4769 across six iterations.Without a space-level stopping predicate, ADAS and AFlow reach the iteration cap in all eight cells, using 84 and 82 committed iterations.
Credit assignment: Log access alone is insufficient for reliable episode-level credit assignment, even when observations, actions, tool calls, planner outputs, self-reports, and simulator traces are available.In injected VoxPoser defect probes, KDLoop detects missing dynamically dispatched LMP sub-call traces, whereas ADAS and AFlow remain focused on scalar outcomes.

5 Limitations and Future Work · Appendix Contents

The paper presents embodied AAS as an initial characterization, limited by executable scope and costly, noisy optimization signals. The appendices document the runtime, search trajectories, harness, implementation, experimental setup, and formal problem formulation.

5 Limitations and Future Work: AgentCanvas covers many embodied agents through typed node-and-wire workflows but excludes systems with dynamically constructed control flow, tools, or memory.This limitation defines the current executable scope of the system.
5 Limitations and Future Work: Future work should extend AAS to richer executable representations and safer edit operators beyond workflow-shaped agents.The proposed direction directly addresses AgentCanvas’s current representation boundary.
5 Limitations and Future Work: Costly, noisy rollouts can obscure the optimization signal during embodied architecture search.The passage identifies rollout expense and noise as limitations of the optimization process.
Appendix Contents: Appendix A documents AgentCanvas’s typed-graph Executor substrate.It provides the runtime details underlying the executor representation.
Appendix Contents: Appendix B provides full search trajectories across the 12-cell grid.These trajectories cover the complete per-cell search record.
Appendix Contents: Appendix C describes the method-agnostic Optimizer-side coding-agent harness.The harness is separated from any particular search method.
Appendix Contents: Appendix D details per-variant proposers and memory, while Appendix E covers executors, evaluation tiers, models, and the rerun protocol.These appendices specify implementation and experimental procedures.
Appendix Contents: Appendix F presents the formal search space and method-seed formulation.The supplied appendix contents identify Appendix F as the formal problem-formulation section.

A AgentCanvas: A Typed-Graph Executor Substrate for Embodied AAS

AgentCanvas is a typed-graph executor substrate that makes embodied-agent architecture spaces editable, runnable, instrumented, and traversable by an optimizer. It defines interfaces for graph editing, static validation, benchmark-scale evaluation, termination, and episode-level evidence for credit assignment.

AgentCanvas substrate: AgentCanvas represents each embodied agent as an editable, runnable, instrumented program, making candidate architectures traversable by an AAS Optimizer.The substrate’s scope excludes the human-facing visual editor, node-authoring API, and wider platform.
Inference interface: The inference interface treats agents as editable typed graphs, with static checks gating edits before simulator rollouts.The graph uses typed nodes, typed-port wires, and shared state accessed through grants.
Evaluation interface: The evaluate interface lets an Optimizer score candidates autonomously at benchmark scale with guaranteed termination, while episode-level evidence supports credit assignment.AgentCanvas’s batch evaluation decouples workers rather than forcing every worker to wait for the slowest module.

A.1 Two interface gaps in current embodied agents … A.5 Episode-level evidence, and its honest limit

AgentCanvas addresses embodied-agent architecture search by representing executors as editable typed graphs with unified structural-edit and batch-evaluation interfaces. It also provides pre-rollout validation and episode-level evidence, while retaining a blind spot for unlogged internal dispatches.

A.1 Two interface gaps in current embodied agents: The unified inference gap requires structural edits to be expressed through a common graph-level interface rather than rewritten separately in idiosyncratic agent code.Examples include moving a stop decision, adding planner voting, or redirecting an observation encoder.
A.1 Two interface gaps in current embodied agents: AgentCanvas replaces method-specific code with a visual node-and-wire canvas and typed-graph substrate exposing one structured-edit surface and one batch-evaluation surface.The fixed forward graph covers perception, memory, planning, and action modules.
A.2 The Executor as an editable typed graph: Each executor is a pure-data GraphDefinition whose typed nodes, edges, state, and access grants fully specify an agent without hidden runtime state.Node types resolve through registries, edges connect named ports, and subgraphs can be wrapped as reusable composites.
A.2 The Executor as an editable typed graph: Legal JSON mutations let the Optimizer add, remove, or retype nodes; rewire edges; tune configuration; edit state, grants, budgets, or termination; and wrap subgraphs.One graph equals one portable, diffable, and loggable agent, making proposals structured patches instead of multi-file rewrites.
A.3 Typed ports and pre-rollout validation: Typed ports gate execution and admission, while connectivity validation rejects structurally invalid proposals in milliseconds before multi-episode GPU rollouts.This converts type-checking from a post-hoc rollout failure into a deterministic pre-flight static check.
A.4 An autonomous, batch-optimized evaluate interface: AgentCanvas evaluates asynchronously with replicated stateful simulators, a shared batchable foundation-model backbone, and time-windowed batching that avoids straggler waiting.One evaluation call returns benchmark metrics and frozen graph and per-episode artifacts, while VRAM admission queues concurrent sessions.
A.5 Episode-level evidence, and its honest limit: Episode logs localize score changes to node firings, but cannot expose internal dispatches when nodes omit optional self-logging.Automatic records include inputs, outputs, timing, errors, model names, and token costs; inner tool or language-model steps may remain invisible.

A.6 Summary: what the substrate asks of the Optimizer

The substrate reduces embodied architecture search to a minimal loop: edit a typed JSON graph, evaluate it on a benchmark split, and inspect per-episode logs. This abstraction removes method-specific execution and evaluation engineering, enabling an Optimizer to run many candidates efficiently with readable evidence.

Substrate contract: The Optimizer’s substrate contract comprises an editable typed JSON graph, one benchmark-scoring evaluation call, and the returned per-episode logs.The loop is simply: edit the graph, issue one evaluate call, and read the result.
Engineering abstraction: The Optimizer avoids maintaining bespoke multi-file implementations, writing simulator glue code, and building method-specific evaluation harnesses.This reduces context spent on executing and scoring candidates when the searched agent is ordinary code with its own runner.
Search consequences: A minimal edit surface, autonomous fast evaluation, and readable per-episode evidence turn embodied AAS from per-method engineering into a runnable search loop.The batched, straggler-avoiding evaluator scores candidates at low wall-clock cost, allowing many evaluations.

B Full Per-Cell Search Trajectories … C.5 Run-directory contract and atomic commit

The full search grid shows directional gains alongside leakage, rerun instability, and failed cells, while the shared coding-agent harness standardizes execution, evaluation, filesystem state, and atomic iteration handling. Its design separates AAS-specific proposal from reusable orchestration and preserves independent per-iteration workspaces and neutral metrics.

B Full Per-Cell Search Trajectories: AFlow × SmartWay had 11 leak-bearing iterations, ADAS × ExploreEQA never beat iter 0, and KDLoop × VoxPoser exited with a substrate bug report and no SR.The headline-selected AFlow iter 11 was leak-bearing, and ExploreEQA ran for 20 iterations without improvement.
B Full Per-Cell Search Trajectories: MapGPT improved for all three optimizers, with AFlow reaching 0.545±0.031 at iter 14, KDLoop 0.540±0.023 at iter 8, and ADAS 0.491±0.032 at iter 9.All three exceeded the 0.469 baseline.
B Full Per-Cell Search Trajectories: SmartWay reruns reduced ADAS from 0.360 to 0.337±0.025 and AFlow from 0.430 to 0.387±0.059, while KDLoop held leak-free at 0.310±0.046.AFlow’s higher selected figure was leak-bearing.
C Coding-Agent Harness as AAS Substrate: The harness is a method-agnostic substrate: swapping an AAS algorithm replaces only the proposer skill while the remaining coding-agent infrastructure is reused.It hosts the AAS family rather than constituting an algorithm itself.
C.1 Pipeline overview: Each iteration uses an orchestrator plus understand, proposer, implementer, and evaluator workers, with the latter two delegating evaluation to the shared /experiment:run runner.Workers run as Markdown-prompt skills within one coding-agent conversation per iteration.
C.2 The loop skill: per-iter orchestration: The loop drives proposer → implementer → evaluator in transient staging, atomically commits only on evaluator success, discards failed iterations, and stops at caps, STOP, or excessive skips.A failed iteration increments consecutive skips and is treated as never having happened.
C.3 Implementer: native editing and the smoke gate: The implementer edits seeded files natively, enforces an edit whitelist, validates JSON and Python, pins model-call nodes to one inference profile, and runs a default five-episode smoke evaluation.The whitelist protects framework and vendored code while warning on out-of-scope graph or nodeset edits.
C.4 Evaluator and the fitness signal: The evaluator runs successful implementations once on a separate profile, typically for 100 episodes, and records run metadata, per-episode accuracy, primary means, and secondary metrics.Variant loops separately convert these neutral records into scalar fitness, means, t-intervals, or bootstrap confidence intervals.

D Variant Implementation Details … E Experiments Setup Detail

The three architecture-search variants share a common substrate but differ in proposer and persistent memory, while KDLoop adds an embodied-specific cycle that exploits richer episode evidence. All experiments use the same harness and per-executor evaluation profile across a 3 × 4 matrix.

D Variant Implementation Details: All three variants share one skeleton and substrate, differing only in proposer skill and memory persisted across iterations.The released implementations live under .claude/commands/architect/ and use a one-line setup followed by a flat loop.
D.1 ADAS: ADAS realizes three Reflexion calls as independent tool-augmented sub-agent spawns sharing one text-rendered message history.Each spawn has full read/grep/shell access and produces a prose change specification; stateless sampling knobs and the smoke-gate trigger are dropped.
D.1 ADAS: ADAS computes bootstrap-confidence-interval fitness in the variant loop at commit and strips the Reflexion-only reflection field before archiving.A zero-scoring embodied run is treated as data, so the gate checks runtime correctness rather than the upstream mean-score trigger.
D.2 AFlow: AFlow selects each iteration’s parent by score-softmax mixing, using α=0.2, λ=0.3, and K=4 from upstream code defaults.Accepted and rejected modifications become avoid-repeating experience; whitespace-normalized duplicates trigger resampling capped at five attempts.
D.3 KDLoop: KDLoop replaces a single propose step with THINK, CRITIC, EXPERIMENT, and DISTILL, plus triggered REFLECT for embodied iterations with richer evidence.THINK emits up to three axis-tagged experiment specs, CRITIC evaluates patched specs, EXPERIMENT applies overlays in parallel, and DISTILL writes lessons to memory.
D.3 KDLoop: KDLoop REFLECT fires on a three-iteration heartbeat, three consecutive iterations sharing one axis, or an empty THINK escalation.It audits search-space coverage, appends coverage to search space.md, and stops when the space is exhausted.
D.3 KDLoop: KDLoop stores typed memory in knowledge.md, hypotheses.jsonl, experience.jsonl, and search space.md, with append and line-delete operations.THINK can add conjectures, experiment designs, and tools, while each iteration polls goal.md and allows up to three intervention-axis specs.
E Experiments Setup Detail: Every cell of the 3 × 4 matrix runs on the same Appendix C harness and per-executor evaluation profile, with variants differing only in proposer and memory.The setup therefore holds execution and evaluation conditions constant across the three variants and four embodied executors.

E.1 Executors and benchmarks

The study evaluates search candidates from published embodied-agent methods using task success rate on each method’s benchmark, with executor-specific secondary metrics logged alongside success. The evaluation covers four executors, including navigation, embodied question answering, and manipulation settings.

Evaluation protocol: Candidates are seeded from published embodied-agent methods and scored by task success rate (SR) on each method’s benchmark.The performance tier is the paper-comparable evaluation set supplying every reported SR.
Evaluation protocol: Navigation additionally logs SPL and (n)DTW, while embodied question answering and manipulation log step count; manipulation also logs cumulative reward.These metrics are secondary to SR and are recorded by executor type.
Benchmarks: The evaluation spans four executors and uses four LIBERO suites—spatial, object, goal, and 10—with ten tasks per suite and five episodes per task for VAS.SR is computed using each benchmark’s own scorer.

E.2 Evaluation tiers … E.5 VoxPoser logging fault

The evaluation uses smoke-gated, paper-comparable performance tiers under fixed model and search settings, with reruns separating selection from variance. A controlled VoxPoser logging fault tests whether optimizers inspect episode-level evidence beyond scalar success rate.

E.2 Evaluation tiers: Each executor has a deterministic smoke tier of 3–8 episodes and a paper-comparable performance tier.Candidates are smoke-gated before performance measurement.
E.2 Evaluation tiers: Smoke evaluation checks runtime correctness: clean exit, at least one step per episode, and a valid metric.Candidates that fail are repaired or skipped, never scored.
E.3 Models and iteration budget: All searched agents use deterministically enforced GPT-5-MINI at temperature 1, except VoxPoser, which runs on GPT-4O.Claude Code supplies the coding-agent session, while Claude Opus 4.7 orchestrates with a 1M-token context.
E.3 Models and iteration budget: ADAS and AFlow run 20 search iterations per cell, whereas KDLoop runs 10, excluding the shared iteration 0 baseline.The search harness remains fixed throughout.
E.4 Baseline and rerun protocol: Each optimizer evaluates the same baseline graph at iteration 0, producing three independent success-rate draws per cell for the Baseline column.Baseline means and sample standard deviations are reported in Table 1.
E.4 Baseline and rerun protocol: Each selected best iteration is re-evaluated three times under the identical performance configuration to separate search-time selection from rerun variance.The reported result is mean±sd with gain ∆ against the baseline.
E.5 VoxPoser logging fault: A controlled VoxPoser fault omits dynamically dispatched LMP sub-calls from the voluntary self-report channel without blocking scalar SR optimization.The fault tests whether optimizers inspect episode-level evidence rather than only scalar outcomes.
E.5 VoxPoser logging fault: KDLoop alone inspects per-episode logs and reports the missing traces; ADAS and AFlow continue optimizing scalar SR despite identical log access.This fault cleanly separates the three optimizers.

F Problem Formulation for Embodied AAS

The formulation defines embodied AAS around a typed, closed-loop graph whose structure, memory, control flow, and environment connections are searchable. Shared rollout evaluation then distinguishes search policies operating over this space using noisy success rates and episode traces.

F Problem Formulation for Embodied AAS: Embodied AAS requires a formulation suited to closed perception–action loops rather than text-agent prompt-and-tool pipelines.The paper argues existing formulations conflate problem definitions with method commitments and do not transfer from text agents to embodied agents.
F.1 AgentCanvas as substrate: the search space: A candidate is a typed, configuration-bearing graph comprising nodes V, shared memory cells Σ, dataflow and memory edges, and per-node configuration γ.Dataflow edges must be type-compatible, while memory cells use typed reducers and lifetimes; configuration stores prompts, sampling parameters, and persist flags.
F.1 AgentCanvas as substrate: the search space: The search space is open and countably infinite, yet type discipline and static validity checks regularize which candidates are well formed.Node types can be authored within a candidate, and validity constrains interactions among nodes, memory, edges, and configuration rather than treating coordinates independently.
F.1 AgentCanvas as substrate: the search space: The graph makes control flow and the simulator ordinary nodes, yielding a self-contained closed loop with no hidden code outside the candidate.The environment node supplies observations and receives actions, while topology represents the candidate’s coarse planning and acting procedure.
F.1 AgentCanvas as substrate: the search space: The search alphabet edits node types, instances, memory cells, dataflow edges, memory edges, and configuration through add, modify, or remove families.Implementation-level changes such as prompt edits or persist-flag changes reduce to SET-CONFIG, while authoring and instantiating node types remain distinct moves.
F.1 AgentCanvas as substrate: the search space: AgentCanvas expresses current workflow-shaped embodied agents through Turing-complete node classes, graph-level loops, branching, and shared state, but not dynamically built deployment-time control flow.The latter is identified as Limitation 1.
F.2 The search policy: ADAS, AFlow, and KDLoop: The three variants share the search space and objective but differ only in search policy and trajectory-memory structure: ADAS and AFlow hill-climb, whereas KDLoop takes a structurally different step.ADAS and AFlow differ in parent selection within their single-proposal archive hill-climb family.
F.3 The evaluation function: Evaluation is shared across variants: batch simulator rollouts over held episode suites return success rate f(c) and logs ℓ for policy memory.Unlike deterministic text-domain scoring, embodied evaluation is expensive and noisy, so static Valid filtering screens malformed candidates before rollout, while traces provide evidence beyond the scalar score.