PaperlayerView exact revision on arXiv

Source-linked AI summary

LACE-CRAFT: Robot Co-Design with Actor Inheritance and Blackboard Collaboration

Yuhan Wen, Jiawei Wang, Qixuan Zhang, Yusen Qin, Lan Xu

arXiv:2610.09283v1cs.RO

TL;DR

Robot co-design must decide how to use control experience when morphology or reward changes, rather than training each design from scratch. LACE–CRAFT compares continued learning with policy adaptation and evidence-guided proposals; its selected policies have higher three-seed evaluation means than D2C on all five tasks.

  • Problem

    Co-design couples morphology search with policy learning, but training each new design from scratch discards acquired control experience.

    Source: §I · p. 1

  • Method

    LACE compares full-state continuation with actor-initialized challengers and incumbent retention, while CRAFT uses shared records and behavioral replays to guide paired proposals.

    Source: §I · pp. 1–2, §VI · p. 7

  • Results

    The selected LACE–CRAFT policies achieve higher three-seed evaluation means than D2C on all five tasks under the stated protocol.

    Source: §VI · p. 7

  • Takeaways & Limitations

    The generative extension connects editable geometry to articulated models and simulation assets, and the fabricated prototype demonstrates indoor walking with the arm removed.

    Source: §VI · p. 7

    Three evaluation seeds quantify rollout variability of fixed policies, not variation across complete searches or training runs, or errors in evidence interpretation versus proposal generation.

    Source: §V · p. 7

Abstract

from arXiv · show

Robot co-design couples morphology search with policy learning, yet training every new design from scratch discards acquired control experience. We present LACE-CRAFT, which compares continued learning on the current robot with policy adaptation to new morphology-reward pairs. LACE resumes the incumbent's full learning state and initializes compatible challengers with its actor parameters and observation statistics. A fixed task metric selects among both branches and the frozen incumbent. CRAFT coordinates Feedback, Morphology, Reward, and Integration roles through shared experimental records and behavioral replays to generate and cross-review paired proposals. A generative extension converts generated meshes into editable articulated models with configured joints, actuator interfaces, and consistently updated simulation assets. Across five locomotion benchmarks, mean scores over three evaluation seeds are 6.4-91.9% higher than D2C. Both methods train 30 new morphology-reward pairs over five rounds; LACE additionally uses four continuation training units. Five-task ablations examine policy inheritance and replay-derived feedback. A fabricated prototype demonstrates indoor walking and illustrates the geometry-to-hardware workflow.

I. Introduction

LACE–CRAFT addresses how to use learned control experience during robot co-design by comparing continued learning on the incumbent with adaptation to new morphology–reward pairs. It also coordinates evidence-informed proposals and extends the morphology representation to generated geometry, within a broader formulation that separates training reward from task evaluation.

I. Introduction

  • CRAFT links tested morphology–reward pairs to scores, checkpoints, and behavioral replays, then uses specialist feedback, reciprocal review, and integration to form later proposals.Feedback, Morphology, Reward, and Integration roles contribute to proposal generation and cross-review.Source: §I · p. 2
  • The generative extension turns generated shapes into editable links with configured joints and actuator interfaces, updating articulated models and simulation assets while preserving LACE’s policy interface.The paper also documents the geometry-to-hardware workflow and a fabricated walking prototype.Source: §I · p. 2
  • LACE compares full-state continuation with actor-initialized morphology–reward challengers under a fixed task metric, while retaining the frozen incumbent as a selection option.This addresses the choice between further learning on the current robot and adapting its policy to a new design.Source: §I · pp. 1–2

II. Related Work

  • Related work spans morphology–control co-design, controller transfer, language-guided design, agent collaboration, and generated geometry; CRAFT grounds role specialization in experimental evidence.The paper positions its approach alongside prior methods in these areas rather than treating any one as the sole basis for co-design.Source: §II · p. 2

A. Problem Formulation

  • The formulation distinguishes the reward that guides finite-budget learning from the fixed task signal that evaluates the resulting robot across candidates.The evaluation metric differs from training return and uses task-environment root-position coordinates.Source: A. Problem Formulation · pp. 2–3

B. LACE: Learning-State-Aware Continuation Evolution

LACE trains a continuation candidate on the incumbent and morphology–reward challengers initialized within a defined transfer boundary. It evaluates the frozen incumbent and new candidates together, retaining the best under a fixed ordering.

LACE: Learning-State-Aware Continuation Evolution

  • Each iteration trains a continuation candidate and design challengers; initialization preserves kinematic connectivity, actuated-joint ordering, and observation and action dimensions and meanings.Direct actor inheritance additionally requires the same policy-network architecture; topology or observation/action semantic changes require another transfer mechanism.Source: B. LACE: Learning-State-Aware… · p. 3
  • The continuation branch resumes the incumbent’s training state on its current morphology and reward for another fixed transition budget.The continuation candidate preserves the incumbent’s morphology, reward, and learner configuration.Source: B. LACE: Learning-State-Aware… · p. 3
  • Challengers inherit the incumbent’s actor parameters and observation statistics, while other learner components are initialized anew and adapt to the proposed dynamics and reward.Each challenger starts from an independent copy of the template; its actor and normalizer continue learning.Source: B. LACE: Learning-State-Aware… · p. 3
  • The comparison set includes newly evaluated candidates and the frozen incumbent; the highest-scoring state is selected, with ties retaining the incumbent or following the fixed candidate order.Incumbent retention makes the cached best score nondecreasing, but does not guarantee improved expected performance on fresh rollouts.Source: B. LACE: Learning-State-Aware… · p. 3

C. CRAFT: Blackboard-Based Collaboration

CRAFT uses shared experiment records and behavioral replays to ground specialist feedback, cross-reviewed morphology and reward proposals, and integration into paired candidates. After evaluation, records and replay evidence inform the next round.

C. CRAFT: Blackboard-Based Collaboration

  • The shared blackboard links evaluated states and scores to their design records, while selection flags and identifiers connect results to the corresponding morphology, reward, and checkpoint.Table II summarizes shared CRAFT evidence and role-specific outputs.Source: C. CRAFT: Blackboard-Based Collaboration · p. 3, Table II · p. 4
  • Roles use the latest five selection records by default, alongside current replay evidence; the complete discussion archive is not automatically included in their inputs.Diagnostic replay is distinct from the SAC replay buffer used for training.Source: C. CRAFT: Blackboard-Based Collaboration · pp. 4–5
  • Feedback uses selected-robot replays and task measurements to diagnose behavior and guide morphology–reward proposals.The roles receive a bounded view of experiment history together with current replay evidence and measurements.Source: C. CRAFT: Blackboard-Based Collaboration · pp. 3–4
  • Morphology and Reward draft independently, cross-review each other’s proposals, and pass revised drafts to Integration to form paired candidates.Integration records joint rationales alongside the candidate pairs.Source: C. CRAFT: Blackboard-Based Collaboration · p. 4
  • After selection, deterministic inference on the chosen robot generates the next diagnostic replay and task measurements for the updated blackboard.The round archive stores requests, responses, and rationales linked to candidates and rounds.Source: C. CRAFT: Blackboard-Based Collaboration · p. 5

D. Extension to Generative Robots

The generative extension turns a generated shape and hardware requirements into editable articulated morphologies and simulation assets. Geometry edits preserve the policy interface, while hardware details and fastening still require configuration or manual checking.

D. Extension to Generative Robots

  • A shape reference and hardware requirements specify the generated morphology, including scale, orientation, joint topology, and motor constraints.Rodin generates the static mesh; the task defines the evaluation objective.Source: D. Extension to Generative Robots · p. 5
  • BANG decomposes the mesh into parts that are aligned and assigned to an articulated kinematic tree with link and joint properties.The illustrated geometry model has two actuated joints per leg; the hardware-adapted quadruped used for walking has three.Source: D. Extension to Generative Robots · p. 5
  • Scripts add configured motor cavities and interfaces, then export collision, inertial, joint-limit, and action-mapping assets for locomotion training.For the arm-equipped robot, policy actions control leg joints while prescribed arm postures remain fixed during training.Source: D. Extension to Generative Robots · p. 5
  • Geometry edits refit affected joint origins and motor slots and regenerate simulation assets while keeping hardware, connectivity, and the policy interface fixed.Fastening details require renewed manual checking after geometry changes.Source: D. Extension to Generative Robots · p. 5

IV. Experiments

Across five locomotion tasks, LACE–CRAFT’s final policies have higher three-seed evaluation means than D2C under the stated benchmark protocol. The experiments compare the methods’ training and evaluation setups and report a historical Ant search trajectory.

A. Experimental Setup

  • Both methods generate 30 new morphology–reward pairs over five rounds, while LACE additionally trains one continuation candidate in each of four later rounds.Final evaluation uses seeds 0, 1, and 2 with 128 parallel episodes per seed; the reported variability is across fixed-policy evaluation seeds, not training seeds or independent searches.Source: A. Experimental Setup · pp. 5–6

B. Overall Performance against D2C

  • 91.9% on Ant, 36.8% on HalfCheetah, 15.0% on Hopper, 46.7% on Swimmer, and 6.4% on Walker2D are LACE–CRAFT’s gains over D2C.These are three-seed evaluation mean comparisons; Figure 3 normalizes means by each task’s fixed D2C mean, and Table III reports raw means and standard deviations.Source: Figure 3 · p. 6, Table III · p. 6, B. Overall Performance against D2C · p. 6
  • In the reported Ant search, a challenger wins round 2, continuation wins rounds 3 and 4, and the incumbent is retained in round 5.The historical selection scores are distinct from the post-selection three-seed means.Source: B. Overall Performance against D2C · p. 6

C. Ablation Study

The ablations compare policy inheritance and replay-derived behavioral feedback across five tasks, while the generative demonstration and limitations describe the hardware workflow and its current boundaries. The conclusion identifies diagnostic reliability, mechanical validation, and quantitative sim-to-real evaluation as future work.

C. Ablation Study

  • 44.0% and 59.1%: No Actor reaches these shares of the full-method mean on Swimmer and Walker2D, while the full method scores higher on all five tasks.The full-method advantage is 8.0–15.7% on Ant, HalfCheetah, and Hopper; the comparisons bundle transferred actor parameters and observation statistics.Source: C. Ablation Study · p. 6, Figure 4 · p. 6
  • 49.6% and 93.9%: No replay reaches these shares of the full-method mean on Swimmer and Hopper, respectively; the full method scores higher on all five tasks.The no-replay condition is closer to the full method on Hopper than Swimmer for these selected policies.Source: Figure 4 · p. 6, C. Ablation Study · p. 7

D. Generative Robot Demonstration

V. Limitations and Future Work

  • Three evaluation seeds measure rollout variability of fixed policies, not variation across complete searches or training runs.They also do not disentangle errors in evidence interpretation from variation in proposal generation; replay-based diagnoses should be corroborated by task measurements.Source: §V · p. 7
  • Fabrication requires manual screw-hole and fastening design; the printed parts had no structural simulation, and thin connections were prone to fracture.Structural analysis under expected loads and durability testing are identified as needed to improve mechanical reliability.Source: §V · p. 7, D. Generative Robot Demonstration · p. 7
  • The simulated and fabricated models differ, and incorporating hardware video and measured joint motions into CRAFT is proposed for future design revisions.Quantitative transfer from the co-design search and walking with the arm remain to be evaluated.Source: §V · p. 7, D. Generative Robot Demonstration · p. 7

VI. Conclusion

  • The conclusion identifies diagnostic reliability, mechanical validation, and quantitative sim-to-real evaluation as future work.Source: §VI · p. 7

Appendix A Training-State Transfer

LACE compares an unchanged incumbent, a full-state continuation, and challengers initialized with the incumbent’s actor and observation statistics. The appendix specifies the learner state and policy-interface conditions for these transfers, alongside the order of CRAFT role calls.

A. Continuation, inheritance, and incumbent retention

  • LACE compares a frozen incumbent, full-state continuation on its morphology and reward, and challengers initialized from the incumbent’s Actor and observation statistics.The incumbent retains its cached score; ties favor the incumbent, then continuation, then ordered challengers, so retained evaluation scores are nondecreasing along the search.Source: A. Continuation, inheritance, and… · p. 9, Table S1 · p. 9

B. PPO continuation state

  • PPO continuation restores the serialized learner state, while a challenger copies the Actor and observation statistics but resets its value function, optimizer, collection state, and counters.The challenger collects trajectories on its new body, so value estimates are learned under the proposed dynamics and training reward.Source: B. PPO continuation state · p. 9

C. SAC continuation state

  • SAC continuation restores critics, targets, entropy state, and replay data with the Actor; a challenger starts these components anew with an empty replay buffer.Its critic updates therefore use transitions collected under its own morphology and reward.Source: C. SAC continuation state · p. 9

D. Policy-interface compatibility

  • Direct Actor transfer requires compatible task interfaces; benchmark adapters preserve observation and action dimensions and actuated-joint ordering.All six challengers receive the same Actor and normalizer transfer; Conservative, Medium, and Aggressive are design-risk groups, not transfer coefficients.Source: D. Policy-interface compatibility · p. 9

A. Role requests and execution order

  • CRAFT’s round order is Feedback first, parallel Morphology and Reward drafts, parallel cross-reviews, then Integration.A round without retries uses six role calls: one Feedback call, two drafts, two reviews, and one Integration call.Source: A. Role requests and execution order · pp. 9–10

B. Output contracts

CRAFT assigns distinct proposal and review contracts to Feedback, Morphology, Reward, and Integration, then checks and records complete candidates. Its blackboard links experiment outcomes, robot assets, and replay evidence to the proposals informed by them.

B. Output contracts

  • Feedback combines replay frames and task measurements to propose testable changes; morphology and reward roles draft complete proposals and cross-review each other’s work.Behavioral explanations about balance, contact, or failure remain hypotheses when those quantities are not directly measured.Source: B. Output contracts · p. 10
  • Integration emits six complete morphology–reward pairs across three design-risk groups, and validation checks identifiers, risk groups, renderable geometry, and the reward interface.Rejected responses are stored with validation errors, and the role pipeline permits up to three stage-level attempts before accepted candidates are trained and externally evaluated.Source: B. Output contracts · p. 10

C. Blackboard and archive contents

  • The blackboard combines experiment records with selected-checkpoint diagnostics, while a separate archive stores full role discussions without automatically expanding later role inputs.After selection, comparison results are appended and a diagnostic rollout produces evidence for the next round.Source: C. Blackboard and archive contents · p. 10, Table S2 · p. 10

D. Linking results, proposals, and replay

  • Selection records link each candidate and score to its checkpoint, morphology, and reward; replay manifests link behavior summaries and saved replays to the selected checkpoint and assets.These links associate measured behavior with the robot that produced it and the proposals generated in that discussion.Source: D. Linking results, proposals, and… · p. 10

A. Selection metric and diagnostic measurements

This section specifies the task metric used for candidate selection, replay diagnostics, learner configurations, baseline evaluations, and morphology constraints. Together, these details define how the five-task experiments were configured and interpreted.

A. Selection metric and diagnostic measurements

  • Selection uses episode_distance_from_origin, the episode-summed root-position norm averaged across evaluation episodes rather than path length, signed forward displacement, or training return.The task coordinates are listed in Table S7.Source: A. Selection metric and diagnostic… · p. 11
  • The deterministic diagnostic replay saves pipeline-state frames and records steps, cumulative reward, root-x displacement, and a finiteness flag; finiteness does not establish balance or task success.Feedback uses replay video and numerical measurements to produce diagnoses for the design roles.Source: A. Selection metric and diagnostic… · p. 11

B. Learner settings

  • PPO and SAC use task-specific configurations, while all tasks use observation normalization, action repeat one, and 128 parallel evaluation environments.Requested transition budgets are aligned to complete learner batches; the configurations are listed in Tables S3–S5.Source: B. Learner settings · p. 11, Table S3 · p. 11, Table S4 · p. 11

C. Networks and software

  • PPO uses four 32-unit actor layers and five 256-unit value layers; SAC uses 256-unit actor and critic layers, with the listed software stack applying to the five-task benchmark.The generated-robot experiments instead use the Isaac Gym adapter.Source: C. Networks and software · p. 11

D. Default baseline and reported checkpoint scores

  • Default trains each task’s original morphology and hand-designed reward from scratch, while the main comparison reports frozen-policy evaluations across three seeds rather than repeated full searches.The historical Walker2D score of 10,125.74 is a stored seed-2 checkpoint score, not the main-text three-seed evaluation mean.Source: D. Default baseline and reported… · pp. 11–12

E. Morphology parameter bounds

  • Morphology vectors contain 6–24 scalars across tasks, with task-specific bounds and additional geometric checks to reject invalid bodies.The indexed scalars describe morphology rather than policy parameters; coordinates and radii are in meters.Source: E. Morphology parameter bounds · p. 12, Table S6 · p. 12
  • Walker2D morphology bounds also enforce ordered vertical anchors, minimum segment lengths, radius-to-length limits, and nonnegative foot clearance.The torso-center constraint is 0.8 < p1 < 2.Source: E. Morphology parameter bounds · p. 12

F. Model calls and failed candidates

The appendix documents model-call and failure handling, training-unit accounting, initialization, and Ant search variability. Its reported search variability is based on three Full searches with one fixed RL seed, not a matched repeated-search comparison with D2C.

F. Model calls and failed candidates

  • Role calls allow three attempts per role, and invalid candidates retain error records; failed training or non-finite evaluation leaves the round unselected rather than assigning a synthetic zero.Completed checkpoint-bound results are preserved on resumption.Source: F. Model calls and failed candidates · p. 12

G. Design counts and total training

  • Both methods generate 30 designs over five rounds, while LACE uses four additional continuation units, for 34 completed training units in total.Incumbent retention uses a cached score and adds no training unit.Source: G. Design counts and total training · p. 12

H. Five-task initialization configurations

  • Each search starts with six fresh designs; later challengers inherit the incumbent Actor and observation statistics, while continuation restores the full learner state.The Conservative, Medium, and Aggressive labels indicate proposal risk, not transfer coefficients.Source: H. Five-task initialization… · p. 12, Table S7 · p. 13

I. Search-level variability and normalized visualization

  • 59,024.51 ± 4,671.80 is the mean and sample standard deviation across three complete Full Ant searches with RL seed fixed at zero.The latest search supplies the Full checkpoint evaluated in the main comparison; these are not confidence limits.Source: §I · p. 13
  • The three Full searches are not a matched repeated-search comparison with D2C; checkpoint rollouts measure one policy, not independent proposal–training–selection searches.The main text separately reports selected-policy evaluations over three evaluation seeds.Source: §I · p. 13

Appendix D Cross-Iteration Evolution

The appendix traces round-wise selection outcomes and defines the ablation comparisons. Walker2D new-design maxima exceed D2C’s in each round, while the Ant records show both challenger and continuation wins; the ablations do not isolate every component or search-level variability.

A. New-design performance across rounds

  • LACE–CRAFT produces a higher-scoring new design in all five Walker2D rounds, although its round-wise maximum dips in round four; incumbent retention prevents the selected score from decreasing.The final selected score is 18,483.50 because selection also admits continuation.Source: A. New-design performance across rounds · p. 13, Table S9 · p. 13

B. Continuation and challenger selection on Ant

  • In the Ant search, the round-2 challenger scores 54,646.39, then continuation wins rounds 3 and 4 with 60,870.41 and 64,252.91.The challenger improves the incoming incumbent by 35.2%; continuation adds 11.4% and 5.6% in rounds 3 and 4, respectively.Source: B. Continuation and challenger… · p. 13, Figure S1 · p. 14

A. Variant definitions

  • The ablations retain all four CRAFT roles and cross-review, comparing Full with No Actor inheritance and No replay evidence under the same five-round, 34-unit schedule.No Actor uses fresh Actor and observation-normalization statistics; No replay removes replay-derived evidence but retains scores, rankings, and design history.Source: A. Variant definitions · pp. 13–14, Table S10 · p. 14

B. Ant round-wise outcomes

C. Scope of the comparisons

  • The No Actor contrast jointly changes Actor and observation-statistics inheritance, and the replay comparison does not isolate the collaboration structure itself.The historical ablation scores are individual search outcomes without variability estimates across independent search seeds.Source: C. Scope of the comparisons · pp. 14–15

Appendix F Generative Geometry and Hardware Interfaces

The appendix describes mesh-to-articulated-model processing and the dependencies needed to keep geometry, joints, and policy interfaces consistent. It also reports hardware deployment details and boundaries, alongside separate single-search Ant progress records.

A. Artifacts at each processing stage

  • The generative workflow separates generated surfaces into body and limb meshes, repairs and aligns link geometry, fits motor cavities and mating interfaces, then compiles the articulated model.The processing stages and their resulting artifacts are catalogued in Table S12.Source: A. Artifacts at each processing stage · p. 15, Table S12 · p. 15

B. Dependencies of a morphology edit

  • Morphology edits require joint origins, motor placement, collision geometry, masses, centers of mass, and inertia tensors to remain consistent with revised link geometry.Direct Actor inheritance also requires fixed observation and action dimensions and consistent actuated-joint ordering.Source: B. Dependencies of a morphology edit · p. 15

C. Raspberry Pi controller and physical recording

  • The R17 policy runs locally on a Raspberry Pi at 20 Hz using a 58-dimensional observation and 12 actions; the recorded indoor trial documents motion, not calibrated displacement or benchmark scores.The hardware-specific controller is distinct from the five Brax benchmark policies; its deployment settings include observation normalization and action filtering.Source: C. Raspberry Pi controller and… · pp. 15–16

D. Manipulation interface

Appendix G Single-search Generation Progress

  • The Ant progress figure tracks selected checkpoint scores after 6, 12, 18, 24, and 30 generation units for one complete search per curve.The Full curve’s stored selection score is distinct from the main tables’ three-seed evaluation of the selected policy.Source: Appendix G · p. 16, Figure S3 · p. 16
Loading 2610.09283v1…