PaperlayerView exact revision on arXiv

Source-linked AI summary

Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning

Junru Song, Yang Yang, Yaqing Xu, Ying Wen, Wei Peng, Guozhen Li, Wei'en Zhou, Wen Yao

arXiv:2608.23100v1cs.ROcs.AI

TL;DR

Robot co-design leaves open how controller learning reciprocally shapes evolutionary morphology, beyond the established role of morphology in accelerating learning. The paper quantifies morphology through learning speed and performance ceiling, then tests adaptive fitness evaluation on simulated voxel-based soft robots. AdaControl counters premature-evaluation bias, matching exhaustive-control efficiency with up to 80% less computation while discovering diverse high-performing designs.

  • Problem

    Robot co-design has established that morphology can facilitate faster control learning, but how control learning shapes morphological evolution remains unexplored.

  • Method

    The paper estimates morphological intelligence and true potential from control-learning curves and uses population-level monitoring to adapt control-learning budgets during evolution.

  • Results

    AdaControl lets a simple genetic algorithm match exhaustive-control optimization efficiency with up to 80% less computation while discovering more diverse high-performing morphologies.

  • Takeaways & Limitations

    Evaluation fidelity, rather than search sophistication, is identified as the primary bottleneck in robot co-design.

  • Takeaways & Limitations

    Dedicated per-morphology control optimization remains computationally expensive, while AdaControl’s threshold selection involves competing effects that constrain design-space coverage.

Abstract

from arXiv · show

Robot co-design via bi-level optimization couples within-lifetime controller learning for fitness evaluation with cross-generational morphological evolution. Prior work has established that well-adapted morphology facilitates faster control learning, a property termed morphological intelligence. Yet how control learning reciprocally shapes morphological evolution remains unexplored. This paper examines both directions for a holistic account of brain-body interplay. We first show that morphological contributions to control learning decouple into two orthogonal dimensions. We formalize the convergence speed as morphological intelligence and identify the performance ceiling as a complementary quantity termed true potential. A concise functional relation is then established to jointly characterize both quantities from individual learning curves, which, when aggregated at the population level, capture evolutionary profiles. Through extensive experiments on simulated voxel-based soft robots, we reveal that premature fitness evaluation systematically underestimates true potential and biases selection towards fast learners. This restricts design space exploration, compromising both optimization efficiency and morphological diversity. Notably, the widely recognized morphological Baldwin effect emerges as an artifact of this bias rather than a general evolutionary tendency. We therefore propose AdaControl, which monitors disproportionate selection for morphological intelligence during evolution and allocates minimally sufficient control learning for unbiased fitness evaluation. With AdaControl, a simple genetic algorithm rivals state-of-the-art generative-model-based co-design methods in discovering diverse high-performing designs while cutting computation by up to 80% versus exhaustive control.

1. Introduction

Robot co-design couples controller learning with evolutionary morphology, but control-learning choices can shape which morphologies survive. This paper decomposes morphological learning profiles, identifies premature evaluation as an evolutionary bias, and introduces AdaControl to reduce that bias efficiently.

  • Motivation: Bi-level co-design combines inner-loop controller learning for fitness evaluation with outer-loop evolutionary refinement of morphological designs.The paper frames these processes as operating on distinct timescales.
  • Learning-profile framework: Morphological intelligence measures control-learning convergence speed, while true potential measures the morphology’s performance ceiling.Both quantities are estimated from individual learning curves and aggregated to characterize evolutionary behavior.
  • AdaControl: AdaControl monitors disproportionate selection for morphological intelligence and extends control learning until the bias diminishes.It begins with a minimal learning budget and seeks unbiased fitness evaluation without exhaustive training.
  • Results: Up to 80% less computation lets a simple genetic algorithm match exhaustive-control optimization efficiency while finding more diverse high-performing morphologies.AdaControl also rivals co-design methods based on deep generative models in simulated voxel-based soft robots.
  • Evolutionary bias: Premature fitness evaluation biases selection toward fast learners, restricting design-space exploration and compromising optimization efficiency and morphological diversity.The paper identifies this bias as the source of the reported morphological Baldwin effect.

2. Preliminaries

Robot co-design jointly optimizes morphology and dedicated controllers through coupled inner learning and outer evolutionary loops. The paper situates this process in voxel-based soft-robot benchmarks and discusses how control-learning configuration can shape evolutionary outcomes.

  • 2.1. Voxel-Based Soft Robots: Voxel-based soft robots use elastic cubic voxels in grid layouts, with soft-material compliance producing complex robot–environment interactions.These properties create expressive design spaces for studying morphology and control.
  • 2.1. Voxel-Based Soft Robots: Experiments use EvoGym’s five material types and three benchmarks spanning manipulation and locomotion: Carrier, Pusher, and BridgeWalker.Carrier and Pusher transport a box, whereas BridgeWalker traverses deformable terrain.
  • 2.2. Robot Co-design: Bi-level co-design optimizes morphology and sensorimotor controllers, with task performance f(x, c) evaluated over morphology and controller spaces X and C.The inner and outer loops operate on distinct timescales.
  • 2.2.1. Inner loop: The inner loop trains a dedicated controller for each morphology, using cumulative episode reward as morphological fitness.Controllers are multilayer perceptrons mapping observations to actuator signals, and reinforcement learning updates them through environment interactions.
  • 2.2.2. Outer loop: The outer loop ranks morphologies by inner-loop fitness, retains top individuals, mutates voxel designs, and repeats this cycle until the evaluation budget is exhausted.The adopted genetic algorithm maintains and iteratively refines a morphology population.
  • 2.2.2. Outer loop: Per-morphology controller optimization is a major computational bottleneck, while inherited-controller strategies can prematurely converge by favoring first-mover morphologies.The paper therefore treats dedicated per-morphology optimization as the most reliable co-design paradigm for its study.
  • 2.3. Morphological Intelligence: Morphological intelligence broadens morphological-computation research by linking morphology to reduced controller demands, but its quantification remains difficult for learning-based soft robots.The paper proposes extracting learning profiles from control curves, separating learning speed from performance ceiling through morphological intelligence and true potential.

3. Morphological Intelligence and True Potential

The paper separates morphology’s contribution to control learning into convergence speed, termed morphological intelligence, and converged performance, termed true potential. A functional model estimates both quantities from individual learning curves and shows that fast learning need not indicate high eventual performance.

  • Qualitative analysis: At 1000 iterations, a fast-learning morphology outperforms a slower morphology whose substantially higher eventual performance remains invisible to selection.
  • True potential is the upper limit of task performance attainable after control learning fully converges.
  • Morphological intelligence is the convergence speed of control learning toward a morphology’s true potential.
  • In reinforcement learning, true potential corresponds to maximal converged cumulative episodic reward, while morphological intelligence corresponds to learning-curve convergence rate.
  • Quantification: The model uses control complexity C, defined as the number of reinforcement-learning iterations, and represents performance with a saturating hyperbolic tangent.
  • The two quantities show no significant correlation among high-performing morphologies, with Spearman’s ρ = 0.05 and p-value = 0.23.

4. AdaControl

AdaControl adaptively extends control learning when tentative selection disproportionately favors fast learners. It seeks unbiased fitness evaluation while avoiding the computational cost of uniformly prolonged learning.

  • Premature control termination biases selection toward fast learners, restricts design-space exploration, and compromises optimization efficiency and morphological diversity.
  • AdaControl begins each generation with a minimal learning budget and tentatively selects survivors using currently evaluated fitness.
  • The algorithm estimates population and survivor morphological intelligence, then monitors their ratio for disproportionate selection of fast learners.

5. Experimental Study

The experimental study evaluates control-learning bias and AdaControl on simulated voxel-based soft robots across manipulation and locomotion tasks. It compares weak, strong, and adaptively scheduled control under a fixed evaluation budget.

  • Experimental setup: Experiments use 5 × 5 voxel-based soft-robot grids, yielding over 10^17 possible morphologies across Carrier-v0, Pusher-v0, and BridgeWalker-v0.
  • The co-design pipeline uses a genetic algorithm for morphology evolution and proximal policy optimization for controller learning.
  • Control complexity varies across 200, 500, 1000, 2000, and 3000 PPO iterations, with 3000 empirically treated as sufficient for most morphologies’ true potential.
  • All experiments use 1000 robot evaluations, population size 25, five independent trials, and no GPU acceleration.
  • Weak-control and AdaControl results are re-evaluated with strong controllers for fair performance comparison, although this is unnecessary in practical deployment.
  • Research questions: The study tests whether weak control underestimates true potential, biases evolution, harms efficiency and diversity, and is corrected by AdaControl.

5.2. Evaluation Metrics

The study evaluates co-design using morphological intelligence, maximal true potential, and morphological diversity. These metrics separately track learning speed, best converged performance, and the variety of high-performing designs.

  • Morphological intelligence measures control-learning convergence speed and supports analysis of evolutionary bias and AdaControl’s dynamic scheduling.
  • Maximal true potential is the best morphology’s performance after evaluation with strong controllers using 3000 PPO iterations, plotted against computational cost.
  • Morphological diversity is the average pairwise edit distance among morphologies exceeding the top 5% true-potential quantile across experiments.

5.3. Baselines

Weak control distorts fitness evaluation and selection by favoring fast-learning morphologies over designs with higher true potential. Stronger control reveals slower but ultimately superior learners, showing that the apparent morphological Baldwin effect can arise from insufficient evaluation.

  • Control complexity: 3000 PPO iterations provide a ground-truth reference because learning curves converge before this budget.Average convergence requires 2200.78, 2216.28, and 1731.16 iterations for Carrier, Pusher, and BridgeWalker, respectively.
  • Underestimation of true potential: Weak controllers consistently underestimate true potential, with significant discrepancies at 200, 500, and 1000 iterations.The discrepancy narrows as control learning is extended.
  • Selection bias: Weaker controllers select survivors that rank progressively lower when re-sorted by true potential.Lower rankings indicate increasing deviation from true-potential-based selection.
  • Selection bias: Weaker control selects morphologies with steeper initial learning curves and systematically higher morphological intelligence.Strong control preserves slower learners that ultimately achieve higher performance.
  • Interpretation: The morphological Baldwin effect appears as a special case of insufficient control learning rather than a general evolutionary tendency.It arises when evaluation cannot distinguish fast learners from morphologies with greater eventual performance.

5.5. Impact of Evolutionary Bias on Co-Design Performance

Evolutionary bias from weak control compromises co-design by favoring fast learners and restricting exploration. AdaControl adaptively allocates learning where bias is detected, achieving strong efficiency and diversity with substantially lower control cost.

  • Morphological intelligence dynamics: Weak control causes population-average morphological intelligence to rise across generations, while strong control keeps it stable.AdaControl substantially reduces MI growth across all three tasks, with near-complete stabilization in Carrier and Pusher.
  • Optimization efficiency: 65–81%: AdaControl reduces control learning cost relative to strong control while achieving comparable or superior efficiency and diversity.The comparison uses 3000 PPO iterations as strong control.
  • Optimization efficiency: AdaControl achieves performance comparable to strong control across all three tasks while using far fewer PPO iterations per evaluation.Its adaptive allocation concentrates additional learning on generations where selection bias is detected.
  • Baseline comparisons: AdaControl enables a simple genetic algorithm to rival MorphVAE and LASeR without deep generative-model training and querying overhead.The results support population-level control scheduling over individual-level heuristics for addressing selection bias.
  • Morphological diversity: AdaControl achieves diversity above the fixed-complexity trend across all three tasks and exceeds strong-control diversity in Carrier and Pusher.In Carrier and Pusher, it uses less than half the computation of strong control.
  • Overall performance: Fixed-complexity schemes face a tension between computational cost and evolutionary performance that AdaControl addresses through dynamic resource allocation.The method invests computation where evolutionary bias is detected.

5.6. Threshold Selection for AdaControl

AdaControl’s threshold controls a trade-off between correcting fast-learner bias and preserving selection stochasticity. A threshold of 1.1 maximizes diversity for Pusher and BridgeWalker, while BridgeWalker retains moderate MI growth under this balance.

  • Threshold selection: rthr ∈{1.04, 1.07, 1.1, 1.13, 1.16, 1.2}: AdaControl evaluates thresholds by plotting diversity against cumulative PPO iterations.The procedure uses morphological diversity to select the threshold rather than setting it subjectively.
  • Threshold selection: rthr = 1.1: diversity peaks for both Pusher and BridgeWalker.Larger thresholds conserve computation but leave MI bias uncorrected and restrict exploration to fast learners.
  • Task-specific trade-offs: BridgeWalker retains moderate MI growth because its optimal threshold balances MI-bias correction against selection stochasticity.Tightening the threshold does not improve design-space coverage.

5.7. Interpretability Analysis

The interpretability analysis links morphological intelligence to energy efficiency and structural composition in Carrier soft robots. Higher MI corresponds to lower cost of work, while empty-voxel count follows an inverted U-shaped relationship with MI.

  • Energy efficiency: Cost of Work is computed from energy expenditure, robot mass, cumulative reward, and gravitational acceleration.Energy is the absolute sum of actuation signals; mass is the number of non-empty voxels, and gravity is omitted because it is constant.
  • Energy efficiency: Higher morphological intelligence corresponds to lower Cost of Work in Carrier.Cost of Work measures energy consumed per unit mass to accomplish the task.
  • Structural composition: The relationship between morphological intelligence and empty voxels is inverted U-shaped.A moderate number of empty voxels appears to reduce structural constraints, whereas excessive sparsity may produce ineffective interaction dynamics.
  • Interpretation: Energy efficiency and structural composition emerge as physical attributes associated with morphological intelligence in voxel-based soft robots.These attributes indicate which design-space regions are favored by biased evolutionary processes.

6. Conclusion

The paper shows that control-learning configuration shapes morphological evolution by separating morphological intelligence from true potential and exposing bias from premature evaluation. AdaControl adaptively extends learning until selection is less biased, while the findings remain bounded to simulated voxel-based soft robots and three tasks.

  • Prematurely terminated control learning biases morphological selection, and decomposing learning profiles into morphological intelligence and true potential exposes this effect.
  • The findings were established on simulated voxel-based soft robots across three tasks, so generalization to other representations, domains, and physical platforms remains unverified.
  • Task-dependent threshold behavior motivates adaptive threshold mechanisms that self-calibrate during evolution.The paper also identifies network architecture and learning algorithms as additional aspects of control complexity for future investigation.
Loading 2608.23100v1…