PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalable Co-Optimization of Morphology and Control in Embodied Machines

Nick Cheney, Josh Bongard, Vytas SunSpiral, Hod Lipson

arXiv:1706.06133v2cs.AI

TL;DR

Jointly optimizing robot morphology and control has seen limited success, in part because body and controller changes can be difficult to coordinate. The paper proposes morphological innovation protection, which gives controllers time to readapt after morphological changes, and reports improved optimization with reduced premature convergence and convergence to similar morphologies across trials.

  • Problem

    Co-optimizing robot morphology and neural controllers has seen very limited success compared with optimizing either subsystem separately.

    Source: §I · p. 1

  • Method

    Morphological innovation protection tracks time since morphological changes and uses age alongside fitness to reduce selection pressure while controllers readapt.

    Source: C. Proposed Method: Morphological… · pp. 2–3

  • Results

    The method supports continued optimization, better resulting fitness, reduced premature convergence, and convergence to similar morphologies across independent trials from random initial conditions.

    Source: §V · pp. 7–8, §VI · p. 9

  • Takeaways & Limitations

    The authors present morphology protection as a short-term improvement for co-optimizing morphology and control in virtual creatures, pending future approaches with body–brain feedback loops.

    Source: §VI · p. 9

    The fitness increase does not conclusively establish the proposed embodied-cognition explanation, because protection also adds population diversity and search dimensionality.

    Source: A. The Effect of Morphological… · p. 4

Abstract

from arXiv · show

Evolution sculpts both the body plans and nervous systems of agents together over time. In contrast, in AI and robotics, a robot's body plan is usually designed by hand, and control policies are then optimized for that fixed design. The task of simultaneously co-optimizing the morphology and controller of an embodied robot has remained a challenge. In psychology, the theory of embodied cognition posits that behavior arises from a close coupling between body plan and sensorimotor control, which suggests why co-optimizing these two subsystems is so difficult: most evolutionary changes to morphology tend to adversely impact sensorimotor control, leading to an overall decrease in behavioral performance. Here, we further examine this hypothesis and demonstrate a technique for "morphological innovation protection", which temporarily reduces selection pressure on recently morphologically-changed individuals, thus enabling evolution some time to "readapt" to the new morphology with subsequent control policy mutations. We show the potential for this method to avoid local optima and converge to similar highly fit morphologies across widely varying initial conditions, while sustaining fitness improvements further into optimization. While this technique is admittedly only the first of many steps that must be taken to achieve scalable optimization of embodied machines, we hope that theoretical insight into the cause of evolutionary stagnation in current methods will help to enable the automation of robot design and behavioral training -- while simultaneously providing a testbed to investigate the theory of embodied cognition.

I. INTRODUCTION

Evolutionary design can search robot bodies and behaviors without copying a particular organism, but jointly optimizing morphology and control has had limited success. Prior work describes small evolved designs and persistent limits in scale, motivating investigation of how body and brain interact during optimization.

I. INTRODUCTION

  • Evolutionary design can avoid separating biologically niche-specific traits from useful design features and can produce machines unlike existing animals.It can also provide a controlled, repeatable test-bed for studying evolutionary, developmental, or behavioral theory.Source: §I · p. 1
  • Joint optimization of robot controllers and body plans has seen very limited success, despite progress optimizing either subsystem separately.Controller optimization has succeeded for fixed morphologies, while morphology optimization has had lesser success with fixed, minimal controllers.Source: §I · p. 1
  • Embodied-cognition accounts propose that body and brain specialize to complement one another, making changes to either subsystem difficult without corresponding changes to the other.This specialization is proposed as a source of fragility under design perturbations.Source: §I · p. 1

II. BACKGROUND

  • Surveys of evolved robots have not shown significant increases in size or complexity beyond early work, despite increased computational power and proposed methods to improve scalability.The passage notes that Sims’ evolved robots averaged 6.042 segments per robot.Source: §II · p. 2

III. METHODS

The section frames co-evolution as a problem of fragile body–controller coupling: a morphology change can initially impair behavior even when the altered body could perform better with an adapted controller. It proposes controller readaptation after morphological changes as a way to assess their potential.

A. The Interdependency of Body and Brain

  • Specialization between a robot’s body plan and controller can create local optima because body changes may be rejected before the controller adapts.Traditional evolutionary evaluation judges a new variation immediately, without readaptation, and uses that fitness to assess its long-term potential.Source: A. The Interdependency of Body and Brain · p. 2
  • A longer-legged robot could take longer strides, yet its unchanged controller may apply insufficient force and disrupt coordinated walking.This illustrates how an initially detrimental evaluation can obscure a morphology’s potential.Source: A. The Interdependency of Body and Brain · p. 2
  • A new morphology may outperform its predecessor when paired with a suitable controller, so the proposed approach allows the controller to readapt after a body change.The foundation is to readapt each controller until the proposed morphology is fitter than its predecessor.Source: A. The Interdependency of Body and Brain · p. 2

B. Controller Readaptation

The method protects newly changed morphologies while their controllers readapt, avoiding a fixed protection duration by tracking morphological age alongside fitness. The reported evolutionary setup uses a (µ, λ)-strategy with specified population and mutation settings.

B. Controller Readaptation

  • A fixed readaptation period is difficult to choose because the required time depends on robot complexity, encoding, task, and current ability, making per-scenario parameter sweeps intractable.The passage notes that ability changes during optimization.Source: B. Controller Readaptation · p. 2

C. Proposed Method: Morphological Innovation Protection

  • The method tracks generations since each morphological mutation and removes an individual when a fitter, younger individual dominates it on age and fitness.This avoids setting a readaptation duration in advance; “morphological age” is the number of generations since the mutation.Source: C. Proposed Method: Morphological… · p. 2
  • Morphological innovation protection reduces selection pressure on new morphologies, allowing poorly adapted controllers time to improve while comparing variants with equal readaptation time.The age-based Pareto approach avoids extended control re-optimization for non-promising morphologies.Source: C. Proposed Method: Morphological… · p. 3

D. Evolutionary Algorithm

  • Trials used a (µ, λ)-evolutionary strategy with 25 parents and 25 mutants for 5,000 generations; each mutation changed morphology or controller, but not both.Crossover was not considered; preliminary mutation-ratio trials without protection did not significantly affect premature convergence or resulting fitness.Source: D. Evolutionary Algorithm · p. 3

E. Genetic Encoding for Soft Robot Morphologies

The robots use CPPNs to encode voxel-based morphologies and controllers, with phenotype resolution determining the number of voxels. Per-muscle-cell control uses phase offsets and a shared oscillator frequency to generate varied coordinated behaviors.

E. Genetic Encoding for Soft Robot Morphologies

  • A CPPN maps voxel coordinates to cell presence and muscle-or-passive-tissue type, with nearby coordinates tending to produce similar outputs and continuous tissue patches.Its varied activation functions can produce regular geometric patterns such as symmetry or repetition.Source: E. Genetic Encoding for Soft Robot… · p. 3

F. Soft Robot Resolution

  • The continuous CPPN morphology encoding can be discretized at different resolutions, with 5 × 5 × 5 used by default and 10 × 10 × 10 for higher-resolution robots.Higher resolution requires more computation, and distance values are in absolute voxel counts.Source: F. Soft Robot Resolution · p. 3

G. Controllers and their Genetic Encoding

  • Each muscle cell has a controller that sets its oscillation phase offset from a global sinusoidal oscillator and contributes a local value used to calculate the global frequency.Controller outputs range from −1 to 1 and change muscle-cell dimensions; passive tissue remains at its original size, though it deforms with intrinsic compliance.Source: G. Controllers and their Genetic… · p. 3
  • The controller encoding can produce synchronized or offset muscle patches, propagating excitation waves, and morphology-specific oscillation speeds.The global frequency can be tuned to morphology properties such as soft-tissue resonance in appendages.Source: G. Controllers and their Genetic… · p. 3
  • Separate CPPNs encode morphology and control, allowing isolated subsystem changes so controller readaptation can proceed without altering morphology..Source: G. Controllers and their Genetic… · p. 3

H. Physics Simulation for Evaluation

The evaluation uses simulated locomotion distance, while morphological innovation protection tracks time since a body-plan mutation to compare individuals with different amounts of controller readaptation. Statistical comparisons use 30 trials and Wilcoxon rank-sum tests.

H. Physics Simulation for Evaluation

  • Robots are evaluated by locomotion distance in VoxCad simulations lasting 20 actuation cycles.Cycle duration may vary with the globally optimized frequency; the authors use cycles rather than elapsed simulation time to normalize the number of steps.Source: H. Physics Simulation for Evaluation · p. 4

I. Morphological Innovation Protection

  • Morphological age resets to zero after a morphological mutation, tracking successive controller mutations without a body-plan change.The age objective compares individuals with similar amounts of controller readaptation; the approach assumes unique morphologies correspond to peaks in the landscape.Source: §I · p. 4
  • Age-zero individuals’ fitness significantly increased from 18.715 at generation 0 to 22.916 at generation 5000, unlike random individuals in standard age-fitness optimization.The linear regression on age-zero individuals from Fig. 2 found p < 0.001.Source: §I · p. 4

J. Statistical Analysis

  • All treatments used 30 independent trials with consistent random seeds, and significance was tested using Wilcoxon rank-sum tests.Plots show means across the most fit individual from each trial, with 95% bootstrapped confidence intervals.Source: J. Statistical Analysis · p. 4

A. The Effect of Morphological Innovation Protection on Fitness

Morphological innovation protection improves locomotion fitness over greedy selection without protection, and its gains continue later in optimization. A controller-protection comparison does not show the same advantage, though the authors caution that the fitness increase alone does not establish their proposed explanation.

A. The Effect of Morphological Innovation Protection on Fitness

  • 31.953 voxels (95% CI 28.157–36.511) with morphological innovation protection versus 21.717 voxels (95% CI 19.457–22.426) without protection; the difference is significant (p = 6.067 ∗10−6).Protection allows individuals to be out-competed only by those with equal or less controller readaptation to their current morphologies.Source: A. The Effect of Morphological… · p. 4
  • The fitness increase supports morphological protection as a more effective optimization method but does not conclusively establish the proposed explanation that morphology encodes the controller.The authors note that added population diversity and the extra age objective are confounds.Source: A. The Effect of Morphological… · p. 4
  • Controller innovation protection averaged 22.049 voxels, did not significantly improve on no protection (p = 0.240), and performed significantly worse than morphological protection (p = 1.211 ∗10−4).This comparison includes the added diversity and multidimensional search effects of protection, so it does not isolate the proposed morphological-language explanation.Source: A. The Effect of Morphological… · p. 4
  • From generation 1000 to 5000, protected fitness rose significantly from 25.925 to 31.953 voxels (p = 0.013), while unprotected fitness rose from 20.988 to 21.717 voxels without significant improvement (p = 0.085).The plotted trajectories show mean fitness over optimization time with 95% bootstrapped confidence intervals.Source: A. The Effect of Morphological… · pp. 4–5, Figure 1 · p. 5

B. The Effect of Morphological Innovation Protection on Population Stagnation

Morphological innovation protection is associated with continued improvement and turnover among the best-so-far body plans, including cases where initially weaker child morphologies later overtake their parents. Without protection, the best individual typically plateaus earlier.

B. The Effect of Morphological Innovation Protection on Population Stagnation

  • The best individual continued improving with protection, whereas without protection the best was found before generation 2000 and fitness had reached 99.6% of its final value by generation 1000.The protected trial’s continued improvement is consistent with the average trend reported in Fig. 1.Source: B. The Effect of Morphological… · p. 5
  • Protected runs averaged 24.179 unique best-so-far morphologies, compared with 10.115 without protection (p = 1.555 ∗10−6).The best individual changed rapidly in the protected trial, with many different morphologies holding the best-so-far position.Source: B. The Effect of Morphological… · p. 5
  • Protection retains a child morphology even when its inherited controller is poorly adapted and its initial fitness is below its parent’s.During the interim, controller optimization proceeds in equal amounts for parent and child; the child can later overtake the parent and the parent is then removed.Source: B. The Effect of Morphological… · p. 5
  • 76.714 morphological overtakes occurred on average with morphological innovation protection, versus 1.432 without protection and 1.333 with controller protection (p ≤6.939 ∗10−10).An overtake occurs when a child starts with worse fitness than its parent but later outperforms it.Source: B. The Effect of Morphological… · p. 5
  • Morphological overtakes were not significantly different in number from controller overtakes under controller protection (p = 0.533), although morphological protection outperformed the other treatments.The authors suggest this contrast points to a greater potential role for morphological than controller overtakes in optimization.Source: B. The Effect of Morphological… · p. 5
  • In the Fig. 2 example, the red child initially underperforms the teal parent but later surpasses it after successive control optimization.Unique morphologies are shown in different colors.Source: Figure 2 · p. 6

C. The Effects of Morphological Innovation Protection on the Progression of Morphologies over Evolutionary Time

With morphological innovation protection, the best morphologies continue changing and improving, and some runs converge on an identical morphology despite different starting points. Without protection, gross morphological changes largely stop early, consistent with premature convergence to early designs.

C. The Effects of Morphological Innovation Protection on the Progression of Morphologies over Evolutionary Time

  • With protection, some runs reached identical final morphologies despite starting differently and finding that form only hundreds or thousands of generations into optimization.The authors interpret the contrast with unprotected runs as suggesting that protection may help escape local optima.Source: C. The Effects of Morphological… · p. 6
  • Without protection, gross morphological changes tended to stop after generation 50, and early cube-like designs often remained the final forms.These gross forms were often already present among the random initial morphologies.Source: C. The Effects of Morphological… · p. 6
  • The authors suggest that without morphological innovation protection, search tends to converge prematurely to early morphologies, whereas protection may help escape local optima and reach “more global” optima.In an unprotected trial with the same initial conditions as Fig. 2, new morphologies that initially performed worse than the current best were rejected and removed from the population.Source: C. The Effects of Morphological… · p. 6, Figure 3 · p. 6
  • Without protection, new morphologies that initially performed worse than the best individual were rejected, whereas protected morphologies could later overtake it.Fig. 3 shows the rejected variants as colored dots beneath the best-performing individual.Source: Figure 3 · p. 6

D. The Potential for Convergence Across Initial Conditions

In the higher-resolution soft-robot setting, morphological innovation protection improved travel over no protection and controller protection; adding a morphology-change threshold produced the strongest reported result in more difficult trials. The authors also report convergence across varied initial conditions with thresholded protection, while noting that the threshold is specific to this voxel-based implementation.

D. The Potential for Convergence Across Initial Conditions

  • As voxel counts rise, a single-voxel change has less effect, making it harder to distinguish functional morphology changes from minor detail changes.The low-resolution implementation did not benefit from a threshold, presumably because its mutations were already large enough to constitute meaningful morphological changes.Source: D. The Potential for Convergence… · pp. 6–7
  • For higher-resolution robots, resetting morphological age only after changes to more than 20% of voxels produced optimal results in the parameter sweep.The threshold parameter is specific to this voxel-based soft-robot implementation, although the general concept of a minimum-change threshold could apply to other robot instantiations.Source: D. The Potential for Convergence… · p. 6
  • A threshold on controller changes was not used because its benefit fell as the threshold increased, with optimal performance at no threshold.Source: D. The Potential for Convergence… · p. 7
  • 60.728 voxels: basic morphological innovation protection out-traveled no protection (32.575, p = 1.208 ∗10−4) and controller protection (35.375, p = 0.018).Thresholded trials had higher average fitness than unthresholded morphological protection (43.340 voxels), but the difference was not significant (p = 0.063).Source: D. The Potential for Convergence… · p. 7
  • 31.313 voxels: in more difficult closed-loop-controller trials, thresholded morphological protection outperformed no protection (13.941), controller protection (15.356), and unthresholded morphological protection (17.224; all p ≤7.287 ∗10−8).Source: D. The Potential for Convergence… · p. 7
  • With a 20% morphology-change threshold, runs converged on the same high-performing morphology across varied initial conditions, while no-protection search stagnated near its starting conditions.The authors describe this as suggesting escape from local optima in this particular soft-robot implementation.Source: D. The Potential for Convergence… · p. 7

V. DISCUSSION

Morphological innovation protection is presented as a way to sustain brain–body co-optimization by giving controllers time to readapt to new morphologies. The authors report improved optimization and convergence across trials, while emphasizing limits in the tested robot class and open questions about how morphology and control are divided.

V. DISCUSSION

  • The method supports longer continued optimization and better resulting fitness than traditional evolutionary methods, helping prevent premature convergence in the rugged search space of robot morphologies and controllers.The authors frame morphological innovation protection as diversity maintenance that gives solutions with long-term potential time to overcome poor immediate fitness.Source: §V · pp. 7–8
  • With a 20% morphological-change threshold, the top 10 of 30 trials converge to the same final morphology, while unprotected search stagnates near its initial local optima.The figure captions identify the displayed trials and the progression over evolutionary time; the accompanying discussion contrasts protection with no protection.Source: Figure 6 · p. 8, Figure 7 · p. 8
  • Protecting morphological innovations is reported to improve optimization more effectively than protecting controller innovations, consistent with an asymmetry between body plan and brain.Source: §V · p. 8
  • Because a morphology’s long-term potential and a mutation’s immediate fitness impact may diverge, reduced selection pressure can help avoid premature convergence and stagnation at suboptimal values.Source: §V · p. 8
  • The authors connect the approach to embodied cognition and argue that body and brain are not cleanly separable; they also offer rough biological parallels in faster brain adaptation and varying selection pressure.They describe this as a first design-automation example using the interdependence of controllers and body plans, while noting that the biological analogies are not entirely clear.Source: §V · pp. 8–9
  • The method was applied to only one robot class, whose distributed sensing, actuation, and processing may make brain–body co-optimization comparatively simple.Source: §V · p. 8
  • For centralized controllers on rigid robots, mutations adding or removing components may require topological changes to the cognitive architecture for controller readaptation.Source: §V · p. 8
  • The efficacy of protection may depend on where morphology and control are divided and what qualifies as a morphological change; broader applicability remains for future work to test.The authors describe the algorithm as simple to implement and believe it may be widely applicable, but future work will test that supposition.Source: §V · p. 9

VI. CONCLUSION

The conclusion presents morphological innovation protection as a way to account for the interdependence of neural controllers and body plans during robot design. Across randomly initialized trials, the technique converges to similar morphologies, though the authors frame it as a short-term improvement that may be surpassed by developmental feedback between body and brain.

VI. CONCLUSION

  • Morphological innovation protection temporarily reduces selection pressure on newly mutated morphologies so agents can readapt their controllers and better escape local optima.Source: §VI · p. 9
  • The technique converges to a similar morphology across many independent trials from randomly initialized conditions, demonstrating an ability to escape local optima in the search space.Source: §VI · p. 9
  • The authors propose the algorithm as a short-term improvement over current co-optimization techniques and hope it will be surpassed by developmental processes with body–brain feedback loops.Source: §VI · p. 9
Loading 1706.06133v2…