PaperlayerView exact revision on arXiv

Source-linked AI summary

Lamarck's Driving School: Discovering Autonomous Driving Training Strategies through Evolutionary Competition

Yichun Ye, He Zhang, Ye Tian, Jian Sun

arXiv:2610.11662v1cs.AI

TL;DR

Training scenarios shape autonomous-driving capabilities, yet realism-, difficulty-, or risk-based selection does not necessarily yield better post-training performance. The paper uses multi-stage evolutionary competition and capability inheritance to discover valuable scenario distributions, then distills recurring patterns into a six-stage strategy. Under the same training budget, that strategy outperforms Replay, Curriculum, and CAT and remains stable across repeated runs.

  • Problem

    Scenario distributions shape learned capabilities, but surrogate criteria such as realism, difficulty, or risk do not necessarily identify distributions that improve post-training performance.

    Source: §I · p. 1

  • Method

    The framework searches stage-wise scenario distributions through evolutionary competition, trains policies on candidate distributions, inherits acquired capabilities, and distills discovered patterns into a lightweight strategy.

    Source: §I · p. 2, B. Training Strategy Optimization… · p. 3, §VI · p. 10

  • Results

    Under the same training budget, the six-stage Lamarckian Training Strategy achieves higher overall performance than Replay, Curriculum, and CAT and remains stable across repeated runs.

    Source: §VI · p. 10

  • Takeaways & Limitations

    High-value distributions can be identified for individual policy-development stages and distilled into a reusable training strategy.

    Source: §I · p. 2, §VI · p. 10

    Experiments primarily use MetaDrive with TD3, so transfer to other algorithms, simulators, and large real-world scenario libraries remains unverified.

    Source: §VI · p. 10

Abstract

from arXiv · show

Autonomous driving capabilities depend strongly on the distribution of scenarios encountered during training. Existing methods commonly construct or dynamically adapt training scenario distributions using surrogate criteria such as realism, difficulty, or risk. However, these predefined surrogates may misrepresent training value, leading to inefficient use of training resources. To address this limitation, we propose a Lamarckian evolutionary framework that replaces surrogate-based guidance with competition among candidate distributions. We formulate training strategy discovery as a multi-stage bilevel optimization problem and use Lamarckian evolution algorithm to approximate its solution. At the outer level, Darwinian crossover, mutation, and selection explore the scenario distribution space; at the inner level, policy learning acquires new capabilities, and Lamarckian inheritance transfers them to subsequent stages, allowing scenario distributions and policy capabilities to co-evolve. The resulting evolutionary trajectories reveal recurring stage-wise regularities among high-value distributions, characterized by capability accumulation through stage-wise challenge rotation. We further distill these regularities into a lightweight, reusable Lamarckian Training Strategy. Experiments show that, compared with the baseline, the complete framework reduces performance loss by up to 25.07%, while the lightweight strategy still achieves a 19.13% reduction. These results demonstrate that evolutionary competition can both discover effective training strategies and reveal reusable stage-wise patterns in how the value of training distributions changes with policy capability. Code is available on GitHub.

I. INTRODUCTION

Training scenario distributions shape learned driving capabilities, but distributions selected by realism, difficulty, or risk may not improve post-training performance. The paper uses performance-based evolutionary competition to discover stage-wise distributions and distills their recurring patterns into a reusable strategy.

I. INTRODUCTION

  • Surrogate criteria such as realism, difficulty, or risk do not necessarily identify distributions that improve post-training performance.This objective mismatch can make training resources inefficient and obscure reusable stage-wise distribution patterns.Source: §I · p. 1
  • Candidate distributions compete according to post-training policy performance, producing high-value paths and records from which reusable training patterns can be discovered.The search is formulated as a multi-stage bilevel problem and approximated through Lamarckian evolution.Source: §I · pp. 1–2
  • The framework combines Darwinian distribution search and selection with policy learning and Lamarckian inheritance of acquired capabilities across stages.Post-training performance determines which candidate individuals survive.Source: §I · p. 2
  • Evolutionary trajectories reveal capability accumulation through stage-wise challenge rotation, which is distilled into a lightweight, reusable Lamarckian Training Strategy.The distilled strategy preserves the discovered training logic without further evolutionary search.Source: §I · p. 2
  • The Lamarckian Training Strategy outperforms Replay and all comparison methods in overall score.The strategy is distilled from patterns found through evolutionary discovery.Source: §I · p. 2

II. RELATED WORK

Related work groups scenario training around naturalistic replay, curricula, and safety-critical generation, generally using surrogate criteria to select distributions. The section also frames policy learning under a given distribution as distinct from optimizing the distribution itself.

A. Naturalistic Scenario Replay

  • Naturalistic replay preserves real traffic distributions, but common low-risk cases dominate and long-tail interactions receive insufficient exposure.These methods generally treat the training distribution as given rather than as an optimizable strategy variable.Source: A. Naturalistic Scenario Replay · p. 2, §II · p. 2

B. Curriculum-Based Methods

  • Curriculum methods organize or adapt scenarios using rules, sampling, policy-dependent generation, regret, or learning potential; population-based training evolves hyperparameters, not data distributions.This paper instead searches distributions and distills recurring elite patterns into a reusable strategy.Source: B. Curriculum-Based Methods · pp. 2–3

C. Safety-Critical Scenario Generation

  • Safety-critical generation increases exposure to challenging scenarios but typically optimizes risk or criticality, which may not reveal reusable distribution sequences that improve task-level performance.Source: C. Safety-Critical Scenario Generation · p. 3

A. Learning Formulation for Autonomous Driving

  • Conventional learning optimizes policy parameters under a given scenario distribution, while differing training methods mainly differ in how they construct that distribution.The paper motivates treating the distribution sequence itself as an optimization variable when distributions lead to different learning outcomes.Source: A. Learning Formulation for Autonomous… · p. 3

B. Training Strategy Optimization Problem

The paper frames training-distribution choice as a strategy optimization problem because training objectives do not necessarily align with final evaluation performance. It approximates sequence-level optimization stage by stage, with each distribution’s value conditional on the current policy state and evaluation protocol.

B. Training Strategy Optimization Problem

  • Training scenario distributions affect final evaluation performance, but optimizing training reward or loss does not necessarily optimize the aggregate evaluation function F.The training objective J(θ, q) and final evaluation objective F are not fully aligned.Source: B. Training Strategy Optimization… · p. 3
  • Stage-wise optimization is a greedy approximation to sequence-level optimization, not an exact decomposition or a guarantee of global optimality.Source: B. Training Strategy Optimization… · p. 3
  • The stage-k value of a distribution is its post-training evaluation performance under the current policy capability and fixed stage-wise training budget.This value is conditional on the inherited policy state and evaluation protocol, not an intrinsic property of the distribution.Source: B. Training Strategy Optimization… · p. 3

IV. METHODOLOGY

The methodology couples outer-level scenario-distribution search with inner-level policy learning and transfers learned policy states across stages. It parameterizes distributions through environment features, making stage-wise distribution selection a search over those parameters.

IV. METHODOLOGY

  • The framework searches for high-value scenario distributions at the outer level, trains policy capability at the inner level, and transfers learned policy states to the next stage.Source: §IV · p. 3

A. Scenario Parameterization

  • The scenario distribution q specifies initial traffic states, environment dynamics, and background behavior, while the ego policy determines actions within the resulting closed-loop trajectories.Source: A. Scenario Parameterization · p. 4
  • Environment parameter ϕ represents initial states, background behavior, and dynamics, while policy parameter θ represents ego-policy capability; stage-wise distribution search therefore operates over ϕ.The induced distribution’s post-training value is conditional on the inherited policy state, fixed training budget, and evaluation protocol.Source: A. Scenario Parameterization · p. 4

B. Lamarckian Evolution Framework

Lamarckian evolution searches scenario distributions through environment-parameter variation while policies learn under those distributions. Candidates are evaluated after training, and selected lineages carry acquired policy capability into subsequent stages.

B. Lamarckian Evolution Framework

  • Candidate distributions compete on post-training evaluation performance after a fixed training budget, which serves as fitness rather than training reward or loss.The stage-wise fitness is measured within each candidate’s inherited policy lineage.Source: B. Lamarckian Evolution Framework · pp. 4–5
  • The search is a multi-environment black-box process because the environment-parameter space is heterogeneous and candidate fitness is observable only after policy training and evaluation.Source: B. Lamarckian Evolution Framework · p. 4
  • Crossover recombines high-fitness environment parameters and mutation perturbs them, while policy learning updates each candidate from its inherited policy state.These evolutionary operations act on environment parameters rather than directly modifying policy parameters.Source: B. Lamarckian Evolution Framework · p. 5
  • Selected candidates preserve policy capability acquired through finite-step training, allowing capability to accumulate along evolutionary paths.Source: B. Lamarckian Evolution Framework · p. 5

C. Scenario Generation and Sampling

The method converts parameterized distributions into executable scenario sets, then distills high-value evolutionary distributions into a fixed-stage training strategy. This avoids repeating the computationally expensive complete evolution run when training subsequent policies.

C. Scenario Generation and Sampling

  • A unified pool combines 400 Waymo source scenarios with TrafficGen and CAT variants derived exclusively from those same sources; sampling matches target distribution statistics as closely as possible.Source: C. Scenario Generation and Sampling · p. 5
  • When the pool lacks target attributes, TrafficGen supplies greater diversity and CAT supplies lower TTC or THW values; neither uses additional independent real-world source scenarios.Source: C. Scenario Generation and Sampling · p. 6
  • GeneLock rejects or projects candidates that violate physical, task-feasibility, or operational-design-domain constraints, keeping evolutionary search within feasible scenarios.It is not intended to increase scenario difficulty.Source: C. Scenario Generation and Sampling · p. 6

D. High-Value Distributions and Training Strategy Distillation

  • The strategy distills high-fitness distributions into fixed training stages, using feature shifts to segment stages and averaging elite parameter vectors within each stage.In this experiment, the six-stage segmentation is descriptive and is not claimed to be unique; GeneLock projects averages back into the feasible domain when needed.Source: D. High-Value Distributions and… · p. 6
  • Complete evolution requires 3.0 million policy-training steps, excluding evaluation, whereas one LTS run uses 300k steps.The discovery cost can be amortized when the fixed strategy is reused for subsequent policy training.Source: D. High-Value Distributions and… · p. 6

V. EXPERIMENTS

The experiments use complete Lamarckian evolution to discover high-value training paths and stage-wise scenario patterns, then distill those patterns into a fixed-stage training strategy. The analysis links distribution changes to capability accumulation through rotating challenges.

A. Experimental Setup

  • The experiments compare discovery, strategy validation under a shared training budget, and ablations of scenario search and capability inheritance.They use MetaDrive and TD3; training sources come from 400 Waymo scenarios, with disjoint fitness and final-test scenario sets.Source: §V · p. 6, A. Experimental Setup · p. 6

B. Lamarckian Evolution Experiment

  • Across 12 generations, selection retains higher-scoring individuals and produces progressively improving evolutionary paths.The search uses a population of 10, with 25,000 training steps per candidate in each generation.Source: B. Lamarckian Evolution Experiment · p. 7, Figure 3 · p. 7
  • Elite distribution features shift across generations, forming six stages whose learning challenges rotate rather than follow a monotonic easy-to-hard or risk-increasing progression.The reported challenges include basic driving, smooth control, traffic interaction, road structure, safety margins, and policy stability; inherited capabilities accumulate across stages.Source: B. Lamarckian Evolution Experiment · p. 7
  • Three policies trained from the same initial policy produce collision, out-of-road failure, and successful completion in the same scenario after one generation.The distributions differ in interaction intensity, motion variability, and safety margins; the cases do not establish a causal effect for any single factor.Source: Figure 5 · p. 8, B. Lamarckian Evolution Experiment · p. 8
  • LTS averages elite distribution vectors within six contiguous stages, distilling generation-wise shifts into a fixed sequence rather than copying the single highest-scoring path.The six stages are grouped using descriptive shifts in elite vectors.Source: B. Lamarckian Evolution Experiment · pp. 7, 9, Table 1 · p. 8

C. Validation of the Lamarckian Training Strategy

LTS achieves the highest mean validation overall score among the compared strategies, and reduces performance loss relative to Replay. Its stage-wise challenge rotation provides a reusable strategy without online evolution.

C. Validation of the Lamarckian Training Strategy

  • The validation compares LTS, Replay, Curriculum, and CAT under the same learning algorithm, maximum training budget, evaluation protocol, and 400 Waymo source scenarios.Five policies per strategy are trained independently from scratch using different random seeds.Source: C. Validation of the Lamarckian… · p. 9
  • 25.07% for Best Evolution and 19.13% for LTS are the respective performance-loss reductions relative to Replay.LTS also achieves the highest mean overall score in validation, outperforming Replay, Curriculum, and CAT.Source: C. Validation of the Lamarckian… · p. 9
  • LTS rotates the dominant training challenge from traffic adaptation and control calibration through interaction, road structure, safety margins, and policy consolidation.Replay uses naturalistic scenarios, while Curriculum and CAT are guided by difficulty and safety-criticality surrogates.Source: C. Validation of the Lamarckian… · p. 9
  • LTS outperforms Replay and the compared methods without online evolution, supporting the conversion of evolutionary analysis into a lightweight training strategy.LTS does not reach the score achieved by Best Evolution.Source: C. Validation of the Lamarckian… · p. 9

D. Ablation Study

The ablations test scenario search and cross-stage capability inheritance separately. Removing either mechanism reduces reported scores, while the inheritance intervention is interpreted as a mechanistic check rather than an isolated effect-size estimate.

D. Ablation Study

  • The outer-search ablation uses random scenario sampling instead of inheritance, crossover, mutation, and elite-pool selection; the inheritance ablation restarts each stage from the fixed initial policy.Both variants use the same computational budget and evaluation setup as the full evolutionary experiment.Source: D. Ablation Study · p. 9
  • Removing outer search reduces the best score from 83.98 to 71.78 and held-out nuScenes success from 78.31% to 56.63%; removing capability inheritance reduces them to 67.89 and 50.60%.The ablation results are reported in Table 3 and Fig. 7.Source: Table 3 · p. 10, Figure 7 · p. 10, D. Ablation Study · p. 10
  • The inheritance ablation repeatedly discards earlier-stage learning, making it a mechanistic check rather than a policy-age-matched estimate of inheritance’s isolated numerical effect.The authors interpret the degradation as a check that high-quality stage distributions alone are insufficient without cross-stage capability accumulation.Source: D. Ablation Study · p. 10

E. Discussion

Complete evolution discovers high-performing paths and distribution patterns, which can be distilled into a low-cost Lamarckian Training Strategy. The study is a controlled-simulator proof of concept, not an engineering study of production-scale online training.

  • Complete evolution serves as a discovery process for high-performing paths and valuable distribution patterns across policy-development stages.Source: E. Discussion · p. 10
  • Those patterns can be distilled into a low-cost Lamarckian Training Strategy used without online population search.Source: E. Discussion · p. 10
  • The study tests strategy discovery under a controlled simulator protocol, rather than production-scale online training.Source: E. Discussion · p. 10

VI. CONCLUSION

The framework couples scenario-distribution search with policy learning and capability inheritance. Experiments report that a six-stage Lamarckian Training Strategy outperforms Replay, Curriculum, and CAT under the same training budget, while transfer and other scope limits remain open.

  • The framework evolves training scenario distributions, trains policies, compares post-training performance, and inherits learned policy state across stages.Source: §VI · p. 10
  • Under the same training budget, the six-stage Lamarckian Training Strategy outperforms Replay, Curriculum, and CAT and remains stable across repeated runs.Source: §VI · p. 10
  • Complete evolution identifies high-value stage distributions for distillation; ablations show that distribution search and capability inheritance contribute to observed gains.Source: §VI · p. 10
  • Experiments primarily use MetaDrive with TD3; transfer to other algorithms, simulators, and large real-world scenario libraries remains unverified.Source: §VI · p. 10
  • Future work targets broader algorithmic transfer, richer generative models, real-world road testing, more efficient search and distillation, and high-fidelity validation.Source: §VI · p. 10

Related papers

Loading 2610.11662v1…