PaperlayerView exact revision on arXiv

Source-linked AI summary

Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy

Huan Rong, Chao Yin, Anouar Imel, Yijie Xia, Tinghuai Ma

arXiv:2609.38016v1cs.AIcs.CVcs.NEcs.RO

TL;DR

Existing constrained-RL methods commonly use static action costs or fixed feasible-region boundaries, limiting how dynamically their constraints reflect vehicle interactions. Brain-SAD uses a fear signal to select long- or short-term policies with dynamic constraints; the authors report higher success, shorter completion and recovery times, and greater stability across complex continuous intersections.

  • Problem

    Existing constrained-RL methods often use static state-to-cost mappings or fixed feasible-region boundaries estimated offline, rather than constraints that adapt to interaction scenarios.

    Source: §1 · p. 2, §2.3 · p. 4

  • Method

    Brain-SAD uses perceived interaction scenarios and predicted evolution to generate a fear signal that selects long- or short-term policies with dynamic action-cost or feasible-region constraints.

    Source: §1 · pp. 2–3

  • Results

    Brain-SAD achieves higher success rates, shorter task-completion and collision-recovery times, and more stable action distributions across continuous intersections of fluctuating complexity.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The fear reaction drives online policy selection and scenario-adapted constraints for both regular scene evolution and urgent braking.

    Source: §1 · p. 3

    Existing constrained-RL methods often rely on static state-mapped costs and fixed boundaries estimated from offline trajectories, rather than constraints coupled online to action impact.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

Constrained Reinforcement Learning has recently gained increasing attention in the field of Safe Autonomous Driving, where the general mechanism is to maximize the expected reward while keeping the overall action risk bounded. In this way, the safety issues arising in AD can be mitigated through constrained actions. However, existing Constrained RL methods still lack dynamics on the imposed constraints. For instance, the action cost adopted by the existing Primal-Dual/soft-constrained methods is often defined as static state-to-cost mapping, and the safe-action projection in hard-constrained methods relies on the static projection with the fixed feasible region boundary estimated from offline demonstrations. The above drawback tightly couples the imposed constraints to the training scenarios, leaving the AD policy hard to handle different interaction scenarios, due to the improper state-level action-cost and the static projection boundary. Consequently, in this paper, we propose Brain-SAD, a brain-inspired safe autonomous driving control framework with dynamic fear-oriented constraints. By perceiving the current vehicle-interaction scene, Brain-SAD generates dynamic fear signal as fear reaction to online decide long-term policy for regular interaction or short-term policy for urgent-collision defense. In such two policy, the above fear-reaction will be constructed as the dynamic fear constraints, respectively reflecting the overall fear cost directly coupled with action-impact, and the dynamic fear boundary of the feasible region derived from different risky neighbors, both of which will in turn serve for the online policy optimization. Experimental results show that Brain-SAD outperforms existing methods, achieving higher success rate in shorter task-completion and collision-recovery time, and exhibits stronger reliability across continuous intersections of fluctuating complexity.

1 INTRODUCTION

Brain-SAD targets constrained-RL methods whose static costs and projection boundaries do not adapt to changing vehicle interactions. It uses a fear signal to select between long- and short-term policies with dynamic constraints, and reports improved driving outcomes and reliability across continuous intersections.

1 INTRODUCTION

  • Safe AD aims to let the policy complete driving tasks while adhering to safety constraints and avoiding collision risk [3].The introduction notes increasingly complex settings such as heavy traffic and continuous intersections.Source: §1 · p. 1
  • Existing soft- and hard-constrained methods can produce similar action distributions and feasible regions across scenes with different safety requirements.The introduction attributes this to static action-cost rules in soft methods and static projection operators in hard methods.Source: §1 · p. 2
  • Dynamic constraints should adapt both action-cost estimation and feasible-region boundaries to interaction scenarios, supporting regular scene evolution and urgent collisions.The introduction proposes integrating these constraints in a unified control framework.Source: §1 · p. 2
  • Brain-SAD perceives interaction scenarios and their predicted evolution to generate a fear reaction that selects long- or short-term policies and constructs dynamic action-cost and feasible-region constraints.The selected constraints are used for online policy optimization.Source: §1 · p. 2
  • For mild fear reactions, the long-term policy uses a dynamic action reward-cost constraint to post-correct the prior action distribution for different scenarios.This policy is intended for regular vehicle interactions.Source: §1 · pp. 2–3
  • For fierce fear reactions, the short-term policy derives a dynamic feasible-region boundary from risky neighbors to select a safe action for emergency braking.The boundary can tighten or loosen the projected feasible region across scenarios.Source: §1 · p. 3
  • Brain-SAD outperforms existing works with higher success rates, shorter task-completion and collision-recovery times, and more stable action distributions across intersections of fluctuating complexity.The reported policy selection matches long- and short-term policies to the current interaction scenario.Source: §1 · p. 3

2 RELATED WORKS

Related work covers cumulative-cost and instantaneous-projection constrained RL, other policy- and environment-oriented constraints, and brain-inspired action control. The discussion identifies static costs and boundaries as a gap in adapting constraints to action impacts.

2.1 Constrained Behaviour Learning in Safe AD

  • Soft-constrained methods optimize reward under cumulative cost limits, but static state-mapped costs can destabilize online policy optimization and require many trajectory samples,,.Examples include a policy-gradient method with stated optimality and constraint-violation guarantees, and methods estimating state cost from actions,.Source: §2.1 · p. 3
  • Hard-constrained methods enforce statewise safe-set constraints [3]; approaches derive feasible regions from policy divergence or offline-regressed Hamilton-Jacobi boundaries, which can reduce reward when overly restrictive.The latter uses offline regression to estimate the safe-state boundary.Source: §2.1 · p. 3
  • Related work groups constrained behaviour learning into ego-vehicle policy-oriented and environment-oriented prior constraints, including CADRE, RLfOLD, imagined-trajectory methods,, and language-model-based methods,,.The cited examples use approaches including online reward or demonstration constraints, imagined trajectories, and traffic-rule or scenario knowledge.Source: §2.1 · p. 3

2.2 Brain-Inspired RL on Agent Action Control

  • Brain-inspired RL agents adopt brain mechanisms to improve policy learning, plausibility, and performance; action-control examples address memory, other-agent behavior prediction, goal decomposition, temporal causality, and multiple-brain-region control,,,,.Source: §2.2 · p. 4
  • FNI-RL uses an adversary that maximizes fear against a reward-maximizing protagonist, but its fear cost is a static state mapping rather than an estimate of future fear after an action.The review therefore classifies it as a static state-level fear constraint.Source: §2.2 · p. 4

2.3 Discussion on Existing Works

  • The discussion identifies a gap: existing costs are static state mappings and feasible-region boundaries often remain fixed from offline trajectories, rather than adapting online to action impacts.The review says existing works are prone to action instability or reward inaccessibility, limiting policy learning to handle collisions.Source: §2.3 · p. 4

3 DETAILS ON OUR PROPOSED BRAIN-SAD

This section describes how Brain-SAD uses scene-responsive fear signals to select and optimize two driving policies under dynamic safety constraints: posterior-supervised long-term action optimization and risk-dependent safe-set projection for urgent defense. It also outlines how online experience feeds an offline-to-online training loop.

3 DETAILS ON OUR PROPOSED BRAIN-SAD

  • The framework motivates dynamic constraints that reflect differing safety requirements across interaction scenarios and adopted actions, treating policy selection, execution, and optimization together.Source: §3 · p. 4

3.1 Current-Scene Imagination for Dual Policy Selection

  • Scene-imagined interactions turn neighbor intent and candidate actions into fear signals that fuse risk and safety to select a long-term or urgent short-term policy online.The framework rolls out future interaction trajectories and uses the resulting dynamic fear signal to derive the online policy-selection signal.Source: §3.1 · pp. 4–5

3.2 Online Long-Term Dynamic Fear-Constrained Policy

  • For regular scenes, Brain-SAD estimates candidate-action reward and action-coupled fear cost, then optimizes a posterior under a fear-cost cap and KL trust region before updating policy weights online.The resulting reward-cost constraint is constructed at the action level to adapt to different interaction scenarios.Source: §3.2 · pp. 6–7
  • The long-term policy’s action distribution is post-constrained using posterior supervision from q* within a KL trust region to prevent dramatic policy shifts.The posterior supervision uses the post-action weights described in equation (10).Source: §3.2 · p. 7

3.3 Online Short-Term Dynamic Fear-Projected Policy

  • For urgent-collision defense, the short-term policy projects its risk action into a safe reachable set and selects the action with minimum shift from the original.The projection is constrained online by a fear boundary that tightens or loosens the set according to neighboring vehicles’ risk intent.Source: §3.3 · pp. 7–8
  • The fear boundary is derived from risk neighbors and their braking demands, with the most negative demand setting the braking limit for the safe action set.Neighbors are identified using relative distance, relative velocity, and time-to-collision criteria; changing neighbors dynamically tightens or loosens the set.Source: §3.3 · p. 8

3.4 Online-to-Offline Optimization Loop

  • Online interaction trajectories, including fear signals, policy labels, and collision outcomes, are stored in a replay buffer to support offline optimization and renewed online exploration.Offline updates are aggregated to update Brain-SAD components before the next online round generates new buffer trajectories.Source: §3.4 · p. 9
  • Offline training first fits an ensemble environment model to next-state shifts and collision labels, then trains policy selection and the long- and short-term policy networks from buffered trajectories.Policy selection is trained against collision and long/short-policy labels; the short-term actor imitates safe actions while its critic evaluates their long-term reward.Source: §3.4 · p. 9

4 ANALYSIS ON EXPERIMENTAL RESULTS

Across the comparative study and six-intersection stress test, Brain-SAD reports stronger task performance than the compared methods. Its ablations favor the full dual-policy version, while its dynamic fear signal supports adaptation across changing interactions.

4.1 Configuration on Simulation Environment

  • The evaluation uses SUMO scenes with increasing traffic interaction complexity and averages metrics across three training rounds.Episodes run for up to 600 steps with early stopping on success or collision; reported metrics include task completion time, success rate, reward, speed-limit compliance, comfort, time-to-collision, and recovery time.Source: §4.1 · pp. 9–10

4.2 Ablation Study

  • In Scenes A and B, the full Brain-SAD version outperforms single long-term and single short-term policies across metrics as interaction complexity increases.The reported trade-offs in TCT and Comfortable vary across the two scenes; the authors conclude that long-term policy alone is inadequate for higher-complexity interaction.Source: §4.2 · p. 10, Table 1 · p. 11, Table 2 · p. 11

4.3 Comparative Study

  • Across Scenes B and C, Brain-SAD reports the shortest TCT and RECT, highest SR, and highest average cumulative Reward among the compared methods.The reported advantage is more explicit in Scene C, with more ahead and aside interactions.Source: §4.3 · p. 11, Table 3 · p. 12, Table 4 · p. 12
  • The authors attribute Brain-SAD's comparative advantage to dual-path policy collaboration and online policy optimization guided by dynamic fear constraints coupled with action impact.They contrast this with state-level fear-cost scoring and approaches that do not directly guide action-distribution optimization.Source: §4.3 · p. 12

4.4 Stress Test by Continuous Intersections

  • Across six continuous intersections, Brain-SAD reports the highest SR and Reward and the shortest TCT and RECT.Its training traces also show more obvious convergence toward the reported optimal level than SVPG, FNI-RL, and EFT-RL.Source: §4.4 · pp. 13–14, Table 5 · p. 14
  • Brain-SAD also reports the best SLC and Comfortable across the six-intersection stress test, while EFT-RL has better Median-TTC.The authors describe EFT-RL's actions as more strongly shaken and its task completion and collision-recovery times as longer.Source: §4.4 · p. 14
  • The stress test reports that Brain-SAD's fear signal and policy selection adapt across intersections of changing complexity, alongside a steadier, higher reward curve over fewer steps.Brain-SAD transitions from short-term to long-term policy as the scene shifts toward regular evolution.Source: §4.4 · pp. 14–15, Figure 11 · p. 15

5 CONCLUSION & FUTURE WORK

Brain-SAD combines dynamic fear perception and constraints with dual-path policy selection for safe autonomous driving. The reported experiments show faster, more successful continuous interactions with more stable action distributions; future work will explore finer-grained constraints in more complex environments.

5 CONCLUSION & FUTURE WORK

  • Brain-SAD’s dynamic fear perception selects long- or short-term policies to better match the current interaction scenario.The framework converts fear perception into dynamic constraints coupled with action impact or value estimation for online policy optimization.Source: §5 · p. 15
  • Dynamic fear constraints coupled with action impact or value estimation support online policy optimization and collision handling under environmental disturbance.The paper reports stronger policy reliability against constant environmental disturbance.Source: §5 · p. 15
  • Brain-SAD completes continuous interactions in shorter time, with higher success and more stable action distributions.These outcomes are attributed in the passage to the dynamic fear constraint.Source: §5 · p. 15
  • Future work will explore finer-grained dynamic fear constraints, including value estimation via LM inference in more complex environments.Source: §5 · p. 15
Loading 2609.38016v1…