PaperlayerView exact revision on arXiv

Source-linked AI summary

CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving

Zhe Huang, Zhaoxin Fan, Shuo Wang, Wenjun Wu, Xuan Zhao, Min Liu

arXiv:2609.04807v1cs.ROcs.CV

TL;DR

Existing negotiation methods can converge prematurely because complex driving scenarios admit multiple feasible decisions, limiting stable consensus. CoLMIN addresses this with multi-decision negotiation and reflective modules, and it outperforms existing methods in challenging CARLA scenarios. Its reported evaluation remains primarily within closed-loop simulation rather than real-world driving.

  • Problem

    Single-decision negotiation can overlook the multi-solution nature of driving decisions, causing premature convergence to suboptimal plans and hindering consensus formation.

  • Method

    CoLMIN generates and jointly evaluates multiple candidate driving intentions, then uses shallow feedback and deep negotiation-history reflection to refine consensus.

  • Results

    CoLMIN achieves the highest driving score across reported interactive scenarios, improving overall driving score by 1.59% over CoLMDriver (90.12% vs. 88.53%).

  • Takeaways & Limitations

    Multi-decision negotiation and reflective reasoning support stable consensus in complex cooperative driving scenarios.

  • Takeaways & Limitations

    Evaluation is mainly limited to closed-loop simulation, which cannot fully reflect real-world perception noise, localization errors, communication latency, human behavior, road structures, or traffic rules.

Abstract

from arXiv · show

Multi-vehicle cooperative autonomous driving enhances the safety and reliability of autonomous driving systems through information sharing among connected vehicles, demonstrating significant potential for improving traffic safety. LLM-based approaches leverage strong reasoning capabilities of LLMs to enable effective inter-vehicle negotiation and improve cooperative driving performance. However, driving decisions in complex traffic scenarios are inherently multi-solution in nature. As a result, existing negotiation-based methods often converge prematurely to suboptimal solutions, hindering consensus formation and limiting the practical deployment of cooperative autonomous driving systems. To address this challenge, we propose CoLMIN, the LLM-based multi-decision path negotiation framework for cooperative autonomous driving, achieving stable decision consensus through multi-decision path negotiation and reflective reasoning. To achieve stable and high-quality consensus in cooperative autonomous driving, CoLMIN consists of three key components: (i) an LLM-based Multi-Intent Negotiation module (LMin), which adopts a Negotiator-Evaluator paradigm and generates multiple candidate driving intentions for joint evaluation; (ii) an Evaluation-based Shallow Reflection Module (ESRM), which analyzes negotiation outcomes and provides feedback to guide subsequent negotiations, thereby accelerating consensus formation; and (iii) an LLM-based Deep Reflection Module (LDRM), which performs long-term reflection over negotiation histories to mitigate cognitive fixation and prevent the system from converging to suboptimal solutions. Experimental results in the CARLA simulation environment demonstrate that CoLMIN significantly outperforms existing methods in challenging interactive driving scenarios.

I. INTRODUCTION

Cooperative autonomous driving benefits from information sharing, but existing single-decision negotiation can overlook multiple feasible solutions and converge prematurely. CoLMIN addresses this with multi-decision path negotiation and reflective reasoning, achieving stronger consensus and reported gains in challenging scenarios.

  • Cooperative autonomous driving uses real-time information sharing and intention communication to mitigate incomplete perception and uncertainty about surrounding vehicles.
  • Optimization-, rule-, and learning-based methods face task-specific constraints, limited robustness, computational complexity, or degraded transfer to unseen environments.
  • LLM-based methods support semantic negotiation, sophisticated decision-making, and flexible interaction in diverse and dynamic traffic scenarios.
  • Single-decision negotiation generates one deterministic plan per round, overlooking multiple feasible solutions and potentially causing deadlock or consensus failure.
  • CoLMIN generates multiple candidate intentions, evaluates them through negotiation, and uses LMin, ESRM, and LDRM to support stable consensus.
  • 13% higher overall success rate and 8% greater safety are reported for CoLMIN in the challenging lane-changing scenario.

II. RELATED WORKS

End-to-end and LLM-based driving methods improve reasoning or decision-making but remain constrained by generalization and negotiation limitations. Reflection and multi-decision negotiation are introduced to address premature convergence and improve consensus behavior.

  • Reinforcement-learning and imitation-learning approaches improve end-to-end driving accuracy but have limited generalization and reasoning in unseen scenarios.
  • LLM-integrated architectures combine perception, reasoning, and decision-making to improve interpretability and support human-like interaction.
  • Single-decision negotiation restricts exploration of feasible strategies, causing premature convergence to suboptimal plans and repeated revisions that hinder consensus formation.

C. LLM-based negotiation

LLM negotiation research motivates applying language-based coordination to cooperative driving, where dynamic conflicts and safety requirements make consensus difficult. The formulation models this process as bounded iterative optimization over multiple candidate plans evaluated for driving quality.

  • LLM-based negotiation: General LLM negotiation frameworks emphasize language generation and interaction quality but do not adequately address dynamic conflicts and consensus in cooperative autonomous driving.
  • LLM-based negotiation: CoLMIN targets intention exchange, conflict resolution, and consensus formation among vehicles in dynamic traffic environments.
  • LLM-based negotiation: Multi-agent negotiation remains challenging in driving because limited mutual understanding can produce unstable decisions and increased collision risks.
  • Problem formulation: The system models cooperative driving as constrained iterative consensus optimization, using an LLM to analyze, evaluate, and coordinate multiple vehicles’ decisions.
  • Problem formulation: At each negotiation round, the LLM generates a finite set of candidate plans representing complete multi-vehicle driving actions, interaction requests, and responses.
  • Problem formulation: Candidate plans are scored and the best-so-far candidate is retained during bounded iterative negotiation until a predefined threshold or maximum round condition is reached.

IV. METHODOLOGY

CoLMIN combines multi-candidate plan generation and evaluation with short-term feedback and long-term history analysis. The architecture iteratively revises plans when quality requirements are unmet and adjusts strategies to avoid repetitive or overconfident decisions.

  • CoLMIN consists of LMin, ESRM, and LDRM, covering multi-intent negotiation, evaluation-based refinement, and deep reflection over negotiation history.

A. Overall Architecture

CoLMIN converts sensor observations into vehicle intentions and detects conflicts before initiating multi-vehicle negotiation. LMin generates and evaluates candidate plans, while reflection modules refine unsuccessful negotiations.

  • Perception and conflict detection: The perception module transforms sensor data into object-level and BEV representations plus concise textual descriptions for intent prediction.BEV features and occupancy maps support planning, while object-level and interaction information are textualized.
  • Perception and conflict detection: The intent predictor infers each vehicle’s initial driving intention and identifies potential conflicts.The inferred intention includes the vehicle’s speed and intended driving action.
  • Negotiation and reflection: When conflicts occur, CoLMIN sends all vehicles’ initial intentions to LMin for multi-vehicle negotiation.LMin evaluates candidate plans using safety, efficiency, and consensus-related criteria.
  • Negotiation and reflection: ESRM supplies short-term refinement suggestions when candidate plans fail quality requirements, while LDRM provides long-term reflection against overconfident or repetitive decisions.Together, the modules guide subsequent negotiation toward consensus.

B. LLM-based Multi-Intent Negotiation Module

LMin uses a Negotiator–Evaluator loop to expand negotiation beyond one deterministic plan. Vehicles generate multiple feasible intentions, which are assessed using waypoint-based safety and efficiency together with LLM-based consensus scoring.

  • Module objective: LMin generates multiple feasible candidate driving intentions to expand the negotiation space and avoid premature convergence to suboptimal decisions.Its Negotiator–Evaluator mechanism supports joint evaluation of alternatives.
  • Negotiator: The negotiator sequentially proposes multiple executable solutions using vehicle states, other vehicles’ intentions, negotiation history, and prior feedback.Speaking order and prompt format remain fixed across experiments.
  • Negotiator: At each round, the LLM generates a finite candidate plan set P(r), with prior negotiation history and previous-round suggestions available as inputs.The first round has no external feedback.
  • Evaluator: The evaluator distributes each joint intention to vehicles, generates future waypoints, and assesses candidate feasibility and quality beyond textual negotiation results.This grounds evaluation in predicted vehicle trajectories.
  • Evaluator: Candidate plans receive safety, efficiency, and consensus scores representing collision avoidance, route progress, and willingness to execute the joint strategy.Safety and efficiency use planner-generated waypoints, while consensus is generated by the LLM under fixed criteria.
  • Consensus control: Negotiation terminates when a candidate exceeds threshold τ; otherwise, the highest-scoring plan guides continued rounds.The process continues until a feasible and consistent plan is identified or the negotiation limit is reached.

C. Evaluation-based Shallow Reflection Module

ESRM addresses stalled negotiations by analyzing low-scoring candidate plans and producing targeted short-term feedback. The feedback steers later rounds away from ineffective strategies and improves local consensus efficiency.

  • Motivation: Negotiator–Evaluator iteration can stall when every candidate in a round fails safety or efficiency thresholds.Without explicit guidance, later rounds may repeat similar ineffective reasoning patterns.
  • ESRM function: ESRM performs short-term corrective reflection on low-score plans and provides targeted suggestions for subsequent negotiation rounds.It is designed to improve plan quality and convergence efficiency.
  • ESRM workflow: In the first round ESRM gives no feedback; after failed evaluation, it analyzes low-scoring plans from safety and efficiency perspectives.Examples include recommending yielding or deceleration for low safety scores.
  • Formalization: The ESRM feedback expression uses aggregated intentions and corresponding safety-and-efficiency evaluation scores to produce concise suggestions for the next round.The resulting suggestions are denoted F^r_sugg.

D. LLM-based Deep Reflection Module

LDRM addresses long-term overconfidence and reasoning fixation that can persist despite ESRM’s round-to-round refinements. It reflects over negotiation histories and detects recurring patterns to correct accumulated cognitive biases.

  • Motivation: Prolonged negotiations may exhibit overconfidence and reasoning fixation, causing stagnation or suboptimal outcomes despite short-term ESRM refinements.Overconfidence is maintaining an inappropriate decision after feedback indicates adjustment is needed.
  • LDRM function: LDRM performs long-term reflection to identify and correct cognitive biases accumulated across multiple negotiation rounds.It activates only when negotiation persists beyond a predefined number of rounds k.
  • LDRM function: LDRM records proposed plans, reasoning traces, and evaluation scores from negotiation history for its reflection process.This historical view distinguishes it from ESRM’s round-to-round refinement.
  • Bias detection: Its detection criteria include repeated strategies, consistently low safety or efficiency scores, overconfident expressions, and deviation from group consensus.These criteria target persistent behavioral patterns rather than a single failed round.
  • Downstream execution: After consensus, the unchanged low-level controller converts negotiated intents into executable throttle, braking, and steering commands.The work focuses on upstream negotiation and decision-making rather than modifying this controller.

V. EXPERIMENTS

CoLMIN is evaluated in closed-loop CARLA/V2Xverse simulations across ten multi-vehicle scenarios using safety, completion, driving-score, and success metrics. It outperforms the compared cooperative-driving methods, remains robust with traffic participants and latency, and improves conflict resolution without inducing deadlocks.

  • Experimental setup: Closed-loop evaluation uses CARLA/V2Xverse simulations with ten multi-vehicle scenarios and four primary metrics: Driving Score, Route Completion, Infraction Score, and Success Rate.The scenarios include realistic traffic settings, and Driving Score combines route completion with safety-violation penalties.
  • Overall performance: CoLMIN achieves the highest driving score across interactive scenarios, improving overall Driving Score by 1.59% over CoLMDriver.The reported scores are 90.12% for CoLMIN versus 88.53% for CoLMDriver.
  • Scenario results: In Lane Changing, CoLMIN surpasses CoLMDriver by 7.99% in Driving Score, reaching 67.20% versus 59.21%.The result is reported for the challenging Lane Changing scenario.
  • Interpretation: CoLMIN combines multi-decision-path negotiation and reflection to prevent intention-conflict collisions while avoiding deadlocks caused by excessive yielding.This balances safety and efficiency in cooperative driving.
  • Robustness: With dynamic traffic participants, performance degrades because interaction complexity and uncertainty increase, but CoLMIN consistently maintains strong performance.The additional participants include pedestrians and cyclists.

C. Qualitative evaluation

Qualitative evaluation illustrates CoLMIN’s multi-vehicle negotiation in complex traffic and compares convergence under single- and multi-decision-path paradigms. The method coordinates conflicting vehicle intentions into feasible, collision-free actions.

  • Qualitative evaluation: In the Cross Change scenario, four vehicles negotiate from their individual states and execute a feasible joint action plan.The visualization depicts coordinated behavior among vehicles with different intended maneuvers.
  • Qualitative evaluation: When planned lane changes and straight movements would cause a collision, multiple negotiation rounds coordinate stopping, slowing, passing, and merging actions.CAV 1 stops, CAV 3 slows behind it, CAV 2 passes, and CAV 4 merges right while maintaining safe distances.
  • Comparative analyses: The section also evaluates convergence differences between single-decision-path and multi-decision-path negotiation paradigms.The comparison is presented in Fig. 6, alongside ablation, inference-time, and failure-case analyses.
  • Qualitative evaluation: The qualitative example shows decision-level multi-agent coordination producing traffic behaviors consistent with the surrounding context while enhancing safety.The coordinated actions resolve the detected conflict without relying on the original colliding plans.

D. Ablation Study

The ablation studies show that CoLMIN’s multi-decision negotiation and reflection modules improve driving performance, consensus convergence, and robustness, while introducing moderate overhead. Additional analyses examine latency, inference time, candidate-intent count, and failure modes.

  • System Component Ablation: Adding multi-decision path negotiation raises Driving Score by 7.07% and Route Completion by 8.19% over single-intent negotiation.Driving Score increases from 70.22% to 77.29%, while Route Completion increases from 87.03% to 95.22%.
  • System Component Ablation: Integrating ESRM and LDRM boosts overall performance by over 10%, while LDRM reduces collisions and improves Infraction Score and Success Rate.The ablation attributes collision reduction specifically to LDRM.
  • Latency Performance: CoLMIN’s score drops only 3% under inference latency and remains above 90, compared with approximately 7%, 4%, and 6% declines for LMDrive, CoDriving, and CoLMDriver.The comparison is conducted in the Lane Merging scenario under ideal and latency conditions.
  • Consensus Convergence: Multi-decision negotiation produces smoother, more stable score distributions across rounds and reduces negotiation oscillation.The evaluator guides exploration of a richer intent space and steadier strategy refinement.
  • Inference Time: CoLMIN averages 29.67 s for complete high-level negotiation and reasoning, while its largest scenario-specific improvement reduces LC time from 41.44 s to 37.83 s.The reported time excludes low-level real-time controller inference.
  • Controlled Process-Level Analysis: 86.25% convergence rate and 1.90 average negotiation rounds indicate more stable consensus than 33.90% and 2.52 rounds for the comparison condition.Average overconfident vehicles also decreases from 2.08 to 1.37.

VI. DISCUSSION ON REAL-WORLD APPLICABILITY.

CoLMIN is validated primarily through CARLA closed-loop simulation, and its evaluator scores closely match objective rollout scores. Real-world applicability remains unvalidated because deployment conditions and interactive closed-loop data are difficult to reproduce.

  • Evaluation Scope: CARLA closed-loop experiments mainly validate the feasibility of CoLMIN’s negotiation mechanism in simulation.The evaluation follows common practice in cooperative autonomous driving studies.
  • Real-World Applicability: Real-world validation must address perception noise, localization errors, communication latency, human driving behavior, road structures, and traffic-rule differences.Many real-world datasets are open-loop and therefore cannot directly support interactive multi-vehicle negotiation testing.
  • Evaluator Validation: The LMin evaluator’s absolute score gaps from objective deployment scores are 0.031 for safety, 0.043 for efficiency, and 0.059 for consensus.These gaps support the reported accuracy and reliability of the evaluator.
Loading 2609.04807v1…