PaperlayerView exact revision on arXiv

Source-linked AI summary

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, Yu Qiao

arXiv:2309.16292v3cs.ROcs.CL

TL;DR

Autonomous-driving systems built from data-driven methods face dataset bias, overfitting, and uninterpretability, raising the question of how to instill human-like knowledge-driven capabilities. DiLu combines LLM-based reasoning and reflection with memory to make decisions and update experiences. Experiments report continuous experience accumulation, comparable performance to extensively trained RL methods with 40 memory items, and stronger generalization across diverse scenarios.

  • Problem

    Data-driven autonomous driving faces dataset bias, overfitting, and uninterpretability, motivating the question of how to instill human-like knowledge-driven capabilities.

  • Method

    DiLu uses an LLM-based Reasoning module, Reflection module, and Memory module to generate decisions from scenario knowledge and experiences, refine unsafe decisions, and store revised experiences.

  • Results

    DiLu continuously accumulates experience, achieves comparable performance to RL-based methods trained over 600,000 episodes with 40 memory items, and generalizes more strongly to diverse scenarios.

  • Takeaways & Limitations

    The framework demonstrates a knowledge-driven approach to autonomous-vehicle decision-making with potential for practical deployment and direct experience acquisition from real-world datasets.

Abstract

from arXiv · show

Recent advancements in autonomous driving have relied on data-driven approaches, which are widely adopted but face challenges including dataset bias, overfitting, and uninterpretability. Drawing inspiration from the knowledge-driven nature of human driving, we explore the question of how to instill similar capabilities into autonomous driving systems and summarize a paradigm that integrates an interactive environment, a driver agent, as well as a memory component to address this question. Leveraging large language models (LLMs) with emergent abilities, we propose the DiLu framework, which combines a Reasoning and a Reflection module to enable the system to perform decision-making based on common-sense knowledge and evolve continuously. Extensive experiments prove DiLu's capability to accumulate experience and demonstrate a significant advantage in generalization ability over reinforcement learning-based methods. Moreover, DiLu is able to directly acquire experiences from real-world datasets which highlights its potential to be deployed on practical autonomous driving systems. To the best of our knowledge, we are the first to leverage knowledge-driven capability in decision-making for autonomous vehicles. Through the proposed DiLu framework, LLM is strengthened to apply knowledge and to reason causally in the autonomous driving domain. Project page: https://pjlab-adg.github.io/DiLu/

1 INTRODUCTION

The paper frames autonomous driving as a knowledge-driven problem, motivated by limitations of data-driven methods and human drivers’ ability to generalize through common sense. DiLu implements this paradigm with LLM-based reasoning, reflection, and memory, and experiments report experience accumulation and stronger generalization than RL-based methods.

  • Data-driven autonomous driving faces dataset bias, overfitting, and uninterpretability, motivating methods for more rational scenario understanding.
  • Human drivers use common sense and explainable reasoning to handle unfamiliar situations, whereas data-driven methods rely on similar data and may limit generalization.
  • DiLu combines an LLM-based Reasoning module, Reflection module, and Memory module to make and refine decisions using accumulated experiences.
  • DiLu achieves comparable performance to RL-based methods trained over 600,000 episodes using only 40 memory items, with stronger generalization to diverse scenarios.
  • Experiments show that DiLu continuously accumulates experience through environment interaction and has potential for practical autonomous-driving deployment.

2 RELATED WORKS

Large language models are Transformer-based models trained on massive text datasets for natural-language understanding and generation. Their embedded common-sense knowledge has motivated research on autonomous agents with human-like capabilities.

  • LLMs are Transformer-based language models with enormous parameter counts, trained on massive text datasets for complex text-generation and comprehension tasks.
  • Their common-sense knowledge has supported adoption across diverse tasks.
  • Research increasingly uses LLMs to create autonomous agents endowed with human-like capabilities.

3 METHODOLOGY

DiLu combines environment observation, LLM-based reasoning, reflection, and experience memory for closed-loop autonomous-driving decisions. It retrieves similar experiences to guide decisions, then evaluates and revises them for continued memory-based improvement.

  • Framework Overview: DiLu consists of Environment, Reasoning, Reflection, and Memory modules connected in a closed-loop driving framework.Reasoning generates actions from observations and retrieved experiences, while Reflection updates revised experiences in Memory.
  • Memory Module: Memory stores scene descriptions with corresponding reasoning processes because out-of-the-box LLMs lack precise reasoning for complex closed-loop driving without few-shot experiences.Initial memories are manually created from selected scenarios and correct reasoning and decisions.
  • Memory Module: At each decision frame, DiLu embeds the current scenario, retrieves the closest stored situations, and supplies their reasoning processes as few-shot guidance.The retrieved memories are provided to the agent through a similarity search over scenario vectors.
  • Reasoning Module: The Reasoning module encodes traffic scenarios, generates prompts, queries the LLM, and decodes its response into an ego-vehicle action.Prompts combine system instructions, textual scenario descriptions, and retrieved few-shot experiences; the action decoder feeds actions back to the environment.
  • Reasoning Module: Chain-of-Thought prompting elicits step-by-step reasoning before the final driving decision, addressing the complexity and variability of driving scenarios.
  • Reflection Module: Reflection records prompts and decisions across a driving session, assesses outcomes as safe or unsafe, and revises unsafe decisions before updating Memory.For hazardous sessions, interpretable chain-of-thought responses help identify causes of dangerous situations; correct reasoning and decisions can also be stored as new memories.

4 EXPERIMENTS

Experiments evaluate DiLu’s experience accumulation, comparison with reinforcement learning, environmental generalization, real-world experience transfer, and Reflection-module memory types. Results show improved performance with more experience and few-shot examples, stronger transfer than GRAD, and benefits from both successful and corrected experiences.

  • Validation of the DiLu framework: Highway-env experiments test DiLu with and without Memory, varying few-shot examples and memory sizes across repeated random seeds.Success Steps counts consecutive collision-free frames, with 30 indicating task completion.
  • Validation of the DiLu framework: As Memory grows, DiLu improves across few-shot settings; with 40 items, 5-shot nearly passes all tests, while 3-shot and 1-shot medians are 27 and 25.The authors attribute this to more diverse experiences and decision strategies supplied as prompts.
  • Comparison with reinforcement learning method: 70% SR for DiLu with 40 experiences matches GRAD’s 69% SR after 600,000 training episodes in lane-4-density-2.In lane-5-density-3 without extra optimization, DiLu retains 35% SR while GRAD undergoes an 85% performance drop.
  • Generalization and transformation: With 20 experiences transferred from lane-4-density-2 to lane-5-density-3, 5-shot DiLu declines from 30 to 23 median SS, compared with 13 to 5 for 3-shot DiLu.The results suggest better generalization with more few-shot experience.
  • Generalization and transformation: CitySim-derived experiences improve DiLu over having no prior experience and show greater robustness than simulation-only memory in more congested environments.The experiment evaluates both memory sources within Highway-env.
  • Effectiveness of two memory types in the Reflection module: The Reflection ablation adds successful and corrected unsafe experiences to a 20-item baseline to evaluate two memory types using Success Steps quartiles.The reported conclusion is that adding both experience types is reasonable and effective.

5 CONCLUSION

DiLu instills human-level, knowledge-driven reasoning into autonomous driving through LLM-based Reasoning and Reflection with memory. Experiments show experience accumulation, stronger generalization than a state-of-the-art RL method, and transfer from real-world datasets, while latency and hallucinations remain limitations.

  • Conclusion: DiLu combines a Memory module with LLM-based Reasoning and Reflection to support knowledge-driven autonomous-vehicle decision-making.The framework records experiences and continually updates them through interaction.
  • Conclusion: DiLu continuously accumulates experience and exhibits stronger generalization than the state-of-the-art reinforcement-learning method.The conclusion summarizes these findings from extensive experiments.
  • Conclusion: DiLu can directly acquire experiences from real-world datasets, supporting its potential for practical autonomous-driving deployment.This consequence is stated as a potential rather than a demonstrated deployment outcome.

A.1 DETAILED SETUP OF THE EXPERIMENT

The experiment uses GPT-3.5 for reasoning, GPT-4 for reflection, and Chroma as a vector-based memory module. Highway-env supplies current-frame vehicle states, while DiLu outputs five discrete meta-actions in closed-loop driving tasks.

  • Model and memory setup: GPT-3.5 performs ego-vehicle decision-making in the Reasoning module, while GPT-4 supports error correction in the Reflection module.Scenario descriptions are embedded with OpenAI’s text-embedding-ada-002 model for memory retrieval.
  • Model and memory setup: Chroma serves as the Memory module, storing scenario descriptions represented as embedding vectors.
  • Driving environment: Highway-env provides each vehicle’s position, speed, and acceleration from the current frame without decision-intent or potential-risk information.
  • Driving environment: The ego vehicle selects among five discrete meta-actions: acceleration, maintaining speed, deceleration, and left or right lane changes.
  • Driving environment: A task succeeds when the ego vehicle navigates traffic for 30 seconds at a reasonable speed without collisions, using decisions at 1Hz.Unless otherwise stated, experiments use a four-lane motorway with vehicle density 2.0.

A.2 PROMPTS EXAMPLE

DiLu’s prompts combine fixed instructions with scenario-specific information and retrieved experiences. Reasoning uses few-shot human-LLM dialogues, while reflection examines erroneous decisions and their faulty reasoning.

  • Reasoning prompts: The reasoning prompt contains system instructions, a scenario description, and few-shot experiences retrieved from memory.The system portion specifies the closed-loop task, input-output instructions, and response formatting.
  • Reasoning prompts: Each retrieved experience is a human-LLM dialogue pairing a scenario question with a correct or revised reasoning process and decision.These dialogues are inserted through few-shot prompting to support in-context learning.
  • Reasoning prompts: The reasoning template keeps grey-box prompts fixed while coloured prompts vary with the current scenario.The few-shot examples are demonstrated separately in Figure 9.
  • Reasoning prompts: DiLu retrieves the three most similar experiences, including two idle and one deceleration decision, and presents them as a human-LLM dialogue to GPT-3.5.
  • Reflection prompts: The reflection prompt combines fixed system instructions with the scenario and faulty reasoning associated with an erroneous decision.Its template is shown in Figure 10, where scenario-dependent prompts occupy the coloured region.

A.3 CASE STUDY

The case studies illustrate how DiLu reasons through action choices and uses reflection to repair unsafe decisions. The examples also situate evaluation in closed-loop traffic settings with varying vehicle density.

  • Reasoning module: The reasoning process evaluates actions sequentially by checking the vehicle ahead, following distance, relative speed, and lane-change feasibility.
  • Reflection module: After a right-lane change causes a collision with car 408, reflection identifies the overlooked relative distance and revises the decision using time to collision.The revised reasoning chooses deceleration because the time to collision is too short for a safe lane change.
  • Evaluation setting: Figure 13 reports episode rewards during GRAD training as averages over five independent runs, smoothed with a window of nine, with shadows showing standard deviations.
  • Evaluation setting: Figure 14 compares highway-env scenes across three traffic densities, where higher density produces more vehicles and reduced spacing and speed.
Loading 2309.16292v3…