PaperlayerView exact revision on arXiv

Source-linked AI summary

Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving

Baojie Chen, Zijun Jia, Jing Zhong

arXiv:2609.08217v1cs.CVcs.RO

TL;DR

Autonomous-driving VLM reasoning lacks evolving scene state, active evidence-seeking, and accumulated cross-scene experience. The paper combines hierarchical memory with online tool use and offline consolidation, and reports strong DriveLMM-o1 results and cross-benchmark generalization.

  • Problem

    Existing approaches lack dynamic scene state, cross-scene driving experience, and active evidence-seeking guided by evidence gaps.

    Source: §1 · p. 2

  • Method

    The framework combines scene-level short-term memory and retrieved long-term experience with adaptive tool use, offline memory consolidation, and training on verified trajectories using SFT and GRPO.

    Source: §1 · p. 2

  • Results

    80.03 reasoning score and 79.09% MCQ accuracy on DriveLMM-o1; the model exceeds same-scale AgentThink by 7.74 MCQ points and generalizes across benchmarks.

    Source: Results on DriveLMM-o1 · p. 5, Cross-Benchmark Generalization · p. 5

  • Takeaways & Limitations

    The results support moving autonomous-driving VLMs toward adaptive systems that accumulate experience and interact with external evidence.

    Source: §5 · p. 7

    Approximately 35 s median latency per query makes the framework suited to offline scene understanding, annotation, and simulation evaluation rather than a real-time driving stack.

    Source: F Limitations · p. 18

Abstract

from arXiv · show

VLMs have shown promise for autonomous driving, yet still suffer from hallucination, weak spatio-temporal perception, and limited generalization. Recent methods improve reasoning and decision-making through CoT explanations, retrieval-augmented generation or the static injection of tool outputs. Although these mechanisms enrich the context, the model neither proactively perceives scene information nor accumulates experience after answering. To overcome these limitations, we present, to our knowledge, the first synergistic framework that tightly couples hierarchical memory with proactive tool invocation in a closed reasoning loop. Our contributions are threefold. (i) Hierarchical Driving Memory: a scene-level short-term memory maintains the dynamic scene state, and an evolving long-term memory retrieves reusable experience and tool strategies. (ii) Memory-Tool Synergistic Reasoning Framework: guided by the scene state and retrieved experience, the model adaptively invokes tools to refine its reasoning at inference time and consolidates reusable experience into a long-term memory pool offline. (iii) Data Generation and Two-stage Training Pipeline: verified memory-tool trajectories built by multi-step teacher rollout are used to train with SFT and GRPO. Our 7B model reaches an overall reasoning score of 80.03 and MCQ accuracy of 79.09% on DriveLMM-o1, surpassing the strongest baseline by 7.74 MCQ points and generalizes strongly across benchmarks. Notably, ablation and analysis studies validate the effectiveness of each component and further reveal the complementary roles of hierarchical memory. Short-term memory strengthens spatio-temporal understanding, improving STSBench accuracy by 24.2 points, while offline long-term memory consolidation yields an additional 3.57-point MCQ gain with all parameters frozen, demonstrating continual self-evolution through accumulated driving experience.

1 Introduction

The paper argues that autonomous-driving reasoning needs evolving scene awareness, active evidence gathering, and experience accumulated across scenes. It proposes a hierarchical-memory framework with proactive tool use and reports strong benchmark results for its 7B model.

1 Introduction

  • Current methods lack dynamic scene state, cross-scene driving experience, and evidence-seeking guided by information gaps.The paper frames these as missing features of a unified reasoning process combining spatio-temporal awareness, proactive perception, and experience accumulation.Source: §1 · p. 2
  • Across three driving benchmarks, the proposed tool-grounded synergistic reasoning framework over hierarchical memory enables its 7B model to outperform the strongest baseline and generalize robustly without further training.Source: §1 · p. 2
  • The framework combines scene-level short-term memory, cross-scene long-term memory, adaptive tool use, and offline consolidation of verified experience.Its training pipeline generates verified memory-tool trajectories through multi-step teacher rollouts and uses SFT followed by GRPO.Source: §1 · p. 2

2 Related Work

Related work spans autonomous-driving VLMs and increasingly demanding benchmarks, then considers reasoning methods that add explanations, retrieved context, or tool outputs. The paper identifies passive information intake as a limitation of these approaches.

2.1 VLMs and Benchmarks for Autonomous Driving

  • Benchmarks have expanded from perception and free-form QA to risk assessment, explicit reasoning, and spatio-temporal understanding, while weaknesses remain in motion trends, interactions, grounding, and safety-critical reasoning.DriveM-LLM evaluates spatial understanding, while STSBench emphasizes spatio-temporal scenario understanding.Source: §2.1 · p. 2

2.2 External Augmentation in Reasoning for Autonomous Driving

  • CoT, retrieval augmentation, and tool-output injection add reasoning steps or external information, but the paper characterizes these methods as passive rather than evidence-responsive.The related work includes AgentThink, which adds tool outputs to context to improve performance and alleviate hallucinations (Qian et al. 2025).Source: §2.2 · p. 2

3 Methodology

The method combines hierarchical memory with iterative tool-grounded reasoning, then trains the policy on verified memory-tool trajectories. Scene context and retrieved experience inform tool use, while the transient belief changes as observations arrive.

3.1 Overview

  • Given a question and six-view observation, the policy conditions on scene-level short-term memory and retrieved long-term-memory entries, then iteratively seeks missing evidence, invokes tools, and updates its scene belief.The loop continues until the model judges the evidence sufficient to answer.Source: §3.1 · p. 2

3.2 Hierarchical Driving Memory

  • Short-term memory represents the current scene using two adjacent six-view observations organized as a graph of entities and their spatial, temporal, and interaction relations.Long-term memory stores reusable experience from verified successful trajectories; retrieval ranks entries using semantic relevance and normalized utility, and the retrieved entries serve as priors rather than direct answer hints.Source: Scene-Level Short-Term Memory · p. 3

3.3 Memory-Tool Synergistic Reasoning

  • Within each query, the model follows a Think–Action–Observation–Update loop without a predefined tool sequence, evolving a transient scene belief while both memories remain fixed.Tool observations are incorporated until the model emits a finish action and answers.Source: §3.3 · p. 4

3.4 Data Generation and Two-Stage Post-Training

4 Experiments

The experiments evaluate reasoning, generalization, component contributions, and offline memory consolidation. Results show strong benchmark performance and gains from hierarchical memory, temporal state organization, and accumulated experience.

4.1 Experimental Setup

4.2 Main Results

  • 80.03 reasoning score and 79.09% MCQ accuracy make the framework best on DriveLMM-o1, exceeding same-scale AgentThink by 7.74 MCQ points.It also ranks first on Rule Adherence and Scene Awareness; a faithful comparison with DriveAgent-R1 was unavailable because its checkpoints and evaluation assets were not fully released.Source: Results on DriveLMM-o1 · p. 5, Table 1 · p. 6
  • DriveMLLM zero-/one-shot AccS is 74.36/80.59, and unseen STRIDE-QA yields the best open-model MLSR of 3.9.On DriveBench qualitative cases, tool evidence and retrieved experience ground answers under visual corruption, including a 41.0 m estimate against 45.22 m ground truth.Source: Cross-Benchmark Generalization · p. 5, Qualitative Analysis · p. 5, Figure 4 · p. 6

4.3 Ablation Study

  • Two-stage SFT raises reasoning/MCQ from 51.77/37.81 to 73.91/67.06, while adding GRPO yields 80.03/79.09.GRPO on the base model reaches only 73.85/64.49, below SFT in MCQ; the authors describe SFT as the prerequisite for GRPO refinement.Source: Post-training stages · p. 5, Table 5 · p. 7
  • Combining STM and LTM gives the best MCQ accuracy at 79.09%, versus 72.16% without memory, 75.20% with LTM alone, and 76.91% with STM alone.The authors attribute STM's larger contribution to questions depending on current scene state and describe LTM as adding reusable experience.Source: Hierarchical memory · p. 6, Table 5 · p. 7
  • On STSBench, temporal STM reaches 55.61%, compared with 50.36% for static STM, 35.64% for raw six-frame input, and 31.38% for the base model.The authors conclude that explicit temporal-state organization is more effective than simply increasing visual context.Source: Temporal Representation Ablation · p. 6, Table 4 · p. 7

4.4 Behavioral and Evolutionary Analysis

  • 97.4% of queries use one to three tools, averaging 2.10 calls; no-tool answering occurs in 0.5% of cases.Tool execution succeeds in 99.74% of calls, and 98.94% of trajectories terminate normally, indicating variable-length rather than fixed-schedule interaction.Source: Online tool behavior · pp. 6–7, Figure 5 · p. 7
  • With all model parameters frozen, sleep-time LTM consolidation raises MCQ from 79.09% to 82.66% by generation 9, a 3.57-point gain.A slight generation-10 decline indicates saturation; consolidation uses training-split and additional non-benchmark scenes, with no test information.Source: Offline LTM evolution · p. 7, Figure 5 · p. 7

5 Conclusion

The paper presents hierarchical memory and adaptive evidence acquisition for autonomous-driving reasoning. It argues for adaptive systems that accumulate experience and interact with external evidence, and identifies scalable consolidation and open-world closed-loop driving as future work.

5 Conclusion

  • The framework combines hierarchical memory, foresight and hindsight, and adaptive evidence acquisition for autonomous-driving reasoning.The authors argue that autonomous-driving VLMs should move toward adaptive systems that accumulate experience and interact with external evidence.Source: §5 · p. 7
  • Future work will explore more scalable memory consolidation and extension toward open-world closed-loop driving.Source: §5 · p. 7
  • The experimental results highlight the importance of systems that accumulate experience and interact with external evidence.Source: §5 · p. 7

A Additional Method Details

The appendix describes how scene-level memory is initialized and refreshed, how cross-scene experience is stored and retrieved, and the callable tools used to add structured observations during reasoning.

A.1 Scene-Level Short-Term Memory Construction

  • The scene-level short-term memory represents the current driving scene as a compact graph of ego state, key objects, spatial sectors, distance bands, motion states, risks, and inter-object relations, refreshed only when a new pair of adjacent observations becomes available.Source: §A.1 · p. 10
  • The scene graph represents ego state, objects, spatial sectors, distance bands, motion, risks, and relations, and refreshes when adjacent observations arrive.Adjacent-timestamp multi-view images initialize the graph; the memory is a read-only scene prior during each reasoning episode.Source: §A.1 · p. 10

A.2 Cross-Scene Long-Term Memory

  • Long-term memory starts with 23,388 OmniDrive-derived entries and adds reusable success patterns and tool strategies abstracted from verified trajectories.Source: §A.2 · p. 10
  • Retrieval embeds the question with scene memory, ranks entries by cosine similarity, returns the top four, and excludes entries from the query's own scene.The returned view strips identity fields and teacher decisions; the store is read-only during benchmark evaluation and updated only in offline consolidation.Source: §A.2 · p. 10

A.3 Driving Tool Library

  • The 27-function library spans seven driving-tool categories, with 20 core tools used in the reported experiments.The functions return structured observations for incorporation into the transient scene belief.Source: §A.3 · p. 10
  • Open-vocabulary detection and instance selection support target grounding, while metric tools estimate 3D location, depth, pairwise distance, and depth order.Monocular depth is approximate and less reliable for long-range targets; pairwise estimates use the same image for relative-scale consistency.Source: §A.3 · p. 10
  • Detection tools return leading, surrounding, front-region, or specified-region objects, while ego-state and forecasting tools provide kinematics, recent trajectory, goals, and predicted waypoints.Source: §A.3 · p. 11
  • Occupancy and map tools query future occupancy, drivable regions, lane features, road edges, and pedestrian-crossing locations.Source: §A.3 · p. 11

B Data Generation and Post-Training Details

The pipeline generates gold-conditioned teacher trajectories with executed tool observations, validates them, and converts verified traces into step-level supervision for two SFT phases.

B.1 Multi-Step Teacher Rollout and Validation

  • The teacher receives the gold answer to anchor trajectory synthesis, while tool observations come from real execution at each step.Source: §B.1 · p. 11
  • Rollouts are checked for action validity, observation provenance, grounding, reasoning-observation consistency, answer correctness, and step or redundancy limits; failures are regenerated.Source: §B.1 · p. 11
  • The teacher run is documented but cannot be replayed bit-exactly because its provider-side model alias may resolve to a different underlying build.The teacher emitted its trace under a JSON response-format constraint without extended-thinking mode; sampling parameters were left at provider defaults.Source: §B.1 · p. 11
  • The teacher's registry-validated tool-call schema is shared with inference, so validated trajectories are executable by the student without translation.Source: §B.1 · p. 11

B.2 Stepwise Conversion and Two-Phase SFT

  • Verified trajectories are decomposed into next-step examples for tool decisions and observation-grounded reasoning, with environment observations masked from the loss.Phase 1 yields 13,679 tool-decision records; Phase 2 yields 20,062 records with complete memory-conditioned contexts and executed observations.Source: §B.2 · pp. 11–12
  • Phase 1 learns valid tool selection and argument generation, then Phase 2 continues from its adapter to learn observation-grounded reasoning.The setup adapts Qwen2.5-VL-7B-Instruct with LoRA while freezing the vision encoder and multimodal aligner.Source: §B.2 · p. 12

B.3 GRPO and Reward Design

The GRPO reward combines answer correctness, legal tool use, and memory citation, with tool-use shaping intended to bootstrap interaction. The reward scheme nevertheless applies no direct reinforcement pressure to open-ended reasoning quality.

B.3 GRPO and Reward Design

  • GRPO scores correctness, legal tool use, and memory citation, with the memory bonus available only for correct answers.Memory citation requires at least two concrete scene-state or experience phrases in the reasoning.Source: §B.3 · p. 12
  • Incorrect trajectories with at least one legal tool call receive a small positive reward to bootstrap tool-interface use and discourage bypassing interaction.The authors say this helps when the policy cannot yet answer and avoids making direct, tool-free answers dominant after supervised warm-up.Source: §B.3 · p. 12
  • The shaping reward is flat in call count, dominated by a correct answer, and yields positive advantage only when a prompt's sample group has no correct trajectory.The authors state that it is active on prompts where the policy has not yet learned to succeed and inactive elsewhere.Source: §B.3 · pp. 12–13
  • Because open-ended items fail the multiple-choice correctness test, GRPO directly pressures multiple-choice correctness and tool discipline but not open-ended reasoning quality.Source: §B.3 · p. 13

B.4 Training and Compute Configuration

The appendix specifies the training and inference configuration, then defines the DriveLMM-o1 benchmark and its reasoning and answer-accuracy evaluation protocols.

B.4 Training and Compute Configuration

  • Inference uses a Think–Action–Observation–Update loop with real tool execution, at most seven tool turns, and a separate terminal answer turn.GRPO uses training questions disjoint from the SFT subset; evaluation freezes model parameters and both memory channels.Source: §B.4 · p. 13
  • Trainer-side randomness is seeded at 42, but third-party teacher and judge APIs and unseeded reinforcement rollouts prevent full bit-reproducibility.Inference is deterministic for a fixed checkpoint; on-policy reinforcement groups may vary across reruns.Source: §B.4 · p. 13
  • Hyperparameters were fixed by compute budget without grid or random search, and checkpoints were selected on a held-out training-file validation partition.The reported search omissions include learning rate, adapter rank, retrieval depth, group size, and interaction budget.Source: §B.4 · p. 13

C.1 DriveLMM-o1 Protocol

  • DriveLMM-o1 contains over 18K training and over 4K test VQA examples, with 4,634 test questions across 539 keyframes from 115 nuScenes scenes.Each sample provides six surround-view images, and the benchmark supplies curated intermediate reasoning steps.Source: §C.1 · p. 14
  • The six views are stitched into a declared 2×3 grid, and the official reasoning instruction is reused for both the model and evaluation harness.Views follow the canonical order from FRONT_LEFT through BACK_LEFT.Source: §C.1 · p. 14
  • Reasoning is judged on twelve 1–10 dimensions, while MCQ accuracy is the mean over 2,391 official-partition items and unparseable answers count incorrect.Table 1 reports five dimensions individually; its “Missing” column denotes Missing Details.Source: §C.1 · p. 14

C.2 Cross-Benchmark Evaluation

The section defines cross-benchmark protocols and leakage safeguards, describes teacher and judge procedures, and reports efficiency and confidence-interval analyses. The answer-accuracy advantage is robust, while overall reasoning is at parity with the strongest baseline.

C.2 Cross-Benchmark Evaluation

  • Evaluation covers single-view spatial tasks on DriveMLLM, temporal multiple-choice reasoning on STSBench, and a disjoint-dataset forecasting test on STRIDE-QA.DriveMLLM uses eight spatial tasks and an unweighted aggregate; STSBench contains 971 questions, while STRIDE-QA has 5,317 QA pairs over 409 scene groups.Source: §C.2 · p. 14

C.3 Leakage Control

  • Training uses DriveLMM-o1 training-split data, while memory entries from shared-corpus benchmarks are restricted to training scenes and evaluation retrieval excludes the query’s own scene.Evaluation memory is read-only, and teacher decisions and dataset identity tokens are stripped from retrieved views.Source: §C.3 · p. 15

D.1 Teacher Rollout Prompts

  • Teacher trajectories use three stages: tool-blind action planning, observation-aware next-step decisions, and trace summarization into memory updates and reasoning.All stages are conditioned on the gold answer, so traces explain a known answer rather than independently attempting it.Source: §D.1 · pp. 15–16

D.2 LLM-Judge Prompt for DriveLMM-o1

  • The judge scores responses on 1–10 dimensions including reasoning alignment and completeness, risk, traffic-rule adherence, scene understanding, and hallucination.The prompt also defines criteria for repetition, semantic coverage, commonsense, missing steps, relevance, and missing details.Source: §D.2 · p. 16

E.1 Tool-Use Behavior and Efficiency

  • Median end-to-end latency is 34.96 s per query, while tool execution adds only 5.4 ms at the median; peak memory is approximately 19 GB on DriveLMM-o1 and 24 GB on DriveMLLM.The authors attribute latency primarily to language-model decoding and describe the overhead from the memory and tool loop as negligible relative to decoding.Source: §E.1 · p. 17, Table 11 · p. 17, Table 12 · p. 17

E.2 Confidence Intervals

  • 71.35 MCQ is the strongest published baseline; the answer-accuracy advantage is robust, while overall reasoning is at parity.The strongest baseline reasoning score, 79.68, falls inside the model’s interval [79.46, 80.57]; bootstrap intervals use 10,000 test-question resamples.Source: §E.2 · pp. 17–18, Table 14 · p. 18

E.3 Other Qualitative Results

The qualitative cases illustrate retrieved experience steering evidence selection and tool use, while tool observations ground answers in geometry and vehicle state.

E.3 Other Qualitative Results

  • On STRIDE-QA, camera-frame 3D localization estimates distances of 13.9 m and 8.8 m against references of 13.26 m and 7.57 m.The baseline’s image-perspective estimates are 5 m and 18 m; on DriveMLLM, retrieved strategies guide pairwise-distance and relative-position evidence selection.Source: §E.3 · p. 18
  • Five cases reinforce retrieved-strategy utility, while the fog case adds a memory entry against decelerating for fog alone before checking speed and nearby hazards.Across the examples, the estimates remain approximate.Source: §E.3 · p. 18

F Limitations

The authors bound the system’s intended use and identify limits in training-run evidence and offline memory consolidation.

F Limitations

  • Approximately 35 s median latency per query makes the framework suited to offline analysis, annotation, and simulation evaluation rather than a real-time driving stack.The paper explicitly does not claim vehicle readiness.Source: F Limitations · p. 18
  • Reported intervals capture test-set sampling variability, not variation across independent training runs, and reinforcement rollouts are not bit-reproducible.The pipeline was trained once.Source: F Limitations · p. 18
  • Offline consolidation improves accuracy for nine generations and then declines, but this pattern comes from a single trajectory with uncharacterized variance or asymptote.The authors identify retrieval reliability and belief-conflict resolution as next steps.Source: F Limitations · p. 18
Loading 2609.08217v1…