PaperlayerView exact revision on arXiv

Source-linked AI summary

Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world

Eugene Vinitsky, Nathan Lichtlé, Xiaomeng Yang, Brandon Amos, Jakob Foerster

arXiv:2206.09889v3cs.MAcs.AIcs.LGcs.RO

TL;DR

Multi-agent driving research needs to address coordination under partial observability, including reaching expert final states without collisions and matching trajectories. Nocturne combines obstructed vector observations with real-world scenes and expert data; its tested baselines struggle in interactive scenes and remain far from human-level coordination.

  • Problem

    The benchmark asks whether policies can reach human experts’ final states without collisions and mimic their trajectories under partial observability and decentralized control.

    Source: §1 · p. 2

  • Method

    Nocturne combines obstructed vector observations and efficient 2D simulation with open-source real-world scenes and expert trajectories for multi-agent driving.

    Source: §1 · pp. 1–2

  • Results

    Tuned RL and imitation-learning baselines struggle to complete highly interactive scenes, while agents remain far from human-level coordination and achieve relatively low distance to expert trajectories.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Substantial work remains to build agents that reach human-level collision and goal rates and to learn such agents efficiently.

    Source: §6 · p. 9

    Waymo scenes can omit unseen vehicles and incomplete expert trajectories, while one-second context and an imperfect 80-meter perception model may make some scenes unsolvable.

    Source: §3.1 · p. 4, §3.4 · p. 6

Abstract

from arXiv · show

We introduce Nocturne, a new 2D driving simulator for investigating multi-agent coordination under partial observability. The focus of Nocturne is to enable research into inference and theory of mind in real-world multi-agent settings without the computational overhead of computer vision and feature extraction from images. Agents in this simulator only observe an obstructed view of the scene, mimicking human visual sensing constraints. Unlike existing benchmarks that are bottlenecked by rendering human-like observations directly using a camera input, Nocturne uses efficient intersection methods to compute a vectorized set of visible features in a C++ back-end, allowing the simulator to run at over 2000 steps-per-second. Using open-source trajectory and map data, we construct a simulator to load and replay arbitrary trajectories and scenes from real-world driving data. Using this environment, we benchmark reinforcement-learning and imitation-learning agents and demonstrate that the agents are quite far from human-level coordination ability and deviate significantly from the expert trajectories.

1 Introduction

Nocturne combines partially observed multi-agent driving with real-world scenes and expert data to study coordination and learning. The paper’s baselines struggle in interactive scenes, and the authors identify substantial distance from human-level coordination.

1 Introduction

  • Nocturne uses obstructed vector observations and efficient visibility checks with a C++ back-end, supporting thousands of simulator steps per second for mixed-motive multi-agent driving.The benchmark targets coordination and cooperation without requiring image-based feature extraction.Source: §1 · p. 1
  • Open-source driving data supplies diverse real-world scenes and expert trajectories for imitation, varying the number of controlled agents, and train-test evaluation of human-driver generalization.Scenes include intersections, roundabouts, parking lots, and highways.Source: §1 · p. 2
  • The benchmark asks policies to reach human experts’ final states with a 0% collision rate and also to mimic expert trajectories under partial observability and decentralized control.The authors identify high-dimensional states and many interacting agents as additional challenges.Source: §1 · p. 2
  • Tuned RL and imitation-learning baselines struggle to complete highly interactive scenes, while agents remain far from human-level coordination and achieve relatively low distance to expert trajectories.The authors also report no apparent zero-shot coordination problem at this level of agent capability.Source: §1 · p. 2

2 Related Work

Related work compares driving simulators by their interaction support, data, observability, and speed, and situates Nocturne among other partially observed benchmarks and open-source trajectory datasets.

2 Related Work

  • Driving simulators are compared by multi-agent interaction, data-driven design, 2D or 3D format, partial-observability mechanism, and observation-construction speed.Table 1 covers visible-object queries, expert data, and baseline human models.Source: §2 · p. 2, Table 1 · p. 4
  • Nocturne’s distinguishing feature is querying visible objects without camera rendering while computing agent dynamics at above 2000 steps-per-second.The comparison discusses BARK [5], SMARTS [42], and MetaDrive [22] as closely related driving simulators.Source: §2 · pp. 2–3
  • Hanabi [3] supports 2–5 agents; SMAC [32] lacks human data, while Melting Pot [20] offers mixed-motive challenges without expert data.The passage notes that many algorithms achieve perfect performance in SMAC [40].Source: §2 · p. 3
  • Nocturne uses Waymo Motion [12] for diverse scenes and cyclist, pedestrian, and traffic-light annotations; Argoverse 2 and nuScenes have similar features.The passage notes that nuScenes has complete traffic-light annotations rather than only ego-observed annotations.Source: §2 · p. 3
  • INTERACTION [41] spans cities and countries, capturing varied driving styles and norms, but is an order of magnitude smaller than other datasets.Source: §2 · p. 4

3 Benchmark construction

Nocturne builds data-driven driving scenes and evaluates decentralized control under standardized partial observability, with benchmark rules governing observations, actions, and episode conditions.

3.1 Defining a Nocturne Scene

  • Nocturne scenes combine map polylines, initial and final road-object positions, and optional trajectories, sourced in this version from Waymo Motion.The dataset contains nine-second snippets, with the first second intended as context and the remaining eight for prediction.Source: §3.1 · p. 4
  • Because Waymo labels only vehicles observed by its sensing car, NocturneV1 excludes traffic-light scenes and vehicles absent at episode start, leaving 134,453 snippets.The remaining scenes include roundabouts, unsignalized intersections, arterial roads, and parking lots.Source: §3.1 · p. 4

3.2 Partial Observability Model and Collision Handling

  • Visibility requires an unobstructed ray within a 120-degree, 80-meter view cone; the rectangular-vehicle model does not capture several complexities of human vision.Drivers can tilt their view, but the model omits effects such as dynamic field of view and seeing over vehicle hoods.Source: §3.2 · p. 5
  • A bounding-volume hierarchy and 2D range tree narrow visibility candidates before ray casting, and the same structures accelerate collision checks.The average scene contains about 30 vehicles, 4,800 road-edge points, and 16,000 road points.Source: §3.2 · p. 5

Definition of the state space

  • Agents may use rasterized or vectorized visible states; the default vectorization describes road objects and uses VectorNet for road points, with goals based on expert final states.Goal achievement requires position within 1 meter, speed within 1 m/s, and heading within 0.3 radians of the expert final state.Source: Definition of the state space · p. 5

Action Space

  • Actions discretize acceleration, steering, and head tilt, while benchmark rules bound acceleration, heading changes to 40 degrees per second, and head tilt to 90 degrees each way.The experimental action sets use 6 acceleration, 21 steering, and 5 head-tilt actions.Source: Action Space · p. 5, §3.4 · p. 6

3.4 Rules of the Benchmark

  • Control begins after a one-second expert warm-up; episodes last eight seconds, and extra information may shape training but only directly observable inputs may be used at test time.The rules may leave some scenes unsolvable under the limited prior context and modeled viewing distance.Source: Environment Dynamics and Goals · p. 6, §3.4 · p. 6

3.5 Unusual features of the Benchmark

4 Experiments Setup

The experiments compare reinforcement-learning and imitation-learning baselines, using a dense progress reward for reinforcement learning and common validation scenes and rollout conditions.

Experiments Setup

  • The experiments test APPO with a shared decentralized policy and a behavior-cloning baseline.The paper also briefly investigates whether policies trained in different runs coordinate poorly when paired.Source: §4 · p. 7
  • Figure 3 reports training-set goal success against environment steps, alongside the percentages of agents reaching goals and colliding.Its training-file labels indicate the number of fixed training scenes used for a policy.Source: Figure 3 · p. 8
  • The reinforcement-learning baseline adds a dense reward to the goal bonus to encourage progress toward target position, speed, and heading.The authors note that this reward affects the optimal policy and is used here to generate good policies, not required by the benchmark.Source: §4 · p. 8
  • All experiments use the same 200 randomly selected validation scenes and an eight-second rollout, with plot widths showing twice the standard error.Method and architecture hyperparameters are provided in Appendix Sec. A.Source: §4 · p. 8

5 Results and Analysis

Across the results, agents struggle most when reaching a goal requires coordination, while more training data narrows the train-test gap and displacement-error gains flatten after 1,000 files.

5.1 Success rate of baselines

  • Goal rate measures vehicles reaching their goals and collision rate measures vehicles colliding with another vehicle or road edge; Table 2 compares APPO and BC, while Figure 4 reports average and final displacement error.The experiments disable removal of vehicles after collision or reaching a goal for displacement-error analysis.Source: §5.1 · p. 8, §5.2 · p. 8, Table 2 · p. 9
  • Adding training data lowers performance through 1,000 files but narrows the train-test gap, closing it at 10,000 files.The passage notes that train-test divergence could re-emerge as agents approach a 100% goal rate on training scenes.Source: §5.1 · p. 8

5.2 Human-agent trajectory similarity

  • The influence of additional training data on displacement error is flat after 1,000 files, although comparisons with other works are difficult.Agents receive goal positions, and expert scenes may include pedestrians or cyclists absent when agents are replayed, so low displacement error may reflect scenes with unique optima.Source: §5.2 · p. 9

5.3 Policy Failure Modes

  • Goal rates decline precipitously and collision rates rise sharply as expert-trajectory intersections increase, indicating poor performance when reaching goals requires coordination.The proxy counts intersections between expert trajectory polylines; close to 25% of vehicles have at least one interaction.Source: §5.3 · p. 9

6 Conclusion

Nocturne is a benchmark for studying human-like decentralized driving coordination, with RL and imitation-learning results that leave substantial challenges in agent performance and learning efficiency.

6 Conclusion

  • Nocturne provides a simulator and benchmark for studying human-like decentralized coordination in driving systems.The conclusion says further work is needed for agents to achieve human collision and goal rates and to learn efficiently.Source: §6 · p. 9
  • Figure 5 plots goal and collision rates against counts of intersections between expert trajectories, grouping scenes with more than three intersections in the three-interaction bin.The proxy captures some interactions, such as crossings and merges, but not complementary left turns and may count following another vehicle.Source: Figure 5 · p. 10
  • Human-like rules and conventions may emerge as properties of driving safely in these settings when agents improve.This is presented as a possibility, not an established result.Source: §6 · p. 10
  • Ego-centric data collection requires removing vehicles after they reach their recorded goals; generative mechanisms could extend trajectories and complete traffic-light states.Such completion could enable inclusion of traffic-light scenes.Source: §6 · p. 10
  • Using Nocturne agents to predict human driving requires a method to infer goals from the one-second context, since agents currently require a goal.The conclusion identifies supervised goal prediction as a future-work direction for decentralized prediction from egocentric observations.Source: §6 · p. 10

7 Reproducibility and Ethical Statement

The benchmark and files needed to run it are publicly available, but trained models are not released because the Waymo Motion dataset restricts their release.

7 Reproducibility and Ethical Statement

  • The benchmark and its required files are publicly available, but trained models are not released because the Waymo Motion dataset restricts their release.The work uses trajectory data, not the dataset's anonymized images.Source: §7 · p. 10

A.1 Architecture and computational resources

The appendix describes APPO and behavior-cloning architectures and compute, their hyperparameters, and a configurable high-dimensional state representation built from visible scene elements.

A.1 Architecture and computational resources

  • APPO experiments run for two days on one V100 GPU and 10 CPUs across five seeds, totaling 1,200 GPU hours and 12,000 CPU hours for RL.Imitation-learning experiments use five seeds for three hours, totaling 15 GPU hours and 100 CPU hours.Source: §A.1 · p. 14
  • APPO uses a two-layer 256-unit Tanh network followed by a 256-state GRU; behavior cloning stacks five states into a three-layer network with acceleration and steering heads.The BC network dimensions are 1025, 256, and 128; its acceleration head has 15 bins and its steering head has 43 bins.Source: §A.1 · p. 14
  • Because BC has no head-tilt action, its visibility cone is extended to π radians, violating the benchmark's cone-shape rule.Adding head tilt to imitation agents remains an open question.Source: §A.1 · p. 14

A.2 Hyperparameters

  • APPO uses a batch size of 7,168 and observations scaled by 10.0; imitation learning uses a 3 · 10^-4 learning rate, batch size 512, and five stacked states.Source: §A.2 · p. 14

A.3 State space Details

  • The state representation sorts visible road points, objects, and stop signs by distance, returning up to 500, 16, and 4 respectively for a 6,727-dimensional state.Counts are configurable; the state is intentionally not pruned, and utilities support querying observed road points and objects.Source: §A.3 · p. 14

A.4 Features

Nocturne represents ego vehicles, other road users, and road geometry through structured features in the observing vehicle’s frame. Cross-seed tests show no apparent zero-shot coordination issue at the agents’ current capability.

A.4 Features

  • Ego features encode speed, goal distance and angle, dimensions, and deviation from target speed and heading.Source: §A.4 · p. 14
  • Road-object features encode speed, relative motion and pose, dimensions, and whether an object is a vehicle, pedestrian, or cyclist.Source: §A.4 · p. 14

B Zero-shot coordination

  • Across five seeds and 100 shared test files per pair, cross-seed play shows no apparent zero-shot coordination issue at current capability.The authors note that incompatible conventions could emerge as agents become more capable.Source: Figure 6 · p. 15, B Zero-shot coordination · p. 15

A.4 Features

  • Road points use approximately 0.5-meter discretized polylines, neighbor vectors, ego-relative positions, and point-type labels.Source: §A.4 · p. 15

C Ablation of the targets

Relaxing the target-speed requirement worsens reported collision, goal-rate, and trajectory-error outcomes. The appendix also specifies the vehicle dynamics and a procedure for estimating infeasible goals.

C Ablation of the targets

  • Relaxing target speed sharply raises collision rate, lowers goal rate versus APPO, and increases ADE and FDE.The authors suggest the speed target may act as a hint or curriculum; the ablation uses 10,000 training files to limit computation.Source: C Ablation of the targets · pp. 15–16, Table 3 · p. 16

D Vehicle Model

  • The vehicle model uses a center-of-gravity kinematic bicycle model, assumes lr = 0.5L, and steps heading and clipped speed.Source: D Vehicle Model · p. 16

E Calculating Infeasible Goal Statistics

F Computing the simulator speed

Nocturne’s speed evaluation measures observation computation and simulation stepping for single agents and groups. The reported results show high single-agent throughput and roughly linear multi-agent scaling; the code and dataset have open licenses.

F Computing the simulator speed

G License Details and Accessibility

Related papers

Loading 2206.09889v3…