PaperlayerView exact revision on arXiv

Source-linked AI summary

Pictura: Perspective-View Self-Play at Scale for Driving

Yuan Yin, Elias Ramzi, Marc Lafon, Valentin Charraut, Victor Bares, Yihong Xu, Éloi Zablocki, Alexandre Boulch, Thibault Buhet, Andrei Bursuc, Matthieu Cord

arXiv:2607.26005v1cs.CVcs.AIcs.RO

TL;DR

Privileged-state self-play leaves a gap with the partial perspective observations available to deployed vehicles. Pictura renders egocentric views inside the self-play loop, and Alberti approaches its privileged counterpart’s performance while transferring zero-shot to WOMD layouts. The rasterized views do not yet match real camera images; appearance alignment is left to future work.

  • Problem

    Large-scale self-play commonly uses privileged simulator states, creating a gap with the partial camera observations available to deployed vehicles.

    Source: §1 · p. 1

  • Method

    Pictura renders each agent’s rasterized egocentric view at every step, allowing Alberti to train through multi-agent self-play PPO directly from images.

    Source: §1 · p. 3, §3.3 · p. 7

  • Results

    Alberti approaches its privileged vectorized counterpart’s performance and transfers zero-shot to WOMD layouts.

    Source: §1 · p. 3, §6 · p. 14

  • Takeaways & Limitations

    Large-scale driving self-play can run directly on perspective images without privileged observations.

    Source: §6 · p. 14

    Pictura’s rasterized frames differ in appearance from real camera images; aligning them is left to future work.

    Source: §1 · p. 3

Abstract

from arXiv · show

Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/

1 Introduction

Pictura brings driving self-play into the perspective view, addressing the gap between privileged simulator-state policies and what a deployed camera can observe. Alberti trains directly on rendered views and approaches its privileged counterpart’s performance while transferring zero-shot to real-world-derived layouts.

1 Introduction

  • Self-play can generate increasingly hard scenarios and learn from action consequences; prior work reports robust behavior and zero-shot transfer without human demonstrations.The prior transfer claim concerns maps and agents unseen during training.Source: §1 · pp. 1–2
  • Privileged vector observations expose occluded agents and exact velocities, while camera-input students can inherit decisions they cannot justify from their own view.The cited failure mode is documented by LEAD and corroborated by the authors’ analysis.Source: §1 · p. 2
  • Perspective-view self-play preserves camera observability while dropping appearance through a cheap rasterized view, rather than relying on photorealistic rendering.Aligning rendered appearance with real imagery is described as a complementary step left to future work.Source: §1 · pp. 2–3
  • Pictura renders each agent’s egocentric view at every step, enabling Alberti to learn perception and control jointly through self-play without a privileged teacher or distillation.Its custom CUDA rasterizer reaches 500 K agent-steps/s (2 M images/s) on a single H100.Source: §1 · p. 3
  • Alberti approaches its privileged vectorized counterpart’s performance and transfers zero-shot to real-world-derived WOMD layouts.The layouts use road maps, agent initial states, and goals drawn from WOMD logged driving and are re-rendered through Pictura.Source: §1 · p. 3

2 Related Work

Prior driving self-play and simulator work established robust policies at scale, but fast vectorized simulators expose privileged state while image-rendering simulators do not reach self-play throughput. Pictura instead makes perspective rasterization practical inside the learning loop.

2 Related Work

  • Driving self-play has been used for negotiation, human-convention regularization [6], and teacher-generated challenging scenarios for long-tail robustness.Self-play is a classic multi-agent RL recipe.Source: §2 · p. 3
  • Gigaflow’s policy achieves state-of-the-art results on CARLA, nuPlan [2], and WOMD without real-world data or annotations, and transfers zero-shot to unseen maps in a simulator stepping billions of transitions.Every agent acts on vectorized privileged state that very few to no on-board sensors can supply.Source: §2 · p. 4
  • Vectorized simulators reach millions of state transitions per second but expose abstract state, while image-rendering simulators do not approach self-play’s required environment counts.The paper states that no existing driving simulator delivers camera-like observations at vectorized throughput.Source: §2 · p. 4
  • Privileged-agent distillation and adapters train camera students to reproduce decisions based on unseen information [25]; Pictura trains the camera-input policy directly.The cited concern is that the student imitates decisions depending on what it cannot see.Source: §2 · p. 4
  • Prior raster use includes conditioning generative models [24] and RAP’s imitation-learning augmentation; Pictura rasterizes on GPU compute cores for scalable self-play.Gigapixel [25] uses Madrona’s batch renderer through the graphics pipeline, whose capacity is reduced on modern computation-oriented hardware.Source: §2 · p. 4

3 Pictura: A Simulator for Perspective-View Self-Play

Pictura layers a GPU rasterizer and image-based policy over a vectorized driving simulator, allowing agents to learn from their own egocentric views in self-play. Its simulator, rendered observations, and PPO training loop are designed to operate together.

3.1 A Gigaflow-Style Vectorized Simulator

  • The vectorized core supports large-scale self-play with diverse scenarios, shared control of vehicles, pedestrians, and cyclists, lane-consistent goals, and terminal infractions.Reward weights and control dynamics are resampled at episode resets and included in each agent’s observation.Source: §3.1 · pp. 5–6

3.2 Perspective Rendering

  • Pictura renders each agent’s egocentric camera view from surrounding agents and map geometry using a lightweight, flat-shaded rasterizer.Its scene includes depth cues, approach-specific traffic signals, occluding walls, parked cars, and depth-tapered road markings; analytic antialiasing keeps thin lane lines visible at low resolution.Source: §3.2 · pp. 6–7
  • A custom CUDA kernel renders views on general-purpose GPU compute cores, keeping images on-device and enabling use on CUDA systems without a graphics stack.Rendering uses the same general-purpose cores as the network, avoiding a host round-trip.Source: §3.2 · p. 7

3.3 Perspective-View Self-Play

  • Alberti is trained from scratch with multi-agent self-play PPO on rendered views, jointly optimizing its image encoder and driving policy without privileged supervision.A ConvNet encodes each camera view, and learned query tokens cross-attend to pool its feature maps; ego state and conditioning remain vector inputs.Source: §3.3 · pp. 7–8

4 Rendering Efficiency Evaluation

Pictura’s CUDA rasterizer improves rendering throughput over prior systems and uses a smaller share of training time than the Madrona reimplementation. The comparison isolates renderer performance and shows the advantage grows on newer hardware.

Throughput against prior renderers

  • 1.4–4.1× faster than Gigapixel’s fastest configuration, Pictura’s CUDA rasterizer leads the prior renderer comparison at matched square resolution and wall-free settings.The Madrona reimplementation rasterizes identical scenes, isolating engine differences; later comparisons use that reimplementation rather than the Gigapixel simulator.Source: Figure 5 · p. 9, §4 · p. 9

Self-play free of the renderer bottleneck

  • About 10% of a Pictura training iteration is rendering, versus over half for Madrona, whose full iteration is 2–3× slower.At 384 × 216, Madrona runs out of memory; the serial micro-benchmark puts the CPU simulator on the critical path, unlike real training, which overlaps it with compute.Source: Figure 5 · p. 9, §4 · pp. 9–10

Scaling with hardware

  • 1.5× faster from A100L to H100, Pictura’s CUDA rasterizer widens its margin over Madrona from 2.2–5.0× to 10–19×.Madrona falls to 0.2–0.7× of its A100L throughput; Pictura runs on general-purpose compute cores, while Madrona uses the fixed-function graphics pipeline.Source: §4 · p. 9

5 Policy Evaluation

The evaluation compares camera-based Alberti with matched vectorized baselines in CARLA self-play and zero-shot WOMD layouts, then probes whether its decisions track visual evidence. Aggregate performance and occlusion analyses cover both driving outcomes and camera grounding.

5.1 Driving Performance at Scale

  • At full WOMD density, Alberti reaches 86% of goals versus 72% for the full privileged agent and 59% for Vectorized†, with 2.5× fewer collisions and 4.8× less off-road than the full agent.The lead holds across 100%, 75%, and 50% road-user densities.Source: Table 1 · p. 11, §5.1 · pp. 11–12
  • At medium density, Alberti nearly matches Vectorized† on off-road rate (0.018 vs. 0.020) and goals reached (11.21 vs. 12.81), but has more collisions (0.060 vs. 0.037).The reported red-light violation rates are 0.058 vs. 0.009, and distance per infraction is 11.61 vs. 26.99 km.Source: §5.1 · pp. 10–11, Table 1 · p. 11
  • Safety metrics remain stable across render resolutions, while goals and distance traveled per infraction improve; throughput falls from ~1.5M agent-steps/s at 32×18 to ~284K at 384×216.The default 96×54 resolution runs at ~1.3M agent-steps/s; each resolution uses a separately trained 10 B-step policy.Source: §5.1 · p. 12, Figure 7 · p. 12

5.2 Grounding in What the Cameras See

  • Across 88K removals in 16 scenes, Alberti’s value and braking-probability responses decrease with increasing occlusion and reach zero when a road user is fully hidden, unlike the privileged policy.In the controlled pedestrian sweep, Alberti’s braking response peaks near 20 m when the pedestrian is fully visible.Source: §5.2 · pp. 12–13, Figure 8 · p. 13
  • At blind corners, Alberti establishes more distance from hidden oncoming traffic before reveal and remains cautious at empty corners, while the privileged agent accelerates there; post-reveal deceleration is similar.In an example takeover, Alberti stops before the corner and proceeds cautiously until the vehicle becomes visible.Source: §5.2 · pp. 13–14, Figure 9 · p. 14

6 Conclusion

The conclusion argues that large-scale driving self-play can run directly on perspective images, while Appendix A outlines the simulator and training details, including reward design and per-agent conditioning.

6 Conclusion

  • Large-scale driving self-play can run directly on perspective images, without privileged observations.Alberti approaches its vectorized counterparts and transfers zero-shot to WOMD layouts.Source: §6 · p. 14

A Simulator and Training Details

  • Appendix A covers reward and conditioning, camera and policy design, optimization, hyperparameters, and hardware.The supplied passage gives the appendix’s scope; the reward and conditioning details are described separately below.Source: A Simulator and Training Details · p. 18

A.1 Reward and Conditioning

  • Most reward coefficients and dynamics multipliers are resampled per agent at episode reset and exposed to the policy through conditioning.The overspeed penalty, added relative to Gigaflow, activates when speed exceeds the lane limit by more than 2 m/s; pedestrian agents use only goal and collision terms.Source: §A.1 · pp. 18–19, Table 2 · p. 19

A.2 Observation and Policy Architecture

The appendix describes the perspective observation and shared policy architecture, the common PPO setup, and a renderer comparison. CUDA rendering preserves image quality at low resolutions where single-sample Madrona aliases.

A.2 Observation and Policy Architecture

  • The perspective policy uses four egocentric cameras, while both modalities share the policy trunk and differ only in their observation encoders.The perspective policy has no vectorized auxiliary slots; the vectorized baselines share a layout, with unobservable slots set to zero in Vectorized†.Source: §A.2 · p. 19, Table 3 · p. 20, Table 4 · p. 21

A.3 Optimization and Hyperparameters

  • All runs use the same single-stage PPO recipe; perspective Alberti runs use 32 H100s, while vectorized baselines use four H100s on one node.The render-resolution sweep ranges from 32 × 18 to 384 × 216, with 96 × 54 as the default.Source: §A.3 · p. 20, Table 5 · p. 22

B CUDA Rasterizer vs. Madrona

  • The comparison uses production antialiased CUDA, native one-sample Madrona, and a 2× supersampled Madrona baseline for quality-matched evaluation.The quality-matched Madrona baseline renders at doubled resolution and box-downsamples; Pictura’s production rendering already includes antialiasing.Source: B CUDA Rasterizer vs. Madrona · p. 20

B.1 Rendering Quality

  • At low resolutions, CUDA preserves sharp lane lines while single-sample Madrona breaks them into aliased fragments; supersampling largely closes the gap.Fig. 10 compares the same scenes across the three rasterizer configurations.Source: §B.1 · p. 21

B.2 Training-Step Time Breakdown

In the controlled single-H100 benchmark, rendering accounts for the wall-clock gap between the CUDA and Madrona rasterizers. Madrona’s per-iteration slowdown grows with resolution, while CUDA rendering scales gently with pixel count.

B.2 Training-Step Time Breakdown

  • The benchmark uses one H100-NVL and a production-sized setup; non-render phases agree within measurement noise, leaving rendering as the source of the wall-clock gap.The setup includes 256 agents, two environments, 128-step rollouts, and a 65,536-transition minibatch; compilation time is excluded and advantage filtering is disabled.Source: §B.2 · p. 22
  • 2.1–3.0× slower end to end for Madrona, rising to 6.1× with antialiasing; rendering takes 58–86% of the iteration.Rendering itself is 13–16× slower on Madrona and 14–41× slower on the antialiased baseline; Pictura reaches 384 × 216, where both Madrona configurations run out of memory.Source: §B.2 · p. 23

B.3 Renderer Memory Usage

The CUDA rasterizer uses substantially less renderer memory than Madrona, while the renderer comparison also reports a wall-clock policy-quality advantage for CUDA. The evaluation-details section outlines the metrics and settings used to track vectorized baselines.

B.3 Renderer Memory Usage

  • About 3× less peak memory than Madrona at each resolution, with CUDA streaming the batch in bounded passes rather than preallocating a renderer heap.Madrona’s heap grows steeply with squared render size; antialiased Madrona fails at 384 × 216, while CUDA and native Madrona reach that resolution before failing at 768 × 432 for different reasons.Source: §B.3 · pp. 23–24

B.4 Wall-Clock Quality per GPU-Hour

  • 3.35× longer for Madrona to reach the 2 B-step cap (10.2 vs. 3.0 h); at equal wall-clock, the CUDA-rendered agent leads on every vehicle-only metric.At matched steps, the two agents reach equivalent driving quality; this comparison uses Madrona’s fastest, non-antialiased configuration, so its speedup is a lower bound versus quality-matched antialiased Madrona.Source: §B.4 · p. 24

C Policy Evaluation Details

C.1 Metric Definitions

Driving quality uses five vehicle-only metrics, with event rates measured per vehicle episode and distance per infraction based on episodes containing at least one infraction.

C.1 Metric Definitions

  • Five vehicle-only metrics are reported: at-fault collision rate, off-road rate, red-light violation rate, goals reached, and distance traveled per infraction.Event rates are fractions of vehicle episodes with at least one occurrence; goals reached is the mean number reached per vehicle.Source: §C.1 · p. 25
  • A collision is an ego-agent bounding-box overlap, off-road means crossing a road edge or leaving the drivable map, and a red-light violation involves crossing or changing lanes into a qualifying red stop line.A collision is not charged as at-fault when the ego is essentially stopped or struck from behind; the red-light rule requires proximity and aligned heading.Source: §C.1 · p. 25
  • Distance per infraction divides controlled-vehicle distance by the number of vehicle episodes containing at least one off-road, collision, or red-light event.Distance is accumulated as speed × ∆t.Source: §C.1 · p. 25

C.2 Evaluation Protocol

Evaluation covers in-domain self-play at three vehicle densities and zero-shot driving on re-rendered WOMD layouts, with a separate renderer wall-clock comparison.

C.2 Evaluation Protocol

  • In-domain evaluation uses eight training maps, three densities of 60, 120, and 150 agents, 200 scenarios per density, and deterministic actions; zero-shot evaluation scores 44,073 WOMD validation scenes.In-domain scenarios run for 3000 steps; WOMD episodes are capped at 600 steps, and zero-shot reports collision, off-road, and goals metrics.Source: §C.2 · p. 26
  • For vectorized zero-shot evaluation, WOMD road polylines are subdivided into equal segments of at most 5 m, while in-domain maps retain their native 10 m chunking.The 5 m span gave the best zero-shot performance in a sweep; perspective observations need no adjustment because the rasterizer projects the polylines themselves.Source: §C.2 · pp. 26–27
  • At 2 B agent steps, the renderers reach equivalent driving quality, while Madrona takes 10.2 h versus Pictura’s 3.0 h.The comparison uses the same perspective agent at 64×36; wall-clock includes startup and compilation.Source: Figure 12 · p. 27

C.3 Vectorized Baselines over Training

The two vectorized baselines improve over training and reach comparable driving quality by 50 B steps; Alberti’s medium-density trajectory closes on them as training proceeds.

C.3 Vectorized Baselines over Training

  • By 50 B agent steps, both vectorized policies reach comparable driving quality, and Alberti’s medium-density trajectory closes on them during training.The figure tracks the two vectorized policies with and without auxiliary features that a perspective agent cannot perceive.Source: §C.3 · p. 27, Figure 13 · p. 28
  • The auxiliary-feature vectorized policy leads on lanekeeping metrics, off-road and red-light violations, which its privileged inputs most directly inform.The auxiliary features are ego lane-center distance, lane-heading, and partner speed.Source: §C.3 · p. 27
Loading 2607.26005v1…