iPad: Iterative Proposal-centric End-to-End Autonomous Driving

• ProFormer iteratively predicts proposals, anchors multi-view feature aggregation at their corner points, and refines proposal queries and features.Source: §1 · p. 2
• iPad scales linearly with proposal count instead of quadratically with dense BEV grids, while planning-aware extraction targets task-relevant information.Source: §1 · p. 2, Figure 1 · p. 2
• Two lightweight auxiliary tasks predict proposal passability and collision-relevant agent futures, aligning intermediate learning with planning while avoiding redundant computation.Source: §1 · pp. 2–3
• iPad centers feature extraction, mapping, prediction, scoring, and planning on sparse, learnable BEV proposals.Source: §1 · p. 2
• iPad achieves state-of-the-art results on NAVSIM and CARLA Bench2Drive and is over 10× more computationally efficient than UniAD.Source: §1 · p. 3

Intermediate representations

• Dense BEV systems support spatial fusion and joint tasks but require high-resolution features that increase computational cost.Source: §2 · p. 3
• Sparse query methods reduce view-transformation costs but can waste computation on irrelevant agents and overlook useful prior knowledge.Source: §2 · p. 3
• iPad integrates planning into intermediate representation learning by iteratively refining proposals and focusing features on planning-relevant content.Source: §2 · p. 3

Auxiliary E2E tasks

• Existing auxiliary tasks often require high-resolution inputs and large models, whereas iPad uses lightweight mapping and prediction tasks focused on proposal-relevant objects.Source: §2 · p. 3

Multi-modal planning

• Deterministic planners and fixed or limited proposal sets constrain multimodal planning, while iPad dynamically predicts and refines proposals to improve trajectory diversity and adaptability.Source: §2 · p. 3

3 Method

• iPad unifies scene encoding, iterative proposal refinement, scoring, mapping, and prediction around sparse candidate trajectories.The framework extracts image and ego features, refines proposals with ProFormer, scores final proposals, and trains proposal-centric auxiliary tasks.Source: §3 · p. 4

3.1 Scene Encoder

• Multi-view images and ego status are encoded as image feature maps and ego features for downstream proposal processing.The image encoder uses a backbone and neck, while ego velocity, acceleration, and future commands are encoded by a linear layer.Source: §3.1 · p. 4

3.2 ProFormer

• ProFormer predicts trajectory proposals from proposal queries, applies proposal-anchored self-attention, and iteratively aggregates image features using proposal geometry.Each proposal is a sequence of future x, y, and heading states; predicted positions anchor deformable attention, while proposal corners anchor multi-view spatial cross-attention.Source: §3.2 · pp. 4–5
• ProFormer shares weights across iterations and supervises proposal prediction with a Minimum over N loss whose discount factor relaxes early-iteration constraints.The loss compares generated proposals with the expert trajectory across iterations, using λ to discount earlier supervision.Source: §3.2 · p. 5

3.3 Scorer

• The scorer predicts each final proposal’s performance and selects the highest-scoring trajectory using pooled proposal features and a BCE objective.Scores are trained from safety, efficiency, and comfort-related NAVSIM sub-metrics obtained through log-replay simulation.Source: §3.3 · p. 5

3.4 Proposal-Centric Mapping and Prediction

• Proposal-centric mapping predicts on-road and on-route probabilities, while prediction models future states of the first at-fault and likely-to-collide agents.These lightweight auxiliary tasks focus on information relevant to each proposal and are trained with BCE, L1, and validity-label losses within fully differentiable end-to-end training.Source: §3.4 · pp. 5–6, §3.5 · p. 6

4.1 Open-Loop NAVSIM Benchmark

• NAVSIM evaluates open-loop plans with closed-loop-oriented metrics derived from a 4-second non-reactive simulation using a kinematic bicycle model and LQR controller.The benchmark emphasizes complex intention-change scenarios rather than predominantly straight driving.Source: §4.1 · p. 6

4.1 Open-Loop NAVSIM Benchmark; 4.2 Closed-Loop Bench2Drive Benchmark

• iPad achieves state-of-the-art NAVSIM and Bench2Drive performance, including all reported NAVSIM metrics, Bench2Drive success rate and driving score, and the best average across five driving abilities.The results do not rely on lidar for NAVSIM or an expert model for Bench2Drive.Source: §4.1 · p. 6, §4.2 · p. 7, Table 1 · p. 7

4.2 Closed-Loop Bench2Drive Benchmark

• Bench2Drive evaluates 220 safety-critical routes across CARLA towns using success rate, driving score, efficiency, and comfortness, plus five driving abilities.A PID controller tracks planned trajectories at 20 Hz during closed-loop evaluation.Source: §4.2 · p. 7

4.3 Ablation Studies

• Replacing ProFormer with BEVFormer yields limited proposal-learning gains, whereas proposal-aware spatial cross-attention significantly improves all planning metrics.Replacing proposal-centric mapping lowers driving-area compliance, while replacing proposal-centric prediction degrades no-at-fault collisions and time-to-collision.Source: §4.3 · p. 8, Table 4 · p. 8

4.4 Scalability

• iPad’s final NAVSIM PDM score follows a power-law scaling trend as proposal number, refinement iterations, and training data size increase.More proposals expand planning-distribution flexibility, more iterations improve proposal accuracy, and more data improves generalization.Source: §4.4 · p. 8, Figure 3 · p. 9

4.5 Qualitative Analysis

• Qualitative results show diverse human-like turning proposals, collision-risk-aware predictions, and conservative collision-free merging plans.Proposal-line brightness represents predicted scores, while predicted-agent-box brightness reflects associated proposals.Source: §4.5 · pp. 8–9, Figure 4 · p. 9

5 Limitations

• iPad omits historical image and status information for efficiency, and its simulated closed-loop evaluation faces a sim-to-real gap that leaves real-world closed-loop performance uncertain.Unmodeled corner cases, unexpected human behavior, and diverse environmental conditions constrain simulation fidelity.Source: §5 · pp. 9–10

6 Conclusion

• iPad places sparse, learnable proposals at the center of perception, prediction, and planning, providing a unified and computationally efficient alternative to dense BEV grids.The framework uses proposals throughout feature extraction, mapping, prediction, and planning.Source: §6 · p. 10
• Experiments on challenging real-world and simulation benchmarks show state-of-the-art performance with significantly greater efficiency than prior work.The same architecture is used for both datasets with downsampled inputs from four camera views.Source: §6 · p. 10, B Model Details · p. 14

A Detailed Mechanism in ProFormer

• The deformable attention mechanism samples offset locations around proposal-derived reference points and combines them with normalized attention weights.The sampled feature locations are obtained by bilinear interpolation.Source: A Detailed Mechanism in ProFormer · p. 13
• ProFormer integrates multi-view image features through proposal-anchored spatial attention while jointly refining BEV queries and proposals.Proposal queries use deformable image features, and projected reference points aggregate features from views hit by each proposal.Source: A Detailed Mechanism in ProFormer · p. 13, Figure 5 · p. 13

C Training Scoring

• Ray multiprocessing is used to obtain ground-truth scores for final proposals efficiently during training.Source: C Training Scoring · p. 14

C.1 NAVSIM Scoring

• NAVSIM evaluation uses an official log-replay simulator with an LQR controller at 10 Hz over a 4-second horizon.Final scores combine the listed official sub-metrics.Source: §C.1 · p. 14
• NAVSIM sets safety-related sub-metrics to zero when at-fault collisions, off-road states, or sub-one-second projected time-to-collision occur.Static-object collisions receive a softer penalty, while non-at-fault collisions in the non-reactive environment are ignored.Source: §C.1 · p. 14
• Comfort is penalized when lateral or longitudinal acceleration, deceleration, jerk, yaw rate, or yaw acceleration exceeds the specified thresholds.The thresholds include lateral acceleration above 4.89 m/s2 and absolute jerk above 8.37 m/s3.Source: §C.1 · p. 14
• Ego Progress measures route-center progress normalized by a safe upper bound from the PDM-Closed planner and clips the ratio to [0, 1].Scores are discarded when the upper bound is below 5 meters or progress is negative.Source: §C.1 · p. 14

C.2 Bench2Drive Scoring

• Bench2Drive uses a log-replay simulator with a perfect controller operating at 2 Hz over a 3-second horizon.Source: §C.2 · p. 15
• Bench2Drive marks trajectories invalid for collisions, off-road states, or projected time-to-collision below 1 second.No Collision considers vehicles, bicycles, pedestrians, signs, cones, and traffic lights.Source: §C.2 · p. 15
• Comfort is zeroed when acceleration or turning rate exceeds expert-trajectory maxima, while progress is normalized by expert progress under collision-free, on-road conditions.Progress ratios above 1 are replaced by their reciprocals.Source: §C.2 · p. 15

C.3 Relations between Open-loop and Closed-loop Scores

• Score and Progress correlate positively with closed-loop driving performance, whereas NC, TTC, open-loop L2, and Comfort correlate negatively.The analysis uses 20 stabilized checkpoints from both shared and non-shared iPad versions; the authors attribute some relationships to reactive-versus-non-reactive behavior and hazardous scenarios.Source: §C.3 · p. 15, Figure 6 · p. 15

D.1 Proposal Refinement

• The proposal-refinement visualizations include one NAVSIM scenario and one Bench2Drive scenario across all iterations.Source: Figure 7 · p. 16, Figure 8 · p. 16
• iPad gradually refines proposals across iterations while preserving multimodality in intersection scenarios.The qualitative evidence covers both NAVSIM and Bench2Drive closed-loop testing scenarios.Source: §D.1 · p. 16, D More Qualitative Results · p. 16

D.2 Mapping

• On-road and on-route probability predictions account for proposal heading and vehicle size, enabling selection of a feasible proposal as the plan.This behavior is demonstrated in both NAVSIM and Bench2Drive scenarios.Source: §D.2 · p. 17
• Figure 9 visualizes NAVSIM mapping results, where proposal lightness represents scores or predicted on-road or on-route probabilities.A proposal is considered off-road if any corner point is off-road.Source: Figure 9 · p. 17
• Figure 10 presents the corresponding Bench2Drive mapping results using proposal-line or point lightness to encode scores or predicted probabilities.The same corner-based off-road criterion applies.Source: Figure 10 · p. 17

D.3 Collision Prediction

• iPad predicts future bounding boxes of at-fault and likely collided agents for dangerous proposals, leading the planner to select a safe centering proposal in NAVSIM.Figure 11 shows proposal and predicted-agent-box lightness according to associated scores.Source: §D.3 · p. 18, Figure 11 · p. 18
• In Bench2Drive parking cut-in scenarios, predicted collision risks trigger selection of a deceleration proposal when the red vehicle’s taillights are illuminated.The predicted future bounding boxes identify the at-fault and likely collided vehicle for dangerous proposals.Source: §D.3 · p. 18, Figure 12 · p. 18

NeurIPS Paper Checklist

• The checklist states that the abstract and introduction clearly present iPad’s motivations, contributions, and scope.The authors answer this checklist question affirmatively.Source: NeurIPS Paper Checklist · p. 19

2. Limitations

• The paper states that its limitations are discussed in section 5, but the supplied checklist passages do not specify those limitations.The checklist separately notes that the work includes experiments and no theoretical results.Source: §2 · pp. 19–20
• The authors report open access to data and code, sufficient reproduction instructions, and adequate training and test details.They also state that all results can be reproduced and that training resources are described.Source: §2 · pp. 20–22
• The experiments do not use significance tests or error bars, although the authors describe NAVSIM and Bench2Drive as large-scale and results as reproducible.The checklist answer is explicitly negative for statistical-significance reporting.Source: §2 · pp. 21–22
• The experiments are conducted on the NAVSIM and Bench2Drive datasets, with results described as stable across multiple trials and accurately reproducible using the provided open-source code.Source: §2 · p. 21
• The checklist states that the work addresses autonomous driving and traffic safety as potential societal impacts and poses no high-risk model-release concerns.The authors report no crowdsourcing, human-subject research, or new asset release.Source: §2 · pp. 22–24