PaperlayerView exact revision on arXiv

Source-linked AI summary

The Decision Value of Perception Compute

Hoang Pham Cong, Ho Viet Duc Luong

arXiv:2609.35910v1cs.LGcs.CV

TL;DR

Adaptive perception often allocates compute according to expected perception improvement, but better perception need not improve a downstream decision. The paper defines decision value as downstream-loss change, introduces DEEP to score pre-escalation allocators against a budget-constrained oracle, and finds harmful escalations and perception–decision mismatch across evaluated settings. These results make full fidelity a baseline rather than an allocation upper bound and show that allocator overhead matters.

  • Problem

    When perception feeds a downstream decision, perception improvement alone does not establish whether extra compute improves the decision, yet adaptive methods commonly evaluate extra compute through perception improvement.

    Source: §1 · p. 1

  • Method

    DEEP benchmarks pre-escalation allocators against a budget-constrained decision oracle under selection, latency, and energy budgets, charging each allocator for its own computation.

    Source: §1 · p. 2

  • Results

    34–54% of escalations that change downstream loss make it worse in monocular-geometry settings, and learned allocators recover part of oracle value but select nearly as much harm as random.

    Source: §5.1 · p. 6, §6 · p. 9

  • Takeaways & Limitations

    Because escalation value can be negative and perception-level gain can disagree with decision value, uniform full-fidelity inference is a baseline rather than an upper bound for allocation.

    Source: §1 · pp. 1–2

    Decision value is realized for one input with the full output replacing the cheap one and a fixed decision rule consuming it, rather than measuring closed-loop value or optimal combination of both outputs.

    Source: §2 · p. 3

Abstract

from arXiv · show

Adaptive perception spends extra computation on inputs where perception is expected to improve. When perception feeds a downstream decision system, a better perception output need not produce a better decision. We define the decision value of perception compute as the change in downstream loss from escalating an input from a cheap to an expensive perception mode. Because this value can be negative, the allocation of perception compute should be judged against a budget-constrained decision oracle, with uniform full-fidelity inference as a baseline rather than an upper bound. We introduce DEEP (Decision Evaluation for Escalated Perception), a benchmark that scores pre-escalation allocators against this oracle under selection, latency and energy budgets, charging each allocator for its own computation. With deployed monocular geometry on KITTI and nuScenes, we find that 34--54% of the escalations that change downstream loss make it worse; harmful escalations also occur for the published PDM-Closed planner, evaluated open-loop on nuPlan with real detector outcomes. On nuScenes, perception-level gain frequently disagrees in sign with decision value. This mismatch has practical consequences: choosing among fixed deployable signals by missed-object perception gain rather than by decision value reduces realized test decision gain by 7.4% of the all-cheap loss on average. Learned allocators recover part of the oracle's value by finding beneficial escalations but select nearly as much harm as random, and once their own computation is charged at a 20% latency budget, only the lightweight routers, at about 3.5\% of a full detector pass, still beat random.

1 INTRODUCTION

Adaptive perception methods commonly value extra computation by its expected improvement in perception, but that is incomplete when outputs feed a decision system. The paper defines decision value by downstream loss, introduces DEEP to evaluate pre-escalation allocators against a budget-constrained oracle, and reports that harmful escalations make full fidelity a baseline rather than an upper bound.

1 INTRODUCTION

  • Prior adaptive-perception methods identify where extra computation may improve prediction quality using signals such as uncertainty, difficulty, criticality, or expected perception gain.The paper’s contribution list frames decision value as signed and reports that learned allocators retain an advantage under measured budgets only at low overhead relative to a full detector pass.Source: §1 · pp. 1–2
  • Additional perception compute can recover objects without changing an action, improve a safety-critical action, or trigger an unnecessary response; its value depends on downstream loss, not perception gain alone.Decision value is the change in downstream loss from escalation, positive when loss decreases.Source: §1 · p. 1
  • Across KITTI and nuScenes, a substantial fraction of escalations that change downstream loss make it worse, and different downstream systems can assign opposite values to the same perception change.Uniform full-fidelity inference, which absorbs harmful escalations together with beneficial ones, is a practical baseline when the budget admits it.Source: §1 · p. 1
  • The decision oracle allocates budget to valuable escalations and skips harmful ones, whereas deployable allocators must decide before escalation from available input and context.The motivating examples show the same kind of recovered object can have opposite decision values depending on its relation to the driving corridor.Source: §1 · p. 2, Figure 1 · p. 2
  • DEEP scores pre-escalation allocators against a budget-constrained decision oracle under selection, latency, and energy budgets, charging each allocator for its own computation.Its core track uses detection-based driving pipelines on KITTI and nuScenes, with published planners on nuPlan in the external track.Source: §1 · p. 2

2 RELATED WORK

Prior work adapts computation using prediction quality, task success, or downstream planning effects. DEEP instead evaluates the marginal value of one additional perception computation for a downstream decision, under a budget and with pre-escalation commitment.

Adaptive inference and task-aware computation

  • Adaptive inference and task-aware computation are generally evaluated by retained perception accuracy or task success at reduced cost, while DEEP values an escalation by its marginal effect on a separate decision system’s loss.Examples include dynamic model selection, image-scale adaptation, critical-region processing, and routing LiDAR frames by scene complexity.Source: §2 · p. 2

Selective offloading and routing

  • Detector-routing methods estimate per-input changes in detection quality, whereas DEEP uses the change in downstream decision loss; on nuScenes, four perception-loss gains disagree in sign with decision value on 25–51% of frames where both are non-zero.The same perception change can also have opposite values for two downstream systems.Source: §2 · pp. 2–3

Planning-aware perception evaluation

  • Planning-aware metrics evaluate perception outputs or failures by downstream planning consequences, while DEEP asks for one input’s marginal value of additional computation under a budget, with allocation before that computation runs.The related work includes PKL, planning-aware metrics, TIP, and task-aware risk estimation.Source: §2 · p. 3

Value of information and computation

  • The formulation operationalizes decision value for cheap-to-full perception escalations using realized downstream-loss change, an oracle reference, and an allocator that commits before escalation.Its value can be negative because the full output replaces rather than supplements the cheap output and is consumed by a fixed decision rule.Source: §2 · p. 3

3 DECISION VALUE OF PERCEPTION COMPUTE

The section defines escalation value by its effect on a downstream system’s loss, rather than by perception quality. Because that value can be signed, allocation is framed against an oracle that chooses beneficial escalations within budget.

3 DECISION VALUE OF PERCEPTION COMPUTE

  • Decision value is the change in downstream loss when an input is escalated from cheap to full perception, positive when the full mode reduces loss and negative when it increases loss.Perception gain and planning-aware gain are distinct signals; the allocation target is decision value.Source: §3 · p. 3
  • The decision oracle selects beneficial escalations under a capacity budget and may leave capacity unused when remaining escalations have negative value.Where loss depends on the previous action, both counterfactual branches use the all-cheap action.Source: §3 · pp. 3–4
  • When all beneficial escalations fit and at least one is harmful, the oracle strictly exceeds all-full inference in decision gain; full fidelity remains a practical baseline when it fits the budget..Source: §3 · p. 4
  • With equal escalation costs, an optimal score ranks every positive-value input ahead of non-positive ones and orders positive values from largest to smallest.Allocation quality depends both on ordering beneficial inputs and separating useful from harmful escalations; a single strict ranking cannot be optimal for two systems with a ranking reversal among beneficial inputs.Source: §3 · p. 4
  • With multiple fidelity modes and allocator costs, allocation becomes a multiple-choice knapsack; the budget is average compute over inputs, not a per-frame deadline.In the cascade, higher-fidelity modes run from scratch, and their additional cost equals their mode cost.Source: §3 · p. 4

4 DEEP: DECISION EVALUATION FOR ESCALATED PERCEPTION

DEEP evaluates allocators, not perception models: each track fixes the perception modes, downstream system, data split, and cost profile. Its design restricts allocators to pre-escalation information and compares them with a budget-matched decision oracle while charging their computation.

4 DEEP: DECISION EVALUATION FOR ESCALATED PERCEPTION

  • DEEP values additional computation by its marginal effect on fixed downstream loss, requires pre-escalation commitment using deployable signals, compares against a budget-matched oracle, and charges allocator computation under measured budgets.Signals from the full output or reference state are diagnostic, not allocator inputs at test time; they may be used as training supervision on non-test units.Source: §4 · p. 4, Figure 2 · p. 5

4.1 TRACKS AND DOWNSTREAM SYSTEMS

  • The core track includes braking, trajectory-control, and learned-planner systems; the nuPlan external track evaluates published PDM-Closed and IDM planners open-loop using real detector outcomes.The planner’s nuScenes loss measures deviation from the logged trajectory rather than safety.Source: §4.1 · p. 5

4.2 SPLITS, STATISTICS AND METRICS

  • Test splits are held out by scene on nuScenes, sequence on KITTI, and log on nuPlan, with learned allocators fit on remaining units and evaluated once on test units.Confidence intervals resample scenes, sequences, or logs rather than frames; selections are evaluated in exact expectation over tie-breaks.Source: §4.2 · pp. 5–6
  • Normalized decision gain measures the fraction of same-budget oracle value realized, while negative values indicate net harm; the oracle may leave capacity unused.Under measured budgets, allocator cost reduces usable escalation capacity, and normalization can be unstable when oracle value is concentrated in few inputs.Source: §4.2 · p. 6

4.3 BUDGETS AND BASELINES

  • Measured budgets charge the cheap pipeline and allocator on every input and the selected higher-fidelity mode on escalated inputs; baselines include random, hand-designed, learned, and diagnostic signals.Selection budgets assess ranking quality, while fixed-rate causal allocation is reported separately.Source: §4.3 · p. 6

5 EMPIRICAL FINDINGS

The empirical findings test whether perception escalations improve downstream decisions, whether pre-escalation signals can predict decision value, and whether allocator advantages survive measured costs. Across settings, escalation can harm decisions, and allocator rankings based on perception gains or free computation can mislead.

5.1 EXPENSIVE PERCEPTION HAS SIGN-VARYING DOWNSTREAM VALUE

  • 34–54% of escalations that change downstream loss make it worse in every monocular-geometry setting, and the table compares all-full inference with a 20%-capacity decision oracle.For nuScenes braking with oracle geometry, helpful escalations reduce all-cheap loss by 24.1% and harmful ones increase it by 10.1%, yielding a 14.0% net reduction under uniform escalation.Source: §5.1 · p. 6, Table 1 · p. 7

5.2 THE PATTERN HOLDS FOR A PUBLISHED PLANNER UNDER REAL DETECTOR OUTCOMES

  • With PDM-Closed, escalation harms 44% of states whose safety loss changes and 38% under scalar loss; decision-changing events are sparse.At 20% capacity, the oracle reduces safety loss by 14.7%, versus 9.8% for uniform escalation; IDM changes safety loss on only 5 of 1,440 states.Source: §5.2 · p. 7

5.3 LEARNED ALLOCATORS FIND PRE-ESCALATION SIGNAL

  • Neither heuristic significantly outperforms random in any cell, while learned allocators win in selected settings, including nuScenes braking and KITTI braking.On nuScenes braking, the gradient-boosted gate reaches 0.482 against 0.132 for random; its paired gain is 0.087 [0.014, 0.154] of the all-cheap loss.Source: §5.3 · p. 8

5.4 RANKING ALLOCATORS BY PERCEPTION GAIN CAN SELECT WORSE ONES

  • Choosing a signal by false-negative gain reduces realized test decision gain by 0.074 [0.010, 0.108] of all-cheap loss at a 20% budget, averaged over eleven cells.Perception and decision objectives share no optimum in 43 of 52 cell–budget pairs; the reduction is not resolved when signals are selected by risk-weighted gain.Source: §5.4 · p. 8

5.5 MEASURED COST REORDERS THE ALLOCATORS

  • At a 20% latency budget, only the MLP detection-list routers beat random among learned allocators; their 0.65 ms cost is about 3.5% of a full pass.They escalate 16.5% of inputs; feature-based gates do not beat random, and learned allocators capture more benefit while selecting nearly as much harm as random.Source: §5.5 · pp. 8–9, Figure 3 · p. 9

6 CONCLUSION

The conclusion frames additional perception compute as having signed, system-dependent downstream value and calls for evaluating allocators against a budget-constrained decision oracle. The appendix formalizes when rankings are optimal, why downstream systems may lack a shared optimum, and how top-k disagreement produces regret.

6 CONCLUSION

  • 34–54% of escalations that change downstream loss make it worse in monocular-geometry settings, while perception-level gain frequently disagrees in sign with decision value on nuScenes.DEEP evaluates pre-escalation allocation against the decision oracle and charges allocator computation; its single-step counterfactual evaluation does not replace closed-loop evaluation, which needs sensor simulation.Source: §6 · p. 9

A.1 PROOF OF PROPOSITION 1

  • With equal escalation costs, an optimal ranking places positive-value inputs first and leaves harmful inputs unselected; allowing abstention preserves optimality across budgets.The conclusion recommends reporting realized decision gain and nDG per downstream system, including after charging allocator cost.Source: §6 · p. 9, §A.1 · p. 13

A.2 NO SHARED OPTIMUM ACROSS DOWNSTREAM SYSTEMS

  • When two downstream systems reverse the relative values of two beneficial inputs, no single strict ranking is optimal for both at every budget.Source: §A.2 · p. 13

A.3 REGRET FROM A TOP-k MISMATCH

  • Top-k disagreement matters most when the decision-value margin at the budget boundary is large, making aggregate correlation a poor summary of allocation quality.The regret comparison is between the oracle-selected set and the score-selected set at a given budget.Source: §A.3 · p. 13

B CROSS-TARGET DIAGNOSTIC ANALYSIS

Across nuScenes cross-target diagnostics, no diagnostic approaches the oracle, and the best signal changes with the downstream target.

B CROSS-TARGET DIAGNOSTIC ANALYSIS

  • Against logged trajectories, PKL's remaining nDG is 0.36 versus 0.06 for random; its self-reference gap contrasts logged-outcome scoring with scoring against its own output.Source: B CROSS-TARGET DIAGNOSTIC ANALYSIS · p. 14
  • The best diagnostic varies by target: risk-weighted gain leads qbrake, TIP leads logged-trajectory planning, and PKL leads self-reference.Under deployed monocular geometry, PKL and TIP reach 0.289 and 0.271 on the braking target, versus 0.135 for random.Source: B CROSS-TARGET DIAGNOSTIC ANALYSIS · pp. 14–15, Figure 4 · p. 14

B.1 LONGITUDINAL BRAKING CONTROLLER

  • At a 20% budget, the qbrake oracle reduces all-cheap loss by 24.07%, versus 14.01% for all-full; 50 of 141 affected frames are harmed.Source: §B.1 · p. 15

B.2 LEARNED PLANNER AGAINST THE LOGGED TRAJECTORY

  • For logged-trajectory qplan, 600 of 1,179 affected frames (50.9%) are harmed; the oracle reduces all-cheap loss by 4.41%, versus 1.31% for all-full.Source: §B.2 · p. 15

B.3 LEARNED PLANNER AGAINST ITS OWN REFERENCE-CONDITIONED OUTPUT

  • Self-reference has 578 of 1,413 affected frames (40.9%) harmed; the oracle reduces loss by 13.67%, versus 7.49% for all-full.Source: §B.3 · p. 15

B.4 LATERAL CORRIDOR CONTROLLER

  • The lateral corridor controller has non-zero decision value on only 23 nuScenes frames, too few for scene-level intervals.Its evaluation appears only in the appendix robustness sweep.Source: §B.4 · p. 15

B.5 TIP SCORE DIRECTION

  • PKL gain is PKLc−PKLf and TIP gain is TIPf−TIPc; both correlate positively with perception gain on the wiring-validation subset.Source: §B.5 · p. 15

C SIGN VARIATION ACROSS CONFIGURATIONS

Harmful escalation persists across detector and operating-point controls, while its prevalence varies with geometry, fidelity gap, and threshold choice.

C SIGN VARIATION ACROSS CONFIGURATIONS

D PER-MODE OPERATING POINTS

  • With per-mode operating points, every nuScenes and moderate-gap KITTI cell has harm rate at least 34% and ρq ≥0.33; across 25 nuScenes threshold pairs, minimum harm is 24.4%.Source: D PER-MODE OPERATING POINTS · p. 16, Table 6 · p. 16
  • Thresholds tuned to each mode's downstream loss yield harm rates of 33–49% and ρq of 0.44–0.84 in nuScenes and moderate-gap KITTI cells.The per-mode operating points are selected using training-unit precision and recall, not decision value.Source: D PER-MODE OPERATING POINTS · p. 16

E BENCHMARK REFERENCE RESULTS

Held-out results show that allocator rankings depend on both the decision objective and the allocator’s own compute cost. Lightweight routers recover some oracle value, but perception-gain selection can reduce decision gain, and even full-fidelity inference can be harmful.

Held-out allocation

Measured budgets

Ranking allocators by perception gain

  • Choosing by missed-object gain reduces realized test decision gain by 0.074 [0.010, 0.108] of all-cheap loss at 20%.On the target-free deployable pool, decision-value and perception-gain objectives share no optimum in 43 of 52 pairs for missed-object gain; median Kendall τ is −0.33.Source: E BENCHMARK REFERENCE RESULTS · p. 20

Training objectives

  • Across ten core cells and four quotas, decision-value training yields more wins against random overall, but no objective performs best for every architecture.Pooled differences between decision-value and perception-gain targets include zero; target effects vary by cell and architecture.Source: E BENCHMARK REFERENCE RESULTS · pp. 20–21

Transfer between consumers

  • Transfer is consumer-dependent: on nuPlan, the PDM-Closed safety-loss gate reaches 0.94 on IDM, while the gate fitted for IDM realizes none of PDM-Closed’s value.Across 528 off-diagonal transfers, median regret is −0.009; the reported counts are descriptive over dependent entries.Source: E BENCHMARK REFERENCE RESULTS · p. 22

Uniform full fidelity

  • At a 50% latency budget, uniform full fidelity beats the best feasible allocator in eight of ten core cells; its two losses occur where escalation raises test loss.Full fidelity becomes feasible only above the per-input cost of running the full mode alone.Source: E BENCHMARK REFERENCE RESULTS · p. 22

Fidelity ladder

  • At a 20% KITTI budget, 640-only allocation reaches 0.73 of the multi-fidelity optimum on braking and 0.57 on the trajectory controller.Intermediate modes have different latency and energy costs, and the deployable multi-fidelity gate cannot run below 30%.Source: E BENCHMARK REFERENCE RESULTS · p. 23

Benefit captured and harm incurred

  • At 20%, the oracle realizes 0.189 of all-cheap loss; learned gates and routers capture 0.060–0.075 and incur 0.007–0.012 harm, versus random’s 0.038 gain and 0.013 harm.The oracle-to-allocator gap is mostly missed benefit on core cells; harmful inputs can still be selected by deployable signals.Source: E BENCHMARK REFERENCE RESULTS · p. 23

F EXTERNAL PLANNER TRACK

The external planner track evaluates planner responses to real detector outcomes in an open-loop, statewise setup. Results and sensitivity checks show that false positives and matching choices materially affect measured decision changes, while the planner-loss cells are related rather than independent.

Setup

  • The evaluation is statewise and open-loop across 1,440 logged states, with separate planner instances for reference, cheap-detector, and full-detector tracks.Proposed trajectories are scored against logged tracks over 0.5–4.0 seconds using five losses.Source: F EXTERNAL PLANNER TRACK · p. 24

Real detector outcomes

  • At 320 versus 640 px, detector recall is 0.29 versus 0.42, while false positives rise from 0.83 to 1.68 per state.Dropping unmatched detections reduces PDM-Closed safety-loss changes from 45 states to 6.Source: F EXTERNAL PLANNER TRACK · p. 24

Sensitivities

  • With false positives removed, PDM-Closed’s safety loss changes on 6 states (harm 33%); count-matching changes 35 (harm 26%), while IoU 0.5 matching changes 51 (harm 47%).A synthetic detection profile predicts a larger recall gap than measured real outcomes and denser decision value for both planners.Source: F EXTERNAL PLANNER TRACK · p. 24

Scope

  • The loss cells are not independent replications, and IDM responds on too few states under real detector outcomes for the stated ranking comparison.The authors interpret the results as two planners across a family of related losses.Source: F EXTERNAL PLANNER TRACK · p. 24

G TIE-BREAKING AND LIFT CONVENTIONS

Top-k transfer comparisons are sensitive to tie-breaking, while cross-system transfer remains asymmetric even under favorable tie choices. The section also clarifies the scope of the nuPlan loss table and the box-center convention used in monocular lifting.

Ties and transfer

  • Table 12 covers five losses for both published planners under real detector outcomes across 1,440 states, but these geometric losses are not nuPlan’s metric suite.It also reports two aggregate losses under the KITTI detection profile; the IDM collision row is one affected state.Source: Table 12 · p. 25
  • 52% of the selected set at a 20% budget is determined by the false-negative-gain tie-break, because 326 frames exceed the cut and 366 share it.For score-free selection, 74% of the braking controller’s top-20% set and 83% of its top-30% set are tied at zero; evaluation therefore uses exact expectation over random tie-breaks.Source: G TIE-BREAKING AND LIFT CONVENTIONS · pp. 24–25
  • The planner’s most favorable completion of its optimal allocation reaches nDGqbrake = 0.067 under oracle geometry and 0.227 under monocular geometry at 20%.In the reverse direction, the braking controller’s heavily under-determined optimum reaches nDGqplan = 0.930 and 0.896, respectively.Source: G TIE-BREAKING AND LIFT CONVENTIONS · p. 25
  • Strictly ranked frame pairs are ordered differently by the two systems at rates of 0.510 under oracle geometry and 0.502 under monocular geometry.Goodman–Kruskal γ is −0.021 and −0.003, respectively.Source: G TIE-BREAKING AND LIFT CONVENTIONS · p. 25
  • The monocular lift reports distance to the box center; using the near face instead would place objects about half an object length closer.That difference is 2.3 m for a car and 5.6 m for a bus.Source: G TIE-BREAKING AND LIFT CONVENTIONS · pp. 25–26

H HARM MECHANISM

Escalation can simultaneously recover missed objects and add false positives, with the latter often driving harmful controller responses. Perception-level gains also frequently disagree in sign with downstream decision value.

Harm mechanism

  • On harmed braking frames, escalation adds +1.980 false positives versus +1.176 on helped frames, while recovering 0.600 misses versus 1.396.On 56.0% of harmed frames it recovers no miss at all; the table reports corresponding detection-error changes by decision-value sign.Source: H HARM MECHANISM · p. 26, Table 13 · p. 26
  • The bird’s-eye view shows harmed frames dominated by false positives added inside the controller’s corridor, while helped frames are dominated by recovered misses.It includes differing objects considered by the controller within 80 m on nuScenes frames with non-zero decision value under oracle geometry.Source: Figure 8 · p. 26

Perception gain and decision value

  • Among affected nuScenes frames with positive perception gain, 16–24% are harmed under braking and about half under the learned planner.Perception-gain/decision-value Spearman correlations are at most 0.11; Table 14 defines the comparison for YOLOv8s 320→640 under oracle geometry and a shared threshold.Source: H HARM MECHANISM · p. 26, Table 14 · p. 27
  • For the planner, sign disagreement is close to 50% for all four perception gains, and 40 of 79 frames with non-zero decision value under both systems have opposite signs.Figure 9 visualizes one gain’s joint distribution; its shaded quadrants mark sign disagreement, at 32.6% for braking and 50.6% for the learned planner.Source: H HARM MECHANISM · pp. 26–27, Figure 9 · p. 27

I ROBUSTNESS ACROSS PERCEPTION AND DOWNSTREAM SETTINGS

Ego speed is a useful control signal in some settings but does not explain the perception-level results on nuScenes; trivial baselines must be assessed per setting.

Robustness across settings

  • Ego speed separates from random on no held-out nuScenes cell at a 20% budget, so ego speed alone does not explain the perception-level results there.The signal uses a backward difference over the previous 0.5 s on nuScenes, current-frame OXTS forward velocity on KITTI, and current ego-state velocity on nuPlan.Source: I ROBUSTNESS ACROSS PERCEPTION AND… · p. 27
  • In KITTI longitudinal settings, braking demand scales as v2/(2g), making ego speed a much stronger baseline because both loss and potential reduction grow with speed.The authors therefore state that trivial baselines must be reported per setting rather than dismissed.Source: I ROBUSTNESS ACROSS PERCEPTION AND… · p. 27
  • Across 16 configurations, false-negative gain never exceeds nDG = 0.337 and is negative in six.Source: I ROBUSTNESS ACROSS PERCEPTION AND… · p. 27

J DOWNSTREAM SYSTEMS AND HARDWARE DETAILS

The downstream evaluations use distinct controller and planner losses, with decision value measured under an all-cheap previous-action convention. The section also specifies sensitivity checks, allocator features, geometry, and device costs.

Downstream systems

  • The braking controller targets the most constraining obstacle within a 1.2 m half-width corridor and 80 m, with reaction time and deceleration capped as specified.Its loss includes safety shortfall, over-braking, command changes, and a collision-sized shortfall.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · p. 27
  • The receding-horizon controller evaluates seven longitudinal accelerations and five lateral offsets over a 3 s horizon, then scores the chosen candidate against reference obstacles.The loss includes clipped clearance shortfall and progress shortfall; the selected action depends on the previous action.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · pp. 28–29

Decision-value convention

  • Decision value escalates one input alone using the all-cheap previous action, and action-change terms account for at most 2.9% of harm and 0.8% of benefit.Braking harm is 73–96% over-braking; receding-horizon harm is almost entirely collision and clearance.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · p. 29

Loss and corridor sensitivity

  • The harm rate remains 33–55% across monocular-geometry loss and corridor variants, while the harm ratio changes more and rises with the over-braking weight.For nuScenes braking under oracle geometry, ρq rises from 0.28 to 0.62; Table 16 reports ranges over variants including the default.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · p. 29, Table 16 · p. 30

Downstream systems

  • The learned planner uses its released published weights and is scored by mean path displacement from the logged future; only 2,655 of 3,376 nuScenes frames have a logged future.The planner emits a BEV occupancy heatmap over 16 future steps, and its decision is the argmax path.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · p. 29

Gates and deployment details

  • Allocator gates use 65 cheap-side features on KITTI and nuScenes, while nuPlan uses 18 analogous cheap-track and ego-state features.Board measurements include 12.71 and 19.35 ms for nuScenes cheap and full pipelines; the gate feature vector costs 3.52 ms and ridge inference 0.41 ms.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · pp. 29–30
  • The deployed monocular lift fuses a ground-plane cue from the detection-box bottom with a class-height prior, while lateral extent comes from box edges.The lift overestimates range by 1.2–1.3 m at 0–15 m and 1.8–2.4 m at 15–30 m; full rigid transformation gives larger median range error in every checked band and fidelity.Source: J DOWNSTREAM SYSTEMS AND HARDWARE… · p. 30

Related papers

Loading 2609.35910v1…