Dream to Assist: Intent-Conditioned World Models for Shared Vehicle Control

1 Introduction

• DREAM2ASSIST extends a recurrent state-space world model with an intent predictor and conditions the assistive policy’s reward on inferred human objectives.In shared vehicle control, the driver and AI use the same steering, throttle, and brake controls.Source: §1 · p. 1
• A preliminary physical-vehicle deployment with three participants demonstrated policy transfer without retraining.Real human partners are expensive and variable, and their behavior differs from synthetic partners.Source: §1 · pp. 1–2
• The paper extends prior synthetic-setting work with human-data augmentation, reward restructuring, and safety filtering for real human partners.It reports more than 40 participants across three formal studies and more than 30 informal pilots, addressing explainability and cognitive load.Source: §1 · pp. 1–2
• The deployment pipeline combines automated policy screening, counterfactual log-replay evaluation, and human-in-the-loop vetting with iterative interface design.The paper also formulates key performance indicators for physical human-robot interaction in multi-agent driving scenarios.Source: §1 · p. 2
• Racing and urban evaluations found measurable gains in torque alignment and collision avoidance over a no-intent baseline; the racing studies included 30 participants and urban study 10.The paper also reports improved overtake success on high-speed straightaways.Source: §1 · p. 2

2 Related Work

• Shared-control and human-robot interaction research spans ergonomics, physiology, language-based teaming, and learning-based approaches.The paper situates its framework at the intersection of machine learning and reinforcement learning for HRI, shared-control driving, and time-critical interfaces.Source: §2.1 · p. 2, §2 · p. 2

2.1 Machine Learning Approaches for HRI

• DREAM2ASSIST embeds intent inference in latent world-model dynamics and conditions the assistive policy’s reward on inferred intent, then evaluates the pipeline with human partners in real time.Prior HRI work has examined joint representations, data efficiency, intent or plan modeling, explanation, alignment, and planning or control approaches.Source: §2.1 · p. 2

2.2 Planning for Shared Control in Driving

• Driving shared control includes planning, learning, and game-theoretic approaches, while prior work has also explicitly inferred driver intent for shared steering, throttle, and brake control.DREAM2ASSIST models discrete intent to support semantically meaningful multimodal behaviors and uses a multi-agent reinforcement-learning framing.Source: §2.2 · p. 2

2.3 HRI and Interfaces in Time-Critical Settings

• Time-critical HRI faces latency and mode-awareness challenges; feedback timing and modality, explainability, and experimental validity also shape system design.The integration of learning-based time-critical systems expands the design space and makes assumptions about user-facing concepts harder to control.Source: §2.3 · pp. 2–3

2.4 Design Metaphors for Shared Control

• DREAM2ASSIST modifies the commander-crew metaphor: the driver leads through action, and the assistive policy infers high-level intent before resolving it into continuous controls.The policy may override when the inferred intent would lead to an unsafe outcome.Source: §2.4 · p. 3

3.1 Preliminaries

• The problem is formulated as a two-player partially observable decision process, with discrete, domain-dependent human intents such as passing or staying behind in racing.Urban intents combine lateral direction and longitudinal response: left or right with go, slow, or stop.Source: §3.1 · p. 3

3.2 DREAM2ASSIST

• The dual-agent RSSM jointly represents behavior and intent, with supervised intent inference conditioning the assistant’s reward on the human objective.Separate human and assistant models support reasoning about human preferences independently of the joint system.Source: §3.2 · p. 4

3.3 Population-Based Human Modeling

• Fictitious co-play trains a population of simulated human policies grouped by objective, including partially trained checkpoints that represent varied, sometimes suboptimal behavior.Each checkpoint represents a different stage of training, with the final checkpoint approximating an optimal policy.Source: §3.3 · p. 5

3.4 Alignment with the Human

• At each timestep, the assistant estimates intent and shapes its reward toward the corresponding expert policy; uncertain intent defaults to task completion with minimized intervention.Training episodes use fixed intent labels, but a moving window of latent states lets the predictor track real-time intent changes.Source: §3.4 · pp. 5–6

3.5 Value-Aligned Assistive Objectives

• The assistant combines task progress and safety rewards, sparse intervention penalties, and intent-conditioned expert alignment; evaluations use reward-only weighting (αr = 1, αa = 0).Expert alignment draws on the optimal human model’s reward and policy, while intervention costs encourage temporally sparse control.Source: §3.5 · p. 6

3.6 Safety Layer

• A control-barrier-function filter enforces forward invariance of a safe vehicle-state set despite unknown future human inputs; human studies applied it to the trajectory visualization, not actuation.The filter may apply to commanded actions, the displayed predicted trajectory, or both.Source: §3.6 · pp. 6–7

3.7 Alignment Metrics

• The metrics measure intervention strength, intent-aligned versus intent-opposing torque, and timesteps with threshold-exceeding torque consistent with human intent.The threshold filters out negligibly small corrections; positive aligned-versus-opposing values indicate more force when aligned with the human’s goal.Source: §3.7 · p. 7
• These metrics provide a common evaluation framework across all experiments reported in Section 6.Source: §3.7 · p. 7

4.1 Racing Domain

• Racing tests high-speed control on the CARLA West track, with the driver’s intent represented as passing or staying behind the lead vehicle.The track includes straight or mildly curved sections and sharply curved sections, each offering opportunities to overtake.Source: §4.1 · pp. 7–8

4.2 Urban Domain

• Urban driving tests everyday hazards under lower-speed, surprise-rich conditions, with lateral and longitudinal intent options but only left-versus-right intents active in the reported evaluations.The scenario includes a wrong-way vehicle, a vehicle emerging from a driveway, and merging traffic at a roundabout; spatial and temporal ambiguity are both present.Source: §4 · p. 7, §4.2 · p. 8

4.3 Simulator Infrastructure

• Both domains use stationary simulator rigs; racing runs in CARLA and urban driving in Unreal Engine, while ROS manages policy inference and logging.Steering assistance is torque-coupled and can be overridden by the driver, whereas throttle and brake assistance is direct but not physically felt at the pedals.Source: §4.3 · p. 8

4.4 GRIP Physical Platform

• On GRIP, the urban policy transfers from simulation without retraining, requiring only minimal steering-hardware parameter adjustments.GRIP’s emulated vehicle dynamics match those used in simulation.Source: Figure 3 · p. 7, §4.4 · p. 8

5 Policy & Interface Design Process

• Policy down-selection and interface design are parallel, coupled pipelines, with their joint output refined through piloting.Interface design can begin with any candidate and narrows as the policy field narrows; late-stage policy voting uses a near-final interface.Source: §5 · pp. 8–9

5.1 Selection Spaces

• Hyperparameter sweeps and random seeds yield about 1K candidates; a three-stage funnel trades evaluation cost for fidelity, narrowing them to ∼500, ∼100, then 5–12 human-evaluated finalists and one deployed policy.Source: §5.1 · p. 9
• Selection combines training-time thresholds, counterfactual replay on fixed human-driver logs, and human review of the remaining policies.Replay measures collision propensity and track-orthogonality on identical logged inputs; urban selection also uses a team vote, while racing relies on offline metrics alone.Source: §5.1 · p. 9, §5 · p. 9

5.2 Interface Design Process

• The interface cycle scopes driver needs, prototypes and parameterizes cues, then iterates through piloting and voting with a near-final policy.A safety filter keeps displayed trajectory paths within safe bounds.Source: §5.2 · p. 9
• Pilots generally preferred visual cues for spatial guidance, while some found high-intensity haptics intrusive; these were informal observations, not a formal modality comparison.The feedback informed interface tuning.Source: §5.2 · p. 10
• The automated-screening, counterfactual-replay, and human-vetting funnel is presented as applicable beyond driving, including manipulation, navigation, and assistive robotics.The authors frame this structure for domains where learned policies must be vetted before deployment with humans.Source: §5.2 · p. 10

6.1 Common Methodology

• Across studies, participants gave informed consent; subjective ratings generally used 7-point Likert scales, with an overall policy rating on a 1–10 scale.Study procedures and configurations varied by evaluation, which built sequentially from racing characterization through refinement and urban validation.Source: §6.1 · p. 11, §6 · p. 11

6.2 Evaluation 1: Racing Characterization

• On straights, overtake success rose from 80% to 89%; on turns it fell from 51% to 21%, with minimum clearance dropping from 1.99 m to 1.01 m.Participants rated intent understanding above neutral overall, but helpfulness did not differ from neutral; the authors identified over-steering as the likely cause of the turn regression.Source: §6.2 · pp. 11–12, Table 4 · p. 11

6.3 Evaluation 2: Racing Policy Refinement

• Refinements targeted turn over-steering through human-data augmentation, action smoothing, and reward restructuring, including bounded progress rewards to discourage always-passing behavior.The authors hypothesized that more repeatable simulated turn behavior contributed to aggressive assistance and that added human data reduced this dynamics gap.Source: §6.3 · pp. 12–13
• In the N=6 refinement pilot, intent-opposing steering decreased from 31.2 to 29.6 deg and the aligned-minus-opposing difference improved from −10.1 to −5.6 deg, neither significantly.In the 4–3 s before an overtake, D2A applied about 50 deg steering versus 25 deg for D2A-Refined; refinement preference and likability were numerically higher, without significant differences.Source: §6.3 · p. 12, Figure 5 · p. 13, Figure 6 · p. 13

6.4 Evaluation 3: Urban Controlled Validation

• D2A-Full yielded 215 vs. 141 torque-alignment events (+52%) and 2.7% vs. 7.7% collision rates against D2A-NoIntent, while subjective ratings split evenly.The No-policy collision rate was 1.7%; five participants rated D2A-Full higher overall and five rated D2A-NoIntent higher.Source: §6.4 · p. 14

6.5 Preliminary On-Vehicle Deployment (N = 3)

• The GRIP deployment transferred the urban policy without retraining, using minimal parameter adjustments and demonstrating qualitatively appropriate lateral and longitudinal assistance.This three-participant deployment was framed as a feasibility demonstration, not a formal quantitative on-vehicle evaluation.Source: §6.5 · pp. 14–15

7.1 Intent Conditioning Improves Human-Robot Coordination

• 52% more torque-alignment events and lower collision rates (2.7% vs. 7.7%) favored D2A-Full over D2A-NoIntent in Eval 3, though collision counts were small.D2A-Full produced more alignment events for 8 of 11 participants who drove both conditions.Source: §7.1 · p. 16
• Intent-conditioned assistance steered with human objectives rather than task completion alone; without intent inference, assistance relied on task objectives and collision rates were higher.The authors attribute directional consistency to combining task completion, intervention sparsity, and expert alignment.Source: §7.1 · p. 16

7.2 Bridging the Sim-to-Real Human Gap

• Synthetic partners omit behaviors such as hesitation, jitter, and closed-loop fighting, so models trained only on them can expect smoother behavior than real drivers provide.In turns, excessive steering torque coincided with overtake success falling from 54% to 27%, despite above-neutral intent-recognition ratings.Source: §7.2 · p. 16

7.3 Structured Pipeline for Deploying Learned Interactive Policies

• The proposed policy-selection pipeline screens candidates with offline metrics, then uses counterfactual replay and human vetting because no automated metric fully captures qualitative considerations.Interface design is vetted separately from policy quality to avoid making evaluation of one dependent on the other.Source: §7.3 · pp. 16–17

7.4 Evaluation Insights: Objective Alignment vs. Subjective Preference

• Objective alignment and subjective preference diverged: Eval 3 had 52% more torque-alignment events under intent conditioning but no significant difference in overall participant ratings.Torque alignment aggregates directional consistency over a trial, whereas subjective ratings reflect retrospective judgments of particular moments.Source: §7.4 · p. 17

7.5 Generalization Across Domains and Platforms

• Intent conditioning showed utility across racing and urban domains, and the policy transferred to the GRIP physical platform without retraining or structural changes.GRIP testing included longitudinal assistance, which exposes drivers to physical interaction forces.Source: §7.5 · p. 17

7.6 Limitations and Future Work

• The three formal evaluations were underpowered for definitive claims about subjective preference, and the preliminary GRIP test does not substitute for a formal on-vehicle evaluation.The human-study safety filter constrained trajectory visualization only and did not modify assistive torque.Source: §7.6 · pp. 17–18

• DREAM2ASSIST extends a recurrent state-space model with intent inference and value-aligned rewards so assistance steers with the human’s objective rather than merely completing the task.The policy infers discrete objectives from observed behavior.Source: §8 · p. 18
• Across three human-subjects evaluations in racing and urban driving, intent conditioning improved measured human-robot coordination.In the double-blind urban evaluation, it yielded 52% more torque-alignment events and fewer collisions (2.7% vs. 7.7%) than the non-intent-conditioned baseline.Source: §8 · p. 18
• The deployment pipeline combines automated policy screening, counterfactual log replay, and human-in-the-loop testing, with interface design resolved before formal policy-quality evaluation.The paper argues that the pipeline generalizes to domains where learned policies must be vetted with human partners.Source: §8 · p. 18
• The fixed-intent training assumption limits applicability when a human’s objective changes within an episode, and the preliminary GRIP transfer still needs validation in a powered on-vehicle study.The authors also identify individual differences in preferred autonomy level as motivation for adaptive intervention styles.Source: §8 · p. 18

Ethical approval

• The research was overseen by WCG IRB on behalf of Toyota Research Institute, and participants gave written informed consent covering study participation, data use, and anonymized-result publication.Source: Ethical approval · p. 18, Informed consent · p. 18

Declaration of conflicting interests

• The authors declared no potential conflicts of interest and disclosed Toyota Research Institute support for the work.Source: Declaration of conflicting interests · p. 18, Funding · p. 18

Supplemental material

• Supplementary videos accompany the article, covering representative racing and urban trials and the GRIP physical-platform demonstration.Source: Supplemental material · p. 18

A Model Architecture

• The appendix describes the RSSM architecture and states that its template is shared across domains, with differences in observation/action spaces, reward configuration, and replay-buffer composition.Both domains use state-based observations rather than pixels, with MLP-based encoders and decoders.Source: A Model Architecture · p. 23, §A.2 · p. 23

A.3 Actor-Critic Architecture

• The actor and critic use two-layer, 512-unit MLPs; the actor outputs a continuous-action distribution, while the critic estimates scalar values using λ-returns and slow-target regularization.The supplied configuration specifies λ = 0.95 and γ = 0.997.Source: §A.3 · p. 23

A.5 FCP Human Population

• Fictitious co-play supplies human-policy populations spanning skill levels for racing intents and left, right, and fast behaviors for urban driving.Urban sampling reflects expected behavior frequencies, and AI training uses population play in all episodes.Source: §A.5 · p. 23

B.1 Policy Shaping

• The reward design combines shared task terms with intent-specific human-objective rewards, configured by domain and training role.Task terms include collision and boundary penalties, action smoothness, and a speed/progress bonus.Source: §A.6 · p. 23, §B.1 · p. 23

B.1.1 Pass-Stay Rewards

• Racing pass/stay reward restructuring supports both intended behaviors and penalizes deviation to address mode collapse toward always passing or always staying behind.The earlier configuration used a dense reward for being ahead within a pass-range window.Source: §B.1.1 · pp. 23–24

B.1.2 Left-Right Rewards

• In urban driving, the intent-conditioned lateral reward penalizes positioning on the side inconsistent with left or right intent when the ego vehicle is near the opponent.Source: §B.1.2 · p. 24

B.1.3 Stop-Go Rewards

• GRIP extends the intent space with go, slow, and stop behaviors, using sparse intent-specific rewards to prevent mode collapse across longitudinal modes.Under stop intent, the default speed/progress bonus is suppressed and collision thresholds account for reduced stopping distance.Source: §B.1.3 · p. 24

B.1.4 Collision / Bound Rewards

• Collision proximity uses a three-circle vehicle approximation and defines inner and outer zones; inner-zone collisions terminate racing episodes but not urban episodes.The appendix gives τ_inner = 3.0 m and τ_outer = 6.0 m; it also specifies hard track-edge and linearly increasing soft-boundary penalties.Source: §B.1.4 · p. 25

B.1.5 Action Smoothness Penalty

• A scheduled penalty on frame-to-frame steering-angle changes discourages jitter, growing from negligible early in training to enforce smoother steering later.The schedule uses μ_max = 20 and λ = 0.0002.Source: §B.1.5 · p. 25

B.1.6 Speed and Progress Bonus

• The tile-based bonus rewards track progress, with an added racing incentive for visiting a tile not yet visited.The bonus is dampened according to how many other cars have already visited the tile; when two agents are used and the ego arrives second, φ = 0.5.Source: §B.1.6 · pp. 25–26

B.1.7 Smoothing Human / AI Policy Outputs

• A causal moving average over the most recent 10 actions smooths fictitious-human outputs and, in later evaluations, AI outputs before hardware and visualization.The filter was introduced in Eval 2 to reduce physically unrealistic high-frequency oscillations and prevent learning from jittery simulated-human behavior.Source: §B.1.7 · p. 26

B.2 Interface Design Considerations

• The HMI combines haptic steering torque, visual trajectory cues, and auditory feedback, with design variations tested to balance transparency and cognitive load.Explored variations included trajectory lines, audio utterances, inferred-intent displays, colors, and line lengths.Source: §B.2 · p. 26

B.2.1 Trajectory Visualization

• The trajectory visualization projects a filtered RSSM rollout of up to seven steps, encoding acceleration or deceleration by color and assistance magnitude by line length in urban driving.In racing, line length remains fixed at the full rollout horizon.Source: §B.2.1 · p. 26

B.2.2 Haptic Feedback

• Steering haptics were tuned so wheel torque communicated assistance without overwhelming driver input; engine noise and line color conveyed longitudinal assistance because pedal haptics were difficult to apply.Source: §B.2.2 · p. 26

B.2.3 Additional Modalities

• Rule-triggered audio utterances were excluded after participants found them delayed, sparse, and difficult to comprehend while driving.Source: §B.2.3 · p. 26

B.2.4 Iterative Piloting & Main Takeaways

• Iterative pilots with 37 participants informed domain-specific choices: audio and inferred-intent information burdened racing drivers, while urban onboarding used staged modalities and comprehension checks.Urban design also applied the safety filter to displayed trajectories and scaled line length by assistance magnitude; Figure 11 shows racing trajectory-line variants tested during piloting.Source: §B.2.4 · pp. 26–27, Figure 11 · p. 27

Procedure

• Participants observed a racing lap, completed one warm-up and five coached laps, then drove five uncoached laps before the overtaking task.The coaching was intended to help participants, many without prior sim-racing experience, maintain basic vehicle control.Source: §C.1 · p. 27

Task design

• Each block paired two opponent-following laps with two overtaking-attempt laps, and the study increased difficulty through a staircase design.Participants advanced only after successfully completing a scenario; scenarios were completed with and without the policy.Source: §C.1 · p. 27

Sample and measures

• The final analysis sample was 24 after two of 33 recruits withdrew because of motion sickness and seven were excluded.After policy interactions, participants rated perceived intent recognition, maneuver help, safety, and whether passing without AI would have been better.Source: §C.1 · p. 27

C.2 Methods for Evaluation 3: Urban Controlled Validation

• Participants drove six laps under each of D2A-Full, D2A-NoIntent, and No-policy, alternating passing side in randomized, double-blind condition order.The urban study used a three-phase onboarding sequence—haptics, trajectory visualization, then both—with comprehension checks before advancing.Source: §C.2 · p. 28

D.1 Eval 1: Racing Characterization

• D2A improved straightaway overtake success from 80% to 89% (p = .069), but turn success fell from 51% to 21% (p < .001).Figure 14 plots paired participant overtake rates by track type, with and without the policy; straightaway clearance increased by 0.47 m while turn speed and clearance decreased.Source: Figure 14 · p. 28, §D.1 · p. 28

5. I felt safe driving with the AI assistant. (Safety)

• Urban validation recruited 12 participants and analyzed 10 after two exclusions; participants rated assistance direction, timing, adaptation, fighting the AI, safety, and teaming.The oncoming-car encounter was specified for the first four ratings.Source: §5 · p. 28, §C.2 · p. 28

D.2 Eval 2: Racing Refinement

• D2A applied larger steering torque than D2A-Refined before overtakes on turns, while their straightaway torque profiles were similar.Figure 16 shows averaged torque traces separated into straight and turn trials.Source: §D.2 · p. 29

D.3 Eval 3: Urban Controlled Validation

• Seven of seven selected participants showed less D2A-Full misalignment than D2A-NoIntent 3 to 1.5 seconds before closest approach, but five showed greater misalignment in the final second.D2A-NoIntent steered on average against the driver's intended avoidance direction before closest approach; five participants were excluded from these plots for data-quality issues.Source: §D.3 · p. 29
• Among 11 participants with both conditions, seven had higher overall torque under D2A-Full and eight experienced more intent-aligned torque.The appendix reports these per-participant patterns as consistent with aggregate alignment-event counts described elsewhere.Source: §D.3 · p. 29

Platform transfer

• The learned policy transferred to the physical GRIP platform without retraining, with examples of steering around an oncoming vehicle and braking on a roundabout approach.The illustrated trials show 13–14 mph lateral assistance and braking from 9 to 4 mph, respectively.Source: §D.4 · p. 29
• Across the two sessions with both policies, D2A-Full crossed the roundabout gate 2–3 mph slower than D2A-NoIntent and kept more distance to the crossing vehicle on average.The reported gate speeds were 8.7 vs. 10.7 mph and 12.2 vs. 15.0 mph; observations are descriptive because N = 3, and GRIP logs do not isolate AI-applied braking.Source: §D.4 · p. 29

Policy interpretation

• The developmental longitudinal variants differed in reward tuning, but all intent-conditioned policies used lateral intents; braking emerged from task-level rewards rather than a separate longitudinal intent space.Source: §D.4 · p. 29

Policy selection

• Candidate policies passed staged screening using training metrics, counterfactual replay of human logs, and internal pilot pass/fail criteria before external deployment.Screening included task progress, collision rate, intent accuracy, intervention sparsity, collision propensity, and control-input track orthogonality.Source: §D.4 · p. 32

Interface design

• Deployment design combined trajectory cues and tuned steering haptics, while longitudinal assistance used engine noise and line color because pedal haptics were impractical.The trajectory display encoded acceleration or deceleration by green or red and scaled line length by assistance magnitude in urban driving.Source: §D.4 · p. 32