Source-linked AI summary
Poster: FedWM-Guard: Thwarting Imagination Poisoning in Federated World Model-based Autonomous Driving
Sheng Liu, Panos Papadimitratos
TL;DR
Federated world-model autonomous driving can learn from distributed vehicle data without centralizing raw logs, but plausible client updates may corrupt long-horizon futures used by the planner. FedWM-Guard screens updates by induced rollouts and audits deployed predictions against later observations, with independent containment for persistent risk. The paper presents this defense architecture and outlines an evaluation plan rather than reporting empirical results.
Problem
Federated WM-based autonomous driving lacks defenses that assess whether plausible client updates alter safety-relevant futures used by the planner.
Source: §1 · p. 1Method
FedWM-Guard screens candidate updates on hidden-canary rollouts, audits deployed predictions against independent observations, and uses an independent fallback when risk remains high.
Source: §4 · pp. 2–3Results
The paper presents a defense that screens updates according to the futures they induce, rather than only their statistical appearance in parameter space.
Source: §7 · p. 3Takeaways & Limitations
The paper identifies federated WM-AD as a security domain with a threat surface and potential countermeasures centered on planner-facing futures.
Source: §7 · p. 3Open questions include calibrating audits across heterogeneous benign vehicles without rejecting rare safety-critical clients and mitigating D3’s counterfactual blind spot.
Source: §6 · p. 3
Abstract
from arXiv · showhide
Federated learning (FL) can improve world model (WM)-based autonomous driving (AD) without centralizing raw private vehicle data, but it also turns model aggregation into a safety-critical integrity boundary. We introduce a new threat in federated WM-AD, namely \emph{imagination poisoning}: compromised vehicles submit bounded WM updates that preserve benign short-horizon predictions yet corrupt long-horizon rollouts (e.g., trigger-conditioned) during training, thereby misleading a downstream planner. We present \emph{FedWM-Guard}, to the best of our knowledge, the first defense to characterize planner-facing rollouts in federated WM-AD, screen authenticated updates in hidden-canary scenarios, audit predicted futures against later observations, and invoke a WM-independent safety shield when persistent inconsistency is detected. Unlike parameter-space defenses, it scores what an update makes the model \emph{imagine}, not only how the update looks. We also outline how we plan to evaluate it under non-IID (not independent and identically distributed) data, adaptive attacks, and benign distribution shift. This work highlights an unexplored domain, federated WM-AD, and its threat surface and potential countermeasures.
1 Introduction
Federated learning lets vehicles improve world models without centralizing raw driving logs, but safety-relevant imagined futures create a security gap that parameter-space defenses do not address. The section introduces imagination poisoning and FedWM-Guard’s screening-and-audit approach.
1 Introduction
- Federated learning lets vehicles train world models on local data and share updates without uploading raw driving data.The motivation is that individual vehicles see only a narrow slice of driving, while rare events are distributed across the fleet.Source: §1 · p. 1
- Imagination poisoning preserves largely normal short-horizon predictions while corrupting selected long-horizon rollouts that influence the planner.Such semantic errors may evade validation-loss and parameter-space anomaly checks despite posing collision risk.Source: §1 · p. 1
- Prior work leaves a gap at the intersection of federated learning, world models, and autonomous driving, especially in detecting unsafe futures induced by plausible updates.The section asks whether bounded, individually plausible client updates can poison federated WM-AD and whether induced futures can reveal that influence.Source: §1 · p. 1
- FedWM-Guard screens updates by their induced futures, audits deployed rollouts against later evidence, and feeds verified audit outcomes into subsequent screening rounds.The section also outlines an evaluation plan before making empirical claims about the defense.Source: §1 · p. 2
2 System Model
The system model encodes recent observations and optional context into a latent state, rolls it forward under candidate actions, and exposes predicted futures to a planner. Participating vehicles train the world-model dynamics locally and submit updates.
2 System Model
- Historical observations and optional conditioning are encoded into a latent state that is rolled forward under a candidate action sequence.Inputs may include camera or LiDAR observations, ego state, and map context.Source: §2 · p. 2
- Prediction heads decode imagined rollouts into planner-facing representations, and the planner scores candidate sequences but executes only the selected sequence’s first action.Representations include future occupancy, object states, BEV semantics, or trajectories.Source: §2 · p. 2
- Each participating vehicle updates trainable world-model dynamics using local data and submits an update to federated training.The model denotes the number of participants as n.Source: §2 · p. 2
3 Threat Model
The threat model assumes authenticated but behaviorally untrusted vehicles that can coordinate or optimize malicious updates. Their objective is to alter planner-facing long-horizon futures while keeping short-horizon behavior and update constraints plausible.
3 Threat Model
- The attacker may control authenticated clients, poison local data or directly optimize updates, coordinate across rounds, and know the aggregation rule and semantic score.Direct actuator takeover and physical sensor destruction are out of scope.Source: §3 · p. 2
- The adversary seeks large deviation from a safe reference future while keeping benign short-horizon loss within ε of nominal and updates inside feasible aggregation and screening constraints.The intended effect is to preserve normal short-horizon predictions while steering the longer futures used by the planner.Source: §3 · p. 2
- The attack objectives include untargeted multi-step degradation and trigger-conditioned backdoors that alter predicted obstacles after several imagined steps.The trigger may involve a route, map, motion, V2X, or physical condition.Source: §3 · p. 2
- Colluding clients can split a malicious direction across benign-looking updates, while stale benign updates can widen the tolerance envelope used by norm and similarity filters.The attacker may time and shape submissions to hide within that envelope.Source: §3 · p. 2
4 FedWM-Guard
FedWM-Guard combines bounded authenticated participation with semantic screening of candidate updates, followed by deployment-time auditing and independent containment. Audit evidence can also inform later server-side screening rounds.
4 FedWM-Guard
- D1 authenticates and bounds client influence, while D2 screens candidate updates using hidden-canary rollout behavior rather than parameter statistics alone.The screening features include short-horizon error, deviation from a trusted or ensemble reference, safety-margin violation, and ensemble disagreement.Source: §4 · p. 2
- A calibrated statistic classifies updates as benign or poisoned, and survivors receive soft trust-weighted aggregation before the tentative global model is checked for coalition effects.The statistic’s weights may be learned on held-out benign and synthetically poisoned updates; its threshold targets a false-positive budget.Source: §4 · p. 2
- The planned comparators include FLTrust, FLAME, parameter-only screening, current-frame checking, and ablations.The poster identifies Table 1 as the evaluation plan, not as a reported result table.Source: §4 · p. 3, Table 1 · p. 3
- D3 replays the deployed model under logged realized actions and compares planner-facing predictions with observations derived independently from onboard sensors.Persistent prediction-reality inconsistency raises a risk alarm; persistence and uncertainty gating suppress transient noise.Source: §4 · p. 3
- When risk remains high, D4 restricts the planner with a fallback controller independent of the attacked latent dynamics, while verified regional alarms can add scenarios to later canary sets.The fallback uses ego kinematics and conservative obstacle envelopes derived through an independent perception pipeline.Source: §4 · p. 3
5 Evaluation Plan
The evaluation plan specifies a federated DreamerV3-style dynamics module across heterogeneous CARLA clients and an adaptive attack protocol. It fixes key evaluation settings before testing.
5 Evaluation Plan
- The planned setup federates a compact DreamerV3-style dynamics module across heterogeneous CARLA clients while keeping the encoder, planner, and shield fixed.Clients differ in maps, weather, and traffic as non-IID sources.Source: §5 · p. 3
- The attacker jointly minimizes clean degradation and defense score while maximizing delayed rollout error and planner risk, with full knowledge of Eqs. (3)-(4).The protocol optionally layers C1/C2.Source: §5 · p. 3
- On-vehicle feasibility assumes idle-time local fine-tuning, not in-drive or from-scratch training.Source: §5 · p. 3
- The plan defines ˆY as H-step BEV occupancy, scores trajectories against predicted occupancy, and compares it with independent CARLA-ground-truth occupancy perturbed by a perception-noise model.This makes the D2 score and D3 residual well-defined.Source: §5 · p. 3
- Budgets, participation, routes, thresholds, and seeds are fixed before evaluation.Source: §5 · p. 3
6 Discussion
The discussion identifies unresolved design and evaluation questions for federated WM-AD security, spanning semantic canaries, audit calibration, secure aggregation, and D3's counterfactual blind spot.
6 Discussion
- The planner-facing representation for semantic canaries must be interpretable yet independent enough.Source: §6 · p. 3
- Audit calibration across heterogeneous benign vehicles must avoid rejecting rare safety-critical clients.Source: §6 · p. 3
- Reconciling D2's per-candidate scoring with secure aggregation raises the open question of inspecting gradients while recovering gradient confidentiality at acceptable cost.Source: §6 · p. 3
- The discussion leaves open how to mitigate D3's counterfactual blind spot.Source: §6 · p. 3
7 Conclusion
Federated WM-AD creates an aggregation-stage risk of compromised clients poisoning planner-facing futures. FedWM-Guard screens updates by the futures they induce rather than by statistical unusualness alone.
7 Conclusion
- At model aggregation, a few compromised clients can poison the futures used for planning in federated WM-AD.Source: §7 · p. 3
- FedWM-Guard screens updates according to the futures they induce, rather than whether they appear statistically unusual.Source: §7 · p. 3
- The screening question is whether an update leads the vehicle to imagine an unsafe world.Source: §7 · p. 3