PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Calibration Attack on Autonomous Vehicles

Liangkai Liu, Qingzhao Zhang, Kang G. Shin

arXiv:2608.28778v1cs.ROcs.CVcs.ETcs.LGeess.SY

TL;DR

Online camera–LiDAR calibration improves runtime alignment but introduces a mutable attack surface that prior AV attacks largely leave trusted. ACA uses one physically realizable poster to trigger recalibration and steer the estimator toward an incorrect transform, producing severe benchmark degradation and safety consequences across simulation and hardware evaluations.

  • Problem

    The paper asks whether a physical adversarial object can compromise online calibration, which prior AV security research largely treats as fixed, accurate, and trusted.

  • Method

    ACA jointly optimizes one printable poster’s geometry and texture to spoof the miscalibration detector and steer the calibration estimator toward an incorrect extrinsic.

  • Results

    33.9° mean marginal rotation damage was induced, and PointPainting Car 3D AP@R40 fell from 51.72 to 0.36 on KITTI; effects were reproduced across nuScenes, CARLA, and a Husky robot.

  • Takeaways & Limitations

    Online calibration is a practical, safety-critical attack surface whose corrupted state can persist across subsequent camera–LiDAR fusion operations.

  • Takeaways & Limitations

    Transferability is architecture-dependent: ACA transfers meaningfully to LCCNet but not to RGGNet, and roadside exploitability varies by scene.

Abstract

from arXiv · show

Autonomous vehicles (AVs) rely on accurate camera-LiDAR calibration for multimodal sensor fusion. In practice, calibration can drift due to vibration, temperature variation, or minor sensor displacement, motivating online calibration algorithms that detect and correct misalignment at runtime while allowing the vehicle to continue operating without a factory visit. Existing AV attacks largely assume correct calibration. We instead identify online sensor calibration as a new attack plane. A corrupted calibration update can persist across subsequent fusion operations, causing system-wide errors that propagate from perception to planning and control. We present Adversarial Calibration Attack (ACA), the first physical attack against camera-LiDAR online calibration. Using a single adversarial poster, ACA first spoofs the miscalibration detector to trigger the calibration process and then steers the calibration estimator toward an incorrect transformation. A unified optimization jointly designs the poster's geometry and texture for both objectives. We evaluate ACA across benchmark datasets, simulation, and physical experiments. On benchmark datasets such as KITTI and nuScenes, ACA induces up to 33.9 degrees mean rotational calibration error, thereby severely degrading object detection. In the CARLA simulator, the attack causes a collision when the corrupted calibration is accepted in vulnerable scenarios crafted by the attacker. On a real Husky robot, a printed adversarial poster successfully reproduces the calibration error. These results demonstrate that online calibration is a practical and safety-critical attack surface for AVs.

1 Introduction

Online camera–LiDAR calibration maintains sensor alignment during deployment but creates a mutable, safety-critical attack surface. ACA exploits the gated pipeline with one physically realizable artifact, corrupting calibration and downstream perception.

  • Motivation: Online calibration compensates for pose changes caused by vibration, thermal expansion, maintenance, and repair while the vehicle operates.Runtime updates reduce reliance on repeated factory service.
  • Attack surface: Prior AV security work largely treats calibration as fixed, accurate, and trusted, leaving adversarial robustness of the calibration process insufficiently studied.The open question is whether a physical object can trigger recalibration and write an incorrect transform despite honest sensor measurements.
  • Attack surface: A corrupted extrinsic misassociates correct LiDAR points with image regions and propagates geometric errors to downstream perception, planning, and control.Figure 1 illustrates projections shifting from vehicles onto roads, sidewalks, and neighboring objects when only the transform changes.
  • ACA: ACA uses one printable poster to spoof the miscalibration detector and steer the calibrator toward an incorrect extrinsic transformation.Joint geometry–texture optimization designs the artifact for both objectives under physically constrained rendering.
  • Evaluation: ACA is evaluated across KITTI, nuScenes, CARLA, and a physical Husky robot, covering calibration corruption, fusion degradation, collisions, and physical reproduction.The evaluation spans benchmark datasets, closed-loop simulation, and hardware experiments.

2 Background and Related Work

Camera–LiDAR fusion depends on an extrinsic transform that maps LiDAR points into the camera frame, while online calibration re-estimates it during deployment. ACA attacks this calibration layer rather than directly targeting sensors or fusion architectures.

  • Camera–LiDAR fusion and extrinsic calibration: The camera–LiDAR extrinsic Tℓ→c maps LiDAR points into the camera frame for cross-modal fusion tasks including detection, BEV perception, tracking, and planning.Cameras contribute appearance and semantics, while LiDAR contributes 3D geometry.
  • Camera–LiDAR fusion and extrinsic calibration: Vibration, thermal variation, maintenance, and mechanical displacement can change sensor pose, motivating online extrinsic re-estimation during operation.Online methods avoid repeated manual or factory calibration.
  • Miscalibration detection and runtime update: Online calibration systems first detect cross-modal inconsistency and then invoke a calibrator to determine the replacement extrinsic.This creates two security-critical decisions: when to update and what value to accept.
  • Prior physical attacks: Prior physical attacks modify sensor observations or downstream recognition, while fusion attacks target particular architectures or modality interactions.These lines of work establish physical-environment manipulation but differ from attacking the calibration module itself.
  • ACA’s distinct layer: ACA targets the module that produces Tℓ→c, so accepted corruption remains upstream of fusion and affects later operations using otherwise legitimate measurements.The attack therefore operates at a different layer from direct sensor-spoofing and fusion-detector attacks.

3 Attack Model

The attack model exposes only the physical sensor-input space and assumes uncompromised vehicle hardware, networks, calibration code, and control stack. ACA seeks to trigger recalibration and persist an incorrect extrinsic under runtime variation.

  • Attack objective: A corrupted camera–LiDAR extrinsic propagates into subsequent BEV occupancy, detection, and tracking steps while sensors continue reporting honest data.The calibration module is upstream of these downstream fusion operations.
  • System model: The AV maintains a current extrinsic Tℓ→c, and a detector invokes calibration when its drift score exceeds threshold τdet.The calibration network then refines the current transform through iterative updates.
  • Attack objective: ACA’s object first induces cross-modal inconsistency, then creates misleading visual–geometric correspondences that steer calibration toward an incorrect estimate.The resulting error persists in later fusion operations until recalibration or reset.
  • Attacker capabilities: The attacker may design the object and select its deployment site but need not generalize across arbitrary road segments or locations.The object must remain effective across victim motion, viewpoint, and sensing variations at the chosen site.
  • Attacker knowledge: The threat model assumes matched-surrogate gray-box knowledge of model families, sensor configuration, operating distribution, and surrogate-training data.Exact deployed parameters are not required according to the supplied assumption passage.
  • Out-of-scope components: The control stack, in-vehicle networks, sensor hardware, synchronization, firmware, calibration code, and detector implementation remain uncompromised.Only physical sensor inputs are exposed, with a raw-data buffer additionally allowed for the digital baseline.

4 Attack Design

ACA uses one physically realizable object to trigger online recalibration and then steer the calibrator toward an incorrect extrinsic. Its joint optimization preserves cross-modal consistency while targeting both gated pipeline stages and deployment variation.

  • Overview: ACA places a single physical object in the joint camera–LiDAR view to trigger recalibration before corrupting the resulting extrinsic estimate.The corrupted estimate is subsequently consumed by downstream fusion.
  • Cross-Modal Rendering: The attack models the artifact with geometry GA, deployment pose TA, and appearance CA, optimizing CA while fixing geometry and nominal pose.The abstraction supports printable surfaces on planar panels or other jointly observable physical objects.
  • Cross-Modal Rendering: The same physical surface generates opaque camera observations and LiDAR returns, preventing independent digital perturbations or transparency-based exploits.Camera texture resolution and LiDAR sampling density are represented separately.
  • Stage 1: Spoof the Miscalibration Detector: Stage 1 minimizes a hinged trigger loss that pushes the detector above τdet by margin m, then becomes inactive after reliable triggering.Clamping the gate objective lets optimization focus on corrupting the calibration estimate.
  • Stage 2: Corrupt the Extrinsic Estimate: Stage 2 unrolls all K calibrator iterations and optimizes the final estimate, so gradients target the converged output rather than an intermediate update.The final calibration error propagates through the refinement trajectory and attacked camera observation to CA.
  • Joint and Robust Optimization: ACA jointly optimizes trigger and corruption objectives over training frames and pose perturbations, targeting robustness within a chosen deployment scenario rather than arbitrary locations.The attacker controls deployment but not the victim’s exact runtime trajectory, distance, or viewpoint.

5 Implementation

The implementation uses a differentiable iterative calibration surrogate and a physically rendered planar artifact. Training samples multiple frames and pose perturbations, with validation revealing overfitting under a short-window setting.

  • Surrogate Calibration Network: The main surrogate is CalibNet with K = 10 differentiable refinement steps, while RGGNet is held out as the transfer target.LCCNet is included only in a print-robustness variant.
  • Physical Artifact: The physical artifact is a 3.6 m × 1.8 m planar board represented by NA = 4,100 LiDAR points on a measured 164 × 25 angular grid.The grid reproduces observed KITTI Velodyne return spacing at approximately 6 m.
  • Physical Artifact: The artifact uses a separate 89 × 90 RGB texture and is rendered as an opaque planar occluder after adding its point set to the LiDAR scene.Opaque area-averaged rendering replaces covered pixels instead of blending them with the background.
  • Training and Validation: Universal-artifact training uses a 16-frame KITTI window, batches of B = 4, and Keot = 8 pose perturbations per frame.The trigger and corruption losses are averaged across EOT samples during optimization.
  • Training and Validation: A 16-frame run overfits, with held-out validation damage falling from +53° to +30°, whereas a 48-frame window at base LR 2×10^-3 removes the effect.Validation is performed every 50 steps with EOT disabled and cached perturbations.

6 Evaluation

ACA is evaluated across benchmark datasets, a physical robot, and a closed-loop simulator using a complete two-stage attack chain. It triggers online recalibration, corrupts the estimated extrinsic, and produces substantial, persistent rotation damage with downstream consequences.

  • Experimental settings: ACA is evaluated on KITTI and nuScenes benchmarks, a Clearpath Husky A300 robot, and the CARLA closed-loop simulator.The benchmark pipeline measures detector triggering, calibration corruption, and downstream perception degradation.
  • Metrics: Rotation damage is measured marginally against the clean baseline, alongside translation damage and ASR> 5◦ over K = 10 refinement iterations.The 5◦ threshold corresponds to the upper bound of nominal in-distribution residual error.
  • Stage 1: Triggering online calibration: The joint artifact fires the global detector at 0.89, while hinged optimization prevents Stage-1 pressure from reducing Stage-2 corruption.Without hinging, Stage-2 damage decays from +43◦ to +8.6◦; the hinged version achieves +14.79◦ versus +14.5◦ for Stage-2-only optimization.
  • Stage 2: Corrupting calibration: At 15◦ initial miscalibration, ACA adds +18.9◦ marginal rotation damage at 96% ASR> 5◦, versus +3.5◦ for the identical gray-board control.Across 5–15◦ initial miscalibration, damage remains +17.1◦ to +19.9◦ with 96–98% success, while translation damage is ≤0.3 m.
  • Universality and mechanism: Across held-out sequences, ACA reaches +33.9◦ rotation damage, while its cross-modal texture contributes +8.6◦ to +8.9◦ beyond the board’s physical presence.The optimized texture is ineffective without LiDAR returns, and returns alone remain near the occlusion floor.

6.4 Downstream Consequences

ACA’s corrupted extrinsic persistently disrupts camera–LiDAR fusion across KITTI, nuScenes, CARLA, and a physical Husky evaluation, producing severe detection failures and safety or availability consequences.

  • KITTI: +58.1° mean marginal rotation damage affected KITTI object-validation frames, with 98% exceeding 10° under the corrupted extrinsic.PointPainting consumes the extrinsic, whereas PointPillars serves as an extrinsic-free control.
  • KITTI: 99.3% loss reduced PointPainting Car 3D AP@R40 from 51.72 to 0.36, while PointPillars remained statistically unchanged at 78.41 → 78.45.The control isolates calibration corruption rather than artifact points as the cause of fusion collapse.
  • Mechanism: 1.51 AP remained when only cross-modal paint lookup was corrupted, showing that incorrect association alone can nearly break fusion despite correct FoV geometry.The full deployment additionally corrupts FoV cropping and can remove real cars from the painted cloud.
  • nuScenes: On nuScenes, the attack drove mean box IoU from 0.38 to near zero and destroyed 96% of cleanly localized objects, compared with 86% for gray-board control.Median camera-feature displacement reached 437 px in a 512 px-wide image.
  • Closed-loop simulation: 7.72° and 7.89° calibration errors caused BEVFusion detections to collapse from 13 to 1 and from 15 to 2 in CARLA scenarios.Accepting the estimate caused collisions, whereas rejecting it caused a permanent stop.
  • Physical evaluation: +6.4° mean rotation damage and 62% ASR> 5° persisted on the physical Husky within ±1.5° placement error; at unseen ±3°, damage remained +4.5° with 59% ASR.The gray control stayed within 0.5° and had at most 1% ASR.

7 Discussion

Existing defenses may raise ACA’s difficulty but do not directly remove the calibration-layer vulnerability, because the attack produces mutually consistent camera and LiDAR observations.

  • Defense limits: Cross-sensor consistency defenses can be bypassed because ACA’s physical object produces consistent camera and LiDAR observations.Similar consistency-based miscalibration detectors can also be triggered by adversarial or benign near-field objects.
  • Defense limits: Tight rejection thresholds are difficult because legitimate calibration drift requires necessary runtime recalibration.The discussion therefore favors independently validating proposed extrinsics before commitment.
  • Defense limits: Adversarial training would need to protect both the detector and calibrator and may not generalize to unseen physical artifacts.Suggested independent checks include trusted historical bounds, geometric evidence, or diverse estimators.

8 Conclusion

The paper establishes online camera–LiDAR calibration as a physical attack plane: one adversarial poster compromises detector triggering and estimation, with persistent downstream and operational consequences.

  • Conclusion: ACA uses one adversarial poster to trigger an unnecessary recalibration and steer the estimator toward an incorrect extrinsic transformation.The corrupted transformation is reused by subsequent fusion operations, creating persistent system-wide misalignment.
  • Conclusion: PointPainting Car 3D AP@R40 fell from 51.72 to 0.36 across the reported KITTI and nuScenes evaluations, while an extrinsic-free detector remained unaffected.This conclusion identifies calibration corruption, rather than artifact points alone, as the fusion-breaking mechanism.
  • Conclusion: CARLA exposed a safety–availability dilemma: accepting corrupted calibration caused collision, while rejecting it could leave the vehicle permanently stopped.A printed board on a real Husky further demonstrated physical-world feasibility.

Open Science

The authors provide an anonymized artifact and reproduction materials while withholding print-ready physical-attack components unless disclosure review determines public release is safe.

  • Artifact: The anonymized repository includes attack implementation, configurations, evaluation scripts, environment specifications, dataset-processing scripts, CARLA configuration, and physical-experiment materials.It also provides metric-computation code and related experimental data.
  • Third-party components: Datasets, pretrained models, simulator assets, and other restricted third-party components are not redistributed; official download links and version information are provided instead.Checksums or configuration information are supplied where permitted.
  • Safety safeguards: The print-ready high-resolution texture and fully automated physical-deployment pipeline remain confidential to reviewers because public release could lower misuse effort.The public artifact is intended to provide a non-printable visualization and evaluation materials instead.

Ethical Considerations

The paper frames online camera–LiDAR calibration as a safety-critical weakness with potential impacts on road users and autonomous-system stakeholders. Experiments were conducted in controlled settings, with disclosure and artifact safeguards intended to reduce misuse.

  • The attack exposes online camera–LiDAR calibration as an upstream safety-critical attack surface.
  • A malicious actor could adapt the technique to disrupt deployed autonomous-system perception, potentially causing property damage, financial loss, or physical harm.
  • Digital evaluations used simulation and public autonomous-driving datasets, while physical experiments used researcher-controlled equipment in a controlled environment without public traffic.
  • The public artifact withholds the print-ready high-resolution texture and fully automated physical-deployment pipeline, while sharing vulnerability details and mitigations with maintainers before release.
  • Residual risk remains because adversaries could reimplement the attack from the technical description or transfer it to other sensor combinations.

A Physical Realizability

ACA is designed as a physically realizable printed poster whose geometry, pose, and texture are constrained by deployment conditions and robustness requirements. Its effectiveness depends strongly on placement near the sensor axis.

  • The artifact is a printed planar poster on a rigid frame, with fixed billboard-like geometry, bounded ground-supported pose, and learnable texture.
  • Optimization includes pose jitter of a few degrees and translation of a few tens of centimeters to model realistic deployment disturbances.
  • The printed surface requires continuous tone, a matte diffuse finish, and opaque rigid mounting, while the trigger survives a 20 mm ink smear across a 3.6 m board.
  • Stage 1 fires on 0.89 of held-out frames on-axis, 0.18 at 1 m lateral offset, and 0.06 at 2 m, matching the gray-board floor at the largest offset.
  • The deployment case is untested for straight-road KITTI windows because genuine roadside billboards are several metres off-axis, whereas the attack requires heading directly toward the board.

B Print Specification

The physical poster uses continuous-tone, matte, diffuse printing with rigid mounting and a specified full-scale geometry. Measurements show that the attack remains effective under common print-chain distortions.

  • The artifact is treated as a manufactured object whose physical requirements are measured rather than selected as preferences.
  • The poster is 3.600×1.800 m, printed 1:1 at 1 mm/px from an 89 × 90 texel grid of 40.4 × 20.0 mm texels.
  • Continuous-tone printing yields +14.79° and 0.96 fire, compared with +10.38° and 0.90 for hard-patch printing.
  • Under 8-bit quantization, the measurements remain 0.89 fire and +14.85°, while aggressive luma compression gives 0.75 fire and +15.65° damage.
  • A realistic ink limit produces 0.85 fire and +15.12°, and both attack stages survive ordinary CMYK printing without requiring a wide-gamut process.
  • Physical placement uses a plane perpendicular to the sensor forward axis, LiDAR-height center, 6.0 m distance, and on-axis alignment.
Loading 2608.28778v1…