Source-linked AI summary
Demo: Vision-Language Model-Guided Online Calibration of an Electromagnetic Digital Twin
Zerui Kang, Yishen Lim, Zhouyou Gu, Seungnyun Kim, Seung-Woo Ko, Tony Q. S. Quek, Jihong Park
TL;DR
Online EM digital-twin calibration must handle changing material conductivities, initialization sensitivity, and the travel cost of measurements. The paper uses two VLM calls to initialize visible materials and choose waypoints online; in an indoor scenario, it reaches low conductivity error with less travel than random waypoint planning.
Problem
Online EM digital-twin calibration must accommodate changing environmental properties, while existing approaches typically optimize offline on pre-collected measurements and fixed positions cannot adapt to evolving RSS errors.
Source: §1 · p. 1Method
Two online VLM calls classify visible materials into conductivity priors for Sionna calibration and select the next waypoint using RSS calibration error and image coverage.
Source: §1 · pp. 1–2, §2 · p. 2Results
1.74 × 10^-4 NMAE is reached within 20.06 m with waypoint planning and VLM priors, versus 3.66 × 10^-4 after 43.69 m with random planning; random initialization stays near 100% NMAE.
Source: §3 · p. 2Takeaways & Limitations
The demonstration indicates that VLM priors support convergence and online waypoint planning reaches calibration targets with less travel than random planning.
Source: §3 · p. 2
Abstract
from arXiv · showhide
An electromagnetic (EM) digital twin gives mobile robots wireless situational awareness but depends on material conductivities that change with the environment. Online calibration faces initialization sensitivity and measurement travel costs. We demonstrate a vision-language model (VLM)-guided framework using a Unitree G1 robot and NVIDIA Sionna, with two VLM calls: material classification maps visible materials through ITU-R P.2040 to conductivity priors for Sionna's gradient descent on accumulated received signal strength (RSS) measurements; waypoint planning selects the next measurement location online using residual RSS calibration error and image coverage. In a real indoor scenario, the framework achieves a normalized mean absolute conductivity error of $1.74\times10^{-4}$ within 20 m of travel; random initialization never converges, while random waypoints require over twice the travel.
1 Introduction
Online EM digital-twin calibration must adapt to changing material properties while limiting measurement effort. The paper proposes coupling VLM-based material initialization with waypoint planning that responds to calibration errors and image coverage.
1 Introduction
- The framework targets wireless situational awareness for mobile robots, whose EM digital-twin fidelity depends on recalibrating environmental properties as they change.Prior methods use differentiable ray tracing on pre-collected measurements, while the cited prior VLM work fixed measurement positions in advance.Source: §1 · p. 1
- The proposed online VLM framework addresses offline calibration and fixed waypoint selection by adapting measurement locations from preceding observations.Its two VLM calls run in each round rather than initializing all parameters and fixing positions before calibration.Source: §1 · pp. 1–2
- Material classification initializes newly observed objects at each waypoint, while RSS calibration errors and image coverage guide the next waypoint.The loop feeds newly collected images and RSS measurements into the next round.Source: §1 · pp. 1–2
2 System Design
At each waypoint, the system classifies visible materials, refines conductivity estimates from RSS, and plans the next measurement using calibration error and image coverage.
2 System Design
- The Unitree G1 measures RSS from a fixed 5G NR transmitter while Sionna models the same carrier frequency and antenna locations.The twin and GPT-5.4 VLM run on a desktop; only material conductivities are calibrated.Source: §2 · p. 2
- Material classification labels segmented objects in four camera views and maps labels through ITU-R P.2040 to conductivity priors.Objects outside the camera views retain their previous estimates; the VLM also records observed-object coverage.Source: §2 · p. 2
- Sionna refines conductivities by gradient descent on measured-versus-simulated RSS error and checks loss and parameter steadiness over 50 iterations.The reported convergence indicator is steadiness of both loss and parameters over the last 50 iterations.Source: §2 · p. 2
- Waypoint planning uses room layout, RSS calibration error, and image coverage to target uncertain materials and unobserved objects.A measurement calibrates materials along propagation paths from the base station to the robot.Source: §2 · p. 2
3 Observations
The ablations show that VLM initialization is needed for convergence, while adaptive waypoint planning reduces travel and yields more accurate calibration than random planning.
3 Observations
- The evaluation compares estimated conductivity against predefined ground truth, with NMAE plotted against traveled distance.Sionna uses the predefined material conductivities to generate the radio map, while those parameters remain unknown to the twin during calibration.Source: §3 · p. 2
- Randomly initialized runs stay near 100% NMAE regardless of planner, whereas both planners converge with VLM priors.The reported explanation is that random values lie too far from the true values for gradient descent to update them in the correct direction; VLM priors place conductivities where the RSS gradient points toward nearby true values.Source: §3 · p. 2
- The proposed scheme reaches 3.66 × 10−4 within 20.06 m, while random planning needs 43.69 m; its calibrated radio maps have 7 × 10−3 dB MAE versus 2.027 dB.Source: §3 · p. 2
4 Demonstration
The demonstration presents the full pipeline in a photorealistic Isaac Sim replica of a meeting room, with an interactive dashboard for inspecting and changing materials or measurement positions.
4 Demonstration
- The Isaac Sim replica reproduces room layout, object placement, robot viewpoints, and measurement locations, while real-world sensing noise and radio-hardware effects remain to be validated.Attendees can change object materials or select measurement positions; the remote setup requires network access and AC power for a laptop and monitor.Source: §4 · p. 2