Coupled Control and Wireless World Models for Resilient Remote Robotic Control

• Continuous transmission of high-dimensional sensory observations increases communication overhead, latency, and energy consumption.Source: §I · p. 1
• Wireless channel variations, packet losses, and environmental disturbances can degrade remote robotic control performance.Source: §I · p. 1
• World models and JEPAs learn compact predictive latent representations for forecasting system states from sensory observations.Source: §I · p. 1
• The paper extends coupled C-JEPA and W-JEPA modeling from a lightweight Gym environment to a realistic Gazebo–ROS–Sionna simulation using transfer learning.Source: §I · p. 2
• The framework is evaluated under wireless disturbances including scatterer changes, access-point handover, carrier-frequency shifts, and intermittent uplink failures.Source: §I · p. 2
• The study investigates raw and structured wireless representations and examines communication-aware latent world models in realistic robotic environments.Source: §I · p. 2, §II · p. 3

• A mobile robot sends visual observations over an uplink, while a remote controller returns control commands through a downlink.Source: §III · p. 3
• The controller uses uplink CSI or alternative RF-derived representations as wireless observations alongside visual observations.Source: §III · p. 3
• A binary scheduling variable ρ_t determines whether the robot transmits an observation or omits the uplink transmission.Source: §III · p. 3
• The framework aims to maintain reliable task performance, reduce communication overhead, exploit favorable channel conditions, and adapt to wireless and environmental changes.Source: §III · p. 4
• The coupled C-JEPA and W-JEPA jointly model robot movement and wireless-condition dynamics in a unified latent representation space.Source: §III · p. 4

• The C-JEPA encodes visual observations and uses an RSSM to represent robot dynamics with deterministic memory and stochastic uncertainty states.Source: A. Joint C-JEPA and W-JEPA Learning · p. 4
• The RSSM recurrent transition, prior, and posterior models update deterministic states and infer observation-conditioned stochastic states.Source: A. Joint C-JEPA and W-JEPA Learning · p. 4
• The learned latent states drive reward, termination, and control-policy models that generate robot actions.Source: A. Joint C-JEPA and W-JEPA Learning · p. 4
• The C-JEPA objective includes KL-divergence, reward-prediction, and terminal-prediction losses.Source: A. Joint C-JEPA and W-JEPA Learning · p. 4
• The W-JEPA encodes raw CSI, spectrogram images, or persistence images and predicts future wireless latent representations conditioned on control latents.Source: A. Joint C-JEPA and W-JEPA Learning · p. 4
• The W-JEPA predicts uplink transmission power, which informs scheduling decisions for transmitting current observations or continuing C-JEPA state prediction.Source: A. Joint C-JEPA and W-JEPA Learning · pp. 4–5

• Historical wireless observations are transformed over a window of length τ to improve identification of temporal changes in wireless conditions.Source: B. Structured RF Representations for… · p. 5
• Spectrogram construction applies windowing and FFT to create a time-frequency image from wireless observations.Source: B. Structured RF Representations for… · p. 5
• Persistence images map time-delay-embedded wireless observations into point clouds and encode topological features through persistent homology.Source: B. Structured RF Representations for… · p. 5
• The W-JEPA accepts raw CSI, spectrogram, or persistence-image inputs, whose performance impact is analyzed experimentally.Source: B. Structured RF Representations for… · p. 5

• Runtime changes considered include global illumination variation and local visual obstruction such as mud patches.Source: C. Adaptive Resilience Mechanism · p. 5
• Perception degradation is detected when the weighted latent prediction discrepancy over K future steps reaches the empirical threshold δ.Source: C. Adaptive Resilience Mechanism · p. 5
• For illumination changes, the system selects a predefined HSV transformation that minimizes latent inconsistency and applies it to incoming observations.Source: C. Adaptive Resilience Mechanism · pp. 5–6
• For local obstruction, a lightweight VAE reconstructs degraded observations using a subset of previously collected data before C-JEPA re-encoding.Source: C. Adaptive Resilience Mechanism · p. 6
• The adaptation ratio r and optimization epochs E trade off adaptation time against reconstruction performance.Source: C. Adaptive Resilience Mechanism · p. 6

• Gazebo supplies robot dynamics, perception, and navigation, while Sionna RT provides physics-based wireless channel modeling through electromagnetic ray tracing.Source: D. Sionna–Gazebo Integration · p. 6
• Because Gazebo and Sionna RT use different scene formats, Blender assigns radio materials and converts the environment for Sionna RT.Source: D. Sionna–Gazebo Integration · p. 6
• Figure 4 depicts the robot, sensing uplink, control downlink, Gazebo scene, and synchronized Sionna RT scene.Source: Figure 4 · p. 6
• The JetBot URDF and RGB camera are integrated into Gazebo, while ROS supplies robot pose to Sionna RT for CSI generation and visual input to C-JEPA.Source: D. Sionna–Gazebo Integration · p. 6

• At each simulation step, ROS synchronizes robot pose and velocity so RGB observations and Sionna-generated CSI represent the same robot state.Source: E. Experimental Setup · p. 6
• The evaluation uses a mobile RGB-camera robot navigating a closed-loop road while maintaining sensing uplink and control downlink communication.Source: E. Experimental Setup · pp. 6–7
• C-JEPA is pretrained in Gym CarRacing because training directly in Gazebo requires several days of continuous interaction before convergence.Source: F. Transfer Learning and Domain… · p. 7
• A major transfer challenge is the mismatch between Gym CarRacing and Gazebo observation spaces, addressed by transforming Gazebo RGB images into Gym-style observations.Source: F. Transfer Learning and Domain… · p. 7
• The Gazebo reward encourages road progress and efficient navigation through crossing-marker rewards and a per-step penalty.Source: F. Transfer Learning and Domain… · p. 7
• Episodes terminate for road deviation, prolonged stopping, overturning, or sufficiently low cumulative reward, with an additional failure penalty.Source: F. Transfer Learning and Domain… · p. 7
• A discrete-to-continuous mapping converts C-JEPA actions into the JetBot's linear and angular velocity commands.Source: F. Transfer Learning and Domain… · p. 7

Experimental setup

• The framework is evaluated in synchronized Gazebo–Sionna experiments comparing navigation and communication performance under limited wireless connectivity.The evaluation examines reliable remote control while reducing communication resource utilization.Source: §IV · p. 7
• All experiments use the same predefined closed-loop trajectory while selectively modifying wireless communication or visual perception conditions.Source: A. Experimental Evaluation Scenarios · pp. 7–8

Wireless perturbations

• E2–E4 test robustness to added scatterers, access-point handover, and carrier-frequency shifts from 2.14 GHz to 1.9 GHz or 2.6 GHz.Source: A. Experimental Evaluation Scenarios · p. 8, Figure 5 · p. 8

Representation and perception tests

• E5 compares raw CSI with spectrogram- and PI-based RF representations, while E6–E8 assess consecutive connectivity failures, lighting changes, and visual obstructions.Source: A. Experimental Evaluation Scenarios · p. 8

Baseline design

• DQN and PID provide representative navigation baselines implemented in the same Gazebo–Sionna environment under identical conditions.DQN uses the proposed transfer-learning strategy, while PID directly follows the reference trajectory from onboard camera observations.Source: B. Baseline Methods · p. 8
• Both baselines continuously transmit RGB observations and lack predictive wireless modeling, transmission scheduling, and wireless power prediction.Source: B. Baseline Methods · p. 8

Overall performance

• 57.1 s completion time, 775.4 cumulative average reward, 0.48 ms uplink latency, and 4.74 J uplink energy were achieved by the proposed framework in E1.DQN and PID used 9.21 J and 11.19 J, respectively, while PID required 77.3 s and achieved 716.1 average reward.Source: C. Overall Task Performance & Robustness · p. 8

Wireless robustness

• The proposed framework shows only minor completion-time variation and the lowest uplink energy across carrier-frequency and scatterer perturbations.DQN degrades more under these wireless changes, whereas PID remains relatively unaffected by its non-learned communication behavior.Source: C. Overall Task Performance & Robustness · pp. 8–9, Figure 6 · p. 9, Figure 7 · p. 9
• 100% navigation completion under access-point handover exceeded DQN’s 56.48% and PID’s 58.99% lap completion.Source: C. Overall Task Performance & Robustness · p. 9

Prediction quality

• Spectrogram- and PI-based W-JEPA representations consistently reduce prediction error versus raw CSI across all prediction horizons.The improvement becomes more evident at longer horizons, with PI-based W-JEPA achieving the lowest error distribution.Source: D. Analysis of The Structured RF… · p. 9
• Longer temporal windows generally produce lower prediction errors and reduced variability for both structured RF representations.Source: D. Analysis of The Structured RF… · p. 9

Computational tradeoffs

• Representation parameters trade prediction accuracy against generation time, while selected spectrogram and PI settings remain computationally efficient for online operation.The spectrogram uses STFT segment length L, whereas the PI uses time-delay embedding dimension m.Source: D. Analysis of The Structured RF… · pp. 9–10, Table III · p. 10
• Raw CSI has lower preprocessing cost, spectrograms offer a favorable accuracy–complexity balance, and PIs provide the lowest errors at higher generation cost.Source: D. Analysis of The Structured RF… · p. 10

Communication efficiency

• The proposed 400-dimensional latent embedding reduces communication payload by about 62-fold relative to the ViT baseline without sacrificing control performance.The ViT baseline requires at least 128 × 196 tokens for comparable performance.Source: E. Communication Efficiency Through… · p. 10, Figure 10 · p. 10

Communication resilience

• The proposed framework sustains stable reward for approximately 13 consecutive missing transmissions, compared with approximately 8 for the ViT baseline.Source: E. Communication Efficiency Through… · p. 10, Figure 11 · p. 10

Global perceptual degradation

• Illumination changes can cause perceptual inconsistency and controller failure, whereas online latent adaptation restores consistency and allows navigation to continue with slight interruption.The comparison is made under global perceptual degradation in E7.Source: F. Adaptive Resilience to Visual… · p. 10

Local visual obstruction

• Recovery from local visual obstruction depends on the historical-data ratio r and adaptation epochs ϵ, both of which increase adaptation time when raised.These parameters govern the computation used for online adaptation in E8.Source: F. Adaptive Resilience to Visual… · pp. 10–11

Recovery–sustainability tradeoff

• Faster recovery with lower r and ϵ is not always sustainable, creating a tradeoff between recovery speed and sustained performance.Maintaining performance for a longer duration requires significant computation in one or both adaptation parameters.Source: F. Adaptive Resilience to Visual… · p. 11

Overall resilience

• The mechanism recovers from global perceptual degradation and local visual obstruction without retraining the complete control policy.Online restoration of latent consistency supports stable navigation and operational continuity under challenging visual conditions.Source: F. Adaptive Resilience to Visual… · p. 11
• The framework jointly models robot and wireless dynamics to schedule communication predictively while maintaining navigation performance under perturbations.Structured RF representations improve wireless prediction accuracy, and the framework outperforms DQN, PID, and raw CSI-based methods across several evaluation dimensions.Source: §V · p. 11

Scope boundary

• The study remains bounded by a single simulated robot, single wireless link, and predefined navigation scenarios.Physical-platform validation and broader multi-robot, network, and environmental settings are identified as future extensions.Source: §V · p. 11

Configuration tables

• Hardware configuration is documented in Table IV.Source: Table IV · p. 12
• Software and wireless configurations are documented in Table V.Source: Table V · p. 12
• Hyperparameters are documented in Table VI.Source: Table VI · p. 13