PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Better Imagined Rollouts Mean Better Robot Control? A Controlled Study of World-Model Evaluation Under Feedback

Dharini Raghavan, Amritpal Singh

arXiv:2609.02811v1cs.RO

TL;DR

This paper examines whether offline prediction metrics reflect closed-loop control performance when predictive state estimators operate with intermittent measurement corrections. Across a differential-drive path-tracking task, it finds that evaluation rankings depend on both rollout horizon and measurement-update schedule, with feedback-matched rollouts remaining more informative than long uncorrected ones.

  • Problem

    Predictive models in robotics are commonly evaluated with offline fidelity or rollout errors, but it remains necessary to determine which measures inform the behavior of complete robotic systems operating in feedback.

  • Method

    The study compares replay estimation error, multi-step measurement-free rollout error, and closed-loop tracking across six observers and 24 sensing conditions, then varies rollout horizon, measurement-update interval, and recurrent-estimator training outages.

  • Results

    Long prediction horizons remain informative with regular measurement corrections, whereas correspondence with closed-loop performance deteriorates substantially when long propagation uses infrequent or absent updates.

  • Takeaways & Limitations

    Predictive-model evaluation should reproduce the sensing and feedback structure of the intended robotic application rather than treating prediction horizon as the sole measure of temporal difficulty.

  • Takeaways & Limitations

    The conclusions are specific to a controlled setting using a three-dimensional state, one simulated differential-drive platform, one reference-path family, and limited controller and model types, without visual predictive models or physical closed-loop experiments.

Abstract

from arXiv · show

Predictive models are increasingly used in robotics for state estimation, planning, control, and policy evaluation, yet they are often judged by open-loop prediction accuracy over a fixed horizon. In closed-loop operation, a robot repeatedly acts, receives new measurements, updates its state estimate, and recomputes control. We study this difference in a differential-drive path-tracking task with biased odometry and intermittent landmark sensing. Six state estimators are evaluated across 24 sensing conditions using trajectory replay, a 20-step measurement-free rollout, and closed-loop tracking. Replay position RMSE correlates more strongly with closed-loop cross-track RMSE than rollout error (Spearman rho = 0.923 vs. 0.774) and selects a different estimator from the closed-loop optimum in 5/24 conditions, compared with 18/24 for the rollout metric. We then vary rollout horizon and measurement-update interval. With H=20, rank agreement decreases from rho = 0.916 with measurements at every step to rho = 0.774 with no measurements. A horizon-update grid shows that long prediction horizons remain informative when regular corrections are retained, whereas long rollouts without correction can produce rankings that differ substantially from closed-loop behavior. We also test recurrent estimators trained on longer sensing outages. This improves the EKF-anchored models under combined sensing degradation, reducing GRU-EKF cross-track RMSE from 1.72 m to 1.06 m, but the gain is not consistent across isolated outages or estimator architectures. These results show that predictive-model evaluation in robotics should specify both prediction horizon and measurement-update schedule. For models used in feedback, offline rollouts are most informative when their sensing and correction pattern reflects closed-loop operation. Code is available at https://github.com/rdharini2001/Robot_World_Model

1 Introduction

The paper examines whether offline prediction metrics reflect closed-loop robot behavior when sensing corrections and feedback alter future trajectories. In a controlled testbed, it compares replay, measurement-free rollout, and closed-loop evaluation while varying measurement schedules and outage training.

  • Predictive-model evaluation matters because robotic systems repeatedly estimate state, act, receive measurements, and recompute control.
  • The study uses a differential-drive robot with biased odometry and intermittent landmark sensing to isolate prediction, correction, and feedback effects.
  • Six estimators are evaluated across 24 sensing conditions using trajectory replay, a 20-step measurement-free rollout, and closed-loop path tracking.
  • At H = 20, rank agreement decreases from ρ = 0.916 with measurements every step to ρ = 0.774 without measurements.
  • Long rollouts remain informative with regular corrections, whereas long observation-free rollouts can produce rankings that differ from feedback behavior.
  • Longer-outage training improves EKF-anchored estimators under combined degradation but does not yield uniform gains across outage conditions or estimator architectures.GRU-EKF cross-track RMSE decreases from 1.72 m to 1.06 m under the reported combined condition.

2 Related Work

The related work situates this study at the intersection of world-model control, predictive accuracy, and state estimation. It narrows those broad questions to a lower-dimensional setting where measurement correction and feedback can be examined separately.

  • World models predict system dynamics for planning, policy learning, and action evaluation, including latent-state rollouts for control-policy learning.
  • The study examines this related accuracy-control issue in a lower-dimensional setting that separates prediction, measurement correction, and feedback.
  • Prior work shows that improved predictive accuracy does not necessarily produce improved policy performance in model-based reinforcement learning.
  • Model-based state estimation commonly combines analytic structures such as the EKF with learned components for modeling errors or residual dynamics.
  • External sensing infrastructure can provide global robot-pose observations that complement or replace portions of onboard localization.

3 Controlled Experimental Testbed

The controlled testbed uses a planar differential-drive robot whose feedback controller relies on an observer combining odometry with intermittent landmark corrections. Analytic and recurrent residual observers are tested under systematically degraded sensing and extended outages.

  • The robot state is planar position and heading, and a pure-pursuit controller follows a lemniscate reference trajectory.
  • Wheel odometry is corrupted by wheel slip and gyroscope bias, while landmark sensing varies in noise, visibility, gyroscope bias, and complete outages.
  • Because control uses the estimated pose, estimation errors alter commands, subsequent motion, and later measurements.
  • Dead reckoning propagates odometry alone, whereas the EKF adds landmark-based measurement updates to the motion model.
  • Recurrent residual observers retain DR or EKF as an analytic reference and learn an additive residual correction using GRU or SSM architectures.
  • Extended-outage training masks landmark measurements while preserving vehicle dynamics and odometry, isolating the effect of missing corrections.

4 Evaluation Protocols

The evaluation compares offline estimation and rollout metrics with online tracking under different information and feedback structures. Replay preserves a fixed trajectory, measurement-free rollout removes landmark corrections, and closed-loop tracking lets observer errors change future behavior.

  • The three protocols compare conventional offline accuracy, measurement-free prediction, and closed-loop control using the same observers.
  • Replay processes a common recorded trajectory with measurement updates, so observer differences do not alter future inputs.
  • Nominally, GRU-EKF has the lowest replay position error and closed-loop cross-track error, while learned residual correction benefits most from an EKF reference.
  • The measurement-free rollout propagates each observer for H = 20 steps using realized odometry while withholding future landmark measurements.
  • Closed-loop tracking feeds each observer’s pose estimate to the controller while landmark measurements arrive according to the tested sensing condition.
  • Offline rankings are compared with closed-loop rankings across 24 sensing conditions using rank correlation, optimum-selection frequency, and selection regret.
  • Across sensing-degradation sweeps, the best observer changes with sensing condition, while EKF-anchored estimators remain the most stable group.

5 Results

Across sensing conditions, replay-based estimator scores track closed-loop performance better than long measurement-free rollouts, while rollout validity depends strongly on measurement corrections. Longer-blackout training helps some EKF-anchored models under combined degradation but produces architecture- and condition-dependent outcomes.

  • Nominal estimator performance: GRU-EKF achieves the lowest replay position error and closed-loop cross-track error under nominal sensing.SSM-EKF also performs well, while DR-anchored recurrent estimators have larger tracking errors and higher divergence rates.
  • Offline criteria versus feedback: ρ = 0.923: replay position RMSE correlates with closed-loop cross-track RMSE across 144 observer-condition pairs, but selects a different estimator in 5 of 24 conditions.Strong aggregate association does not guarantee identical estimator selection for each operating condition.
  • Offline criteria versus feedback: ρ = 0.774 versus ρ = 0.923: the 20-step measurement-free rollout has weaker closed-loop rank agreement and selects a non-optimal observer in 18 versus 5 of 24 conditions.Maximum selection regret rises from 0.0347 m for replay position RMSE to 0.1208 m for the measurement-free rollout.
  • Longer-blackout training: GRU-EKF cross-track RMSE decreases from 1.717 m to 1.061 m under combined degradation after extended-blackout training, but gains are not uniform.SSM-EKF also improves under combined degradation, while DR-anchored SSM worsens and SSM-EKF degrades across the isolated outage sweep.
  • Horizon and correction schedule: ρ = 0.916 at k = 1 versus ρ = 0.774 with corrections withheld: less frequent measurement updates reduce agreement with closed-loop rankings at H = 20.Selection failures increase from 5 of 24 conditions at k = 1 to 18 of 24 when measurements are withheld entirely.
  • Horizon and correction schedule: At H = 80, ρ = 0.32 with no measurement corrections, compared with ρ = 0.90 when measurements are incorporated every step.Long horizons remain informative with regular updates, whereas long observation-free rollouts can substantially change estimator rankings.

6 Implications for World-Model Evaluation in Robotics

Offline evaluation should reflect the information pattern of the intended robotic application, including sensing, state updates, and control recomputation. Prediction accuracy should therefore be assessed alongside agreement with closed-loop behavior.

  • Offline benchmarks may differ from deployment because robots receive new measurements and update state estimates during operation.
  • Evaluation should specify both rollout horizon and measurement-update frequency rather than treating horizon as a complete protocol description.
  • Prediction error should be reported together with closed-loop agreement, including ranking agreement, selection accuracy, regret, or task success.
  • State propagation and measurement correction should be evaluated separately because new measurements can substantially alter the state estimate and subsequent control action.
  • Long-horizon evaluation remains relevant for planning, model-predictive control, and policy learning when those applications rely on extended model propagation.

7 Limitations

The study isolates feedback evaluation in a deliberately controlled setting, limiting how broadly its conclusions can be generalized. Several experiments also constrain interpretation of training and outage-matching results.

  • The primary experiments use one simulated differential-drive platform, one reference-path family, a pure-pursuit controller, and three-dimensional state without image observations.
  • Training variability is estimated from three seeds for the principal EKF-anchored recurrent estimators, fewer than the five or more runs suggested for stronger assessment.
  • Matching training blackout distributions to evaluation conditions does not consistently improve closed-loop behavior, so blackout matching alone is insufficient.
  • The combined sensing-degradation experiment represents one difficult condition rather than a full factorial combination of disturbance levels.
  • The study does not establish a universal metric for large visual world models or show that measurement-free rollouts are generally inappropriate for robotic planning.

8 Conclusion

The study finds that offline evaluation agrees with closed-loop observer performance only when its sensing and correction pattern resembles feedback operation. Training for longer outages helps under combined degradation, but benefits vary by regime and architecture.

  • Replay position RMSE is more consistent with closed-loop observer ranking than the 20-step measurement-free rollout, despite the rollout evaluating multi-step propagation.
  • The relevant evaluation variable is the combination of prediction horizon and measurement-update schedule, not horizon alone.
  • Long prediction horizons remain informative with regular corrections, whereas long propagation with infrequent or absent updates can diverge from feedback rankings.
  • GRU-EKF cross-track RMSE decreases from 1.72 m to 1.06 m under combined sensing degradation after training on longer sensing outages.
  • The improvement is not uniform across isolated outages or estimator architectures, and blackout-duration matching alone does not ensure better closed-loop behavior.

B Policy-Coupled Rollout Evaluation

A policy-coupled rollout feeds predicted states back into the controller, but this does not improve agreement with closed-loop observer rankings. Reproducing feedback requires matching the deployed sensing and update schedule, not merely including the controller.

  • The measurement-free rollout replays realized future odometry, so predicted states do not influence subsequent offline control inputs.
  • The policy-coupled rollout computes each next control input from the current predicted pose and propagates the trajectory using that command.
  • The comparison covers conventional replay, periodically corrected rollout with k = 5, measurement-free rollout, and policy-coupled rollout across 24 conditions and six observers.
  • ρ = 0.743 is the policy-coupled rollout's rank correlation, and 0.0479 m is its largest mean selection regret among the compared protocols.
  • Including the controller explicitly can still yield poor closed-loop ranking when the offline sensing and measurement-update schedule differs from deployment.

C Effect of Estimation-Error Structure on Control

Control performance depends not only on estimation-error magnitude but also on its temporal structure and direction. Counterfactual rollouts show that persistent displacement and heading bias can cause substantially different tracking errors than position RMSE alone suggests.

  • The counterfactual rollout held the trajectory, controller, dynamics, and position-error magnitude fixed while varying only error structure.
  • 0.30 m position RMSE produced tracking errors from 0.25 m for oscillatory error to 0.64 m for persistent post-outage displacement.The two sequences had identical position RMSE but differed in temporal structure.
  • Persistent displacement yielded approximately 2.5× the tracking error of high-frequency oscillatory error despite equal position RMSE.The controller partially attenuated rapidly alternating error but remained biased by persistent displacement.
  • Zero position RMSE from a heading-only bias still produced 0.29 m of tracking error because orientation error changes the steering command.
  • These temporal and directional effects help explain why offline estimation metrics can rank models differently from feedback-control performance.

D Controlling for Training-Set Size and Outage Distribution

The study separates training-set size from outage-distribution effects by comparing matched 300-trajectory variants. Extended-outage training helps under combined sensing degradation but does not consistently improve isolated-outage tracking.

  • 300-trajectory variants isolate training-data size from outage-distribution effects while holding architecture, optimizer, batch size, and initialization fixed.
  • Under combined degradation, Long-blackout-300 achieved 1.42 m versus 2.41 m for Standard-300 and 2.92 m for Deployment-matched-300 with SSM-EKF.
  • Under combined degradation, GRU-EKF errors were 1.06 m for Long-blackout-300, 1.40 m for Standard-300, and 2.12 m for Deployment-matched-300.
  • Under isolated long outages, Standard-300 performed best for both recurrent architectures, at 0.56 m for SSM-EKF and 0.22 m for GRU-EKF.
  • The benefits of extended-outage training depend on estimator architecture and sensing condition rather than training-set size or outage matching alone.

E Condition-Level Ranking Analysis and Statistical Significance

Condition-level analyses confirm that replay position RMSE agrees more closely with closed-loop behavior than the 20-step measurement-free rollout. Bootstrap and permutation tests show that the regret difference persists across sensing conditions.

  • The condition-level analysis computes ranking statistics separately within each of 24 sensing conditions rather than relying only on pooled observer-condition rankings.
  • Replay position RMSE had median per-condition Spearman correlation 0.943 and correctly ordered 90.3% of observer pairs.
  • Replay position RMSE had CVaR-20% regret of 0.0244 m versus 0.0590 m for the measurement-free rollout.CVaR-20% regret averages selection regret over the five highest-regret conditions.
  • The paired bootstrap produced a 95% confidence interval of [−0.033, −0.011] m, while the sign-flip permutation test gave p = 0.0001.
  • Comparing measurement-update endpoints gave the same −0.0214 m mean regret difference because the endpoints selected the same observers as replay and measurement-free rollout.
Loading 2609.02811v1…