PaperlayerView exact revision on arXiv

Source-linked AI summary

Betting for Sim-to-Real Performance Certificates

Yujia Chen, Bowen Weng

arXiv:2608.21572v1cs.RO

TL;DR

Robot performance certificates are often loose because real-world trials are expensive and scarce. The paper introduces sim-to-real betting certificates that use simulator-informed sequential bets and proves anytime validity regardless of simulator quality. Across synthetic and robot experiments, the method narrows certificate width by 51.6% ± 16% against applicable baselines, including a 32.26% ± 8% reduction with ≤30 samples.

  • Problem

    Expensive, scarce real-world robot trials make performance certificates loose, while existing sim-to-real approaches can require restrictive paired simulation-real observations and lack practical simulator-design guidance.

  • Method

    The framework uses a scalable simulator bank to determine predictable betting stakes for e-process confidence sequences, while real-world observations provide the statistical evidence.

  • Results

    51.6% ± 16% certificate-width reduction was achieved against nine applicable baselines, with 32.26% ± 8% reduction for ≤30 rounds.

  • Takeaways & Limitations

    The confidence sequence remains simultaneously valid over time regardless of simulator-bank accuracy, while wealth-regret bounds provide guidance for simulator-bank design and configuration.

Abstract

from arXiv · show

Consider a typical test of a robot system: one observes a sequence of outcomes concerning some aspect of interest (crash or no crash, tracking error, time to completion), and reports a mean (crash risk, average error, mean time to completion) and, more importantly, an interval guaranteed to contain that mean at a prescribed confidence, referred to as a performance certificate. Given expensive real-world trials, the sample size is therefore small, and the certificate is often loose. Now consider the same procedure, except that before each real outcome is revealed, the operator ``peeks'' at a large bank of simulated results, and places a bet on where the real outcome will land. As the real outcomes settle the bets, the operator gains or loses wealth. One's ``trust'' over simulators also shifts within the portfolio. This paper develops that idea into a sim-to-real betting certificate framework with three contributions: (i) An algorithm that links a scalable bank of simulators to effective bets, and the accumulated betting wealth to the certificate. (ii) A proof that the returned certificate is anytime valid, covering the true mean with the prescribed probability, using any simulator bank. (iii) The guaranteed wealth-regret bounds yield configuration principles for the proposed algorithm and simulator bank design to deliver tight certificates. Experiments across synthetic distributions and real-world robot tests, covering both replayed standardized testing outcomes and online runtime evaluation, show the proposed method narrows the certificate by $51.6\%\pm16\%$ against classic and state-of-the-art baselines, and by $32.26\%\pm8\%$ in the extremely limited-sample regime ($\leq30$ samples).

I. INTRODUCTION

Robot performance certification must balance tight uncertainty bounds against the scarcity and cost of real-world trials. The paper motivates using inexpensive simulators to support probabilistic certificates while retaining prescribed confidence.

  • Sim-to-real motivation: The paper proposes leveraging increasingly abundant and inexpensive simulators to reduce reliance on costly real-world tests while providing guaranteed performance certification.The motivating setting includes bounded performance measures such as risk, success rate, and normalized tracking error.
  • Motivation: Finite real-world samples can substantially deviate from the target mean, making uncertainty quantification difficult in robot testing.This issue is especially relevant when sampling is extremely limited.
  • Certificate formulation: A performance certificate is an interval containing the unknown mean, with width measuring certificate tightness.The interval is denoted C = [r_l, r_u], and its width is r_u − r_l.
  • Concentration-based certificates: Classical certificate construction uses concentration inequalities whose confidence requirement is δ(γ, t) ≤ α.Hoeffding’s inequality uses the observation range, while empirical Bernstein additionally incorporates empirical variance.

2) Hypothesis test based certificate:

Hypothesis-test certificates retain candidate means that are not rejected, while e-process betting supports sequential evidence accumulation through nonnegative wealth processes. The reviewed methods differ in assumptions, stopping validity, and how quickly evidence can tighten the certificate.

  • Hypothesis-test certificate: Hypothesis-test certificates evaluate each candidate mean µ and exclude it when its test statistic exceeds an α-determined threshold.The certificate is the set of candidate values whose null hypotheses remain accepted.
  • E-value approach: E-processes test candidates sequentially using wealth processes that remain nonnegative supermartingales under the null hypothesis.Ville’s inequality provides the associated time-uniform control.
  • Betting interpretation: Betting stakes are placed before each observation, with positive or negative payoffs depending on how the outcome deviates from the candidate mean.Wealth tends to grow when the tested candidate is false, accelerating its rejection.
  • Stake selection: Ideal Kelly chooses stakes from the true mean and variance, while practical methods substitute predictable empirical estimates and clip the stake for valid wealth factors.The aggressiveness parameter κ controls the oracle’s betting strength.
  • Motivation for simulation: The reviewed real-observation-only methods are fundamentally limited in certificate tightness by the scarcity and cost of real-robot trials.This motivates supplementing real-world evidence with inexpensive, scalable simulations.

B. Probabilistic Certificate (with “Simulators”)

The paper introduces sim-to-real betting certificates that use simulator-guided, predictable bets while relying on real-world observations for statistical evidence. The resulting confidence sequence is anytime valid regardless of simulator quality, and wealth-regret analysis guides simulator-bank and algorithm design for tighter certificates.

  • Relation to prior approaches: The framework avoids requiring paired simulation and real-world trials, unlike approaches that estimate sim-to-real corrections from matched outcomes.Paired trials can impose substantial practical constraints and may be impossible when reliable real-to-simulation correspondence is unavailable.
  • Certificate construction: The method constructs e-process-based confidence sequences rather than only point estimates by integrating simulator-guided betting with e-process hypothesis testing.The algorithm outputs a sequence of certificates over testing rounds.
  • Coverage: Coverage remains simultaneously valid over time whenever stakes are predictable and safely clipped, regardless of simulator quality.The simulator bank can affect certificate tightness without affecting the coverage guarantee.
  • Design guidance: Wealth-regret bounds quantify how simulator-bank and algorithm choices approach an optimal simulator and translate that gap into design guidance.The analysis targets simulator design, bank construction, and hyperparameter selection.
  • Algorithm: Sim-to-real betting uses a scalable simulator bank to determine bets while using only real-world observations as statistical evidence.Simulator moments are combined through trust weights to set the direction and magnitude of predictable bets against candidate means.
  • Algorithm: The algorithm updates simulator trust from how well each predicted performance distribution explains incoming observations, then updates e-process wealth using those observations.Before each new observation, the current trust-weighted mean and variance determine the bet against each candidate value.

B. The coverage guarantee

The coverage guarantee comes from inverting an e-process into a sequence of candidate-mean certificates. It permits continuous inspection and data-dependent reporting or stopping while preserving the prescribed simultaneous coverage, independent of simulator accuracy.

  • Coverage guarantee: The confidence sequence satisfies P(µ∗∈C_t for all t ∈ Z_T) ≥ 1 − α.This is the stated anytime-valid coverage guarantee for the certificates returned by Algorithm 1.
  • Coverage guarantee: Anytime validity allows experimenters to inspect certificates continuously and select reporting or stopping rounds based on observed data without invalidating coverage.This contrasts with methods valid only at predetermined sample sizes or times.
  • Coverage guarantee: Simulator accuracy does not affect coverage, although inaccurate simulator banks may produce certificates that are not as tight as desired.Simulator quality therefore influences tightness rather than validity.
  • Certificate inversion: The e-process accumulates evidence against each candidate mean and rejects a candidate once its wealth reaches 1/α.The certificate consists of candidates that have not been rejected.
  • Wealth regret: Wealth regret compares Algorithm 1 with ideal Kelly through a normalized worst-candidate positive log-wealth difference over real observations.The positive part retains only cases where Algorithm 1 has less wealth, and division by t gives the average accumulation rate.
  • Illustration: Figure 2 compares expected average wealth regret and theorem bounds for dense and separated simulator banks, alongside their means and variances.The two simulator sets yield similar regret while satisfying different theorem structures.

2) The expected wealth-regret bound in general:

The general expected wealth-regret bound decomposes performance into simulator approximation, identification, and score-fluctuation costs, with sensitivity factors linking moment errors to regret. These terms yield concrete principles for simulator-bank density and learning-rate selection.

  • General bound: The first bound requires no additional structural assumptions on the simulator bank beyond those stated in Algorithm 1.It applies to a general simulator bank through a lemma connecting moment-estimation error to wealth regret.
  • Stability: Moment-estimation errors contribute through sensitivity constants measuring mean and variance error, with c_stab combining their effects in the wealth-regret bound.The stability lemma evaluates Algorithm 1 and ideal Kelly on the same real-observation sequence.
  • General bound: The analysis reduces wealth-regret control to how closely trust-weighted simulator moments track the unknown real mean and variance.It compares the trust-weighted score with the best fixed simulator, converts score differences into moment error, and applies moment-to-wealth stability.
  • Bound terms: The approximation term ϵ_K is the expected Gaussian-loss difference between the best fixed simulator and the true moment pair, and is zero when the true moment is in the bank.A denser bank can reduce this term by increasing the chance of including a moment-accurate simulator.
  • Bound terms: The identification cost log K/(ηt) decreases with more real observations, while the fluctuation term ηb_score^2/8 increases with learning rate.A larger learning rate reacts more aggressively to score fluctuations during trust-weight updates.
  • Design principles: The learning rate should balance bank size, testing horizon, and score variation: larger banks call for moderately larger η, whereas larger T or greater variation calls for smaller η.This guidance follows the competing identification and fluctuation terms in the regret bound.

3) The expected wealth-regret bound under simulator separation:

Under simulator separation, trust concentrates geometrically on a distinguishable reference simulator, yielding a tighter wealth-regret bound when its moments are accurate. The dense-bank and separated-bank analyses provide complementary explanations of Algorithm 1 under different bank structures.

  • Simulator separation: Each competitor’s expected relative-trust multiplier must be bounded by a common ρ < 1, causing expected trust outside the reference to decrease geometrically.Smaller ρ implies faster trust concentration and a faster-decreasing simulator-selection term.
  • Simulator separation: A separated simulator bank requires an accurate reference simulator whose mean and variance are close to the real-world moments.Persistent moment errors are measured by h_m and h_v.
  • Wealth-regret bound: When the reference simulator matches the real-world mean and variance, persistent regret terms vanish and expected average wealth regret decreases at O(1/t).The tighter rate depends on the stronger separation assumption, which may fail when simulators receive nearly identical scores.
  • Comparison of bank structures: The dense-bank analysis does not require a distinguished simulator, whereas the separated-bank analysis obtains a tighter theoretical bound from trust contraction.In the toy example, both bounds have similar empirical performance despite relying on different bank structures.
  • Practical configuration: Theorem 1 is useful with little prior information about simulator accuracy, while Theorem 2 applies when a locally randomized simulator approximately mirrors the real system.The separated setting is relevant when physical parameters remain uncertain but the simulator design is already approximately matched.

III. EXPERIMENT

The experiments evaluate Algorithm 1 across synthetic distributions, online humanoid command tracking, and replayed standardized robot tests. Across tasks and sample sizes, the proposed methods generally produce substantially narrower certificates while retaining the relevant coverage comparison.

  • Cross-task results: Algorithm 1 yields the tightest certificate across nearly all tasks and sample sizes, spanning synthetic distributions and physical robot performance measures.The physical evaluations include mobile-manipulator, quadruped, and humanoid-robot systems.
  • Coverage comparison: The t-test occasionally produces narrower bounds but lacks the coverage guarantee provided by Algorithm 1.The comparison concerns a fixed-sample parametric method versus the proposed anytime-valid guarantee.
  • Cross-task results: 51.6% ± 16% certificate-width reduction is achieved against 9 applicable baselines across the shared comparisons.The worst proposed variant, Sim 7 biased, still reduces width by 22.9% ± 60%.
  • Limited-sample regime: 32.26% ± 8% certificate-width reduction is achieved for sample sizes of ≤30 rounds.The corresponding reductions are 10% ± 3% for ≤10 rounds and 37.3% ± 12% for ≤50 rounds.
  • Simulator-bank design: With three appropriately separated MuJoCo simulators, Sim MuJoCo 3 performs on par with denser banks and converges faster early under limited samples.The ablation also supports using a moderately large η for denser banks.

V. PROOFS

The proofs establish how Algorithm 1 converts moment and stake discrepancies into wealth-regret bounds. They rely on bounded stakes, safe clipping, and summation of per-round log-wealth differences.

  • Proof strategy: Moment errors alter the betting stake, and stake errors alter the one-round logarithmic wealth factor.The proof bounds sensitivity to inserted mean and variance changes before combining the two coordinate changes.
  • Proof strategy: Safe clipping keeps intermediate wealth factors positive by ensuring 1 + λ(y_τ − µ) > δ.This permits a bound on the change in one-round log wealth caused by stake differences.
  • Proof conclusion: Summing the one-round bounds over time yields the wealth-regret inequality, and taking square roots establishes its companion bound.The final steps use the product definitions of the wealth processes and the definition of R_t.
  • Proof conclusion: The proof concludes once the per-round inequalities are combined across the sequence of observations.The argument is completed after applying the derived stake and log-wealth bounds.

C. Proof of Theorem 1

Theorem 1’s proof compares trust-weighted simulator losses with the true-moment oracle and converts excess loss into weighted moment error. Convexity and finite-sum inequalities then produce the wealth-regret bound.

  • Trust-update analysis: Simulator trust-update losses lie in a bounded interval because outcomes and simulator moments are bounded.Their negatives, accumulated by Algorithm 1, have the same range, enabling the exponential-sum argument.
  • Trust-update analysis: The trust-update analysis compares cumulative trust-weighted loss with the smallest cumulative simulator loss in the bank.Each simulator’s expected trust-update loss is related to the loss obtained from the true moments.
  • Moment-error conversion: Nonnegative trust weights summing to one bound each weighted moment’s squared error by the corresponding trust-weighted average individual squared errors.This converts excess trust-update loss into weighted mean and variance errors.
  • Moment-error conversion: Jensen’s inequality and Cauchy–Schwarz convert the per-round weighted moment bounds into cumulative bounds over the finite testing horizon.Combining these inequalities with Lemma 1 completes the proof.

D. Proof for Lemma 2

The proof invokes Lemma 2’s definition, the dependence structure of the betting terms, and i.i.d. fresh observations to complete the argument.

  • The proof begins from Lemma 2’s given definition.
  • The betting term depends only on observations before round t, while Gk depends on the fresh observation y_t.
  • Using the stated dependence and i.i.d. sampling assumptions, the proof concludes.

E. Proof for Theorem 2

The proof separates reference and competitor errors, bounds the competitor contribution, applies Lemma 1, and bounds S_t to establish the claimed result.

  • The proof first separates reference error from competitor error by definition.
  • The competitor contribution is bounded using prior equations, defined quantities, and the triangle inequality.
  • Lemma 2 and the definition of S_t are then applied.
  • The final step bounds S_t using the ceiling function, completing the proof of equation (21).

VI. EXPERIMENT DETAILS

The experiments detail reproducibility support, real-world and paired simulation data collection, and robot-system test setups spanning mobile manipulation and quadruped stability.

  • Experiment Details: The section supplements the main empirical results with details on C2 and C3 and the released reproducibility code base.
  • Experiment Details: Real-world velocity-error samples use joystick commands at 50 Hz and measurements from the OptiTrack Motion Capture System.
  • Experiment Details: The same recorded commands are replayed in MuJoCo to establish paired real-world and simulation data for SureSim.
  • Experiment Details: A diverse MuJoCo simulation bank is constructed by varying the environment’s physical parameters.
  • C3 - peg-in-hole: The Type B CMMA uses a mobile robot cart transporter and manipulator with an end-effector laser sensor for fiducial interception.
  • C3 - peg-in-hole: The fiducial-interception experiment varies search algorithm and mobile-base translation speed in a full 2^3 factorial design.
  • C3 - peg-in-hole: Performance is measured by fiducial interception rate and average fiducial search time.
  • C3 - pushover: The Unitree Go2 pushover test uses a pendulum-driven steel-ball impact, with varying body pose contributing to performance-measure uncertainty.
Loading 2608.21572v1…