Betting for Sim-to-Real Performance Certificates

Sim-to-real motivation

• The paper proposes leveraging increasingly abundant and inexpensive simulators to reduce reliance on costly real-world tests while providing guaranteed performance certification.The motivating setting includes bounded performance measures such as risk, success rate, and normalized tracking error.Source: §I · p. 1

Motivation

• Finite real-world samples can substantially deviate from the target mean, making uncertainty quantification difficult in robot testing.This issue is especially relevant when sampling is extremely limited.Source: A. Probabilistic Certificate (without… · p. 1

Certificate formulation

• A performance certificate is an interval containing the unknown mean, with width measuring certificate tightness.The interval is denoted C = [r_l, r_u], and its width is r_u − r_l.Source: A. Probabilistic Certificate (without… · p. 1

Concentration-based certificates

• Classical certificate construction uses concentration inequalities whose confidence requirement is δ(γ, t) ≤ α.Hoeffding’s inequality uses the observation range, while empirical Bernstein additionally incorporates empirical variance.Source: 1) Concentration inequality based… · p. 2

Hypothesis-test certificate

• Hypothesis-test certificates evaluate each candidate mean µ and exclude it when its test statistic exceeds an α-determined threshold.The certificate is the set of candidate values whose null hypotheses remain accepted.Source: 2) Hypothesis test based certificate · p. 2

E-value approach

• E-processes test candidates sequentially using wealth processes that remain nonnegative supermartingales under the null hypothesis.Ville’s inequality provides the associated time-uniform control.Source: 2) Hypothesis test based certificate · p. 2

Betting interpretation

• Betting stakes are placed before each observation, with positive or negative payoffs depending on how the outcome deviates from the candidate mean.Wealth tends to grow when the tested candidate is false, accelerating its rejection.Source: 2) Hypothesis test based certificate · p. 2

Stake selection

• Ideal Kelly chooses stakes from the true mean and variance, while practical methods substitute predictable empirical estimates and clip the stake for valid wealth factors.The aggressiveness parameter κ controls the oracle’s betting strength.Source: 2) Hypothesis test based certificate · p. 2

Motivation for simulation

• The reviewed real-observation-only methods are fundamentally limited in certificate tightness by the scarcity and cost of real-robot trials.This motivates supplementing real-world evidence with inexpensive, scalable simulations.Source: 2) Hypothesis test based certificate · p. 3

Relation to prior approaches

• The framework avoids requiring paired simulation and real-world trials, unlike approaches that estimate sim-to-real corrections from matched outcomes.Paired trials can impose substantial practical constraints and may be impossible when reliable real-to-simulation correspondence is unavailable.Source: B. Probabilistic Certificate (with… · p. 3, §II · p. 3

Certificate construction

• The method constructs e-process-based confidence sequences rather than only point estimates by integrating simulator-guided betting with e-process hypothesis testing.The algorithm outputs a sequence of certificates over testing rounds.Source: §II · p. 3, A. The sim-to-real betting for… · p. 4

Coverage

• Coverage remains simultaneously valid over time whenever stakes are predictable and safely clipped, regardless of simulator quality.The simulator bank can affect certificate tightness without affecting the coverage guarantee.Source: §II · p. 3

Design guidance

• Wealth-regret bounds quantify how simulator-bank and algorithm choices approach an optimal simulator and translate that gap into design guidance.The analysis targets simulator design, bank construction, and hyperparameter selection.Source: §II · pp. 3–4

Algorithm

• Sim-to-real betting uses a scalable simulator bank to determine bets while using only real-world observations as statistical evidence.Simulator moments are combined through trust weights to set the direction and magnitude of predictable bets against candidate means.Source: A. The sim-to-real betting for… · p. 4
• The algorithm updates simulator trust from how well each predicted performance distribution explains incoming observations, then updates e-process wealth using those observations.Before each new observation, the current trust-weighted mean and variance determine the bet against each candidate value.Source: A. The sim-to-real betting for… · p. 4

Coverage guarantee

• The confidence sequence satisfies P(µ∗∈C_t for all t ∈ Z_T) ≥ 1 − α.This is the stated anytime-valid coverage guarantee for the certificates returned by Algorithm 1.Source: B. The coverage guarantee · p. 4
• Anytime validity allows experimenters to inspect certificates continuously and select reporting or stopping rounds based on observed data without invalidating coverage.This contrasts with methods valid only at predetermined sample sizes or times.Source: B. The coverage guarantee · p. 4
• Simulator accuracy does not affect coverage, although inaccurate simulator banks may produce certificates that are not as tight as desired.Simulator quality therefore influences tightness rather than validity.Source: B. The coverage guarantee · p. 4

Certificate inversion

• The e-process accumulates evidence against each candidate mean and rejects a candidate once its wealth reaches 1/α.The certificate consists of candidates that have not been rejected.Source: C. The tightness guarantee · p. 4

Wealth regret

• Wealth regret compares Algorithm 1 with ideal Kelly through a normalized worst-candidate positive log-wealth difference over real observations.The positive part retains only cases where Algorithm 1 has less wealth, and division by t gives the average accumulation rate.Source: 1) Defining the wealth regret · pp. 4–5

Illustration

• Figure 2 compares expected average wealth regret and theorem bounds for dense and separated simulator banks, alongside their means and variances.The two simulator sets yield similar regret while satisfying different theorem structures.Source: Figure 2 · p. 5

General bound

• The first bound requires no additional structural assumptions on the simulator bank beyond those stated in Algorithm 1.It applies to a general simulator bank through a lemma connecting moment-estimation error to wealth regret.Source: 2) The expected wealth-regret bound in… · p. 5

Stability

• Moment-estimation errors contribute through sensitivity constants measuring mean and variance error, with c_stab combining their effects in the wealth-regret bound.The stability lemma evaluates Algorithm 1 and ideal Kelly on the same real-observation sequence.Source: 2) The expected wealth-regret bound in… · p. 5

General bound

• The analysis reduces wealth-regret control to how closely trust-weighted simulator moments track the unknown real mean and variance.It compares the trust-weighted score with the best fixed simulator, converts score differences into moment error, and applies moment-to-wealth stability.Source: 2) The expected wealth-regret bound in… · p. 5

Bound terms

• The approximation term ϵ_K is the expected Gaussian-loss difference between the best fixed simulator and the true moment pair, and is zero when the true moment is in the bank.A denser bank can reduce this term by increasing the chance of including a moment-accurate simulator.Source: 2) The expected wealth-regret bound in… · pp. 5–6
• The identification cost log K/(ηt) decreases with more real observations, while the fluctuation term ηb_score^2/8 increases with learning rate.A larger learning rate reacts more aggressively to score fluctuations during trust-weight updates.Source: 2) The expected wealth-regret bound in… · p. 6

Design principles

• The learning rate should balance bank size, testing horizon, and score variation: larger banks call for moderately larger η, whereas larger T or greater variation calls for smaller η.This guidance follows the competing identification and fluctuation terms in the regret bound.Source: 2) The expected wealth-regret bound in… · p. 6

Simulator separation

• Each competitor’s expected relative-trust multiplier must be bounded by a common ρ < 1, causing expected trust outside the reference to decrease geometrically.Smaller ρ implies faster trust concentration and a faster-decreasing simulator-selection term.Source: 3) The expected wealth-regret bound… · p. 6
• A separated simulator bank requires an accurate reference simulator whose mean and variance are close to the real-world moments.Persistent moment errors are measured by h_m and h_v.Source: 3) The expected wealth-regret bound… · p. 6

Wealth-regret bound

• When the reference simulator matches the real-world mean and variance, persistent regret terms vanish and expected average wealth regret decreases at O(1/t).The tighter rate depends on the stronger separation assumption, which may fail when simulators receive nearly identical scores.Source: 3) The expected wealth-regret bound… · p. 6

Comparison of bank structures

• The dense-bank analysis does not require a distinguished simulator, whereas the separated-bank analysis obtains a tighter theoretical bound from trust contraction.In the toy example, both bounds have similar empirical performance despite relying on different bank structures.Source: 3) The expected wealth-regret bound… · pp. 6–7

Practical configuration

• Theorem 1 is useful with little prior information about simulator accuracy, while Theorem 2 applies when a locally randomized simulator approximately mirrors the real system.The separated setting is relevant when physical parameters remain uncertain but the simulator design is already approximately matched.Source: 3) The expected wealth-regret bound… · pp. 6–7

Cross-task results

• Algorithm 1 yields the tightest certificate across nearly all tasks and sample sizes, spanning synthetic distributions and physical robot performance measures.The physical evaluations include mobile-manipulator, quadruped, and humanoid-robot systems.Source: §III · p. 8

Coverage comparison

• The t-test occasionally produces narrower bounds but lacks the coverage guarantee provided by Algorithm 1.The comparison concerns a fixed-sample parametric method versus the proposed anytime-valid guarantee.Source: §III · p. 8

Cross-task results

• 51.6% ± 16% certificate-width reduction is achieved against 9 applicable baselines across the shared comparisons.The worst proposed variant, Sim 7 biased, still reduces width by 22.9% ± 60%.Source: §III · p. 8

Limited-sample regime

• 32.26% ± 8% certificate-width reduction is achieved for sample sizes of ≤30 rounds.The corresponding reductions are 10% ± 3% for ≤10 rounds and 37.3% ± 12% for ≤50 rounds.Source: §III · p. 8

Simulator-bank design

• With three appropriately separated MuJoCo simulators, Sim MuJoCo 3 performs on par with denser banks and converges faster early under limited samples.The ablation also supports using a moderately large η for denser banks.Source: §III · pp. 7–8

Proof strategy

• Moment errors alter the betting stake, and stake errors alter the one-round logarithmic wealth factor.The proof bounds sensitivity to inserted mean and variance changes before combining the two coordinate changes.Source: B. Proof for Lemma 1 · p. 10
• Safe clipping keeps intermediate wealth factors positive by ensuring 1 + λ(y_τ − µ) > δ.This permits a bound on the change in one-round log wealth caused by stake differences.Source: B. Proof for Lemma 1 · p. 10

Proof conclusion

• Summing the one-round bounds over time yields the wealth-regret inequality, and taking square roots establishes its companion bound.The final steps use the product definitions of the wealth processes and the definition of R_t.Source: B. Proof for Lemma 1 · pp. 10–11
• The proof concludes once the per-round inequalities are combined across the sequence of observations.The argument is completed after applying the derived stake and log-wealth bounds.Source: B. Proof for Lemma 1 · pp. 10–11

Trust-update analysis

• Simulator trust-update losses lie in a bounded interval because outcomes and simulator moments are bounded.Their negatives, accumulated by Algorithm 1, have the same range, enabling the exponential-sum argument.Source: C. Proof of Theorem 1 · p. 11
• The trust-update analysis compares cumulative trust-weighted loss with the smallest cumulative simulator loss in the bank.Each simulator’s expected trust-update loss is related to the loss obtained from the true moments.Source: C. Proof of Theorem 1 · pp. 11–12

Moment-error conversion

• Nonnegative trust weights summing to one bound each weighted moment’s squared error by the corresponding trust-weighted average individual squared errors.This converts excess trust-update loss into weighted mean and variance errors.Source: C. Proof of Theorem 1 · p. 12
• Jensen’s inequality and Cauchy–Schwarz convert the per-round weighted moment bounds into cumulative bounds over the finite testing horizon.Combining these inequalities with Lemma 1 completes the proof.Source: C. Proof of Theorem 1 · pp. 12–13

• The proof begins from Lemma 2’s given definition.Source: D. Proof for Lemma 2 · p. 13
• The betting term depends only on observations before round t, while Gk depends on the fresh observation y_t.Source: D. Proof for Lemma 2 · p. 13
• Using the stated dependence and i.i.d. sampling assumptions, the proof concludes.Source: D. Proof for Lemma 2 · p. 13

• The proof first separates reference error from competitor error by definition.Source: E. Proof for Theorem 2 · p. 13
• The competitor contribution is bounded using prior equations, defined quantities, and the triangle inequality.Source: E. Proof for Theorem 2 · p. 13
• Lemma 2 and the definition of S_t are then applied.Source: E. Proof for Theorem 2 · pp. 13–14
• The final step bounds S_t using the ceiling function, completing the proof of equation (21).Source: E. Proof for Theorem 2 · p. 14

Experiment Details

• The section supplements the main empirical results with details on C2 and C3 and the released reproducibility code base.Source: §VI · p. 14
• Real-world velocity-error samples use joystick commands at 50 Hz and measurements from the OptiTrack Motion Capture System.Source: A. C2 - command tracking · p. 14
• The same recorded commands are replayed in MuJoCo to establish paired real-world and simulation data for SureSim.Source: A. C2 - command tracking · p. 14
• A diverse MuJoCo simulation bank is constructed by varying the environment’s physical parameters.Source: A. C2 - command tracking · p. 14

C3 - peg-in-hole

• The Type B CMMA uses a mobile robot cart transporter and manipulator with an end-effector laser sensor for fiducial interception.Source: B. C3 - peg-in-hole · pp. 14–15
• The fiducial-interception experiment varies search algorithm and mobile-base translation speed in a full 2^3 factorial design.Source: B. C3 - peg-in-hole · p. 15
• Performance is measured by fiducial interception rate and average fiducial search time.Source: B. C3 - peg-in-hole · p. 15

C3 - pushover

• The Unitree Go2 pushover test uses a pendulum-driven steel-ball impact, with varying body pose contributing to performance-measure uncertainty.Source: C. C3 - pushover · p. 15