PaperlayerView exact revision on arXiv

Source-linked AI summary

When Do Intrinsic Rewards Lead to Exploration?

Scott W. Viteri, Laura Gomezjurado Gonzalez, Clark Barrett

arXiv:2610.02159v1cs.LG

TL;DR

Intrinsic rewards may favor experience that does not maximize counterfactual information. The paper compares policies by whether their histories can reproduce histories from alternative interactions, characterizes intrinsic-objective optima, and constructs a score that rewards strict exploration improvements.

  • Problem

    Intrinsic rewards can value prediction error or learning progress even when maximizing them need not produce the most informative experience available.

    Source: §1 · pp. 1–2

  • Method

    The paper defines exploration through reproducibility of alternative-policy histories and characterizes exact optima of intrinsic objectives against this criterion.

    Source: §1 · pp. 1–2

  • Results

    Specified count-based, prediction-error, empowerment, and information-gain objectives have maximizing policies that are Pareto-suboptimal at acquiring counterfactual information in the constructed examples.

    Source: §1 · p. 2, §3.1 · p. 6

  • Takeaways & Limitations

    Undiscounted cumulative information, Brier, and related belief-based objectives reward every strict exploration improvement on finite classes with positive prior mass at each world.

    Source: §1 · p. 2

    The native score depends on the interface and weights, has no general computable evaluator, and broad empirical superiority is unestablished.

    Source: §7 · p. 11

Abstract

from arXiv · show

Intrinsic rewards are designed to guide exploration in reinforcement learning by assigning value to an agent's experience, for example through prediction error or learning progress. However, maximizing these rewards need not produce the most informative experience available. We propose a formal criterion for exploration that compares policies by the counterfactual information they acquire: how well their histories can substitute for experience under alternative policies. We construct a single, simple environment in which specified count-based, prediction-error, empowerment, and information-gain objectives have maximizing policies that are Pareto-suboptimal at acquiring counterfactual information. We explain these failures and establish conditions under which existing intrinsic rewards successfully encourage optimal exploration. We also construct an objective that assigns a higher value whenever exploration strictly improves under our criterion.

1 Introduction

The paper defines exploration by whether one policy’s histories can reproduce the histories of alternative policies, then tests intrinsic-reward optima against this criterion. It identifies failures, conditions for successful exploration, and a score that rewards every strict improvement.

1 Introduction

  • The alarm example shows that varied or unpredictable observations can earn intrinsic rewards without revealing which world is actual.Inspection reveals whether the alarm will ever ring; play leaves the display active, whose known color distribution is identical across worlds.Source: §1 · p. 1, Figure 1 · p. 2
  • Exploration is compared by whether randomized transformations of one policy’s histories can reproduce alternative-policy histories uniformly across worlds.The transformation may use additional collection time and must meet the requested accuracy without receiving the world label.Source: §1 · p. 1
  • The paper characterizes exact intrinsic-objective optima on the alarm and separate examples, including objectives whose maximizing policies lose counterfactual information.It also studies a finite planning comparison of six objectives or controls across 22 world classes.Source: §1 · pp. 2–3
  • Undiscounted cumulative information, Brier, and related belief-based objectives reward every strict exploration improvement on finite classes with positive prior mass at each world.The paper also constructs a bounded score rewarding every strict improvement across countably many finite experiments.Source: §1 · pp. 2–3
  • Complete exploration may be infeasible, and the proposed scores have limits on computability.The paper identifies feasibility and computability as separate limits of its approach.Source: §1 · p. 3

2 Comparing the evidence policies acquire

The paper treats a policy’s finite interaction history as evidence and compares policies by whether one history can simulate another across worlds. It defines a preorder for this comparison and relates monotonicity of objectives to their maximizing policies.

2.1 From interaction histories to experiments

  • A policy and world response law together induce a distribution over finite action-observation histories, treated as a statistical experiment with the history as its signal.A finite native experiment is available by acting through the environment’s own interface from the empty history.Source: §2.1 · p. 3

2.2 Reproducing another experiment

  • A randomized decoder simulates a target experiment by transforming its signal, with deficiency measuring the smallest worst-world total-variation error.Replacing a target signal with a decoded simulation reduces the best expected payoff for bounded decision problems by at most that error.Source: §2.2 · pp. 3–4
  • The finitary process preorder compares policies by whether each fixed finite target can be reproduced arbitrarily accurately after enough collection time.Collection time and decoder may vary with target length and accuracy, but must work uniformly over worlds; a sufficient policy dominates every policy.Source: §2.2 · p. 4

2.3 Selecting policies by exact objective optimization

  • A strictly monotone objective gives greater value to any policy that strictly dominates another, excluding dominated maximizers in every feasible policy family.If a sufficient policy is feasible, exactly the sufficient policies maximize such a score; the paper’s comparisons otherwise study exact objective optima.Source: §2.3 · p. 4

3 Objectives with dominated maximizers

Across the alarm and separate MDP examples, several intrinsic objectives favor observations or learning paths that do not maximize reproducible evidence. Their outcomes depend on the reward and return horizon; some belief-based objectives succeed at finite horizons but tie policies in the complete return.

3.1 Variation without evidence about the world

  • For every H ≥64, maximizing the specified count bonus skips inspection, while complete-return and long-run-average objectives tie all policies.Keeping the display active sustains visits to more labels, although every policy’s complete return is +∞ and long-run average is zero.Source: §3.1 · p. 6
  • For every proper discount, Shannon and predictive Square select exactly the policies that skip inspection, rewarding known display randomness without learning about the world.Empowerment likewise selects policies that never inspect for each fixed probe length; state-visitation entropy and DIAYN have suboptimal optima in separate examples, while MOP optima read the unknown bit with probability 1/3.Source: §3.1 · p. 6

3.2 The path of learning can outweigh the evidence acquired

  • On the alarm, all four posterior-update rewards favor inspection, but Hellinger and Absolute prefer PARTIAL to FULL in the sixteen-world example, with eventual FULL deficiency 1/2.Information and Brier instead prefer FULL; the smaller sequential updates can earn more movement reward despite revealing strictly less.Source: §3.2 · p. 7

3.3 A finite-time advantage can disappear in the complete return

  • On the alarm, complete information and Brier returns tie all policies, including deficient ones, although both favor inspection at every positive finite horizon and proper discount.In the four-bit example, Hellinger and Absolute prefer PARTIAL both near discount one and at one; compression progress has a deficient complete-return optimum only under a fixed description language, initialization, and supplied background.Source: §3.3 · p. 7

4 Posterior objectives on finite and compact world classes

The paper relates cumulative belief-based rewards to posterior-score gains and asks when these objectives preserve exploration comparisons. On finite classes, strict convexity gives a monotone score; on infinite classes, average and uniform error comparisons need additional conditions.

4 Posterior objectives on finite and compact world classes

  • Undiscounted information and Brier rewards equal expected gains in negative entropy and squared-posterior scores, respectively.For continuous convex Ψ, the expected posterior-score gains increase and are bounded, so their limit exists.Source: §4 · p. 8
  • On finite nonempty world, action, and observation sets, a full-support prior and continuous strictly convex Ψ make JΨ strictly monotone; feasible maximizers are exactly sufficient policies.The theorem does not require that a policy identify the actual world with certainty.Source: §4 · pp. 8–9
  • On infinite classes, positive prior mass at every world alone does not make prior-average error imply uniform error; the alarm example retains uniform error 1/2 without inspection.The alarm's vanishing average uncertainty therefore does not establish uniform exploration comparison.Source: §4 · p. 9
  • If the behavior class is compact under convergence of finite-prefix masses, a prior positive at every world makes average and uniform finitary preorders coincide.With finite prior entropy, complete information is finite and strictly monotone for the uniform preorder; these assumptions restrict the class to be countable.Source: §4 · p. 9

5 A score that preserves every strict improvement

A rich countable family of finite target experiments represents exploration comparisons through their simulation errors. Positive weighted aggregation yields a bounded strictly monotone score, although complete exploration may be infeasible and score evaluation may be uncomputable.

5 A score that preserves every strict improvement

  • A rich countable family of finite native experiments represents the finitary preorder exactly: one policy dominates another iff every target's eventual simulation error is no larger.The family can approximate every finite native experiment arbitrarily closely and uniformly over worlds.Source: §5 · p. 9
  • Any positive weights summing to one give a native score bounded in [0, 1] and strictly monotone for the finitary preorder.The weights set priorities among simulation errors for different targets.Source: §5 · p. 9
  • When a sufficient policy is feasible, exactly the sufficient policies maximize the native score, with value one.Without feasibility, strict monotonicity still selects undominated policies.Source: §5 · p. 9
  • Complete exploration can be infeasible: in a four-world, two-bit example, every policy is undominated, yet none eliminates both reading errors.A strictly monotone score can select a compromise but cannot remove this constraint.Source: §5 · p. 9
  • No algorithm can evaluate every computable policy's real score to arbitrary precision in two deterministic computable worlds when the score is weakly monotone and favors startup revelation over waiting.Restricted settings may still permit effective optimization.Source: §5 · pp. 9–10

6 Comparing finite information-gathering choices

The finite-planning comparison tests whether short-interaction objectives acquire evidence useful for reproducing longer interactions. Native objectives often reduce held-out simulation error in these cases, but the evidence does not establish universal superiority or eventual exploration.

6 Comparing finite information-gathering choices

  • The study computes three-step policies for 22 classes of four or eight worlds and evaluates them against the hardest of 32,768 four-step targets excluded from optimization.The comparisons use finite planning objectives, not the eventual score Jw, and use no sampled training.Source: §6 · p. 10
  • Native mean has lower four-step evaluation error than ordinary Brier in 18 of 22 classes, with two ties and two losses.Native maximum, under a fixed native-mean policy's Brier return constraint, records 15 wins, one numerical tie, and six losses against that reference; this fixed-reference comparison does not isolate the reward constraint.Source: §6 · p. 10
  • In 21 of 22 reward-matched pairs, neither experiment can simulate the other exactly; the inspected-case results establish neither universal superiority nor eventual exploration.The authors note that the presentation and controls were designed after inspecting these cases.Source: §6 · p. 10

7 Related work and limits

The paper situates its criterion among experiment-comparison and exploration methods, distinguishing its initial-interface, uniform-simulation focus. It emphasizes that empirical superiority and tractable objectives remain unestablished.

7 Related work and limits

  • Orseau et al. (2013, Theorem 7) address prediction of alternative continuations from reached histories under prior and horizon assumptions; this paper targets initial-interface uniform simulation.The latter includes opportunities already lost.Source: §7 · p. 10
  • Blackwell comparison and positive weighted scalarization are established tools, while reward-free exploration gives acquisition and planning guarantees in finite episodic MDPs; this paper studies objective values assuming optimization succeeds.The comparison criterion uses Blackwell comparison and Le Cam deficiency (Blackwell, 1953; Le Cam, 1964; Torgersen, 1991).Source: §7 · p. 10
  • Broad empirical superiority is unestablished, the native score has no general computable evaluator, and validating world-class assumptions and developing tractable objectives remain open.The earlier sixteen-target benchmark failed its prespecified screen, with no wins over Brier.Source: §7 · p. 11

Ethics statement

The work is theoretical analysis and synthetic-environment experiments, with no human participants or personal data.

Ethics statement

  • The work involves no human participants or personal data.It consists of theoretical analysis and experiments in synthetic environments.Source: Ethics statement · p. 11

Reproducibility statement

The appendices document proofs, objective definitions, assumptions, and details needed to reproduce and verify the finite study.

Reproducibility statement

  • Appendix K records the finite study’s response laws, seeds, planning objectives, evaluation budgets, tolerances, and adverse results.The appendix distinguishes numerical certificates from an exact rational-table proof and describes the formal proofs’ scope; code, selected data, Lean sources, and reproduction instructions are also provided.Source: Reproducibility statement · p. 11

AI usage statement

The appendix develops the finite-history comparison between policies, showing how native experiments represent it and how complete information relates to prior-average simulation. It also gives conditions for strict scores and a counterexample where no policy is sufficient or dominated.

AI usage statement

  • AI assistants contributed to framing, claims, proofs, experiment design and execution, interpretation, exposition, and the Lean formalization; the authors take responsibility for the final work.The schematic in Figure 1 was also AI-generated.Source: AI usage statement · p. 11

A An irreversible choice with no sufficient policy

  • In the two-bit irreversible-choice example, no policy is natively sufficient, yet every policy is undominated because different initial reading probabilities trade off the two native targets.At least one of the two limiting decoding errors is at least 1/4; policies with the same initial reading probability can reproduce each other’s finite-history distributions.Source: A An irreversible choice with no… · p. 15

B Structural proofs

  • The finitary process preorder compares policies by whether sufficiently long histories can reproduce each finite target history uniformly over worlds, and the comparison is reflexive and transitive.Randomized finite native experiments and deterministic contingent plans impose the same zero-error limit requirement.Source: B Structural proofs · pp. 15–16

B.1 Why countably many native targets suffice

  • A countable family of actual finite native experiments represents the finitary preorder through coordinatewise eventual-error comparisons.The construction enumerates dense finite-depth policy tables; longer histories can reproduce shorter ones, but a maximizing score need not provide a common acquisition deadline when no sufficient policy is feasible.Source: §B.1 · pp. 16–17
  • The bounded weighted native score preserves the finitary preorder and strictly increases under every strict improvement; its attained maximizers are undominated, and are exactly the sufficient policies when one is feasible.The score combines eventual errors for the countable native targets using strictly positive weights summing to one.Source: §B.1 · pp. 16–17

B.2 Complete information and average simulation

  • For finite complete-information totals, under prior-average simulation, equality of complete information is equivalent to reverse prior-average simulation.No prior-support or identifying-policy hypothesis is required; the result distinguishes average simulation from the uniform comparison used by the finitary preorder.Source: §B.2 · p. 17
  • The strictness test for complete information requires that a uniformly comparable pair with reverse average simulation also be comparable in the reverse finitary direction.For potentially infinite information totals, strict uniform pairs whose two scores are both infinite must additionally be excluded.Source: §B.2 · p. 18

B.3 Positive guarantees for finite and compact classes

Posterior-based objectives strictly respect finitary improvements under stated finite or compact-class conditions, but terminal equivalence can erase differences in finite access. The appendix also specifies the prior and aggregation conventions for the alarm objectives.

B.3 Positive guarantees for finite and compact classes

  • With finite worlds and a prior positive on every world, continuous strictly convex posterior potentials are strictly finitarily monotone, including complete information and Brier returns.If a sufficient policy is feasible, the exact maximizers are precisely the feasible sufficient policies; strictness does not require attainable full revelation or a sufficient policy.Source: §B.3 · p. 19
  • For a compact Hausdorff world space with continuous controlled behavior and positive prior mass at every singleton, prior-average and uniform finitary preorders agree.Complete information strictly respects strict comparisons when the dominating total is finite; finite prior entropy suffices for finite totals, while complete Brier return is strictly monotone on countable classes without an entropy assumption.Source: §B.3 · pp. 19–20
  • Positive singleton mass is stronger than topological full support, and the guarantee is eventual for each fixed target rather than a common acquisition deadline or a method for finding a sufficient policy.Source: §B.3 · p. 20

B.4 Why terminal information can erase finite access

  • A countable-world pulse interface has policies that are strictly ordered by finite access but whose terminal experiments are Blackwell equivalent, so terminal-equivalence-invariant scores tie that pair.The complete pulse stream identifies the world, although at every finite time a no-pulse history is compatible with a later pulse.Source: §B.4 · p. 21
  • The terminal-equivalence result establishes a tied strict pair, not global optimality for every terminal-invariant score; a timestamped infinite-path score may retain prefix information.The shared alarm construction’s global complete-information and Brier ties are established separately by their objective formulas.Source: §B.4 · p. 21

C The alarm construction and specified objectives

  • The alarm-objective comparisons use a geometric prior with mass 1/2 on infinity and 2^−(k+2) on k, except for explicitly fixed-world quantities.Logarithmic rewards are in bits, nonnegative complete returns are limits of finite returns, and discounted returns weight the startup decision by one.Source: C The alarm construction and specified… · p. 21

C.1 The interface and its exact native error

On the alarm interface, several familiar objectives can maximize policies that fail to reproduce inspection histories, while fixed-horizon information and Brier returns and the native score favor sufficient inspection. The paper also specifies an ideal same-archive compression-progress score.

C.1 The interface and its exact native error

  • The common audit measures whether randomized transformations of one policy’s histories can reproduce another policy’s history distributions across worlds.This comparison is framed as a statistical experiment over finite histories.Source: §C.1 · pp. 21–22

C.2 Continuing pseudo-count bonuses at arbitrarily large horizons

  • For every H ≥64, every maximizing pseudo-count policy skips inspection and has inspection-target deficiency 1/2; complete and long-run average returns instead tie all policies.The threshold 64 is a sufficient, not optimal, bound.Source: §C.2 · p. 23

C.3 Information and Brier returns

  • At every fixed positive finite horizon, inspection maximizes information and Brier returns, with equality exactly when s = 1; complete-return maximizing ties can be insufficient.The contrast is between a fixed finite horizon and a complete return on the same countable world class.Source: §C.3 · p. 24

C.4 Sensor empowerment

  • For every positive probe horizon, all and only policies with s = 0 maximize long-run average sensor empowerment, yet sufficient inspection strictly dominates them.The optimizer set is unchanged as probe length varies; only the phase of alternating endpoint capacities changes.Source: §C.4 · pp. 24–25

C.5 The native score on this interface

  • The native score is maximized exactly by sufficient policies, those with s = 1.Source: §C.5 · p. 25

D Compression progress and acquired evidence

  • Compression progress scores an update by comparing old and updated compressors on the same current archive.The example fixes the description language and background before selecting which button reveals the unknown world.Source: D Compression progress and acquired… · p. 25

D.1 The ideal learner and its return

  • The ideal learner minimizes a two-part model-plus-residual description cost on each archive, then scores the resulting update; its oracle updates have unlimited computation and no model-size or codebook restriction.Source: §D.1 · p. 26

D.2 A two-button example and a complete optimum

With a fixed ideal learner and background, the two-button example yields a wiring with a deficient optimum for complete compression return. Further interpreter constructions show that complete scores may be infinite or finite without removing the obstruction.

D.2 A two-button example and a complete optimum

  • For one of the two wirings, every fixed ideal learner has an attained complete-return maximizer that is deficient relative to the revealing policy.A policy choosing the nonrevealing button has a history independent of the world forever, giving deficiency 1/2 for finite targets.Source: §D.2 · pp. 26–28
  • Complete-return maximizers exist whether the supremum of finite-horizon scores is finite or infinite.When finite, the nondecreasing finite-horizon maxima attain the supremum; when infinite, a mixture of policies in a fixed-root family has infinite return.Source: §D.2 · p. 27
  • The result gives one deficient optimum for one wiring, not failure of every optimum or of one wiring for every learner.The learner and background remain fixed across wirings; supplying the wiring to the scorer can change its history score.Source: §D.2 · pp. 26, 28

D.3 Finite values and the description language

  • One constructed optimal prefix interpreter makes every policy’s complete score infinite by rewarding updates that copy each nonempty archive as its model.Source: §D.3 · p. 28
  • A different constructed interpreter gives all policies uniformly bounded complete scores yet still permits a strictly dominated complete-return maximizer.In this construction, the reward comes entirely from correcting initialization; it does not establish a finiteness guarantee for all interpreters.Source: §D.3 · p. 28

E Discounted predictive Shannon and Square rewards

This section examines whether predictive, posterior-movement, state-visitation, and DIAYN objectives promote counterfactual information, using a shared-alarm example and additional constructions. It also establishes a limitation on uniformly computing objectives that strictly prefer revelation and specifies a finite planning comparison, without reporting its outcomes.

E Discounted predictive Shannon and Square rewards

  • For every 0 < γ < 1, discounted predictive Shannon and Square rewards are maximized exactly by policies that skip initial inspection, leaving deficiency 1/2 and being strictly dominated by sufficient inspection.Increasing γ toward one does not change the maximizing startup; this result concerns exact optima, not approximate planning or other prediction-error rewards.Source: E Discounted predictive Shannon and… · pp. 29–30

F.1 The published rewards and their aggregation

  • For the published Hellinger and Absolute rewards of Orseau et al. (2013, Section 3, equations (3)–(5)), world-independent random observations pay zero, while a fair binary query pays c and 1, respectively.The aggregation calculation covers randomized history policies and discounted or complete returns.Source: §F.1 · pp. 30–31

F.2 Both rewards succeed on the shared alarm

  • On the shared alarm, every maximizing policy for either published Hellinger or Absolute reward inspects initially and has zero eventual deficiency for 0 < γ ≤ 1.Geometrically discounted posterior information gain and Brier rewards likewise select initial inspection for every 0 < γ < 1, while their complete undiscounted objectives tie all policies at two bits and 2/3, respectively.Source: §F.2 · pp. 31–32

F.3 Three incremental bits can outscore four bits at once

  • For 9/10 ≤ γ ≤ 1, Hellinger and Absolute rewards favor PARTIAL, whereas information, ordinary Brier, and native Jw favor sufficient FULL policies.For t ≥ 3, the exact deficiency is (1 − s)/2; these failures compare different paths of the same posterior and do not contradict continuation prediction guarantees.Source: §F.3 · pp. 32–33

G MOP in a fully observed finite MDP

  • MOP’s stationary entropy objective (Ram´ırez-Ruiz et al., 2024) is maximized by choosing READ with probability 1/3, leaving deficiency 1/3 at every positive collection time and over the infinite history.This holds for each per-world objective and the worst-world objective; pure READ is sufficient and strictly dominates every maximizer.Source: G MOP in a fully observed finite MDP · pp. 33–34

H State-visitation entropy in a three-state MDP

  • Every state-visitation entropy maximizer chooses RANDOM initially, leaving exact deficiency 1/2 at every finite collection time; TEST is sufficient and strictly dominates these optimizers.The deficiency remains 1/2 for every proper discount, even as γ approaches one.Source: H State-visitation entropy in a… · p. 35

I DIAYN on actual observed states

  • DIAYN’s maximizing skill families achieve log 2 + α log 3 but read with probability at most 1/2, leaving deficiency (1 − s)/2 to the READ target.Every deficiency from 1/4 to 1/2 is attained; READ strictly dominates every maximizing family. The result concerns exact objective optimization, not neural-parameter optima or training performance.Source: I DIAYN on actual observed states · p. 36

J Computing objective values of arbitrary policies

  • No objective that is weakly monotone for finitary simulation and strictly prefers revelation to waiting is uniformly computable on all total computable policies in the stated worlds.A uniform evaluator would distinguish whether a program halts by evaluating its policy, deciding the halting problem; the obstruction comes from unbounded postponement of acquisition.Source: J Computing objective values of… · p. 37

K Finite planning comparison

  • The finite comparison selects three-step policies to simulate four-step experiments, then selects four- and five-step policies against fixed three-step references and measures reproduction cost.These are numerical planning comparisons over full history-dependent policies with shared response laws, not sampled training or learned dynamics; the finite native objectives do not implement the eventual score Jw.Source: K Finite planning comparison · p. 37

K.1 Worlds and available information

  • The comparison covers 22 heterogeneous cases with four or eight hidden worlds, and its inspected cases are not independent replications or confirmation on unseen classes.Policies observe only action–observation histories, not the physical state or world label; evaluated and target policies start from state zero in the same world.Source: §K.1 · p. 38

K.2 Objectives, planning, and evaluation

  • Information, whole-world Brier, and categorical count policies use backward dynamic programming; native planning uses linear programs in realization weights with causal-flow constraints, evaluated against all 32,768 depth-four targets.The categorical pseudo-count baseline scores raw observation symbols. Each evaluated policy receives only three observations; the audit does not extend to arbitrarily long experiments.Source: §K.2 · pp. 38–39

K.3 Fixed-budget results and reward-matched controls

Across these finite-budget studies, simulation-error comparisons vary with the reference, reward constraint, and budget, while reproducing selected references does not ensure success on the full native family. The results do not establish a generally superior practical native objective.

K.3 Fixed-budget results and reward-matched controls

  • 6 wins, 1 numerical tie, and 15 losses favor native mean over the fixed-reference native-maximum control; this comparison does not isolate the return-floor effect.The control minimizes the native depth-three maximum while retaining the fixed reference’s Brier return.Source: Table 2 · p. 40, §K.3 · p. 40
  • 21 of 22 matched policy pairs are numerically incomparable and one is equivalent, so scalar rankings do not establish strict Blackwell dominance or a universally superior objective.Information and Brier strictly respect Blackwell improvement on these finite full-support world classes; maximum target error need only be weakly monotone.Source: §K.3 · p. 40

K.4 More collection against fixed references

  • At four steps Brier beats both native methods at threshold .02, while native methods win more often at five steps; at .01, five-step Brier and native mean tie.A first successful tested budget is neither a minimum acquisition time nor a guarantee for subsequently selected policies.Source: §K.4 · p. 43

K.5 Return cost of reproducing a reference

  • Among 34 eligible comparisons in four pilot classes, 12 have numerically zero cost and 22 have positive lower bounds; these repeated comparisons are not a failure rate across all 22 classes.Each reference may require a different compatible policy.Source: §K.5 · p. 43
  • Reproducing all six fixed references within 10^-12 still leaves the balanced five-step native-maximum policy’s hardest three-step target error at [.2262799, .2262801].Success on the six references is weaker than success on the entire native family.Source: §K.5 · p. 44
  • For Delay at four steps, reproducing Brier’s own reference costs 0.47% of maximum Brier gain; in Irreversible, a compatible five-step policy has numerically zero information-gain cost.The Figure 4 alternatives are selected afresh at each budget rather than tracking one continuing policy.Source: Figure 4 · p. 44, §K.5 · p. 44

K.6 Verification, reproducibility, and scope

  • The studies use unmatched computation and finite budgets, and do not establish a generally superior practical native objective or eventual dominance.The compression constructions also assume a supplied complete optimal prefix interpreter whose existence is not formalized.Source: §K.6 · p. 45

Related papers

Loading 2610.02159v1…