PaperlayerView exact revision on arXiv

Source-linked AI summary

Minimal Recurrent Behavioral Memory for Imitation under Partial Observability

Xianyao Li, Fang Xu, Rui Min, Ruitong Tian, Jing Du

arXiv:2609.25757v1cs.LGcs.ITcs.RO

TL;DR

Under partial observability, the paper asks which expert-behavior distinctions must persist in recurrent memory. It characterizes a minimal behavioral target and a protocol for measuring it, then finds learned codes near the stated requirements while showing that acquiring sufficient memory remains difficult.

  • Problem

    The paper asks which distinctions determine a specified expert's behavior now and which must persist because future observations will not restore them before use.

    Source: §1 · p. 1

  • Method

    A compatibility relation characterizes recurrent behavioral memory, while a sole-carrier protocol separates behavioral sufficiency, excess code rate, and information in observations or other memory paths.

    Source: §1 · p. 1, §3 · p. 4

  • Results

    Learned code rates remain near 0- and 2-bit requirements as hidden-mode counts grow to 512, while future-behavior supervision improves acquisition in tested settings but can retain surplus information.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The characterization supplies a minimal-memory target, while the learning experiments show a gap between information sufficiency and acquiring its representation.

    Source: §7 · p. 9

    The exact quotient formula requires history-determined expert behavior and transitive compatibility; general exact computation is limited to small instances or matching finite-instance bounds, and experimental rates concern symbolic behavior under stated conventions and occupancy.

    Source: §7 · p. 9

Abstract

from arXiv · show

What is the least recurrent memory needed to reproduce a specified expert under partial observability? The instantaneous requirement is the conditional entropy of the expert's behavioral quotient, but recurrence must also preserve distinctions that future observations will not restore before use. We characterize this minimal recurrent behavioral memory by a compatibility relation: under transitivity its classes attain the exact minimum, while the general case is an entropy minimization over closed compatible state assignments, with exact certificates on finite instances. A sole-carrier measurement protocol separates behavioral sufficiency, excess code rate, and information carried by observations or other memory paths; experimental bit requirements refer to the induced symbolic behavioral model under the stated occupancy. Across manipulation tasks, learned code rates remain near zero- and two-bit requirements as hidden modes grow to $512$, and anticipatory memory follows a $2\to1\to0$ requirement despite zero instantaneous demand during waiting. Learning this representation remains difficult: event-agnostic future-behavior supervision yields $36/40$ sufficient seeds with one frozen configuration and improves the longest-horizon pixel setting from $0/8$ to $6/8$ sufficient held-out seeds (closed-loop success from $0.08$ to $0.57$). On unmodified community benchmarks, the protocol certifies delay-independent requirements, which sufficient codes match at mid-delay. The supervision aids commitment but can induce predictive surplus; annealing it lets imitation and rate training reduce that surplus, separating the information-theoretic target from the ability to learn it.

1 INTRODUCTION

The paper asks what expert-behavior distinctions must persist in memory under partial observability, then characterizes a recurrent target that preserves distinctions future observations will not restore before use. Its learning experiments test whether compact codes can realize that target.

Motivation

  • A waiting robot need not act on grasp or placement cues, yet must retain them until their decisions unless a future observation reveals them again first.After grasping, only the placement cue remains necessary.Source: §1 · p. 1

Core characterization

  • The recurrent target retains distinctions only while future observations cannot restore them before use, and under transitive compatibility its conditional entropy is the exact minimum among zero-distortion realizations.Without transitivity, closed compatible assignments replace the class variable, with exact search for small instances and matching bounds for finite corridor instances.Source: §1 · p. 1

Relation to prior targets

  • Existing recurrent, control-state, predictive-state, and system-identification targets can preserve distinctions the specified expert treats identically; this target instead tracks expert responses as observations arrive.Future-action prediction can retain a re-displayed cue when its decoder receives no future observations.Source: §1 · pp. 1–2

Measurement

  • The measurement protocol uses a sole temporal carrier and explicit observation and occupancy conventions, and checks behavioral sufficiency before minimality or attribution.The reported experimental bits concern the induced symbolic behavioral model, not hardware storage or exact continuous control.Source: §1 · p. 2

Learning experiments

  • Learned code rates remain near 0- and 2-bit requirements as hidden-mode counts grow to 512, while event-agnostic future-behavior supervision improves acquisition in tested settings but can retain surplus information.The paper distinguishes this learning surrogate from the minimal-memory characterization.Source: §1 · p. 2

2 MINIMAL RECURRENT BEHAVIORAL MEMORY

The paper separates the current-action behavioral quotient from recurrent memory, which must preserve distinctions until use unless observations restore them. It characterizes the exact minimum under transitivity and gives a closed-assignment formulation and finite-instance certificates for the general case.

2.1 THE INSTANTANEOUS PROBLEM

  • The behavioral quotient groups states with the same observation and expert action distribution, and encoding it achieves the instantaneous zero-distortion bound, but does not ensure recursive updatability.For uniform hidden modes forming R equal behavioral classes, log2(M/R) bits are unnecessary for the current action.Source: §2.1 · pp. 2–3

2.2 FROM CURRENT BEHAVIOR TO AN EXACT RECURRENT TARGET

  • In A′, recurrent memory must retain two independent binary cues through waiting despite zero instantaneous demand, then falls from 2 bits to 1 after grasp and 0 after placement.The recurrent realization uses a discrete state with deterministic updates and a policy conditioned on the observation and state.Source: §2.2 · p. 3
  • When compatibility is transitive, its history classes admit deterministic updates, every zero-distortion realization determines them, and using those classes attains the minimum simultaneously at every step.The anticipatory gap is H(Γt | Ot) − H(GE,t | Ot), measuring memory needed despite being unnecessary for the current action.Source: §2.2 · p. 3

2.3 WHY THE GENERAL CASE IS HARDER

  • Compatibility may be nontransitive when histories share different futures, so taking its transitive closure can merge histories that conflict on a common future.The general object is a closed compatible state assignment whose successor cells remain compatible under common continuations.Source: §2.3 · p. 3
  • For the general case, small-instance partition search and matching entropy bounds certify exact finite corridor rates of 2 and 1 bits, but do not provide a general polynomial-time algorithm.For nonzero distortion, the instantaneous rate-distortion function remains a lower bound; the full recurrent frontier is not characterized.Source: §2.3 · p. 4

3 WHEN DOES A BIT COUNT AS MEMORY?

The measurement protocol distinguishes behavioral sufficiency, excess rate, and information carried through observations or other memory paths. A discrete sole-carrier realization makes the code auditable, while reported rates remain tied to the stated observation convention and symbolic model.

3 WHEN DOES A BIT COUNT AS MEMORY?

  • The protocol asks whether the code with credited observations determines required behavioral memory, how much rate exceeds that requirement, and whether information instead travels through another carrier or occupancy.Sufficiency is checked before minimality, so matching a lower bound is evidence of absent redundancy only after sufficiency is established.Source: §3 · p. 4
  • The theoretical rate, learned code rate, codebook capacity, and storage footprint are distinct quantities, and reported bits describe the finite symbolic behavioral model under expert occupancy.The observation convention credits behaviorally decodable classes to current observation; these rates do not quantify exact continuous-control requirements.Source: §3 · p. 4
  • Low code rate or high task success alone does not establish minimal internal memory because persistent hidden state, policy pose, observations, or occupancy can carry information.The protocol therefore uses body-memory probes, raw-observation leak probes, and matched-side-information controls.Source: §3 · p. 5

3.1 A MEASURABLE RECURRENT REALIZATION

  • The discrete sole-carrier policy updates Ct from its prior code, observation, and previous action, and its action head receives only observation and code.Its expected rate penalty is H(Ct | Ot) plus a conditional-prior KL term; reported rates are plug-in entropies of hard codes.Source: §3.1 · p. 5

4 DOES LEARNED MEMORY MATCH THE BEHAVIORAL REQUIREMENT?

The evaluation compares learned code rates with certified behavioral-memory requirements under explicit sole-carrier and occupancy conventions. Across the tasks, rates track zero-, two-, and anticipatory-memory requirements, while acquisition reliability and surplus vary by setting.

4.1 EVALUATION PROTOCOL

  • The protocol checks sufficiency before minimality using a sole temporal carrier, explicit observation and occupancy conventions, and probes for continuous bypasses, body memory, and current observations.Reported experimental bits concern the induced symbolic model, not hardware storage or exact continuous control; β is selected by closed-loop fidelity before rates are inspected for the plain learner.Source: §4.1 · p. 5

4.2 WORLD COMPLEXITY: ZERO AND NONZERO REQUIREMENTS

  • Task A’s grasp requirement remains zero bits as hidden masses grow to 512; sufficient plain codes carry 0.00–0.10 bit and achieve 0.91–0.99 closed-loop success.At M = 512, the grasp rate is 0.03 bit; the mass changes dynamics and later observation support but not the expert’s action choice.Source: §4.2 · p. 5
  • Readout-2 requires exactly two first-gap bits at M = 32, 128, and 512, and sufficient learned codes remain near two bits with 0.85–0.96 closed-loop success.Plain training passes on 8/8, 7/8, and 8/8 seeds; task-informed supervision passes on 8/8 throughout.Source: §4.2 · pp. 5–6
  • The identification control’s learned rates rise with larger codebooks, while a separate carrier restores Task A success and carries additional information, distinguishing identification burden from shared-carrier interference.Rates grow from 2.25 to 8.71 bits on Task A and 4.82 to 7.49 on readout-2; the separate carrier adds 4.1–6.0 bits.Source: §4.2 · p. 6
  • On the unmodified memory chain and Passive T-maze, the protocol certifies delay-independent requirements; 87 of 192 runs pass the gate and match the requirement at mid-delay.The memory-chain requirement is zero during cue-visible steps, b bits from the first cue-free step until the query, then one bit once the query index appears.Source: §4.2 · p. 6
  • Community-benchmark agreement has a task-structure boundary: at mid-delay, deterministic context coding makes exact sufficiency equal context entropy, and none of 32 two- or three-bit-context runs passes at delay 100.These tasks validate protocol transfer and acquisition, whereas Task A and readout test compression amid additional world information.Source: §4.2 · p. 6

4.3 ANTICIPATION, USE, AND RE-OBSERVATION

  • On A′, required memory falls from 2 to 1 to 0 bits despite zero instantaneous demand in both waiting gaps; sufficient supervised runs carry 2.03–2.09 bits before grasp and 1.00 afterwards.Across two horizons and two rate weights, all 32 runs pass, with 0.96–1.00 closed-loop success.Source: §4.3 · p. 6
  • The re-reveal toy’s requirement drops from one bit before its first decision to zero afterwards, while an open-loop future-action target retains 2.06 and 1.29 bits.The physical weighing task also has a certified 2-to-0 requirement because the next lift re-reveals mass; its closed-loop failure is documented in the appendix.Source: §4.3 · pp. 6–7

5 CAN A COMPACT LEARNER FIND THE MINIMAL MEMORY?

The section tests whether compact learners can acquire sufficient memory and approach its certified rate. Future-behavior supervision improves acquisition in state and pixel settings, but may leave predictive surplus that annealing can reduce.

5.1 A TEMPORAL LEARNING DIFFICULTY

  • Across 320 sole-carrier A′ runs, sufficiency is negatively associated with delay (−0.146/step, CI [−0.18, −0.12]); controlled sweeps also show negative within-learner slopes and lower reliability under larger memory load.The pooled association is descriptive because configurations differ; full-history attention recovers tested decisions under teacher forcing and reaches 0.88–1.00 closed-loop success, but its persistent history excludes its code readout from the rate comparison.Source: §5.1 · p. 7

5.2 EVENT-AGNOSTIC FUTURE-BEHAVIOR SUPERVISION

  • The frozen event-agnostic configuration yields 36/40 sufficient seeds at first-gap rates of 2.00–2.04 bits against a two-bit requirement, with every verdict reproduced on fresh recordings.The event-informed reference reaches 37/40 and lower post-use rates, but requires event structure; the two surrogates have different information requirements, not different definitions of Γt.Source: Table 1 · p. 8, §5.2 · p. 8
  • Matching a full-history teacher’s internal state yields only 0–2/8 sufficient seeds, while future-behavior supervision improves acquisition at the same capacity without identifying a unique optimization mechanism.Longer training, curriculum, and code refinement leave the longest A′ horizon at 0–1/8; direct recorded-action targets support the task-informed reference, not teacher-free performance of the frozen event-agnostic configuration.Source: §5.2 · p. 8
  • Readout-3 requires three bits and at least eight states, yet sufficient-seed counts rise from 0/24 at K = 8 to 23/24 at K = 24, separating required state count from reliable learning capacity.Sufficient seeds average 2.99–3.14 bits; small continuous RNNs also fail on the external T-maze, though these controls do not isolate quantization from network size and optimization.Source: §5.2 · p. 8

5.3 PIXELS AND PREDICTIVE SURPLUS

  • On held-out pixel seeds, event-agnostic supervision yields 6/8 sufficient policies and 0.57 closed-loop success, versus 0/8 and 0.08 for plain training.A schedule fixed before pixel runs yields 12/16 sufficient policies overall; grasp-side accuracy is high for all learners, while later placement differs and pixel control precision still limits success.Source: Table 2 · p. 8, §5.3 · p. 8
  • Without annealing, event-agnostic supervision acquires memory at gap 20 (7/8), but the second-gap rate remains 2.14 bits from states and 2.01 from pixels against a one-bit requirement.The expired grasp class contributes at most 0.02 bit with or without annealing.Source: §5.3 · p. 8
  • Removing positional jitter reduces unannealed surplus from 1.16 to 0.23 bit, supporting initial-position variability as an important driver; with original jitter, annealing reduces surplus to 0.17–0.21 bit.With original jitter, annealing reduces information about x to 0.01–0.03 bit; the account attributes the improvement to letting imitation and rate training reduce predictive surplus after supervision.Source: §5.3 · pp. 8–9
  • On the memory chain, sufficient runs with two- and three-bit contexts retain mean rates of 1.91 and 2.69 bits after the query lowers the requirement to about one bit.Because the query appears only at the final step, the rate term can reward forgetting at that step alone.Source: §5.3 · p. 9

6 RELATED REPRESENTATION TARGETS

The paper distinguishes its fixed-expert behavioral target from representations that preserve future processes, control structure, or hidden parameters. Its target compares expert responses on common reachable continuations while treating future observations as side information.

6 RELATED REPRESENTATION TARGETS

  • Predictive-state approaches preserve future observation distributions, whereas this fixed-expert target compares responses on common reachable continuations and can require fewer distinctions than full-process prediction.Related approaches include causal states and predictive-state representations and the ϵ-transducer.Source: §6 · p. 9
  • Bisimulation and related control or identification targets preserve reward, dynamics, or Markov structure, but a specified expert can ignore those distinctions.The paper compares targets under matched objectives rather than comparing full published algorithms.Source: §6 · p. 9
  • The paper’s characterization combines recurrent realizability, future observations as side information, and entropy minimality, including an anticipatory gap beyond action-relevant information bounds.The discussion situates this alongside bounded-memory control, decision-centric memory, history dependence, memory architectures, credit assignment, and distillation.Source: §6 · p. 9

7 SCOPE AND CONCLUSION

The paper characterizes the memory needed to reproduce an expert across time, distinguishing current behavioral requirements from distinctions that must persist until use. It also separates that information-theoretic target from the difficulty of learning it and from limits of the experimental evidence.

7 SCOPE AND CONCLUSION

  • On physical weighing, expert-led observation generation raised grasp-side accuracy from 0.30–0.32 to 0.93–0.99, while subsequent control remained imprecise.Event-agnostic prediction extends the compact carrier's horizon on the external Passive T-maze but still fails at length 100; a 128-dimensional GRU is more reliable.Source: §7 · p. 9
  • The central result is a recurrent behavioral-memory target that tracks distinctions future observations will not restore before use, alongside evidence of a gap between memory sufficiency and learning it.Experimental scope is symbolic behavior under the stated observation convention and occupancy; reliable closed-loop evidence is strongest through two bits.Source: §7 · p. 9

A PROOFS AND EXACT CERTIFICATES

  • The proof conventions assume reachable histories and history-determined expert behavior; exact solver use additionally requires enumerable hidden variables and deterministic symbolic observations.The solver certifies finite enumerated models; the general exact computation remains limited to small instances or matching finite-instance bounds.Source: §7 · p. 9, REPRODUCIBILITY STATEMENT · p. 10, A PROOFS AND EXACT CERTIFICATES · p. 14

A.1 INSTANTANEOUS MINIMALITY

  • At a single step, the zero-distortion rate is H(GE,t | Ot), while hidden modes within the same behavioral class need not be distinguished.For uniform behavioral classes independent of Ot, the stated closed form gives RE(D) as a function of class count and distortion.Source: §A.1 · pp. 14–15

A.2 RECURRENT MINIMALITY AND SUPPORT CONDITIONS

  • Under (A1)–(A4), the compatibility classes attain the minimum recurrent memory rate H(Γt | Ot) simultaneously at every step.The relation is an equivalence under (A4), and the classes admit a deterministic update.Source: §A.2 · p. 16
  • Without transitivity, compatible assignments can require an intermediate rate: the three-history example has Rmem_t(0) = h2(1/3), strictly between 0 and log2 3.The general characterization uses closed compatible state assignments, with dynamic programming for small instances.Source: §A.2 · p. 17

A.3 THE GENERAL CASE: CLOSED COMPATIBLE STATE ASSIGNMENTS

For non-transitive compatibility, the paper characterizes recurrent memories as closed compatible state assignments and minimizes their conditional entropy. It also gives lower bounds, finite-instance certificates, and a future-behavior supervision objective with explicit scope limits.

A.3 THE GENERAL CASE: CLOSED COMPATIBLE STATE ASSIGNMENTS

  • On finite POMDPs, zero-distortion recurrent memories correspond to closed compatible state assignments, and minimizing their conditional entropy gives the exact minimum.A backward dynamic program computes small instances; its unpruned search grows rapidly, so it is not a general algorithm.Source: §A.3 · p. 18
  • For drifting corridors, verified constructions tighten the hall-wise brackets to [0, 2] and [0, 1] bits, with learned codes matching the 2-bit first-hall construction and a 0.2–0.4-bit second-hall surplus.For W = 3, 5, and 7, finite-instance coloring lower bounds match certified upper bounds at every step.Source: §A.3 · pp. 18–19

A.4 NONZERO DISTORTION

  • For nonzero distortion, recurrent rate is non-increasing and bounded below by the instantaneous rate–distortion function, but the bound can be strict even at zero distortion.The paper claims neither convexity nor equality with the instantaneous frontier for positive distortion.Source: §A.4 · p. 19

A.5 BEHAVIORAL FUTURE SUFFICIENCY

  • Full-horizon behavioral future sufficiency makes zero loss equivalent to compatibility across common reachable continuations; under (A4), its rate-minimal code has entropy H(Γt | Ot).Predicting every prefix charges the first step where expert behavior diverges, even when longer continuations are no longer common.Source: §A.5 · p. 20

A.6 CLOSED-FORM VERIFICATION OF THE THEORETICAL QUANTITIES

  • Before learning experiments, partition-refinement quantities were checked against closed-form toy results; the measurement design distinguishes symbolic requirements, code entropy, codebook capacity, and physical storage.The general exact dynamic program has more limited scope than the refinement pass.Source: §A.6 · p. 21, B EXPERIMENTAL DESIGN AND MEASUREMENT · p. 21

B.1 BENCHMARKS AND THE SYMBOLIC OBSERVATION CONVENTION

The benchmark appendix defines the symbolic observations and finite-model procedures used to measure requirements, then specifies recurrent variants and how their rates are accounted for.

B.1 BENCHMARKS AND THE SYMBOLIC OBSERVATION CONVENTION

  • Held-out class labels match the frozen binner on 100% of separable steps, while the Task A sag convention credits a partly decodable, behaviorally irrelevant mass-half symbol during placing.The binner returns a null symbol when class-conditional means are not separated by more than 5 cm or 25 sensor-noise standard deviations; its reported upper-bound relationship follows from coarsening raw observations.Source: §B.1 · pp. 21–22
  • The solver applies backward partition refinement to the recorded finite POMDP, checks its assumptions and transitivity, and returns exact conditional entropies only when the relation is transitive.The appendix also reports that 13 of 14 datasets record every latent configuration; Task A at M = 32 reconstructs two missing mass-slot pairs without modifying recorded labels.Source: §B.1 · pp. 22–23

B.2 REALIZATION AND TRAINING VARIANTS

  • The realization uses a quantized code as its only learned recurrent state, while table variants distinguish plain rate training, behavioral future sufficiency, and a scaffold annealed away before evaluation.A retained continuous bypass is excluded from sole-carrier minimality claims; future-behavior objectives differ in decoder side information.Source: §B.2 · p. 23

B.3 RATE ACCOUNTING

  • The paper’s per-step rate accounting avoids both free passage of nuisance information through prior codes and over-penalization of information that remains visible.At zero distortion, H(Ct | Ot) charges information that must persist beyond its visibility; “pay” denotes representational burden, not communication cost.Source: §B.3 · p. 23

B.4 SIDE-INFORMATION AND ESTIMATOR CONTROLS

The controls probe estimator and side-information effects, test learned sufficiency under stricter read-side designs, and check whether results persist across gates and recordings. They also show that benchmark performance alone cannot certify the absence of observation leaks.

B.4 SIDE-INFORMATION AND ESTIMATOR CONTROLS

  • Refining observation conditioning lowers estimated first-gap rates, but the same decrease appears after code permutations that destroy position dependence; the cross-validated raw-observation bound provides no evidence of additional prediction beyond the symbolic observation.In the second gap, four low-β seeds show 0.04–0.08 bit above the permutation null, attributed to end-effector position.Source: §B.4 · pp. 23–24
  • Miller–Madow corrections change SΓ by at most 0.014 across 160 runs, and sufficient-seed counts remain identical at thresholds 0.8, 0.9, and 0.95.Sufficient runs score 1.000 on every second-gap step, while the best insufficient run scores 0.37.Source: §B.4 · p. 24
  • With forecast supervision, all 64 strict read-side runs pass the reported sufficiency checks, whereas the same architectures without that target pass on 0/8 seeds in every cell.For the intent architecture, nearest-class decoding is only a diagnostic proxy, not the action executed by its controller.Source: §B.4 · p. 24, Table 6 · p. 24

B.5 SUFFICIENCY-GATE SENSITIVITY

  • Of 2,211 A′-family runs, 928 pass the relaxed gate and 879 pass the full gate; a bypass can restore a placement class late and pass the relaxed check despite an incorrect recurrent state earlier.Every full-gate sufficient seed carries 2.00–2.15 bits in the first gap.Source: §B.5 · p. 25

B.6 OBSERVATION SIDE INFORMATION: AN INJECTED-LEAK AUDIT

  • At δ = 8 mm, injected leakage improves success and slot accuracy, while the pre-specified low-rate flag detects only 1/96 runs; the exploratory flags are not leakage certificates.The full-gate flag also fires on some clean correct policies, often because they lost the grasp class while retaining the place class; the authors identify mismatches between the correctness criterion and gate.Source: §B.6 · pp. 25–26

B.7 INDEPENDENT-RECORDING REPLICATION

  • DIACRITIC remains sufficient on every fresh-recording Task A seed (8/8, 8/8, and 16/16 at M = 4, 32, 512), with grasp rates of 0.02–0.10 bits; large-M P-I sufficiency is not stable across recordings.In fully covered Task A, DIACRITIC and system identification are both place-sufficient on 8/8 seeds, but their closed-loop success is 0.99 and 0.23, respectively.Source: §B.7 · p. 26

B.8 INFERENCE LATENCY AND STORAGE FOOTPRINT

  • The Transformer re-encodes its full token history at every step, with carried storage growing from 8.2 kB at T = 33 to 262 kB at T = 1024; recurrent carriers keep a fixed 128 or 129 bytes.The comparison isolates committing information in advance and is not presented as a deployment-architecture recommendation.Source: §B.8 · p. 27, Table 8 · p. 27

B.9 TRAINING-TIME COST OF FORECAST SUPERVISION

The appendix reports the training-time cost of forecast supervision and the compute used for data collection, training, and evaluation.

B.9 TRAINING-TIME COST OF FORECAST SUPERVISION

  • The default forecaster adds about 1.5% to student training time, while fitting the measured 100-step random-offset forecaster takes under 0.2%.Task-informed controls at A′ gap 20 reached 8/8 sufficient seeds with a 100-step forecaster and 7/8 with recorded-action targets; these results do not establish performance for the frozen event-agnostic configuration.Source: §B.9 · p. 27, Table 9 · p. 27

B.10 COMPUTE AND DATA COLLECTION

  • Metered work totals approximately 651 GPU-hours across training, closed-loop evaluation, and independent re-recording, with additional local development, data collection, and pixel evaluation unmetered.The authors do not report a timing comparison between the two interchangeable training GPU types.Source: §B.10 · p. 27

C.1 FINITE TOY VALIDATION

Toy validation confirms the exact rate–distortion characterization and the recurrent 2→1→0 requirement, while learner tests expose surplus, collapse, and limits of the behavioral quotient under stochastic actions.

C.1 FINITE TOY VALIDATION

  • The solver returns a 2→1→0 recurrent-memory staircase on the gap and re-reveal toys, while bounds replace construction on a hand-built non-transitive instance.On re-reveal, Γ falls from 2 to 1 to 0 while Γs remains at 2.Source: §C.1 · p. 28
  • Blahut–Arimoto matches the closed-form reduced-source curve to 7.8 × 10−16, and the history source matches it across M = 4, 8, and 16 to 1.3 × 10−15.The deterministic and stochastic frontiers differ, with the deterministic frontier lying above the stochastic curve.Source: §C.1 · p. 28, Figure 4 · p. 28
  • For β ∈ [0, 0.1], both objectives recover the enumerated [0, 0, 1, 2] requirements exactly; at β = 0.3, the code collapses.The finite-sample tests also show that stronger rate pressure reduces nuisance information and surplus, but excessive pressure loses sufficiency.Source: §C.1 · pp. 28–29
  • The continuous carrier succeeds only at β = 0.003, but its KL bound is not a comparably tight minimality measurement; a bypass can also make readout rate underestimate memory rate.The toy ablation reports a 6.9-bit KL bound and 0.32 bit of nuisance information for the continuous carrier.Source: §C.1 · p. 29
  • Under Bernoulli(0.3) reveal loss, the solver flags (A2); the history-conditional observational relation recovers its own rate but not the demonstrator’s privileged coupling.The observational rate is [0, 0, 1.58, 2.58], and the passage states that privileged-coupling reproducibility lies outside the theorem’s assumptions.Source: §C.1 · p. 29
  • Across toy gaps, future-sufficiency-triggered refinement does not consistently increase sufficient-seed counts, while the annealed scaffold is modestly more reliable.The reported counts vary across gaps and seeds, so the passage does not establish a uniform refinement benefit.Source: §C.1 · p. 29

C.2 A STOCHASTIC EXPERT: THE QUOTIENT IS DEFINED ON ACTION LAWS

  • For a stochastic expert, states with the same action law are merged: the quotient entropy is 0.918 bit rather than log2 3 = 1.585 bits.The learner found the quotient on only 4/8 seeds at p = 0.7 without rate pressure and none with β ≥ 0.01; learned failures collapsed codes rather than separating equal-law states.Source: §C.2 · p. 29

C.3 TASK A AND READOUT CONTROLS

Task A and readout controls compare behavioral memory with system identification and inverse objectives as hidden-world complexity grows. Behavioral codes stay near the required rates in tested settings, while system-identification rates grow and control outcomes depend on the carrier and evaluation setting.

Inverse-objective controls

  • Inverse-only memory is insufficient in Task A at every tested M, but adding inverse prediction to imitation and rate training has no measurable effect on the behavioral code’s rate or sufficiency.The inverse-only objective obtains 0/8 at every M and retains 0.3–0.6 bit about mass; the combined objective reaches 8/8 sufficiency.Source: §C.3 · p. 30

Task A controls

  • At M = 128 and 512 with full coverage, system identification is place-sufficient on all seeds but stores 8.2 bits and succeeds on 0.23 of episodes, versus 0.99 success for DIACRITIC.The capacity-relieved system-identification rate rises with log2 M, while Task A controls report DIACRITIC rates ≤0.13 bit and the system-identification rate approximately linear in log2 M.Source: Figure 5 · p. 30, §C.3 · p. 31, Table 11 · p. 31
  • With full mass coverage, system identification at K = 1024 remains separated from DIACRITIC in storage and closed-loop success, so the reported separation is not a coverage artifact.The passage reports 8.2 bits and 0.23 success for system identification against 0.99 success for DIACRITIC.Source: §C.3 · p. 31

Readout task

  • For readout-2, behavioral learners match the 2.00-bit first-gap and 1.00-bit transport requirements across M = 32, 128, and 512, with sufficiency on 7/8–8/8 seeds.Forecast-supervised closed-loop success is 0.95, 0.90, and 0.96; the corresponding −R values are 0.94, 0.85, and 0.87.Source: §C.3 · p. 31, Table 12 · p. 32
  • For readout-3, forecast supervision reaches sufficiency on 6/8, 5/8, and 4/8 seeds with sufficient codes carrying 3.03–3.17 bits against the 3.00-bit prediction.Closed-loop success remains 0.15–0.28 across all seeds, and a K = 8 carrier reaches sufficiency on 0/24 seeds while K = 16 does so on 15/24.Source: §C.3 · p. 32, Table 12 · p. 32

System-identification controls

  • Separating the identification code from the behavioral carrier yields 0.90–0.97 closed-loop success, compared with 0.07–0.52 for displayed shared-carrier controls.Behavioral codes carry 0.28–0.49 bit and at most 0.03 bit about mass, while identification codes carry 4.1–6.0 bits.Source: §C.3 · p. 32, Table 13 · p. 32

Task A controls

  • At M = 512, the capacity-relieved system-identification rate is 8.71 bits, and mean closed-loop success is 0.19 despite sufficiency on 7/8 seeds.The teacher-forced sufficiency count falls to 4/8 on fresh expert recordings, and the evaluation gate does not certify continuous-action accuracy or policy-induced-observation stability.Source: §C.3 · pp. 32–33

C.4 SIGNPOST CORRIDOR: EXACT REQUIREMENTS AND LEARNED RATES

The signpost corridor extends the requirements test to exact and non-transitive settings, then examines learned rates, nuisance retention, and temporal distance. Behavioral rates track the certified staircase, but sufficiency declines with longer horizons for plain realizations.

C.4 SIGNPOST CORRIDOR: EXACT REQUIREMENTS AND LEARNED RATES

  • At W = 1, the solver certifies the 2→2→1 staircase; for W > 1, finite-instance certificates tighten the hall requirements to exactly 2 and 1 bits despite non-transitivity.The signpost corridor shares the solver and learner with A′ but tests drift variants and different transitivity conditions.Source: §C.4 · p. 33
  • Every sufficient seed carries 2.00–2.07 bits in hall 1 across M = 4, 16, and 64, while history grows from 3 to 7 bits.The −R, −RF, and β = 0 sufficient-seed counts vary by M; inverse-only memory reaches 0/8 at every M.Source: §C.4 · p. 33
  • Plain realizations lose sufficiency with reveal-to-use distance, with a −0.11/step coefficient, while the annealed scaffold reaches 8/8 through distance 25 in this corridor.Increasing hall-1 length also reduces −R sufficiency to 2/8 and then 0/8.Source: §C.4 · p. 33, Figure 6 · p. 34
  • With drift modes W = 3, 5, and 7, DIACRITIC retains ≤0.08 bit about drift, whereas multi-step-inverse memory retains 0.30–0.71 bit and never reaches sufficiency.The inverse objective depends on drift because the observed lateral displacement is needed to infer a sidestep; behaviorally equivalent action randomness makes that target mismatch observable.Source: §C.4 · pp. 33–34
  • Inverse-only memory remains at 0/8 even with K = 64, while adding the inverse head to imitation and rate training leaves sufficiency matching −R and reduces drift information.At W = 5, inverse-only retains approximately 1.0 bit about drift; the combined objective retains 0.09–0.12 bit.Source: §C.4 · p. 34
  • At K = 64, system identification reaches sufficiency on 4/8 seeds versus 0/8 at K = 16, but the continuous GRU bypass’s successful readout carries 1.5 bits where Γ requires 2.0.A non-sole memory carrier therefore does not yield a valid memory-rate measurement.Source: §C.4 · p. 34

C.5 PLAIN LEARNED CODE TRAJECTORIES

  • Figure 7 distinguishes seeds that pass the full sufficiency gate from those that fail, and plots certified behavioral-memory and instantaneous requirements under expert occupancy.Its caption warns that a low code rate on a failed seed is not evidence of compression at preserved behavior.Source: Figure 7 · p. 35, §C.5 · p. 35

D LEARNING COMPACT BEHAVIORAL MEMORY

  • Learning comparisons use the full sufficiency gate unless a relaxed or class-specific diagnostic is explicitly identified, and separate rates conditional on passing from all-seed control outcomes.The experimental section concerns acquisition of the independently defined target.Source: D LEARNING COMPACT BEHAVIORAL MEMORY · p. 35

D.1 EVENT-AGNOSTIC FUTURE-BEHAVIOR SUPERVISION

Event-agnostic future-behavior supervision improves acquisition of recurrent behavioral memory in the tested settings, while predictive targets can retain information beyond the symbolic requirement. Across larger hidden-mode counts and additional domains, sufficient learned codes often approach certified rates, but learning reliability remains bounded by task and training conditions.

D.1 EVENT-AGNOSTIC FUTURE-BEHAVIOR SUPERVISION

  • 36/40 seeds were sufficient with the frozen configuration, while held-out pixel sufficiency rose from 0/8 with plain training to 6/8 with event-agnostic supervision.For held-out pixels, closed-loop success rose from 0.08 to 0.57; the task-informed forecast reached 7/8 sufficient seeds.Source: §D.1 · pp. 35–36, Table 14 · p. 36, Table 15 · p. 36
  • The un-annealed objective retained 1.02 bits of surplus from pixels and 1.16 from states, versus 0.13 and 0.17 after annealing and 0.00 for the task-informed forecast.Initial-position variation is one supported source of predictive surplus: removing its jitter reduced surplus from 1.16 to 0.23 bit under retraining.Source: §D.1 · pp. 36–37

D.2 TASK-INFORMED SUPERVISION AND TEACHER-STATE CONTROLS

  • At the four-bit join, task-informed supervision yielded sufficient codes on 2/8 seeds only at K = 64, so slack in codebook size did not ensure reliable learning.On the re-reveal toy, open-loop action-only targets carried 2.06 bits in the first gap and 1.29 after use, while the side-information decoder carried 1.12 and 0.08 bits.Source: §D.2 · p. 37, §D.3 · p. 38

D.3 FUTURE-BEHAVIOR TARGETS WITHOUT OBSERVATION SIDE INFORMATION

  • Applying future-behavior distillation every second or fourth step still yielded 8/8 and 7/8 sufficient seeds, while several other tested interventions did not remove the learning barrier.At readout-3, sufficient-seed counts rose from 0/24 at K = 8 to 23/24 at K = 24; the authors distinguish this state-count bound from reliable acquisition.Source: §D.3 · p. 38, §D.4 · p. 38, §D.5 · p. 39

D.6 OBSERVED RATE–FIDELITY TRADE-OFFS

  • Among sufficient policies, closed-loop success fell from 0.96–1.00 at β ≤3×10^-4 to 0.70–0.90 at β = 2×10^-3 as post-use rate approached the one-bit boundary.In temporal-distance sweeps, placement sufficiency fell to 0/16 at gap 20 for both −RF and scaffold-RF; 43 of 45 contrasts in other reported sweeps did not reach Fisher p < 0.05.Source: §D.6 · p. 39, §D.7 · p. 39

D.8 LARGE HIDDEN-MODE COUNTS ON P-I

  • With forecast supervision, successful P-I seeds retained exactly 2.00 bits through M = 32, while late-phase sufficiency declined to 3/8 at M = 512.At M = 128 and 512, late-gate sufficient-seed means were 2.17 and 2.24 bits, against the 2.00-bit theory requirement.Source: §D.8 · p. 40, Figure 9 · p. 40

E.1 PIXEL OBSERVATIONS AND OBSERVATION-TIMING CHECKS

  • For pixel A′, plain training recovered the exact rate staircase on 2/8 seeds, but pixel policies had lower action precision and lower closed-loop success than state policies.A capture-time offset initially made the place class readable in gap frames; after correction, a single-frame probe predicted it at 0.45 versus a 0.52 majority baseline.Source: §E.1 · p. 41, Table 18 · p. 41

E.2 CUE–CORRIDOR T-MAZE

  • In the cue corridor, sufficient codes carried at least the predicted log2 R bits, but two-bit-cue sufficiency declined from 6/8 at L = 5 to 0/8 at L = 40 and 80.The one-bit-cue curve was non-monotone; the authors present these results as observations within one task family, not a general distance law.Source: §E.2 · pp. 41–42, Table 19 · p. 42

E.3 PHYSICAL WEIGHING AND RE-OBSERVATION

The physical weighing task requires two bits between identification and the grasp decision, then zero after re-observation; learning that memory does not by itself ensure reliable continuous control. On the unmodified passive T-maze, the compact carrier benefits from supervision but remains less reliable at the longest tested horizon than the 128-dimensional GRU.

E.3 PHYSICAL WEIGHING AND RE-OBSERVATION

  • The certified weighing-task requirement is 2 bits from put-down to grasp and 0 afterwards, as the second lift re-reveals mass.Plain training was sufficient on 1/8 and 0/8 seeds at gaps 6 and 20; step-0 forecast supervision reached 8/8 and 7/8, respectively.Source: §E.3 · p. 42
  • No tested compact-carrier learner solved the weighing task reliably in closed loop; even hybrid rollouts remained limited by imprecise physical grasp and transport.With scripted weighing, grasp-side accuracy rose to 0.99 at gap 6 and 0.93 at gap 20, but task success was 0.38 and 0.13.Source: §E.3 · p. 43, Table 20 · p. 43

E.4 EXTERNAL PASSIVE T-MAZE

  • On the unmodified passive T-maze, the 128-dimensional GRU was reliable through L = 50, while the compact carrier still failed at L = 100.The tested 4- and 8-dimensional GRUs failed at L = 20, 50, and 100, but their coupled network-size changes do not isolate state-size effects.Source: §E.4 · pp. 43–44, Table 21 · p. 43

E.5 CERTIFIED REQUIREMENTS ON COMMUNITY BENCHMARKS

The protocol certifies delay-independent memory requirements on two unmodified community benchmarks, and sufficient learned codes match the certified mid-delay rate. At the query, however, some codes retain distinctions that have become unnecessary, and reliability declines with delay in the tested settings.

E.5 CERTIFIED REQUIREMENTS ON COMMUNITY BENCHMARKS

  • The memory-chain requirement is 0 bits while the cue is visible, rises to b bits during the cue-free delay, and falls to one bit after the query; it is independent of memory length.The passive T-maze requires one bit through its cue-free corridor.Source: §E.5 · p. 44
  • All 87 sufficient runs matched the certified mid-delay rate, with a largest measured deviation of 0.00 bit.The figure describes the certified requirement as delay-independent and shows that every sufficient seed at mid-delay matches it.Source: Figure 10 · p. 44, §E.5 · p. 45, Table 22 · p. 45
  • At the query step, sufficient two- and three-bit memory-chain runs retained means of 1.91 and 2.69 bits although the requirement was one bit.No seed passed for two or three bits at delay 100; the empirical scope is finite symbolic tasks with enumerable hidden variables and the environments’ own observations.Source: §E.5 · p. 45

F REPRESENTATION TARGETS OF RELATED WORK

  • Table 23 compares representation targets across fixed-demonstrator fidelity, observation side information, entropy minimality, and non-transitive compatibility.The paper identifies the combination of these properties—not any individual property—as its contribution.Source: F REPRESENTATION TARGETS OF RELATED WORK · p. 45, Table 23 · p. 46
Loading 2609.25757v1…