MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation

• The benchmark spans 90 language-conditioned tasks across 10 memory types.It includes 10 reactive controls whose relevant cues remain visible.Source: MIKASA-Robo-VLA · p. 1
• The release contains 22,500 oracle episodes, 6.3 million recorded frames, and 175 hours of data in RLDS and LeRobotDataset v3.Episode horizons range from 25 to 2,160 steps.Source: MIKASA-Robo-VLA · p. 1
• For 28 of the 70 tasks with environment-determined gaps, the cue is absent longer than the surveyed 16-frame observation window.The longest measured gap is 1,000 steps.Source: MIKASA-Robo-VLA · p. 1

• A robot may need to recall object identity, reproduce a sequence, or track completed steps after relevant information disappears from view.The instruction specifies the goal, while observation history determines how to achieve it.Source: §1 · p. 2
• MIKASA-Robo-VLA expands MIKASA-Robo [Cherepanov et al., 2026a] from 32 tasks and 4 memory types to 90 tasks and 10 types, with language instructions and a shared interface.Tasks vary memory content, difficulty, and configured episode horizons from 25 to 2,160 steps.Source: §1 · p. 2
• A common evaluation protocol fixes the policy interface, episode seeds, and success metric.The paper also reports a reference π0.5 baseline on 14 tasks.Source: §1 · p. 2
• The suite provides 22,500 oracle demonstration episodes across all tasks in RLDS and LeRobotDataset v3, with dense rewards for subsequent RL fine-tuning.The demonstrations comprise 6,306,953 timesteps and were collected using PPO policies and motion planning.Source: §1 · p. 2

• RL memory research ranges from delay probes such as T-Maze and bsuite [Bakker, 2001, Osband et al., 2020] to broader environments such as POPGym and Memory Gym [Morad et al., 2023, Pleines et al., 2025].Other work separates memory from credit assignment [Ni et al., 2023] or develops memory-capability taxonomies [Cherepanov et al., 2025].Source: §2 · p. 2
• Robot policies use limited observation windows, while explicit-memory designs include perceptual storage, recurrent state, experience retrieval, and multiscale visual-textual memory.Examples include OpenVLA [Kim et al., 2024], RT-1, Octo, and RoboVLMs [Brohan et al., 2023, Octo Model Team et al., 2024, Li et al., 2026], as well as MemoryVLA [Shi et al., 2026], µVLA, MemER [Sridhar et al., 2025], and MEM [Torne et al., 2026].Source: §2 · p. 3
• CALVIN [Mees et al., 2022], LIBERO [Liu et al., 2023], SIMPLER [Li et al., 2024], and RoboCasa [Nasiriany et al., 2024] do not systematically vary memory content and retention delay.A long task may remain locally observable, while a short task may depend on a vanished cue.Source: §2 · p. 3
• Memory-focused suites cover varied demands, including object-centric factors in LIBERO-Mem [Chung et al., 2026] and keyframe-based memory in RoboTwin-MeM.MIKASA-Robo-VLA adds 90 parameterized tasks over 10 memory types, demonstrations, dense rewards, horizon stratification, and GPU-parallel execution.Source: §2 · p. 3

3 MIKASA-Robo-VLA Benchmark

• The 90 ManiSkill 3 tasks make cue-dependent action explicit, while 10 reactive controls keep the relevant information visible.A typical memory-dependent task presents a cue, hides it, and then opens the dependent action phase, with tracking and timed tasks as exceptions.Source: §3 · p. 3

3.1 Memory taxonomy and task families

• The 10 memory categories cover operations such as retaining identities, locations, sets, sequences, changes, and pending intentions across 26 task families.Variants adjust factors such as distractor count, retained items, and sequence length; the design is not fully factorial.Source: §3.1 · p. 3, Table 1 · p. 5

3.2 Temporal structure and horizon splits

• Configured horizons span 25 to 2,160 steps across Short, Medium, and Long splits, but episode length is not itself a measure of retention.Longer-horizon variants can extend the cue-to-action interval while preserving task semantics, though the release is not fully crossed.Source: §3.2 · p. 4, Table 2 · p. 5

3.3 Extension over MIKASA-Robo

• The suite extends MIKASA-Robo to a full-suite language interface, adds temporal variants and standardized evaluation, and releases demonstrations in two dataset formats.Task parameters can be re-specified to test different memory update rates.Source: §3.3 · p. 4

3.4 Observations, actions, and language

• The shared policy interface provides top-down and wrist-camera images, proprioception, and seven-dimensional delta-pose and gripper actions at 10 Hz.Thirty instruction templates cover 90 tasks; most are shared within families, with task-specific values in some instructions.Source: §3.4 · p. 4

3.5 Environment interface and evaluation

• The benchmark provides fixed-seed evaluation, a default of 50 episodes per task, and success_once credit when the task condition holds at any step.The shared interface may freeze actions during cue presentation or waiting phases when required by the collection curriculum.Source: §3.5 · pp. 4–5

3.6 Rewards and success criteria

• Every task has a dense RL reward and a separate binary success condition assessed directly from task conditions.Reward shaping can combine reaching and placement terms with task-specific progress signals and penalties.Source: §3.6 · pp. 5–6

4 Datasets

• The release contains 22,500 episodes and 6,306,953 timesteps—175.2 hours at 10 Hz—with demonstrations for every benchmark task.Both formats use the policy interface specified for evaluation.Source: §4 · p. 6

4.1 Demonstration collection

• Each task contributes 250 successful oracle episodes, generated by PPO for 34 tasks and scripted motion planners for 56.Initial scenes use the environment reset procedure, and simulator seeds are retained in RLDS metadata.Source: §4.1 · p. 6
• The demonstrations retain successful behavior but do not systematically cover failure recovery or states induced by learned-policy errors.The reported oracle success rate of 1.000 describes the success-filtered dataset, not unconditional oracle reliability.Source: §4.1 · p. 6

4.2 Release formats

• RLDS stores PNG camera frames and per-step rewards, while LeRobot uses AV1 video and Parquet arrays; their size difference reflects both encoding and schema.LeRobot omits per-step rewards, and without shared episode IDs, records cannot be joined by index; reward-dependent training should use RLDS.Source: §4.2 · p. 6

4.3 Dataset composition

• Equal episode counts per task do not equalize timesteps, and horizon splits describe task duration rather than a fixed train/test partition.Evaluation must specify held-out seeds or tasks separately.Source: §4.3 · p. 6

5.1 Information gap and observation context

• Environment phase boundaries define gaps for 70 tasks; 28 determined gaps exceed 16 observations, while the gap measures only the provably cue-absent interval.Every memory-dependent task still requires retention by construction, even when its measured gap is short; 15 tasks have zero gaps, and some retention intervals are undetermined.Source: §5.1 · pp. 6–7

5.2 Reference baseline

• 0.211 ± 0.044 mean task success was reported for reference π0.5 using current images and proprioception, without observation history or an explicit memory module.Table 3 reports the unweighted aggregate across 20 episodes per task.Source: Table 3 · p. 8, §5.2 · p. 8

5.3 Memory architecture comparison

• A related µVLA comparison reports that adding a 64-token recurrent memory module raises mean success from 0.42 to 0.84 on five trained tasks and from 0.07 to 0.23 on 18 held-out tasks.A separate execution-regime analysis reports that open-loop chunking shifts per-task success by up to 0.650, a confound for the Long-split result.Source: §5.3 · p. 8

5.4 Performance across memory demands and horizons

• The 9 tasks with determined gaps had mean success 0.167 versus 0.290 for the other 5, but neither this comparison nor median-gap correlation establishes an association at this sample size.The other group includes reactive controls and tasks with undetermined retention intervals; excluding Long tasks reduces the difference to 0.048.Source: §5.4 · p. 8
• 1 success in 60 episodes (0.017) on the 3 evaluated Long-split tasks compares with 58 in 220 (0.264) on the other 11; task-level permutation p = 0.008.The split follows horizon bands, not task-name suffixes.Source: §5.4 · p. 9
• The Long-split contrast concentrates failures in the longest tasks, but its cause is unresolved because duration is coupled with memory-type composition and open-loop chunking.The Long tasks include the subset’s only Capacity and Sequential tasks and one Procedural task.Source: §5.4 · p. 9

6 Limitations and Future Work

• The reference evaluation covers one training run and checkpoint on 14 of 90 tasks; the separate architecture comparison cannot establish which tasks discriminate under identical training.Eight-action open-loop chunks and successful demonstrations with limited recovery leave long-task failures attributable to memory, motor error, or distribution shift.Source: §6 · p. 9
• Phase timing captures explicit information gaps, not all policy-dependent retention demands, and does not rule out shortcuts; the release is not a dense delay sweep.TimedTransfer measures elapsed-time tracking rather than recall of a hidden visual cue.Source: §6 · p. 9
• All environments and demonstrations are simulated, so physical-robot transfer is unmeasured; the release also contains no human demonstrations.The instruction set covers 80 tasks with one instruction each and no held-out paraphrases, limiting conclusions to task execution rather than linguistic generalization.Source: §6 · p. 9

7 Conclusion

• The 90-task suite spans 10 memory types and three horizon splits, and releases 22,500 oracle trajectories in RLDS and LeRobotDataset v3.For 28 tasks, the median information gap exceeds the widest fixed-context window surveyed; every memory-dependent task requires retention by construction.Source: §7 · p. 9
• The reference π0.5 baseline reaches 0.211 ± 0.044 mean success on 14 tasks, but this single result does not establish a ceiling for memory-free policies.Open-loop chunking and the memory types represented in the evaluated Long split confound interpretation of lower Long-split success.Source: §7 · p. 9
• The benchmark provides a fixed target for comparing memory-augmented VLA policies against a documented baseline using shared per-family reward and success definitions.The paper frames the suite as infrastructure for studying memory, rather than an answer to that question.Source: §7 · p. 9

Appendix Overview

• The appendix overview lists materials on dataset schemas, observation and action details, evaluation, task statistics, the reference baseline, external evaluations, and rollout videos.It also lists a datasheet, benchmark comparison, and task-family details.Source: Appendix Overview · p. 12

A Datasheet

• The datasheet follows the framework of Gebru et al. [2021].Source: A Datasheet · p. 12

Motivation

• The dataset supports training and evaluation of language-conditioned manipulation policies across task semantics and documented memory demands.Its stated purpose includes comparing recent-observation, recurrent-state, and external-memory policies across retention intervals.Source: Motivation · p. 12

Composition

• The release contains 90 tasks and 22,500 episodes, a fixed sample of 250 oracle rollouts per task rather than an exhaustive state-space enumeration.Episodes include two RGB streams, proprioception, actions, and an instruction; the release does not impose a single train/test split.Source: Composition · pp. 12–13

Collection Process

• Oracle rollouts use PPO policies on 34 tasks and scripted motion planners on 56, with observations re-rendered through the VLA-facing wrapper stack.Collection dates and PPO checkpoints are not recorded in the release, and reproducing initial states also requires matching simulator software, assets, and configuration.Source: Collection Process · p. 13

Preprocessing, Cleaning, and Labeling

• Only successful episodes are retained, and the release provides wrapped VLA-facing observations rather than the underlying simulator scene state.No other filtering, augmentation, or relabeling is applied.Source: Preprocessing, Cleaning, and Labeling · p. 13

Uses

• The demonstrations are controlled task-specific data from privileged-state policies and planners, not a proxy for real-deployment demonstrations or a source of failure-recovery trajectories.The dataset has been used to train and evaluate the paper's reference π0.5 baseline on 14 tasks.Source: Uses · p. 14

Distribution

• The benchmark code and trajectories are distributed in RLDS and LeRobot formats under the MIT License, subject to separate upstream software and asset licenses.Source: Distribution · p. 14

Maintenance

• Post-release corrections to the manifest or extraction scripts are to be documented in the repository changelog rather than silently changing released files.Source: Maintenance · p. 14

B Dataset Schemas

• Actions are normalized to [−1, 1]7 while proprioception remains in raw simulator units, and the gripper action is an absolute target rather than a delta.The six pose channels use controller-specific scaling, while a stored gripper value of zero commands the midpoint of its range, not an unchanged position.Source: B Dataset Schemas · pp. 14–15, C Observation and Action Space Details · p. 17
• The RLDS branch uses TFDS records plus sidecar metadata, while LeRobotDataset v3 uses Parquet data and AV1 videos; their schemas define the release-specific keys.Table 4 gives the LeRobot feature schema, and Table 5 gives the RLDS per-step schema with corresponding LeRobot keys where available.Source: B Dataset Schemas · pp. 14–15, Table 4 · p. 15, Table 5 · p. 16
• RLDS stores normalized dense per-step rewards, but LeRobot's own metadata omits rewards, success flags, and simulator seeds.The LeRobot schema documentation strings for reward and terminal status do not describe the dense per-step values actually written.Source: B Dataset Schemas · p. 15

C Observation and Action Space Details

• The observation dictionary includes two intended policy inputs and two diagnostic privileged-state channels, which must be dropped on six tasks to prevent answer leakage.Those channels are absent from released episodes, but the VLA wrapper does not remove them during live evaluation.Source: C Observation and Action Space Details · pp. 16–17

D Evaluation Protocol

• The released evaluation code fixes the wrapper construction, seed, episode count, and success criterion; reproducing reported numbers requires running that module.Saving videos also enables debug overlays, making recorded runs a different configuration from the default.Source: D Evaluation Protocol · p. 17, §D.1 · p. 18

D.1 What the released evaluation code fixes

• An episode counts as successful if its task success condition holds at any step, even if that condition is later undone.Source: §D.1 · p. 18

D.2 How the reference baseline’s evaluation differed

• Reproducing the baseline’s Long-variant numbers requires adding the cue-phase freeze wrapper, which the released evaluation module omits by default.The wrapper holds the arm still through the cue phase.Source: §D.2 · pp. 18–19
• The baseline’s split simulator backends produce potentially different trajectories, and the paper reports no paired comparison to bound the discrepancy’s size or direction.Motion-planned tasks use CPU simulation and PPO-sourced tasks use GPU simulation, matching their demonstration-collection backends.Source: §D.2 · p. 19

D.3 Chance floor: what a uniform-random policy scores

• 1 of 4,500 random-policy episodes succeeded; 89 tasks scored zero, and the sole success was absent at the episode’s final step.ShellGameShuffleTouch-Long-VLA-v0 scored 0.02 with a binomial standard error of 0.020.Source: §D.3 · p. 20
• The random-policy floor does not establish that a task requires memory or bound what a competent policy without retention could achieve.The selection floor is conditional on competent manipulation; the GatherAndRecall floor is conditional on manipulation succeeding first.Source: §D.3 · p. 20
• 0 of 7 selection-task rates clear 1/k by more than their own standard error, leaving the reference arm indistinguishable from the forgetful floor.The pooled result was 33 successes versus 35.3 expected at the floor, within the floor’s 5.1 spread.Source: Table 6 · p. 20, §D.3 · p. 20

E Wrapper Cookbook

• Environment construction requires six explicit settings, and the curriculum wrapper expects delta end-effector pose control rather than another control mode.The six settings include observation, control, render, reward, and simulator-backend modes, plus the number of parallel environments.Source: E Wrapper Cookbook · p. 21
• Two diagnostic observation channels must be removed before policy input; on six tasks, one otherwise persists unchanged from reset across steps.The instruction is returned separately, not as part of the observation dictionary.Source: E Wrapper Cookbook · p. 21
• The wrapper function defaults overlays on, while released evaluation defaults them off; applying the wrapper to an unlisted task id raises an error.Overlays alter rendering only, not the policy’s observation.Source: E Wrapper Cookbook · p. 21

F.1 Batteries Checker Easy

• Batteries Checker Easy requires testing each battery, remembering working ones, and confirming them; the cue remains visible for only 7 steps after a working hit.Automatic return advances the task to confirmation, whose timing depends on the agent’s motion; the easy variant reuses the hard variant’s dense reward.Source: §F.1 · p. 22

F.2 Batteries Checker Hard

• Batteries Checker Hard requires testing, returning, and confirming every battery; success requires discovering all working batteries through the full procedure.Its stage-gated reward shapes insertion, return, and confirmation actions, with event bonuses, an out-of-stage confirmation penalty, and a success bonus.Source: §F.2 · p. 23

F.3 Blink Count Button Press

• Blink Count Button Press requires counting blue-lamp blinks, pressing the red button exactly that many times, and submitting; its information gap is 0 steps.Success requires an exact match at submission; its reward is phase-decomposed across cue, counting, lift-and-release, and submit stages.Source: §F.3 · p. 24

F.4 Bunch Of Colors

• Bunch Of Colors requires touching every color shown during the cue, in any order; its long variants have a median 225-step gap, ranging from 138 to 313.Non-long variants have a median 3-step gap (2–4); success requires all target colors and no wrong selections.Source: §F.4 · pp. 24–25

F.5 Chain Of Colors

• Chain Of Colors requires reproducing the cue’s color sequence in order; its long variants have a median 225-step gap, ranging from 138 to 313.Non-long variants have a median 3-step gap (2–4); success requires the full correct order without wrong or out-of-order selections.Source: §F.5 · pp. 25–26

F.6 Find Imposter Color

• Find Imposter Color asks the robot to remember the cue’s colors and touch the cube whose color was absent; non-long variants have a median 3-step gap (2–4).Success is reaching the imposter cube during the manipulation phase.Source: §F.6 · pp. 26–27

F.7 Find Imposter Shape

• Find Imposter Shape asks the robot to remember the cue’s shapes and touch the object whose shape was absent; non-long variants have a median 3-step gap (2–4).Its reach-to-goal reward code matches Find Imposter Color.Source: §F.7 · pp. 27–28

F.8 Find Imposter Shape And Color

• Find Imposter Shape And Color requires remembering shape-color combinations and touching the novel combination, so a familiar shape or color alone does not rule out an imposter.Source: §F.8 · p. 28
• Find Imposter Shape And Color variants begin manipulation after a reset-drawn empty phase, yielding a 3-step median information gap (2–4).The gap is the empty phase itself, drawn uniformly from 1 to 5 steps.Source: §F.8 · p. 28

F.9 Gather And Recall

• Gather And Recall requires placing cubes while remembering a brief lamp flash, then pressing the matching color button; its cue-to-action gap is undetermined.The flash lasts 8–14 steps, but the button task starts only after all cubes are placed, so the gap depends on how quickly the remaining cubes are placed; each variant contributes 250 retained demonstrations with oracle success 1.000.Source: §F.9 · p. 29, Table 15 · p. 29

F.10 Intercept; F.11 Intercept Grab

• Intercept tasks deflect a rolling ball toward a target, while Intercept Grab tasks catch or stop it; in both families the ball remains continuously observable, so no retention gap is measured.Intercept Grab’s success condition requires the ball to be grasped and the robot to be static at the episode’s end.Source: §F.10 · pp. 29–30, “Intercept the rolling ball and grasp… · p. 30

F.12 Remember Color

• Remember Color variants require delayed matching by color, with short variants at a 3-step median gap (2–4) and Long variants at 250 steps (150–350).Both gaps are the reset-drawn empty phase before manipulation; short gaps draw 1–5 steps and Long gaps draw 50–450.Source: §F.12 · p. 31

F.13 Remember Shape; F.14 Remember Shape And Color

• Remember Shape requires delayed shape matching, while Remember Shape And Color requires matching both attributes; both offer short and Long variants with reset-drawn empty phases.For both families, the short phase draws 1–5 steps and the Long phase draws 50–450 steps; their manipulation begins after that phase.Source: §F.13 · p. 32, §F.14 · pp. 32–33

F.15 Rotate Lenient

• Rotate Lenient asks for a peg angle with a lenient center-position criterion, and the target angle stays available through an instruction recomputed at every step.Because the target is never hidden, these variants have no cue-hide phase over which to measure a retention gap.Source: “Rotate the peg by {angle_deg} degrees… · pp. 33–34

F.16 Rotate Strict

• Rotate Strict uses a narrow angle criterion and additionally requires the peg to stay near its initial XY position; its target angle also remains continuously available in the prompt.The target is set at reset and never hidden, so no cue-hide retention gap is measured.Source: §F.16 · p. 34

F.17 Seq Of Colors

• Seq Of Colors requires retaining which colored cubes appeared during the cue and their order, then touching the cubes and pressing the center button.Source: §F.17 · p. 35
• Seq Of Colors presents colors sequentially, but implemented success requires touching them in cue order despite order-insensitive shaping and an instruction allowing any order.The docstring describes set membership only; the success condition and released instruction therefore disagree.Source: §F.17 · p. 35

F.18 Shell Game Color Lamp Touch; F.19 Shell Game Push

• Shell Game Color Lamp Touch requires recalling a color-to-cup mapping to touch the lamp-matching cup, while Shell Game Push requires recalling the ball-hiding cup and pushing it forward.The color-lamp variant has a 0-step gap; the push variant also begins manipulation as the ball cue is hidden.Source: §F.18 · p. 36, §F.19 · pp. 36–37

F.20 Shell Game Shuffle Color Lamp Touch

• Shell Game Shuffle Color Lamp Touch requires tracking cup-color associations through shuffling before touching the cup matching the lamp color.Its standard and Long variants have gaps of 28 (24–31) and 250 (175–325) steps, respectively.Source: §F.20 · p. 37

F.21 Shell Game Shuffle Touch

• Shell Game Shuffle Touch requires tracking the ball-hiding cup through shuffles and touching its final position.The standard and Long gaps are 28 (24–31) and 250 (175–325) steps; its reward shaping is otherwise the same as shell_game_touch, including a cup-displacement penalty.Source: §F.21 · p. 38

F.22 Shell Game Touch

• Shell Game Touch requires remembering which cup covers the hidden ball and touching that cup, with no intervening empty phase.The gap is 0 steps, and shaping includes a penalty for displacing any of the three cups.Source: §F.22 · p. 39

F.23 Take It Back

• Take It Back requires pushing a cube to a target, then returning it to its original position after the target changes.Its information gap is undetermined because the return stage starts when the agent achieves the first goal, not at a fixed or sampled time.Source: §F.23 · pp. 39–40

F.24 Timed Transfer

• Timed Transfer requires counting from a lamp cue and placing a cube on the red disc within a target-step tolerance; delays are 100, 150, 200, 300, 500, or 1000 steps.The reward structure tests elapsed-time tracking rather than retention of a hidden visual cue, and placement too early or late fails.Source: §F.24 · pp. 40–41

F.25 Trace Shape

• Trace Shape asks the robot to watch a cube trace a shape and reproduce the demonstrated path with another cube.The task requires remembering the trajectory, not only its start and end points.Source: §F.25 · p. 41
• The Trace Shape variants require visiting every demonstrated-path checkpoint and closing the contour near its start; the red-cube cue disappears when manipulation begins, yielding a 0-step gap.The checkpoint-tracing reward includes reach and grasp terms, progress over visited checkpoints, closure reward, a smoothness penalty, and a fixed success bonus.Source: §F.25 · pp. 41–42

F.26 Trace Shape Seq

• Trace Shape Seq requires reproducing several demonstrated shapes in order, closing each contour before moving on and pressing a button after the sequence.Each variant contributes 250 retained demonstrations with oracle success rate 1.000; shaping adds sequence-progress and submit-button terms to checkpoint tracing.Source: §F.26 · p. 42, Table 32 · p. 42

G Full Per-Task Statistics

• Five *-Medium-VLA-v0 environment IDs are excluded from the 90-task benchmark because they belong to a different project.They are excluded from every count, table, and figure in the paper.Source: G Full Per-Task Statistics · p. 43
• Figure 29 ranks the 90 tasks by total released frames, and its numeric values are listed in Table 33.The same color denotes the same memory type across the paper’s figures.Source: G Full Per-Task Statistics · p. 43, Figure 29 · p. 44
• Table 33 reports full per-task statistics for all 90 canonical tasks, with 250 episodes per task and lengths, horizons, and gaps measured in environment steps.Its Gap column distinguishes fixed values, episode-varying values, state-dependent undetermined intervals, and tasks whose cues are never hidden.Source: Table 33 · p. 45

H Reference Baseline: Full Configuration

• Appendix H records the reference π0.5 baseline configuration, and its training and evaluation lanes use distinct simulator seed ranges.Distinct recorded seeds do not by themselves establish distinct initial states across simulator versions.Source: H Reference Baseline: Full Configuration · p. 48, Table 34 · p. 48

I Comparison to Existing Benchmarks

• Table 35 compares dedicated memory-dependent manipulation benchmarks across scale, controlled memory-difficulty variation, and environment and learning support.Its legend defines support as explicit, partial or entangled, or not documented or provided suite-wide; benchmark-specific type taxonomies are not directly comparable.Source: Table 35 · p. 48, I Comparison to Existing Benchmarks · p. 48

J Supplementary External Evaluations

• The three supplementary external evaluations use separate protocols and episode counts, so they are not a single leaderboard or part of the canonical 90-task evaluation.Each table caption specifies its own protocol and episode count.Source: J Supplementary External Evaluations · p. 48

J.1 Memory architecture comparison

• Table 36 presents OpenVLA-OFT with and without a 64-token recurrent memory module, fine-tuned on five tasks and evaluated on 18 held-out tasks under an earlier release protocol.The no-memory ShellGameTouch evaluation did not complete and is marked N/A.Source: Table 36 · p. 49, §J.1 · p. 49

J.2 Execution-regime comparison

• Table 37 compares two independently trained π0.5 checkpoints under eight-action open-loop and one-action-per-inference regimes, with 100 episodes per task per cell.This comparison is separate from the canonical benchmark evaluation.Source: Table 37 · p. 49, §J.2 · p. 49

J.3 Benchmark customization case study

• 0.540 success for the K = 8 window versus 0.080 for K = 2 on RememberColor5-Medium-VLA-v0, while neither training run reached its full step budget.These five custom variants are outside the canonical 90-task suite, and the aggregate means are not a matched final comparison.Source: §J.3 · p. 50, Table 38 · p. 50

K Rollout Contact Sheet

• Figures 30–39 provide one rollout-filmstrip sheet per memory type, with one row per task and eight evenly spaced frames per rollout.The filmstrips use the render channel and include debug text absent from released observations.Source: K Rollout Contact Sheet · p. 50, Figure 30 · p. 51, Figure 31 · p. 52