Orca: The World is in Your Mind

Yihao Wang; Yuheng Ji; Mingyu Cao; Yanqing Shen; Runze Xiao; Huaihai Lyu; Senwei Xie; Euan Liu; Klara Tian; Tianfeng Long; Yichi Zhang; Zhengliang Cai; Ruike Chen; Jifan Zhao; Ruochuan Shi; Zihan Tang; Jing Lyu; Wenxing Tan; Ningbo Zhang; Yangtao Hu; Yuming Gao; Xiansheng Chen; Junkai Zhao; Congsheng Xu; Boan Zhu; Ziqi Wang; Yupu Feng; Qiongqiong Zhang; Yingli Zhao; Yulong Ao; Shaoxuan Xie; You Liu; Guocai Yao; Leiduo Zhang; Xiaodan Liu; Yunyan Zhang; Yance Jiao; Xinyan Yang; Jiaxing Wei; Xu Liu; Tengfei Pan; Shaokai Nie; Chunlei Men; Sen Cui; Xiaojie Jin; Hongyang Li; Jianlan Luo; Yao Mu; Yunchao Wei; Jun Yan; Hang Zhao; Xiaolong Zheng; Jiaming Li; Yonghua Lin; Tiejun Huang; Zhongyuan Wang; Pengwei Wang

Source-linked AI summary

Orca: The World is in Your Mind

Yihao Wang, Yuheng Ji, Mingyu Cao, Yanqing Shen, Runze Xiao, Huaihai Lyu, Senwei Xie, Euan Liu, Klara Tian, Tianfeng Long, Yichi Zhang, Zhengliang Cai, Ruike Chen, Jifan Zhao, Ruochuan Shi, Zihan Tang, Jing Lyu, Wenxing Tan, Ningbo Zhang, Yangtao Hu, Yuming Gao, Xiansheng Chen, Junkai Zhao, Congsheng Xu, Boan Zhu, Ziqi Wang, Yupu Feng, Qiongqiong Zhang, Yingli Zhao, Yulong Ao, Shaoxuan Xie, You Liu, Guocai Yao, Leiduo Zhang, Xiaodan Liu, Yunyan Zhang, Yance Jiao, Xinyan Yang, Jiaxing Wei, Xu Liu, Tengfei Pan, Shaokai Nie, Chunlei Men, Sen Cui, Xiaojie Jin, Hongyang Li, Jianlan Luo, Yao Mu, Yunchao Wei, Jun Yan, Hang Zhao, Xiaolong Zheng, Jiaming Li, Yonghua Lin, Tiejun Huang, Zhongyuan Wang, Pengwei Wang

arXiv:2606.30534v3Jun 29, 2026cs.CV

TL;DR

Building a general model of the world requires learning latent state transitions from multimodal signals rather than optimizing isolated tasks. Orca learns such a latent space through next-state prediction and multimodal supervision, and its text, image, and action readouts improve as pre-training scales.

Problem
The paper asks how to build a general world model that continuously learns multimodal latent state transitions relevant to understanding, prediction, and action.
Method
Orca learns a unified world latent space through unconscious learning of dense video transitions and conscious learning of sparse language-described events and VQA supervision.
Results
Orca’s training losses decrease with model size and data scale, while text, image, and action readouts consistently improve as pre-training scales.
Takeaways & Limitations
Orca provides an early exploratory milestone toward general-purpose world foundation models with a shared latent interface for multimodal downstream readouts.
Takeaways & Limitations
Experiments are limited mainly to 0.8B and 4B models, and current training uses only one-tenth of the 125K-hour, 160M-annotation inventory.

Abstract

from arXiv · show

We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multimodal readout interfaces. Rather than optimizing isolated next-token, next-frame, or next-action prediction, we are centered on Next-State-Prediction modeling, offering a unified state-transition modeling route toward understanding, predicting, and acting upon the world. Orca learns through two complementary paradigms: unconscious learning captures dense natural state transitions from continuous videos, and conscious learning models sparse meaningful state transitions by language-described events and VQA supervision. For pre-training, we construct a large-scale world-learning inventory data, including 125K hours of video data and 160M event annotations. After pre-training, Orca learns a unified world latent space. To examine whether the learned latent supports downstream, we evaluate it by three representative downstream readouts: text generation, image prediction, and embodied action generation. Orca's backbone is frozen, and only the lightweight modality-specific decoders are trainable. Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts. Orca outperforms similar-sized specialized baselines. These results show that Orca, as a general world foundation model, presents a promising approach to understanding, predicting, and acting upon the world. Finally, we discuss the current limitations, aiming to provide useful insights and inspiration for the community.

1. Introduction

Orca learns a world latent space from multimodal signals as a general interface for downstream text, image, and action readouts. It combines unconscious learning of dense video transitions with conscious learning of meaningful, instruction-constrained event transitions.

Contribution: Orca learns a world latent space from multimodal world signals that supports multimodal downstream readouts.The decoder reads out text, images, and actions through general-purpose interfaces.
Learning paradigms: Unconscious learning captures natural, dense state transitions from continuous videos through next-frame latent prediction without labeled tags.The process uses self-supervision to learn natural evolution and internalize state transitions.
Learning paradigms: Conscious learning captures meaningful, sparse event-level state transitions under textual constraints associated with decisions and task outcomes.Instructions provide the constraints for learning meaningful transitions rather than dense natural evolution.
Readout evaluation: Orca freezes its backbone and trains only lightweight readout modules to test whether the paradigms are feasible and scalable and whether stronger world modeling improves downstream readouts.The readouts are intended to examine these core questions rather than pursue task-specific state-of-the-art performance.

2. Orca

Orca models the world as latent state abstraction and state transition from multimodal signals, using visual and language inputs in this version. It combines unconscious learning of dense natural transitions with conscious learning of sparse, meaningful transitions conditioned on events, tasks, or causal premises.

Macro: Orca maps multimodal world signals to latent states that capture invisible dynamics and explicit conditions for predicting future or past states.The latent dynamics may represent physical laws, object properties, scene dynamics, and environmental forces, while conditions include observed human instructions.
Details: Orca uses visual and language signals with two complementary paradigms: unconscious learning and conscious learning.The encoder learns a unified world latent space for state abstraction and transition modeling.
Unconscious learning: Unconscious learning derives dense, natural state transitions from observation alone, without explicit semantic conditions.It learns patterns such as object motion, natural dynamics, physical regularities, and scene transitions from nearby future observations.
Conscious learning: Conscious learning derives sparse, meaningful transitions from explicit language conditions describing future or past events, task intentions, or causal premises.It also supports VQA response generation, while event-conditioned training uses segmented videos paired with instruction descriptions.

3. Training

Orca trains a unified world latent through large-scale multimodal pre-training, then freezes its backbone while training lightweight modality-specific readouts for language, vision, and action. Its objectives combine dense observation-only transitions, language-conditioned transitions, and VQA supervision, supported by real-world video, event, and VQA data.

Training Stages: Training proceeds in two stages: multimodal pre-training learns the world latent, while downstream post-training freezes Orca’s backbone and trains only modality-specific readout modules.The readouts target language, vision, and action information.
Learning Objectives: Orca pre-training combines observation-only state transitions, event-conditioned state transitions, and VQA response generation.The first two objectives use learnable query vectors and supervise the frozen vision-encoder latent, while VQA uses the backbone’s LM head with next-token loss.
Pre-training Data: The training inventory includes 125K hours of general video, 160M event annotations, and 11.5M general VQA examples, with one-tenth of the video used in this version.Video covers egocentric interaction, exocentric manipulation, action-free robot execution, and natural dynamics; event and VQA data are built from these observations.
Downstream Readouts: Language readout reuses the LM head, vision readout maps latent representations to images through an MLP and frozen SD3.5, and action readout uses a trainable DiT-based Action Expert.The action module conditions on latent representations, robot proprioception, and noisy actions to generate action chunks.
Training Components: Throughput increases from 0.66 to 2.91 Samples/Sec/GPU, achieving approximately 4.4× acceleration over StarVLA.These optimizations are described as training-throughput improvements, with details and results provided in Appendix D.

4. Evaluation

Orca’s learning paradigm scales with model size and video data, while stronger pretrained world latents improve frozen-backbone readouts for text, images, and actions. Across downstream evaluations, Orca demonstrates strong overall performance and benefits from combining complementary pretraining objectives.

Scaling Evaluation: Orca’s loss decreases with more pretraining video data, while larger models achieve lower objective loss, demonstrating scalable learning.The 0.8B and 4B models both benefit continuously from increased data rather than converging quickly.
Scaling Evaluation: Stronger pretrained world latents improve text, image, and action readouts even when Orca’s backbone is frozen.These downstream evaluations are zero-shot, and action gains emerge despite using no action labels during pretraining.
Downstream Evaluation: Orca achieves the best overall result among same-size VLMs and large-size world models, demonstrating advantages of its learning paradigm.The evaluation uses no benchmark-specific training data or benchmark tuning.
Image Prediction: Orca improves image prediction on PRICE and preserves robot morphology, scene consistency, contact relationships, and instruction following under interaction-conditioned changes.Its learned latent contains predictive information about future visual states in real-world interactions.
Action Generation: Orca outperforms Qwen3.5 in all OOD action settings from scratch, reaches nonzero success from 0%, and remains comparable to pretrained 𝜋0.5.Its trajectories advance farther, stagnate less, and recover more effectively after execution errors.
Ablation Study: Jointly using 𝜆obs, 𝜆evt, and 𝜆vqa gives the most balanced readouts, with observation-only supervision especially helping action and event conditioning helping vision.VQA supervision preserves language interfaces and strengthens semantic grounding and commonsense constraints.

5. Conclusion

Orca learns a world latent space from multimodal signals and exposes it through dedicated readouts rather than optimizing isolated downstream tasks. The authors identify limitations in modalities, supervision, scale, evaluation, horizons, readouts, losses, and embodiment, and propose directions toward native, broader, and self-evolving world models.

Conclusion: Orca learns internal world-state representations from multimodal signals, then exposes them through dedicated readout interfaces instead of optimizing isolated downstream tasks.This modeling paradigm shifts the target toward learning a world latent space before supporting language, vision, and action readouts.
Discussion & Limitation: Orca mainly uses vision and language, omitting audio, tactile, force, light, and proprioceptive signals needed to represent broader physical state transitions.The authors propose aligning richer modalities to the same underlying state and constraining transitions with physical laws.
Discussion & Limitation: Its frozen ViT supervision aligns the learned state space with semantic representations, whereas a general world model should learn a unified space directly from multi-source signals.The proposed future direction is native world-state modeling trained from scratch without constraints from existing ViT or embedding spaces.
Discussion & Limitation: Experiments remain limited by 4B and 0.8B model scales, a narrow PRICE-V0.1 benchmark, short-horizon event annotations, and limited language, vision, and action readouts.The authors also note trade-offs among readout performance, insufficient evaluation diversity, inadequate long-term state evolution, and missing fields such as hearing, quantum circuits, and proteins.
Discussion & Limitation: Orca’s three-loss training is not fully consistent with Next-State-Prediction, while stringent but short and easy embodied tasks limit current conclusions.The authors call for simpler supervision and more comprehensive embodied evaluations.
Future Works: Future work proposes unified state-transition evaluation, a model-data-evaluation self-evolutionary loop, and extensions from embodied intelligence to scientific and complex systems.The evaluation framework would cover state prediction, intervention response, physical quantifiability, and counterfactual inference; the loop would generate, filter, train on, and improve data.

6. Author List · Appendix

The appendix lists contributors by role, covering core model development, infrastructure, evaluation, robotics, operations, expert consultation, and research leadership.

6.1. Core Contributors: The core contributor list names Yihao Wang, Yuheng Ji, Mingyu Cao, Yanqing Shen, and Runze Xiao among its principal contributors.The passage marks several names with asterisks and identifies Yuheng Ji with an additional dagger symbol.
6.1. Core Contributors: Core contributors covered model pre-training, data infrastructure, evaluation, real-robot work, downstream post-training, and infrastructure.These roles were assigned to the listed core contributors across the main development workflow.
6.2. Contributors: Additional contributors supported infrastructure, real-robot data, product and operations, brand management, platform management, and system management.The appendix distributes these responsibilities across the listed contributors.
6.3. Expert Consultant (Ordered by English last name alphabetically): Expert consultants are listed as Sen Cui, Xiaojie Jin, Hongyang Li, Jianlan Luo, Yao Mu, Yunchao Wei, Jun Yan, Hang Zhao, and Xiaolong Zheng.The names are ordered alphabetically by English last name.
6.4. Research Leads: The research leads are Jiaming Li, Yonghua Lin, Tiejun Huang, Zhongyuan WangB, and Pengwei WangB.The appendix presents these five names under the research-lead category.

A. Orca Conception … B.2. Next Token Prediction

Orca centers modeling on multimodal next-state prediction, learning a unified world latent through complementary implicit-dynamics and semantic-condition pathways. The related-work discussion distinguishes Orca from latent-prediction and next-token paradigms by treating language as a condition on world-state transitions rather than the sole prediction target.

A. Orca Conception: Orca models the latent world state as the central object, treating language, vision, and action as observations or readouts that support understanding, prediction, and intervention.This shifts the target from next-token, next-frame, and next-action prediction to next-state prediction.
A. Orca Conception: Orca combines unconscious learning of dense natural state transitions from continuous visual experience with conscious learning of meaningful transitions conditioned on language, events, instructions, and questions.The two modes capture physical regularities alongside causal explanations and task intentions.
A. Orca Conception: The learned world latent supports multimodal readouts to language for explanation and reasoning, vision for prediction and imagination, and action for intervention.These interfaces expose the same underlying representation across downstream capabilities.
B. Related Work: Related work is organized by each paradigm’s primary learning objective rather than by the full set of downstream capabilities its models may exhibit.Unified models spanning multiple capability domains are categorized by their dominant training formulation.
B.1. Self-Supervised Learning: JEPA-style models predict semantic representations instead of pixels, whereas Orca broadens latent prediction into multimodal world-state abstraction and state transitions under implicit dynamics and explicit semantic conditions.Orca’s formulation connects observation-only transitions with semantically conditioned transitions.
B.2. Next Token Prediction: Unified multimodal systems such as Emu3, Emu3.5, and BAGEL extend next-token prediction across modalities, blurring the boundary between token and frame prediction.The supplied discussion describes Emu3.5 as optimizing image-generation speed and performance within this autoregressive paradigm.
B.2. Next Token Prediction: Unlike next-token models that organize knowledge through autoregressive language modeling, Orca uses language as an explicit semantic condition guiding transitions toward target states.VQA response generation retains the language interface while strengthening commonsense and semantic grounding.

B.3. Next Frame Prediction … C.1.1. Pre-Training Objective Definitions

Orca distinguishes world modeling from frame synthesis and action-centered learning by prioritizing physically constrained state transitions and a world-learning-first foundation. Its pre-training combines observation-only transitions, event-conditioned transitions, and VQA generation using frozen-encoder latent supervision and weighted objectives.

B.3. Next Frame Prediction: Orca differs from image and video generation models by prioritizing action execution, physical plausibility, consistency, contact relationships, and instruction following in interaction-conditioned state transitions.Rather than functioning primarily as a painter, Orca models whether target states follow real-world physical constraints and interaction processes.
B.4. Next Action Prediction: VLA models provide a path toward stronger embodied generalization and multitask learning, while world-action models jointly model actions and future world states.The related approaches include OpenVLA, which uses a pre-trained VLM with large-scale robot data, and systems combining understanding, video generation, and action experts.
B.4. Next Action Prediction: Video action models introduce visual dynamics from video generation to reduce the action expert’s burden of simultaneously modeling visual dynamics and control information.The motivation is that a static VLM backbone can make action learning harder by requiring the action expert to learn both components together.
B.4. Next Action Prediction: VLA and world-action models center embodied learning on action or joint video-action prediction, whereas Orca first learns scene and object changes through state-transition modeling without pre-training action labels.This world-learning-first approach targets temporal changes, object motion, and local physical interactions as a general foundation.
C. Training Settings: The appendix provides implementation details for the training procedure described in Section 3.This material covers the loss definitions and sampling choices for Orca pre-training.
C.1. Pre-Training Settings: Orca’s pre-training objective contains observation-only state transition, event-conditioned state transition, and VQA response-generation losses.The first two terms use visual latents extracted by the frozen vision encoder, while VQA uses the language-modeling head with standard next-token prediction.
C.1.1. Pre-Training Objective Definitions: For state transitions, Orca matches predicted visual-token latents to ground-truth latents from the frozen vision encoder, predicting the next frame or adjacent event states.Event conditioning directs prediction toward either an earlier or later event state, and the event loss averages both transition directions.
C.1.1. Pre-Training Objective Definitions: The final objective is Lpre = 0.1 Lobs + 0.5 Levt + 0.4 Lvqa, with state-transition and VQA samples mixed at an approximate 5 : 1 ratio.Event-conditioned transition receives the largest loss weight, followed by VQA generation and observation-only transition.

C.1.2. Query-Based Implementation … C.2.3. Action Readout

The appendix specifies Orca’s query-based state-transition implementation, pre-training settings, and downstream readouts. Language, vision, and action interfaces expose the learned latent through autoregressive text, image generation, and flow-matching action prediction.

C.1.2. Query-Based Implementation: Observation-only queries predict the latent of a temporally next frame, while event-conditioned queries predict a random frame within an instruction-specified target event.All queries are trained from scratch; Query 2 reads the instruction-conditioned predictive state, with event directions computed from L_evt.
C.1.3. Pre-Training Hyperparameters: The appendix reports Orca’s model-scale, optimization, and objective-specific pre-training hyperparameters in Table C1.The supplied passage identifies Table C1 as the main pre-training hyperparameter summary but provides no numerical settings.
C.2. Downstream Readout Post-Training Settings: Downstream readout training implements language, SD3.5-based vision, and DiT-based action interfaces, with the overall architecture illustrated in Figure 4.These settings extend the readout training described in Section 3.2.
C.2.1. Language Readout: The language readout reuses the VLM backbone’s language-modeling head to autoregressively produce responses for VQA, event interpretation, and causal explanation.It introduces no additional trainable module and takes a visual observation and instruction as inputs.
C.2.2. Vision Readout: The vision readout maps Orca’s predicted visual latent into image space using a pretrained SD3.5 decoder, with only an MLP adaptor and decoder-attention LoRA trainable.The SD3.5 VAE and MMDiT weights remain frozen; the adaptor projects the latent into token-level and pooled conditioning spaces, and targets are resized to 768 × 768.
C.2.3. Action Readout: The action readout combines Orca’s predictive query state, noisy time-embedded actions, and robot proprioception to generate short-horizon manipulation action chunks.A DiT-based Action Expert uses flow matching, with proprioception including joint and end-effector information.
C.2.3. Action Readout: The Action Expert is trained by perturbing ground-truth action chunks with Gaussian noise and predicting the corresponding velocity under a flow-matching loss.Its architecture and training settings are summarized in Table C3.

D. Infrastructure … E.2. Image Prediction

The paper combines system-level infrastructure optimizations for scalable Orca training with text-generation evaluation settings spanning video understanding, temporal dynamics, and 3D spatial reasoning.

D. Infrastructure: Orca’s training infrastructure targets distributed sharding, memory-efficient execution, and communication-efficient scheduling to address the higher memory and communication costs of its multimodal branches.The pipeline integrates visual embedding, language modeling, future visual-latent prediction, and action-related branches.
D. Infrastructure: FSDP2 enables flexible parameter, gradient, and optimizer-state sharding, while resharding reduces redundant copies and unsharded lightweight blocks limit communication overhead.These choices improve multi-GPU efficiency and maintain training stability.
D. Infrastructure: Activation recomputation checkpoints selected boundaries and reconstructs intermediates during backpropagation, trading computation for memory savings that support larger batches and higher throughput.This is designed for memory-constrained training settings.
D. Infrastructure: Chunked cross-entropy computes token blocks separately to avoid materializing the full logits tensor and reduce peak memory under long sequences and large vocabularies.The optimization addresses the log-softmax intermediate tensor’s memory spike in the VLM forward stage.
D. Infrastructure: Forward/backward pre-fetching overlaps upcoming-layer parameter all-gathers with current-layer computation, reducing communication stalls and GPU idle time.The scheduling improves overall device utilization under FSDP2.
D. Infrastructure: 2.91 samples/sec/GPU on H100 GPUs, a 3.0× gain over the FSDP2 baseline and 4.4× over the StarVLA training pipeline, demonstrates the optimized infrastructure’s throughput advantage.The comparison is reported in Table D1.
E.1. Text Generation: Text generation is evaluated with MVBench (Li et al., 2024), TemporalBench, and 3DSRBench (Ma et al., 2025), covering video understanding, fine-grained temporal dynamics, and 3D spatial reasoning.The benchmarks include multiple-choice video QA, temporal-frequency and event-order analysis, and reasoning about height, location, orientation, and multiple objects.
E.1. Text Generation: The text-generation baselines include V-JEPA 2.1 (Mur-Labadia et al., 2026), a self-supervised latent world-model baseline, and Emu3, a unified discrete-token multimodal model.V-JEPA 2.1 emphasizes spatially grounded and temporally consistent visual understanding, whereas Emu3 uses next-token prediction across images, text, and videos.

E.2.1. Benchmarks … E.3. Action Generation

The paper evaluates instruction-conditioned image-to-image state prediction with PRICE-V0.1, using multimodal judging criteria and comparisons against open-source image-generation baselines. PRICE emphasizes realistic, instruction-following state changes while preserving scene consistency.

E.2.1. Benchmarks: PRICE-V0.1 evaluates whether a model can generate the target state image resulting from an instructed real-world interaction.The benchmark uses an initial state image and an action instruction in an instruction-conditional image-to-image generation task.
E.2.1. Benchmarks: PRICE combines robot and first-person interaction data from AgiBot-World, HomeInteract, PE-Video, and PSI-Ego.Each sample contains an instruction, an initial state image, and a target state image; HomeInteract is closed-source data collected by a dual-arm wheeled robot in home scenes.
E.2.2. Metrics: Evaluation uses Gemini 3.1 Pro, GPT 5.4, and Doubao Seed 2.0 Pro, with open-source Gemma 4-31B serving as a reproducible judge.The judge reads the initial image, instruction, and generated target image, then assigns an integer score from 1 to 5 based on action execution and scene consistency.
E.2.2. Metrics: The judge penalizes outcomes that appear teleported rather than executed, while allowing imperfect progress, minor occlusion, detail loss, blur, texture shifts, and small artifacts.For agent-action instructions, the agent’s pose, position, or contact state should visibly change when appropriate.
E.2.2. Metrics: The metric assesses scene preservation, correct action-induced state changes, and the absence of physically implausible content.It prioritizes preserving the input environment, viewpoint, and major-object layout while reflecting changes such as objects being moved, opened, picked up, or lifted.
E.2.3. Baselines: Image-prediction comparisons include OmniGen2 and FLUX.1-Kontext as generative baselines for unified image generation, editing, and in-context generation.OmniGen2 uses separate text and image readout pathways, a decoupled image tokenizer, and task-specific data pipelines; FLUX.1-Kontext is a flow-matching model accepting text and image inputs.

E.3.1. Real-Robot Benchmark … F. More Visualization

Orca is evaluated on five real-robot manipulation tasks under environment and object OOD settings, using rule-based completion scores and trajectory-level PRM-as-a-Judge diagnostics. Detailed results and visualizations examine task progress, failure recovery, and execution quality.

E.3.1. Real-Robot Benchmark: The real-robot benchmark evaluates five manipulation tasks on a dual-arm wheeled humanoid across environment and object OOD settings.Each task uses 200 real-robot trajectories for Action Expert post-training; environment OOD changes tablecloths or backgrounds, while object OOD replaces objects with semantically related unseen instances.
E.3.2. Metrics: Real-robot performance is measured with task-specific rule-based completion scores and dense trajectory-level PRM-asa-Judge diagnostics.Rule-based scoring records the highest completed stage before termination, while PRM-asa-Judge assesses progress and execution quality.
E.3.4. Detailed Real-Robot Results: Table E3 reports detailed rule-based task completion results under the real-robot OOD settings.The appendix presents task-level completion under manually designed scoring criteria, while PRM-as-a-Judge results are used in the main-text analysis.
E.3.4. Detailed Real-Robot Results: Evaluation stops within 60 seconds after severe collision lockup or an unrecoverable object fall, counting only the highest score achieved beforehand.These stopping and scoring conditions define the detailed rule-based evaluation protocol.
E.3.5. Additional Qualitative Visualizations: Additional trajectory visualizations show Orca maintaining higher intermediate progress in failures and recovering from grasp failures.Examples cover Stamp, Pull Out Tissue, Stacked Bowls, and Scoop Sugar, contrasting Orca with Qwen3.5, π0.5, or JEPA.
F. More Visualization: The additional visualizations document process-level failure progress and recovery behavior across representative real-robot trajectories.Orca retries or eventually succeeds after spoon-grasp failures, whereas comparison systems remain stagnant, shake without effective re-grasping, or fail earlier.

F.1. Cross-Benchmark Capability Analysis for Text Generation

Orca exhibits cross-benchmark capabilities in state transition, commonsense reasoning, spatial relations, and dynamic motion. Representative examples show stronger answers than Qwen on several temporal, motion, and spatial reasoning questions.

State transition: Orca correctly predicts immediate outcomes in state-transition questions, answering that copying begins processing and that the door opens first, unlike Qwen.These tasks assess causal temporal dynamics and physical state changes, with especially evident improvement on irreversible processes.
Commonsense reasoning: Orca’s cross-benchmark capability analysis also includes commonsense reasoning, particularly for complex VQA scenarios requiring inference beyond the visible scene and hypothetical outcomes.The analysis identifies state transition, commonsense reasoning, spatial relations, and dynamic motion as generalized capability dimensions transcending benchmark boundaries.
Dynamic motion: Orca demonstrates stronger dynamic-motion reasoning, identifying the yellow cube’s movement as down and to the left rather than Qwen’s up and to the left.The proposed unconscious learning paradigm is described as acquiring temporal continuity and motion inertia for forward simulation of dynamic object behavior.
Spatial relations: Orca handles spatial relations robustly, correctly determining that the tank and flags are far apart and identifying the truck’s left side facing the stop sign.The examples involve complex occlusions and multi-object spatial reasoning; both models answer the traffic-light proximity question identically.