Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI

Motivation

• Kairos defines a control-sufficient state as a compact representation preserving action consequences, failure boundaries, task progress, contact conditions, and deployment-relevant risk.The state is intended to support action selection, risk assessment, failure anticipation, and recovery planning without simulating all sensory information.Source: §1 · pp. 5–6

Regret formulation

• The paper formalizes representation-induced regret as the excess expected physical cost incurred when planning from compressed state Z_t instead of the full observation–action history H_t.This cost includes task failure, collision, unsafe contact, recovery effort, and discrepancies between imagined and real outcomes.Source: §1 · p. 6

Cross-Embodiment Data Curriculum

• Kairos organizes open-world videos, human behavior, and robot interactions by increasing intervention strength to move from physical observation toward action–outcome causation.Open-world videos provide broad physical priors, human data provides intentional task structure, and robot data grounds embodiment-specific constraints and perception–action alignment.Source: §1 · pp. 7, 8, 12

Hybrid Linear Temporal Attention

• Hybrid Linear Temporal Attention separates local, mid-range, and global pathways to maintain contact dynamics, subtask continuity, object permanence, task progress, and failure history across timescales.The design combines Sliding-Window Attention, Dilated Sliding-Window Attention, and Gated Linear Attention for efficient persistent state maintenance.Source: §1 · pp. 7, 9, 10

Deployment-Aware System Co-Design

• Kairos treats latency, memory, communication, and hardware compatibility as modeling constraints because efficient inference determines whether the state can participate in real observation–action–feedback loops.Hardware-aware kernels, quantization, token streaming, timestep distillation, and runtime optimization are presented as prerequisites for processing control-relevant information before action execution.Source: §1 · pp. 8, 10, Figure 1 · p. 9

Native Understanding–Generation–Prediction Architecture

• Its unified architecture maintains semantic understanding, visual generation, physical prediction, and action prediction in a shared world-action state, with future actions modeled as part of world evolution.World Generation regularizes physical consistency, while World Prediction jointly models environmental dynamics and future robot actions through coupled video and action streams.Source: §1 · pp. 8, 9, 12

Deployment-Aware System Co-Design

• Deployment-Aware System Co-Design integrates runtime optimization, memory efficiency, hardware-aware execution, quantization, and scalable inference to support practical observation–action–feedback loops.The design targets imagined-rollout evaluation, failure anticipation, unsafe-action filtering, and policy refinement from real feedback.Source: §1 · p. 12

Control-Sufficient Information Chain

• The contributions form a control-sufficient information chain in which data curriculum, unified state compression, temporal attention, and action-conditioned models connect information acquisition to alternative-action responsiveness.The Cross-Embodiment Data Curriculum determines information sources; the unified architecture compresses visual, semantic, physical, and action-related information; Hybrid Linear Temporal Attention maintains state over time; World Prediction and Action DiT respond to alternative actions.Source: §1 · p. 12

Results

• Evaluations across embodied world-model, world-action, general world-model, long-horizon generation, and inference-efficiency settings provide evidence for several capabilities relevant to regret-aware Physical AI.These include physical consistency, instruction grounding, joint world-action prediction, long-horizon state maintenance, and efficient deployment; they are proxy evidence rather than direct estimates of RegH or proof of closed-loop regret minimization.Source: §1 · p. 12

Limitations and Future Work

• Reduced RegH in real-robot deployment remains future work, requiring closed-loop experiments on rollout correlation, pre-execution failure prediction, safety filtering, recovery learning, and policy improvement.These evaluations are needed to determine whether Kairos can progress from proxy capability to real-world regret reduction.Source: §1 · pp. 12–13

2 Model

• Kairos defines Zt as a control-sufficient state that compresses observation–action history and task goals into variables relevant to embodied decision-making rather than complete future visual detail.The target variables include object state, spatial relations, contact conditions, task progress, action consequences, failure boundaries, safety risk, and deployment uncertainty.Source: §2 · p. 13, §2.1 · p. 13

2.1 Native Architecture with Unified Understanding, Generation, and Prediction

• The native architecture treats World Understanding, World Generation, and World Prediction as coupled interfaces to one shared world-action state rather than independent modules.Understanding constructs Zt, Generation imagines physically plausible futures to probe and regularize it, and Prediction maps it to future state-action trajectories and executable action tokens.Source: §2.1 · pp. 13–14, Figure 4 · p. 14

2.1.1 World Understanding: Control-Sufficient State Construction

• World Understanding constructs Zt from heterogeneous multimodal history and task instructions, preserving semantic, physical, task-progress, and uncertainty variables for downstream control.The implementation uses visual latent tokens and language-conditioned features as the first compression stage of the world-action stack.Source: §2.1.1 · pp. 14–15
• World Understanding is task-aware and history-aware: it prioritizes control-relevant information such as contact, affordances, safety margins, delayed effects, and task progress under the current goal.The state is maintained through temporal modeling so it can retain motion, object persistence, and progress information unavailable from a single frame.Source: §2.1.1 · p. 15

2.1.2 World Generation: Control-State Regularization

• World Generation serves as control-state regularization, using future-observation modeling to test and reinforce physical plausibility, object permanence, task consistency, and long-horizon temporal coherence.Its value is not photorealism alone, but whether generated futures reveal that Zt preserves dynamics and information useful for physical decision-making.Source: §2.1.2 · pp. 16–17

2.1.3 World Prediction: Joint World-Action Modeling

• World Prediction formulates future visual dynamics and robot actions as a unified World-Action Model, coupling action tokens to the same state and physical priors used for world modeling.Its Video DiT and Action DiT jointly model future visual and action tokens, moving beyond observation prediction toward embodied control.Source: §2.1.3 · pp. 17–18
• Kairos supports deployment-time action-only inference: future video synthesis can be skipped while retaining jointly learned world dynamics, reducing diffusion and attention computation within the robot control loop.This asymmetric design trains video and action branches together, but does not require future video tokens to generate future actions.Source: §2.1.3 · pp. 17–19

2.2 Efficient Diffusion Transformer with Hybrid Linear Attention

• Hybrid Linear Temporal Attention factorizes control-state maintenance across local dynamics, mid-range dependencies, and global causal memory within a unified DiT architecture.SWA handles motion, contact, and collision; DSWA captures interaction history and subtask transitions; GLA preserves task progress, scene memory, delayed effects, and failure history without dense attention over all tokens.Source: §2.2 · p. 20

2.2.1 Global Pathway: Gated Linear Attention as Persistent Causal Memory

• GLA provides linear-complexity persistent global causal memory, propagating object permanence, task progress, delayed effects, scene context, and long-range dependencies beyond the local temporal window.Implemented with GatedDeltaNet, its delta update removes outdated key–value associations, while gating adaptively controls forgetting and preserves relevant long-term information.Source: §2.2.1 · pp. 20–22

2.2.2 Local and Mid-Range Pathways: Sliding-Window and Dilated Sliding-Window Attention

• SWA and DSWA restrict attention to local or dilated temporal neighborhoods, efficiently capturing fast physical dynamics and intermediate event continuity while preserving geometry with RoPE-based relative positions.Interleaving SWA with DSWA dilation factors d ∈ {6, 12} expands the receptive field without quadratic cost; GLA remains responsible for global positional and causal information.Source: §2.2.2 · p. 22
• The modular hybrid design supports interactive world modeling, long-horizon generation through recurrent state passing, and cross-modal integration with constant per-step local memory and compressed global history.Action tokens can update the GLA state or gating mechanism for latency-aware control, while additional streams fuse local features through SWA/DSWA and maintain global alignment through GLA.Source: §2.2.2 · pp. 22–23

2.3.1 Necessity of Persistent Internal States

• Theorem 1 shows that purely local temporal models incur positive excess risk exactly when the full-history predictor is not recoverable from the recent window.This limitation is information-theoretic: scaling parameters or training compute cannot remove the gap without preserving supra-window information.Source: §2.3.1 · pp. 23–24

2.3.2 Approximate Sufficiency of Hybrid Multi-Scale Temporal Memory

• Under branch-wise approximation error ε and contractive global-memory updates with factor ρ, Theorem 2 bounds hybrid long-horizon excess risk by approximation and global-memory perturbation terms.The global perturbation contribution is bounded by LG ξ̄/(1 −ρ), and contraction prevents one-step errors from accumulating arbitrarily.Source: §2.3.2 · pp. 24–25
• In the realizable case ε = 0 and ξ̄ = 0, the learned hybrid state exactly recovers the Bayes-optimal predictor, establishing exact sufficiency under the theorem’s assumptions.The hybrid design assigns local and mid-range branches to shorter temporal structures and GLA to persistent causal context.Source: §2.3.2 · p. 25

2.3.3 Interpretation for Kairos

• The theory motivates SWA, DSWA, and GLA as short-, mid-, and global-timescale pathways for maintaining control-relevant state across fast motion, intermediate events, and supra-window causal context.It supports persistent memory and hybrid factorization under stated assumptions, while direct real-robot validation and closed-loop regret minimization remain future work.Source: §2.3.3 · p. 25

2.3.4 From Theoretical State Maintenance to Regret-Aware Physical AI

• For regret-aware Physical AI, persistent memory preserves local contact and motion, mid-range interaction history and subtask progress, and global context such as object permanence, delayed effects, failures, and unfinished tasks.Forgetting such information can leave predictions visually plausible but control-irrelevant, making long-horizon state maintenance a prerequisite for future closed-loop regret reduction.Source: §2.3.4 · pp. 25–26

3 Native Pretraining Paradigm for Physical AI

• CEDC organizes pretraining by intervention strength, progressing from open-world physical priors to human intentional behavior and finally robot-specific action–outcome grounding.This developmental pathway addresses interventional generalization by moving beyond passive observation toward embodied action causation.Source: §3 · pp. 26–27

3.1 Native Pretraining with Cross-Embodiment Data Curriculum

• CEDC uses open-world videos, human-centric behavior, and robot interactions as distinct stages because each supplies different control-relevant information and progressively reduces the gap between visual forecasting and physical execution.The curriculum first acquires broad physical regularities, then task structure, and finally embodiment-specific signals such as actions, proprioception, tactile or force cues, constraints, errors, and feedback.Source: §3.1 · pp. 27–28

3.2 Stage I: Physical Pretraining

• Stage I trains Video DiT on large-scale open-world images and videos to learn spatial–temporal regularities and broad physical priors before robot embodiment or executable actions.Progressive image-to-video training connects spatial structure with motion dynamics and extends temporal modeling, making the backbone a stronger state regularizer for Z_t.Source: §3.2 · pp. 29–31

3.3 Stage II: Embodied Pretraining with Human-centric Data

• Stage II uses human-centric behavioral data to make Video DiT task-sensitive and instruction-aware by learning goal-directed actions, task structure, behavioral causality, and environmental consequences before robot action grounding.It provides transferable action-relevant priors but does not yet solve robot control or bind the representation to a specific robotic action space.Source: §3.3 · p. 33
• Stage II uses first- and third-person human data, robot-centric visuals, and evolving captions to build task-structured, instruction-conditioned embodied dynamics while preserving physical grounding.Human data supplies transferable priors about tool use, manipulation, task decomposition, and recovery, but does not replace robot data or provide final action grounding; target-embodiment fine-tuning can specialize the model further.Source: §3.3 · pp. 33–34

3.4 Stage III: Regret-Aware World-Action Training

• Stage III aligns world prediction with executable robot action by combining regret preference supervision from execution outcomes with joint training of Video DiT and Action DiT.Failures, recoveries, unsafe contacts, unstable executions, and prediction–observation mismatches shape shared representations toward safer, more stable, recoverable, and execution-consistent futures.Source: §3.4 · p. 34

3.4.1 Regret Alignment Training

• Regret Alignment Training converts high-control-information robot experience into DPO-style preference pairs based on task completion, physical plausibility, contact stability, safety margin, recoverability, and rollout agreement.Failures and boundary cases are retained as contrastive evidence about physically costly futures rather than filtered as ordinary bad data.Source: §3.4.1 · p. 36

3.4.2 Joint World-Action Training

• Joint World-Action Training temporally aligns video and action trajectories so shared states support both visual forecasting and executable action prediction, enabling counterfactual outcomes for different action candidates.This coupling grounds preference alignment in the same representation used for action prediction, while addressing contact-rich phenomena such as slip boundaries, end-effector corrections, and force–torque-sensitive interactions.Source: §3.4.2 · pp. 36–37

3.5 Training Infrastructure for Control-Sufficient Pretraining

• Kairos uses operator-level, fused, and adaptive parallelism to train long, high-resolution, heterogeneous Video DiT–Action DiT systems despite hybrid attention’s sequential dependencies and communication costs.The infrastructure supports 720P and 15-second sequences while aligning training, memory, action interfaces, token streaming, and action-only inference with deployment-aware world-action modeling.Source: §3.5 · pp. 37–38

3.6 Summary: From Flat Data Scaling to a Staged Cross-Embodiment Curriculum

• The Native Pretraining Paradigm organizes data by intervention strength: Stage I supplies passive physical priors, Stage II adds task-structured embodied visual dynamics, and Stage III grounds them in robot actions, outcomes, and execution preferences.Its purpose is to turn heterogeneous experience into a developmental pathway for the control-relevant world-action variables needed by regret-aware Physical AI, rather than merely scaling data or retrofitting generic video generation.Source: §3.6 · p. 38

4 Data

• Kairos evaluates data by control information density: information gained about control-relevant variables per unit acquisition, annotation, compute, safety, or deployment cost.The current definition is a conceptual proxy for the regret objective, not yet an implemented optimization target.Source: §4 · p. 39
• Near-boundary failures and recoveries are prioritized over near-boundary successes, contact-rich data, ordinary successful trajectories, and ordinary observation videos because they expose controllability limits and recovery conditions.The ordering is diagnostic rather than label-based; scale remains necessary for coverage and physical priors but is insufficient for failure anticipation and action–outcome reasoning.Source: §4 · p. 39, §8.1.5 · p. 88

4.1 Data Collection: Multi-Source Experience Acquisition

• Kairos combines public datasets, compliant Internet acquisition, and first-person human manipulation data to cover passive observation, intentional intervention, and embodied interaction regimes.Its in-house Internet-scale system spans human, robot, general-scene, and physical-phenomena domains, while ego-centric data captures reachability, contact transitions, task progress, and recovery.Source: §4.1 · pp. 40–41
• The collection pipeline standardizes raw videos into searchable training assets through segmentation, duration rules, and large-scale processing, yielding hundreds of millions of clips for later control-relevant filtering and sampling.PySceneDetect achieves over 95% segmentation precision and approximately 80% recall; retained segments span 5–40 seconds, with longer shots split into 20-second clips.Source: §4.1 · p. 41

4.2 Data Curation: From Quality Filtering to Control-Relevant Filtering

• Kairos uses two-level curation: basic quality filtering followed by control-relevant event filtering, because visually clean clips are not necessarily informative for physical control.Current filtering emphasizes visual quality, temporal coherence, safety, motion, and redundancy; explicit control-information-density filtering remains a future extension.Source: §4.2 · pp. 41–42, Control-Relevant Event Filtering · p. 43

Basic Quality Filtering

• Quality filtering removes low-aesthetic, excessively low- or high-motion, low-quality synthetic, unsafe, blurry, and near-duplicate clips before training, using specialized predictors, sharpness assessment, and embedding-based deduplication.The pipeline uses a CLIP-based aesthetic predictor, RAFT for optical flow, a ViT-Large discriminator, Falconsai, Laplacian sharpness assessment, and CLIP embeddings; deduplication retains only the higher-resolution or higher-quality version when similarity exceeds a threshold.Source: Basic Quality Filtering · pp. 42–43

Control-Relevant Event Filtering

• Future control-relevant filtering targets near-boundary failures, recovery, near-boundary successes, contact transitions, safety anomalies, and long-horizon dependencies because these events reveal failure boundaries and corrective strategies.The current report presents these filters as a guiding extension that could use detectors, human annotation, simulation labels, robot logs, and tactile–force signals.Source: Control-Relevant Event Filtering · p. 43

4.3 Tagging: Structured Indexing for Control-Relevant Sampling

• Structured tags make heterogeneous videos searchable and stage-specifically samplable, covering attributes, domains, intervention levels, physical events, object–contact–action relations, task progress, failures, safety, and embodiment.Tags provide weak supervision and retrieval handles for the Cross-Embodiment Data Curriculum, while tag-conditioned captioning produces more complete, less ambiguous, control-aligned descriptions.Source: §4.3 · pp. 44–45

4.4 Captioning: From Visual Description to Control-State Supervision

• Captioning converts raw video into scalable weak supervision for control-relevant variables, using structured tags, multimodal ensembles, and semantic, physical, embodied, task-progress, failure, and safety objectives.The current implementation supports semantic, physical, and long-horizon task-oriented captions, while explicit failure–recovery and safety-risk captions remain future extensions; captioning does not replace action labels, tactile data, or robot rollouts.Source: §4.4 · pp. 46–47

4.5 Enhanced Text with Control-Oriented Chain of Thought

• Enhanced text reinforces physical laws and long-horizon causal structure, encouraging preservation of physical relations, task dependencies, failure boundaries, and recovery logic for control-sufficient state learning.Physics-centric captions explain motion, collisions, gravity, and friction, while long-horizon annotations decompose complex tasks into executable sub-steps with explicit dependencies and causal relationships.Source: §4.5 · pp. 47–48, Figure 13 · p. 47

4.6 Data Engineering Infrastructure for Scalable Control-Information Processing

• Kairos improves data-processing throughput by over 30× end-to-end on one 8×4090-GPU machine through parallelization, I/O optimization, and task scheduling.Shot detection reaches 8,640.0 hours/day at 7.4× throughput with 77.44% recall, while frame filtering and captioning reach 29.9× and 34.0× improvements, respectively.Source: §4.6 · pp. 48–49

4.7 Limitations and Future Data Directions

• Kairos data limitations include weak control-value metrics, incomplete alignment across internet, human, and robot data, potentially hallucinated annotations, and unvalidated synthetic experience.Future infrastructure should measure failures, recovery, safety, imagined–real rollout discrepancies, and whether data improves closed-loop policy behavior.Source: §4.7 · pp. 49–50

4.8 Summary: A Data Engine for Control-Sufficient World Modeling

• Kairos reframes data infrastructure as a control-sufficient engine that prioritizes information about action consequences, contact dynamics, failure boundaries, recovery strategies, and safety risks over raw scale.Its capabilities span multi-source collection, clip standardization, hierarchical curation, structured physics and task annotations, and high-throughput processing, linking data-side control information to future regret-aware decision support.Source: §4.8 · p. 50

5 Inference

• Kairos defines inference as regret-relevant information throughput, prioritizing timely control-relevant information under time, memory, compute, communication, and hardware constraints over visually rich but operationally delayed rollouts.This supports action selection, risk assessment, failure anticipation, recovery planning, and future closed-loop improvement.Source: §5 · pp. 50–51

5.1 Toward Self-Evolution: Proxy Rollout–Evaluation–Refinement

• Kairos’s unified understanding–generation–prediction architecture enables rollout–evaluation–refinement loops that simulate candidate futures, rank them by plausibility, progress, and risk, and refine decisions or prompts.The framework is intended to support future self-evolution that reduces predicted and eventually real physical cost, but current validation remains a controlled prompt- and generation-level proxy without demonstrated real-robot policy improvement.Source: §5.1 · pp. 51–52, Figure 14 · p. 52

5.2 Prompt Self-Alignment

• Prompt Self-Alignment uses a rewriting agent and multidimensional reward function to iteratively generate, evaluate, and refine prompts for physical plausibility, task completion, instruction alignment, and visual quality.The tested prompt-refinement loop can operate without human intervention after initialization, but validation of real-robot policy improvement remains pending.Source: §5.2 · pp. 52–53

5.3 Inference Efficiency

• Kairos addresses inference-efficiency trade-offs across cloud and consumer hardware through Timestep Distillation and Hardware-Aware Inference Optimization.These vectors reduce diffusion sampling latency and imagined-rollout cost while co-designing computational kernels and memory footprints for hardware utilization.Source: §5.3 · p. 53

5.3.1 Timestep Distillation

• With only four inference steps, the distilled Kairos generator qualitatively preserves much of the teacher’s spatial structure, motion dynamics, physical interactions, and temporal consistency on PAI-Bench.The method transfers distributional knowledge and trajectory dynamics for efficient high-fidelity embodied video generation, while quantitative teacher comparisons remain future work.Source: Figure 15 · p. 54, §5.3.1 · p. 56

5.3.2 Hardware-Aware Inference Optimization

• Hardware-aware optimization combines mixed parallelism, cache reuse, compilation and kernel fusion, low-precision attention, and tiled streaming access to reduce latency, memory usage, and long-horizon out-of-memory risk.The design targets both low-latency cloud generation and cost-effective inference on consumer-grade, low-memory devices.Source: §5.3.2 · pp. 57–58
• Hardware-aware inference combines INT4 weight-only text-encoder quantization with preserved high-precision activations to reduce memory and enable sub-millisecond keyword grounding.The approach targets standard edge-computing hardware while maintaining negligible semantic-accuracy degradation.Source: §5.3.2 · p. 58

5.3.3 Efficiency Comparison

• Kairos-4B uses 23.5 GB memory, 2.3 PFlops, and 43/9-second inference on 1/4 GPUs, outperforming three comparison models across evaluated efficiency metrics.The comparison uses TI2V at 720P for 5 seconds on an NVIDIA A800 server.Source: Table 5 · p. 58, §5.3.3 · p. 59
• Kairos-4B has the lowest latency across tested resolutions and durations, with 28×–85× speedups over Cosmos-Predict2.5-14B and 2.5×–3.7× over Wan2.2-5B.Its latency scales linearly as resolution and duration increase, while other models exhibit exponential growth; 480P generation on four NVIDIA A800 GPUs reaches real-time throughput.Source: §5.3.3 · pp. 58–59, Table 4 · p. 58

5.4 Inference Modes for Kairos

• Kairos supports full-visual, latent-evaluation, action-only, and proxy self-alignment modes, trading visual fidelity and interpretability against latency and compute cost.Action-only deployment disables future-video generation and is expected to reduce attention and diffusion costs, but quantitative savings are not reported; proxy self-alignment is validated without demonstrating autonomous robot policy improvement.Source: §5.4 · pp. 59–60

5.5 Summary: From Fast Generation to Deployment-Aware World-Action Operation

• Inference turns control-sufficient states into deployment-aware operation through rollout–evaluation–refinement, prompt self-alignment, timestep distillation, and hardware optimization, while real-robot closed-loop validation remains future work.The next stage requires validating imagined-versus-real rollouts, failure prediction, uncertainty calibration, safety filtering, recovery, and policy improvement; efficiency matters because predictions must enter the observation–action–feedback loop before execution.Source: §5.5 · p. 60, §5.4 · p. 60

6.2 Embodied World Model Benchmarks

• Kairos-4B achieves leading results across embodied world-model benchmarks, including first-place performance on WorldModelBench-Robot, DreamGen, and small-scale PAI-Bench-Robot comparisons.It scores 9.30 overall on WorldModelBench-Robot, AVG_PA 0.538 and AVG_Score 0.618 on DreamGen, and Domain Score 88.59 and Overall Score 82.57 on PAI-Bench-Robot.Source: §6.2.1 · p. 62, §6.2.2 · p. 63, Table 7 · p. 63

6.2.4 Human Evaluation

• Human evaluators preferred Kairos-4B over larger baselines, with win rates reaching 60.2% against Cosmos-Predict2.5-14B, 86.7% against Wan2.2-5B, and 88.8% against Wan2.2-5B on DreamGen.The evaluation used anonymized outputs ranked by 10 volunteers across PAI-Bench, WorldModelBench, and DreamGen.Source: §6.2.4 · pp. 65–66

6.2.5 Ablation Studies: Human-Centric Scaling and VLM Selection

• Human-centric data scaling raises WorldModelBench-Robot instruction following from 2.10 to 2.33 and total score from 9.08 to 9.25, while upgrading the VLM raises them further to 2.36 and 9.30.The improvements are attributed to richer behavioral priors and stronger multimodal alignment between language instructions and visual context.Source: §6.2.5 · pp. 66–67, Table 10 · p. 68, Table 11 · p. 68

6.3.1 RoboTwin 2.0

• On RoboTwin 2.0, Kairos leads the Clean setting at 96.9 and achieves the highest average success rate at 96.1, while MotuBrain leads Randomized at 96.1.The results support jointly modeling environment dynamics and action evolution for long-horizon bimanual manipulation planning and execution.Source: §6.3.1 · p. 68, Table 12 · p. 68

6.3.2 LIBERO-Plus

• Kairos achieves state-of-the-art performance on LIBERO-Plus after fine-tuning, indicating robustness to scene variation, visual distribution shifts, compositional reasoning, and long-horizon policy demands.The benchmark emphasizes scene-level generalization and policy stability under perturbed evaluation settings.Source: §6.3.2 · pp. 68–69, Table 13 · p. 69

6.3.3 Ablation Studies: Human-Centric Pretraining and Joint World-Action Training

• Jointly denoising future video and action tokens improves action-prediction performance from 89.0 to 90.8, while human-centric pretraining and generation-prediction joint training provide additional gains.Human-centric data improves LIBERO-Plus performance, and omitting generation supervision consistently degrades action prediction.Source: §6.3.3 · pp. 69–70, Table 14 · p. 70, Table 15 · p. 70

6.4.1 PAI-Bench

• On general PAI-Bench, Kairos-4B reaches an Overall Score of 80.8, matching the 2B and 14B Cosmos-Predict2.5 models at 81.0 while achieving the best i2v-background score.Qualitative samples show prompt adherence, accurate first-frame conditioning, and physical consistency across autonomous-driving, industrial, indoor-human, and robotic scenarios.Source: §6.4.1 · pp. 70–71, Table 16 · p. 70, Figure 20 · p. 71

6.4.2 WorldModelBench

• On the full WorldModelBench, Kairos remains close to Cosmos-Predict2.5-14B at 4B parameters and achieves the best scores on several physics and common-sense metrics in TI2V.Qualitative results show instruction-aligned dynamic processes and plausible physical behaviors in both TI2V and T2V settings.Source: §6.4.2 · p. 72, Table 17 · p. 72, Figure 21 · p. 72

6.4.3 VideoPhy

• 45.55 average score on VideoPhy is highest among the compared models, with 4B-parameter Kairos outperforming Cosmos-Predict2.5-14B.The benchmark evaluates semantic adherence and physical commonsense across 344 generated videos, and qualitative cases show coherent fluid, fracture, rainfall, and waterfall dynamics.Source: Table 18 · p. 73, §6.4.3 · pp. 73–74

6.5 Long-Horizon Generation

• 79.9 overall score on PAI-Bench-15s surpasses Cosmos-Predict2.5-2B/14B at 77.2/76.2 and Wan2.2-5B at 77.8, while retaining strong domain and consistency metrics.Kairos records autonomous driving 66.7, robot 80.4, industry 86.8, human 83.2, i2v-bg 97.1, and i2v-s 95.6; absolute preservation beyond 15 seconds is not evaluated.Source: Table 19 · p. 76, §6.5 · p. 76

6.6 Efficiency–Capability Trade-off as Deployment Proxy

• Under 720P, 5-second TI2V, Kairos-4B uses 23.5 GB memory, requires 2.3 PFlops, and runs in 43 seconds on 1 GPU or 9 seconds on 4 GPUs.It is reported as more efficient than larger and similarly scaled baselines, with lower latency across 480P and 720P and approximately linear scaling with video duration.Source: §6.6 · p. 76

6.7 Regret-Relevant Cases

• In two representative manipulation cases, Kairos better preserves task-directed future rollouts than Cosmos-Predict2.5-2B, maintaining clothes-to-washer progress and container-to-bag insertion intent.These cases suggest preservation of goal-conditioned control information relevant to physical cost, rather than direct validation of regret reduction.Source: §6.7 · pp. 76–77, Figure 25 · p. 77

6.8 Current Evaluation Limitations and Future Closed-Loop Validation

• Current evaluations do not directly test imagined–real rollout correlation, counterfactual actions, failure prediction, recovery learning, policy improvement, or uncertainty calibration in real-robot closed loops.Future protocols should measure these capabilities, including unsafe-event reduction, recovery robustness, imagined-experience gains, and prediction-risk calibration before claiming validated regret reduction.Source: §6.8 · p. 78

6.9 Summary

• Across the evaluated benchmarks, Kairos demonstrates physical plausibility, manipulation performance, long-horizon state maintenance, and favorable deployment efficiency, while remaining proxy evidence rather than direct measurement of regret reduction.The reported evaluation supports prerequisites for control-sufficient world-action modeling, but full closed-loop validation remains outstanding.Source: §6.9 · pp. 78–79

7.1 Video Generation Models

• Video generation research has progressed from 3D-convolutional and spatial-temporal diffusion models to scalable DiT-based systems, while autoregressive models remain competitive for long-term consistency and streaming.Representative advances include LTX-Video [17], HunyuanVideo [16], Wan [82], VideoPoet [8], and MAGI-1 [9].Source: §7.1 · pp. 79–80

7.2 World Models

• World models evolved from latent “dream” environments for reinforcement learning toward action-conditioned video simulators and general-purpose physical-world foundations.World Models [20], PlaNet, and Dreamer [22] [23] [24] established latent dynamics, while UniSim, GAIA-1 [161], Genie, and Cosmos extended interactive simulation.Source: §7.2 · pp. 80–81

7.3 World Action Models

• World-action models extend reactive VLAs by jointly modeling future states and actions, with cascaded and joint designs trading planning interpretability, fidelity, latency, and deployment efficiency.Cascaded approaches use explicit visual or geometric futures or implicit latent plans, whereas joint autoregressive and diffusion families address modality coupling and inference cost through masking, latent modeling, lightweight heads, asynchronous sampling, and speculative decoding.Source: §7.3 · pp. 81–82

7.4 Efficient Attention Mechanisms

• Efficient attention research replaces quadratic self-attention with kernelized, sparse, recurrent, state-space, or gated linear mechanisms that support scalable long-sequence modeling.Linear Transformers, Performer [184], RetNet [185], Mamba [67] [74], Gated Delta Networks, and GLA [69, 187] target reduced complexity, with GLA supporting O(N) complexity [68] [71].Source: §7.4 · p. 82

8 Conclusion and Future Works

• The Cross-Embodiment Data Curriculum organizes passive observation, intentional human behavior, and robot interaction by intervention strength to support control-relevant state learning.The Native Understanding–Generation–Prediction Architecture shares a world-action state across multimodal understanding, physically plausible generation, and joint future visual-state/action prediction.Source: §8 · p. 83
• Hybrid Linear Temporal Attention assigns local, mid-range, and global pathways to contact dynamics, interaction dependencies, and persistent task memory, with theory supporting the design but not guaranteeing real-world correctness.Deployment co-design adds distillation, optimization, parallelism, quantization, caching, and action-only pathways, improving deployment readiness without proving real-time closed-loop control.Source: §8 · p. 83
• Kairos advances control-sufficient world modeling by integrating cross-embodiment pretraining, unified world-action states, hybrid temporal memory, deployment-aware inference, and scalable data engineering, while leaving closed-loop robot validation for future work.Its evaluations show strong performance across embodied and general world-model, world-action, long-horizon generation, and inference-efficiency benchmarks, but do not establish real-world closed-loop regret minimization.Source: §8 · pp. 82–84

8.1.1 Direct Evaluation of Control-Sufficient States

• Future evaluation should test whether Z_t is control-sufficient by probing task progress, failure, action consequences, safety risk, and imagined–real discrepancy rather than relying on visual plausibility alone.Suggested protocols combine representation probes, rollout prediction, and downstream policy comparisons using metrics such as progress accuracy, failure precision/recall, risk calibration, action error, and discrepancy calibration.Source: §8.1.1 · pp. 84–85

8.1.2 Counterfactual Action Validation

• Counterfactual validation should execute alternative actions from matched initial states and test whether predicted outcome branches, risks, and action rankings match real outcomes.The objective is action–outcome causation: predicting control-relevant consequences such as object motion, contact stability, task progress, failure, and risk rather than perfect pixels.Source: §8.1.2 · pp. 85–86

8.1.3 Interventional Generalization Across Observation, Human Intervention, and Robot Intervention

• Interventional generalization studies should compare passive, human, and robot data contributions across environments, tasks, and embodiments under intervention-induced distribution shift.Metrics should include action success and prediction, task progress, failure prediction, data efficiency, and perturbation robustness, testing whether staged CEDC outperforms individual sources or flat mixtures.Source: §8.1.3 · pp. 86–87

8.1.4 Multi-Timescale Memory Beyond 15-Second Generation

• Memory evaluation should extend beyond 15-second video consistency to seconds-, minutes-, hours-, and day-scale task-state maintenance, measuring whether memory improves action selection, reduces repeated mistakes, and supports recovery.Ablations should associate sliding-window, dilated, and gated-linear pathways with local, mid-range, and persistent temporal dependencies.Source: §8.1.4 · p. 87

8.1.5 Control-Information-Density Data Engine

• A future control-information-density engine should prioritize near-boundary failures and recoveries, near-boundary successes, and contact-rich data, scoring segments by uncertainty reduction about outcomes, failures, recovery, contact, and safety.Ordinary trajectories remain useful, while extreme nondiagnostic failures should not dominate when learning action consequences and recovery strategies.Source: §8.1.5 · p. 88
• Future data should align human, robot, and simulation experiences through shared event labels, such as a common taxonomy for contact failures.This would enable Kairos to learn more uniformly from heterogeneous physical experience.Source: §8.1.5 · p. 88
• The data engine should improve failure prediction, safety filtering, and recovery by exposing action failures, unsafe or uncertain states, and post-error behavior.It should eventually close the data loop by using deployment feedback to select valuable data, prioritize annotation, update the model, and improve policy evaluation.Source: §8.1.5 · p. 88

8.2 Outlook

• Kairos should be evaluated by task-progress prediction, action-consequence discrimination, cross-intervention generalization, long-horizon state maintenance, and data-engine effects on failure handling.Advancing toward directly validated regret-aware Physical AI requires careful experiments, calibrated evaluation, and real-robot validation.Source: §8.2 · p. 89

A Contributors

• Dacheng Tao and Xiaogang Wang are listed as advisors.Source: A Contributors · p. 104
• Fei Wang, Shan You, and Qiming Zhang are listed as project leads.Source: A Contributors · p. 104
• Tao Huang and Zuoyi Fu are listed as core contributors.Source: A Contributors · p. 104
• Zhisheng Zheng, Yunlong Xi, Feng Lv, Xiaoming Wu, Zeyu Liu, and Cong Wan are listed as contributors.Source: A Contributors · p. 104
• Pu Li, Ruiqing Yang, Xiaoou Li, Wei Wang, Kangkang Zhu, Yuwei Zhang, Shi Fu, Zheng Zhang, Xiaoning Wu, and Xuzeng Fan are also listed as contributors.Source: A Contributors · p. 104

Necessity Result

• Finite-window predictors incur strictly positive irreducible excess risk whenever long-horizon Bayes-optimal prediction depends on history outside the recent window, establishing the necessity of persistent internal state.This applies to targets involving object permanence, delayed physical effects, and multi-stage task variables.Source: B Theoretical Analysis · p. 105

Sufficiency Result

• When the Bayes predictor factorizes across short-range, mid-range, and contractive global-memory branches, hybrid temporal memory achieves an explicit excess-risk bound.The bound is controlled by branch-wise approximation errors and a geometrically discounted global-memory perturbation term.Source: B Theoretical Analysis · p. 105

Architectural Implications

• Together, the theorems justify the architecture: local pathways capture localized motion, while persistent global memory propagates supra-window context with controlled drift.The necessity theorem explains degradation from purely local mechanisms, and the sufficiency theorem mathematically supports the proposed hybrid design for long-horizon capabilities.Source: B Theoretical Analysis · p. 105

Problem Setup

• The interaction stream is modeled as a discrete-time partially observed controlled process, with observations O_t and actions A_t influencing future evolution.Targets may include future latent-frame coordinates, object permanence, delayed physical effects, or task progress.Source: §B.1 · p. 105
• The complete history H_t contains observations through time t and actions through time t−1, while a recent window captures only the latest w steps.The setup explicitly distinguishes full-history information from bounded local context through their generated σ-fields.Source: §B.1 · p. 106

State Definitions

• A persistent internal state recursively propagates compressed historical information instead of recomputing predictions from scratch from a bounded local context.The state is defined through measurable update maps Φ_t, with an initial state M_1.Source: §B.1 · p. 106
• An exact sufficient state recursively retains precisely the historical information relevant to predicting the target Y through measurable updates and a decoder.Its definition requires state recursion, transition maps, and a measurable decoder mapping the state to the prediction.Source: §B.1 · pp. 106–107

Theoretical Scope

• The necessity theorem depends only on the process-history filtration and assumes neither a particular architecture nor the existence of an exact sufficient state.The sufficiency theorem instead factorizes the state into shared predictive, short-range, mid-range, and global recurrent components corresponding to local, dilated, and global temporal pathways.Source: §B.1 · p. 107
• For any sub-σ-field, conditional expectation is the unique L2-optimal predictor under squared risk.The result follows from the orthogonal risk decomposition, with equality only when the competing predictor equals the conditional expectation almost surely.Source: §B.1 · p. 107

• Long-horizon futures can differ despite identical recent observations and actions because earlier occlusions, delayed effects, or task stages remain unobserved.Thus, reliance on recent windows is fundamentally insufficient, not merely harder to train.Source: §B.2 · p. 107
• Theorem 3 establishes that whenever the Bayes predictor is not recoverable from the latest w steps, every window-restricted predictor incurs strictly positive irreducible excess risk.The theorem characterizes the full-history and recent-window risks and gives an exact excess-risk identity.Source: §B.2 · p. 108
• Persistent world state is necessary whenever future consistency depends on information beyond the visible short-range past.The information-theoretic interpretation makes this necessity quantitative through the measurable excess-risk gap.Source: §B.2 · p. 110
• For recursive world models, preserving supra-window information naturally requires a propagated internal memory pathway rather than only local attention.The result concerns required information preservation, not a specific neural implementation.Source: §B.2 · p. 110
• When the historical cause of a future evaluation target is unrecoverable from the recent window, purely local models inevitably incur a nonzero prediction gap regardless of optimization quality.The target may be a future latent coordinate, object-identity consistency, collision outcome, or task-progress indicator.Source: §B.2 · p. 110

• Hybrid multi-scale temporal memory is approximately sufficient under a matching Bayes-predictor decomposition and stable global recurrent-memory dynamics.The decomposition formalizes complementary roles for a shared predictive representation, short-range local branch, mid-range branch, and global memory branch.Source: §B.3 · p. 111
• The abstract decomposition maps to SWA and DSWA short- and mid-range pathways, GLA global gated memory, and a shared substrate reused across tasks.The shared predictive substrate supports understanding, generation, and prediction.Source: §B.3 · p. 111
• The global-memory branch uses a gated delta update with decay gate α ∈(0, 1) and writing strength β ∈(0, 1).For state S, value v, and key k, the update is F(S; α, β, v, k) := αS + β(v − Sk)k⊤.Source: §B.3 · p. 111

• Definition 7 introduces non-global branch approximation errors at time t for the hybrid predictor.These errors characterize approximation quality outside the global branch.Source: Then the exact and learned global… · p. 112
• The shared predictive substrate error measures approximation quality, while the mid-range branch error is tracked separately.The passages identify distinct error measures for the shared substrate and mid-range branch.Source: Then the exact and learned global… · p. 112
• Definition 8 specifies one-step gate approximation errors.Source: Then the exact and learned global… · p. 112

Lemma 3

• The gated delta update is contractive when ρ = α + β∥k∥2 < 1, ensuring discrepancies between memory states decay geometrically.The contraction is measured in Frobenius norm and applies uniformly to any pair of memory states.Source: Also define the initial global-memory… · pp. 113–114

Theorem 4

• Theorem 4 bounds global-memory error and long-horizon excess risk under Bayes-factorization, Lipschitz-decoder, and contractivity assumptions.The asymptotic degradation combines non-global approximation error with a discounted contribution from global-memory perturbations.Source: Also define the initial global-memory… · pp. 115–116

Corollary 4

• Exact hybrid-state recovery together with an initially aligned global-memory state yields zero memory error and exact sufficiency at every time step.The realizable conditions require εU_t = 0 for all t and e0 = 0.Source: Also define the initial global-memory… · p. 118

Interpretation

• Long-horizon prediction error is controlled by non-global branch approximation and accumulated global-memory perturbations, with contraction preventing uncontrolled drift.The global contribution is discounted by the contraction factor, while branchwise approximation quality determines the remaining error.Source: Also define the initial global-memory… · p. 118

Architectural relevance

• The theorem provides architectural sufficiency, explaining how SWA, DSWA, and GLA branch quality supports long-horizon world modeling without guaranteeing any specific checkpoint’s performance.Together with the necessity result, it forms a template in which persistent memory is necessary when recent windows are insufficient and approximately sufficient when the global branch is contractive.Source: Also define the initial global-memory… · p. 119