PaperlayerView exact revision on arXiv

Source-linked AI summary

Foundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation

Zizhe Wang, Jia Deng, Xin Lu, Zehao Jia, Yi Huang, Yao Zhu, Jianing Li

arXiv:2610.23068v1cs.RO

TL;DR

Space robots must act under unfamiliar dynamics, difficult sensing, and limited human intervention, while evidence for transferring terrestrial pretraining benefits to space conditions remains insufficient. This review traces model contributions through physical execution and proposes iterative validation from simulation through ground tests to space; its conclusion is that specific pretraining benefits are supported on the ground, but need testing under space dynamics and scarce interaction data.

  • Problem

    Space robots operate with unfamiliar dynamics, difficult sensing, and limited opportunities for human intervention, while representative floating-base contacts and recovery trajectories are difficult to acquire.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    The review compares reused models, adaptation, physical execution, and validation, and proposes an iterative sim-to-ground-to-space pathway with a capture protocol separating generalization from bounded adaptation.

    Source: §8.2 · p. 15, §8.5 · p. 18

  • Results

    Terrestrial comparisons support specific pretraining benefits for Cosmos Policy and OpenVLA, but those benefits need testing under space dynamics and scarce interaction data.

    Source: §9.1 · p. 19

  • Takeaways & Limitations

    Event sensing and spiking computation offer design options, while physical tests, telemetry diagnosis, and closed-loop revalidation connect model reuse to complete-task performance and resource needs.

    Source: §8.2 · p. 15, §9.3 · pp. 19–20

    Runtime-assurance guarantees depend on adopted dynamics, bounded disturbances, actuator limits, and a verified backup set; satellite capture additionally needs corresponding contact and momentum models and validated recovery.

    Source: §6.5 · p. 12

Abstract

from arXiv · show

Space robots must interpret tasks and act despite unfamiliar dynamics, difficult sensing conditions, and limited opportunities for human intervention. Foundation models offer reusable perceptual and behavioral knowledge, but their value depends on how that knowledge reaches physical execution. This critical review follows space-specific requirements through perception, prediction, planning, and action to deployment and validation, emphasizing orbital manipulation and proximity operations. Terrestrial comparisons establish benefits of particular pretraining stages and model interfaces; their transfer to space dynamics and constrained onboard operation remains insufficiently tested. Event cameras offer timely motion observations across strong illumination contrasts, while spiking neural networks provide possible routes to sparse local computation. We examine how sensor degradation, state estimation and computing choices affect feedback and recovery. Higher action throughput need not yield fresher feedback; lower processor power can also coexist with higher energy per inference. We organize validation as an iterative sim-to-ground-to-space pathway, connecting physical tests with telemetry-based diagnosis and closed-loop revalidation. A satellite-capture example links observations to approach, contact and recovery, while a corresponding protocol separates generalization from adaptation with limited target-domain interaction. These comparisons identify what models already contribute and the physical evidence needed to support autonomous space operation.

1 Introduction and scope

The review asks whether reusable foundation-model knowledge remains useful when space robotics changes the dynamics, sensing, and computation available to physical execution. It focuses on orbital manipulation and proximity operations, comparing transferable terrestrial methods with space-specific requirements and validation needs.

1 Introduction and scope

  • Space robots face unfamiliar dynamics, difficult sensing, and limited human intervention; satellite capture couples tracking, approach, contact, and stabilization.The review asks whether reusable foundation-model knowledge remains useful under these conditions.Source: §1 · p. 1

1.1 Definitions and scope

  • A foundation model is defined by broad pretraining and downstream reuse through a shared model or backbone, not size or architecture alone.The review assesses transfer to held-out tasks separately.Source: §1.1 · p. 1
  • The review emphasizes orbital manipulation and proximity operations, using cabin and planetary studies to compare shared problems and physical requirements.It distinguishes stabilized spacecraft, free-floating manipulators, pressurized cabins, and lunar rovers.Source: §1.1 · p. 2

1.2 Position relative to existing surveys

  • The review connects reusable components and physical requirements through comparisons of adaptation, execution, and validation, with event cameras and SNNs receiving particular attention.It proposes a sim-to-ground-to-space pathway and a capture protocol separating held-out generalization from bounded adaptation.Source: §1.2 · p. 2

1.3 Literature selection

  • Purposive selection limits completeness, and task differences preclude pooled performance rankings.The review uses targeted retrieval and source checks, retaining studies for distinct operational interfaces, physical mechanisms, or validation settings.Source: §1.3 · p. 2

2 From space operating conditions to technical requirements

Space operating conditions alter action consequences, observation quality, feedback timing, and recovery requirements. The section traces those changes from dynamics and hardware constraints to limited interaction data and a satellite-capture example.

2.1 Dynamics and contact

  • A free-floating arm exchanges momentum with its base, so the same joint motion can produce different end-effector motion than with a fixed base.Capture changes mass distribution and motion, making state and dynamics requirements mission-stage dependent [17] [18].Source: §2.1 · p. 3
  • Orbital manipulation state includes joint configuration, base attitude and velocity, target-relative motion, and contact mode; reduced gravity also changes soil traction and sinkage [19] [20].Matched wheel-load parabolic-flight experiments found reduced traction and increased sinkage relative to ground tests.Source: §2.1 · p. 3

2.2 Perception and hardware constraints

  • Illumination, thermal change, and radiation can degrade measurements, while vacuum thermal limits and radiation effects constrain sustained onboard computation [22] [25] [26] [27] [28] [29].Calibration and masks address characterized errors, but cannot restore unique measurements lost to saturation or occlusion; alternative viewpoints or modalities may be needed.Source: §2.2 · pp. 3–4

2.3 Communication and limited interaction data

  • Hardware access and failure consequences restrict interaction data, leaving representative floating-base contacts and recovery trajectories difficult to acquire from terrestrial pretraining [3] [31].Studies should identify physical variations absent from pretraining and the additional interaction needed to handle them.Source: §2.3 · p. 4

2.4 Connecting requirements to a task

  • In satellite capture, approach depends on target-relative motion and observation age, contact adds force evidence, and stabilization requires the coupled base–target state [18].Degraded or stale observations may require a slower approach or reacquisition; evaluation should distinguish adaptation across targets from contact loads, base disturbance, and stabilization.Source: §2.4 · p. 4

3 An embodied closed loop for model integration

The section treats model integration as a closed loop whose interfaces, timing, feedback, and supervision determine how pretrained components contribute to physical execution. It distinguishes model updates from recovery using existing capabilities.

3.1 Functional interfaces and feedback

  • Embodied autonomy links observation, state and memory, prediction and planning, action generation, and monitored execution, using unified VLA or modular model-controller designs [11].Inputs, outputs, update rates, and feedback paths expose each model's contribution.Source: §3.1 · p. 5

3.2 How models are coupled to execution

  • Models can select skills, generate programs, issue motion commands, learn observation-to-action policies, or predict outcomes for training or online candidate evaluation [34] [36] [37] [38] [39] [40] [41] [42].Selecting a whole skill delegates its feedback loop; successive motion commands make model latency part of that loop.Source: §3.2 · pp. 5–6

3.3 Timing, supervision, and model updates

  • Prediction horizon, execution interval, inference delay, and observation age describe different aspects of responsiveness; local supervision can interrupt actions and updates need regression tests.Evaluation of changed weights or skills is separate from recovery using existing capabilities.Source: §3.3 · p. 6

4 Multimodal perception and state representation

Perception must connect semantic and sensor representations to calibrated, timed physical state that action selection can use. The section compares visual, event-based, photon-counting, tactile, and proprioceptive inputs and notes where space-robot evidence remains limited.

4 Multimodal perception and state representation

  • Planning and control require target-relative motion, spacecraft-base motion, joint states, and contact, collected as a task state that pretrained features must connect to physical quantities.The state representation includes relative state, joint configuration and velocity, and contact.Source: §4 · p. 6

4.1 Connecting semantic features to physical state

  • Semantic features can support metric motion through geometric estimation, while FoundationPose needs a CAD model or reference views and novel-object transfer depends on geometric information [45–47].A specialist orbital-state pipeline tracks relative state, attitude, and angular velocity, propagating state between measurements and adapting process-noise covariance [18].Source: §4.1 · p. 6

4.3 Force, touch, and proprioceptive encoding

  • Tactile representation compatibility and policy use are distinct problems: AnyTouch aligns sensor representations, while FuSe connects heterogeneous observations to task semantics [26] [59].For floating robots, wrench interpretation also depends on sensor frame, calibration, joint motion, and base response.Source: §4.3 · p. 8

5 Prediction and task planning

Prediction and planning connect action consequences to feasible motion and decisions about which capabilities to use. The reviewed approaches show benefits from pretrained initialization and feedback-guided planning, while their physical assumptions and tested settings bound the evidence.

5.1 World models: prediction targets and control roles

  • World models can generate training experience, support action learning, or evaluate candidate actions, but learned predictions can inherit transition or action-inference errors.UniSim supplies action-conditioned future observations for simulated policy learning, while DreamGen recovers pseudo-actions from generated robot videos.Source: §5.1 · p. 8
  • V-JEPA 2-AC plans toward goal representations by executing the first receding-horizon action and replanning, but its demonstrated setup uses scheduled subgoals and takes 16 seconds per action.The reported experiments use Franka pick-and-place; camera sensitivity and externally supplied subgoals also shape demonstrated autonomy.Source: §5.1 · p. 8
  • With equal numbers of gradient updates, Cosmos Policy’s pretrained initialization outperforms random initialization (Table 6).Its planning mode uses predicted state and value to assess candidates; direct-policy evaluation discards those predictions.Source: §5.1 · p. 8

5.2 Grounding language in available capabilities

  • Language-grounded systems connect instructions to available skills, tools, execution feedback, or spatial objectives, with orbital capture requiring motion prediction and whole-body constraints beyond VoxPoser’s mostly static-scene assumptions.VoxPoser composes spatial affordance and avoidance maps for a motion planner.Source: §5.2 · p. 9
  • SpaceMind’s skill evolution revises task-specific recovery instructions, but shows no cross-task transfer or benefit where baselines already succeed.Its evaluation includes 174 UE5 runs and 18 planar ground-robot runs, without orbital dynamics; one of six inspection runs collided.Source: §5.2 · p. 9

5.3 From mission specifications to feasible motion

  • OrbitTAMP parses instructions into a mission configuration, completes behavior and waypoint plans, then optimizes trajectories under constraints such as flight time, ∆v budget, and keep-out zones.The physical problem enters through the planning representation and optimizer.Source: §5.3 · p. 9
  • Two verifier-guided revisions raise Qwen3.5-9B exact recovery from 75% to 88% on 100 development commands, but structurally valid semantic errors can escape verification.Five revisions add no benefit, and the version-1 trajectories are nominal and open-loop.Source: §5.3 · p. 9

5.4 Autonomous decisions and requests for assistance

  • Supervision can allocate planning and execution between humans and robots, while KnowNo’s conformal guarantee covers semantic choices under its calibrated scenario distribution and help mechanism.Physical uncertainty and execution deviations need additional responses, including renewed observation or controller-level recovery, beyond that semantic guarantee.Source: §5.4 · pp. 9–10

6 Action generation and physical execution

Physical execution depends on matching action interfaces and feedback timing to the robot’s dynamics, sensing, and contact constraints. Terrestrial comparisons offer evidence about policies and interfaces, while orbital capture demands coupled-state estimation, constrained recovery, and validation of the execution layer.

6.1 Direct commands and learned action policies

  • On 50 paired RoboDojo instances, hybrid control completes 24 tasks versus 13 for direct control, with 85.6% of hybrid execution steps from the learned policy.The ranking also depends on policy fit: the task-finetuned π0.5 checkpoint is used in RoboDojo, while a DROID-trained checkpoint is used zero-shot on RoboLab, where Direct performs better.Source: §6.1 · p. 10
  • Raw torque selection completes 0/20 planar-arm episodes, versus 9/20 using skills and external simulation; the comparison changes both action abstraction and predictive support.It does not show that Jev itself learned a dynamics model.Source: §6.1 · p. 10

6.2 Action chunks and feedback timing

  • Cosmos Policy’s direct ALOHA evaluation pauses for 0.95 seconds of blocking inference, then executes 50 steps at 25 Hz for two seconds; shorter chunks help only when new actions arrive in time.Action horizon, execution interval, inference delay, and observation age are distinct responsiveness measures.Source: §6.2 · p. 10
  • Table 3 compares model interfaces by their physical assumptions and feedback responsibilities; its orbital adaptations are proposed extensions, and prediction can support any interface.Real-time chunking overlaps inference with execution and soft-inpaints the remaining chunk, with its reported ablation on Kinetix and robot tests in terrestrial bimanual manipulation.Source: §6.2 · p. 10, Table 3 · p. 11

6.3 Incorporating space dynamics

  • Free-floating arm motion changes spacecraft-base motion and control-resource use, while capture contact can dissipate kinetic energy but cannot remove total momentum from the coupled system.State conditioning, training adaptation, or external control can each incorporate space dynamics; capture additionally needs accurate base-motion and contact-mode representations.Source: §6.3 · p. 11
  • Prior-policy-guided free-floating motion planning varies base mass and target rotation but stops short of post-contact force control, which stabilized capture would still require.SpaceOctopus combines multi-arm policies with a lower-level PD controller in MuJoCo.Source: §6.3 · p. 11

6.4 Contact sensing, action updates, and compliant control

  • ForceVLA conditions action decoding on pose, gripper width, and a six-axis wrench, while Reactive Diffusion Policy updates actions within a chunk using recent tactile or force observations.Reactive Diffusion Policy demonstrates feedback in ground contact tasks but does not impose a force bound.Source: §6.4 · p. 11

6.5 Constrained execution and runtime monitoring

  • Approach clearance must account for pose and tracking error, sensing and computation during closing motion, and braking distance; after contact, grip retention, contact load, and combined motion become relevant.The proposed capture example links image semantics to target selection and geometric/contact estimates to admissible motion; any hold or retreat must remain feasible under current dynamics and contact constraints.Source: §6.5 · p. 12, Figure 5 · p. 12

7 Learning, adaptation, and onboard deployment

Adaptation and deployment choices must account for which space-specific changes are represented, what computation can be sustained, and how physical performance is measured. The reviewed evidence includes promising efficiency and adaptation results, but hardware, task, and validation boundaries prevent broad conclusions about onboard space operation.

7.1 Choosing what to adapt

  • Space-LLaVA adapts visual-language knowledge and SPADES supplies spacecraft events, but neither provides complete servicing trajectories.Open X-Embodiment supplies diverse terrestrial trajectories but does not cover floating-base interaction; generated videos can broaden visual scenarios, while physical consistency and executable actions still require checking.Source: §7.1 · p. 12

7.2 Reducing deployment cost

  • On Jetson AGX Orin, QwenVLA/OpenVLA-OFT peak memory is 7.39/19.20 GB, while separately evaluated LIBERO success is 78.8/95.3%.BitVLA reports 1.4 GB model memory and 96.0% LIBERO success, but its 73 ms per 25-action chunk is measured on A100 and physical manipulation uses remote A800 inference.Source: §7.2 · p. 13
  • OpenVLA-OFT parallel decoding raises A100 throughput from 4.2 to 15.9 generated actions/s, and adding eight-action chunking raises it to 108.8 actions/s while query latency rises from 62.9 to 73.5 ms.These are ablation variants, not the final continuous-action L1 configuration; fully executing each chunk before replanning means more generated actions do not imply faster response to new observations.Source: §7.2 · p. 13

7.3 Spiking computation for local autonomy

  • For event-to-frame input, Akida V1 draws 0.919 W dynamic power at 16 FPS versus 2.519 W at 330 FPS for the floating-point Jetson Orin Nano baseline, yielding 57.4 versus 7.63 mJ/frame.On the full test set, quantization raises mean pose score from 0.0361 to 0.1009 and rejected poses from 1 to 162; the means exclude rejections, and hardware benchmarking uses a distinct 4,000-input set.Source: §7.3 · p. 13
  • Replacing Spike-driven VLA’s ANN encoders with spiking encoders lowers LIBERO success from 96.3% to 92.4% and estimated arithmetic energy from 179.7 to 20.9 mJ per twelve-action chunk.The estimate excludes memory and other hardware overhead, and encoder capacity and pretrained priors also change; SpikingVLA’s internal steps process the same observation, with more steps improving navigation but increasing estimated energy.Source: §7.3 · p. 13

7.4 Continual learning and recovery

  • After rollback, restored parameters, recurrent memory, and state estimates must be reconciled with the robot’s current physical state; evaluation should include retained and regained performance, update cost, and switching behavior.The cited space approach selectively updates components after simulated failures, and its authors report online optimization exceeds space-grade embedded-computing power constraints.Source: §7.4 · pp. 13–14

7.5 Thermal operation and energy measurement

  • Thermal throttling or reduced duty cycles can slow inference and leave controllers acting on older observations, so deployment must sustain the task’s feedback rate within thermal limits.Task-level energy accounting should specify measurement boundaries and configuration, and include failures and idle periods; Nahuku preloaded-input measurements differ from the flight system’s Kapoho Bay configuration.Source: §7.5 · p. 14

7.6 Radiation tolerance and model robustness

  • Applying vision-transformer soft-error studies to control requires matching the fault model to the device and tracing output errors into actions.Compression or adaptation can change numerical sensitivity and fault propagation, so protection needs reassessment; irradiation and closed-loop tests should establish whether corruption is detected before actuation and restart restores coherent control.Source: §7.6 · p. 14

8 Simulation, physical validation, and evaluation

Operational claims require evidence that connects model behavior to the hardware, environment, and task stage. The section proposes iterative validation and a capture protocol that distinguish transfer, adaptation, recovery, and resource costs.

8.1 From benchmark performance to operational claims

  • Event-based lunar-descent simulation reached 50 m, but near-surface oscillations and unrealistic thrust requests remained; sensor timing and image limits still constrain the result.Sensor tests should measure event statistics and motion-estimation error, while task tests should assess effects on approach or recovery; buffering and execution can consume latency advantages.Source: §8.1 · p. 15
  • Benchmarks cover distinct capabilities, while grasping, contact, retention, and stabilization remain separate milestones; SPEED+ adaptation also changes the untouched-domain protocol.The dual-arm study stops short of post-contact force control [20], and interpreting SPEED+ transfer requires tracking data exposure across pretraining, adaptation, and model selection [83].Source: §8.1 · p. 15

8.2 From simulation through ground tests to space

  • The proposed sim-to-ground-to-space pathway uses ground analogues and HIL to test selected dynamics, contact, sensing, and computing, then diagnoses discrepancies before revalidation.Planar and spherical testbeds, gravity offloading, and HIL preserve selected mechanisms but introduce constraints; Figure 6 depicts the pathway, and Table 5 lists proposed calibration quantities.Source: §8.2 · p. 15, Figure 6 · p. 16, Table 5 · p. 16
  • 8/25 physical grasps with full randomization versus 2/25 with reduced randomization tested visual and manipulation transfer under terrestrial gravity, not lunar gravity or free-capture momentum management [117].The episodes came from policies selected from three training seeds, not independent trained deployments.Source: §8.2 · pp. 16–17

8.3 Interpreting flight and hardware evidence

  • Flight and processor tests establish different evidence: LeLaR telemetry exposed delayed actuation and intermittent unresponsiveness, while Falcon Neuro reconstructed Earth features from events [119].LeLaR’s independent monitor did not intervene in the reported experiments; its idle design assumed InnoCube remained power-positive while tumbling.Source: §8.3 · p. 17
  • Loihi 2 reduced measured inference energy relative to the GPU ANN but increased final position and orientation errors; Figure 7 separates tested capabilities rather than providing a readiness score [114].The configurations shown have no reported thermal-vacuum or irradiation tests.Source: §8.3 · p. 17, Figure 7 · p. 18

8.4 Environmental and resource testing

  • Environmental and resource tests should connect device-specific faults, thermal conditions, latency, and energy to embodied task behavior and recovery.Radiation evidence for Loihi does not establish tolerance of the Loihi 2 control configuration [114]; fault tests should report exposure, monitored outputs, and interventions.Source: §8.4 · p. 17

8.5 A reusable evaluation protocol

  • The capture protocol crosses seen and unseen task compositions with seen and unseen physics, then separates fixed-checkpoint generalization from budgeted target-physics adaptation.It calls for held-out evaluation trajectories, matched comparisons of model and controller changes, and separate scoring of approach, contact, retention, stabilization, recovery, latency, and energy.Source: §8.5 · pp. 17–18

9 Synthesis and research priorities

Terrestrial comparisons support specific pretraining benefits, but their transfer to space dynamics and scarce interaction data remains untested. The review identifies comparisons and physical tests needed to evaluate reuse, adaptation, and deployment together.

9.1 What the current results support

  • Video initialization improves Cosmos Policy at matched gradient steps [42], while OpenVLA robot-trajectory pretraining improves over directly adapting Prismatic with vision–language pretraining retained in both [33].These specific pretraining benefits still require testing under space dynamics and scarce interaction data.Source: §9.1 · p. 19
  • Interface and adaptation comparisons isolate different effects, while throughput gains, query latency, capacity, and priors require distinct explanations [17] [67] [64] [87] [90].Joint tests of task reuse, coupled dynamics, sustained computation, and recovery remain a gap for foundation-model systems [36] [84] [114] [119].Source: §9.1 · p. 19

9.2 Research questions that follow from these comparisons

  • The proposed capture tests compare state conditioning, policy updates, and controller changes at matched budgets as inertia or contact varies.Held-out task–physics combinations should extend through contact, retention, and stabilization, retaining failures to locate whether recovery depends on policy, controller, or operator.Source: §9.2 · p. 19
  • Deployment comparisons should include preprocessing, memory, idle periods, recovery, feedback timing, task energy, and sustained thermal performance.Transfer discrepancies should be diagnosed before an update is chosen and tested in the closed loop.Source: §9.2 · p. 19

9.3 Conclusion

  • Foundation models contribute reusable representations, task interfaces, and learned behaviors; sim-to-ground-to-space validation can test these options against complete-task performance, recovery, and resource needs.The conclusion calls for separating generalization from bounded target-domain adaptation.Source: §9.3 · pp. 19–20
Loading 2610.23068v1…