Source-linked AI summary
Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots
Ling Xu, Chuyu Han, Borui Li, Hao Wu, Shiqi Jiang, Ting Cao, Chuanyou Li, Sheng Zhong, Shuai Wang
TL;DR
Embodied AI deployment lacks a unified runtime for heterogeneous, closed-loop systems. Embodied.cpp provides a portable, layered C++ runtime, achieving 100.0% VLA task success and reducing WAM block memory from 312.2 MiB to 88.1 MiB.
Problem
Embodied AI deployment remains fragmented across model-specific stacks and backend assumptions, while existing runtimes lack unified support for embodied execution on heterogeneous devices.
Method
Embodied.cpp uses a portable five-layer C++ architecture with modular multi-rate execution, latency-first fused inference, and pluggable embodied interfaces.
Results
100.0% success rate was achieved for HY-VLA, while the LingBot-VA block benchmark reduced resident weight memory from 312.2 MiB to 88.1 MiB.
Takeaways & Limitations
The runtime supports different VLA architectures in one C++ path and provides initial evidence of substantial WAM block memory savings with limited output drift.
Takeaways & Limitations
Full LingBot-VA closed-loop results are not included because the complete model was not stable on the constrained local edge setup.
Abstract
from arXiv · showhide
Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied$.$cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied$.$cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied$.$cpp on two VLA models, HY-VLA and pi0.5, and on a preliminary WAM benchmark using a LingBot-VA Transformer block. The VLA deployments achieve successful closed-loop execution with 100.0% and 91.0% task success rates, respectively. The WAM benchmark reduces block memory from 312.2 MiB to 88.1 MiB. These results show that Embodied$.$cpp improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.
1. Introduction
Embodied AI deployment is fragmented because conventional runtimes do not match closed-loop, multi-rate robot control. Embodied.cpp addresses this gap with a portable, layered C++ runtime spanning VLA and WAM architectures.
- Runtime requirements: Embodied deployment requires multi-rate execution, latency-first closed-loop control, and extensible interfaces beyond fixed token I/O.These requirements arise because perception, backbones, predictive branches, and action heads may execute at different rates inside one control loop.
- Runtime architecture: Embodied.cpp organizes inference into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters.The architecture keeps a reusable execution core while allowing model-specific heads, interfaces, and backends to remain pluggable.
- Deployment design: Embodied.cpp provides modular multi-rate execution, latency-first fused inference, and extensible operator and embodied I/O support across heterogeneous edge devices.These capabilities replace one-off Python stacks, backend-specific paths, sensor wrappers, and platform-specific control glue.
- Architectural analysis: A shared execution path covers representative VLA and WAM models, with major divergences confined to pluggable heads and predictive modules.This model-side analysis motivates infrastructure that is uniform at the deployment boundary but extensible internally.
2. Related Work and Motivation
Embodied models now include structurally diverse VLA and WAM families, while existing runtimes remain poorly aligned with their deployment needs. The resulting gap concerns model coverage, modular execution, edge support, heterogeneous hardware, and direct robot or simulator integration.
- Model families: VLA models primarily map perception to action, whereas WAMs explicitly incorporate future prediction into online control.This distinction changes whether a runtime executes an action policy alone or jointly manages predictive state and action generation.
- VLA models: VLA architectures range from shared-backbone autoregressive action generation to VLM-backed continuous-action heads and hierarchical modular execution.Examples include RT-2 and OpenVLA, Octo and pi0.5, and systems where high-level planners produce subgoals for low-level controllers.
- WAM models: WAMs differ in how world modeling couples to action generation, including explicit predict-then-act, unified autoregressive, shared-backbone, and latent-space designs.These structures expose predictive computation and intermediate futures as runtime concerns rather than hidden model details.
- Runtime gap: Existing inference runtimes emphasize general execution or serving, but embodied deployment additionally requires closed-loop control, persistent state, batch-1 latency, and direct robot or simulator integration.Simulation platforms support training and benchmarking, while vla.cpp advances portable VLA execution but remains VLA-centric.
- Deployment gap: Existing systems remain either generic runtimes without first-class embodied support or embodied runtimes specialized to one model family.The unresolved target is joint VLA and WAM support with modular optimization and deployment across edge devices, heterogeneous hardware, robots, and simulators.
3. Project Overview
Embodied.cpp reframes deployment around closed-loop control rather than request-response serving. Its architecture separates shared runtime infrastructure from pluggable model components and supports heterogeneous embodied systems.
- Runtime contract: Embodied inference must support closed-loop control, heterogeneous inputs and outputs, and deployment on robot-side hardware.These requirements differ from conventional token-serving runtimes optimized for synchronous requests and throughput.
- Design principles: Multi-rate execution lets perception, predictive, and action modules refresh at different frequencies within one control loop.Explicit execution units, shared state or feature pools, and configurable refresh policies avoid forcing a single synchronous path.
- Design principles: Latency-first fused execution targets stable batch-1 control performance across heterogeneous devices.The design includes graph replay, buffer reuse, operator fusion, backend-specific dispatch, and careful host-device data movement.
- Design principles: Extensible operator and I/O support provides typed embodied interfaces, pluggable heads, and deployment adapters for evolving model paradigms.New embodied models can be implemented without rebuilding the surrounding runtime.
- Runtime architecture: The runtime uses one shared execution zone for VLA models, WAMs, and future variants, with adapters connecting sensors, datasets, simulators, and robots.Its five-layer organization includes input adapters, sequence builders, backbone execution, head plugins, and deployment adapters.
4. Evaluation
The evaluation demonstrates that Embodied.cpp runs two VLA deployments while preserving task behavior and tests WAM portability through a quantized Transformer block. The WAM result is preliminary because only one block is benchmarked on a constrained edge device.
- VLA evaluation: 100.0% success rate is achieved by HY-VLA on the evaluated RoboTwin place_empty_cup task.HY-VLA uses a larger Hunyuan-VL backbone, three-view inputs, and a video-history/MEM vision path, increasing latency relative to pi0.5.
- VLA evaluation: Both HY-VLA and pi0.5 run correctly through the C++ runtime while preserving task behavior.Latency and memory depend on backbone size, visual input complexity, and action chunking.
- WAM evaluation: MAE below 3.3 × 10−2 and cosine similarity above 9.997 × 10−1 indicate limited output drift for the quantized block.The comparison uses 100 random input samples after measuring the corresponding block implementations.
5. Conclusion
Embodied.cpp treats the shared execution path across embodied models as reusable infrastructure while isolating task-specific differences in plugins. The current evidence quantitatively validates two VLA deployments and positions WAM support through architectural analysis and a preliminary benchmark.
- Conclusion: Embodied.cpp separates a stable runtime core from pluggable task-specific components as embodied model families diversify.Its five-layer architecture keeps interaction patterns, I/O semantics, objectives, and deployment boundaries explicit.
- Conclusion: Two VLA deployments provide quantitative validation of the C++ inference path, while WAM support is currently supported by architectural analysis.The conclusion distinguishes current VLA validation from the status of WAM evaluation.