Source-linked AI summary
EVA-Client: A Unified Data Collection, Inference, and Deployment Framework for Embodied Policies on Real Robots
Heqing Yang, Yang Yi, Liyao Wang, Linqing Zhong, Donglin Yang, Ruipu Wu, Zitong Bai, Fengjiao Chen, Manyuan Zhang, Linjiang Huang, Si Liu
TL;DR
Real-robot deployment and evaluation remain fragmented across method- and robot-specific scripts, making execution and assessment difficult to inspect and reproduce. EVA-Client unifies deployment, inference, data collection, and evaluation in a decoupled client, with illustrative deployments showing asynchronous scheduling enables a table-tennis rally that synchronous execution cannot sustain.
Problem
Real-robot deployment and evaluation remain fragmented across method- and robot-specific scripts, limiting inspectability, reproducibility, and systematic comparison.
Method
EVA-Client unifies robot execution, policy inference, debugging, data collection, and evaluation through decoupled backends, workflows, and configurable inference strategies.
Results
Asynchronous scheduling enables the table-tennis rally to proceed, whereas synchronous execution stalls and fails to track the fast ball in this illustrative deployment.
Takeaways & Limitations
EVA-Client turns real-robot deployment, collection, and evaluation into an inspectable, reproducible, and comparable process that feeds recorded rollouts back into external training.
Takeaways & Limitations
The reported task outcomes are illustrative real deployments rather than a controlled study, and the released tooling does not include the collected data.
Abstract
from arXiv · showhide
We present EVA-Client, an open-source framework for deployment, data collection, and evaluation of trained manipulation policies on real robots. Sitting between a policy server and the physical hardware, EVA-Client unifies the real-robot stages of the policy iteration loop within a single codebase. It makes three contributions. First, a component-decoupled architecture in which robot backends, inference strategies, and transport middlewares form an orthogonal grid: adding a robot or a strategy touches only its own layer. Second, inspectable execution through Debug, Collect, and Eval workflows, with modes ranging from open-loop simulation to continuous real-time control. Third, every evaluation run doubles as a data collection, recording full rollouts in training-ready format alongside exhaustive logs and a side-by-side comparison viewer, so each evaluation feeds the next round of training rather than ending as an unrecorded impression. EVA-Client further consolidates major real-time inference strategies, synchronous and asynchronous execution, ACT-style temporal ensembling, Real-Time Chunking, and a naive-async ablation baseline, behind a single configuration surface.
1 Introduction
EVA-Client addresses the underdeveloped real-robot deployment layer for trained manipulation policies by unifying inference, debugging, data collection, deployment, evaluation, and logging in one client. It complements existing training frameworks while remaining model-agnostic and focused on serial-arm manipulation.
- Real-robot deployment requires observation-command interfaces, action scheduling, latency compensation, action-space conversion, and rollout logging beyond querying a policy server.These deployment components can determine whether a trained policy runs safely and smoothly, despite usually not being treated as model contributions.
- Robot-specific cameras, state feedback, middleware, and action spaces prevent code built for one robot from transferring directly to another.The control loop remains tied to robot integration even when policies share checkpoint formats or inference APIs.
- EVA-Client unifies the real-robot stages of the policy lifecycle, spanning hardware integration, data collection, external training, real-time deployment, and evaluation.It sits between signal sources and robot execution as a single codebase for inference, debugging, data collection, and deployment.
- A single client combines data collection, deployment, chunk smoothing, evaluation, and recorded feedback, addressing the fragmented real-robot tooling surrounding shared training frameworks.The framework closes the gap between consolidated training infrastructure and scattered per-policy, per-robot scripts.
- EVA-Client is a model-agnostic client and deployment layer for trained policies, not a policy, benchmark, or dataset, with inverse-kinematics support targeting serial-arm manipulators.It assumes an existing policy server and is positioned as complementary infrastructure to mature training frameworks.
2 Related Work
EVA-Client sits at the intersection of policy models, inference techniques, and embodied infrastructure. Its related work spans language-conditioned and open Vision-Language-Action policies, chunk-based execution methods, and community systems for robot control, kinematics, and visualization.
- Overview: EVA-Client connects three related-work lines: policy models, inference techniques, and embodied infrastructure.These correspond to the policy models it serves, the inference techniques it consolidates, and the embodied infrastructure it builds on.
- Policy techniques: Language-conditioned robotic transformers formulated control as sequence prediction over discretized actions, followed by reproducible open Vision-Language-Action models such as OpenVLA.The passage identifies Brohan et al. (2023a,b) and Kim et al. (2024) as representative works.
- Execution modes: Its execution modes progress from safe, fully observable open-loop simulation to fully real-time hardware execution, while evaluation and data collection reuse the same console and backends.Table 1 describes evaluation as running on top of these modes and data collection as reusing them.
- Inference techniques: Chunk-based policies require real-time scheduling of successive queries and reconciliation of overlapping predictions, with ACT popularizing exponentially weighted temporal ensembling.Diffusion policies likewise emit short action sequences per query.
- Embodied infrastructure: EVA-Client reuses mature components: ROS and ROS2 for transport, PyRoki for kinematics, and Viser for interactive visualization.The surrounding embodied-systems ecosystem provides observation and command movement, inverse kinematics, and console rendering.
3 Framework Overview
EVA-Client is a thin client between policy or teleoperation signal sources and robot execution, organizing deployment around modular layers and a closed control loop. Its declarative configurations and inspectable workflows make runs reproducible and observable without code changes.
- Architecture: EVA-Client mediates between trained policy servers or human teleoperation and physical robot execution, forwarding observations and instructions and returning actions or action chunks.Model-based sources receive observations and language instructions, while teleoperation supplies operator actions through the same execution path.
- Design principles: EVA-Client makes deployments reproducible and observable by recording what was observed, inferred, and executed and allowing simulation preview, chunk stepping, trajectory visualization, and intervention.These inspections occur at each stage without rewriting code.
- Architecture: The client separates transport, robot description, policy, and workflow concerns behind narrow interfaces, so workflows share the same internal layers.The transport layer supports real robots, datasets, and socket-based fake nodes, while robot descriptions specify actuators, cameras, observations, mappings, and optional kinematics.
- Configuration: A declarative configuration fixes the robot, transport, policy endpoint, action-space modes, loop rates, prompts, and inference strategy, while command-line flags can override individual fields.The evaluation subsystem records the configuration so a deployment can be reconstructed later.
4 Backends and Hardware
EVA-Client separates robot-agnostic control logic from transport, robot-description, action-space, and inverse-kinematics layers. It supports hardware, dataset, and simulated execution while exposing configurable representations and continuous IK, with non-ROS camera input remaining a limitation.
- Transport backends: Transport backends decouple control logic from observation and command movement across ROS1, ROS2, dataset replay, and ZMQ execution.ROS1 and ROS2 align buffered topic streams; dataset replay enables open-loop inference without hardware, while ZMQ can synthesize observations through a fake execution-layer node.
- Robot support: Robot-description objects declare actuator groups, observation schemas, mappings, and optional kinematics, allowing generic backends to support heterogeneous platforms.The reference dual-arm Piper has 2 × (6 joints + 1 gripper) = 14 action dimensions.
- Action spaces and inverse kinematics: EVA-Client independently separates observation, policy-output, and publication action spaces, each of which can use joint or end-effector representations.End-effector actions include position, orientation, and gripper commands; orientations are canonicalized from quaternion, roll-pitch-yaw, or 6D rotation formats.
- Continuous IK: Continuous IK uses PyRoki’s differentiable kinematics and Levenberg–Marquardt least-squares solves with pose, rest, and velocity costs under hard joint limits.Sequential warm starts and anchoring the first frame to measured joints promote continuity, although discontinuities can still occur near singularities or joint limits.
5 Operation Model
EVA-Client frames deployment as an inspectable debugging process managed through a unified web console. Its operation model progresses from risk-free simulation and human-supervised stepping to continuous real-time execution.
- Unified console: A single web console exposes Debug, Collect, Eval, Replay, and read-only Result tabs, sharing controls, a 3D scene, and synchronized camera streams.The persistent tab bar unifies the framework’s operation model across its primary workflows and result inspection.
- Open-loop simulation: Open-loop simulation routes policy actions only to visualization, letting users verify checkpoint motion before risking physical hardware.Observations may be live or synthetic, while actions remain confined to the Viser-based 3D view.
- Supervised stepping: Real single-chunk stepping advances exactly one action chunk per command, keeping the robot stationary between chunks for failure localization.Users can inspect each result and associate failures such as bad grasp poses with specific inference steps.
- Supervised stepping: Single-chunk sim-to-real stepping previews each chunk in simulation before the user commits or cancels physical execution.This human checkpoint supports checkpoint validation and operation near fragile objects.
- Continuous execution: Continuous execution publishes actions at the configured rate while the selected inference strategy schedules and smooths chunks in the background.This is EVA-Client’s full real-time deployment mode for actual task execution.
6 Inference Strategies
EVA-Client unifies synchronous and asynchronous inference strategies for chunk-based policies, addressing latency, stale actions, and discontinuities between overlapping predictions. Its selectable methods include latency-aware blending, ACT-style temporal ensembling, naive asynchronous replacement, and Real-Time Chunking.
- Strategy overview: Chunk-based real-time control must schedule inference, handle delayed actions, and combine overlapping predictions into a smooth command trajectory.Inference latency causes consecutive action chunks to overlap and potentially disagree at their boundaries.
- Execution modes: Synchronous execution is reproducible but pauses the robot during every forward pass, whereas asynchronous scheduling keeps control running during inference.On table tennis, synchronous pause-and-go motion prevents the rally from getting started, while asynchronous execution allows it to proceed; these are illustrative deployments, not a controlled benchmark.
- Asynchronous smoothing: Asynchronous prefetch uses a background inference thread, latency trimming, and linear-overlap blending to align new chunks with the buffered trajectory.The method discards leading actions based on estimated delay, blends over an overlap window, and appends the new chunk’s unoverlapped tail.
- Asynchronous smoothing: ACT-style temporal ensembling retains predictions for each absolute timestep and exponentially aggregates them, with default decay coefficient m=0.01.Earlier predictions receive larger weights and later predictions are exponentially discounted; this weighting follows ACT’s default rather than an EVA-Client-specific design choice.
- Baselines and RTC: Naive asynchronous replacement provides a no-blending control baseline, while Real-Time Chunking reduces seams at the source by sending the previously committed chunk back to the server.Replacement compensates for latency through global-timestep indexing, isolating the contribution of smoothing; RTC instead uses server-side conditioning.
7 Data Collection
EVA-Client’s Collect mode turns teleoperated robot operation into training-ready demonstrations using the same console, robot descriptions, and transport backends as deployment. It records synchronized state-action episodes, stores them in LeRobot format, and validates and flags data for review.
- Collection workflow: Collect mode records teleoperated episodes directly into training-ready datasets while reusing EVA-Client’s deployment console, robot descriptions, and transport backends.Collect runs no policy: a human drives the robot during recording.
- Collection workflow: Teleoperation recordings pair each frame’s measured robot state with its commanded action, represented as joint angles and end-effector pose.Capture requires an explicit activation step, preventing accidental recording.
- Training-ready output: Episodes are written in LeRobot format with per-step observations and actions, one H.264 video per camera, and dataset metadata.Exact-interval timestamps satisfy fixed-rate requirements while jittery real capture time is preserved separately.
- Quality control: Every finished episode undergoes frame-level checks for timestamp, camera, vector, finite-value, and video-table consistency, with problematic data flagged for review.Offending fields are zero-filled to preserve regularity rather than silently discarding flagged episodes.
8 Evaluation and Logging
EVA-Client makes real-robot evaluation systematic by standardizing scored trials and preserving structured records, transforming subjective checkpoint impressions into reproducible, comparable measurements. Exhaustive action-stream logging and a read-only viewer make policy execution inspectable and enable side-by-side checkpoint analysis.
- Motivation: Evaluation replaces subjective checkpoint impressions with recorded, comparable measurements through a core subsystem for evaluation and logging.The framework addresses evaluation that is otherwise subjective, unrecorded, and impossible to reproduce or audit.
- Scored trials and scenes: Each scene pairs an object configuration with a prompt, and each trial receives milestone-based graded scoring plus a structured persistent record.Trials are keyed by scene, position, and trial, with milestone outcomes and optional free-text notes.
- Multi-checkpoint comparison: Multiple checkpoints can be evaluated on identical scenes and milestones, with separate logs enabling side-by-side comparison in the result viewer.Switching the active model re-targets the policy endpoint and re-runs warmup before evaluation.
- Complete logging and result viewer: EVA-Client records raw predictions, strategy-smoothed actions, and executed robot actions as timestamped streams, while a read-only viewer links trials, videos, milestones, and breakdowns.Each step tracks its source chunk, and exported action-stream tables are organized by prompt, mode, and strategy.
9 Limitations and Roadmap
EVA-Client is deployment infrastructure rather than a policy or benchmark, with illustrative rather than controlled task outcomes and several current hardware-scope limitations. Its roadmap extends the substrate toward reinforcement-learning data collection, agentic policies, fine-grained annotation, and broader robot morphologies, while releasing tooling rather than collected data.
- Limitations: EVA-Client does not train models or provide a benchmark, and its reported task outcomes are illustrative observations from real deployments rather than a controlled study.Live transport currently covers ROS1, ROS2, and ZMQ; cameras on non-ROS robots still depend on middleware-specific image sources.
- Limitations: The inverse-kinematics solver targets serial-arm manipulators, while other morphologies are supported only through the stated current limitations.
- Roadmap: The roadmap aims to turn evaluation rollouts and reward or outcome labels into reinforcement-learning and interactive-fine-tuning signals, including recorded human corrections.Operators may take over or nudge the arm mid-rollout, with each correction recorded as targeted training signal.
- Roadmap: The same client is intended to host agentic-policy controllers while exposing planner hooks for issuing sub-goals and reading execution state.
- Roadmap: Collect mode is planned to support fine-grained task and sub-task annotation, segmenting long-horizon episodes into labeled units and milestones within the same LeRobot dataset.This is intended to make collections reusable at the level of individual manipulation phases.
- Roadmap: Across these directions, EVA-Client will release tooling that produces and labels data, not the data itself.
10 Conclusion
EVA-Client is presented as a deployment counterpart to the policy-training ecosystem, unifying real-robot deployment into an inspectable, reproducible, and comparable process. Its decoupled substrate supports teleoperated data collection, scored evaluation, and exhaustive action logging.
- EVA-Client serves as the deployment counterpart to an increasingly mature policy-training ecosystem.
- Its unified client decouples signal sources, robot backends, action spaces, and inference strategies.
- The framework turns real-robot deployment from one-off scripts into an inspectable, reproducible, and comparable process.
- The same substrate supports teleoperated data collection, scored evaluation, and exhaustive action logging.
Author List
The paper is authored by Fengjiao Chen, Manyuan Zhang, Linjiang Huang, and Si Liu.
- Author List: The author list comprises Fengjiao Chen, Manyuan Zhang, Linjiang Huang, and Si Liu.Fengjiao Chen is marked with †, while Linjiang Huang and Si Liu are marked with B.