Source-linked AI summary
JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Models
Haoran Sun, Wentao Zhang, Junyang Hua, Hedan Yang, Yongjian Guo, Yifei Zhang, Xiaolong Xiang, Mingxi Luo, Jing Long, Chen Zhao, Chen Zhou, Wanting Xu, Qiming Yang, Hui Zhang, Song Wang, Xiaodong Bai, Shuai Di, Xu Chu, Xiaotie Deng, Yicheng Gong, Junwu Xiong
TL;DR
Existing VLA compute services burden users with infrastructure management and inefficient fixed card-hour allocation for small or bursty workloads. JoyNexus provides a multi-tenant service architecture with shared model and environment services, and experiments show reduced aggregate GPU time and improved training and inference utilization.
Problem
Existing VLA compute services require tenants to manage complex infrastructure, while fixed card-hour pricing is inefficient for small, iterative, and bursty workloads.
Method
JoyNexus separates Training Model, Inference Model, and Environment Services, routing isolated tenant workloads through shared infrastructure and supporting SFT, RL, rollout, and evaluation.
Results
JoyNexus reduces aggregate GPU time and improves training and inference utilization compared with isolated single-tenant execution.
Takeaways & Limitations
JoyNexus supports tenant-private VLA post-training while letting providers manage shared resources, scheduling, routing, and service state.
Takeaways & Limitations
Greater user flexibility introduces security, correctness, and efficiency risks, requiring future safeguards for extensions, sandboxing, attack detection, and performance contracts.
Abstract
from arXiv · showhide
The post-training of Vision-Language-Action (VLA) models is essential due to the diversity of simulators, robot embodiments, and task objectives. Existing compute services, whether offered as direct accelerator rental or batch-workload submission, typically allocate an exclusive set of GPU and CPU resources to a single tenant. While this paradigm maximizes client flexibility, it burdens users with infrastructure adaptation, and the fixed card-hour accounting model renders short or bursty workloads both expensive for tenants and inefficient for the service provider. To address these challenges, we present JoyNexus, a unified service for multi-tenant VLA supervised fine-tuning, reinforcement learning, and evaluation. JoyNexus decouples the Training Model Service, Inference Model Service, and Environment Service, each accessed through APIs and backed by resident shared base models with tenant-specific slots. Tenants can directly invoke high-level semantic APIs for training, rollout, and evaluation, or compose custom algorithms using lower-level APIs and their assigned endpoints. Multiple tenants submit workloads concurrently; their action modules, optimizers, rollout records, and policy versions remain isolated, and the service is scheduled by the global Training Queue and Inference Queue. To further improve multi-tenant training efficiency, JoyNexus introduces group batching for heterogeneous VLA data schemas that share a compatible model-facing prefix, enabling a single shared backbone forward pass over grouped samples. Finally, we evaluate JoyNexus through workload simulation and a group-batching pipeline in a realistic embodied scenario. Results show that, compared with isolated single-tenant execution, JoyNexus reduces aggregate GPU time and improves service utilization via cross-tenant scheduling on shared resources.
1 Introduction
JoyNexus reframes VLA post-training as a multi-tenant service that separates model and environment services while isolating tenant-specific computation. Its unified backend supports SFT, RL, and evaluation, improving resource utilization over serial single-tenant processing.
- Motivation: JoyNexus addresses the infrastructure burden of dedicated rental and shared-cluster workload paradigms, which leave tenants responsible for complex dependencies.The challenge is particularly pronounced for VLA training across heterogeneous model and simulator environments.
- JoyNexus service abstraction: JoyNexus reframes embodied VLA post-training as a client-server multi-tenant service with server-controlled allocation, placement, and routing.The abstraction separates tenant-private computation from shared infrastructure while maintaining workload isolation.
- Unified service architecture: The backend separates Training Model, Inference Model, and Environment Services to support SFT, RL, and evaluation.Base models remain memoryresident, while tenant-specific modules are mounted in isolated slots.
- Results: JoyNexus achieves higher efficiency than classic single-tenant serial workload processing through improved resource utilization.The contribution is reported as an experimental result comparing JoyNexus with serial single-tenant execution.
2 Related Work
Related work spans service-oriented post-training systems, distributed training substrates, multi-tenant model serving, and VLA-specific training and evaluation workflows. JoyNexus builds on these directions while targeting client-server boundaries, tenant isolation, dual-path scheduling, policy routing, and VLA environment interaction as first-class service objects.
- Service-oriented post-training: Tinker-style systems expose post-training through programmatic APIs while hiding distributed execution, separating user-composed learning programs from service-managed resources, residency, and scheduling.This separation lets users avoid manually managing worker placement, distributed initialization, and artifact movement.
- Service-oriented post-training: Recent language-model systems extend service-oriented post-training to agentic reinforcement learning, rollout-as-a-service, managed serving, and asynchronous separation of rollout generation from policy training.Examples include OpenTinker, ProRL Agent, MinT, AReaL, and MARLaaS.
- Distributed training substrates: Distributed frameworks supply the computational substrate for service execution through tensor, data, pipeline, sequence, heterogeneous, and memory-efficient parallelism.Megatron-LM, ZeRO, DeepSpeed, and Colossal-AI represent these capabilities, while Ray provides distributed execution support.
- Multi-tenant systems: Multi-tenant training and serving systems share base models or adapters and improve batching, multiplexing, memory use, and scheduling, but JoyNexus addresses a broader service layer.JoyNexus additionally treats service boundaries, training and inference queues, tenant-private state, artifact visibility, policy routing, and VLA environment interaction as first-class objects.
- VLA post-training and evaluation: VLA post-training research combines supervised fine-tuning, reinforcement learning, rollout storage, simulator interaction, model updates, and evaluation across reusable datasets and tools.VLA foundation models motivate these workloads, while recent methods demonstrate downstream adaptation and online policy-training loops.
3 Preliminaries
VLA models map multimodal context to actions through a shared perception-language backbone and embodiment-specific action modules, while JoyNexus abstracts post-training as reusable workflows. These workflows cover supervised fine-tuning, reinforcement learning, and fixed-policy evaluation, sharing core system components despite differing data sources and parameter updates.
- VLA Model Abstraction: VLA models encode visual observations, language instructions, robot state, and optional history before an action module predicts robot actions.Action modules may use diffusion, flow-matching, or lightweight action experts.
- VLA Model Abstraction: A Lego-style decomposition separates the vision-language base model from action-specific heads, enabling flexible combinations of VLMs and action experts.VLA post-training commonly keeps the base VLM fixed because narrow task data may degrade general vision-language comprehension.
- Post-Training Workflows: JoyNexus abstracts SFT, RL, and evaluation as workflows that reuse data access, inference, training updates, parameter export, and artifact storage.The workflows differ in data origin and whether model parameters are updated.
- Post-Training Workflows: SFT trains on demonstration records, RL learns from environment-generated rollout records, and evaluation runs inference on a fixed policy while recording metrics.SFT updates tenant trainable modules; RL updates tenant parameters and synchronizes them, whereas evaluation does not change policy parameters.
4 JoyNexus Service Architecture
JoyNexus organizes multi-tenant VLA post-training around a Master Service control plane and resident, decoupled model and environment services. Its APIs, runtime safeguards, and group batching support isolated concurrent workloads while improving utilization for compatible small requests.
- Workflow composition: Shared backend primitives compose RL, SFT, evaluation, and custom workflows while separating user intent from backend-specific process layout and communication details.RL connects rollout, inference, environment, and training; SFT uses an offline-data producer; evaluation reuses inference and environment services without optimization.
- Service architecture: JoyNexus comprises user-facing workload specifications, a Master Service control plane, and resident model and environment services forming the execution plane.The Master Service validates user intent and translates specifications into tenant-scoped workloads.
- Multi-tenant isolation: Tenant-specific action modules, optimizer states, policy versions, environment sessions, and checkpoints remain separately identified while Training and Inference Queues dispatch work across workloads.Resident base models are reused across workloads sharing a base model, with tenant-specific action modules maintained separately.
- Runtime capabilities: JoyNexus provides centralized monitoring, service-level fault isolation, and elastic scaling as runtime capabilities for long-running multi-tenant post-training workloads.A Health Manager supports in-place restart of failed roles, while decoupled inference and training enable asynchronous rollout-capacity scaling.
- Group batching: Group batching increases the physical batch presented to the shared base model when per-tenant batches are small, amortizing the dominant shared forward pass.The pipeline determines a canonical VLAFeature shape for accepted requests and partitions outputs afterward.
5 Experiments
JoyNexus is evaluated on realistic mixed VLA post-training workloads and controlled group-batching experiments. The results show improved overlap and shared-forward efficiency while preserving tenant-specific optimization behavior.
- Mixed Workload Evaluation: The realistic workload mixes three RL tenants and one SFT tenant on a shared 8-GPU node across LIBERO and ManiSkill simulator workloads.The deployment uses a 2–2–4 layout for actor training, inference, and environment simulation, respectively.
- Mixed Workload Evaluation: Compared with sequential isolated execution, JoyNexus overlaps asynchronous rollouts and fills idle actor intervals with SFT work.The isolated baseline exposes complementary idle periods between inference, actor training, and simulator interaction.
- Group Batching: Group batching canonicalizes compatible tenant inputs for one shared base-model forward pass while retaining tenant-private losses, backward passes, and updates.The evaluation isolates shared-forward efficiency rather than claiming end-to-end training-throughput improvements.
- Group Batching: Group-batching speedup generally increases with more tenants and smaller local batches, especially when small tenant batches underutilize the resident GPU service.For QwenGR00T, the benefit also grows with VLM scale; OpenPI shows a similar trend at comparable scale and architecture.
- Group Batching: In the eight-tenant, local-batch-size-4 setting, group execution reduces repeated VLM computation because base-VLM forward time dominates tenant-specific action-module computation.This reduction can shorten iterations when shared forward computation is a major component of iteration time.
- Training Correctness: Across four OpenPI tenants training on LIBERO and CALVIN streams, grouped loss trajectories closely track the resident serial baseline while retaining stable per-tenant optimization behavior.Each tenant maintains separate flow-matching action-head parameters and optimizer state.
6 Conclusion
JoyNexus presents a multi-tenant service architecture that makes VLA post-training workloads tenant-private while managing shared model and environment services. The conclusion identifies future work in adaptive scheduling, pricing, user flexibility, isolation, and service-aware algorithm design.
- Conclusion: JoyNexus organizes SFT, RL, rollout, and evaluation as tenant-private workloads over shared model and environment services.Users express learning intent while the provider manages sessions, routing, action-module state, artifacts, and policy revisions.
- Dynamic Resource Adjustment: JoyNexus should dynamically adjust service routing and resource allocation using queue pressure, simulator latency, and tenant priority.The current design leaves workloads largely tied to fixed routes and resource assignments during execution.
- Pricing and Productization: Future pricing mechanisms must balance tenant cost, platform utilization, and service guarantees through usage-based and priority-aware schemes.The proposed schemes would account for resource consumption, workload characteristics, and latency requirements.
- User Flexibility and Isolation: JoyNexus must support migration paths for platform simulators, uploaded Docker environments, external environment APIs, and customizable algorithm components.Potential overrides include data processing, rewards, rollout logic, and training code, with flexibility balanced against isolation.
- Service-Aware Algorithm Design: With tenant permission and privacy safeguards, JoyNexus could use related cross-user datasets, tasks, or action schemas for augmentation, transfer, or initialization.Richer schedulers could form dynamic training groups beyond the current FIFO-like queues with compatibility checks.
A Composition of VLA Workloads
The appendix expands the high-level workflows into implementation-oriented pseudocode. It clarifies service composition while remaining independent of deployment size and model-specific implementation details.
- The appendix provides an implementation-oriented expansion of the high-level workflows in Section 4.2.
- The pseudocode follows the current control flow but omits Ray remote-call syntax, distributed collectives, and model-specific tensor operations.
- These omissions make service composition explicit without tying the description to a particular deployment size.
A.1 Dual-Queue Scheduling Abstraction
JoyNexus separates optimization and latency-sensitive prediction into independent Training and Inference Queues, while coordinating them only when updated tenant policies are published for inference.
- Dual-Queue Scheduling Abstraction: The Training Queue handles optimization work whose payload remains available across rollout generation, preprocessing, and actor consumption.The Inference Queue instead handles latency-sensitive predictions returned directly to rollout or evaluation sessions.
- Dual-Queue Scheduling Abstraction: The Training and Inference Schedulers operate without a global barrier, allowing reinforcement-learning action requests, pending trajectories, and evaluation work to proceed concurrently.Evaluation can use the Inference Queue without creating optimization work.
- Dual-Queue Scheduling Abstraction: The sole cross-queue dependency is policy publication, through which the Training Model Service sends updated tenant parameters to the Inference Model Service after reinforcement-learning updates.
A.2 Training Job Descriptor and Data Binding
JoyNexus schedules RL and SFT through a shared training-job descriptor paired with a retained data partition, while tenant–task admission and priority-aware execution regulate concurrency. Sequence fields and state leases preserve ordering and prevent duplicate preprocessing. واق
- Training job binding: RL and SFT share a training-job descriptor and retained data partition as the unit of Training Scheduler and Training Queue orchestration.The descriptor carries a partition_id that links producers, preprocessing stages, and actors to the corresponding payload without containing training tensors.
- Admission control: Tenant–task admission uses priority, in-flight-job limits, and optional completion targets to control reservations.Reservations are rejected after an in-flight or completion bound is reached, causing the producer to try another eligible tenant or wait.
- Scheduling order: The actor selects the highest-priority ready job and uses ready_seq to preserve FIFO order among equal-priority jobs.created_seq records system entry, whereas ready_seq records satisfaction of all producer-side dependencies.
- Job state progression: Intermediate computing_* states act as preprocessing-service leases that prevent concurrent replicas from processing the same stage.Actor-forward is optional for RL algorithms that do not require newly computed action log-probabilities or values.
A.3 Inference Queue Composition
The Inference Queue batches environment-step requests in arrival order when they share a compatible frozen model prefix, while preserving tenant- and session-specific policy execution. A shared encoder pass is partitioned and decoded by tenant action modules, with responses returned to originating sessions.
- Request composition: Each inference request carries tenant, session, schema, observation, robot-state, language-instruction, and policy-version metadata.These fields support request routing and preserve policy context across environment steps.
- Scheduling: The scheduler forms the largest FIFO-compatible prefix subject to batching timeout and maximum batch size.Compatibility is evaluated at the shared frozen-prefix boundary rather than by tenant identity.
- Group batching: Compatible requests are canonicalized and padded before one shared inference-model encoder pass produces batched features.The worker uses the configured batching timeout and shared-prefix compatibility predicate.
- Response processing: After encoding, features are partitioned by recorded slices and decoded by the corresponding tenant action modules.This preserves tenant-specific decoding while reusing the shared encoder computation.
- Response processing: Decoded results resolve to originating sessions, preserving environment-loop request–response semantics without retaining an Inference Queue partition after delivery.The scheduler and encode-batching worker may therefore be combined without changing the logical dual-queue design.
A.4 Producer-Side Composition
JoyNexus composes RL and SFT producers around tenant-scoped scheduling and model-facing metadata. RL rollouts preserve policy versions and require post-processing before readiness, whereas SFT batches can be consumed immediately.
- RL Producer: The RL producer records the current policy version before environment interaction and computes samples as environments multiplied by the interaction horizon.Each RL job carries its behavior-policy version and tenant schema signature.
- RL Producer: Multiple RL jobs run asynchronously under global and per-tenant concurrency limits, while completed rollouts advance independently without waiting for slower tenants.If admission capacity is unavailable, rollout generation continues without reserving a training job.
- Sample Composition: SFT and RL producers attach tenant identity and schema metadata to model-facing samples, with RL samples additionally carrying the behavior-policy version.SFT sample counts derive from the configured SFT sample count, while RL counts use environment count times interaction horizon.
- Readiness States: SFT examples terminate ready because they already contain supervised targets, whereas RL partitions require actor-forward fields, advantages, and returns before readiness.The two producer paths deliberately terminate at different states.
A.5 RL Preprocessing Stages · A.6 Actor Consumption and Version Consistency
JoyNexus preprocesses RL rollouts through optional actor-forward and advantage stages that preserve partitioned samples while attaching learning fields through queue-mediated metadata. A shared actor then consumes RL and SFT jobs with tenant-aware batching, policy-version checks, isolated updates, and terminal cleanup.
- A.5 RL Preprocessing Stages: The optional actor-forward stage evaluates rollout behavior actions and attaches log-probability and value fields required by the learning objective.The advantage stage computes PPO-, GRPO-, or return-based advantages and returns from Training Queue batches, then writes derived fields back to the same sample metadata.
- A.5 RL Preprocessing Stages: The advantage stage obtains batch metadata and tensors through the Training Queue’s two-step interface, computes derived fields, and marks the job ready.On failure, the scheduler records the error and re-raises it.
- A.5 RL Preprocessing Stages: Stage leases apply only to control metadata, while rollout samples remain in their original partition and derived tensors stay associated with sample metadata.This allows actor-forward and advantage workers to be independently deployed or omitted without changing the actor’s final input contract.
- A.6 Actor Consumption and Version Consistency: The actor is the common consumer for ready RL and SFT jobs, but it checks RL behavior-version staleness against the tenant’s current actor version before consumption.The check is omitted for SFT because demonstrations are independent of the current policy.
- A.6 Actor Consumption and Version Consistency: A leased RL partition is dropped when the tenant’s actor-version advance exceeds the configured maximum policy staleness.The stale partition is cleared and the training scheduler records the job as dropped for policy staleness.
- A.6 Actor Consumption and Version Consistency: Each distributed optimizer microbatch contains one tenant–schema group, activating that tenant’s action module and optimizer for task-specific loss computation and updates.Only the corresponding tenant’s policy version is incremented after training.
- A.6 Actor Consumption and Version Consistency: Partitions are cleared after successful consumption, staleness drops, and failures, while terminal training-job state remains scheduler metadata after tensors are released.The actor validates tenant and schema annotations and records completion, failure, or drop outcomes through the Training Scheduler.