Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies

• VLM backbones with billions of parameters incur latency and energy costs that matter for reactive manipulation at 25–50 Hz.Action chunking and asynchronous execution accommodate slow inference but do not shorten perception-to-action computation.Source: §I · p. 1
• On 18 RoboCasa tasks with held-out language paraphrases and randomized scenes, DEM reached 55.6% success, versus 56.9% and 54.6% for VLM-backbone policies.It reached the VLM policies’ success range at 6.1 ms per call, eight to seventeen times less latency. GPU energy was estimated at 2.07 J per policy call.Source: Figure 1 · p. 1, §I · p. 2
• The study asks how much success modern standalone encoders retain and what inference cost they save, holding demonstrations, training steps, tasks, evaluation, and measurement platform fixed.It compares seven VLM-backbone and decoupled policies under that shared protocol.Source: §I · pp. 1–2
• DEM combines a fine-tuned DINOv3 vision encoder and frozen NeoBERT language encoder with a MeanFlow head that generates an action chunk in one forward pass.The proposed alternative separates image and instruction representations that condition action prediction.Source: §I · pp. 1–2
• Under the study protocol, fine-tuning vision has a far larger observed effect than fine-tuning language, and the representation remains effective across regression-, diffusion-, and flow-based action heads.Source: §I · p. 2

• Early generalist policies RT-1, Octo, and BAKU [15] used separate encoders, while later VLM-backed recipes include OpenVLA, the π series [4], and GR00T [3].VLM latency mitigation methods retain decoder-only structure; other compact-policy approaches avoid VLMs through training, observation, viewpoint, or attention changes.Source: §II · p. 2
• DINOv3 [12] and renewed encoder pretraining, including NeoBERT [13], strengthen standalone representations, but whether these encoders yield VLM-policy success remains the tested question.Prior comparisons report encoders outperforming decoders on representation tasks, including a 150M encoder above a 400M decoder on MNLI.Source: §II · p. 2
• Concurrent TurboVLA [31] and ReactVLA [32] use similar designs, while this work varies vision, language, and action components under one protocol on RoboCasa and a physical system.The cited LIBERO result for a 0.54M task-index-conditioned policy is 95.1% [33], which the passage says separates methods poorly there.Source: §II · p. 2

III. DEM: DECOUPLED EMBODIMENT MODEL

• DEM consists of separate vision and language networks whose outputs meet through cross-attention in the action head; the selected modules are DINOv3 ConvNeXt-B, NeoBERT, and MeanFlow.MeanFlow generates the action chunk in one forward pass, and the language pathway runs only when the instruction changes.Source: §III · p. 2, Figure 2 · p. 3

A. Vision encoder

• Each camera frame yields 256 stage-3 tokens from DINOv3 ConvNeXt-B, producing 512 vision tokens per control step from the scene and wrist cameras.The vision encoder is initialized from the public checkpoint and fine-tuned with the head.Source: A. Vision encoder · p. 2

B. Language encoder

• NeoBERT supplies 32 instruction-token hidden states and remains frozen; DEM caches them until a new instruction arrives, unlike the evaluated VLMs’ per-step joint encoding.The language pathway is independent of the images, allowing its cached tokens to be reused between instruction changes.Source: B. Language encoder · pp. 2–3, Figure 2 · p. 3

C. Action head

• MeanFlow generates an action chunk by learning average velocity over an interval, so displacement across that interval is a single product.At inference, the head runs once from noise using the displacement identity with r = 0 and t = 1.Source: C. Action head · p. 3
• The network is trained to satisfy the MeanFlow identity, using the known interpolation velocity and a total derivative that costs one Jacobian-vector product.Source: C. Action head · p. 3
• The training loss uses an adaptive weight, and interval endpoints are sampled logit-normal and set equal with probability 0.5.When the endpoints are equal, the loss reduces to flow matching and anchors training.Source: C. Action head · p. 3

D. Assembling the modules

• The action head receives vision tokens, language tokens, and proprioceptive state as projected, modality-tagged context.The configuration contains 545 context tokens per step; noisy action tokens enter separately, alongside an embedding of the interval.Source: D. Assembling the modules · p. 3
• Each head block self-attends among action tokens, then reads context through cross-attention, with a zero-initialized per-channel gate controlling context use.A final layer normalization and zero-initialized linear layer map the block output to the predicted average velocity.Source: D. Assembling the modules · p. 3
• The vision encoder is fine-tuned from its public checkpoint, the language encoder remains frozen, and the action head trains from scratch.The head alone consumes context, allowing encoder swaps to change only their projections while retaining the same head.Source: D. Assembling the modules · p. 3
• Fig. 3 presents one simulated-task demonstration frame per task, grouped by the skill categories in Table I.Source: Figure 3 · p. 4

IV. EXPERIMENTAL SETUP

• Every simulated policy is trained for 300k steps on the same demonstrations and evaluated over 100 rollouts per task across 18 RoboCasa atomic tasks.Evaluation uses held-out instruction paraphrases and randomized layouts; the configurations are trained once, so reported success differences are descriptive.Source: §IV · p. 4
• Seven baselines are fine-tuned on the same demonstrations using official implementations and default recipes, while retaining their native fusion, action module, and training recipe.The comparison is therefore system-level.Source: §IV · p. 4
• Latency is the median batch-size-one policy forward-pass time, while energy is mean GPU power times median latency, with idle draw unsubtracted.Energy is a device-level GPU estimate that excludes CPU, cameras, communication, and actuation; latency excludes camera capture, preprocessing, and robot communication.Source: §IV · p. 4
• Real-world tests use an xArm 7 on a linear rail for three FurnitureBench assembly tasks, with 200 teleoperated episodes per task and 50 evaluation rollouts per task and method.Each policy receives 100k training steps; the real-robot comparison includes GR00T N1.7, π0.5, and TurboVLA.Source: §IV · p. 4

A. Overall benchmark

• 55.6% mean success for DEM is close to GR00T N1.7 at 56.9% and π0.5 at 54.6%; the experiment treats them as a similar observed range.The rollout-only 95% interval for the DEM–GR00T difference is −1.3 ± 3.3 points; training-seed variability was not estimated.Source: A. Overall benchmark · p. 4, Table I · p. 5
• On the knob task, the reported deficit is 37.0% against 49.0%; TurboVLA scores 46.3%, ReactVLA 41.9%, and Octo is last at 25.6%.Source: A. Overall benchmark · p. 5
• TurboVLA reaches 46.3%, ReactVLA 41.9%, and Octo 25.6%, below the VLM-backbone group in this benchmark.Octo pairs 2024-era encoders with a diffusion head.Source: A. Overall benchmark · p. 5

B. Forward-pass latency and estimated GPU energy

• 6.1 ms per action chunk gives DEM a maximum throughput of 163 policy calls per second and an estimated 2.07 J per call.GR00T N1.7 and π0.5 reach 19.3 and 9.7 calls per second and spend 13.7 and 31.7 J per call, respectively.Source: B. Forward-pass latency and estimated… · p. 5, Table II · p. 5
• With language cached, a control step pays for vision encoding, cross-attention, and one MeanFlow evaluation; running NeoBERT every step raises latency to 10.4 ms and energy to 3.09 J.The cached instruction tokens are encoded when the instruction arrives.Source: B. Forward-pass latency and estimated… · p. 5

C. Vision encoder ablation

• 55.6% for fine-tuned DINOv3 contrasts with 32.6% for frozen DINOv3, while fine-tuned vision encoders range from 51.8% to 55.6%.The VLM vision towers reach 53.9% and 51.8% with about five times the parameters, three times the latency, and four times the energy per inference.Source: C. Vision encoder ablation · p. 5, Table III · p. 5
• The evaluated VLM towers share an attention stack and re-encode instructions each step, whereas DEM’s separate towers write to static context, allowing language tokens to be cached.The vision ablation uses the language encoder, fusion, and head fixed.Source: C. Vision encoder ablation · p. 5, Table III · p. 5

D. Language encoder ablation

• 55.6% for NeoBERT exceeds T5-base at 44.5%, BERT-base at 47.8%, and mmBERT-base at 50.6%; text encoders resolve tasks from held-out paraphrases.Without language tokens success is 11.3%, and replacing the instruction with a one-hot code raises it to 37.4%; fine-tuned NeoBERT reaches 55.1%, a difference the single-run design cannot resolve.Source: D. Language encoder ablation · pp. 5–6, Table IV · p. 6

E. Action head

• 55.6% for MeanFlow is the highest observed head success, while all four heads remain within 3.2 points and their computational costs differ substantially.ACT and MeanFlow need one pass per chunk and reach about 163 calls per second for about 2.1 J; flow matching and Diffusion Policy need ten and sixteen passes and reach 40 and 27 calls per second for 8.5 and 12.9 J.Source: E. Action head · p. 6, Table V · p. 6

F. Robustness beyond MeanFlow

• 54.7% for flow matching, 53.8% for Diffusion Policy, and 52.4% for ACT keep the decoupled representation near π0.5 at 54.6% and GR00T N1.7 at 56.9%.The comparison supports a competitive, more efficient alternative in this trained-task regime, not a causal claim that joint vision-language encoding is inferior.Source: F. Robustness beyond MeanFlow · p. 6, Table V · p. 6

G. Chunk length

• 55.6% success at chunk length 16 and 55.9% at 8 declines to 50.6% when DEM replans every control step, still at 6.1 ms per step.Chunk length and replanning interval change together in the sweep, so their effects are not separated; longer chunks remain the better choice on these tasks.Source: G. Chunk length · p. 6

H. Real-world tasks

• 66.0% success for DEM across three real-robot tasks is close to GR00T N1.7 at 68.0%; the three-trial difference does not support an ordering.DEM succeeds in 99 of 150 trials and GR00T N1.7 in 102; the authors interpret the study as evidence of transfer to the physical system, not equivalence.Source: H. Real-world tasks · pp. 6–7, Table VII · p. 7
• DEM reaches 163 policy calls per second and 2.07 J per call on the real robot, compared with GR00T N1.7 at 19.3 calls per second and 13.7 J.The architectures, inference stack, and workstation are unchanged from simulation; the setup uses an xArm 7 on a linear rail and tasks involving a drawer, lamp, and cabinet assembly.Source: H. Real-world tasks · p. 7, Figure 4 · p. 7

VI. DISCUSSION

• GR00T N1.7 has the highest success point estimate, while DEM has far lower measured inference cost; other action heads retain near-VLM-range success with the same decoupled representation.MeanFlow turns that representation into inference efficiency by replacing iterative action generation with a single head evaluation.Source: §VI · p. 7
• Fine-tuning the vision encoder is worth 23 points, while fine-tuning the language encoder changes nothing.The discussion relates this asymmetry to robot-specific visual elements absent from web pretraining and short instructions using ordinary English.Source: §VI · p. 7
• Evidence is limited to trained skills; baseline recipes remain native, each configuration is trained once, and efficiency measures cover one workstation GPU’s policy forward pass.The experiments do not test unseen tasks or compositional instructions, and the measurements do not establish end-to-end control frequency or total system energy.Source: §VI · p. 7, §VII · p. 7

VII. CONCLUSION

• Under the trained-task protocol, DEM reaches a success range similar to the strongest VLM-backbone policies while requiring far less forward-pass computation.The conclusion limits this finding to skills represented in the demonstrations and says open-world generalization remains untested.Source: §VII · p. 7