PaperlayerView exact revision on arXiv

Source-linked AI summary

FlowVLA: Learning Flow-Aware Physical Completion for Quantitative Robotic Pouring

Liwen Yang, Yutian Zhang, Jiaqi Zhai, Li Yang, Tutian Tang, Qiwei Shan, Yancheng Bi, Qiankuan Shangguan, Yangkun Zhu, Ce Hao, Dong Wei, Qiaojun Yu, Dibo Hou

arXiv:2610.29692v1cs.RO

TL;DR

During steady pouring, images and robot pose can remain nearly unchanged while transferred mass accumulates, making return timing ambiguous for a whole-task VLA policy. FlowVLA separates motor generation from target-independent completion estimation, using multimodal histories to estimate mass and residual outflow for its return decision. FlowVLA reduces MacroMAE to 13.1 g and raises overall quantitative success from 31.0% for BELIEF-VLA to 53.7%.

  • Problem

    During steady pouring, images and robot pose can remain nearly unchanged while transferred mass accumulates, making return timing ambiguous for a whole-task VLA policy.

    Source: §I · p. 1

  • Method

    FlowVLA separates motor generation from target-independent completion estimation, using multimodal histories to estimate mass and residual outflow for its return decision.

    Source: §I · pp. 1–2

  • Results

    FlowVLA reduces MacroMAE to 13.1 g and raises overall quantitative success from 31.0% for BELIEF-VLA to 53.7%.

    Source: B. Main Results and Mechanism Analysis · p. 7

  • Takeaways & Limitations

    The results support modeling completion as a persistent physical state separate from motor generation; target-independent estimation enables held-out target interpolation, and independently trained models cover distinct flow regimes.

    Source: §V · p. 8

    The approach still requires domain-specific training and offline electronic-scale supervision.

    Source: §V · p. 8

Abstract

from arXiv · show

During steady pouring, images and robot pose may remain nearly unchanged while transferred mass accumulates, making the return decision ambiguous for a whole-task vision-language-action (VLA) policy. We present FlowVLA, which separates motor generation from physical-completion estimation. A shared $\pi_{0.5}$ action expert learns approach, pour, and return with boundary-aware supervision. A target-independent estimator fuses visual, joint, wrist-wrench, and recurrent force-belief histories to maintain an estimate of transferred mass. The controller initiates return when this estimate plus predicted residual outflow reaches the requested mass. In coffee-pot/water experiments, FlowVLA achieves 13.1 g goal-balanced mean absolute error (MacroMAE), lower than all tested VLA baselines. Across all attempts, 53.7% both complete the task and finish within +/-10% of the requested mass. The same model interpolates to held-out 150/250-g targets without retraining. Independently trained models further evaluate the formulation in three additional container-fluid settings.

I. INTRODUCTION

Quantitative pouring requires a robot to match transferred mass to a requested amount without online scale feedback, but visually similar states can require continued pouring or return. FlowVLA separates motor generation from persistent physical-completion estimation and predicts residual outflow to guide return.

I. INTRODUCTION

  • Without online scale feedback, visually similar pouring states may require either continued pouring or return as transferred mass accumulates.The paper calls this ambiguity physical-completion aliasing; initiating return also does not immediately stop liquid transfer.Source: §I · p. 1
  • Force feedback can guide action generation, but wrist wrench varies with motion, pose, grasp geometry, and liquid dynamics, so it does not define accumulated transferred mass.The unresolved challenge is integrating quantitative completion into a learned whole-task policy that also grasps, transports, pours, returns, and places.Source: §I · p. 1
  • FlowVLA estimates transferred mass and residual outflow separately from motor generation, then triggers return when predicted terminal mass reaches the target.Its completion estimators are target-independent, while the requested mass conditions the POUR prompt and completion threshold.Source: §I · pp. 1–2
  • The paper evaluates FlowVLA against whole-task VLA and load-based baselines, with mechanism ablations, held-out target interpolation, and additional container–fluid regimes.The additional container–fluid models are independently trained.Source: §I · pp. 1–2
  • Boundary-aware action learning combines multirate force observations and recurrent belief with stage-specific prompts in a shared π0.5 action expert.Row-wise masking and exact-return-boundary sampling supervise the transition from steady pouring to return.Source: §I · p. 2

A. Robotic Precision Pouring

Prior precision-pouring methods estimate liquid level, source-mass loss, or overpour, while force-aware and progress-aware policies add interaction feedback or task-progress representations. FlowVLA is positioned as combining physical-quantity estimation with learned whole-task manipulation.

A. Robotic Precision Pouring

  • Precision-pouring methods estimate receiver liquid level or volume, plan with simplified fluid dynamics, or learn pouring skills through self-supervised trials.Source-container sensing includes mass-loss estimation and overpour prediction, but these approaches generally assume prescribed grasps or restricted motions.Source: A. Robotic Precision Pouring · p. 2

B. Force-Aware and Progress-Aware Robot Policies

  • Force-aware policies add tactile, wrench, or torque feedback; FM-VLA encodes force histories as memory tokens for action generation.The passage identifies FM-VLA as the closest temporal-force reference.Source: B. Force-Aware and Progress-Aware… · p. 2
  • Progress-aware policies estimate remaining task horizon or visual progress, whereas FlowVLA maintains flow in g/s and mass in g across perceptually similar states.FlowVLA uses those estimates to assess whether returning now will achieve the requested quantity.Source: B. Force-Aware and Progress-Aware… · p. 2

A. Problem Formulation

The task is to transfer a requested mass by grasping, pouring, returning upright, and placing the source container. Because mass and flow are not directly observed and similar observations can require different actions, the formulation emphasizes persistent physical state and predictive return.

A. Problem Formulation

  • Given a target mass, the robot must grasp a source container, pour into a receiver, return upright, and place the source down.Source: A. Problem Formulation · p. 2
  • The action expert predicts 50-step absolute joint-position chunks at 30 Hz, conditioned on visual and joint inputs, force history, and recurrent belief.Source: A. Problem Formulation · p. 2
  • Cumulative transferred mass and instantaneous mass flow are not directly observed online, and similar observations can require continued pouring or return.This is the physical-completion aliasing described by the paper.Source: A. Problem Formulation · p. 2
  • The design principles separate motor modes for continued pouring and return, preserve progress as a physical state, and account for liquid that continues flowing during return.The architecture caption describes the return criterion as estimated transferred mass plus return tail.Source: A. Problem Formulation · pp. 2–3, Figure 2 · p. 3

B. Persistent State and Boundary-Aware Action Generation

FlowVLA maintains a recurrent physical-state belief because a single wrist-force sample cannot reliably distinguish liquid flow from arm inertia and vibration. Boundary-aware action learning prevents an action chunk from blending continued pouring with return.

Persistent State

  • The recurrent observer fuses fast and medium force histories with joint position and velocity, carrying belief across stages to distinguish flow from arm inertia and vibration.It encodes 32 force samples over 0.25 s and 32 low-pass samples spanning 1 s.Source: B. Persistent State and Boundary-Aware… · p. 3
  • The observer’s physical state includes integrated mass, corrected flow, flow activity and onset, and estimates of flow and mass uncertainty and force reliability.Its onset latch activates after three consecutive valid updates meeting activity-weighted flow and active-probability thresholds, then persists until reset.Source: B. Persistent State and Boundary-Aware… · p. 3

Boundary-Aware Action Generation

  • The shared action expert conditions 50-step arm–hand action chunks on force and belief tokens, stage prompts, and discretized joint state.Prompts specify the domain and stage; only the POUR prompt includes the target mass.Source: B. Persistent State and Boundary-Aware… · p. 3
  • Boundary-aware supervision masks action rows after the next stage boundary and samples exact return boundaries, avoiding chunks that mix incompatible motion modes.An exact boundary anchor is the first desired return frame and retains the complete return horizon.Source: B. Persistent State and Boundary-Aware… · p. 3

C. Structured Physical-Completion Estimation

FlowVLA estimates physical completion by combining causal multimodal histories into flow and mass estimates, then comparing predicted terminal mass—not just current mass—with the target.

Predictive Completion

  • Return is triggered when estimated transferred mass plus predicted residual outflow reaches the target, rather than when current mass alone reaches it.Before one second of history is available, the controller instead uses current mass; a persistent stall can request a retry or safe return.Source: C. Structured Physical-Completion… · pp. 3–4

Completion Head

  • Visual features, joint and wrench histories, observer belief, and elapsed time feed a fusion network that predicts flow and a mass correction.The completion head uses a six-axis sensor-frame wrench history, unlike the action model’s shorter three-axis base-frame force window.Source: C. Structured Physical-Completion… · pp. 3–4

Gating and Integration

  • A hard-zero gate suppresses false-flow drift, and gated flow plus correction updates a monotonic mass estimate.During approach, buffered predicted flow initializes mass at onset; this accounts for transfer before onset detection.Source: C. Structured Physical-Completion… · p. 4

Residual Outflow

  • A causal GRU predicts residual mass that will leave the source during return from recent joint, wrench, and completion-head flow histories.This tail estimate accounts for liquid transfer continuing after the container begins returning.Source: C. Structured Physical-Completion… · pp. 3–4

D. Supervision and Execution

The system trains its observer, completion head, gate, and tail predictor with separate supervision tailored to flow, mass, false-flow suppression, and residual outflow. At execution, stage transitions select new prompts and request fresh action predictions.

Supervision

  • The belief observer is supervised with flow and mass losses, integral and signed-bias penalties, flow-uncertainty likelihood, and activity labels.Reliability and mass-observation uncertainty are learned through the recurrent mass update without separate labels.Source: D. Supervision and Execution · p. 4
  • The completion head combines confidence-weighted flow and mass interval losses with false-flow suppression, positive-bias penalties, and residual regularization.Its primary ensemble averages four separately trained heads and remains frozen during automatic evaluation.Source: D. Supervision and Execution · p. 4
  • Gate training uses confidence-weighted activity labels and auxiliary losses to suppress zero-flow leakage while preserving nonzero flow and frozen-head predictions.A straight-through estimator supplies gradients through the hard gate decision.Source: D. Supervision and Execution · p. 4

Execution

  • The tail predictor is trained at annotated return boundaries, while execution advances APPROACH→POUR→RETURN and requests a fresh prediction after each transition.The onset latch triggers POUR; the terminal-mass criterion or stall branch triggers RETURN.Source: D. Supervision and Execution · p. 4

A. Experimental Setup

The evaluation compares learned and load-threshold controllers in coffee-pot/water pouring and tests held-out targets and three additional container–fluid settings. Offline scale measurements supervise and evaluate the models but are unavailable at deployment.

Platform and Tasks

  • The robot uses 30-Hz head and wrist images and 500-Hz wrist-wrench observations, while the 10-Hz receiving-container scale supplies offline labels and evaluation only.The scale has 0.1-g specified accuracy and is unavailable to the deployed policy.Source: A. Experimental Setup · p. 4
  • The primary comparison uses coffee-pot/water targets of 100/200/300 g, with 150/250-g targets withheld; additional tasks use a kettle, soy-sauce dispenser, and oil dispenser.Each additional container–fluid setting uses an independently trained FlowVLA model.Source: A. Experimental Setup · p. 4

Data and Baselines

  • The shared coffee-pot/water dataset has 758 episodes, and the primary domain uses a target-stratified 608/75/75 train/validation/test split for its physical estimators.The primary comparisons include π0.5, a data-scale control, ForceVLA, TA-VLA-Wrench, BELIEF-VLA, and two trajectory-replay load-threshold controls.Source: A. Experimental Setup · pp. 4–5

Evaluation Scope

  • Table I covers primary coffee-pot/water results at 100/200/300 g, contrasting replayed held-out paths in original layouts with learned policies using randomized initial positions.Table II reports mechanism ablations; Table III covers held-out targets with the unchanged primary model and additional tasks with independently trained models.Source: Table I · p. 6, Table II · p. 7, Table III · p. 7

Protocol and Metrics

  • Trials randomize method and goal order; successful completion requires timely return, no excessive spillage or tipping, and stable placement and release.Quantitative success requires final mass within ±10% of target, and overall quantitative success combines accuracy and completed tasks over all attempts.Source: A. Experimental Setup · p. 6
  • MacroMAE averages the 100-, 200-, and 300-g goal-specific MAEs equally; mass errors use completed trials with recorded final mass.Flow MAE is time-weighted within episodes and averaged equally across episodes, using valid labels before recorded return onset.Source: A. Experimental Setup · p. 6

B. Main Results and Mechanism Analysis

FlowVLA improves quantitative pouring outcomes over force-feedback and recurrent-belief baselines, while mechanism tests show distinct contributions from boundary-aware supervision, zero-flow gating, and residual-outflow prediction.

Main Results

  • FlowVLA reduces MacroMAE to 13.1 g and raises overall quantitative success from 31.0% for BELIEF-VLA to 53.7%.BELIEF-VLA reaches 26.5 g MacroMAE, while ForceVLA and TA-VLA-Wrench reach 50.3 and 101.6 g, respectively.Source: B. Main Results and Mechanism Analysis · p. 7

Mechanism Analysis

Main Results

  • Scaling π0.5 from 758 to 1,453 demonstrations raises grasp success from 65.2% to 73.3%, but post-grasp success changes only from 33.3% to 36.4%, with 72.9 g MacroMAE.The larger dataset improves grasping without resolving return timing.Source: B. Main Results and Mechanism Analysis · p. 7
  • With prescribed motion in original layouts, TR-LT calibration reduces MacroMAE from 95.56 to 19.87 g, while FlowVLA has lower error at every target.Remaining TR-LT error reflects that wrist-load changes alone do not isolate transferred mass during dynamic pouring, when container orientation, arm motion, vessel oscillation, and sloshing affect wrench.Source: B. Main Results and Mechanism Analysis · p. 7

Mechanism Analysis

  • Removing boundary-aware supervision raises MacroMAE from 13.15 to 20.27 g and reduces post-grasp success from 94.0% to 84.6%.Partial return followed by renewed pouring under the POUR prompt occurred in 9 of 33 completed trials, with multiple reversals in 4.Source: B. Main Results and Mechanism Analysis · p. 7
  • Without the gate, MacroMAE reaches 16.92 g and bias shifts from +2.65 to −5.73 g, consistent with false accumulated mass prompting early return.Gate and trigger variants retain 94.6–97.5% post-grasp success, but their mass errors increase.Source: B. Main Results and Mechanism Analysis · p. 7
  • With policy and mass estimator fixed, current-mass, fixed (0.2 s) bft, and learned-tail triggers yield 19.06, 15.79, and 13.15 g MacroMAE, respectively.Predicted residual mass triggers return before estimated mass reaches 300 g; final mass is 298.60 g.Source: B. Main Results and Mechanism Analysis · p. 7

C. Held-Out Goals and Additional Domains

The primary model interpolates to held-out target amounts without retraining, while independently trained models evaluate additional container–fluid settings with distinct flow profiles and outcomes.

Held-Out Goals

Additional Domains

  • The top-handle kettle has 78.1% grasp success and 75.0% E2E success, while its 15.0% goal-balanced relative MAE is the highest among the three additional tasks.Its wide opening produces the fastest observed flow profile, consistent with greater sensitivity to return timing; the oil task reaches 57.4% E2E success and has a longer return tail.Source: C. Held-Out Goals and Additional Domains · pp. 7–8
  • Saturated brine yields 9.05 g completed-trial MAE but 35.5% E2E success, with failures mainly involving difficult grasping and spillage from the soy-sauce dispenser’s spout.Source: C. Held-Out Goals and Additional Domains · p. 8

V. CONCLUSION

The conclusion frames quantitative pouring as a physical-completion problem: similar visual states may require different actions as mass accumulates. FlowVLA separates that persistent state from motor generation, but still depends on domain-specific training and offline scale supervision.

Conclusion

  • Physical-completion aliasing means nearly unchanged images and robot poses can require different actions as mass accumulates.Source: §V · p. 8
  • More demonstrations and force feedback do not by themselves yield precise quantitative stopping; FlowVLA combines boundary-aware action learning, persistent mass estimation, and residual-outflow prediction.Source: §V · p. 8
  • The results support modeling completion as a persistent physical state, separately from motor generation, and target-independent estimation enables held-out target interpolation.Source: §V · p. 8
  • The approach requires domain-specific training and offline electronic-scale supervision; future work aims to learn transferable flow, mass, and residual-outflow representations.The stated goal is to reduce retraining and dependence on instrumented demonstrations.Source: §V · p. 8

Related papers

Loading 2610.29692v1…