PaperlayerView exact revision on arXiv

Source-linked AI summary

Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation

Haoyuan Deng, Haichao Liu, Wenkai Guo, Yuan Ling, Zaijia Yang, Yuanjiang Xue, Haosheng Sun, Liangzi Wang, Ziwei Wang

arXiv:2609.01596v1cs.ROcs.LG

TL;DR

Sub-millimeter robotic assembly requires precise, compliant interaction because visually similar motions can produce different contact outcomes. Facet-0 predicts and values action consequences, then adapts locally; on five electronics-assembly tasks, it reaches 82% mean success versus 15% for the strongest baseline, with 0.5 mm accuracy and 50 ms latency.

  • Problem

    Sub-millimeter assembly remains difficult because success depends on contact outcomes that visual progress and broad task competence do not reliably distinguish.

  • Method

    Facet-0 combines semantic–contact representation learning, joint action–wrench prediction, distributional Action–Wrench Critic valuation, contact-selective refinement, and bounded local adaptation.

  • Results

    82% mean success across five sub-millimeter tasks versus 15% for the strongest baseline, with 0.5 mm placement accuracy and 50 ms command latency.

  • Takeaways & Limitations

    A shared semantic–contact representation supports both global value-guided policy refinement and local transfer under changed part dynamics.

  • Takeaways & Limitations

    The controlled evaluation is limited to wrist-sensed, parallel-gripper electronics assembly on a shared chassis fixture, with broader embodiments and contact regimes left for future work.

Abstract

from arXiv · show

Real-world robotic assembly at sub-millimeter tolerances demands spatial precision, compliant interaction, and robustness to contact failures. We present Facet-0, a robotic foundation model that predicts and values the contact consequences of its actions. Facet-0 unifies multimodal representation learning and reinforcement learning (RL) post-training around a joint action-wrench proposal: a causal wrench history is aligned with vision-language semantics and kinematic state, and flow matching generates each action chunk together with the future wrist-wrench profile it is expected to induce. Deployment rollouts train a distributional Action-Wrench Critic to distinguish motions with similar task progress but different contact outcomes, while phase-aware rewards and contact-selective credit concentrate policy improvement on decisive interactions. To accommodate part-specific dynamics, a lightweight bounded actor reuses the frozen representation for on-robot adaptation; RL remains defined over executable Cartesian actions, while an auxiliary wrench head preserves predictive, non-commanded action-contact coupling. Trained on ManuFacet-1K, a 1,000-hour force-synchronized corpus spanning three embodiments and multiple manufacturing cells, the bounded task-adapted system reaches 82% mean success on five sub-millimeter computer-assembly tasks, compared with 15% for the strongest baseline, with 0.5 mm placement accuracy and 50 ms command latency.

1 Introduction

Facet-0 addresses the final-millimeter challenge in contact-rich assembly by modeling contact as both an anticipated consequence of action and a quantity valued during deployment. Its force-synchronized training and value-guided refinement yield strong precision-assembly performance.

  • Sub-millimeter assembly failures arise because small pose errors can create lateral force, friction, or jamming at contact.
  • Facet-0 aligns vision-language semantics and kinematic state with causal wrist-wrench history to predict future contact consequences.
  • ManuFacet-1K supplies 1,000 hours of force-synchronized demonstrations and deployment rollouts across three embodiments and multiple manufacturing cells.
  • Joint action-wrench flow matching connects task intent to anticipated physical consequences within the policy.
  • Value-guided policy improvement refines decisive contact behavior, while bounded local adaptation reuses the learned representation under changed part dynamics.

2 Related Work

Prior work combines vision-language-action policies, contact sensing, classical force control, real-world reinforcement learning, and broad manipulation datasets. Facet-0 is positioned against gaps in contact-aware valuation and force-synchronized precision-assembly data.

  • Modern VLA policies generalize across scenes and instructions but usually learn motion while treating contact as an observation rather than a consequence.
  • Force-aware methods incorporate wrist wrench, tactile reasoning, reactive force feedback, or force-conditioned flow matching into learned policies.
  • Classical controllers regulate force and position or shape contact dynamics through hybrid, impedance, admittance, variable-impedance, and energy-tank formulations.
  • Real-world RL advances include sample-efficient off-policy learning, human corrections, intervention selection, imitation-plus-offline/online RL, and VLA post-training.
  • ManuFacet-1K contributes approximately 1,000 hours of force-synchronized demonstrations and closed-loop rollouts organized around contact-aware precision manipulation.
  • Large manipulation corpora prioritize behavioral breadth, whereas existing assembly resources provide smaller-scale contact-rich trajectories or benchmarks.

3 The ManuFacet-1K Dataset

ManuFacet-1K is a force-synchronized precision-manipulation corpus combining demonstrations, closed-loop rollouts, shared sensing, and contact-phase annotations. Its collection and curation support phase-aware learning and action-wrench conditioning.

  • ManuFacet-1K contains approximately 1,000 hours of force-synchronized demonstrations and closed-loop rollouts across multiple sites and installation tasks.
  • The corpus comparison retains each source’s native scale because trajectory counts cannot reliably be converted to hours without duration statistics.
  • Its acquisition uses torque-controlled arms, parallel grippers, wrist six-axis force/torque sensing, three RGB cameras, joint proprioception, and force-feedback teleoperation.
  • Each frame combines end-effector pose, gripper opening, and wrench, with 0.5 mm action resolution and VLM-generated sub-task and instruction pairs.
  • Skill-Phase Taxonomy and Statistics: Seven phases—approach, align, insert, press, seat, fasten, and retreat—make temporal interaction regimes addressable for pretraining, reward decomposition, and action-wrench conditioning.
  • Collection and Curation: Collection curation synchronizes modalities, extracts valid segments, samples action chunks by saliency, adds VLM supervision, and retains corrected trajectories.

4 Semantic–Contact Alignment

Facet-0 aligns task semantics with partially observed contact dynamics by encoding visual-language context, robot state, and wrench history together. Flow matching then generates executable Cartesian actions paired with predicted future wrist wrenches.

  • Because wrist force/torque reveals interaction only after execution, Facet-0 models each proposed motion jointly with its anticipated wrench.
  • The base policy combines a PaliGemma vision-language backbone with a flow-matching action expert.
  • Robot state separates end-effector pose and gripper opening from the measured six-dimensional wrist wrench.
  • The latest 10 wrenches encode contact onset, persistent off-axis moments, and rising wrench during stalled motion as temporal events.
  • Vision-language, kinematic-state, and wrench-history encoders are fused into a learned semantic-contact representation.
  • Each proposal row pairs a Cartesian action and gripper command with its anticipated future wrist wrench, but only the action is executable.
  • Conditional flow matching trains proposals by interpolating action-wrench targets with Gaussian noise before inference integrates the learned field back to data.

5 Value-Guided Policy Refinement

Facet-0 refines a shared action–wrench policy by valuing predicted contact consequences and emphasizing decisive interactions. A separate bounded actor adapts the frozen representation locally while executing only Cartesian actions.

  • 5.1 Interaction Valuation: The coarse and refinement experts update the same action–wrench object, with the refinement expert operating near contact.The critic guides the 20 Hz refinement expert using the predicted proposal.
  • 5.1 Interaction Valuation: Phase-aware rewards combine sub-goal completion, completion time, and normalized wrench violations into one per-step signal.The phase-specific wrench envelope treats values above one as violations, while the time budget rewards faster completion.
  • 5.1 Interaction Valuation: The Action–Wrench Critic scores joint action–wrench proposals rather than geometric progress alone, separating clean insertions from visually similar jams.Its distributional value preserves distinct contact-related success and failure modes instead of averaging them into a single unattained mean.
  • 5.1 Interaction Valuation: Contact-selective credit ranks frames within contact and free-space regimes, then tags positive interactions for guided policy refinement.This avoids selecting mostly free-space frames because decisive contact occupies few frames and advances measured progress slowly.
  • 5.3 Local Policy Adaptation: For part-specific transfer, a lightweight actor consumes frozen semantic–contact features and outputs a bounded seven-dimensional Cartesian action plus an auxiliary next-wrench prediction.Only the bounded action is executed; the wrench prediction is supervised as a contact-consequence target and is not a force command.
  • 5.3 Local Policy Adaptation: The local critic evaluates only the executable action, while the auxiliary wrench head remains predictive and non-commanded during compliant control.The force loop uses measured wrist F/T feedback, and safety filtering applies workspace and per-step motion limits before execution.

6 Experiments

Experiments evaluate Facet-0 on precision assembly, deployment post-training, on-robot adaptation, transfer, and wrench-role ablations. Results show contact-critical interactions dominate failures, while contact-selective refinement and bounded adaptation improve success, recovery, autonomy, and intervention outcomes.

  • 6.2 Main Comparison and Component Gains (Q1): 82% mean success across five assembly tasks versus 15% for the strongest baseline, a 5.5× improvement.Per-task gains over the best baseline range from +45 to +75 points.
  • 6.2 Main Comparison and Component Gains (Q1): Free-space pick remains strong, while align and place sub-goals account for the measured performance deficit at contact-critical stages.Displayed pick rows range from 60% to 100%, whereas RAM and GPU alignment reach 95% and 85% for Facet-0 versus 10% for π0.5.
  • 6.3 Post-Training from Deployment Experience (Q2): Contact-selective post-training raises Disk success from 20% to 65% and recovery from 44% to 81% against matched AWR.Human intervention falls from 47% to 24%.
  • 6.4 Adaptation: Paradigms and Few-Shot Transfer (Q3): Both Facet-0 adaptation variants reach the success ceiling, while autonomy rises as supervisor takeovers decrease over adaptation episodes.DSRL remains at the floor; the actor–critic variant ends at full success on both tasks after a dip on RAM insert.
  • 6.4 Adaptation: Paradigms and Few-Shot Transfer (Q3): After an off-axis collision, the adapted policy backs off, re-aligns, and seats the module on a second attempt.The executable correction comes from the action head; auxiliary wrench prediction remains non-commanded.
  • 6.5 Ablation Studies (Q4): Removing prediction, valuation, and adaptation together cuts RAM success by 50 points and increases normalized peak off-axis force from 0.2× to 1.0×.Individual removals differ by only one to two successes in 20 trials, so their ordering is unresolved.

7 Discussion and Limitations

The evaluation supports semantic–contact representation learning, value-guided refinement, and bounded adaptation within a controlled electronics-assembly setting. Broader evaluation across embodiments, parts, and contact geometries remains future work.

  • Scope: The controlled five-task evaluation uses wrist-sensed, parallel-gripper electronics assembly on a shared chassis fixture.The training corpus spans three embodiments and two chassis families.
  • Future work: Future work will broaden evaluation to further end effectors, parts, and contact geometries.The paper also identifies force-conditioned policy distillation for cells without wrist-mounted six-axis sensing.

8 Conclusion

Facet-0 links task semantics with anticipated contact and refines the resulting behavior through value-guided reinforcement learning and bounded local adaptation. On five electronics-assembly tasks, the full model achieves 82% mean success while reducing intervention and increasing recovery relative to a matched baseline.

  • Conclusion: 82% mean success is achieved across five sub-millimeter electronics-assembly tasks, versus 15% for the strongest baseline.The full model also reports 0.5 mm placement accuracy and 50 ms command latency.
  • Conclusion: Value-guided post-training reduces intervention from 47% to 24% relative to the matched AWR baseline.This comparison is reported alongside increased recovery from 44% to 81%.
  • Conclusion: Recovery increases from 44% to 81% relative to the matched AWR baseline after value-guided post-training.The reported intervention reduction is from 47% to 24%.
  • Evaluation setting: The five-task suite uses a shared chassis fixture while exposing distinct contact geometries.LEVER is evaluated as an isolated contact-only CPU socket lever closure rather than a full pick-to-place chain.
  • Evaluation setting: The host-computer assembly suite contains 23 sub-tasks, with fourteen identified as contact-critical.Clearance is defined by the tightest fit among each task’s critical sub-goals.

A.1 Evaluation Protocol

The evaluation uses multi-stage electronics-assembly instructions decomposed into reusable sub-goals, with force-synchronized data collected across embodiments and manufacturing cells. Data curation removes episodes and segments with synchronization, hardware, or visibility problems.

  • Task suite: The five tasks comprise 23 sub-tasks drawn from a shared seven-verb vocabulary.The vocabulary includes pick, hold, rotate, align, place, push, and press, enabling skills learned in one task to be addressed in another.
  • Baselines and variants: Facet-0 compares against generalist backbones and aligned variants that aggregate wrist force/torque history or add semantic–contact alignment and value-guided refinement.The Full variant additionally includes local policy adaptation.
  • Data collection: Each sensing cell includes a torque-controlled arm, parallel gripper, wrist six-axis force/torque sensor, three RGB cameras, and joint proprioception.Kinesthetic teleoperation with force feedback supplies demonstrations, and hardware timestamps establish a common 15 Hz training timeline.
  • Data curation: Episodes with excessive cross-channel skew are rejected rather than interpolated.Teleoperation faults, truncated chunks, sensor saturation, calibration drift, and sustained manipulated-part occlusion are removed at segment level.

C Architecture and Execution Details

The architecture separates predictive action–wrench modeling, contact-rate refinement, compliant control, and bounded local adaptation. Wrench history supplies temporal contact context, while predicted wrench remains a supervised consequence variable rather than an executed command.

  • Wrench representation: A 10-frame wrench window captures contact-onset steps, jams, and persistent off-axis moments that are properties of short trajectories.The window is described as the shortest one over which contact onset is detectable above sensor noise at the corpus sampling rate.
  • Wrench representation: Predicting wrench consequences makes contact information available before execution instead of relying only on measured contact after prior actions.The wrench decoding term rewards representing the contact consequence of proposed motion.
  • Figure 10: Figure 10 presents conceptual distributions for input conditioning, Action–Wrench valuation, and bounded local adaptation.The distributions are intended to illustrate optimization roles rather than report empirical measurements.
  • Execution hierarchy: The joint action–wrench proposal is decoded at 5–10 Hz, then refined at 20 Hz to roughly 0.5 mm resolution.The coarse and fine stages differ in precision, not in kind.
  • Compliant control: A compliant controller runs at 200 Hz, regulating force along task-frame directions and position along the remaining directions.Its force loop closes on measured wrist force/torque feedback.
  • Local adaptation: Local adaptation updates only a bounded actor while keeping the refinement expert frozen.The actor output is confined to a per-task admissible action box before being passed through the task safety mapping.

D Method Details

Facet-0’s method shares a semantic–contact representation across alignment, value-guided refinement, and local adaptation. It jointly predicts executable Cartesian actions and future wrench consequences, then refines contact behavior while preserving predictive coupling.

  • Method architecture: Three method modules share the semantic–contact representation h_c^t: alignment, value-guided refinement, and local policy adaptation.The representation supports a joint action–wrench proposal and is compressed for adaptation to new parts.
  • Optimization: 32K action-branch updates use four action–wrench updates for every one VQA update under alternating optimization.Action and wrench losses use separate masks and normalization, while VQA uses answer-token cross-entropy.
  • Semantic–contact alignment: The action–wrench branch aligns visual-language inputs with kinematic state and causal wrench history before generating a joint proposal.The proposal contains an action chunk and future wrist-wrench predictions, while only the action is executed.
  • Value-guided policy refinement: The Action–Wrench Critic uses distributional returns and auxiliary heads for near-future wrench, contact intensity, within-contact progress, and success ranking.These heads distinguish motions with similar task progress but different contact states, such as gentle alignment versus ramming.
  • Value-guided policy refinement: Contact-selective refinement uses regime-conditioned thresholds and classifier-free-style positive-interaction conditioning to focus policy updates on decisive contacts.Progress-calibrated scores and truncated, undiscounted credit are used to avoid over-rewarding early free-space frames.
  • Local policy adaptation: Local adaptation freezes the representation and safety-related components while adapting only a lightweight actor and twin critics over executable Cartesian actions.The auxiliary wrench head predicts the measured consequence of the applied motion without making wrench a command.

E Reported Per-Sub-Goal Results

The per-sub-goal decomposition separates local contact quality from full-chain completion. For the full system, each task chain is limited by its weakest displayed contact-critical sub-goal.

  • Reported Per-Sub-Goal Results: For the full system, each chain success rate matches its weakest displayed contact-critical sub-goal.Alignment limits RAM, CPU, Disk, and GPU, while the first press limits LEVER.
  • Reported Per-Sub-Goal Results: The decomposition distinguishes local interaction quality from end-to-end reliability without letting longer tasks dominate the diagnostic average.This comparison uses separate contact-event and assembly-level aggregates.

F Supporting Evaluation Data

Supporting evaluation data reports faster deployment, precision and adaptation reference measurements, and behavioral mechanisms for recovery after contact failures. The analysis links wrench observations to corrective actions within the same sub-goal.

  • Supporting Evaluation Data: Facet-0 completes the task 40% faster than expert teleoperation on the same fixtures.The statement is presented alongside deployment-property data moved from the main narrative.
  • Supporting Evaluation Data: Placement accuracy is measured at the align-to-insert transition, while Table 10 reports adaptation values over a two-hour window with a 30-minute checkpoint.Violations cover all training episodes, and a dash indicates that 90% success was not reached.
  • Behavioral analysis: On RAM and CPU, the adapted actor uses off-axis wrench feedback to retreat, correct laterally, and re-enter after visually aligned insertion contacts the slot rim.The correction stays within the same sub-goal and runs at the contact-control rate.
  • Behavioral analysis: During press-and-lock, wrist force distinguishes a stalled misseat from successful seating, enabling the system to flag the jam, re-issue the sub-goal, and complete insertion.Force rises while displacement stalls in the misseat case.
  • Behavioral analysis: Per-sub-task reward evaluation gives recovered trajectories credit for their corrective segment instead of discarding them with the preceding failure.This is described as protected recovery credit in the behavioral analysis.
Loading 2609.01596v1…