PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixture of Flow Matching and Particle Filter for Proprioception-Based Online Contact Estimation

Seo Wook Han, Min Jun Kim

arXiv:2610.29568v1cs.RO

TL;DR

Proprioceptive contact estimation must infer detailed contact states from internal measurements, while existing approaches face limitations in representing multi-modal beliefs and tracking events. ContactFlowFilter combines a particle filter with generative proposals; comparative studies report the highest accuracy and lowest dissimilarity to the target posterior among the baselines. Its validation spans simulation and real-world settings.

  • Problem

    Proprioceptive sensing requires inferring detailed contact states from internal measurements, and particle filters alone have limitations in representing multi-modal beliefs and handling contact events.

    Source: §I · p. 2, F. Limitation of CPF · p. 10

  • Method

    ContactFlowFilter mixes standard particles from particle-filter prediction with dual particles generated by a generative model to address multi-modality, temporal dependency, and contact-event detection.

    Source: §VIII · p. 19

  • Results

    Comparative studies report that ContactFlowFilter outperforms the baselines in contact estimation, achieving the highest accuracy and lowest dissimilarity to the target posterior.

    Source: §VIII · p. 20

  • Takeaways & Limitations

    Generated dual particles can populate high-belief regions missed by particle-filter particles, while generative estimates help mitigate sensitivity in contact-event detection.

    Source: §VIII · p. 19

    The CPF algorithm can converge to only a single mode and therefore cannot properly represent the distribution.

    Source: F. Limitation of CPF · p. 10

Abstract

from arXiv · show

In this article, we propose ContactFlowFilter, a novel approach for contact estimation that determines the number of active contacts, their locations, and the associated force vectors. We consider a robot manipulator equipped with joint torque sensors and a base force/torque sensor; such a proprioceptive-only setup presents a challenging inverse problem characterized by (i) multi-modality from non-unique solutions, (ii) temporal dependencies in state evolution, and (iii) discrete contact events involving contact addition/removal. Existing approaches, such as particle filters (PFs) and learning-based methods, often struggle to address these challenges simultaneously. To overcome this, ContactFlowFilter integrates a generative model into the PF framework. The recursive structure of the PF captures temporal dependencies, while the flow matching model serves as an auxiliary proposal distribution to track multi-modal beliefs and detect contact events. Extensive validation in both simulation and real-world experiments confirms the effectiveness of the proposed method. In real-world settings, ContactFlowFilter achieves the lowest dissimilarity from the target distribution, with errors of only 0.36 cm and 0.88 cm for single- and dual-contact scenarios, respectively, while operating at 0.69 ms and 2.69 ms.

I. INTRODUCTION

Contact estimation from proprioceptive sensors is an ambiguous inverse problem: it must recover contact count, locations, and forces despite multi-modality, temporal dependence, and discrete contact events. ContactFlowFilter combines a particle filter with a flow-matching generative model to address these challenges; real-world experiments report the lowest dissimilarity to the target distribution alongside fast execution.

I. INTRODUCTION

  • Proprioceptive contact estimation infers contact count, locations, and force vectors from sparse internal measurements, with scope limited to up to two point contacts.Distinct contact states can produce identical sensor measurements, so the solution distribution may be multi-modal.Source: §I · p. 3

A. Contributions of the Article

  • ContactFlowFilter combines recursive particle-filter prediction with generative proposals to represent multi-modal beliefs and detect contact events while using temporal context.Its event-detection method combines temporal context with generative estimates.Source: §I · p. 3, A. Contributions of the Article · p. 4
  • A paired-contact representation and spatially aligned kinematics, joint-torque, and base force/torque inputs are processed by a transformer to extract cross-modal correlations.The fixed-size paired-contact representation supports both single- and dual-contact cases.Source: A. Contributions of the Article · p. 4
  • Flow matching provides the generative backbone for few-step inference using linearized probability paths, alongside a motion model built from pre-processed mesh data.The authors select these components to support real-time performance and efficient motion modeling.Source: A. Contributions of the Article · p. 4
  • 0.36 cm localization error and 0.69 ms average execution time for single-contact scenarios, and 0.88 cm and 2.69 ms for dual-contact scenarios, accompanied the lowest real-world dissimilarity to the target distribution.The paper also reports validation in simulation and real-world settings.Source: A. Contributions of the Article · p. 4

II. RELATED WORK

Proprioception-based contact methods trade off temporal reasoning, multimodal estimation, and force recovery. The paper motivates advancing particle filters while addressing their contact-event threshold sensitivity.

II. RELATED WORK

  • Contact estimation distinguishes detection and isolation from localization and estimation, the latter seeking contact position and force together.The section focuses on proprioception-based localization and estimation from external force/torque values attributed to contacts.Source: §II · p. 5

A. Model-Based Approaches

  • The line-of-force method gives a deterministic single-contact solution; dual-contact use in relies on redundant base and end-effector force/torque sensing, not temporal reasoning.Its geometric construction is limited to single-contact scenarios without that redundant sensing setup.Source: A. Model-Based Approaches · p. 5
  • Particle filters incorporate temporal information but greedy resampling can cause particle starvation, collapsing the set onto one high-belief region and losing multimodality.Non-greedy alternatives, handle multimodality, but extending them beyond one contact is not straightforward.Source: A. Model-Based Approaches · p. 5
  • Existing multi-contact PF methods depend on a sensitive threshold to manage particle sets, risking false positives or missed contacts as contacts change.The paper contrasts this with a particle-set management scheme using flow-matching output.Source: A. Model-Based Approaches · p. 5

B. Learning-Based Approaches

  • Learning-based localization methods estimate contact probability, classify predefined surface points, or regress location, but generally do not estimate force or reason temporally.The diffusion method assumes the first contact remains fixed; ContactFlowFilter combines generative modeling with PF inference and reports 15.97 ms → 2.69 ms inference.Source: B. Learning-Based Approaches · pp. 5–6

A. Particle Filter

  • The paper describes the particle filter as a non-parametric Bayes filter that estimates the posterior over robot states from sensor data using a finite set of samples.Its recursion relies on the Markov assumption and state evolution without control input in this setup.Source: A. Particle Filter · p. 6

1) Sampling:

The sampling procedure proposes and weights particles to approximate the belief, while dual-sampling mixes observation-based proposals with recursive PF proposals. Flow matching provides a way to generate samples along a learned probability path.

1) Sampling:

  • The PF sampling step draws particles through a proposal distribution, then uses the measurement model to weight them according to the target distribution.The importance weight is the target-to-proposal ratio.Source: 1) Sampling · p. 6

2) Dual-Sampling:

  • Dual-sampling draws particles from observations alone, without recursion, and assigns weights using a form matching the motion-model distribution.Its weight evaluates a pair of particles rather than a single particle.Source: 2) Dual-Sampling · p. 6

3) Importance Resampling:

  • Importance resampling draws a new particle set from the weighted particles so the belief is represented by particles without retaining weights.Source: 3) Importance Resampling · p. 7

B. Flow Matching Model

  • Flow matching connects Gaussian noise to the data distribution through a probability path and generates samples by integrating a learned vector field with an ODE.The path endpoints are p[0] = N(0, I) and p[1] approximately the target distribution.Source: B. Flow Matching Model · p. 7

C. Contact Induced Sensor Measurements

The paper relates contact locations and forces to proprioceptive measurements, estimates external torques and base wrench, and frames contact localization as a belief over points conditioned on observations over time.

C. Contact Induced Sensor Measurements

1) External Joint Torque Estimation:

  • A momentum-based disturbance observer estimates external joint torque from the flexible-joint robot dynamics and measured joint torque.Its output is a first-order low-pass filtered estimate of actual external torque.Source: 1) External Joint Torque Estimation · p. 7

2) External Base Wrench Estimation:

  • The estimated external base wrench is obtained by subtracting a nominal contact-free wrench, computed with RNEA, from the base force/torque sensor measurement.The nominal acceleration is calculated by removing the estimated external joint torque from robot dynamics.Source: 2) External Base Wrench Estimation · p. 7

IV. PARTICLE FILTER FOR CONTACT ESTIMATION

  • The particle-filter section introduces notation for particle sets and states that the baseline PF motivates the subsequent method.Source: §IV · p. 7

A. Problem Statement

  • The contact-localization belief is a distribution over contact points conditioned on observations up to the current time, and the PF approximates it with particle sets.The measurement model simultaneously addresses force identification; the setup retains observability at Jacobian rank deficiency using the base force/torque sensor.Source: A. Problem Statement · pp. 7–8

B. Motion Model

The motion model evolves contact particles over the robot surface, using mesh-based representation and geodesic distances to avoid off-surface samples and costly projection.

B. Motion Model

  • A random-walk Gaussian motion model perturbs the prior particle, but samples may fall off the robot surface and require additional processing.The proposed alternative samples directly on the surface.Source: B. Motion Model · p. 8

1) Mesh Preprocessing:

  • Particles are represented by link and face indices on a mesh whose non-contact faces are removed and whose remaining faces are remeshed uniformly.The described mesh preprocessing uses a target edge length of approximately 0.36 cm.Source: 1) Mesh Preprocessing · p. 8

2) Approximating the Gaussian on the Surface:

C. Measurement Model

The measurement model scores contact-point particles by how well their force solution explains the observed wrench and joint torques. QP-based weights and particle-set management use this fit to update beliefs and adjust the number of contact hypotheses.

1) QP Formulation:

  • The QP identifies contact forces that best explain the observation for given contact-point locations, subject to a pushing-only friction-cone constraint.The friction cone is approximated by a polyhedral cone to make the optimization tractable.Source: 1) QP Formulation · p. 9

2) Importance Factor:

  • A particle with smaller QP error is treated as more probable, with an exponential weight whose sensitivity is controlled by α.The force-identification QP takes candidate contact points as inputs; the particle weight is based on the resulting fit.Source: 1) QP Formulation · p. 9, 2) Importance Factor · p. 9
  • For multiple contacts, particle weights are evaluated jointly while holding representative particles from the other sets fixed, reducing the combinations that must be tested.This relies on the assumption that the other particle sets have converged to their true contact points; random representatives are sampled according to weight.Source: 2) Importance Factor · p. 9

D. Particle Set Management

The contact particle filter maintains one particle set per estimated contact, adding or removing sets according to whether the current observation can be explained. The conventional filter has limitations in multimodal beliefs, cross-link motion, and contact changes; the proposed method combines flow matching with the PF and introduces robust particle-set management.

D. Particle Set Management

  • Particle-set management adds a uniformly initialized set when the current collection cannot explain the observation, and removes a set when the observation remains explainable without it.The method uses a QP error threshold to make these decisions.Source: D. Particle Set Management · p. 9

E. Contact Particle Filter

  • The CPF recursively updates particle sets as contact-point beliefs, and derives each contact force from the QP solution used to update that set’s particle weights.Each set corresponds to one contact point.Source: E. Contact Particle Filter · pp. 9–10

F. Limitation of CPF

  • Particle starvation can trap the CPF in a local minimum or prevent it from representing multiple belief modes; its motion model also cannot move particles across links.The cross-link limitation means particles absent from the actual contact link cannot reach its high-belief regions.Source: F. Limitation of CPF · p. 10

2) Particle Drift:

  • When contact count changes, low-pass adaptation of the external-wrench estimate can cause particle drift, delaying or preventing event detection and potentially violating the particle-set convergence assumption.Lowering the detection threshold may reduce delay but can cause false positives.Source: 2) Particle Drift · p. 10

V. MIXTURE OF FLOW MATCHING AND PARTICLE FILTER

  • The proposed method integrates flow matching into the PF through dual-sampling, using the PF’s temporal prediction alongside a generative proposal to represent multimodal beliefs.It also uses temporal context and generative estimates for contact-event detection.Source: §V · p. 10

A. Contact Dataset

The flow-matching model uses paired contact-point samples for up to two contacts and observations aligned along the robot’s kinematic chain. This representation keeps the model input size fixed while encoding contact count and spatial relations.

A. Contact Dataset

  • The dataset represents each example with contact-point locations and their corresponding observations, using a fixed-size paired-point tensor for one or two contacts.For one contact the point is duplicated; for two contacts each row contains the pair.Source: A. Contact Dataset · p. 11
  • The paired representation keeps input dimension constant across one- and two-contact cases, provides samples for each contact distribution, and lets the model infer contact count from point relationships.The representation therefore supports contact-event detection without changing the model’s input size.Source: A. Contact Dataset · p. 11

2) Observation:

  • Observations align each link’s pose and virtual wrench in that link’s frame, forming a sequence along the kinematic chain for the network.The virtual wrench projects external wrench and joint-torque information into the corresponding link frame.Source: 2) Observation · p. 11

B. Neural Network Model

  • The paired-point representation encodes contact count directly, unlike a per-contact-row representation that requires a separate classifier to distinguish single from dual contact.The paired samples are processed by the flow-matching network described in the neural-network section.Source: 2) Observation · p. 11, B. Neural Network Model · p. 12

1) Global Feature Encoder:

The flow-matching network combines linkwise observation features with surface-relative point features, then predicts a vector field conditioned on both. Generated samples are allocated to PF particle sets and weighted for consistency with the previous belief.

1) Global Feature Encoder:

  • The global encoder stacks each link’s pose and virtual-wrench features, applies self-attention across links, and average-pools them into one observation embedding.The number of links sets sequence length, while each link’s features form its embedding.Source: 1) Global Feature Encoder · p. 12

2) Pointwise Feature Encoder:

  • The pointwise encoder embeds each query point using its distances and directions to link surfaces, then combines the surface embedding with the point embedding.This yields one feature vector per query point.Source: 2) Pointwise Feature Encoder · p. 12

3) Decoder:

  • The decoder uses MLPs with skip connections and FiLM conditioning, with modulation determined by the observation embedding and a Fourier encoding of flow-matching time.The resulting affine modulation is applied to each pointwise feature.Source: 3) Decoder · p. 12

C. Dual-Sampling

  • Generated samples are assigned to current PF particle sets by proximity to the previous time step’s sets, and dual-particle weights favor consistency with the prior motion model.The flow-matching output is treated as a sample from the dual-proposal distribution.Source: C. Dual-Sampling · p. 13

D. Flow-Guided Particle Set Management

Flow-Guided-Management combines QP criteria with the dual-particle distribution to adjust contact particle sets. ContactFlowFilter also combines temporal prediction and generative sampling, with an implementation and training pipeline designed for fast inference.

D. Flow-Guided Particle Set Management

  • Contact-set addition requires both the QP condition and dual-particle spread exceeding the motion model’s standard deviation; removal requires no dual-particle allocation and QP support.The method combines the conventional QP criterion with generative estimates, using a logical AND to avoid erroneous initialization.Source: D. Flow-Guided Particle Set Management · p. 13

E. ContactFlowFilter

  • ContactFlowFilter combines temporal prediction and dual-sampling, merges the resulting particles, resamples each set to M particles, then adjusts the number of sets to match contact count.Each set may contain varying numbers of standard and dual particles before importance resampling.Source: E. ContactFlowFilter · p. 13

A. Dataset for Dual-Sampling

  • The flow-matching model is trained on 102.4M simulated data points, with 70% dual-contact cases empirically giving optimal performance.The dataset uses one or two forces of 10–25 N and adds sensor noise to facilitate sim-to-real transfer.Source: A. Dataset for Dual-Sampling · p. 13

B. Training Details for Dual-Sampling

  • The shortcut flow-matching model uses K = 128 and two ODE steps for inference after training for 5M steps on a batch size of 1024.Training took approximately 66 hours; the network has 18.1M parameters.Source: B. Training Details for Dual-Sampling · p. 13

C. Asynchronous Sampling and Execution Time

A. Validation of the Proposed Architecture for Dual-Sampling

  • Experiments randomize robot configuration and contact location across the manipulator surface, and Experiment A compares dual-sampling against a baseline and Flow Matching against DDIM.Experiment A evaluates sample quality and computational efficiency across different ODE-step counts.Source: §VII · p. 14, A. Validation of the Proposed… · p. 14

1) Metrics:

The experiments evaluate generated samples and particle-set management using RMSE and event-detection measures. Results favor the proposed architecture and management method, while qualitative trials identify particle starvation, drift, and adjacent-contact ambiguity as relevant failure modes.

1) Metrics:

  • M-RMSE measures coverage of ground-truth contact points, whereas C-RMSE measures convergence to each corresponding contact point.The evaluation uses two variants of root mean squared error for generated samples.Source: 1) Metrics · p. 14

2) Results and Analysis:

  • The proposed architecture has lower RMSE than the baseline with either Flow Matching or DDIM, while the baseline’s high C-RMSE reflects failure to converge to ground-truth contacts.The baseline cannot encode contact count, leading to single-versus-dual-contact misclassification and samples spread away from a single contact location.Source: 2) Results and Analysis · p. 14, Table II · p. 15
  • Two-step Flow Matching achieves M-RMSE of 0.17 cm for single contact and 0.72 cm for dual contact, matching or surpassing 16-step methods at one-eighth the integration time.Its inference takes 2.57 ms; the Flow Matching trajectories are more linear than DDIM’s, preserving consistency between 2- and 128-step results.Source: 2) Results and Analysis · pp. 14–15, Figure 13 · p. 14, Table II · p. 15

B. Performance Evaluation of Particle Set Management

  • Experiment B compares proposed and baseline contact-event detection over 2,000 trials, each with a contact added at 1 s and removed at 2 s.Success Rate measures matching particle-set count to contact count during each phase; Adjustment Delay and Fault Rate measure event response and mismatches.Source: B. Performance Evaluation of Particle… · p. 15, 1) Metrics · p. 15

2) Quantitative Analysis:

  • The proposed method’s Fault Rate is 0.75%, versus 16.5% for the baseline, and its Success Rate is higher by 2.3% in Phase 1 and 8.1% in Phase 2.Adjustment Delay is comparable in Phase 1 and markedly reduced in Phase 2.Source: Table III · p. 15, 2) Quantitative Analysis · p. 15

3) Qualitative Analysis:

  • Baseline particle starvation can trigger erroneous second-set initialization and subsequent removal failure; the proposed method avoids this by generating particles in high-belief regions.This scenario accounts for less than 2.7% of Phase 1 errors, but false-positive set addition is the primary reason for the baseline’s higher Fault Rate.Source: 3) Qualitative Analysis · pp. 15–16, Figure 15 · p. 16
  • Particle drift can prevent the baseline’s QP-based removal condition from identifying which set to remove, while dual particles support management despite drift.The comparison is shown for contact addition and removal phases.Source: 3) Qualitative Analysis · p. 16, Figure 16 · p. 16
  • Both methods can fail when adjacent contacts are indistinguishable from a single resultant force, with approximately 90.9% of proposed-method Phase 1 failures occurring on adjacent links.This ambiguity is attributed to sparse proprioceptive sensing.Source: 3) Qualitative Analysis · pp. 16–17, Figure 17 · p. 16

C. Comparative Study of Contact Estimation Algorithms

Experiment C compares ContactFlowFilter with CPF, MCP-EP, and a dual-particle-only baseline. It evaluates particle-distribution dissimilarity and contact-state estimation error using likelihood-based and RMSE measures.

C. Comparative Study of Contact Estimation Algorithms

  • The comparison includes CPF, MCP-EP, and Dual-Particle only, which respectively vary the motion model, replace dual-sampling with random sampling, or use generated dual particles alone.CPF’s projection-based motion model takes 33.93 ms and 69.08 ms for single- and dual-contact scenarios; MCP-EP matches the proposed method’s 0.69 ms and 2.69 ms.Source: C. Comparative Study of Contact… · p. 17

1) Metrics:

  • For dual contacts, the likelihood evaluates the second contact over the robot surface while fixing the first contact at its ground-truth location.This setup assesses temporal dependency by treating the first contact as known historical information.Source: 1) Metrics · p. 17, Figure 18 · p. 17
  • Particle log-Likelihood Score measures whether particles lie in high-likelihood regions, whereas Earth Mover’s Distance measures the cost of transporting one distribution into another.PLS can appear favorable when particles occupy only one mode; EMD requires particles to populate all modes to minimize the metric.Source: 1) Metrics · p. 17
  • Estimation error is assessed with M-RMSE and W-RMSE for particle-set locations and contact forces.W-RMSE measures the highest-weight particle’s error relative to ground truth.Source: 1) Metrics · pp. 17–18

2) Quantitative Analysis:

ContactFlowFilter outperforms baselines in simulation and real-world evaluation, while combining recursive particle-filter updates with flow-matching proposals to address multimodality, temporal dependence, and contact events.

2) Quantitative Analysis:

  • ContactFlowFilter outperforms baselines across all metrics in simulation, with the lowest PLS, EMD, and contact-point and force RMSE, and demonstrates superior performance in real-world evaluation.The paper’s conclusion also reports highest accuracy and lowest dissimilarity to the target posterior across simulation and real-world studies.Source: 2) Quantitative Analysis · p. 18, D. Real-World Experiment · p. 19, §VIII · p. 20

3) Qualitative Analysis:

  • ContactFlowFilter places particles in both high-belief regions of multi-modal single-contact distributions, while CPF particles converge to one region.Flow-matching samples outside high-belief contact regions can occur because predicted contact points are not constrained to the robot surface; low weights suppress them during dual sampling and the QP-based measurement update.Source: 3) Qualitative Analysis · p. 18
  • When a second contact occurs, drift in CPF’s first particle set can cause cascading error and leave the second set outside its target distribution’s high-belief region.CPF’s measurement-weight update assumes other particle sets have converged to their true contact points.Source: 3) Qualitative Analysis · pp. 18–19, Figure 20 · p. 18

D. Real-World Experiment

  • The real-world evaluation transferred the simulation-trained flow-matching model and used a dedicated ground-truth dataset from 200 trials across 20 robot configurations.The trials applied two sequential contacts; ground-truth contact locations and forces were acquired with a specialized tool.Source: D. Real-World Experiment · p. 19

VIII. SUMMARY, CONCLUSION, AND DISCUSSION

  • ContactFlowFilter combines recursive PF belief updates with generative-model dual particles to address temporal dependence, particle starvation, and contact-event detection.The generative estimates also help the approach remain robust to particle drift that can cause contact-state tracking failure.Source: §VIII · p. 19
  • The flow-matching model uses a linearized probability path, with a spatially aligned representation and tailored transformer architecture for rapid estimation and multi-modal contact-state modeling.These capabilities are described as supporting high-fidelity temporal tracking.Source: §VIII · pp. 19–20
  • The current implementation estimates at most two contacts because its generative component uses a fixed-size training-sample array capped at two contacts.Beyond two contacts, single-observation estimation is ill-posed because distinct contact states can yield identical sensor measurements.Source: §VIII · p. 20

Related papers

Loading 2610.29568v1…