PaperlayerView exact revision on arXiv

Source-linked AI summary

HINT-Blimp: Human INTent Inference from Multimodal Cues for Robotic Blimps

Subhadeep Koley, Benjamin Greenberg, Yifei Simon Shao, Juan Aceros, Nadia Figueroa, David Saldaña

arXiv:2609.27154v1cs.RO

TL;DR

Traditional interfaces can require attention and add latency, while brief human cues may leave goal and motion ambiguous. HINT-Blimp represents intent as an LDS and estimates it online from multimodal cues with a particle filter. It achieves 86% goal identification and produces curved trajectories around obstacles known only to the human.

  • Problem

    Traditional interfaces can add latency and require operators’ attention, while short physical and verbal cues may be ambiguous about the intended goal or motion.

    Source: §I · p. 1

  • Method

    HINT-Blimp represents intent as an LDS encoding goal and motion behavior, then estimates its parameters online by combining physical pushes and directional cues in a particle filter.

    Source: §I · p. 2, §III · p. 2

  • Results

    86% goal identification was achieved by fusing multimodal cues, and the inferred dynamical systems produced curved trajectories that avoid obstacles known only to the human.

    Source: §I · p. 2

  • Takeaways & Limitations

    Experiments on a physical blimp showed that combining pushes and voice commands improves on either alone, while the LDS eigenvalue parameterization lets the filter infer curved trajectories around obstacles known only to the human.

    Source: §VI · p. 7

Abstract

from arXiv · show

In human-robot interaction, traditional interfaces such as joysticks and handheld tablets introduce latency into navigation tasks and require the operator's explicit attention on the device, instead of the robot. We propose a new human-robot interaction framework in which a human communicates intent directly through sparse multimodal signals such as physical pushes and spoken commands. Human intent is represented as a parameterized linear dynamical system (LDS) that encodes the desired goal and motion behavior. The robot estimates this intent (parameters) online using a particle filter, where each particle represents a candidate LDS hypothesis and is reweighted online as new information becomes available. We validate this framework on a robotic blimp, whose inherent compliance and collision tolerance make it well-suited for repeated physical interaction. Experiments with multiple participants across 300 trials show that combining pushes and voice commands identifies the intended goal in 86% of trials within at most five interactions, with most trials resolved in two. The inferred dynamical systems can also produce curved trajectories that avoid obstacles known only to the human.

I. INTRODUCTION

Sparse pushes and verbal cues can be ambiguous, motivating a robot that treats each as partial evidence about a human’s intended goal and motion. HINT-Blimp combines those cues in an online intent-inference framework, validated on a blimp, while its LDS representation supports curved trajectories.

I. INTRODUCTION

  • Traditional interfaces can add latency and demand attention, while short physical or verbal cues are often ambiguous across goals and motion behaviors.The framework treats each cue as partial evidence rather than requiring a complete command.Source: §I · p. 1
  • Prior aerial-robot touch methods primarily treated force as compliant control input, leaving high-level intent from sparse physical cues relatively underexplored for mobile robots.Blimps’ low-speed dynamics, compliance, and collision tolerance suit repeated physical interaction.Source: §I · pp. 1–2
  • 86% goal identification was achieved by fusing multimodal cues, and inferred dynamical systems produced curved trajectories that avoid obstacles known only to the human.The framework was validated on a physical blimp with multiple participants.Source: §I · p. 2

II. PRELIMINARIES

  • The LDS represents goal-directed motion; its eigenstructure can yield curved trajectories, while the paper restricts the matrix to the stable-node case.The illustrated LDS examples vary parameters (ϑ1, ϑ2, r).Source: §II · p. 2, Figure 2 · p. 2

III. PROBLEM STATEMENT

The robot must infer both the intended destination and the desired motion behavior from sparse, ordered interaction cues. The formulation represents intent as an LDS and combines scene priors with pushes and directional hints.

III. PROBLEM STATEMENT

  • Intent comprises a candidate goal and LDS parameters, with the goal encoding where to go and A encoding how to get there.The robot infers these from an ordered sequence of observations and follows the estimated LDS.Source: §III · pp. 2–3
  • Scene context supplies candidate goal locations and prior plausibilities, which encode background knowledge about relevant destinations.The prior probabilities sum to one.Source: §III · p. 2
  • Physical pushes provide motion evidence, while speech or gestures provide directional hints relative to the blimp’s heading.The modalities may arrive in any order and at any time.Source: §III · pp. 2–3

IV. INFERRING HUMAN INTENT

The framework incrementally estimates intent parameters from each new observation using a particle filter. It parameterizes the LDS matrix through its eigendecomposition to enforce stability and structure the search.

IV. INFERRING HUMAN INTENT

A. Eigenvalue Parameterization

  • Each LDS matrix is parameterized by eigenvector angles and a relative contraction rate, rather than by its four entries directly.Fixing one eigenvalue and requiring a minimum angular separation enforces stability and avoids nearly parallel eigendirections, yielding a lower-dimensional, structured search space.Source: A. Eigenvalue Parameterization · p. 3

B. Particle Filter for Intent Estimation

The particle filter maintains a weighted belief over candidate LDS intent hypotheses and updates it as interactions arrive. Its model treats intent as nearly static between interactions and uses modality-dependent observation likelihoods.

B. Particle Filter for Intent Estimation

  • A weighted set of particles represents the robot’s belief over candidate intent hypotheses.Each particle is an LDS hypothesis with an importance weight.Source: B. Particle Filter for Intent Estimation · p. 3
  • The filter iterates through process-model sampling, observation-likelihood weighting, and resampling based on particle weights.The likelihood depends on the observation modality.Source: B. Particle Filter for Intent Estimation · p. 3
  • Intent is assumed static between interactions, so the process model is a near-static random walk; particles are initialized from the scene prior over candidate goals.Each particle stays assigned to one candidate goal throughout.Source: B. Particle Filter for Intent Estimation · p. 3

1) Process Model (Line 2)

Intent is treated as nearly static between interactions, while the goal remains fixed and uncertainty evolves over motion-policy shape parameters.

1) Process Model (Line 2)

  • Between interactions, the intent follows a near-static evolution model.The process model has no control input.Source: 1) Process Model (Line 2) · p. 3
  • The goal component stays fixed at its initialized candidate value, so goal belief changes through reweighting and resampling.Goals form a discrete hypothesis set.Source: 1) Process Model (Line 2) · p. 3
  • Process noise perturbs the motion-policy shape components and is scaled by one minus the push-dependent confidence from the previous filter step.The stated shape components are (ϑ1, ϑ2, r).Source: 1) Process Model (Line 2) · p. 3

2) Observation Likelihood (Line 3)

The observation likelihood incorporates physical pushes and heading-relative directional hints, alongside a scene prior over candidate goals. Push alignment also informs subsequent exploration of motion-policy shape.

2) Observation Likelihood (Line 3)

  • Scene context initializes particles in proportion to prior plausibility across candidate goals, with the prior representing knowledge available before human cues.The prior may come from scene understanding, task context, or operator-specified destinations.Source: 2) Observation Likelihood (Line 3) · pp. 3–4
  • Push trajectories arrive incrementally over window τ and are scored by cosine alignment between observed and LDS-predicted velocities, normalized by sample count.The increments cover the push segment without overlap, and their total log-likelihood is proportional to mean cosine similarity over the complete segment.Source: 2) Observation Likelihood (Line 3) · p. 4
  • Push confidence is mean cosine alignment at the current estimate; well-explained pushes suppress later shape exploration, while poorly explained pushes retain more process noise.Only physical pushes provide kinematic evidence about (ϑ1, ϑ2, r); scene context and directional cues constrain only the goal.Source: 2) Observation Likelihood (Line 3) · p. 4
  • Spoken commands and pointing become heading-relative direction vectors processed as the same evidence type, and the formulation is not limited to named commands.The experiments use four voice commands: straight, right, left, and back.Source: 2) Observation Likelihood (Line 3) · p. 4
  • Directional hints receive a Gaussian score based on angular difference from the commanded bearing, with poorly aligned goals downweighted rather than excluded.A directional hint enters the filter once, when decoded, and is not reapplied thereafter.Source: 2) Observation Likelihood (Line 3) · p. 5

3) Resampling (Line 8)

Systematic resampling replicates particles that better explain the observations and discards low-weight particles, while preserving representation of every candidate goal.

3) Resampling (Line 8)

  • After each evidence-bearing update, systematic resampling draws a new particle set from the weighted distribution, making better-explaining particles more likely to be replicated.Resampling accumulates evidence; entering particles carry uniform weights, so posterior belief is encoded by particle density in parameter space.Source: 3) Resampling (Line 8) · p. 5
  • A floor of m particles per candidate goal is enforced after resampling to prevent depletion of the discrete goal set.The induced uniform component Km/N is described as negligible relative to the evidence.Source: 3) Resampling (Line 8) · p. 5
  • Without the floor, one misleading cue could extinguish the true goal’s particle cluster, leaving no mechanism for recovery.Source: 3) Resampling (Line 8) · p. 5

4) Commitment and Execution (Line 6)

The filter commits after the current observation is complete, selecting a goal by its cluster’s total weight and taking motion-policy shape from that cluster’s highest-weight particle.

4) Commitment and Execution (Line 6)

  • For a push, commitment waits until the final increment of the τ = 5 s window; for a directional hint, it occurs when the hint is decoded.Source: 4) Commitment and Execution (Line 6) · p. 5
  • Goal commitment selects the candidate whose cluster has the largest total weight, representing that goal’s posterior probability.The cluster population records past evidence, while particle weights record the current cue.Source: 4) Commitment and Execution (Line 6) · p. 5
  • The motion-policy shape comes from the highest-weight particle in the selected goal cluster, and the robot follows the resulting LDS policy until another interaction arrives.The policy matrix A* is reconstructed via equation (4).Source: 4) Commitment and Execution (Line 6) · p. 5

V. EXPERIMENTS AND RESULTS

The experiments used a physical SBlimp, with intent inference running off-board and velocity commands sent to its onboard controller. Trials assumed the robot lacked obstacle knowledge and counted success only if it reached the target in fewer than six interactions.

V. EXPERIMENTS AND RESULTS

  • The SBlimp converts the inferred LDS into desired translational velocity, using its pendulum-like stability to avoid an attitude loop.The system is localized with OptiTrack, and commands are sent through crazyswarm over crazyradio.Source: §V · p. 5
  • The particle filter runs on a CPU, detects pushes from motion-capture velocity, and processes voice and gesture inputs on the workstation.Only the velocity controller runs on the blimp itself.Source: §V · p. 5
  • The robot has no obstacle knowledge, and a trial succeeds only if it reaches the predesignated goal in fewer than six interactions.Experiments 1 and 2 use a uniform prior over candidate goals; the evaluated modalities are pushes and voice commands, not gestures separately.Source: §V · p. 5

A. Experiment 1: Goal Identification and Modality

The experiment tested goal identification across three interaction modalities, using randomly selected targets among eight candidate goals. Combined pushes and voice commands produced a two-stage interaction pattern, while participants reported favoring that combination.

A. Experiment 1: Goal Identification and Modality

  • Five participants completed 60 trials each, conveying randomly selected targets among eight goals through push-only, voice-only, or freely combined input.The setup is shown in Fig. 3.Source: A. Experiment 1: Goal Identification… · pp. 5–6
  • 37/100 voice-only trials failed; combined successes clustered at two interactions, while push-only successes often resolved in one but ambiguous pushes required physical correction.The authors interpret the combined pattern as a push establishing coarse direction followed by voice disambiguating among remaining goals.Source: A. Experiment 1: Goal Identification… · p. 6
  • All five participants rated ease and confidence at least 4/5, four preferred combined input, and all reported using a push-then-voice strategy.Participants identified central Goals 3 and 6 as difficult because of their proximity to neighboring goals.Source: A. Experiment 1: Goal Identification… · p. 6

B. Experiment 2: Curved Trajectories Around Obstacles

The experiments examined curved obstacle-avoiding trajectories and sensitivity to goal priors, then concluded that multimodal evidence supports intent inference on a physical blimp. The study’s future work includes scaling to more participants and complex environments.

B. Experiment 2: Curved Trajectories Around Obstacles

  • The obstacle experiment used three goals and one obstacle, with participants guiding the blimp to a goal whose straight-line path was blocked.The inferred dynamics can curve around obstacles without the human explicitly communicating their location.Source: B. Experiment 2: Curved Trajectories… · p. 6
  • A push toward Goal 1 followed by a rightward voice command led the filter to commit to Goal 2 and execute a curved path around the obstacle.The human knew the obstacle layout, while the robot did not; the straight-line route to Goal 2 would have collided.Source: B. Experiment 2: Curved Trajectories… · p. 6

C. Experiment 3: Importance of priors

  • 1.88 interactions under the adversarial prior matched the uniform baseline’s 1.75, while the helpful prior required 1.45 interactions in simulation.Each condition had 40 trials; the authors caution that absolute success rates and convergence speeds may differ in real-world trials.Source: C. Experiment 3: Importance of priors · p. 7, Figure 6 · p. 7

VI. CONCLUSION

  • The framework represents intent as an LDS and estimates its parameters online with a particle filter; physical blimp experiments combined pushes and voice commands, and inferred curved obstacle-avoiding trajectories.The conclusion attributes the curved trajectories to the eigenvalue parameterization.Source: §VI · p. 7
  • The authors identify larger participant samples and more complex environments as future study needs.Source: §VI · p. 7
Loading 2609.27154v1…