PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACER: Adaptive Multi-Robot Social Navigation via Joint Human-Response Prediction and Interaction-Aware Replanning

Lan Hu, Minghui Liwang, Wenbo Zhu, Xinlei Yi, Wei Gong, Yiguang Hong, Seyyedali Hosseinalipour

arXiv:2609.18776v1cs.RO

TL;DR

Multi-robot navigation methods often address action-conditioned prediction, planning, and online adaptation separately, despite the interaction between robot motions and entity responses. TRACER jointly predicts per-entity responses for candidate team plans and updates persistent identity-bound beliefs from executed interactions. Experiments report improved response prediction and replanning, along with better collision-free completion than an independent-robot baseline on SocialGym2.

  • Problem

    Existing work often treats action-conditioned prediction, multi-robot planning, and online adaptation separately, rather than jointly modeling robot–entity interactions and adapting decisions from executed outcomes.

    Source: A. Motivation · p. 1

  • Method

    TRACER predicts per-entity responses for candidate joint trajectories, then updates persistent identity-bound response-mode beliefs from executed robot prefixes and synchronized entity observations.

    Source: A. Motivation · p. 2

  • Results

    Experiments report more accurate non-additive interaction prediction, improved response prediction and downstream replanning from persistent evidence, and improved collision-free completion over GoAlone on SocialGym2.

    Source: A. Motivation · p. 2, C. Backward Coupling: Retaining… · p. 7, D. From Belief to Action:… · p. 7

  • Takeaways & Limitations

    The reported results support using joint interaction modeling and persistent evidence from executed interactions in closed-loop multi-robot social navigation.

    Source: §V · p. 8

Abstract

from arXiv · show

Multi-robot navigation in human-shared spaces is inherently interactive: coordinated robot motions influence how nearby entities respond, while those responses provide valuable information for subsequent robot decisions. However, existing methods typically address action-conditioned prediction, multi-robot planning, or online adaptation separately, and therefore lack a unified mechanism for modeling joint robot-entity interactions and adapting future decisions from executed interaction outcomes. To address this gap, we propose TRACER, a bi-directional receding-horizon framework that closes the loop between prediction and adaptation. TRACER evaluates candidate (i.e., alternative feasible future motion plans for the robot team) trajectories using a per-entity probabilistic response model that separates individual-robot effects from non-additive pairwise interactions; after executing the selected trajectory prefix, it updates persistent identity-bound beliefs over latent response modes using the synchronized observed responses. These updated beliefs then guide subsequent candidate evaluation under probabilistic safety and response-aware cost criteria. Experiments show that (i) TRACER more accurately captures non-additive multi-robot interaction effects than a capacity-matched additive predictor, (ii) persistent identity-consistent evidence improves response prediction and downstream replanning, and (iii) the complete TRACER framework improves collision-free completion over an independent-robot baseline on the SocialGym2 multi-robot social-navigation benchmark.

I. INTRODUCTION

TRACER frames multi-robot navigation as a closed loop: candidate joint plans shape entity responses, and observed executed interactions inform later decisions. It combines per-entity response prediction, persistent identity-bound beliefs, and candidate evaluation, addressing capabilities that prior work often treated separately.

I. INTRODUCTION

  • TRACER targets future navigation in which humans and embodied agents adapt to one another over repeated interactions.The introduction describes this as a closed-loop process that may reshape walking habits, yielding conventions, and perceived safety and risk.Source: §I · p. 1

A. Motivation

  • Prior work covers action-conditioned prediction, multi-robot planning, and online belief updates, but these capabilities are largely addressed in isolation.The paper identifies the need to jointly model how robot plans influence entity responses and how observed responses inform subsequent decisions.Source: A. Motivation · p. 1
  • TRACER’s experiments support joint-response modeling and identity-consistent evidence, and report improved collision-free completion over GoAlone on SocialGym2.The framework evaluates candidate joint trajectories using predicted entity responses, then updates identity-bound posteriors from executed interactions to shape future selection.Source: A. Motivation · p. 2

B. Literature Review

  • Related action-conditioned predictors commonly use a single robot or fixed participant set, limiting per-entity comparison of alternative variable-size multi-robot plans.Related multi-robot planners typically generate controls or scene predictions; TRACER instead evaluates supplied candidate joint trajectories through per-entity response prediction.Source: B. Literature Review · p. 2
  • TRACER’s adaptation couples a persistent identity-bound posterior, evidence from executed motion and synchronized entity response, and posterior-guided evaluation of later candidates.The related-work discussion distinguishes this combination from methods that infer latent interaction quantities or use active probing and dual control.Source: B. Literature Review · p. 2

II. PROBLEM FORMULATION

  • Each decision window evaluates candidate joint trajectories, executes a short prefix, and updates entity-specific response beliefs from observed responses.The formulation allows robot and entity counts to vary; the upstream candidate generator supplies alternatives, while only the selected trajectory is executed.Source: §II · p. 2, A. Scene and Candidate Trajectories · pp. 2–3

A. Scene and Candidate Trajectories

  • The selected plan is replanned after its first L steps, with candidate trajectories represented in fixed robot slots and inactive slots masked.Candidate feasibility checks include robot dynamics, static-obstacle avoidance, and robot–robot non-overlap; entity safety is handled probabilistically.Source: A. Scene and Candidate Trajectories · p. 3

B. Response Modes and Evidence

TRACER maintains identity-bound beliefs over latent entity response modes. It updates them using synchronized observations from executed interactions, not counterfactual predictions.

B. Response Modes and Evidence

  • Each entity has a latent response mode, and its future velocity response is represented over the next T prediction steps.At test time, the mode is latent and inferred from executed interactions.Source: B. Response Modes and Evidence · p. 3
  • Execution evidence pairs the robot position–velocity prefix actually executed with the synchronized entity response over the same L steps.Predictions for unexecuted candidates and the selected plan’s unexecuted suffix are counterfactual and excluded from the evidence history.Source: B. Response Modes and Evidence · p. 3
  • The identity-bound posterior weights mode-conditioned response distributions, while observed execution evidence changes those mixture weights for later candidate evaluation.The posterior is carried between decision windows and updated from the executed prefix and observed real responses.Source: B. Response Modes and Evidence · p. 3, Figure 1 · p. 3

C. Navigation Objective

TRACER formulates navigation as a belief-conditioned, receding-horizon problem. It selects a candidate plan, executes a prefix, and uses the resulting interaction to inform the next decision window.

C. Navigation Objective

  • The objective balances goal progress, residual interaction risk, and response burden using nonnegative weights.The exact expressions for these objective terms are specified later in the method.Source: C. Navigation Objective · p. 3
  • The navigation problem couples candidate response prediction with belief updates from executed interactions across successive decision windows.The selected candidate advances robots toward their destinations while maintaining robot–entity safety and limiting unnecessary entity motion.Source: C. Navigation Objective · p. 3
  • After the selected joint trajectory’s first L steps are executed, measured robot motion and synchronized entity responses are added as evidence for the next posterior.That posterior is then used in the next decision window’s candidate evaluation.Source: C. Navigation Objective · p. 3

III. METHODOLOGY

TRACER links candidate joint trajectories to predicted entity responses, then feeds executed interaction evidence back into later planning. Its response model separates individual-robot effects from pairwise interaction residuals.

A. Overview of the Bi-Directional Coupling Mechanism

  • At each decision window, TRACER predicts entity responses for supplied candidates, filters and ranks them, executes a prefix, and updates identity-bound beliefs from the observed interaction.The updated posterior influences evaluation and selection of candidates in the next window.Source: A. Overview of the Bi-Directional… · p. 4

B. Forward Coupling: From CJTs to Selected Plan

1) Forward Response Model (FRM):

  • The forward response model encodes recent scene history and robot trajectories, then conditions a probabilistic response distribution on context, candidate motion, and response mode.Its conditional mean combines a reference response, individual-robot effects, and pairwise second-order residuals; interactions above second order are omitted.Source: 1) Forward Response Model (FRM) · p. 4
  • A shared LSTM, two heterogeneous graph transformer layers, and a trajectory encoder represent node histories, typed interactions, and candidate robot motion.The robot-pair interaction residual isolates the additional mean entity response induced by joint deviations, after accounting for effects attributable to either robot alone.Source: 1) Forward Response Model (FRM) · p. 4

2) Selection under Predicted Responses:

TRACER first filters candidate joint trajectories (CJTs) by probabilistic entity-safety constraints, then ranks safe candidates using progress and predicted interaction costs.

2) Selection under Predicted Responses:

  • A C JT is unsafe for an entity if any robot comes within the hard separation distance at any prediction step, with response-mode posteriors weighting violation probability.The safe set enforces a maximum violation probability; if it is empty, the system uses a braking fallback.Source: 2) Selection under Predicted Responses · pp. 4–5
  • Among safe candidates, the ranking cost includes goal progress relative to a reference joint trajectory.The progress cost uses the candidate and reference robots' terminal positions.Source: 2) Selection under Predicted Responses · p. 5
  • Residual interaction risk penalizes the largest shortfall from preferred separation, so a near encounter cannot be obscured by averaging across robots or prediction steps.The preferred separation distance exceeds the hard separation distance.Source: 2) Selection under Predicted Responses · p. 5
  • The ranking also accounts for candidate-induced entity motion relative to the reference interaction, and selects the minimum-cost safe candidate.Risk is averaged over response modes and mode-conditional responses, while response burden depends only on the latent response mode.Source: 2) Selection under Predicted Responses · p. 5

C. Backward Coupling: Obtaining the Next-Window Belief

Backward coupling extracts evidence from the executed plan to refine entity-specific beliefs and carry them into the next decision window.

C. Backward Coupling: Obtaining the Next-Window Belief

  • Backward coupling refines entity-specific beliefs using evidence from the executed part of the selected plan and propagates them to the next window.It consists of collecting synchronized evidence and updating the identity-bound posterior.Source: C. Backward Coupling: Obtaining the… · p. 5
  • The backward-coupling process has two steps: collect synchronized evidence, then update the identity-bound posterior.Source: C. Backward Coupling: Obtaining the… · p. 5

1) Evidence Collection from the Executed Prefix:

Evidence collection records the executed robot motion and synchronized entity response over the same horizon, together with entity-local context.

1) Evidence Collection from the Executed Prefix:

2) Posterior Update and Replanning:

TRACER scores latent response modes against the observed executed-prefix response, updates their identity-bound posterior, and uses it in subsequent replanning.

2) Posterior Update and Replanning:

  • For each latent mode, the FRM evaluates the observed entity response's log-likelihood over the executed prefix.The FRM density is restricted to the first L prediction steps, using the same parameters as full-horizon prediction.Source: 2) Posterior Update and Replanning · p. 5
  • Bayes' rule gives greater posterior weight to response modes that better explain the observed interaction.Source: 2) Posterior Update and Replanning · p. 5
  • The updated posterior guides the next window's response prediction and CJT evaluation; without synchronized observations it carries forward unchanged and is discarded when the identity track ends.A newly observed entity is initialized with its prior response-mode belief.Source: 2) Posterior Update and Replanning · p. 5

D. Offline Training

The FRM is trained jointly to predict full-horizon responses and score executed prefixes, using paired robot-motion branches under matched conditions. Its objective combines the two likelihood tasks, with a tunable weight for prefix training.

D. Offline Training

  • The FRM jointly learns full-horizon response prediction and shorter-prefix likelihoods using the same model parameters.Both capabilities support inference: full-horizon responses evaluate candidate joint trajectories, while prefix likelihoods support posterior updates.Source: D. Offline Training · p. 5
  • Training examples pair reference, single-robot-change, and joint-change branches under matched context, initial state, exogenous randomness, and response-mode labels.Source: D. Offline Training · p. 5
  • Prefix lengths are sampled from P(pref) over 1 through T, and full-horizon and prefix log-likelihoods are normalized by velocity coordinates and time steps.Source: D. Offline Training · p. 5
  • The response objective weights prefix training with λ(pref) and is minimized end-to-end over FRM parameters by gradient descent.Source: D. Offline Training · p. 5

IV. EXPERIMENTS

The experiments isolate joint-response prediction and persistent belief updates with controlled synthetic protocols, then assess how these mechanisms relate to replanning and generalization. Results test whether joint robot actions capture non-additive responses and whether executed interaction evidence remains useful across windows.

A. Evaluation Protocol

  • The synthetic protocols isolate joint-response effects and persistent-mode updates, using matched interventions or fixed response modes over eight replanning windows.The causal-response protocol uses 2 robots and 12 entities; the persistent-mode protocol uses 3 robots and 10 entities, with 3 observed per window.Source: A. Evaluation Protocol · p. 6

B. Forward Coupling: Joint Actions Recover Non-additive Responses

  • Matched four-intervention scenes vary neither, either, or both robot motions; the second difference cancels reference and individual-robot effects to isolate non-additive response.The comparison includes a capacity-matched additive predictor whose residual parameter count differs from TRACER by 0.31%.Source: B. Forward Coupling: Joint Actions… · p. 6, Figure 2 · p. 6
  • TRACER achieves the lowest pair RMSE (0.343) and FDE (0.480), reducing pair RMSE by 23.94% versus MatchedAdditive in the nonlinear-response regime.It has lower pair RMSE in 1,271 of 1,280 matched scene-seed records.Source: Table I · p. 6, Figure 2 · p. 6

C. Backward Coupling: Retaining Executed Evidence

  • TRACER’s identity-bound beliefs become more concentrated across replanning windows, and it achieves the lowest NLL, ADE, and FDE on both ID and OOD splits.Reset returns toward nearly uniform beliefs when cross-window evidence is discarded; Fig. 3(b) compares changes in task progress, safety cost, human-response cost, and collisions against TaskOnly on ID/OOD.Source: C. Backward Coupling: Retaining… · pp. 6–7, Figure 4 · p. 7, Figure 3 · p. 7

D. From Belief to Action: Evidence-Informed Replanning

The section reports that retaining identity-consistent interaction evidence improves response-aware replanning, then tests TRACER on recorded interactions and external navigation benchmarks. External evaluations show improved collision-free multi-robot navigation relative to baselines, and the conclusion frames closed-loop prediction and adaptation as an effective basis for socially-aware navigation.

D. From Belief to Action: Evidence-Informed Replanning

  • TRACER reduces safety and human-response costs on ID and OOD splits relative to TaskOnly, with a moderate loss in task progress; retaining evidence also improves decisions over Reset.Oracle provides an upper reference with privileged access to the true response mode.Source: D. From Belief to Action:… · p. 7
  • In a matched-initial-state example, TRACER completes the task with a positive minimum sampled gap, while GoAlone enters contact.Source: Figure 5 · p. 7, E. Generalization Beyond Controlled… · p. 8

E. Generalization Beyond Controlled Simulation

  • On SocialGym2, TRACER raises collision-free completion from 22.5% to 31.6% and team completion from 40.9% to 47.5%, while lowering collision-episode rate from 53.8% to 30.3% versus GoAlone.On CrowdSimPlus, TRACER achieves 100% collision-free completion and a 0.951 progress score, the highest among kinematically matched non-privileged controllers.Source: Table III · p. 7, E. Generalization Beyond Controlled… · p. 8
  • Action conditioning improves NLL, ADE, and FDE over HistoryOnly on both NavWareSet and Bi3 recorded-interaction datasets.NavWareSet and Bi3 test recordings include up to two humans.Source: E. Generalization Beyond Controlled… · pp. 7–8, Table IV · p. 8

V. CONCLUSION

  • TRACER closes the loop by combining joint-trajectory-conditioned response prediction with persistent identity-bound belief updates from executed interactions.The conclusion reports improved response prediction and downstream decisions, alongside improved collision-aware navigation on external benchmarks.Source: §V · p. 8

Related papers

Loading 2609.18776v1…