Active Inference for Interaction-Mediated Control of a High-Dimensional Robotic Arm

• High-dimensional, dynamically coupled controls can make direct user control burdensome, especially when low-bandwidth interaction cannot expose a tractable interface,.Source: §1 · p. 2
• Interaction-mediated control treats user input as evidence about latent task preferences, which an interaction broker infers and passes to an autonomous Active Inference controller.The broker queries and infers preferences; the controller controls the plant under that information.Source: §1 · p. 2
• The architecture is formally specified and computationally realized in a simulated multi-link arm, where a clutch-style binary evaluative channel conveys target preferences without joint-torque commands.The experiments provide an initial demonstration and motivate further HCI investigation.Source: §1 · p. 2

2.1 Mediated Interaction as a General Architectural Response

• Interaction-mediated control places a mediating mechanism between user and plant, allowing user input to provide evaluative evidence about task preferences rather than direct plant commands.Source: §2.1 · p. 2

2.2 Active Inference as an Interaction-Mediating Mechanism

• Active Inference maintains probabilistic beliefs about latent preferences, uses user-facing queries to reduce uncertainty, and applies inferred preferences to guide plant-facing actions.Source: §2.2 · pp. 2–3
• Unlike prior Active Inference work on transduction, perception, reaching, torque control, or pointing, this study infers a simulated user's objective from evaluative interaction before autonomous plant control.Source: §2.2 · p. 3

2.3 Decomposing the AIF-Mediator: Inferential Division of Labour

• The Interaction Broker Agent queries the user and performs Bayesian preference inference, while the Low-Level Control Agent autonomously controls the plant using posterior-marginal preferences.The simulated Human/User Agent has private access to the objective and provides evaluative feedback; the components communicate through restricted interfaces.Source: §2.3 · p. 3

3.1 Task Definition

• The planar arm must simultaneously cover every object in a private target subset, while nontarget coverage is unpenalized and no unique pose is prescribed.The arm has N equal-length links and uses per-joint torques.Source: §3.1 · pp. 3–4

3.2 Local Models, Objectives, and Interface Contracts

• The three components implement distinct inference, evaluation, and control processes, with local models, operative computations, and interface contracts specified for each.Full implementation details are placed in Appendices A–C.Source: §3.2 · p. 4

Human/User Agent

• The simulated HUA privately knows the target subset and returns binary satisfaction or dissatisfaction feedback without communicating target identity or motor commands.The HUA is not implemented as an Active Inference agent and performs no VFE minimisation or EFE planning.Source: Human/User Agent · pp. 4–5

Interaction Broker Agent

• The IBA selects queries for expected information gain, updates its categorical belief by Bayes’ rule after feedback, and sends object-wise preference marginals to the LLCA.It does not issue torque commands or choose an execution rollout.Source: Interaction Broker Agent · p. 5

Low–Level Control Agent

• The LLCA exclusively controls the arm, maintains Gaussian beliefs over latent state, and samples candidate torque plans propagated through its internal dynamics model.It performs perceptual correction by variational free-energy minimisation and selects actions using receding-horizon random-shooting control.Source: Low–Level Control Agent · p. 6

3.3 Complete Outer Interaction Cycle

• The LLCA’s first planning call supplies a query rollout pool, but the plant advances only after the second planning call for execution.The first torque of the selected execution rollout is applied for an action-repeat block before replanning.Source: §3.3 · p. 7, Low–Level Control Agent · p. 7

1. Query planning. The IBA supplies pIBA,pre

• After the HUA returns a binary evaluation, the IBA updates its posterior and marginalises it into object-wise preferences for the next planning stage.The IBA carries the posterior into the next interaction cycle.Source: §1 · pp. 7–8
• Using the updated preferences, the LLCA generates a fresh execution pool, selects a rollout, and applies only its first torque to the plant.Fig. 2 depicts the execution pool and the realised post-action pose.Source: §1 · p. 8, Figure 2 · p. 8
• The LLCA supplies a query rollout pool, and the IBA selects the candidate with greatest expected information gain rather than executing that query rollout.In Fig. 2, the LLCA’s pool is ranked by pragmatic value, while the IBA ranks the same pool by information gain.Source: §1 · p. 7, Figure 2 · p. 8

4.1 Experimental Question and Task

• The evaluation uses four circular objects in a 2 × 2 grid, with exactly two targets, and tests arms with 3–6 links across all six target subsets.Each subset is repeated over 24 episodes per dimensionality, for 144 episodes per dimensionality and 576 per condition.Source: §4.1 · p. 8

4.2 Matched Experimental Conditions

• The direct-preference reference supplies true object-wise preferences to the LLCA, whereas the interaction-mediated condition infers preferences from binary feedback; other task and controller settings are matched.The simulated HUA and IBA use the same response-model family, so the comparison assumes matched models rather than real human behaviour or user-model mismatch.Source: §4.2 · p. 9

4.3 Evaluation

• Success requires simultaneous coverage of all target objects before termination, and timing is reported only for successful episodes using generative-process and outer-cycle indices.Neither timing measure captures wall-clock computation or HUA response latency; posterior probability, entropy, MAP accuracy, and feedback frequency are separate interaction diagnostics.Source: §4.3 · p. 9

5 Results

• Median time-to-first-success was longer under interaction mediation at every tested dimensionality among successful episodes.Table 1 reports timing summaries for successful episodes only.Source: §5 · p. 10, Table 1 · p. 10
• 97.9% (564/576) of direct-preference episodes succeeded, versus 73.4% (423/576) under interaction-mediated control.The direct-preference reference achieved higher success at every tested arm dimensionality.Source: §5 · pp. 9–10, Table 1 · p. 10
• MAP target accuracy ranged from 0.167 to 0.472 across arm dimensionalities, while mean final posterior entropy remained non-zero.Exact target identification and task completion are distinct outcomes because the controller acts on posterior-marginal object preferences.Source: §5 · p. 10

6 Discussion and Conclusion

• Interaction-mediated control succeeded in 423/576 episodes (73.4%), compared with 564/576 (97.9%) under direct preference access.The architecture succeeded across all tested arm dimensionalities despite imperfect target-subset identification.Source: §6 · p. 10
• The simulated user’s feedback matched the response-model family assumed by the mediator, so robustness to human-model mismatch and actual human interaction behaviour was not established.This matched-model assumption was adopted to evaluate an initial realisation under controlled conditions.Source: §6 · pp. 10, 12
• The fixed object layout and target cardinality, single expected-information-gain query strategy, and instantaneous interaction constrain the evaluated setting.The simulation does not model human response latency or the real-time consequences of pausing plant execution during interaction.Source: §6 · p. 12
• The authors call for human-in-the-loop experiments and evaluations across other interactive systems to establish practical applicability and broader generality.Figure 3 presents one illustrative successful episode, not an aggregate trend.Source: Figure 3 · p. 11, §6 · p. 12

HUA Variables and Interface

• The HUA’s private target indicator defines its effective object-preference vector, which is not directly observable by the IBA or LLCA.The HUA receives the full rollout-pool context and the IBA-selected query index.Source: HUA Variables and Interface · p. 13
• The public binary response denotes satisfaction or dissatisfaction with a selected preview relative to the current rollout pool.An absent dissatisfaction response is interpreted as satisfaction rather than as a missing observation.Source: HUA Variables and Interface · p. 13

HUA Feedback Process

• The stochastic HUA samples feedback using its privately known target and selected query, under the same pool-contextual response-model family as the IBA.The HUA performs no VFE minimisation, EFE optimisation, or temporal planning.Source: HUA Feedback Process · p. 14

B.1 Latent Intent Model and Query Selection

• The IBA models the private target subset as a latent hypothesis, with six uniformly weighted hypotheses in the reported K = 4, Ktar = 2 experiments.Its query objective is purely epistemic and has no separate pragmatic term.Source: §B.1 · p. 14
• For each candidate query, the IBA evaluates mutual information between the latent target hypothesis and prospective binary feedback.It selects the query expected to provide the greatest information about the latent target subset.Source: §B.1 · p. 14

B.2 Posterior Inference and Preference Communication

• After observing feedback, the IBA updates its categorical belief by exact Bayesian inference over the enumerated hypothesis space.It sends object-wise posterior target probabilities to the LLCA rather than the full posterior or MAP hypothesis.Source: §B.2 · p. 15

C.1 Generative Model: States, Observations, and Actions

• The LLCA receives object-indexed preferences from the IBA and determines the joint torques needed to realise them, without inferring latent targets or modelling the interaction.The simulated plant is a planar N-link arm; its transition is treated as a black box unavailable to the LLCA.Source: §C.1 · p. 15
• The LLCA’s internal model uses planar serial-link kinematics, link mass and length, inertia, and a semi-implicit Euler transition with implicit damping.The model’s transition includes process uncertainty and model mismatch.Source: §C.1 · pp. 15–16
• The plant provides joint angles unchanged, while the LLCA omits angular velocities, arm geometry, and task variables from its observation vector.No stochastic observation noise is added to the plant observation stream; the reported internal observation uncertainty uses σθ = 10^-4 and Σobs = 10^-8I_N.Source: §C.1 · p. 16

C.2 LLCA State Inference

• After full-state initialisation, the LLCA infers unobserved angular velocities through Unscented Transform prediction and angle-based variational perceptual updates.Each repeated plant step receives prediction and perceptual correction; reported updates use 50 posterior samples and 25 Adam steps at learning rate 0.05.Source: §C.2 · pp. 16–17, Figure 4 · p. 20

C.3 LLCA Rollout Generation

• Candidate torque plans are propagated into predicted state-belief trajectories, with unweighted object-wise coverage bases computed from predicted joint-angle means before preferences are applied.Reported candidate torques are sampled from [−1.5, 1.5], independently of the preference vector.Source: §C.3 · p. 17

C.4 LLCA Preference Model and Pragmatic Action Selection

• The LLCA evaluates whole-arm coverage deterministically at the predicted joint-angle mean, weighting object-wise basis terms by preferences and neglecting predicted-state covariance.With information gain disabled and λpv = 1, rollout selection maximises mean whole-horizon pragmatic value; applied torques are clipped to [−1, 1].Source: §C.4 · pp. 18–19

D Complete Interaction Algorithm

• The outer cycle includes a user query that is not executed, then repeats the selected torque with prediction, plant transition, and angle-based correction until success or block completion.The LLCA carries its belief forward between cycles and replans after the action-repeat block.Source: D Complete Interaction Algorithm · p. 19, §C.4 · p. 19