Predictive-Coding-Based Autonomous Regulation of Internally Generated and Externally Coupled Processing in Human-Robot Interaction

1 Introduction

• In PV-RNN, learned prior dynamics generate temporal predictions while posterior inference accommodates observations, and the meta-prior regulates posterior departure from the prior.Stronger regularization constrains inference toward learned dynamics; weaker regularization permits greater sensory-driven adaptation.Source: §1 · p. 2
• Persistent mismatch with reliable sensory observations may favor stronger sensory accommodation, while prior dynamics may be advantageous when sensory information is unreliable or unavailable.Source: §1 · p. 2
• Existing approaches generally examine a given precision, meta-prior, or interaction regime, or externally specify its change, leaving online regulation during evolving interaction unresolved.Source: §1 · p. 2
• The proposed mechanism uses reconstruction error accumulated over a recent window to choose between predefined meta-prior regimes while keeping learned generative-model parameters fixed.Persistent mismatch promotes a lower meta-prior; sufficiently low mismatch promotes a higher one.Source: §1 · p. 2
• Across three interaction tasks, the mechanism links recent reconstruction mismatch to online regime selection and reports consequences for prospective prediction and robot-side physical interaction.Task 3 includes conditions where recent sensory observations are accommodated without ensuring accurate prediction of subsequent interaction.Source: §1 · pp. 2–3

2.1 Predictive-Coding-Inspired Variational Recurrent Neural Network

• PV-RNN combines hierarchical recurrent dynamics with variational inference: learned priors generate temporal predictions, while posterior inference accommodates sensory observations.During interaction, learned network parameters remain fixed and only adaptive posterior variables are optimized within the inference window.Source: §2.1 · p. 3

2.2 Internally Generated and Externally Coupled Processing

• A larger meta-prior more strongly constrains posterior departure from learned dynamics; a smaller value permits greater sensory accommodation, and neither regime is inherently preferable.Stronger prior constraint preserves learned temporal organization, whereas weaker constraint enables greater accommodation of ongoing sensory information.Source: §2.2 · pp. 3–4

2.3 Reconstruction-Error-Driven Meta-Prior Regulation

• Temporally averaged reconstruction error measures retrospective reconstruction of recent observations and probabilistically regulates transitions between high- and low-meta-prior regimes using separate thresholds.Hysteresis separates the thresholds, while transition temperature controls their probabilistic sharpness; reconstruction error does not directly measure future prediction accuracy.Source: §2.3 · p. 4
• Online selection uses endogenous reconstruction dynamics without an external switching command, but the regulatory policy and its hyperparameters are predefined.The selected meta-prior changes the conditions of the following inference step, not the posterior that produced the current reconstruction error.Source: §2.3 · p. 4

2.4 Closed-Loop Online Regulation During Interaction

• The closed loop updates posterior states from new sensorimotor data, evaluates accumulated mismatch, and uses it to regulate subsequent prior constraint without changing learned network parameters.The mechanism uses reconstruction mismatch both in current-window posterior optimization and as a higher-order signal regulating later inference.Source: §2.4 · pp. 4–5

3.1 Humanoid Robot and Sensorimotor Variables

• Torobo has 16 degrees of freedom, including 6 in each arm; four arm-joint angles, two per arm, served as PV-RNN observations.The selected joint angles constituted the sensorimotor observations processed by the model.Source: §3.1 · p. 5
• The PV-RNN generated target joint trajectories integrated with force-feedback control while a human operator physically guided the robot in a closed loop.The robot supports compliant kinesthetic interaction through force-feedback control.Source: §3.1 · p. 5
• Physical interaction conflict was measured using the robot’s excess-torque signal, a robot-side indicator of relative conflict rather than human force or calibrated biomechanical force.Larger excess-torque values indicate stronger disagreement between internally generated motor commands and externally imposed motion.Source: §3.1 · p. 5

3.2 Training Data and Model Training

• The model learned four-dimensional joint-angle trajectories from PFSM-generated sequences for Tasks 1 and 2 and continuous kinesthetic interaction data for Task 3.Source: §3.2 · p. 5

3.2.1 Structured Training Data for Tasks 1 and 2

• Tasks 1 and 2 used four cyclic primitives, A–D, with 20-step cycles and stochastic transitions; ten sequences each contained 200 cycles.Source: §3.2.1 · pp. 5–6

3.2.2 Less-Constrained Training Data for Task 3

• Task 3 used four kinesthetic-interaction datasets of approximately 1300–1500 time steps, with naturally varying timing, amplitude, and transitions.Source: §3.2.2 · p. 6

3.2.3 Network Architecture and Training

• The two-layer PV-RNN used 30 deterministic units and four latent variables in its upper layer, and 60 deterministic units and two latent variables in its lower layer.The training meta-prior was fixed at wtr = 0.01 for both layers.Source: §3.2.3 · pp. 6–7
• The PV-RNN generated target joint trajectories for force-feedback control, while reconstruction error accumulated over a sliding window modulated subsequent inference.The experiments used a 100-step inference window.Source: Figure 1 · p. 6, Figure 2 · p. 7
• Training ran for 50,000 epochs with Adam at a learning rate of 0.001; the meta-prior stayed fixed, so the online switching policy was not learned during training.Source: §3.2.3 · p. 7

3.3 Human–Robot Interaction Tasks

• All tasks used a 100-step inference window and regulated the meta-prior online; calibrated parameters were fixed within each task and across model realizations.The timing of regime transitions was determined online from evolving reconstruction-error history.Source: §3.3 · pp. 7–8

3.3.1 Task 1: Fixed Structured Interaction

• Task 1 paired 100 time steps of free generation with 200 steps of human guidance along one fixed learned primitive, repeated ten times.The setup produced sustained structured mismatch when human motion differed from robot dynamics.Source: §3.3.1 · p. 8

3.3.2 Task 2: Changing Structured Interaction

• Task 2 kept the same generation and interaction durations but changed the human-imposed primitive every 100 interaction steps, repeatedly violating learned transition expectations.Source: §3.3.2 · p. 8

3.3.3 Task 3: Less-Constrained Human–Robot Interaction

• Task 3 involved continuous human guidance without scheduled free-generation segments or prescribed primitive transitions, with naturally varying motion over approximately 900–1200 steps.“Less-constrained” still denotes interaction bounded by robot morphology, four sensorimotor dimensions, and the control architecture.Source: §3.3.3 · p. 8

• Interaction samples were classified by high- or low-meta-prior regime, and evaluation combined inference, prospective prediction, robot-side interaction, and latent-organization measures.Source: §3.4 · p. 8
• Reconstruction error measured accommodation of observations within the inference window, while raw KL divergence measured posterior departure from learned prior dynamics.The reported KL divergence was unweighted by the meta-prior.Source: §3.4 · pp. 8–9
• Prospective prediction error compared the first prior-generated sensory sample beyond the inference window with the next observed input, using mean absolute difference across sensorimotor dimensions.Source: §3.4 · p. 9
• Latent organization was visualized with kernel PCA, using a common embedding for Tasks 1 and 2 and a separate embedding for Task 3.Generated outputs were plotted using KPC1 against its one-step change.Source: §3.4 · p. 9
• Statistical summaries used paired model-level values from four independently trained realizations; confidence intervals were descriptive estimates, not formal hypothesis tests.For Tasks 1 and 2, summaries excluded free-generation periods and the first five steps after regime transitions.Source: §3.4 · p. 9

4.1 Task 1: Fixed Structured Interaction

• In Task 1, low-w processing reduced reconstruction error, prospective prediction error, and excess torque while increasing raw KL divergence in both layers.Across four models, reconstruction error fell from 0.0287 to 0.0094, prospective prediction error from 0.3276 to 0.1152, and excess torque from 6.86 N·m to 3.57 N·m; raw KL divergence rose in Layers 1 and 2.Source: §4.1 · p. 9, Figure 4 · p. 10

4.2 Task 2: Changing Structured Interaction

• In Task 2, repeated changes to the movement primitive drove recurrent mismatch and high/low-w transitions, whose timing varied across models.Low w reduced reconstruction and prospective prediction errors and excess torque, while raw KL divergence increased in both layers.Source: §4.2 · pp. 9–10, Figure 5 · p. 10

4.3 Task 3: Less-Constrained Human–Robot Interaction

• Without a prescribed movement sequence or transition schedule in Task 3, accumulated mismatch still prompted autonomous transitions to low-w sensory accommodation.Low w reduced reconstruction error and increased raw KL divergence in both layers.Source: §4.3 · pp. 10–11, Figure 6 · p. 11

4.4 Prospective Prediction and Physical Interaction in Task 3

• In Task 3, recent human motion could be accommodated through posterior inference even as subsequent motion departed from the model’s prior-generated future trajectory.Low w was also associated with reduced prospective prediction error and excess torque across models.Source: §4.4 · p. 11, Figure 7 · p. 12

4.5 Latent-Dynamical Organization Across Interaction Conditions

• Kernel-PCA representations showed partially different high- and low-w occupancy patterns; Task 3 was broader and less discretely organized than Tasks 1 and 2.Tasks 1 and 2 shared an embedding for direct comparison, while Task 3 was embedded separately; its high- and low-w samples overlapped more.Source: §4.5 · p. 11, Figure 8 · p. 12

4.6 Summary Across Interaction Conditions

• Across all three tasks, low-w processing produced lower reconstruction and prospective prediction errors, greater posterior–prior KL divergence, and lower robot-side physical interaction conflict than high-w processing.Quantitative effect magnitudes are interpreted within each task because Task 3 used different regulatory parameters.Source: §4.6 · p. 13

5.1 Reconstruction Error as a Higher-Order Regulatory Signal

• Lowering the meta-prior weakens the posterior–prior divergence penalty, allowing greater sensory accommodation and producing lower reconstruction error with larger raw KL divergence.This verifies that the two predefined regimes produce the intended difference in posterior constraint.Source: §5.1 · p. 13
• Across all three tasks, regime-dependent changes appeared in prospective prediction and robot-side physical interaction, beyond the reconstruction objective.Task 3 also showed that recent observations could be accommodated while subsequent human motion still departed from the model’s prior-generated trajectory.Source: §5.1 · p. 13, §5 · p. 13

5.2 Regulation Between Internally Generated and Externally Coupled Processing

• Stronger prior constraint preserves learned temporal dynamics, while weaker constraint accommodates unexpected input; the value of sensory coupling depends on sensory reliability.Earlier embodied studies examined fixed meta-prior settings, whereas this framework uses endogenous reconstruction dynamics to select between two operating regions along a broader continuum.Source: §5.2 · p. 13

5.3 From Structured to Less-Constrained Human–Robot Interaction

• In Tasks 1 and 2, lowering w reduced reconstruction error, prospective prediction error, and physical interaction conflict for trajectories drawn from learned movement primitives.Stronger sensory accommodation also supported accurate prediction of those trajectories’ subsequent evolution.Source: §5.3 · p. 14
• In Task 3, low w again reduced reconstruction and prospective prediction errors while increasing posterior–prior divergence, despite no prescribed movement sequence or transition schedule.Recent observations could be accommodated even when subsequent human motion departed from the model’s prior-generated future trajectory.Source: §5.3 · p. 14

5.4 Relation to Focused and Mind-Wandering Processing

• The framework extends autonomous processing transitions studied in simulated cognitive tasks to embodied physical interaction, but the correspondence to mind wandering and focused attention is functional, not phenomenological.The regimes are not equivalent to mind wandering or focused attention.Source: §5.4 · p. 14

5.5 Limitations and Future Directions

• The regulatory policy is manually specified and switches between two predefined meta-prior values; its advantage over fixed-w operation was not established.Direct comparisons with fixed-w baselines are needed to determine whether dynamic regulation provides functional benefits.Source: §5.5 · p. 14
• The evaluation leaves generative-model adaptation, sensory reliability, and broader behavioral generalizability unresolved.Error regression updates posterior inference but not learned model parameters; reconstruction error does not represent sensory reliability. Experiments used four arm-joint dimensions, one humanoid platform, and four model realizations, and Task 3 remained physically and kinematically constrained.Source: §5.5 · pp. 14–15

6 Conclusion

• Recent interaction-history reconstruction error modulates the PV-RNN meta-prior, regulating the relative influence of learned prior dynamics and ongoing sensory information.The generative-model parameters remain as described in the framework; the regulation concerns conditions of inference.Source: §6 · p. 15
• Across fixed structured, changing structured, and less-constrained interaction, low w was associated with lower reconstruction error and greater posterior–prior divergence, while regime selection also tracked prospective prediction and robot-side interaction.The latter quantities are not directly optimized by the reconstruction term.Source: §6 · p. 15
• Temporally accumulated mismatch functions as a higher-order signal that regulates the conditions of subsequent inference.The resulting feedback loop connects recent inference mismatch to online prior-constraint regulation and consequences for prospective and embodied interaction dynamics.Source: §6 · p. 15