Source-linked AI summary
Shifting Relational Paradigms for Affective Computing: Affective Resonance, Vitality Affects, and Vocal Interaction Fields
Cy Gorman, Yihang Yao
TL;DR
The paper addresses the limits of individual-state affective computing for interaction by asking how relational affective organisation can be modeled. It proposes vocal interaction fields using affective resonance, vitality dynamics, and ARDO/AARI design frameworks, and presents a proof-of-concept study of directional expressive coupling. Coupling is regime-specific, concentrated at sub-second lags, and reduced under exclusive-speech controls, while the framework remains limited by its AMI-only evidence and other stated scope constraints.
Problem
Individual-state pipelines model emotion labels or arousal/valence from speakers, leaving relational affective organisation insufficiently addressed in interaction.
Method
The paper combines affective-resonance and vitality-affect theory with continuous vocal representations and regime-based directional coupling analyses, introducing ARDO and AARI as interaction-field design frameworks.
Results
Directional expressive coupling was reliably detected at sub-second lags under mutual orientation, shifted toward zero outside mutual co-presence, and showed approximately +6.6 and +7.4 percentage-point calibrated excess rejections in Tier A micro conditions.
Takeaways & Limitations
The findings support treating vocal affective dynamics as regime-conditioned interactional-field phenomena rather than only as individual trajectories or categorical emotion outputs.
Takeaways & Limitations
The results are AMI-only, Tier B macro analysis is underpowered, and the directionality score depends on FDR and minimum run-fraction parameters.
Abstract
from arXiv · showhide
Affective computing has largely followed an individual-state paradigm, extracting discrete emotion labels or arousal/valence from isolated speakers. We argue this framing is incomplete for interaction. Drawing on affective resonance and vitality-contour accounts, we propose a relational framework in which the primary unit of affective analysis is the interactional field constituted within vocal dynamics. As a proof of concept, we present a preliminary empirical study using continuous self-supervised speech representations to detect directional expressive coupling in multi-party conversation. Coupling is regime-specific, concentrated at sub-second timescales, and collapses under exclusive-speech negative controls, consistent with a relational account of affective dynamics. We introduce design frameworks for Artificial Affective Resonance Intelligence grounded in Affective Resonance Dynamic Ontologies, supported by null-calibrated directional coupling analyses across interaction regimes.
1. Introduction
Affective computing has mainly modeled individual emotion states from signals, while this paper motivates relational dynamics as a complementary focus for interaction. Vocal dynamics are used as a tractable domain for examining that relational field.
- Affective computing pipelines commonly map acoustic, linguistic, physiological, or biometric signals to discrete emotion categories or continuous arousal/valence.
- Relational dynamics are proposed as a complementary primary site of affective organisation within interaction.
- Vocal dynamics provide the paper’s theoretically motivated and tractable domain for examining relational affective organisation.
- Existing coupling work often operationalises interaction as increasing similarity between speaker trajectories rather than directionality, configuration, or field-level emergence.
2. Theoretical foundations
The paper grounds its relational framework in affective resonance, vitality affects, and enactivist participatory sense-making. It then defines ARDO and AARI as interaction-field-oriented alternatives to systems that classify individual affective states.
- Affective resonance: Affective resonance treats affective experience as an emergent interactional dynamic rather than transmission between independently existing individual states.It is described as relational, immanent, and creatively generative rather than governed by pre-formed emotion categories.
- Affective resonance: Affective resonance differs from imitation, synchrony, and mimicry because it does not require congruent form, only connectivity among forces constituting the interactional field.
- Vitality affects: Vitality affects describe dynamic, noncategorical qualities whose temporal forms include surging, fading, rushing, and bursting.Their central dimensions are temporality, intensity, and form, with voice serving as a continuous carrier.
- Enactivism: Enactivist participatory sense-making holds that meaning is jointly constituted through embodied interaction, with the relational subject emerging within interaction.
- ARDO and AARI: ARDO models interaction-level organisation in sustained vocal dynamics, while AARI denotes systems that map vitality trajectories and generate responses calibrated to their modulation.Both frameworks treat discrete affective states as downstream products rather than preconditions of resonance.
3. Empirical illustration
The study tests whether directional expressive coupling depends on interactional configuration, using regime-decomposed four-speaker AMI audio and null-calibrated analyses. Coupling is strongest at sub-second lags during overlapping speech and weakens toward the null under exclusive-speech controls.
- Rationale: The framework predicts directional expressive coupling during mutual co-presence and weaker coupling when overlapping interaction is removed.Tier A captures simultaneous co-presence; Tier B pools non-overlapping activity, while Tier C separates it by speaker direction.
- Data and features: Four-speaker AMI headset audio was analysed in synchronised 60-second windows with voice-activity masks mapped to approximately 20 ms WavLM feature frames.Results are reported for AMI only, with cross-corpus generalisation reserved for future work.
- Data and features: Expressiveness was derived from cross-layer WavLM activation dispersion and residualised against energy within regime-contiguous episodes before directional testing.The analysis also produced aligned semantic-magnitude and topic-drift proxies across the same regimes.
- Analysis: The analysis used signed Directionality Support Scores after false-discovery-rate control and compared observed rejection behaviour with matched circular-shift nulls.Null-calibrated excess rejection was reported as OBS − NULL percentage points; negative values indicate fewer rejections than the calibrated null.
- Results: At q = 5%, Tier A micro effects were approximately +6.6 percentage points for the bivariate model and +7.4 points under energy-controlled conditional analysis.At q = 1%, effects attenuated but remained regime-specific: Tier A was strongest, Tier B weaker, and Tier C approached the null.
- Results: Directional coupling was reliably detectable at sub-second lags in mutual-orientation regimes, while pooled and directional exclusive activity shifted strongly toward zero support.Tier A directional wins were near-symmetric across directions, and the result was not reducible to synchrony in the scalar energy proxy alone.
4. Design implications
The paper reframes affective-system design around participation in an interactional field rather than inference of individual affective states. It proposes relational evaluation and vocal dynamics as design resources for responsive systems.
- Relational design: The interactional field leaves measurable traces because coupling is constituted by conversational configuration rather than carried into interaction by individual speakers.This motivates systems that respond to the evolving relational dynamic rather than to diagnoses of individual trajectories.
- Relational design: ARDO-oriented systems are evaluated by the quality of relational coupling they sustain rather than by classification error against discrete emotion labels.Emotion labels remain useful as diagnostic and benchmark interfaces, but are not the primary object of ARDO.
- Relational design: AARI systems are designed to maintain asymmetric, responsive attunement to the user’s affective tempo within the interactional field.The proposed attunement is compared with dynamics described in Stern’s infant–caregiver model.
- Vocal interaction: Treating vocalics as a design medium opens analysis of moment-to-moment interactional contours using concepts such as tension, release, consonance, and dissonance.The paper presents this as a working hypothesis consistent with evidence that affective processing precedes semantic processing.
5. Conclusions
The paper concludes that affective computing should model interactional fields as primary sites of affective organisation rather than individual affective states. Its empirical illustration supports this reorientation by showing regime-conditioned, temporally localised vocal coupling that collapses when mutual orientation is removed.
- Affective computing should treat interactional fields, rather than individual states, as the primary object of affective modelling.
- Vocal coupling is regime-conditioned and temporally localised, and collapses when mutual orientation is removed.
- ARDO and AARI translate this reorientation into designs that treat vocalics as expressive responsiveness rather than a diagnostic channel.
7. Generative AI Use Disclosure
Generative AI tools assisted with editing, structural refinement, analysis, summarisation, workflow documentation, and visualisation scripts. The authors retain responsibility for the theoretical framing, experimental design, rationale, and implementation.
- Generative AI tools assisted with editing, structural refinement, analysis, summarisation, workflow documentation, and visualisation scripts.
- The authors produced the theoretical framing, experimental design, rationale, and implementation, retaining intellectual ownership and responsibility.