Source-linked AI summary
Humanoid Musical Robots as Experimental Interfaces for Music-Evoked Emotion
Vincent K. M. Cheung, Jia-Yeu Lin
TL;DR
Music-emotion research has largely relied on auditory-only, pre-recorded stimuli, limiting controlled study of embodied, multimodal, and interactive mechanisms. This position paper proposes humanoid robots as reproducible experimental interfaces and uses WAS-5 as technical feasibility evidence. The paper concludes that these systems can support future controlled investigations while complementing human-musician studies.
Problem
Pre-recorded auditory paradigms inadequately support systematic manipulation of the embodied, multimodal, and interactive aspects involved in music-evoked emotion.
Method
The paper positions humanoid musical robots as reproducible interfaces for controlling acoustic, embodied, and interaction variables, supported by a WAS-5 case study and proposed experimental paradigms.
Results
The WAS-5 case study demonstrates reproducible control of embodied acoustic parameters and closed-loop interaction with human behaviour, supporting future emotion experiments.
Takeaways & Limitations
Musical humanoid robots can serve as complementary methodological platforms for controlled investigation of multimodal and social mechanisms in music-evoked emotion.
Takeaways & Limitations
The feasible experimental state-space is constrained by current robot technology, while construction and calibration add technical and logistical hurdles.
Abstract
from arXiv · showhide
Advances in technology have led to increasingly sophisticated musical humanoid robots. However, their use has largely been limited to performance and related research in human-robot interaction. In this position paper, we propose a novel perspective: musical humanoid robots as experimental interfaces for investigating music-evoked emotions. We argue that current research is constrained by paradigms relying on pre-recorded auditory stimuli, which fail to capture the multimodal, embodied, and interactive nature of real-world musical experience. Building on existing theories of music cognition and emotion, we identify mechanisms that require controlled manipulation of both acoustic and non-acoustic variables. We show that humanoid robots are well-suited as they enable parametric control of performance variables, reproducibility across trials, and the decoupling and recombination of auditory, visual, and interactive components. We illustrate the technical feasibility of this perspective through a case study of the WAseda Saxophonist Robot 5 (WAS-5), demonstrating reproducible control of acoustic and interaction variables that are prerequisites for future music-emotion experiments. Our work positions musical humanoid robots as a methodological platform that enables future controlled investigations of music-evoked emotions.
1. INTRODUCTION
Music cognition research seeks to explain how musical structures produce affective responses, but conventional auditory-only paradigms overlook embodied, multimodal, and interactive experience. The paper therefore proposes musical humanoid robots as controllable experimental interfaces and uses WAS-5 to demonstrate technical feasibility.
- Music cognition research aims to explain how musical structures are perceived, integrated, and linked to affective responses.
- Laboratory studies commonly use pre-recorded stimuli and assume auditory information captures expressive content, overlooking visual, motor, and social cues.
- Musical robots have mainly supported performance and human–robot interaction, leaving their potential as experimental tools underdeveloped.
- The paper proposes humanoid musical robots as interfaces that translate expressive intent into physically instantiated, controllable actions for hypothesis-driven emotion research.
- The paper contributes a robot-based experimental perspective, a WAS-5 feasibility case study, and theoretically grounded paradigms for music-emotion experiments.
2. ROBOTS AS EXPERIMENTAL INTERFACES
Existing music-emotion paradigms largely isolate pre-recorded sound despite evidence that perception involves visual, somatosensory, and social information. The paper argues that humanoid robots provide reproducible experimental interfaces for manipulating these components while complementing, rather than replacing, human-musician studies.
- Pre-recorded auditory paradigms have yielded insights but remain limited in manipulating embodied and interactive performance aspects under experimental control.
- An experimental interface observes changes in user responses after controlled variable manipulation, with robots providing structured and reproducible embodiments.
- Humanoid robots enable parametric control, repeatable performances, and decoupling or recombination of acoustic, visual, and interaction modalities.
- Robots are proposed as complementary tools for isolating specific multimodal mechanisms, not as replacements for the richness or ecological validity of human performance.
3. COGNITIVE MECHANISMS OF MUSIC-EVOKED EMOTIONS
Music-evoked emotion research must account for interacting tonal, metrical, affective, embodied, and social mechanisms. The paper argues that humanoid robots can dissociate and reproducibly manipulate these components to support controlled investigation.
- Tonal and metrical representations are shaped by sensory-acoustic constraints and learned stylistic regularities, while their interaction generates expectations that may elicit emotion.
- Music-emotion models describe responses from core affect and categorical emotions to higher-level evaluative processes.
- The BRECVEMA framework includes brain-stem reflex, expectancy, entrainment, visual imagery, emotional contagion, evaluative conditioning, episodic memory, and aesthetic appraisal.
- Because these mechanisms integrate auditory, visual, motor, and social information, research requires multimodal and embodied experimental approaches.
- Humanoid musical robots can reproducibly manipulate movement, gesture, perceived agency, and acoustic output to investigate mechanisms such as entrainment, contagion, and social interaction.
4. WHAT HUMANOID ROBOTS ENABLE
Humanoid robots can support controlled investigation of multimodal perception, emotion contagion, and social dynamics in music-evoked emotion. Their reproducibility and controllable physical actions make these mechanisms experimentally testable.
- Multimodal integration: Humanoid robots enable controlled study of music perception across auditory, visual, vibrotactile, and social modalities.They can reproduce performances exactly while reducing variability associated with human performers.
- Emotion contagion: A robot’s physical form provides a strong, modifiable prior for testing how anthropomorphism affects perceived expressive intent.This supports investigation of the boundary between musical expression and emotion contagion.
- Emotion contagion: Robot movement offers a controllable framework for testing which performer actions communicate musical expression.The approach can examine expressive touch, key motion, and conductor-like gestures systematically.
- Social dynamics: Multiple musical robots can create controlled social contexts for studying how synchrony and group interaction modify music-evoked emotions.Relevant outcomes include trust, cooperation, and differences between group and solo listening.
5. CASE STUDY: WAS-5 AS TECHNICAL FEASIBILITY EVIDENCE
The WAS-5 case study demonstrates technical feasibility for controlled music-emotion research through parametric acoustic control and closed-loop interaction. Its mechanisms support reproducible manipulation of sound production, agency, and interpersonal synchronization.
- System capabilities: WAS-5 is a humanoid saxophone robot with 31 degrees of freedom that mimics respiratory airflow and human facial and finger movements.It is designed to play a standard alto saxophone using biomimetic mechanisms.
- Acoustic control: The robot uses an air pump, proportional valves, and elastomeric tongue and lips to control airflow through the mouthpiece and oral cavity.These components simulate aspects of the human respiratory tract and embouchure.
- Acoustic control: Up to 33.8% increased sound pressure range produced dynamic ranges of 24.3 dB at E4 and 26.5 dB at F4.The eight-directional lip mechanism improves on the previous two-direction configuration.
- Closed-loop interaction: WAS-5 reliably adapted its tempo to participants in the Follower condition and systematically influenced their tapping in the Leader condition.The experiment demonstrates controlled manipulation of agency and interpersonal synchronization, not music-evoked emotion itself.
- Implications: Together, WAS-5 provides precise acoustic control, reproducible performance, and closed-loop interaction capabilities for manipulating multimodal and social variables.These capabilities constitute technical feasibility evidence for future music-emotion experiments.
6. EXPERIMENTAL PARADIGMS ENABLED BY THE SYSTEM
The proposed paradigms use humanoid robots to test multimodal integration, emotion contagion, and social interaction through exact performance reproduction and controlled manipulation of embodiment and synchrony.
- Research directions: The paradigms are experimentally testable research directions targeting multimodal integration, emotion contagion, and social interaction.Robots are used where exact reproduction, anthropomorphism control, and movement synchronization are required.
- Multimodal integration: One paradigm compares live robot performance with matched speaker presentation under isolated and visible conditions.The design tests effects of robotic embodiment while holding the musical performance as constant as possible.
- Emotion contagion: Another paradigm varies robot anthropomorphism through predefined changes in body sway, facial movements, and breathing-related torso motion while auditory output remains constant.Participants’ responses are collected after repeated presentations.
- Social interaction: A social-interaction paradigm varies whether audience robots synchronize in phase, out of phase, or independently with a performing robot.Measures include body movement, electrodermal activity, engagement, and arousal.
7. LIMITATIONS
The proposal is bounded by current robot technology, construction and calibration demands, and variability in people’s attitudes toward robots. These constraints affect feasible questions, implementation, and generalizability.
- Technical constraints: Current materials, sensors, and actuators constrain the experimental state-space and may restrict realistically testable questions.The authors note that the proposed paradigms remain within existing robots’ capabilities.
- Implementation constraints: Robot construction and calibration introduce technical and logistical hurdles that must be addressed before experiments.The paper frames this as a trade-off against the design freedoms offered by virtual reality.
- Generalizability: Participants’ receptivity to robots varies individually and culturally, limiting generalization across listeners, locations, and time.The authors recommend a multicultural approach to address this boundary.
- Scope of evidence: Table 1 summarizes the proposed paradigms, their targeted mechanisms, and the psychological outcomes that could be measured.It organizes the experimental scope rather than reporting completed empirical results.
8. CONCLUSION
The paper positions musical humanoid robots as experimental interfaces for music-evoked emotions. The WAS-5 case study demonstrates reproducible control enabling systematic manipulation of multimodal and social performance aspects.
- The paper introduces musical humanoid robots as experimental interfaces for investigating music-evoked emotions.
- The WAS-5 case study demonstrates reproducible control of embodied acoustic parameters and closed-loop interaction with human behaviour.
- This control allows systematic manipulation of multimodal and social aspects of musical performance.
- The paper outlines experimentally testable paradigms targeting embodiment, emotion contagion, and social co-regulation.