Source-linked AI summary
Physiology-Informed Audio Quality and Listening Effort Estimation
Niall McGuire, Dimitra Emmanouilidou
TL;DR
MOS and audio-based prediction do not capture individual differences in perceived quality, while prior physiological decoding has relied on laboratory equipment. This study compares laboratory and consumer sensing across thirty listeners; wearable and optical streams decode quality, while listening effort is recovered mainly after listener calibration.
Problem
MOS is costly and collected after playback, while audio-based predictors yield population-average scores that discard subject-specific variation.
Source: §1 · p. 1Method
The study records laboratory EEG, wearable EEG, fNIRS, and eye tracking from listeners hearing graded-quality audio, relating physiological features to objective quality labels and each listener’s effort ratings.
Source: §1 · p. 1Results
Wearable EEG, fNIRS, and eye tracking exceeded chance for unseen-listener binary quality prediction at balanced accuracy 0.60 to 0.65; effort decoding appeared mainly under listener-dependent prediction.
Source: §3 · p. 3Takeaways & Limitations
Physiological sensing may help personalize audio experiences to both quality perception and the effort required to process a clip.
Source: §1 · p. 1The same clips recur across evaluation folds, so the results establish cross-listener transfer rather than generalisation to unseen audio.
Source: §2.4 · pp. 2–3
Abstract
from arXiv · showhide
Audio systems aim to maximise the listener's perceived quality, traditionally evaluated through mean opinion scoring (MOS), a subjective process that is slow, costly, and collected only after playback. Machine learning now predicts MOS directly from the audio, but in generalising across participants it tends to discard the subject variance that defines individual perception. Neurophysiological measures may offer a complementary signal taken directly from the listener, objective, implicit, and continuous, though prior work has relied on laboratory-grade EEG and fNIRS. We ask how practical consumer-grade devices compare, and whether they might recover population-level audio-quality trends and, when calibrated per listener, track that listener's effort. Recording 32-channel laboratory EEG alongside wearable EEG, fNIRS, and eye tracking from thirty listeners, our results suggest commercial wearable and optical sensing can decode audio quality at statistically significant levels and recover listening effort per listener. These findings may point towards platforms tuned not only to perceived quality but to the effort a clip demands, read from the listener's own response as they listen.
1. INTRODUCTION
The paper asks whether practical physiological sensing can recover audio-quality trends and listening effort, addressing the limits of population-average MOS prediction and laboratory-bound prior work. It compares laboratory and consumer sensing for quality and effort, with effort framed as a dimension that quality ratings alone can miss.
1. INTRODUCTION
- MOS varies with listeners’ preferences, experience, and scale interpretation, requiring costly panels and post-playback ratings, while audio-based predictors still discard subject-specific variation.Reference-based metrics also require a clean reference unavailable during live playback; reference-free models use only degraded audio.Source: §1 · p. 1
- EEG and fNIRS offer ongoing, implicit physiological signals, with EEG resolving millisecond activity and fNIRS’s slower response potentially tracking effort or demand.Prior work had used laboratory-grade equipment; this study asks whether consumer-grade sensing can recover quality and effort.Source: §1 · p. 1
- Listening effort can differ between clips with the same quality rating, and sustained effort may accumulate into fatigue and disengagement.The paper notes that pupil dilation can increase with degradation even when trials reach the same correct answer.Source: §1 · p. 1
- The feasibility-focused comparison records laboratory EEG, wearable EEG, fNIRS, and eye tracking while listeners hear graded-quality audio, relating features to objective quality labels and each listener’s effort ratings.Evaluation includes unseen and calibrated listeners.Source: §1 · p. 1
2. METHODS
The study records four physiological streams while thirty listeners hear speech and music clips with varying quality, rate quality or effort, and complete a concurrent visual task. It extracts modality-specific features and evaluates binary and graded quality and effort prediction under unseen-listener and within-listener regimes.
2.1. Participants, Apparatus, and Stimuli
- Thirty listeners were considered across tasks; recordings combined 32-channel laboratory EEG, four-channel wearable EEG, wearable fNIRS, and eye tracking, with 144 speech and music clips.The wearable EEG and fNIRS came from a consumer headband; stimuli were loudness-normalised and playback gain was held fixed.Source: §2.1 · p. 2
2.2. Procedure
- Listeners rated quality and effort on graded trials and made categorical quality judgements with ordinal effort ratings on extreme trials, while tracking moving visual targets.Quality targets came from corpus MOS or clean/degraded labels; effort targets came from the listener’s ratings.Source: §2.2 · p. 2
2.3. Physiological Processing and Features
- Signals were baseline-corrected and processed into modality-specific features, including EEG band power and covariance, fNIRS response measures, and gaze and pupil measures.The wearable fNIRS optodes sit over prefrontal cortex, and the device lacks a short-separation reference, so systemic haemodynamics may mix with the cortical signal.Source: §2.3 · p. 2
2.4. Prediction Tasks and Evaluation
- Four tasks predict binary or five-point audio quality and listening effort, scored with balanced accuracy for binary targets and correlation and RMSE for continuous targets.Quality targets use objective corpus labels, whereas effort targets use listener ratings.Source: §2.4 · p. 2
- Leave-one-subject-out evaluation tests unseen listeners, while pooled within-subject folds test listener-calibrated prediction; both reuse the fixed clip set, so they do not establish generalisation to unseen audio.The within-subject pooled correlation can be inflated by stable between-listener offsets; modality and support-condition differences were not tested against each other.Source: §2.4 · pp. 2–3
3. RESULTS AND DISCUSSION
Physiological signals recover audio quality across listeners, while effort is mainly decodable after listener calibration. Results also vary by modality and content, and objective-MOS supervision improves pooled quality metrics while leaving individual listener ordering unresolved.
Recovering audio quality (RQ1)
- Wearable EEG, fNIRS, and eye tracking exceed chance for unseen listeners at balanced accuracy 0.60 to 0.65; wearable EEG and fNIRS correlate with corpus MOS at r 0.38 and 0.36.For five-point quality prediction, RMSE is near 1.18 to 1.20 MOS points; subject-dependent correlations rise to 0.41 to 0.43. Laboratory EEG does not exceed chance or correlate with MOS for unseen listeners.Source: §3 · p. 3
From laboratory to wearable sensing (RQ3)
- In the matched-trial condition, fNIRS alone exceeds chance (0.65) and correlates with MOS (r 0.19) for unseen listeners.The broader modality comparison is confounded by differences in trial counts, channel counts, protocol, and electrode stability, so the authors do not claim wearables outperform laboratory EEG.Source: §3 · p. 3
Decoding listening effort (RQ2)
- Effort decoding appears mainly after listener calibration; eye tracking is strongest for graded ratings (r 0.37, RMSE 0.89), while unseen-listener five-point decoding is nonsignificant after correction.For unseen listeners, only fNIRS reaches significance on binary effort, with a small effect at 0.52; rated effort also correlates negatively with quality (r −0.57).Source: §3 · p. 3
Supplementary analyses
- Objective-MOS supervision improves pooled RMSE and correlation over listener-rating supervision, but the comparison is descriptive and does not isolate supervision as the cause.Objective-trained predictions show 55 to 94 percent lower within-MOS dispersion, yet within-listener rank correlation falls from around 0.79 to between 0.12 and 0.33, leaving individual ordering unresolved.Source: §3 · p. 4, Table 4 · p. 4
Speech versus music
- Speech carries the quality signal more strongly than music: participant-mean balanced accuracy is 0.69 to 0.83 for speech versus 0.38 to 0.62 for music.This is a descriptive, single-seed comparison, and equalised trial counts leave some content-modality groups with very few listeners.Source: §3 · p. 4, Table 3 · p. 4
4. CONCLUSION
Physiological sensing decoded binary audio quality above chance across unseen listeners, while graded MOS correlations were lower but significant. Effort was decodable per listener, especially with eye tracking and fNIRS, but did not yet transfer to unseen listeners.
4. CONCLUSION
- Binary quality was decoded above chance across unseen listeners with wearable EEG, fNIRS, and eye tracking, while graded MOS showed lower but significant correlations.With support equalised, fNIRS was most consistent; laboratory EEG highlighted practical costs of high-density recording.Source: §4 · p. 4
- Effort was decodable per listener, most strongly with eye tracking and fNIRS, but did not yet transfer to unseen listeners.The signals recovered the corpus-level quality axis and, after listener calibration, that listener’s effort ratings.Source: §4 · p. 4
- The results do not establish recovery of an individual listener’s own quality ordering, which remains open.The authors describe these as feasibility results, noting modest correlations and uneven cohorts across modalities.Source: §4 · p. 4
- The authors identify DNSMOS stimulus-ceiling bounds and leave-one-clip-out testing for unseen-audio generalisation as natural next steps.They suggest the findings may inform systems tuned to both perceived quality and processing effort.Source: §4 · p. 4