Source-linked AI summary
Evaluating perceptual transparency of lossy music audio compression with Opus codec
Benjamin Martin, Romain Hennequin
TL;DR
High-bitrate Opus transparency for music remains insufficiently evaluated, motivating a two-phase listening study with mixed-effects and equivalence analyses. The results show that 160 kbps withstands the critical-listener stress-test, while 128 kbps does not.
Problem
Method
The study uses two-phase ABC/HR testing, models Difference Grades with an LMM, and selects critical listeners and challenging samples using BLUPs.
Source: §4.1 · p. 2, §4.2 · p. 2, §4.4 · p. 3Results
160 kbps had no significant distinguishability in Phase 2 (p = 0.747) and passed strict equivalence testing (p = 0.024), whereas 128 kbps was detected as degraded (p = 0.030).
Source: §5 · pp. 3–4, §6 · p. 4Takeaways & Limitations
Under the tested critical-listening stress-test, 160 kbps was the minimum condition established as perceptually transparent to the reference.
Source: §5 · p. 3, §6 · p. 4The LMM assumes the subject, sample, and trial-level random components are mutually independent.
Source: §4.4 · p. 3
Abstract
from arXiv · showhide
Although the open-source Opus audio codec is widely used for digital audio streaming, its perceptual transparency threshold for high-bitrate music compression remains insufficiently evaluated. To establish this threshold, this study evaluates Opus across a two-phase ITU-R BS.1116-3 double-blind framework, analyzing Difference Grades using a Linear Mixed-Effects Model (LMM) and Two One-Sided Tests (TOST) for statistical equivalence. In a first phase with 21 qualified listeners and 23 music excerpts, 128 kbps achieves statistical in-distinguishability ($p = 0.456$) but narrowly fails strict equivalence to the uncompressed reference ($p = 0.064$). In Phase 2, Best Linear Unbiased Predictors (BLUPs) isolate the 10 most critical listeners and 8 most challenging samples for a stress-test. Under these worst-case conditions, 128 kbps is conclusively detected as degraded ($p = 0.030$), whereas 160 kbps successfully reaches perceptual transparency ($p = 0.747$) and strict statistical equivalence ($p = 0.024$). The study concludes that 160 kbps demonstrates perceptual transparency to uncompressed CD quality in critical listening environments. For reproducibility, we publicly release results for both phases, anonymized subject background info and sample characteristics.
1. INTRODUCTION
Perceptual audio coding enables efficient digital music distribution by exploiting limits of human hearing. This study asks what minimum Opus bitrate makes music subjectively indistinguishable from a lossless CD-quality reference.
1. INTRODUCTION
- Perceptual audio coding supports digital music distribution and streaming by exploiting physiological and cognitive limits of human hearing.Source: §1 · p. 1
- Opus combines SILK speech technology with CELT, designed for high-quality music, to scale from narrowband speech to fullband stereo music.Source: §1 · p. 1
- Results for the study are available from the linked Zenodo record.Source: §1 · p. 1
- The study asks what minimum bitrate lets Opus achieve perceptual transparency against a lossless CD-quality reference.The question addresses a gap in empirical evaluation of high-bitrate music compression.Source: §1 · p. 1
2. RELATED WORK
Prior Opus evaluations emphasize speech and lower bitrates, while music studies establish codec competitiveness but do not formally assess high-bitrate transparency with reproducible peer-reviewed results.
2. RELATED WORK
- Existing Opus perception studies predominantly concern lower-bitrate speech, while music evaluations have not formally established high-bitrate transparency in reproducible peer-reviewed results.Source: §2 · p. 1
- Hoene et al. tested speech and music at 64–96 kbps with MUSHRA, reporting Opus superiority to HE-AAC, Vorbis, and iLBC.Source: §2 · p. 1
- Other studies and Hydrogenaudio testing report Opus competitiveness for vocal and music signals, with the latter using ABC/HR.The paper identifies this study as the first formal assessment of high-bitrate Opus transparency for music.Source: §2 · p. 1
3. PERCEPTUAL TEST DESIGN
The test uses sensitive double-blind ABC/HR trials, controlled listening conditions, and challenging music excerpts to assess subtle compression artifacts.
3. PERCEPTUAL TEST DESIGN
- The study adopts ITU-R BS.1116-3 ABC/HR because the authors argue MUSHRA anchors can bias high-bitrate evaluation through range equalization and ceiling effects.Source: §3 · p. 2
- Listeners compare a disclosed uncompressed reference with two randomized blind excerpts, score on a 1–5 scale, and receive guided training before grading.Training uses increasing-difficulty material and encourages assigning exactly 5.0 to one blind stimulus; grading sessions are capped at 30 minutes with rest between sessions.Source: §3 · p. 2
- Grading takes place in isolated booths using calibrated headphones, with room noise and headphone isolation kept below the specified noise threshold.Source: §3 · p. 2
- The 23-excerpt corpus combines seven challenging benchmark samples with 16 curated excerpts spanning genres and difficult acoustic properties.Encoding uses libopus 1.6.1 in CELT music mode with unconstrained VBR; all tracks are normalized to -23 LUFS.Source: §3 · p. 2
- The volunteer panel comprises 26 Deezer employees with varied musical and audio experience, including 18 musicians and two self-described expert listeners.Source: §3 · p. 2
4. EXPERIMENTS
The experiments progress from a broad bitrate evaluation to a targeted stress-test, using screened listeners, Difference Grades, and a mixed-effects model for the sparse randomized observations.
4.1. Two phases
- Phase 1 evaluates 26 subjects and 23 excerpts at 32, 64, and 128 kbps; Phase 2 tests the 10 most critical listeners and eight most artifact-revealing excerpts across 32, 96, 128, and 160 kbps.Phase 2 uses 32 trials per subject to test the codec under worst-case conditions.Source: §4.1 · p. 2
4.2. Diffgrades
- Difference Grades subtract the reference grade from the compressed-sample grade, with negative values indicating correctly identified artifacts and transparency corresponding to an expected mean of zero.Source: §4.2 · p. 2
4.3. Subject post-screening
- Post-screening removes listeners with excessive undecided ties or who fail a one-sided 32 kbps discrimination test at α = 0.05.Source: §4.3 · p. 3
4.4. Linear mixed model
- The LMM models bitrate as a fixed effect and subject and sample as crossed random intercepts, accommodating sparse randomized observations while separating degradation from listener and stimulus variability.Cell-means coding estimates an independent fixed effect for each bitrate; the random components are assumed mutually independent.Source: §4.4 · p. 3
5. RESULTS
Phase 2’s worst-case test found that 128 kbps was detectably degraded, while 160 kbps met the study’s perceptual-transparency criteria. Phase 1 showed no significant degradation at 128 kbps, but it narrowly failed strict equivalence.
5. RESULTS
- Phase 1’s 128 kbps mean Diffgrade was −0.082 (p = 0.456), but it narrowly failed strict equivalence (TOST p = 0.064).The analysis followed screening that left 21 participants.Source: §5 · pp. 3–4
- In Phase 1, 64 kbps showed mild impairment (mean Diffgrade −0.438, p < 0.001), while 32 kbps showed severe artifacts (mean −1.661, p < 0.001).Both estimates are from the analysis of 21 participants.Source: §5 · p. 3
- BLUPs selected the 10 most critical listeners and 8 most artifact-revealing samples for the Phase 2 stress-test.Figure 1 orders subjects by criticality and samples by audibility of artifacts, with highlighted regions marking the selected candidates.Source: §5 · p. 3, Figure 1 · p. 4
- Phase 2 variance estimates were σsubject = 0.118 (SE = 0.020) and σtrack = 0.195 (SE = 0.040), indicating a uniformly strict panel and challenging stimuli.The authors describe these reduced variances as validating the BLUP selection.Source: §5 · pp. 3–4
- 160 kbps alone passed Phase 2 strict equivalence (TOST p = 0.024), while 128 kbps was detected as degraded (p = 0.030) and failed equivalence (p = 0.450).The Phase 2 mean Diffgrades were −0.035 for 160 kbps and −0.236 for 128 kbps.Source: §5 · pp. 3–4
6. CONCLUSION
The study concludes that 160 kbps is required to reach perceptual equivalence with uncompressed audio under its critical listening test conditions. The authors note that broader equipment and expert sampling would help assess transparency across real-world acoustic scenarios.
6. CONCLUSION
- A two-phase ABC/HR evaluation and LMM were used to isolate codec degradation from subject and sample variance.The analysis also used TOST to assess statistical equivalence.Source: §6 · p. 4
- 160 kbps was the only condition established as perceptually equivalent under the critical Phase 2 test (TOST p = 0.024).Figure 2 presents TOST equivalence results for all bitrate conditions against ∆ = ±0.25; green intervals indicate equivalence to uncompressed quality.Source: §6 · p. 4, Figure 2 · p. 4
- The authors recommend further study with varied audiophile equipment and broader expert sampling before extending the conclusion across real-world acoustic scenarios.Source: §6 · p. 4