Source-linked AI summary
SEMAC: Spherical Energy Matching Ambisonics Codec for Efficient Higher-Order Ambisonics Compression
Christoph Hold, Jan Buethe, Jean-Marc Valin, Jan Skoglund, Toni Hirvonen, Nils Peters
TL;DR
HOA channel-wise coding becomes costly as channel count grows quadratically with order. SEMAC codes a full-order directional energy envelope alongside fewer CELT-coded transport channels and restores the envelope after decoding. In fifth-order MUSHRA comparisons, 16-channel SEMAC outperforms full-order CMF 3 and equal-count order truncation.
Problem
HOA’s channel count grows quadratically with spherical-harmonic order, making channel-wise coding costly at fixed per-channel rate.
Source: §1 · p. 1Method
SEMAC codes a full-order directional energy envelope separately from reduced-dimensional waveform transport and reapplies it after decoding.
Source: §1 · p. 1Results
At equal transport count SEMAC exceeds order-truncated CMF 3, and 16-channel SEMAC exceeds 36-channel full-order CMF 3 at both tested bitrates.
Source: §3.2 · p. 4Takeaways & Limitations
At the tested rates, allocating bitstream capacity to a directional energy envelope is more effective than coding every full-order waveform channel at low precision.
Source: §3.2 · p. 4At fifth order, envelope quantization can exceed 100 kbit/s, while reduced transport loses within-band directional structure and cannot restore absent phase relationships or detail.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
Higher-order Ambisonics (HOA) is an attractive format for immersive spatial audio, but its high channel count and directional properties present significant audio coding challenges. The widely deployed Opus audio codec supports HOA through Channel Mapping Families 2 and 3 (RFC 8486), which code and transmit every Ambisonics channel, so the bitrate grows quadratically with the spherical harmonic order. We introduce SEMAC, which instead represents an Ambisonics signal by a quantized full-order directional energy envelope together with fewer CELT-coded transport channels. CELT separates a per-band energy envelope from the signal shape and restores it after quantization, and SEMAC generalizes this principle to directional signals. Since the reduced transport spans only a lower-dimensional subspace, the decoder re-imposes the envelope observed at the encoder input, restoring the directional energy distribution at full order. A MUSHRA test on fifth-order material compares SEMAC against Opus CMF~3 in matched configurations at 512 and 768 kbit/s. SEMAC with 16 of 36 transport channels is rated above full-order CMF~3 at both rates, and at equal transport channel count above order truncation. The proposed SEMAC strategy of coding fewer transport channels with the directional envelope is found perceptually preferable to coding all HOA channels at low precision.
1. INTRODUCTION
HOA channel counts grow quadratically with spherical-harmonic order, making channel-wise coding costly; SEMAC instead sends a full-order directional energy envelope alongside fewer waveform transport channels. It builds on preserving directional energy as a key perceptual feature and evaluates envelope restoration against Opus CMF 3.
Motivation
- With fixed per-channel rate, HOA’s C = (N+1)^2 channels make aggregate coding rate grow quadratically with order.CMF 2 codes channels independently, while CMF 3 beamforms them before coding; both transmit all C signals.Source: §1 · p. 1
- Existing channel-reduction approaches use inter-channel transforms or parametric scene descriptions, whose reconstruction quality generally depends on estimated scene parameters.Examples include PCA or covariance-based transforms and estimates of source count, direction, or diffuseness.Source: §1 · p. 1
- SEMAC treats directional energy composition as a key perceptual feature and separately codes its full-order envelope and waveform transport.It generalizes CELT’s separate coding and restoration of band energy and within-band shape to the directional domain.Source: §1 · p. 1
Contributions
- The work reduces transport channels, restores the full-order directional envelope on the reduced reconstruction, and evaluates fifth-order SEMAC against fifth-order Opus CMF 3.The matched configurations are intended to isolate the perceptual benefit of envelope restoration.Source: §1 · p. 1
2. SEMAC CODEC
SEMAC analyzes HOA signals as directional beams, codes per-band directional magnitudes separately from reduced waveform transport, and restores those magnitudes after transport expansion. Its codec combines CELT-style envelope coding with a reduced-dimensional reconstruction and full-order directional correction.
2.1. Directional Analysis and Envelope
- For each beam and CELT band, SEMAC stores a sampled directional envelope, with total rate equal to CELT transport rate plus envelope side-information rate.The envelope records per-band magnitudes, not explicit direction-of-arrival or source-count estimates.Source: §2.1 · p. 2
2.2. Envelope Coding
- SEMAC predicts coarse log-energy from the preceding frame and band, entropy-codes residuals on a 6 dB grid, and uses fine bits to refine selected bands.Fine-bit precision follows available per-transport-channel rate; envelope rate also rises with input order, so the waveform/metadata split matters.Source: §2.2 · p. 2
2.3. Reduced Audio Channel Transport
- Transport projects the signal to fewer channels and reconstructs a rank-limited signal; SEMAC permits SH-domain or directionally selective transport, while experiments use beams to match CMF 3.Directionally selective channels can represent few-source scenes compactly; SEMAC also biases CELT allocation using decoded band magnitudes without allocation side information.Source: §2.3 · p. 3
2.4. Envelope-Guided Reconstruction
- The decoder measures the expanded transport at full directional resolution, then rescales each beam to its decoded input envelope before projecting back to the SH domain.The correction restores the full-order directional energy distribution on a rank-limited reconstruction.Source: §2.4 · p. 3
2.5. Relation to CELT and CMF 3
- Unlike CMF 3’s fixed projection with all waveform dimensions, SEMAC reduces transport dimensions and restores spherical directional band magnitudes after expansion.CELT controls each transport channel’s spectral magnitude, while SEMAC controls directional band magnitudes after full-order reconstruction.Source: §2.5 · p. 3
3. PERCEPTUAL EVALUATION
The evaluation compares fifth-order SEMAC with matched Opus CMF 3 configurations in a MUSHRA test at two bitrates. SEMAC with 16 transport channels outperformed full-order CMF 3 at both rates, while order truncation and limits in the current implementation qualify the findings.
3.1. Setup
- The test used 11 screened listeners, seven preselected spatial-audio items, and fifth-order 48-kHz signals rated at 512 and 768 kbit/s.The hidden reference and a 3.5-kHz low-pass anchor appeared in each trial; outputs were rendered binaurally and scored against the uncoded fifth-order signal through the same path.Source: §3.1 · pp. 3–4, §3.2 · p. 4
3.2. Results and Discussion
- SEMAC ratings rose with transport-channel count; 16 channels reached the Excellent range, while 9- and 4-channel configurations performed similarly across bitrates.For the 16-channel configuration, 768 kbit/s suggested improvement over 512 kbit/s; the authors suggest that channel count, rather than bitrate, bounds quality with greatly reduced transport.Source: §3.2 · p. 4
- Full-order CMF 3 showed no clear advantage over nine-channel CMF3-SH2 despite using four times as many channels, and its ratings improved with bitrate.The authors interpret the pattern as consistent with insufficient per-channel bitrate when distributing the rate across 36 channels.Source: §3.2 · p. 4
- At both tested bitrates, 16-channel SEMAC had higher medians and tighter rating spread than full-order CMF 3 with 36 channels.The comparison used fifth-order input; Fig. 2 shows the MUSHRA rating distributions.Source: §3.2 · p. 4, Figure 2 · p. 4
- At equal transport-channel count, SEMAC was rated above order-truncated CMF 3, while naive truncation showed clear quality deterioration.The paper identifies the comparison between CMF3-SH2 and SEMAC-BEAM9, both with nine transport channels.Source: §3.2 · p. 4
- At fifth order, envelope quantization could exceed 100 kbit/s and starve the core coder; the optimal waveform–metadata rate split remains unresolved.The decoder cannot recover phase relationships or directional detail absent from the reduced transport.Source: §3.2 · p. 4
- A more comprehensive evaluation and comparison, despite informal listening and objective-model assessment on a larger set including loudspeaker playback, was left for future work.Source: §3.2 · p. 4
4. CONCLUSION
SEMAC combines reduced CELT waveform transport with a quantized directional energy envelope measured at the encoder input. In a fifth-order MUSHRA test, its 16-channel configuration outperformed full-order Opus CMF 3 at both tested bitrates and beat order truncation at equal transport-channel count.
4. CONCLUSION
- In a fifth-order MUSHRA test, 16-channel SEMAC was rated above full-order Opus CMF 3 at both tested bitrates and above order truncation at equal transport-channel count.Source: §4 · p. 4
- SEMAC adds a quantized directional energy envelope measured at the encoder input to mono and stereo CELT waveform coding.Source: §4 · p. 4
- The decoder re-imposes the envelope to restore the directional energy distribution after reduced transport introduces directional error.Source: §4 · p. 4