PaperlayerView exact revision on arXiv

Source-linked AI summary

Measurement-Based Bitrate-Energy-Quality Analysis of Neural Audio Codec Decoders on Laptop and Phone Platforms

Seunghyeon Shin, Seokjin Lee

arXiv:2609.34524v1eess.ASeess.SP

TL;DR

Neural codecs can offer similar objective quality at lower bitrates, but their decoder cost may offset that advantage on battery-powered devices. This paper measures codec energy, quality, and bitrate across laptop and phone paths, finding that matched-quality neural points generally use more decoder energy than conventional references. The results make the energy trade-off dependent on the effective decoder path and transmission-energy coefficient.

  • Problem

    Prior codec evaluations commonly report bitrate, quality, complexity, or latency, while evidence on decoder-side energy and effective execution on target devices is limited.

    Source: §I · p. 1

  • Method

    The study measures decoder energy and original-reference ViSQOL for neural and conventional codecs, selects matched-quality or explicitly evaluated bitrate points, and screens runtime paths for effective execution.

    Source: §I · p. 1, §III · p. 3

  • Results

    At matched-quality points, neural codecs achieved similar ViSQOL at lower bitrates but generally required more decoder energy; EnCodec had the lowest neural break-even thresholds in both cohorts.

    Source: §VII · p. 9, §VIII · p. 10

  • Takeaways & Limitations

    Whether lower bitrate yields an energy benefit depends on decoder complexity, effective runtime and device mapping, execution validity, accelerator availability, and the transmission-energy coefficient.

    Source: §I · p. 1, §VIII · p. 10

    Measurements came from one laptop and one phone, and ViSQOL operating points do not establish exact subjective equivalence.

    Source: §VII · p. 10

Abstract

from arXiv · show

Neural audio codecs can achieve similar objective quality at lower bitrates than conventional codecs, but their decoder-side computational cost may offset this bitrate advantage on battery-powered client devices. This paper presents a measurement-based rate--energy--quality analysis of four neural audio codecs--EnCodec, DAC, HILCodec, and SNAC--and two conventional baselines, AAC-LC and Opus, on laptop and phone platforms. Speech and music are evaluated separately using the original-reference ViSQOL protocol. For the main comparison, operating points are matched by selecting the measured point nearest to the midpoint of the common overlap in treatment-level mean ViSQOL. A secondary analysis includes only explicitly evaluated bitrate settings. Decoder energy is measured as idle-subtracted energy per second of audio (J/s), and results are summarized as the median of three repeated runs for execution-valid runtime and device paths. Pairwise break-even transmission-energy thresholds are then derived analytically from the measured decoder energy and bitrate. At the matched-quality operating points, the evaluated neural codecs achieved similar ViSQOL scores at lower bitrates but generally required more decoder-side energy than the applicable conventional codecs. EnCodec produced the lowest neural break-even thresholds in both matched-quality cohorts on both platforms. By contrast, several DAC and SNAC comparisons on the Phone XNNPACK CPU path exceeded 200 mJ/kbit, and their full-band Phone configurations required more than 1 s to decode 1 s of audio. These results show that lower bitrate alone does not guarantee an energy benefit: deployment efficiency also depends on decoder complexity, the effective runtime and device mapping, execution validity, accelerator availability, and the transmission-energy coefficient.

I. INTRODUCTION

The paper frames decoder energy and effective device execution as gaps in neural-codec evaluation, since bitrate and quality advantages may not translate into lower client energy. It extends this evaluation to neural and conventional codecs, whose architectures and operating points shape the comparison.

I. INTRODUCTION

  • The study measures decoder energy alongside original-reference quality and bitrate, while checking whether accelerator-targeted graphs execute on the requested device.Earlier neural-codec studies commonly report bitrate, quality, model complexity, or latency, but not decoder-side energy or effective device execution.Source: §I · p. 1
  • For repeated playback, transmission and client decoding recur, so bitrate savings must compensate for any additional decoder-side energy to yield an overall energy benefit.Server-side encoding is typically performed once per content item, unlike transmission and client-side decoding during playback.Source: §I · p. 1

A. Audio Codec Architectures and Operating Points

  • EnCodec uses streamable residual vector quantization, DAC extends RVQGAN, HILCodec uses causal depthwise-separable convolutions, and SNAC quantizes at multiple temporal resolutions.These designs are described alongside SoundStream’s fully convolutional encoder-decoder and residual vector quantization.Source: A. Audio Codec Architectures and… · p. 2
  • Reducing transmitted codes may not reduce waveform-synthesis operations proportionally, so decoder cost is measured directly rather than inferred from quantizer configuration.The evaluated rate ladders mainly vary active RVQ depth for EnCodec, DAC, and HILCodec; SNAC distributes retained codes across temporal resolutions.Source: A. Audio Codec Architectures and… · p. 2
  • AAC-LC and Opus provide practical conventional references, evaluated at nominal encoder settings spanning 12–128 and 12–96 kbps, respectively.AAC-LC uses MDCT-based transform coding; Opus combines SILK linear-prediction tools with the MDCT-based CELT layer.Source: A. Audio Codec Architectures and… · p. 2

B. Decoder Energy and Runtime Mapping

The paper extends decoder-energy measurement to neural and conventional audio codecs across runtime and device paths, recording effective execution. Its framework models decoder and transmission energy, selects operating points by quality overlap or explicitly evaluated bitrate, and screens invalid execution paths.

B. Decoder Energy and Runtime Mapping

  • The study directly measures decoder-side energy across neural and conventional audio codec paths, selects discrete points in original-reference ViSQOL space, and records effective execution.Transmission energy remains a parametric sensitivity term.Source: B. Decoder Energy and Runtime Mapping · p. 2

III. ANALYSIS FRAMEWORK

  • The main comparison selects measured operating points by overlap in ViSQOL space, while the secondary bitrate-specified analysis uses only explicitly evaluated points without interpolation or substitution.This avoids treating equal bitrate as equivalent perceptual quality.Source: §III · p. 2
  • Total modeled energy combines client-side decoder energy with transmission energy, with the execution configuration specifying platform, runtime, target device, and precision where applicable.The analysis considers one-to-many playback and excludes server-side encoding, which is performed once per content item.Source: §III · p. 2
  • Decoder energy is normalized by decoded-audio duration after subtracting idle energy, yielding edec in joules per second of audio.This is numerically equivalent to average additional decoding power under 1× playback, but does not imply that every path meets real-time latency requirements.Source: §III · p. 2
  • Pairwise break-even thresholds equate transmission savings from bitrate reduction with the decoder-energy difference, treating the transmission-energy coefficient as a parameter.A lower-bitrate configuration has lower modeled transmission-plus-decoding energy when the coefficient exceeds its threshold.Source: §III · p. 3
  • Paths that fail or do not meet execution-validity criteria are excluded from the main comparison and reported separately as diagnostics.Latency and fallback ratio are execution diagnostics rather than primary metrics.Source: §III · p. 3

A. Hardware Platforms

The study benchmarks neural and conventional decoder paths on Laptop and Phone, whose energy measurements use different telemetry layers. Runtime paths are evaluated separately, with only execution-valid paths included in the main comparison.

A. Hardware Platforms

  • Laptop energy uses processor package-level telemetry, while Phone energy uses device battery-discharge telemetry, so absolute energy values are compared only within each platform.Laptop measurements were taken with external power connected; Phone measurements were taken on battery with airplane mode enabled and other listed features disabled.Source: A. Hardware Platforms · p. 3

B. Software Stack and Runtime Configuration

  • Neural decoders use ONNX Runtime and OpenVINO on Laptop and ONNX Runtime on Phone, while conventional decoders use FFmpeg on Laptop and MediaCodec on Phone.Phone CPU paths use XNNPACK, and accelerator-targeted paths use QNN.Source: B. Software Stack and Runtime… · p. 4
  • Only paths with successful session creation and completed decoding enter the main comparison; accelerator-targeted paths are excluded when runtime evidence shows the requested target was not effectively used.Partial non-target execution is retained and classified as mixed execution.Source: B. Software Stack and Runtime… · p. 4

C. Codec Instances and Evaluated Operating Points

The evaluation compares four neural codec families with AAC-LC and Opus using their evaluated operating points. Speech and music are treated as separate quality domains, with matched-quality and explicitly bitrate-specified analyses.

C. Codec Instances and Evaluated Operating Points

  • EnCodec, DAC, and HILCodec operating points vary active RVQ depth, whereas SNAC points retain subsets of scales or codebooks from released checkpoints.SNAC configurations are treated as retained-scale-derived points, not as separately trained and officially released low-bitrate checkpoints.Source: C. Codec Instances and Evaluated… · p. 4
  • AAC-LC and Opus operating points are defined by encoder bitrate presets, with decoder inputs held constant across runtime paths.Table III summarizes the evaluated codec instances and bitrate settings.Source: C. Codec Instances and Evaluated… · p. 4, Table III · p. 4

D. Evaluation Datasets, Quality Protocol, and Operating-Point Selection

  • Speech uses VCTK with ViSQOL speech mode, while music uses MUSDB18-HQ with ViSQOL audio mode; their scores are not pooled.Each domain uses 100 evaluation clips of 10 seconds, resampled to each codec instance’s native sampling rate before processing.Source: D. Evaluation Datasets, Quality… · p. 4
  • Original-reference ViSQOL compares decoded output with the original clip resampled to the mode’s canonical rate, so codec bandwidth loss contributes to measured quality degradation.Speech is evaluated at 16 kHz and music at 48 kHz; decoded signals are resampled from the codec’s native sampling rate.Source: D. Evaluation Datasets, Quality… · p. 4
  • Matched-quality points use the measured bitrate point nearest the midpoint of the common treatment-level mean ViSQOL overlap; the secondary analysis includes only explicitly evaluated bitrate settings.The secondary analysis does not interpolate, substitute a neighboring bitrate, or use nearest-bitrate matching; codecs without a point are omitted from that condition.Source: D. Evaluation Datasets, Quality… · p. 5

V. MEASUREMENT PROTOCOL

The benchmark measures steady-state decoder energy after subtracting an idle baseline. Laptop and Phone use platform-specific telemetry, and the main metric normalizes energy by decoded-audio duration.

A. Benchmark Procedure

  • The benchmark cycles through 32 pregenerated one-second artifacts and measures steady-state decoding through reconstructed audio output, excluding setup, diagnostic profiling, and post-run logging.Neural paths receive code indices and optional scale values; conventional paths receive preloaded compressed inputs.Source: A. Benchmark Procedure · p. 5
  • Each configuration proceeds through initialization, warm-up, timing probe, cooldowns, idle-baseline acquisition, and measured decoding; probe results can increase iterations to satisfy the minimum measurement duration.Session-creation and execution failures are retained as failure records.Source: A. Benchmark Procedure · p. 5

B. Energy Measurement

  • The main energy metric is idle-subtracted decoder energy per decoded-audio duration, derived from run energy and idle power.The common recorded quantities include telemetry samples and timestamps, run duration, decoded-audio duration, and execution status.Source: B. Energy Measurement · p. 5
  • Laptop energy integrates processor package-level power samples between effective measurement boundaries, with idle power estimated from repeated five-second intervals.Power samples are treated as interval averages, and the main metric is net energy after idle-baseline subtraction.Source: B. Energy Measurement · p. 5
  • Phone energy integrates sampled current and voltage using the trapezoidal rule, then subtracts an idle-power baseline estimated from repeated five-second intervals.Phone current and voltage telemetry is sampled at one-second intervals.Source: B. Energy Measurement · p. 5
  • Absolute energy values are interpreted only within each platform because Laptop and Phone measurements use different telemetry layers.Laptop measurements use processor package-level energy; Phone measurements use device-level battery-discharge energy.Source: B. Energy Measurement · p. 5

C. Result Aggregation

Decoder energy is summarized across repeated runs, while quality anchors are based on overlap in treatment-level mean ViSQOL. The selected discrete points are not exactly quality-equivalent, which qualifies bitrate and break-even comparisons.

C. Result Aggregation

  • Decoder energy is represented by the median of three runs, with the observed minimum and maximum retained as run-to-run diagnostics.Bootstrap confidence intervals from 10,000 source-track resamples visualize rate–quality uncertainty but are not used to select operating points.Source: C. Result Aggregation · p. 5
  • Pairwise break-even thresholds use selected bitrate and platform-specific median decoder energy; bitrate-specified panels omit codecs without evaluated points and require at least two series.Energy-figure whiskers show observed minimum and maximum, not confidence intervals.Source: C. Result Aggregation · p. 6

A. Rate–Quality Curves and Overlap Feasibility

  • 3.858 MOS-LQO is the full-band music matched-quality anchor, the midpoint of the 3.774–3.943 common overlap interval.The selected point for each codec instance is its measured discrete bitrate point nearest that anchor.Source: A. Rate–Quality Curves and Overlap… · p. 6, Figure 3 · p. 6
  • 3.357 MOS-LQO is the 24-kHz speech matched-quality anchor, the midpoint of the 2.903–3.810 common overlap interval.The cohort includes AAC-LC, EnCodec, DAC, and HILCodec; Opus lacks a point in the overlap and SNAC’s attainable quality is below it.Source: A. Rate–Quality Curves and Overlap… · p. 6, Figure 4 · p. 6
  • Speech selected EnCodec at 6 kbps, HILCodec at 3 kbps, DAC at 6 kbps, and AAC-LC at 16 kbps; DAC lies above and AAC-LC below the anchor.The selected points are measured discrete operating points.Source: A. Rate–Quality Curves and Overlap… · p. 6
  • Selected points are not exactly quality-equivalent: music AAC-LC and Opus lie above both the anchor and EnCodec, while speech AAC-LC lies below the anchor and neural points are at or above it.The resulting thresholds apply to selected measured points rather than exact perceptual equivalence; the point positions can bias apparent neural bitrate savings in opposite directions across the two domains.Source: A. Rate–Quality Curves and Overlap… · p. 6

B. Quality-Anchor Decoder Energy

At the selected quality-anchor points, neural codecs generally used more decoder energy than the applicable conventional codecs on both platforms. EnCodec had the lowest decoder energy among the selected neural speech codecs on Phone.

B. Quality-Anchor Decoder Energy

  • On Laptop music, EnCodec, DAC, and SNAC used 0.123, 0.947, and 1.47 J/s, versus 0.0103 J/s for AAC-LC and 0.0242 J/s for Opus.The DAC GPU-requested path included approximately 45% non-target execution and is interpreted as mixed execution.Source: B. Quality-Anchor Decoder Energy · p. 7, Figure 5 · p. 7
  • On Phone music, EnCodec, DAC, and SNAC used 0.570, 6.21, and 6.96 J/s, versus 0.0142 J/s for AAC-LC and 0.0162 J/s for Opus.Neural codecs used XNNPACK CPU; conventional codecs used the MediaCodec system path.Source: B. Quality-Anchor Decoder Energy · p. 7, Figure 5 · p. 7
  • On Laptop speech, EnCodec, HILCodec, and DAC used 0.0527, 0.319, and 0.529 J/s, versus 0.00750 J/s for AAC-LC.The DAC path included approximately 47% non-target execution and is interpreted as mixed execution.Source: B. Quality-Anchor Decoder Energy · p. 7, Figure 6 · p. 7
  • On Phone speech, EnCodec, HILCodec, and DAC used 0.257, 1.74, and 4.08 J/s, versus 0.00909 J/s for AAC-LC; EnCodec was the lowest-energy selected neural codec.The observed AAC-LC range was 0.00866–0.0251 J/s.Source: B. Quality-Anchor Decoder Energy · p. 7, Figure 6 · p. 7

C. Decoder Energy at Specified Bitrate Settings

At explicitly evaluated bitrate settings, decoder energy and ViSQOL varied across codecs and bitrates, and lower energy did not always accompany higher quality. These comparisons do not align codecs by perceptual quality.

C. Decoder Energy at Specified Bitrate Settings

  • At 12 kbps for full-band music, EnCodec scored 4.22 ViSQOL versus 1.49 for AAC-LC and 1.93 for Opus, while requiring more decoder energy on both platforms.EnCodec used 0.123 J/s on Laptop and 0.593 J/s on Phone; the corresponding AAC-LC and Opus values were lower.Source: C. Decoder Energy at Specified Bitrate… · p. 7
  • At 3 kbps for 24-kHz speech, HILCodec scored highest at 3.41 ViSQOL; at 6 kbps, DAC and HILCodec scored 3.86 and 3.83, above EnCodec's 3.41.EnCodec nevertheless had the lowest neural decoder energy at 6 kbps: 0.0527 J/s on Laptop and 0.257 J/s on Phone.Source: C. Decoder Energy at Specified Bitrate… · p. 7, Figure 8 · p. 8
  • For 24-kHz speech at 12 kbps, Opus achieved higher ViSQOL than EnCodec (4.01 versus 3.70) while requiring less decoder energy on both platforms; DAC had the highest score, 4.27.At 24 kbps, AAC-LC and Opus continued to require relatively low decoder energy, whereas DAC scored 4.46 at the cost of higher decoder energy.Source: C. Decoder Energy at Specified Bitrate… · p. 8, Figure 8 · p. 8
  • Across evaluated neural operating points, decoder energy varied less with codebook depth than ViSQOL, a trend limited to the evaluated runtime configurations.The observed trend should not be assumed to generalize to other decoder architectures.Source: C. Decoder Energy at Specified Bitrate… · p. 8

D. Parametric Transmission-Energy Break-Even Analysis

The analysis derives pairwise transmission-energy break-even thresholds from measured decoder energy and bitrate, with lower thresholds favoring the neural point at a lower modeled transmission-energy coefficient. EnCodec had the lowest reported neural thresholds across the matched-quality cohorts and platforms.

D. Parametric Transmission-Energy Break-Even Analysis

  • EnCodec had the lowest music break-even thresholds on Laptop: 2.50 mJ/kbit versus AAC-LC and 7.60 mJ/kbit versus Opus.DAC and SNAC thresholds were higher against both baselines; the DAC values came from a GPU-requested path with partial non-target execution.Source: Figure 9 · p. 8, D. Parametric Transmission-Energy… · p. 8
  • For Phone music, EnCodec's thresholds were 12.4 mJ/kbit versus AAC-LC and 42.6 mJ/kbit versus Opus, lower than DAC's and SNAC's thresholds against both baselines.DAC and SNAC Opus-referenced thresholds were 493.7 and 517.6 mJ/kbit, above the 200 mJ/kbit plotting range.Source: Figure 10 · p. 8, D. Parametric Transmission-Energy… · p. 8
  • The lower AAC-LC-referenced music thresholds reflect larger bitrate reductions against the selected 48-kbps AAC-LC point than against the 16-kbps Opus point.Threshold values also depend on the corresponding decoder-energy differences.Source: D. Parametric Transmission-Energy… · pp. 8–9
  • For 24-kHz speech, EnCodec had the lowest AAC-LC-referenced threshold on both Laptop (4.52 mJ/kbit) and Phone (24.8 mJ/kbit).HILCodec and DAC thresholds were 24.0 and 52.2 mJ/kbit on Laptop, and 132.9 and 407.3 mJ/kbit on Phone.Source: Figure 11 · p. 8, D. Parametric Transmission-Energy… · p. 8, Figure 12 · p. 9
  • Speech thresholds are AAC-LC-referenced because Opus had no evaluated matched-quality point in the selected overlap cohort, so they do not rank neural codecs against all conventional speech codecs.Thresholds are model-derived, not measurements of a particular wireless interface.Source: D. Parametric Transmission-Energy… · pp. 8–9

E. Execution Validity and Latency Diagnostics

Matched-quality neural points had lower bitrates but higher decoder energy than conventional points, while execution-path behavior and the discrete matched points constrain deployment conclusions.

E. Execution Validity and Latency Diagnostics

  • On Laptop, NPU paths had the lowest family-level energy for EnCodec, HILCodec, and SNAC; DAC’s lowest GPU FP16-requested result included substantial non-target execution.For the OpenVINO GPU FP16-requested paths, non-target shares were 98% for SNAC, 47% for DAC, 21% for HILCodec, and 8% for EnCodec.Source: E. Execution Validity and Latency… · p. 9, Figure 14 · p. 9
  • All selected Laptop paths decoded faster than real time, but full-band DAC and SNAC on Phone XNNPACK CPU required 1.32 and 1.63 s, respectively, per second of audio.Latency was used only as a feasibility diagnostic.Source: E. Execution Validity and Latency… · p. 9

VII. DISCUSSION

  • At matched quality, selected neural points had lower bitrates but higher decoder energy than applicable conventional points; full-band Phone DAC and SNAC also exceeded 1 s per second of audio.EnCodec had the lowest neural break-even thresholds in both cohorts on both platforms.Source: §VII · pp. 9–10
  • Phone measurements characterize XNNPACK CPU execution, not mobile accelerator paths; DirectML failed validity criteria and Phone QNN showed no effective requested-accelerator use.DAC’s GPU FP16-requested result is mixed execution, not pure GPU decoding.Source: §VII · p. 10
  • At 12 kbps, EnCodec had higher full-band ViSQOL than evaluated AAC-LC and Opus, while at 24 kbps Opus reached a similar region with substantially lower decoder energy.The observed trade-off depends on service bitrate range and the client decoder path.Source: §VII · p. 10
  • Discrete bitrate ladders create mismatched quality anchors, making bitrate savings and break-even thresholds potentially more favorable in music and oppositely mismatched in speech.The protocol also retains bandwidth loss in measured quality degradation rather than bandwidth-matching the reference.Source: §VII · p. 10
  • Results cover one Laptop and one Phone, use different platform telemetry layers, and rely on three runs that do not characterize the full run-to-run distribution.ViSQOL anchors are objective-metric points rather than exact subjective equivalence.Source: §VII · p. 10
  • The codec set excludes closed or non-exportable systems; SNAC low-rate points derive from retained scales, and the measured pairwise thresholds do not establish a codec-wide ranking.Opus lacked a matched-quality speech point in the evaluated ladder, and server-side encoding energy was outside the playback model.Source: §VII · p. 10

VIII. CONCLUSION

The paper measures rate, decoder energy, and objective quality for neural and conventional audio codecs on Laptop and Phone. Neural codecs achieved similar matched-quality scores at lower bitrates but generally used more decoder energy, and the results do not establish a codec-wide ranking.

VIII. CONCLUSION

  • The analysis measured four neural codecs and two conventional baselines on Laptop and Phone, evaluated speech and music separately with original-reference ViSQOL, and compared matched-quality and explicitly evaluated bitrate settings.Decoder energy was idle-subtracted J/s audio, execution-valid paths were included, and pairwise break-even thresholds were derived from measured energy and bitrate.Source: §VIII · p. 10
  • Across matched-quality cohorts, neural points achieved similar ViSQOL at lower bitrates but required more decoder energy; EnCodec had the lowest neural thresholds, while several DAC and SNAC Phone CPU comparisons exceeded 200 mJ/kbit.Full-band Phone DAC and SNAC also required more than 1 s to decode 1 s of audio; DAC’s Laptop GPU-requested result included substantial non-target execution.Source: §VIII · p. 10
  • A low bitrate alone does not establish an energy benefit: the effective decoder path must be efficient and transmission-energy coefficient must exceed the pairwise threshold.Failed DirectML and invalid Phone QNN paths limit conclusions about alternative accelerator-assisted deployment; future work calls for more devices, repeated measurements, valid mobile accelerator paths, and subjective listening tests.Source: §VIII · p. 10
Loading 2609.34524v1…