SwinDS-BWE: A Parameter-Efficient Swin-1D Lattice Generator with Decision-Science Discriminators for Speech Bandwidth Extension

1. INTRODUCTION

• BWE restores HF content lost in compression, but heavy backbones or global mixing scale poorly and mean-aggregated supervision can average away rare HF failures.Generic sharpness rewards may also be gamed by excessive HF amplification, producing bright but unrealistic artifacts.Source: §1 · p. 1
• The Swin-1D lattice replaces AP-BWE's heavy AP ConvNeXt refiners with shifted-window attention, enabling local modeling and cross-window flow without global quadratic attention.The passage attributes the generator-size reduction without performance sacrifice to this design.Source: §1 · p. 1
• Three lightweight critics target distinct failures: CVaRD tail-pools worst segments, CCD barriers HF over-boost, and MCUD learns convex weights over spectral criteria; the critics train in lockstep.These mechanisms respectively emphasize poor segments, regulate HF-energy exceedance, and aggregate spectral flatness, centroid, and rolloff criteria.Source: §1 · p. 1
• 74% reduction in overall model size versus prior AP-BWE (72M vs. 18.45M) while maintaining strong perceptual scores across extension ranges.The authors also report favorable performance and say the ablation supports complementary HF polishing with stabilizing global amplitude-phase feedback.Source: §1 · p. 1

2.1. Generator: Swin-1D with Lattice Coupling (see Alg. 1)

• Two Swin-1D lattice blocks refine phase and magnitude streams through learned cross-stream coupling, with shifted windows giving O(TW) rather than O(T^2) attention complexity.The generator predicts wideband magnitude as a residual correction; its phase head uses a complex-valued prediction without a residual connection.Source: §2.1 · pp. 1–2

2.2. Decision-Science Discriminators and Terminology

• The decision-science critics use depthwise-separable convolution and hinge losses, returning critic scores and intermediate feature maps for feature matching.The losses provide adversarial feature-matching feedback to the generator.Source: §2.2 · p. 2

2.2.1. CVaR Discriminator (CVaRD): For tail-risk pooling for rare HF artifacts (see Algorithm 2)

• CVaRD combines top-k temporal evidence with global average pooling, directing adversarial feedback toward worst reconstructed segments while retaining global context.Its tail size is k = max(1, floor(αT′)).Source: §2.2.1 · p. 2

2.2.2. Chance-Constraint HF Discriminator (CCD): For barrier trace to suppress over-boost (see Algorithm 3)

• CCD converts per-frame HF-energy-ratio exceedance into a smooth barrier, using average and maximum pooling to capture persistent overshoot and rare spikes.The stated threshold is τ = 0.4; the barrier’s sharpness is controlled by κ.Source: §2.2.2 · p. 2

2.2.3. Multi-Criteria Utility Discriminator (MCUD): For criteria-to-utility aggregation for HF realism (see Algorithm 4)

• MCUD instance-normalizes spectral flatness, centroid, and roll-off, then combines them with learned convex weights into a utility trace for critic scoring.The criteria are extracted from magnitude spectra, and the roll-off uses p = 0.85.Source: §2.2.3 · pp. 2–3

2.3. Loss Functions

• The discriminator objective uses six losses, and its total is defined in Equation 1.The passage names the CVaRD loss among those terms.Source: §2.3 · p. 3

• The discriminator losses are CVaRD, CCD, MCUD, Multi-Resolution Amplitude (MRAD), and Multi-Resolution Phase Discriminator (MRPD), respectively.Source: D , and LMRPD · p. 3

3. ANALYSIS, RESULTS, AND ABLATION STUDY

• Evaluation uses VCTK and French MLS; models train for 50 epochs with window size 8 and batch size 16.Audio is converted to mono, silence is removed, and segments are downsampled, interpolated, and aligned.Source: §3 · p. 3

3.1. Comparative Analysis with Baselines with & without Noise

• SwinDS-BWE achieves the highest NISQA-MOS across three ranges, with similar AP-BWE STOI and LSD using 18.45M parameters versus AP-BWE’s 72M.It also outperforms AP-BWE in NISQA-MOS, SI-SNR, and LSD on noisy VCTK.Source: §3.1 · p. 3

3.2. Ablation Study

• Among Rows 1–3, CVaRD achieves the highest NISQA-MOS (3.36), versus 2.99 for CCD and 3.01 for MCUD, with LSD 1.06 versus 1.25/1.18, respectively.MCUD has the best PESQ among Rows 1–3 at 2.63.Source: §3.2 · p. 4
• Adding MRAD or MRPD raises NISQA-MOS from 3.23 to 4.04 or 4.16, while improving LSD and SI-SNR relative to the three-critic-only setup.The authors interpret the global reconstruction components as stabilizing reconstruction before local HF polishing.Source: §3.2 · p. 4
• With discriminator feedback fixed, Swin-1D raises NISQA-MOS from 4.13 to 4.24 and uses 1.9x fewer parameters than the ConvNeXt alternative.PESQ falls from 2.25 to 2.20, while LSD changes from 1.00 to 1.03.Source: §3.2 · p. 4

3.3. Results Across Different Bands

• Across frequency ranges, the broadest 2–48 kHz setting is hardest; performance improves as the reconstruction-band gap shrinks.The reported trend spans objective metrics, with distortion dropping and intelligibility and quality rising as the gap is reduced.Source: §3.3 · p. 4, Table 3 · p. 4

3.4. Computational Complexity

• Relative to AP-BWE, SwinDS-BWE reduces generator parameters by 42.6%, discriminator parameters by 96.8%, and MACs and FLOPs by 63%, but has slightly higher, still-real-time RTF.The authors attribute the RTF difference to window partitioning and reversal, masking, and related shifted-attention operations.Source: §3.4 · p. 4, Table 4 · p. 4

3.5. Subjective Analysis

• A panel of 10 rated SwinDS-BWE significantly higher than AP-BWE for female speakers and overall.Ratings used a 5-point MOS scale.Source: §3.5 · p. 4

4. CONCLUSION AND LIMITATION

• 17.09 M parameters and 5.28 G MACs reduce the generator from 14.24 G MACs while maintaining strong performance across extension bands.The Swin-1D shifted-window AP lattice preserves amplitude–phase coupling and replaces heavier refiners with locality-aware attention.Source: §4 · p. 4
• Three lightweight decision-science critics target sparse, perceptually dominant HF errors through tail pooling, a chance-constraint barrier, and convex utility over spectral criteria.Together, they shift BWE training from mean-frame matching toward tail-event realism, with a reported 74% reduction in parameter count.Source: §4 · p. 4
• The current design does not use data-driven calibration for the hyperparameters of CCD, MCUD, and CVaRD.Source: §4 · p. 4