HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis

Motivation

• Higher sampling rates improve expressive singing but create wider frequency bands and longer, finer-grained waveforms that challenge acoustic and vocoder modeling.Earlier systems either use coarse-grained MFCCs with slow autoregressive vocoders or non-neural vocoders, limiting high-sampling-rate quality.Source: §1 · pp. 1–2

Approach

• HiFiSinger combines a FastSpeech-based acoustic model with a Parallel WaveGAN-based vocoder for fast training, fast inference, and high voice quality.The system applies multi-scale adversarial training to both models.Source: §1 · p. 2
• SF-GAN splits the 80-dimensional mel spectrum into sub-bands and trains separate discriminators to model high-frequency details.ML-GAN similarly uses separate discriminators for randomly cropped waveform sequences of different lengths.Source: §1 · p. 2
• HiFiSinger adds F0 and V/UV features, selects suitable window and hop sizes, and enlarges the vocoder receptive field to improve high-fidelity singing modeling.These designs target pitch, voiced/unvoiced structure, acoustic-feature alignment, and long vowels.Source: §1 · p. 2

Evaluation

• Experiments on 11 hours of 48kHz recordings show higher quality than previous singing voice synthesis systems, with ablations supporting the individual designs.The supplied passages establish the dataset scope and overall experimental conclusion but not the abstract's MOS values.Source: §1 · p. 2

High-Fidelity SVS

• At 48kHz, the representable frequency band extends to 24kHz rather than 12kHz at 24kHz, adding high-frequency signals that are more complicated and less predictive.This follows the sampling-rate relationship described in the paper.Source: §2 · p. 2
• A one-second 48kHz waveform contains 48,000 sampling points, increasing the temporal sequence length that vocoders must model.The example illustrates why higher sampling rates create longer waveform sequences.Source: §2 · p. 2

SVS versus TTS

• SVS synthesizes singing from lyrics together with note pitch and duration, emphasizing expression and emotion more than ordinary speech synthesis.These requirements distinguish SVS from TTS and motivate specialized modeling choices.Source: §2 · p. 3

High-Fidelity SVS

• High-fidelity singing commonly uses 44.1kHz or higher sampling rates, whereas many prior SVS systems use 16kHz or 24kHz settings inherited from TTS.Higher sampling rates are used to convey expressive detail.Source: §2 · p. 3
• Prior high-sampling-rate SVS systems use coarse MFCCs with slow autoregressive vocoders or non-neural vocoders, which do not fully exploit the available bandwidth.The paper motivates HiFiSinger as a neural alternative for high-fidelity synthesis.Source: §2 · p. 3

3.1 SYSTEM OVERVIEW

• HiFiSinger converts music-score lyrics, pitch, and duration into acoustic features using a FastSpeech-based acoustic model and generates waveform with a parallel vocoder.The score is phonemized, pitch is represented by MIDI IDs, and duration is quantized into mel-spectrogram frames.Source: §3.1 · pp. 3–4
• The system architecture contains a parallel acoustic model, a parallel vocoder, SF-GAN for mel-spectrograms, and ML-GAN for waveforms.Figure 1 summarizes these components.Source: §3.1 · p. 3, Figure 1 · p. 4

3.2 MODELING WIDE FREQUENCY WITH SF-GAN

• Increasing mel bins alone did not improve voice quality; the paper instead targets modeling diverse details across the wide frequency range, with a 20ms window selected experimentally.The paper notes that increasing frequency resolution also trades against time resolution.Source: §3.2 · p. 4
• SF-GAN applies separate discriminators to multiple overlapping mel-frequency bands so each discriminator guides its sub-spectrogram toward less smoothing and closer ground truth.For an 80-dimensional mel spectrum, the example uses low, middle, and high bands with overlap.Source: §3.2 · p. 5

3.3 MODELING LONG WAVEFORM WITH ML-GAN

• ML-GAN uses multiple discriminators for different waveform lengths, reducing the difficulty of long-sequence modeling and capturing variable singing phoneme durations.Its discriminators operate on different time-length waveform sequences.Source: §3.3 · p. 5

3.4 OTHER SYSTEMATIC DESIGNS

• The vocoder uses mel-spectrogram, pitch, and V/UV features, while window-hop choices balance acoustic-model and vocoder difficulty and a larger receptive field covers long vowels.The selected window and hop sizes are 20ms and 5ms.Source: §3.4 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• The evaluation uses 11 hours of 48kHz singing recordings, with MOS judgments from at least 20 listeners on 100 test pieces.The dataset contains 6,817 pieces from a female Mandarin pop singer, split into training, validation, and test sets.Source: §4.1 · p. 6, §4.2 · p. 7

4.2 AUDIO QUALITY

• 0.32 MOS over the 48kHz baseline and 0.83 MOS over XiaoiceSing show HiFiSinger’s strongest reported audio-quality gains at 48kHz.HiFiSinger also gains 0.29 MOS when moving from 24kHz to 48kHz, compared with 0.12 MOS for the baseline.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 ABLATION STUDIES

• Three SF-GAN discriminators outperform zero, one, and five-discriminator settings, while 120 mel bins provide only a 0.02 CMOS gain over 80 bins.The three discriminators cover overlapping low, middle, and high-frequency mel bands; adding SF-GAN produces more high-frequency detail and less smoothing.Source: §4.1 · p. 6, §4.3 · p. 8, Table 2 · p. 8
• ML-GAN outperforms single-length waveform discrimination and produces stable long vowels instead of glitches found with a single 1-second discriminator.Different discriminator lengths trade expressive voice against electronic noise, whereas ML-GAN combines their advantages.Source: §4.1 · p. 6, §4.3 · p. 8, Table 3 · p. 8
• Removing pitch and V/UV causes CMOS drops of 0.34 and 0.28, respectively, while pitch and voiced/unvoiced features improve controllability and robustness.Without these inputs, unvoiced and silence regions become less informative and over-smoothed, producing electronic noise in the authors’ observations.Source: §4.3 · p. 9, Table 4 · p. 9, Figure 4 · p. 9
• Larger or smaller window/hop sizes reduce quality, whereas a larger vocoder receptive field improves audio quality for singing with variable, especially long, vowel durations.The experiments keep the window-to-hop ratio at 4:1 and vary the absolute sizes; the receptive-field study varies convolution kernel size.Source: §4.3 · pp. 9–10

• HiFiSinger combines SF-GAN for wider frequency bands, ML-GAN for longer waveform sequences, and additional designs for high-fidelity singing synthesis.The system also incorporates findings intended to improve singing modeling, including pitch and V/UV features, window/hop selection, and vocoder receptive-field design.Source: §5 · p. 10