Source-linked AI summary
JVS corpus: free Japanese multi-speaker voice corpus
Shinnosuke Takamichi, Kentaro Mitsui, Yuki Saito, Tomoki Koriyama, Naoko Tanji, Hiroshi Saruwatari
TL;DR
The paper addresses the need for Japanese voice data suited to more general speech research than a single-speaker reading corpus. It constructs JVS with multiple speakers, styles, and utterance types; the resulting corpus contains 30 hours of recordings. The authors present it as a resource for multi-speaker and multi-style speech research.
Problem
The earlier JSUT corpus contained 10 hours of reading-style speech from one speaker, while more general speech-related research motivated a new corpus.
Source: §1 · p. 1Method
JVS combines 100 native Japanese professional speakers with normal, whisper, and falsetto recordings, including parallel and non-parallel utterances.
Source: §1 · p. 1, §2.1.1 · p. 2, §2.1.2 · p. 2Results
30 hours of voice data comprise 26 hours of normal voices and 4 hours of other-style voices.
Source: §3.2.1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
Thanks to improvements in machine learning techniques, including deep learning, speech synthesis is becoming a machine learning task. To accelerate speech synthesis research, we are developing Japanese voice corpora reasonably accessible from not only academic institutions but also commercial companies. In 2017, we released the JSUT corpus, which contains 10 hours of reading-style speech uttered by a single speaker, for end-to-end text-to-speech synthesis. For more general use in speech synthesis research, e.g., voice conversion and multi-speaker modeling, in this paper, we construct the JVS corpus, which contains voice data of 100 speakers in three styles (normal, whisper, and falsetto). The corpus contains 30 hours of voice data including 22 hours of parallel normal voices. This paper describes how we designed the corpus and summarizes the specifications. The corpus is available at our project page.
1. Introduction
The paper introduces JVS as a Japanese voice corpus for more general speech-related research, building on the earlier single-speaker JSUT corpus. JVS combines professional speakers and multiple voice styles with accessible, annotated data.
1. Introduction
- JVS is introduced for more general speech-related research than the single-speaker JSUT corpus, which provided 10 hours of reading-style speech.The authors note that accessible voice corpora can accelerate speech research and improve reproducibility.Source: §1 · p. 1
- The corpus includes 100 native Japanese professional speakers and recordings in a studio controlled by a professional sound director.Audio is sampled at 24 kHz, encoded at 16 bit, and stored in RIFF WAV format.Source: §1 · p. 1
- JVS contains 30 hours of voice data in normal, whisper, and falsetto styles, with parallel and non-parallel utterances and annotations including transcriptions and phoneme alignments.It is freely downloadable and free to use for research in academic institutions and commercial companies.Source: §1 · p. 1
2. Corpus design
JVS is organized into four sub-corpora that vary in utterance commonality and speaking style. Their design pairs parallel and non-parallel material across normal, whisper, and falsetto voices, and includes speaker and utterance annotations.
2. Corpus design
- The four sub-corpora are parallel100, nonpara30, whisper10, and Falsetto10, with the number indicating utterances per speaker.The corpus directory identifies speakers by IDs from 1 through 100.Source: §2 · p. 1
2.1.1. parallel100
- Parallel100 uses 100 phonetically balanced JSUT voiceactress100 sentences shared across speakers, with transcriptions and phoneme alignments.The paper identifies parallel voices as useful for voice conversion, speaker factorization, and multi-speaker modeling.Source: §2.1.1 · p. 2
2.1.2. nonpara30
- Nonpara30 gives each speaker 30 randomly selected JSUT utterances that differ across speakers and are not phonetically balanced.The authors describe this non-parallel setting as more realistic but challenging compared with parallel voices.Source: §2.1.2 · p. 2
2.1.3. whisper10
- Whisper10 contains five utterances parallel with parallel100 and five parallel with nonpara30, linking whispered and normal speech.Thus, all ten whispered utterances are parallel between whispered and normal voices.Source: §2.1.3 · p. 2
2.1.4. Falsetto10
- Falsetto10 includes five utterances parallel across speakers, ten parallel between normal and falsetto voices, and five parallel between whisper and falsetto voices.The paper describes falsetto as a register with an F0 range higher than normal voices.Source: §2.1.4 · p. 2
2.2. Tags
- The tags include manually annotated F0 ranges per speaker for normal voices, perceptual similarity scores for same-gender speaker pairs, and duration and speech-rate information.Phoneme-level duration is calculated from phoneme alignments.Source: §2.2 · p. 2
3. Results of data collection
The corpus comprises normal, whispered, and falsetto recordings from 100 speakers, with duration statistics and perceptual similarity scores reported. Speaker durations vary, and subjective scoring identifies both a most-similar pair and a most-dissimilar speaker.
3.1. Corpus specs
- 100 native Japanese professional speakers—49 male and 51 female—were recorded in a studio, with each speaker recorded within one day.A professional sound director controlled the recordings; audio was downsampled from 48 kHz to 24 kHz.Source: §3.1 · p. 2
3.2.1. Duration
- 26 hours of normal voices and 4 hours of other-style voices make up the corpus; each speaker contributed approximately 15.7 minutes normal, 1.24 minutes whispered, and 1.18 minutes falsetto.In parallel100, jvs084 spoke 1.8 times slower than jvs020 despite sharing the transcription.Source: §3.2.1 · p. 2
3.2.2. Perceptual speaker similarity
- Subjective scoring found jvs019 and jvs096 to be the most similar pair, while jvs010 was most dissimilar from the other speakers.Ten listeners scored each speaker pair on a scale from −3 to +3; 1,000 listeners participated overall.Source: §3.2.2 · p. 2, §3.1 · p. 2
4. Conclusion
The paper concludes that JVS was constructed for speech research using multi-speaker and multi-style voices. Its stated terms distinguish text, tags, and audio use, with commercial-use terms described on the project page.
4. Conclusion
- JVS was constructed as a corpus for speech-related research using multi-speaker and multi-style voices.Text data follows the license specified in the JSUT corpus LICENCE file, and tags are licensed under CC BY-SA 4.0.Source: §4 · p. 2
- Audio use includes academic research, non-commercial research including within commercial organizations, and personal use; commercial-use terms are described on the project page.The listed non-commercial examples also include personal blog posts.Source: §4 · pp. 2–3