Source-linked AI summary
Assessing the Reusability of Public Speech Resources for Low-Resource Languages: A Central Kurdish Case Study
Hiwa Asadpour
TL;DR
Central Kurdish lacks well-documented public speech technology, and this review examines whether a released dataset, models, code, and paper correspond. It finds substantial artifact mismatches and a narrower supported scope than the release suggests, while noting that most documentation problems are repairable but licensing constrains public reuse.
Problem
Central Kurdish remains poorly represented in publicly available speech technology resources despite its substantial speaker population.
Method
The review checks the published paper against the released audio, transcripts, checkpoints, vocabulary, configuration, inference script, evaluation code, and upstream components.
Results
MOS scores for the three systems range from 4.01 to 4.08, while the release contains mismatches including stale configuration settings, an unlabeled evaluation split, and faulty long-number handling.
Takeaways & Limitations
The release supports Central Kurdish read-speech synthesis by three single-speaker systems, but better records would make the work easier for others to check and reuse.
Takeaways & Limitations
The systems were trained on read material from three single speakers, and the evaluation design cannot separate data-source effects from narrator and speaker effects.
Abstract
from arXiv · showhide
Kurdish is spoken by millions of people, but little technology can read it aloud. A recent study released three Kurdish voices, 35 hours of recorded speech, and a paper describing the work, all free to download. This review checks how well those public files match the paper. The research is careful about its limits, but the files contain several problems: a settings file lists equipment that was never used, test recordings are left unlabeled among training data, and a coding fault mishandles long numbers. The download page also claims a stronger result than the paper reports and recommends one voice for general use. That recommendation matters because Kurdish has major regional and written variation, while these voices were built from three people reading prepared texts. The process therefore removes much everyday and regional speech. English and German benefit from long traditions of dictionaries and linguistic description that help identify wrong pronunciations; Kurdish has far less such support, so software choices can go unchecked. The voices sound fluent, but they represent the reading styles of their speakers rather than Kurdish as a whole. Most of these issues can be fixed using information the team already has, without changing the reported results. Better records would mainly make the work easier for others, especially community linguists, to check and reuse. The license is the main exception: whether audiobook owners allow corrected versions to be shared will affect whether future Kurdish voices can build on this work or must start again.
1. Introduction and background information
Central Kurdish has substantial speaker numbers but remains poorly served by public speech resources. The release improves availability while exposing documentation, reproducibility, and licensing gaps between the paper and its artifacts.
- 35 hours and 20 minutes of speech, three trained text-to-speech models, and published evaluation code make this release unusually substantial for Central Kurdish.
- The paper’s account of fine-tuning the English F5-TTS checkpoint is more specific than the release’s description of models based on the F5-TTS architecture.These descriptions imply different starting points for interpreting the low-resource achievement.
- The phoneme-conversion pipeline addresses missing Kurdish characters in the baseline vocabulary but embeds contested orthographic decisions before training.
- CC BY-NC-ND 4.0 permits access but restricts modified redistribution, limiting public verification through retraining, ablations, or adapted models.
3. Where the released configuration and the paper diverge
The released configuration conflicts materially with the paper’s reported training setup. Its stale hardware and epoch settings could mislead users attempting to reproduce the models.
- The configuration specifies eight GPUs, whereas the paper reports fine-tuning each dataset independently on one RTX8000 GPU.The file lists a batch size of 38,400 frames per GPU and an eight-GPU comment.
- Eleven configured epochs cannot explain learning curves reaching roughly 275,000 steps, while the paper places convergence during the first 100,000 iterations.The discrepancy spans orders of magnitude and is consistent with an inherited upstream template.
4. What the evaluation supports, and what the model card claims
The paper reports audiobook-trained Kurdish voices as competitive with studio-trained data, while the model card presents a stronger claim than the paper supports. The evaluation’s single-speaker design limits what can be attributed to audiobook data or generalized to Kurdish speech.
- The paper reports audiobook-trained systems as competitive with high-quality studio data, not as definitively superior.The female audiobook system is described as “marginally better” on subjective evaluation, while the paper says the systems perform similarly overall.
- The model card claims the female audiobook system achieved the highest overall subjective score and is suitable for general-purpose applications.That wording is stronger than the paper’s cautious description of a marginal difference.
- Eighty-eight listeners supplied 3,101 ratings across seven categories, evaluating each system on 140 sentences.Listeners rated naturalness from 1 to 5 without being told that real speech was included.
- The reported MOS ranking rests on 4.08 versus 4.06, while category confidence intervals range from ±0.14 to ±0.19.The paper therefore calls the difference marginal rather than claiming statistical significance.
- Because each condition used one speaker, audiobook data cannot be separated from narrator, material, recording environment, or speaker gender.The design cannot establish whether the top score reflects audiobook data or qualities specific to the female narrator.
5. A measurement that depends on its own training data
The paper’s intelligibility measure depends on an automatic recognizer that also produced the training transcripts, and human-control results expose that dependency. The aggregate error rates therefore retain internal ranking value but overstate what they establish about intelligibility.
- The recognizer used to score synthetic speech also transcribed the audiobook recordings that became training text, creating a circular evaluation dependency.The same fine-tuned Seamless system generated the transcripts and measured the speech, although those transcripts were manually revised.
- Human recordings used as controls also receive nonzero character error rates, showing that the recognizer contributes measurable error without synthesis.
- Table 2 reports averages and 95% confidence intervals for metrics across training datasets and categories.
- The model card’s 3.7%, 4.7%, and 5.6% character error rates preserve internal ranking but should not be read as straightforward intelligibility estimates.The paper’s claim that error remains below 8% across speakers conflicts with a Table 3 row of 0.09.
6. An undocumented preprocessing step
The released inference script mishandles unbroken digit strings of nine or more characters. Its placeholder substitution has no restoration step and can introduce a character absent from the model vocabulary.
- Nine-or-more-digit runs are replaced with <NUM:...> placeholders before normalization, but the script never restores the original numbers.The placeholder therefore reaches the model as literal text rather than being converted back after processing.
- The placeholder includes “<”, which is absent from vocab.txt, while the remaining symbols come from the English checkpoint rather than Kurdish training data.
- The failure is directly established by the released substitution rule, the missing reversing step, and the vocabulary inspection.
- The pipeline’s Central Kurdish orthographic assumptions also limit reuse with other Kurdish varieties and scripts.The review notes that Kurmanji Latin characters such as ḧ and ẍ are not covered by the relevant vocabulary, although this case was not tested.
7. What can and cannot be reproduced
The release makes much of the method inspectable, but key artifacts and metadata still prevent full reproduction. The most consequential gaps are the inaccessible Kurdish recognizer, missing evaluation sentences, and absent split labels.
- The paper and release provide the architecture, starting checkpoint, hardware, preprocessing chain, metrics, audio, transcripts, checkpoints, vocabulary, configuration, and inference script.
- The Kurdish ASR checkpoint needed for intelligibility evaluation is inaccessible, so Table 3 cannot be recomputed from the released materials.The evaluation code is public and the recognizer-building recipe is documented, but the fine-tuned checkpoint is not available without authentication.
- Missing split labels could cause future users to train on evaluation data when adding speakers, producing silent contamination and misleading improvements.
- Publishing the 140 evaluation sentences per system would make the listening test supporting the central claim repeatable.
- The paper and dataset card list 17,671 utterances, whereas the Hugging Face viewer reports 17,738 rows.
8. The license and its practical reach
The release’s licensing constraints arise from audiobook rights and upstream model weights, not solely from the researchers’ choices. The no-derivatives term directly complicates sharing corrected or adapted resources.
- The audiobook data are released under CC BY-NC-ND 4.0 because writers and narrators granted permission under that license.
- The pretrained F5-TTS weights add a separate non-commercial restriction, so licensing constraints are inherited from both the data and the upstream model.
- The no-derivatives restriction creates a practical tension between intended voice-cloning or TTS use and distributing models adapted from the dataset.The dataset card lists those intended uses while the license prohibits distributing modified versions.
- Training a model on the data and releasing it may be considered distribution of something adapted from the licensed material under a literal reading.
9. One rights holder named, one absent
The release documents permission for the female audiobook’s text and voice, but does not identify a rights holder for the male audiobook’s text.
- The female audiobook has separately identified permissions from its writer and narrator, while the male audiobook names only its narrator.The male audiobook’s writer, publisher, or other text-rights holder is not identified, although the paper refers generally to writers and narrators granting permission.
- The missing male-text rights information limits whether others can confidently build on that corpus.
10. The pronunciation decisions embedded in the models
The models inherit pronunciation choices from a grapheme-to-phoneme converter whose treatment of contested Kurdish orthography is not discussed in the paper.
- The converter resolves contested Kurdish orthographic cases before training, including the unwritten short vowel /i/ and specific letters.Those decisions are embedded in the training data, so every input receives the converter’s consistent treatment.
- Because the converter standardizes pronunciation before modeling, fluent output reflects one software rule system and one deliberate reading register.
11. The direction of accommodation
The release repeatedly narrows linguistic variation through read-source selection, limited speakers, transcription, and normalization, but does not clearly document those commitments.
- Read audiobooks and prepared sentences remove much spontaneous, regional, and everyday variation before later processing begins.The three speakers’ varieties are also unrecorded, leaving the corpus’s attested range unclear.
- The pipeline’s stages standardize Kurdish in the same direction, making the cumulative narrowing more consequential than any single defensible decision.
- Kurdish linguistic references would help readers distinguish settled pipeline commitments from contested ones and interpret the released voices accordingly.
- Documenting the written-standard orientation, converter settings, discarded syllable markers, and vocabulary coverage would make these choices visible to users.
12. Which variety, and whose judgement
The release provides detailed evaluation evidence, but its three-speaker, prepared-reading design and missing variety information constrain how broadly listeners and developers can interpret the results.
- The release does not record which Kurdish variety each speaker represents, limiting voice selection and interpretation of listening-test judgements.
- Table 3 reports 95% confidence intervals across seven categories and three systems, supporting proportional interpretation of the reported 0.02 margin.
- The evaluation code reproduces the shipped reference values across all seven metrics, establishing a concrete basis for checking the release.
- Each evaluation condition uses one speaker reading prepared material, while the paper acknowledges too few speakers for a multi-speaker model.
14. Conclusion
The public release does not consistently match the paper, leaving key details about training, evaluation, linguistic coverage, and reuse unresolved. Most issues could be clarified with information the authors already hold, but audiobook rights may constrain redistribution of corrected models.
- The paper identifies mismatches between its careful reporting and public artifacts, including inaccurate hardware specifications, omitted dataset splits, and a faulty undocumented preprocessing step.These mismatches affect how the release can be used or verified.
- The release leaves unresolved which hardware, training checkpoint, commands, manifests, and audio preparation produced the public models.The configuration file conflicts with the paper’s reported hardware, and checkpoint selection is not reported.
- The evaluation cannot be reliably reproduced because held-out test utterances are not identifiable and the evaluation sentence categories may not be publishable.Users fine-tuning on the released corpus may therefore train on the paper’s evaluation set.
- The release does not document which Central Kurdish varieties the speakers represent, the raters’ dialect backgrounds, or the grapheme-to-phoneme settings used.The treatment of syllable markers is also not identified as deliberate or inherited.
- Audiobook rights may determine whether corrected or adapted models can be redistributed for non-commercial research.The recordings would remain under their existing terms, while permission for model redistribution is unresolved.
Model
The release is accompanied by a model repository and dataset, along with related Kurdish resources from the same account.
- The model and dataset are publicly hosted on Hugging Face, with related Kurdish ASR and resource repositories linked from the same account.The cited materials include model-card files, a dataset card, and related resources.