PaperlayerView exact revision on arXiv

Source-linked AI summary

Lipi-Ghor: A Large-Scale, Diarized, Multi-Domain Bengali Speech Corpus and a Scalable Pipeline for Building It

Sanjid Hasan, A H M Fuad, Risalat Labib, Bayazid Hasan Simon

arXiv:2610.36184v1cs.CL

TL;DR

Open Bengali ASR resources lack broad domain coverage together with diarization and long-form audio, despite the language’s substantial speaker population. Lipi-Ghor constructs an 885-hour, 17-domain Bengali corpus from YouTube with segment-level diarization and caption alignment. The paper reports consistent degradation on emotive, dialectal, and long-form speech, while fine-tuning demonstrates the corpus’s utility for Bengali ASR.

  • Problem

    Open Bengali ASR resources are narrow in domain and confined to short read or crowd-sourced speech, lacking diarization, speaker annotations, and long-form audio.

    Source: §1 · p. 1

  • Method

    Lipi-Ghor uses targeted YouTube channel curation and a replicable pipeline to build a Bengali corpus with diarization aligned to caption transcripts.

    Source: §1 · p. 2, §3.4 · p. 3

  • Results

    Experiments reveal consistent degradation on emotive, dialectal, and long-form speech, while fine-tuning demonstrates the corpus’s utility for Bengali ASR.

    Source: §8 · p. 8

  • Takeaways & Limitations

    Targeted channel curation across domains yields an 885-hour weakly supervised Bengali corpus and a construction pipeline for other languages with sufficient captioned YouTube content.

    Source: §1 · p. 2, §8 · p. 8

    Caption-derived transcripts are not fully human-verified, speaker labels are automatically generated rather than a manually validated diarization benchmark, and regional dialect coverage is sparse.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Bengali, spoken by over 230 million people, remains severely under-resourced in spoken language research. Existing open Bengali ASR corpora are confined to short, read or crowd-sourced speech and lack speaker diarization and long-form audio. We present Lipi-Ghor, a large-scale weakly-supervised Bengali speech corpus comprising 885 hours of audio sourced from 998 YouTube videos across 596 unique channels, spanning 17 diverse domains including audiobooks, drama, religious sermons, news, and regional dialects. Each recording is annotated in SSTT format (Speaker, Speech, Transcription, Timestamp), which combines automated diarization with Bengali YouTube caption tracks. To our knowledge, Lipi-Ghor is the first Bengali speech corpus to pair large-scale coverage with speaker-diarized, long-form audio annotations, both of which are absent from existing open Bengali resources. We also present the construction methodology as a replicable pipeline for any language with a sufficient captioned YouTube presence. Targeted channel curation across a handful of rich domains yields corpus scales that were previously out of reach for low-resource languages. We benchmark four widely-used ASR models under zero-shot and fine-tuned conditions. All four degrade consistently on emotive, dialectal, and long-form content. Because caption targets are noisy, we also release a human-annotated gold evaluation split, verified by 17 volunteer annotators over 30 hours of human-in-the-loop correction and stratified across all 17 domains. We release our original contributions (video-ID lists, diarization output, pipeline code, and the gold annotations) under CC-BY-4.0, and distribute caption-derived transcripts for research use only; no raw audio is redistributed.

1 Introduction

Bengali speech research lacks open resources that combine broad domain coverage with diarization and long-form audio. Lipi-Ghor addresses this gap with a large-scale YouTube corpus and a replicable construction pipeline.

1 Introduction

  • Existing open Bengali ASR resources are narrow in domain, limited to short read or crowd-sourced speech, and lack diarization, speaker annotations, and long-form audio.Bengali also has dialectal variation, English code-switching, register shifts, and high speaker density in long-form recordings.Source: §1 · p. 1
  • Lipi-Ghor combines large-scale Bengali speech coverage with speaker-diarized, long-form audio in a single corpus built through a scalable, replicable pipeline.The pipeline uses domain-aware channel curation, caption-track filtering, distributed transcript fetching, and automated diarization; the Bengali corpus contains 885 hours across 17 domains and 596 channels.Source: §1 · p. 2
  • Figure 1 provides an overview of the Lipi-Ghor construction process.Source: §1 · p. 2

2 Related Work

Prior Bengali speech resources vary in scale and annotation richness, but the supplied comparison describes them as lacking the combination of large-scale training audio, diarization, and long-form recordings. Web-based corpora offer scale, while Bengali content and speaker-attributed long-form coverage remain limited.

Bengali ASR Datasets

  • Open Bengali resources do not pair large-scale training audio with speaker diarization and long-form recordings.SHRUTI has tens of hours of read speech; Common Voice Bengali has several hundred hours of crowd-sourced read speech; BengaliAI offers broader coverage without speaker-attributed long-form audio.Source: §2 · p. 2

Web- and YouTube-Based Corpus Construction

  • YODAS harvested over 500k hours of YouTube subtitles across more than 100 languages, but contains only 27 hours of Bangla; cited efforts are English-centric or provide only segment-level transcripts without speaker diarization.Recurring pipeline challenges include filtering for target-language caption quality and managing API rate constraints at scale.Source: §2 · p. 2

Speaker Diarization Corpora

  • Bengali has no coverage in the cited diarization benchmarks, while Lipi-Ghor brings diarized annotation to Bengali at training scale.The cited benchmarks are AMI, DIHARD III (Ryant et al., 2021), and VoxConverse (Chung et al., 2020).Source: §2 · p. 2

Weakly Supervised ASR

  • Noisy web transcripts and multilingual speech models are established research directions, and auto-generated captions have error patterns distinct from human transcription.The paper addresses this caption limitation through a gold evaluation split.Source: §2 · p. 2

3 The Lipi-Ghor Pipeline

The Lipi-Ghor pipeline builds a Bengali speech corpus through targeted channel curation, caption-based selection, audio and subtitle extraction, and segment-level diarization alignment. Its released outputs include an 885-hour SSTT corpus and ASR-ready chunks.

3 The Lipi-Ghor Pipeline

  • The pipeline combines domain-aware channel curation, transcript availability filtering, audio and caption extraction, and SSTT annotation.Source: §3 · pp. 2–3

3.1 Stage 1: Domain-Aware Channel Curation

  • Channel curation begins with high-volume domain channels, then adds long-tail sources to improve speaker, topic, and recording diversity.The passage identifies domain-aware channel discovery—finding channels with enough Bengali caption coverage—as the bottleneck.Source: §3.1 · p. 3

3.2 Stage 2: Transcript Availability Filtering

  • Videos are retained only when they have an existing Bengali caption track, and video IDs are processed in parallel batches to manage YouTube API rate constraints.Source: §3.2 · p. 3

3.3 Stage 3: Audio and Caption Extraction

  • 1,019 audio-caption recordings were extracted, and cross-domain deduplication yielded 998 unique videos for the corpus.Only Bengali-track captions are retained; auto-translated tracks are discarded.Source: §3.3 · p. 3

3.4 Stage 4: SSTT Annotation

  • 7 of 17 domains account for roughly 83% of corpus audio duration, reflecting where Bengali caption coverage is densest on YouTube.Source: Figure 2 · p. 3
  • Precision-2 diarization is aligned with Bengali captions into segment-level SSTT annotations, with the released 885-hour corpus and 364 videos pre-chunked for ASR training.The three released splits comprise raw diarization output, diarization aligned with transcription, and segment-chunked ASR-ready data.Source: §3.4 · p. 3

4 Dataset Description

Lipi-Ghor combines broad domain coverage with a sharply skewed distribution, varied linguistic profiles, and diarization-derived views of conversational structure.

4.1 Scale and Coverage

  • Three domains account for 58.6% of audio, while seven of seventeen account for 83.0%, reflecting uneven coverage across the corpus.Audiobooks, talkshows, and movies contribute the largest shares; domain balance depends on Bengali-caption availability.Source: §4.1 · p. 3

4.2 Linguistic Analysis

  • Average speaking rates range from 170.17 WPM in Service/Promo to 59.87 WPM in Regional/Dialect, while audiobook OOV reaches 50.35% versus 15–16% in news and political content.The corpus contains 181,648 unique word types.Source: §4.2 · p. 4
  • 262 utterances (0.026% of segments) contain Latin-script characters from Bengali–English code-switching and are retained with metadata flags; transcripts use weakly supervised YouTube captions.Caption tracks mix manually contributed and auto-generated text, and the corpus is intended for pretraining rather than evaluation.Source: §4.2 · p. 4

4.3 Conversational Structure

  • At least one detected speaker segment covers 639 of 885.34 audio hours (72.2%); the remainder contains silence, music, ambient sound, or other non-speech.Speech ratio is the fraction of each recording covered by a speaker turn, averaged over recordings.Source: §4.3 · p. 4, Table 3 · p. 4
  • The median recording has four speakers, but 25.9% are single-speaker narration and 31% have ten or more speakers, mostly talkshows and debates.Narration domains are mostly single-speaker and have the longest segments; performance domains have the highest speaker counts and shortest segments.Source: §4.3 · p. 4
  • The median diarized segment lasts 1.54 seconds, and 62.6% are shorter than two seconds, despite source recordings averaging roughly 52 minutes.The duration distribution shows nearly two-thirds of 984,647 segments under two seconds.Source: §4.3 · pp. 4–5, Figure 3 · p. 5

5 Gold Evaluation Split

The gold split provides held-out human transcript and speaker-label annotations for evaluation, addressing the weakness of caption-referenced assessment. It also separates transcript-reference disagreement from ASR model error and retains automatically generated diarization output.

5 Gold Evaluation Split

  • Seventeen volunteer native Bengali-speaking annotators corrected transcripts for 500 stratified clips, producing approximately 30 hours of human annotation across all 17 domains.The split includes human transcriptions and speaker labels and is held out from training data.Source: §5 · p. 5
  • Table 5 compares domain-wise disagreement between YouTube captions and human-corrected references using WER and CER.These values characterize transcript noise, not ASR system performance.Source: Table 5 · p. 5, §5 · p. 5
  • The gold split supplies human-verified transcript references, while released diarization output remains automatically generated for future diarization analysis.Automatic speaker segmentation can involve over-segmentation and false speaker assignments, affecting speaker-attributed transcript quality, especially with frequent speaker changes.Source: §5 · p. 5

6 Baseline Experiments

Lipi-Ghor fine-tuning outperforms the tested zero-shot systems on global WER, while zero-shot error varies with domain and segment structure. The evaluations also distinguish ASR performance from caption-reference noise.

6 Baseline Experiments

  • 20.70% WER for fine-tuned Faster Whisper Medium is below the best zero-shot result, Hishab TITU at 30.51%.Fine-tuned Fast Conformer scores 24.67% WER globally.Source: §6 · p. 5, Figure 4 · p. 6, §6.2 · p. 7

6.1 Zero-Shot Evaluation

  • Zero-shot WER ranges from 30.51% to 84.53%, and errors are lowest on audiobooks and news but highest on emotive drama, comedy, and regional dialects.Seamless M4T-v2 exceeds 100% WER on Regional/Dialect; Whisper large-v3 stays near 85% across domains.Source: §6.1 · p. 6, Table 7 · p. 6, Figure 5 · p. 7
  • Across seven domains, shorter median diarized segments associate with higher Hishab TITU WER (Spearman ρ = −0.75), but the relationship weakens when the smallest domains are dropped.Across all 17 domains, turn rate alone does not survive extension (ρ = +0.42, p = 0.09); Figure 6 plots segment duration against WER.Source: §6.1 · p. 6, Figure 6 · p. 7
  • Only the very-long utterance quartile shows a consistent WER uptick for TITU, from 28.5% to 32.6%.The effect is modest because diarization has already segmented long recordings into short turns and absorbed most long-form difficulty.Source: §6.1 · p. 6, Table 8 · p. 6

6.2 Fine-Tuned Baselines

  • Fine-tuned Faster Whisper Medium scores 20.70% WER and 11.90% CER globally after training on approximately 58 hours of Lipi-Ghor data.External benchmark results warrant caution because public models may have prior exposure to related Bengali resources.Source: §6.2 · p. 7, Table 9 · p. 7

6.3 Gold-Split Evaluation

  • Caption-versus-human transcript disagreement varies by domain and measures reference noise rather than ASR system performance.Table 5 reports WER and CER for the 500-clip human-corrected split.Source: §6.3 · p. 7

7 Discussion

The discussion identifies diarization artifacts that constrain overlap and speaker-count interpretation, and presents the pipeline as potentially generalizable beyond Bengali but validated only on Bengali.

7 Discussion

  • The diarizer caps simultaneous speakers at two, so overlap figures represent detected overlap under that ceiling rather than all genuine multi-speaker overlap.Talkshows and debates can contain three or more overlapping speakers.Source: §7 · p. 8
  • Speaker counts are more reliable at the low end and inflated at the high end, making the median of 4 more trustworthy than the mean of 8.9.The likely cause is over-segmentation; 21.8% of speaker labels cover under ten seconds, and one 105-label recording has a dominant-speaker share of 0.11.Source: §7 · p. 8
  • The pipeline is designed for languages with substantial YouTube presence and native-language captions, but cross-lingual transfer remains unvalidated.In this collection, 3–5 rich channels per domain yielded 20–50 hours, suggesting comparable scales may be achievable with modest curation effort.Source: §7 · p. 8

8 Conclusion

The conclusion presents Lipi-Ghor as an 885-hour, 17-domain weakly supervised Bengali corpus built with a scalable YouTube pipeline. Experiments show consistent degradation on several challenging speech types, while fine-tuning demonstrates utility for Bengali ASR.

8 Conclusion

  • Lipi-Ghor comprises 885 hours of weakly supervised Bengali speech across 17 domains, built with a scalable YouTube-based construction pipeline.The paper releases corpus metadata, annotations, pipeline code, and a human-annotated gold evaluation split.Source: §8 · p. 8
  • ASR experiments reveal consistent degradation on emotive, dialectal, and long-form speech.Source: §8 · p. 8
  • Fine-tuning demonstrates the corpus’s utility for Bengali ASR.Source: §8 · p. 8

Limitations

The corpus has limitations in transcript verification, diarization validation, dialect coverage, and exact reproducibility.

Limitations

  • Precision-2 speaker labels are automatically generated metadata, not a manually validated diarization benchmark, and the commercial API-gated model makes diarization not fully open.Released diarization output means downstream users need not rerun the model.Source: Limitations · p. 8
  • Regional dialect coverage is sparse at 4 videos and 1.29 hours, which is insufficient for dialect-robust training.Source: Limitations · p. 8
  • Exact reproduction is constrained by YouTube’s evolving API rate policies and the partially manual collection process.Video identifiers and pipeline scripts are released to mitigate this constraint.Source: Limitations · p. 8

9 Ethical Considerations

The release excludes raw audio and applies research-use and privacy restrictions, while providing a process for removal requests.

9 Ethical Considerations

  • Raw audio is not redistributed; video identifiers and pipeline scripts let users reconstruct audio locally, while original contributions are released under CC-BY-4.0.Caption-derived transcripts remain subject to uploader rights and are distributed for non-commercial research use only.Source: §9 · p. 8
  • Rights holders, uploaders, and people whose voices appear in source videos may request removal by providing video identifiers.Valid requests lead to removal, an updated dataset version, and a public changelog of removed identifiers.Source: §9 · p. 8
  • Speaker labels are video-local and do not support cross-video re-identification; the release prohibits re-identification and surveillance applications.Source: §9 · p. 8
  • Lipi-Ghor is intended for ASR pre-training, spoken language modeling, and multilingual NLP research, not high-stakes production ASR without additional human verification.Source: §9 · p. 8

A Extended Dataset Statistics

The extended statistics tables describe domain and caption-type distributions, with domain counts reported before deduplication and hours after it. The annotation toolkit supports correction of machine output, uses video to help disambiguate speaker turns, and preserves edit provenance; diarization outputs are released under CC-BY-4.0.

A Extended Dataset Statistics

  • Table 10 reports raw per-domain video counts before cross-domain deduplication and hours on the deduplicated corpus; Table 11 presents the distribution of YouTube caption types.Deduplication reduces 1,019 raw domain-counted videos to 998 unique videos.Source: Table 10 · p. 11, Table 11 · p. 11

B The Annotation Toolkit

  • Annotators correct existing transcript and diarization output in an interface with playback, speaker-turn visualization, and editable segments; video serves as an audio-subordinate aid for ambiguous speaker attribution.The audio remains authoritative because off-screen speakers can make visual cues misleading.Source: B The Annotation Toolkit · p. 11, Figure 7 · p. 12
  • Each submitted job records the annotator, edit timestamps, and starting machine baseline, and exports corrected jobs in the corpus’s SSTT schema.This provenance supports segment-level analysis and reproducible evaluation; gold clips can replace noisy counterparts without changing the training pipeline.Source: B The Annotation Toolkit · pp. 11–12

C Diarization Output Licensing

  • Generated Lipi-Ghor diarization outputs are released under CC-BY-4.0, but that license does not cover the proprietary Precision-2 model or service.The service’s Terms of Use assign rights in generated results to the customer.Source: C Diarization Output Licensing · p. 12

Related papers

Loading 2610.36184v1…