Source-linked AI summary
M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
Fan Yu, Shiliang Zhang, Yihui Fu, Lei Xie, Siqi Zheng, Zhihao Du, Weilong Huang, Pengcheng Guo, Zhijie Yan, Bin Ma, Xin Xu, Hui Bu
TL;DR
Meeting transcription lacks large, richly annotated public datasets despite difficult overlap, acoustics, and multi-speaker conditions. This paper releases the AliMeeting Mandarin corpus and launches M2MeT with diarization and multi-speaker ASR tracks, reporting baseline evaluations while defining constrained and expanded-data settings. The resulting open corpus and common testbed support reproducible evaluation of meeting transcription systems.
Problem
Large labeled meeting corpora are limited, while meeting transcription requires precise annotations for speaker identity, context, and timing under complex acoustic conditions.
Method
The paper constructs AliMeeting and organizes M2MeT around speaker diarization and multi-speaker ASR tracks using constrained and extra-data sub-tracks, evaluation methods, and baselines.
Results
The AliMeeting corpus contains 120 hours of Mandarin meeting data, and baseline results are reported for speaker diarization and multi-speaker ASR.
Takeaways & Limitations
AliMeeting and M2MeT provide open data and a common testbed for reproducible research in meeting transcription and related speech processing.
Takeaways & Limitations
The constrained sub-track restricts system development to AliMeeting, Aishell-4, and CN-Celeb, whereas the expanded sub-track permits additional data that must be documented.
Abstract
from arXiv · showhide
Recent development of speech processing, such as speech recognition, speaker diarization, etc., has inspired numerous applications of speech technologies. The meeting scenario is one of the most valuable and, at the same time, most challenging scenarios for the deployment of speech technologies. Specifically, two typical tasks, speaker diarization and multi-speaker automatic speech recognition have attracted much attention recently. However, the lack of large public meeting data has been a major obstacle for the advancement of the field. Therefore, we make available the AliMeeting corpus, which consists of 120 hours of recorded Mandarin meeting data, including far-field data collected by 8-channel microphone array as well as near-field data collected by headset microphone. Each meeting session is composed of 2-4 speakers with different speaker overlap ratio, recorded in rooms with different size. Along with the dataset, we launch the ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) with two tracks, namely speaker diarization and multi-speaker ASR, aiming to provide a common testbed for meeting rich transcription and promote reproducible research in this field. In this paper we provide a detailed introduction of the AliMeeting dateset, challenge rules, evaluation methods and baseline systems.
1. INTRODUCTION
Meeting transcription addresses who speaks what and when in challenging multi-speaker conditions, but progress is constrained by limited, costly-to-create meeting data. The paper responds with the AliMeeting corpus and M2MeT challenge as a common testbed for speaker diarization and multi-speaker ASR.
- Motivation: Meeting transcription combines speaker identification, diarization, recognition, and front-end processing to resolve who speaks what and when.The task is difficult because meetings involve overlap, unknown speaker counts, far-field signals, noise, and reverberation.
- Motivation: Limited sizable labeled meeting data is a major bottleneck because corpus construction requires complex recording setups and expensive, precise annotations.Existing corpora also vary in size, speaker coverage, recording conditions, content, and language.
- Contributions: M2MeT uses AliMeeting and Aishell-4 to establish speaker diarization and multi-speaker ASR challenge tracks.Each track includes a constrained-data sub-track and a sub-track allowing extra constrained data, with rules and baselines released alongside the corpus.
2. THE ALIMEETING CORPUS
AliMeeting is a multi-venue Mandarin meeting corpus designed around varied room acoustics, microphone conditions, speaker counts, and overlap ratios. It combines far-field array recordings with synchronized near-field headset signals and detailed session annotations.
- Dataset composition: AliMeeting contains 118.75 hours split into 104.75 training, 4 evaluation, and 10 test hours across meeting sessions.Train and Eval include 212 and 8 sessions, respectively, with 456 and 25 participants and balanced gender coverage.
- Recording conditions: Recordings span 13 small, medium, and large venues ranging from 8 to 55 m2, providing varied acoustic properties, layouts, and furnishings.Participants sit around a table-mounted microphone array, with microphone-speaker distances from 0.3 to 5.0 m and naturally occurring indoor noise.
- Meeting content: Each meeting has 2 to 4 participants, with Train and Eval average speech-overlap ratios of 42.27% and 34.76%, respectively.Topics cover medical treatment, education, business, organization management, industrial production, and daily routine meetings.
- Recording conditions: The corpus pairs far-field multi-speaker array audio with each participant’s near-field headset signal on a synchronized common timeline.Session scripts are prepared in TextGrid format and include session duration and speaker information.
3. TRACK SETTING AND EVALUATION
The challenge evaluates speaker diarization and multi-speaker ASR on far-field meeting audio under defined scoring procedures. Both tracks provide constrained-data and open-data sub-tracks, with track-specific output and metrics.
- Speaker Diarization: Track 1 requires participants to determine the speaker at each time point and submit an RTTM file for every session.The test set provides only eight-channel far-field audio and sentence-segmentation timestamps, not headset audio or transcriptions.
- Speaker Diarization: DER sums speaker confusion, false alarm, and missed detection durations relative to total duration.The evaluation aligns hypothesis outputs with references using the Hungarian algorithm and applies a 0.25-second no-score collar around reference boundaries.
- Multi-Speaker ASR: Track 2 requires transcription of each speaker in overlapped speech but does not require identifying the speaker corresponding to each transcript.Its test audio is the same as Track 1, and performance is measured by Character Error Rate.
- Multi-Speaker ASR: CER is based on the minimum character insertions, substitutions, and deletions needed to transform a hypothesis into the reference transcript.The metric includes spaces and normalizes the three error counts by the total number of reference characters.
- Multi-Speaker ASR: Two CER schemes address permutation-invariant overlapping speech: utterance-based FIFO and speaker-based concatenation over possible patterns.FIFO orders reference utterances by start time and joins them with the <sc> token, whereas the speaker-based scheme combines utterances from each speaker.
- Sub-Tracks: Both tracks have a constrained sub-track limited to AliMeeting, Aishell-4, and CN-Celeb, and an open sub-track allowing additional disclosed data.Participants using manually simulated data in Sub-track II must describe the simulation scheme in detail.
4. BASELINE SYSTEMS
The baselines cover speaker diarization, single-speaker ASR, and multi-speaker ASR using Kaldi and Conformer-based systems, with comparisons across near-field, far-field, and beamformed data. Results show that external-data augmentation benefits far-field single-speaker ASR, while SOT improves multi-speaker recognition.
- 4.1. Speaker Diarization Baseline: The diarization baseline uses speaker embeddings, agglomerative hierarchical clustering, and VBx clustering, with oracle speech-activity timestamps provided to participants.The embedding network is ResNet-based and trained on CNCeleb; AHC initializes labels before VBx clustering.
- 4.1. Speaker Diarization Baseline: Diarization DER increases with more speakers and higher overlap because the baseline assumes each speech frame belongs to only one speaker.Short-time overlap near segment boundaries particularly increases DER under a 0-second collar; challenge scoring uses a 0.25-second collar.
- 4.2. Multi-Speaker ASR Baseline: The single-speaker ASR baselines use Conformer encoders, with training variants covering near-field data, far-field data, external speech data, and simulated noise and reverberation.The systems use ESPnet, 71-dimensional mel-filterbanks, and a 12-layer encoder with a 6-layer decoder.
- 4.2. Multi-Speaker ASR Baseline: The multi-speaker ASR baseline applies Serialized Output Training to generate multiple speakers’ transcriptions sequentially, using speaker-change symbols and utterance-based FIFO ordering.SOT models dependencies among different speakers’ outputs without limiting the maximum number of speakers.
- 4.2. Multi-Speaker ASR Baseline: ConformerD substantially improves far-field single-speaker ASR while causing only a small degradation on the near-field Eval set.ConformerD adds simulated noise and reverberation to external near-field data to reduce the acoustic mismatch with real far-field meetings.
- 4.2. Multi-Speaker ASR Baseline: SOT significantly improves the multi-speaker Eval set over the single-speaker model, while beamforming also benefits the Ali-far-bf Eval set.Adding Train-Ali-near data to the multi-speaker model improves both Eval sets.
5. CONCLUSIONS
M2MeT introduces datasets, evaluation procedures, metrics, tracks, and baseline systems for a multi-channel multi-party meeting transcription challenge. Its released open data and common testbed are intended to advance meeting transcription and related speech processing tasks.
- 5. CONCLUSIONS: M2MeT describes the datasets, tracks, evaluation methods, metrics, and baseline systems for meeting transcription research.The challenge is presented as a common testbed built around released open data.