Source-linked AI summary
Flexible and Interpretable Accent Distance Measurements
Charles McGhee, Mark J. F. Gales, Kate M. Knill
TL;DR
Accent comparison requires methods that are both interpretable and applicable beyond limited paired recordings, while embedding-based approaches are flexible but difficult to analyze. This paper combines articulatory representations from articulatory inversion with optimal transport to compare accents across arbitrary recording types. It achieves similar accent-classification performance to accent-classification models through alignment, extends comparison to arbitrary recordings with slightly lower accuracy using optimal transport, and identifies rhoticity differences interpretably.
Problem
Accent comparison needs interpretable phonetic measures without the time-consuming, limited recordings required by vowel comparisons, while accent embeddings are less readily interpretable.
Method
The paper uses deep articulatory representations for pronunciation comparison and optimal transport for utterance-independent comparison of unpaired speech.
Results
Articulatory alignment achieves similar accent-classification performance to accent-classification models, while optimal transport extends comparison to arbitrary recordings with slightly lower accuracy and identifies rhoticity differences.
Takeaways & Limitations
Articulatory features provide an interpretable basis for accent comparison, and optimal transport broadens that comparison beyond paired recordings.
Takeaways & Limitations
The evaluation assumes accent and speaker identity are mostly disentangled, while acknowledging that some global speaker properties may help predict accent identity.
Abstract
from arXiv · showhide
Determining the differences between two speakers' accents is a fundamental task in linguistics and speech technology research. The methodology used to measure these differences depends on the specific research area. A phonetics researcher may demonstrate accent variation by comparing vowel formants in paired recordings of individual words. These results will be interpretable, but the recordings will be time-consuming to collect and may not be representative of connected speech. Accented Text-to-Speech (TTS) research has pushed towards using accent embeddings derived from accent classification tasks. These embeddings can be produced from any speech recording, but are not readily interpretable. In this paper, we demonstrate that articulatory representations created through articulatory inversion can be used as an interpretable basis for accent comparison and that optimal transport provides a framework for accent comparison across arbitrary recording types.
1. Introduction
Accent comparison must balance interpretability with coverage of natural speech: traditional phonetic measures are interpretable but costly and limited, while embeddings are flexible but harder to analyze. The paper combines deep articulatory representations with optimal transport to bridge these approaches.
- Motivation: Accent measures should capture regional or social pronunciation differences while remaining interpretable across realistic speech contexts.The paper frames accent as segmental and suprasegmental pronunciation properties that identify a speaker’s region or social class.
- Motivation: Vowel-formant comparisons provide interpretable phonetic evidence but require time-consuming recordings and may not represent connected or conversational speech.The motivation specifically contrasts minimal-set vowel measurements with the demands of broader speech data.
- Motivation: Accent embeddings can compare speech at a glance across recordings, but their learned representations are less readily analyzable than vowel-distance measures.Embedding models such as GenAID are trained on national-level accent classification labels and are intended to capture a broad definition of accent.
- Contribution: The paper uses deep articulatory representations and optimal transport to compare specific phonetic differences while supporting unpaired, utterance-independent accent comparison.The proposed combination is designed to retain phonetic interpretability while moving beyond paired recordings.
2. Background
The paper derives speaker-comparable articulatory features from speech and uses optimal transport to compare unpaired feature distributions. Under equal-sized, uniformly weighted inputs, the transport solution becomes a permutation-based distance between speakers’ feature spaces.
- 2.1. Deep Articulatory Representations: Acoustic-to-Articulatory Inversion estimates articulation from speech, with speaker-consistent representations designed to map similar phonetic categories similarly across speakers.The representations build on regression from speech to articulatory targets and include VVN features capturing voicing and nasality.
- 2.2. Optimal Transport: Optimal transport compares unpaired speech-feature distributions by finding the minimum-cost transformation between two speakers’ feature sets.This framework is motivated by settings where paired speech data are impractical or impossible to collect.
- 2.2. Optimal Transport: For feature sets X ∈ R^n×f and Y ∈ R^m×f, a Euclidean cost matrix assigns pairwise costs between the speakers’ feature points.Each matrix entry represents the cost of comparing one point from X with one point from Y.
- 2.2. Optimal Transport: The optimal coupling matrix distributes mass between feature points while minimizing total transport cost subject to the input weightings.The coupling constraints are defined over distributions a and b on the probability simplex.
- 2.2. Optimal Transport: With equal numbers of uniformly weighted points, the coupling is a permutation matrix and its optimal cost becomes the distance between the two feature spaces.The optimal solution is computed with the network simplex algorithm.
3. Evaluation
The evaluation compares accent and phonetic-distance measures across identity and pronunciation axes using classification, speaker-distance correlation, and cross-method comparisons. It includes vowel baselines, paired-utterance alignment, and optimal transport with clustered speech features.
- 3.1. Identity: The evaluation tests whether accent distances separate accent identity from speaker identity using classification and correlation with speaker-embedding distances.Speaker-embedding correlation is measured with averaged Spearman rank correlations across the speaker set.
- 3.1. Identity: Accent classification uses three-way national-level labels for Scottish, English, and Irish speakers from VCTK.VCTK provides paired and unpaired recordings, substantial speech per speaker, and regional information for later analyses.
- 3.2. Phonetic Distance: Phonetic-distance evaluation compares vowel, utterance-alignment, and sets-of-utterances measures to expose differences between pronunciation and embedding distances.The evaluation proceeds from vowel distances to paired-utterance alignment and then broader comparisons.
- 3.2. Phonetic Distance: Vowel distances provide a baseline by measuring midpoint articulatory, WavLM, and formant features while excluding suprasegmental and consonantal differences.The Montreal Forced Aligner identifies vowel positions within utterances.
- 3.2. Phonetic Distance: Articulatory and WavLM alignment uses dynamic time warping, with a modified step pattern chosen to avoid skipping sections with substantially different pronunciation.The comparison uses Euclidean distance for articulatory features and cosine distance for final-layer WavLM features.
- 3.3. Optimal Transport: Optimal transport is computed after k-means reduction because the 50-Hz features from roughly 12.5 minutes per VCTK speaker are too large for direct transport.The implementation uses the POT package and k=1000 clustering to reduce the number of input points.
4. Results
Articulatory alignment matched GenAID’s accent-classification performance while showing lower correlation with speaker attributes, and OT extended comparison to unpaired utterances. The analyses also illustrate interpretable pronunciation differences, including rhoticity, while showing sensitivity to input quantity.
- 4.1. Vowel Distances and Alignment: Articulatory-feature alignment achieved the same accent-classification results as GenAID while having lower correlation with speaker attributes.The comparison used accent classification accuracy and speaker-embedding correlation; all methods failed on speaker p285, which appeared mislabelled.
- 4.1. Vowel Distances and Alignment: DTW alignment outperformed vowel-distance metrics, likely because it captured consonantal and suprasegmental details omitted by vowel-only comparisons.Final-layer WavLM features correlated more strongly with speaker embeddings than the 14-dimensional articulatory representation.
- 4.2. Optimal Transport: Optimal transport with articulatory features had slightly worse classification performance than alignment but higher performance than pure vowel distances.It also showed higher correlation with other phonetic distance measures than GenAID, while GenAID and WavLM had higher correlation with speaker-embedding distances.
- 4.2. Optimal Transport: Alignment and OT with articulatory features encoded fairly similar information about geographically close speakers, although Scottish–Irish boundaries were less clear than English boundaries.Figure 1 compares normalized distances from a target speaker using articulatory alignment/OT and GenAID.
- 4.3. Interpretability: OT exposed interpretable rhoticity-related differences: rhotic tongue positions aligned with clusters that moved farther during transport, though the position was not definitively a rhotic /ô/.For the rhotic speaker, the /ô/ tongue position was retracted; for the non-rhotic speaker, the vowel position was lowered.
- 4.4. Optimal Transport Ablations: OT classification accuracy peaked with 7.5 minutes of speech per speaker and 100 clusters, while alignment SRC improved with more data and clusters.The authors suggest focusing on articulatory targets as a possible route to better OT data efficiency and interpretability.
5. Conclusion
The paper shows that articulatory features can support accent classification and, with optimal transport, compare arbitrary recordings while retaining phonetic interpretability.
- Articulatory features achieve similar accent-classification performance to accent-classification models through feature alignment.
- Optimal transport extends articulatory accent comparison to arbitrary recording types, with slightly lower classification accuracy.
- Alignment and optimal transport provide more interpretable accent representations than supervised-task embeddings by identifying rhoticity differences between speakers.
- Future work will improve OT efficiency and interpretability by clustering articulatory targets and use alignment or OT distances to control accented TTS.
6. Generative AI Use Disclosure
The paper reports that generative AI was not used in its production.
- Generative AI was not used in producing the paper.