Source-linked AI summary
A Novel Sentence Stress Detection Framework Leveraging Auxiliary Word-Stress Modeling and Loss Optimization
Tien-Hong Lo, Fong-Chun Tsai, Ting-An Hung, Yu-Hsuan Hsieh, Yao-Ting Sung, Berlin Chen
TL;DR
Prior work models sentence and word stress detection independently, despite their shared reliance on prosodic cues. STRAW combines token-level sentence stress detection with auxiliary phone-level word stress detection and a word-span regularizer; on TinyStress-15K, it achieves the best reported SSD result among the evaluated configurations.
Problem
Prior work models sentence stress detection (SSD) and word stress detection (WSD) independently despite their shared reliance on prosodic cues.
Source: §1 · p. 1Method
STRAW uses a frozen Whisper backbone with task-specific SSD and auxiliary phone-level WSD branches, plus a word-span regularizer that concentrates SSD probabilities within stressed word spans.
Source: §1 · pp. 1–2, §5 · p. 4Results
0.934 F1, improving over WhiStress by 2.5 points (0.909 →0.934), with consistent gains in both precision and recall.
Source: §4.1 · p. 4Takeaways & Limitations
The design supports sentence- and word-level stress analysis while addressing within-word ambiguity induced by subword tokenization.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Prosodic stress is a crucial aspect of automatic pronunciation assessment (APA), encompassing both sentence stress detection (SSD) and word stress detection (WSD). SSD highlights semantically salient words that shape discourse meaning, while WSD identifies the primary stressed syllable within each word to ensure lexical clarity. However, most prior work treats SSD and WSD as independent tasks, overlooking their shared reliance on prosodic cues such as pitch, duration, and intensity. To address this gap, we propose an effective SSD approach combining SSD with auxiliary WSD via a novel modeling paradigm. In addition, we introduce a word-span stress regularizer (WSR) that concentrates token-level SSD probabilities within each stressed word span. Experiments on the TinyStress-15K benchmark show that the proposed method outperforms strong baselines, with the complete configuration achieving the best SSD result.
1. Introduction
Sentence stress detection (SSD) identifies semantically salient words, while word stress detection (WSD) locates primary stress within a word. Although both rely on prosodic cues, prior work has modeled them independently; STRAW combines them and adds a word-span stress regularizer.
1. Introduction
- Prior work models SSD and WSD independently despite their shared reliance on prosodic cues, leaving the potential benefits of a unified framework unexplored.Prosodic correlates include duration, intensity, and pitch.Source: §1 · p. 1
- STRAW combines token-level SSD and auxiliary phone-level WSD using a frozen Whisper backbone, with separate task-specific branches.The branches share the backbone but do not directly exchange hidden states or predictions.Source: §1 · p. 1, Figure 2 · p. 2
- The word-span stress regularizer discourages diffuse SSD probabilities across the subword tokens of a stressed word.Source: §1 · p. 2
2. Method
The method uses a frozen Whisper backbone with task-specific SSD and WSD heads, then combines their objectives with a word-span regularizer. SSD predicts stress at the token level, WSD at the phone level, and the regularizer sharpens SSD probabilities within stressed word spans.
2. Method
- Utterances are represented as log-mel spectrograms and word, token, and phone sequences; a frozen Whisper backbone supplies contextualized encoder and decoder states.Only task-specific heads and the phone embedding layer are trained.Source: §2 · p. 2
2.1. Sentence Stress Detection
- The SSD head uses decoder states as queries and encoder states as keys and values, then predicts token-level stress with cross-entropy loss.Evaluation aggregates token predictions to the word level, labeling a word stressed if any token is predicted stressed.Source: §2.1 · p. 2, §2 · p. 2
2.2. Word Stress Detection
- The WSD branch uses phone embeddings as queries over encoder states and predicts stress labels for phones with cross-entropy loss.The phone sequence is produced using a grapheme-to-phoneme converter with CMU-style stress-marked phones.Source: §2.2 · p. 2, §2 · p. 2
2.3. Word-Span Stress Regularizer
- The word-span stress regularizer addresses multiple elevated token probabilities within a stressed word by encouraging one dominant position and suppressing the others.It weights peak sharpening by how far total SSD stress probability within the word deviates from one.Source: §2.3 · p. 2
2.4. Optimization
- The final objective combines SSD, WSD, and WSR losses using nonnegative weighting coefficients α, β, and λ.The framework’s loss is L = α · LSSD + β · LW SD + λ · LW SR.Source: §2.4 · p. 3
3. Experiment Setup
The experiments use TinyStress-15K and compare STRAW with alignment-based and alignment-free baselines. They describe the training setup and evaluate stressed-class precision, recall, and F1 for SSD and WSD.
3.1. Dataset
- TinyStress-15K provides synthetic speech with word-level stress annotations; WSD labels mark primary-stressed phones, and 10% of training data is held out for validation.The official training and test partitions are retained; the training split contains 13.5k utterances and 171k tokens.Source: §3.1 · p. 3
3.2. Comparison Methods
- STRAW is compared with timestamp- or MFA-aligned prosodic-feature pipelines and alignment-free WhiStress, extending the latter with auxiliary WSD and WSR.The alignment-based systems use duration, energy, and pitch features with a BLSTM classifier.Source: §3.2 · p. 3
3.3. Implementation Details
- The experiments freeze whisper-small2 and train only the SSD head, WSD head, and phone embedding layer; the WSD branch can be disabled at inference.The heads are optimized in one training run, and the checkpoint with the highest validation SSD binary F1 is selected.Source: §3.3 · p. 3
3.4. Evaluation Metrics
- Precision, recall, and F1 are reported for the stressed class; SSD predictions are mapped to words, while WSD is evaluated over stressed phone labels.A word is marked stressed if any constituent token is predicted stressed.Source: §3.4 · p. 3
4. Experimental Results
On TinyStress-15K, STRAW achieves the strongest reported SSD result, while ablations, WSD results, and POS error analysis characterize its components and behavior.
4.1. SSD Performance Comparison with Baselines
- F1 0.934 is STRAW’s best overall SSD result, 2.5 points above WhiStress (0.909 →0.934), with consistent precision and recall gains.Table 2 summarizes SSD performance on TinyStress-15K.Source: §4.1 · pp. 3–4
4.3. Error Analysis
- Figure 3 reports POS-specific FNR and FPR alongside sample counts; STRAW modestly reduces baseline false negatives on PART and DET, with further gains on AUX and ADP.False positives remain low, while SCONJ false negatives persist where sample counts are limited.Source: Figure 3 · p. 4, §4.3 · p. 4
4.1. SSD Performance Comparison with Baselines
- Removing WSD, WSR, or both reduces SSD F1 to 0.922, 0.929, and 0.915, respectively.The separate trainable heads mean the WSD ablation does not establish direct knowledge transfer to SSD; the WSR result is consistent with its intended role of limiting diffuse within-word predictions.Source: §4.1 · p. 4
4.2. WSD Performance
- STRAW attains 0.924 precision, 0.916 recall, and 0.920 F1 for WSD; removing WSR changes WSD F1 only marginally, from 0.920 to 0.921.Comparable external WSD baselines are limited because TinyStress-15K is primarily an SSD benchmark.Source: §4.2 · p. 4
5. Conclusion
STRAW combines an auxiliary WSD head with WSR to constrain token-level SSD predictions within stressed word spans. The TinyStress-15K results favor the complete configuration, while the conclusion identifies limits in stress representation, task interaction, and evaluation scope.
5. Conclusion
- STRAW combines token-level SSD with auxiliary WSD and WSR, constraining SSD predictions within stressed word spans to mitigate subword-tokenization ambiguity.The complete configuration has the highest F1 among the evaluated configurations and improves over strong baselines on TinyStress-15K.Source: §5 · p. 4
- WSD simplifies lexical stress to one primary position per word and does not explicitly model syllabic structure or secondary stress..Source: §5 · p. 4
- The frozen backbone and separate task-specific heads provide no direct interaction between SSD and WSD.The evaluation is also confined to synthetic speech because large-scale, fine-grained stress annotations are scarce.Source: §5 · p. 4
6. Generative AI Use Disclosure
The authors disclose using ChatGPT for language editing and rephrasing selected passages, while retaining responsibility for and verifying the technical content.