Source-linked AI summary
Cross-Attention is Half Explanation in Speech-to-Text Models
Sara Papi, Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli
TL;DR
Cross-attention is widely used in speech-to-text, but its explanatory reliability remains underexplored. The paper compares it with feature-attribution saliency across S2T settings and finds moderate-to-strong alignment that still leaves substantial relevance unexplained.
Problem
Method
The study compares cross-attention with SPES input saliency and encoder-output saliency across ASR and ST models and settings.
Source: §1 · p. 1, §3.2 · p. 3Results
Cross-attention moderately to strongly aligns with saliency but captures only about half of input relevance and 52–75% of encoder-output saliency.
Source: §7 · p. 8Takeaways & Limitations
Cross-attention provides informative cues but is an incomplete explanation proxy and should not be treated as a stand-alone XAI tool.
Source: §6 · p. 8, §7 · p. 8The experiments cover ASR and ST, multilingual analysis is limited to English and Italian, and SpeechLLMs are not evaluated.
Source: §8 · p. 8
Abstract
from arXiv · showhide
Cross-attention is a core mechanism in encoder-decoder architectures, widespread in many fields, including speech-to-text (S2T) processing. Its scores have been repurposed for various downstream applications--such as timestamp estimation and audio-text alignment--under the assumption that they reflect the dependencies between input speech representation and the generated text. While the explanatory nature of attention mechanisms has been widely debated in the broader NLP literature, this assumption remains largely unexplored within the speech domain. To address this gap, we assess the explanatory power of cross-attention in S2T models by comparing its scores to input saliency maps derived from feature attribution. Our analysis spans monolingual and multilingual, single-task and multi-task models at multiple scales, and shows that attention scores moderately to strongly align with saliency-based explanations, particularly when aggregated across heads and layers. However, it also shows that cross-attention captures only about 50% of the input relevance and, in the best case, only partially reflects how the decoder attends to the encoder's representations--accounting for just 52-75% of the saliency. These findings uncover fundamental limitations in interpreting cross-attention as an explanatory proxy, suggesting that it offers an informative yet incomplete view of the factors driving predictions in S2T models.
1. Introduction
Cross-attention scores are widely used in S2T applications as indicators of input-output dependencies, but their explanatory reliability in speech has not been directly assessed. This study compares them with feature-attribution saliency maps and finds meaningful but incomplete alignment.
1. Introduction
- Cross-attention scores are used for S2T applications including audio-text alignment, speaker identification, timestamp estimation, and simultaneous ASR and ST guidance.These uses assume that attention indicates which input signal content matters during output generation.Source: §1 · p. 1
- Whether cross-attention reliably indicates input dependencies remains unverified, and encoder context mixing may obscure its alignment with the original audio.Prior S2T explainability research focused mainly on self-attention or the effects of context mixing, not cross-attention’s explanatory potential.Source: §1 · p. 1
- Across ASR and ST, monolingual, multilingual, and multitask models, cross-attention aligns moderately to strongly with saliency but captures only about 50% of input relevance and 52–75% of encoder-output saliency.The study compares attention with SPES input saliency and encoder-output saliency to examine context mixing.Source: §1 · p. 1
2. Related Works
Speech explainability research has grown more slowly than work in vision and language, while attention’s explanatory value remains debated. Cross-attention in speech-to-text is a largely unexplored part of that debate.
2. Related Works
- Speech signals’ time-frequency complexity and variable output lengths are identified as challenges for explainability research.The text notes progress in speech classification and S2T generation despite these challenges.Source: §2 · p. 1
- Perturbation-based attribution can produce S2T saliency maps, but its computational cost motivates exploring inference-time cross-attention as a lightweight alternative.SPES identifies spectrogram regions influencing predictions in autoregressive S2T models.Source: §2 · p. 2
- Prior attention-explanation research has debated attention’s reliability and examined aggregation, but has focused mainly on self-attention rather than speech cross-attention.Related work also explores combining attention with attribution methods or using attention as a training-time regularizer.Source: §2 · p. 2
3. Methodology
The methodology compares cross-attention with input and encoder-output saliency maps to assess alignment and the possible impact of context mixing. It extracts attention and attribution patterns, aligns their temporal granularity, and measures their correlation.
3. Methodology
- The analysis compares cross-attention with input saliency and encoder-output saliency, using differences between the comparisons to assess context-mixing effects.Figure 1 depicts which model components are covered by the two saliency maps.Source: §3 · p. 2, Figure 1 · p. 2
3.1. Cross-Attention in Speech-to-Text
- Cross-attention relates decoder token states to encoder outputs; the model downsamples speech inputs before the encoder produces representations used for autoregressive decoding.Attention can be examined per head and layer or averaged across selected heads and layers.Source: §3.1 · p. 2
3.2. Feature Attribution for Speech-to-Text
- SPES perturbs clustered spectrogram regions and maps output-distribution changes back to input saliency, while encoder-output saliency perturbs hidden states to estimate their contribution to each token.The input perturbations mask regions; the hidden-state perturbations set encoder states’ features to zero.Source: §3.2 · p. 3
3.3. Correlation
- Input saliency is aggregated over frequency and downsampled in time to match cross-attention’s granularity before the maps are normalized and compared.The study uses Pearson correlation because saliency magnitudes, not just their ordering, are meaningful.Source: §3.3 · p. 3
4. Experimental Settings
The experiments use monolingual and multilingual speech-to-text models, test data, and an evaluation pipeline to compare cross-attention with input explanations. They select 2D max pooling for input explanations and report output and explanation quality across models.
4.1. Models
- The study uses a 125M-parameter monolingual English ASR model and multilingual, multitask FAMA models with 474M and 878M parameters.All use Conformer encoders and Transformer decoders; the FAMA models are trained first for ASR and then jointly for ASR and ST.Source: §4.1 · pp. 3–4
4.2. Data
- Training data range from about 3,000 hours for the monolingual model to more than 150,000 hours for the multilingual models; EuroParl-ST supplies the test set.The test set covers English and Italian ASR and English-to-Italian and Italian-to-English ST.Source: §4.2 · p. 4
4.3. Evaluation Process
- Evaluation generates hypotheses with beam search, computes Pearson correlations across test samples, and uses SPES perturbation-based saliency maps for input explanations.The SPES explanation generation takes about 27 hours for base, 3–4 days for small, and 6–8 days for large, depending on language.Source: §4.3 · p. 4
4.4. Effect of Aggregation Functions on Input Explanations
- 2D max pooling produces the best-quality explanations and highest correlations; 2D average pooling performs worst, while 2-step pooling approaches but does not match 2D max.The overall layer-averaged correlation is 0.572 for 2D max and 0.565 for 2-step pooling; averaging can blur localized frequency saliency.Source: §4.4 · p. 5, Table 1 · p. 5
4.5. Quality Metrics for the Reported Models
- Across models, output scores are mostly competitive; ASR deletion scores approach 100 (97% on Italian), ST deletion scores are near 0, and explanation sizes range from 28.2 to 30.6.The models outperform popular large-scale models in English ASR with the base model and English-to-Italian ST with the large model.Source: §4.5 · pp. 5–6, Table 2 · p. 6
5. Results
Across model sizes, languages, and tasks, the results compare cross-attention with input and encoder-output saliency. Aggregation improves alignment, but cross-attention captures only part of the relevance in either representation.
5.1.1. Head-wise and Layer-wise Correlations
- At the individual-head level, correlations are generally low, while head averaging improves alignment; layer 6 is most aligned, followed by layer 5 and the all-layer average.The strongest reported correlation reaches 0.588; averaging across both heads and layers is close but not the overall best, as indiscriminate layer aggregation can dilute relevance.Source: §5.1.1 · p. 6
5.1.2. Multitask and Multilingual Correlations
- The best layer varies by model size: layer 5 for small and layers 8–10 for large, while the final layers remain closely aligned across tasks and languages.Across all models, the last decoder layers provide stronger or near-optimal correlations than the first layers for encoder-output saliency as well.Source: §5.1.2 · p. 6, §5.2 · p. 7
- Averaging across heads and layers yields the best or near-best input-saliency correlations in almost all configurations.The exceptions are small-model English and Italian ASR, where selective-layer extraction improves correlation by 0.006 and 0.001, respectively.Source: §5.1.2 · pp. 6–7
- 49–63% of input relevance is aligned with cross-attention across multilingual and multitask configurations, despite moderate-to-strong correlations after aggregation.English ASR has the highest correlations; Italian ASR also exceeds ST, and the authors relate language differences to English's larger training share.Source: §5.1.2 · pp. 6–7
5.2. What Is the Impact of Context Mixing?
- 52–75% of encoder-output relevance is captured by cross-attention, which remains an incomplete explanation even after context mixing is factored out.Correlations with encoder-output saliency exceed those with input saliency by ρ differences of 0.03–0.18, quantifying context-mixing effects at 6.6–16.7%.Source: §5.2 · p. 7
6. Discussion
Cross-attention provides lightweight, complementary cues, but it is not a faithful stand-alone explanation of speech-to-text predictions. Its deletion results and partial saliency alignment point to weaker, coarser attributions than feature-attribution maps.
6. Discussion
- 41.2 for cross-attention, versus 52.9 for frequency-aggregated SMX and 91.3 for full-resolution maps, on the deletion metric.The authors report that cross-attention discards fine-grained time-frequency cues and produces weaker attributions even under identical aggregation.Source: §6 · p. 7
- Cross-attention correlates moderately with aggregated SMX, peaking around 0.45–0.55, but does not capture full input relevance even when context mixing is factored out.The trend holds across correlation and deletion scores, which the authors interpret as an intrinsic constraint of attention as an explanatory signal.Source: §6 · p. 7
- Averaging across layers, especially heads, can improve the robustness of attention-based downstream methods, but cross-attention should remain an auxiliary cue rather than a substitute for attribution.The paper identifies training-time alignment between attention and attribution as a promising direction for interpretability and task performance.Source: §6 · pp. 7–8
7. Conclusions
Across tasks, languages, and model scales, cross-attention moderately to strongly aligns with saliency, especially when aggregated across heads and layers. It captures only about half of input relevance and remains a partial explanation of S2T predictions.
- Cross-attention captures only about half of input relevance; when encoder-output analysis disentangles context mixing, it explains just 52-75% of saliency.Source: §7 · p. 8
- Across tasks, languages, and model scales, cross-attention moderately to strongly aligns with saliency, especially when averaged across heads and layers.Source: §7 · p. 8
- Cross-attention provides informative cues but only a partial view of the factors driving S2T predictions.Source: §7 · p. 8
8. Limitations
The analysis is limited to ASR and ST, English and Italian, and speech foundation models rather than SpeechLLMs. It uses SPES as a silver reference, which may introduce some margin of error.
- The evaluation covers ASR and ST, but not downstream tasks such as spoken question answering or speech summarization.The authors note that other tasks may involve different decoder-attention dynamics.Source: §8 · p. 8
- The multilingual analysis is limited to English and Italian because broader large-scale model training is computationally costly.Source: §8 · p. 8
- The study focuses on speech foundation models, not SpeechLLMs, and uses retrained or open-science models with documented training data to avoid contamination concerns.Source: §8 · p. 8
- SPES serves as a silver reference rather than a gold or human reference, and may introduce some margin of error.The authors report that SPES has high explanation quality and is more faithful than alternatives such as gradient norms.Source: §8 · p. 8