Source-linked AI summary
Behind the [MASK]: Disentangling Representation and Faithfulness in DAPF-Based Dementia Detection
Pardis Ranjbar-Noiey, Natalie Parde
TL;DR
Spoken-language ADRD detection is promising but cross-corpus generalization and model interpretability remain challenging. This paper analyzes DAPF and transcript-based baselines, finding that diagnosis information is concentrated in DAPF’s [MASK] representation even though its token-level attributions are not consistently faithful.
Problem
Cross-corpus generalization is difficult because datasets vary in size, demographics, and elicitation tasks, while high corpus-specific accuracy may reflect linguistic or task-specific patterns rather than ADRD-relevant information.
Method
The study interprets BERT-based DAPF for ADRD detection alongside transcript-only and prompt-as-input baselines using classification, probing, attribution, and perturbation analyses.
Results
DAPF achieves higher accuracy and macro-F1 than control baselines, with diagnosis more recoverable from its final-layer [MASK] representation than from its [CLS] representation or baseline [CLS] representations.
Takeaways & Limitations
Representation-level diagnosis encoding can coexist with unfaithful token-level explanations, so interpretability should be evaluated alongside performance in low-resource prompt-based models.
Takeaways & Limitations
All experiments use a single low-resource CCC → ADReSS transfer setting, and larger perturbations may conflate token importance with sensitivity to distribution shift.
Abstract
from arXiv · showhide
Spoken-language analysis via prompt-based domain-adaptive models is a promising direction for low-resource, non-invasive dementia screening, but such models remain internally opaque. We study the interpretability of the Domain-Adapted models via Prompt-based Fine-tuning (DAPF) framework, which casts dementia detection as diagnosis-related masked-token prediction. We interpret DAPF and strong baselines using a variety of probing and analysis techniques, finding that DAPF achieved the best overall performance (accuracy=0.83 and macro-F1=0.83) with diagnosis most recoverable from its [MASK] representation. However, this representational advantage did not extend to token-level explanation faithfulness. DAPF attributions primarily reflected language task vocabulary, discourse markers, and transcription artifacts, with perturbation tests showing weak or negative effects. This suggests that its masked-token interface determines diagnosis information without producing faithful token-level explanations.
1 Introduction
Spoken language offers a low-cost, non-invasive medium for ADRD detection, but cross-corpus performance and interpretability remain challenging. This paper studies whether DAPF’s masked-token interface encodes diagnosis information faithfully at the representation and token levels.
- Motivation: Spoken dialogue enables low-cost, non-invasive ADRD detection without neuroimaging, cerebrospinal fluid, plasma, or blood-based assays.Language samples can be elicited through picture description, story recall, or conversational interviews.
- Research gap: Cross-corpus generalization is difficult because datasets differ in size, demographics, and elicitation tasks.High accuracy on one corpus does not establish that a model learned ADRD-relevant linguistic patterns.
- Approach: DAPF frames cross-domain detection as cloze-style masked-token prediction using domain information, diagnosis-related label words, and a verbalizer.The model predicts label words at a [MASK] position and maps them to class labels.
- Approach: Interpretability experiments combine representation probing, Integrated Gradients, attribution aggregation, deletion, insertion, and paired error analysis.These analyses examine recoverable diagnosis information, token contributions, faithfulness, and model decisions.
- Findings: DAPF outperformed control baselines and concentrated more diagnosis information in its final-layer [MASK] representation than in compared [CLS] representations.Its highlighted tokens nevertheless did not consistently support the DEMENTIA prediction under perturbation.
2 Related Work
Prior work establishes spoken-language ADRD detection and prompt-based adaptation as active research areas, while highlighting risks from corpus-specific cues and opaque explanations. This paper focuses on whether DAPF’s diagnosis representations and token attributions are reliable.
- Spoken-language ADRD detection: Speech-based ADRD detection commonly uses DementiaBank and its Pitt Corpus, with ADReSS variants extending standardized or speech-only evaluation.CCC provides another dataset based on longer health-oriented conversational interviews.
- Prompt-based adaptation: Prompt learning adapts pretrained language models through manual, automatic, or continuous prompts, including domain-specific prompts.DAPF applies prompt-based masked-token prediction to spoken-language ADRD detection.
- Research focus: The paper studies how a prompt-based ADRD detector represents diagnosis information and whether predictions can be interpreted reliably.Its focus differs from prior emphasis on clinical plausibility or explanation usefulness.
- Interpretability methods: Representation probing tests recoverable information in hidden states using simple auxiliary classifiers and control tasks.The paper uses logistic regression and a random-label control to limit probe-capacity concerns.
- Interpretability risks: Token attribution methods can highlight dataset-specific cues, preprocessing artifacts, or clinically irrelevant tokens rather than faithful evidence.Deletion and insertion tests assess whether highlighted tokens affect predictions as expected.
3 Task and Model Background
The study evaluates DAPF against two BERT baselines for binary ADRD detection in a CCC-to-ADReSS transfer setting. DAPF uses domain and class prompts to predict diagnosis-related label words at [MASK].
- Task: The task is binary ADRD detection from spoken-language transcripts using DAPF, BERT-CLS, and BERT-CLS+Prompt.The models are evaluated with CCC as source domain and ADReSS as target domain.
- Task: The CCC → ADReSS setting uses the ADReSS train split for target-domain adaptation and the held-out ADReSS test split for evaluation.Both datasets are small, making token-level explanations sensitive to participants, repeated task vocabulary, and initialization.
- DAPF: DAPF introduces domain-specific and class-specific prompt text and predicts masked label words mapped to classes by a verbalizer.The design aims to align representations with the task and source-target domain differences.
- DAPF: The DAPF prompt places a transcript and domain placeholder before a diagnosis [MASK] position.The masked position is the interface through which diagnosis-related label prediction is made.
- Baselines: BERT-CLS predicts from [CLS], while BERT-CLS+Prompt retains the DAPF prompt but uses a [CLS] classification head without masked label prediction or a verbalizer.The controls separate conventional classification, prompt text, and masked-token prediction effects.
4 Interpretability Methodology
The methodology tests where diagnosis information is encoded and whether token attributions reflect the model’s decisions. It combines probing, attribution aggregation, and perturbation-based faithfulness diagnostics under controlled model comparisons.
- Overview: The study combines representation probing, token- and word-level attributions, perturbation diagnostics, and paired error analysis.These analyses target representation content, token contributions, faithfulness, and output differences.
- Representation probing: Representation probing trains logistic regression on frozen hidden states with stratified five-fold cross-validation and macro-F1 scoring.DAPF is probed at [CLS] and [MASK], while the baselines are probed at the positions they contain; layer-wise probing tracks recoverability across layers.
- Attribution: Integrated Gradients estimates each token’s contribution to P(DEMENTIA), using masked-token verbalizer scores for DAPF and [CLS] classification outputs for baselines.Positive scores support DEMENTIA, while negative scores oppose it.
- Attribution: Attributions are aggregated by normalized token type using mean signed and mean absolute scores to summarize direction and magnitude.The analysis applies shared preprocessing and excludes rare tokens using a stricter frequency threshold.
- Controlled comparison: All models share the same backbone, data split, training configuration, class weighting, and random seeds, while their input and prediction interfaces differ.Evaluation includes balanced accuracy, macro F1, AUROC, and ECE.
- Faithfulness: Deletion masks highly ranked tokens and insertion restores them to test whether predictions weaken and recover as expected.Positive and absolute attribution rankings are evaluated at the token-occurrence level.
5 Experimental Setup
The controlled comparison fixes the backbone, data split, training configuration, class weighting, and random seeds while evaluating multiple performance and calibration metrics.
- The comparison holds the backbone, data split, training configuration, class weighting, and random seeds constant across models.
- Models are evaluated using balanced accuracy, macro F1, AUROC, and Expected Calibration Error.Lower ECE indicates better calibration.
6 Results
DAPF performs best on accuracy and macro-F1, with diagnosis information especially recoverable from its supervised [MASK] representation. However, its token attributions emphasize task and transcription cues and show weak faithfulness under perturbation tests.
- DAPF achieves the highest accuracy and macro-F1 among the baselines, while BERT-CLS achieves the highest AUROC.ECE is similar across all models.
- DAPF is uniquely correct in 20 cases, whereas BERT-CLS and BERT-CLS+Prompt are uniquely correct in three and two cases, respectively.The classifier baselines are both correct in 17 cases where DAPF is wrong, indicating a modest but distinct decision-making advantage for DAPF.
- DAPF’s final-layer [MASK] representation is more diagnostic than its [CLS] representation, with a mean paired macro-F1 improvement of 0.044.The bootstrap 95% CI is [0.034, 0.056], and the one-sided exact Wilcoxon signed-rank test gives p=0.031.
- Random-label controls reduce probe macro-F1 from 0.755 to 0.474 for [CLS] and from 0.799 to 0.470 for [MASK].This indicates that the probes recover diagnosis-related rather than arbitrary structure.
- DAPF’s [MASK] representation becomes informative in middle layers and peaks in the final layer, unlike the baselines’ different layer-wise patterns.BERT-CLS [CLS] peaks around layer 5, while the auxiliary BERT-CLS+Prompt [MASK] is strongest around layers 8–10.
- DAPF attributions primarily highlight task vocabulary, discourse markers, fillers, and transcription artifacts rather than clearly interpretable ADRD-related features.Among 20 tokens compared across Integrated Gradients and Partition SHAP, 11 (55%) had the same mean direction, with Spearman ρ = 0.362.
- Layer 4–5 probing should be interpreted cautiously because it may encode lexical or task-related group differences without revealing what drives the separation.Future work could probe nuisance variables such as transcript length or task-related content-word counts.
- DAPF shows weak deletion effects and consistently negative insertion effects, despite having the strongest final-layer diagnosis representation.BERT-CLS and BERT-CLS+Prompt show positive deletion and insertion effects, making prompt text alone an unlikely explanation for DAPF’s weak insertion faithfulness.
7 Discussion
DAPF encodes diagnosis information strongly at its supervised [MASK] position, but this representation-level advantage does not yield faithful token-level explanations. Its attribution patterns are shaped by prompt-related and transcription cues, motivating broader interpretability analyses beyond individual tokens.
- DAPF’s final-layer [MASK] representation is more diagnosis-informative than its [CLS] representation and baseline representations.This supports concentration of diagnosis information at the supervised prompt prediction position.
- BERT-CLS positive attributions respond more under insertion than DAPF’s, revealing divergence between diagnosis encoding and token-level faithfulness.The comparison links strong representation-level diagnosis information with weaker perturbation behavior.
- DAPF’s prompt-based prediction combines transcript, prompt wording, domain information, and verbalizer words, complicating word-level attribution.These interactions may not be localized to individual words.
- BERT-CLS+Prompt behaves more like BERT-CLS than DAPF, indicating prompt wording alone does not explain DAPF’s representation or faithfulness patterns.
- Future analyses should examine broader transcript patterns and clinically meaningful feature changes rather than relying only on high-attribution tokens.Suggested directions include informativeness, lexical specificity, discourse organization, and concept-based methods.
8 Conclusion
The paper evaluates whether DAPF’s diagnosis representations and token attributions are interpretable in cross-domain ADRD detection. DAPF concentrates diagnosis information at [MASK], but its token explanations are less faithful under perturbation, cautioning against equating performance with explanation reliability.
- DAPF achieves slightly higher accuracy and unique correct decisions while concentrating diagnosis information at its supervised [MASK] representation.
- High-attribution tokens primarily reflect task-specific vocabulary, discourse markers, and transcription artifacts rather than clinically meaningful ADRD markers.
- Partition SHAP recovers broad cue categories but only partially agrees with Integrated Gradients on individual token direction and ranking.
- DAPF token attributions are less faithful under deletion and insertion than classifier baselines.
Limitations
The study’s interpretability claims are bounded by one low-resource transfer setting and by perturbation sensitivity to distribution shift. Strong diagnosis information in DAPF’s [MASK] representation also does not establish faithful word-level explanations.
- All experiments use a single low-resource CCC → ADReSS transfer setting, limiting generalization to speech-based ADRD detection.
- Large transcript masking may create out-of-distribution inputs, so effects at larger perturbation fractions may reflect distribution-shift sensitivity as well as token importance.
- Testing one DAPF-style prompt leaves open whether other templates, domain descriptions, verbalizers, or prompt-tuning methods produce different patterns.
- DAPF’s strong [MASK] diagnosis information does not necessarily make its token attributions faithful word-level evidence.
Ethical Considerations
The study uses ethically approved access procedures for CCC and ADReSS and frames the models as research tools rather than clinical diagnostic systems. Its interpretability analyses distinguish diagnosis information recoverability from faithful token-level evidence.
- Ethics and research use: CCC access required institutional ethics review at both participating institutions, while ADReSS access required DementiaBank approval and an additional institutional ethics determination.
- Ethics and research use: Token attributions reflecting task-specific vocabulary or transcription artifacts may mislead readers if interpreted as clinical evidence.
- Interpretability methods: Deletion masks selected high-attribution tokens, whereas insertion restores them after masking to test whether predictions weaken and recover as expected.
- Interpretability methods: Representation probing tests whether diagnosis is recoverable from frozen hidden states, while Integrated Gradients assigns token-level contributions to P(DEMENTIA).
- Interpretability methods: The 50% perturbation condition is a stress test for stronger perturbation, not a precise estimate of token importance.
- Interpretability methods: Layer-wise probing compares DAPF with transcript-only BERT-CLS and prompt-as-input BERT-CLS+Prompt using macro-F1 averaged across five seeds.
H Partition SHAP Sensitivity Analysis
The Partition SHAP sensitivity analysis finds broad agreement with Integrated Gradients about cue categories but only moderate agreement on individual token importance. This supports stable qualitative patterns while limiting confidence in precise word-level explanations.
- Sensitivity setup: Partition SHAP was applied to one DAPF seed and 48 ADReSS test transcripts to assess whether attribution patterns were qualitatively consistent across methods.
- Attribution agreement: Only moderate agreement on individual token importance indicates that broader qualitative interpretation is more stable than precise word-level importance.
- Token-level instability: Partition SHAP rankings included Cookie Theft words, discourse forms, and tokenization fragments, with semantically related forms sometimes receiving opposite attribution directions.
- Broad attribution patterns: Partition SHAP and Integrated Gradients both highlighted picture-description vocabulary, discourse markers, and transcription artifacts.
- Faithfulness comparison: DAPF showed negative deletion and insertion effects overall when tokens were ranked by absolute attribution magnitude.
- Faithfulness comparison: Absolute-ranking faithfulness results should not be interpreted as target-specific rationales.