PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Rank Aggregation for Multimodal Speech-Based Alzheimer's Disease Detection

Zemin Jin, Tomoko Matsui

arXiv:2609.09948v1cs.SD

TL;DR

Multimodal speech classifiers can produce posterior probabilities on mismatched scales, making direct probability averaging unreliable. The paper aggregates normalized prediction ranks and adds a confidence-gated Random Forest correction; on both ADReSS benchmarks, the framework outperforms probability averaging and achieves competitive accuracy.

  • Problem

    Posterior probabilities from classifiers trained on different representations or regularization strengths may not be numerically comparable, complicating conventional probability averaging.

    Source: §1 · p. 1

  • Method

    The framework aggregates normalized prediction ranks and uses a confidence-gated Random Forest with handcrafted linguistic features to selectively correct residual errors.

    Source: §1 · p. 1, §4 · p. 4

  • Results

    The framework consistently outperforms probability averaging and achieves competitive accuracy on ADReSS2020 and ADReSSo2021.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The authors conclude that rank aggregation offers an alternative to probability averaging when heterogeneous classifiers have incomparable output scales.

    Source: §4 · p. 4

Abstract

from arXiv · show

Speech-based Alzheimer's disease (AD) detection has recently benefited from multimodal foundation-model representations that integrate complementary acoustic and linguistic information. However, conventional probability averaging over these complementary classifiers is unreliable, because their posterior probabilities exhibit mismatched scales: identical values may reflect different confidence levels across models. We propose a robust rank aggregation framework that aggregates normalized prediction ranks instead of posterior probabilities. Each subject is scored by its percentile within a fixed training-cohort distribution of out-of-fold predictions; since rank ordering is invariant to monotonic transformations, this avoids probability-scale mismatch while preserving classifier confidence ordering. A confidence-gated Random Forest further corrects residual errors using clinically interpretable linguistic features, overriding the rank prediction only when the two disagree and the RF is highly confident, without additional deep model training or explicit posterior-probability calibration. On ADReSS2020 and ADReSSo2021, the method achieves accuracies of 95.83% and 90.14%, respectively, comparing favorably with previously reported results.

1. INTRODUCTION

The paper addresses posterior probability-scale mismatch in multimodal speech classifiers by aggregating normalized prediction ranks and selectively correcting residual errors with a confidence-gated Random Forest. On ADReSS2020 and ADReSSo2021, the framework outperforms probability averaging and achieves competitive accuracy.

1. INTRODUCTION

  • Different classifiers’ posterior probabilities may not be numerically comparable, so identical values can reflect different confidence levels.Averaging them assumes their probability scales are comparable.Source: §1 · p. 1
  • The proposed rank aggregation addresses this mismatch by aggregating normalized prediction ranks instead of posterior probabilities.Source: §1 · p. 1
  • A confidence-gated Random Forest selectively corrects highly confident residual errors with handcrafted linguistic features.The correction is interpretable and identifies which speech dimensions drive the final decision.Source: §1 · p. 1
  • The framework consistently outperforms probability averaging and achieves competitive accuracy on ADReSS2020 and ADReSSo2021.Source: §1 · p. 1

2. PROPOSED METHOD

The method processes speech and its transcript through representation extraction, rank aggregation, and confidence-gated correction. Its acoustic, semantic, and handcrafted linguistic representations are processed by classifiers before aggregation and selective correction.

2. PROPOSED METHOD

  • The framework processes speech and its transcript through representation extraction and classifier outputs, rank aggregation, and confidence-gated correction.Source: §2 · p. 2
  • Robust rank aggregation mitigates mismatched posterior-probability scales while preserving classifier confidence ordering, and a confidence-gated Random Forest selectively corrects highly confident residual errors.Source: §2 · p. 2

2.1. Representation and Classifier Outputs

  • The three representations are acoustic, semantic, and handcrafted linguistic features extracted from the recording and transcript.Source: §2.1 · p. 2

Audio Representation

The audio branch encodes the waveform with frozen HuBERT-large and forms an utterance-level embedding from frame-level representations.

Audio Representation

  • Frozen HuBERT-large encodes the waveform into frame-level representations.Source: Audio Representation · p. 2
  • The HuBERT representation captures prosody, hesitation, and disfluency as non-lexical cues that complement the text channel.Source: Audio Representation · p. 2
  • The utterance-level audio embedding concatenates the temporal mean and standard deviation of the frame representations.Source: Audio Representation · p. 2

Text Representation

The text branch uses frozen RoBERTa-base to encode the transcript as a sentence-level embedding.

Text Representation

Handcrafted Linguistic Features

The framework extracts 30 handcrafted acoustic and linguistic features spanning pauses, timing, content, and lexical information. Mutual information selects the top-K features for the correction stage.

Handcrafted Linguistic Features

  • The 30-feature pool covers pause statistics, utterance timing, speech rate, and disfluency markers.These capture pause duration and counts, utterance duration, words per utterance, repetitions, and a disfluency-pause interaction.Source: Handcrafted Linguistic Features · p. 2
  • Content features measure mentions and ratios of people, objects, and actions, alongside overall content density.The feature list also includes people ratio, objects per utterance, and action-to-object ratio.Source: Handcrafted Linguistic Features · p. 2
  • Lexical features include type-token-ratio variants, unique-word counts, and word-length statistics; mutual information selects the top-K features.The selected vector is used in the correction stage.Source: Handcrafted Linguistic Features · p. 2

Classifier Outputs

The section describes classifier outputs whose probability scales vary, then defines rank aggregation and a confidence-gated Random Forest correction. The RF gate is tuned using training-side out-of-fold predictions.

Classifier Outputs

  • Changing HuBERT’s regularization strength C alters posterior-probability spread while largely preserving ordering; standard deviations range from 0.11 to 0.47 across 48 ADReSS2020 samples.The audio and text embeddings are each processed by logistic-regression classifiers trained with different L2 regularization strengths.Source: Classifier Outputs · p. 2, Figure 2 · p. 3

2.2. Rank Aggregation

  • Each test probability becomes its classifier-specific percentile in the training OOF distribution; averaging these ranks yields a score thresholded at 0.5.The training reference scores are generated with leave-one-out cross-validation.Source: §2.2 · pp. 2–3

2.3. Confidence-Gated RF Correction

  • A Random Forest trained on handcrafted features overrides the rank prediction only on a disagreement with confidence above τ.The rule limits corrections to high-confidence disagreements to prevent unreliable corrections from degrading the rank ensemble.Source: §2.3 · p. 3
  • The gate threshold τ is selected by maximizing training OOF accuracy over candidate values, with ties resolved in favor of the larger τ.The candidates are evaluated on training OOF pseudo-ranks.Source: §2.3 · p. 3

3. EXPERIMENTS

The framework achieved the best accuracy among compared methods on both benchmarks. Ablations found contributions from both modalities, rank aggregation, and RF correction, while selected features were dominated by pause and content categories.

3.1. Experimental Setup

  • Experiments used frozen HuBERT-large and RoBERTa-base with LOOCV, selecting 7/5 LR arms and K=12/5 RF features for ADReSS2020/ADReSSo2021.The LR arms were selected from 66 classifiers; RF feature counts were selected by maximizing RF LOOCV OOF accuracy.Source: §3.1 · pp. 3–4

3.2. Comparison with Existing Methods

  • The proposed method attains the best accuracy among compared methods on both ADReSS2020 and ADReSSo2021.The reported comparison is in Table 1.Source: §3.2 · p. 4

3.3. Ablation Study

  • On ADReSSo2021, removing RoBERTa/HuBERT drops accuracy 7.04/5.63, probability averaging instead of rank aggregation drops 4.22, and removing RF correction costs 1.41.The reported ablations indicate the largest drop when the text branch is removed, with a smaller but consistent contribution from RF correction.Source: §3.3 · p. 4

3.4. Feature Analysis

  • Pause and content features rank highest on both datasets, complemented by disfluency and lexical features, despite different selected-feature counts.Figure 4 shows the top-12 features for ADReSS2020 and top-5 for ADReSSo2021, with colors denoting feature categories.Source: §3.4 · p. 4, Figure 4 · p. 4

4. CONCLUSION

The paper proposes rank aggregation to address posterior probability-scale mismatch, with a confidence-gated Random Forest for selective correction. Its ablations support contributions from both modalities and rank aggregation, while the framework is presented as applicable to other multimodal classification settings with incomparable posterior scales.

4. CONCLUSION

  • Normalized prediction ranks replace posterior probabilities to handle scale mismatch, while a confidence-gated RF uses handcrafted linguistic features to correct sufficiently confident disagreements.The RF correction is selective: it activates only when its prediction is sufficiently confident and disagrees with the primary ensemble.Source: §4 · p. 4
  • Ablations support contributions from both text and audio modalities and rank aggregation; selected handcrafted features concentrate in pause and content categories.The feature selection outcome provides an interpretable correction signal for the final prediction.Source: §4 · p. 4
  • The framework is proposed for multimodal classification beyond Alzheimer’s detection when heterogeneous classifiers produce incomparable posterior probability scales.The authors describe it as lightweight, using frozen feature extractors and small classifiers, and plan evaluation on larger cohorts, additional languages, and other classifier families.Source: §4 · p. 4
Loading 2609.09948v1…