ADReasoner: Acoustic-Linguistic Cue-Aware Reinforcement Learning for Explainable Alzheimer's Disease Detection from Speech

Motivation

• Speech and language impairments are prominent early AD signs, making their analysis a non-invasive, scalable detection approach.Traditional diagnostics such as lumbar puncture are described as invasive, costly, and resource-intensive.Source: §1 · p. 1

Problem

• Conventional speech-based AD models typically output binary labels without explaining the evidence behind their predictions.Existing SpeechLLM approaches largely retain categorical classification; SFT-generated rationales may remain surface-level and fail to connect fine-grained cues to clinical significance.Source: §1 · p. 1

Challenges

• The paper identifies data scarcity, missing reasoning annotations, weak cue perception, and insufficient rule-based RL rewards as four challenges for explainable AD speech detection.The reward limitation can leave reasoning paths suboptimal and misalign generated reasoning with predicted labels.Source: §1 · p. 2

Contributions

• ADReasoner combines ADCoT-7K, cue-perception-enhanced SFT, and stabilized four-reward GRPO to produce cue-grounded rationales; it outperforms prior state-of-the-art models in detection and explanation quality across three datasets.Its reasoning and reasoning-label consistency rewards are paired with shrinkage and progressive gating.Source: §1 · p. 2

2.1 Speech-Based AD Detection

• Speech-based AD methods progressed from handcrafted features to pretrained speech models and SpeechLLMs, yet most still produce binary labels or surface-level explanations without fine-grained cue grounding.The reviewed SFT explanation approaches do not autonomously explore reasoning paths or ground explanations in detailed acoustic and linguistic cues.Source: §2.1 · p. 2

2.2 RL for Speech Reasoning

• General-purpose RL speech-reasoning methods often leave intermediate reasoning unconstrained, while AD diagnosis depends on precise integration of specialized acoustic-linguistic cues.The passage motivates mechanisms that supervise intermediate reasoning and ensure these cues are captured accurately and comprehensively.Source: §2.2 · p. 2

Formulation

• Given participant audio and a task query, ADReasoner generates an interpretable rationale followed by an AD-or-HC diagnosis in a structured output format.The output is represented as <think>... </think><answer>AD|HC</answer>.Source: §3 · p. 2

Pipeline

• The four-stage pipeline comprises controlled text-speech data synthesis, ADCoT-7K construction, cue-centric two-stage SFT, and stabilized four-reward GRPO optimization.The stages are presented as the method's overall workflow.Source: §3 · p. 2

3.1 Controlled Text-Speech Data Synthesis

• The synthesis pipeline is designed to increase training-data diversity and alleviate the limited scale of AD speech data.It uses controlled text generation, speech synthesis, and text/speech quality filtering.Source: §3.1 · p. 2, Figure 2 · p. 3

Text Generation

• Four fine-tuned LLMs generate synthetic descriptions using scene, diagnosis, demographic, and linguistic-cue information as inputs.Each model produces descriptions at 10× the size of the real training set; cue profiles are mapped to ordinal levels and generated texts undergo multi-stage filtering before speech synthesis.Source: Text Generation · p. 3

Speech Synthesis

• Fine-tuned CosyVoice2 synthesizes speech from filtered text using label-matched reference speech and label-conditioned instructions, yielding 6,545 filtered text-speech pairs.Speech filtering includes duration checks, ASR–text agreement evaluation, and binary-classifier filtering.Source: Speech Synthesis · p. 3

3.2 ADCoT-7K Construction

• Existing AD speech datasets contain discrete diagnostic labels but lack supervision for structured reasoning.Source: §3.2 · pp. 3–4
• ADCoT-7K contains approximately 7,000 speech-reasoning pairs spanning about 110 hours of audio.It is described as the first richly annotated AD speech reasoning dataset.Source: §3.2 · p. 4
• The annotation pipeline extracts six acoustic and six linguistic cues, discretizes them into five ordinal levels, and uses them with labels and participant details to prompt CoT generation.Generated traces undergo format, length, and reasoning-label consistency checks before inclusion.Source: §3.2 · p. 4

3.3 Cue-Centric Two-Stage SFT

• Preliminary experiments found that Qwen2.5-Omni-7B lacked fine-grained perception of AD-related acoustic-linguistic cues, motivating two-stage SFT before RL.The paper notes that cue perception is a prerequisite for reliable AD speech reasoning.Source: §3.3 · p. 4

Stage I: Multi-Task SFT

• Stage I uses 17 tasks spanning cue-level prediction, joint cue and diagnosis prediction, ASR, and audio-only or text-only classification.The mixed corpus contains approximately 120K samples and 1,700 hours of audio, and is used to fine-tune the audio encoder, audio adapter, and LLM backbone.Source: Stage I: Multi-Task SFT · p. 4

Stage II: CoT Cold-Start SFT

• Stage II applies LoRA fine-tuning to the LLM backbone on a randomly sampled 1K ADCoT-7K subset to teach basic <think>-<answer> reasoning patterns.The audio encoder and audio adapter remain frozen, yielding ADReasoner-Base.Source: Stage II: CoT Cold-Start SFT · p. 4

3.4 Stabilized Four-Reward GRPO Optimization

• GRPO optimization combines format, label, reasoning, and reasoning-label consistency rewards to supervise output structure, diagnosis, and reasoning quality.Source: §3.4 · p. 4

Format Reward

• The format reward checks whether output places reasoning and prediction in the predefined <think> and <answer> tags.It awards 1 for a correctly formatted output and 0 otherwise.Source: Format Reward · p. 4

Label Reward

• The label reward RL supervises whether the predicted label is correct relative to the ground-truth label y⋆, and is defined using the ground-truth label y⋆ and predicted label ˆy.Source: Label Reward · p. 4

Reasoning Reward

• The reasoning reward scores generated rationales using both similarity to a reference rationale and coverage of its paraphrased cues.Cue coverage counts matches between generated cue expressions and the reference’s paraphrased cue dictionary.Source: Reasoning Reward · p. 4

Reasoning-Label Consistency Reward

• The reasoning-label consistency reward assesses whether a generated rationale supports its predicted diagnosis, beyond checking label correctness alone.A BERT-based binary classifier trained on ADCoT-7K rationale-label annotations estimates this support probability.Source: Reasoning-Label Consistency Reward · pp. 4–5

Reasoning Shrinkage Mechanism

• A shrinkage factor adaptively suppresses reasoning-reward signals when incorrect responses in a GRPO group receive higher reasoning rewards than correct responses.The adjusted reasoning reward is the original reward multiplied by the shrinkage factor.Source: Reasoning Shrinkage Mechanism · p. 5

Progressive Gating Mechanism

• Training initially uses only format and label rewards; after validation accuracy reaches threshold δ, reasoning rewards are gradually activated.Both reasoning rewards share the trigger and schedule, can be disabled independently, and use γ warmup steps.Source: Progressive Gating Mechanism · p. 5

Overall Reward

• The overall GRPO reward weights the four reward components, applying gating coefficients to the reasoning rewards and a shrinkage factor to the reasoning reward.Source: Overall Reward · p. 5

Datasets

• Pitt is the in-domain benchmark, while PROCESS-1 and PROCESS-2 are used only for out-of-domain evaluation.Pitt has 552 samples and a participant-independent 7:1:2 train-validation-test split; PROCESS-1 and PROCESS-2 contain 98 and 250 samples, respectively.Source: Datasets · p. 5

Baselines

• Baselines include conventional supervised classifiers, a reasoning-SFT SpeechLLM, and 15 open-source plus two closed-source zero-shot SpeechLLMs.Source: Baselines · p. 5

Evaluation Metrics

• Detection is measured with macro-F1 and accuracy, while reasoning is assessed through format parsing, reasoning, consistency, and LLM-based scores.GPT-5.4 scores four reasoning dimensions from 1 to 5, and human evaluation is also conducted.Source: Evaluation Metrics · p. 5

Implementation Details

• Results are averaged over five independent runs, with test splits reserved for evaluation and training using synthetic data and designated training splits.RL post-training uses 6K ADCoT-7K samples for 4,000 steps on four A800 GPUs.Source: Implementation Details · p. 5

Comparison with Prior Baselines

• 80.72% F1 and 81.10% Acc on Pitt, with ADReasoner also leading reasoning metrics; on OOD data, it achieves 65.65% F1 versus GPT-audio-1.5’s 50.04%.Its Pitt LLM-R is 3.70 versus 3.27 for the strongest baseline, and its OOD accuracy is 74.38% versus GPT-audio-1.5’s 75.97%.Source: Comparison with Prior Baselines · pp. 6–7, Table 1 · p. 6

Human Evaluation

• ADReasoner receives the highest human reasoning-evaluation scores, and model rankings closely align with automatic judgments.Five independent reviewers assessed anonymized outputs from eight models on 20 class-balanced instances per dataset, using the same four-dimensional rubric.Source: Table 2 · p. 6, Human Evaluation · p. 7

Cue Perception Comparison and Effectiveness of Synthetic Data

• Cue-centric SFT improves cue-perception performance over Qwen2.5-Omni-7B, while adding quality-filtered synthetic data yields the best results across tasks.Figure 4 reports accuracy except for ASR, which uses one minus word error rate.Source: Cue Perception Comparison and… · p. 7, Figure 4 · p. 7

Ablation Study

• Reasoning shrinkage raises F1 from 79.38% to 80.28% and LLM-R from 3.52 to 3.65; progressive gating improves F1 by 2.75 points and LLM-R by 0.18 over V3.Adding the RLC reward raises RLC from 86.12% to 90.43%; progressively activating it yields further gains, and the complete ADReasoner performs best overall.Source: Ablation Study · p. 7, Table 3 · p. 7

• Across three AD speech datasets, ADReasoner achieves state-of-the-art performance in both AD detection and reasoning quality.Source: §5 · p. 7
• ADCoT-7K integrates real and quality-filtered synthetic data with fine-grained acoustic-linguistic cues and diagnostic reasoning annotations.Source: §5 · p. 7
• Cue-centric SFT builds ADReasoner-Base with enhanced cue perception, while four-reward GRPO optimizes reasoning quality and reasoning-label consistency using shrinkage and progressive reward gating.Source: §5 · p. 7