PaperlayerView exact revision on arXiv

Source-linked AI summary

DiaWhisper-DPO: Role-Attributed Transcription of Clinical Interviews via Failure-Mined Preference Optimization

Weiming Li, Ana Catarina Fidalgo Barata, Miguel Constante, João Miguel Sanches

arXiv:2609.16661v1cs.CL

TL;DR

Clinical interview screening requires knowing who said what, but cascaded diarization and role-assignment stages can propagate errors. The paper fine-tunes Whisper for joint transcription and role attribution, then refines it with genuine decoding failures as DPO negatives. DiaWhisper-DPO leads the reported baselines on both datasets, and its DAIC-WOZ DER is 0.119 versus 0.431 for the strongest baseline.

  • Problem

    Clinical interview screening requires correctly attributing utterances because patient signals and their clinical relevance depend on who spoke.

    Source: §1 · p. 1

  • Method

    DiaWhisper fine-tunes Whisper for role-tagged transcription with an auxiliary role head, and DiaWhisper-DPO applies DPO using genuine decoding failures as rejected completions.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    DiaWhisper-DPO leads baselines in role accuracy, DER, and AER on both datasets; its DAIC-WOZ DER is 0.119 versus 0.431 for the strongest baseline, and PDCH-HAMD DER is 0.313 versus Pyannote's 0.574.

    Source: §3.2 · pp. 3–4

  • Takeaways & Limitations

    The results support failure-mined preference optimization for structured transcription in the evaluated English and Chinese clinical-interview settings.

    Source: §4 · p. 4

    The real-versus-synthetic check covers only the DAIC-WOZ patient side because real interviewer audio is unavailable, and outputs still require human verification.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Automated depression screening from clinical interviews requires attribution of utterances to the clinician or patient. We evaluate two datasets: DAIC-WOZ, where participant-only recordings require re-synthesizing both sides for controlled two-party evaluation, and PDCH-HAMD, comprising voice-converted real Chinese interviews for cross-lingual validation. Cascaded systems combine speaker diarization with role-assignment heuristics, so errors can propagate across stages. We propose an end-to-end model, which we named DiaWhisper, that fine-tunes Whisper-large-v3 with LoRA and an auxiliary frame-level role head for transcription and attribution, together with DiaWhisper-DPO, a failure-mined refinement that uses genuine decoding failures as DPO rejected completions without human preference annotation. On 29 DAIC-WOZ test sessions, DiaWhisper-DPO achieves 0.973 role accuracy and 0.119 DER, 72% below the strongest cascaded baseline, and reduces seed variation from {\sigma} = .205 to .002. Retrained on PDCH-HAMD, it achieves 0.757 role accuracy and improves all 78 session-seed pairs.

1. INTRODUCTION

Clinical interview analysis depends on correctly attributing utterances because speaker identity changes the relevance of screening signals. The paper introduces a two-stage role-tagged transcription approach and evaluates it across English and Chinese clinical-interview corpora.

1. INTRODUCTION

  • Correct attribution matters because patient word choice, hesitation, response latency, and turn-taking can indicate depressive symptoms only when assigned to the right speaker.The preceding interviewer question can also shape the signals' interpretation, while misattributed turns distort downstream screening signals.Source: §1 · p. 1
  • The study evaluates DAIC-WOZ, which requires two-party re-synthesis from participant-only recordings, and voice-converted real Chinese PDCH-HAMD interviews.Traditional diarization followed by heuristic role assignment can propagate errors between stages.Source: §1 · p. 1
  • On DAIC-WOZ, DiaWhisper-DPO improves seed stability from σ=.205 to .002 and reduces DER by 72% relative to the strongest cascaded baseline.When independently trained and evaluated on voice-converted PDCH-HAMD, it improves role attribution in all 78 matched session-seed cases.Source: §1 · p. 1
  • DiaWhisper-DPO refines DiaWhisper in a second training stage, and both models use the same inference procedure.The pipeline figure depicts Stage 1 producing DiaWhisper and Stage 2 refining it into DiaWhisper-DPO.Source: Figure 1 · p. 2

2. METHOD

The method constructs two-party clinical-interview data, fine-tunes Whisper for joint transcription and role attribution, then refines it using genuine training failures. DiaWhisper and DiaWhisper-DPO share an inference procedure.

2.1. Dataset Construction

  • DAIC-WOZ dialogues are reconstructed from participant-only recordings by synthesizing both roles, while voice-converted PDCH data use patient-level splits to prevent speaker identity crossing splits.DAIC-WOZ uses four cross-combined VCTK voice variants per session, with one selected deterministically.Source: §2.1 · p. 2

2.2. Architecture

  • DiaWhisper fine-tunes Whisper-large-v3 with LoRA and role-tagged timestamp–text targets, alongside a training-only frame-level role head.The auxiliary head predicts Doctor, Patient, Interject, or None and is discarded at inference; frame confidence weights down-weight ambiguous-frame gradients.Source: §2.2 · pp. 2–3

2.3. Training

  • Stage 1 jointly optimizes serialized-target cross-entropy and an auxiliary diarization loss, then Stage 2 applies DPO using ground truth as chosen completions and genuine model failures as rejected completions.About 500 training windows per seed are mined using the 8-token repetition criterion; only a newly initialized policy LoRA adapter is optimized in Stage 2.Source: §2.3 · p. 3

2.4. Inference and Decoding

  • At inference, audio is decoded in chunks with retries after repetition cutoffs, and the resulting role, time, and text triplets are concatenated per session.Both checkpoints share this procedure; retries resume with sampling rather than greedy decoding.Source: §2.4 · p. 3

3. EXPERIMENTAL RESULTS AND ANALYSIS

DiaWhisper-DPO leads cascaded baselines on both datasets, with lower DER and stronger attribution results. Stability analyses show gains across DAIC-WOZ and PDCH-HAMD, while the remaining error patterns differ by dataset.

3.1. Experimental Setup

  • DAIC-WOZ uses 132/28/29 train/dev/test sessions and synthesized two-party speech; PDCH-HAMD uses a patient-wise 51/11/11 split, with matched real-versus-XTTS patient WER of 0.136 vs. 0.106 across 189 sessions.PDCH-HAMD provides independent validation on voice-converted real Chinese interviews.Source: §3.1 · p. 3

3.2. Main Results

  • 21.5 points on DAIC-WOZ and 17.1 on PDCH-HAMD separate DiaWhisper-DPO from the best oracle-mapped baseline, while oracle mapping changes baseline accuracy by only 0–7 points.The authors interpret this as evidence that baseline errors are mainly diarization errors rather than role-assignment errors.Source: §3.2 · p. 4
  • 0.119 DER on DAIC-WOZ and 0.313 DER on PDCH-HAMD lead the strongest baselines, whose respective DERs are 0.431 and 0.574.DiaWhisper-DPO also leads all baselines in role accuracy and AER.Source: §3.2 · pp. 3–4, Table 1 · p. 4

3.3. Seed Stability and Error Analysis

  • DPO reduces DAIC-WOZ seed variation from σ=.205 to .002; synthetic corruptions reach 0.910 role accuracy but have 18× the seed variability (σ=.037 vs. .002).Ablations also show that removing retry barely changes DiaWhisper-DPO role accuracy (0.973 to 0.971), while replacing DPO with SFT reaches 0.684.Source: §3.3 · p. 4, Table 2 · p. 4
  • Across all 87 matched DAIC-WOZ session-seed evaluations, role accuracy improves +0.220 to +0.442; on 2694 chunks, uncovered speech falls from 33.6% to 3.2%.Matched-turn role accuracy rises from 95.4% to 98.3%, with non-overlapping 95% confidence intervals.Source: §3.3 · p. 4
  • On PDCH-HAMD, seed variation falls from σ=.052 to .005 and all 78 matched evaluations improve (mean +0.188 role accuracy); residual error is 0.2% missing versus 25.8% wrong-role.Matched-turn role accuracy rises from 62.3% to 75.9%, with non-overlapping 95% confidence intervals.Source: §3.3 · p. 4
  • Real-versus-synthetic validation covers only the DAIC-WOZ patient side because real interviewer audio is unavailable; the output still requires human verification.The authors note that incorrect role attribution could distort depression-screening signals.Source: §3.3 · p. 4

4. CONCLUSION

DiaWhisper-DPO combines LoRA fine-tuning of Whisper-large-v3 with an auxiliary role head and failure-mined DPO to transcribe and attribute speech jointly. The paper reports that it outperforms cascaded baselines on both datasets and reduces seed variability.

4. CONCLUSION

  • DiaWhisper-DPO applies DPO to genuine decoding failures after fine-tuning Whisper-large-v3 with LoRA and an auxiliary role head.It jointly transcribes and attributes roles without separate diarization or human preference annotation.Source: §4 · p. 4
  • Joint transcription and role attribution are performed without separate diarization or human preference annotation.Source: §4 · p. 4
  • Across independent training on DAIC-WOZ and PDCH, DiaWhisper-DPO outperforms cascaded baselines and greatly reduces seed variability.Source: §4 · p. 4

5. COMPLIANCE WITH ETHICAL STANDARDS

The study reports ethics approval and informed consent for the PDCH collection and for the original DAIC-WOZ collection. It used the existing de-identified DAIC-WOZ dataset under its data-use terms.

5. COMPLIANCE WITH ETHICAL STANDARDS

  • PDCH was collected at Beijing Anding Hospital with ethics approval and written informed consent from all participants.Source: §5 · p. 5
  • DAIC-WOZ was originally collected with USC IRB approval and informed consent.Source: §5 · p. 5
  • This study used the existing de-identified DAIC-WOZ dataset under its data-use terms.Source: §5 · p. 5

Related papers

Loading 2609.16661v1…