Source-linked AI summary
Evidence-Bounded Mental Health Reasoning from Heterogeneous Speech Protocols
Chengyuan Gao, Jiang Wu, Tao Lu, Jiayan Guo, Mingkun Xu, Tianyi Zang, Shangyang Li
TL;DR
Existing omni-modal screening systems can ignore protocol-specific evidence boundaries, while unconstrained reasoning may worsen boundary violations. This paper introduces evidence-bounded benchmarking and EviBound, achieving strong held-out depression screening performance with auditable evidence consistency.
Problem
The central challenge is the absence of explicit evidence-bound control for heterogeneous speech protocols, whose evidence cannot be treated uniformly.
Method
The paper builds a 1,870-package benchmark and proposes EviBound, which routes evidence by protocol, applies permission constraints, and aggregates five acoustic tools conservatively.
Results
0.8658 AUROC and 100% EBCPass with zero boundary violations; EviBound gains +0.0811 AUROC over the strongest direct LMM baseline.
Takeaways & Limitations
Clinical multimodal systems should be evaluated for auditable evidence consistency and protocol-aware reasoning reliability alongside predictive accuracy.
Takeaways & Limitations
The benchmark remains moderate in scale, with anxiety evaluation dominated by interview records and restrictive profiles having limited sample sizes.
Abstract
from arXiv · showhide
Computational mental health screening using multimodal speech and text has shown great promise. However, existing models often assume all clinical speech protocols carry equivalent evidentiary validity. In reality, heterogeneous protocols, from free interviews to fixed reading tasks, support fundamentally different evidence. Forcing uniform reasoning flattens these boundaries, causing models to hallucinate symptoms from irrelevant text or overclaim support. Even advanced long chain-of-thought LLMs fail to resolve this issue, as free-form reasoning can exacerbate boundary violations. To address this, we reformulate multimodal screening as an evidence-bounded reasoning problem. We introduce the Evidence Package Benchmark, integrating 1,870 packages across six heterogeneous sources with explicit modality masks and evidence permissions. We further propose EviBound, a protocol-aware evidence control framework. Unlike direct LLM prompting, EviBound uses a profile-aware planner to restrict reasoning scope, orchestrates evidence tools via five-way acoustic consensus, and enforces a boundary critic to suppress unsupported claims. Empirical results show EviBound achieves a held-out test Depression AUROC of 0.8658, exceeding the strongest direct omni-modal baseline by +0.0811 AUROC while maintaining zero claim violations. Our work moves beyond unconstrained accuracy toward evidence-consistent, protocol-aware systems for safer clinical NLP research.
1 Introduction
The paper frames heterogeneous speech protocols as distinct evidence boundaries rather than interchangeable inputs. It proposes evidence-bounded screening and EviBound to constrain reasoning, route evidence, and validate claims.
- Evidence boundaries: Speech protocols differ in which acoustic, semantic, discourse, and missing-evidence claims they license.Free interviews support acoustic, semantic, and discourse claims; prompted speech restricts semantic inference; fixed reading supports prosody only; text-only discourse supports language-based semantic claims.
- Problem: Ignoring protocol boundaries causes epistemic flattening, unsupported interpretations, and claims attributed to absent modalities.Larger models and longer chain-of-thought reasoning can increase instability, computational cost, and boundary violations.
- Evidence-bounded reasoning: The paper reformulates screening as evidence-bounded reasoning using protocol profiles, modality masks, observations, and admissible evidence boundaries.The objective combines predictive accuracy with consistency between conclusions, observable evidence, and protocol constraints.
- EviBound: EviBound uses a profile-aware planner, parallel evidence modules with five-way acoustic consensus, and a boundary critic.These stages restrict reasoning scope, cross-validate signals, suppress unsupported claims, and enforce modality-mask consistency.
- Results: 0.8658 AUROC and 100% EBCPass are reported for EviBound on held-out packages, with zero boundary violations.The benchmark contains 1,870 standardized evidence packages from six clinical sources with modality masks and claim permissions.
2 Related Work
Prior work spans multimodal mental-health modeling, diverse clinical speech resources, and trustworthy medical AI. This paper distinguishes its contribution by making protocol-aware evidence validity and boundary evaluation central.
- Mental-health screening: Speech-based mental-health research includes interview modeling, multimodal fusion, audiovisual risk assessment, and large multimodal reasoning systems.The cited literature covers several modeling paradigms for psychiatric analysis.
- Clinical resources: Public mental-health resources differ substantially in language, modality availability, and acquisition protocol.The related datasets include DAIC-WOZ/E-DAIC, CMDC, EATD, MODMA, MMPsy, DISCOURSE-UWO, and DAIS-C.
- Clinical evaluation: Clinical AI benchmarks increasingly emphasize reliability and reasoning beyond a single predictive metric.HealthBench, MedHELM, MentalBench, and PsychiatryBench are cited as examples.
- Speech evidence: Recent work converts acoustic landmarks, self-supervised representations, digital phenotypes, and psychological knowledge into LLM-usable evidence.Standard speech pipelines also use openSMILE/eGeMAPS and encoders such as wav2vec 2.0, HuBERT, and WavLM.
- Positioning: Unlike prior work focused mainly on reasoning capability or factual grounding, this paper focuses on protocol-aware evidence validity and boundary-aware metrics.Its evaluation uses CVR, MHR, and EBCPass within an Agent-Evidence Interface.
3 Methodology
The methodology reframes multimodal mental health screening as evidence-bounded reasoning over protocol-specific evidence packages. It combines restricted route planning, auditable evidence extraction, and read-only report validation to keep claims consistent with available modalities and permissions.
- Evidence-Bounded Reasoning: Evidence packages encode protocol profile, modality mask, observed inputs, and admissible evidence boundaries for protocol-consistent screening.The objective includes evidence consistency under heterogeneous acquisition protocols, not only predictive performance.
- Evidence Package Benchmark: The benchmark contains 1,870 packages from six heterogeneous mental health resources, with model-visible inputs separated from evaluator-only labels.Packages may contain audio, transcripts, feature vectors, or text-only discourse, while evaluation tasks remain distinct.
- Protocol-Aware Routing: A deterministic planner selects compatible evidence routes before reasoning, while missing modalities are explicitly registered in the report contract.Interviews may activate lexical, acoustic, and feature routes; prompted or fixed-reading speech cannot use scripted text as symptom history, and text-only records cannot support acoustic claims.
- Evidence Tools and Risk Routing: Audio-supported records use five-way acoustic consensus across openSMILE/eGeMAPS, segmented openSMILE, wav2vec2, HuBERT, and WavLM representations.Each branch produces calibrated risk and uncertainty signals for conservative aggregation under disagreement, with inapplicable routes masked before scoring.
- Boundary Validation: The validator removes or rewrites modality hallucinations, protocol misuse, and unsupported diagnosis, treatment, prognosis, or recommendation claims.Risk scores are frozen before report repair, so validation is evaluated independently from predictive classification performance.
4 Experimental Setup
The experiments compare protocol-compatible baselines and ablations under a shared package interface, evaluating predictive utility separately from evidence consistency on held-out packages.
- Evaluation Protocol: All systems operate on identical package-visible inputs while labels, thresholds, and unavailable modalities remain evaluator-side or masked.Audit procedures verify that evaluator-only information is absent from prompts, cached outputs, and intermediate traces.
- Baselines and Ablations: The study compares direct omni-modal LMMs, long-reasoning prompting, acoustic and feature baselines, and controlled harness ablations.Ablations separately remove planning, acoustic consensus, feature routing, and boundary validation.
- Metrics: Predictive evaluation uses AUROC, validation-threshold F1, and QWK across separate depression, anxiety, and severity-oriented held-out splits.Secondary diagnostics include AUPRC, Brier score, and calibration-oriented operating points.
- Metrics: Evidence consistency is measured independently with CVR, MHR, and EBCPass, which require protocol-valid claims, missing-evidence handling, and schema compliance.The evaluation deliberately avoids collapsing predictive and boundary-aware metrics into one aggregate score.
- Evaluation Protocol: Validation splits fix routing, consensus weights, thresholds, blending ratios, and canonical configurations before held-out testing.Exploratory settings that fail validation remain appendix-only analyses, and deterministic scripts support replayability.
5 Results
EviBound improves held-out depression screening over direct omni-modal baselines and maintains strict evidence consistency, with the largest gains appearing under restrictive protocols. The results also qualify anxiety and acoustic-consensus comparisons because of split composition and uncertainty.
- Main Held-Out Results: 0.8658 pooled depression-eligible AUROC is achieved by EviBound, a +0.0811 gain over Gemini 3.5 Flash and a +0.1942 gain over Qwen3-Omni-Flash.Against Qwen3-Omni-Flash, F1 also increases from 0.5032 to 0.6557.
- Main Held-Out Results: The evaluated long-reasoning baseline is lower than direct Qwen3-Omni-Flash by 0.0499/0.0322 AUROC on depression/anxiety-eligible tasks.The anxiety split is substantially narrower and 256 of 264 eligible packages originate from MMPsy interview/feature records.
- Protocol Stratification: 0.9779 prompted-speech AUROC is reached by EviBound versus 0.5993, with semantic interpretation suppressed in favor of admissible acoustic evidence.Fixed-reading rows show a similar trend despite limited statistical support.
- Paired Bootstrap Checks: Depression-eligible improvements over direct omni-modal baselines remain above zero, but anxiety intervals require caution because the split is dominated by MMPsy interview records.The pooled depression result is computed over eligible evidence profiles.
- Paired Bootstrap Checks: Confidence intervals overlap zero for EviBound versus the 5-way acoustic consensus on both eligible tasks, qualifying that comparison.The supported interpretation is preserved acoustic strength with added evidence-bound routing and report validation.
- Evaluation Interpretation: Predictive accuracy and evidence-valid reasoning are evaluated jointly because strong classification can coexist with unsupported modality claims or missing-evidence omissions.Secondary diagnostics remain independent from boundary-consistency metrics.
6 Ablation and Analysis
Ablation and control analyses separate predictive evidence routes from report-validation components, showing strong acoustic contributions while attributing EviBound’s added value to evidence governance. Replay checks and shortcut controls test whether outputs respect protocol and source boundaries.
- Ablation Study: The 5-way acoustic consensus is the strongest non-harness acoustic baseline, while final EviBound preserves essentially identical AUROC after adding routing and boundary validation.The validator cannot alter predictive scores because risk estimates are frozen before report repair.
- Output-Scope Replay Checks: Table 3 organizes held-out comparisons by depression-eligible, anxiety-eligible, and MMPsy-dominated anxiety tasks, reporting predictive and boundary-consistency outcomes.Best and second-best results are visually distinguished in the table.
- Protocol-Stratified Analysis: Table 4 stratifies held-out AUROC by evidence profile, enabling comparison of EviBound’s delta across heterogeneous protocol settings.The table’s protocol interpretation is supported by profile-level reporting of interview-centered, prompted, and fixed-reading packages.
- Output-Scope Replay Checks: 0% CVR and 100% EBCPass are obtained after flagged reports are repaired or blocked before emission.Replay diagnostics target acoustic hallucinations, prompted-speech semantic overreach, unsupported listening claims, and feature-scope misuse.
- Shortcut and Source-Prior Controls: Interviewer prompts alone reach AUROC 0.6410 versus 0.6471 for participant answers, motivating manifest-level separation of participant evidence from prompt/control text.Source and protocol identifiers are retained for routing and reproducibility but disallowed as participant-level clinical evidence.
- Ablation Study: External baselines are tiered by manifest compatibility, and adapter-wrapped acoustic backbones approach EviBound’s performance.The analysis attributes EviBound’s primary contribution to routing, permission control, missing-evidence handling, and reasoning rather than acoustic backbones alone.
7 Conclusion
The paper introduces evidence-bounded reasoning for heterogeneous speech-based mental health screening through the Evidence Package Benchmark and EviBound. It argues that protocol-aware evidence routing and boundary validation improve evidence consistency without abandoning predictive performance.
- The Evidence Package Benchmark and EviBound provide a protocol-aware harness with explicit evidence routing and boundary validation.
- Scaling up or extending LLM reasoning cannot resolve epistemic flattening caused by ignoring protocol boundaries.
- EviBound achieves 0% CVR and 100% EBCPass while preserving strong predictive performance.
- Clinical multimodal systems should be evaluated by predictive accuracy together with auditable evidence consistency and protocol-aware reasoning reliability.
Limitations
The paper’s evaluation is bounded by moderate benchmark scale, uneven protocol and disorder coverage, deterministic known-protocol rules, offline structured screening, and possible cultural and linguistic bias. Prospective validation and broader data are needed before generalization or deployment.
- Benchmark scale and protocol coverage: The benchmark has moderate scale, anxiety records dominated by MMPsy interviews, and limited fixed-reading samples used mainly as boundary-stress tests.The benchmark includes six heterogeneous sources, but restrictive profiles have limited sample sizes.
- Benchmark scale and protocol coverage: Generalization requires expanded longitudinal sampling, multilingual coverage, and broader psychiatric conditions.
- Deterministic rules vs. learned boundaries: Deterministic boundary rules guarantee zero violations for known protocols but cannot catch failures outside the defined manifest schema.The paper identifies open-domain conversational hallucinations as an example outside the current schema.
- Screening utility vs. clinical deployment: The work targets offline structured screening rather than prospective clinical deployment and produces risk assessments, not clinical diagnoses.
- Screening utility vs. clinical deployment: Real-world workflows require clinician-in-the-loop auditing, robustness to noisy inputs, and rigorous prospective validation.
- Cultural and linguistic bias: English and Chinese resources acquired under different populations and protocols may transmit cultural and linguistic biases that do not transfer across settings.
Ethical Statement
The ethical statement emphasizes de-identified public data, evaluator-side separation of sensitive labels, and evidence-consistency controls. It also acknowledges inherited demographic and cultural biases and the need for human oversight before deployment.
- The benchmark uses de-identified public resources and does not collect or access private health information.
- EviBound may inherit biases in speech and text data across languages, genders, and cultural groups despite enforcing evidence consistency.
- Practical use requires human-in-the-loop auditing, prospective clinical validation, and explicit demographic-bias mitigation.
- The benchmark separates model-visible observations from evaluator-only PHQ, GAD, and diagnosis-derived labels.
- Compared methods receive identical package interfaces, modality masks, missing-evidence fields, and output schemas.
B Boundary Validation and Replay
The boundary layer deterministically checks report validity without changing predictive scores. Audits show complete compliance across the frozen benchmark and perfect held-out replay performance.
- The boundary layer verifies modality usage, blocked claims, missing-evidence disclosure, and protocol permissions without modifying predictive scores.This preserves the AUROC, F1, and QWK scores used for evaluation.
- The claim-permission audit covers 1,870 packages and outputs with zero failed records, zero score changes, and zero permission violations.Screening-risk reporting is permitted, while formal diagnosis and treatment recommendation are globally blocked.
- The output-contract audit checks blocked claims, source attribution, and missing-evidence handling across all 1,870 packages.
- Held-out exact match, macro F1, and micro F1 are all 1.0000 for the boundary-action replay audit.The replay suite contains 3,955 action-labeled instances, including 786 held-out instances.
- The cleanup guard removed 168 unsupported acoustic route/tool actions while removing zero gold actions.It changed 84 rows in the audited data.
C Boundary Replay Cases
Table 8 presents label-free boundary replay cases under a frozen evidence-package interface, using cached fragments for interpretability rather than model selection. The cases illustrate evidence-scoped reporting and blocking unsupported claims without changing predictive scores.
- Boundary Replay Cases: Table 8 uses label-free cached fragments to study boundary replay for interpretability, not model selection.The cases are evaluated under a frozen evidence-package interface.
- Boundary Replay Cases: Local package assembly, routing, replay, validation, and metric computation used a Linux workstation with two NVIDIA RTX PRO 6000 GPUs.Commercial direct LMM baselines were executed through vendor APIs and cached, while local hardware handled supporting computations.
- Boundary Replay Cases: Real supported cases show when EviBound permits evidence-scoped reporting.
- Boundary Replay Cases: Counterfactual stressors show unsupported claims being blocked without modifying the predictive score.