Source-linked AI summary
A Safety-Gated Multimodal AI Backend for Mental-Health Support: Hierarchical State Representation, Conservative Risk Fusion, and Controlled Generation in Anian
Lei Wang, Xiao Wang, Lei Li
TL;DR
Mental-health support systems need to distinguish when supportive conversation is appropriate from when free-form generation should be blocked. Anian uses hierarchical state representation, conservative multimodal risk fusion, and generation gating to control that decision. Internal prototype results support technical feasibility, while the paper emphasizes the need for human-validated and real-world evidence.
Problem
Safety-critical mental-health support requires deciding when ordinary generation should not occur, because fluent systems may miss high-risk or ambiguous safety signals.
Method
Anian maps user input into linked emotion, psychosocial, safety-risk, and intervention-route states, then uses conservative multimodal fusion and gating before generation.
Results
The internal prototype achieved micro-F1 scores of 0.9604 for L1 emotion classification, 0.9144 for L2 psychosocial constructs, and 0.9742 for L4 routing; L3 high-risk recall was 1.0000 in 233 controlled stress-test samples.
Takeaways & Limitations
Anian provides a structured safety-gated pathway for support routing and escalation, with ordinary generation unavailable at moderate or high fused risk.
Takeaways & Limitations
The evidence supports internal prototype feasibility only: the data are not a perinatal clinical gold-standard corpus, labels are partly weakly supervised or rule-derived, and real-world safety remains untested.
Abstract
from arXiv · showhide
Safety-critical mental-health support systems must distinguish when supportive conversation is appropriate from when free-form generation should be blocked. This paper presents Anian, a safety-gated multimodal AI backend for perinatal mental-health support and mindfulness-intervention routing. Anian is not intended to diagnose psychiatric conditions or replace clinical care or crisis intervention. Its modular pipeline places generative AI downstream of structured state representation, conservative risk fusion, and response gating. User text or voice-derived ASR transcripts are mapped into four linked layers: L1 emotion states, L2 psychosocial constructs, L3 safety risk, and L4 intervention routes. Local text- and rule-based safety evidence is fused with external voice-derived evidence using a highest-risk-priority rule, S_fusion = max(S_local, S_external). At moderate or high fused risk, ordinary AI-generated responses and text-to-speech delivery are blocked and replaced by fixed safety content and prompts for human support. An internal prototype evaluation used approximately 858,295 normalized records from public emotion, dialogue, mental-health-related, and Chinese dialogue corpora within a weak-label and rule-derived framework. Micro-F1 scores were 0.9604 for L1 emotion classification, 0.9144 for L2 psychosocial constructs, and 0.9742 for L4 routing. In a controlled safety stress test of 233 samples, the L3 rule engine achieved high-risk recall of 1.0000 within predefined scenarios. These findings support the internal feasibility of the label framework and gating logic but do not establish clinical validity, diagnostic accuracy, real-world safety, or effectiveness. We report the architecture, ontology, safety-fusion mechanism, prototype evaluation, error-analysis plan, and roadmap for expert-reviewed and real-world validation.
1 RYTECH, Wuhan, 430074, China; 2 City University of Hong Kong, Hong Kong SAR, China
The paper concerns natural language processing and AI safety for multimodal mental-health support. Its focus includes controlled generation, safety triage, risk fusion, perinatal support, and mindfulness routing.
- The paper applies natural language processing to mental-health support.
- Its safety focus includes multimodal AI, safety triage, and risk fusion.
- The intended application includes perinatal support and mindfulness routing with controlled generation.
1 Introduction
The introduction frames mental-health support as a safety-gating problem: systems must determine when helpful generation should be blocked. Anian addresses this through structured state representation, conservative safety assessment, intervention routing, and downstream generation control.
- Fluent, empathic generation is insufficient when systems may miss self-harm, harm-to-others, loss-of-control, or ambiguous safety signals.
- Perinatal mental-health support requires pathways connecting screening with assessment, treatment, follow-up, and safety response.
- Anian first constructs structured states, assesses safety, selects an intervention route, and then determines whether ordinary generation is allowed.
- The paper contributes an L1-L4 representation separating affect, psychosocial constructs, safety risk, and support routing.
- Its contributions also include hybrid model-rule safety control, conservative multimodal fusion, generation gating, and a staged validation roadmap.
2 Related work
Related work establishes both the potential and the safety gap of digital mental-health tools and conversational AI. Anian responds by making generation conditional on risk-aware routing and by blocking ordinary generation under elevated risk.
- Perinatal AI research suggests potential for early identification and risk stratification but remains limited by under-representative data and insufficient external validation.
- Safety-critical support requires endpoints such as high-risk recall, false-negative rate, false-positive burden, escalation appropriateness, and generation leakage.
- Conversational AI can produce inappropriate reassurance or fail to follow escalation boundaries, motivating safety, transparency, oversight, accountability, and governance.
- Anian treats generation as a constrained expression component whose availability depends on upstream risk fusion.
- Mindfulness and smartphone applications show potential short-term effects, while engagement and long-term outcomes remain uncertain.
- Anian makes intervention routing state-dependent and safety-aware, with moderate or high risk overriding ordinary content routing.
3 System design and methods
Anian is a safety-first backend that represents user expression across affect, psychosocial constructs, safety risk, and intervention routing before controlling generation. It uses multimodal highest-risk fusion and blocks ordinary responses for moderate or high risk, while its current evidence remains an internal prototype evaluation rather than clinical validation.
- Study design and intended use: Anian is an internal prototype backend, not a diagnostic system, medical device claim, clinical screening conclusion, treatment recommendation, or crisis-intervention service.Its stated purpose is support routing and safety control rather than diagnosis.
- Overall architecture: The pipeline accepts text or voice, processes voice through ASR and speech analysis, then combines local state predictions with external speech-derived safety evidence.The local pipeline follows a safety-first order before routing and generation decisions.
- Generation gating: Ordinary AI-generated responses and ordinary text-to-speech delivery are blocked when fused risk is moderate or high; low-risk states may receive ordinary support routes.The generation gate permits ordinary generation only below the moderate threshold.
- Hierarchical L1-L4 state representation: L1 captures affective states, L2 intervention-relevant psychosocial constructs, L3 safety levels, and L4 actionable support routes.Labels describe expressed states or themes and do not imply psychiatric diagnoses.
- Safety rules and routing: L3 detects safety expressions and suppresses false positives, while moderate or high risk overrides ordinary L4 routing and forces safety escalation.Examples of non-risk contexts include negation, quotations, third-party discussion, colloquial exaggeration, and commercial idioms.
- Conservative safety fusion: S_fusion = max(S_local, S_external) preserves the highest-risk modality signal, preventing calm or unavailable external speech evidence from downgrading high-risk text.The risk order is high > moderate > low > none > unknown.
- Data sources and evidence boundaries: The prototype uses public multi-source data with weak-label and rule-derived evidence, and these corpora are not perinatal clinical interview data or a diagnostic gold standard.The evaluation framework combines original dataset labels, Anian rule mappings, and AI/rule-assisted automatic labels.
4 Prototype technical evaluation
The prototype evaluation tested weak-label state learning, routing, and predefined safety-gating behavior rather than clinical validity or real-world safety. Model metrics were strong on internal tasks, while the L3 stress test achieved complete recall within its predefined scenarios.
- Model performance: 0.9604 micro-F1 was achieved for L1 emotion classification, 0.9144 for L2 psychosocial constructs, and 0.9742 for L4 intervention routing.These results evaluate the current weak-label framework and routing policy.
- Safety stress testing: 1.0000 high-risk recall was achieved by the L3 rule engine across 233 predefined safety stress-test samples.The test also reported zero mismatches and included self-harm, harm-to-others, acute loss-of-control, and false-positive scenarios.
- Safety stress testing: The L3 stress-test result indicates coverage of designed scenarios, not real-world zero false negatives.Broader language variation, expert gold labels, ASR perturbation, dialect and accent testing, multi-turn accumulation, and deployment-condition evaluation remain necessary.
- Planned evaluation: A formal ablation study is planned to compare text-only, rule-only, text-plus-rule, voice-only, full fusion, and ungated systems.The design targets high-risk recall, false-positive burden, escalation accuracy, and unsafe generation exposure.
- Error analysis: Subsequent error analysis will categorize semantic mixture, negation, quotation or third-party context, ASR, multi-turn accumulation, and route-acceptability errors.The framework is intended to supplement aggregate metrics with categorized failure analysis.
5 Discussion
Anian’s discussion frames safety-gated multimodal support as a structured action-selection problem, with ordinary generation downstream of risk assessment and routing. The evidence remains internal and prototype-level, requiring expert-reviewed and real-world validation before clinical or deployment claims.
- Principal findings: Anian places structured state representation, conservative safety fusion, and generation gating before ordinary generative interaction.The architecture is presented as a safety-orchestration approach rather than an LLM-first design.
- Safety fusion: Highest-risk-priority fusion preserves critical safety evidence when text, ASR, and voice-derived signals conflict.The design accepts potentially higher false positives to reduce high-risk false negatives.
- Perinatal support: Mindfulness practices are routed as support resources, mapping states such as worry, sleep distress, self-blame, and isolation to different interventions.Moderate or high safety risk maps to safety escalation rather than ordinary mindfulness support.
- Evidence boundaries: The current evidence supports internal technical feasibility only, not clinical validity, diagnostic accuracy, real-world safety, usability, or effectiveness.The corpus is not a perinatal clinical interview gold-standard dataset, and several validation domains remain incomplete.
- Validation roadmap: Future validation requires human gold-standard data, expert adjudication, ASR robustness testing, ablations, workflow studies, and prospective clinical evaluation.The roadmap identifies these as later phases beyond the current internal prototype evaluation.
6 Limitations
The paper identifies substantial evidence boundaries around data representativeness, labeling, safety testing, speech robustness, human evaluation, and expert review before real-world use.
- Data and validation: The evaluation uses public emotion, dialogue, mental-health-related, and Chinese dialogue datasets rather than a perinatal clinical interview gold-standard dataset.This limits the current evidence to prototype technical evaluation.
- Data and validation: L2 and L4 labels are partly weakly supervised or rule-derived, so reported metrics mainly indicate learning of the current label framework.They do not establish clinical construct validity.
- Safety testing: The L3 result comes from a controlled stress test and should not be interpreted as real-world zero false negatives.The tested scenarios do not substitute for broader deployment-condition evaluation.
- Speech robustness: ASR errors, low-volume or emotional speech, noise, accent and dialect variation, and abnormal speech rate were not systematically tested.Speech robustness remains an open validation requirement.
- Real-world evaluation: The evaluation lacks human gold-standard annotation, expert adjudication, prospective users, clinical outcomes, safety-event monitoring, and workflow integration.Fixed scripts, support prompts, and human-support pathways also require independent expert review before real-world use.
7 Future work
Future work centers on independent perinatal human evaluation, component-level safety analysis, speech robustness, and prospective usability and clinical assessment.
- Human validation: The next phase will establish a perinatal Chinese human gold-standard dataset with independent annotation and expert adjudication.L1-L4 labels should be evaluated separately within this validation phase.
- Safety evaluation: L3 evaluation will emphasize high-risk recall, false-negative rate, false-positive rate, and categorized error examples.These endpoints are intended to characterize safety performance beyond aggregate metrics.
- Speech robustness: Speech robustness testing will report ASR character error rate together with safety recall.The planned testing targets speech conditions relevant to safety detection.
- Ablation analysis: A formal ablation study will quantify contributions from text models, rules, external voice safety, and generation gating.This separates the effects of the major safety-system components.
- Prospective evaluation: A later prospective study will evaluate usability and real-world clinical support outcomes.The supplied passage introduces this prospective phase but does not report its results.
8 Conclusion
Anian combines structured state representation, rule-based safety control, multimodal fusion, and generation gating for mental-health support and perinatal mindfulness routing. The current evidence supports internal prototype feasibility, while expert-reviewed and real-world validation remains necessary.
- Anian provides a structured pathway from user expression to support routing and safety escalation.
- The system’s current evidence supports internal prototype feasibility, not clinical effectiveness or deployment readiness.
- The next validation step is human-validated, expert-reviewed, and real-world testing of support, escalation, and generation-blocking decisions.
Declarations
The study used public datasets and internal, non-clinical prototype evaluations without human participants, prospective clinical intervention, or identifiable patient-level outcomes. Data and code availability are limited, and one author is affiliated with the developer and potential commercializer.
- Ethics and registration: The study used public datasets, internal engineering test cases, and non-clinical prototype evaluation without human participants or prospective clinical intervention.
- Ethics and registration: Clinical trial registration was not applicable because the manuscript does not report a clinical trial.
- Data and code availability: The merged corpus, weak labels, stress-test prompts, preprocessing manifest, and some processed data are not publicly released in this version.
- Data and code availability: Production deployment configuration, full safety-trigger libraries, server addresses, and internal orchestration code are not publicly released.
- Funding and competing interests: The internal prototype technical evaluation received no external funding, and the authors declare no competing interests apart from one disclosed affiliation.
- Author contributions and AI assistance: General-purpose AI tools assisted with language polishing, structural revision, and formatting, while the authors retained responsibility for the manuscript.
Appendix: Ontology, safety rules, and evaluation protocols
The appendix organizes Anian’s emotion and psychosocial ontologies, routing and safety-rule taxonomies, annotation workflow, robustness protocols, ablations, and gate-audit metrics. Its examples link user expressions to intervention routes while requiring safety interpretation through L3.
- Ontology: The appendix includes separate ontologies for L1 emotion states and L2 psychosocial constructs.
- Ontology: Examples map expressions such as crying, worry, anger, self-blame, and feeling lighter to low-mood, overload, self-compassion, or grounding-related interpretations.
- Intervention routing: Intervention routes include light check-ins, breathing or sleep grounding, self-compassion, support-seeking prompts, and caregiving-pressure downshift.
- Safety rules and gating: Moderate or high L3 risk triggers fixed safety content, support pathways, and blocked ordinary generation.
- Safety rules and gating: The safety-rule taxonomy addresses false positives, including slang or idioms containing high-risk words, especially in Chinese contexts.
- Evaluation protocols: Evaluation protocols cover human annotation, gold-set metrics, ASR robustness, acceptable-route agreement, ablations, subgroup analysis, and gate-audit measures.
- Evaluation protocols: Gate-audit metrics distinguish blocked-sample proportion, safety recall, low-risk overblocking, missed blocks, and correct downstream actions.
- Evaluation protocols: Audit records track pass, revise, or reject decisions alongside model-rule, ASR, service, and robustness-report versions.