PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Risk Prediction: Evidence Grounding and Psychosocial Factor Verification for Explainable Suicide Risk Assessment

Tianle Hu, Chen Peng, Yi-Hsin Tsai, Takshing Andy Tung, Bingyang Sun, Yenjou Wang

arXiv:2610.08842v1cs.CLcs.AIcs.LG

TL;DR

Risk classification alone offers limited insight into the textual evidence and psychosocial factors behind suicide-risk predictions from SNS posts. The paper presents a coordinated framework combining risk assessment, evidence grounding, and factor identification. It reports results across all three tasks, with factor-verifier fusion outperforming either verifier alone.

  • Problem

    Risk classification alone provides limited insight into the textual evidence and psychosocial factors underlying suicide-related SNS posts.

    Source: §I · pp. 1–2

  • Method

    A factor-centered framework uses Risk Assessment and Evidence Grounding as supporting components for Psychosocial Factor Identification.

    Source: §I · p. 2

  • Results

    Factor Identification achieved Macro F1 0.5562, Risk Assessment achieved OOF Weighted F1 0.8088, and Evidence Grounding achieved held-out test Macro row F1 0.7605.

    Source: §VI · p. 7

  • Takeaways & Limitations

    Combining complementary factor-verification signals can provide a more fine-grained and interpretable assessment beyond risk-level prediction alone.

    Source: §VI · p. 7

    The analysis is post-level and cannot model longitudinal user changes; its 3ST analysis uses selected factor labels as proxies, not direct psychological constructs.

    Source: A. Limitations · p. 7

Abstract

from arXiv · show

Identifying suicide risk from social networking services (SNS) posts is important for detecting suicide-related signals in online environments. However, risk classification alone provides limited insight into the textual evidence and psychosocial factors behind a prediction. Based on the IEEE BigData 2026 Explainable Suicide Risk Detection Challenge, this study presents a framework consisting of Risk Assessment, Evidence Grounding, and Factor Identification. Risk Assessment uses length-based routing to accommodate posts of different lengths. Evidence Grounding identifies supporting phrases and uses a Risk-Evidence constraint to maintain consistency with the Risk prediction. For Factor Identification, two verifiers are used. The Taxonomy Verifier focuses on factor semantics, whereas the Evidence-Aware Verifier uses factor-specific lexical-semantic cues to select informative positive training units. Their prediction probabilities are combined to produce the final factor predictions. The three tasks are evaluated using task-specific F1 score measures. Risk Assessment achieved a Weighted F1 of 0.8088, Evidence Grounding achieved a test Macro row F1 of 0.7605, and Factor Identification achieved a Macro F1 of 0.5562. The results show that the framework can provide risk predictions, along with supporting textual evidence and fine-grained information on psychosocial factors. Overall, the proposed framework extends suicide-risk assessment beyond risk-level prediction and provides a more interpretable analysis of SNS posts.

I. INTRODUCTION

Suicide-related SNS posts can contain multiple psychosocial factors, while risk labels alone do not identify the textual evidence or factors behind a prediction. This study proposes a coordinated, factor-centered framework to provide risk assessment, evidence grounding, and psychosocial factor identification.

I. INTRODUCTION

  • Complex posts may contain multiple psychological, social, behavioral, and protective factors, making reliable and interpretable assessment difficult.A risk prediction alone does not show which text supports the decision or which psychosocial factors are reflected.Source: §I · p. 1
  • The framework coordinates length-aware Risk Assessment, risk-consistent Evidence Grounding, and psychosocial Factor Identification.Risk Assessment and Evidence Grounding support the factor-centered framework.Source: §I · p. 2
  • Factor Identification combines taxonomy-oriented verification with cue-guided positive-unit selection under strict cross-fitted evaluation.Source: §I · p. 2
  • The resulting factor profiles support theory-informed interpretation beyond risk-level prediction.Source: §I · p. 2

II. RELATED WORK

Prior work has advanced suicide-risk classification and separately explored evidence extraction and psychosocial factors, but these outputs are often modeled independently. This study retains risk and evidence as supporting components while focusing on definition-conditioned and cue-guided factor identification.

A. Social Media-Based Suicide Risk Assessment

B. Evidence-Grounded and Psychosocial Factor-Aware Modeling

  • Evidence-extraction approaches – improve interpretability, but separate risk and evidence models can produce inconsistent outputs.Source: B. Evidence-Grounded and Psychosocial… · p. 2
  • Psychosocial factor modeling [15], faces overlapping labels and implicit factors, motivating definition-conditioned verification and cue-guided selection.The 3ST is used only to interpret selected group-level factor contrasts, not to constrain training or define computational labels.Source: B. Evidence-Grounded and Psychosocial… · p. 2

III. METHODOLOGY

The methodology combines Risk Assessment, Evidence Grounding, and Psychosocial Factor Identification as complementary views of SNS posts. Risk routing adapts to post length, while evidence extraction is constrained to remain consistent with the risk prediction.

III. METHODOLOGY

  • The framework combines risk level, supporting evidence, and psychosocial factors as complementary views of an SNS post.Source: §III · p. 2, Figure 1 · p. 3

A. Supporting Risk Assessment and Evidence Grounding

  • Posts at or below the routing threshold use a Short-Context Ensemble; longer posts use a Long-Context Specialist.The short branch averages three DeBERTa-v3-large seeds at a 384-token maximum; the long branch uses a matched-seed model with a 640-token maximum.Source: A. Supporting Risk Assessment and… · pp. 2–3
  • Evidence Grounding scores candidate phrases around suicide-related lexical anchors, thresholds and ranks them, then discards lower-probability overlapping candidates.Hard negatives are manually reviewed, and ambiguous cases are excluded from the hard-negative training set.Source: A. Supporting Risk Assessment and… · p. 3
  • A Risk–Evidence constraint prevents an Indicator prediction from being accompanied by suicide-related evidence.Source: A. Supporting Risk Assessment and… · p. 3

B. Psychosocial Factor Identification

Psychosocial Factor Identification predicts a 24-label profile using two verifiers that assess factor semantics and select informative text units. Their post-level probabilities are combined, and selected predictions are interpreted post hoc through 3ST.

B. Psychosocial Factor Identification

  • The task predicts 24 labels—19 risk factors and five protective factors—as a binary vector for each post.Multiple factors may be expressed within a single post.Source: B. Psychosocial Factor Identification · p. 3
  • The Taxonomy Verifier checks text-unit compatibility with a factor definition, while the Evidence-Aware Verifier selects positive training units using factor-specific lexical-semantic cues.The Evidence-Aware Verifier retains up to three units scoring at least 1.0, or the highest-scoring unit as a fallback.Source: B. Psychosocial Factor Identification · pp. 3–4
  • The verifiers’ post-level probabilities are combined with weights of 0.60 for TV and 0.40 for EAV, then factor-specific thresholds determine label activation.Thresholds are determined without using the target fold; a 0.50 global floor and 0.60 floors for selected factors are applied.Source: B. Psychosocial Factor Identification · p. 4
  • Selected factor profiles are interpreted using 3ST themes of hopelessness, connectedness, and capability, not as a training constraint or direct label definition.Source: B. Psychosocial Factor Identification · p. 4

A. Dataset and Evaluation Protocol

The study uses the IEEE BigData 2026 challenge dataset and evaluates risk, evidence, and factor prediction with task-specific protocols. Risk and factor evaluation are user-grouped, whereas evidence evaluation uses a post-level split.

A. Dataset and Evaluation Protocol

  • The challenge dataset supports two subtasks—risk prediction with textual evidence and psychosocial-factor identification—which the study organizes into three system components.Source: A. Dataset and Evaluation Protocol · p. 4
  • The labeled corpus contains 1,635 posts from 153 users; 378 additional unlabeled samples lack publicly available gold labels, so analyses primarily use the labeled corpus.Source: A. Dataset and Evaluation Protocol · p. 4
  • Risk and factor evaluation use user-grouped five-fold protocols, while evidence uses a separate 70/10/20 post-level split on 1,611 cleaned posts.The evidence split prevents a post from appearing in multiple subsets; official primary metrics are Weighted F1 for risk, Phrase F1 for evidence, and Macro F1 for factors.Source: A. Dataset and Evaluation Protocol · p. 4

B. Supporting Risk and Evidence Results

The routed risk system achieves an OOF Weighted F1 of 0.8088, with context-length and routing comparisons showing different class-level effects. Evidence Grounding data is cleaned before model evaluation.

B. Supporting Risk and Evidence Results

  • 0.8088 OOF Weighted F1 was achieved by the routed system; it scored 0.8088 versus 0.8067 for the ML384 ensemble OOF, but did not outperform that ensemble on the blind leaderboard.Source: B. Supporting Risk and Evidence Results · pp. 4–5, Table I · p. 5
  • 0.9327 Indicator precision for the routed system exceeded the ML384 ensemble’s 0.9245, with nearly unchanged recall of 0.8412 versus 0.8396.The routed configuration reduced false Indicator predictions from 42 to 37, while true Indicator predictions changed from 514 to 513.Source: B. Supporting Risk and Evidence Results · p. 5
  • 1,611 usable evidence posts remained after removing 23 Indicator posts with non-none annotations and one full-post annotation, and revising 28 identifiable paraphrased or mismatched annotations.Source: B. Supporting Risk and Evidence Results · p. 5

C. Factor Identification Results

Fusion improves aggregate factor-identification performance over either standalone verifier, while per-factor strengths differ and gains vary across folds. The analysis uses strict cross-fitting and examines sensitivity to fusion weights.

C. Factor Identification Results

  • Factor identification was evaluated as a 24-label task using five-fold strict cross-fitting, with held-out posts scored only by models that excluded them.Factor-specific thresholds for each target fold were determined using the remaining folds.Source: C. Factor Identification Results · p. 5
  • Fusion led all four aggregate F1 measures, raising Macro F1 from 0.5467 to 0.5562 over TV.The reported mean fold-level Fusion–TV Macro F1 difference was +0.0092±0.0206, with Fusion higher in four of five folds.Source: C. Factor Identification Results · p. 5, Table II · p. 6
  • Fusion's average predicted label cardinality was 3.489, compared with 2.919 in the gold labels.TV averaged 3.554 and EAV 4.113 predicted labels per post.Source: C. Factor Identification Results · pp. 5–6
  • Macro F1 stayed between 0.5562 and 0.5600 across tested TV weights from 0.45 to 0.75; the adopted 0.60 weight was not selected on OOF results.The highest value was 0.5600 at α = 0.50.Source: C. Factor Identification Results · pp. 5–6
  • Fusion achieved the highest F1 on 13 factors, tied the highest value for SORI, and improved over TV on 16 of 24 factors.Standalone strengths differed by factor: EAV led on low self-esteem, traumatic experience, poor school performance, and meaning in life; TV led on six other listed factors.Source: C. Factor Identification Results · p. 6

D. Exploratory Risk-Factor Distributional Analysis

The post-hoc analysis compares selected factor profiles across risk groups in gold annotations and Fusion predictions, using 3ST themes as an interpretive lens. Agreement varies across factors, and the findings are limited to empirical group-level patterns.

D. Exploratory Risk-Factor Distributional Analysis

  • Uncertainty was estimated with 5,000 user-level cluster-bootstrap resamples of fixed OOF predictions, so intervals reflect user resampling rather than model-training uncertainty.The 95% interval used the 2.5th and 97.5th percentiles of bootstrap contrasts.Source: D. Exploratory Risk-Factor… · p. 6
  • Hopelessness (+.165 gold, +.169 Fusion) and dysfunctional family (+.048 gold, +.050 Fusion) showed close Ideation–Indicator agreement.Poor social support was negative in both annotations and predictions, with a larger magnitude in Fusion; the interpersonal-difficulty confidence interval included zero.Source: D. Exploratory Risk-Factor… · p. 6, Table IV · p. 7
  • For suicide means, the (Behavior+Attempt)–Ideation contrast was +.477 in gold and +.400 in Fusion; for PSST, it was +.252 and +.214.For PSST, the Attempt–Ideation contrast was +.809 in gold and +.506 in Fusion, preserving direction but compressing magnitude.Source: D. Exploratory Risk-Factor… · p. 6, Table IV · p. 7
  • Contrasts generally followed gold-annotation directions, but agreement varied; these group-level patterns do not validate 3ST, directly measure its constructs, or establish longitudinal progression.Selected factors were treated as empirical proxies for themes related to hopelessness, connectedness, and capability.Source: D. Exploratory Risk-Factor… · p. 6

V. DISCUSSION

The discussion connects risk-model behavior to downstream evidence extraction and explains how the two factor verifiers contribute different signals. It also emphasizes uneven factor performance and limits the theory-informed analysis to exploratory group-level patterns.

V. DISCUSSION

  • Additional context improved aggregate F1 measures and Attempt recall but reduced Behavior recall; deterministic routing's OOF gain was slight and absent on the blind leaderboard.The routed configuration did not outperform the ML384 ensemble in standalone Risk performance on the blind leaderboard.Source: §V · p. 7
  • False Indicator predictions can suppress all downstream evidence, so the routed system's reduction in such errors matters alongside its nearly unchanged Indicator recall.The discussion argues that Risk should be assessed in interaction with Evidence Grounding, not only through aggregate classification scores.Source: §V · p. 7
  • TV checks semantic compatibility with factor definitions, while EAV uses factor-specific cues to favor more directly supportive positive training units.Their fusion combines different inductive biases, allowing EAV's factor-specific signals to contribute despite weaker standalone aggregate performance.Source: §V · p. 7
  • Indirect expression, low frequency, and semantic overlap make fine-grained factor boundaries difficult, potentially contributing to over-prediction and uneven per-factor performance.EAV's higher predicted label cardinality is consistent with broader factor activation; TV's stronger taxonomy signal partially controls it in fusion.Source: §V · p. 7
  • Selected Fusion contrasts preserve some gold-annotation patterns, but directional agreement does not ensure accurate contrast magnitudes or direct measurement of 3ST constructs.The analysis does not validate 3ST or establish longitudinal progression from ideation to suicidal action.Source: §V · p. 7

A. Limitations

The framework combines risk assessment, evidence grounding, and psychosocial factor identification, but its evaluation and analysis have several stated limitations. The conclusion reports results across all three tasks and identifies priorities for future work.

A. Limitations

  • The analysis is post-level, cannot model longitudinal changes within users, and uses selected factor labels as proxies for broader 3ST themes rather than direct psychological constructs.The long-context branch receives only 43 of 1,635 OOF samples, and its standalone Risk advantage does not consistently transfer across evaluation settings.Source: A. Limitations · p. 7

VI. CONCLUSION

  • 0.5562 Macro F1 for Factor Identification, 0.8088 OOF Weighted F1 for Risk Assessment, and 0.7605 held-out test Macro row F1 for Evidence Grounding were reported.The conclusion also reports that factor-verification fusion outperformed either verifier alone.Source: §VI · p. 7
  • Future work targets rare-factor identification, systematic evaluation of cue-guided positive-unit selection, and longitudinal user-level analysis.The reported results indicate that complementary factor-verification signals can provide finer-grained assessment beyond risk-level prediction alone.Source: §VI · p. 7
Loading 2610.08842v1…