PaperlayerView exact revision on arXiv

Source-linked AI summary

MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer Accuracy

Mengxi Xiao, Kailai Yang, Pengde Zhao, Enze Zhang, Ziyan Kuang, Zhiwei Liu, Weiguang Han, Shu Liao, Lianting Huang, Guojun Xiong, Victor Gutierrez Basulto, Jinpeng Hu, Min Peng, Qianqian Xie, Sophia Ananiadou

arXiv:2512.09636v3cs.CL

TL;DR

Mental-health judgments must keep their specificity, certainty, severity, and actionability warranted by limited evidence, while existing evaluations leave reasoning trajectories under-specified. MIRAMIND evaluates outcomes and evidence-to-judgment trajectories across six task families; evaluations reveal a recurring restraint gap, and Mindora achieves the best overall rank among evaluated systems with more balanced trajectories.

  • Problem

    Mental-health judgments must keep their specificity, certainty, severity, and actionability warranted by limited, subjective, and ambiguous evidence, while benchmark evaluation of reasoning trajectories remains under-specified.

    Source: §1 · pp. 1–2

  • Method

    MIRAMIND evaluates final decisions and evidence-to-decision trajectories across six task families, while Mindora uses hard-case supervision, structured trajectory rewriting, and consistency-aware hybrid SFT–RL training.

    Source: §1 · p. 2, §3 · p. 6

  • Results

    Evaluations of 20 LLMs reveal a recurring restraint gap, while Mindora achieves the best overall rank among evaluated systems and produces more balanced reasoning trajectories.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    MIRAMIND exposes evidence-to-judgment failures that final-answer correctness alone cannot capture.

    Source: §5 · p. 8

    Strong benchmark performance does not imply clinical safety, treatment validity, deployment readiness, or regulatory compliance.

    Source: Ethical Statement · p. 9

Abstract

from arXiv · show

Mental-health reasoning with large language models (LLMs) is an evidence-constrained judgment problem: models must transform limited, subjective, and often ambiguous evidence into interpretations, decisions, or claims whose specificity, certainty, severity, and actionability remain warranted. Existing benchmarks mainly evaluate specific clinical roles or final answers, leaving the reliability of explicit reasoning trajectories under-specified. We introduce MiraMind, a unified benchmark spanning six task families and 13 datasets across appraisal, diagnosis, intervention, abstraction, and verification. MiraMind evaluates both task outcomes and the trajectories connecting evidence to judgment through usability, logical structure, and informational contribution. Evaluating 20 LLMs reveals a restraint gap, in which the specificity or certainty of model judgments exceeds what limited evidence supports. We further train Mindora, an 8B model that targets evidence-to-judgment transitions through hard-case supervision, structured trajectory rewriting, and consistency-aware optimization. Mindora achieves the best average rank on MiraMind, improves over its backbone across all six task families, and produces more balanced reasoning trajectories. These results show that MiraMind can expose shared restraint failures and evaluate whether targeted post-training improves evidence-constrained mental-health reasoning.

1 Introduction

MIRAMIND frames mental-health reasoning as evidence-constrained judgment and evaluates both answers and the trajectories connecting evidence to judgment. Evaluations reveal recurring restraint failures, while Mindora’s results support targeted post-training as a way to improve control across task families.

1 Introduction

  • MIRAMIND addresses benchmarks focused on specific roles or endpoints by evaluating how evidence is transformed into judgment, not only whether the final answer is correct.Prior benchmarks cover diagnosis, counseling, or safety, while final-answer metrics can miss unsupported assumptions, excessive certainty, and unnecessary pathologization.Source: §1 · p. 2
  • The benchmark scores final-answer performance alongside trajectory usability, logical structure, and informational contribution.This shared structure is intended to make evidence-to-judgment failures comparable across otherwise isolated mental-health tasks.Source: §1 · p. 2
  • Across 20 LLMs, MIRAMIND reveals a shared restraint gap: models overread weak cues, introduce generic clinical assumptions, or give rationales that do not support their conclusions.Varying evidence sources and required judgments, while evaluating trajectories as well as endpoints, makes these failures visible.Source: §1 · p. 2
  • Mindora achieves the highest overall rank among evaluated systems and produces more balanced reasoning trajectories.Its gains across multiple task families support the interpretation that targeted post-training improves transferable control rather than fitting a single dataset.Source: §1 · p. 3
  • MIRAMIND formulates reliable mental-health reasoning as evidence-constrained judgment across six task families and 13 datasets.Its common evaluation structure combines refined resources, task-specific dataset versions, and a new expert-annotated evidence-summarization dataset.Source: §1 · pp. 2–3

2 MIRAMIND Benchmark

MIRAMIND treats mental-health reasoning as evidence-constrained judgment and evaluates both task outcomes and the trajectories connecting evidence to decisions. Its benchmark construction preserves uncertainty where possible and tests related judgment demands across varied tasks.

2 MIRAMIND Benchmark

  • MIRAMIND organizes six task families under five aspects and evaluates whether judgment strength remains proportional to available evidence.Its construction principles require consequential evidence-to-judgment mapping, preserve incompleteness or uncertainty where possible, and assess both endpoints and explicit reasoning trajectories.Source: §2 · p. 3

2.1 Task Definition and Dataset Curation

  • Appraisal, diagnosis, intervention, and psychiatry QA test judgments from short personal statements, informal narratives, counseling cases, and clinical questions.The benchmark preserves non-identical appraisal taxonomies and dataset-specific clinical labels, and separates intervention training from real-demonstration testing.Source: §2.1 · pp. 3–4
  • Abstraction tests whether psychiatric-review summaries preserve findings’ direction, magnitude, and certainty, while verification tests claims against noisy online evidence.PSRS contains 115 Cochrane abstracts, with two psychiatrists rating generated summaries; MentalMisinfo uses manually annotated video-derived content.Source: §2.1 · p. 5

2.2 Reasoning Trajectory Evaluation

  • Trajectory evaluation scores usability, logical structure, and informational contribution alongside task-specific final-answer metrics.An LLM judge segments trajectories into reasoning units and labels their relations; the protocol was validated on 100 trajectories independently annotated by two trained evaluators. These measures do not establish clinical correctness, safety, or faithfulness to latent computation.Source: §2.2 · pp. 5–6

3 Mindora: Targeted Post-Training for Evidence-Constrained Judgment

Mindora tests whether post-training can improve evidence-to-judgment control across heterogeneous mental-health tasks. Its pipeline targets difficult cases, explicit evidence-to-inference transitions, and consistency between reasoning and final judgments.

3 Mindora: Targeted Post-Training for Evidence-Constrained Judgment

  • Mindora uses hard cases where the base model fails under zero-shot prompting to create supervision for evidence-sensitive distinctions.The authors describe these examples as less likely to be solved through surface cues and therefore higher-signal for learning.Source: §3 · p. 6
  • Structured rewriting turns trajectories into concise, non-backtracking explanations with explicit evidence, intermediate inferences, and final judgments.The rewriting is intended to reduce unsupported transitions, omitted qualifiers, and generic reasoning templates.Source: §3 · p. 6
  • A hybrid SFT–RL objective uses rewards for format and length validity, internal consistency, and task-specific correctness.The reward discourages trajectories whose intermediate reasoning is disconnected from or inconsistent with the final judgment.Source: §3 · p. 6

4 Experiments

Across 20 LLMs, MIRAMIND tests answer performance and the reliability of explicit reasoning trajectories across heterogeneous tasks. Mindora leads on average rank and improves over its backbone across all six task families, while the evaluations expose recurring restraint failures.

4.1 Experimental Setup

  • The evaluation covers proprietary, open-weight, reasoning-oriented, and mental-health-specific models, prompting each to provide both a final answer and a structured trajectory.The study examines whether answer competence accompanies evidence-constrained judgment, whether restraint failures recur across task families, and whether targeted post-training improves performance.Source: §4.1 · p. 6

4.2 Main Benchmark Results

  • Strong performance does not transfer consistently across task families: models with high psychiatry QA scores can perform weakly on intervention.GPT-o4-mini and DeepSeek-R1 score 0.7402 and 0.7459 on psychiatry QA, respectively, but 0.2086 and 0.2020 on intervention; Qwen2.5-72B scores 0.6527 on condition detection and 0.9555 on abstraction, but 0.2796 on intervention.Source: §4.2 · pp. 6–7

4.3 Key Findings of MIRAMIND

  • High answer competence does not ensure well-controlled reasoning: informative trajectories can still be long, inadequate, irrelevant, or contradictory.DeepSeek-R1 has Informativeness of 0.9958 and a Contradiction Rate of 0.0410; Qwen-plus has Step Adequacy of 0.4500 and averages 9.2035 steps.Source: §4.3 · pp. 7–8
  • The restraint gap recurs across appraisal, condition detection, intervention, psychiatry QA, and misinformation verification, including overinterpretation and unsupported clinical assumptions.Mindora’s ablations worsen average rank when hard-case filtering, structured rewriting, or the consistency reward is removed; removing structured rewriting causes the largest degradation.Source: §4.3 · p. 8

4.2 Main Benchmark Results

  • Mindora achieves the best average rank (4.92) and improves over Qwen3-8B across all six task families.The main benchmark results are grouped by task family, with average scores across evaluated datasets also reported.Source: §4.2 · p. 7, Table 3 · p. 7, §4.3 · p. 8

4.4 Reasoning-Trajectory Evaluation

  • On trajectory evaluation, Mindora raises terminology parsimony from 0.6792 to 0.9042 and step adequacy from 0.4208 to 0.8625 versus Qwen3-8B, and lowers contradiction rate from 0.0410 to 0.0154 versus DeepSeek-R1.Conditioned on correct final predictions, Mindora produces shorter trajectories than Qwen3-8B, with higher Step Adequacy and lower Irrelevance Rate.Source: §4.4 · p. 8

5 Conclusion

MIRAMIND evaluates evidence-constrained mental-health judgment and explicit reasoning trajectories across six task families and 13 datasets. Its evaluations reveal recurring restraint failures, while Mindora’s results support targeted post-training as an intervention for evidence-to-judgment control.

5 Conclusion

  • MIRAMIND frames reliable reasoning as keeping the specificity, certainty, severity, and actionability of judgments proportional to available evidence.Across 20 LLMs, it identifies recurring overinterpretation, unsupported assumptions, and rationales that do not justify conclusion strength.Source: §5 · p. 8
  • Mindora achieves the best average rank and improves over its backbone across all six task families, with results supporting targeted control of evidence-to-judgment transitions.The intervention combines hard-case supervision, structured trajectories, and consistency-aware optimization; the paper presents MIRAMIND as an evaluation framework and research substrate.Source: §5 · p. 8
  • Real-world deployment remains outside the benchmark’s scope and requires clinical validation, safety monitoring, and human oversight.Source: §5 · p. 8

Limitations

MiraMind’s benchmark coverage and data sources constrain what its results can establish about mental-health practice.

Limitations

  • The benchmark tests single-instance reasoning, leaving longitudinal therapy, multi-session risk monitoring, clinician–patient collaboration, culturally situated care, and multimodal signals for future work.Some tasks also inherit source-specific assumptions, and broader expert review and cross-cultural validation would strengthen the trajectory protocol.Source: Limitations · p. 9

Ethical Statement

The paper distinguishes benchmarked reasoning from clinical deployment and describes how the benchmark’s scope, data handling, and evaluation setting are bounded. It also positions MiraMind alongside clinical, counseling, and safety benchmarks.

Ethical Statement

  • Mindora is a benchmark-guided post-training case study; benchmark performance does not establish clinical safety, treatment validity, deployment readiness, or regulatory compliance.Real-world applications require clinical validation, human oversight, safety monitoring, and safeguards against unsupported advice, over-pathologizing, and crisis mismanagement.Source: Ethical Statement · p. 9

A.1 Psychiatric Diagnosis and Clinical Decision-Making Benchmarks

  • Psychiatric benchmarks mainly assess models as practitioners, whereas MiraMind spans broader operations including appraisal, diagnosis, intervention, abstraction, and verification.These include interpreting ambiguous narratives, identifying risk without over-pathologizing distress, summarizing relevant content, and assessing advice credibility.Source: §A.1 · p. 13

A.2 Counseling and Psychotherapy Competence Benchmarks

  • Counseling benchmarks assess competencies, response quality, or multi-turn support, while MiraMind targets clinically aligned reasoning across heterogeneous task families.Its trajectory assessment includes grounding, coherence, conciseness, hallucination avoidance, and internal consistency.Source: §A.2 · p. 13

A.3 Safety, Crisis, and Ethical Reasoning Benchmarks

  • Crisis, ethics, and adversarial-counseling benchmarks target specific high-risk domains; MiraMind instead assesses reasoning reliability across ordinary and high-stakes tasks.It examines whether reasoning remains coherent, cautious, and consistent across appraisal, diagnosis, intervention, abstraction, and verification.Source: §A.3 · pp. 13–14

B.1 Dataset Licenses and Access Conditions

  • For restricted or platform-derived datasets, the authors follow original access terms and do not redistribute raw text unless explicitly permitted.For datasets without an explicit public license, they follow the original access terms and cite the source.Source: §B.1 · p. 15

B.2 Usage Declaration

  • MiraMind is intended for academic research, and derived prompts, metadata, labels, scripts, or annotations are released only where source terms permit.For Cochrane-derived materials, full copyrighted abstract text is not redistributed unless allowed by the original terms.Source: §B.2 · p. 15

B.3 Personally Identifying Information and Sensitive Content

  • The authors treat user-generated and dialogue data as potentially sensitive, avoid adding identifiers, and withhold raw text unless redistribution is permitted.Client utterances are compressed into task-focused summaries; users are asked to apply appropriate ethical and institutional safeguards to sensitive research materials.Source: §B.3 · p. 15

B.4 Model Size and Computational Budget

  • Provider-side compute budgets for API-accessed models are unavailable; local evaluations used about 16 GPU-hours, and benchmark experiments cost approximately $20.The local models were at the 7–8B scale; Mindora is initialized from Qwen3-8B and its inference is included in the local compute budget.Source: §B.4 · p. 15

B.5 Appraisal Dataset Details

This section details MiraMind’s appraisal, intervention, psychiatry QA, and PSRS resources, including their construction and evaluation. The resources combine existing datasets, task-specific adaptations, and expert-reviewed material.

B.5 Appraisal Dataset Details

  • CognitiveReframing combines simulated thoughts and self-reports with professional annotations, PatternReframe uses crafted persona-based statements and independent ratings, and Therapist Q&A uses real interactions with clinical annotations.The three datasets use slightly different cognitive-error taxonomies.Source: §B.5 · pp. 15–16
  • Seven cognitive-error labels are shared across all three appraisal datasets, while other labels are dataset-specific.Therapist Q&A uniquely includes Emotional Reasoning, Mental Filtering, Magnification, and No Distortion; CognitiveReframing and PatternReframe also have distinct label sets.Source: §B.5 · p. 16

B.6 Intervention Dataset Details

  • The intervention task covers thirteen counseling strategies, using compressed client summaries and counselor-derived labels reviewed by an expert counselor.PsyDTCorpusM is reserved for training and AnnoMIM for evaluation, testing transfer from a partially synthetic corpus to real counseling demonstrations.Source: §B.6 · pp. 16–17

B.7 Multi-step Dataset Details

  • Psychiatry QA combines symptoms, case details, medical knowledge, and possible interventions into decisions that should follow from evidence rather than unsupported assumptions.Its four datasets are MHQA, MedQAen, MedMCQA, and PubMedQA.Source: §B.7 · p. 17

B.8 PSRS Dataset Construction

  • PSRS fills a benchmark gap by evaluating abstraction of clinically relevant conclusions from psychiatric systematic-review evidence through structured summary generation and expert assessment.It uses 115 mental-health Cochrane abstracts, sampled for diversity in conditions, interventions, and populations, without patient- or trial-level records.Source: §B.8 · p. 17, §B.8.1 · p. 17

B.8.2 LLM Summary Generation

  • Four LLMs generated four structured summaries per Cochrane abstract, yielding 460 summaries focused on the Main Results field, with low-temperature decoding and no external content.The task abstracts direction, magnitude, and certainty from statistical and methodological details into a concise clinical interpretation.Source: §B.8.2 · p. 17

B.8.3 Human Annotation

  • Two psychiatrists assess Main Results on a 0–2 scale for accuracy and completeness, provide rationales for non-maximal scores, and record minimal corrective actions.The evaluation uses a single-dimension rubric; full double-annotation was considered too expensive with clinical experts.Source: §B.8.3 · p. 17, §B.8.4 · p. 18

B.8.5 Inter-Annotator Agreement and Quality Assurance

  • A cross-check of 120 summaries reports absolute disagreement counts, with disagreements often arising from borderline judgments about effect reporting or over-strong wording; PSRS releases outputs, scores, rationales, and QA signals.The QA subset contains 30 reviews per model, as specified in Table 6.Source: §B.8.5 · p. 18, Table 6 · p. 18, §B.8.6 · p. 18

C Reasoning Trajectory Evaluation

MiraMind evaluates explicit reasoning trajectories for usability, logical structure, and informational contribution, alongside task answers. The section also describes Mindora’s trajectory-focused post-training pipeline.

C Reasoning Trajectory Evaluation

  • The protocol evaluates reasoning chains along usability, logical structure, and informational contribution, but these observable properties do not establish clinical correctness, safety, or faithfulness to latent computation.The dimensions target whether judgments are clear, supported, and advanced by decision-relevant information.Source: C Reasoning Trajectory Evaluation · p. 19

C.3 Scoring Metrics

  • Usability measures terminology parsimony and step adequacy; logical structure assesses coherence and relation-derived irrelevance and contradiction rates; informational contribution measures whether steps advance the conclusion.The protocol scores these dimensions independently because a trajectory can be concise but unsupported, informative but contradictory, or coherent yet opaque.Source: C Reasoning Trajectory Evaluation · p. 19, §C.3.2 · p. 21, §C.3.3 · p. 21

C.1 Evaluation Settings

  • An LLM judge scores trajectories with structured rubrics, and a human-validation study independently annotates 100 sampled trajectories across four rubric metrics.The authors plan to release judge outputs and describe the protocol as reproducible with alternative judge models.Source: §C.1 · p. 19, §C.2 · p. 19, §C.2.1 · p. 20

C.2.2 Inter-Annotator Agreement

  • Human raw agreement ranged from 0.83 to 0.92 across four metrics, while the judge matched human consensus perfectly on Logical Coherence in the 70-trajectory pilot subset and showed moderate agreement on the other metrics.A representative disagreement concerned whether repeating a speaker’s subjective framing introduced a minor grounding problem.Source: §C.2.2 · p. 20, §C.2.3 · p. 20, Table 8 · p. 20

D.1 Overview

  • Mindora is initialized from Qwen3-8B and trained in two stages: difficult-case trajectory generation followed by hybrid SFT–RL with a consistency-aware reward.The reward encourages task-relevant, internally consistent reasoning aligned with the final answer.Source: §D.1 · p. 22

D.2 Reasoning Trajectory Generation

  • Trajectory generation filters for training instances answered incorrectly by LLaMA-3-8B-Instruct, then uses verifier-guided search and retains successful rationale content supporting the correct answer.This targets useful supervision rather than simply increasing training volume.Source: §D.2 · p. 22

D.3 Structured Rewriting for Non-backtracking Trajectories

  • Mindora’s selected search trajectories are rewritten as standardized, non-backtracking rationales because raw traces can contain self-corrections, speculative branches, and backtracking artifacts.The rewritten format uses domain-relevant sections and a final conclusion explicitly mapped to the answer, strengthening coherence, alignment, and interpretability.Source: §D.3 · p. 22

D.4 Hybrid SFT–RL Post-training

Mindora combines supervised trajectory imitation with reward-driven refinement, including rewards for consistency and task correctness. It is presented as a benchmark-guided post-training case study, not a deployable mental-health system.

D.4 Hybrid SFT–RL Post-training

  • Mindora uses hybrid SFT–RL: SFT imitates structured trajectories, while RL refines behavior against task-specific correctness and reasoning-quality constraints.Its consistency-aware reward checks format, length, internal consistency, and task-specific correctness; an auxiliary Qwen3-32B model assesses factual or logical inconsistencies.Source: §D.4 · pp. 22–23
  • The consistency term penalizes fluent reasoning with unsupported or self-conflicting claims, targeting concise, grounded, internally consistent trajectories rather than endpoint correctness alone.This aligns training with MiraMind’s evaluation of whether reasoning remains evidence-proportional and decision-relevant.Source: §D.4 · p. 23

D.5 Role of Mindora in This Work

  • Mindora is a benchmark-guided post-training case study, not a deployable mental-health system.Deployment-oriented safety, treatment validity, and clinical efficacy remain outside the work’s scope.Source: §D.5 · p. 23

E.1 Evaluated LLMs

  • The evaluation covers proprietary and open-source GPT, DeepSeek, Qwen, and LLaMA models, plus psychology-focused EmoLLM and Psyche-R1.The passage describes reasoning-oriented and chat-oriented systems, models at multiple scales, and mental-health-focused baselines.Source: §E.1 · p. 24

E.2 Model Size and Computational Budget

  • Most large or hosted models were evaluated through APIs with undisclosed provider compute budgets; local evaluations used about 16 GPU-hours, and benchmark experiments cost approximately $20.Mindora is initialized from Qwen3-8B and has approximately 8B parameters.Source: §E.2 · p. 24

E.3 Group-wise Performance Analysis

Mindora ranks consistently across task groups, while trajectory comparisons and external evaluation provide additional evidence about its performance. On correctly answered instances, its trajectories are shorter and more adequate than those of Qwen3-8B.

E.3 Group-wise Performance Analysis

  • 4.92 overall average rank places Mindora ahead of evaluated systems, with strong ranks across appraisal, diagnosis, and multi-step reasoning.Its ranks are 3.00 on appraisal, 3.67 on diagnosis, and 5.75 on multi-step reasoning; lower rank indicates better performance.Source: §E.3 · p. 24, Table 9 · p. 25

E.4 External Evaluation on GoEmotions

  • On 1,012 GoEmotions test instances, Mindora achieves accuracy 0.4407 versus Qwen3-8B’s 0.3528.The paper notes that GoEmotions does not evaluate the full scope of mental-health reasoning.Source: §E.4 · p. 25
  • On a GoEmotions case where both models predict the correct labels, Mindora connects evidence directly, while Qwen3-8B repeats evidence and rejects unsupported alternatives.The example illustrates that equal final-answer correctness can conceal differences in trajectory length, evidence selection, and informational contribution.Source: §E.4 · p. 25

E.5 Trajectory Evaluation Conditioned on Correct Predictions

  • Mindora’s conditioned trajectories average 5.0079 steps versus Qwen3-8B’s 7.6168, with Step Adequacy rising from 0.8785 to 1.5238 and Irrelevance Rate falling from 0.0556 to 0.0124.Informativeness is comparable; the comparison excludes incorrect predictions rather than assigning them zero trajectory scores, so these results are not directly comparable in magnitude with unconditional averages.Source: §E.5 · p. 27, Table 10 · p. 28
  • Without structured rewriting, average trajectory length rises from 5.0079 to 9.8136 steps, while Step Adequacy falls from 1.5238 to 0.3644 and Irrelevance and Contradiction Rates rise.The passage attributes these trajectory differences to structured rewriting after controlling for final-answer correctness.Source: §E.5 · p. 27

F Case Study

The case study contrasts Mindora’s evidence-to-label mapping with trajectories that trade off brevity, structure, and decision relevance. The ablation overview describes the training components as complementary design choices.

F Case Study

  • The example identifies Overgeneralization as extrapolating from one negative incident to a broad belief about all people’s intentions.The case contrasts that pattern with alternatives such as catastrophizing, personalization, and mind reading.Source: F Case Study · pp. 29, 31
  • Mindora’s case-study trace anchors to the client’s thought, isolates the core cue, and maps evidence to a label with minimal redundancy.Other baselines may use generic brief templates or exhaustive option-scanning, which can omit linking steps or add verbosity without decision signal.Source: F Case Study · p. 29

G Ablation Details

  • Removing any of S1 hard-sample filtering, S2 structured rewriting, or S3 consistency reward degrades average MiraMind performance, supporting their complementary roles.The overview associates S1 with challenging cases, S2 with standardized trajectories, and S3 with evidence integration and reasoning reliability.Source: G Ablation Details · p. 33

G.1 Hard-sample Filtering (S1)

Ablations examine hard-case filtering, structured rewriting, consistency rewards, and few-shot prompting. The reported results indicate distinct contributions from the three training components, while few-shot gains vary across datasets.

G.1 Hard-sample Filtering (S1)

  • Without hard-case filtering, average MiraMind score drops by 0.0202, including 0.0358 on diagnosis and 0.0093 on multi-step psychiatry QA.The variant also has lower step adequacy (0.1667) and informativeness (0.1042), despite often reaching the same final answer.Source: §G.1 · p. 33

G.2 Structured Rewriting (S2)

  • Removing structured rewriting lowers overall performance by 0.0468; Step Adequacy drops by 0.6708, while Irrelevance and Contradiction Rates rise by 0.0095 and 0.0229.The authors link standardized modular rewriting to clearer supervision for distilling benchmark-aligned reasoning into an 8B-scale model.Source: §G.2 · p. 35

G.3 Consistency Reward (S3).

  • Removing S3 lowers Avgall from 0.6933 to 0.6788 and PSRS from 0.8379 to 0.6948; logical coherence and informativeness also decline.The paper highlights verification- and abstraction-style tasks requiring stable evidence integration, including PSRS and MentalMisinfo.Source: §G.3 · p. 37

G.4 Few-shot In-Context Learning Analysis

  • Few-shot prompting improves some structured classification and QA scores, but effects vary across datasets and can include declines.The analysis concludes that few-shot prompting is not a general solution, whereas Mindora’s targeted post-training gains are described as more consistent.Source: §G.4 · p. 39

Related papers

Loading 2512.09636v3…