PaperlayerView exact revision on arXiv

Source-linked AI summary

Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

Yunqi Zhu, Wensheng Zhang, Xuebing Yang

arXiv:2609.05090v1cs.AI

TL;DR

Medical-agent evaluation often measures only final-answer correctness, leaving the trustworthiness of intermediate reasoning underexamined. MedTraj constructs and evaluates structured reasoning trajectories, injects controlled errors, filters steps by marginal contribution, and uses quality-weighted context learning; across three datasets, trajectory context improves coherence, while CECMed results show higher correctness and lower hallucination. The framework remains limited by its reliance on LLMs for both trajectory generation and evaluation.

  • Problem

    Answer-centric medical benchmarks overlook the quality of intermediate reasoning, even though fabricated evidence or incoherent logic can make a correct clinical answer dangerous.

  • Method

    MedTraj constructs structured trajectories, scores five quality dimensions, injects controlled errors, filters steps by marginal contribution, and feeds evaluations into quality-weighted context learning.

  • Results

    +0.029 to +0.041 coherence gains over a zero-shot baseline were observed across CareQA, PubMedQA, and CECMed; CECMed reached 73.8% correctness with an 87% hallucination reduction.

  • Takeaways & Limitations

    Trajectory context consistently improves reasoning quality across medical QA tasks, while step-level analysis identifies a small set of high-impact steps and a practical four-step reasoning-length boundary.

  • Takeaways & Limitations

    The pipeline relies entirely on LLMs for trajectory generation and evaluation, so model biases and internally coherent hallucinations may pass through undetected.

Abstract

from arXiv · show

Evaluation of medical artificial intelligence agents remains predominantly answer-centric, assessing only the correctness of final outputs while overlooking the quality of intermediate reasoning. In clinical settings, however, a correct answer reached through fabricated evidence or incoherent logic is as dangerous as an incorrect one. We propose MedTraj, a framework that treats reasoning trajectories as critical objects for construction, evaluation, and optimization. The pipeline generates structured multi-step reasoning chains from medical reasoning sources. Each trajectory is then parsed into clinical observations, evidence, numbered reasoning steps, and a final conclusion, and scored across five quality dimensions: coherence, evidence support, hallucination, completeness, and traceability. Controlled error injection introduces targeted faults into otherwise correct trajectories to establish causal links between specific reasoning failures and measurable quality degradation. Building on this, step-level filtering based on marginal contribution identifies which individual reasoning steps drive or undermine trajectory quality. Finally, quality-weighted context learning feeds trajectory evaluations back into the model at inference time, allowing it to learn from both strong and weak reasoning demonstrations. Experiments across CareQA, PubMedQA, and CECMed demonstrate that trajectory context consistently improves reasoning coherence, with gains of +0.029 to +0.041 over a zero-shot baseline. On CECMed, quality-weighted context nearly doubles the correctness over the zero-shot baseline while cutting the hallucination ratio by 87%. Marginal-contribution analysis further shows that a small minority of reasoning steps carry most of the quality signal, and that extending chains beyond four steps yields diminishing returns.

I. INTRODUCTION

Medical-agent evaluation is shifting beyond final-answer correctness because clinically dangerous reasoning can remain hidden behind a correct answer. MedTraj addresses this gap by constructing, evaluating, and optimizing reasoning trajectories, with experiments showing consistent quality gains.

  • Motivation: Answer-centric benchmarks overlook whether intermediate clinical reasoning is coherent, evidence-supported, and trustworthy.A correct conclusion reached through fabricated evidence or incoherent logic can still be clinically dangerous.
  • Framework: MedTraj generates structured multi-step clinical reasoning chains and scores them across coherence, evidence support, hallucination, completeness, and traceability.The pipeline parses observations, evidence, numbered reasoning steps, and conclusions before evaluation.
  • Results: +0.029 to +0.041 coherence gains over a zero-shot baseline were observed across CareQA, PubMedQA, and CECMed with trajectory context.The experiments cover three medical benchmarks and also analyze correctness, hallucination control, reasoning length, and step-level quality dynamics.
  • Framework: Controlled error injection introduces targeted faults into correct trajectories to connect specific reasoning failures with measurable quality degradation.The framework uses these corrupted trajectories to support causal analysis of reasoning-quality changes.
  • Framework: Step-level filtering uses marginal contribution to identify reasoning steps that drive or undermine overall trajectory quality.This treats individual steps as potentially unequal contributors to trajectory success.

B. Multi-Step Diagnostic Reasoning

Multi-step clinical reasoning systems combine structured deliberation, retrieval, tool use, and reflection, but medical evaluation still often centers on final answers. MedTraj operationalizes trajectory generation and quality-aware context learning across three medical datasets.

  • Multi-Step Diagnostic Reasoning: Chain-of-thought, tree-of-thought, and ReAct-style systems expose or expand clinical reasoning through sequential inference, parallel hypotheses, or external actions.These approaches improve transparency or iterative refinement but can remain vulnerable to errors along reasoning paths.
  • In-Context Learning: In-context learning adapts agents by conditioning on input–output demonstrations, with performance sensitive to demonstration selection and ordering.Prior work links example quality, label distribution, input similarity, and retrieval to in-context performance.
  • Medical Evaluation: Medical benchmarks commonly assess reference-question answering, while workflow simulations evaluate communication, decision-making, dialogue diagnosis, or broader clinical environments.These evaluation settings address aspects of sequential and interactive clinical decision-making beyond conventional answer scoring.
  • MedTraj: MedTraj generates structured trajectories, parses and evaluates them, filters steps by marginal contribution, and feeds quality feedback into inference-time context learning.Its five-stage pipeline uses data from three medical QA benchmarks and structured outputs from an instruction-following model.
  • Datasets: CareQA represents free-form diagnostic reasoning, PubMedQA requires biomedical-evidence-supported answers, and CECMed targets geriatric severity assessment.The study samples 2,000 examples for generation and training and holds out 500 for evaluation, except CECMed’s 1,908 training examples.

D. Structured Parsing and Quality Dimensions

MedTraj converts free-form trajectories into standardized clinical components and evaluates them across multiple quality dimensions. Controlled perturbations and composite scoring support analysis of how structural and local errors affect trajectory quality.

  • Structured Parsing: Each trajectory is parsed into clinical points, evidence, numbered reasoning steps, and a final conclusion.The decomposition supports subsequent trajectory evaluation and recombination.
  • Structured Parsing: Parsed trajectories exhibit six structural types ranging from minimal to fully elaborated forms.These structural variants are summarized in Table I.
  • Quality Dimensions: The quality scheme measures coherence, evidence support, hallucination, completeness, and traceability.These dimensions assess step connections, grounding, unsupported assertions, coverage of necessary reasoning, and derivability of the conclusion.
  • Controlled Error Injection: Seven controlled injection modes create targeted faults in otherwise correct trajectories for error analysis and negative-sample construction.Structural disruptions alter chain organization, local perturbations modify individual steps, and positional modes compare misleading prefixes with suffixes.
  • Composite Evaluation: Trajectory evaluation adds average step score and answer match score to the five quality dimensions.Average step score captures clinically relevant tokens and adjacent-step coherence, while answer match compares the conclusion with the reference answer.
  • Composite Evaluation: The composite trajectory value combines normalized positive dimensions with penalties for hallucination and invalid steps, then maps their difference to [0, 1] through a sigmoid.Positive dimensions include coherence, evidence, step, completeness, traceability, and match.

G. Step-Level Filtering and Classification

The analysis estimates each reasoning step’s marginal contribution and classifies steps by their effect on trajectory quality and answer accuracy. It also compares context-learning strategies that feed trajectory information back to the model.

  • Step-Level Filtering and Classification: Marginal-contribution analysis isolates the effect of a divergent step by comparing trajectories that share a common prefix.For example, trajectories [a, b, c] and [a, b, c, k] reveal the marginal effect δ of step k.
  • Step-Level Filtering and Classification: Steps are classified as key driver, effective, detrimental, hallucinatory, or invalid using marginal δ and host-trajectory answer match.Key drivers require δ ≥+0.05 and MS≥0.8, whereas detrimental steps have δ < −0.02.
  • Step-Level Filtering and Classification: Key drivers comprise 15.3% of steps, while detrimental steps comprise 18.5%, showing that relatively few steps disproportionately influence trajectory outcomes.Step 2 has the highest mean δ (+0.261), and key-driver plus hallucinatory labels make up 69% of its classifications.
  • Step-Level Filtering and Classification: Step 2 is a critical decision point, while steps 3–5 have stable, gradually diminishing contributions and steps 6 and beyond show a sharp quality collapse.These patterns motivate treating longer reasoning chains cautiously rather than assuming that additional steps improve quality.
  • Context-Learning Strategies: The compared inference strategies include no context, QA context, trajectory context, quality-weighted context, supervised fine-tuning, self-consistency, and Best-of-N.Trajectory Context supplies high-value reasoning demonstrations, whereas Quality-Weighted Context mixes high- and low-quality trajectories with scores.
  • Context-Learning Strategies: Quality-weighted context presents scored high- and low-quality trajectories so the model can learn distinctions between strong and weak reasoning.The comparison also tests supervised fine-tuning and sampling-based alternatives that modify weights or generate multiple responses.

IV. EXPERIMENTS

The experiments evaluate seven inference or training strategies across three medical datasets using correctness, coherence, evidence, and hallucination metrics. Trajectory context improves coherence across datasets, while correctness gains vary by task structure.

  • Experimental Setup: Experiments use DeepSeek-R1-Distill-Qwen-7B as the base model, Qwen3-32B for trajectory generation, and Qwen3-8B as the LLM judge.Generation uses a maximum of 4096 new tokens, temperature 0.3, and top-p 0.9.
  • Experimental Setup: The evaluation compares seven strategies and reports correctness, coherence, evidence score, and hallucination ratio.Hallucination is lower-is-better; the other three metrics are higher-is-better.
  • Main Results: Trajectory context raises coherence by +0.029 on CareQA, +0.041 on PubMedQA, and +0.039 on CECMed versus no context.Evidence scores also improve by +0.020, +0.051, and +0.075, respectively.
  • Main Results: CECMed shows the largest correctness gain, with trajectory context increasing correctness from 0.390 to 0.642 (+25.2%).PubMedQA improves by +0.052, whereas CareQA correctness changes only +0.015 despite coherence and evidence improvements.
  • Main Results: In open-ended QA, improved reasoning does not necessarily produce a more precise answer because the model may reason well about the wrong diagnosis.This explains why coherence and evidence improvements can coexist with small correctness gains.

C. Step-Level Analysis

The step-level and strategy analyses show that quality-aware trajectory context can outperform answer-only or sampling-based alternatives, but its benefits depend on task structure. The strongest gains occur on CECMed, while open-ended tasks show a clearer separation between reasoning quality and correctness.

  • Context Strategies: QA-only context is generally ineffective, whereas switching from QA context to trajectory context yields correctness gains of +1.2%, +5.8%, and +20.6% across the three datasets.Providing reasoning demonstrations is more effective than providing answer demonstrations alone.
  • Context Strategies: Supervised fine-tuning on random trajectories does not selectively reinforce high-quality reasoning and leaves hallucination near baseline on PubMedQA and CECMed.On PubMedQA, correctness is 0.656 versus NC and hallucination is 0.145 versus 0.140; on CECMed, correctness is 0.408 with hallucination 0.175.
  • Inference-Time Strategies: Sampling-based methods provide limited gains: self-consistency and Best-of-N help PubMedQA modestly but fail to improve CareQA and CECMed reliably.On CECMed, their hallucination ratios are 0.203 and 0.208, the highest in the table.
  • Quality-Weighted Context: Quality-weighted context achieves CECMed correctness of 0.738 versus 0.642 for trajectory context, while hallucination falls from 0.063 to 0.025.This is a +9.6-percentage-point correctness gain and a further 60% hallucination reduction over trajectory context.
  • Quality-Weighted Context: On CareQA, quality-weighted context reaches the highest coherence and evidence scores, but correctness is only 0.2 percentage points above trajectory context.This decoupling is consistent with the finding that coherent reasoning does not guarantee an accurate open-ended answer.
  • Quality-Weighted Context: On PubMedQA, quality-weighted and trajectory context are nearly identical across all four metrics, suggesting limited extra signal from quality labels and low-quality counterexamples.Sampling more responses helps PubMedQA but produces more hallucinations on CareQA and CECMed.

C. Error Injection Analysis

Controlled error injection shows that reasoning-chain organization and error position substantially affect trajectory quality, while step composition matters more than length alone. Marginal-contribution analysis identifies a small set of influential steps, especially the second step.

  • Error Injection: Structural disruptions reduce trajectory value by 27–51%, whereas flipping numerical values or removing key points causes only 4–7% drops.The reasoning chain is more fragile to reorganization than to individual factual errors.
  • Error Injection: Trailing errors hurt more than leading errors: correct-then-wrong yields trajectory value 0.477, versus 0.540 for wrong-then-correct.The result suggests that later chain content has disproportionate influence on quality judgment.
  • Step-Level Contributions: Effective steps comprise 58.9%, while key driver and detrimental steps comprise 15.3% and 18.5%, showing that a minority carry most quality signal.These categories demonstrate that reasoning steps are not uniformly valuable.
  • Step-Level Contributions: Step 2 has the highest mean marginal contribution δ (+0.261) and widest spread, with key driver and hallucinatory steps comprising 69% of its labels.This position can either lock the trajectory onto a correct direction or send it off course.
  • Step-Level Contributions: Key driver steps produce the steepest cumulative quality gains, detrimental steps the steepest declines, and effective steps a gradual climb.The divergence widens with each additional step, indicating that step composition matters more than step count alone.
  • Reasoning Length: Beyond step 4, trajectory quality worsens: at step 6 trajectory value reaches 0.649, hallucination rises to 0.392, and answer score falls to 0.656.Steps 1–4 maintain trajectory value at 0.94–0.95 while hallucination declines from 0.123 to 0.113.

E. Trajectory Generation Statistics

Trajectory-generation statistics reveal that most generated trajectories are effective, but coherent-looking hallucinations and invalid low-quality chains remain important failure clusters. The framework uses filtering and quality feedback to manage these risks, while acknowledging scope limitations.

  • Trajectory Clusters: 82% of generated trajectories belong to the high-quality effective cluster.The generation pipeline produced 87k valid trajectories comprising 349k step instances across seven injection modes.
  • Trajectory Clusters: The coherent-but-wrong cluster comprises 12.4% of trajectories and has hallucination ratio 0.425, compared with 0.029 for the high-quality group.These trajectories appear structurally sound but rely on fabricated medical knowledge.
  • Trajectory Clusters: The low-quality invalid cluster comprises 5.5% of trajectories, with trajectory value 0.131 and hallucination ratio 0.826.The step-level filter is designed to catch and exclude these trajectories from training data.
  • Limitations: The pipeline relies entirely on LLM generation and evaluation, so internally coherent hallucinations may enter the trajectory pool and evade detection.The authors identify human expert review as a stronger future validity check.
  • Limitations: The controlled injection setting establishes links between fault types and quality drops, but its mapping to natural clinical errors is approximate.Real clinical reasoning errors may be more complex and nuanced than the isolated injected faults.
  • Framework: MedTraj combines multidimensional quality evaluation, controlled error injection, composite trajectory value, and step-level filtering to optimize medical reasoning trajectories.The framework is presented as infrastructure for using trajectory quality feedback in model improvement.

APPENDIX

The appendix documents supplementary metric distributions and the prompts used to generate and parse structured clinical reasoning trajectories. It also specifies the evaluator outputs used to score trajectory quality.

  • Supplementary Figures: Figure 5 displays distributions of correctness, coherence, evidence score, and hallucination across methods and datasets, with red diamonds marking means.It complements the mean values reported in Table III.
  • Supplementary Figures: Figure 6 displays evaluation-dimension distributions under each error-injection mode.It complements the growth curves reported in Figure 1.
  • Prompts: The trajectory-generation prompt requires salient clinical findings, supporting evidence, numbered reasoning steps, and a final clinical conclusion.The output is constrained to a single JSON object with structured explanation steps.
  • Prompts: The parsing prompt converts free-form trajectories into clinical points, evidence, reasoning steps with sub-reasoning, and a structured final result.The final result contains diagnosis, treatment, and prognosis fields.

C. LLM Trajectory Evaluation

The LLM evaluation pipeline compares a question, reference answer, prediction, and parsed trajectory, then returns structured scores for correctness, matching, coherence, evidence, hallucination, completeness, and traceability.

  • Evaluation Inputs: The evaluator receives the question, reference answer, model prediction, and parsed clinical trajectory as inputs.The parsed trajectory includes clinical points, reasoning steps, and the final result.
  • Evaluation Outputs: The evaluator reports whether the prediction is correct and assigns a 0.0–1.0 match score measuring similarity to the reference.Correctness is represented as a boolean or null when no prediction exists.
  • Evaluation Outputs: Logical coherence measures consistency between reasoning steps and clinical points, while evidence score measures support for claims.Both are returned as scores ranging from 0.0 to 1.0.
  • Evaluation Outputs: Hallucination ratio measures the proportion of assertions not grounded in clinical points, and completeness measures coverage of points, steps, and conclusion.These metrics are also specified on a 0.0–1.0 scale.
  • Evaluation Outputs: Traceability measures how well the final conclusion can be traced back to the reasoning steps.This captures linkage between intermediate reasoning and the final result.

D. Context Learning Baselines

The baselines compare question-answer examples, trajectory examples, and quality-weighted trajectory examples as context for answering clinical questions. The quality-weighted prompt explicitly distinguishes high- and low-quality reasoning to guide the target response.

  • Quality-Weighted Context: The quality-weighted prompt asks the model to answer a clinical question with numbered reasoning steps in JSON format.The target question is paired with quality-scored examples and an instruction to produce structured reasoning.
  • QA Context: The QA baseline provides example clinical questions and answers before presenting a new question.It includes example pairs and then asks the model to answer the target question.
  • Trajectory Context: The trajectory baseline provides multi-step clinical reasoning examples and asks the model to follow their step-by-step process.The examples include numbered reasoning steps and final answers across multiple high-quality trajectories.
  • Quality-Weighted Context: The quality-weighted context prompt presents scored reasoning trajectories, using higher-quality examples as desired patterns and lower-quality examples as cautionary references.Examples are labeled with quality values and match scores, including a high-quality example with Value=0.85 and Match=0.90 and a low-quality example with Value=0.25 and Match=0.30.
  • Quality-Weighted Context: The baseline examples balance high- and low-quality trajectories across k total examples.The low-quality examples are described as potentially containing errors or gaps, while high-quality examples are presented as patterns to follow.
Loading 2609.05090v1…