PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Deployable Tree-Relational Structure Enhancement for Oncology LLMs

Jizheng Lai, Yingyun Li, Ying Qin, Haiyang Qian

arXiv:2609.35810v1cs.CLcs.AI

TL;DR

Medical LLMs have advanced, yet evaluations reveal gaps between benchmark performance and robust clinical reasoning, and oncology decisions depend on structured clinical relationships. TRACE learns and refines an oncology tree-relational structure offline and retrieves compact evidence at inference. It improves accuracy across the reported oncology evaluations and outperforms vanilla RAG and generic GraphRAG in the same-corpus comparison.

  • Problem

    Medical LLM evaluations show gaps between benchmark performance and robust real-world clinical reasoning, while oncology decisions involve structured clinical relationships and evidence prioritization.

    Source: §1 · p. 1

  • Method

    TRACE learns and refines an oncology tree-relational structure offline using LM-loss-derived evidence, then retrieves compact evidence for prompt augmentation.

    Source: §1 · p. 1, §2.1 · p. 2

  • Results

    TRACE improves accuracy across zero-shot, fine-tuning, retrieval-baseline, leakage-controlled, and deployment-oriented evaluations, and outperforms vanilla RAG and generic GraphRAG in the same-corpus comparison.

    Source: §1 · p. 2, §4 · p. 6

  • Takeaways & Limitations

    The results suggest explicit, updatable medical structure can make oncology LLMs more accurate, auditable, and practical for constrained clinical deployment settings.

    Source: §4 · p. 6

    The evaluation is limited to oncology, lacks systematic clinical-expert assessment of retrieved evidence, and establishes benchmark performance rather than prospective clinical validity.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Large language models are increasingly used in oncology applications, but their predictions are often weakly grounded in explicit medical structure. We present TRACE, a deployable tree-relational enhancement framework for oncology LLMs. TRACE separates expensive offline structure learning from lightweight online inference: oncology concepts and relations are organized into an updatable tree-relational structure, refined using LM-loss-derived evidence, and retrieved at inference time as compact prompt evidence. This design supports task-adaptive evidence selection without requiring supervised labels in the zero-shot setting. Across ten oncology classification tasks and one MedQuAD CancerGov QA benchmark, TRACE improves both label-free evaluation and supervised fine-tuning. Additional analyses show that TRACE improves over vanilla RAG and generic GraphRAG, remains useful under leakage-controlled METABRIC inputs, and produces interpretable evidence paths aligned with clinical reasoning. These results suggest that explicit, updatable medical structure is a practical path toward more accurate and auditable oncology LLM deployment.

1 Introduction

Oncology reasoning draws on structured relationships among disease, biomarkers, treatment, and outcomes, while medical LLM evaluations reveal gaps between benchmark performance and robust clinical reasoning. TRACE learns and refines oncology structure offline, then retrieves compact evidence for downstream prompts.

1 Introduction

  • Medical LLMs have advanced in question answering and consultation-style reasoning, but evaluations still show a gap between benchmark performance and robust real-world clinical reasoning.The introduction also argues that models should identify and prioritize evidence needed for specific clinical decisions.Source: §1 · p. 1
  • Oncology reasoning involves structured interactions among disease categories, histology, molecular markers, staging, treatment, recurrence, and prognosis.OncoKB and CIViC organize genomic alterations with diagnostic, prognostic, and therapeutic implications; PrimeKG integrates biomedical entities and relations.Source: §1 · p. 1
  • TRACE separates offline structure learning from lightweight online retrieval and prompt augmentation for downstream oncology predictions.Its offline stage includes concept-query retrieval, structure-oriented adaptation, and LM-loss-based refinement.Source: §1 · pp. 1–2
  • Initial-to-refined structure improves knowledge-intensive oncology QA, and TRACE outperforms vanilla RAG and generic GraphRAG on the 10-task macro average.The comparisons evaluate retrieval baselines built from the same oncology corpus.Source: §1 · p. 2
  • TRACE is evaluated for leakage-controlled robustness, inference overhead, and offline structure-update cost as a lightweight online enhancement with offline maintenance.Source: §1 · p. 2

2 Method

TRACE learns an oncology tree-relational structure offline using model-derived evidence, refines its node and edge support, and retrieves selected evidence online for downstream tasks. Supervised settings can additionally learn task-branch weights.

2 Method

  • TRACE treats the medical structure as an offline learnable artifact, then retrieves relevant tree paths and graph relations for prompt augmentation without updating the structure during serving.The structure can be versioned and maintained independently of the serving backbone.Source: §2 · p. 2, Figure 1 · p. 3

2.1 Tree-Relational Oncology Structure

  • The concept tree organizes oncology concepts hierarchically, while a relation graph connects the same concepts; corpus and label-free input text seed the structure, and Concept Query LoRA supports LM-loss scoring.The tree covers branches such as cancer type, pathology, staging, biomarkers, treatment, and outcome.Source: §2.1 · p. 2

2.2 Bayesian Structure Refinement

  • With tree topology fixed, TRACE aggregates descendant evidence into Beta-Bernoulli node support, allowing supported fine-grained concepts to strengthen their interpretive ancestors.Source: §2.2 · pp. 2–3
  • For graph edges, LM evidence updates a Beta prior initialized from edge confidence; posterior weights confirm, retain, or exclude edges for retrieval.The update is described as bounded, interpretable smoothing rather than an assumption of intrinsic superiority over deterministic weighting.Source: §2.2 · p. 3

2.3 Tree-Relational RAG and Task Integration

  • Online retrieval uses input-side information and combines sample matching with updated structural confidence; supervised fine-tuning can also optimize task-branch weights, unlike zero-shot TRACE.Classification uses candidate log-likelihood scoring, while generation uses standard decoding.Source: §2.3 · p. 4

3 Experiments

Experiments evaluate TRACE across ten oncology classification tasks and MedQuAD CancerGov QA, including label-free inference, fine-tuning, retrieval baselines, deployment measurements, and structural evidence cases. TRACE improves benchmark results, while deployment measurements and case studies examine serving costs and evidence traceability.

3.1 Zero-Shot Oncology Task Evaluation

  • 0.748 versus 0.728: TRACE achieves the best 10-task zero-shot task-macro average, improving over Qwen3.5-27B.The comparison uses label-free evaluation without supervised target-task labels.Source: §3.1 · p. 4, Table 1 · p. 5

3.2 Fine-Tuning on Public Oncology Benchmarks

  • 0.797 versus 0.761 in METABRIC task-macro accuracy, and 0.579 versus 0.546 in task-macro F1; TRACE also leads CancerGov QA on ROUGE-L and BERTScore.The METABRIC comparisons are against the strongest baseline; the supplied passages do not give CancerGov metric values.Source: §3.2 · pp. 4–5, Table 2 · p. 6

3.3 Ablation Study

  • Tree-Relational RAG provides the main zero-shot gain, TRACE beats vanilla passage RAG and generic GraphRAG, refinement mainly helps examination QA, and tree-plus-graph performs best among structural views.The retrieval comparisons use the same oncology corpus; the combined tree-plus-graph variant achieves the best 10-task macro accuracy.Source: §3.3 · p. 5, Table 3 · p. 6

3.4 Deployment-Oriented Evaluation

  • 2.71 ms median structural-retrieval latency; prompt augmentation raises end-to-end latency from 305.0 ms to 404.2 ms, while projected refinement costs are 28 and 110 GPU-hours at 5,000 and 20,000 elements.The larger-structure costs are linear projections, not empirically profiled measurements.Source: §3.4 · p. 6

3.5 Structural Evidence Case Study

  • In representative cases, tree paths expose clinical hierarchies and graph edges connect relevant clues, including tumor size and lymph-node status, to early-stage breast cancer.The METABRIC example removes treatment- and outcome-related fields.Source: §3.5 · p. 6

4 Conclusion

TRACE learns and refines explicit oncology structure offline, then uses lightweight online retrieval and prompt augmentation. Across the reported evaluations, it improves accuracy while preserving interpretable structural evidence.

4 Conclusion

  • TRACE learns and refines an explicit oncology structure offline, then retrieves and injects evidence online for downstream prediction.The conclusion describes the online stage as lightweight.Source: §4 · p. 6
  • Across zero-shot, fine-tuning, retrieval-baseline, leakage-controlled, and deployment-oriented evaluations, TRACE improves accuracy while preserving interpretable structural evidence.Source: §4 · p. 6
  • The authors suggest explicit, updatable medical structure as a practical path toward more accurate and auditable oncology LLM deployment in constrained clinical settings.Source: §4 · p. 6

Limitations

TRACE’s structure refinement has coverage and backbone-dependent limits, while the evaluation remains confined to oncology and does not establish prospective clinical validity.

Limitations

  • Refinement recalibrates only nodes and edges already represented in the candidate structure; extending coverage requires separate offline candidate extraction or updates.That expansion is distinct from the current confidence-refinement stage.Source: Limitations · p. 6
  • Prompt-based evidence injection varies across backbones, with some models showing negative transfer on certain tasks.Evidence formatting and calibration require further study.Source: Limitations · pp. 6–7
  • Evaluation is limited to oncology, and retrieved evidence has not been systematically assessed by clinical experts for correctness, relevance, harmful omissions, or safety.The results establish benchmark performance and evidence inspectability, not prospective clinical validity.Source: Limitations · p. 7

Ethical Considerations

The paper describes safeguards for data use and boundaries for any real-world deployment of TRACE.

Ethical Considerations

  • Private clinical-record tasks use de-identified records, and task-facing data exclude labels, reference answers, outputs, and target fields to reduce answer leakage.Real-world use would still require institutional approval and privacy and security review.Source: Ethical Considerations · p. 7
  • Real-world use would require prospective validation and workflow integration studies, with final review by qualified clinicians.Retrieved evidence and model predictions may remain incomplete, outdated, or incorrect.Source: Ethical Considerations · p. 7
  • TRACE is intended as a decision-support system, not a substitute for clinical judgment.Source: Ethical Considerations · p. 7

A Implementation Details

The appendix specifies TRACE’s implementation, dataset preparation, and evaluation setup, including offline refinement costs and inference-time latency. It also describes the CancerGov QA fine-tuning benchmark and its evaluation metrics.

A Implementation Details

  • TRACE uses a 300-concept updated structure, direct prompt evidence, and a default retrieval budget of six concepts and six graph edges per sample.The default backbone is Qwen3.5-27B; supervised task LoRA fine-tuning uses one epoch.Source: A Implementation Details · p. 9
  • One LM-loss Bayesian refinement round takes about 97 minutes end-to-end and approximately 3.1–3.3 GPU-hours; this cost is incurred offline when the structure is refreshed.Initial construction is reported by scale and provenance rather than GPU-hours because profiling logs were not retained; projections for larger structures are analytical, not measured.Source: A Implementation Details · p. 10
  • Retrieval has 2.71 ms median latency, while TRACE’s end-to-end single-sample latency is 404.24 ms versus 304.99 ms for the base prompt.The median overhead is 101.50 ms, mainly from adding a median 610 prompt tokens; profiling used 100 samples and one generated output token.Source: A Implementation Details · p. 10, Table 5 · p. 11

B Dataset Details

  • The oncology structure draws on textbooks, guidelines, expert consensus documents, and biomedical literature; METABRIC tasks remove their target fields, with an additional leakage-controlled setting excluding treatment- and outcome-related fields.The private clinical-record datasets are de-identified and are not publicly released due to institutional and privacy constraints.Source: B Dataset Details · pp. 10–11

C CancerGov QA Fine-Tuning Results

D Additional Deployment-Oriented Evaluation

The deployment-oriented evaluations examine robustness to noisy clinical inputs and the accuracy-latency tradeoff as retrieved evidence grows. The section also defines classification and generation metrics.

D Additional Deployment-Oriented Evaluation

  • The noise evaluation perturbs EMR-style inputs with abbreviations, token deletion, word-order perturbation, and irrelevant note fragments while preserving the gold label.The evidence-budget sweep varies the number of retrieved structural evidence items to characterize operational behavior.Source: D Additional Deployment-Oriented… · p. 12
  • Under severe corruption, accuracy is 0.6386 for LoRA-only, 0.6575 for Tree-Relational RAG, and 0.6618 for TRACE; TRACE has the best accuracy and macro-F1 at all three noise levels.Tree-Relational RAG offers a cheaper robustness gain with fewer prompt tokens.Source: D Additional Deployment-Oriented… · p. 12, Table 9 · p. 13
  • Accuracy rises from 0.7213 at k = 0 to 0.7449 at k = 16 as latency increases, with gains beginning to saturate after k = 8.The table characterizes an accuracy-latency tradeoff; its latency measurements should not be directly compared with the separate single-token latency microbenchmark.Source: D Additional Deployment-Oriented… · p. 13, Table 10 · p. 13

E Evaluation Protocol and Metrics

  • Classification selects the candidate label with the highest conditional log-likelihood, while CancerGov QA generation is evaluated using ROUGE-L and BERTScore.Task-macro accuracy averages task accuracies without weighting by dataset size; overall accuracy pools test examples.Source: E Evaluation Protocol and Metrics · p. 13

F Additional Zero-Shot F1 Results

The appendix reports additional zero-shot F1 and accuracy analyses, including task-level gains, confidence intervals, and a comparison of tree and graph evidence. The ablation indicates that combining both structural views yields the best macro average.

F Additional Zero-Shot F1 Results

G Bootstrap Confidence Intervals for Main Results

  • Zero-shot accuracy confidence intervals use 10,000 task-stratified bootstrap samples and 95% percentile intervals.Examples are resampled within each task before group-level and overall task-macro averages are computed.Source: G Bootstrap Confidence Intervals for… · pp. 13–14

H Full Zero-Shot Analysis

  • TRACE improves most zero-shot tasks over Qwen3.5-27B, with the largest gains on Professional Practice, Breast Stage Prediction, relapse status, and ER status.METABRIC stage and PR status show negative or near-zero changes, where Qwen3.5-27B is already a strong baseline.Source: H Full Zero-Shot Analysis · p. 14

I Tree–Graph Ablation

  • Tree-only and graph-only retrieval both help, while their combination achieves the best macro average, indicating complementary hierarchy and relation evidence.Tree evidence provides hierarchical concept grounding; graph evidence exposes cross-branch clinical relations.Source: I Tree–Graph Ablation · p. 14

J Additional Ablations

The additional ablations examine structure-weight updates, cross-backbone transfer, and task-branch weighting, alongside a comparison of retrieval baselines. Results include cross-backbone gains with one METABRIC exception and a benefit from task-branch weighting.

J Additional Ablations

  • The structure-update ablation compares retaining initial weights, linear interpolation, and TRACE’s Beta-Bernoulli update while holding the structure, retrieved evidence, and inference procedure constant.The supplied passage describes the comparison setup but does not report its outcome.Source: J Additional Ablations · p. 14
  • The same learned structure improves 10-task macro average for five backbones, while DeepSeek-V4-Flash decreases on METABRIC, reflecting dependence on backbone calibration and prompt-evidence sensitivity.The reported gains span Baichuan-M2-32B, Baichuan-M3-235B, Llama-4-Scout, HuatuoGPT-3-32B, and Qwen3-32B.Source: J Additional Ablations · p. 14
  • Removing task-branch weighting lowers overall accuracy from 0.7969 to 0.7614 and task-macro F1 from 0.5792 to 0.5376.The authors interpret this as evidence that weighting helps select task-appropriate structural evidence across shared METABRIC tasks.Source: J Additional Ablations · p. 14

K Retrieval Baseline F1 Comparison

  • Generic GraphRAG improves task-macro F1 over Vanilla RAG on three examination QA tasks and two clinical-record tasks using the same oncology corpus.The authors suggest graph-structured retrieval organizes evidence more usefully than passage retrieval alone.Source: K Retrieval Baseline F1 Comparison · p. 14

L Leakage-Controlled METABRIC Analysis

The leakage-controlled METABRIC analysis removes treatment- and outcome-related input fields and finds that TRACE retains the best task-macro accuracy and macro-F1. Structural case studies show tree paths and graph relations linking input clues to predictions.

L Leakage-Controlled METABRIC Analysis

  • The stricter METABRIC inputs remove treatment- and outcome-related fields, while withholding the target label as in the main evaluation.Removed fields include therapy, surgery, recurrence, survival, follow-up, and death-related information.Source: L Leakage-Controlled METABRIC Analysis · p. 14

M Additional Structural Evidence Case Studies

L Leakage-Controlled METABRIC Analysis

  • 0.729 task-macro accuracy for TRACE exceeds 0.704 for Qwen3.5-27B and 0.707 for Vanilla RAG; task-macro F1 improves from 0.497 to 0.527.After field removal, TRACE retains the best task-macro accuracy and macro-F1; several methods have lower absolute performance.Source: L Leakage-Controlled METABRIC Analysis · p. 14, Table 19 · p. 18

Related papers

Loading 2609.35810v1…