PaperlayerView exact revision on arXiv

Source-linked AI summary

RetiJev: Fast and Source-Grounded Ophthalmic Decision Models

Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu

arXiv:2610.30947v1cs.CLcs.AI

TL;DR

Ophthalmic point-of-care decisions need fast answers among predefined actions, but autoregressive LLM inference has latency and computational costs. RetiJev introduces source-grounded data and candidate-scoring models; it outperforms both baselines on guideline grading across model sizes and reports its strongest clinical-decision accuracy with RetiJev-9B.

  • Problem

    Autoregressive LLMs incur latency and computational overhead for predefined-action ophthalmic decisions, while point-of-care support also raises privacy concerns.

    Source: §1 · pp. 1–2

  • Method

    The paper builds source-grounded ophthalmic decision data and trains 0.8B–9B models to score candidate answers directly without autoregressive decoding.

    Source: §1 · p. 2

  • Results

    Across sizes, RetiJev outperforms both baselines on guideline grading by 10.4–23.0 percentage points; RetiJev-9B has the highest clinical-decision accuracy and matches them on diagnosis.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Direct candidate scoring provides an efficient alternative to generative inference for structured clinical decisions, with RetiJev answering a case in 158 ms on one GPU.

    Source: §1 · p. 2, §7 · p. 10

    Decision labels are source-grounded and cross-verified but not clinician-adjudicated, patient states are synthetic, and prospective evaluation has not been conducted; real-world clinician-labelled evaluation is needed before clinical use.

    Source: §6 · p. 9

Abstract

from arXiv · show

Ophthalmic clinical decision support requires fast, accurate, and privacy-preserving inference, yet autoregressive large language models (LLMs) incur substantial latency and computational overhead for decisions among predefined actions. We therefore investigate a decision-oriented alternative to generative reasoning. Specifically, we first introduce RetiJev-Data, a source-grounded ophthalmology corpus comprising 78,158 clinical-decision and diagnosis questions derived from articles in PubMed Central and a MeSH vocabulary covering 531 eye diseases, and 50,074 guideline-grading questions spanning 11 grading systems, along with a 24,662-question evaluation benchmark. Based on this, we develop RetiJev, a family of 0.8B-9B decision models that directly score candidate answers without autoregressive decoding. Across all model sizes, RetiJev outperforms both a 27B LLM and the commercial Jev 1.13 decision model on guideline grading by 10.4-23.0 points. RetiJev-9B further achieves the highest accuracy on clinical decisions, improving over both baselines by 5.4-6.5 points while matching them on diagnosis. It answers a case in 158 ms on one GPU, 3.5x faster than LLM generation and 470x faster than reasoning. Models are released at https://github.com/Awenbocc/RetiJev.

1. Introduction

Ophthalmic decision support needs rapid answers among predefined actions, but generative LLMs bring latency, compute, and privacy concerns. RetiJev pairs source-grounded data with candidate-scoring models and reports stronger benchmark results with fast inference.

Motivation

  • Point-of-care decisions often select among predefined actions, while autoregressive generation can add latency, require substantial resources, and raise cloud privacy concerns.The introduction also notes that free-form generation is poorly aligned with predefined clinical choices and guideline adherence remains inconsistent.Source: §1 · p. 2

Contributions

  • RetiJev-Data supplies source-grounded ophthalmic supervision, with 78,158 clinical-decision and diagnosis questions and 50,074 guideline-grading questions.The test split forms a 24,662-question benchmark.Source: §1 · p. 2

Results

  • 10.4–23.0 percentage points: RetiJev outperforms both baselines on guideline grading across all model sizes; RetiJev-9B leads clinical-decision accuracy and matches them on diagnosis.The introduction also reports clinical-decision improvements over both baselines at 2B and 9B.Source: §1 · p. 2
  • 158 ms per case: RetiJev answers on one GPU versus 546 ms for direct generation and 74.5 s for reasoning-based generation by the 27B LLM.The 0.8B model uses 3.3 GB of GPU memory and runs on eight CPU cores, enabling local deployment without transmitting patient data externally.Source: §1 · p. 2

Contributions

  • RetiJev is a 0.8B–9B family that directly scores candidate answers without autoregressive decoding.The work presents this as an alternative to generative reasoning for ophthalmic decisions.Source: §1 · p. 2

2. Related Work

Prior ophthalmic LLM work has emphasized knowledge assessment and patient queries, while efficient inference research includes both faster generation and direct prediction for predefined answers. RetiJev builds on candidate scoring and source-grounded, independently checked supervision.

LLMs for clinical knowledge and decisions

  • Exam performance does not necessarily translate into good clinical decisions: LLMs on inpatient cases failed to follow guidelines and were sensitive to information order.Much ophthalmic AI has focused on images, from disease screening to retinal foundation models and multimodal benchmarks.Source: §2 · p. 2

Decision models and efficient inference

  • Efficient inference work either accelerates autoregressive decoding or avoids generation for predefined-answer tasks through letter scoring, encoder classifiers, and Jev-style decision models.MedJev focuses on clinical variable extraction; RetiJev extends candidate scoring to multiple ophthalmic decision tasks using a shared patient representation and single prefill.Source: §2 · pp. 2–3

Synthetic data and model-verified labels

  • RetiJev grounds clinical-decision labels in explicit source evidence and verifies them across independent model families, while guideline labels are computed from published rules.Generated patient descriptions for guideline grading are independently checked against the underlying findings.Source: §2 · p. 3

3. RetiJev-Data: Source-Grounded Ophthalmic Decisions

RetiJev-Data combines source-grounded clinical decision and diagnosis questions with guideline-grading cases. Its construction uses evidence-linked rules, synthetic cases, predefined answer options, verification, and distinct benchmark sets.

3. RetiJev-Data: Source-Grounded Ophthalmic Decisions

  • RetiJev-Data combines clinical decisions and diagnosis with guideline grading; its 24,662-question test split defines RetiJev-Bench.The corpus includes 78,158 clinical-decision and diagnosis questions and 50,074 guideline-grading questions, with counts spanning training, development, and test splits.Source: §3 · p. 3, §3.3 · p. 5

3.1. Clinical Decision Tasks

  • 65,759 full-text articles were retained after retrieval and screening, and relevant material was organized into disease-specific evidence bundles with source identifiers and excerpts preserved.Clinical decision rules were extracted from literature, retaining applicability conditions, exceptions, recommendation strength, and traceable evidence; unresolved conflicts were excluded from unique-answer supervision.Source: §3.1 · p. 3, §3.1.1 · p. 3

3.1.1. FROM ARTICLES TO SYNTHETIC CASES

  • 6,303 rule families link patient findings to supported diagnoses or actions, and split-before-generation partitioning separates familiar from held-out decision principles.Synthetic cases vary decision-relevant conditions; Seen cases use training-represented principles, whereas Unseen cases use held-out principles, with related variants kept within one split.Source: §3.1.1 · pp. 3–4

3.1.2. QUESTIONS AND VERIFIED ANSWERS

  • Four action tasks use a clinician-reviewed catalog of 53 options, with preferred-action and candidate-acceptability questions preserving both best choices and reasonable alternatives.Labels are filtered and independently checked; unresolved candidates are omitted from binary questions, and disagreements are retained only under specified multi-model agreement and evidence conditions.Source: §3.1.2 · p. 4

3.2. Guideline Grading Task

  • Eleven grading systems are encoded as functions that compute labels from structured findings, with cases emphasizing values near decision thresholds.Generated notes are retained only when independent parameter re-extraction exactly recovers the label-generating parameters; additional tests use out-of-distribution note styles and harder cases.Source: §3.2 · pp. 4–5

3.3. Training Data and Test Benchmark

  • The 24,662-question test split includes six main sets for decisions, diagnosis, and grading plus two grading reference sets.Diagnosis selection is converted to a single-best-diagnosis question only when exactly one candidate is labeled the leading diagnosis.Source: §3.3 · p. 5

4. RetiJev: A Single-Pass Decision Model

RetiJev formulates ophthalmic questions as candidate-scoring decisions, using a shared pointer head and training objective across task types. At inference, it reuses one patient representation to score independent question branches in parallel.

4.1. Problem Formulation and Candidate Scoring

  • RetiJev uses a shared pointer head to score candidate answers across question types, with predictions based only on each patient state and its own question.The predicted answer is the highest-probability candidate.Source: §4 · p. 5, §4.1 · p. 5

4.2. Training Objective

  • Every question receives label supervision, while ordered ratings also use an ordinal loss for diagnosis-support ratings, urgency levels, and guideline grades.Only LoRA adapters and the pointer head are optimized; base weights remain frozen.Source: Figure 3 · p. 6, §4.2 · pp. 6–7
  • Clinical-decision training can add teacher distributions and paraphrase-consistency targets, while diagnosis-coverage augmentation adds synthetic cases from a 531-disease MeSH vocabulary.Teacher distillation is not used for guideline grading, whose targets are computed from encoded criteria.Source: §4.2 · pp. 6–7, Figure 3 · p. 6
  • Training averages question losses within each case before averaging across cases, giving equal weight to cases regardless of their question counts.Source: §4.2 · p. 7

4.3. Shared-Context Inference

  • RetiJev encodes the patient state once, then batches independent question branches using cached representations to score candidates without autoregressive decoding.This reduces processed token positions from K S + Σ B_k to S + Σ B_k.Source: §4.3 · pp. 7–8, Figure 4 · p. 7

5. Experiments

RetiJev is evaluated against a 27B LLM and Jev 1.13 on guideline grading, clinical decisions, and diagnosis, with speed, calibration, and training-objective ablations also assessed.

5.1. Setup

  • The evaluation uses micro accuracy, and pairwise differences are considered significant when the 95% confidence interval of the paired difference excludes zero.Baselines are evaluated zero-shot; the 27B LLM uses letter scoring with reasoning disabled, while Jev 1.13 is accessed through an API.Source: §5.1 · p. 8

5.2. Main Results

  • 10.4–23.0 percentage points: every RetiJev model significantly outperforms both zero-shot baselines on Grading-OOD and Grading-Hard.Accuracy ranges are 95.9%–97.3% on Grading-OOD and 89.8%–96.3% on Grading-Hard.Source: §5.2 · p. 8
  • 5.4–6.5 percentage points: RetiJev-9B improves on both baselines across Decision-Unseen and Decision-Seen; the 2B model also gains significantly.RetiJev-0.8B is comparable on Decision-Unseen and significantly outperforms both baselines on Decision-Seen.Source: §5.2 · p. 8
  • 84.8%: RetiJev-9B matches both baselines on Diagnosis-Unseen, while the smaller models remain below them on unseen diagnosis.Diagnosis-set comparisons have limited precision because both sets are small.Source: §5.2 · pp. 8–9

5.3. Speed

  • 158 ms: RetiJev-9B answers a case on one H200, versus 546 ms for Qwen generation and 74.5 s for reasoning-based generation.Latency is median per-case latency; Table 3 reports latency percentiles and peak GPU memory.Source: §5.3 · p. 9, Table 3 · p. 9

5.4. Calibration and Deferral

  • At 80% coverage, RetiJev-0.8B and RetiJev-9B reach 87.8% and 92.1% accuracy, while deferring 20% containing 51% and 58% of their errors.The models' Decision-Unseen ECE values are 0.033 and 0.016, respectively.Source: §5.4 · p. 9

5.5. Ablation of the Training Objective

  • 79.7%: teacher distillation raises Decision accuracy from 77.8% and reduces ECE from 0.186 to 0.045; adding paraphrase consistency yields 80.3% Decision accuracy and 0.036 ECE.Paraphrase consistency also raises Diagnosis accuracy to 68.9%, while Grading returns to 95.9%.Source: §5.5 · p. 9, Table 4 · p. 9

6. Limitations

The evaluation has not established clinical validity: decision labels are not clinician-adjudicated, patient states are synthetic, and prospective evaluation has not been conducted.

6. Limitations

  • Decision labels are source-grounded and cross-verified but not clinician-adjudicated, all patient states are synthetic, and RetiJev has not been evaluated prospectively.The authors state that clinician-labelled, real-world evaluation is needed before clinical use.Source: §6 · p. 9

7. Conclusion

RetiJev uses candidate scoring rather than text generation for ophthalmic clinical decisions. Its supervision supports strong rule application and clinical action selection, while disease recognition is more sensitive to model scale and disease coverage.

7. Conclusion

  • Direct decision scoring provides an efficient alternative to generative inference for structured clinical decisions.Disease recognition remains more sensitive to model scale and disease coverage than rule application and clinical action selection.Source: §7 · p. 10

Impact Statement

The paper presents compact decision models as a potential route to faster, more local ophthalmic decision support, particularly where specialist availability or connectivity is limited. It also emphasizes risks from errors, automation bias, and distribution shift, and characterizes RetiJev as a research prototype requiring independent clinical validation.

Impact Statement

  • Compact decision models could reduce cloud dependence, improve responsiveness, and support access to structured ophthalmic knowledge where specialist availability or connectivity is limited.The authors caution that incorrect or overconfident recommendations, automation bias, and distribution shift beyond the synthetic cases and source material remain risks.Source: Impact Statement · p. 10

A. Action Catalog

  • Action questions use case-specific candidates; urgency is ordinal, while diagnosis questions include candidate ratings and, when one diagnosis leads, a single-best-diagnosis choice.The action catalog supplies the options from which relevant candidates are selected.Source: A. Action Catalog · p. 13, Table 5 · p. 13

B. Guideline Grading Systems

  • The guideline benchmark covers 11 systems, with cases sampled near clinical thresholds, labels computed from published rules, and notes rendered in standard and paraphrased styles.Generated notes are retained only when extracted clinical parameters match the original case exactly.Source: B. Guideline Grading Systems · pp. 13–14, Table 6 · p. 14

C.1. Source coverage and rule extraction

  • Clinical-decision rules are extracted from relevant article passages, consolidated into 6,303 families, and split into training, development, and held-out pools.Seen cases use training rule families; unseen cases use held-out families.Source: §C.1 · p. 15

C.2. Clinical-decision generation and verification

  • Generated clinical-decision questions are cross-family verified against source evidence, and unresolved disagreements are excluded rather than labeled negative.Retained questions require a direct supporting quote and no insufficient-evidence judgment.Source: §C.2 · p. 15

C.3. External-disease distillation

  • Diagnosis training combines synthetic cases based on 531 MeSH eye diseases with source-derived diagnosis-choice cases; development and test texts are withheld from the teacher.Synthetic cases are retained only under the stated diagnosis-agreement and text-similarity criteria.Source: §C.3 · p. 15

D. Latency Measurement Protocol

  • Latency is measured end-to-end per case at batch size 1 on development data; shared-state encoding keeps RetiJev-0.8B latency from 138 ms to 146 ms as K rises from one to twenty.Independently processed questions reach approximately 656 ms for the 0.8B backbone and 1,747 ms for Qwen3.8-27B; Figure 5 compares latency as question count grows.Source: D. Latency Measurement Protocol · pp. 15–16, Figure 5 · p. 16

E. Test Set Details; F. Guideline Grading in Detail

G. Data Quality Criteria

Clinical-decision instances are filtered for source fidelity, label consistency, answer leakage, and well-formed questions. Labeling preserves soft recommendations and requires explicit support for leading diagnoses.

G. Data Quality Criteria

  • Instances are rejected when rule quotes lack verbatim page support, candidates or label sets are invalid, answers leak into states, or text violates language and length requirements.Near-verbatim quotes may be repaired when similarity is at least 0.9; states must be English and 20–200 words.Source: G. Data Quality Criteria · p. 17
  • Choice questions must contrast preferred and inappropriate options, urgency questions must have one level, and leading diagnoses require a verbatim supporting phrase.A possible diagnosis without that evidence is relabelled as not supported or excluded.Source: G. Data Quality Criteria · p. 17
  • Soft recommendation language such as “may” and “can be considered” is preserved when assigning preferred and acceptable labels under explicit rules.Source: G. Data Quality Criteria · p. 17

H. Robustness to Wording

The grading pilot tests whether paraphrase augmentation improves robustness to wording shifts while preserving performance on standard wording. The final RetiJev models use the augmentation.

H. Robustness to Wording

  • The pilot compares two 0.8B models, with and without one paraphrase per training case, on held-out cases rendered by familiar and unseen generators.The two training conditions preserve the same underlying cases and labels; questions are restricted to those answered by all four systems.Source: H. Robustness to Wording · p. 17, Table 10 · p. 18
  • The standard and paraphrase renderings use DeepSeek-V4.1-Flash, while Qwen3.8-Flash supplies the unseen-generator paraphrases.The 27B and Jev 1.13 systems are zero-shot baselines, and the pilot models are distinct from the final models in Table 2.Source: Table 10 · p. 18
  • Paraphrase augmentation improves accuracy under both wording shifts while preserving in-distribution performance, and all final RetiJev models use it.Standard-only training performs worse than either baseline under both shifts despite high accuracy on familiar wording.Source: H. Robustness to Wording · p. 18

I. Additional Analyses

Additional analyses report task-specific Decision-Unseen comparisons, adaptation-setting results, and a task-imbalance caveat for generator-family comparisons. The input encoding also isolates question branches while preserving their causal context.

I. Additional Analyses

  • For management direction, 0.8B significantly outperforms both baselines; for plan safety, 9B significantly outperforms both, while urgency and referral samples are insufficient for reliable comparisons.Investigation follows the pattern that 0.8B outperforms Jev 1.13 and performs comparably to the 27B model; 9B follows the same pattern.Source: Table 11 · p. 18
  • Rank 64 at a learning rate of 3 × 10−5 has the highest development accuracy among tested adaptation settings, while higher ranks or full fine-tuning do not by themselves improve accuracy.The selected 0.8B configuration is applied unchanged to the 2B and 9B models.Source: §I · p. 18, Table 12 · p. 18
  • On 2,221 DeepSeek-written Decision-Unseen questions, 0.8B scores 85.0% versus 82.3% for 27B and 82.8% for Jev 1.13; on 2,404 Qwen-written questions, scores are 76.0%, 77.0%, and 76.5%, respectively.The 9B model scores 88.6% and 81.5% on those groups; because the Qwen-written group contains all 961 per-candidate diagnosis questions, task imbalance prevents attributing the difference to model-family bias alone.Source: §I · p. 18

J.1. Input Encoding and Branch Isolation

  • Each question branch restarts position indices after the state, and a separate copy of the cached attention and recurrent states gives it the causal context of an independent state–question sequence.The Qwen3.5 DeltaNet layers carry recurrent information that an attention mask alone cannot isolate; median case length is 120 state tokens and 2.57 questions per case.Source: §J.1 · p. 19
  • The pointer head projects final hidden states for question and option delimiters, supports 2–255 options, and training reshuffles choice options each epoch.Source: §J.1 · p. 19

J.2. Training Objectives

Training combines question-type-specific supervision with consistency targets for matching paraphrased clinical-decision questions. The shared training set covers clinical actions, diagnosis selection, and guideline grading.

J.2. Training Objectives

  • Cross-entropy applies to every question, while ordinal supervision also applies to score questions such as diagnosis-support ratings, urgency levels, and ordinal guideline grades.Source: §J.2 · p. 19
  • Distillation applies only to clinical-decision questions when the teacher distribution covers every current candidate, with candidate identities aligning teacher probabilities to student logits.Teacher probabilities are clamped below at 10−6, raised to 1/τ, and normalized; paraphrases reuse their original case’s teacher distribution.Source: §J.2 · p. 19
  • Consistency supervision uses a detached moving average of prior predictions only when a different rendering has matching candidate identities and order.A missing matching target is initialized with the current detached prediction; paraphrase consistency is restricted to clinical-decision cases, and neither consistency nor distillation applies to deterministically computed grading labels.Source: §J.2 · p. 19
  • Active loss terms are averaged over questions within each case and then over cases.Source: §J.2 · p. 19

J.3. Training Supervision

  • The shared training set covers clinical actions and diagnostic support, most-likely diagnosis, and guideline criteria, including source-derived cases and synthetic diagnosis cases.Diagnosis selection uses 6,882 questions from source-derived cases and 6,933 from synthetic cases covering 531 external disease terms; guideline labels come from 11 encoded systems and rules.Source: §J.3 · p. 20, Table 14 · p. 20

K. Examples of Every Task

The examples demonstrate how RetiJev frames ophthalmic cases as decisions among predefined urgency, management, diagnosis, referral, investigation, and grading options. Each case pairs clinical findings with a supported answer or grading label.

K. Examples of Every Task

  • A cherry-red spot with sudden painless vision loss calls for immediate emergency management, while improving dysthyroid optic neuropathy calls for expedited evaluation within days.The examples apply urgency categories to distinct clinical states.Source: K. Examples of Every Task · p. 20
  • Referral examples direct persistent isotretinoin-related eye symptoms to general ophthalmology and infant retinal detachment with nonclearing vitreous hemorrhage to retina subspecialty care.The infant case notes that vitrectomy may be needed and often requires consultation with a pediatric retina-trained surgeon.Source: K. Examples of Every Task · pp. 20–21
  • For suspected traumatic intraocular foreign body or retinal pigment epithelial tear, the examples accept imaging options including CT or MRI, ocular ultrasound, OCT, and angiography.The trauma case accepts CT or MRI and ocular ultrasound; the RPE-tear case prefers OCT while also accepting angiography.Source: K. Examples of Every Task · p. 21
  • Diabetic macular edema with ischemia favors injection therapy, while persistent epiblepharon symptoms with unassessed keratopathy call for completing assessment before deciding.Both examples ask for a management direction rather than a specific procedure.Source: K. Examples of Every Task · pp. 21–22
  • Plan-safety examples reject atropine for traumatic hyphema with an occludable angle and defer LASIK while dry-eye findings remain unchanged.The cases distinguish contraindication from deferral pending improvement.Source: K. Examples of Every Task · p. 22
  • The differential examples rank preseptal cellulitis as leading over orbital cellulitis and identify Sorsby macular dystrophy as the leading inherited diagnosis.The Sorsby case also rates pattern dystrophy as possible.Source: K. Examples of Every Task · p. 23
  • The ICDR example labels the right eye moderate NPDR and the left severe NPDR, with central-involving edema only in the left eye and referral to ophthalmology.The case provides eye-specific findings and grading labels.Source: K. Examples of Every Task · p. 23
  • Other grading examples cover inactive thyroid eye disease by CAS, ETROP treatment status, Chandler group III infection, SUN cell grade, and EVS scope.The EVS example falls outside the study population, so its rule does not directly apply.Source: K. Examples of Every Task · pp. 23–24

Related papers

Loading 2610.30947v1…