Source-linked AI summary
OralGPT-Omni: A Versatile Dental Multimodal Large Language Model
Jing Hao, Yuci Liang, Lizhuo Lin, Yuxuan Fan, Wenkai Zhou, Kaixin Guo, Zanting Ye, Yanpeng Sun, Xinyu Zhang, Yanqi Yang, Qiankun Li, Hao Tang, James Kit-Hon Tsoi, Linlin Shen, Kuo Feng Hung
TL;DR
Dentistry lacks reliable, transparent MLLM analysis and broad multimodal evaluation. OralGPT-Omni combines dental-specific data, TRACE-CoT reasoning, and staged training, and is reported to outperform current state-of-the-art MLLMs on MMOral-Uni and MMOral-OPG.
Problem
Dental MLLMs face limited domain-specific data and expert annotation, modality-specific modeling gaps, and reliability and transparency concerns.
Source: §1 · pp. 1–2Method
OralGPT-Omni combines TRACE-CoT diagnostic reasoning with a four-stage training strategy and introduces the MMOral-Uni benchmark.
Source: §1 · pp. 2–3, §2.2 · p. 4Results
Takeaways & Limitations
The work provides a unified dental multimodal benchmark and explicit diagnostic reasoning chains for dental image analysis.
Source: §5 · p. 9Report generation lags behind GPT-4V, which the authors hypothesize may reflect difficulty capturing structures and multidimensional conditions.
Source: §4.3 · p. 7
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have exhibited immense potential across numerous medical specialties; yet, dentistry remains underexplored, in part due to limited domain-specific data, scarce dental expert annotations, insufficient modality-specific modeling, and challenges in reliability. In this paper, we present OralGPT-Omni, the first dental-specialized MLLM designed for comprehensive and trustworthy analysis across diverse dental imaging modalities and clinical tasks. To explicitly capture dentists' diagnostic reasoning, we construct TRACE-CoT, a clinically grounded chain-of-thought dataset that mirrors dental radiologists' decision-making processes. This reasoning supervision, combined with our proposed four-stage training paradigm, substantially strengthens the model's capacity for dental image understanding and analysis. In parallel, we introduce MMOral-Uni, the first unified multimodal benchmark for dental image analysis. It comprises 2,809 open-ended question-answer pairs spanning five modalities and five tasks, offering a comprehensive evaluation suite to date for MLLMs in digital dentistry. OralGPT-Omni achieves an overall score of 51.84 on the MMOral-Uni benchmark and 45.31 on the MMOral-OPG benchmark, dramatically outperforming the scores of GPT-5. Our work promotes intelligent dentistry and paves the way for future advances in dental image analysis. All code, benchmark, and models will be made publicly available.
1. Introduction
Dentistry lacks reliable, transparent MLLM analysis and sufficiently broad evaluation, amid scarce expert annotations and heterogeneous imaging. OralGPT-Omni combines dental-specific data, TRACE-CoT reasoning, and staged training; MMOral-Uni adds a unified benchmark.
1. Introduction
- Dentistry research faces limited and inconsistent datasets, modality heterogeneity, scarce expert annotation, and insufficiently transparent or reliable MLLM responses.Existing dental MLLMs are described as lacking consistency, completeness, and clarity, with hallucinated responses also reported.Source: §1 · pp. 1–2
- OralGPT-Omni targets comprehensive analysis across eight dental imaging modalities and five tasks, producing explicit diagnostic rationales that mirror clinical workflows.The paper presents the rationales as improving transparency and interpretability.Source: §1 · p. 2
- MMOral-Uni provides 2,809 open-ended question–answer pairs across five modalities and five tasks, with all pairs validated and refined by two experienced dentists.The benchmark is presented as a unified evaluation suite for dental multimodal imaging analysis.Source: §1 · p. 3
- 51.84 on the Moral-Omni benchmark and 45.31 on MMOral-OPG are reported for OralGPT-Omni, surpassing GPT-5.The introduction reports these overall scores as evidence of performance on both dental benchmarks.Source: §1 · p. 3
2.1. Training Corpus Construction
- The training corpus combines about 3.21 million text tokens, 59,658 images, and 90 videos across eight modalities, five tasks, and samples from at least ten countries.It draws on 31 public datasets and one dental-hospital dataset; dentists refine annotations from datasets rated moderate or high risk of bias.Source: §2.1 · pp. 3–4
2.2. TRACE-CoT Data Generation
- TRACE-CoT structures diagnosis as image inspection, hypothesis generation, clinical-knowledge reference, feature-based verification, and an evidence-informed conclusion.The authors generated 36,777 chains for three imaging modalities and had two dentists evaluate 300 instances.Source: §2.2 · p. 4
2.3. Model & Training Strategy
- Training proceeds through dental-knowledge injection, image-caption alignment, full-model supervised fine-tuning, and final reinforcement-learning tuning.The RLT stage selects medium-difficulty cases and uses TRACE-based, answer, and format rewards.Source: §2.3 · p. 5
3. MMOral-Uni Benchmark
MMOral-Uni is designed to evaluate dental MLLMs across varied imaging and clinical tasks. Its open-ended QA pairs undergo dentist review, and model outputs are scored by an LLM judge using few-shot examples.
3.1. Benchmark Composition
- MMOral-Uni contains 2,809 open-ended QA pairs spanning five dental imaging modalities and five tasks.The tasks include abnormality diagnosis, CVM staging, treatment planning, tooth localization and counting, and treatment-video comprehension.Source: §3.1 · p. 5
3.2. Evaluation Metrics
- Two experienced dentists validate and refine the benchmark QA pairs, while GPT-5-mini scores open-ended predictions from 0 to 1 using five in-context examples.The evaluation reports scores by modality and overall; the authors also describe ablation experiments as validating the LLM-judge approach's feasibility and stability.Source: §3.1 · p. 5, §3.2 · p. 6
4. Experiments
Experiments compare OralGPT-Omni across MMOral-Uni and MMOral-OPG and ablate its staged training and TRACE-CoT. It leads overall on both benchmarks, while treatment planning and report generation are weaker areas.
4.1. Experimental Setup
- The evaluation covers 27 MLLMs on MMOral-Uni and also assesses OralGPT-Omni on MMOral-OPG; training uses Qwen2.5-VL-7B-Instruct and four stages.Training used two NVIDIA A100 80G GPUs for approximately 90 hours.Source: §4.1 · p. 6
4.2. Comparisons on MMOral-Uni Benchmark
- 51.84 overall on MMOral-Uni, significantly above GPT-5’s 15.42, with OralGPT-Omni highest among the evaluated models.The benchmark results are summarized across dental imaging modalities.Source: §4.2 · p. 6, Table 1 · p. 7
4.3. Comparisons on MMOral-OPG Benchmark
- 45.31 overall on MMOral-OPG; OralGPT-Omni exceeds GPT-4V on five dimensions but lags in report generation.The five dimensions are Teeth, Patho, His, Jaw, and Summ.Source: Table 2 · p. 7, §4.3 · p. 7
4.4. Ablation and In-Depth Study
- Case studies show stronger abnormality diagnosis than GPT-5 and Lingshu-7B in intraoral, periapical X-ray, and histopathological images, with explicit TRACE-CoT reasoning.Clinical assessments of responses from three leading models were conducted by a radiologist with over ten years of experience.Source: Figure 6 · p. 8, §4.4 · pp. 8–9
- The four-stage strategy raises MMOral-Uni from 22.88 to 24.00 after DKI/DCT, to 48.67 after SFT, then RLT adds 3.17 points.SFT strengthens instruction following and prepares the model for explicit reasoning in RLT.Source: Table 3 · p. 8, §4.4 · p. 8
- TRACE-CoT during SFT raises the overall score by 4.36 points versus answer-only training, with largest gains in II-Dx-I, II-Dx-R, and PI.TRACE-CoT annotations were included only for II-Dx-I, II-Dx-R, PA, and PI.Source: §4.4 · p. 8, Table 3 · p. 8
5. Conclusion
OralGPT-Omni combines dental-specific multimodal data, TRACE-CoT reasoning supervision, and four-stage training; the paper also introduces MMOral-Uni as a unified evaluation suite for digital dentistry.
5. Conclusion
- The conclusion presents OralGPT-Omni as a dental-specialized MLLM and MMOral-Uni as a comprehensive evaluation suite for multimodal dental imaging.The model combines dental-specific data, TRACE-CoT reasoning, and four-stage training.Source: §5 · p. 9
1. Related Works
- Prior dental MLLMs cover narrower modalities or provide limited explanations, while TRACE-CoT is designed to follow radiologists’ diagnostic decision-making.Earlier medical reasoning approaches include reinforcement learning and synthesized reasoning data; the paper positions TRACE-CoT as a clinically aligned alternative.Source: §1 · p. 12
2. Details on Training Data Construction
- The TRACE-CoT pipeline uses sparse label categories, abnormality appearance, and dental knowledge from publications and Wikipedia to instruct GPT-5-mini, generating 36,777 reasoning chains with dataset-specific prompts tailored to differing taxonomies.The training corpus contains approximately 3.21 million text tokens, 59,658 images, and 90 videos.Source: §2 · p. 12
3.1. Hyperparameters of four-stage training paradigm
- Training progresses through four stages—DKI, DCA, SFT, and RLT—to strengthen multimodal understanding and reasoning.The first three stages use LLaMA-Factory; RLT uses ms-swift.Source: §3.1 · p. 13
3.2. Reinforcement Learning Tuning
- GRPO tuning compares multiple sampled outputs for each prompt using group-relative advantages, without a separate value function.The reference policy is the preceding SFT model with the TRACE-CoT pattern.Source: §3.2 · pp. 13–14
3.3. Reward Computation for GRPO
- The TRACE-based reward scores factual soundness, logical coherence, and answer consistency, complementing answer and format rewards.TRACE-based reward is averaged across its three dimensions and contributes only when the answer is not completely wrong.Source: §3.3 · pp. 14–15
4. MMOral-Uni Benchmark
MMOral-Uni is a unified benchmark for dental multimodal imaging, covering five modalities and five dental tasks through open-ended question–answer pairs.
4. MMOral-Uni Benchmark
- MMOral-Uni contains 2,809 open-ended question–answer pairs spanning five modalities and five tasks.The pairs are intended to simulate realistic user interactions.Source: §4 · p. 15
- Its modalities include intraoral photographs, periapical and cephalometric radiographs, pathological images, intraoral videos, and interleaved image–text inputs.The tasks cover diagnosis, CVM stage prediction, treatment planning, tooth localization and counting, and dental treatment video comprehension.Source: §4 · p. 15
- Abnormality diagnosis spans 40 categories, with task and modality quantities detailed in Table 10.The benchmark also includes example items in Figures 25–29.Source: §4 · p. 15
5. LLM as the judges for MMOral-Omni: A Feasibility Analysis
Repeated evaluations indicate stable LLM-judge scores, while dentist comparisons show broad agreement on overall performance alongside subjective differences in some subcategories.
Effectiveness
- Approximately 2 points: dentists’ differences from LLM-based “Overall” scores were around this size for GPT-5 and Lingshu-7B.Dentist B’s overall-score difference was 2.43 points.Source: §5 · p. 15
- Dentist–LLM score differences varied by subcategory, with larger differences in TP and IV for Dentist A and generally larger differences for Dentist B.The authors associate subcategory fluctuations with subjective evaluator perceptions of the scoring criteria.Source: §5 · p. 15
Stability
- 0.212 or less: overall-score standard deviations across four LVLMs over five GPT-5-mini evaluations did not exceed this value.GPT-5 scores had a standard deviation of 0.179, and Qwen3-VL-8B scores had 0.039.Source: §5 · p. 15
6. Case Study and Clinical Validity of OralGPT-Omni
The paper presents case studies and a radiologist-led clinical assessment of leading models, reporting superior performance and potential clinical value for OralGPT-Omni.
6. Case Study and Clinical Validity of OralGPT-Omni
- The clinical assessment compared OralGPT-Omni with GPT-5 and Lingshu-7B across four representative modalities.Clinical validation examples are shown in Figures 33–36.Source: §6 · p. 16
- Additional case studies illustrating model outputs are presented in Figures 30–32.The authors frame these examples as evidence of OralGPT-Omni’s superiority.Source: §6 · p. 16
- The clinical assessment describes OralGPT-Omni’s outputs as demonstrating superior performance and potential clinical value.The assessment was conducted by an oral-maxillofacial radiology faculty member with over ten years of diagnostic imaging experience.Source: §6 · p. 16
1. Logical Coherence
Logical Coherence evaluates whether the reasoning in <Think> is consistent with <Caption>, including whether its steps contain unsupported jumps or contradictions.
1. Logical Coherence
- The evaluation checks whether reasoning in <Think> steps 1–3 is logically consistent with the information in <Caption>.It looks for jumps, contradictions, and illogical conclusions.Source: §1 · p. 34
- A score of 0 indicates a clear logical inconsistency or contradiction.Source: §1 · p. 34
- A score of 1 indicates fully coherent reasoning.Source: §1 · p. 34
2. Factual Knowledge Accuracy
Factual Knowledge Accuracy assesses the professional knowledge in step 2 of <Think> against established oral medicine or pathology facts.
2. Factual Knowledge Accuracy
- The evaluation assesses whether step 2 of <Think> presents professional knowledge that is correct, complete, and consistent with established oral medicine or pathology facts.Source: §2 · p. 34
- A score of 0 indicates incorrect or seriously inaccurate professional knowledge.Source: §2 · p. 34
- Scores of 1 and 2 indicate partially correct and fully correct knowledge, respectively.Source: §2 · p. 34
3. Answer Consistency
Answer Consistency checks whether the reasoning trace in step 3 justifies the gold-standard answer without contradiction or unsupported information.
3. Answer Consistency
- The evaluation checks whether the reference answer can be directly derived from the description in step 3 of <Think>.Source: §3 · p. 34
- It checks that the reasoning trace fully justifies the reference without contradictions, unstated information, or unsupported claims.Source: §3 · p. 34
- Scores of 0, 1, and 2 indicate inaccurate, partially accurate, and fully accurate reasoning, respectively.Source: §3 · p. 34
4. Overall Thinking Reward
The overall thinking reward scores clinical equivalence between a model answer and a gold-standard reference, then assesses semantic or diagnostic equivalence of their text.
4. Overall Thinking Reward
- The reward assigns a 0–1 score for clinical equivalence between the Candidate answer and the Reference.Source: §4 · p. 35
- The Reference may include bounding boxes, tooth numbers, or a diagnostic description.Source: §4 · p. 35
- The content score measures semantic or diagnostic equivalence; minor omissions incur a small penalty, while contradictions or false positives incur a large penalty.Source: §4 · p. 35
2. Box/Tooth Score (0–1)
This material presents an answer-reward prompt and examples from MMOral-Uni, pairing dental images and task-specific questions with dentist-validated ground truths.
2. Box/Tooth Score (0–1)
- Figure 24 presents the prompt used to compute answer reward with GPT-5-nano.Source: Figure 24 · p. 35
MMOral-Uni Benchmark
- Benchmark questions include tooth counting and localization, disease identification, oral pathology classification, and cephalometric landmark identification with CVM staging.Source: MMOral-Uni Benchmark · pp. 36, 37, 38, 41, 46, 47
- MMOral-Uni example figures pair images with questions, categories, and ground truths validated by experienced dentists.Source: Figure 25 · p. 36