PaperlayerView exact revision on arXiv

Source-linked AI summary

DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice

Zijie Meng, Jin Hao, Xiwei Dai, Yang Feng, Jiaxiang Liu, Bin Feng, Huikai Wu, Xiaotang Gai, Hengchuan Zhu, Tianxiang Hu, Yangyang Wu, Hongxia Xu, Jin Li, Jun Xiao, Xiaoqiang Liu, Joey Tianyi Zhou, Fudong Zhu, Zhihe Zhao, Lunguo Xia, Bing Fang, Jimeng Sun, Jian Wu, Zuozhu Liu

arXiv:2509.23344v1cs.CVcs.AI

TL;DR

Dental practice requires multimodal assessment, but many existing tools address isolated tasks and specialized dental VLM applications remain nascent. DentVLM combines bilingual dental image-question data with staged training for 36 tasks across seven modalities. It outperformed leading models and showed diagnostic and workflow benefits in dentist studies and practical scenarios.

  • Problem

    Existing digital dental tools often address diagnosis or care delivery in isolation, while comprehensive practice requires synthesis across imaging modalities and clinical information.

    Source: §1 · p. 2

  • Method

    DentVLM is a dental vision-language model trained in two stages on bilingual dental data to address 36 diagnostic tasks across seven imaging modalities.

    Source: §1 · p. 2, §3 · p. 7, §4.4 · p. 16

  • Results

    DentVLM exceeded leading models by 19.61% in oral-disease accuracy and 27.91% in malocclusion accuracy, and surpassed junior dentists on 21 of 36 tasks and senior dentists on 12.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors position DentVLM as a clinical decision-support tool with potential to enhance primary dental care and access to specialized expertise.

    Source: §3 · p. 9

    Broader generalizability requires additional imaging modalities and larger, higher-quality datasets, especially datasets with longitudinal annotations.

    Source: §3 · pp. 8–9

Abstract

from arXiv · show

Diagnosing and managing oral diseases necessitate advanced visual interpretation across diverse imaging modalities and integrated information synthesis. While current AI models excel at isolated tasks, they often fall short in addressing the complex, multimodal requirements of comprehensive clinical dental practice. Here we introduce DentVLM, a multimodal vision-language model engineered for expert-level oral disease diagnosis. DentVLM was developed using a comprehensive, large-scale, bilingual dataset of 110,447 images and 2.46 million visual question-answering (VQA) pairs. The model is capable of interpreting seven 2D oral imaging modalities across 36 diagnostic tasks, significantly outperforming leading proprietary and open-source models by 19.6% higher accuracy for oral diseases and 27.9% for malocclusions. In a clinical study involving 25 dentists, evaluating 1,946 patients and encompassing 3,105 QA pairs, DentVLM surpassed the diagnostic performance of 13 junior dentists on 21 of 36 tasks and exceeded that of 12 senior dentists on 12 of 36 tasks. When integrated into a collaborative workflow, DentVLM elevated junior dentists' performance to senior levels and reduced diagnostic time for all practitioners by 15-22%. Furthermore, DentVLM exhibited promising performance across three practical utility scenarios, including home-based dental health management, hospital-based intelligent diagnosis and multi-agent collaborative interaction. These findings establish DentVLM as a robust clinical decision support tool, poised to enhance primary dental care, mitigate provider-patient imbalances, and democratize access to specialized medical expertise within the field of dentistry.

1 Introduction

Dental care faces major access inequities, while existing digital tools often address diagnosis and care delivery separately rather than supporting multimodal practice. DentVLM is introduced as a bilingual model for dental diagnosis, with reported advantages over other models and potential clinical utility.

1 Introduction

  • About 3.5 billion people are affected by oral diseases, while dentist-to-population ratios are approximately 1:360,000 in LMICs and 1:2,000 in high-income countries.The passage links constrained access with delayed diagnoses, suboptimal management, and ineffective referral pathways.Source: §1 · p. 2
  • Existing oral-health assessments can be inefficient and variable, while digital tools often handle diagnosis or care delivery in isolation rather than integrating multimodal information.Dental practice draws on clinical photographs, radiographs, and patient histories; the passage notes that reliance on scarce clinical expertise further restricts digital tools.Source: §1 · p. 2
  • DentVLM targets 36 diagnostic tasks across seven 2D oral imaging modalities, using a large bilingual dataset and a two-stage training strategy.The dataset contains 110,447 images and over 2.4 million VQA pairs; training begins with vision-language alignment and then instruction tuning.Source: §1 · p. 2
  • DentVLM exceeded leading models by 19.61% in oral-disease accuracy and 27.91% in malocclusion accuracy, and showed benefits in dentist collaboration and practical-use scenarios.In a study of 25 dentists, it surpassed junior dentists on 21 of 36 tasks and senior dentists on 12; collaboration reduced diagnostic time by 15–22%.Source: §1 · p. 2
  • The authors present DentVLM as a clinical decision-support tool with potential to improve primary dental care and access to specialized expertise.They also describe the work as a framework for specialty-specific models integrating imaging and clinical workflows.Source: §1 · p. 2

2 Results

DentVLM combines a broad dental task and image scope with staged VQA training, then evaluates benchmark performance, generalizability, clinical collaboration, and practical deployment scenarios.

2.1 Problem formulation and dataset

  • The task matrix covers 36 dental conditions across seven imaging modalities, and the dataset includes 2.46 million bilingual VQA pairs.The conditions are grouped into oral diseases and malocclusions, with image modalities assigned by a senior-dentist consensus framework.Source: §2.1 · p. 3

2.2 Model overview and experimental setup

  • DentVLM generates answers, rationales, and optional anatomical locations through simple-VQA alignment followed by rationale- and location-aware instruction tuning.The bilingual test set covered all 36 tasks and included 910 patients and 34,942 VQA pairs; a separate clinical set included 1,946 patients and 3,105 QA pairs.Source: §2.1 · p. 3, §2.2 · pp. 3–4

2.3 Overall performance of DentVLM and baselines

  • DentVLM surpassed baselines across 34 bilingual tasks and beat the best baseline by 23.76% on average across four settings.For disease localization, mean IoU was 38.44% in English and 40.31% in Chinese, exceeding GPT-4o by 6.85% and 4.8%, respectively.Source: §2.3 · p. 4

2.4 Model generalizability and ablation analysis

  • DentVLM variants based on Qwen2-VL-2B, Qwen2-VL-7B, and Gemma3-4B each outperformed GPT-4o by more than 25% across four test settings.Gemma3-4B was slightly worse on malocclusion tasks but surpassed the default version by 6.45% on oral-disease tasks; increasing training data improved malocclusion and Chinese-context performance, while unseen rare conditions reached 52.03%–64.44% average accuracy.Source: §2.4 · pp. 4–5
  • Ablations found that full-parameter fine-tuning and mixed oral-disease/malocclusion training outperformed alternatives, while bilingual training performed best and results remained robust to resolution and instruction phrasing.Training only on either diagnostic category failed to achieve competitive performance; English-only or Chinese-only training had smaller differentials than bilingual training.Source: §2.4 · p. 5
  • Accuracy was comparable between English and Chinese across modalities, but 1–5 tasks per modality fell below 70%, and some panoramic-X-ray diagnoses remained weak.Calculus, caries, and midline deviation were examples of tasks with low accuracy on panoramic X-rays.Source: §2.4 · p. 5

2.5 Clinical study and human evaluation

  • DentVLM achieved 78.26% mean accuracy on oral disease-related tasks, significantly exceeding junior dentists (P < 0.001), while AI answers and rationales improved both junior and senior dentists’ accuracy and efficiency.With answers and rationales, oral-disease accuracy rose from 72.25% to 77.90–79.29% for juniors and from 77.23% to 81.43–81.62% for seniors (all P < 0.001); assistance enhanced efficiency for both cohorts (all P < 0.001).Source: §2.5 · p. 6

2.6 Practical utility of DentVLM in real-world deployment

  • Three practical evaluations covered home health management, hospital orthodontic diagnosis, and multi-agent interaction, with diagnosis matching scores of 78.22% and 85.33% under majority voting in the first two settings.The multi-agent interaction evaluation received scores above 3 across seven dimensions, with fairness, acceptability, and readability approaching 4.Source: §2.6 · p. 7

3 Discussion

The discussion frames DentVLM as a unified dental model spanning seven imaging modalities and interprets its dentist comparisons, assisted-use trade-offs, practical scenarios, and remaining validation needs.

3 Discussion

  • The two-stage strategy combines 2.37 million simple VQA pairs for alignment with 88,616 rationale-enhanced pairs for refinement, and ablations support using both stages.The authors present this as a way to limit the costly burden of extensive expert annotation.Source: §3 · p. 7
  • DentVLM unifies seven major 2D dental imaging modalities, addressing prior research’s narrower, task-tailored focus and improving on general-purpose and medical-domain VLMs in dentistry.The discussion positions this breadth as a step toward general-purpose dental analysis and clinical deployment.Source: §3 · pp. 7–8
  • DentVLM underperformed dentists on only 4/17 oral-disease tasks against juniors and 8/17 against seniors, while outperforming them on 8/19 and 3/19 malocclusion tasks, respectively.The task-level comparisons suggest particular promise for orthodontic applications.Source: §3 · p. 8
  • DentVLM assistance degraded dentists’ performance in only 3/36 tasks, but rationale-plus-answer assistance underperformed answer-only assistance for juniors on 16/36 and seniors on 14/36 tasks.Rationale review also increased diagnosis duration across most tasks; the authors suggest adapting rationale delivery to task complexity and expertise.Source: §3 · p. 8
  • Home-based health management achieved a diagnosis matching score of up to 93.91%, while intelligent hospital diagnostics achieved 88.14%.The multi-agent framework’s interaction records scored over 3 and approached 4 in dentists’ subjective evaluation.Source: §3 · p. 8
  • Broader generalizability will require additional modalities, larger higher-quality datasets with longitudinal annotations, and multi-center trials across diverse populations.The authors also identify clinical text, structured knowledge bases, and 3D imaging as needed integrations for expanded capabilities.Source: §3 · p. 9
  • The authors conclude that DentVLM demonstrates the potential of specialty-specific multimodal models for diverse dental needs and provides a framework for other specialties with integrated imaging and complex workflows.They describe possible use across expertise levels and healthcare settings, including resource-limited regions.Source: §3 · p. 9

4 Methods

DentVLM was built from a large bilingual dental dataset, uses a two-stage training pipeline, and was evaluated across diagnostic tasks and practical clinical scenarios. The methods also describe its architecture, evaluation sets, and ethical approvals.

4.2 Training dataset construction

  • 20,741 patients, 110,447 images, and 2,458,084 bilingual VQA pairs from 31 Chinese provinces formed the training dataset for 36 tasks.Oral-disease and malocclusion data used distinct image and annotation pipelines; six expert dentists reviewed diagnoses and localization for a subset, while 28 orthodontic technicians corrected malocclusion pairs. Training, validation, and test data were anonymized and authorized, and the study received institutional ethics approval.Source: §4.2 · pp. 15–16, §4.1 · p. 15

4.3 Details of DentVLM architecture

  • DentVLM uses a Vision Transformer encoder and a 7B-parameter Qwen2 language decoder, removing the conventional connector and using multimodal rotational position embedding.Dynamic-resolution image encoding and a patch merger that combines adjacent 2×2 tokens are used to limit resizing degradation and reduce resource overhead.Source: §4.3 · p. 16

4.4 Two-stage training pipeline of DentVLM

  • Training proceeds from answer-only vision-language alignment on roughly 2.4 million low-resolution image-question pairs to instruction tuning on about 88.6k rationale-and-location samples.In stage two, dentist-annotated and double-validated images are encoded at original resolution, the vision encoder is frozen, and the language decoder is updated.Source: §4.4 · pp. 16–17

4.5 Downstream evaluation details

  • The bilingual test set covers all 36 tasks, while the filtered clinical set contains 3,105 QA pairs from 1,946 patients.The clinical set was derived from 3,600 initial pairs through expert annotation and filtering; four expert dentists labeled cases, with low-confidence or tied answers removed.Source: §4.5 · p. 17
  • The 36 tasks are evaluated with accuracy for 34 multi-class diagnosis tasks, hit-rate for multi-label diagnosis, IoU for disease localization, and human scores for rationale generation.The rationale-generation evaluations ask dentists to assess model responses across multiple dimensions.Source: §4.5 · p. 18
  • Baseline comparisons include proprietary, open-source, and medical MLLMs, while additional experiments examine base-model adaptability, data scale, and zero-shot performance on four diseases absent from training.The clinical study compares dentists working independently or with answer-only or rationale-bearing model assistance; response quality is rated on six dimensions.Source: §4.5 · pp. 18–19

4.6 Practical utility for DentVLM in real-world demonstrations

  • The practical demonstrations aggregate image-level predictions for home screening and hospital malocclusion workflows, and pass DentVLM responses to DeepSeek-R1 for multi-turn refinement.The multi-agent evaluation used 50 VQA pairs spanning 31 tasks and all modalities; three experienced dentists rated dialogues on seven dimensions.Source: §4.6 · p. 20

5 Computing hardware and software

Training and downstream evaluations ran on an eight-GPU single-node server with a specified software stack. The paper also reports separate inference and web-application technologies.

5 Computing hardware and software

  • Training used an eight-GPU single-node server with DeepSpeed ZeRO-2, LLaMA-Factory, PyTorch 2.4.0, CUDA 12.4, and Transformers 4.45.2.The server ran Ubuntu 20.04 and used 8 × 80 GB GPUs.Source: §5 · p. 20
  • Evaluation used vLLM and vLLM-flash-attn for DentVLM, while proprietary and open-source comparison models were deployed through their APIs or inference implementations.The paper names OpenAI Batch API, Google Developer API, and model-specific official inference approaches.Source: §5 · pp. 20–21
  • Web applications used Flask, MySQL, Docker, Docker Compose, and Nginx for backend, database, containerization, and reverse-proxy functions.The passage assigns these respective roles to the listed technologies.Source: §5 · p. 21

7 Author contributions

The author contributions span study design, data and model development, clinical evaluation, analysis, and manuscript preparation.

7 Author contributions

  • Study design and data acquisition were led by a subset of authors, while preprocessing involved a larger group.The passage names the contributors to each activity.Source: §7 · p. 21
  • Model development and web application systems were handled by separate author groups.The passage identifies the model developers and the web-system developers.Source: §7 · p. 21
  • Experiments, clinical deployment, statistical analysis, figure generation, drafting, and revision were distributed across overlapping author groups.The contribution statement lists authors for each activity.Source: §7 · p. 21
Loading 2509.23344v1…