Source-linked AI summary
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Zheng Chen, Zhicheng Du, Haoxuan Li, Yingshan Liang, Peiwu Qin
TL;DR
TCM’s holistic I-AOI-P diagnosis has been difficult to align with quantitative, standardized medical systems. The paper introduces UFMD to structure multimodal diagnostic records and fine-tunes LingLan-14B to emulate the I-AOI-P workflow. It reports a 103.5% improvement over the baseline and an F1-score up to 82%.
Problem
TCM’s holistic, subjective I-AOI-P diagnosis is difficult to align with quantitative and standardized medical systems, while multimodal data integration and high-quality dataset availability remain challenges.
Source: §I · p. 1, A. Unification Framework for… · p. 2Method
UFMD structures tongue and pulse image information into clinical descriptions and unified I-AOI-P records, which are used to fine-tune LingLan-14B to emulate the diagnostic workflow.
Source: §I · pp. 1–2Results
A 103.5% improvement over the baseline and an F1-score as high as 82% are reported for I-AOI-P-based supervised fine-tuning.
Source: §I · p. 2Takeaways & Limitations
The authors conclude that structured data integration and fine-tuning enhance TCM diagnostic accuracy, with LingLan-14B reaching state-of-the-art diagnostic performance.
Source: §V · p. 5
Abstract
from arXiv · showhide
Though artificial intelligence (AI) increasingly transforms modern medicine, its integration into Traditional Chinese Medicine (TCM) has been relatively slow, primarily due to TCM's reliance on holistic, subjective diagnostic methods---namely Inspection, Auscultation and Olfaction, Inquiry, and Palpation(I-AOI-P)---which are difficult to align with quantitative, standardized medical systems. In this work, we introduce a Unification Framework for Multimodal Data (UFMD), which automatically processes tongue and pulse images into structured, clinically standard descriptions, integrating multi-source diagnostic information into a unified digital record of I-AOI-P process. Building on this structured data, we create LingLan-14B, a TCM-specific large language model fine-tuned via supervised learning to emulate the diagnostic logic and workflow of I-AOI-P process. Experimental results show that our method significantly enhances diagnostic accuracy, achieving a relative improvement of 103.5% over the baseline (62.72% vs. 30.82%) and reaching an F1-score of up to 82%.
I. INTRODUCTION
TCM’s holistic I-AOI-P diagnosis relies on subjective experiential methods, while current AI-assisted diagnosis faces accuracy, multimodal integration, and data-quality challenges. The paper presents UFMD and LingLan-14B, reporting a 103.5% improvement over the baseline and an F1-score up to 82%.
I. INTRODUCTION
- TCM’s holistic, subjective I-AOI-P approach is difficult to align with modern medicine’s quantitative analysis and standardized pathways.The introduction also identifies insufficient accuracy and stability in AI-assisted TCM diagnosis, alongside limits in multimodal fusion and public dataset quality.Source: §I · p. 1
- UFMD processes tongue and pulse images into standardized clinical descriptions and integrates I-AOI-P information into a unified digital case record.The framework is presented as an MLLM-based pipeline for turning multi-source information into a structured representation.Source: §I · pp. 1–2
- LingLan-14B is fine-tuned on the unified structured representations to learn the logic and workflow of TCM’s I-AOI-P diagnostic process.The paper describes this as reframing supervised fine-tuning around the diagnostic data framework.Source: §I · p. 2
- A 103.5% improvement over the baseline and an F1-score as high as 82% are reported for I-AOI-P-based supervised fine-tuning.The authors attribute the improvement to the model’s enhanced ability to diagnose TCM.Source: §I · p. 2
II. RELATED WORK
Prior TCM data-unification work combines authoritative databases, NLP, and manual curation, while AI diagnosis studies include LLM, knowledge-graph, and corpus-based approaches. The paper identifies manual-curation inefficiency and limited consideration of I-AOI-P as remaining challenges.
1) Multimodal Data Unification
- Existing TCM data projects combine multiple databases and NLP extraction with manual curation of clinical or historical records.Examples include Liu et al.’s use of BioBERT to extract and validate relationships from over 30 million documents, and Li et al.’s integration of authoritative databases and literature.Source: §II · p. 2
- The cited data-unification approaches still face inefficient cleansing of large-scale real-world data and time-intensive manual curation.Source: §II · p. 2
2) TCM Diagnosis Improvement
- AI-based TCM diagnosis includes OpenTCM’s combination of LLMs, knowledge graphs, and GraphRAG, and TCM-GPT-7B’s large-scale corpus, but these approaches overlook comprehensive I-AOI-P diagnosis.The paper states that this omission results in limited generalization capability in practical applications.Source: §II · p. 2
III. METHODOLOGY
TCM records combine heterogeneous image and text data, and their aggregation remains challenging. UFMD converts multimodal records into structured data and uses semantic-relevance checks and expert proofreading to support data reliability.
A. Unification Framework for Multimodal Data
- TCM records span tongue images, pulse images, and unstructured or semi-structured records, making aggregation and use challenging.The paper notes that digitizing historical paper records and images often still depends on inefficient manual extraction.Source: A. Unification Framework for… · p. 2
- UFMD uses DeepSeek-OCR to parse complex medical-record images, tongue images, and pulse images into structured data, with manual proofreading and expert verification.The framework’s overview depicts feature extraction, MLLM review and expert validation, then persistent storage of verified structured data.Source: A. Unification Framework for… · pp. 2–3, Figure 2 · p. 3
- UFMD checks extracted medical-record text against its raw source using cosine similarity between Sentence-BERT embeddings to flag potential hallucinations or errors.A predefined threshold determines whether extracted data is considered semantically consistent; the paper notes this is intended for narratives with paraphrasing and synonymy.Source: A. Unification Framework for… · pp. 2–3
- The sentence-embedding equation defines the vector representation used in the semantic-relevance check.The subsequent similarity calculation compares the extracted and raw-text embeddings.Source: A. Unification Framework for… · p. 2
B. LingLan: SFT Model for TCM Diagnosis
LingLan’s TCM diagnosis model adapts Qwen3-14B using PEFT with LoRA and trains on records structured to reflect the complete I-AOI-P diagnostic process. The authors describe this setup as computationally efficient and aligned with holistic syndrome differentiation.
B. LingLan: SFT Model for TCM Diagnosis
- LingLan adapts Qwen3-14B with PEFT based on LoRA to specialize it for TCM clinical records while avoiding the computational cost of full fine-tuning.Source: B. LingLan: SFT Model for TCM Diagnosis · p. 3
- Training records combine quantitative tongue and pulse features with inquiry information in a structured template to align learning with holistic I-AOI-P reasoning.The authors say this helps the model relate tongue and pulse patterns to TCM syndromes, moving beyond symptom-text-only descriptions.Source: B. LingLan: SFT Model for TCM Diagnosis · p. 4
- The reported experimental configuration uses PEFT and compares its efficiency with BianCang-14B-Instruct, with hyperparameters and hardware listed in Table I.The authors state that their approach reduces training time and computational resource requirements while maintaining competitive performance metrics.Source: B. LingLan: SFT Model for TCM Diagnosis · p. 4
A. Experimental settings
The evaluation reserves a stratified 20% subset of TCM medical records for testing, with rare-class handling and diagnoses standardized to GB/T 15657-2021.
1) Dataset:
- 20% of the TCM medical-record dataset was reserved as the test set, using stratified sampling by diagnostic label.The sampling aims to keep diagnosis proportions consistent between training and test sets.Source: 1) Dataset · p. 4
- Classes with one sample were excluded from the test set; classes with two samples contributed at least one test sample.The stated aim was to prevent evaluation bias from a class appearing only in training or testing.Source: 1) Dataset · p. 4
- Diagnoses followed GB/T 15657-2021, and the test-data distribution is presented in Fig. 3.Category statistics are also identified as appearing in Table II.Source: 1) Dataset · p. 4, Figure 3 · p. 4
2) Baselines:
The study compares LingLan with general-purpose Chinese-language LLMs and a TCM-specific model, using standard classification metrics alongside two diagnosis-specific indicators.
2) Baselines:
- The baselines include Qwen3-14B, larger general-purpose models, and the TCM diagnosis model BianCang-14B-Instruct.The listed larger models are Qwen3-32B, DeepSeek-R1-Distill-32B, and DeepSeek-R1 (671B).Source: 2) Baselines · p. 4
3) Metrics:
- Accuracy, recall, and F1-score evaluate fundamental TCM prediction ability, while UDI and PCDI add diagnosis-specific measures.The authors motivate the added metrics by noting that unconstrained general-purpose model answers can be overly broad and inaccurate.Source: 3) Metrics · p. 4
- UDI assesses prediction without constraints in generalized scenarios, while PCDI tests prediction for specific symptoms after instruction constraints are added.The supplied text describes these metrics' evaluation settings, not their formulas.Source: 3) Metrics · pp. 4–5
B. Results
LingLan-14B outperformed the baselines across five evaluation metrics on the TCM medical-record test set; its confusion matrix also reports higher recall accuracy than BianCang-14B-Instruct.
1) Overall Performance:
- LingLan correctly classified 2,850 BN, 5,406 BF, and 1,703 BG instances, with 703 BN–BW, 487 BN–BY, and 1,103 BF–BG confusions.The matrices show predominant diagonal concentration, with the listed errors among the off-diagonal cases.Source: 1) Overall Performance · p. 5
- LingLan-14B outperformed all baseline models on Accuracy, Recall, F1-score, UDI, and PCDI on the TCM medical-record test set.The comparison is reported in Table III.Source: Table III · p. 5, 1) Overall Performance · p. 5
- 82.03% recall accuracy for LingLan exceeded BianCang-14B-Instruct's 69.51%.The confusion matrices compare true labels on the vertical axis with predicted labels on the horizontal axis.Source: Figure 4 · p. 5, 1) Overall Performance · p. 5
2) Ablation Study:
The ablation compares structured-data fine-tuning with raw-data fine-tuning under the same baseline model, parameters, and test set.
2) Ablation Study:
- Fine-tuning Qwen3-14B on structured data performed significantly better than fine-tuning the same baseline on raw data.The comparison kept model parameters and the test set the same; accuracy and recall were calculated separately for diagnosis and syndrome differentiation.Source: 2) Ablation Study · p. 5
V. CONCLUSION
UFMD integrates multimodal I-AOI-P information, and LingLan-14B is fine-tuned to emulate that diagnostic workflow. The authors report state-of-the-art diagnostic performance, while noting data-access and model-evaluation constraints and plans to test larger models.
V. CONCLUSION
- LingLan-14B achieves state-of-the-art diagnostic performance, and experiments support the effectiveness of structured data integration and fine-tuning.The conclusion characterizes performance as state of the art but supplies no metric values in these passages.Source: §V · p. 5
- UFMD integrates holistic multimodal I-AOI-P information with quantitative medical systems, while LingLan-14B is supervised-fine-tuned to emulate the diagnostic workflow.Source: §V · p. 5
VI. FUTURE WORK
- Patient privacy currently prevents public release of the dataset, and cybersecurity constraints prevented empirical evaluation of non-open-source models on TCM diagnosis.Source: §VI · pp. 5–6
- The authors plan to extend fine-tuning to larger models to further assess its effectiveness and scalability.Source: §VI · p. 6