Source-linked AI summary
Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey
Zirui Song, Bin Yan, Yuhan Liu, Miao Fang, Mingzhe Li, Rui Yan, Xiuying Chen
TL;DR
General-purpose LLMs are limited in specialized applications that require precise domain expertise. This survey organizes domain-knowledge integration into four paradigms, compares their trade-offs and applications, and identifies challenges including knowledge consistency and uneven coverage across domains.
Problem
General-purpose LLMs often lack the specialized knowledge needed for accurate and reliable performance in domains such as healthcare, chemistry, and legal analysis.
Method
The survey categorizes domain-knowledge integration into dynamic injection, static embedding, modular adapters, and prompt optimization, then reviews applications, datasets, benchmarks, and comparisons.
Results
The surveyed paradigms offer different trade-offs in flexibility, updateability, training cost, inference speed, adaptation, and performance dependence on data quality.
Takeaways & Limitations
Domain-specific LLM research is moving toward combining complementary integration strategies while requiring standardized evaluation and methods for maintaining consistency across injected knowledge.
Takeaways & Limitations
Coverage is limited for some emerging or less-studied areas, and targeted comparisons were feasible only under controlled conditions within biomedicine.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have demonstrated remarkable success in various tasks such as natural language understanding, text summarization, and machine translation. However, their general-purpose nature often limits their effectiveness in domain-specific applications that require specialized knowledge, such as healthcare, chemistry, or legal analysis. To address this, researchers have explored diverse methods to enhance LLMs by integrating domain-specific knowledge. In this survey, we provide a comprehensive overview of these methods, which we categorize into four key approaches: dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization. Each approach offers unique mechanisms to equip LLMs with domain expertise, balancing trade-offs between flexibility, scalability, and efficiency. We discuss how these methods enable LLMs to tackle specialized tasks, compare their advantages and disadvantages, evaluate domain-specific LLMs against general LLMs, and highlight the challenges and opportunities in this emerging field. For those interested in delving deeper into this area, we also summarize the commonly used datasets and benchmarks. To keep researchers updated on the latest studies, we maintain an open-source at: https://github.com/abilliyb/Knowledge_Injection_Survey_Papers, dedicated to documenting research in the field of specialized LLM.
1 Introduction
General-purpose LLMs perform strongly across broad tasks but require domain-specific knowledge for accurate and reliable specialized applications. This survey reviews the growing field, organizing injection methods into four paradigms and identifying evaluation and scalability challenges.
- Specialized domains such as healthcare, chemistry, and legal analysis require domain-specific knowledge for accurate and reliable LLM performance.
- Knowledge injection integrates external or embedded expertise to bridge general-purpose language understanding and specialized task requirements.
- The survey addresses rapidly growing research and limited methodological standardization by reviewing techniques, applications, datasets, benchmarks, and future challenges.
- The review frames knowledge injection around dynamic runtime injection, static training-time embedding, modular adapters, and prompt optimization.
2 Background
Domain-specific knowledge provides field-specific expertise needed for specialized tasks, motivating research on methods for incorporating it into LLMs. Knowledge may be represented in structured graphs, unstructured text, or vector spaces, while prior surveys differ in focus and scope.
- Domain-specific knowledge is specialized information or expertise for a particular field or application, unlike general knowledge spanning multiple domains.
- Researchers have explored methods for incorporating domain-specific knowledge into LLMs, and this paper surveys those injection methods.
- Knowledge graphs encode entities and relationships for reasoning, whereas text sources provide rich unstructured information for broad contextual understanding.
- Prior surveys address scientific article classification, knowledge editing, or model-centric categorization, differing in focus and methodological scope.
3 Paradigms of Knowledge Injection
The survey organizes domain-knowledge integration into four paradigms distinguished by when knowledge enters the system and whether model parameters change. They span runtime retrieval, parameter-based embedding, modular adaptation, and prompt-based guidance, with different trade-offs in cost, speed, flexibility, and updateability.
- The four paradigms are Dynamic Knowledge Injection, Static Knowledge Embedding, Modular Knowledge Adapters, and Prompt Optimization.
- Dynamic Knowledge Injection: Dynamic injection retrieves external knowledge during inference while keeping model parameters unchanged, enabling updates without retraining but depending on retrieval and knowledge-base quality.Input-length limits and retrieval efficiency can constrain performance.
- Static Knowledge Embedding: Static embedding learns domain knowledge in model parameters through full or partial fine-tuning, enabling fast inference but making updates costly and less flexible.Embedding large or frequently changing knowledge bases also requires substantial computational resources.
- Modular Knowledge Adapters: Modular adapters train small plug-and-play modules while typically freezing the base model, reducing computational requirements and preserving general-purpose capabilities.Their effectiveness depends on adapter design, hyperparameters, and data quality.
- Prompt Optimization: Prompt optimization guides the model with carefully designed inputs without changing parameters, avoiding retraining while relying on internal knowledge and manual prompt design.Long prompts can reduce available input space because of context-length limits.
4 Applications
Applications of domain-specific LLMs span finance, biomedicine, materials science, chemistry, mental health, education, and social science. These studies use varied combinations of static embedding, dynamic injection, prompt optimization, and tool integration, with materials and chemistry beginning to move toward interactive reasoning.
- Finance: Financial LLMs follow two main paths: fine-tuning general-purpose models or training models from scratch on domain-specific corpora.Examples range from PIXIU’s 136K financial instruction samples and FinGPT’s 50K LoRA-tuned samples to BloombergGPT’s 5B Bloomberg-specific tokens and XuanYuan 2.0’s 366B-token training corpus.
- Biomedicine: Biomedical LLMs commonly use specialized corpora and static embedding, while newer systems combine pretraining, instruction tuning, vector prompts, and tool integration.PMC-LLaMA further pretrained LLaMA 7B on 4.9 million PubMed Central articles, while vector prompts support retrieval of medical knowledge during inference.
- Materials and Chemistry: Materials science and chemistry have largely relied on static embedding, but recent work adds chemistry tools and planning prompts for synthesis, drug discovery, and complex execution.Darwin 1.5 uses two-stage training, ScholarChemQA supports chemical reasoning, ChemCrow adds tools, and ChemAgent uses planning prompts.
- Materials and Chemistry: Materials and chemistry are transitioning from static embedding toward interactive and tool-augmented reasoning, although this direction remains at an early stage.The transition is presented as having strong potential for future development.
- Human-Centered Science: Human-centered applications use personalized and context-aware models across mental health, education, social behavior analysis, and misinformation research.Examples include SoulChat and MeChat for counseling, EduChat for educational support, and models combining injection methods or prompt optimization for social-network analysis.
5 Tools, Resources, and Analysis
The survey presents four representative open-source frameworks and compares knowledge-injection paradigms using domain resources and biomedical benchmarks. The analysis emphasizes framework diversity, practical trade-offs, and the importance of pretraining data and task alignment.
- Knowledge Injection Framework: The survey organizes four open-source frameworks by paradigm: KnowGPT for dynamic injection, StructTuning for static embedding, K-Adapter for modular adapters, and SelfLift for prompt optimization.Together, they illustrate the survey’s unified framework for understanding and applying the four knowledge-injection methods.
- Knowledge Injection Framework: KnowGPT selects relevant knowledge-graph subgraphs with reinforcement learning, converts triples into prompts, and reduces LLM API-call costs while improving domain-task performance.Its prompt templates make the selected graph knowledge interpretable and usable by language models.
- Knowledge Injection Framework: StructTuning embeds domain knowledge through structure-aware continual pretraining followed by supervised fine-tuning on structured question-answering tasks.The framework reports improvements on relation classification and question answering while balancing generality and efficiency.
- Knowledge Injection Framework: K-Adapter freezes the base model and trains independent task-specific adapter modules, helping preserve existing knowledge while adding specialized representations.The modular design is intended to mitigate catastrophic forgetting by preventing new knowledge from overwriting pretrained knowledge.
- Resources and Analysis: Biomedical resources include PubMed, PubMedQA, and BioASQ, whereas materials and chemistry have more limited datasets; biomedical comparisons use MedQA, PubMedQA, and MedMCQA.GPT-4 with Medprompt achieves strong performance despite lacking domain-specific training, and finance experiments yield similar conclusions to the medical comparison.
6 Challenges and Opportunities
The survey identifies integrated-knowledge consistency and cross-domain transfer as major challenges. Retrieved information may conflict with model knowledge, while transferring expertise across fields must reconcile different terminologies, ontologies, and reasoning patterns.
- Integrated Knowledge Consistency: Knowledge injection can produce inconsistent outputs when retrieved facts conflict with pretrained representations or with one another.The survey highlights healthcare treatment protocols and legal precedents as settings where such conflicts can undermine decision reliability and trustworthiness.
- Integrated Knowledge Consistency: Addressing knowledge conflicts requires reliable-source prioritization, domain rules, ensemble techniques, and alignment or validation modules.These mechanisms are proposed to detect conflicts, resolve them, and ensure retrieved knowledge fits the model’s reasoning.
- Cross-Domain Knowledge Transfer: Cross-domain transfer is difficult because domains differ in terminology, ontology, data structure, and reasoning framework.The survey uses chemistry-to-healthcare transfer as an example requiring reconciliation of distinct structures and reasoning patterns.
- Cross-Domain Knowledge Transfer: The survey proposes modular representations, transfer learning, hybrid static-and-dynamic methods, and standardized benchmarks as directions for cross-domain knowledge transfer.These approaches aim to support flexible adaptation without sacrificing domain depth and to enable consistent evaluation.
7 Conclusion
The survey systematically reviews knowledge-injection systems, their representations and integration strategies, and mechanisms for preserving model generality. It consolidates applications, datasets, benchmarks, challenges, and future opportunities for domain-specific LLM research.
- Conclusion: The survey reviews knowledge representation, integration strategies, and mechanisms for preserving model generality across domain-specific LLM systems.Its application coverage includes biomedicine, chemistry, and computational social science.
- Conclusion: The survey aims to provide a resource for research on knowledge-enhanced LLMs by consolidating datasets, benchmarks, challenges, and future opportunities.The stated goal is to inspire further exploration of domain-specific challenges.
Limitation
The survey covers several major domains but gives limited attention to less-studied or emerging areas. Comparisons were conducted only under controlled biomedical conditions, leaving broader empirical evaluation for future work.
- Less-studied areas, including low-resource languages, cross-cultural education, and niche disciplines, receive relatively limited attention.
- The survey primarily summarizes methodological principles and representative models from existing literature.
- Because architectures, domains, training data, and evaluation protocols vary substantially, comparisons were limited to controlled biomedical settings using commonly adopted datasets.
- A more systematic and broad-based empirical evaluation across knowledge injection methods remains an important direction for future work.
A Performance Comparison of 4 Paradigms
The survey compares representative knowledge injection paradigms on biomedical and financial benchmarks. Across these domains, targeted knowledge injection generally outperforms general-purpose baselines, while performance varies with corpus-task alignment and adaptation strategy.
- The financial comparison evaluates representative models under four knowledge injection paradigms on benchmarks including FPB, FiQA-SA, and TFNS.
- Across biomedical and financial comparisons, effectiveness depends on architectural design, adaptation strategy, and corpus-task alignment.
- Finetuned GPT-4 consistently outperforms other financial-domain models, while LLaMA2-7B lags behind across most tasks.
- Static knowledge models such as FinBERT and FLANG perform well on some financial tasks but show significant variability.
- FinGPT achieves competitive performance while maintaining adaptability through lightweight adapter-based knowledge injection.
B.1 Integrated Knowledge Consistency
Knowledge-enhanced LLMs face consistency challenges when injected information conflicts with internal representations or other retrieved sources. Proposed responses include contradiction detection, confidence-aware retrieval, structured checking, and modular strategies for managing domain complexity and integration costs.
- Integrated Knowledge Consistency: Injected knowledge may contradict the model’s internal representations or other retrieved information, creating risks in healthcare and legal applications.
- Integrated Knowledge Consistency: Post-retrieval contradiction detection, confidence-aware reranking, weighted retrieval, and ensemble voting prioritize more reliable information sources.
- Integrated Knowledge Consistency: Neural-symbolic consistency checking aligns injected knowledge with predefined ontologies or verifies it through structured reasoning paths.
- Integrated Knowledge Consistency: Alignment-aware reranking filters retrieved documents according to their alignment with the LLM’s intermediate beliefs.
- Cross-Domain Knowledge Transfer: Cross-domain transfer must handle incompatible ontologies, domain languages, causal assumptions, and data formats.
- Cross-Domain Knowledge Transfer: Adapters, meta-learning, and continual pretraining offer strategies for selectively activating domain components, adapting with limited supervision, and improving robustness.
- Knowledge Integration Efficiency: Large external knowledge sources create computational and memory bottlenecks, motivating sparse retrieval, compression, caching, and modular architectures.
B.4 Evaluation and Hallucination Detection
Evaluation remains difficult because standardized benchmarks and automatic metrics for factual consistency, coverage, and reasoning depth are limited. Hallucinations can persist despite accurate augmentation, while existing evaluation approaches often do not scale across domains or languages.
- Knowledge-enhanced LLMs lack standardized benchmarks and automatic metrics for factual consistency, coverage, and reasoning depth.
- LLMs may hallucinate facts even when augmented with accurate knowledge, complicating trust in high-stakes applications.
- FactScore, entailment-based verification, and human-in-the-loop schemes provide emerging evaluation approaches.
- Few existing evaluation methods scale across domains or languages, motivating task-specific and fine-grained metrics.