PaperlayerView exact revision on arXiv

Source-linked AI summary

Intelligent Identification and Repair of Design Defects in BIM via Domain-Specific Large Language Models

Jia-Rui Lin, Yun-Hong Cai, Xiang-Rui Ni, Peng Pan

arXiv:2608.28629v1cs.CLcs.AI

TL;DR

BIM defect identification and repair lack a unified, generalizable approach across diverse defect types. This study develops an integrated domain-specific LLM framework combining BIM-to-Text, prompt learning, RAG, and hallucination control. The framework reports improved identification and repair-suggestion outcomes, while experiments identify low generation efficiency and residual hallucinations as limitations.

  • Problem

    BIM design-defect identification and repair lack a unified, generalizable method, while engineering applications require high precision and reliability.

  • Method

    The framework uses BIM-to-Text with component-balanced chunking, prompt learning, RAG, and hallucination control for multi-category defect identification and repair.

  • Results

    85% identification accuracy versus 70% for traditional rule checking accompanied a 94% rate of reasonable repair suggestions.

  • Takeaways & Limitations

    The study establishes an end-to-end prototype for multi-category BIM design-defect identification and repair.

  • Takeaways & Limitations

    The approach has relatively low generation efficiency and continues to exhibit hallucination phenomena.

Abstract

from arXiv · show

Existing methods lack a generalized approach to efficiently identify and resolve the diversity of design defects in BIM. Therefore, this study proposes an integrated framework to identify and repair various defects in BIM via domain-specific LLMs. Firstly, a BIM-to-Text method with component-balanced chunking is introduced to bridge BIM data with LLMs. Then, prompt learning with rule injection, few-shot prompting and RAG is proposed to identify defects and generate repair suggestions. Meanwhile, a hallucination control strategy combining key identifier validation and token-length thresholds is introduced to ensure reliability. Experiments show capability expansion yields 85% identification accuracy versus 70% for traditional rule checking, achieving a 94% rate of reasonable repair suggestions. Moreover, the proposed hallucination control further increased accuracy from 64% to 85%, eliminating 92.5% of hallucinations in a single intervention round. This study establishes an end-to-end prototype from raw BIM data input, through defect identification, to repair suggestion generation.

1 Introduction

BIM defect identification and repair remain fragmented, manual, and difficult to generalize across diverse design problems. The study proposes a domain-specific LLM framework combining BIM-to-Text, prompt learning, RAG, and hallucination control for end-to-end defect handling.

  • Research gap: Existing BIM defect checking relies heavily on manual inspection and limited automated methods.Rule-based and collision-detection approaches typically target particular defect categories or scenarios.
  • Research gap: Diverse BIM quality issues include semantic defects that rule-based and collision-detection methods may miss or misclassify.These methods struggle with implicit design conventions, semantic context, and defects involving functionality, materials, or fire resistance.
  • Research gap: Automated repair remains scarce because effective solutions must synthesize regulatory constraints with complex engineering expertise.Current repair processes often require interdisciplinary teams and substantial manual effort.
  • Proposed direction: The study develops a domain-specific LLM framework for identifying and repairing multiple BIM design-defect types.Its stated goal is an end-to-end prototype supporting intelligent and efficient engineering applications.
  • Proposed direction: The framework converts BIM models to text with component-balanced chunking and combines rule injection, few-shot prompting, RAG, and hallucination control.Hallucination control uses key-identifier validation and token-length thresholds to mitigate unreliable outputs.

2 Literature Review

The literature review organizes BIM design defects and examines existing identification and repair approaches. It finds persistent dependence on manual rule engineering, limited semantic coverage, and comparatively little automation for repair suggestion generation.

  • Defect significance: Design defects can contribute to rework, schedule delays, cost overruns, stakeholder disputes, and project performance decline.The review characterizes defect identification and prevention as core BIM application issues.
  • Defect classification: BIM design defects are categorized here as integrity, rationality, and compliance defects.Integrity concerns missing components or attributes, rationality concerns deviations from design common sense, and compliance concerns violations of standards or regulations.
  • Identification methods: Rule-based systems formalize domain knowledge and detect explicit regulatory violations but require substantial manual encoding and maintenance.Their workload and applicability constraints increase as data requirements and rule complexity expand.
  • Identification methods: Machine-learning approaches identify abnormal or inconsistent BIM elements but remain limited for semantically rich information and large-scale deployment.They often require large training datasets, iterative feature optimization, and high computational costs.
  • Repair methods: Research on automated BIM defect repair and repair-suggestion generation remains limited and largely unexplored.Existing integrity-defect repair methods include semantic augmentation, expert systems, and machine-learning approaches, but manual adjustment remains important.
  • LLM direction: The study focuses on identifying and repairing integrity, rationality, and compliance defects using parameter-free prompt learning and RAG.This approach targets the contextual relationships requiring complex knowledge interpretation and multi-step reasoning.

3 Methodology

The methodology integrates BIM textualization, prompt-based defect identification, RAG-supported repair suggestions, and validation in a residential-building BIM scenario.

  • The framework targets multi-category BIM defect identification and repair suggestion generation through an integrated domain-specific LLM architecture.
  • Design defect identification: Rule-injection prompts and few-shot examples guide the domain-specific LLM to identify integrity, rationality, and compliance defects.The framework was validated experimentally in a residential building BIM scenario.
  • Defect repair and validation: RAG incorporates domain knowledge to generate grounded repair suggestions, while key-identifier validation and token-length thresholds improve output accuracy and stability.
  • BIM-to-Text Method: BIM-to-Text converts component information into semi-structured natural language while preserving hierarchy and selected spatial and engineering attributes.The representation is designed to improve comprehension while limiting token growth from raw geometric data.
  • BIM-to-Text Method: Component-balanced chunking randomly samples a fixed number of component entries per block to keep model inputs manageable.Blocks are evaluated independently and remain within a specified component limit, although sampled components may not be spatially adjacent.

3.2 Design defect identification

The defect-identification method textualizes BIM chunks and combines system, rule-injection, and few-shot prompts to guide domain-specific LLM outputs.

  • Output validation: Generated identification outputs are checked for hallucinations and accepted or regenerated, with final accuracy verified through manual inspection.
  • Each BIM data chunk concatenates component descriptions before system, rule-injection, and few-shot prompts are added.
  • Prompt design specifies professional role, reasoning guidance, output-format constraints, and representative examples to improve consistency.
  • Rule-Injection Prompts: Rule injection combines structured representations, representative code examples, and explicit inspection requirements rather than expanding every provision individually.The study reports that this reduces input length and lowers the likelihood of hallucinations.
  • Few-Shot Prompts: Few-shot examples are created by hiding selected answer information, including values or reasoning details, so examples do not directly determine the response.

3.3 Defect Repair Suggestion Generation Based on RAG

The repair-suggestion method uses RAG with a domain-specific knowledge base to retrieve relevant regulatory knowledge and generate BIM defect repairs.

  • The method builds a defect-repair knowledge base to support RAG, improve repair-suggestion quality, and enhance suggestion rationality.
  • Knowledge base: The knowledge base primarily supports integrity and compliance repairs, while rationality defects mainly rely on BIM design common sense.
  • Knowledge base: Five building codes and standards provide domain knowledge, whose materials are organized into structured thematic chunks for retrieval.
  • Knowledge base: The knowledge base contains 221 vectors mapped to titles and knowledge text, with data from different standards exhibiting clear clustering patterns.
  • RAG retrieval: Qdrant retrieves the top ten text segments, which are re-ranked and reduced to the three most relevant segments as final RAG content.
  • Suggestion generation: The complete model input combines textualized BIM data, system and defect-repair prompts, few-shot prompts, retrieved knowledge, and a RAG prompt.

3.4 Hallucination Control Method

The hallucination-control method addresses fabricated BIM information and reasoning loops through identifier validation, length thresholds, and iterative regeneration.

  • BIM-related hallucinations include fabricated information beyond the input scope and logical loops during reasoning.
  • Key-identifier validation: Key-identifier validation compares extracted component IDs, names, attributes, and type information against the input data.An identifier absent from the input indicates fabricated content.
  • Key-identifier validation: The identifier constraint enhances the accuracy and coverage of hallucination detection.
  • Token-length threshold: A token-length threshold flags potentially hallucinated outputs when generated text exceeds 4,000 characters under a 4,096-token output limit.
  • Iterative control: Iterative generation–validation regenerates outputs containing fabricated or inconsistent information until a hallucination-free result is produced.The mechanism applies to both defect identification and repair stages.

3.5 Verification and Experiments

The study evaluates an integrated BIM defect-identification and repair framework using a residential building model with artificially injected fire-protection defects. Experiments compare rule-based checking, a base LLM, and domain-tuned Qwen-BIM across identification, repair suggestions, and hallucination control.

  • Experimental design: 100 fire-protection defects were injected into a residential BIM model across integrity, rationality, and compliance categories.The defects comprised 25 integrity, 30 rationality, and 45 compliance instances.
  • Experimental design: Repair-suggestion evaluation compared Qwen-BIM and the base model under RAG and non-RAG conditions using rationality as the evaluation criterion.Suggestions covered all 100 injected defects, with acceptability judged against combustibility and fire-resistance requirements.
  • Reliability verification: Hallucination control was evaluated by testing abnormal-output suppression during repeated defect-identification rounds.The mechanism combined key-identifier validation and token-length thresholds to improve generation stability.
  • Experimental setting: The residential BIM model contained 3,389 components across walls, slabs, evacuation stairs, roofs, doors, and windows, processed in component-balanced chunks.Every ten components were grouped into one chunk, yielding 339 chunks in total.
  • Rule construction: The rule-based inspection encoded fire-protection code provisions into textual rules refined for BIM component categories and attributes.Doors and windows were excluded because the Code specified no corresponding requirements for them.

4 Results and Discussion

The domain-specific LLM framework broadens BIM defect checking beyond syntactic rules by identifying rationality issues while maintaining strong performance on compliance and integrity defects. RAG and hallucination control further improve repair-suggestion rationality and output reliability, although generation efficiency and hallucinations remain limitations.

  • Identification performance: 85% overall identification accuracy was achieved by Qwen-BIM versus 70% for the traditional rule-based method.The 15% gap is attributed primarily to Qwen-BIM’s ability to address rationality defects, which represented 30% of the test set.
  • Identification performance: Qwen-BIM slightly outperformed rule checking on compliance and integrity defects while adding rationality-defect identification.The framework therefore expanded inspection scope rather than merely improving accuracy on identical tasks.
  • Limitations: The framework’s current limitations are low generation efficiency and persistent hallucination phenomena.Suggested future directions include quantization, acceleration techniques, and larger-parameter models for long-text understanding and hallucination reduction.
  • Repair suggestions: 94% repair-suggestion rationality was achieved by Qwen-BIM with RAG, increasing from 79%.RAG introduced normative information and prompted step-by-step analysis based on the relevant texts.
  • Hallucination control: Hallucination control eliminated 92.5% of Qwen-BIM’s hallucinated chunks after one intervention round and removed them completely after four cycles.The first round contained 40 hallucinated chunks, while only 3 remained after re-identification.
  • Model enhancement: Fine-tuning and hallucination control increased identification accuracy from 64% for the base model to 85%.The study reports a synergistic effect improving semantic comprehension, stability, and accuracy.

5 Conclusion

The study presents an end-to-end domain-specific LLM framework for identifying and repairing multiple BIM design defects. It combines BIM-to-Text transformation, prompt learning, RAG, and hallucination control, achieving stronger identification and repair performance in the evaluated scenarios.

  • Integrated framework: The framework closes the workflow from raw BIM data through multi-category defect identification to repair suggestion generation.It combines BIM-to-Text transformation with prompt learning, RAG, and domain-adapted hallucination control.
  • Defect identification: 85% overall identification accuracy versus 70% for traditional rule checking demonstrates capability expansion across defect categories.The improvement is primarily attributed to automatically addressing rationality defects and complementing syntactic rule checking.
  • Repair suggestion generation: 94% rationality rate versus 79% shows that RAG strengthened normative grounding and external-knowledge utilization for repair suggestions.The RAG-based repair method uses a vectorized knowledge base containing 221 design-code data nodes.
  • Hallucination control: 85% identification accuracy versus 64% followed hallucination control based on key-identifier validation and token-length thresholds.A single intervention round eliminated 92.5% of hallucinations and reduced fabricated information and excessive generation.
  • Implications: The prototype provides a structural foundation for integrating generative AI into rigorous engineering workflows.The authors describe the resulting paradigm as reliable and reproducible within the demonstrated scope.
  • Future work: Future evaluation should test complex and diverse engineering projects, novel defect types, stronger hallucination mitigation, and multidisciplinary repair.The present study was validated primarily in limited scenarios; integration with BIM software for real-time modification is also proposed.

CRediT authorship contribution statement

The authors’ contributions span conceptualization, supervision, methodology, writing, funding, project administration, data curation, analysis, visualization, validation, and investigation.

  • Jia-Rui Lin contributed conceptualization, supervision, methodology, writing review, funding acquisition, and project administration.
  • Yun-Hong Cai contributed original drafting, writing review, methodology, data curation, formal analysis, and visualization.
  • Xiang-Rui Ni contributed original drafting, methodology, validation, investigation, and data curation.
  • Peng Pan contributed conceptualization, supervision, project administration, funding acquisition, and writing review.

A.1 Defect Identification Prompt Template

The defect-identification prompt template instructs the model to apply fire-code rules to specified BIM components and return only formatted defect identifiers and descriptions.

  • The template frames the model as a professional architectural designer performing stepwise compliance inspection.It asks the model to examine each component against specified rules and identify missing or abnormal attribute values.
  • Fire-code rules map component categories and fire-resistance ratings to required combustibility and resistance limits.Separate provisions cover load-bearing walls, exterior and interior walls, floors, stairs, and accessible flat roofs.
  • The prompt limits inspection to the given components and requires missing or abnormal values to be reported as defects.
  • The required output lists each problematic component ID and issue, separating fields with commas and components with semicolons.The template also requires a [Final Answer]: marker and excludes additional reasoning from the final response.

A.2 Repair Suggestion Generation Prompt Template

The repair-suggestion prompt supplies BIM component attributes, detected defects, and retrieved code provisions, then requests formatted repair-oriented outputs.

  • The input describes component identity, location, geometry, wall type, fire-resistance rating, combustibility, and fire-resistance limit.Examples include multiple aerated-concrete wall components with compliant, missing, or noncompliant fire-safety attributes.
  • The final answer must follow a fixed format containing only problematic component IDs and their specific issues.The prompt requests a [Final Answer]: marker and prohibits additional information or reasoning in the final response.
  • The prompt presents identified defects such as missing combustibility information, combustible materials, and insufficient fire-resistance limits.Example defect records identify components 350236, 350838, and 350925 as problematic.
  • The prompt requires the model to determine whether retrieved provisions are relevant before answering based on them.
Loading 2608.28629v1…