Source-linked AI summary
AAS-RAIL: Improving Information Extraction for Asset Administration Shells through Retrieval-Augmented In-Context Learning
Janek Groß, Jens Heidrich
TL;DR
Generating AAS from heterogeneous datasheets is difficult because technical terminology and schema conventions vary across companies. AAS-RAIL retrieves LLM-generated extraction helpers from similar AAS to provide instance-specific guidance without fine-tuning. Across evaluated models, it consistently improves extraction quality, with reported relative gains of 30.4–52.4% over conventional prompting.
Problem
Datasheet-based AAS generation requires extracting technical information from heterogeneous documents while handling company-specific terminology, conventions, and possible missing or ambiguous values.
Method
AAS-RAIL dynamically retrieves property-specific extraction helpers from semantically similar AAS and adds them to LLM prompts as company-specific in-context guidance.
Results
30.4–52.4% relative extraction improvements over conventional prompting were reported across evaluated open- and closed-weight LLMs.
Takeaways & Limitations
RAIL supports company-specific AAS generation without model fine-tuning, and reasonably sized open-weight models can provide a practical local-deployment option.
Takeaways & Limitations
The evaluation covers the TechnicalData submodel, datasheets from four manufacturers, a limited model selection, and assumes existing examples for the retrieval database.
Abstract
from arXiv · showhide
The Asset Administration Shell (AAS) is a cornerstone of Industry 4.0 and the Digital Product Passport, providing standardized digital representations of industrial assets. While manufacturers already maintain extensive technical product documentation, generating AAS instances from existing product datasheets remains a labor-intensive task because technical information is extracted from heterogeneous document structures and often involves company-specific terminology and conventions. In this work, we present AAS-RAIL, a retrieval-augmented information extraction (IE) approach that automatically generates Asset Administration Shells from PDF product datasheets using large language models (LLMs). Instead of relying on a fixed set of few-shot examples, the proposed retrieval-augmented in-context learning (RAIL) approach retrieves LLM-generated extraction helpers from similar Asset Administration Shells to provide instance-specific in-context learning (ICL). This enables the model to adapt its extraction behavior to company-specific naming conventions and formatting styles without fine-tuning. Our core contribution is the dynamic selection of company-specific AAS examples for each datasheet, replacing static prompting with an extraction pipeline that adapts to instances and combines semantic retrieval and structured information extraction. The proposed approach is evaluated on a collection of industrial product datasheets using a selection of open- and closed-weight LLMs. Experimental results show that RAIL consistently improves extraction quality over conventional few-shot prompting, yielding relative improvements of 30.4-52.4%. These results demonstrate that our approach provides an effective improvement for company-specific AAS generation.
1 Introduction
AAS provides standardized digital asset representations, but creating AAS from legacy or third-party datasheets requires extracting heterogeneous technical information under company-specific conventions. AAS-RAIL addresses this challenge by dynamically retrieving extraction helpers, improving company-specific generation without fine-tuning.
- Motivation: AAS standardizes digital representations of hardware and software assets through submodels, technical properties, and associated metadata.The TechnicalData submodel contains properties such as rated voltage, operating temperature, and protection class, commonly linked to ConceptDescriptions.
- Motivation: Manual extraction is required when legacy or third-party assets cannot be generated from internal engineering databases, because datasheets are heterogeneous and semi-structured.The resulting information extraction task transforms technical documentation into machine-readable AAS representations.
- Challenge: Company-specific property names and schema conventions make static prompting unreliable, while fine-tuning requires curated data and repeated retraining as conventions evolve.The same product identifier may be modeled as ArticleNumber by one manufacturer and OrderCode by another.
- Approach: RAIL retrieves offline-generated, property-specific extraction helpers from semantically similar AAS and incorporates them as instance-specific in-context guidance.The helpers guide property identification, schema alignment, naming conventions, and output formatting without modifying model parameters.
- Contribution: 30.4–52.4% relative extraction improvements over conventional prompting were reported across evaluations of open- and closed-weight LLMs.The study also examines how retrieved-example count and prompt context affect extraction performance and practical applicability.
2 Related Work
Related work moves from task-specific extraction pipelines toward instruction-tuned LLMs, structured decoding, and retrieval-augmented prompting. AAS-RAIL extends this direction by retrieving extraction helpers that adapt AAS extraction to organization-specific conventions.
- Information Extraction: Classical information extraction combines task-specific models, domain rules, and post-processing, requiring labeled data and retraining for new domains or schemas.These systems can be accurate in well-defined domains but are costly to adapt.
- Information Extraction: Instruction-tuned LLMs enable end-to-end generative extraction by identifying entities, inferring relationships, and producing target representations in one inference process.This reduces prototyping effort for new extraction domains.
- In-Context Learning: Grammar-constrained decoding enforces machine-readable output syntax, but extraction quality still depends on how the task is presented in the prompt.In-context learning adapts models through prompt examples rather than parameter updates.
- Retrieval-Augmented Prompting: Retrieval-augmented generation dynamically selects relevant database entries for prompt context, addressing the dependence of few-shot performance on demonstration choice.The retrieved context adapts to each query without retraining the model.
- AAS Generation: RAIL combines retrieval-augmented prompting with AAS information extraction by retrieving helpers from semantically similar AAS instances to adapt to organization-specific schemas and naming conventions.Unlike fixed demonstrations, the approach preserves prompt-based flexibility without fine-tuning.
3 Study Design
The study constructs an offline retrieval database from existing AAS and uses it during online extraction to select company- and product-relevant helpers. Evaluation compares RAIL with fixed few-shot prompting across industrial datasheets, models, retrieval settings, and context lengths.
- Online Inference: The online pipeline converts each datasheet to text, retrieves helpers using manufacturer, ECLASS taxonomy, and datasheet similarity, and prompts an LLM to generate AAS values.The generated values are compared with reference values after extraction.
- ICL Database Creation: The offline phase transforms existing AAS into a database containing property metadata, datasheet embeddings, and LLM-generated extraction helpers.AASX archives are converted into RDF so technical properties and metadata can be retrieved as structured resources.
- ICL Database Creation: Each helper describes how a technical property is identified and extracted from datasheet evidence rather than merely providing an input-output example.Helper generation occurs once during database construction and does not add inference-time cost.
- Retrieval: Layered retrieval prioritizes same-company properties, then deeper ECLASS taxonomy similarity, and finally cosine similarity between datasheet embeddings.The top-k helpers are inserted into the prompt as instance-specific guidance.
- Evaluation: The evaluation uses 200 datasheet–AAS pairs from four companies, with 40 disjoint pairs reserved for the retrieval database.Each datasheet is processed with and without RAIL under otherwise matched prompting conditions.
- Evaluation: Performance is measured as property extraction accuracy, with grammar-constrained decoding producing exactly one value per predefined property.Nonnumeric matches require cosine similarity ≥0.88, while numeric values may differ by no more than 1%.
4 Results
RAIL improved property extraction accuracy across all evaluated models, with gains also reflected in per-property outcomes and ablation results. Performance generally benefited from more retrieval examples, while batch size had a smaller effect.
- 4.1 In-Context Learning Results: 30.4%–52.4% relative improvements were achieved across evaluated models when RAIL was compared with the baseline.Results were averaged over individual AAS instances so each product and company contributed equally regardless of property count.
- 4.1 In-Context Learning Results: RAIL improved all evaluated models, with open-weight models gaining 40.2–52.4% and closed-weight models gaining 30.4%–43.8%.With RAIL enabled, accuracy ranged from 58.0% to 79.3%.
- 4.1 In-Context Learning Results: RAIL converted 22.4% of previously incorrect property predictions into correct ones, while 2.0% of previously correct predictions became incorrect.Both approaches were correct for 52.2% of properties, and both were incorrect for 23.5%.
- 4.2 Ablation Study: 62.6% accuracy was achieved with values from different AAS as prompt examples, compared with 50.2% for the alternative configuration.Replacing extraction helpers with example values reduced accuracy by 11.2 percentage points, while random retrieval reduced it by 3.7 percentage points.
- 4.3 Effect of Context Length: Accuracy generally increased with more retrieval examples, peaking at four to eight helpers per property, whereas batch-size effects were less pronounced.Differences among larger batch-size configurations were comparatively small.
5 Discussion
RAIL consistently improves extraction across evaluated LLMs by using dynamically retrieved, company-specific guidance, while remaining limited by source-document quality and requiring human oversight. Retrieved helpers improve performance beyond simply adding examples, and larger retrieved contexts helped within the evaluated setting.
- RAIL improved extraction accuracy across all evaluated LLMs by conditioning models on AAS reflecting organization-specific terminology and conventions.This adaptation does not require model fine-tuning.
- Ablation results indicate that richer LLM-generated extraction helpers, rather than merely additional examples, account for a substantial part of the improvement.Layered retrieval adds further benefit by selecting more relevant demonstrations.
- 79.3% accuracy in the best-performing configuration remains insufficient for fully automated AAS generation.Source documents often omit, distribute, or implicitly describe technical information, so human evaluation and correction remain necessary.
- Increasing the number of retrieved helpers consistently improved extraction quality without substantial degradation from longer prompt contexts in the evaluated setting.The optimal configuration still depended somewhat on extraction batch size.
- 22.4% of previously incorrect extractions were corrected by RAIL, while 2.0% of previously correct predictions became incorrect.The improvement considerably exceeded these infrequent retrieval-related failures.
- Externalizing company-specific knowledge into a retrieval database allows new AAS instances to be incorporated incrementally without retraining.
6 Threats to Validity
The validity of the reported improvements is constrained by evaluation assumptions, possible retrieval confounds, limited coverage, and incomplete uncertainty quantification. The evidence is strongest for the evaluated within-manufacturer TechnicalData setting rather than broader industrial generalization.
- Construct validity: Embedding-based evaluation may classify related but incorrect values as correct because of the selected similarity threshold.The evaluation also assumes that reference AAS instances represent the intended ground truth.
- Construct validity: Reported results are point estimates without confidence intervals or statistical significance tests.Paired bootstrap intervals or McNemar’s test could quantify uncertainty in the observed improvements.
- Internal validity: Retrieval quality, prompt structure, and context length may interact, so improvements cannot be attributed exclusively to the retrieval strategy.
- Internal validity: The retrieval database contains 40 representative AAS examples, which may not reflect industrial retrieval collections’ full size and diversity.
- Internal validity: Near-duplicate documents or product variants could make retrieval easier and inflate the observed benefit.
- External validity: The evaluation covers the TechnicalData submodel, datasheets from four manufacturers, and a limited selection of current models.Results may not directly generalize to other submodels, document types, industrial domains, multilingual documentation, or future models.
- External validity: The study does not establish performance for new manufacturers without prior AAS instances.
- Conclusion validity: Conclusions rely on a single evaluation dataset and fixed prompt configurations, while closed-weight APIs may change over time.Broader datasets, retrieval strategies, repeated executions, and recorded model versions would strengthen reproducibility and generality.
7 Conclusion and Future Work
The paper presents RAIL as a retrieval-augmented approach for adapting LLM-based AAS extraction to company-specific naming conventions without fine-tuning. Evaluation improved extraction for every tested model, while future work proposes combining RAIL with agentic document exploration and self-evaluation.
- RAIL dynamically retrieves guidance from relevant AAS instances and incorporates it into prompts for datasheet-based AAS generation.This enables adaptation to company-specific naming conventions without model fine-tuning.
- RAIL improved extraction quality for every model in the evaluated configuration without major practical drawbacks.
- Reasonably sized open-weight models achieved competitive performance, supporting local deployment as a practical option for company-specific AAS generation.
- Future work should integrate RAIL with agentic information-extraction systems that iteratively explore documents and self-evaluate.The proposed combination is intended to improve the reliability and completeness of automatically generated AAS.